Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Feature]: Create custom message for Any Playwright/Test step, not just expects #33730

Open
ethan-bm opened this issue Nov 22, 2024 · 2 comments

Comments

@ethan-bm
Copy link

ethan-bm commented Nov 22, 2024

🚀 Feature Request

I leverage the Reporter API's "onStepEnd" to feed in a custom message for step.title, which I then use in a custom report. I would like to be able to pass in a custom message, which I can use for step.title, on any step.

Example

await page4.locator('div.sidemenu').click();
await expect(page4.locator('div.nav[id*=steps]'), 'Result Step 4 of 4: Expect Equipment step view to be visible').toBeVisible({ timeout: 30000 });
Appears in my report as:

  • locator.click(div.sidemenu)
  • Result Step 4 of 4: Expect Equipment step view to be visible

I would like to create a custom statement for the "await page4.locator('div.sidemenu').click();" step.

If I did something like:
await page4.(locator('div.sidemenu'), 'Result Step 3 of 4: Click Equipment step view menu').click();
Then it appears in my reporter as:

  • Result Step 3 of 4: Click Equipment step view menu

Motivation

I believe it will help expand custom reporters and/or pave for enhanced reports built into Playwright, such as the HTML reporter.

I also use my custom report to report timings on certain steps that are problematic to management. My custom report publishes the results to both JSON and CSV. The CSV is for human readable, easy sharing, and the JSON is to push to dashboard software.

Having an exact statement for specific steps will make running analytics for dashboarding significantly easier. As of now, if a step is repeated in a test, such as "await page4.locator('div.sidemenu').click();", it makes identifying where the exact occurrence we want to report timings on harder. This is made worse if the test is repeated but does not have the same number of steps due to timeouts or failures, as ordinal positioning can't be used as a unique identifier.

@mxschmitt
Copy link
Member

We have test.step(title) for that, have you tried using it?

@ethan-bm
Copy link
Author

Yes, it creates a nested output in the report that makes data parsing for dashboards wonky. Like so:

Result Step 3 of 4: Click Equipment step view menu :15ms
{
locator.click(div.sidemenu) :281ms
}
Result Step 4 of 4: Expect Equipment step view to be visible :2999ms

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants