Menu

1 / 5

Playwright 1.65 onlyFailures: Smaller Reports That Say Zero Tests Passed

DEV Community: javascript·Gary Parker·2 days ago
#3iTp302S
#devtojavascript#devto#dev#tests#report
Reading 0:00
15s threshold

Playwright 1.65 adds an onlyFailures option to the HTML, JSON and JUnit reporters, and a new failures terminal reporter. The release note says the option keeps only tests with unexpected outcomes, flaky tests and interrupted tests. I wanted three things before turning it on: how much smaller the reports get, whether the run gets faster, and what the report no longer contains.

I measured it on a 2,000-test suite. The harness and raw results are in the PoC repo .

The setup

100 spec files with 20 tests each. Each test has 3 steps. The tests use no browser.

@playwright/test 1.65.1, on Node 24 and macOS, with workers: 4 and retries: 0 .

0, 20, 100 or 400 of the 2,000 tests fail, each on a toEqual of two objects.

Eight reporter setups: dot , failures , and JSON, JUnit and HTML with and without onlyFailures . The file reporters ran beside dot , with CI=1 set.

5 repeats of every cell: 160 runs.…

Continue reading — create a free account

Join HashtagPLUS to read full articles, follow hashtags, vote, and join the conversation.