Skip to content

Update dependency @playwright/test to v1.53.1 #5366

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

openverse-bot
Copy link
Collaborator

@openverse-bot openverse-bot commented Feb 1, 2025

This PR contains the following updates:

Package Type Update Change
@playwright/test (source) devDependencies minor 1.49.1 -> 1.53.1

Release Notes

microsoft/playwright (@​playwright/test)

v1.53.1

Compare Source

Highlights

https://github.com/microsoft/playwright/issues/36339 - [Regression]: Click can fail when scrolling requiredhttps://github.com/microsoft/playwright/issues/363077 - [Regression (Chromium)]: Under some scenarios filling a textarea doesn't fill
https://github.com/microsoft/playwright/issues/36294 - [Regression (Firefox)]: setViewportSize times out
https://github.com/microsoft/playwright/pull/36350 - [Fix]: Display HTTP method for fetch trace entries

Browser Versions

  • Chromium 138.0.7204.23
  • Mozilla Firefox 139.0
  • WebKit 18.5

This version was also tested against the following stable channels:

  • Google Chrome 137
  • Microsoft Edge 137

v1.53.0

Compare Source

Trace Viewer and HTML Reporter Updates

  • New Steps in Trace Viewer and HTML reporter: New Trace Viewer Steps

  • New option in 'html' reporter to set the title of a specific test run:

    import { defineConfig } from '@​playwright/test';
    
    export default defineConfig({
      reporter: [['html', { title: 'Custom test run #​1028' }]]
    });

Miscellaneous

  • New option kind in testInfo.snapshotPath() controls which snapshot path template is used.

  • New method locator.describe() to describe a locator. Used for trace viewer and reports.

    const button = page.getByTestId('btn-sub').describe('Subscribe button');
    await button.click();
  • npx playwright install --list will now list all installed browsers, versions and locations.

Browser Versions

  • Chromium 138.0.7204.4
  • Mozilla Firefox 139.0
  • WebKit 18.5

This version was also tested against the following stable channels:

  • Google Chrome 137
  • Microsoft Edge 137

v1.52.0

Compare Source

Highlights

  • New method expect(locator).toContainClass() to ergonomically assert individual class names on the element.

    await expect(page.getByRole('listitem', { name: 'Ship v1.52' })).toContainClass('done');
  • Aria Snapshots got two new properties: /children for strict matching and /url for links.

    await expect(locator).toMatchAriaSnapshot(`
      - list
        - /children: equal
        - listitem: Feature A
        - listitem:
          - link "Feature B":
            - /url: "https://playwright.dev"
    `);

Test Runner

  • New property testProject.workers allows to specify the number of concurrent worker processes to use for a test project. The global limit of property testConfig.workers still applies.
  • New testConfig.failOnFlakyTests option to fail the test run if any flaky tests are detected, similarly to --fail-on-flaky-tests. This is useful for CI/CD environments where you want to ensure that all tests are stable before deploying.
  • New property testResult.annotations contains annotations for each test retry.

Miscellaneous

  • New option maxRedirects in apiRequest.newContext() to control the maximum number of redirects.
  • New option ref in locator.ariaSnapshot() to generate reference for each element in the snapshot which can later be used to locate the element.
  • HTML reporter now supports NOT filtering via !@​my-tag or !my-file.spec.ts or !p:my-project.

Breaking Changes

  • Changes to glob URL patterns in methods like page.route():
    • ? wildcard is not supported any more, it will always match question mark ? character.
    • Ranges/sets [] are not supported anymore. We recommend using regular expressions instead.
  • Method route.continue() does not allow to override the Cookie header anymore. If a Cookie header is provided, it will be ignored, and the cookie will be loaded from the browser's cookie store. To set custom cookies, use browserContext.addCookies().
  • macOS 13 is now deprecated and will no longer receive WebKit updates. Please upgrade to a more recent macOS version to continue benefiting from the latest WebKit improvements.

Browser Versions

  • Chromium 136.0.7103.25
  • Mozilla Firefox 137.0
  • WebKit 18.4

This version was also tested against the following stable channels:

  • Google Chrome 135
  • Microsoft Edge 135

v1.51.1

Compare Source

Highlights

https://github.com/microsoft/playwright/issues/35093 - [Regression]: TimeoutOverflowWarning: 2149630.634 does not fit into a 32-bit signed integer
https://github.com/microsoft/playwright/issues/35138 - [Regression]: TypeError: Cannot read properties of undefined (reading 'expectInfo')

Browser Versions

  • Chromium 134.0.6998.35
  • Mozilla Firefox 135.0
  • WebKit 18.4

This version was also tested against the following stable channels:

  • Google Chrome 133
  • Microsoft Edge 133

v1.51.0

Compare Source

StorageState for indexedDB

  • New option indexedDB for browserContext.storageState() allows to save and restore IndexedDB contents. Useful when your application uses IndexedDB API to store authentication tokens, like Firebase Authentication.

    Here is an example following the authentication guide:

    // tests/auth.setup.ts
    import { test as setup, expect } from '@​playwright/test';
    import path from 'path';
    
    const authFile = path.join(__dirname, '../playwright/.auth/user.json');
    
    setup('authenticate', async ({ page }) => {
      await page.goto('/');
      // ... perform authentication steps ...
    
      // make sure to save indexedDB
      await page.context().storageState({ path: authFile, indexedDB: true });
    });

Copy prompt

New "Copy prompt" button on errors in the HTML report, trace viewer and UI mode. Click to copy a pre-filled LLM prompt that contains the error message and useful context for fixing the error.

Copy prompt

Filter visible elements

New option visible for locator.filter() allows matching only visible elements.

// example.spec.ts
test('some test', async ({ page }) => {
  // Ignore invisible todo items.
  const todoItems = page.getByTestId('todo-item').filter({ visible: true });
  // Check there are exactly 3 visible ones.
  await expect(todoItems).toHaveCount(3);
});

Git information in HTML report

Set option testConfig.captureGitInfo to capture git information into testConfig.metadata.

// playwright.config.ts
import { defineConfig } from '@​playwright/test';

export default defineConfig({
  captureGitInfo: { commit: true, diff: true }
});

HTML report will show this information when available:

Git information in the report

Test Step improvements

A new TestStepInfo object is now available in test steps. You can add step attachments or skip the step under some conditions.

test('some test', async ({ page, isMobile }) => {
  // Note the new "step" argument:
  await test.step('here is my step', async step => {
    step.skip(isMobile, 'not relevant on mobile layouts');

    // ...
    await step.attach('my attachment', { body: 'some text' });
    // ...
  });
});

Miscellaneous

Browser Versions

  • Chromium 134.0.6998.35
  • Mozilla Firefox 135.0
  • WebKit 18.4

This version was also tested against the following stable channels:

  • Google Chrome 133
  • Microsoft Edge 133

v1.50.1

Compare Source

Highlights

https://github.com/microsoft/playwright/issues/34483 - [Feature]: single aria snapshot for different engines/browsershttps://github.com/microsoft/playwright/issues/344977 - [Bug]: Firefox not handling keepalive: true fetch requesthttps://github.com/microsoft/playwright/issues/3450404 - [Bug]: update snapshots not creating good difhttps://github.com/microsoft/playwright/issues/34507507 - [Bug]: snapshotPathTemplate doesnt work when multiple projehttps://github.com/microsoft/playwright/issues/344624462 - [Bug]: updateSnapshots "changed" throws an error

Browser Versions

  • Chromium 133.0.6943.16
  • Mozilla Firefox 134.0
  • WebKit 18.2

This version was also tested against the following stable channels:

  • Google Chrome 132
  • Microsoft Edge 132

v1.50.0

Compare Source

Test runner

  • New option timeout allows specifying a maximum run time for an individual test step. A timed-out step will fail the execution of the test.

    test('some test', async ({ page }) => {
      await test.step('a step', async () => {
        // This step can time out separately from the test
      }, { timeout: 1000 });
    });
  • New method test.step.skip() to disable execution of a test step.

    test('some test', async ({ page }) => {
      await test.step('before running step', async () => {
        // Normal step
      });
    
      await test.step.skip('not yet ready', async () => {
        // This step is skipped
      });
    
      await test.step('after running step', async () => {
        // This step still runs even though the previous one was skipped
      });
    });
  • Expanded expect(locator).toMatchAriaSnapshot() to allow storing of aria snapshots in separate YAML files.

  • Added method expect(locator).toHaveAccessibleErrorMessage() to assert the Locator points to an element with a given aria errormessage.

  • Option testConfig.updateSnapshots added the configuration enum changed. changed updates only the snapshots that have changed, whereas all now updates all snapshots, regardless of whether there are any differences.

  • New option testConfig.updateSourceMethod defines the way source code is updated when testConfig.updateSnapshots is configured. Added overwrite and 3-way modes that write the changes into source code, on top of existing patch mode that creates a patch file.

    npx playwright test --update-snapshots=changed --update-source-method=3way
  • Option testConfig.webServer added a gracefulShutdown field for specifying a process kill signal other than the default SIGKILL.

  • Exposed testStep.attachments from the reporter API to allow retrieval of all attachments created by that step.

  • New option pathTemplate for toHaveScreenshot and toMatchAriaSnapshot assertions in the testConfig.expect configuration.

UI updates

  • Updated default HTML reporter to improve display of attachments.
  • New button for picking elements to produce aria snapshots.
  • Additional details (such as keys pressed) are now displayed alongside action API calls in traces.
  • Display of canvas content in traces is error-prone. Display is now disabled by default, and can be enabled via the Display canvas content UI setting.
  • Call and Network panels now display additional time information.

Breaking

Browser Versions

  • Chromium 133.0.6943.16
  • Mozilla Firefox 134.0
  • WebKit 18.2

This version was also tested against the following stable channels:

  • Google Chrome 132
  • Microsoft Edge 132

Configuration

📅 Schedule: Branch creation - Between 12:00 AM and 03:59 AM, on day 1 of the month ( * 0-3 1 * * ) (UTC), Automerge - At any time (no schedule defined).

🚦 Automerge: Enabled.

Rebasing: Whenever PR is behind base branch, or you tick the rebase/retry checkbox.

🔕 Ignore: Close this PR and you won't be reminded about this update again.


  • If you want to rebase/retry this PR, check this box

This PR has been generated by Renovate Bot.

@openverse-bot openverse-bot added dependencies Pull requests that update a dependency file 💻 aspect: code Concerns the software code in the repository 🟨 tech: javascript Involves JavaScript 🟩 priority: low Low priority and doesn't need to be rushed 🧰 goal: internal improvement Improvement that benefits maintainers, not users 🧱 stack: frontend Related to the Nuxt frontend labels Feb 1, 2025
@openverse-bot openverse-bot requested a review from a team as a code owner February 1, 2025 03:10
Copy link

github-actions bot commented Feb 1, 2025

Latest k6 run output1

     ✓ status was 200

     checks.........................: 100.00% ✓ 408      ✗ 0   
     data_received..................: 98 MB   407 kB/s
     data_sent......................: 53 kB   221 B/s
     http_req_blocked...............: avg=73.4µs   min=2.4µs    med=4.97µs   max=1.53ms   p(90)=149.09µs p(95)=457.68µs
     http_req_connecting............: avg=33.45µs  min=0s       med=0s       max=1.45ms   p(90)=96.1µs   p(95)=121.03µs
     http_req_duration..............: avg=159.54ms min=17.55ms  med=93.21ms  max=1.05s    p(90)=390.54ms p(95)=502.62ms
       { expected_response:true }...: avg=159.54ms min=17.55ms  med=93.21ms  max=1.05s    p(90)=390.54ms p(95)=502.62ms
   ✓ http_req_failed................: 0.00%   ✓ 0        ✗ 408 
     http_req_receiving.............: avg=179.67µs min=58µs     med=151.35µs max=764.17µs p(90)=292.47µs p(95)=371.6µs 
     http_req_sending...............: avg=27.09µs  min=8.8µs    med=22.63µs  max=146.82µs p(90)=39.82µs  p(95)=62.56µs 
     http_req_tls_handshaking.......: avg=0s       min=0s       med=0s       max=0s       p(90)=0s       p(95)=0s      
     http_req_waiting...............: avg=159.33ms min=17.41ms  med=93.03ms  max=1.05s    p(90)=390.31ms p(95)=502.36ms
     http_reqs......................: 408     1.69444/s
     iteration_duration.............: avg=863.58ms min=246.49ms med=868.92ms max=1.8s     p(90)=1.2s     p(95)=1.47s   
     iterations.....................: 76      0.315631/s
     vus............................: 2       min=0      max=6 
     vus_max........................: 60      min=60     max=60

Footnotes

  1. This comment will automatically update with new output each time k6 runs for this PR

@openverse-bot openverse-bot force-pushed the gha-renovateplaywright-monorepo branch 12 times, most recently from 9dba4c5 to c30a008 Compare February 6, 2025 17:06
@openverse-bot openverse-bot force-pushed the gha-renovateplaywright-monorepo branch from c30a008 to 957e79e Compare February 13, 2025 12:47
@openverse-bot openverse-bot force-pushed the gha-renovateplaywright-monorepo branch 3 times, most recently from 64e6bde to 64d38b5 Compare March 6, 2025 18:09
@openverse-bot openverse-bot changed the title Update dependency @playwright/test to v1.50.1 Update dependency @playwright/test to v1.51.0 Mar 6, 2025
@openverse-bot openverse-bot force-pushed the gha-renovateplaywright-monorepo branch 3 times, most recently from 7653711 to 413ffe8 Compare March 10, 2025 05:07
@openverse-bot openverse-bot force-pushed the gha-renovateplaywright-monorepo branch from 413ffe8 to 7e2c1dd Compare March 17, 2025 17:07
@openverse-bot openverse-bot changed the title Update dependency @playwright/test to v1.51.0 Update dependency @playwright/test to v1.51.1 Mar 17, 2025
@openverse-bot openverse-bot force-pushed the gha-renovateplaywright-monorepo branch 4 times, most recently from db9da32 to a930a05 Compare April 11, 2025 14:08
@openverse-bot openverse-bot force-pushed the gha-renovateplaywright-monorepo branch 2 times, most recently from 2f27554 to 929133c Compare April 17, 2025 17:38
@openverse-bot openverse-bot changed the title Update dependency @playwright/test to v1.51.1 Update dependency @playwright/test to v1.52.0 Apr 17, 2025
@openverse-bot openverse-bot force-pushed the gha-renovateplaywright-monorepo branch 2 times, most recently from 19721fa to a9d29b7 Compare April 24, 2025 15:09
@openverse-bot openverse-bot force-pushed the gha-renovateplaywright-monorepo branch 3 times, most recently from 530dc94 to 5815e1c Compare May 7, 2025 06:44
@openverse-bot openverse-bot force-pushed the gha-renovateplaywright-monorepo branch 4 times, most recently from 7e96a8e to 6e83392 Compare May 31, 2025 09:07
@openverse-bot openverse-bot force-pushed the gha-renovateplaywright-monorepo branch 5 times, most recently from 30adf27 to 0d8e5de Compare June 10, 2025 18:43
@openverse-bot openverse-bot changed the title Update dependency @playwright/test to v1.52.0 Update dependency @playwright/test to v1.53.0 Jun 10, 2025
@openverse-bot openverse-bot force-pushed the gha-renovateplaywright-monorepo branch from 0d8e5de to 7dcae05 Compare June 11, 2025 12:48
@openverse-bot openverse-bot force-pushed the gha-renovateplaywright-monorepo branch from 7dcae05 to f8f0a40 Compare June 18, 2025 18:10
@openverse-bot openverse-bot changed the title Update dependency @playwright/test to v1.53.0 Update dependency @playwright/test to v1.53.1 Jun 18, 2025
Copy link

Playwright failure test results: https://github.com/WordPress/openverse/actions/runs/15740394825

It looks like some of the Playwright tests failed. If you made changes to the frontend UI without updating snapshots, this might be the cause. You can download zipped patches containing the updated snapshots alongside a general trace of the tests under the "Artifacts" section in the above page. They're named in the form *_snapshot_diff and *_test_results respectively.

You can read more on how to use these artifacts in the docs.

If the test is flaky, follow the flaky test triage procedure.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
💻 aspect: code Concerns the software code in the repository dependencies Pull requests that update a dependency file 🧰 goal: internal improvement Improvement that benefits maintainers, not users 🟩 priority: low Low priority and doesn't need to be rushed 🧱 stack: frontend Related to the Nuxt frontend 🟨 tech: javascript Involves JavaScript
Projects
Status: 👀 Needs Review
Development

Successfully merging this pull request may close these issues.

1 participant