Skip to content

👷 run latest firefox and webkit in CI, default local e2e to chromium#4618

Open
thomas-lebeau wants to merge 1 commit into
mainfrom
thomas.lebeau/e2e-latest-browsers-in-ci
Open

👷 run latest firefox and webkit in CI, default local e2e to chromium#4618
thomas-lebeau wants to merge 1 commit into
mainfrom
thomas.lebeau/e2e-latest-browsers-in-ci

Conversation

@thomas-lebeau
Copy link
Copy Markdown
Collaborator

@thomas-lebeau thomas-lebeau commented May 13, 2026

Motivation

We only ran the latest Chromium in CI e2e, alongside pinned variants of Chromium 120, Firefox 119 and WebKit 17.4 (added in #4612). This left a gap: regressions on the latest Firefox/WebKit versions weren't caught until they shipped. Locally, running all browsers also made the e2e feedback loop slow.

Changes

  • .gitlab-ci.yml: add firefox and webkit to the e2e job's BROWSER matrix (in addition to the existing chromium/*-pinned projects). Bump CURRENT_CI_IMAGE to 108.
  • Dockerfile: install Playwright's firefox and webkit browsers (with deps) in the CI image so the new matrix entries have what they need.

Test instructions

  • CI: confirm the e2e job runs the new firefox and webkit matrix entries successfully on the freshly built image 108.
  • Local: yarn test:e2e should now run chromium only. yarn test:e2e --project=firefox still works for opting in to other browsers.

Checklist

  • Tested locally
  • Tested on staging
  • Added unit tests for this change.
  • Added e2e/integration tests for this change.
  • Updated documentation and/or relevant AGENTS.md file

@thomas-lebeau thomas-lebeau requested a review from a team as a code owner May 13, 2026 13:42
Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 544f22ee07

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread package.json Outdated
@thomas-lebeau thomas-lebeau force-pushed the thomas.lebeau/e2e-latest-browsers-in-ci branch from 544f22e to 4ea9e76 Compare May 13, 2026 13:48
@cit-pr-commenter-54b7da
Copy link
Copy Markdown

cit-pr-commenter-54b7da Bot commented May 13, 2026

Bundles Sizes Evolution

📦 Bundle Name Base Size Local Size 𝚫 𝚫% Status
Rum 169.51 KiB 169.51 KiB 0 B 0.00%
Rum Profiler 5.97 KiB 5.97 KiB 0 B 0.00%
Rum Recorder 21.23 KiB 21.23 KiB 0 B 0.00%
Logs 54.70 KiB 54.70 KiB 0 B 0.00%
Rum Slim 127.85 KiB 127.85 KiB 0 B 0.00%
Worker 22.99 KiB 22.99 KiB 0 B 0.00%
🚀 CPU Performance
Action Name Base CPU Time (ms) Local CPU Time (ms) 𝚫%
RUM - add global context 0.0017 0.0027 +58.82%
RUM - add action 0.0097 0.0149 +53.61%
RUM - add error 0.0098 0.0133 +35.71%
RUM - add timing 0.0005 0.0007 +40.00%
RUM - start view 0.0093 0.0117 +25.81%
RUM - start/stop session replay recording 0.0008 0.0012 +50.00%
Logs - log message 0.0181 0.0184 +1.66%
🧠 Memory Performance
Action Name Base Memory Consumption Local Memory Consumption 𝚫
RUM - add global context 52.78 KiB 38.42 KiB -14.36 KiB
RUM - add action 69.61 KiB 67.15 KiB -2.45 KiB
RUM - add timing 43.02 KiB 38.76 KiB -4.26 KiB
RUM - add error 71.47 KiB 71.49 KiB +21 B
RUM - start/stop session replay recording 51.15 KiB 45.87 KiB -5.28 KiB
RUM - start view 492.44 KiB 487.17 KiB -5.27 KiB
Logs - log message 58.34 KiB 70.08 KiB +11.74 KiB

🔗 RealWorld

@datadog-prod-us1-6
Copy link
Copy Markdown

datadog-prod-us1-6 Bot commented May 13, 2026

Tests

Fix all issues with BitsAI or with Cursor

⚠️ Warnings

🧪 8 Tests failed

Session Stores › Cookies › trackSessionAcrossSubdomains: false › stores the cookie on the full host name › async from sessionStore.scenario.ts   View in Datadog   (Fix with Cursor)
sessionStore.scenario.ts:50:17 async

[webkit] › sessionStore.scenario.ts:50:17 › Session Stores › Cookies › trackSessionAcrossSubdomains: false › stores the cookie on the full host name › async 

    Error: expect(received).toEqual(expected) // deep equality

    Expected: ObjectContaining {"domain": "foo.bar.localhost"}
    Received: undefined

      54 |           const cookies = await browserContext.cookies()
...
Session Stores › Cookies › trackSessionAcrossSubdomains: false › stores the cookie on the full host name › bundle from sessionStore.scenario.ts   View in Datadog   (Fix with Cursor)
sessionStore.scenario.ts:50:17 bundle

[webkit] › sessionStore.scenario.ts:50:17 › Session Stores › Cookies › trackSessionAcrossSubdomains: false › stores the cookie on the full host name › bundle 

    Error: expect(received).toEqual(expected) // deep equality

    Expected: ObjectContaining {"domain": "foo.bar.localhost"}
    Received: undefined

      54 |           const cookies = await browserContext.cookies()
...
Session Stores › Cookies › trackSessionAcrossSubdomains: false › stores the cookie on the full host name › npm from sessionStore.scenario.ts   View in Datadog   (Fix with Cursor)
sessionStore.scenario.ts:50:17 npm

[webkit] › sessionStore.scenario.ts:50:17 › Session Stores › Cookies › trackSessionAcrossSubdomains: false › stores the cookie on the full host name › npm 

    Error: expect(received).toEqual(expected) // deep equality

    Expected: ObjectContaining {"domain": "foo.bar.localhost"}
    Received: undefined

      54 |           const cookies = await browserContext.cookies()
...
View all

ℹ️ Info

No other issues found (see more)

❄️ No new flaky tests detected

🎯 Code Coverage (details)
Patch Coverage: 100.00%
Overall Coverage: 76.96% (+0.00%)

Useful? React with 👍 / 👎

This comment will be updated automatically if new data arrives.
🔗 Commit SHA: 0c3132a | Docs | Datadog PR Page | Give us feedback!

@thomas-lebeau thomas-lebeau force-pushed the thomas.lebeau/e2e-latest-browsers-in-ci branch from 4ea9e76 to 573ba97 Compare May 15, 2026 15:21
Adds firefox and webkit to the CI e2e matrix alongside chromium so we
catch cross-browser regressions on the latest browser versions (in
addition to the existing pinned variants). Installs the matching
Playwright browsers in the CI image and bumps CURRENT_CI_IMAGE to 108.

Defaults local `yarn test:e2e` to chromium only to keep the local
feedback loop fast; other browsers can still be run via `--project=`.
@thomas-lebeau thomas-lebeau force-pushed the thomas.lebeau/e2e-latest-browsers-in-ci branch from 573ba97 to 0c3132a Compare May 15, 2026 19:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants