Repository navigation
test: harden flaky tests that are no longer retried on CI - #922
Merged
Merged
Conversation
Swift Testing failures aren't retried on the iOS simulator job, so these now fail CI outright: - ExceptionSteps: clear this token's folder in getSut; a previous test's $exception could be re-sent and read as events.first. - QueueTest: record an issue instead of trapping on a missing request or extra upload, and only count this queue's /batch requests. - ApplicationViewLayoutPublisher: raise the wait ceiling from 1s to 5s. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
dustinbyrne
approved these changes
Oct 8, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
💡 Motivation and Context
CI retries failed XCTest cases on the iOS simulator job (#882), but Swift Testing failures aren't retried. Since #915 moved the remaining specs to Swift Testing, these tests now fail CI outright when they flake:
storage.reset()keeps the event queue, so a previous test's$exceptioncould still be on disk, get sent, and be read asevents.first.getSutnow deletes this token's app folder first. It's scoped to the token because some tests run two SDKs at once.server.batchProjectTokenso it only counts its own queue's/batchrequests. The stub is process-global, so a stray request from another SDK instance shifted the request count.waitUntiltimeout raised from 1s to 5s. It returns as soon as the condition holds, so the higher ceiling only matters on a slow runner.The replay queue migration test flake is fixed separately in #920.
💚 How did you test it?
make testfiltered toPostHogExceptionStepsTest,PostHogQueueTestandPostHogQueueUploadDispositionTest: 35 tests pass on macOS.ApplicationViewLayoutPublisherTestis iOS-only; the change only raises the timeout default. Covered by thetest-ios-simulatorjob.📝 Checklist
🤖 Agent context
Autonomy: Human-driven (agent-assisted)
Written with Claude Code. Test-only changes, so there's no changeset.
🤖 Generated with Claude Code