Stop three tests failing on every pull request

[mihow]

Summary

Three tests have been failing on every pull request regardless of what the pull request changes, so nothing can merge on a green check. This makes those three tests pass again by removing the two things that made them unreliable: one test depended on a website that changed its behaviour, and two others depended on the order files happen to come back from the filesystem. No application code changes — the fixes are entirely in the tests and their helpers.

The failures were not caused by any of the open pull requests. main itself is red: the last time the test workflow ran on main was 2026-04-14, and everything since has been Dependabot runs, so the suite has been broken for months without anyone noticing.

Two of the three failures turned out not to be network problems at all. The two API tests asked for "the first" trap image and then checked that the ML pipeline found insects in it. The helper picked that image from an unsorted directory listing, so which image a test got depended on arbitrary filesystem ordering, which differs between a developer's checkout and a fresh CI checkout. One of the three Vermont sample images is an empty frame with nothing for the detector to find, and CI kept picking exactly that one. The pipeline was behaving correctly and returning no detections; the test simply had nothing to assert on. On a developer machine the ordering usually landed on an image with insects in it, which is why this looked like a CI-only problem.

The third failure is a genuine external dependency: the test downloaded a thumbnail from Wikimedia, which now refuses arbitrary thumbnail widths and returns an HTTP 400 pointing at its allowed-sizes policy.

List of Changes

#	Change (what it means for the reader)	How (implementation)
1	Tests that check the ML pipeline found insects now always run against an image that actually contains insects, instead of whichever image the filesystem happened to return first.	test_logits_in_classification_response and test_config_num_classification_predictions pass an explicit filename via a new IMAGE_WITH_DETECTIONS constant in trapdata/api/tests/utils.py.
2	Every test that pulls sample images gets the same images on every machine, so a test cannot pass locally and fail in CI purely because of directory order.	get_test_image_urls sorts the glob result, and accepts an optional filenames argument for tests that need specific images. The argument is added last, so existing callers are unaffected.
3	The source image URL test no longer reaches out to the public internet, so it cannot break again when a third-party host changes what it will serve.	test_url writes an image to a temporary directory and serves it through StaticFileTestServer, which the repository already uses for this purpose. The same download-and-open code path is exercised.

Verification

Run against this branch with the GPU disabled, matching the CPU-only CI runners:

• Full suite: 39 passed, 1 skipped, 0 failed.
• The three previously failing tests pass.
• Negative control: temporarily pointing IMAGE_WITH_DETECTIONS at the empty frame reproduces the exact CI failure (AssertionError: No detections found in response), confirming the image choice is what these tests turn on rather than something incidental.
• Pinned tooling clean on the changed files: black 22.3.0, isort 5.11.5, flake8 4.0.0 with bugbear and comprehensions, autoflake 1.4.

Notes for reviewers

The detector threshold and the empty Vermont frame are both left as they are. An image with no insects in it is a reasonable thing to keep in the sample set, and the detector returning nothing for it is correct behaviour; the bug was that a test could land on it by accident.

Worth flagging separately: the ML tests still download model weights from a remote object store at test time. That is a real remaining dependency on an external service, but it is fundamental to what those tests do and out of scope here.

[coderabbitai]

Warning

Review limit reached

@mihow, you've reached your PR review limit, so we couldn't start this review.

Next review available in: 58 minutes

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

How can I continue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews.

How do review limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window.

Please refer docs for additional details.

Review details

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: 703f2ee9-4ea3-4b8a-8dd5-8dbf4bfef8b7

📥 Commits

Reviewing files that changed from the base of the PR and between a33746a and af68bcb.

📒 Files selected for processing (3)

• trapdata/api/tests/test_api.py
• trapdata/api/tests/test_models.py
• trapdata/api/tests/utils.py

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}