Skip to content

Pull requests: OpenRouterTeam/benchmark-harness

Pull request creation is restricted
13 Open 44 Closed
13 Open 44 Closed
Author
Filter by author
Loading
Label
Filter by label
Loading
Use alt + click/return to exclude labels
or ⇧ + click/return for logical OR
Projects
Filter by project
Loading
Milestones
Filter by milestone
Loading
Reviews
Filter by reviews
No reviews Review required Approved review Changes requested
Assignee
Filter by who’s assigned
Assigned to nobody Loading
Sort
Sort by
Newest Oldest Most commented Least commented Recently updated Least recently updated Best match
Most reactions
👍 👎 😄 🎉 😕 ❤️ 🚀 👀

Pull requests list

deps: bump the dependencies group across 1 directory with 17 updates
#57 opened Aug 25, 2026 by dependabot Bot Loading…
1
refactor(providers): drop the chat-completions client
#55 opened Aug 25, 2026 by abhinav-pola Contributor Loading…
5 tasks done
@abhinav-pola
3
feat(agent-cli): add Prime Agent and package agent runtimes
#54 opened Aug 25, 2026 by abhinav-pola Contributor Loading…
5 tasks done
@abhinav-pola
7
feat: persist completed sample outcomes so interrupted runs resume instead of restarting
#48 opened Aug 21, 2026 by jamespsterling • Draft
4 of 5 tasks
@jamespsterling
33
feat(runner): accept caller trace headers for OpenRouter API requests
#45 opened Aug 20, 2026 by jamespsterling Loading…
3 tasks done
@jamespsterling
7
ECO-2484: feat(datasets): cache HF rows pages and task repo checkouts on disk
#44 opened Aug 19, 2026 by abhinav-pola Contributor Loading…
6 tasks done
25
fix(search): resolve server-tool generation ids
#32 opened Aug 14, 2026 by ayush-or Contributor Loading…
fix: recover unclassified Responses output items
#29 opened Aug 13, 2026 by bjoerndanz Contributor Loading…
5 tasks done
@bjoerndanz
3
feat: add agent_dx benchmark for OpenRouter agent-experience evals
#15 opened Aug 7, 2026 by devin-ai-integration Bot Loading…
5 tasks done
7
feat(benchmarks): chess — full games vs Stockfish, first-party original
#4 opened Aug 5, 2026 by LukasParke Loading…
2
feat(benchmarks): EvalManifestV1 — unified eval description schema + first-party manifests
#3 opened Aug 5, 2026 by LukasParke Loading…
2
feat(benchmarks): custom_eval — declarative user evals as run-config data
#2 opened Aug 5, 2026 by LukasParke Loading…
2
feat(harness): durable per-(sample, epoch) result store + fiber-scoped request context
#1 opened Aug 5, 2026 by LukasParke Loading…
3
ProTip! Mix and match filters to narrow down what you’re looking for.