All pull requests

Pull requests

• fix(dataset): preserve quoted text when repairing JSON commas

  #3536·arin30 opened Oct 27, 2026·

  ·
  1

• fix(dataset): preserve expected labels in local dataset files

  #3535·raederhans opened Oct 27, 2026·

  ·
  1

• fix(tool_correctness): distinguish JSON booleans from numbers

  #3534·DevChiniwala opened Oct 26, 2026·

  ·
  3

• fix: read MCP 2.x structured_content in tool interactions

  #3533·anh-nguyenn opened Oct 26, 2026·

  ·
  1

  2

• fix: cap Azure top_logprobs to the Azure limit

  #3532·anh-nguyenn opened Oct 26, 2026·

  ·
  1

  2

• fix(benchmarks): score EquityMedQA without inverting BiasMetric and over all goldens

  #3529·inchang-ing opened Oct 25, 2026·

  ·
  1

  1

• fix: replace legacy HF dataset ids in ARC, GSM8K, TruthfulQA, BoolQ and HumanEval benchmarks

  #3527·Sivadheeraj5356 opened Oct 24, 2026·

  ·
  1

  1

• chore: format the ten files that fail the lint jobs

  #3526·Eapp1e opened Oct 24, 2026·

  ·
  2

• fix(g_eval): ignore multi-digit score prefixes in weighted scoring (Py + TS)

  #3525·zzrs-xxxr opened Oct 24, 2026·

  ·
  1

  2

• Blog: Top 10 LLM observability tools in 2026, compared

  #3524·gorakhnath-confident opened Oct 23, 2026·3 tasks done·

  ·
  1

• fix(g-eval): weight the last matching score token in TypeScript (#3516)

  #3523·Eapp1e opened Oct 23, 2026·3 tasks done·

  ·
  1

  3

• fix: correct quoting and grammar in BBQ prompt and HF callback messages

  #3519·Avicennasis opened Oct 23, 2026·

  ·
  1

• docs(faithfulness): say that unsupported claims only count with penalize_ambiguous_claims

  #3515·KritiikaaR opened Oct 22, 2026·

  ·
  1

• fix(evaluate): apply hyperparameters before cache lookup

  #3514·sushan5140 opened Oct 22, 2026·

  ·
  1

• fix(test-run): keep scores of metrics without a threshold in metricsScores

  #3513·TanishGudise opened Oct 22, 2026·

  ·
  1

• fix(evaluate): don't report the previous case's result for a sync timeout

  #3512·TanishGudise opened Oct 22, 2026·

  ·
  1

• fix(cache): recover null lookup maps

  #3511·devgprime opened Oct 22, 2026·

  ·
  2

• fix: invalidate tool metric cache when scoring options change

  #3510·Ricardo-M-L opened Oct 22, 2026·

  ·
  2

• fix: stop GoalAccuracyMetric duplicating earlier turns inside later steps

  #3509·Wilson0511 opened Oct 22, 2026·

  ·
  1

• fix: regenerate auto-generated assessment_questions per measure

  #3508·Wilson0511 opened Oct 22, 2026·

  ·
  1

• fix(g_eval): raise when the judge scores outside the score range

  #3507·astroknot-sheep opened Oct 21, 2026·

  ·
  2

• chore: Update Integration tests

  #3506·A-Vamshi opened Oct 21, 2026Collaborator·

  ·
  1

• chore: Migrate existing integrations to use confident-trace SDK

  #3505·A-Vamshi opened Oct 21, 2026Collaborator·

  ·
  1

• fix: clear stale error/skipped in prepare_measure so reused metrics recover

  #3504·Wilson0511 opened Oct 21, 2026·

  ·
  1

• fix: apportion tokens when _split_cost meets EvaluationCost(None)

  #3503·Wilson0511 opened Oct 21, 2026·

  ·
  1