Skip to content

flaky: eval::inference execute_python_test_diagnostics_tests::assertion_failure_reports_nonzero_and_traceback — python exit code read as None once in two runs #3043

Description

@noahgift

Observed 2026-09-07 on the intel dev box while verifying PR #3041 (R-0b), branch agent/R-0b at 5b5116d, cargo test -p apr-cli --lib (default features):

  • run 1 (13:37Z): test result: FAILED. 7228 passed; 1 failed — the one failure:
    ---- commands::eval::inference::execute_python_test_diagnostics_tests::assertion_failure_reports_nonzero_and_traceback stdout ----
    thread '…' panicked at crates/apr-cli/src/commands/eval/inference.rs:1406
    assertion `left == right` failed
      left: None
     right: Some(1)
    
  • run 2 (13:39Z, same tree plus one unrelated commit in chat_generate_session_02.rs): 7229 passed; 0 failed.

The test asserts a spawned python assertion failure reports exit status 1; None means the child had no exit code (terminated by a signal, or never ran). Not root-caused; a rerun is not a pass (Verification Discipline #7 / I7). Filed so it is a ticket and not a silent flake; owner: eval::inference.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    pp-066PP-066 (0.66) DAG row

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions