Pipeline.run() passes the inputs of a variadic socket in a fixed order. Pipeline.run_async() and run_async_generator() pass them in the order the sending components finish. When the senders run concurrently, the order can differ between runs, platforms and Python versions.
Where it comes from
In async mode, each component runs as its own task, and sync components run in a thread via asyncio.to_thread. The task's _runner writes the outputs to the downstream inputs as soon as its component finishes (pipeline.py#L780-L796).
A variadic socket such as DocumentJoiner.documents, AnswerJoiner.answers or a custom Variadic[...] input therefore receives its lists in completion order. This is visible to users:
DocumentJoiner with the default concatenate mode and documents without scores returns documents in that input order.
- Any component that relies on the position of a variadic input gets a run-dependent result.
How it surfaced
While verifying Python 3.15 support in #13054, three async scenarios in test/core/pipeline/features/test_run.py failed on every ubuntu and Windows CI run with 3.15 because of swapped orders:
- "multiple branches that merge into a component with a single variadic input"
- "linear with conditional branching and multiple joins"
- "file conversion pipeline with two joiners"
I then added a random 0–10 ms delay before each component run. Over 40 seeds, three more scenarios became order-dependent:
- "variadic component that receives partial inputs"
- "variadic component that receives partial inputs in a different order"
- "answer joiner variadic component"
#13054 relaxes those expectations with AnyOrder, as #10634 did for Python 3.14. That makes the tests pass but leaves the behavior undefined.
Question
Should run_async() guarantee the same variadic input order as run()?
- If yes: the outputs written to a variadic socket could be placed in a fixed order, for example by sender, instead of being appended on arrival. The
AnyOrder relaxations in test_run.py could then be reverted.
- If no: the
Pipeline.run_async() docs should state that the order of variadic inputs is not guaranteed, so users don't rely on it, for example for joiner outputs without scores.
👋 Hello there! This issue will be handled internally and isn't open for external contributions. If you'd like to contribute, please take a look at issues labeled contributions welcome or good first issue. We'd really appreciate it!
Pipeline.run()passes the inputs of a variadic socket in a fixed order.Pipeline.run_async()andrun_async_generator()pass them in the order the sending components finish. When the senders run concurrently, the order can differ between runs, platforms and Python versions.Where it comes from
In async mode, each component runs as its own task, and sync components run in a thread via
asyncio.to_thread. The task's_runnerwrites the outputs to the downstream inputs as soon as its component finishes (pipeline.py#L780-L796).A variadic socket such as
DocumentJoiner.documents,AnswerJoiner.answersor a customVariadic[...]input therefore receives its lists in completion order. This is visible to users:DocumentJoinerwith the defaultconcatenatemode and documents without scores returns documents in that input order.How it surfaced
While verifying Python 3.15 support in #13054, three async scenarios in
test/core/pipeline/features/test_run.pyfailed on every ubuntu and Windows CI run with 3.15 because of swapped orders:I then added a random 0–10 ms delay before each component run. Over 40 seeds, three more scenarios became order-dependent:
#13054 relaxes those expectations with
AnyOrder, as #10634 did for Python 3.14. That makes the tests pass but leaves the behavior undefined.Question
Should
run_async()guarantee the same variadic input order asrun()?AnyOrderrelaxations intest_run.pycould then be reverted.Pipeline.run_async()docs should state that the order of variadic inputs is not guaranteed, so users don't rely on it, for example for joiner outputs without scores.👋 Hello there! This issue will be handled internally and isn't open for external contributions. If you'd like to contribute, please take a look at issues labeled contributions welcome or good first issue. We'd really appreciate it!