Define how jq output streams map to runtime-expression values

[maver1ck]

What would you like to be added?

Specify how an OWS runtime must convert jq’s output stream into the single value expected by a runtime-expression field.

The specification requires jq support and describes an expression’s “result,” but jq filters can emit zero, one, or many values. Current implementations therefore make incompatible choices.

The clarification should cover:

• Zero emitted values.
• Multiple emitted values.
• The distinction between no output and JSON null.
• Whether evaluation stops after selecting a result.
• Whether an error occurring after an emitted value must still fail evaluation.

Proposal(s)

Define runtime-expression evaluation as equivalent to:

first(<expression>)

with these rules:

jq behavior	OWS runtime-expression result
Emits one value	Return that value
Emits multiple values	Return the first value and stop evaluation
Emits no values	Raise the standard expression error with status 400
Emits JSON null	Return JSON null
Emits a value followed by an error	Return the first value; the later error is not evaluated

Examples:

Input: [1, 2, 3]
Expression: .[]
Result: 1

Expression: empty
Result: expression error

Expression: null
Result: null

Expression: 1, error("boom")
Result: 1

This provides a stable single-value contract without silently changing a scalar-producing expression into an array. It also preserves jq’s distinction between empty and null.

Add corresponding CTK scenarios so implementations produce portable results.

Alternative(s)

1. Require exactly one emitted value and raise an expression error for zero or multiple values.

2. Collect multiple values into an array and convert zero values to null. This resembles the Java SDK reference implementation, but implicitly changes the result type and makes empty indistinguishable from an emitted JSON null.

3. Preserve jq’s output stream. This would require defining stream handling independently for every OWS field that consumes a runtime expression.

Additional info

The OWS runtime-expression section requires jq support but does not define output cardinality.

The jq manual defines filters as generators: .[] can produce multiple results, while empty produces none.

The Java SDK’s JQExpression returns one emission directly, collects subsequent emissions into an array, and returns null when nothing is emitted. Other jq-based runtimes consume only the first emission and treat no emission as an error.

Without a normative rule and CTK coverage, the same valid workflow can produce different output or failure behavior across conforming runtimes.

Community Notes

• Please vote by adding a 👍 reaction to the feature to help us prioritize.
• If you are interested in working on this feature, please leave a comment.

[ricardozanini]

Thank you for raising this — the portability concern is valid and the analysis is thorough.

However, we don't think imposing implicit first() semantics is the right approach. Here's our reasoning:

The spec should not override jq's natural behavior

jq's fundamental model is that every filter is a generator that can emit zero, one, or many values. This is well-documented and well-understood by jq users. If a workflow author writes .[] on [1, 2, 3], they should expect three values — not a silent truncation to 1.

If the author wants a single value, jq already provides the tools: first(.[]) , .[0], limit(1; .[]). The expression language itself is the right place to control cardinality, not the runtime.

Industry alignment

We surveyed how major tools and workflow engines handle multi-output expressions:

Approach	Adopted by
Collect all outputs into array/list	jackson-jq (used by our Java SDK), pyjq, AWS Step Functions (JSONPath), FEEL/BPMN (Camunda), Terraform, Argo Workflows
Stream/iterator (caller collects)	libjq, gojq, jaq, jq CLI
Single-valued by design	GitHub Actions, Terraform data sources

No mainstream tool imposes implicit first() semantics. The closest is pyjq, which offers first() as an opt-in API alongside all(). Terraform's one() function errors on multiple results rather than silently picking one.

The "collect all into array" pattern is the dominant industry convention for embedded jq/expression contexts. This is what the Java SDK's JQExpression already does, and it aligns with how jackson-jq, AWS Step Functions, and FEEL all behave.

Our position

• Multiple values: collected into an array. The author chose a multi-output filter and should expect an array downstream. This preserves all information and makes cardinality explicit.
• Zero values / empty: returns null. This is a valid outcome, not an error — it doesn't break the data flow processing downstream.
• null: returns null.
• Single value: returned as-is.

The workflow author owns the expression and its output shape. The spec gives users the ability to decide how their workflow should behave, and that's the correct design choice.

What we will do

We agree that the current documentation should be clearer about how multi-output jq filters are handled. We'll:

1. Enhance the runtime expression documentation in dsl.md to clarify output cardinality behavior
2. Add CTK scenarios covering zero, single, and multi-output expressions

This is a documentation clarification, not an API or behavioral change.

Waiting for others to give a thumbs up and I'll open a PR.