Skip to content

fix: array_sort doesnt respect array inputs for sort order/nulls first - #24502

Open
JSOD11 wants to merge 8 commits into
apache:mainfrom
JSOD11:jsod/array-sort-options-08-19-26
Open

JSOD11 wants to merge 8 commits into
apache:mainfrom
JSOD11:jsod/array-sort-options-08-19-26

Conversation

@JSOD11

@JSOD11 JSOD11 commented Aug 19, 2026 •

Copy link
Copy Markdown
Contributor

Which issue does this PR close?

Rationale for this change

array_sort now evaluates its optional order and nulls_order arguments per row instead of reading only the first row of each option column.

Previously, queries like this would use the first row's option value for the whole batch:

array_sort(arr, order)

So if row 1 used 'asc' and row 2 used 'desc', row 2 could still be sorted ascending.

This PR keeps the existing primitive and non-primitive sorting paths, but threads the option arrays through the sort logic so each output row uses that row's own options.

What changes are included in this PR?

Sort order can vary per row

query ?
select array_sort(arr, order)
from values
    (make_array(5, 2, 3), 'asc'),
    (make_array(5, 2, 3), 'desc'),
    (make_array(10, 20, 30), 'desc') as t(arr, order);

Before this PR, all rows used the first row's 'asc' option:

[2, 3, 5]
[2, 3, 5]
[10, 20, 30]

Now each row uses its own option:

[2, 3, 5]
[5, 3, 2]
[30, 20, 10]

NULL option values are handled per row

query ?
select array_sort(arr, sort_order)
from values
    (make_array(2, 1), 'ASC'),
    (make_array(2, 1), NULL),
    (make_array(2, 1), 'DESC') as t(arr, sort_order);

Before this PR, the NULL and 'DESC' rows still used the first row's 'ASC' option:

[1, 2]
[1, 2]
[1, 2]

Now the NULL option only affects its own row:

[1, 2]
NULL
[2, 1]

Null placement can vary per row

query ?
select array_sort(arr, order, null_order)
from values
    (make_array(5, 2, NULL, 3, NULL), 'asc', 'NULLS FIRST'),
    (make_array(NULL, 5, NULL, 3, null), 'desc', 'NULLS LAST'),
    (make_array(10, NULL, null, 20, 30, NULL), 'desc', 'NULLS LAST') as t(arr, order, null_order);

Before this PR, later rows could incorrectly reuse the first row's sort options.

Now each row gets the correct ordering:

[NULL, NULL, 2, 3, 5]
[5, 3, NULL, NULL, NULL]
[30, 20, 10, NULL, NULL, NULL]

Additional coverage

This also covers:

• primitive arrays without element nulls
• primitive arrays with element nulls
• non-primitive arrays, such as strings
• NULL order arguments
• NULL nulls_order arguments
• preserving valid empty lists as []

Are these changes tested?

cargo test -p datafusion-sqllogictest --test sqllogictests -- array_sort

Are there any user-facing changes?

Just the bug fixes described.

@github-actions github-actions Bot added the sqllogictest SQL Logic Tests (.slt) label Aug 19, 2026
@codecov-commenter

codecov-commenter commented Aug 19, 2026 •

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 87.23404% with 18 lines in your changes missing coverage. Please review.
✅ Project coverage is 82.68%. Comparing base (bd86190) to head (c9881ce).

Files with missing lines Patch % Lines
datafusion/functions-nested/src/sort.rs 87.23% 8 Missing and 10 partials ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main   #24502      +/-   ##
==========================================
- Coverage   82.69%   82.68%   -0.01%     
==========================================
  Files        1147     1147              
  Lines      447204   447300      +96     
  Branches   447204   447300      +96     
==========================================
+ Hits       369793   369865      +72     
- Misses      55002    55014      +12     
- Partials    22409    22421      +12     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@github-actions github-actions Bot added the functions Changes to functions implementation label Aug 28, 2026
@JSOD11
JSOD11 marked this pull request as ready for review October 5, 2026 21:54
@JSOD11

JSOD11 commented Oct 6, 2026

Copy link
Copy Markdown
Contributor Author

cc @Jefffrey for review when you get a moment

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

functions Changes to functions implementation sqllogictest SQL Logic Tests (.slt)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

array_sort doesnt respect array inputs for sort order/nulls first

2 participants