Repository navigation
fix: reject unsupported DELETE LIMIT - #25005
Conversation
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #25005 +/- ##
=======================================
Coverage 82.73% 82.73%
=======================================
Files 1147 1147
Lines 449213 449208 -5
Branches 449213 449208 -5
=======================================
- Hits 371634 371632 -2
+ Misses 54929 54925 -4
- Partials 22650 22651 +1 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
|
Should this PR really close the issue ? |
|
Agreed—the issue should remain open for full bounded DELETE support. I updated the PR description so this no longer closes #24998. |
|
The only remaining failing check is |
alamb
left a comment
There was a problem hiding this comment.
Thank you for the contribution @ryux1 and @martin-g
However, I am not sure about this one
DELETE with LIMIT currently appears to plan successfully, but the table provider receives only the filters and deletes every matching row. Rejecting the unsupported clause prevents a bounded delete from silently becoming an unbounded one and matches the existing UPDATE LIMIT behavior.
Can you please provide a reproducer that shows this bug behavior? The code in this PR appears to show a limt passed to the input.
It seems like maybe your problem is somewhere else , but just forbidding such deletes and removing the planning code doesn't see right 🤔
|
I reproduced this on the exact pre-fix parent, create table t as values (1), (2), (3);
delete from t limit 1; -- count = 3
select * from t; -- no rows
create table u as values (1), (2), (3);
delete from u where column1 > 1 limit 1; -- count = 2
select * from u; -- only 1 remainsThe So the planner input does retain the limit, but it cannot reach the provider. The rejection in this PR prevents the observed unbounded delete until bounded-delete semantics and a provider API for them are implemented. |
|
@ryux1 What do you think about my suggestions ? |
Removed unsupported delete queries with limit from test file.
|
I am working on adding support for |
As explained at #24998 (comment) adding support for DELETE+LIMIT seems to be a bad idea. I am going to apply my suggestions to this PR and merge it! |
This way it will be consistent with the other errors for unsupported use cases. Also, as agreed in Discord, DELETE+LIMIT won't be supported at all in DataFusion. Co-authored-by: Martin Grigorov <martin-g@users.noreply.github.com>
## Which issue does this PR close? - Part of apache#24998; this PR adds a safe rejection until bounded DELETE is implemented. ## Rationale for this change DELETE with LIMIT currently appears to plan successfully, but the table provider receives only the filters and deletes every matching row. Rejecting the unsupported clause prevents a bounded delete from silently becoming an unbounded one and matches the existing UPDATE LIMIT behavior. ## What changes are included in this PR? - reject DELETE LIMIT during SQL planning - remove the unreachable limit construction from delete_to_plan - replace the misleading EXPLAIN snapshots with error regressions, both with and without WHERE - directly cover both planner rejection branches in the SQL integration suite ## What is the testing strategy for this PR? - cargo test -p datafusion-sqllogictest --test sqllogictests -- delete - cargo test -p datafusion-sql --lib (88 passed) - cargo test -p datafusion-sql --test sql_integration plan_delete_rejects_limit (2 passed) - cargo clippy -p datafusion-sql --all-targets -- -D warnings - cargo fmt --all -- --check ## Are there any user-facing changes? Yes. DELETE statements containing LIMIT now return a not-implemented planning error instead of accepting the limit and potentially deleting every matching row. There is no public Rust API change. Implementation and validation were completed with AI coding assistance under the account owner’s direction. --------- Co-authored-by: Andrew Lamb <andrew@nerdnetworks.org> Co-authored-by: Martin Grigorov <martin-g@users.noreply.github.com> Co-authored-by: Martin Tzvetanov Grigorov <mgrigorov@apache.org>
Which issue does this PR close?
Rationale for this change
DELETE with LIMIT currently appears to plan successfully, but the table provider receives only the filters and deletes every matching row. Rejecting the unsupported clause prevents a bounded delete from silently becoming an unbounded one and matches the existing UPDATE LIMIT behavior.
What changes are included in this PR?
What is the testing strategy for this PR?
Are there any user-facing changes?
Yes. DELETE statements containing LIMIT now return a not-implemented planning error instead of accepting the limit and potentially deleting every matching row. There is no public Rust API change.
Implementation and validation were completed with AI coding assistance under the account owner’s direction.