Repository navigation
Reject negative ignore_rows and ignore_columns in CSVDocumentCleaner - #13088
Premvkmishra wants to merge 5 commits into
Conversation
|
@Premvkmishra is attempting to deploy a commit to the deepset Team on Vercel. A member of the Team first needs to authorize it. |
|
Hi @Premvkmishra, thanks a lot for your contribution! 🙏 We noticed that the Contributor License Agreement (CLA) check ( To get your PR reviewed, please sign the CLA via the link in the |
|
Hi @Premvkmishra, just a friendly reminder: this PR is still in draft because the Contributor License Agreement (CLA) hasn't been signed yet. We'd love to review your contribution! Please sign the CLA via the link in the |
|
Thanks for signing the CLA, @Premvkmishra! 🎉 This PR is now ready for review again and the reviewer has been re-assigned. |
Related Issues
Proposed Changes:
In
CSVDocumentCleaner.__init__, negative values forignore_rowsorignore_columnswere stored unvalidated. In_clean_df, pandasiloc[ignore_rows:, ignore_columns:]treated negative integers as offsets from the end of the DataFrame, leading to silent data loss without restoring ignored sections.CSVDocumentCleaner.__init__to raise aValueErrorifignore_rows < 0orignore_columns < 0, matching the style ofCSVDocumentSplitterandDocumentSplitter.:raises ValueError:entry in the component__init__docstring and updated parameter descriptions indocs-website/docs/pipeline-components/preprocessors/csvdocumentcleaner.mdx.renorelease note withfixesandupgradesections.How did you test it?
test_negative_ignore_rows_raises,test_negative_ignore_columns_raises, andtest_negative_ignore_values_raise_from_dict) intest/components/preprocessors/test_csv_document_cleaner.py.hatch run test:unit test/components/preprocessors/test_csv_document_cleaner.py(24 passed).hatch run test:types(mypy) andhatch run fmt(ruff).Notes for the reviewer
__init__before attribute assignment.0) and positive values behave unchanged.Checklist
fix:,feat:,build:,chore:,ci:,docs:,style:,refactor:,perf:,test:and added!in case the PR includes breaking changes.