Reviewing docs vs actual code regarding fastq validation during submission with webin-cli, I noticed discrepancies between what the docs say and what the webin-cli code actually does.
Is there any validation done server side, or is the code the ground truth and docs just out of date?
This is the relevant section of docs: https://ena-docs.readthedocs.io/en/latest/submit/fileprep/reads.html#fastq-format
Differences between what the code does vs what the docs say:
- Base alphabet is much broader than documented. The docs specify
^([ACGTNactgn.]*?)$, while the code accepts the full IUPAC set plus U and -: ACGTURYSWKMBDHVN.-. It also applies an undocumented ≤50% non-AUTCG composition rule.
- The average-quality check is not enforced client-side. The requirement that at least
50% of reads have an average quality score of ≥30 is computed, but its enforcement is commented out.
- “No technical reads” has no client-side check.
- Read-name suffix handling is broader than documented. The code accepts
., :, /, and _ separators with indices 1–4, including 10x-style names, rather than only /1 and /2.
- The client performs additional undocumented checks: duplicate read-name detection, a
≥20% pairing threshold for paired files, empty-read detection, empty-quality detection, and base-sequence versus quality-string length mismatch detection.
- Validation is bounded rather than whole-file. Only the first
100,000 reads in quick mode or 100,000,000 reads in extended mode are inspected.
Reviewing docs vs actual code regarding fastq validation during submission with webin-cli, I noticed discrepancies between what the docs say and what the webin-cli code actually does.
Is there any validation done server side, or is the code the ground truth and docs just out of date?
This is the relevant section of docs: https://ena-docs.readthedocs.io/en/latest/submit/fileprep/reads.html#fastq-format
Differences between what the code does vs what the docs say:
^([ACGTNactgn.]*?)$, while the code accepts the full IUPAC set plusUand-:ACGTURYSWKMBDHVN.-. It also applies an undocumented≤50%non-AUTCGcomposition rule.50%of reads have an average quality score of≥30is computed, but its enforcement is commented out..,:,/, and_separators with indices1–4, including 10x-style names, rather than only/1and/2.≥20%pairing threshold for paired files, empty-read detection, empty-quality detection, and base-sequence versus quality-string length mismatch detection.100,000reads in quick mode or100,000,000reads in extended mode are inspected.