Skip to content

Mismatch between code of webin-cli fastq validation and what the docs say: is there server side validation? #232

Description

@corneliusroemer

Reviewing docs vs actual code regarding fastq validation during submission with webin-cli, I noticed discrepancies between what the docs say and what the webin-cli code actually does.

Is there any validation done server side, or is the code the ground truth and docs just out of date?

This is the relevant section of docs: https://ena-docs.readthedocs.io/en/latest/submit/fileprep/reads.html#fastq-format

Differences between what the code does vs what the docs say:

  • Base alphabet is much broader than documented. The docs specify ^([ACGTNactgn.]*?)$, while the code accepts the full IUPAC set plus U and -: ACGTURYSWKMBDHVN.-. It also applies an undocumented ≤50% non-AUTCG composition rule.
  • The average-quality check is not enforced client-side. The requirement that at least 50% of reads have an average quality score of ≥30 is computed, but its enforcement is commented out.
  • “No technical reads” has no client-side check.
  • Read-name suffix handling is broader than documented. The code accepts ., :, /, and _ separators with indices 1–4, including 10x-style names, rather than only /1 and /2.
  • The client performs additional undocumented checks: duplicate read-name detection, a ≥20% pairing threshold for paired files, empty-read detection, empty-quality detection, and base-sequence versus quality-string length mismatch detection.
  • Validation is bounded rather than whole-file. Only the first 100,000 reads in quick mode or 100,000,000 reads in extended mode are inspected.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions