Skip to content

Feature/sanitize jsons - #1535

Open
allanbolipata wants to merge 6 commits into
developfrom
feature/sanitize_jsons
Open

Feature/sanitize jsons#1535
allanbolipata wants to merge 6 commits into
developfrom
feature/sanitize_jsons

Conversation

@allanbolipata

@allanbolipata allanbolipata commented Jul 10, 2026

Copy link
Copy Markdown
Collaborator

Let's have everyone sign off on this before merging.

This contains small PR from #1534, so let's merge that first.

Summary

  • Replaced patient IDs, project IDs, person names, and email addresses across ~130 fixture JSON files, test Python files, and operator reference JSON files with clearly dummy values
  • Renamed fixture files on disk to match sanitized project IDs (e.g., 100XX_D99990_D)
  • All 286 tests pass (0 failures, 0 errors, 2 pre-existing skips)

What was sanitized

Category Example (before → after)
Patient IDs C-123456C-PPPPP3
Project/Request IDs 100XX_D99990_D
Email addresses *@mskcc.org*@example.org
Person names John SmithLab Head B

What was preserved

  • Public identifiers: MSKCC sequencing center, gene panel names (IMPACT468, HemePACT_v4), public GitHub URLs
  • Sanitized IDs respect format constraints enforced by PRIMARY_ID_REGEX (e.g., single-letter project suffixes must be [B-Z], not A)
  • Hyphen/underscore structure in sample names preserved for operator parsing compatibility (e.g., DONOR00-TP keeps the hyphen that operators split on)

Key fixes during sanitization

  • patient.cmo fields in DMP metadata use bare patient IDs without the C- prefix — these required separate replacement from the C-XXXXX form
  • Fixture filenames needed renaming in addition to content replacement (Django loaddata resolves fixtures by filename)
  • Added BEAGLE_FASTQ_DEFAULT_LOCATION_PREFIX and BEAGLE_FASTQ_IRIS_LOCATION_PREFIX to .env.local to fix pre-existing test failures unrelated to sanitization

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants