Release PR - #115
Open
LuisHeinzlmeier wants to merge 50 commits into
Open
Conversation
* hasheddrops inserted and they run in the pipeline * insert local preprocessing module for htodemux and multiseq (not tested yet) * adding htodemux and multiseq to the pipeline * small improvements * add parameters of modules also to the pipeline * address * run pre-commit for modules/local/preprocessing_for_htodemux_multiseq * add htodemux visualization (not tested yet) * visualization added to the pipeline (works locally) * add visualization params to config files * change strings to type boolean for htodemux visualization * improve visualization * change data types for preprocessing * change data types of nf-core modules if possible * Prettier * Pre-commit * Fix nf-core modules not matching remote * Fix PULL_REQUEST_TEMPLATE issue based on this thread: https://nfcore.slack.com/archives/CQY2U5QU9/p1747475193145499 * Solve remaining prettier issues * One more prettier fix :D * Revert "Fix nf-core modules not matching remote" This reverts commit ac3ed24. * Make test profile work * Patch modules * Prettier * Gitignore test intermediates * Add pipeline test snapshot * Make sure hash diffs are visible * Ignore some more test snaps * Enable new hash-based tools * Set prefix to prevent file name collisions * nft-ignore some outputs --------- Co-authored-by: LuisHeinzlmeier <luis.heinzlmeier@tum.de>
* first steps * current changes (not tested yet) * save changes * htodemux and multiseq in hash_summary.py (without mudata and anndata) and debugging of ch_hashing_summary * solve summary input with join * pipeline finally calls the script * empty inputs solved * add all inputs to main.nf (empty inputs still work) * update htodemux in script * script passed and produced results * correct results * solve incorrect merging * add demuxem (runs but merging to the summary table is not working) * remove the duplicate variable name * add demuxem to summary (has more barcodes than htodemux and multiseq) * add hasheddrops * add gmmdemux to summary * first steps to improve hasheddrops outputs * fix hasheddrops output * fix using unknown as value * remove comments * adding hashsolo doesn't work (cannot access the correct cell_hashing_columns) * add bff summary function * bff works * join into barcodes from raw hto file to adress demuxem having more output barcodes that input barcodes * pipeline pass with class Arguments * move testing to Argument class * restructure hash_summary.py * mudata and anndata outputs are created * hashsolo works * replace manual str with args variable * update hasheddrops * update modules and improve linting * Skip “prettier” failing – covered in #69 * clean up code 1 * clean code 2 (remove hard-coded parameters) * clean code 3 (hash_sumary.py) * update modules bff and hashsolo * update schema * remove properties created by nf-core pipelines schema build --no-prompts * structure hashing paramters of nextflow.config * update hasheddrops (removed one TODO in hash summary, hardcoding of mapping from id to hash) * remove hardcoded hash_list * remove linting error * use left join for classification and small improvements for summary csv * remove debugging outputs * Fix demuxem MPLCONFIGDIR * Add hash_summary environment variables * Comment out arg printing in hash_summary * Update demuxem patch file * Prettier * update snapshot --------- Co-authored-by: Nico Trummer <nictru32@gmail.com>
* Template update for nf-core/tools version 3.3.2 * Template update for nf-core/tools version 3.3.2 * Update multiQC module * Fix vireo_cell_range issue
* Template update for nf-core/tools version 3.3.2 * Template update for nf-core/tools version 3.3.2 * Template update for nf-core/tools version 3.3.2 * Fix some linting problems * Use github runners instead of self-hosted Updated runner configuration to use 'ubuntu-latest' for nf-test jobs. * Fix runner configuration in nf-test workflow * Template update for nf-core/tools version 3.4.1 * Fix logo images
* save first steps * join outputs * gene_summary merges results for vireo, demuxlet and freemuxlet * merge on Barcode * all modules work for gene_summary but demuxlet has still empty values after merging * prepare donor matching * try understanding donor match * debugging donor match * donor_match produces something * donor_match produces something * donor_match seems to produce correct output files now, but it still needs to be tested with properly joined data * pipeline is running with new input data and donor_match produces correct output * first updates to souporcell * update souporcell * final versions.yml * save anndata/mudata, output before joining and left join to rna barcodes * couln't fix demuxEM error, opened an issue at developer's github * gene/hash_summary: save mudata/anndata, print results overview fuction and clean code * include all methods in donor_match and update bff() in hash_summary * improve the readability of hadge.nf * use fasta from iGenomes for souporcell but not fully tested (see #77) * remove prints and linting errors * update snapshot * Merge origin/dev * update souporcell * Fix image files * update snapshot * update snapshot 2 * fasta updates * update snapshot with 3 modules each (hash/gene) * update fasta description * update snapshot * replace set with list to keep the same order * snapshot from codespace * add souporcell outputs to .nftignore * Add trailing newline to .nftignore * fix end of file * add freemuxlet/*/*.clust1.vcf.gz to .nftignore * Incorporate review comments * update snapshot --------- Co-authored-by: nictru <nictru32@gmail.com>
* implementation of find_variants * subset gt_donors in mode 'donor_match' and update nextflow schema * clean find_variants * clean subset_gt_donors (update env) * clean donor_match * fix donor_match * add versions.yml (closes #87) * remove own TODOs and add error message for HTO names (closes #75, closes #73) * remove rename_genes_to_features (not used anymore) * add an own anndata/mudata module and remove it from summary modules (closes #83) * code and dependency cleanup of summary scripts * code and dependency cleanup of create_anndata_mudata.py * cleanup outdir * add a comment * closes #76 * add method overview output (closes #86) * add warning method mentioned in #77 * add test for genetic mode (see #95) * add tests for hashing and donor_match mode (see #95) * update nf-core modules vireo and souporcell (closes #94) * update input paths and add new tests to the github workflow (closes #95) * remove duplicate parameter * fix linting errors * update snapshots for rescue and donor_match * codespace snapshot default.nf.test * fix create anndata/mudata issue (hto==null in genetic mode) * add contributors * change defaultBranch * update contributor information * add another contributor * Add intro page * add rescue and genetic mode to the documentation * update samplesheet and add usage documentation * restructure steps of the pipeline * outputs for hashing, genetic and summary * close details * place output files below the headline * update and describe donor_match and find_variant outputs * push pipeline png * add citation of the relevant tools and adjust outdir * add seo's orcid * move checking the params and samplesheet inputs to 'utils_nfcore_hadge_pipeline' * move the validation of hto_names to utils_nfcore_hadge_pipeline with a function * add ext.prefix of JOIN_RESULTS with regex * add specific profiles to the tests and test vcf files * add nft-vcf plugin * update snapshot for rescue and donor match mode * update profile name of donor_match * update default snap with codespaces * ingore cluster_genotypes.vcf, sort AnnData columns and mention test profiles in docs * bash script to update all test snapshots * update snapshots with codespaces * update snapshot from test_genetic * update ignore souporcell vcf * add a seed for reproducible t-SNE and PCA * add extract hashes env.yml * nf-core pipelines lint --fix rocrate_readme_sync * nf-core pipelines lint --dir . --fix files_unchanged * Revert "nf-core pipelines lint --dir . --fix files_unchanged" This reverts commit 9bdae0b. * add comment for ignoring cluster_genotypes.vcf * Samplesheet Input Requirements by Module * small fix in docs * remove rna_matrix from genetic mode as it is not needed * simplify preprocessing channels * clean up formatting * remove barcodes from donor_match because it was already joined on barcodes.tsv in gene_summary.py * simplify tuple nesting issue in ch_donor_match * update docs * test new snapshots * update donor_match snapshot * fix donor_match snapshot issue * test new jpeg image * remove the random seed from htodemux (snapshot differences are due to OS variations) * update snapshot hashing * update rescue snapshot * update genetic snapshot * only update *_tSNE_htodemux.jpeg snapshot and date (multiqc throws an error in codespaces) * incorporate feedback * nf-core pipelines lint --fix rocrate_readme_sync * own line for each sentence
* Template update for nf-core/tools version 3.3.2 * Template update for nf-core/tools version 3.3.2 * Template update for nf-core/tools version 3.5.2 * Update multiqc
* add larger nf-core runners * add zenodo and citation * update citation * add missing bam files to docs * add random comment to trigger CI tests * remove nf-core TODO's * nf-core pipelines lint --fix rocrate_readme_sync * use BibTeX for citations * try another style for citations * try another style for citations (pre-commit) * change dropdown syntax to html * adjust syntax * add detailed docs for demuxlet * update format * update format 2 * Fix Numba caching error in container by setting NUMBA_CACHE_DIR * setup test_full * fix hash modules for full_test compatibility * remove demuxem TODOs * update tests and snapshots * current test setup * reduce computing/storage resources of test and test_full * closes #107 * update nextflow_schema.json * update docs * update test_full input * nf-core pipelines lint --fix rocrate_readme_sync * update snapshots * remove outdir from config files * update snapshots * increase volume of self-hosted runners * Remove aws.client.anonymous from global nextflow.config in concordance with template * fine tune vireo configurations * increase runner volume and update test snapshot with all tools * set plp as path input * use plp dir as input to fix aws error * new try * new try 2 * remove time constraint for souporcell * increase time for souporcell * update readme --------- Co-authored-by: Nico Trummer <nictru32@gmail.com>
* Template update for nf-core/tools version 3.3.2 * Template update for nf-core/tools version 3.3.2 * Template update for nf-core/tools version 4.0.0 * Template update for nf-core/tools version 4.0.2 * Fix nextflow linting errors * Update multiQC module * Fix nextflow lint warnings and remove redundant samplesheet input parameter. Use explicit closure parameters, prefix unused bindings, and drop dead HTODEMUX channel assignments. PIPELINE_INITIALISATION reads params.input directly, so the take input is no longer needed. * Use elvis operator for simplification * Update container config files --------- Co-authored-by: Nico Trummer <nictru32@gmail.com>
…cable (#110) * Update nf-core modules and fix bam_qc for new samtools API. Refresh installed modules from nf-core/modules, adapt BAM_QC to the updated samtools view/sort inputs, and regenerate module patch files. Drop obsolete hasheddrops and multiseqdemux patches that no longer differ from upstream. * Drop manual version mixing for nf-core topic outputs. Rely on the versions topic collected before MultiQC instead of passing samtools, umitools, untar, csvtk, bff, and hasheddrops versions through subworkflows. * Emit local module versions via topic and keep patched nf-core mixing. Local modules publish to the versions topic so subworkflows no longer mix their outputs manually, while patched nf-core modules without topic support continue to use traditional ch_versions plumbing. * Restore Harshil alignment on local module emit statements. Re-align output emit columns after adding topic: versions during the topics migration. * Fix compatibility issues * Simplify hto null branching * Fix hto branch logic while preserving dev hash channel behaviour. Use a side-channel branch so ch_hto stays intact for joins, and mix is_null rows with EXTRACT_HASHES hashes rather than not_null hto paths. * Update nf-test snapshots for topic-based version collection. Refresh pipeline software versions YAML after local modules publish to the versions topic. * Ignore non-deterministic tSNE plots in nf-test snapshots. tSNE htodemux JPEGs vary across CI and local Docker runs; exclude them from stable-path md5 checks.
Correct meta output types in bff/meta.yml after the nf-core module update and sync modules.json.
Co-authored-by: nf-core-bot <core@nf-co.re>
* Template update for nf-core/tools version 3.3.2 * Template update for nf-core/tools version 3.3.2 * Template update for nf-core/tools version 4.0.0 * Template update for nf-core/tools version 4.0.2 * Template update for nf-core/tools version 4.0.3 * resolve missing merge conflicts * fix linting errors * update mulitqc * nf-core pipelines lint --dir /workspaces/hadge --fix container_configs * Fix conda container config file linting * Align linting workflow * use stable_name in test * use input instead of params.input --------- Co-authored-by: Nico Trummer <nictru32@gmail.com> Co-authored-by: LuisHeinzlmeier <luis.heinzlmeier@tum.de>
* bump-version * update changelog * resolve lint errors/warnings * udpate snapshots
Member
|
Warning Newer version of the nf-core template is available. Your pipeline is using an old version of the nf-core template: 4.0.3. For more documentation on how to update your pipeline, please see the nf-core documentation and Synchronisation documentation. |
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.