Mostly opening this one for discussion. While upcoming nvd versions introduce some query sequence shard-parallelism to our BLAST searches, MEGABLAST and BLASTN remain extremely slow when run against Core NT. The latest versions of nvd take 12-15 hours on our big wastewater datasets, and a lot of that time is spent BLASTing.
There are a number of things we could explore to make BLAST faster or replace it with something that provides mostly equivalent results. For example:
- Simply replacing MEGABLAST and/or BLASTN with mmseqs2 for nucleotide search. - This introduces a multiple challenges, including that we now need to convert the latest core NT blast database back to a FASTA, build that into an mmseqs2 database that we maintain, and also figure out how to build an mmseqs2 taxonomy database for Core NT that gives equivalent results. It's also unclear whether mmseqs2, operating as we operate BLAST, would actually be that much faster.
- Shard BLAST executions further - This may be worth exploring, but official BLAST documentation recommends against splitting up BLAST searches, and each BLAST search pays a very heavy expense for loading big indexes like Core NT.
- Replace BLASTN with a protein search - This pathway doesn't save time on MEGABLAST, but is alluring for BLASTN. DIAMOND has some exciting new bells and whistles, and mmseqs2 is best-known for fast protein searches and even has a GPU mode we could use with our CHTC GPUs. That said, this is the biggest can of worms on the list. What do we search against? Would any available indexes be adequately sensitive for viruses? How do we know what reading frame of the six possible should be searched for wastewater fragments? Is it even worth optimizing BLASTN when MEGABLAST does most of the work for us? MEGABLAST itself surely could not be replaced with protein search.
- Something we haven't thought of yet.
This exploration is low-priority but potentially high-benefit. The slowness and memory-intensity of BLAST is arguably nvd's single greatest weakness. It's certainly the design choice that raises the most eyebrows when we reveal that we simply BLAST tens of thousands of read-length sequences!
Mostly opening this one for discussion. While upcoming nvd versions introduce some query sequence shard-parallelism to our BLAST searches, MEGABLAST and BLASTN remain extremely slow when run against Core NT. The latest versions of nvd take 12-15 hours on our big wastewater datasets, and a lot of that time is spent BLASTing.
There are a number of things we could explore to make BLAST faster or replace it with something that provides mostly equivalent results. For example:
This exploration is low-priority but potentially high-benefit. The slowness and memory-intensity of BLAST is arguably nvd's single greatest weakness. It's certainly the design choice that raises the most eyebrows when we reveal that we simply BLAST tens of thousands of read-length sequences!