Module to download CDD databases #9582
Conversation
vagkaratzas
left a comment
vagkaratzas
left a comment
There was a problem hiding this comment.
Leaving some comments for now. Will have to figure out if we prefer to have such a module here, rather than using download and decompress modules directly in pipelines. My vote is yes, since this can be a nice wrapper for preparing the cdd blast dbs for ease of use.
| wget https://ftp.ncbi.nih.gov/pub/mmdb/cdd/cddid.tbl.gz -O ./data/cddid.tbl.gz && gzip -d ./data/cddid.tbl.gz | ||
| wget https://ftp.ncbi.nih.gov/pub/mmdb/cdd/cdtrack.txt -O ./data/cdtrack.txt | ||
| wget https://ftp.ncbi.nih.gov/pub/mmdb/cdd/family_superfamily_links -O ./data/family_superfamily_links | ||
| wget https://ftp.ncbi.nih.gov/pub/mmdb/cdd/cddannot.dat.gz -O ./data/cddannot.dat.gz && gzip -d ./data/cddannot.dat.gz | ||
| wget https://ftp.ncbi.nih.gov/pub/mmdb/cdd/cddannot_generic.dat.gz -O ./data/cddannot_generic.dat.gz && gzip -d ./data/cddannot_generic.dat.gz | ||
| wget https://ftp.ncbi.nih.gov/pub/mmdb/cdd/bitscore_specific.txt -O ./data/bitscore_specific.txt |
There was a problem hiding this comment.
Are these necessary to be downloaded? Next to every db every time?
There was a problem hiding this comment.
One set of them is enough for any database. These are "universal"
There was a problem hiding this comment.
I am wondering how this module will handle multiple dbs. You'll have to call it as many times as dbs you want, and then all these files will be downloaded again, and then you'll have multiple copies of the cdd_databases folder.
I am not familiar with the tree structure, but what's the plan for moving everything in a single cdd_databases folder to be used by the pipeline for multiple dbs? Or is the idea to only use one db at a time?
There was a problem hiding this comment.
The rpsblast will run with one databse at the time. The Cdd database contains all the rest of the DBs. In case someone want to run the analysis versus all the databases, should choose Cdd or a the corresponding subset. The idea is to do the check if exists in the calling workflow and run the downloader only when the database is not present. I am not sure how to tell the pipeline to fix the location of the downloaded DB. We probably should do something like this:
https://nf-co.re/funcscan/3.0.0/parameters/#database-downloading-options
There was a problem hiding this comment.
I will add an input for the downloading dir and checks for existing dbs
There was a problem hiding this comment.
I have added handlers to deal with existing db folders
This module was developed to download and process the CDD databases necessary to run rpsblast from the FTP site of the NCBI.
PR checklist
versions.ymlfile.labelnf-core modules test blast/cdddownloader --profile docker