Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Repo info
Activity
    Matt MacManes
    @macmanes
    Memory or hard drive space?
    ulrichkudahl
    @ulrichkudahl
    Memory, but I am also curious to know how much space is required
    *hard drive space is required
    Matt MacManes
    @macmanes
    about use of GFF3. I also use in IGV for viz purposes. Honestly I’ve neved kept track of RAM usage for making te databases. My dammit database folder is 59G
    Camille Scott
    @camillescott
    the databases can be quite memory intensive to generate; expect ~16gb for the orthodb one
    not sure what is necessary for uniref90, though i can try to find out for you
    Camille Scott
    @camillescott
    currently dammit doesn't have an option to custom tune parameters
    however, if you're willing to get a little dirty, you can tune them yourself
    if you can find your site-packages directory, you can edit the file. something like /path/to/python/lib/python2.7/site-packages/dammit
    there's a file in there called .config.json
    one group is called "last", and has an entry for "lastdb"
    you can add extra arguments to its params entry, and they will be used in the run
    there is a bunch of info about tuning here: http://last.cbrc.jp/doc/last-tuning.html
    the only extra param included by default is -w3
    this reduces memory usage
    you can up that to -w5 or more
    there are tradeoffs, explained in the last docs i linked
    hope that helps @ulrichkudahl :)
    future version will probably allow for more tuning (ie passing in your own config file)
    Matthew J. Oldach
    @moldach

    Hello Camille,

    I hope this message finds you well.

    I'm running into an issue with dammit annotate and hoping you can help.

    I ran dammit annotate /home/ubuntu/V10.Trinity.fasta --busco-group eukaryota --database-dir /home/ubuntu/dammit/ --full and got the following error:

    --- Checking PATH for dependencies
    
              [x] HMMER
    
              [x] Infernal
    
              [x] BLAST+
    
              [x] BUSCO
    
              [x] TransDecoder
    
              [x] LAST
    
              [x] crb-blast
    
    
    --- Dependency results
    
              All dependencies satisfied!
    
    
    --- Checking for database prep (dir: /home/ubuntu/dammit)
    
              [x] download_and_gunzip:Pfam-A.hmm
    
              [x] hmmpress:Pfam-A.hmm
    
              [x] download_and_gunzip:Rfam.cm
    
              [x] cmpress:Rfam.cm
    
              [x] download_and_gunzip:aa_seq_euk.fasta
    
              [x] lastdb:aa_seq_euk.fasta.db
    
              [x] download_and_gunzip:ODB8_EukOGs_genes_ALL_levels.txt
    
              [x] download_and_untar:dammit-eukaryota
    
              [x] download_and_gunzip:uniref90.fasta
    
              [x] lastdb:uniref90.fasta.db
    
    
    --- Database results
    
              All databases prepared!
    
    
    --- Running annotate!
    
              Transcriptome file: V10.Trinity.fasta
    
              Output directory: /home/ubuntu/dammit/V10.Trinity.fasta.dammit
    
              [x] V10.Trinity.fasta
    
              [x] transcriptome_stats:V10.Trinity.fasta
    
              [x] busco:V10.Trinity.fasta-eukaryota
    
              [x] TransDecoder.LongOrfs:V10.Trinity.fasta
    
              [ ] hmmscan:longest_orfs.pep.x.Pfam-A.hmm
    
    Some tasks failed![dammit.annotate:ERROR]
    TaskError - taskid:hmmscan:longest_orfs.pep.x.Pfam-A.hmm[dammit.annotate:ERROR]
    Command error: 'cat V10.Trinity.fasta.transdecoder_dir/longest_orfs.pep | None --round-robin --pipe -L 2 -N 10000 --gnu -j 1 -a V10.Trinity.fasta.transdecoder_dir/longest_orfs.pep /usr/bin/hmmscan --cpu 1 --domtblout /dev/stdout -E 1e-05 -o V10.Trinity.fasta.transdecoder_dir/longest_orfs.pep.pfam.tbl.out /home/ubuntu/dammit/Pfam-A.hmm /dev/stdin > V10.Trinity.fasta.transdecoder_dir/longest_orfs.pep.pfam.tbl' returned 127
    [dammit.annotate:ERROR]

    As per your earlier comments with @ulrichkudahl I tried which parallel and found that it was not installed. After sudo apt-get install parallel I re-ran the dammit annotate command and a new error was returned.

    ...
    [ ] hmmscan:longest_orfs.pep.x.Pfam-A.hmm
    
    Some tasks failed![dammit.annotate:ERROR]
    TaskError - taskid:hmmscan:longest_orfs.pep.x.Pfam-A.hmm[dammit.annotate:ERROR]
    Command error: 'cat V10.Trinity.fasta.transdecoder_dir/longest_orfs.pep | /usr/bin/parallel --round-robin --pipe -L 2 -N 10000 --gnu -j 36 -a V10.Trinity.fasta.transdecoder_dir/longest_orfs.pep /usr/bin/hmmscan --cpu 1 --domtblout /dev/stdout -E 1e-05 -o V10.Trinity.fasta.transdecoder_dir/longest_orfs.pep.pfam.tbl.out /home/ubuntu/dammit/Pfam-A.hmm /dev/stdin > V10.Trinity.fasta.transdecoder_dir/longest_orfs.pep.pfam.tbl' returned 255
    [dammit.annotate:ERROR]

    Your help is greatly appreciated.
    Thanks,
    Matt

    Matt MacManes
    @macmanes
    Hey @moldach, which version of gnu parallel do you have installed? I remember that dammit requires a newer version than I initially had. I used Linuxbrew to install.
    Matthew J. Oldach
    @moldach
    Thanks for the tip @macmanes brew install parallel worked!
    b4ph0m3t
    @b4ph0m3t

    hello I am trying to run dammit over the provided sample data in a CentOS machine.
    It runs until

    
              [ ] lastal:cdna_nointrons_utrs.fa.x.orthodb.maf
    
    Some tasks failed![dammit.annotate:ERROR]
    TaskFailed - taskid:lastal:cdna_nointrons_utrs.fa.x.orthodb.maf[dammit.annotate:ERROR]
    Command failed: 'cat cdna_nointrons_utrs.fa | /usr/bin/parallel --round-robin --pipe -L 2 -N 10000 --gnu -j 1 -a cdna_nointrons_utrs.fa "/home/labgen/Software/last-658/src/lastal -F15 -D100000.0 /home/labgen/.dammit/databases/aa_seq_euk.fasta.db" > cdna_nointrons_utrs.fa.x.orthodb.maf' returned 1
    [dammit.annotate:ERROR]

    happens.
    Is it a dependency issue?
    Thanks

    b4ph0m3t
    @b4ph0m3t

    solved the error by deleting .prj file from /databases/ folder and re-running

    dammit databases --install

    there was some strange issue with lastal reading .prj file (some lines were missing)

    Matthew J. Oldach
    @moldach

    It appear like dammit is working, although maybe not a 100% since not all of the boxes were checked:

    ubuntu@ip-172-31-2-75:~/dammit$ dammit annotate /home/ubuntu/good.V10.Trinity.fasta --busco-group metazoa --database-dir /home/ubuntu/dammit/ --full
    ========================================
    dammit! a tool for easy de novo transcriptome annotation
    Camille Scott 2015
    ========================================
    
    ~~~~~~~~~~~~~~~~~~~~~~~~

    submodule: annotate

    
    --- Checking PATH for dependencies
    
              [x] HMMER
    
              [x] Infernal
    
              [x] BLAST+
    
              [x] BUSCO
    
              [x] TransDecoder
    
              [x] LAST
    
              [x] crb-blast
    
    
    --- Dependency results
    
              All dependencies satisfied!
    
    
    --- Checking for database prep (dir: /home/ubuntu/dammit)
    
              [x] download_and_gunzip:Pfam-A.hmm
    
              [x] hmmpress:Pfam-A.hmm
    
              [x] download_and_gunzip:Rfam.cm
    
              [x] cmpress:Rfam.cm
    
              [x] download_and_gunzip:aa_seq_euk.fasta
    
              [x] lastdb:aa_seq_euk.fasta.db
    
              [x] download_and_gunzip:ODB8_EukOGs_genes_ALL_levels.txt
    
              [x] download_and_untar:dammit-metazoa
    
              [x] download_and_gunzip:uniref90.fasta
    
              [x] lastdb:uniref90.fasta.db
    
    
    --- Database results
    
              All databases prepared!
    
    
    --- Running annotate!
    
              Transcriptome file: good.V10.Trinity.fasta
    
              Output directory: /home/ubuntu/dammit/good.V10.Trinity.fasta.dammit
    
              [x] good.V10.Trinity.fasta
    
              [x] transcriptome_stats:good.V10.Trinity.fasta
    
              [x] busco:good.V10.Trinity.fasta-metazoa
    
              [x] TransDecoder.LongOrfs:good.V10.Trinity.fasta
    
              [x] hmmscan:longest_orfs.pep.x.Pfam-A.hmm
    
              [x] remap_hmmer:longest_orfs.pep.pfam.tbl
    
              [ ] TransDecoder.Predict:good.V10.Trinity.fasta
    
              [x] cmscan:good.V10.Trinity.fasta.x.Rfam.cm
    
              [x] lastal:good.V10.Trinity.fasta.x.orthodb.maf
    
              [x] lastal:good.V10.Trinity.fasta.x.uniref.maf
    
              [x] maf_best_hits:good.V10.Trinity.fasta.x.orthodb.maf-
              good.V10.Trinity.fasta.x.orthodb.maf.best.csv
    
              [x] maf-gff3:good.V10.Trinity.fasta.x.orthodb.maf.gff3
    
              [x] maf_best_hits:good.V10.Trinity.fasta.x.uniref.maf-
              good.V10.Trinity.fasta.x.uniref.maf.best.csv
    
              [x] maf-gff3:good.V10.Trinity.fasta.x.uniref.maf.best.csv.gff3
    
              [x] hmmscan-gff3:good.V10.Trinity.fasta.pfam.csv.gff3
    
              [x] cmscan-gff3:good.V10.Trinity.fasta.rfam.tbl.gff3
    
              [ ] gff3-merge:good.V10.Trinity.fasta.dammit.gff3
    
              [x] fasta-annotate:good.V10.Trinity.fasta.dammit.fasta

    Does this mean TransDecoder.Predict and gff3-merge failed to run?
    Any suggestions for how to force them to complete if so? I tried running the command a number of times to no avail

    Thank you

    b4ph0m3t
    @b4ph0m3t
    from my understanding checked boxes mean that the output file of the command is already present and updated so it is not necessary to execute the related command.
    not 100% sure tho
    Adam Stuckert
    @AdamStuckert
    Hi @camillescott, thanks for a great program! I looked for information on citing dammit, but I couldn't find it on your website or the github page. Help?
    shrhops
    @shrhops
    HI @camillescott , I've run into an issue with dammit's database installation. It's the same issue mentioned here: dib-lab/dammit#229
    When I try to gunzip the files uploaded by @hoelzer, I get the error "invalid compressed data--format violated". So I'm missing orthoDB and sprot, but can't install them with the databases --install command, and the files won't unzip properly. Any ideas how to solve this?