Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Activity
    rebeccarosetm
    @rebeccarosetm
    Ah ok, thanks for that suggestion!
    1 reply
    hariszaf
    @hariszaf
    :thumbsup:
    Matthias Obst
    @biomobst
    @hariszaf we now bought a PC and installed Docker. Almost got everything to run, but one error message:
    root@f71f06b4874d:/home# ./PEMA_v1.bds
    Picked up JAVA_TOOL_OPTIONS: -XX:+UseContainerSupport
    Fatal error: ./PEMA_v1.bds, line 68. Directory '/mnt/analysis/mydata' does not exists
    do you recognise the problem?
    hariszaf
    @hariszaf
    Hi @Matthias!
    Do you have your raw data in a folder called mydata inside your analysis directory?
    Matthias Obst
    @biomobst
    yes, the folder is on the Desktop
    it contains fastq files
    hariszaf
    @hariszaf
    So you have something like this
    /Users/matthias/Desktop/my_analysis/
    and there you have your parameters file and a folder called
    mydata
    is that right?
    Matthias Obst
    @biomobst
    Desktop/analysis_directory/mydata/
    the parameters file is under analysis_directory, and the fastq file is under mydata
    hariszaf
    @hariszaf
    and you run
    docker run -it -v /Desktop/analysis_directory:/mnt/analysis
    I am asking because you need to mount the whole analysis_directory not just your data.
    Matthias Obst
    @biomobst

    i hadnavigates to the desktop therefore i only had "/anlaysis_directory" as path. Now i copied your command and get this: "docker run" requires at least 1 argument.
    See 'docker run --help'.

    Usage: docker run [OPTIONS] IMAGE [COMMAND] [ARG...]

    Run a command in a new container

    hariszaf
    @hariszaf
    docker run -it -v /Desktop/analysis_directory:/mnt/analysis hariszaf/pema
    if you run this what do you get?
    Matthias Obst
    @biomobst
    now i am in, i get: root@3069a1f860d3:/home#
    hariszaf
    @hariszaf
    ok. could you run
    ls -la /mnt
    and then
    ls -la /mnt/analysis
    Matthias Obst
    @biomobst
    root@3069a1f860d3:/home# ls -la /mnt
    total 12
    drwxr-xr-x 1 root root 4096 May 24 16:19 .
    drwxr-xr-x 1 root root 4096 Jun 17 12:41 ..
    drwxr-xr-x 2 root root 40 Jun 17 12:41 analysis
    drwxr-xr-x 2 root root 4096 May 24 16:19 databases
    hariszaf
    @hariszaf
    :thumbsup:
    Matthias Obst
    @biomobst
    total 4
    drwxr-xr-x 2 root root 40 Jun 17 12:41 .
    drwxr-xr-x 1 root root 4096 May 24 16:19 ..
    hariszaf
    @hariszaf
    this is from the second command???
    Matthias Obst
    @biomobst
    yes
    hariszaf
    @hariszaf
    ok please run exit.
    then go to your desktop, run pwd and please tell me the exact path you get
    Matthias Obst
    @biomobst
    C:\Users\xobsma\Desktop>pwd
    'pwd' is not recognized as an internal or external command,
    operable program or batch file.
    18 replies
    Matthias Obst
    @biomobst
    Hi Haris, I am back. I found the problem with the Mac installation. You have to add a "~" before the path to the analysis directory in the github tutorial
    '''docker run -it -v ~/Desktop/analysis_directory/:/mnt/analysis hariszaf/pema''',
    ...and i got another error when executing PEMA_v1.bds:
    Fatal error: ./PEMA_v1.bds, line 161, pos 1. Task/s failed.
    29 replies
    hariszaf
    @hariszaf
    Hi @/all ! FYI I am about to integrate the convertIllumunaRawDataToEnaFormat.sh script in the main PEMA code. It seems like an unessecary burden! If you have any other suggestions in terms of improvement of PEMA there are more than welcome!
    ajiwahyu
    @ajiwahyu
    @hariszaf . I got this error and i feel that this has something to do with the parameter.tsv file

    all the first steps are done! clustering is about to start!
    Fatal error: /home/PEMA_v1.bds, line 557, pos 84. Map 'paramsOfTable' does not have key 'clusteringAlgoForCOI_ITS'.
    PEMA_v1.bds, line 557 : if ( paramsOfTable{'clusteringAlgoFor16S_18SrRNA'} == "algo_swarm" || paramsOfTable{'clusteringAlgoForCOI_ITS'} == " algo_SWARM") {

    ProgramCounter.pop(100): Node ID does not match!
    PC : PC: size 4 / 0, nodes: 1422 -> 4690 -> 4691 -> 4698
    Node Id : 4699
    bdsNode Id : 4698
    ProgramCounter.pop(100): Node ID does not match!
    PC : PC: size 3 / 0, nodes: 1422 -> 4690 -> 4691
    Node Id : 4698
    bdsNode Id : 4691
    ProgramCounter.pop(100): Node ID does not match!
    PC : PC: size 2 / 0, nodes: 1422 -> 4690
    Node Id : 4691
    bdsNode Id : 4690
    ProgramCounter.pop(100): Node ID does not match!
    PC : PC: size 1 / 0, nodes: 1422
    Node Id : 4690
    bdsNode Id : 1422

    i ahve checked the
    ajiwahyu
    @ajiwahyu
    i have checked the clusteringAlgoForCOI_ITS and set it into algo_CROP, but the error still persist. Also I am not really sure if I have to comment (#) the 'clusteringAkgofor_16S_18SrRNa part as I am using only COI for the moment . Thanks @hariszaf . So sorry for the trouble...
    hariszaf
    @hariszaf
    Hi @ajiwahyu ! Thank you for the feedback! You should not comment out any of the parameters! PEMA needs to know that it has access to all of them even if you do not use them. So, you could check that all the parameters are on and make sure that your parameters file is tab separated. I would suggest to go clusteringAlgoFor16S_18SrRNA algo_vsearch and clusteringAlgoForCOI_ITS algo_SWARM for a first run; especially if you have many samples as the CROP algorithm will take quite a long!
    It is essential to have a tab seperated file.. This is a quite common error but for the time being we cannot avoid this format.. :/
    ajiwahyu
    @ajiwahyu
    Thanks @hariszaf. I just tried your suggestion and the pipeline seemed working with SWARM but spit the following error when I used CROP. Also a question with Database, for COI what database are you using?. Is there any setting that I need to change if I want to use different data base?.Also for setting a targetlength at the trimmomatic part, how should one decide it?. IAm using COI, with COI primer dgHCO2198, expecting a final merge reads at 300-330 bp.

    This is the error i got when using CROP COI; cropPath is: /mnt/analysis/16S_final_test/7.gene_dependent/gene_COI/CROP
    /home/scripts/croParameters.sh: line 49: cd: tools/: No such file or directory
    toolsPath is: /mnt/analysis
    /home/scripts/croParameters.sh: line 57: /mnt/analysis/CROP/CROPLinux: No such file or directory
    cp: cannot stat 'CROP_output.cluster.list': No such file or directory
    sed: can't read CROP_output.cluster.list: No such file or directory
    split: cannot open 'CROP_output.cluster.list' for reading: No such file or directory
    rm: cannot remove 'CROP_output.cluster.list': No such file or directory

    sed: can't read /mnt/analysis/16S_final_test/7.gene_dependent/gene_COI/CROP/abundances/x: No such file or directory
    rm: cannot remove '/mnt/analysis/16S_final_test/7.gene_dependent/gene_COI/CROP/abundances/x
    .abundances': No such file or directory
    cat: 'x': No such file or directory
    rm: cannot remove 'x
    ': No such file or directory
    tail: cannot open 'CROP_output.cluster' for reading: No such file or directory
    cp: cannot stat 'CROP_output.cluster.list': No such file or directory
    sed: can't read SWARM.swarms: No such file or directory
    Fatal error: /home/PEMA_v1.bds, line 971, pos 3. Exec failed.
    Exit value : 2
    Command : bash /home/scripts/cropOutputTranformations.sh
    PEMA_v1.bds, line 804 : if ( paramsOfTable{'gene'} == 'gene_COI' || paramsOfTable{'gene'} == "gene_ITS") {
    PEMA_v1.bds, line 927 : if ( paramsOfTable{'clusteringAlgoForCOI_ITS'} == 'algo_CROP' ) {
    PEMA_v1.bds, line 971 : sys bash $path/scripts/cropOutputTranformations.sh

    hariszaf
    @hariszaf
    Hi @ajiwahyu! Thanks a lot for this! There was a bug on the script that calculates the CROP parameters. I am about to rebuild an image fixing that.
    Regarding the COI marker gene, PEMA uses the MIDORI database, which you may find here.
    For the time being you cannot use an in-house reference database, however that is going to be included in the next version of PEMA.
    hariszaf
    @hariszaf
    When it comes to the Trimmomatic parameters, you should always check on its manual
    As you can see there, targetLength is a MAXINFO related parameter. I think that by reading this and by combining this with the special feautures of your dataset, you may end up with a value for this parameter.
    Some users have mentioned that they use a value a bit smaller than the merged sequence they're expecting, but I do not know if that is always the case.
    ajiwahyu
    @ajiwahyu
    Thanks @hariszaf !, ill be waiting the update from the bug!
    10 replies
    hariszaf
    @hariszaf
    :plus1:
    thanks again for the feedback! i ll fix it asap!
    Miguel
    @migdesmarais
    After running PEMA with vsearch, the pipeline stopped because my metadata file was named as metadata1.csv. I renamed it as metadata.csv and restarted the pipeline at phyloseq.chp. It resulted in an empty OTU table and so phyloseq did not run. As we discussed, I believe I should have restarted at tableStep.chp?
    hariszaf
    @hariszaf
    Hi @migdesmarais! it is most likely that this is what went bad. If you run the tableStep.chp and it is ok it would be great if you'd share
    hariszaf
    @hariszaf
    Hi @ajiwahyu , hi @all.
    PEMA v.1.3 is now up and running.
    The CROP bug has been resolved and now you can run CROP in multiple threads as well.
    @ajiwahyu many thanks for the feedback once again.
    ajiwahyu
    @ajiwahyu
    thanks @hariszaf !. Am i correct to assume that i need to do another "singularity pull" again so my PEMA container is up to date?
    M Danie Al Malik
    @daniealmalik354
    Hello @hariszaf, I have trouble to run my data on Windows OS from Docker. When I run the data the result like this: Fatal error: ./pema_latest.bds, line 80, pos 27. Map 'paramsFirstAssignment' does not have key 'outputFolderName'. Please help me if you have a suggestion. Thank you