Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Activity
    hariszaf
    @hariszaf
    if you run this what do you get?
    Matthias Obst
    @biomobst
    now i am in, i get: root@3069a1f860d3:/home#
    hariszaf
    @hariszaf
    ok. could you run
    ls -la /mnt
    and then
    ls -la /mnt/analysis
    Matthias Obst
    @biomobst
    root@3069a1f860d3:/home# ls -la /mnt
    total 12
    drwxr-xr-x 1 root root 4096 May 24 16:19 .
    drwxr-xr-x 1 root root 4096 Jun 17 12:41 ..
    drwxr-xr-x 2 root root 40 Jun 17 12:41 analysis
    drwxr-xr-x 2 root root 4096 May 24 16:19 databases
    hariszaf
    @hariszaf
    :thumbsup:
    Matthias Obst
    @biomobst
    total 4
    drwxr-xr-x 2 root root 40 Jun 17 12:41 .
    drwxr-xr-x 1 root root 4096 May 24 16:19 ..
    hariszaf
    @hariszaf
    this is from the second command???
    Matthias Obst
    @biomobst
    yes
    hariszaf
    @hariszaf
    ok please run exit.
    then go to your desktop, run pwd and please tell me the exact path you get
    Matthias Obst
    @biomobst
    C:\Users\xobsma\Desktop>pwd
    'pwd' is not recognized as an internal or external command,
    operable program or batch file.
    18 replies
    Matthias Obst
    @biomobst
    Hi Haris, I am back. I found the problem with the Mac installation. You have to add a "~" before the path to the analysis directory in the github tutorial
    '''docker run -it -v ~/Desktop/analysis_directory/:/mnt/analysis hariszaf/pema''',
    ...and i got another error when executing PEMA_v1.bds:
    Fatal error: ./PEMA_v1.bds, line 161, pos 1. Task/s failed.
    29 replies
    hariszaf
    @hariszaf
    Hi @/all ! FYI I am about to integrate the convertIllumunaRawDataToEnaFormat.sh script in the main PEMA code. It seems like an unessecary burden! If you have any other suggestions in terms of improvement of PEMA there are more than welcome!
    ajiwahyu
    @ajiwahyu
    @hariszaf . I got this error and i feel that this has something to do with the parameter.tsv file

    all the first steps are done! clustering is about to start!
    Fatal error: /home/PEMA_v1.bds, line 557, pos 84. Map 'paramsOfTable' does not have key 'clusteringAlgoForCOI_ITS'.
    PEMA_v1.bds, line 557 : if ( paramsOfTable{'clusteringAlgoFor16S_18SrRNA'} == "algo_swarm" || paramsOfTable{'clusteringAlgoForCOI_ITS'} == " algo_SWARM") {

    ProgramCounter.pop(100): Node ID does not match!
    PC : PC: size 4 / 0, nodes: 1422 -> 4690 -> 4691 -> 4698
    Node Id : 4699
    bdsNode Id : 4698
    ProgramCounter.pop(100): Node ID does not match!
    PC : PC: size 3 / 0, nodes: 1422 -> 4690 -> 4691
    Node Id : 4698
    bdsNode Id : 4691
    ProgramCounter.pop(100): Node ID does not match!
    PC : PC: size 2 / 0, nodes: 1422 -> 4690
    Node Id : 4691
    bdsNode Id : 4690
    ProgramCounter.pop(100): Node ID does not match!
    PC : PC: size 1 / 0, nodes: 1422
    Node Id : 4690
    bdsNode Id : 1422

    i ahve checked the
    ajiwahyu
    @ajiwahyu
    i have checked the clusteringAlgoForCOI_ITS and set it into algo_CROP, but the error still persist. Also I am not really sure if I have to comment (#) the 'clusteringAkgofor_16S_18SrRNa part as I am using only COI for the moment . Thanks @hariszaf . So sorry for the trouble...
    hariszaf
    @hariszaf
    Hi @ajiwahyu ! Thank you for the feedback! You should not comment out any of the parameters! PEMA needs to know that it has access to all of them even if you do not use them. So, you could check that all the parameters are on and make sure that your parameters file is tab separated. I would suggest to go clusteringAlgoFor16S_18SrRNA algo_vsearch and clusteringAlgoForCOI_ITS algo_SWARM for a first run; especially if you have many samples as the CROP algorithm will take quite a long!
    It is essential to have a tab seperated file.. This is a quite common error but for the time being we cannot avoid this format.. :/
    ajiwahyu
    @ajiwahyu
    Thanks @hariszaf. I just tried your suggestion and the pipeline seemed working with SWARM but spit the following error when I used CROP. Also a question with Database, for COI what database are you using?. Is there any setting that I need to change if I want to use different data base?.Also for setting a targetlength at the trimmomatic part, how should one decide it?. IAm using COI, with COI primer dgHCO2198, expecting a final merge reads at 300-330 bp.

    This is the error i got when using CROP COI; cropPath is: /mnt/analysis/16S_final_test/7.gene_dependent/gene_COI/CROP
    /home/scripts/croParameters.sh: line 49: cd: tools/: No such file or directory
    toolsPath is: /mnt/analysis
    /home/scripts/croParameters.sh: line 57: /mnt/analysis/CROP/CROPLinux: No such file or directory
    cp: cannot stat 'CROP_output.cluster.list': No such file or directory
    sed: can't read CROP_output.cluster.list: No such file or directory
    split: cannot open 'CROP_output.cluster.list' for reading: No such file or directory
    rm: cannot remove 'CROP_output.cluster.list': No such file or directory

    sed: can't read /mnt/analysis/16S_final_test/7.gene_dependent/gene_COI/CROP/abundances/x: No such file or directory
    rm: cannot remove '/mnt/analysis/16S_final_test/7.gene_dependent/gene_COI/CROP/abundances/x
    .abundances': No such file or directory
    cat: 'x': No such file or directory
    rm: cannot remove 'x
    ': No such file or directory
    tail: cannot open 'CROP_output.cluster' for reading: No such file or directory
    cp: cannot stat 'CROP_output.cluster.list': No such file or directory
    sed: can't read SWARM.swarms: No such file or directory
    Fatal error: /home/PEMA_v1.bds, line 971, pos 3. Exec failed.
    Exit value : 2
    Command : bash /home/scripts/cropOutputTranformations.sh
    PEMA_v1.bds, line 804 : if ( paramsOfTable{'gene'} == 'gene_COI' || paramsOfTable{'gene'} == "gene_ITS") {
    PEMA_v1.bds, line 927 : if ( paramsOfTable{'clusteringAlgoForCOI_ITS'} == 'algo_CROP' ) {
    PEMA_v1.bds, line 971 : sys bash $path/scripts/cropOutputTranformations.sh

    hariszaf
    @hariszaf
    Hi @ajiwahyu! Thanks a lot for this! There was a bug on the script that calculates the CROP parameters. I am about to rebuild an image fixing that.
    Regarding the COI marker gene, PEMA uses the MIDORI database, which you may find here.
    For the time being you cannot use an in-house reference database, however that is going to be included in the next version of PEMA.
    hariszaf
    @hariszaf
    When it comes to the Trimmomatic parameters, you should always check on its manual
    As you can see there, targetLength is a MAXINFO related parameter. I think that by reading this and by combining this with the special feautures of your dataset, you may end up with a value for this parameter.
    Some users have mentioned that they use a value a bit smaller than the merged sequence they're expecting, but I do not know if that is always the case.
    ajiwahyu
    @ajiwahyu
    Thanks @hariszaf !, ill be waiting the update from the bug!
    10 replies
    hariszaf
    @hariszaf
    :plus1:
    thanks again for the feedback! i ll fix it asap!
    Miguel
    @migdesmarais
    After running PEMA with vsearch, the pipeline stopped because my metadata file was named as metadata1.csv. I renamed it as metadata.csv and restarted the pipeline at phyloseq.chp. It resulted in an empty OTU table and so phyloseq did not run. As we discussed, I believe I should have restarted at tableStep.chp?
    hariszaf
    @hariszaf
    Hi @migdesmarais! it is most likely that this is what went bad. If you run the tableStep.chp and it is ok it would be great if you'd share
    hariszaf
    @hariszaf
    Hi @ajiwahyu , hi @all.
    PEMA v.1.3 is now up and running.
    The CROP bug has been resolved and now you can run CROP in multiple threads as well.
    @ajiwahyu many thanks for the feedback once again.
    ajiwahyu
    @ajiwahyu
    thanks @hariszaf !. Am i correct to assume that i need to do another "singularity pull" again so my PEMA container is up to date?
    M Danie Al Malik
    @daniealmalik354
    Hello @hariszaf, I have trouble to run my data on Windows OS from Docker. When I run the data the result like this: Fatal error: ./pema_latest.bds, line 80, pos 27. Map 'paramsFirstAssignment' does not have key 'outputFolderName'. Please help me if you have a suggestion. Thank you
    hariszaf
    @hariszaf

    thanks @hariszaf !. Am i correct to assume that i need to do another "singularity pull" again so my PEMA container is up to date?

    Yes you are absolutely right. You now need to run
    singularity pull shub://hariszaf/pema:v.1.3

    hariszaf
    @hariszaf

    Hello @hariszaf, I have trouble to run my data on Windows OS from Docker. When I run the data the result like this: Fatal error: ./pema_latest.bds, line 80, pos 27. Map 'paramsFirstAssignment' does not have key 'outputFolderName'. Please help me if you have a suggestion. Thank you

    Hi @daniealmalik354 ! Download the parameters.tsv file from here. It would be better to do that using the wget command.
    There is a chance that you have completed the parameters file using spaces instead of tabs. Please make sure that you are using tabs.

    3 replies
    hariszaf
    @hariszaf
    Hi @/all !
    The convertion of Illumina's raw data to the ENA formt is now part of the PEMA code.
    pema:v.1.3.1 is now available as both Docker and Singularity containers and from now on you do not have to convert your raw data on your own! You just need to set the parameter EnaData to No and PEMA will do the convertion automatically.
    ajiwahyu
    @ajiwahyu
    @hariszaf, Thanks for the hard work!
    Matthias Obst
    @biomobst
    @hariszaf, Hi Haris, I am back. We are getting a new problem on PC. installation seems fine but when we execute the script it says: error: ./PEMA_v1.bds line 68, 'mnt/analysis/mydata' does not exist. do you knwo what could be the problem?
    111 replies
    Matthias Obst
    @biomobst
    @hariszaf, back to Mac. The analysis runs fine until line 1186. I get the error: /PEMA_v1.bds, line 1186, pos 3. Task/s failed. When I look at the "output 7. gene dependent" everythyning is there but the file "tax_assign_swarm_COI_temp.txt" " is empty. Do you know that problem?
    15 replies
    hariszaf
    @hariszaf
    Hi @/all
    For your information, all bugs mentioned until now have been fixed and the convertion of Illumina raw data to ENA format is now automatically implemented.
    You can find this version of pema as the ones with the latest tag, both as Docker and Singularity images.
    Thank you all for your feedback - and patience sometimes! :smiley:
    hariszaf
    @hariszaf
    Hi @/all !
    FYI I just found out about this awesome recommendation!
    Hope you ll find it useful!
    naurasd
    @naurasd
    @hariszaf, Hi Haris, I got the same issue running PEMA on PC as mentioned by @biomobst Matthias last month. When executing the script, this is what happens: "Fatal error: ./pema_latest.bds, line 68. Directory '/mnt/analysis/mydata' does not exists". I do have the latest Version of PEMA and execute ./pena_latest.bds, there is a mydata folder named exactly like it is supposed to be in my analysis_directory folder and it only contains 2 fastq.gz files. Any idea what could be the issue?
    33 replies
    hariszaf
    @hariszaf
    This message was deleted