Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Activity
    hariszaf
    @hariszaf
    Thanks again and for anything might be needed please let me know! :)
    ajiwahyu
    @ajiwahyu
    Hi there. Do you have any examples of TruSeq2-PE.fa file(s) and how should it be written or arranged?.
    hariszaf
    @hariszaf
    Hi @ajiwahyu!
    These are the adapaters that Illumina uses. You do not have to use them as input in any step of your analysis.
    You just need to let the tools know which adapters were used to remove these parts.

    Here is the list of the adapters Trimmomatic includes:

    haris@hydrogen:~/metabar_pipeline/PEMA/tools/Trimmomatic-0.36/adapters$ ls
    NexteraPE-PE.fa  TruSeq2-PE.fa  TruSeq2-SE.fa  TruSeq3-PE-2.fa  TruSeq3-PE.fa  TruSeq3-SE.fa

    and here is how the TruSeq2-PE.fa looks like

    haris@hydrogen:~/metabar_pipeline/PEMA/tools/Trimmomatic-0.36/adapters$ more TruSeq2-PE.fa 
    >PrefixPE/1
    AATGATACGGCGACCACCGAGATCTACACTCTTTCCCTACACGACGCTCTTCCGATCT
    >PrefixPE/2
    CAAGCAGAAGACGGCATACGAGATCGGTCTCGGCATTCCTGCTGAACCGCTCTTCCGATCT
    >PCR_Primer1
    AATGATACGGCGACCACCGAGATCTACACTCTTTCCCTACACGACGCTCTTCCGATCT
    >PCR_Primer1_rc
    AGATCGGAAGAGCGTCGTGTAGGGAAAGAGTGTAGATCTCGGTGGTCGCCGTATCATT
    >PCR_Primer2
    CAAGCAGAAGACGGCATACGAGATCGGTCTCGGCATTCCTGCTGAACCGCTCTTCCGATCT
    >PCR_Primer2_rc
    AGATCGGAAGAGCGGTTCAGCAGGAATGCCGAGACCGATCTCGTATGCCGTCTTCTGCTTG
    >FlowCell1
    TTTTTTTTTTAATGATACGGCGACCACCGAGATCTACAC
    >FlowCell2
    TTTTTTTTTTCAAGCAGAAGACGGCATACGA
    ajiwahyu
    @ajiwahyu
    Thanks @hariszaf . OK, so basically we just need to provide adapter's name without the need to supply the actual fa file when running the pipeline?. Thanks again.
    hariszaf
    @hariszaf
    Absolutely right! Just be sure which one was used for your run! :)
    ajiwahyu
    @ajiwahyu
    Anyway, what do you suggest to be used for 16S earth microbiome protocol?. I Just want to make sure, I know it uses truseq2, but there is a part of the protocol which doesn't reflect the adapter that is written in TruSeq2-PE.fa. Thanks again @hariszaf !!
    hariszaf
    @hariszaf
    In our lab, we are using TruSeq2-PE.fa for 16S MiSeq data from environmental samples. So, I think it should be for you too! :)
    ajiwahyu
    @ajiwahyu
    Thanks so much @hariszaf !
    hariszaf
    @hariszaf
    @ajiwahyu :)
    enjoy your analyses!
    rebeccarosetm
    @rebeccarosetm
    Hi! I'd like to run PEMA on my dataset; however, I have only single end Illumina files. I tried to run PEMA according to the instructions on the github page, and I got through the first steps, but then this error message:
    all the first steps are done! clustering is about to start!
    rm: cannot remove 'se.': No such file or directory
    rm: cannot remove 'nospace.
    ': No such file or directory
    Fatal error: ./PEMA_v1.bds, line 517, pos 1. Exec failed.
    I'm wondering if that's because PEMA won't accept single end data? Or is this due to something else? Thanks for your help!
    hariszaf
    @hariszaf
    @rebeccarosetm hi! :)
    My apologies that I did not see that earlier.
    Unfortunately PEMA can deal only with paired end files for the time being.
    I guess you could try to get the reverse complements of your fastq files and then handle your "paired end files" as they have a complete (100%) overlap.
    I have not tried this before but I think it makes sense. :)
    In a next version of PEMA, the handling of single-end files will be included for sure!
    hariszaf
    @hariszaf
    To make reverse complements of your fastq files, you can check over here
    rebeccarosetm
    @rebeccarosetm
    Ah ok, thanks for that suggestion!
    1 reply
    hariszaf
    @hariszaf
    :thumbsup:
    Matthias Obst
    @biomobst
    @hariszaf we now bought a PC and installed Docker. Almost got everything to run, but one error message:
    root@f71f06b4874d:/home# ./PEMA_v1.bds
    Picked up JAVA_TOOL_OPTIONS: -XX:+UseContainerSupport
    Fatal error: ./PEMA_v1.bds, line 68. Directory '/mnt/analysis/mydata' does not exists
    do you recognise the problem?
    hariszaf
    @hariszaf
    Hi @Matthias!
    Do you have your raw data in a folder called mydata inside your analysis directory?
    Matthias Obst
    @biomobst
    yes, the folder is on the Desktop
    it contains fastq files
    hariszaf
    @hariszaf
    So you have something like this
    /Users/matthias/Desktop/my_analysis/
    and there you have your parameters file and a folder called
    mydata
    is that right?
    Matthias Obst
    @biomobst
    Desktop/analysis_directory/mydata/
    the parameters file is under analysis_directory, and the fastq file is under mydata
    hariszaf
    @hariszaf
    and you run
    docker run -it -v /Desktop/analysis_directory:/mnt/analysis
    I am asking because you need to mount the whole analysis_directory not just your data.
    Matthias Obst
    @biomobst

    i hadnavigates to the desktop therefore i only had "/anlaysis_directory" as path. Now i copied your command and get this: "docker run" requires at least 1 argument.
    See 'docker run --help'.

    Usage: docker run [OPTIONS] IMAGE [COMMAND] [ARG...]

    Run a command in a new container

    hariszaf
    @hariszaf
    docker run -it -v /Desktop/analysis_directory:/mnt/analysis hariszaf/pema
    if you run this what do you get?
    Matthias Obst
    @biomobst
    now i am in, i get: root@3069a1f860d3:/home#
    hariszaf
    @hariszaf
    ok. could you run
    ls -la /mnt
    and then
    ls -la /mnt/analysis
    Matthias Obst
    @biomobst
    root@3069a1f860d3:/home# ls -la /mnt
    total 12
    drwxr-xr-x 1 root root 4096 May 24 16:19 .
    drwxr-xr-x 1 root root 4096 Jun 17 12:41 ..
    drwxr-xr-x 2 root root 40 Jun 17 12:41 analysis
    drwxr-xr-x 2 root root 4096 May 24 16:19 databases
    hariszaf
    @hariszaf
    :thumbsup:
    Matthias Obst
    @biomobst
    total 4
    drwxr-xr-x 2 root root 40 Jun 17 12:41 .
    drwxr-xr-x 1 root root 4096 May 24 16:19 ..
    hariszaf
    @hariszaf
    this is from the second command???
    Matthias Obst
    @biomobst
    yes
    hariszaf
    @hariszaf
    ok please run exit.
    then go to your desktop, run pwd and please tell me the exact path you get
    Matthias Obst
    @biomobst
    C:\Users\xobsma\Desktop>pwd
    'pwd' is not recognized as an internal or external command,
    operable program or batch file.
    18 replies
    Matthias Obst
    @biomobst
    Hi Haris, I am back. I found the problem with the Mac installation. You have to add a "~" before the path to the analysis directory in the github tutorial
    '''docker run -it -v ~/Desktop/analysis_directory/:/mnt/analysis hariszaf/pema''',
    ...and i got another error when executing PEMA_v1.bds:
    Fatal error: ./PEMA_v1.bds, line 161, pos 1. Task/s failed.
    29 replies
    hariszaf
    @hariszaf
    Hi @/all ! FYI I am about to integrate the convertIllumunaRawDataToEnaFormat.sh script in the main PEMA code. It seems like an unessecary burden! If you have any other suggestions in terms of improvement of PEMA there are more than welcome!
    ajiwahyu
    @ajiwahyu
    @hariszaf . I got this error and i feel that this has something to do with the parameter.tsv file

    all the first steps are done! clustering is about to start!
    Fatal error: /home/PEMA_v1.bds, line 557, pos 84. Map 'paramsOfTable' does not have key 'clusteringAlgoForCOI_ITS'.
    PEMA_v1.bds, line 557 : if ( paramsOfTable{'clusteringAlgoFor16S_18SrRNA'} == "algo_swarm" || paramsOfTable{'clusteringAlgoForCOI_ITS'} == " algo_SWARM") {

    ProgramCounter.pop(100): Node ID does not match!
    PC : PC: size 4 / 0, nodes: 1422 -> 4690 -> 4691 -> 4698
    Node Id : 4699
    bdsNode Id : 4698
    ProgramCounter.pop(100): Node ID does not match!
    PC : PC: size 3 / 0, nodes: 1422 -> 4690 -> 4691
    Node Id : 4698
    bdsNode Id : 4691
    ProgramCounter.pop(100): Node ID does not match!
    PC : PC: size 2 / 0, nodes: 1422 -> 4690
    Node Id : 4691
    bdsNode Id : 4690
    ProgramCounter.pop(100): Node ID does not match!
    PC : PC: size 1 / 0, nodes: 1422
    Node Id : 4690
    bdsNode Id : 1422

    i ahve checked the
    ajiwahyu
    @ajiwahyu
    i have checked the clusteringAlgoForCOI_ITS and set it into algo_CROP, but the error still persist. Also I am not really sure if I have to comment (#) the 'clusteringAkgofor_16S_18SrRNa part as I am using only COI for the moment . Thanks @hariszaf . So sorry for the trouble...
    hariszaf
    @hariszaf
    Hi @ajiwahyu ! Thank you for the feedback! You should not comment out any of the parameters! PEMA needs to know that it has access to all of them even if you do not use them. So, you could check that all the parameters are on and make sure that your parameters file is tab separated. I would suggest to go clusteringAlgoFor16S_18SrRNA algo_vsearch and clusteringAlgoForCOI_ITS algo_SWARM for a first run; especially if you have many samples as the CROP algorithm will take quite a long!