These are chat archives for dereneaton/ipyrad

1st
Oct 2016
Shea Lambert
@SheaML
Oct 01 2016 00:48

I'll let you know if I can replicate it.

Meanwhile, I can report that the whole pipeline is working for me now, and steps 3 and 6 are indeed much, much faster! Thanks

Edgardo M. Ortiz
@edgardomortiz
Oct 01 2016 08:36

A couple of questions about the changes in step 2. Looking at the code of rawedit.py may I assume that now to apply the quality filters you must set parameter [16] to 1? are the bases under the quality threshold still being converted to Ns? I couldn't find Ns in the edited reads when parameter [16] was set to 0.
After changing parameter [16] to 1 I get the following error:

 -------------------------------------------------------------
  ipyrad [v.0.4.0]
  Interactive assembly and analysis of RAD-seq data
 -------------------------------------------------------------
  New Assembly: ast-r1
  host compute node: [48 cores] on nid00017

  Step 1: Linking sorted fastq data to Samples
        Linking to demultiplexed fastq files in:
            /scratch/02728/emo347/ddRAD/astereae/e_denovo-rad_ipyrad/ast-r1_fastqs/*_R1_.fq.gz

    44 new Samples created in 'ast-r1'.
    44 fastq files linked to 44 new Samples.


  Step 2: Filtering reads
  [####################] 100%  processing reads      | 0:03:11

  Encountered an error, see ./ipyrad_log.txt.
  UnboundLocalError(local variable 'cmdf1' referenced before assignment)

The edits folder is empty in this case. Also, the errors are not being written to ipyrad_log.txt, the file is empty as well.

James McDaniel
@jlmcdaniel
Oct 01 2016 16:40
@dereneaton @isaacovercast Hi Deren and Isaac, I just wanted to follow-up... I implemented the helpful tips that you provided to me and my dataset seems to be running faster on the cluster! Hopefully I won't encounter anymore problems, but I'll let you know if I do. Also, I have ipyrad on my desktop computer, and when I installed the update I got the same cutadapt error as @SheaML. I'm not sure what is going on, but I am using the previous version of ipyrad for the moment on the cluster and it is working. Thanks for your help!
Isaac Overcast
@isaacovercast
Oct 01 2016 16:42
@jlmcdaniel Glad to hear its working better on the cluster. For your desktop and the cutadapt error, is your desktop mac or linux?
James McDaniel
@jlmcdaniel
Oct 01 2016 16:54
@isaacovercast My desktop is mac. I forgot that there are two versions of the program. Could that be the issue?
Isaac Overcast
@isaacovercast
Oct 01 2016 17:02
@edgardomortiz Thanks for catching that bug, I fixed this and pushed a new conda package: 0.4.1
Edgardo M. Ortiz
@edgardomortiz
Oct 01 2016 17:09
@isaacovercast no problem. So cutadapt does all the quality trimming now? It just removes Ns that were in the original fastq but low quality bases are not converted to Ns, right?
Isaac Overcast
@isaacovercast
Oct 01 2016 17:19
Yeah, cutadapt is doing the heavy lifting for step 2 now. But it does look like there is a slight change of behavior here with low qual bases. cutadapt filters reads if the number of low qual bases exceeds max_low_qual_bases, but it doesn't convert low qual bases to Ns if the read passes this filter.
Edgardo M. Ortiz
@edgardomortiz
Oct 01 2016 17:24
Yeah, I'm losing many reads now... would it be possible to have an option to first convert low quality bases to Ns and then pass those edited reads to cutadapt?
Isaac Overcast
@isaacovercast
Oct 01 2016 17:30
I think that's probably a good idea. Let me think about the best way to do this...
Edgardo M. Ortiz
@edgardomortiz
Oct 01 2016 17:50
Cool! That would be very useful, specially to compare results with previous analyses
Deren Eaton
@dereneaton
Oct 01 2016 18:05
Yeah, that's a good idea, we could do that.