These are chat archives for dereneaton/ipyrad

8th
Nov 2018
AliceLedent
@AliceLedent
Nov 08 2018 08:27
Hi again,
Any idea of why the step 2 could result in paired-end fastq file of different length even if the s2_rawedits_stats.txt file seems normal? And this especially since i've been working with running cutadapt before ipyrad (cutadapt outputs are perfect) and set the parameter "filter_adapter" to 0 compared to when i was not running cutadapt before ipyrad and set this parameter to 2?
Thank you!
Isaac Overcast
@isaacovercast
Nov 08 2018 16:01
@AliceLedent What are the other parameters you're using?
How are you figuring out the files are different length before and after step 2?
AliceLedent
@AliceLedent
Nov 08 2018 16:22
Hi Here is the overview of the params file i used.
I compared the size of the Read1 and Read2 with this command line:
zcat file.1.fastq.gz| wc -l and zcat file.2.fastq.gz| wc -l
I did so on the output files from the 1st step in the folder _fastqs (the size were identical) and then on the output files from the 2nd step in the folder _edits (the size were different). Leading to this error in step 3 : "Invalid line 10716930 in FASTQ file: Unexpected end of file\n'))".
"------- ipyrad params file (v.0.7.28)-------------------------------------------
Micro_cutNEW ## [0] [assembly_name]: Assembly name. Used to name output directories for assembly steps
/gpfsuser/home/users/a/l/aledent/ipyrad/Belgium/Plate1/Plate1_demul_cutNEW_ipyrad ## [1] [project_dir]: Project dir (made in curdir if not present)
Merged: Micro_Plate1_cutNEW, Micro_Plate2_cutNEW, Micro_Plate3_cutNEW, Micro_Plate4_cutNEW, Micro_Plate5_cutNEW ## [2] [raw_fastq_path]: Location of raw non-demultiplexed fastq files
Merged: Micro_Plate1_cutNEW, Micro_Plate2_cutNEW, Micro_Plate3_cutNEW, Micro_Plate4_cutNEW, Micro_Plate5_cutNEW ## [3] [barcodes_path]: Location of barcodes file
Merged: Micro_Plate1_cutNEW, Micro_Plate2_cutNEW, Micro_Plate3_cutNEW, Micro_Plate4_cutNEW, Micro_Plate5_cutNEW ## [4] [sorted_fastq_path]: Location of demultiplexed/sorted fastq files
denovo ## [5] [assembly_method]: Assembly method (denovo, reference, denovo+reference, denovo-reference)
/ ## [6] [reference_sequence]: Location of reference sequence file
pairgbs ## [7] [datatype]: Datatype (see docs): rad, gbs, ddrad, etc.
CWG, ## [8] [restriction_overhang]: Restriction overhang (cut1,) or (cut1, cut2)
5 ## [9] [max_low_qual_bases]: Max low quality base calls (Q<20) in a read
33 ## [10] [phred_Qscore_offset]: phred Q score offset (33 is default and very standard)
5 ## [11] [mindepth_statistical]: Min depth for statistical base calling
3 ## [12] [mindepth_majrule]: Min depth for majority-rule base calling
10000 ## [13] [maxdepth]: Max cluster depth within samples
0.85 ## [14] [clust_threshold]: Clustering threshold for de novo assembly
0 ## [15] [max_barcode_mismatch]: Max number of allowable mismatches in barcodes
0 ## [16] [filter_adapters]: Filter for adapters/primers (1 or 2=stricter)
35 ## [17] [filter_min_trim_len]: Min length of reads after adapter trim
1 ## [18] [max_alleles_consens]: Max alleles per site in consensus sequences
5, 5 ## [19] [max_Ns_consens]: Max N's (uncalled bases) in consensus (R1, R2)
0, 0 ## [20] [max_Hs_consens]: Max Hs (heterozygotes) in consensus (R1, R2)
4 ## [21] [min_samples_locus]: Min # samples per locus for output
20, 20 ## [22] [max_SNPs_locus]: Max # SNPs per locus (R1, R2)
0, 0 ## [23] [max_Indels_locus]: Max # of indels per locus (R1, R2)
0 ## [24] [max_shared_Hs_locus]: Max # heterozygous sites per locus (R1, R2)
0, 0, 0, 0 ## [25] [trim_reads]: Trim raw read edges (R1>, <R1, R2>, <R2) (see docs) 0, 0, 0, 0 ## [26] [trim_loci]: Trim locus edges (see docs) (R1>, <R1, R2>, <R2)
G, a, g, k, m, l, n, p, s, u, t, v ## [27] [output_formats]: Output formats (see docs)
/ ## [28] [pop_assign_file]: Path to population assignment file"
i just added the / in the line ##[6] and ## [28] cause it was making a weird appearance on this window but they are empty in the real params file.
Thank you very much in advance!
Amely Martins
@amelymartins
Nov 08 2018 17:49

Hi @isaacovercast and @dereneaton . The computer cluster I've been working for a while was updated recently and now I'm getting an error when trying to run ipyrad. I've reinstalled conda and ipyrad, but keep getting the following error:
Traceback (most recent call last):
File "/home1/03114/abm2462/miniconda2/bin/ipyrad", line 11, in <module>
load_entry_point('ipyrad==0.7.28', 'console_scripts', 'ipyrad')()
File "/home1/03114/abm2462/miniconda2/lib/python2.7/site-packages/pkg_resources/init.py", line 484, in load_entry_point
return get_distribution(dist).load_entry_point(group, name)
File "/home1/03114/abm2462/miniconda2/lib/python2.7/site-packages/pkg_resources/init.py", line 2707, in load_entry_point
return ep.load()
File "/home1/03114/abm2462/miniconda2/lib/python2.7/site-packages/pkg_resources/init.py", line 2325, in load
return self.resolve()
File "/home1/03114/abm2462/miniconda2/lib/python2.7/site-packages/pkg_resources/init.py", line 2331, in resolve
module = import(self.module_name, fromlist=['name'], level=0)
File "/home1/03114/abm2462/miniconda2/lib/python2.7/site-packages/ipyrad/init.py", line 20, in <module>
from . import load as _load
File "/home1/03114/abm2462/miniconda2/lib/python2.7/site-packages/ipyrad/load/init.py", line 14, in <module>
from .load import test_assembly
File "/home1/03114/abm2462/miniconda2/lib/python2.7/site-packages/ipyrad/load/load.py", line 13, in <module>
from ipyrad.assemble.util import *
File "/home1/03114/abm2462/miniconda2/lib/python2.7/site-packages/ipyrad/assemble/init.py", line 9, in <module>
from . import consens_se
File "/home1/03114/abm2462/miniconda2/lib/python2.7/site-packages/ipyrad/assemble/consens_se.py", line 33, in <module>
import h5py
File "/opt/apps/intel18/cray_mpich_7_7/python2/2.7.15/lib/python2.7/site-packages/h5py/init.py", line 26, in <module>
from . import _errors
ImportError: libhdf5.so.10: cannot open shared object file: No such file or directory

Any help on this?

Amely Martins
@amelymartins
Nov 08 2018 17:56
@isaacovercast and @dereneaton I've solved the problem I've described before by adding: export PYTHONPATH="";
Isaac Overcast
@isaacovercast
Nov 08 2018 17:59
@amelymartins dereneaton/ipyrad#231 It's probably the conda install is messed up, easiest to blow away your conda install and start fresh.
@AliceLedent Can you dropbox me the R1 and R2 of one of the samples that's failing?