These are chat archives for dereneaton/ipyrad

26th
Feb 2018
Glib Mazepa
@mazepago_twitter
Feb 26 2018 12:52

@isaacovercast I ran the s7 with -d flag, it is gross, the tail is below. Also, I checked the _across directory that contains results of s6, is it possible that during execution of s7 it created the tmp.h5 files, I thought that the s6 has been finished successfully and those tmps were removed before I launched s7? If the s6 has not been finished correctly, is it possible that there is no warning when I am launching s7? : 2018-02-26 10:11:09,836 pid=32223 [write_outfiles.py] INFO
seqx
seq1
seq2

2018-02-26 10:11:09,961 pid=32223 [write_outfiles.py] INFO Writing .vcf file
2018-02-26 10:11:09,962 pid=32223 [assembly.py] ERROR [Errno 5] Input/output error
2018-02-26 10:11:12,576 pid=32223 [assembly.py] INFO shutting down engines
2018-02-26 10:11:12,673 pid=32223 [assembly.py] INFO finished shutdown
2018-02-26 10:11:12,900 pid=32223 [init.py] INFO debugging turned off

Ollie White
@Ollie_W_White_twitter
Feb 26 2018 13:16

Hello, I would like to assemble paired GBS reads. The sequencing facility that provided our data said they used two different restriction enzymes instead of one which wouldn't work with the pairgbs method in ipyrad so I opted for pairddrad as it seems closest to the data type that we have. After attempting to de-multiplex our data I got an error saying that the lengths of the forward and reverse files were different. See log file output below.

-------------------------------------------------------------
  ipyrad [v.0.7.21]
  Interactive assembly and analysis of RAD-seq data
 -------------------------------------------------------------
  Begin run: 2018-02-22 12:36
  Using args {'preview': False, 'force': True, 'threads': 16, 'results': False, 'quiet': False, 'merge': None, 'ipcluster': None, 'cores': 0, 'params': 'params-des.txt', 'branch': None, 'steps': '12', 'debug': False, 'new': None, 'download': None, 'MPI': False}
  Platform info: ('Linux', 'green0506', '2.6.32-642.11.1.el6.x86_64', '#1 SMP Wed Oct 26 10:25:23 EDT 2016', 'x86_64')2018-02-22 12:37:26,280   pid=16029       [assembly.py]    ERROR   R1 and R2 files are not the same length.

I checked the length of the files too and they are the same

zcat RAW_GBS00289_L7_R1_data.fq.gz | wc -l
106985232
zcat RAW_GBS00289_L7_R2_data.fq.gz | wc -l
106985232

Has anyone any suggestions on what might be the issue?
Cheers
Ollie

Isaac Overcast
@isaacovercast
Feb 26 2018 14:47
@Ollie_W_White_twitter Hm, yeah that looks okay. Can you rerun with the -d flag and email me the ipyrad_log.txt file?
@mazepago_twitter Yes, the tmp-* files are getting created during step 7 and they aren't getting cleaned up because it's crashing. The relevant line is:
2018-02-26 10:11:09,962 pid=32223 [assembly.py] ERROR [Errno 5] Input/output error
Isaac Overcast
@isaacovercast
Feb 26 2018 14:52
This is almost always a disk space issue. Are you sure you have enough disk to construct the vcf file?
Ollie White
@Ollie_W_White_twitter
Feb 26 2018 15:07
HI Isaac, thanks for the reply. The debug mode output is below
ipyrad -p params-des.txt -s 1 -d -f

  ** Enabling debug mode **

 -------------------------------------------------------------
  ipyrad [v.0.7.21]
  Interactive assembly and analysis of RAD-seq data
 -------------------------------------------------------------
  New Assembly: des
  establishing parallel connection:
  host compute node: [16 cores] on green0217

  Step 1: Demultiplexing fastq data to Samples
  [force] overwriting fastq files previously created by ipyrad.
  This _does not_ affect your original/raw data files.
INFO:ipyrad.assemble.demultiplex:zcat is using optim = 8000000
  [####################] 100%  chunking large files  | 0:01:35  ERROR:ipyrad.core.assembly:R1 and R2 files are not the same length.

  Encountered an unexpected error (see ./ipyrad_log.txt)
  Error message is below -------------------------------
R1 and R2 files are not the same length.
INFO:ipyrad.core.assembly:  shutting down engines
INFO:ipyrad.core.assembly:  finished shutdown
INFO:ipyrad:debugging turned off
Glib Mazepa
@mazepago_twitter
Feb 26 2018 20:53
@isaacovercast there should be some 4+ TB on /scratch/, could this issue be linked to the $HOME installation directory though? The vcf is not starting to be even filled, just 19K in size...