These are chat archives for dereneaton/ipyrad

7th
Oct 2016
leclearnm
@leclearnm
Oct 07 2016 00:37
Hi @dereneaton ,
I am having an inconsistent error running step 7 from ipyrad . My goal is to generate 5 data sets with an increasing minimum locus threshold. I branched my params file, and changed only parameter 21 to the desired value and 27 to an *.
Below I have the error message from the log file, and I can also show the contents of the outfile directory.
The strange thing is that it worked for 2 out of 5 the first time, and then a third one after running it again. The other 2 have failed in the same way after trying multiple times.
login1.ls5(9)$ cat s7_88-fwd32.log

 -------------------------------------------------------------
  ipyrad [v.0.4.1]
  Interactive assembly and analysis of RAD-seq data
 -------------------------------------------------------------
  loading Assembly: fwd32
  from saved path: /work/03530/nleclear/lonestar/pyrad/data/ja16512/relaxed/fwd32.json
  host compute node: [8 cores] on nid01218

  Step 7: Filter and write output files for 65 Samples
  [####################] 100%  filtering loci        | 0:00:25 
  [####################] 100%  building loci/stats   | 0:00:03 
  [####################] 100%  building vcf file     | 0:00:17 

  Encountered an error, see ./ipyrad_log.txt. 
   error in vcf build chunk 58905: ValueError(all the input arrays must have same number of dimensions)
Here are the contents of the directory
login1.ls5(4)$ ls fwd32_outfiles/
fwd32.hdf5          fwd32.vcf.gz.15939  fwd32.vcf.gz.24255  fwd32.vcf.gz.32571  fwd32.vcf.gz.40887  fwd32.vcf.gz.48510  fwd32.vcf.gz.56826
fwd32.loci          fwd32.vcf.gz.16632  fwd32.vcf.gz.24948  fwd32.vcf.gz.33264  fwd32.vcf.gz.4158   fwd32.vcf.gz.49203  fwd32.vcf.gz.57519
fwd32_stats.txt     fwd32.vcf.gz.17325  fwd32.vcf.gz.25641  fwd32.vcf.gz.33957  fwd32.vcf.gz.41580  fwd32.vcf.gz.49896  fwd32.vcf.gz.58212
fwd32.vcf.gz.0      fwd32.vcf.gz.18018  fwd32.vcf.gz.26334  fwd32.vcf.gz.3465   fwd32.vcf.gz.42273  fwd32.vcf.gz.50589  fwd32.vcf.gz.58905
fwd32.vcf.gz.10395  fwd32.vcf.gz.18711  fwd32.vcf.gz.27027  fwd32.vcf.gz.34650  fwd32.vcf.gz.42966  fwd32.vcf.gz.51282  fwd32.vcf.gz.6237
fwd32.vcf.gz.11088  fwd32.vcf.gz.19404  fwd32.vcf.gz.2772   fwd32.vcf.gz.35343  fwd32.vcf.gz.43659  fwd32.vcf.gz.51975  fwd32.vcf.gz.693
fwd32.vcf.gz.11781  fwd32.vcf.gz.20097  fwd32.vcf.gz.27720  fwd32.vcf.gz.36036  fwd32.vcf.gz.44352  fwd32.vcf.gz.52668  fwd32.vcf.gz.6930
fwd32.vcf.gz.12474  fwd32.vcf.gz.2079   fwd32.vcf.gz.28413  fwd32.vcf.gz.36729  fwd32.vcf.gz.45045  fwd32.vcf.gz.53361  fwd32.vcf.gz.7623
fwd32.vcf.gz.13167  fwd32.vcf.gz.20790  fwd32.vcf.gz.29106  fwd32.vcf.gz.37422  fwd32.vcf.gz.45738  fwd32.vcf.gz.54054  fwd32.vcf.gz.8316
fwd32.vcf.gz.1386   fwd32.vcf.gz.21483  fwd32.vcf.gz.29799  fwd32.vcf.gz.38115  fwd32.vcf.gz.46431  fwd32.vcf.gz.54747  fwd32.vcf.gz.9009
fwd32.vcf.gz.13860  fwd32.vcf.gz.22176  fwd32.vcf.gz.30492  fwd32.vcf.gz.38808  fwd32.vcf.gz.47124  fwd32.vcf.gz.5544   fwd32.vcf.gz.9702
fwd32.vcf.gz.14553  fwd32.vcf.gz.22869  fwd32.vcf.gz.31185  fwd32.vcf.gz.39501  fwd32.vcf.gz.47817  fwd32.vcf.gz.55440
fwd32.vcf.gz.15246  fwd32.vcf.gz.23562  fwd32.vcf.gz.31878  fwd32.vcf.gz.40194  fwd32.vcf.gz.4851   fwd32.vcf.gz.56133
leclearnm
@leclearnm
Oct 07 2016 01:06
Just as a followup to that, I updated to the latest release and have the same error
Deren Eaton
@dereneaton
Oct 07 2016 01:07
@leclearnm thanks, damn, ipyrad should be deleting all of those tmp files, and that error is obviously a problem. I think I know the source. Can probably fix it by tomorrow.
leclearnm
@leclearnm
Oct 07 2016 01:09
OK, thank you!
Carlos Munoz-Ramirez
@radagast1980
Oct 07 2016 13:47
Hello, I am trying to run ipyrad in a hpc, but it gives me this error when trying to run: Error: Steps >1 (234567) requested but no current assembly found - /scratch/lsa_flux/carmunoz/heraldo/ipyrad/acor_c90d6 I created the assemble right in the working directory, but still gives me the error.
Isaac Overcast
@isaacovercast
Oct 07 2016 15:37
Hi @radagast1980 It looks like step 1 didn't run or didn't run to completion. Can you try just running ipyrad with -s 1, this will just run step 1.
Carlos Munoz-Ramirez
@radagast1980
Oct 07 2016 15:38
This message was deleted
@isaacovercast I'll give it a try
Carlos Munoz-Ramirez
@radagast1980
Oct 07 2016 15:47
@isaacovercast , It didn't work. Same error as before.
Isaac Overcast
@isaacovercast
Oct 07 2016 15:49
Can you email me your params file? iovercast@gc.cuny.edu
Also, can you copy and paste the command you're running?
Carlos Munoz-Ramirez
@radagast1980
Oct 07 2016 15:54
@isaacovercast sent, thanks for taking a look at it
Isaac Overcast
@isaacovercast
Oct 07 2016 15:57
ipyrad -p /scratch/lsa_flux/carmunoz/heraldo/ipyrad/params-acor_c90d6.txt -s 1
This will fix it. The problem is that even though you are bringing in demultiplexed data, you still need to run step 1 so ipyrad can import the samples.
Carlos Munoz-Ramirez
@radagast1980
Oct 07 2016 15:58
@isaacovercast oh I see. So if I run all the steps at once from s1-7 should be fine? It will only run from step 2?
Isaac Overcast
@isaacovercast
Oct 07 2016 16:00
Yeah, you can say:
ipyrad -p /scratch/lsa_flux/carmunoz/heraldo/ipyrad/params-acor_c90d6.txt -s 1234567
Carlos Munoz-Ramirez
@radagast1980
Oct 07 2016 16:01
@isaacovercast Great, thanks, its working!
Isaac Overcast
@isaacovercast
Oct 07 2016 16:01
Awesome.
Deren Eaton
@dereneaton
Oct 07 2016 20:16
Another option is to set your 'project_dir' param to your scratch directory location and to keep your params.txt file in your local directory. It's a bit cleaner way to do things in my opinion. But both work.