These are chat archives for dereneaton/ipyrad

30th
Jan 2018
Rebecca Tarvin
@frogsicles_twitter
Jan 30 2018 00:46 UTC
Hi @isaacovercast and @dereneaton - I am having an issue merging assemblies that have completed steps 1-3. When I merge them and run step 4, I get a similar error as @edgardomortiz did on Nov 3 2016 (post is at 11:08). The clustS.gz files are all copied to the directory where the json file expects them. I haven't changed the mindepth parameters between steps 3 and 4, so I think it's some kind of bug. Here are the log file and the out file. In the results file, the het sites are calculated only for the first sample and the rest say NaN. Oddly, I can get other assemblies working with this version of ipyrad, so I'm not sure what's going on. Any help would be appreciated.
 -------------------------------------------------------------
  ipyrad [v.0.7.1]
  Interactive assembly and analysis of RAD-seq data
 -------------------------------------------------------------

  A new version of ipyrad is available (v.0.7.21). To upgrade run:
    conda install -c ipyrad ipyrad

  ** Enabling debug mode ** 

 -------------------------------------------------------------
  ipyrad [v.0.7.1]
  Interactive assembly and analysis of RAD-seq data
 -------------------------------------------------------------

  loading Assembly: all_clust82
  from saved path: /scratch/02576/rdtarvin/ddrad_rana-Nov17/all_clust82.json
  host compute node: [64 cores] on nid02000

  Step 4: Joint estimation of error rate and heterozygosity
  [####################] 100%  inferring [H, E]      | 0:06:13  

  Encountered an unexpected error (see ./ipyrad_log.txt)
  Error message is below -------------------------------
The truth value of an array with more than one element is ambiguous. Use a.any() or a.all()
  ipyrad [v.0.7.1]
  Interactive assembly and analysis of RAD-seq data

 -------------------------------------------------------------

  Begin run: 2018-01-29 14:42
  Using args {'preview': False, 'force': True, 'threads': 4, 'results': False, 'quiet': False, 'merge': None, 'ipcluster': True, 'cores': 0, 'params': 'params-all_clust82.txt', 'branch': None, 'steps': '4567', 'debug': True, 'new': None, 'MPI': False}
  Platform info: ('Linux', 'nid02000', '3.0.101-0.47.86.1.11753.0.PTF-default', '#1 SMP Wed Oct 19 14:11:00 UTC 2016 (56c73f1)', 'x86_64')2018-01-29 14:42:06,266     pid=10499     [load.py]    DEBUG     skipping: no svd results present in old assembly

2018-01-29 14:42:09,385     pid=10204     [jointestimate.py]    INFO      mindepth setting changed: recalculating clusters_hidepth and maxlen
2018-01-29 14:42:09,443     pid=10252     [jointestimate.py]    INFO      mindepth setting changed: recalculating clusters_hidepth and maxlen
2018-01-29 14:42:09,451     pid=10217     [jointestimate.py]    INFO      mindepth setting changed: recalculating clusters_hidepth and maxlen
2018-01-29 14:42:09,460     pid=10282     [jointestimate.py]    INFO      mindepth setting changed: recalculating clusters_hidepth and maxlen
.
.
.
2018-01-29 14:42:19,917     pid=10178     [jointestimate.py]    INFO     (491068,) 285.11142856 25.0631346515
2018-01-29 14:42:23,128     pid=10211     [jointestimate.py]    INFO     (584467,) 286.203397283 24.714325636
2018-01-29 14:42:23,638     pid=10172     [jointestimate.py]    INFO     (770703,) 285.532792788 25.879264084
.
.
.
2018-01-29 14:48:21,247     pid=10499     [jointestimate.py]    ERROR       Sample Rber_T1113b failed with error IndexError(index 143213 is out of bounds for axis 0 with size 143213)
2018-01-29 14:48:21,249     pid=10499     [jointestimate.py]    ERROR       Sample Rbla_D2864 failed with error IndexError(index 126826 is out of bounds for axis 0 with size 126826)
2018-01-29 14:48:21,249     pid=10499     [jointestimate.py]    ERROR       Sample Rbla_D2865 failed with error IndexError(index 121892 is out of bounds for axis 0 with size 121892)
2018-01-29 14:48:21,387     pid=10499     [assembly.py]    ERROR     The truth value of an array with more than one element is ambiguous. Use a.any() or a.all()
2018-01-29 14:48:21,447     pid=10499     [__init__.py]    INFO     debugging turned off
Ollie White
@Ollie_W_White_twitter
Jan 30 2018 12:36 UTC
Hello, I repeated a previous ipyrad assembly and got a slightly different number of SNPs at the end. I'm fairly sure my commands have stay the same. Could the difference be due to a more recent version by any chance? The differences are small admittedly. For example number of clusters changing from 62306 to 62310 in one sample. Cheers
Eaton Lab
@eaton-lab
Jan 30 2018 15:29 UTC
@Ollie_W_White_twitter My guess is that it's probably due to different random seeds being used in one or more steps of assembly. For example, we randomize input sequences before clustering. We haven't yet found a way to ensure everything can be run from a single starting seed since there's a mix of libraries that use different seeds. And, I guess we would need to store the seed from the start, since if you were to run it in steps, stopping and restarting, or even branching, it gets complicated for which random seed to use. Something we'll have to think about more. I should raise a ticket for us to work on it though, it should be doable.
Isaac Overcast
@isaacovercast
Jan 30 2018 16:29 UTC
@frogsicles_twitter Can you try updating to the newest version and running step 4 again? There have been many bug fixes since 0.7.1 and updating is the easiest way to see if this issue has already been fixed.
Rebecca Tarvin
@frogsicles_twitter
Jan 30 2018 19:01 UTC
@isaacovercast Sure I can. Will update once I try that.