These are chat archives for dereneaton/ipyrad

Apr 2016
Isaac Overcast
Apr 28 2016 03:53
denovo+reference is working again. the only problem is the sloppy way i created the simulated reference (a handful of overlapped reads) causes some of them to fail due to high indels
denovo-reference and reference both work again as well, for SE (contingent on the above prob getting fixed)
Deren Eaton
Apr 28 2016 12:08
Heck yeah. That's awesome to hear.
I'm finding that step6 does not scale well to very big data sets. Filling the h5 data base takes a long time, and it becomes huuuge (79GB). I have some ideas for reducing its size.
Deren Eaton
Apr 28 2016 19:53
Ran into the following problem on the latest pull when running denovo:
  ipyrad [v.0.1.87]
  Interactive assembly and analysis of RADseq data
  loading Assembly: cli
  from saved path: ~/Documents/ipyrad/tests/cli/cli.json
  ipyparallel setup: Local connection to 4 Engines

  Step6: Clustering across 12 samples at 0.85 similarity
INFO:ipyrad.assemble.cluster_across:creating input files
INFO:ipyrad.assemble.cluster_across:concatenating sequences into _catcons.tmp file
INFO:ipyrad.assemble.cluster_across:sorting sequences into len classes
INFO:ipyrad.assemble.cluster_across:shuffling sequences within len classes & sampling alleles
INFO:ipyrad.assemble.cluster_across:sort/shuf/samp took 1 seconds
  [####################] 100%  clustering across 1/4  | 0:00:00 
INFO:ipyrad.assemble.cluster_across:building consens clusters
INFO:ipyrad.assemble.cluster_across:building reads file -- loading utemp file into mem
ERROR:ipyrad.core.assembly:IOError: File /home/deren/Documents/ipyrad/tests/cli/cli_consens/cli.utemp does not exist
  Saving Assembly.

Traceback (most recent call last):
  File "/home/deren/anaconda/bin/ipyrad", line 9, in <module>
    load_entry_point('ipyrad', 'console_scripts', 'ipyrad')()
  File "/home/deren/Documents/ipyrad/ipyrad/", line 361, in main, force=args.force, preview=args.preview)
  File "/home/deren/Documents/ipyrad/ipyrad/core/", line 1360, in run
  File "/home/deren/Documents/ipyrad/ipyrad/core/", line 1318, in step6
    randomseed], 45)
  File "/home/deren/Documents/ipyrad/ipyrad/core/", line 815, in _clientwrapper
  File "/home/deren/Documents/ipyrad/ipyrad/core/", line 1110, in _step6func
    force, randomseed, ipyclient)
  File "/home/deren/Documents/ipyrad/ipyrad/assemble/", line 954, in run
    clustbits = build_reads_file(data)
  File "/home/deren/Documents/ipyrad/ipyrad/assemble/", line 790, in build_reads_file
    updf = pd.read_table(uhandle, header=None)
  File "/home/deren/anaconda/lib/python2.7/site-packages/pandas/io/", line 498, in parser_f
    return _read(filepath_or_buffer, kwds)
  File "/home/deren/anaconda/lib/python2.7/site-packages/pandas/io/", line 275, in _read
    parser = TextFileReader(filepath_or_buffer, **kwds)
  File "/home/deren/anaconda/lib/python2.7/site-packages/pandas/io/", line 590, in __init__
  File "/home/deren/anaconda/lib/python2.7/site-packages/pandas/io/", line 731, in _make_engine
    self._engine = CParserWrapper(self.f, **self.options)
  File "/home/deren/anaconda/lib/python2.7/site-packages/pandas/io/", line 1103, in __init__
    self._reader = _parser.TextReader(src, **kwds)
  File "pandas/parser.pyx", line 353, in pandas.parser.TextReader.__cinit__ (pandas/parser.c:3246)
  File "pandas/parser.pyx", line 591, in pandas.parser.TextReader._setup_parser_source (pandas/parser.c:6111)
IOError: File /home/deren/Documents/ipyrad/tests/cli/cli_consens/cli.utemp does not exist
INFO:ipyrad.core.parallel:Shutting down [ipyrad-19256] remote Engines
seems like a vsearch problem?
Isaac Overcast
Apr 28 2016 23:14
uploaded a new sim_mt_genome in the ipsimdata.tar.gz. refmapping is golden (at least for SE). Gonna tackle PE next.
Deren Eaton
Apr 28 2016 23:27
Isaac Overcast
Apr 28 2016 23:29
I know, phew. No sense of how broken PE is yet, but i'll let you know when if figure it out... Did you do your talk at UArk? How'd it go?