These are chat archives for dereneaton/ipyrad

28th
Apr 2016
Isaac Overcast
@isaacovercast
Apr 28 2016 03:53
denovo+reference is working again. the only problem is the sloppy way i created the simulated reference (a handful of overlapped reads) causes some of them to fail due to high indels
denovo-reference and reference both work again as well, for SE (contingent on the above prob getting fixed)
Deren Eaton
@dereneaton
Apr 28 2016 12:08
Heck yeah. That's awesome to hear.
I'm finding that step6 does not scale well to very big data sets. Filling the h5 data base takes a long time, and it becomes huuuge (79GB). I have some ideas for reducing its size.
Deren Eaton
@dereneaton
Apr 28 2016 19:53
Ran into the following problem on the latest pull when running denovo:
 --------------------------------------------------
  ipyrad [v.0.1.87]
  Interactive assembly and analysis of RADseq data
 --------------------------------------------------
  loading Assembly: cli
  from saved path: ~/Documents/ipyrad/tests/cli/cli.json
  ipyparallel setup: Local connection to 4 Engines

  Step6: Clustering across 12 samples at 0.85 similarity
INFO:ipyrad.assemble.cluster_across:creating input files
INFO:ipyrad.assemble.cluster_across:concatenating sequences into _catcons.tmp file
INFO:ipyrad.assemble.cluster_across:sorting sequences into len classes
INFO:ipyrad.assemble.cluster_across:shuffling sequences within len classes & sampling alleles
INFO:ipyrad.assemble.cluster_across:sort/shuf/samp took 1 seconds
INFO:ipyrad.assemble.cluster_across:clustering
  [####################] 100%  clustering across 1/4  | 0:00:00 
INFO:ipyrad.assemble.cluster_across:building consens clusters
INFO:ipyrad.assemble.cluster_across:building reads file -- loading utemp file into mem
ERROR:ipyrad.core.assembly:IOError: File /home/deren/Documents/ipyrad/tests/cli/cli_consens/cli.utemp does not exist
  Saving Assembly.

Traceback (most recent call last):
  File "/home/deren/anaconda/bin/ipyrad", line 9, in <module>
    load_entry_point('ipyrad', 'console_scripts', 'ipyrad')()
  File "/home/deren/Documents/ipyrad/ipyrad/__main__.py", line 361, in main
    data.run(steps=steps, force=args.force, preview=args.preview)
  File "/home/deren/Documents/ipyrad/ipyrad/core/assembly.py", line 1360, in run
    self.step6(force=force)            
  File "/home/deren/Documents/ipyrad/ipyrad/core/assembly.py", line 1318, in step6
    randomseed], 45)
  File "/home/deren/Documents/ipyrad/ipyrad/core/assembly.py", line 815, in _clientwrapper
    stepfunc(*args)
  File "/home/deren/Documents/ipyrad/ipyrad/core/assembly.py", line 1110, in _step6func
    force, randomseed, ipyclient)
  File "/home/deren/Documents/ipyrad/ipyrad/assemble/cluster_across.py", line 954, in run
    clustbits = build_reads_file(data)
  File "/home/deren/Documents/ipyrad/ipyrad/assemble/cluster_across.py", line 790, in build_reads_file
    updf = pd.read_table(uhandle, header=None)
  File "/home/deren/anaconda/lib/python2.7/site-packages/pandas/io/parsers.py", line 498, in parser_f
    return _read(filepath_or_buffer, kwds)
  File "/home/deren/anaconda/lib/python2.7/site-packages/pandas/io/parsers.py", line 275, in _read
    parser = TextFileReader(filepath_or_buffer, **kwds)
  File "/home/deren/anaconda/lib/python2.7/site-packages/pandas/io/parsers.py", line 590, in __init__
    self._make_engine(self.engine)
  File "/home/deren/anaconda/lib/python2.7/site-packages/pandas/io/parsers.py", line 731, in _make_engine
    self._engine = CParserWrapper(self.f, **self.options)
  File "/home/deren/anaconda/lib/python2.7/site-packages/pandas/io/parsers.py", line 1103, in __init__
    self._reader = _parser.TextReader(src, **kwds)
  File "pandas/parser.pyx", line 353, in pandas.parser.TextReader.__cinit__ (pandas/parser.c:3246)
  File "pandas/parser.pyx", line 591, in pandas.parser.TextReader._setup_parser_source (pandas/parser.c:6111)
IOError: File /home/deren/Documents/ipyrad/tests/cli/cli_consens/cli.utemp does not exist
INFO:ipyrad.core.parallel:Shutting down [ipyrad-19256] remote Engines
deren@oud:~/Documents/ipyrad/tests$
seems like a vsearch problem?
Isaac Overcast
@isaacovercast
Apr 28 2016 23:14
uploaded a new sim_mt_genome in the ipsimdata.tar.gz. refmapping is golden (at least for SE). Gonna tackle PE next.
Deren Eaton
@dereneaton
Apr 28 2016 23:27
Yesssss
Isaac Overcast
@isaacovercast
Apr 28 2016 23:29
I know, phew. No sense of how broken PE is yet, but i'll let you know when if figure it out... Did you do your talk at UArk? How'd it go?