These are chat archives for dereneaton/ipyrad

26th
Jan 2016
Isaac Overcast
@isaacovercast
Jan 26 2016 02:32
Hmm. Agreed. I shall marinate upon it....
Deren Eaton
@dereneaton
Jan 26 2016 02:33
I just got an email about that workshop. ..
Deren Eaton
@dereneaton
Jan 26 2016 02:57
They want to put the code on their system well ahead of time... pressure is on.
Isaac Overcast
@isaacovercast
Jan 26 2016 14:32
Did they give you a deadline?
Isaac Overcast
@isaacovercast
Jan 26 2016 15:35
“I don't need time, I need a deadline.”
― Duke Ellington
Did a full demultiplex of one plate of gbs data. Raw file is 25GB .gz, full-run_fastqs directory is 16GB? Should I be worried or is it normal for the data to shrink by this much during step 1?
Isaac Overcast
@isaacovercast
Jan 26 2016 15:46
raw_file                          total_reads    cut_found  bar_matched
D25GWACXX_6_fastq                        297132667    249086024    222233136
I guess that looks normal
Deren Eaton
@dereneaton
Jan 26 2016 20:36
Let's aim for a deadline of the third week of Feb to have all simulated data sets running as tests, to have clean runs on real data sets of all types, and to have solid docs.
Deren Eaton
@dereneaton
Jan 26 2016 20:41
Those step1 results seem reasonable. How long does it take to run?
Deren Eaton
@dereneaton
Jan 26 2016 20:52
getting the conda install working is a big deal too, I suppose.
Deren Eaton
@dereneaton
Jan 26 2016 21:00
or do we plan to just provide a channel like: conda install -c https://conda.anaconda.org/iovercast ipyrador by listing a pypi channel? I forget where we left off on this...
Deren Eaton
@dereneaton
Jan 26 2016 21:24
but I guess we still have the problem that pip doesn't install all of the dependencies we want, even when we don't account for numba
Isaac Overcast
@isaacovercast
Jan 26 2016 22:27
Step1 took ~6hrs on my shitty mac desktop. I made a wiki page for tracking runtimes on real data, figure this is something folks are interested in:
pip is going to be tricky to support. Conda should be easy. I will have to continue marinating on the best way to get pip working.
Deren Eaton
@dereneaton
Jan 26 2016 22:33
6 hrs isn't too bad. I guess that would be only about 1 hour if you had 40 cores, which is pretty satisfying.
Isaac Overcast
@isaacovercast
Jan 26 2016 22:34
I'll eventually test the whole pipeline on a better box. Yeah 1 hour would be tight.