These are chat archives for dereneaton/ipyrad

9th
May 2016
Deren Eaton
@dereneaton
May 09 2016 01:10
Fought with h5py in step7 all day. Fixed a few bugs, but it was painful.
roneytan
@roneytan
May 09 2016 15:12
Hi Guys, this is Ron. I provided the four PE ddRAD samples for ipyrad development. I'm putting ipyrad through the paces on a couple of Macs I have. I have some questions, comment, bug reports, etc. Do you mind if I chime in?
Deren Eaton
@dereneaton
May 09 2016 15:19
yeah, please go ahead.
roneytan
@roneytan
May 09 2016 15:37
So, I did manage to get ipyrad working on my Mac Book Pro (running Yosemite). I had an installation of anaconda3 on there and was using an environment where I had Python 2.7.11 installed. I kept getting error messages when trying to use the Mac version, even when in the 2.7.11 environment. The odd thing is that my MacPro has miniconda3 installed with the same 2.7.11 environment and it worked fine. So, I removed anaconda3 from my laptop, installed the latest miniconda2 release, and now everything works.
Not sure what was going on. Conflicts with anaconda?
I've run through the same dataset you used to test ipyrad and it gets hung up on the same step (step 6)
I receive the following error: ERROR:ipyrad.core.assembly:index 1000 is out of bounds for axis 0 with size 1000
Caught unknown exception - index 1000 is out of bounds for axis 0 with size 1000
The log file reads: 2016-05-08 12:47:08,151 pid=87386 [assembly.py] ERROR index 1000 is out of bounds for axis 0 with size 1000
Deren Eaton
@dereneaton
May 09 2016 16:09
Hey Ron, the installation was fixed. We just had a bad version up for the mac. I will install v.0.2.3 today that should fix the other problem you encountered.
Deren Eaton
@dereneaton
May 09 2016 16:39
ugh, ok I'm testing on a mac and it seems the hangup on step6 is some mac/linux difference. I'm working on it.
roneytan
@roneytan
May 09 2016 16:48
ok. sounds good.
Deren Eaton
@dereneaton
May 09 2016 16:52
@isaacovercast have you run into this problem? I'm testing side by side now and for some reason subprocess.wait() hangs on Mac.
Deren Eaton
@dereneaton
May 09 2016 16:59
It might have to do with the pseudo-tty as well.
Isaac Overcast
@isaacovercast
May 09 2016 17:40
I'm rerunning step six on glenn's gbs
I haven't see this behavior on mac, but it's been a while since I ran step 6 (been working on step3 mostly).
ps @dereneaton re: step6 on the gbs, the progress bars are really useful. I mean comforting :+1:
Isaac Overcast
@isaacovercast
May 09 2016 17:47
@roneytan Thanks for the feedback, if you see anything else or if you have suggestions yeah definitely let us know!
Deren Eaton
@dereneaton
May 09 2016 17:48
@roneytan, OK things are working for v.0.2.5 now up on Anaconda for Mac.
revise: --MPI option does not appear to be working on the latest Mac install.
but it's working without it.
Deren Eaton
@dereneaton
May 09 2016 17:53
@isaacovercast I've only done testing of MPI on linux. Have you had success with it on Mac? I'll do some more testing.
Isaac Overcast
@isaacovercast
May 09 2016 18:32
I haven't done much work with MPI on mac, i can do some testing too.
Deren Eaton
@dereneaton
May 09 2016 18:33
OK, I think I figured out the problem. I was testing in a separate conda env.
## this usually wont work
$ conda create -n testenv python=2
$ source activate testenv
(testenv) $ conda install -c ipyrad ipyrad 
(testenv) $ ipcluster start 

## this works
$ conda create -n testenv python=2 -c ipyrad ipyrad
$ source activate testenv
(testenv) $ ipcluster start
Deren Eaton
@dereneaton
May 09 2016 18:44
nvm,
not working
Deren Eaton
@dereneaton
May 09 2016 19:30
hmm, I want to test whether the ipyrad installation is working from a fresh install. I can't tell whether it's not working b/c the conda recipe is broken, or if it's not working because there's something wrong with how I'm setting it up in a new env.
Deren Eaton
@dereneaton
May 09 2016 20:38
I'm testing in a docker container now.
Isaac Overcast
@isaacovercast
May 09 2016 20:41
Oh dude, so cool. We should totally distribute a docker container, that would be legit.
Deren Eaton
@dereneaton
May 09 2016 20:44
yeah, for sure.
well... I'm trying to install in docker... for some reason the download rate from anaconda just dropped to 4kB/s, so it's taking quite a while...
Deren Eaton
@dereneaton
May 09 2016 20:51
now 3kB/s
wtf, I'll do it later.
On a separate note: I've been thinking about removing clusters_hidepth from the stats summary, since I think it gives the impression clusters are being filtered out at step3 when in fact they are not being filtered until step5.
Instead maybe we should report the mean_depth.
Isaac Overcast
@isaacovercast
May 09 2016 20:55
Agreed. Maybe we can do mean and upper/lower 95%? If you think that'd be useful.
Deren Eaton
@dereneaton
May 09 2016 21:04
adding more things like upper/lower, etc, would be good for the full s3 stats output for sure, I think mean and std is not very informative for read depths.
Isaac Overcast
@isaacovercast
May 09 2016 21:05
Makes sense
Deren Eaton
@dereneaton
May 09 2016 22:36
OK, we're good. Everything install great into a docker container. Installing into other env in conda is something we might want to figure out eventually, but it may be a conda problem, not ours.