These are chat archives for dereneaton/ipyrad

9th
Mar 2016
Isaac Overcast
@isaacovercast
Mar 09 2016 01:44
re: hackathon Saturday yeah we could do that, i don't have any specific plans. sounds like a good idea.
Where were you thinking location-wise?
Re: --mpi flag, i haven't really fucked with it that much, so it's hard to say. What little i've tried on the cuny cluster and on the lsu cluster hasn't seemed to work (for CLI), at least across cluster nodes, which is when it's really useful.
Isaac Overcast
@isaacovercast
Mar 09 2016 01:53
Also, question about memory usage, denovo step 3 is really 2 steps, clustering and alignment. Is your impression that clustering is much more memory intensive than alignment? That's my impression, and it raised some red flags (or maybe yellow flags) on the LSU cluster...
Deren Eaton
@dereneaton
Mar 09 2016 06:26
Cool. No idea, where's good for hacking? I haven't tried large connections with MPI recently, it's probably not up to par yet.
alignment is low memory, the speed is limited by our inefficient I/O to muscle. Vsearch is pretty low memory too, unless you give it unlimited threads. But maybe if too many are running concurrently the memory gets too high. I suppose in the case that is the problem the user should just use fewer cores. I've been thinking too that it would be kind of nice to just make the aligning a separate step instead of this state=2.5 business.
Loading the utemp into memory to build the clusters could crank up memory too, but I would expect that problem to occur in step6 more dramatically than step3.
Isaac Overcast
@isaacovercast
Mar 09 2016 19:32
re: saturday, idk, somewhere with lots of megahertz. And coffee. There's a couple coffee shops downtown that are decent. Could also work at the lab at city college, or at the grad center (midtown). lots of options..
Deren Eaton
@dereneaton
Mar 09 2016 21:55
I'm planning to meet up with a friend in Brooklyn later Saturday night. So unless you're planning to head up north we could meet closer to you.
Isaac Overcast
@isaacovercast
Mar 09 2016 21:55
Sounds good.