These are chat archives for dereneaton/ipyrad

Dec 2015
Deren Eaton
Dec 01 2015 18:36
I'm thinking again changing how we launch ipcluster

I know I keep going back and forth on this, but I'm really thinking that ipcluster should be launched at ip.__init__ instead of during Assembly.__init__. It just feels too clunky otherwise.

This would be really easy for the CLI in that we just launch ipcluster in __main__, and allow an argument from the command line to specify the controller (e.g., Local is default, MPI, PBS), and the number of engines (All is default).

Problem is lauching ipcluster in __init__ is a mess when running interactive since each launched Engine also runs ip.__init__. An easy solution is have users run ipcluster themselves before interactive sessions, but we were shying away from that before.

Do you think having an integrated ipcluster for the CLI, and a user-run ipcluster for API is reasonable?

Deren Eaton
Dec 01 2015 22:56
Alternatively, the connections happen behind the scenes enough that they could remain in Assembly and simply not be reported explicitly, and it would seem less clunky. As long as users can change the connection type fairly seamlessly then it would be nice to avoid having to make a separate bash call to ipcluster start.