These are chat archives for dereneaton/ipyrad

31st
Oct 2016
LinaValencia85
@LinaValencia85
Oct 31 2016 15:40
Hi @dereneaton any luck trying to figure out the memoryerror on s6? As others in the group I keep getting this error on s6: [ ] 3% indexing clusters | 0:00:45
Encountered an unexpected error (see ./ipyrad_log.txt)
Error message is below -------------------------------
tuple index out of range
and wehn I check the ipyrad_log.txt I get the following:
2016-10-29 23:07:48,085 pid=26342 [cluster_across.py] ERROR error in singlecat (CPB104) MemoryError()
Jenny Archibald
@jenarch
Oct 31 2016 21:37

Hi @dereneaton and @isaacovercast

I am trying to run two datasets with very similar settings (both paired radseq data). Both are being stopped by errors, but with different errors. I've used pyrad successfully in the past but so far can't get ipyrad to work.

One dataset has been repeatedly crashing with a memory error, possibly related to the discussion above but it looks like this:


Step6: Clustering across 288 samples at 0.9 similarity
[####################] 100% concat/shuf input | 0:09:32
[####################] 100% clustering across | 3 days, 12:49:03
[####################] 100% aligning clusters | 1:41:08
Caught unknown exception - OSError([Errno 12] Cannot allocate memory)
Saving Assembly.

Traceback (most recent call last):
[portion deleted to save space, please let me know if you want to see it]
File "/tools/cluster/6.2/ipyrad/0.3.5/lib/python2.7/site-packages/ipyparallel/client/asyncresult.py", line 166, in get
raise self.exception()

ipyparallel.error.RemoteError: OSError([Errno 12] Cannot allocate memory)"

I've tried increasing the requested memory several times (up to 200gb in the last run), but so far with no luck. I have another run in the queue now where I explicitly tell ipyrad how many cores to use, in case that is the problem?

The other dataset crashes almost immediately with this error:


Traceback (most recent call last):
[portion deleted to save space, please let me know if you want to see it]
File "/tools/cluster/6.2/ipyrad/0.3.5/lib/python2.7/site-packages/ipyrad/core/assembly.py", line 648, in set_params
param = self.paramsdict.keys()[int(param)]

IndexError: list index out of range

Can you give any suggestions on how to get these running? Thanks!

Isaac Overcast
@isaacovercast
Oct 31 2016 21:45
The second one is easy, looks like a malformed params file.
Also, looks like you're using 0.3.5, I would definitely upgrade conda install -c ipyrad ipyrad
If you send me your params file i'll take a look at it: iovercast@gc.cuny.edu