These are chat archives for dereneaton/ipyrad

25th
Oct 2016
LinaValencia85
@LinaValencia85
Oct 25 2016 13:01

Hi @isaacovercast @dereneaton I have a silly question. I have run iPYRAD before and has worked just fined but now I am trying to run it again with a new data set and I keep having an error when running s1 by itself and s12 together -------------------------------------------------------------
ipyrad [v.0.4.2]

Interactive assembly and analysis of RAD-seq data

New Assembly: nwm-s1_sub
host compute node: [48 cores] on nid00008

Step 1: Loading sorted fastq data to Samples
[####################] 100% loading reads | 0:00:29

Encountered an unexpected error (see ./ipyrad_log.txt)
Error message is below -------------------------------
IOError(CRC check failed 0x950f2c24 != 0xfd5df48eL)

Any ideas of why I getting this? I am using ipyrad 0.4.2
Isaac Overcast
@isaacovercast
Oct 25 2016 15:24
Hi @LinaValencia85 I have seen that problem before. It means there's a problem with the raw data, probably one of the files was corrupted or malformed. You can verify the integrity of your raw data with gunzip -tv *.gz. this will tell you which file is bad and then hopefully you can redownload it.
NB: You have to be in the directory your raw data files are in for that to command to work.
LinaValencia85
@LinaValencia85
Oct 25 2016 16:30
Perfect! Thanks a lot @isaacovercast
Sara Lipshutz
@slipshut
Oct 25 2016 17:45

Hi @isaacovercast @dereneaton, I'm running the beginner's tutorial and ran into this problem at step 6: (my_root)[slipshut@cypress1 tutorial]$ ipyrad -p params-iptest.txt -s 6


ipyrad [v.0.4.4]

Interactive assembly and analysis of RAD-seq data

loading Assembly: iptest
from saved path: /lustre/project/ederrybe/ipyrad-sara/tutorial/iptest.json
local compute node: [40 cores] on cypress1

Step 6: Clustering at 0.85 similarity across 12 samples
[####################] 100% concat/shuffle input | 0:00:03
[####################] 100% clustering across | 0:00:00
[####################] 100% building clusters | 0:00:00
[####################] 100% aligning clusters | 0:00:11
[####################] 100% database indels | 0:00:00
[ ] 0% indexing clusters | 0:00:04 ERROR:ipyrad.assemble.cluster_across: error in singlecat (1B_0) UntypedAttributeError(Caused By:
Traceback (most recent call last):
File "/share/apps/anaconda/2/2.5.0/lib/python2.7/site-packages/numba/compiler.py", line 246, in run
stage()
File "/share/apps/anaconda/2/2.5.0/lib/python2.7/site-packages/numba/compiler.py", line 472, in stage_nopython_frontend
self.locals)
File "/share/apps/anaconda/2/2.5.0/lib/python2.7/site-packages/numba/compiler.py", line 803, in type_inference_stage
infer.propagate()
File "/share/apps/anaconda/2/2.5.0/lib/python2.7/site-packages/numba/typeinfer.py", line 587, in propagate
raise errors[0]
UntypedAttributeError: Unknown attribute "any" of type Module(<module 'numpy' from '/share/apps/anaconda/2/2.5.0/lib/python2.7/site-packages/numpy/__init__.pyc'>)
File "../../../../../home/slipshut/.conda/envs/my_root/lib/python2.7/site-packages/ipyrad/assemble/cluster_across.py", line 883

Failed at nopython (nopython frontend)
Unknown attribute "any" of type Module(<module 'numpy' from '/share/apps/anaconda/2/2.5.0/lib/python2.7/site-packages/numpy/__init__.pyc'>)
File "../../../../../home/slipshut/.conda/envs/my_root/lib/python2.7/site-packages/ipyrad/assemble/cluster_across.py", line 883)

Encountered an unexpected error (see ./ipyrad_log.txt)
Error message is below -------------------------------
tuple index out of range

Isaac Overcast
@isaacovercast
Oct 25 2016 20:35
Hi @slipshut Welcome!!!!!
Sara Lipshutz
@slipshut
Oct 25 2016 20:36
Hey @isaacovercast, super excited to be using ipyrad!
Isaac Overcast
@isaacovercast
Oct 25 2016 20:36
I've seen this error before, some computers have a problem with one of the fancy python libaries we use to make it run faster. It looks like you're running on a cluster my bet is this is the problem. I'm trying to figure out/remember how to fix this...
Are you running the jacana data again? or is this something new?
Sara Lipshutz
@slipshut
Oct 25 2016 20:37
I'm running the jacana data with a genome from Erich Jarvis's 10x genomes project
And I'm working with another phd student, Emily, on golden crowned sparrows using the white-throated sparrow genome
We actually didn't install ipyrad through the Tulane HPC people, and instead are using "conda create -n my_root --clone=/share/apps/anaconda/2/2.5.0"
Isaac Overcast
@isaacovercast
Oct 25 2016 20:40
Oh cool. yeah that should work.
Can you re-run step 6 with the -d flag:
ipyrad -p params-iptest.txt -s 6 -d
This will create a file called ipyrad_log.txt. Can you email me that file after the next run fails?
Sara Lipshutz
@slipshut
Oct 25 2016 20:43
will do! i'm using a params file called params-data1.txt fyi
Isaac Overcast
@isaacovercast
Oct 25 2016 20:46

Also, this would be helpful to check the version of numba installed. You can do it like this at a terminal:

python

This will put you in a python shell, then type this to get the numba version:

 >>> import numba
 >>> numba.__version__
 0.29.0+0.gead0dd3.dirty'
(Mine is 0.29.0)
Also numpy:
>>> import numpy
>>> numpy.__version__
Sara Lipshutz
@slipshut
Oct 25 2016 20:51
Looks like we got numba version 0.24.0
and numpy version 1.10.4
Isaac Overcast
@isaacovercast
Oct 25 2016 20:53
I'll bet that's the problem. Since you are using conda env you should be able to update these:
conda install numpy numba
That should prompt you to update to the newest version. I'll bet that'll fix it.
It's weird because our package should enforce versions on dependencies...
Sara Lipshutz
@slipshut
Oct 25 2016 21:00
Hmm, definitely updated those packages but step 6 still had an error. I'll email the new log
Isaac Overcast
@isaacovercast
Oct 25 2016 23:00
@slipshut I'm pretty sure it must be some package version in the environment. When you created the conda env you passed the --clone flag which looks like it copied the local conda repo. You don't have to use the --clone flag though, if you just do something like this then it will create a new blank conda env, switch to it and install all the right ipyrad dependencies:
conda create --name ipyrad python=2.7
source activate ipyrad
conda install -c ipyrad ipyrad
You can do this to list all available environments and make sure you are using the right one:
conda info --envs
Try that and let me know how it goes.
Deren Eaton
@dereneaton
Oct 25 2016 23:19
@slipshut @isaacovercast I agree, cloning the system libraries could speed things up slightly, but it is much easier to just install a local anaconda library for yourself, as in the ipyrad installation docs. That way you know that all of the software is exactly the version you want it to be.
@slipshut @isaacovercast especially since we're using some bleeding edge python libraries like numba.