These are chat archives for dereneaton/ipyrad

29th
Sep 2016
jianlizhao
@jianlizhao
Sep 29 2016 01:50
Hi Isaac/Eaton I got a error massage at step 7. Do you have any suggestion. Thank you. : 2016-09-29 09:43:10,383 pid=25106 [load.py] WARNING skipping: no svd results present in old assembly
2016-09-29 09:43:10,513 pid=25106 [parallel.py] INFO ['ipcluster', 'start', '--daemonize', '--cluster-id=ipyrad-25106', '--engines=Local', '--profile=default', '--n=20']
2016-09-29 09:43:22,844 pid=25106 [assembly.py] INFO "Unable to open object (Object 'seqs' doesn't exist)"
2016-09-29 09:43:22,868 pid=25106 [assembly.py] INFO shutting down engines
Deren Eaton
@dereneaton
Sep 29 2016 17:41
@jianlizhao it seems like your step 6 might have not finished. Did it raise any errors?
James McDaniel
@jlmcdaniel
Sep 29 2016 18:12
@dereneaton @isaacovercast Hi, I'm at UW-Madison and I am trying to get ipyrad to run efficiently on CHTC's high-throughput computing. Would you recommend that I switch to CHTC's high performance computer cluster or is there a way to get it running efficiently the other way?
Isaac Overcast
@isaacovercast
Sep 29 2016 18:51
Hi @jlmcdaniel We have lots of users working on clusters, so i'm sure it should work for you. Have you tried following the ipyrad hpc guide: http://ipyrad.readthedocs.io/HPC_script.html?highlight=hpc
Deren Eaton
@dereneaton
Sep 29 2016 18:52
@jlmcdaniel Hi I was about to say the same. Our directions are mostly focused on TORQUE systems, but we have a short note on SLURM as well, which appears to be what CHTC uses. I see more notes about how to use SLURM on the CHTC website (http://chtc.cs.wisc.edu/HPCuseguide.shtml). I'm not sure of the differences between the two clusters you mention, but ipyrad should work on either one. The only real trick for HPC setups is to be sure to use the --MPI flag if you are connecting to CPUs across multiple nodes, this will make sure that ipyrad finds all of them.
James McDaniel
@jlmcdaniel
Sep 29 2016 19:22
@dereneaton @isaacovercast Thanks for the advice! I had been using the original pyrad program with our HTC system, but I would always run into an issue of completing the job within the time limit of 3-days with reverse complement clustering turned on. I got around the time constraint by splitting up the samples across multiple jobs on step 3 and then re-combining them for steps 4-7. With ipyrad and the new .json files, it seems that it would be difficult to split up the samples across multiple jobs, but maybe I am missing something? HPC/Slurm also seems to be different than the HTC system because the HTC system only allows you to connect to one computer "slot" at a time with multiple cores (http://chtc.cs.wisc.edu/helloworld.shtml). Sorry for the long response!
Isaac Overcast
@isaacovercast
Sep 29 2016 19:26
@jlmcdaniel We actually have quite a nice feature for "branching" subsets of samples and then "merging" them again (which there are docs for on the readthedocs site). The other good thing about ipyrad is that it is much faster than pyrad v3. How big is the dataset you want to analyse? How many lanes of data? How long are the reads?
James McDaniel
@jlmcdaniel
Sep 29 2016 19:37
@isaacovercast That's awesome! I should have read that part of the manual more carefully. I thought the branching method was just for running multiple parameters (i.e., clustering at 85% and 90%) at the same time. My dataset is ~50 samples, 100-bp single end reads. The entire dataset is ~60million reads and samples range from <1million reads up to 5million reads.
Isaac Overcast
@isaacovercast
Sep 29 2016 19:39
Oh yeah, don't even worry about branching. Depending on how many cores you can get that dataset should fly, no problems.
James McDaniel
@jlmcdaniel
Sep 29 2016 19:44
@isaacovercast Hmm, I wonder why I have so much trouble with steps 3 and 6? When I have reverse complement clustering turned off, I can get the dataset through all of the steps in less than 12 hours. When it is turned on, I was never able to get it to run all of the samples through step 3 before the 3-day limit on our HTC system. We can ask for up to 32 cores on a single job, but all of the cores are constrained to a single physical computer. Could this be the issue?
Isaac Overcast
@isaacovercast
Sep 29 2016 20:05
Not sure what's going on with steps3 and 6 with pyrad v.3, but if you get 32 cores ipyrad will crush it, no problem.
James McDaniel
@jlmcdaniel
Sep 29 2016 20:15

@isaacovercast Thanks again for the advice! I was having the same issues with ipyrad during step 3 as I was with pyrad v.3. Do you mind taking a quick look at my executable file?

!/bin/bash

run ipyrad

/mnt/gluster/jlmcdaniel/miniconda2/bin/python /mnt/gluster/jlmcdaniel/miniconda2/bin/ipcluster start --n 32 --profile=ipyrad --daemonize
sleep 60
/mnt/gluster/jlmcdaniel/miniconda2/bin/python /mnt/gluster/jlmcdaniel/miniconda2/bin/ipyrad -p /mnt/gluster/jlmcdaniel/ipyrad/porroglossum/params-porroglossum.txt -s 3 -c 32 --ipcluster --MPI

Isaac Overcast
@isaacovercast
Sep 29 2016 20:30
Looks fine. You don't need the MPI flag if all your cores are on the same compute node. It might help figure out what's going on if you add the -d flag to the ipyrad run, then we can look at the ipyrad_log.txt to make sure stuff is actually happening. Like this:
/mnt/gluster/jlmcdaniel/miniconda2/bin/ipyrad -p /mnt/gluster/jlmcdaniel/ipyrad/porroglossum/params-porroglossum.txt -s 3 -c 32 -d --ipcluster
Try running it again, then do tail -f ipyrad_log.txt, it should show you tons of output.
Also, send me your params file so i can check it out iovercast@gc.cuny.edu
James McDaniel
@jlmcdaniel
Sep 29 2016 20:51
@isaacovercast Thanks! I just sent the params file to you. I will work on getting a new job submitted to the cluster with the -d flag now.
Deren Eaton
@dereneaton
Sep 29 2016 21:07
@jlmcdaniel it looks like you are opening python in your run ipyrad script, which will cause a problem.
James McDaniel
@jlmcdaniel
Sep 29 2016 21:25

@dereneaton Hmm, when I remove python from the run-ipyrad script, I get an error message stating that it can't find python (at least I think that's what the message is conveying):

/var/lib/condor/execute/slot1/dir_133060/condor_exec.exe: /mnt/gluster/jlmcdaniel/miniconda2/bin/ipcluster: /var/lib/condor/execute/slot1/dir_3556041/miniconda2/bin/python: bad interpreter: No such file or directory