These are chat archives for dereneaton/ipyrad

15th
Aug 2016
Cycadales
@Cycadales
Aug 15 2016 10:03
Hi guys quick question with regards to running ipyrad and a virtual online HPC. I know you guys have put out a HPC script (http://ipyrad.readthedocs.io/HPC_script.html?highlight=nodes) is there anything I should running an assembly using ipyrad on a HPC?
Deren Eaton
@dereneaton
Aug 15 2016 14:47

@Cycadales Hey, sorry that HPC script needs to be updated. It is much simpler than that to run ipyrad on HPC now. Simply use the MPI flag when you run the command line and tell it how many cores you have available. e.g.,

ipyrad -p params-iptest.txt -s 123 -c 64 --MPI

Depending on your HPC system it might be necessary to source your .bashrc file again (remember this was a step during installation) so that your session can find your conda directory. If it wasn't sourced you would get an error that looks something like 'ipyrad: command not found'. If so, just add this command before ipyrad.

source ~/.bashrc
Cycadales
@Cycadales
Aug 15 2016 16:01
@dereneaton excellent thank you so much this is really helpful. I have one quick question too if you don't mind. I have been running a large run of 72 samples however, an error occurred during the final step six. I am unsure if this is a bug as the two samples which failed in step six also came up with NaN after step four (estimation of heterozygosity) from the step four outputs. To me this indicates something is not quite right with these samples (maybe polyploidy) or it could just be a bug?
Isaac Overcast
@isaacovercast
Aug 15 2016 17:28
@Cycadales You aren't the only one seeing this weird problem with a couple samples in step4, i'm working on figuring it out. haven't made much progress yet, but will let you know when i fix it.
Deren Eaton
@dereneaton
Aug 15 2016 17:33
@isaacovercast Hmm, maybe there is a zero division problem somewhere if samples have too little data.
@isaacovercast Anyway, we should get it working so that samples with no data just get skipped over (with a warning message) without step6 failing as a consequence.
Cycadales
@Cycadales
Aug 15 2016 18:46
@dereneaton @isaacovercast I am not sure is there would be zero data but they are the only two this happened to in a 72 sample data set. For the first samples that failed it had 2064088 filtered reads PE, 1363839 clusters and 22115 hi depth clusters and the second one is very similar. I do not get too many hi depth clusters due to such large genome sizes. You can see here: Ccal-15-18-7_S30_001_PairedTrim 19310 0.067297 0.029480
Ccal-15-18-8_S16_001_PairedTrim 29508 0.078264 0.032666
Ccal-15-18-9_S32_001_PairedTrim 29240 0.065740 0.031500
Ccal-15-19-10_S34_001_PairedTrim 24820 0.073103 0.031194
Ccal-15-19-1_S33_001_PairedTrim 20677 0.073399 0.028918
Ccal-15-19-2_S35_001_PairedTrim 26229 0.067964 0.030893
Ccal-15-19-3_S36_001_PairedTrim 18386 0.074341 0.023790
Ccal-15-19-4_S37_001_PairedTrim 16906 0.067374 0.031275
Ccal-15-19-5_S38_001_PairedTrim 18287 0.069497 0.033255
Ccal-15-19-6_S39_001_PairedTrim 20924 0.071255 0.031390
Ccal-15-19-7_S40_001_PairedTrim 23877 0.084965 0.027303
Ccal-15-19-8_S41_001_PairedTrim 18905 0.068206 0.032376
Ccal-15-19-9_S42_001_PairedTrim 20367 0.073678 0.033244
Ccal-15-21-10_S44_001_PairedTrim 20270 0.074325 0.030129
Ccal-15-21-1_S43_001_PairedTrim 29433 0.072993 0.029077
Ccal-15-21-2_S45_001_PairedTrim 22115 NaN NaN
Ccal-15-21-3_S46_001_PairedTrim 21545 0.067196 0.029985
Ccal-15-21-4_S47_001_PairedTrim 16840 0.069844 0.030484
Ccal-15-21-5_S48_001_PairedTrim 28666 0.069447 0.031449
Ccal-15-21-6_S49_001_PairedTrim 17702 0.073530 0.028057
Ccal-15-21-7_S50_001_PairedTrim 20689 0.070413 0.029614
Ccal-15-21-8_S51_001_PairedTrim 22181 0.071271 0.030211
Ccal-15-21-9_S52_001_PairedTrim 46150 0.068804 0.031108
Ccal-15-22-10_S54_001_PairedTrim 33247 0.078422 0.029145
Ccal-15-22-1_S53_001_PairedTrim 20979 0.072515 0.029580
Ccal-15-22-2_S55_001_PairedTrim 17832 0.069513 0.030224
Ccal-15-22-3_S56_001_PairedTrim 22862 0.073217 0.028919
Ccal-15-22-4_S57_001_PairedTrim 17372 0.070965 0.030438
Ccal-15-22-5_S58_001_PairedTrim 23808 0.072599 0.031368
Ccal-15-22-6_S59_001_PairedTrim 24148 NaN NaN
Ccal-15-22-7_S60_001_PairedTrim 21304 0.070204 0.033073
Ccal-15-22-8_S61_001_PairedTrim 22150 0.069753 0.031287
Ccal-15-22-9_S62_001_PairedTrim 25762 0.072899 0.031701
Ccal-15-23-10_S64_001_PairedTrim 24948 0.070189 0.028724
Ccal-15-23-1_S63_001_PairedTrim 29491 0.074741 0.031638
Ccal-15-23-2_S65_001_PairedTrim 20028 0.075471 0.027946
Ccal-15-23-3_S66_001_PairedTrim 25180 0.074473 0.032364
Ccal-15-23-4_S67_001_PairedTrim 33162 0.070073 0.030220
Deren Eaton
@dereneaton
Aug 15 2016 22:54
@Cycadales I see, thanks, we'll look into it. BTW, when entering a table or code into the gitter chat box if you enter your text in between three back tick marks (```) before and after it will print with monospaced font and look a lot cleaner.