These are chat archives for dereneaton/ipyrad

24th
Jul 2018
carol-rowe666
@carol-rowe666
Jul 24 2018 01:00
I have 4 closely related plant species: two diploids and two allotetraploids. I don’t want to split the data, as I want to find the origins of the allotetraploids. Also, I do not have a reference sequence. What assembly parameters would you recommend? (In particular, what would I enter for max_alleles_consens, what would be the best clustering approach, etc.?) Thanks for your input.
Isaac Overcast
@isaacovercast
Jul 24 2018 13:50
@ahaponski_twitter Yep, that's right, monomorphic loci won't be included in the .u.snps.phy file or in the vcf.
tommydevitt
@tommydevitt
Jul 24 2018 16:19
@isaacovercast So the ipcluster engine is running, all input files are created, but it doesn't seem Structure is running because no output is ever created. What am I missing?
Isaac Overcast
@isaacovercast
Jul 24 2018 16:24
@tommydevitt What is in your workdir? Can you see running processes in top? It could just be taking a long time to run, depending on what your data looks like structure can take quite some time.
tommydevitt
@tommydevitt
Jul 24 2018 16:29
@isaacovercast The only items in the workdir are the input files created for each K. The only process running in top that I recognized was ipcluster. The dataset isn't huge (500 loci, ~140 individuals) and I was only doing a test run of 1,000 burn-in and 5,000 reps. Gave it 6-8 hours (on 4 cores) but nothing resulted. I would think that would be more than enough time based on previous structure analyses, but maybe I'm wrong.
Isaac Overcast
@isaacovercast
Jul 24 2018 16:35
Can you show me the output of ls -ltr in your workdir?
tommydevitt
@tommydevitt
Jul 24 2018 16:37
-rw------- 1 tdevitt G-815494 361338 Jul 23 13:02 tmp-quick-3-3.strfile.txt
-rw------- 1 tdevitt G-815494 803 Jul 23 13:02 tmp-quick-3-3.mainparams.txt
-rw------- 1 tdevitt G-815494 2568 Jul 23 13:02 tmp-quick-3-3.extraparams.txt
-rw------- 1 tdevitt G-815494 361338 Jul 23 13:02 tmp-quick-3-2.strfile.txt
-rw------- 1 tdevitt G-815494 803 Jul 23 13:02 tmp-quick-3-2.mainparams.txt
-rw------- 1 tdevitt G-815494 2568 Jul 23 13:02 tmp-quick-3-2.extraparams.txt
-rw------- 1 tdevitt G-815494 361338 Jul 23 13:02 tmp-quick-3-1.strfile.txt
-rw------- 1 tdevitt G-815494 803 Jul 23 13:02 tmp-quick-3-1.mainparams.txt
-rw------- 1 tdevitt G-815494 2568 Jul 23 13:02 tmp-quick-3-1.extraparams.txt
-rw------- 1 tdevitt G-815494 361338 Jul 23 13:02 tmp-quick-3-0.strfile.txt
-rw------- 1 tdevitt G-815494 803 Jul 23 13:02 tmp-quick-3-0.mainparams.txt
-rw------- 1 tdevitt G-815494 2568 Jul 23 13:02 tmp-quick-3-0.extraparams.txt
-rw------- 1 tdevitt G-815494 361338 Jul 23 13:02 tmp-quick-2-3.strfile.txt
-rw------- 1 tdevitt G-815494 803 Jul 23 13:02 tmp-quick-2-3.mainparams.txt
-rw------- 1 tdevitt G-815494 2568 Jul 23 13:02 tmp-quick-2-3.extraparams.txt
-rw------- 1 tdevitt G-815494 361338 Jul 23 13:02 tmp-quick-2-2.strfile.txt
-rw------- 1 tdevitt G-815494 803 Jul 23 13:02 tmp-quick-2-2.mainparams.txt
-rw------- 1 tdevitt G-815494 2568 Jul 23 13:02 tmp-quick-2-2.extraparams.txt
-rw------- 1 tdevitt G-815494 361338 Jul 23 13:02 tmp-quick-2-1.strfile.txt
-rw------- 1 tdevitt G-815494 803 Jul 23 13:02 tmp-quick-2-1.mainparams.txt
-rw------- 1 tdevitt G-815494 2568 Jul 23 13:02 tmp-quick-2-1.extraparams.txt
-rw------- 1 tdevitt G-815494 361338 Jul 23 13:02 tmp-quick-2-0.strfile.txt
-rw------- 1 tdevitt G-815494 803 Jul 23 13:02 tmp-quick-2-0.mainparams.txt
-rw------- 1 tdevitt G-815494 2568 Jul 23 13:02 tmp-quick-2-0.extraparams.txt
Isaac Overcast
@isaacovercast
Jul 24 2018 16:43
When you all call the run() function on the structure object it'll create async results for each replicate, you can query status, also are you sure you're connected to the ipcluster?
ipyclient = ipp.Client()
print("Connected to {} cores".format(len(ipyclient)))
What does this say?
tommydevitt
@tommydevitt
Jul 24 2018 16:43
Connected to 4 cores
Isaac Overcast
@isaacovercast
Jul 24 2018 16:43
Assuming your structure object is called struct what does this say:
for async in struct.asyncs:
    print(async.ready()),
tommydevitt
@tommydevitt
Jul 24 2018 16:50
Hang on, forgot to unload python2 and am getting errors. Just a sec.
tommydevitt
@tommydevitt
Jul 24 2018 17:06
That returned
True True True True True True True True
Isaac Overcast
@isaacovercast
Jul 24 2018 17:39
Well this indicates that all the runs have completed.
tommydevitt
@tommydevitt
Jul 24 2018 17:41
Shouldn't there be output files in the workdir?
Isaac Overcast
@isaacovercast
Jul 24 2018 17:41
perhaps the runs failed.
tommydevitt
@tommydevitt
Jul 24 2018 17:43
Hm, wonder why? Is there a log file created somewhere?
Isaac Overcast
@isaacovercast
Jul 24 2018 17:43
Are you sure structure is installed?
which structure
I did conda remove structure and now i'm seeing exactly the behavior you're seeing.
tommydevitt
@tommydevitt
Jul 24 2018 17:45
which structure gives me /home1/02745/tdevitt/miniconda2/bin/structure
Isaac Overcast
@isaacovercast
Jul 24 2018 17:51
Can you show me the full path to your workdir?
tommydevitt
@tommydevitt
Jul 24 2018 17:55
/work/02745/tdevitt/lonestar/Eurycea/JA17735/structure-analysis/
Isaac Overcast
@isaacovercast
Jul 24 2018 17:56
You could try running structure by hand to see if you can figure out what's going on:
    cmd = ["structure",
           "-m", mname,
           "-e", ename,
           "-K", str(kpop),
           "-D", str(seed),
           "-N", str(ntaxa),
           "-L", str(nsites),
           "-i", sname,
           "-o", outname]
You can call this at the command line, but you'll need to substitute in the appropriate values.
tommydevitt
@tommydevitt
Jul 24 2018 18:02
OK, I'll give this a try later. Thanks for your help!