These are chat archives for dereneaton/ipyrad

18th
Jun 2018
Robin Sleith
@R0cknRobin_twitter
Jun 18 16:20
@isaacovercast I am still have problems restarting step 6 from a checkpoint. I have tried upgrading to the newest version to no avail. The jobs are killed by the scheduler so I am not sure exactly how the shutdown process is occurring.
Isaac Overcast
@isaacovercast
Jun 18 18:33
@R0cknRobin_twitter Im willing to bet the schedule is not asking nicely when it kills ipyrad at wall time limit. In this case the assembly would not be able to clean up and record its state, which would explain this behavior. One way you might work around this is to let it run for 47 hours and then kill it nicely (for example with a ctrl+c, or a SIGINT (e.g. kill -2 <ipyrad_pid>), and let it clean itself up and record state. Then you can start again and it will start from the checkpoint.
Nathan Layman
@northbynate
Jun 18 20:04
Any chance the contig name can be included in the .loci file so we can calculate the length of each contig reported in the VCF when using a reference?
Robin Sleith
@R0cknRobin_twitter
Jun 18 20:44

@isaacovercast I am using a TORQUE scheduler, so I tried qsig -s 2 <ipyrad_jobid> . This gave me:

 -------------------------------------------------------------
  ipyrad [v.0.7.23]
  Interactive assembly and analysis of RAD-seq data
 -------------------------------------------------------------
  loading Assembly: texastest
  from saved path: /N/dc2/scratch/rsleith/ipyrad/texastest.json
  establishing parallel connection:
  host compute node: [24 cores] on c24
  host compute node: [24 cores] on c1
  host compute node: [24 cores] on c51
  host compute node: [24 cores] on c5
  host compute node: [24 cores] on c32

  Step 6: Clustering at 0.85 similarity across 155 samples
  [####################] 100%  concat/shuffle input  | 0:04:25  
  [####################] 100%  clustering across     | 6 days, 10:18:16  
  [####################] 100%  building clusters     | 0:02:50  
  [####################] 100%  aligning clusters     | 0:08:27  
  [####################] 100%  database indels       | 0:23:21  
  [####################] 100%  indexing clusters     | 0:16:46  
  [#########           ]  48%  building database     | 1 day, 8:48:07  
  Keyboard Interrupt by user
Traceback (most recent call last):
  File "/N/u/rsleith/Carbonate/miniconda2/bin/ipyrad", line 11, in <module>
    load_entry_point('ipyrad==0.7.23', 'console_scripts', 'ipyrad')()

The Keyboard Interrupt by user was a welcome sight, but when I restarted the job it started at the beginning.

Isaac Overcast
@isaacovercast
Jun 18 21:58
Is there anything in the *_across directory?
That kind of should work. If the right files are in the _across directory maybe we can just trick it.
        ## close client when done or interrupted
        finally:
            try:
                ## save the Assembly
                self.save()
First thing it does on keyboard interrupt is save itself, which should include the checkpoint
There should be a texastest.json file in your working directory. Can you open this and look for "_checkpoint "
Robin Sleith
@R0cknRobin_twitter
Jun 18 22:34
"_checkpoint":0,
I will have to wait for clustering across to finish, looking back I definitely should have zipped or saved the _across directory because now that it started over it looks like everything that was in there is gone