These are chat archives for dereneaton/ipyrad

23rd
Apr 2018
Jean-Rémi Trotta
@jrtrottablanc
Apr 23 2018 08:08 UTC

Hi @dereneaton & @isaacovercast I'm currently running an assembly of 600 paired-end GBS samples and something seems wrong with the step6 "Building database" part:

host compute node: [24 cores] on cnc6

  Step 3: Clustering/Mapping reads
  [####################] 100%  dereplicating         | 0:00:00  
  [####################] 100%  clustering            | 15:04:19  
  [####################] 100%  building clusters     | 0:01:17  
  [####################] 100%  chunking              | 0:00:07  
  [####################] 100%  aligning              | 9:13:30  
  [####################] 100%  concatenating         | 0:00:55  

  Step 4: Joint estimation of error rate and heterozygosity
  [####################] 100%  inferring [H, E]      | 0:19:35  

  Step 5: Consensus base calling 
  Mean error  [0.00487 sd=0.00184]
  Mean hetero [0.02345 sd=0.00408]
  [####################] 100%  calculating depths    | 0:05:32  
  [####################] 100%  chunking clusters     | 0:05:43  
  [####################] 100%  consens calling       | 2:48:17  

  Step 6: Clustering at 0.86 similarity across 600 samples
  [####################] 100%  concat/shuffle input  | 0:11:55  
  [####################] 100%  clustering across     | 4 days, 11:16:46  
  [####################] 100%  building clusters     | 0:19:43  
  [####################] 100%  aligning clusters     | 3:05:12  
  [####################] 100%  database indels       | 1:16:41  
  [####################] 100%  indexing clusters     | 1:23:42  
  [                    ]   0%  building database     | 3 days, 18:22:21

As you can see after more than 3 days building database is still at 0%, did you already face this issue? Thanks!

Isaac Overcast
@isaacovercast
Apr 23 2018 14:21 UTC
@jrtrottablanc This step can take quite some time, especially on a very large dataset. If you inspect top you should see one ipyrad process working. You should be able to inspect the progress withls -ltr *_across. Check back with me in another day or two, or if you see there is no process running in top or you see there are no changes to the *_across directory.