These are chat archives for dereneaton/ipyrad

1st
May 2016
Isaac Overcast
@isaacovercast
May 01 2016 02:17
w00t PE refmap "working" again
There's a slight problemo tho. With PE when we pull in the reference sequence to improve alignment if the reads do not overlap (which they normally won't at least w/ ddrad) then the reference sequence is much longer than the merged PE so there are hella indels and all sequences get filtered.... :-/
For the moment i'm going to disable including the reference sequence for PE, just so i can keep hacking, but we should figure out a strategy for this
Isaac Overcast
@isaacovercast
May 01 2016 02:23
oof, fucking hallelujah, disabling including the reference sequence for alignment and we're back in business for PE! :rocket: :fire:
still a few bugs to work out tho :bug:
Deren Eaton
@dereneaton
May 01 2016 05:00
I think bc we normally split the pairs for alignment if they aren't merged we need to split the pulled in reference too.
Deren Eaton
@dereneaton
May 01 2016 17:24
@isaacovercast Can you send me the musMT.fa file?
Deren Eaton
@dereneaton
May 01 2016 17:29
sorry if you already did a while back. I might have lost it.
I'm thinking about making the storage of depth information optional. It's an awesome feature, but most people doing phylogenetics aren't going to use the VCF output, or the depth information at all. And so by storing it we're just wasting a lot of time and disk space. But, should we make storing depths the default, or not storing the default? Probably not storing...
Isaac Overcast
@isaacovercast
May 01 2016 18:00
You want the one w/ simulated reads inserted?
re: depth info, yeah i think not storing by default is probably fine, agree most people won't use it.
Deren Eaton
@dereneaton
May 01 2016 18:13
no, without insert
Isaac Overcast
@isaacovercast
May 01 2016 18:33
done
Isaac Overcast
@isaacovercast
May 01 2016 19:17
Is there a way to set noreverse=True for CLI? shouldn't this just be a paramsdict item?
Deren Eaton
@dereneaton
May 01 2016 19:20
thanks
yeah... or hackerzdict. I don't expect it will be used very often.
Isaac Overcast
@isaacovercast
May 01 2016 19:21
something is revcomping the R2 reads before they get mapped, any quick idea where this could be?
Deren Eaton
@dereneaton
May 01 2016 19:21
in which case it could also just be left as an option that only API users can access.
util line 327?
Isaac Overcast
@isaacovercast
May 01 2016 19:24
yep! Good eye... That bug has been giving me serious trouble. I was mapping PE by hand and it'd work great, but in the pipeline it'd fsck. omg i was about to quit my phd and move to the hills....
Deren Eaton
@dereneaton
May 01 2016 19:24
lol
Isaac Overcast
@isaacovercast
May 01 2016 19:25
dude i'm serious, i thought i was going crazy.
Deren Eaton
@dereneaton
May 01 2016 19:25
sorry about that, pretty sure I stuck it in there.
Isaac Overcast
@isaacovercast
May 01 2016 19:26
np, just glad i figured it out finally... you think it's worth having a flag for merge_pairs to tell it not to revcomp?
for PE
Im gonna do it, bcz pe r2 shoudn't be revcomp before mapping, and merging mapped reads after mapping shouldn't be revcomp either, bcz samtools already does it for us.
Deren Eaton
@dereneaton
May 01 2016 19:28
sounds good. It seems like there might be cases where the either need to be or not depending on map vs not mapping, perhaps.
Deren Eaton
@dereneaton
May 01 2016 19:33
finished step6 on a 84 taxon 2 lanes of data assembly.
  Step6: Clustering across 84 samples at 0.85 similarity
  [####################] 100%  clustering across 1/3  | 7:12:38
  [####################] 100%  aligning clusters 2/3  | 0:39:23
  [####################] 100%  building database 3/3  | 2:57:40
  Saving Assembly.
using 16 cores on HPC. 3 hours is not too bad for building the full depth data base.
...seems like the alignment is going almost too fast tho.
Deren Eaton
@dereneaton
May 01 2016 19:49
but the results look good.
Isaac Overcast
@isaacovercast
May 01 2016 19:50
Thinking about upgrading to 0.2.0 mostly in honor of the massive rewrite of step3 w/ apply(). what do you think?
Deren Eaton
@dereneaton
May 01 2016 19:50
I was thinking the same
Isaac Overcast
@isaacovercast
May 01 2016 19:50
:+1:
Deren Eaton
@dereneaton
May 01 2016 19:50
and we're getting pretty high in the 0.1 numbers
Isaac Overcast
@isaacovercast
May 01 2016 19:50
i know!
added the revcomp flag to merge_pairs() and pe started working. Thank you lord jesus good christ.
p cool you can actually edit the releases on github to make notes on the changes..