These are chat archives for broadinstitute/hail

17th
Aug 2016
Laurent Francioli
@lfrancioli
Aug 17 2016 17:36
Will I need to reimport if I fast-forward my forked master to the current BI master?
cseed
@cseed
Aug 17 2016 17:36
No.
Laurent Francioli
@lfrancioli
Aug 17 2016 17:36
OK, great!
thanks!
cseed
@cseed
Aug 17 2016 17:36
Err, I mean, probably not. How long ago did you branch off?
Laurent Francioli
@lfrancioli
Aug 17 2016 17:36
haha
after the last reimport fest
cseed
@cseed
Aug 17 2016 17:37
No recent changes require reimport, not since the last major one.
Laurent Francioli
@lfrancioli
Aug 17 2016 17:37
OK, so the BGEN / dosage stuff shouldn’t require reimport?
cseed
@cseed
Aug 17 2016 17:37
Nope!
Oh, wait.
Just double-checked with Jackie, will not require reimport.
Laurent Francioli
@lfrancioli
Aug 17 2016 17:38
great, thanks!
Laurent Francioli
@lfrancioli
Aug 17 2016 17:47
I’m getting this :( hail: fatal: read: corrupt VDS: invalid metadata Missing field `isDosage' Recreate VDS with current version of Hail.
Tim Poterba
@tpoterba
Aug 17 2016 17:48
okay, we can fix that
Laurent Francioli
@lfrancioli
Aug 17 2016 17:49
like now?
Or should I rollback?
cseed
@cseed
Aug 17 2016 17:49
Like right now.
Laurent Francioli
@lfrancioli
Aug 17 2016 17:49
so fast!
cseed
@cseed
Aug 17 2016 17:50
Tim is typing in the fix as we speak!
Laurent Francioli
@lfrancioli
Aug 17 2016 17:51
pushing/pulling is the bottleneck it seems :)
Also Tim, since you are around, should someStruct.filter(Set(“va.info.XXX”)) work and keep only va.info.XXX(and delete va.info.YYY and va.ZZZ) ?
Tim Poterba
@tpoterba
Aug 17 2016 17:52
that looks like it won’t typecheck
filter takes a (Field) => Boolean or a Set[String]
Laurent Francioli
@lfrancioli
Aug 17 2016 17:53
oops, let me edit
there
Tim Poterba
@tpoterba
Aug 17 2016 17:54
that won’t do what you want.
are you calling this on va or va.info?
Laurent Francioli
@lfrancioli
Aug 17 2016 17:55
va
Tim Poterba
@tpoterba
Aug 17 2016 17:55
there’s no way to filter out info.XXX in a filter operation on va
at least, I don’t think so
Laurent Francioli
@lfrancioli
Aug 17 2016 17:55
OK, that’s what I thought
cseed
@cseed
Aug 17 2016 17:56
What about {“va”: {“info”: {“XXX”: va.info.XXX } } }?
Laurent Francioli
@lfrancioli
Aug 17 2016 17:56
I can filter info but not info.XXX)
Tim Poterba
@tpoterba
Aug 17 2016 17:56
right @lfrancioli
cseed
@cseed
Aug 17 2016 17:57
or va = {info: {XXX: va.info.XXX}}}
I admit, nested select would be nice.
Laurent Francioli
@lfrancioli
Aug 17 2016 17:58
@cseed : that could potentially work…
cseed
@cseed
Aug 17 2016 18:01
or va = {info: select(va.info, XXX)}
We should really be able to write select(va, info.XXX, info.YYY, rsID, …) and select those and only those.
Laurent Francioli
@lfrancioli
Aug 17 2016 18:02
that’s actually pretty nice! I’m just thinking now how to best integrate this in my use-case
I might give a shot at the nested select if I am not able to resolve it cleanly!
Tim Poterba
@tpoterba
Aug 17 2016 20:55
the bgen back compatibility fix is in @lfrancioli
Laurent Francioli
@lfrancioli
Aug 17 2016 21:08
awesome, thanks @tpoterba !!!
Tim Poterba
@tpoterba
Aug 17 2016 21:15
  def evalNumeric[T](g: (T) => Any)(implicit conv: NumericConversion[T]): (Any) => Any = {
    (a: Any) => {
      if (a == null)
        null
      else g(conv.to(a))
    }
  }

  def evalComposeNumeric[T](ec: EvalContext, subexpr: AST)
    (g: (T) => Any)
    (implicit convT: NumericConversion[T]): () => Any = {
    val f = subexpr.eval(ec)
    val wrapped = evalNumeric[T](g)
    () => wrapped(f())
  }

  def evalNumeric[T1, T2](g: (T1, T2) => Any)(implicit conv1: NumericConversion[T1], conv2: NumericConversion[T2]): (Any, Any) => Any = {
    (a1: Any, a2: Any) => {
      if (a1 == null || a2 == null)
        null
      else g(conv1.to(a1), conv2.to(a2))
    }
  }

  def evalComposeNumeric[T1, T2](ec: EvalContext, subexpr1: AST, subexpr2: AST)
    (g: (T1, T2) => Any)
    (implicit convT1: NumericConversion[T1], convT2: NumericConversion[T2]): () => Any = {
    val f1 = subexpr1.eval(ec)
    val f2 = subexpr2.eval(ec)
    val wrapped = evalNumeric[T1, T2](g)

    () => wrapped(f1(), f2())
  }