Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Activity
  • Apr 01 07:37

    BenFradet on master

    Update sbt to 1.3.9 (compare)

  • Apr 01 07:37
    BenFradet closed #66
  • Mar 31 20:32
    scala-steward opened #66
  • Feb 21 09:27

    BenFradet on master

    Update scalatest to 3.1.1 (compare)

  • Feb 21 09:27
    BenFradet closed #65
  • Feb 21 07:32
    scala-steward opened #65
  • Feb 10 10:26

    BenFradet on master

    Update sbt to 1.3.8 (compare)

  • Feb 10 10:26
    BenFradet closed #63
  • Feb 10 10:06

    BenFradet on master

    Update spark-sql to 2.4.5 (compare)

  • Feb 10 10:06
    BenFradet closed #64
  • Feb 06 23:08
    scala-steward opened #64
  • Feb 04 01:21
    scala-steward opened #63
  • Jan 16 07:50

    BenFradet on master

    Update sbt to 1.3.7 (compare)

  • Jan 16 07:50
    BenFradet closed #62
  • Jan 16 05:47
    scala-steward opened #62
  • Dec 26 2019 19:48

    BenFradet on master

    Update sbt to 1.3.6 (compare)

  • Dec 26 2019 19:48
    BenFradet closed #61
  • Dec 26 2019 19:14
    scala-steward opened #61
  • Dec 16 2019 11:05

    BenFradet on master

    Update sbt to 1.3.5 (compare)

  • Dec 16 2019 11:05
    BenFradet closed #60
Gregory Barry
@gregosaurus
Good Morning Ben !
Ben Fradet
@BenFradet
hey @gregosaurus
Gregory Barry
@gregosaurus
I just stumble upon your struct-type-encoder yesterday and so far it works very nicely, but i was wondering if i can make it a bit more generic
(btw, are you French? - just saw the flag on your github profile)
Ben Fradet
@BenFradet
yes I am :D, more generic in what sense?
Gregory Barry
@gregosaurus
i would like if possible to have something like :
StructTypeEncoder[T].encode where T is is my base data model (a trait).
The compiler complains there isn't any implicit for T defined, which quite make sense
i tried using ClassTag and TypeTag but still i can't figure it out
Ah cool ! Je suis Francais aussi
Ben Fradet
@BenFradet
let's keep the convo in English so it is useful for others
Gregory Barry
@gregosaurus
sure
Ben Fradet
@BenFradet
you mean something along the lines of the use case described in #4 ?
Gregory Barry
@gregosaurus
hmmm no,
it'll be more something like:

trait A
case class B(g: Int) extends A
case class C(g: String) extends A

object Encoder {
  def encode[A](a: A): StructType = {
    StructTypeEncoder[A].encode
  }
}



def main() {
  val s = C("hello")
  val schema = Encoder.encode(s)
}
Ben Fradet
@BenFradet
Don't B and C extend A?
Gregory Barry
@gregosaurus
yes (fixed)
Ben Fradet
@BenFradet
so yes it's pretty much the same thing as #4, it's still an open issue I haven't had time to really dig into unfortunately
just out of curiosity, what schema would you expect to get?
Gregory Barry
@gregosaurus
You mean in the example ?
Ben Fradet
@BenFradet
yes
Gregory Barry
@gregosaurus
something like :
{ "type" : "struct", "fields" : [ { "name" : "g", "type" : "String", "nullable" : true, "metadata" : { } } } (if rendered as json using StructType.json )
Ben Fradet
@BenFradet
so no indication of whether it was a C or a B, cool
Gregory Barry
@gregosaurus
Nope, whether it's C or B i don't really care :)
I just made an ugly pattern matching for now with all the possible subtypes of A (refering to the example).
i would love to contribute to the suggestions, but i'm afraid my scala knowlegde (at least for this case scenario) is a bit limited :worried:
Ben Fradet
@BenFradet
yup, I might have some time to dive into this issue in the coming weeks since I'm on holidays but no promises unfortunately
yeah no worries
this shapeless stuff is pretty hard
Gregory Barry
@gregosaurus
this is my next target actually, learning Shapeless for good
Ben Fradet
@BenFradet
do you own the type astronaut's guide to shapeless?
I mean the book
Gregory Barry
@gregosaurus
i've ran into in yesterday
Ben Fradet
@BenFradet
welp you're in good hands then :D
Gregory Barry
@gregosaurus
:)