These are chat archives for nrinaudo/kantan.csv

24th
May 2017
jportway
@jportway
May 24 2017 17:16
Hi - is there a way using this library to deal with rows that have more than 22 columns? I have a format where the first two columns are discrete values, and then there are 24 columns following that should be treated as a list, but as far as I can tell I can't express this using the standard RowDecoder.decoder() mechanism - is there a way to do it?
Nicolas Rinaudo
@nrinaudo
May 24 2017 17:31
Absolutely. You need to hand-write your decoder instance, but it's definitely possible. I'll write some sample code when I'm near a computer in a couple of hours
jportway
@jportway
May 24 2017 17:31
that would be great - thank you!
Nicolas Rinaudo
@nrinaudo
May 24 2017 17:33
If you want me to make my sample relevant to your use case, could you provide the types?
For cell 0, 1, and then what goes in the list
jportway
@jportway
May 24 2017 17:33
wow. Service!
[Long],[String],[Long], 24 x [Long]
Nicolas Rinaudo
@nrinaudo
May 24 2017 17:34
I have few users, I try to treat them right :)
jportway
@jportway
May 24 2017 17:34
there were 3 columns first - i forgot about one
Nicolas Rinaudo
@nrinaudo
May 24 2017 17:35
Not an issue. Do you want to fail if there are more than 24 cells following the initial 3?
jportway
@jportway
May 24 2017 17:35
basically each row has a numeric ID, a name, a total counter, and then 24 counters, one for each hour in the day
Nicolas Rinaudo
@nrinaudo
May 24 2017 17:36
We could bake the check that your hourly counters add up to the total counter in the decoder if you need that
jportway
@jportway
May 24 2017 17:36
failing if the data violates the format would be fine - it shouldn't ever do that
I think this would be useful sample code for users in general, though, and the more stuff that is specific to my use case the less useful it would be for ithers I think
Nicolas Rinaudo
@nrinaudo
May 24 2017 17:38
I'm also not sure whether this kind of validation has its place in the decoder. Up for debate
Decoding only, no encoding?
jportway
@jportway
May 24 2017 17:39
At the moment, though encoding would be nice too. If it's useful for you. As far as I can tell there isn't any documentation on the site for rolling your own decoder like this, so if it would make useful documentation that would be great, but I don't want to take up too much of your time.
If there is documentation that I've missed just send me the URL
Nicolas Rinaudo
@nrinaudo
May 24 2017 17:42
Use cell decoders for your first 3 cells, a row decoder for the list without its first three elements, glue the whole thing together in a for comprehension
It's a few lines, but not ones that I want to write on a phone :)
jportway
@jportway
May 24 2017 17:44
:)
Nicolas Rinaudo
@nrinaudo
May 24 2017 17:45
Alright I need to be off for a bit, be back in a couple of hours
jportway
@jportway
May 24 2017 17:45
thank you
Nicolas Rinaudo
@nrinaudo
May 24 2017 19:02
@jportway right, so, here's a first simple implementation:
import kantan.csv._
import kantan.csv.ops._

case class Foo(l1: Long, s1: String, total: Long, hourly: List[Long])

implicit val decoder: RowDecoder[Foo] = RowDecoder.from {
  case i1c +: s1c +: totalc +: tail => for {
    i1     <- CellDecoder[Long].decode(i1c)
    s1     <- CellDecoder[String].decode(s1c)
    total  <- CellDecoder[Long].decode(totalc)
    hourly <- RowDecoder[List[Long]].decode(tail)
  } yield Foo(i1, s1, total, hourly)

  case ss => DecodeResult.typeError(s"Not a valid foo: $ss")
}
Proof that it works:
"1,foo,3,1,2,3,4,5,6,7,8,9,10".readCsv[List, Foo](rfc)
// res4: List[kantan.csv.ReadResult[Foo]] = List(Success(Foo(1,foo,3,List(1, 2, 3, 4, 5, 6, 7, 8, 9, 10))))
jportway
@jportway
May 24 2017 19:04
Tha's fantastic, thank you
Nicolas Rinaudo
@nrinaudo
May 24 2017 19:04
this does not include the length check, which you can do on hourly in the for-comprehension
jportway
@jportway
May 24 2017 19:05
It's much simpler than I though, after looking at the methods in RowDecoder - it would be great if this was in the Docs.
I'm using this in a new iteration of of this : http://www.blackshoals.net/
can I add you to the list of "thanks to" ?
Nicolas Rinaudo
@nrinaudo
May 24 2017 19:07
of course!
jportway
@jportway
May 24 2017 19:07
thanks a lot for you help - you've probably saved me hours of hassle
Nicolas Rinaudo
@nrinaudo
May 24 2017 19:07
you're right about the documentation - maybe RowDecoder provides too many helpers, which hides the fact that it's actually quite trivial
well that's a problem right there, it's such an easy problem, it shouldn't take you more than, say, 10 to 15 minutes to work out the internals and implement it
I guess it's back to the doc for me then :/
jportway
@jportway
May 24 2017 19:09
yes - I assumed it was a lot more complex than that. Maybe if I'd seen the source for the generated RowDecoders I would have realised it was easier, but since they're generated I couldn't find the source code
Nicolas Rinaudo
@nrinaudo
May 24 2017 19:09
oh - you're looking for them in github? Yeah, you won't (easily) find it here
if you load the project through intelliJ or download the source artifacts they're exactly where you'd expect them though
jportway
@jportway
May 24 2017 19:10
yes, i just hadn't got that far
Nicolas Rinaudo
@nrinaudo
May 24 2017 19:10
yeah I mean you shouldn't have to
I guess there really needs to be a section in the documentation that shows how to write a decoder (and encoder) from scratch, without using one of the helpers
jportway
@jportway
May 24 2017 19:12
iI definitely wouldn't have been bothering you if I'd seen anything like that example
Nicolas Rinaudo
@nrinaudo
May 24 2017 19:13
but it's good that you did - kantan.csv (and other kantan libraries)'s design are so evident to me by now that I don't necessarily think about the fact that it might not be for everybody else
which, I guess, is a pretty good reason why a library's author shouldn't write the library's documentation
jportway
@jportway
May 24 2017 19:14
:)