These are chat archives for nrinaudo/kantan.csv

10th
Feb 2017
Mike Limansky
@limansky
Feb 10 2017 14:20
Hi! What is the best way to convert List[A] to List[String] (assuming I have encoder for A)?
Andrew Roberts
@aroberts
Feb 10 2017 15:35
val elems: List[A]
elems.map(_.asCsv(‘,’))
Nicolas Rinaudo
@nrinaudo
Feb 10 2017 16:03
@limansky we'd need a bit more context. What's the A? A row type? a cell type?
if you could tell us just a bit more of what you're trying to achieve, I'm pretty sure we can help you out
Mike Limansky
@limansky
Feb 10 2017 20:13
@nrinaudo A is a case class.
Nicolas Rinaudo
@nrinaudo
Feb 10 2017 20:13
right. So you want to turn a list of case classes into a list of strings, where each string is a CSV row?
Mike Limansky
@limansky
Feb 10 2017 20:14
exactly
Nicolas Rinaudo
@nrinaudo
Feb 10 2017 20:14
ok. And I assume you have an implicit instance of RowEncoder[A] in scope, either through the generic module or hand-crafted?
in that case, you should be able to just write:
values.map(RowEncoder[A].encode)
is that what you had ind mind, or did I misunderstand your question?
Mike Limansky
@limansky
Feb 10 2017 20:16
scala> implicitly[RowEncoder[Foo]].encode(Foo(4, "aaa"))
res0: Seq[String] = List(4, aaa)
do you mean I need to call mkString(",")?
Nicolas Rinaudo
@nrinaudo
Feb 10 2017 20:17
err...
I don't understand your question, I'm afraid
oooh
yes, I understand your question, sorry
Mike Limansky
@limansky
Feb 10 2017 20:18
:)
Nicolas Rinaudo
@nrinaudo
Feb 10 2017 20:18
I'm being dense tonight, @aroberts asked the exact same question a while ago and I gave the exact same incorrect answer
Mike Limansky
@limansky
Feb 10 2017 20:19
I mean if I call map on the list of Foo, I'll get List[List[String]] instead of List[String]
Nicolas Rinaudo
@nrinaudo
Feb 10 2017 20:19
so, @aroberts' answer was the correct one
values.map(_.asCsv(','))
(provided you've imported kantan.csv.ops._)
Mike Limansky
@limansky
Feb 10 2017 20:20
scala> import kantan.csv._
import kantan.csv._

scala> import kantan.csv.ops._
import kantan.csv.ops._

scala> import kantan.csv.generic._
import kantan.csv.generic._

scala> case class Foo(i: Int, s: String)
defined class Foo

scala> implicitly[RowEncoder[Foo]].encode(Foo(4, "aaa"))
res0: Seq[String] = List(4, aaa)

scala> Foo(4, "aaa").asCsv(',')
<console>:23: error: value asCsv is not a member of Foo
       Foo(4, "aaa").asCsv(',')
Should I import something else?
Nicolas Rinaudo
@nrinaudo
Feb 10 2017 20:20
!@#
I'm being unusually dense.
so asCsv works on sequences of "things that can be encoded as rows"
Foo(4, "aaa") is not a sequence.
Try it with Seq(Foo(4, "aaa"))
Mike Limansky
@limansky
Feb 10 2017 20:22
scala> List(Foo(4, "aaa")).asCsv(',')
res2: String =
"4,aaa
"
Nicolas Rinaudo
@nrinaudo
Feb 10 2017 20:22
ah, yes, the trailing space.
ok, so before I suggest you trim the result - can I ask why you want a Seq[String]?
because if you're trying to write your Seq[A] as CSV to a file or a stream or a socket or whatever, there are better, cleaner ways
(kantan.csv doesn't really work on the row level - it encodes data as rows, and then serialises bunches of rows)
Mike Limansky
@limansky
Feb 10 2017 20:25
well, I have a stream of data. I'd like to convert each item to csv string and put it into Kafka. At some moment another application take a required amount of lines from Kafka to create a report.
Nicolas Rinaudo
@nrinaudo
Feb 10 2017 20:26
not knowing much about Kafka, the way I'd model this is to put the instances of A into Kafka, and then serialise entire reports to CSV
but I do not know whether that's possible
tell you what. If it's not, you can use asCsv and trim the output. Open a ticket on kantan.csv asking for the possibility of turning a single row into a CSV string, and I'll put it in the next release
Mike Limansky
@limansky
Feb 10 2017 20:29
Right, that was another option. But in this case I need to use same serialization protocol on both sides. Something like protobuf or JSON. I suppose it would be easier to put strings to Kafka than serialize and deserialize case classes.
ok, I'll create a ticket.
Nicolas Rinaudo
@nrinaudo
Feb 10 2017 20:30
yeah - I can see both sides of the coin. On the one hand, it's "cleaner" to send data objects in Kafka and let it do the serialisation. On the other hand, it sounds suspiciouly like work
Mike Limansky
@limansky
Feb 10 2017 21:57
@nrinaudo nrinaudo/kantan.csv#78
Nicolas Rinaudo
@nrinaudo
Feb 10 2017 22:37
Thanks!