These are chat archives for nrinaudo/kantan.csv

2nd
May 2016
Simeon H.K. Fitch
@metasim
May 02 2016 14:01
How does one pull a CellDecoder[Option[T]] out of implicit scope? The following generates a "could not find implicit value" error:
    implicit val dateDecoder: CellDecoder[Date] = CellDecoder(s ⇒ DecodeResult(stringToDate(s)))
    def maybeD(s: String) = CellDecoder[Option[Date]].decode(s)
(In 1.8 this sort of thing seemed to work.)
Nicolas Rinaudo
@nrinaudo
May 02 2016 14:04
does this not work?
wait. First. Did you try that in the REPL, or in "actual" code?
Simeon H.K. Fitch
@metasim
May 02 2016 14:04
Actual code.
My imports are:
import kantan.csv.ops._
import kantan.csv._
Nicolas Rinaudo
@nrinaudo
May 02 2016 14:05
let me have a look
Simeon H.K. Fitch
@metasim
May 02 2016 14:05
Thanks!
Nicolas Rinaudo
@nrinaudo
May 02 2016 14:05
(as a side note, I don't remember whether I documented it, but if Date is java.util.Date, you should be able to put an implicit DateFormat instance in scope and get a free CellDecoder)
Simeon H.K. Fitch
@metasim
May 02 2016 14:06
Cool... I'm trying to upgrade from 1.8 to 1.9, so this is part of an incremental upgrade.
I'll try the DateFormat thing
Nicolas Rinaudo
@nrinaudo
May 02 2016 14:07
yeah, I'm afraid I'm quite bad at providing upgrade instructions and the API is still in a bit of a flux, at least where signatures are concerned
Simeon H.K. Fitch
@metasim
May 02 2016 14:07
NP. Still worth it! :)
Nicolas Rinaudo
@nrinaudo
May 02 2016 14:07
let me know if you get stuck with anything, 0.1.9 should be able to do whatever 0.1.8 does, and then some
(I'm not changing the API just for kicks either, it's just that as I write kantan.xpath and kantan.regex, I learn more about the problem and wish to unify signatures)
Simeon H.K. Fitch
@metasim
May 02 2016 14:08
This is basically where I'm stuck... I had a shortcut function to reduce boilerplate noise on my end, and this no longer compiles:
    implicit val dateDecoder: CellDecoder[Date] = CellDecoder.fromUnsafe(stringToDate)
    // No, this is not a collateralized debt obligation
    private def cdo[T: CellDecoder](ss: Seq[String], i: Int) = CellDecoder[Option[T]].decode(ss, i)
So I'm basically trying to rewrite cdo.
Nicolas Rinaudo
@nrinaudo
May 02 2016 14:09
yeah. It's not compiling on my end either, which is absolutely a regression. Crap.
Simeon H.K. Fitch
@metasim
May 02 2016 14:10
I found RowDecoderInstances.optionRowDecoder, but nothing similar for CellDecoder.
Nicolas Rinaudo
@nrinaudo
May 02 2016 14:11
oh crap, I know where this comes from. That was ill thought of me
Hang on, let me see if I can find a quick workaround for you, and then a more permanent one for the API
Simeon H.K. Fitch
@metasim
May 02 2016 14:11
(BTW, your drive to unify signatures is what I like about the API, so I'd keep going with your intuition on that)
Nicolas Rinaudo
@nrinaudo
May 02 2016 14:13
Ok, first, to address your issue. Can you write stringToDate as an instance of DateFormat ?
if so, this compiles:
implicit def dateFormat: DateFormat = ???
CellDecoder[Option[Date]]
If not, there's a roundabout way to get this working. I'm not happy about this, but:
import kantan.codecs.strings._
import kantan.codecs._

def stringToDate(str: String): Date = ???
implicit def stringDecoder: StringDecoder[Date] = StringDecoder(str => Result.nonFatal(stringToDate(str)))
CellDecoder[Option[Date]]
Nicolas Rinaudo
@nrinaudo
May 02 2016 14:19
I can explain if you want, but I'm not sure it's all that interesting.
Simeon H.K. Fitch
@metasim
May 02 2016 14:19
Nah, I can figure it out. I really appreciate you providing a work-around.
Trying it now
Nicolas Rinaudo
@nrinaudo
May 02 2016 14:20
no worries - I have all of three users, including me, I'm going to cherish them :)
Simeon H.K. Fitch
@metasim
May 02 2016 14:20
Lemme know when you want more advertising. I moderate a google group, and have a number of scala-land followers on twitter
Nicolas Rinaudo
@nrinaudo
May 02 2016 14:21
well, not before I've fixed your issue, that's for sure. I'm sorry about this, I thought I had tested things better than that
Simeon H.K. Fitch
@metasim
May 02 2016 14:21
No worries!
shoot.... crashing the compiler now :/
Nicolas Rinaudo
@nrinaudo
May 02 2016 14:22
eh?
Simeon H.K. Fitch
@metasim
May 02 2016 14:23
StackOverflow in compiler type specialization...
I might be behind one scala release.
Nicolas Rinaudo
@nrinaudo
May 02 2016 14:23
that's a first. Ok, let me see if I can come up with a less convoluted fix then
Simeon H.K. Fitch
@metasim
May 02 2016 14:23
Let me investigate more.
Might be my bad.
Nicolas Rinaudo
@nrinaudo
May 02 2016 14:26
before you spend too much time on that, here's another, clearer workaround:
  implicit def optionDecoder[A](implicit ca: CellDecoder[A]): CellDecoder[Option[A]] =
    CellDecoder { s ⇒
      if(s.isEmpty) Result.success(None)
      else          ca.decode(s).map(Some.apply)
    }
Simeon H.K. Fitch
@metasim
May 02 2016 14:27
Ah, I like that.
Nicolas Rinaudo
@nrinaudo
May 02 2016 14:27
Just stick that in your original code and see how that works out
that's probably going to be the final fix, too
Simeon H.K. Fitch
@metasim
May 02 2016 14:27
Looks just like the RowDecoder version
Nicolas Rinaudo
@nrinaudo
May 02 2016 14:28
mmm... good point...
Simeon H.K. Fitch
@metasim
May 02 2016 14:28
The SOE is something else.... I'm getting it even after stashing my changes.
Nicolas Rinaudo
@nrinaudo
May 02 2016 14:30
Let me know if I can help with that, but it sounds like it might not be kantan.csv related - maybe you didn't get it before because compilation failed too soon due to the regression?
Simeon H.K. Fitch
@metasim
May 02 2016 14:30
Yeh, something like that. There's a bug for it: https://issues.scala-lang.org/browse/SI-9078
Just don't know how I'm getting down that path.
Probably because it's Monday...
Nicolas Rinaudo
@nrinaudo
May 02 2016 14:33
a compiler bug? sounds like something for @milessabin then, he seems to be squashing them one after the other these days
Simeon H.K. Fitch
@metasim
May 02 2016 14:34
Yeh. Bumping to 2.11.8 to see if I get something different. Very odd.
Nicolas Rinaudo
@nrinaudo
May 02 2016 14:35
are you doing or using much type level trickery?
Nicolas Rinaudo
@nrinaudo
May 02 2016 14:40
I've opened #36 for your issue. I should be able to get a fix out soon, or you can rely on the above optionDecoder fix until the next release
Simeon H.K. Fitch
@metasim
May 02 2016 14:51
:boom: optionDecoder works great!
I'm good to go with 1.9 for the time being (happy to test out the next release when avail, but not hurry on my end)
Nicolas Rinaudo
@nrinaudo
May 02 2016 14:52
the next release might be sooner than I expected, your issue needs urgent fixing in an official release
Simeon H.K. Fitch
@metasim
May 02 2016 14:52
OK. Feel free to ping me and I'll give it a spin.
Nicolas Rinaudo
@nrinaudo
May 02 2016 14:52
cheers, I'll let you know
Nicolas Rinaudo
@nrinaudo
May 02 2016 14:59
@metasim on a completly unrelated subject, if you find yourself having to work with XML or extract well typed data from regular expressions, would you mind test driving 2 other libraries I'm working on?
Simeon H.K. Fitch
@metasim
May 02 2016 15:55
Oooh, fascinating.
Do quite a lot with regexes. Not sure if strongly typed matches my current use cases, but something like that may jog my thinking about the problem space.
Links?
Nicolas Rinaudo
@nrinaudo
May 02 2016 15:57
Well, kantan.regex is a work in progress and has no published artifacts yet, but I'm writing the documentation now so it should be good to go soon: https://github.com/nrinaudo/kantan.regex
The idea is to be able to write code like this:
"\\((\\d+), (\\d+)\\)".regex[(Int, Int)]
this is then able to parse "(1, 2) and then (3, 4)" into an Iterator[(Int, Int)]
Simeon H.K. Fitch
@metasim
May 02 2016 15:58
And I assume this:
case class Foo(i1: Int, i2: Int)
"\\((\\d+), (\\d+)\\)".regex[Foo]
Nicolas Rinaudo
@nrinaudo
May 02 2016 15:59
yes, absolutely
although I've yet to work out how to plug shapeless in there, so you'd have to write a decoder manually
but that's as simple as MatchDecoder.decoder(1, 2)(Foo.apply)
Simeon H.K. Fitch
@metasim
May 02 2016 15:59

Have you considered this signature instead:

"\\((\\d+), (\\d+)\\)".r.as[(Int, Int)]

So the enrichment is on type of scala.util.Regex?

To be honest, I think shapeless can be more trouble that it's worth sometimes, at least if it's exposed to the API user.
Type signature errors are just too hard to decode without a lot of experience.
Nicolas Rinaudo
@nrinaudo
May 02 2016 16:01
Interesting idea .I'm not a big fan of scala.util.Regex as it's very inefficient, but there might be something there
Simeon H.K. Fitch
@metasim
May 02 2016 16:02
Only because it's built in and people are used to working with it. You could always pull the pattern out and put it in your own implementation.
Just thinking of easy migration path for existing Regex users.
Nicolas Rinaudo
@nrinaudo
May 02 2016 16:03
No you're right, there's probably something to do there, even if it's just adding support for your idea on top of the low-level stuff
Simeon H.K. Fitch
@metasim
May 02 2016 16:03
(e.g. just call foo.pattern.pattern())
Nicolas Rinaudo
@nrinaudo
May 02 2016 16:04
oh my implementation already relies on java.util.Pattern, just like scala.util.Regex, it's just that I generate far fewer intermediate strings
Simeon H.K. Fitch
@metasim
May 02 2016 16:04
But really what you want is something on a pre-compiled Pattern. Without macros, I'm assuming "\\((\\d+), (\\d+)\\)".regex[(Int, Int)] would do a regex compilation each time.
LOL.
Nicolas Rinaudo
@nrinaudo
May 02 2016 16:05
well, it generates a single instance of Regex[(Int, Int)], which you can think of as String => DecodeResult[(Int, Int)]
and then you're free to re-use that as much as you want without having to re-compile it
Simeon H.K. Fitch
@metasim
May 02 2016 16:05
Ah, good point.
Nicolas Rinaudo
@nrinaudo
May 02 2016 16:05
it's very similar to the way kantan.xpath generates a Query that you can then re-use without ever having to re-compile it
this one is meant to do stuff like this:
new URI("http://some.server.com").evalXPath[List[Int]]("//h1/span[@class='num']")
Simeon H.K. Fitch
@metasim
May 02 2016 16:07
What's your take on Neko HTML vs JSoup?
Nicolas Rinaudo
@nrinaudo
May 02 2016 16:07
(where List[Int] could obviously be any collection of any type, such as Set[Tuple2[Int, String]]
I'm a big neko html fan
Simeon H.K. Fitch
@metasim
May 02 2016 16:07
Yeh, that's super cool
Nicolas Rinaudo
@nrinaudo
May 02 2016 16:08
the last time I tried JSoup, it struggled with modern HTML and sort of... gave up, a lot.
Simeon H.K. Fitch
@metasim
May 02 2016 16:08
Good to know.
Nicolas Rinaudo
@nrinaudo
May 02 2016 16:08
that was a long time ago though, to be honest I thought the project was not maintained anyway
kantan.xpath has a NekoHTML module, but adding a JSoup one would take a couple of lines of code only, I think
Simeon H.K. Fitch
@metasim
May 02 2016 16:09
Did you consider a signature like this?:
new URI("http://some.server.com").XPath("//h1/span[@class='num']").as[List[Int]]
I'm thinking of how some of the JSON libraries handle stuff.
Nicolas Rinaudo
@nrinaudo
May 02 2016 16:10
well, you have something similar: "//h1/span[@class='num']".xpath[List[Int]].eval(new URI("http://some.server.com"))
why is your code properly highlighted and not mine?
Simeon H.K. Fitch
@metasim
May 02 2016 16:10
I could imagine wanting to try to convert the same XPath to a few different types to see what works.
tripple backticks?
Nicolas Rinaudo
@nrinaudo
May 02 2016 16:11
oh, right. Yours is not inline.
Simeon H.K. Fitch
@metasim
May 02 2016 16:11
I like the xpath enrichment.
but could imagine wanting to delay type specification until eval
Nicolas Rinaudo
@nrinaudo
May 02 2016 16:12
Anyway, I really must be off, the wife and baby are probably going to be upset soon. If you're interested, both these projects have their own gitter, and kantan.xpath is already available
Simeon H.K. Fitch
@metasim
May 02 2016 16:12
Thanks much!
Thanks for your time today.
Nicolas Rinaudo
@nrinaudo
May 02 2016 16:12
ah, yes, that's how the first version of kantan.xpath worked. It didn't work out in the end, I can get into why later if you're interested
no problem, you're quite welcome. Talk to you later!
Nicolas Rinaudo
@nrinaudo
May 02 2016 19:17
@metasim I need to thank you. The issue turned out to be relatively easy to work around, present in all the kantan libraries, and fixing it makes the code much more generic
Simeon H.K. Fitch
@metasim
May 02 2016 19:18
Fantastic!!