Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Repo info
Activity
Travis Brown
@travisbrown
circe's codec type will also have an additional configuration type parameter very soon (something we need to help guide generic derivation, among other things—I've got a blog post about it here).
Nicolas Rinaudo
@nrinaudo
I'm thinking of some of my use cases - XPath evaluation for example: it would be a shame to only be able to decode XML documents as Strings, as it would force you to parse the document once for each expression you want to evaluate.
let's say I'm scrapping some website and am interested in two different parts of a given page. I'd rather parse the thing as a Node once and evaluate my expressions on this
if that makes sense
Travis Brown
@travisbrown
Sorry, I misspoke—in both circe and Finch the Decoder has a fixed input type, but in circe it's a parsed JSON value. Neither is generic over the input type, but they do have different input types.
Nicolas Rinaudo
@nrinaudo
oh, I see
but they would need to become generic over the input type if they were to use the Decoder type class you have in mind, right?
Travis Brown
@travisbrown
Right.
Nicolas Rinaudo
@nrinaudo
thank you. That's quite a lot for me to digest, but it's certainly taken my brain in unusual directions
Travis Brown
@travisbrown
Thanks for the encouragement to start thinking about this stuff again!
Nicolas Rinaudo
@nrinaudo
wow, thanks for the phantom type idea, this is making the code so much easier to read and follow than my previous F-bound implementation!
laws in particular are becoming quite a bit clearer
Nicolas Rinaudo
@nrinaudo
not having a specialised Decoder per supported format does make it inconvenient to provide default instances though, there's no implicit context in which to stick them...
Travis Brown
@travisbrown
What do you mean? You could have a generic instance method that provides instances for any format.
Nicolas Rinaudo
@nrinaudo
mmm, I'm not sure I follow. When creating a type class instance, say, CsvDecoder, I usually stick CsvDecoder[Int], CsvDecoder[String]... in the CsvDecoder companion object
if Decoder becomes generic, something like Decoder[T, A] where T is the content type and A the decoded type, I have no specific companion object in which to put these default instances
Travis Brown
@travisbrown
Oh, got it. Yes, that's kind of inconvenient, but something like export-hook makes it less so.
Nicolas Rinaudo
@nrinaudo
right, but unless I'm mistaken, export-hook will make sure default instances are available at the right priority level, but will still require an explicit import
not the end of the world, but a bit inconvenient
Travis Brown
@travisbrown
Agreed.
Nicolas Rinaudo
@nrinaudo
of course, one way to work around that would to have the free T parameter not be a type level string but a singleton object, and put default instances in there. Not sure which is the lesser of the two evils there.
Travis Brown
@travisbrown
Ah, hadn't thought of that.
Nicolas Rinaudo
@nrinaudo
you lose the format unicity constraint that content-types were buying you though - you're all but guaranteed that you'll get different type tags for identical formats.
then again, if we use a fuller Decoder signature (Decoder[Encoded, Decoded, Tag]), you already have pretty good odds that different libraries will use different Encoded types...
Vladimir Kostyukov
@vkostyukov
I've started to working on something similar to this for Finch (finagle/finch#541) - only the encoder part though.
Nicolas Rinaudo
@nrinaudo
@vkostyukov that looks pretty amazing, but how did you solve the default instance problem? Does this code require an explicit import, or did you manage to sneak them in the implicit context?
and yes, it'd make sense that finch only need the Encoder part. Decoder would be more useful to featherbed, right?
Nicolas Rinaudo
@nrinaudo
I'm struggling to convince myself that specialising instances on type level content types brings enough benefits to forfeit default instances in the implicit scope. Having tried to teach type class based libraries to relative beginners, anything that looks like a magical incantation can have a really negative impact
Nicolas Rinaudo
@nrinaudo
on the other, it is really elegant...
Nicolas Rinaudo
@nrinaudo
I've just pushed a new branch, which implements Encoder, Decoder and Codec with an unconstrained type parameter rather than an F-Bound one
A very simple implementation can be found in the tests. Note that this uses a singleton object to tag encoders / decoders rather than type level strings, but there's nothing in Encoder or Decoder that makes that a necessity
Nicolas Rinaudo
@nrinaudo
(the build fails, but that's because the documentation doesn't compile anymore, all the tests pass)
Nicolas Rinaudo
@nrinaudo
unexpected advantage: implicit resolution works so much better, thanks @travisbrown
Igor
@ishubelko
Hi, guys
Is there someone who can answer question about kantan.xpath?
The question is: can I use kantan.xpath to query org.w3c.dom.Element?
Nicolas Rinaudo
@nrinaudo
The kantan.xpath room would probably be a better place to ask
What so you mean by querying? Do you have a code example that I could take a look at?
Igor
@ishubelko

Sure, I'm getting org.w3c.dom.Element elements from streaming parser, them I'm extracting data like:

def extractProposalDetails(domElement: org.w3c.dom.Element): Proposal =
  Proposal(
    id = domElement.getAttribute("ID"),
    member = domElement.getAttribute("Member"),
    account = ProviderSid.parse(domElement.getAttribute("Account")),
    offerDate = LocalDateTime.parse(domElement.getAttribute("OfferDate")),
    ptype = domElement.getAttribute("Type"),
    currency = Currency.getInstance(domElement.getAttribute("Currency")),
    details = domElement
      .getElementsByTagName("Details")
      .map(od => proposalAttributesFactory(od)) // proposalAttributesFactory - doing a lot `getAttribute`, `getElementsByTagName` etc
      .toList
  )

I'd like to describe all these getAttribute, getElementsByTagName with kantan xpath queries like:

  implicit private val proposalDetailsDecoder: NodeDecoder[Proposal] =
    NodeDecoder.decoder(
      xp"@ID",
      xp"@Member",
      xp"@Account",
      xp"@OfferDate",
      xp"@Type",
      xp"@Currency",
      xp"Details"
    )(Proposal.apply)
// .. other decoders

and then simply domElement.evalXPath[List[Proposal]](xp"//Proposal")

Nicolas Rinaudo
@nrinaudo
And does that not work?
(I’m not in front of a computer right now)
Igor
@ishubelko
Just double-checked - always getting Left(NotFound: no matched node), when domElement.getElementsByTagName("Proposal") - returns DeepNodeListImpl
Nicolas Rinaudo
@nrinaudo
So that means it works but your query returns no result, doesn’t it?
Igor
@ishubelko
yes, looks like that
Igor
@ishubelko
My question was more about is it supposed and correct use the library in such way?
Nicolas Rinaudo
@nrinaudo
Absolutely, yes
Igor
@ishubelko
Got it, thanks!
Igor
@ishubelko
More background: we successfully used kantan xpath for processing xml files, but when we started to receive files up to 50Mb - parsing was not able to process such sizes. So we moved to streaming parser which returns, as I said org.w3c.dom.Element, but all these getAttribute, getElementsByTagName and others is not you want to dealing with after simple and elegant Xpath queries..
Nicolas Rinaudo
@nrinaudo
Ah, yes, xpath doesn’t work in streaming, you need the entire dom to evaluate a query
Or, well, unless you have a tricky library able to emulate a fully loaded elem :)
Igor
@ishubelko
Yeah it looks exactly like you described,
thanks for very nice library and answers!
Have a great weekend.