Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Repo info
Activity
  • Jan 20 09:47
    scala-steward opened #256
  • Jan 05 19:53
    scala-steward closed #248
  • Jan 05 19:53
    scala-steward commented #248
  • Jan 05 19:53
    scala-steward opened #255
  • Dec 24 2020 10:57
    scala-steward closed #249
  • Dec 24 2020 10:57
    scala-steward opened #254
  • Dec 20 2020 17:53
  • Dec 20 2020 17:30
    scala-steward closed #241
  • Dec 20 2020 17:30
    scala-steward opened #253
  • Dec 20 2020 03:55
    scala-steward closed #246
  • Dec 20 2020 03:55
    scala-steward opened #252
  • Dec 18 2020 19:26
    scala-steward closed #247
  • Dec 18 2020 19:25
    scala-steward opened #251
  • Dec 17 2020 20:44
    scala-steward closed #238
  • Dec 17 2020 20:44
    scala-steward opened #250
  • Dec 14 2020 09:00
    scala-steward opened #249
  • Dec 02 2020 16:21
    scala-steward opened #248
  • Nov 26 2020 09:24
  • Nov 26 2020 08:15
    scala-steward opened #247
  • Nov 25 2020 07:56
Travis Brown
@travisbrown

You can make things a little cleaner by adding a type parameter rather than using subtyping to distinguish "formats":

trait Decoder[C, A] {
  def map[B](f: A => B): Decoder[C, A]
  // ...
}

The C type parameter could be completely unconstrained, could be a type-level string representing a content type, etc.

Nicolas Rinaudo
@nrinaudo
@travisbrown I understand the bit about the unconstrained type parameter, and it's quite clever. I see how I would retain all or most of the safety granted by F-bound types while keeping things much less verbose
I also like the idea of not having to extend Decoder for each new decoded type, everything could just be a Decoder[E, D, F, C], where E is the encoded type, D the decoded type, F the failure type (the left part of an Either like structure) and C solely used for specialisation
I must confess that I'm a bit more fuzzy on the type level string, though, and will need to read up on that. Is it simply a singleton type, or is there more to it than that?
Travis Brown
@travisbrown
@nrinaudo Here's the first place I started experimenting with the type-level content-type string idea: finagle/finch#329

The idea is that you could enumerate all possible formats that could be used as the C type with something like this:

class ContentType(val contentType: String)
case object ApplicationJson extends ContentType("application/json")
case object PlainText extends ContentType("text/plain")
...

But that's hugely verbose and not extensible in the right ways.

These are essentially phantom types—we do have instances, but really we're just using them as markers to distinguish decoding formats.
The Scala compiler has singleton types for every string literal—there's just no way to refer to them without something like Shapeless or a custom macro. That means that instead of enumerating all of our content types (or formats, or whatever we want to call them), we can just use the string literal singleton types.
Travis Brown
@travisbrown
So for example circe might provide Decoder["application/json".type, A] instances, an XML library could provide Decoder["application/xml".type, A] instances, etc.
(Where "application/json".type is shorthand for shapeless.Witness.`"application/json"`.T or whatever technique you're using to refer to the type-level string.)
Nicolas Rinaudo
@nrinaudo
I understand the general principle, but I'm not sure about using an explicit content type - this wouldn't be accessible anyway and could very well just be a random CsvTag tag for the exact same effect, wouldn't it?
I mean, you'd presumably type alias things - type CsvDecoder[A] = Decoder[CsvTag, A] and never even really expose CsvTag
well, you'd expose it, but callers would never need to know it exists
do you see the explicit content type as some sort of type-level documentation?
Travis Brown
@travisbrown
Mostly it's about not having to define types for every single possible format, but it has other advantages, like the fact that you don't have to worry about two different JSON libraries each having their own JsonTag.
Nicolas Rinaudo
@nrinaudo
on the other hand, I'd see the appeal of being able to provide default implementations - Decoder["text/*".type, A] that all text types could inherit from. There aren't that many ways to serialise an int. But I don't see how to get that to work
oh, I see
Travis Brown
@travisbrown
You can also get a runtime string from the typelevel string, which can be useful in some contexts. I'm not sure "text/*" is a good idea—I'd have to think about it.
Nicolas Rinaudo
@nrinaudo
right, using content-types is sort of an implicit enumeration, but not one that needs to be maintained in the code..
it probably is not a good idea, but it's true that some decoders are going to have lots of duplicated code. I have a CSV library, for instance, that defines a decoder for most primitive types, and another XPath one that defines almost exactly the same ones
the only difference is in the failure type and, if we assume content types at the type level, the content-type
One downside I see to the content-type approach is that it might not be specific enough. What exactly is an XML decoder? is it something that knows how to decode XML attributes, which surely is not quite the same thing as decoding results of XPath expressions, or an XML document itself?
Travis Brown
@travisbrown
You could have a type class that characterizes content types and provide generic codec instances for instances of that type class.
About your last question: the rule of thumb would be "if you'd serve some content using this content type, this decoder can decode it to type X".
Nicolas Rinaudo
@nrinaudo
I can't really come up with a concrete example, but I can't help but feel that this might be a bit too fuzzy - you could easily end up mixing instances from different APIs.
that's part of the beauty of your idea, really: a Decoder instance for a give content type would have the same type, regardless of the library that implements it
but you might find yourself having part of your JSON document extracted by circe instances and others by Argonaut ones, if you have weird dependencies, and I feel this might not be desirable
Travis Brown
@travisbrown
I think that's an issue that's best solved at other levels. I write my code generically with no constraints other than the need for a Decoder["application/json".type, Foo] instance, and it's the responsibility of callers to get an appropriate instance into scope.
…they could do that with import io.circe.generic.auto._ or by defining an Argonaut instance, but that doesn't (and shouldn't have to) matter to my code.
Nicolas Rinaudo
@nrinaudo
ok. I need to play with this a bit, get my head around the concept, but I appreciate you taking the time to discuss this. It certainly not something I considered - nor something I could have considered, really, it's the first time the idea of using phantom types to specialise type class instances even registered as something that one could do
in all your example, you only have 2 type parameters - content-type and decoded type. Does that mean that you're expecting all Decoders to work from in-memory strings, or is that just a shortcut?
Travis Brown
@travisbrown
It's shorthand, but so far that's the way both circe and Finch work.
circe's codec type will also have an additional configuration type parameter very soon (something we need to help guide generic derivation, among other things—I've got a blog post about it here).
Nicolas Rinaudo
@nrinaudo
I'm thinking of some of my use cases - XPath evaluation for example: it would be a shame to only be able to decode XML documents as Strings, as it would force you to parse the document once for each expression you want to evaluate.
let's say I'm scrapping some website and am interested in two different parts of a given page. I'd rather parse the thing as a Node once and evaluate my expressions on this
if that makes sense
Travis Brown
@travisbrown
Sorry, I misspoke—in both circe and Finch the Decoder has a fixed input type, but in circe it's a parsed JSON value. Neither is generic over the input type, but they do have different input types.
Nicolas Rinaudo
@nrinaudo
oh, I see
but they would need to become generic over the input type if they were to use the Decoder type class you have in mind, right?
Travis Brown
@travisbrown
Right.
Nicolas Rinaudo
@nrinaudo
thank you. That's quite a lot for me to digest, but it's certainly taken my brain in unusual directions
Travis Brown
@travisbrown
Thanks for the encouragement to start thinking about this stuff again!
Nicolas Rinaudo
@nrinaudo
wow, thanks for the phantom type idea, this is making the code so much easier to read and follow than my previous F-bound implementation!
laws in particular are becoming quite a bit clearer
Nicolas Rinaudo
@nrinaudo
not having a specialised Decoder per supported format does make it inconvenient to provide default instances though, there's no implicit context in which to stick them...
Travis Brown
@travisbrown
What do you mean? You could have a generic instance method that provides instances for any format.
Nicolas Rinaudo
@nrinaudo
mmm, I'm not sure I follow. When creating a type class instance, say, CsvDecoder, I usually stick CsvDecoder[Int], CsvDecoder[String]... in the CsvDecoder companion object
if Decoder becomes generic, something like Decoder[T, A] where T is the content type and A the decoded type, I have no specific companion object in which to put these default instances
Travis Brown
@travisbrown
Oh, got it. Yes, that's kind of inconvenient, but something like export-hook makes it less so.
Nicolas Rinaudo
@nrinaudo
right, but unless I'm mistaken, export-hook will make sure default instances are available at the right priority level, but will still require an explicit import