Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Repo info
Activity
  • Apr 09 15:14
    joroKr21 closed #1073
  • Apr 09 15:14
    joroKr21 commented #1073
  • Apr 09 15:14
    joroKr21 closed #1077
  • Apr 09 15:14
    joroKr21 commented #1077
  • Apr 09 12:31

    joroKr21 on v2.3.4

    (compare)

  • Apr 09 12:13

    joroKr21 on 2.3

    Set versionScheme to pvp Merge pull request #1106 from j… (compare)

  • Apr 09 12:13
    joroKr21 closed #1106
  • Apr 09 12:12
    milessabin review_requested #1106
  • Apr 09 11:58
    joroKr21 edited #1106
  • Apr 09 11:54
    joroKr21 opened #1106
  • Apr 09 09:19

    joroKr21 on 2.3

    Binary compatible and cross com… Merge pull request #1105 from j… (compare)

  • Apr 09 09:19
    joroKr21 closed #1105
  • Apr 09 08:11
    joroKr21 synchronize #1105
  • Apr 08 18:15
    joroKr21 review_requested #1105
  • Apr 08 18:14
    joroKr21 synchronize #1105
  • Apr 08 18:09
    joroKr21 ready_for_review #1105
  • Apr 08 18:02
    joroKr21 synchronize #1105
  • Apr 08 17:38
    joroKr21 opened #1105
  • Apr 07 11:02

    joroKr21 on main

    sbt slash syntax (the old one i… Merge pull request #1104 from j… (compare)

  • Apr 07 11:02
    joroKr21 closed #1104
Julien Richard-Foy
@julienrf
Hello! What is the status of Shapeless 3? What are the tradeoffs of using Shapeless 3 for doing type class derivation, versus using the built-in derivation capabilities of Scala 3?
Nathan Fischer
@nrktkt:matrix.org
[m]
@julienrf: I believe if you just want type class derivation then you can use scala 3 directly, you won't need the additional stuff in shapeless 3
Julien Richard-Foy
@julienrf
I was wondering if there were some benefits in using Shapeless (e.g., less boilerplate, or less asInstanceOf calls)
But yes, I’m only interested in type class derivation
Jeff May
@jeffmay

I am working on an open-source project that abstracts over an tagless final algebra for performing calculations that looks similar to native Scala code, but the end result is a GADT tree that can be visited / interpreted into a Scala function and evaluated by providing a FactTable. Once it is all working, the facts from the fact table will propagate along with any derived values to produce an output with Evidence. The project is mostly working (there are bugs with evidence tracking), but I am having trouble defining operations with HList types.

The root type of the GADT is called Expr[V, R, P]. V is the input value, R is the return type, and P is a custom post-processing parameter that can be extracted from the expression (it can basically be ignored).

In order to avoid repeating the V and P type parameters (which are fixed and invariant -- thus have to be passed along), I created this HList-like structure called HL[V, P] which has two sub-classes HLNil[V, P] and HLCons[V, H, L <: HList, P]. I was able to get a different version of this class to support applying a visitor in a manner that would walk a HLCons[V, M[H0], HLCons[V, M[H1], HLNil[V, P], P], P] with a Expr.Visitor[V, P, G] where G[_] is both Functor and Semigroupal and produced a G[M[H0 :: H1 :: HNil]]. It did so by requiring that M[_] have both Align and Functor instances, and would zip over the head of the HList and the head of the tail HL (which would be recursively visited to zip with it's tail) and basically the zip would stop at the shortest M. You can see how this works here: https://github.com/jeffmay/vapors/blob/prototype/core/src/main/scala/com/rallyhealth/vapors/core/algebra/HExprList.scala#L49-L65 and the tests are here https://github.com/jeffmay/vapors/blob/prototype/core/src/test/scala/com/rallyhealth/vapors/core/interpreter/ZipOutputSpec.scala#L30-L39

I got stuck trying to implement a .zipWithPadding or .zipWithDefaults version, because I would have to be able to represent an HL that effectively takes M[H0] :: M[H1] :: HNil as well as the defaults H0 :: H1 :: HNil and zips them together such that M[H0 :: H1] is defined for all elements of M[H0] and M[H1] by either taking the appropriately aligned element from the other functor or the default value at that location in the HList. I can't represent that in my current implementation of the NonEmptyExprHList that I linked. After looking at shapeless code more, I realize that I am probably constraining things too much with all these type parameters on the NonEmptyExprHList and that I should mimic the parameter-free HList with HListOps that takes implicit operators pattern.

Here is what I have for HL:

sealed trait HL[V, +L <: HList, P]
object HL {
  type Any[V, P] = HL[V, HList, P]
  @inline final def nil[V, P]: HLNil[V, P] = HLNil()
}

final case class HLCons[V, RH, +RT <: HList, P](
  head: Expr[V, RH, P],
  tail: HL[V, RT, P],
) extends HL[V, RH :: RT, P]

sealed abstract class HLNil[V, P] extends HL[V, HNil, P]
object HLNil {
  final val instance = new HLNil[Any, Any] {}
  @inline final def apply[V, P](): HLNil[V, P] = instance.asInstanceOf[HLNil[V, P]]
}
1 reply
And here is what I have for writing my .zipWithDefaults method.
trait CanZipWithDefaults[V, P, M[_], RL <: HList, L <: HL.Any[V, P]] {
  type Out

  def applyZipWithDefaults[G[_] : Functor : Semigroupal](
    inputExpr: L,
    defaults: RL,
    v: Expr.Visitor[V, P, G],
  ): G[Out]
}

object CanZipWithDefaults {

  implicit def hlist1CanZipWithDefaults[
    V,
    P,
    M[_] : Functor,
    RH,
  ]: CanZipWithDefaults[V, P, M, RH :: HNil, HLCons[V, M[RH], HNil, P]] = {
    new CanZipWithDefaults[V, P, M, RH :: HNil, HLCons[V, M[RH], HNil, P]] {
      override type Out = M[RH :: HNil]
      override def applyZipWithDefaults[G[_] : Functor : Semigroupal](
        inputExpr: HLCons[V, M[RH], HNil, P],
        defaults: RH :: HNil,
        v: Expr.Visitor[V, P, G],
      ): G[M[RH :: HNil]] = {
        inputExpr.head.visit(v).map(_.map(_ :: HNil))
      }
    }
  }

  // TODO: How do I write the non-HNil case?
}
I'm pretty new to Shapeless, but I've been writing Scala for a long time
Gosha Emelyanov
@goshaQ_gitlab

Hello, can someone help with short question? I have a graph defined as case class. What I'm trying to achieve is to return HList of graph components (either nodes or relationships). In order to do this I need to filter elements of HList after cast to HList, but the types of graph components have type arguments and it is not clear how to reflect this in the filtering condition. So it looks like

case class Graph(
  factories: Node[Factory],  // Factory extends ENode trait
  cars: Node[Car],  // Car extends ENode trait
  producedBy: Relationship[ProducedBy]   // ProducedBy extends ERelationship trait
) extends TGraph[Graph]

trait TGraph[A <: TGraph[A]] { self: A =>
  def nodes[Repr <: HList]()(
    implicit
    gen: Generic.Aux[A, Repr],
    filter: Filter[Repr, Node[ENode]]  // Issue appear here as the type does not exact match
  ): HList = {
    filter(gen.to(self))
  }
}

At the moment it returns HNil, because the filtering condition does exact match on the type NodeTable[ENode], is there a way to relax this constraint to grab all types that match NodeTable[E <: ENode]? Thanks!

wpoosanguansit
@wpoosanguansit
hi, i have been using Coproduct typeclass and we get quite a hit with the compile time with each additional type we add to the definition. right now we have 11, the compile time is Total time: 302 s (05:02), completed Mar 8, 2021, 5:16:16 PM. when we add the next one - Total time: 421 s (07:01), completed Mar 8, 2021, 5:24:17 PM. 2mins additional time on average. is there any suggestion what can be done to improve on the compile time? appreciate your help. we are using shapeless 2.3.3 on scala 2.13.
Ethan
@esuntag
What specifically are you doing?
In general, moving from automatic to a semi-auto style of derivation should speed up compile times
1 reply
Alessandro
@ImGrayMouser_twitter

Hello everyone,
I am playing / experimenting with Shapeless and would like to solve the following use case. Given a case
class with at most one annotated field (the annotation being custom), derive a typeclass that at runtime returns the
value of the attributes marked with the annotation. For example, given the following code

trait Identity[T] {
    def apply(t: T): Long
}

final case class IdAnnot() extends scala.annotation.StaticAnnotation

with the following random case class and instances

case class User(@IdAnnot id: Long, account_id: Int, name: String, updated_at: java.time.Instant)

val u1 = User(1009, 101, "Alessandro", Instant.now())
val u2 = User(2009, 201, "Mario", Instant.now().plus(10, ChronoUnit.SECONDS))
val u3 = User(3009, 301, "Luigi", Instant.now().plus(100, ChronoUnit.SECONDS))

val identity: Identity[User] = Identity.gen[User]
println(s"u1: ${identity(u1)}, u2: ${identity(u2)}, u3: ${identity(u3)}")

it should print 1009 , 2009 and 3009.

My idea is to save, in a val, the unique Key corresponding to the attribute marked with annotation and at a runtime use a
shapeless Selector to return its instance value. So far I managed to put together something that binds annotation and
case class attributes key, and I think to filter it on the presence of the annotation and map to the Key value (it the
attribute name Symbol) but I am now stuck.

object Collector extends Poly1 {
  implicit def some[K <: Symbol, B] = at[(K, Some[IdAnnot])] { case (k, _) => k }
  implicit def none[K <: Symbol, B] = at[(K, None.type)] { case (k, _) => 'None }
}

implicit def collect[A, HL <: HList, AL <: HList, KL <: HList, VL <: HList, ZL <: HList, P <: Poly1, H](
    implicit
    generic: LabelledGeneric.Aux[A, HL],
    annots: Annotations.Aux[IdAnnot, A, AL],
    keys: Keys.Aux[HL, KL],
    values: Values.Aux[HL, VL],
    zip: hlist.Zip.Aux[KL :: AL :: HNil, ZL],
    f: hlist.Filter.Aux[ZL, H, ZL] // <------
    ) = 
 new Identity[A] {
    val zipped: ZL = zip(keys() :: annots() :: HNil)

    // here I want to return the HList Const with the Annotation,  mapped to the K (ie the field name)
    val attributeWithAnnotation = zipped  filtered by IdAnnot

    override def identity(a: A): Long = {
      // summon a Selector instance and use it to
      generic.to(a).gen(attributeWithAnnotation)
    }
}

The code is broken in many ways. I would return an Option of K because there might not be an annotation at all,
but to simplify the intention I am pretending to return an hardcoded sentinel value ie 'None

The compiler can't fine an implicit for the mapper and don't know if what I am doing makes sense at all.
This line fails the compilation
f: hlist.Filter.Aux[ZL, H, ZL]

with error

could not find implicit value for .....

If I comment that lines out the compiler succeed.

Any hint is greatly welcomed. Thanks everyone

Alessandro
@ImGrayMouser_twitter

Hi all,
with respect my previous post I made some progress, almost there.
I now get the field name of the case class attribute marked with the annotation.
Not sure whether what I did is good or bad, basically it is the result of my frustration.
By the way, I have now in a Symbol the attribute name I need to return the value of at runtime

  object Mapper extends Poly1 {
    implicit def some[K <: Symbol]: Case.Aux[(K, Some[IdAnnot]), Option[K]] = at[(K, Some[IdAnnot])] {
      case (k, _) => Some(k)
    }
    implicit def none[K <: Symbol]: Case.Aux[(K, None.type), Option[K]] = at[(K, None.type)] {
      case (k, _) => Option.empty[K]
    }
  }

  implicit def collect[
    A, HL <: HList, AL <: HList, KL <: HList, ZL <: HList, ML <: HList,
  ](
    implicit
    generic: LabelledGeneric.Aux[A, HL],
    annots: Annotations.Aux[IdAnnot, A, AL],
    keys: Keys.Aux[HL, KL],
    zip: hlist.Zip.Aux[KL :: AL :: HNil, ZL],
    mapper: hlist.Mapper.Aux[Mapper.type, ZL, ML],
    ev0: hlist.ToList[ML, Option[Symbol]],
  ) = new Debug[A] {

    val zipped: ZL = zip(keys() :: annots() :: HNil)
    val x: ML      = mapper.apply(zipped)

    val symbol: Symbol = x.to[List].find(_.isDefined).get match {
      case Some(symbol) => symbol
      case _            => throw new Exception(s"Class  has no attribute marked with @IdAnnot")
    }

    println(s"""
               |zipped: ${zipped}
               |mapped: ${x}
               |symbol: $symbol
               |""".stripMargin)

    override def print(a: A): Unit = {
      val repr     = generic.to(a)
      val value = repr.get(Witness(symbol))

      println(s"""
                 |Generic ${generic.to(a)}
                 |value: $value
      """.stripMargin)
    }
  }

However, the compiler fails the build with

No field this.symbol.type in record HL
[error]       val value = repr.get(Witness(symbol))

Now I am totally lost and a bit afraid that all my strategy is wrong!!

Do you have any advice / suggestion ?

Julien Richard-Foy
@julienrf
I don’t have much time to continue, but I’ve tried to update shapeless to the latest Scala release: milessabin/shapeless#1085
Can someone help me to finish this work?
Miles Sabin
@milessabin
@julienrf I'll sort that out this week.
D Cameron Mauch
@DCameronMauch
Is this code being actively maintained?
I asked a question about it here, in StackOverflow, and in the Shapeless Github issues - with no responses of any kind
Fabio Labella
@SystemFw
I think the reason is that it involves spark, and has no repro whatsoever
Markus Appel
@markusa380

I have a problem with building a generic method to convert a tuple of zio.test.Assertions (Assertion[A1], ..., Assertion[AN]) into a Assertion((A1, ..., AN)).

The method I would like to use to solve this issue is Assertion.hasField[(A1, ..., AN), A1]("_1", _._1, assertion1) && ... && Assertion.hasField[(A1, ..., AN), AN]("_n", _._n, assertionN)

How would I go about this? I've seen UnaryTCConstraint and played with Polys to get the type (A1, ..., AN) from (Assertion[A1], ..., Assertion[AN]) but no success...

Markus Appel
@markusa380
To ask a much simpler question, is there an easy way to get A :: B :: C :: HNil from F[A] :: F[B] :: F[C] :: HNil?
Ethan
@esuntag:matrix.org
[m]
@markusa380: In general, I don't think that's possible unless you have a CoMonad instance for F. I think you should be able to get that to F[A :: B :: C :: HNil] though
Christopher Davenport
@ChristopherDavenport
Given applicative for F, which is present in kittens.
Adding to how to go the other way than you want to
Ethan
@esuntag:matrix.org
[m]
@ import cats.sequence._
import cats.sequence._

@   import cats.implicits._
import cats.implicits._

@   import shapeless._
import shapeless._

@   val l: Option[Int] :: Option[String] :: HNil = Some(3) :: Some("ASD") :: HNil
l: Option[Int] :: Option[String] :: HNil = Some(3) :: Some("ASD") :: HNil

@   val s: Option[Int :: String :: HNil] = l.sequence
s: Option[Int :: String :: HNil] = Some(3 :: "ASD" :: HNil)
Using kittens
That should get you what you want as long as there's an Applicative for Assertion
Aλexander Zafirov
@z4f1r0v
Hi, I have been looking at this write up - https://stackoverflow.com/a/20315002/1450817 - and have a question. Is it possible to pass a HList to a function with a generic (set) of selectors such that we can get arbirary fields to the growing or shrinking HList? I'm sorry if the question is phrased wrongly or I am talking nonsense. I am happy to elaborate my thoughts if needed be :)
Fabio Labella
@SystemFw
it's possible to produce an HList of Options were the options are going to be Some based on the value of the strings in the selector
it is not possible to return an HList with the right type (e..g Int :: String :: HNil or String :: String :: HNil) depending on which selectors are there
because that would mean you could change a type (a compile time entity) based on a runtime value (the Set of selectors)
whereas with an HList of Option the type will always be the same, but the values in that type (i.e. whether the Option are Some or None) can be determined at runtime (although implementing it requires a nice trick)
it is also possible to have the proper type if the selectors are known constants at compile time
which is what shapeless records already do
Aλexander Zafirov
@z4f1r0v
Thank you for the elaborate answer, Fabio!
D Cameron Mauch
@DCameronMauch
With Play’s JsObject, is there any way to sort map fields before calling .toString? For some reason, we store the JSON in our Spark Datasets as String. And in the tests, {“a”: 1, “b”: 1} != {“b”: 2, “a”: 1}
D Cameron Mauch
@DCameronMauch
Also, what the hell does this mean? A needed class was not found. This could be due to an error in your runpath. Missing class: scala/Product$class
Ethan
@esuntag:matrix.org
[m]
That's usually caused by some sort of version mismatch issue
D Cameron Mauch
@DCameronMauch
Yeah, this is part of a big Scala 2.11 to 2.12 update to our Spark code
but I can’t find any Scala 2.11 reminants anywhere
Ethan
@esuntag:matrix.org
[m]
Hm... are you using a version of 2.12 that's incompatible with one of your libraries or something?
D Cameron Mauch
@DCameronMauch
Not sure, everything compiles
This only happens when running a test
Looked at the full dependency tree - there is nothing Scala 2.11 in there
I’m specifically going to 2.12.12 - there is some weird issue with WartRemover on the latest version that supports Scala 2.12.13, where tests won’t run in IntelliJ
How can I find out?
Ethan
@esuntag:matrix.org
[m]
No easy way. Try scala 2.12.11, I vaguely recall issues going above that when we were upgrading, not sure if it was related to spark specifically
D Cameron Mauch
@DCameronMauch
Sure, I’ll give that a whirl - thanks!
Miles Sabin
@milessabin
Thanks @joroKr21 for releasing 2.3.4: https://github.com/milessabin/shapeless/releases/tag/v2.3.4.
Vladimir Ivanov
@vivanov

Hi. I'd appreciate some advice.

In order to persist data represented as a case class with Doobie in a generic way we have TableContextIO case class defined which captures table name, list of columns names and finally a function that converts case class to something that has Composite instance (result of this function is wrapped into IO as there are additional IO operations that have to be performed when this function is called to get more data):
https://gist.github.com/vivanov/368960402c52ee943a7e1acfd6ac8023

Then inside singleton object for this case class there are several PolyN functions defined to:

  • extract list of column names
  • extract function taking for example case class instance (and possibly some additional data) as input and returning value of some field of this case class (or performing some operation on additional data)
  • apply such function to a case class instance to get the property value

as well as number of apply methods for different number of columns names/functions and finally getContext method that applies these PolyN functions having evidences provided by Shapeless (and Composite instance for the final result).

It allows to get TableContextIO having a case class, table and column names as well as functions to get property values from a case class.

This implementation in particular allows to avoid specifying result type (which is an HList of column types to persist) on a call site, but I have following questions about it:

  • We'd like to use tagless final encoding instead of explicit IO, something like this:
    case class TableContextF[F[_]: Functor, S, T: Composite](tableName: String, columns: List[String], cc2compositeIO: S => F[T])

but it's not possible due to nested java.lang.Objects and corresponding compiler error (we are on Scala 2.12): can't existentially abstract over parameterized type F[T]
Is there a way to avoid these nested Objects and have something that preserves type constructor?

  • Function that TableContextIO case class expects is essentially a flatMap, but I guess it can't have instance of Cats FlatMap typeclass due to Composite instance requirement (and laws), am I right here?

Thanks!

1 reply