alexarchambault/scalacheck-shapeless

Generation of arbitrary case classes / ADTs with scalacheck and shapeless

Georgi Krastev

@joroKr21

with mapN

In theory it would be easy to make it work with LabelledGeneric

And customize your generators based on the field name

But I don't think it exists in scalacheck-shapeless ATM

Akhtiam Sakaev

@asakaev

Hello. I am getting compiler "stackoverflow" when trying to derive generators for scalapb data.

at scala.tools.nsc.typechecker.Implicits$ImplicitSearch.scala$tools$nsc$typechecker$Implicits$ImplicitSearch$$typedImplicit(Implicits.scala:622)
        at scala.tools.nsc.typechecker.Implicits$ImplicitSearch$ImplicitComputation.rankImplicits(Implicits.scala:1213)
        at scala.tools.nsc.typechecker.Implicits$ImplicitSearch$ImplicitComputation.findBest(Implicits.scala:1248)
        at scala.tools.nsc.typechecker.Implicits$ImplicitSearch.searchImplicit(Implicits.scala:1305)
        at scala.tools.nsc.typechecker.Implicits$ImplicitSearch.bestImplicit(Implicits.scala:1730)
        at scala.tools.nsc.typechecker.Implicits.inferImplicit1(Implicits.scala:112)
        at scala.tools.nsc.typechecker.Implicits.inferImplicit(Implicits.scala:91)
        at scala.tools.nsc.typechecker.Implicits.inferImplicit$(Implicits.scala:88)
        at scala.tools.nsc.Global$$anon$5.inferImplicit(Global.scala:483)
        at scala.tools.nsc.typechecker.Implicits.$anonfun$inferImplicit$1(Implicits.scala:144)
        at scala.tools.nsc.typechecker.Implicits.inferImplicit(Implicits.scala:144)
        at scala.tools.nsc.typechecker.Implicits.inferImplicit$(Implicits.scala:142)
        at scala.tools.nsc.Global$$anon$5.inferImplicit(Global.scala:483)
        at scala.reflect.macros.contexts.Typers.inferImplicitValue(Typers.scala:56)
        at scala.reflect.macros.contexts.Typers.inferImplicitValue$(Typers.scala:54)
        at scala.reflect.macros.contexts.Context.inferImplicitValue(Context.scala:18)
        at scala.reflect.macros.contexts.Context.inferImplicitValue(Context.scala:18)
        at shapeless.LazyMacros$DerivationContext$State$.resolveInstance(lazy.scala:303)
        at shapeless.LazyMacros$DerivationContext.resolve0(lazy.scala:421)
        at shapeless.LazyMacros$DerivationContext.$anonfun$deriveLowPriority$1(lazy.scala:449)
        at scala.Option.flatMap(Option.scala:283)
        at shapeless.LazyMacros$DerivationContext.helper$2(lazy.scala:446)
        at shapeless.LazyMacros$DerivationContext.deriveLowPriority(lazy.scala:483)
        at shapeless.LazyMacros$DerivationContext.derive(lazy.scala:494)
        at shapeless.LazyMacros$DerivationContext$State$.deriveInstance(lazy.scala:327)
        at shapeless.LazyMacrosCompat.deriveInstance(lazymacros.scala:50)
        at shapeless.LazyMacrosCompat.deriveInstance$(lazymacros.scala:33)
        at shapeless.LazyMacros$.deriveInstance(lazy.scala:573)
        at shapeless.LazyMacros.mkImpl(lazy.scala:216)
        at shapeless.LazyMacros.mkStrictImpl(lazy.scala:209)
        at shapeless.LazyMacrosRef.mkStrictImpl(lazyref.scala:39)
        at scala.reflect.macros.runtime.JavaReflectionRuntimes$JavaReflectionResolvers.$anonfun$resolveJavaReflectionRuntime$5(JavaReflectionRuntimes.scala:45)

Can I do something with this or I have big nested type that can not be derived?

Georgi Krastev

@joroKr21

You could try bumping up the stack

I usually go for -Xss2m

Akhtiam Sakaev

@asakaev

@joroKr21 4m already, doesn't helps 🤷‍♂️

Georgi Krastev

@joroKr21

But do you see a loop in the stack trace? From the part you shared I don't.

Also how big is this nested type?

One thing you might try is break it up, i.e. derive generators for the inner types first and then for the outer ones.

Pritam Kadam

@kpritam

Hi,. In the following example, is there a way to create one instance of each subtype predictably, I want to remove randomness and create one instance for each subtype in ADT's?

import org.scalacheck.ScalacheckShapeless._

sealed trait Base
case class BaseIntString(i: Int, s: String) extends Base
case class BaseDoubleBoolean(d: Double, b: Boolean) extends Base

val arb = implicitly[Arbitrary[Base]]
println(arb.sample)
println(arb.sample)
println(arb.sample)

here arb.sample gives random instance for one of the subtype

Felix Bjært Hargreaves

@hejfelix

Trying to derive an arbitrary for a big type from an external library, but getting:

Could not write class arbitraries/package$anon$derivedArbitrary$macro$175$1 because it exceeds JVM code size limits. Method inst$macro$139$lzycompute's code too large!

/**
  * Case class representing a canonical Snowplow event.
  *
  * @see https://github.com/snowplow/snowplow/wiki/canonical-event-model
  */
case class Event(app_id:                   Option[String],
                 platform:                 Option[String],
                 etl_tstamp:               Option[Instant],
                 collector_tstamp:         Instant,
                 dvce_created_tstamp:      Option[Instant],
                 event:                    Option[String],
                 event_id:                 UUID,
                 txn_id:                   Option[Int],
                 name_tracker:             Option[String],
                 v_tracker:                Option[String],
                 v_collector:              String,
                 v_etl:                    String,
                 user_id:                  Option[String],
                 user_ipaddress:           Option[String],
                 user_fingerprint:         Option[String],
                 domain_userid:            Option[String],
                 domain_sessionidx:        Option[Int],
                 network_userid:           Option[String],
                 geo_country:              Option[String],
                 geo_region:               Option[String],
                 geo_city:                 Option[String],
                 geo_zipcode:              Option[String],
                 geo_latitude:             Option[Double],
                 geo_longitude:            Option[Double],
                 geo_region_name:          Option[String],
                 ip_isp:                   Option[String],
                 ip_organization:          Option[String],
                 ip_domain:                Option[String],
                 ip_netspeed:              Option[String],
                 page_url:                 Option[String],
                 page_title:               Option[String],
                 page_referrer:            Option[String],
                 page_urlscheme:           Option[String],
                 page_urlhost:             Option[String],
                 page_urlport:             Option[Int],
                 page_urlpath:             Option[String],
                 page_urlquery:            Option[String],
                 page_urlfragment:         Option[String],
                 refr_urlscheme:           Option[String],
                 refr_urlhost:             Option[String],
                 refr_urlport:             Option[Int],
                 refr_urlpath:             Option[String],
                 refr_urlquery:            Option[String],
                 refr_urlfragment:         Option[String],
                 refr_medium:              Option[String],
                 refr_source:              Option[String],
                 refr_term:                Option[String],
                 mkt_medium:               Option[String],
                 mkt_source:               Option[String],
                 mkt_term:                 Option[String],
                 mkt_content:              Option[String],
                 mkt_campaign:             Option[String],
                 contexts:                 Contexts,
                 se_category:              Option[String],
                 se_action:                Option[String],
                 se_label:                 Option[String],
                 se_property:              Option[String],
                 se_value:                 Option[Double],
                 unstruct_event:           UnstructEvent,
                 tr_orderid:               Option[String],
                 tr_affiliation:           Option[String],
                 tr_total:                 Option[Double],
                 tr_tax:                   Option[Double],
                 tr_shipping:              Option[Double],
                 tr_city:                  Option[String],
                 tr_state:                 Option[String],
                 tr_country:               Option[String],
....

any way to circumvent this?

Georgi Krastev

@joroKr21

@hejfelix does it also happen if you just try to get shapeless.Generic[Event]?

Georgi Krastev

@joroKr21

how many fields does it have?

Nicolás Cañibano

@ncanibano

I'm facing the same issue as @hejfelix, while using implicitly[Arbitrary[BigCaseClass]].arbitrary:

Could not write class xxx because it exceeds JVM code size limits. Method inst$macro$135$lzycompute's code too large!

The case class has 130 fields.
Any workaround to solve this issue? Thanks!

Georgi Krastev

@joroKr21

Did you try with just Generic?

What kind of fields does it have? Primitives or other case classes?

Nicolás Cañibano

@ncanibano

shapeless.Generic[BigCaseClass] compiles successfully, and the case class has primitive fields only, no nested case classes, any clues how to solve this?

Georgi Krastev

@joroKr21

I'm not sure

It sounds like shapeless is not the problem, and nested case classes are not the problem

So a bit difficult situation

Nicolás Cañibano

@ncanibano

I found a workaround using generics and implicit generators:

// primitive generators
implicit def stringGen(implicit arbString: Arbitrary[String]): Gen[String] = arbString.arbitrary
implicit def intGen(implicit arbInt: Arbitrary[Int]): Gen[Int] = arbInt.arbitrary

// hlist generator
implicit def hnilGen: Gen[HNil] = Gen.const(HNil)
implicit def hlistGen[H, T <: HList](implicit headGen: Gen[H], tailGen: Gen[T]) = headGen.flatMap(head => tailGen.map(tail => head :: tail))

// case class generator
implicit def caseClassGen[T, L <: HList](implicit generic: Generic.Aux[T, L], lGen: Gen[L]): Gen[T] = lGen.map(generic.from)
def generator[A](implicit gen: Gen[A]): Gen[A] = gen

Then you can generate arbitrary big case classes like this:

implicit val generic = Generic[YourBigCaseClass]

val gen = generator[YourBigCaseClass]

val sample = Gen.nonEmptyListOf(gen).sample.get

I hope that helps anyone facing this limitation.

Georgi Krastev

@joroKr21

But I wonder what in scalacheck-shapeless generates so much bytecode.

Dario Abdulrehman

@dabd

I'm trying to write a test using ScalaTest's ShrinkLowPriority trait but it seems to conflict with the internals of scalacheck-shapeless. I get

Error:(27, 88) ambiguous implicit values:
 both method derivedShrink in trait DerivedInstances of type [T](implicit ev: shapeless.LowPriority.Ignoring[String("shrinkAny")], implicit underlying: shapeless.Strict[org.scalacheck.derive.MkShrink[T]])org.scalacheck.Shrink[T]
 and method shrinkAny in trait ShrinkLowPriority of type [T]=> org.scalacheck.Shrink[T]
 match expected type ...

How can I disable shrinking in my own tests?

Looking at the source there is a

implicit def derivedShrink[T]
   (implicit
     ev: LowPriority.Ignoring[Witness.`"shrinkAny"`.T],
     underlying: Strict[MkShrink[T]]
   ): Shrink[T] =
    underlying.value.shrink

But I don't understand what it does.

Jack Low

@wjlow

Is it possible to generate an Arbitrary instance using scalacheck-shapeless for my ADT that has a generic type in one of its cases?

sealed trait A
case class Foo[T](value: T) extends A
case class Bar(value: String) extends A

4 replies

Adrian

@adrian-salajan

Hello, can scalacheck-shapeless work with case classes with in turn have fields like Instant / Date / Curency / scala enumeratum, if there are "manual" arbitraries provided for these ?
I can't seem to make it work
the closest i get is to a

 scala.UninitializedFieldError: Uninitialized field

edit: it works just have to have all the implicit imports near
import org.scalacheck.ScalacheckShapeless._

Howy Perrin

@howyp

Hey @alexarchambault - is there a plan to release scalacheck-shapeless for Scalacheck 1.15 ?

Buk Lau

@mk9023_gitlab

Hi :wave:

I am experiencing an issue with the library that is described in this ticket: alexarchambault/scalacheck-shapeless#46

Will this be fixed in 1.15 or is not going to be fixed?

I have a case class and one of the fields is an Option of a type which is an ADT (a sealed trait with sum types) but when I get runtime exceptions when it tries to generate instances for it

If I take out the Option with that sealed trait all is fine

Buk Lau

@mk9023_gitlab

Is this version: "com.github.alexarchambault" %% "scalacheck-shapeless_1.14" % "1.2.5" compatible with

"org.scalatest"              %% "scalatest"                      % "3.2.3"          % Test,
"org.scalatestplus"          %% "scalacheck-1-15"                % "3.2.3.0"        % Test,

Buk Lau

@mk9023_gitlab

I fixed the issue - it only works with "org.scalatestplus" %% "scalacheck-1-14"

Artem Nikiforov

@nikiforo

Hi all,
I'm using "com.github.alexarchambault" %% "scalacheck-shapeless_1.14" % "1.2.5", scala 2.13.5.
I expect implicitly[Arbitrary[IntWr]] to use arb in the code below, but it doesn't.

  case class IntWr(int: Int)

  val intGen = Gen.choose(5, 9)

  def shapelessCustom = {
    import org.scalacheck.ScalacheckShapeless._
    implicit val arb: Arbitrary[Int] = Arbitrary(intGen)
    mkList(implicitly[Arbitrary[IntWr]])
  }

  private def mkList[T](gen: Arbitrary[T]): List[T] =
    List.fill(10)(gen.arbitrary.sample.get)

For instance, here the correct Arbitrary[Int] is used:

def custom: List[Int] = {
    implicit val arb: Arbitrary[Int] = Arbitrary(intGen)
    mkList(implicitly[Arbitrary[Int]])
  }

Is this behavior expected?
How can I use custom arbitrary for a case class attribute?

Georgi Krastev

@joroKr21

How do can you tell that it's not used?

Artem Nikiforov

@nikiforo

How do can you tell that it's not used?

two evidences:

Warning that isn't used

Println on both lists gives:

// List(7, 5, 5, 8, 6, 8, 7, 5, 7, 6)
// List(IntWr(-2147483648), IntWr(1263246479), IntWr(1791721042), IntWr(0), IntWr(-90407902))

Georgi Krastev

@joroKr21

Hmm ok 1. btw it could also warn even when it's used depending on your configuration. Because an implicit which is used only after a macro has expanded might be detected too late. But the println is a good indicator :thumbsup:

❯ scala -Wmacros:help
Usage: -Wmacros:<mode> where <mode> choices are none, before, after, both (default: before).
  none    Do not inspect expansions or their original trees when generating unused symbol warnings.
  before  Only inspect unexpanded user-written code for unused symbols.
  after   Only inspect expanded trees when generating unused symbol warnings.
  both    Inspect both user-written code and expanded trees when generating unused symbol warnings.

Georgi Krastev

@joroKr21

Well ... it works on my machine :smile: https://scastie.scala-lang.org/7Ji2AhrCT5GK2FGyJ3yG8A

Artem Nikiforov

@nikiforo

❯ scala -Wmacros:help
Usage: -Wmacros:<mode> where <mode> choices are none, before, after, both (default: before).

Wow, I didn't know that compiler flags show some help this way. Neither I knew about flag -Wmacros :)).
Your message is about the first "evidence"? That without the second one my argument would be week?

Well ... it works on my machine :smile: https://scastie.scala-lang.org/7Ji2AhrCT5GK2FGyJ3yG8A

hmmmmm

Thanks!
Interesting... I'll look closer and try to narrow it somehow

Georgi Krastev

@joroKr21

:thumbsup:

Where communities thrive

People

Repo info

Activity