alexarchambault/scalacheck-shapeless

Generation of arbitrary case classes / ADTs with scalacheck and shapeless

Dario Abdulrehman

@dabd

I may be using it incorrectly but I attempted to use scalacheck-shapeless to automatically generate data for a sealed trait hierarchy. It turned out that the generators are very generic and not very useful for my purposes. For example a case class with a String field apparently uses Arbitrary.arbString and that is too generic for my application. Is there a way to customise the generators for a field using the library?

Dario Abdulrehman

@dabd

The only way around it I found is to use tagged types and write Arbitrary implicits.

Georgi Krastev

@joroKr21

I guess depending on your taste you could also just import your own implicit Arbitrary[String]

If you don't mind having orphan typeclass instances

Dario Abdulrehman

@dabd

I suppose @edmondo1984 was asking a similar question above

Dario Abdulrehman

@dabd

@joroKr21 and what if your case class has two different fields of type String which need different instances of Arbitrary. I think in this case you have to use tagged types.

Georgi Krastev

@joroKr21

I'm not sure I would have to check

Dario Abdulrehman

@dabd

The implicit mess + tagged types makes it easier to just hand-craft the case class generators.

Georgi Krastev

@joroKr21

yeah if you want customization might be easier to handroll it

you could use https://christopherdavenport.github.io/cats-scalacheck/

with mapN

In theory it would be easy to make it work with LabelledGeneric

And customize your generators based on the field name

But I don't think it exists in scalacheck-shapeless ATM

Akhtiam Sakaev

@asakaev

Hello. I am getting compiler "stackoverflow" when trying to derive generators for scalapb data.

at scala.tools.nsc.typechecker.Implicits$ImplicitSearch.scala$tools$nsc$typechecker$Implicits$ImplicitSearch$$typedImplicit(Implicits.scala:622)
        at scala.tools.nsc.typechecker.Implicits$ImplicitSearch$ImplicitComputation.rankImplicits(Implicits.scala:1213)
        at scala.tools.nsc.typechecker.Implicits$ImplicitSearch$ImplicitComputation.findBest(Implicits.scala:1248)
        at scala.tools.nsc.typechecker.Implicits$ImplicitSearch.searchImplicit(Implicits.scala:1305)
        at scala.tools.nsc.typechecker.Implicits$ImplicitSearch.bestImplicit(Implicits.scala:1730)
        at scala.tools.nsc.typechecker.Implicits.inferImplicit1(Implicits.scala:112)
        at scala.tools.nsc.typechecker.Implicits.inferImplicit(Implicits.scala:91)
        at scala.tools.nsc.typechecker.Implicits.inferImplicit$(Implicits.scala:88)
        at scala.tools.nsc.Global$$anon$5.inferImplicit(Global.scala:483)
        at scala.tools.nsc.typechecker.Implicits.$anonfun$inferImplicit$1(Implicits.scala:144)
        at scala.tools.nsc.typechecker.Implicits.inferImplicit(Implicits.scala:144)
        at scala.tools.nsc.typechecker.Implicits.inferImplicit$(Implicits.scala:142)
        at scala.tools.nsc.Global$$anon$5.inferImplicit(Global.scala:483)
        at scala.reflect.macros.contexts.Typers.inferImplicitValue(Typers.scala:56)
        at scala.reflect.macros.contexts.Typers.inferImplicitValue$(Typers.scala:54)
        at scala.reflect.macros.contexts.Context.inferImplicitValue(Context.scala:18)
        at scala.reflect.macros.contexts.Context.inferImplicitValue(Context.scala:18)
        at shapeless.LazyMacros$DerivationContext$State$.resolveInstance(lazy.scala:303)
        at shapeless.LazyMacros$DerivationContext.resolve0(lazy.scala:421)
        at shapeless.LazyMacros$DerivationContext.$anonfun$deriveLowPriority$1(lazy.scala:449)
        at scala.Option.flatMap(Option.scala:283)
        at shapeless.LazyMacros$DerivationContext.helper$2(lazy.scala:446)
        at shapeless.LazyMacros$DerivationContext.deriveLowPriority(lazy.scala:483)
        at shapeless.LazyMacros$DerivationContext.derive(lazy.scala:494)
        at shapeless.LazyMacros$DerivationContext$State$.deriveInstance(lazy.scala:327)
        at shapeless.LazyMacrosCompat.deriveInstance(lazymacros.scala:50)
        at shapeless.LazyMacrosCompat.deriveInstance$(lazymacros.scala:33)
        at shapeless.LazyMacros$.deriveInstance(lazy.scala:573)
        at shapeless.LazyMacros.mkImpl(lazy.scala:216)
        at shapeless.LazyMacros.mkStrictImpl(lazy.scala:209)
        at shapeless.LazyMacrosRef.mkStrictImpl(lazyref.scala:39)
        at scala.reflect.macros.runtime.JavaReflectionRuntimes$JavaReflectionResolvers.$anonfun$resolveJavaReflectionRuntime$5(JavaReflectionRuntimes.scala:45)

Can I do something with this or I have big nested type that can not be derived?

Georgi Krastev

@joroKr21

You could try bumping up the stack

I usually go for -Xss2m

Akhtiam Sakaev

@asakaev

@joroKr21 4m already, doesn't helps 🤷‍♂️

Georgi Krastev

@joroKr21

But do you see a loop in the stack trace? From the part you shared I don't.

Also how big is this nested type?

One thing you might try is break it up, i.e. derive generators for the inner types first and then for the outer ones.

Pritam Kadam

@kpritam

Hi,. In the following example, is there a way to create one instance of each subtype predictably, I want to remove randomness and create one instance for each subtype in ADT's?

import org.scalacheck.ScalacheckShapeless._

sealed trait Base
case class BaseIntString(i: Int, s: String) extends Base
case class BaseDoubleBoolean(d: Double, b: Boolean) extends Base

val arb = implicitly[Arbitrary[Base]]
println(arb.sample)
println(arb.sample)
println(arb.sample)

here arb.sample gives random instance for one of the subtype

Felix Palludan Hargreaves

@hejfelix

Trying to derive an arbitrary for a big type from an external library, but getting:

Could not write class arbitraries/package$anon$derivedArbitrary$macro$175$1 because it exceeds JVM code size limits. Method inst$macro$139$lzycompute's code too large!

/**
  * Case class representing a canonical Snowplow event.
  *
  * @see https://github.com/snowplow/snowplow/wiki/canonical-event-model
  */
case class Event(app_id:                   Option[String],
                 platform:                 Option[String],
                 etl_tstamp:               Option[Instant],
                 collector_tstamp:         Instant,
                 dvce_created_tstamp:      Option[Instant],
                 event:                    Option[String],
                 event_id:                 UUID,
                 txn_id:                   Option[Int],
                 name_tracker:             Option[String],
                 v_tracker:                Option[String],
                 v_collector:              String,
                 v_etl:                    String,
                 user_id:                  Option[String],
                 user_ipaddress:           Option[String],
                 user_fingerprint:         Option[String],
                 domain_userid:            Option[String],
                 domain_sessionidx:        Option[Int],
                 network_userid:           Option[String],
                 geo_country:              Option[String],
                 geo_region:               Option[String],
                 geo_city:                 Option[String],
                 geo_zipcode:              Option[String],
                 geo_latitude:             Option[Double],
                 geo_longitude:            Option[Double],
                 geo_region_name:          Option[String],
                 ip_isp:                   Option[String],
                 ip_organization:          Option[String],
                 ip_domain:                Option[String],
                 ip_netspeed:              Option[String],
                 page_url:                 Option[String],
                 page_title:               Option[String],
                 page_referrer:            Option[String],
                 page_urlscheme:           Option[String],
                 page_urlhost:             Option[String],
                 page_urlport:             Option[Int],
                 page_urlpath:             Option[String],
                 page_urlquery:            Option[String],
                 page_urlfragment:         Option[String],
                 refr_urlscheme:           Option[String],
                 refr_urlhost:             Option[String],
                 refr_urlport:             Option[Int],
                 refr_urlpath:             Option[String],
                 refr_urlquery:            Option[String],
                 refr_urlfragment:         Option[String],
                 refr_medium:              Option[String],
                 refr_source:              Option[String],
                 refr_term:                Option[String],
                 mkt_medium:               Option[String],
                 mkt_source:               Option[String],
                 mkt_term:                 Option[String],
                 mkt_content:              Option[String],
                 mkt_campaign:             Option[String],
                 contexts:                 Contexts,
                 se_category:              Option[String],
                 se_action:                Option[String],
                 se_label:                 Option[String],
                 se_property:              Option[String],
                 se_value:                 Option[Double],
                 unstruct_event:           UnstructEvent,
                 tr_orderid:               Option[String],
                 tr_affiliation:           Option[String],
                 tr_total:                 Option[Double],
                 tr_tax:                   Option[Double],
                 tr_shipping:              Option[Double],
                 tr_city:                  Option[String],
                 tr_state:                 Option[String],
                 tr_country:               Option[String],
....

any way to circumvent this?

Georgi Krastev

@joroKr21

@hejfelix does it also happen if you just try to get shapeless.Generic[Event]?

Georgi Krastev

@joroKr21

how many fields does it have?

Nicolás Cañibano

@ncanibano

I'm facing the same issue as @hejfelix, while using implicitly[Arbitrary[BigCaseClass]].arbitrary:

Could not write class xxx because it exceeds JVM code size limits. Method inst$macro$135$lzycompute's code too large!

The case class has 130 fields.
Any workaround to solve this issue? Thanks!

Georgi Krastev

@joroKr21

Did you try with just Generic?

What kind of fields does it have? Primitives or other case classes?

Nicolás Cañibano

@ncanibano

shapeless.Generic[BigCaseClass] compiles successfully, and the case class has primitive fields only, no nested case classes, any clues how to solve this?

Georgi Krastev

@joroKr21

I'm not sure

It sounds like shapeless is not the problem, and nested case classes are not the problem

So a bit difficult situation

Nicolás Cañibano

@ncanibano

I found a workaround using generics and implicit generators:

// primitive generators
implicit def stringGen(implicit arbString: Arbitrary[String]): Gen[String] = arbString.arbitrary
implicit def intGen(implicit arbInt: Arbitrary[Int]): Gen[Int] = arbInt.arbitrary

// hlist generator
implicit def hnilGen: Gen[HNil] = Gen.const(HNil)
implicit def hlistGen[H, T <: HList](implicit headGen: Gen[H], tailGen: Gen[T]) = headGen.flatMap(head => tailGen.map(tail => head :: tail))

// case class generator
implicit def caseClassGen[T, L <: HList](implicit generic: Generic.Aux[T, L], lGen: Gen[L]): Gen[T] = lGen.map(generic.from)
def generator[A](implicit gen: Gen[A]): Gen[A] = gen

Then you can generate arbitrary big case classes like this:

implicit val generic = Generic[YourBigCaseClass]

val gen = generator[YourBigCaseClass]

val sample = Gen.nonEmptyListOf(gen).sample.get

I hope that helps anyone facing this limitation.

Georgi Krastev

@joroKr21

But I wonder what in scalacheck-shapeless generates so much bytecode.

Dario Abdulrehman

@dabd

I'm trying to write a test using ScalaTest's ShrinkLowPriority trait but it seems to conflict with the internals of scalacheck-shapeless. I get

Error:(27, 88) ambiguous implicit values:
 both method derivedShrink in trait DerivedInstances of type [T](implicit ev: shapeless.LowPriority.Ignoring[String("shrinkAny")], implicit underlying: shapeless.Strict[org.scalacheck.derive.MkShrink[T]])org.scalacheck.Shrink[T]
 and method shrinkAny in trait ShrinkLowPriority of type [T]=> org.scalacheck.Shrink[T]
 match expected type ...

How can I disable shrinking in my own tests?

Looking at the source there is a

implicit def derivedShrink[T]
   (implicit
     ev: LowPriority.Ignoring[Witness.`"shrinkAny"`.T],
     underlying: Strict[MkShrink[T]]
   ): Shrink[T] =
    underlying.value.shrink

But I don't understand what it does.

Jack Low

@wjlow

Is it possible to generate an Arbitrary instance using scalacheck-shapeless for my ADT that has a generic type in one of its cases?

sealed trait A
case class Foo[T](value: T) extends A
case class Bar(value: String) extends A

4 replies

Adrian

@adrian-salajan

Hello, can scalacheck-shapeless work with case classes with in turn have fields like Instant / Date / Curency / scala enumeratum, if there are "manual" arbitraries provided for these ?
I can't seem to make it work
the closest i get is to a

 scala.UninitializedFieldError: Uninitialized field

edit: it works just have to have all the implicit imports near
import org.scalacheck.ScalacheckShapeless._

Howy Perrin

@howyp

Hey @alexarchambault - is there a plan to release scalacheck-shapeless for Scalacheck 1.15 ?

Buk Lau

@mk9023_gitlab

Hi :wave:

I am experiencing an issue with the library that is described in this ticket: alexarchambault/scalacheck-shapeless#46

Will this be fixed in 1.15 or is not going to be fixed?

I have a case class and one of the fields is an Option of a type which is an ADT (a sealed trait with sum types) but when I get runtime exceptions when it tries to generate instances for it

If I take out the Option with that sealed trait all is fine

Buk Lau

@mk9023_gitlab

Is this version: "com.github.alexarchambault" %% "scalacheck-shapeless_1.14" % "1.2.5" compatible with

"org.scalatest"              %% "scalatest"                      % "3.2.3"          % Test,
"org.scalatestplus"          %% "scalacheck-1-15"                % "3.2.3.0"        % Test,

Buk Lau

@mk9023_gitlab

I fixed the issue - it only works with "org.scalatestplus" %% "scalacheck-1-14"

Where communities thrive

People

Repo info

Activity