alexarchambault/scalacheck-shapeless

Generation of arbitrary case classes / ADTs with scalacheck and shapeless

Felix Bjært Hargreaves

@hejfelix

/**
  * Case class representing a canonical Snowplow event.
  *
  * @see https://github.com/snowplow/snowplow/wiki/canonical-event-model
  */
case class Event(app_id:                   Option[String],
                 platform:                 Option[String],
                 etl_tstamp:               Option[Instant],
                 collector_tstamp:         Instant,
                 dvce_created_tstamp:      Option[Instant],
                 event:                    Option[String],
                 event_id:                 UUID,
                 txn_id:                   Option[Int],
                 name_tracker:             Option[String],
                 v_tracker:                Option[String],
                 v_collector:              String,
                 v_etl:                    String,
                 user_id:                  Option[String],
                 user_ipaddress:           Option[String],
                 user_fingerprint:         Option[String],
                 domain_userid:            Option[String],
                 domain_sessionidx:        Option[Int],
                 network_userid:           Option[String],
                 geo_country:              Option[String],
                 geo_region:               Option[String],
                 geo_city:                 Option[String],
                 geo_zipcode:              Option[String],
                 geo_latitude:             Option[Double],
                 geo_longitude:            Option[Double],
                 geo_region_name:          Option[String],
                 ip_isp:                   Option[String],
                 ip_organization:          Option[String],
                 ip_domain:                Option[String],
                 ip_netspeed:              Option[String],
                 page_url:                 Option[String],
                 page_title:               Option[String],
                 page_referrer:            Option[String],
                 page_urlscheme:           Option[String],
                 page_urlhost:             Option[String],
                 page_urlport:             Option[Int],
                 page_urlpath:             Option[String],
                 page_urlquery:            Option[String],
                 page_urlfragment:         Option[String],
                 refr_urlscheme:           Option[String],
                 refr_urlhost:             Option[String],
                 refr_urlport:             Option[Int],
                 refr_urlpath:             Option[String],
                 refr_urlquery:            Option[String],
                 refr_urlfragment:         Option[String],
                 refr_medium:              Option[String],
                 refr_source:              Option[String],
                 refr_term:                Option[String],
                 mkt_medium:               Option[String],
                 mkt_source:               Option[String],
                 mkt_term:                 Option[String],
                 mkt_content:              Option[String],
                 mkt_campaign:             Option[String],
                 contexts:                 Contexts,
                 se_category:              Option[String],
                 se_action:                Option[String],
                 se_label:                 Option[String],
                 se_property:              Option[String],
                 se_value:                 Option[Double],
                 unstruct_event:           UnstructEvent,
                 tr_orderid:               Option[String],
                 tr_affiliation:           Option[String],
                 tr_total:                 Option[Double],
                 tr_tax:                   Option[Double],
                 tr_shipping:              Option[Double],
                 tr_city:                  Option[String],
                 tr_state:                 Option[String],
                 tr_country:               Option[String],
....

any way to circumvent this?

Georgi Krastev

@joroKr21

@hejfelix does it also happen if you just try to get shapeless.Generic[Event]?

Georgi Krastev

@joroKr21

how many fields does it have?

Nicolás Cañibano

@ncanibano

I'm facing the same issue as @hejfelix, while using implicitly[Arbitrary[BigCaseClass]].arbitrary:

Could not write class xxx because it exceeds JVM code size limits. Method inst$macro$135$lzycompute's code too large!

The case class has 130 fields.
Any workaround to solve this issue? Thanks!

Georgi Krastev

@joroKr21

Did you try with just Generic?

What kind of fields does it have? Primitives or other case classes?

Nicolás Cañibano

@ncanibano

shapeless.Generic[BigCaseClass] compiles successfully, and the case class has primitive fields only, no nested case classes, any clues how to solve this?

Georgi Krastev

@joroKr21

I'm not sure

It sounds like shapeless is not the problem, and nested case classes are not the problem

So a bit difficult situation

Nicolás Cañibano

@ncanibano

I found a workaround using generics and implicit generators:

// primitive generators
implicit def stringGen(implicit arbString: Arbitrary[String]): Gen[String] = arbString.arbitrary
implicit def intGen(implicit arbInt: Arbitrary[Int]): Gen[Int] = arbInt.arbitrary

// hlist generator
implicit def hnilGen: Gen[HNil] = Gen.const(HNil)
implicit def hlistGen[H, T <: HList](implicit headGen: Gen[H], tailGen: Gen[T]) = headGen.flatMap(head => tailGen.map(tail => head :: tail))

// case class generator
implicit def caseClassGen[T, L <: HList](implicit generic: Generic.Aux[T, L], lGen: Gen[L]): Gen[T] = lGen.map(generic.from)
def generator[A](implicit gen: Gen[A]): Gen[A] = gen

Then you can generate arbitrary big case classes like this:

implicit val generic = Generic[YourBigCaseClass]

val gen = generator[YourBigCaseClass]

val sample = Gen.nonEmptyListOf(gen).sample.get

I hope that helps anyone facing this limitation.

Georgi Krastev

@joroKr21

But I wonder what in scalacheck-shapeless generates so much bytecode.

Dario Abdulrehman

@dabd

I'm trying to write a test using ScalaTest's ShrinkLowPriority trait but it seems to conflict with the internals of scalacheck-shapeless. I get

Error:(27, 88) ambiguous implicit values:
 both method derivedShrink in trait DerivedInstances of type [T](implicit ev: shapeless.LowPriority.Ignoring[String("shrinkAny")], implicit underlying: shapeless.Strict[org.scalacheck.derive.MkShrink[T]])org.scalacheck.Shrink[T]
 and method shrinkAny in trait ShrinkLowPriority of type [T]=> org.scalacheck.Shrink[T]
 match expected type ...

How can I disable shrinking in my own tests?

Looking at the source there is a

implicit def derivedShrink[T]
   (implicit
     ev: LowPriority.Ignoring[Witness.`"shrinkAny"`.T],
     underlying: Strict[MkShrink[T]]
   ): Shrink[T] =
    underlying.value.shrink

But I don't understand what it does.

Jack Low

@wjlow

Is it possible to generate an Arbitrary instance using scalacheck-shapeless for my ADT that has a generic type in one of its cases?

sealed trait A
case class Foo[T](value: T) extends A
case class Bar(value: String) extends A

4 replies

Adrian

@adrian-salajan

Hello, can scalacheck-shapeless work with case classes with in turn have fields like Instant / Date / Curency / scala enumeratum, if there are "manual" arbitraries provided for these ?
I can't seem to make it work
the closest i get is to a

 scala.UninitializedFieldError: Uninitialized field

edit: it works just have to have all the implicit imports near
import org.scalacheck.ScalacheckShapeless._

Howy Perrin

@howyp

Hey @alexarchambault - is there a plan to release scalacheck-shapeless for Scalacheck 1.15 ?

Buk Lau

@mk9023_gitlab

Hi :wave:

I am experiencing an issue with the library that is described in this ticket: alexarchambault/scalacheck-shapeless#46

Will this be fixed in 1.15 or is not going to be fixed?

I have a case class and one of the fields is an Option of a type which is an ADT (a sealed trait with sum types) but when I get runtime exceptions when it tries to generate instances for it

If I take out the Option with that sealed trait all is fine

Buk Lau

@mk9023_gitlab

Is this version: "com.github.alexarchambault" %% "scalacheck-shapeless_1.14" % "1.2.5" compatible with

"org.scalatest"              %% "scalatest"                      % "3.2.3"          % Test,
"org.scalatestplus"          %% "scalacheck-1-15"                % "3.2.3.0"        % Test,

Buk Lau

@mk9023_gitlab

I fixed the issue - it only works with "org.scalatestplus" %% "scalacheck-1-14"

Artem Nikiforov

@nikiforo

Hi all,
I'm using "com.github.alexarchambault" %% "scalacheck-shapeless_1.14" % "1.2.5", scala 2.13.5.
I expect implicitly[Arbitrary[IntWr]] to use arb in the code below, but it doesn't.

  case class IntWr(int: Int)

  val intGen = Gen.choose(5, 9)

  def shapelessCustom = {
    import org.scalacheck.ScalacheckShapeless._
    implicit val arb: Arbitrary[Int] = Arbitrary(intGen)
    mkList(implicitly[Arbitrary[IntWr]])
  }

  private def mkList[T](gen: Arbitrary[T]): List[T] =
    List.fill(10)(gen.arbitrary.sample.get)

For instance, here the correct Arbitrary[Int] is used:

def custom: List[Int] = {
    implicit val arb: Arbitrary[Int] = Arbitrary(intGen)
    mkList(implicitly[Arbitrary[Int]])
  }

Is this behavior expected?
How can I use custom arbitrary for a case class attribute?

Georgi Krastev

@joroKr21

How do can you tell that it's not used?

Artem Nikiforov

@nikiforo

How do can you tell that it's not used?

two evidences:

Warning that isn't used

Println on both lists gives:

// List(7, 5, 5, 8, 6, 8, 7, 5, 7, 6)
// List(IntWr(-2147483648), IntWr(1263246479), IntWr(1791721042), IntWr(0), IntWr(-90407902))

Georgi Krastev

@joroKr21

Hmm ok 1. btw it could also warn even when it's used depending on your configuration. Because an implicit which is used only after a macro has expanded might be detected too late. But the println is a good indicator :thumbsup:

❯ scala -Wmacros:help
Usage: -Wmacros:<mode> where <mode> choices are none, before, after, both (default: before).
  none    Do not inspect expansions or their original trees when generating unused symbol warnings.
  before  Only inspect unexpanded user-written code for unused symbols.
  after   Only inspect expanded trees when generating unused symbol warnings.
  both    Inspect both user-written code and expanded trees when generating unused symbol warnings.

Georgi Krastev

@joroKr21

Well ... it works on my machine :smile: https://scastie.scala-lang.org/7Ji2AhrCT5GK2FGyJ3yG8A

Artem Nikiforov

@nikiforo

❯ scala -Wmacros:help
Usage: -Wmacros:<mode> where <mode> choices are none, before, after, both (default: before).

Wow, I didn't know that compiler flags show some help this way. Neither I knew about flag -Wmacros :)).
Your message is about the first "evidence"? That without the second one my argument would be week?

Well ... it works on my machine :smile: https://scastie.scala-lang.org/7Ji2AhrCT5GK2FGyJ3yG8A

hmmmmm

Thanks!
Interesting... I'll look closer and try to narrow it somehow

Georgi Krastev

@joroKr21

:thumbsup:

Paul Snively

@paul-snively

Hi everyone! Does anyone know when we can look forward to a Scalacheck 1.15.x release? :-)

skestle

@skestle

If I provide an implicit arbitrary for a deep field in a nested type, should scalacheck-shapeless use it? (Mine isn't, wondering if that's expected behaviour)

skestle

@skestle

Yes, yes it does; My specific arbitrary implementation to produce valid data did not, in-fact, produce valid data. Now that it's fixed, everything works as hoped.

dantb

@dantb

Hey, are there any plans to move this library to Scala 3 (using shapeless 3 or otherwise)?

dantb

@dantb

paging @alexarchambault . We have started looking at doing it, don't want to duplicate any work though

skestle

@skestle

@dantb Do you think you could give an update on what you did with Scala 3 over at alexarchambault/scalacheck-shapeless#241

Where communities thrive

People

Repo info

Activity