Where communities thrive

• Join over 1.5M+ people
• Join over 100K+ communities
• Free without limits
Activity
Nicolas Rinaudo
@nrinaudo

I have a question about the default Shrink[Int] instance.

From what I can see, it "simply" halves its input it reaches 0. This obviously works, but isn't very precise. Let's say, for example, that you have the following property:

  val propTest = forAll { i: Int =>
i < 156
}

If you get an input of, say, 482, it'll get halved to 241, then get stuck there - it has no way of getting back "up" from (241 / 2) to 156.

An alternative implementation would be to halve the input, but then explore the larger part of the halved space:

  implicit val intShrink: Shrink[Int] = {

def halves(inf: Int, sup: Int): Stream[Int] = {
if(math.abs(sup - inf) <= 1) Stream.empty
else {
val mid = inf + (sup - inf) / 2
mid #:: halves(mid, sup)
}
}

Shrink(i => halves(0, i))
}

I've ran some tests and this seems to consistently find the most precise value possible.

Was there any reason it wasn't implemented that way? For example, "hoping" for a value closer to 0 and avoiding getting stuck in a local optimum?

Aaron S. Hawley
@ashawley
@nrinaudo Well, complexity would be a concern, but I presume it was implemented this way since it was the simplest thing. Others have raised making this improvement of shrinking on a known range and not just zero. It seems worth pursuing.
Fortunately, not a lot of property testing is numerical it's structural or algebraic. Not that it's worth improving, but might explain why this doesn't come up, often.
To be fair to the current implementation, numerical test generators are supposed to run with increasing values so a failure would cause shrinking to use decreasing values, but it's not hard to find trivial cases, as you have, where that doesn't work optimally.
Nicolas Rinaudo
@nrinaudo

@ashawley yeah I've actually pushed the experience further and it's not actually a good idea. Binary search works because you keep a running context - [a, b]. With shrinking, you do have that context while looking for the next failing test case, but you lose it as soon as you find that test case.

That is, you can be looking for your local minimum in [100, 150], and you'll end up searching in [0, 125].

You end up spending far more time looking at values you've already explored, and I'm not at all convinced it's safe - there *must be some infinite loop scenario

Basavaraj Kalloli
@scalolli
Hello i have just updated Scala Version to 2.13.4, trying to get rid off these Stream warnings:
def noShrink[T]: Shrink[T] = Shrink(_ => Stream.empty)
But I see the scalacheck library still expects a Stream, whats the recommendation here?
João Costa
@JD557
you might have to use @nowarn, not sure if that's the only way
Aaron S. Hawley
@ashawley
@scalolli Doesn't the warning suggest using LazyList?
Actually, we couldn't overload that method to support LazyList. We had to add a new method withLazyList in #627.
3 replies
Aaron S. Hawley
@ashawley
If you're having to support 2.12 or earlier, then there may be nothing you can do about the warning.
Artem Nikiforov
@nikiforo

Hi, I want to include some properties into ordinary AnyFunSuite, for example

MyAwesomeTest extends AnyFunSuite {
test("ordinary test") {
assert(true)
}

test("scalacheck test" {
val prop = forAll { b: Boolean =>
b
}
prop.check()
}
}

How can I launch property check in that scenario? I want to have a red failed test instead of

! Falsified after 0 passed tests.
> ARG_0: 0
> ARG_0_ORIGINAL: 71

I haven't found the answer in https://github.com/typelevel/scalacheck/blob/master/doc/UserGuide.md

Aaron S. Hawley
@ashawley
@nikiforo That looks like Scalatest. You might refer to their docs https://www.scalatest.org/user_guide/writing_scalacheck_style_properties
You can probably just write
test("scalacheck test") {
forAll { b: Boolean =>
b
}
}
Keir Lawson
@keirlawson
Hey, I'm wanting to generate a string up to a given number of characters, what's the best way to do this? I see I can use listOfN for a fixed length, but not sure how to get the "up to"
Dmitry Polienko
@nigredo-tori
@keirlawson,
Gen.choose(0, max)
.flatMap(Gen.stringOfN(_, arbitrary[Character]))
Basavaraj Kalloli
@scalolli
Another deprecation warnings I am trying to fix with Scala 2.13.4 is with these:
Auto-application to () is deprecated. Supply the empty argument list () explicitly to invoke method dispatch,
or remove the empty argument list from its definition (Java-defined methods are exempt).
In Scala 3, an unapplied method like this will be eta-expanded into a function.
forAll {}
I cannot find an alternative to the forAll {} syntax, is there any obvious alternative that I am missing in the library?
2 replies
Aaron S. Hawley
@ashawley
@scalolli What are you doing with forAll that produces that warning?
2 replies
Nicolas Rinaudo
@nrinaudo
@keirlawson i think that’s what the sized combinator is meant for
Nicolas Rinaudo
@nrinaudo
The resize combinator, I mean
Nicolas Rinaudo
@nrinaudo

Here's a different attempt at shrinking ints (basically a Scala version of the QuickCheck implementation) which seems to have better results than the default one: https://scastie.scala-lang.org/XbfRtWfFRG6rAKy6mUVbjg

Note that, contrary to the default ScalaCheck instance, it does not explore negative values, simply because QuickCheck doesn't.

Is there any reason the QuickCheck implementation wasn't ported over as is?

If not, I'm happy to make a more generic version and submit a PR...

Guillaume Martres
@smarter
that looks nice!
João Costa
@jd557:matrix.org
[m]
fwiw, you have a typo in your example (you have written "Old shrink" twice) :P
Nicolas Rinaudo
@nrinaudo
yeah I realised afterwards, and was kind of hoping no one would pick up on it. Clearly, that was in vain :)
a couple of important things to note:
• this implementation can be centered on any number, not just 0, which can be useful when you're working with, say, ints between 50 and 100.
• it starts by attempting the destination (0, here), because that's the largest possible shrink and will often end up short-circuiting the entire process
• I take exactly 0 credit for it, it's really just me ripping it from Hedgehog (which, itself, ripped it from QuickCheck)
Nicolas Rinaudo
@nrinaudo
I'd have pasted a sample shrink tree, but since this is so much more thorough than the default shrink, they end up being huge and not terribly useful.
Yuliban
@yupegom_gitlab

Hi everyone. I'm trying to generate an ADT that is previously validated. So, instead of returning the actual ADT it is returning a Validate[ADT]. This is the generator:

val coordinatesGen: Gen[ValidationResult[Coordinates]] = for {
latitude <- arbitrary[Double].suchThat(BigDecimal(_).scale <= 8).suchThat(_.abs < 90)
longitude <- arbitrary[Double].suchThat(BigDecimal(_).scale <= 8).suchThat(_.abs < 180)
} yield {
Coordinates(
latitude = latitude,
longitude = longitude
)
}

implicit val arbCoordinatesGen: Arbitrary[ValidationResult[Coordinates]] = Arbitrary(coordinatesGen)

And this is the test:

property("coordinates") = forAll { coordinates: ValidationResult[Coordinates] =>
coordinates.isValid
}

This is not working tho, it seems like no test is passing: Generating coordinates.coordinates: Gave up after only 0 passed tests. 501 tests were discarded.. Is this related to the fact that I'm returning a ValidationResult instead of the actual ADT?

Christopher Davenport
@ChristopherDavenport
You want to use restricted generators rather than using filters like that .
Gen.between I think is the one I'm thinking of
Yuliban
@yupegom_gitlab
Thanks @ChristopherDavenport . When you say "rather than using filters like that" you mean this filter: coordinates.isValid?
Yuliban
@yupegom_gitlab
Oh I got it you meant avoiding using suchThat filters basically. Why those wouldn't work?
Christopher Davenport
@ChristopherDavenport
Because those just throw away the value, if it's too strict then it will fail like your case did above
Aaron S. Hawley
@ashawley
@nrinaudo Yes, feel free to open a PR for improved numeric shrinking. Missed your message earlier. I'm not sure why ScalaCheck has a different shrink from Quickcheck. It could be that Scalacheck fell behind the times.
Nicolas Rinaudo
@nrinaudo
@ashawley will do. Am I ok to basically port the quickcheck version, or do you want to retain the scalacheck twist of swapping signs?
On an unrelated subject, does anyone know why Arbitrary exists? I assumed because it does in QuickCheck, but in QuickCheck, it’s purpose is to aggregate Gen and Shrink, which I don’t think ScalaCheck does?
Aaron S. Hawley
@ashawley
No, it wouldn't seem that it's necessary to arbitrarily swap signs if the halving strategy is smartly shrinking across the negative domain.
Nicolas Rinaudo
@nrinaudo
It does not - if the initial failing test case is positive, and the “center” of the gen’s interval is 0+, you’ll never explore negative values. But that seems desirable to me
Exploring values on the other side of the interval’s “zero” feels like looking for a different failure symptom. But it can definitely be argued both ways, and it’s easy enough to implement sign swapping, so if you’d rather keep it for scenarios where zero is 0, the smallest positive failing test case is 123456789 and the smallest negative one -1, then I can definitely do that
Anton Sviridov
@velvetbaldmime:matrix.org
[m]

Apologies in advance for the super vague question - still investigating.

We have some tests that were setting Seed(0) generating several instances of a fairly large generator.

After doing some rounds of upgrades (Scala 2.12 patch from 12 to 13), we noticed that the values generated are no longer the same.

So my question is - are all Seeds born equal? Could there be something special about our poor choice of seed 0?

Scalacheck is 1.14.0, but 1.15.4 has the same behaviour. I'm really clutching at straws here, everything else failed :)

(note that we don't use the generated values in a property, we just generate the values and compare them against a pre-defined set. It's not a good test.)
Aaron S. Hawley
@ashawley
@velvetbaldmime:matrix.org Using seed in a property test is usually an anti-pattern. ScalaCheck does it in its test suite to verify its implementation. Now that ScalaCheck prints the failing seed, it seems less necessary for a user to use it.
I don't recall what happened to the Seed implementation in 1.14.0 or earlier. I know in 1.15.0 there was #674 and #651.
Anton Sviridov
@velvetbaldmime:matrix.org
[m]
Yeah, this whole test is sadly an anti-pattern and we're trying to convert it, while attempting to understand what could possibly have affected it :D
Artem Nikiforov
@nikiforo

Hi all,
I want generated values to be unique by some field.
For instance,

case class Entity(id: Int, code: String)

check { entities: List[Entity] =>
// I want every generated entity in a list to have unique code
}

Is it possible?

Aaron S. Hawley
@ashawley
@nikiforo This is one way to do it
  property("genEntity") = {
val genEntity: Gen[List[Entity]] = {
for {
id <- Arbitrary.arbitrary[Int]
codes <- Gen.containerOf[Set,String](Arbitrary.arbitrary[String])
code <- codes.toList
} yield {
Entity(id, code)
}
}
Prop.forAllNoShrink(genEntity) { (entities: List[Entity]) =>
Prop.collect(entities.size) {
entities.size >= 0
}
}
}
Artem Nikiforov
@nikiforo

Hi all,
Consider this code:

  var seen = 0
var filtered = 0
private val genInt =
Arbitrary
.arbInt
.arbitrary
.map { v => seen += 1; v }
.filter(_ % 2 == 1)
.map { v => filtered += 1; v }

private implicit val arbInt: Arbitrary[Int] = Arbitrary(genInt)

test("uniqueGen") {
var listed = 0
check { ints: List[Int] =>
listed += ints.length
println(s"$seen/$filtered/\$listed")
true
}
}

running tests results in Gave up after 5 successful property evaluations. 51 evaluations were discarded.

And the logs are:

0/0/0
4/3/3
114/43/3
310/100/4
0/0/0
0/0/0
4/1/1
19/7/7
155/47/11

From my POV this check should succeed, shoudn't it?

Aaron S. Hawley
@ashawley
It should, but creating a collection (i.e. List) of Ints has a filter for size (see definition of Gen.buildableOfN) and then your Gen[Int] has a filter for oddsthat causes ScalaCheck to exhaust the number of attempts. You might try retryUntil instead of filterbut that suggests a different implementation.