Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Repo info
Activity
    Gabriele Petronella
    @gabro
    ah, thanks for clarifying that @griggt ­čÖĆ Looking forward to 3.0.1 then
    Milan van der Meer
    @milanvdm
    Is there an out-of-the-box way to retry tests failing with a specific error?
    Ólafur Páll Geirsson
    @olafurpg
    @milanvdm you can write a test transformer
    if you want this behavior for all tests then itÔÇÖs best to have an abstract super class that defines the test transformer, and all suites extend that class
    Karthik Krishnamurthy
    @kkrs
    Hi, I'm trying to find how to run test suite like I'd do with scalacheck
    using Prop.check for example
    Gabriele Petronella
    @gabro
    @kkrs have you read the documentation of the scalacheck integration? Does it cover it?
    Daniel Esik
    @danicheg
    Hi there. It would be awesome if someone could review this scalameta/munit#393. Thanks :pray:
    Karthik Krishnamurthy
    @kkrs
    @gabro - i'm not looking to define property tests. Just want to know how I can run an munit test from main or an ammonite script for example. Prop.check allows me to do that.
    Ólafur Páll Geirsson
    @olafurpg
    @kkrs this should probably be documented on the website. You can use JUnit main runners with MUnit
    I donÔÇÖt have a ready example but you should
    be able to find some via google for how to run JUnit tests programmatically
    Gerry Fletcher
    @gerryfletch
    Hi all. What would be the best way to extend the test function in FunFixtures to accept an additional implicit parameter, and pass it to the body of the test?
    Essentially I have an F[Trace[F]], and I'd like to flatMap the Trace[F] into an implicit argument for the test body. I'm using cats-effect-munit.
    Ólafur Páll Geirsson
    @olafurpg
    @gerryfletch the best way to achieve that is to implement a new method with a different name
    Gerry Fletcher
    @gerryfletch
    I gave that a go but I was pretty tired; i'll try again!
    Jens Halm
    @jenshalm
    Is it expected behaviour for munit that, when run with sbt and parallel execution on, it logs the tests of multiple suites intertwined and not, like ScalaTest for example, grouped by suite? Is there a way to change the configuration for this? It's something that could potentially stop me from migrating to munit.
    Ólafur Páll Geirsson
    @olafurpg
    @jenshalm I think Test / testOptions += Tests.Argument(TestFrameworks.MUnit, "+l") configures MUnit to use the standard sbt loggers instead of println
    it's disabled by default because sbt adds an [info] prefix to the output that messes up with how MUnit formats error messages
    the +l name is cryptic, I can't remember exactly why it's not self-descriptive. The flag parsing logic started as a fork of sbt/junit-interface that uses single character flag names
    Jens Halm
    @jenshalm
    Ok, thanks, I'll try that, but it sounds like it would just bring a different set of downsides. There is no way the munit implementation could change so that it does the proper grouping without sbt's logging? Meaning, is it something that would be difficult/impossible to implement, or just something that hasn't been done yet? If it's the latter I might try to look into it at some point. Good output in sbt is quite important for me as I cannot run the tests properly in IntelliJ either (their support for JVM/JS cross projects seems to be very poor - with any test framework)
    Ólafur Páll Geirsson
    @olafurpg
    It's probably possible to implement a buffering logger outside of the sbt system that flushes the output after each test suite completes
    You may need to implement the same logic twice, once in Scala and once in Java
    not 100% sure if that's necessary here, but some internals are duplicate to support Scala.js and Native
    Jens Halm
    @jenshalm
    Oh, that sounds fun. :-) Thanks for the pointers. I'll first check how quickly I can make sense of the implementation (I don't know much about JUnit or sbt's test framework, so it might take a while).
    after every test suite has finished running
    Jens Halm
    @jenshalm
    Isn't it more complicated though? Wouldn't the buffering need to happen with one buffer per suite in case of parallel execution?
    Jens Halm
    @jenshalm
    I only briefly skimmed over the code in the two classes, I might take a closer look over the weekend. But I did not spot any easy opportunity to add this behaviour, as EventDispatcher only seems to have a way of detecting the first test of the suite and not the last.
    Another little thing I noticed (although I might be missing something) is that RichLogger. popCurrentTestClassName() might be prone to race conditions in parallel execution.
    Ólafur Páll Geirsson
    @olafurpg

    is that RichLogger. popCurrentTestClassName() might be prone to race conditions in parallel execution.

    Good catch. That field can probably be removed because it's not really used

    That code was originally forked from junit-interface, which had a different way to highlight relevant stack trace elements
    I think you can buffer log messages in something like private final ConcurrentHashMap<Description, ConcurrentLinkedDeque<String>> buffers = new ConcurrentHashMap<>();
    Ólafur Páll Geirsson
    @olafurpg
    And then override the testSuiteFinished hook to flush logs
    System.out is already synchronized so everything should work in a concurrent setting as long as you use a call println() or write() once per test suite
    Ólafur Páll Geirsson
    @olafurpg
    feel free to commandeer that branch and open a PR
    I'm open to enabling this by default
    Jens Halm
    @jenshalm

    Ah great, I certainly would have needed much longer. I'll play with this on the weekend.

    My main hurdle for quickly understanding the fairly simple code of RichLogger or EventDispatcher was merely the scoping, meaning how many instance of those exists, whether they are global, per suite, per test or some other scope. Which determines how they need to be synchronized (if at all). The use of synchronization in EventDispatcher suggests it is not "per suite", however it is only ever instantiated in JUnitTask.execute which kind of felt to me like it is scoped to a single suite.

    Jens Halm
    @jenshalm
    @olafurpg did a quick check on my project and it seems to work great (apart from a minor issue of reverse order which is trivial to fix). I'll turn this into a PR over the weekend and also add some docs (I'd suggest to the existing section "Run tests in parallel" as it matters the most in that context).
    Jens Halm
    @jenshalm
    btw. I think my confusion was justified. :-) Out of curiosity I added a lot of low-level logging to EventDispatcher and the result is: 1) I get one instance per suite, 2) the reportedSuites property never has more than one item in it (which is always the suite the dispatcher belongs to), 3) even with parallel execution each dispatcher is always invoked by the same thread. It's not super-important performance-wise, but from this observation it seems that some of the synchronization in EventDispatcher and RichLogger may not be necessary.
    Ólafur Páll Geirsson
    @olafurpg
    Interesting
    I only did the minimal changes to the original junit-interface implementation to tweak the error message reporting to my preferences
    Looks like JUnitReporter is the Scala-equiavalent for EventDispatcher (they should probably be named the same)
    I don't think we need to tweak the Scala implementation, it's only used for Scala.js and Native
    Jens Halm
    @jenshalm
    Yes, I'll look into general improvements after I did the PR for buffered logging. I think EventDispatcher and RichLogger should be cleaned up. They are implemented for multi-suite, multi-thread use, but they are A) not used in that way in practice, B) would be buggy if used that way as there is a race condition.
    Jens Halm
    @jenshalm

    @olafurpg I did the PR for buffered logging here: scalameta/munit#397 - I did not make this option the default yet, that's kind of your call, happy to add that change if you want. I'm also open to duplicate the work for JS and Native, maybe in a separate PR.

    I also opened an issue for several concurrency issues I found in the junit-interface module (more than the ones I already mentioned here): scalameta/munit#399. It's easy to fix those in the way suggested in the ticket, but I did not start yet, as I wanted to check whether you have any objections. The work is also best done after buffered logging is merged.

    Ross A. Baker
    @rossabaker:matrix.org
    [m]
    I don't have a minimization yet, but http4s has lost its test output since upgrading to 0.7.28. I get it back if I turn off the buffered logging.
    Ross A. Baker
    @rossabaker:matrix.org
    [m]