Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Repo info
Activity
  • 18:22
    marcphilipp commented #1972
  • 14:53
    kriegfrj commented #1980
  • 14:53
    kriegfrj commented #1980
  • 14:51
    kriegfrj commented #1989
  • 14:49
    kriegfrj commented #1980
  • 12:07
    marcphilipp assigned #1959
  • 12:07
    marcphilipp unassigned #1959
  • 12:07
    marcphilipp assigned #1959
  • Aug 23 12:54
    sbrannen labeled #1959
  • Aug 23 12:53
    sbrannen commented #1959
  • Aug 23 10:12

    junit-buildmaster on gh-pages

    Generated by gradle-git-publish. (compare)

  • Aug 23 09:58
    sbrannen commented #1990
  • Aug 23 09:58
    sbrannen milestoned #1990
  • Aug 23 09:58
    sbrannen labeled #1990
  • Aug 23 09:58
    sbrannen labeled #1990
  • Aug 23 09:58
    sbrannen labeled #1990
  • Aug 23 09:58
    sbrannen labeled #1990
  • Aug 23 09:57
    sbrannen commented #1986
  • Aug 23 09:57
    sbrannen commented #1990
  • Aug 23 09:44
    sbrannen commented #1972
Sam Brannen
@sbrannen
By default, Gradle doesn't log anything about skipped/disabled tests to the console.
Stéphane Nicoll
@snicoll
Exactly and it would log a warning for something that might just be very sensible
IMO, the semantic of ignored tests (the way build systems and other tools handle them) has changed
I have a concrete example. I need to write a test slightly differently according to the JDK. I have two tests that exclude each other. The net result is that I end up with 1 test ignored and a warning. Other tools like SonarQube would probably report a warning either.
Sam Brannen
@sbrannen

Exactly and it would log a warning for something that might just be very sensible

I don't think Gradle logs a warning, or at least I never noticed that it's a "warning" per se.

Stéphane Nicoll
@snicoll
Ok whatever. Some tools? Maven does by default
Sam Brannen
@sbrannen
If that's the case for Maven, I'd say Maven can be improved.
I agree that skipped and disabled tests are often not a sign of something going potentially wrong.
Stéphane Nicoll
@snicoll
It’s not the only tool. I am asking here as the semantic of @Ignored has changed a lot and it would be worth reaching out to these communities
Sam Brannen
@sbrannen
Especially with the kind of support one gets for disabling tests with Spock or JUnit Jupiter.
Even for JUnit 4, a failed assumption (AssumptionViolatedException) is not the same as an @Ignored test.

It’s not the only tool. I am asking here as the semantic of @Ignored has changed a lot and it would be worth reaching out to these communities

Do you have other communities in mind other than Maven Surefire?

Stéphane Nicoll
@snicoll
SonarQube
Sam Brannen
@sbrannen
I think Gradle, IDEA, and Eclipse currently do the right thing.
Stéphane Nicoll
@snicoll
But they’re reading the reports of the build I would assume so potentially different
Sam Brannen
@sbrannen
OK, so Maven Surefire and SonarQube.

But they’re reading the reports of the build I would assume so potentially different

In the de facto JUnit 4 XML reports, it's just a <skipped/> element within a <testcase>.

Stéphane Nicoll
@snicoll
Yeah I guess what I am trying to say is that it’s an on/off switch with loose semantic. Even more now that we have plenty of features to do conditional tests that are effectivly skipped based on that
Sam Brannen
@sbrannen
Agreed
Stéphane Nicoll
@snicoll
It’s encouraging people to skip more tests which increases the surface of the problem IMO
Sam Brannen
@sbrannen
Are you saying we need finer grained reporting for skipped/disabled tests and containers?
Stéphane Nicoll
@snicoll
I have no idea what the right solution to that problem is. Blindly treating that as a warning looks wrong to me though
Sam Brannen
@sbrannen
Yes, we agree on that: skipped tests should no longer be logged as a warning by those tools.
Sam Brannen
@sbrannen
Related issues:
Stéphane Nicoll
@snicoll
Thanks
jaroslavpadivy
@jaroslavpadivy
@sbrannen is there any other way to execute the test in a order? we will use jenkins
Christian Stein
@sormuras
Does the @TestMethodOrder annotation serve your needs? https://junit.org/junit5/docs/current/user-guide/#writing-tests-test-execution-order
jaroslavpadivy
@jaroslavpadivy
@sormuras the problem is we have more classes... I think the only thing we can do is make one class with SMOKE-tests and put the @Order in it. we have about 20 smoke test that rely on each other. (Just for the smoke tests) In junit 4/3 you could create a test suit.
Christian Stein
@sormuras
Ah, I see. That's an open issue for Jupiter: junit-team/junit5#330 and the related one junit-team/junit5#744
Sam Brannen
@sbrannen
@jaroslavpadivy, there's also the recent discussion about a possible @TestClassOrder: junit-team/junit5#1948
choojoykin
@choojoykin
hello
could u tell me how to do it right
i have 2 test classes with 2 test methods with my custom annotation eg TmsID(1) and TmsID(2)
i want to run test with TmsID(1) from maven eg mvn test -Dtmsid=1
so i implemented ExecutionCondition interface. in evaluateExecutionCondition method i parse ExtensionContext by condition eg TmsID annotation present and this annotation contains my id from -Dtmsid=1
so when its true => i return ConditionEvaluationResult.enabled else => ConditionEvaluationResult.disabled
in this case test with TmsID(2) marked as skipped
so my querstion how to make this test not marked as skipped and not present overall in report
ty and ssory for my eng
Sam Brannen
@sbrannen
If you implement an ExecutionCondition and that returns disabled for a test method, that method will always be reported as disabled/skipped.
There is no way around that.
If you want to "skip" something and not have it show up in reports, your only option is to use tags instead of conditions.
For example, annotate your tests with @Tag("tmsid_1") and @Tag("tmsid_2") (though likely better implemented via custom composed annotations @TmsID1 and @TmsID2), and include the "tmsid_1" or "tmsid_2" as appropriate.
See:
choojoykin
@choojoykin
thank you
soura62
@soura62
hey can anyone help with Test suite and how to implement it in JAVA 8
Daniel Rodriguez Hernandez
@drodriguezhdez
Hello,
Is there any possibility to get the value of the parameters in a @ParameterizedTest in TestExecutionListener methods like “executionStarted”, “executionFinished”, etc..
Thanks in advance!
joerg1985
@joerg1985
Hi, we wrote an Extension/AfterAllCallback which is registered to our BaseTestCase and now we try to write a self test of this Extension.
The SelfTestCase is derived from BaseTestCase and needs to check stuff done by the Extension/AfterAllCallback, we tried to attatch to SelfTestCase using @AfterAll or AfterAllCallback.
But theses are executed before the Extension/AfterAllCallback is executed. How is it possible to write a self test for Extensions?
Sam Brannen
@sbrannen
The best way to test extensions is to use the Engine Test Kit: https://junit.org/junit5/docs/current/user-guide/#testkit
choojoykin
@choojoykin

hello
if i need to run junit5 tests by tag from maven eg mvn test -tags/groups?=fast what i need to add in pom.xml surefire plugin configuration?
i saw many examples and none of them didnt work - no tests are being executed
in officical docs https://junit.org/junit5/docs/current/user-guide/#running-tests-build-maven-filter-tags i saw example of config

<artifactId>maven-surefire-plugin</artifactId>
            <version>2.22.2</version>
            <configuration>
                <groups>acceptance | !feature-a</groups>
                <excludedGroups>integration, regression</excludedGroups>
            </configuration>

if i try to customize this like

 <groups>${tags}</groups>

and run like mvn test -Dtags=fast it didnt work to me

Jan Jansen
@farodin91
Hi
Try implement a ConditionExtension which evaluate a condition after beforeeach. Any how i can do this?
choojoykin
@choojoykin

hi. im use junit5 (not junit4 with vintage)
i wanna execute tests by tag and default exec with mvn test -Dtest=TestClass
if i run tests by mvn test -Dgroups=fast or mvn test -Dtest=TestClass - everything ok
but when i try to override groups to tags in maven-surefire-plugin config props like

                <configuration>
                    <properties>
                        <groups>${tags}</groups>
                    </properties>
                </configuration>

and run test like mvn test -Dtags=fast its ok but if i wanna run like default style mvn test -Dtest=TestClass junit
maven throw an exception - org.apache.maven.plugins:maven-surefire-plugin:2.22.2:test (default-test) on project junittag: No tests were executed!
additionaly im added to surefire plugin section this dependency

<dependency>
                        <groupId>org.apache.maven.surefire</groupId>
                        <artifactId>surefire-junit-platform</artifactId>
                        <version>2.22.2</version>
                    </dependency>

so my question is - what can i do in this case?