Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Repo info
Activity
  • 08:02
    kriegaex commented #1298
  • 07:55
    Vampire commented #1298
  • 03:11
    kriegaex commented #1298
  • Jun 11 13:47
    milo2048 synchronize #1345
  • Jun 11 13:44
    milo2048 commented #1345
  • Jun 11 13:43
    milo2048 synchronize #1345
  • Jun 11 13:41
    milo2048 synchronize #1345
  • Jun 10 19:21
    leonard84 commented #1345
  • Jun 10 17:04
    milo2048 opened #1345
  • Jun 09 21:46
    codecov[bot] commented #1344
  • Jun 09 21:46
    codecov[bot] commented #1344
  • Jun 09 21:44
    codecov[bot] commented #1344
  • Jun 09 21:44
    codecov[bot] commented #1344
  • Jun 09 21:42
    codecov[bot] commented #1344
  • Jun 09 21:42
    codecov[bot] commented #1344
  • Jun 09 21:41
    codecov[bot] commented #1344
  • Jun 09 21:41
    codecov[bot] commented #1344
  • Jun 09 21:41
    codecov[bot] commented #1344
  • Jun 09 21:37
    leonard84 opened #1344
  • Jun 09 14:37
    kriegaex commented #111
Michael "M3" Lasevich
@mlasevich
I think I found a workaround though
Leonard Brünings
@leonard84
@mlasevich anything worth sharing?
Michael "M3" Lasevich
@mlasevich
Basically if you load it via Script, it will not work, so I made a copy of the class in the classpath, and then have my script extend the classpath copy with no body. Hacky AF, but gets the intended results
I imagine there is a way to game it by adding actual script to classpath and creating a fake script to load
Björn Lindahl
@blindahl
Hi, We are experiencing weird behaviour in our spock tests. We suspect concurrency issues but struggling to find detailed information on how specs are run, how variable instances on spec level are isolated between runs and more in depth information. Also if it's possible to get more information on runner level what test is ran when and on which thread etc. The documentation we've found about SAME_THREAD, Stepwise etc was really basic and didn't really answer our questions.
Can some point me to any resource describing some of those topics more in detail?
Björn Lindahl
@blindahl

We have problem that test fail sometimes. We're trying to pin point what causes it. We suspect concurrency but we are not completely sure.
Right now the it works consistently if we add @Execution(ExecutionMode.SAME_THREAD) on one of the spec. (Just one, that's enough).
If we add

runner {
    parallel {
        enabled false
    }

it sometimess fails as well which really confuses us. Could someone shed some light on this would be appreciated.

Leonard Brünings
@leonard84
Leonard Brünings
@leonard84
concurrency doesn't change spocks model in regards to fields, i.e., normal fields are per instance, and @Shared/static fields are shared between instances
Most of the concurrency problems stem from modifying access to a shared resource, that is spock offers @ResourceLock
Björn Lindahl
@blindahl
@leonard84 yes I've read that documentation. We've removed all @Shared instances and have no static instances.
It's weird that SAME_THREAD behaves different from runner.parallel.enabled=false since it states that they are similar as I read it, right?
Is there a way to see more output what happens inside spock when it runs the specs? Print its properties etc.
Leonard Brünings
@leonard84
Spock has no logging
its not officially released yet, but you can give it a go
if you don't want to write the extension yourself you can grab it from the solution branch https://github.com/spockframework/workshop/tree/2.0-advanced-solution
Alexander Kriegisch
@kriegaex

@blindahl , just to throw in another train of thought, not contradicting what Leonard said in any way but offering an alternative way of thinking about the problem:

IMO if you run Spock without explicitly enabling parallel execution and believe to have concurrency issues, logic implies that you ought to search for the root cause of those issues outside of Spock and inside of your classes or applications under test, unless you explicitly start threads in your Spock tests.

BTW, that you have flaky tests could have all kinds of reasons, concurrency being only one of them. Timing issues, e.g. due to high system load, could be another one.

Alexander Kriegisch
@kriegaex
Also, it would be interesting to know what kind of tests are failing: unit tests, integration tests, maybe Geb tests for web a UI?
Björn Lindahl
@blindahl
Unit tests. Absolutely. It could be other reasons. But we can't reproduce errors locally. They only occur in CI (jenkins linux agent) and only one in 5-10th of our runs. The type of flakyness not seldom relates to concurrency and since they go away when explicitly setting SAME_THREAD it indicates even more that it might be issue with concurrency.
Leonard Brünings
@leonard84
Are they timing related? But yeah as @kriegaex said, without more info of what is failing, it is hard to help.
Björn Lindahl
@blindahl
Yeah, I'm not asking you to solve our issue. I'm aware that it's not half the information needed to do that. I was mainly looking for more indepth information about sputnik etc. I'll look more closely on the links you provided. Thanks.
Alexander Kriegisch
@kriegaex

I agree after this additional explanation. There is a strong indicator. No proof, but an indicator justifying investigation.

Can you share any further details about your tests or even some (anonymised, generalised) version of your tests, if necessary in a private GitHub repo, for us to inspect? Seeing what we are trying to help you debug would be helpful.

I know you are not asking us to solve your issue, but it certainly is intriguing and finding the root cause might not just help you but maybe also the Spock maintainers uncover any potential problem in Spock itself.

Björn Lindahl
@blindahl
It's tricky to share some relevant code. Need too much work to anonymize I'm afraid.
Alexander Kriegisch
@kriegaex

private GitHub repo

Those used to be only for paying customers, but AFAIK they are free now and you could just invite Leonard and me.

Björn Lindahl
@blindahl
Sorry private/public repo doesn't make any difference unfortunately.
Alexander Kriegisch
@kriegaex
Well, then it is a matter of trust or at some point you shall be on your own. Just speaking for myself, I have absolutely no problem signing an NDA.
Good luck then.
Björn Lindahl
@blindahl
Appreciate your proposition though.
Alexander Kriegisch
@kriegaex
Feel free to report back whenever you can come up with anything resembling or even reproducing your problem. Leonard's knowledge is far superior to mine, he might be able to help you with way less information than I.
Björn Lindahl
@blindahl
We'll stick to going with SAME_THREAD and see if the error pop up somewhere else in the application leading us towords the issue. We're intrigued with why it occures as well so we won't ignore it completely.
Yeah, will for sure report back if it has to do with Spock.
Alexander Kriegisch
@kriegaex
FWIW, I love tricky problems. The easy ones can be boring to debug. ;-)
Björn Lindahl
@blindahl
Hehe :)
Björn Lindahl
@blindahl
A couple of other things with mentioning:
It's always same the tests that fail when they fail. They all fail when any of them fail. It's enough to annotate one spec file with SAME_THREAD to not make the tests fail. It doesn't have to be any spec containing the tests that fail occasionally but that's maybe a result of how SAME_THRED is implemented? I haven't fully grasped all detaild yet.
Alexander Kriegisch
@kriegaex
You might learn something about your problem if you abstract it, i.e. condense it down into a new project with similar characteristics. This is called an MCVE. While creating one, you might already find out what your problem is. If you don't but can reproduce the problem with the MCVe on your local machine or CI system, then the nice side effect is that you can share the MCVE with us and have a few more pairs of eyes inspecting it.
Is there any chance that you might have multiple Spock configurations on your CI system, maybe overriding each other in a strange way, making SAME_THREAD even necessary of effective beside runner.parallel.enabled=false? And are you sure, your Spock config is being picked up? You can just add diagnostic println statements to it, it is a Groovy file. Sorry for the unstructured brainstorming style of response, but I guess at this point any educated guess might be more helpful than none at all.
Björn Lindahl
@blindahl
Absolutely, every idea is welcome at this point :)
Leonard Brünings
@leonard84
None of the parallel extensions have any effect if spock isn't running in parallel mode
All they do is set meta data, that is used by the JUnit Platform and in the non parallel mode that meta data is ignored
Björn Lindahl
@blindahl
I have three config files right now, one for each module in our project but I only added them to add runner.parallel.enabled=false. Right now I added a println to them as well which was printed when running locally. Will try on CI as well.
Alexander Kriegisch
@kriegaex
Do you have any parallel configuration e.g. for Maven builds in general, Maven Surefire in particular or the Gradle equivalents, something independent of Spock? (Maybe a desperate guess and I am not even sure how this would work in combination with Spock 2.)
Leonard Brünings
@leonard84
runner.parallel.enabled=false is the default you need to opt-in to parallel execution with runner.parallel.enabled=true
Alexander Kriegisch
@kriegaex

Exactly! That's what I meant when I said earlier:

IMO if you run Spock without explicitly enabling parallel execution and believe to have concurrency issues, logic implies that (...)

Leonard Brünings
@leonard84

btw spocks config scrips are groovy so you can do stuff like

runner {
    parallel {
        enabled !System.getenv('CI')
    }
}

Instead of having 3 different files

Björn Lindahl
@blindahl
Hmm, have to check how maven surefire build
Ah ok, can use one config file then that way.
We didn't think it was running in parallel either but we wanted to set it explicitely.
Leonard Brünings
@leonard84
Also be aware that Surefire has some issues with the JUnit Platform
Björn Kautler
@Vampire
This friday Gr8 tech talk with Leonard about cool new things in Spock 2.0: https://twitter.com/gr8conf/status/1351127175286706177
sevenlist
@sevenlist

Hello! Using Spock, I try to use Mockito's static mocking feature. Here's a simple example for mocking an SLF4J/Logback Logger:

given:
Logger loggerMock = Mock()
MockedStatic loggerFactoryMockitoMock = Mockito.mockStatic(LoggerFactory)
loggerFactoryMockitoMock.when(LoggerFactory.getLogger(ClassThatLogs)).thenReturn(loggerMock)

and:
def objectThatLogs = new ClassThatLogs()

when:
objectThatLogs.logSomethingWithInfo()

then:
1 * loggerMock.info('some expected message')

cleanup:
loggerFactoryMockitoMock.close()

This works, but not with @Unroll tests, because from the second test on the class ClassThatLogs still references the loggerMock from the first test. To me it looks like as if the class ClassThatLogs has not been reloaded. Do you how I could make it work?

(BTW: ClassThatLogs has a private static final Logger LOG = LoggerFactory.getLogger(ClassThatLogs.class);)

4 replies