Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Repo info
Activity
  • Nov 27 20:32
    rprouse commented #3677
  • Nov 27 14:31
    jnm2 commented #3376
  • Nov 27 14:30
    jnm2 commented #3376
  • Nov 27 14:30
    jnm2 commented #3376
  • Nov 27 14:29
    jnm2 commented #3376
  • Nov 27 14:20
    jnm2 commented #3677
  • Nov 27 14:15
    jnm2 labeled #3677
  • Nov 27 14:08
    jnm2 commented #3376
  • Nov 27 14:08
    jnm2 commented #3376
  • Nov 27 07:49
    manfred-brands commented #3677
  • Nov 27 07:40
    Dreamescaper commented #3677
  • Nov 27 07:39
    manfred-brands commented #3677
  • Nov 27 07:35
    manfred-brands commented #3677
  • Nov 27 07:15
    Build #6920 passed
  • Nov 27 07:10
    manfred-brands commented #3376
  • Nov 27 07:09
    manfred-brands synchronize #3677
  • Nov 27 07:03
    Dreamescaper commented #3677
  • Nov 27 06:58
    Build #6919 passed
  • Nov 27 06:58
    manfred-brands commented #3376
  • Nov 27 06:55
    manfred-brands commented #3376
Rohithnm
@Rohithnm
@ChrisMaddock Thanks for the reply, it would be helpful if you could share any reference for this retry logic
Oleksandr Liakhevych
@Dreamescaper
@Rohithnm There is no built in logic, you'd need to implement it yourself. The simplest example would be some loop with try-catch inside, e.g.
for(int i = 0; i < 3; i++)
{
      try 
      {
             // Setup...
             return;
      }
      catch
      {
             // Handle error
      }
}
Rohithnm
@Rohithnm
@Dreamescaper Thanks for the response, i thought is there any specific class in Nunit which needs to be overridded with custom logic. as we do for the test attributes
almazmr
@almazmr

Hi, i am using VSTest to run tests via Azure DevOps. Tests run successfully, but using the option "rerun failed tests: true", an error appears during the restart phase.

vstest.console.exe "C:\agent2.172.2\_work\r1\a\UITest\drop\Tests.Web\bin\Release\netcoreapp3.1\Tests.Web.dll"
/Settings:"C:\agent2.172.2\_work\_temp\3utv233tymm.runsettings"
/Logger:"trx"
/TestAdapterPath:"C:\agent2.172.2\_work\r1\a\UITest\drop\Tests.Web\bin\Release\netcoreapp3.1"
/TestCaseFilter:"FullyQualifiedName=Tests.Web.Tests._5.CourierModuleTest.N1_SendingTest.Id_5_1_01_TransferToCourierModule(Chrome)|FullyQualifiedName=Tests.Web.Tests._3.IssuanceOfDocuments.InformationOnTheApplicationIdentificationOfRecipient.N2_RecipientIdentificationTest.Id_3_2_13_RegisterAddressByFiasTest(Chrome)"
NUnit Adapter 3.16.1.0: Test execution started
An exception occurred while invoking executor 'executor://nunit3testexecutor/': Incorrect format for TestCaseFilter Missing Operator '|' or '&'. Specify the correct format and try again. Note that the incorrect format can lead to no test getting executed.

How can I fix this error and successfully restart the tests in DevOps?

Chris Maddock
@ChrisMaddock
@almazmr I'd suggest posting that as an issue on the nunit3-vs-adapter repo. The experts will be able to help there. :-)
@Rohithnm Yep - I agree with @Dreamescaper here - best to implement something in your own code to workaround this, as far as I can think.
Ace Olszowka
@aolszowka
@OsirisTerje @ChrisMaddock Back in July I mentioned a project to attempt to invoke NUnit3-Console shortly after the Test Project was built in MSBuild (after the Build target) as discussed above it did not seem like anything like that existed out of the box. Therefore I have created and pushed https://github.com/aolszowka/NUnit.MSBuild I wanted to solicit feedback from some of the thought leaders for NUnit to see if this is something that is already duplicated or something that is novel. I am using it internally with a lot of great results so far. This is MIT licensed and I am willing to respond to reasonable pull requests.
Mikkel Nylander Bundgaard
@mikkelbu
Can anyone point me in a direction of a project with a lot of NUnit3 tests and asserts, so we can test possible performance improvements to the NUnit analyzers. Currently, the "largest" I can think of is the NUnit project itself. See https://github.com/nunit/nunit.analyzers/pull/282#issuecomment-672288135 for some context.
Oleksandr Liakhevych
@Dreamescaper
Yep, I though about NUnit project itself as well :)
Julien
@LiohAu
Can anybody tell me if this binary : /Applications/Visual Studio.app/Contents/Resources/lib/monodevelop/AddIns/MonoDevelop.UnitTesting/NUnit3/NUnitRunner.exe is developped by the NUnit, Microsoft or MonoDevelop team ?
Chris Maddock
@ChrisMaddock
@LiohAu That's not an executable produced by the NUnit organization, as far as I'm aware.
@mikkelbu Try NodaTime perhaps? I think that's got a pretty sizable test library...and I wonder if @jskeet might even be interested in a contribution, if everything works well! :-)
@aolszowka Looks nice! I haven't personally used MSBuild properly in a good few years, but would have always relied on the MSBuildTasks tool before - it's good to have a replacement out there, if the former is no longer maintained!
Julien
@LiohAu
@OsirisTerje what happens behind the scenes when you select the .runsettings in the IDE ? where is the code that handles that ? in the NUnit VS adapter ? or in VS itself ?
Richard Adleta
@radleta

Running into a weird behavior trying to Debug Tests using VS 2019 16.7.2. I can get a breakpoint to stop but when I try to Continue or Step Into or Step Over it cancels the debugging session and outputs the below error. I can see a ThreadAbortException thrown as well. Any suggestions on how to fix or what I'm doing wrong?

Message: 
    Test cancelled by user
  Stack Trace: 
    RuntimeMethodHandle.InvokeMethod(Object target, Object[] arguments, Signature sig, Boolean constructor)
    RuntimeMethodInfo.UnsafeInvokeInternal(Object obj, Object[] parameters, Object[] arguments)
    RuntimeMethodInfo.Invoke(Object obj, BindingFlags invokeAttr, Binder binder, Object[] parameters, CultureInfo culture)
    Reflect.InvokeMethod(MethodInfo method, Object fixture, Object[] args)
    MethodWrapper.Invoke(Object fixture, Object[] args)
    TestMethodCommand.InvokeTestMethod(TestExecutionContext context)
    TestMethodCommand.RunTestMethod(TestExecutionContext context)
    TestMethodCommand.Execute(TestExecutionContext context)
    SimpleWorkItem.PerformWork()

Packages

    <PackageReference Include="NUnit" Version="3.12.0" />
    <PackageReference Include="NUnit.Console" Version="3.11.1" />
    <PackageReference Include="NUnit3TestAdapter" Version="3.17.0" />
Julien
@LiohAu
Is there an architecture diagram somewhere that would explain which NUnit omponent uses the other components ?
Julien
@LiohAu
At first I assumed that NUnit adapter was calling the NUnit console runner, but I don't see any reference to that .exe in the code, only references to vstest
so are NUnit Adapter and NUnit Console using vstest under the hood ?
Ace Olszowka
@aolszowka
@LiohAu I don't believe that there is a block diagram, NUnit Adapter is using the actual NUnit Engine under the hood, the NUnit Console also uses that same engine but has a CLI Front end to it.
Julien
@LiohAu
@aolszowka my concerns is with the runsettings file. I just followed the vstest code, and it seems that the runsettings is given to an MSBuild .target file, then it goes into the VsTest code, and it is then given to NUnit by the NUnit classes that respect the VSTest contrat.
the thing I am trying to understand is why runsettings is ignored when using VS Mac :(
Ace Olszowka
@aolszowka
VS Mac isn't the same as VS Proper I thought it was a reskinning of Android Studio or whatever that they purchased, it might be that they do not properly support the IRunSettings interface, are you setup to debug this?
Chris Maddock
@ChrisMaddock
@radleta Does your test have a timeout attirbute by any chance? This is fixed in master so the timeout doesn't apply when the debugger is attached, but it's not released yet
Julien
@LiohAu
@aolszowka behind VS mac, it is the monodevelop IDE and no i'm not setup to debug it ^^
But I assume everything is calling vstest.console.exe in the end. So if I can submit an issue or a PR on the NUnit MonoDevelop AddIn, to pass the NUnit file which is defined in the .csproj, that could be enough
Richard Adleta
@radleta
@ChrisMaddock Unfortunately, no it does not have a timeout attribute.
Terje Sandstrom
@OsirisTerje
@LiohAu The runsettings is sent down from whoever calls the adapter. I have sent a question to the MS product group about this, referring to the issue raised here nunit/nunit3-vs-adapter#778 . Please add whatever other comments you may have to that issue, MS will read it :-)
Julien
@LiohAu
@OsirisTerje It seems that people using .NET core projects in Windows based VS or in VS Mac/MonoDevelop are all depending on VsTest. But if you are not using a .NET core based project and that NUnit is in your project, the NUnit addin in VSMac/MonoDevelop, seems to fallback to an internal runner, and I did not find yet how to provide the settings...
Terje Sandstrom
@OsirisTerje
@LiohAu Just for others to see, MS has confirmed this doesn't yet work on VS Mac, see https://github.com/nunit/nunit3-vs-adapter/issues/778#issuecomment-679292768. They suggest a command line work around until the issue is resolved
Stephan
@zpqrtbnk
Quick Q before filing an issue... running tests with dotnet test ... and nunit3 adapter 3.16.1 does produce a complete Xml file, upgrading to adapter 3.17.0 produces an Xml file that only contains the <command-line> element - rings a bell / known issue?
Terje Sandstrom
@OsirisTerje
@zpqrtbnk No. Please raise an issue with repro :-)
Stephan
@zpqrtbnk
Issue nunit/nunit3-vs-adapter#780 - reproduces on my project, going to try to reproduce on a smaller-scale project that I can share
thanks
Marius Morar
@marius02
Hello, Is there a way to load a dll that is targeting .net framework 4.7.2 into an asp.net core web application? I have the following error when exploring the nunit tests( Tests = new TestNode(TestEngineRunner.Explore(TestFilter.Empty));):
            LoadPackage(testFile);

            TestEngine = TestEngineActivator.CreateInstance();

            TestEngineRunner = TestEngine.GetRunner(_package);

            Tests = new TestNode(TestEngineRunner.Explore(TestFilter.Empty));
The error:
NUnit.Engine.NUnitEngineException: 'An exception occurred in the driver while loading tests.'
FileNotFoundException: Could not load file or assembly 'TestingFramework, Version=9.2.2.0, Culture=neutral, PublicKeyToken=null'. The system cannot find the file specified.
Chris Maddock
@ChrisMaddock
Hi @marius02 - not yet. See nunit/nunit-console#465 for tracking
Alberto Bar-Noy
@AlbertoBN
Hello, I have a weird problem with NUnit3-console. I am trying to run my tests via command line and have NUnit.Allure for reports in my code as well. When I call the command line the console runner is stuck and neither runs tests not produces any output even when --trace=Verbose is used... Any insights? (same question goes to allure gitter. I am at a total loss here)
The project is a full .Net framework one...
Chris Maddock
@ChrisMaddock
@AlbertoBN try running with --labels=All, see if you get stuck on a particular test?
Alberto Bar-Noy
@AlbertoBN
@ChrisMaddock No joy. Stuck on the get go... waiting 10 minutes to see a label ... :(
Alberto Bar-Noy
@AlbertoBN
OK so... I went a little bit crazy and run the project with vstest.console... and I noticed that allure is dependent on Newtonsoft.Json 12.0.0 But this dependency was not mentioned in the package. I added it and run with NUnit again and TADAAAAAAAAAA (the windows 95 one)
Chris Maddock
@ChrisMaddock
@AlbertoBN How strange - I'd have expected that to crash rather than hang. Glad you're sorted, either way! :-)
Marius Morar
@marius02
@ChrisMaddock thanks for the reply , you have any suggestions to workaround this issue(nunit/nunit-console#465)? maybe to create a wcf service and to consume it via asp.net core web application, or an web api/rest app ?
Chris Maddock
@ChrisMaddock
Hmm...what's your usecase, why do you need to run an netfx test assembly from a .NET Core test runner? Or is it that you have a single test runner and want to be able to run both .NET Core and netfx test assemblies from it?
Chris Maddock
@ChrisMaddock
Here's a thought, have you considered using NUnitLite for you .NETFX test suites? That was your test runner could just start them up in a separate process and retrieve the results? It would mean you wouldn't really be using the engine for the netfx stuff of course.
Marius Morar
@marius02
the use case: I want to parse the test dll(NETFX ) and display the tests in a asp net core web app(kind of a NUnit Gui for the web, but customized), I'm not able to alter the test dll for now, but I think I will change my approach
Daniel Abrahamsson
@danabr
I have an issue with the NUnit test agent crashing ("Remote test agent exited with non-zero exit code"). What can I do to start debugging it? My tests run fine when I run them through VS.
Daniel Abrahamsson
@danabr
It is interesting to note that I don't get any crash if I use process=Single.
Ah. I found an issue in one of my test assemblies. A StackOverflowException, so naturally the process crashes.