Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Repo info
Activity
  • 11:52
    manfred-brands commented #3376
  • 02:50
    manfred-brands commented #3376
  • 00:04
    ChrisMaddock closed #833
  • 00:04
    ChrisMaddock commented #833
  • 00:03
    ChrisMaddock labeled #839
  • 00:03
    ChrisMaddock labeled #839
  • 00:03
    ChrisMaddock labeled #839
  • 00:03
    ChrisMaddock opened #839
  • Nov 28 23:58
    ChrisMaddock labeled #838
  • Nov 28 23:58
    ChrisMaddock labeled #838
  • Nov 28 23:58
    ChrisMaddock labeled #838
  • Nov 28 23:58
    ChrisMaddock opened #838
  • Nov 28 23:54
    ChrisMaddock milestoned #837
  • Nov 28 23:54
    ChrisMaddock labeled #837
  • Nov 28 23:54
    ChrisMaddock labeled #837
  • Nov 28 23:54
    ChrisMaddock assigned #837
  • Nov 28 23:54
    ChrisMaddock opened #837
  • Nov 28 23:53
    ChrisMaddock commented #824
  • Nov 28 23:51
    Build #4960 passed
  • Nov 28 23:47
    ChrisMaddock labeled #829
Terje Sandstrom
@OsirisTerje
@aolszowka MSBuild - many years since I used that for anything advanced... You have all the moving parts here, and you know your own situation, so I would just go and create your own nuget package to wrap this up. About stopping at first error, there is a StopOnError option which does that.
Chris Maddock
@ChrisMaddock
@aolszowka For MSBuild, I think https://github.com/loresoft/msbuildtasks is maybe what you're looking for? Be warned it's unmaintained.
Ace Olszowka
@aolszowka
@OsirisTerje Sounds fair thank you for the follow up, I was looking for more a confirmation that nothing like that already existed out of the box. Thank you!
@ChrisMaddock Yup I've actually contributed to that back in the day, thanks for the link, sad to hear that he is no longer maintaining it :(
Joseph Musser
@jnm2

@mikkelbu The other tests in the file have a level of indentation like this:

            var testCode = TestUtility.WrapClassInNamespaceAndAddUsing(@"
    class AnalyzeArgumentHasImplicitConversion
    {
        [TestCase(uint.MaxValue)]
        public void Test(long e) { }
    }");

If I add a using statement, how would you want that indented?

Rohithnm
@Rohithnm
Hi All, is there any way were we can re-execute OneTimeSetup on failure
Mikkel Nylander Bundgaard
@mikkelbu
@jnm2 I think it is fine to indent the using directive to the same level as the class. The reason for the indentation was so that the resulting code looked nicely formatted, but I'm not sure we have been that consistent with the indentation.
Chris Maddock
@ChrisMaddock
@Rohithnm - the short answer: "no"! :-) I'd recommend implementing any retry logic at that level yourself.
Rohithnm
@Rohithnm
@ChrisMaddock Thanks for the reply, it would be helpful if you could share any reference for this retry logic
Oleksandr Liakhevych
@Dreamescaper
@Rohithnm There is no built in logic, you'd need to implement it yourself. The simplest example would be some loop with try-catch inside, e.g.
for(int i = 0; i < 3; i++)
{
      try 
      {
             // Setup...
             return;
      }
      catch
      {
             // Handle error
      }
}
Rohithnm
@Rohithnm
@Dreamescaper Thanks for the response, i thought is there any specific class in Nunit which needs to be overridded with custom logic. as we do for the test attributes
almazmr
@almazmr

Hi, i am using VSTest to run tests via Azure DevOps. Tests run successfully, but using the option "rerun failed tests: true", an error appears during the restart phase.

vstest.console.exe "C:\agent2.172.2\_work\r1\a\UITest\drop\Tests.Web\bin\Release\netcoreapp3.1\Tests.Web.dll"
/Settings:"C:\agent2.172.2\_work\_temp\3utv233tymm.runsettings"
/Logger:"trx"
/TestAdapterPath:"C:\agent2.172.2\_work\r1\a\UITest\drop\Tests.Web\bin\Release\netcoreapp3.1"
/TestCaseFilter:"FullyQualifiedName=Tests.Web.Tests._5.CourierModuleTest.N1_SendingTest.Id_5_1_01_TransferToCourierModule(Chrome)|FullyQualifiedName=Tests.Web.Tests._3.IssuanceOfDocuments.InformationOnTheApplicationIdentificationOfRecipient.N2_RecipientIdentificationTest.Id_3_2_13_RegisterAddressByFiasTest(Chrome)"
NUnit Adapter 3.16.1.0: Test execution started
An exception occurred while invoking executor 'executor://nunit3testexecutor/': Incorrect format for TestCaseFilter Missing Operator '|' or '&'. Specify the correct format and try again. Note that the incorrect format can lead to no test getting executed.

How can I fix this error and successfully restart the tests in DevOps?

Chris Maddock
@ChrisMaddock
@almazmr I'd suggest posting that as an issue on the nunit3-vs-adapter repo. The experts will be able to help there. :-)
@Rohithnm Yep - I agree with @Dreamescaper here - best to implement something in your own code to workaround this, as far as I can think.
Ace Olszowka
@aolszowka
@OsirisTerje @ChrisMaddock Back in July I mentioned a project to attempt to invoke NUnit3-Console shortly after the Test Project was built in MSBuild (after the Build target) as discussed above it did not seem like anything like that existed out of the box. Therefore I have created and pushed https://github.com/aolszowka/NUnit.MSBuild I wanted to solicit feedback from some of the thought leaders for NUnit to see if this is something that is already duplicated or something that is novel. I am using it internally with a lot of great results so far. This is MIT licensed and I am willing to respond to reasonable pull requests.
Mikkel Nylander Bundgaard
@mikkelbu
Can anyone point me in a direction of a project with a lot of NUnit3 tests and asserts, so we can test possible performance improvements to the NUnit analyzers. Currently, the "largest" I can think of is the NUnit project itself. See https://github.com/nunit/nunit.analyzers/pull/282#issuecomment-672288135 for some context.
Oleksandr Liakhevych
@Dreamescaper
Yep, I though about NUnit project itself as well :)
Julien
@LiohAu
Can anybody tell me if this binary : /Applications/Visual Studio.app/Contents/Resources/lib/monodevelop/AddIns/MonoDevelop.UnitTesting/NUnit3/NUnitRunner.exe is developped by the NUnit, Microsoft or MonoDevelop team ?
Chris Maddock
@ChrisMaddock
@LiohAu That's not an executable produced by the NUnit organization, as far as I'm aware.
@mikkelbu Try NodaTime perhaps? I think that's got a pretty sizable test library...and I wonder if @jskeet might even be interested in a contribution, if everything works well! :-)
@aolszowka Looks nice! I haven't personally used MSBuild properly in a good few years, but would have always relied on the MSBuildTasks tool before - it's good to have a replacement out there, if the former is no longer maintained!
Julien
@LiohAu
@OsirisTerje what happens behind the scenes when you select the .runsettings in the IDE ? where is the code that handles that ? in the NUnit VS adapter ? or in VS itself ?
Richard Adleta
@radleta

Running into a weird behavior trying to Debug Tests using VS 2019 16.7.2. I can get a breakpoint to stop but when I try to Continue or Step Into or Step Over it cancels the debugging session and outputs the below error. I can see a ThreadAbortException thrown as well. Any suggestions on how to fix or what I'm doing wrong?

Message: 
    Test cancelled by user
  Stack Trace: 
    RuntimeMethodHandle.InvokeMethod(Object target, Object[] arguments, Signature sig, Boolean constructor)
    RuntimeMethodInfo.UnsafeInvokeInternal(Object obj, Object[] parameters, Object[] arguments)
    RuntimeMethodInfo.Invoke(Object obj, BindingFlags invokeAttr, Binder binder, Object[] parameters, CultureInfo culture)
    Reflect.InvokeMethod(MethodInfo method, Object fixture, Object[] args)
    MethodWrapper.Invoke(Object fixture, Object[] args)
    TestMethodCommand.InvokeTestMethod(TestExecutionContext context)
    TestMethodCommand.RunTestMethod(TestExecutionContext context)
    TestMethodCommand.Execute(TestExecutionContext context)
    SimpleWorkItem.PerformWork()

Packages

    <PackageReference Include="NUnit" Version="3.12.0" />
    <PackageReference Include="NUnit.Console" Version="3.11.1" />
    <PackageReference Include="NUnit3TestAdapter" Version="3.17.0" />
Julien
@LiohAu
Is there an architecture diagram somewhere that would explain which NUnit omponent uses the other components ?
Julien
@LiohAu
At first I assumed that NUnit adapter was calling the NUnit console runner, but I don't see any reference to that .exe in the code, only references to vstest
so are NUnit Adapter and NUnit Console using vstest under the hood ?
Ace Olszowka
@aolszowka
@LiohAu I don't believe that there is a block diagram, NUnit Adapter is using the actual NUnit Engine under the hood, the NUnit Console also uses that same engine but has a CLI Front end to it.
Julien
@LiohAu
@aolszowka my concerns is with the runsettings file. I just followed the vstest code, and it seems that the runsettings is given to an MSBuild .target file, then it goes into the VsTest code, and it is then given to NUnit by the NUnit classes that respect the VSTest contrat.
the thing I am trying to understand is why runsettings is ignored when using VS Mac :(
Ace Olszowka
@aolszowka
VS Mac isn't the same as VS Proper I thought it was a reskinning of Android Studio or whatever that they purchased, it might be that they do not properly support the IRunSettings interface, are you setup to debug this?
Chris Maddock
@ChrisMaddock
@radleta Does your test have a timeout attirbute by any chance? This is fixed in master so the timeout doesn't apply when the debugger is attached, but it's not released yet
Julien
@LiohAu
@aolszowka behind VS mac, it is the monodevelop IDE and no i'm not setup to debug it ^^
But I assume everything is calling vstest.console.exe in the end. So if I can submit an issue or a PR on the NUnit MonoDevelop AddIn, to pass the NUnit file which is defined in the .csproj, that could be enough
Richard Adleta
@radleta
@ChrisMaddock Unfortunately, no it does not have a timeout attribute.
Terje Sandstrom
@OsirisTerje
@LiohAu The runsettings is sent down from whoever calls the adapter. I have sent a question to the MS product group about this, referring to the issue raised here nunit/nunit3-vs-adapter#778 . Please add whatever other comments you may have to that issue, MS will read it :-)
Julien
@LiohAu
@OsirisTerje It seems that people using .NET core projects in Windows based VS or in VS Mac/MonoDevelop are all depending on VsTest. But if you are not using a .NET core based project and that NUnit is in your project, the NUnit addin in VSMac/MonoDevelop, seems to fallback to an internal runner, and I did not find yet how to provide the settings...
Terje Sandstrom
@OsirisTerje
@LiohAu Just for others to see, MS has confirmed this doesn't yet work on VS Mac, see https://github.com/nunit/nunit3-vs-adapter/issues/778#issuecomment-679292768. They suggest a command line work around until the issue is resolved
Stephan
@zpqrtbnk
Quick Q before filing an issue... running tests with dotnet test ... and nunit3 adapter 3.16.1 does produce a complete Xml file, upgrading to adapter 3.17.0 produces an Xml file that only contains the <command-line> element - rings a bell / known issue?
Terje Sandstrom
@OsirisTerje
@zpqrtbnk No. Please raise an issue with repro :-)
Stephan
@zpqrtbnk
Issue nunit/nunit3-vs-adapter#780 - reproduces on my project, going to try to reproduce on a smaller-scale project that I can share
thanks
Marius Morar
@marius02
Hello, Is there a way to load a dll that is targeting .net framework 4.7.2 into an asp.net core web application? I have the following error when exploring the nunit tests( Tests = new TestNode(TestEngineRunner.Explore(TestFilter.Empty));):
            LoadPackage(testFile);

            TestEngine = TestEngineActivator.CreateInstance();

            TestEngineRunner = TestEngine.GetRunner(_package);

            Tests = new TestNode(TestEngineRunner.Explore(TestFilter.Empty));
The error:
NUnit.Engine.NUnitEngineException: 'An exception occurred in the driver while loading tests.'
FileNotFoundException: Could not load file or assembly 'TestingFramework, Version=9.2.2.0, Culture=neutral, PublicKeyToken=null'. The system cannot find the file specified.
Chris Maddock
@ChrisMaddock
Hi @marius02 - not yet. See nunit/nunit-console#465 for tracking
Alberto Bar-Noy
@AlbertoBN
Hello, I have a weird problem with NUnit3-console. I am trying to run my tests via command line and have NUnit.Allure for reports in my code as well. When I call the command line the console runner is stuck and neither runs tests not produces any output even when --trace=Verbose is used... Any insights? (same question goes to allure gitter. I am at a total loss here)
The project is a full .Net framework one...
Chris Maddock
@ChrisMaddock
@AlbertoBN try running with --labels=All, see if you get stuck on a particular test?
Alberto Bar-Noy
@AlbertoBN
@ChrisMaddock No joy. Stuck on the get go... waiting 10 minutes to see a label ... :(
Alberto Bar-Noy
@AlbertoBN
OK so... I went a little bit crazy and run the project with vstest.console... and I noticed that allure is dependent on Newtonsoft.Json 12.0.0 But this dependency was not mentioned in the package. I added it and run with NUnit again and TADAAAAAAAAAA (the windows 95 one)