dotnet/BenchmarkDotNet

Tebjan Halm

@tebjan

@tebeco i've created an issue: dotnet/BenchmarkDotNet#1445

Sebastiano Mandalà

@sebas77

Jumping in quickly to ask what config you would suggest me to use to have fast benchmark iterations without the result being meaningless. Running a benchmark on simple code takes minutes

Jason Bock

@JasonBock

I just tried to create a couple of benchmark tests, and....I got this error:

error MSB4086: A numeric comparison was attempted on "$(LangVersion)" that evaluates to "latest" instead of a number, in condition "'$(LangVersion)' == '' Or '$(LangVersion)' < '7.3'".

I've never seen that one before :). Any ideas why I'm getting this, and how to get rid of it?

Jason Bock

@JasonBock

Seems like if I have <LangVersion> set to latest, that's why I'm getting that error. Sounds like a bug to me. I'll submit something to the GitHub issues list.

TeBeCo

@tebeco

Bad idea of the day ....
Can BenchmarkDotnet do bisect so that we could detect threshold in the code,

[BisectParam]
public ValueTuple<int, int> RequestSize = (8, 4096)

[Benchmarck]
[BisectPivot(MemoryDiagnosterResult.Gen0)]
public void Foo()
{
  //Code will run with 
  // 0 => Alloc ?
  // 4096  => Alloc ?
   // (4096-0)/2 => Alloc ?
  // ....
}

i'm not sure if that would makes any sense to be fair

it would be a bit like a FlatMap/Reduce

Florian Verdonck

@nojaf

Hello

I created my first benchmark today and now I'm wondering what to do with it.
How many times do I want to run it? And what is the best strategy for saving the results after a CI build?
Because I guess you want to compare the results with previous runs, so what is the best way to do that.

Eugene Krapivin

@EugeneKrapivin

Hey guys!
is there a support planned for TeamCity? storing the files and comparing between builds and presenting the delta?
Thanks!

VittenaDev007

@VittenaDev007

I automatic create word document in server side
I use microsoft.introp.word not working on IIS
show this error
at System.RuntimeTypeHandle.CreateInstance(RuntimeType type, Boolean publicOnly, Boolean wrapExceptions, Boolean& canBeCached, RuntimeMethodHandleInternal& ctor, Boolean& hasNoDefaultCtor) at System.RuntimeType.CreateInstanceDefaultCtorSlow(Boolean publicOnly, Boolean wrapExceptions, Boolean fillCache) at System.RuntimeType.CreateInstanceDefaultCtor(Boolean publicOnly, Boolean skipCheckThis, Boolean fillCache, Boolean wrapExceptions) at System.Activator.CreateInstance(Type type, Boolean nonPublic, Boolean wrapExceptions)
please help me

DrGriff

@DrGriff

Hi everyone. I created a Visual Studio Console project for benchmarking, and ran this just fine (by doing Ctrl+F5 to run it without the debugger). I then tried using the command line (as Adminstrator) c:\.......\myapp.exe and I got the following error: Generate Exception: Access to the path 'C:\WINDOWS\system32\LogFiles\WMI\RtBackup' is denied.. What am I missing?

1 reply

Johannes Rössel [yWorks]

@jroessel

Good morning; I've got a bit of a problem with a ParamsSource that requires STA. It seems as if the parameters are queried at a point where the STAThread attribute of the benchmark (for understandable reasons, actually) is not yet "active". Right now I'm working around this by using the parameter as string and then using reflection in a GlobalSetup method to pick the correct instance, but is there a nicer way? Attributing the method that's called for ParamsSource with STAThread doesn't help either. In my case I'd be fine if everything ran with STA anyway, so some global option would also be okay.

gpetrou

@gpetrou

Is it possible to Setup some code only once per assembly, not per test method as GlobalSetup does?
Similarly, is it possible to initialize some code per class?
We have some third party libraries that can only be initialized/terminated once and we also generate some data that should be shared between tests.

Johannes Rössel [yWorks]

@jroessel

@gpetrou Wouldn't a static initializer of any type that's used in benchmarks do the trick?

gpetrou

@gpetrou

@jroessel I assume you are talking about the code per class case? I will check that. But assuming that this will work, how can Cleanup happen once per class or assembly?

Ivan Kochurkin

@KvanTTT

Hi! Is it possible to export to csv raw nanoseconds 18 instead of formatted form like 0.0184 ms?

Reason: I'm comparing several runtimes and need in normalized benchmark time.

Adam Sitnik

@adamsitnik

hi @KvanTTT SummaryStyle is what you are looking for

var summaryStyle = new SummaryStyle
{
    PrintUnitsInHeader = true,
    PrintUnitsInContent = false,
    SizeUnit = SizeUnit.B,
    TimeUnit = TimeUnit.Nanosecond
};

config = config.With(summaryStyle);

Ivan Kochurkin

@KvanTTT

Yes, I've already found it, but thank you! :thumbsup:

Jason Bock

@JasonBock

Is there any way to run a project with benchmark tests where LangVersion is set to preview? I want to test some stuff I'm writing for .NET 5.0 and I need to use C# 9 features, but when I have preview in play, I get:

error MSB4086: A numeric comparison was attempted on "$(LangVersion)" that evaluates to "preview" instead of a number, in condition "'$(LangVersion)' == '' Or '$(LangVersion)' < '7.3'".

Jason Bock

@JasonBock

Nevermind....I set it to 9.0 and apparently that worked :)

sanjaysoni5657

@sanjaysoni5657

Can we use dependency injection setup same as we do in dotnet core project for Benchmark Test?

trampster

@trampster

I've run into an issue trying to create benchmarks for a .NET 5 c# source generator. It appears that it doesn't accept the LangVersion being set to Preview, but this is currently required for c# source generators

dotnet/BenchmarkDotNet#1537

It doesn't work if set to 9.0 like what worked for @JasonBock

Clinton Ingram

@saucecontrol

What does it do if you set it to 9.0? That's the correct solution

trampster

@trampster

If I set it to 9.0 it doesn't compile because the Source Generator doesn't generate any source

Clinton Ingram

@saucecontrol

the roslyn build in the 5.0 RC1 SDK definitely recognizes langversion 9.0. are you compiling with an older VS version instead?

trampster

@trampster

No I'm on linux

I'm running 5.0.100-rc.1.20452.10

I'm using vscode and running from the commandline

trampster

@trampster

The Source Generator announcement says that it requires Preview: https://devblogs.microsoft.com/dotnet/introducing-c-source-generators/

Why do I need to use the Preview LangVersion to consume a Source Generator?
Although Source Generators are not technically a C# language feature, they are in preview. Rather than introduce a new setting just for Source Generators, we decided it would be easier to just use the existing switch that enables preview language features for the C# compiler.

Given that Preview features are things that could benefit most from benchmarking, it seems an odd choice to not support them.

trampster

@trampster

Roslyn doesn't have any problems recognizing langversion 9.0, its just that langversion 9.0 doesn't seem to include Source Generators yet.

Clinton Ingram

@saucecontrol

That's very strange... At the time that announcement was made, Roslyn didn't recognize 9.0, only preview. I believe SDK preview 7 was the first to recognize 9.0, so I would have expected all 9.0 roslyn features to fall under that flag at that point

trampster

@trampster

I've asked about it on the roslyn gitter, my guess is that Source Generators wasn't deemed ready in time for c# 9.0 so stayed in preview.

The .net 5 RC1 announcement didn't mention Source Generators, which I thought was strange at the time but could be that it has been delayed.

Clinton Ingram

@saucecontrol

ohhhhhh

you can probably set up an inproc benchmark then https://benchmarkdotnet.org/articles/samples/IntroInProcess.html

Martz

@Martz2804_twitter

hi all, I have a question regarding benchmarkdotnet setup, read the docs but I can't seem to find the solution for it. Maybe this isn't the right tool or I (plan to) use it wrong.

Say you have a library you want to perf test with FunctionA() which calls into a lot of code internal to the lib. You want to benchmark the perf of FunctionA() accross multiple versions of the library. Is there a way to automate this "version switching" with benchmarkdotnet? I can't seem to find it

Clinton Ingram

@saucecontrol

You can do it if the library is in a nuget package: https://benchmarkdotnet.org/articles/samples/IntroNuGet.html

TeBeCo

@tebeco

i think theres a sample about nuget and newtonsoft doing this exactly

oO faster

Clinton Ingram

@saucecontrol

lol

TeBeCo

@tebeco

Martz

@Martz2804_twitter

I don't know how I missed this... thank you folks, for the quick answer :) seems to be what I'm after!

TeBeCo

@tebeco

perfect ;)

Where communities thrive

People

Repo info

Activity