Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Repo info
Activity
  • Dec 08 15:53
    AndreyAkinshin labeled #1324
  • Dec 08 15:53
    AndreyAkinshin commented #1324
  • Dec 08 14:51
    valentinoNguyen opened #1324
  • Dec 07 08:41
    AndreyAkinshin commented #1315
  • Dec 07 07:17
    sywhang commented #1315
  • Dec 06 21:16
    AndreyAkinshin labeled #1316
  • Dec 06 21:16
    AndreyAkinshin unlabeled #1316
  • Dec 06 21:16
    AndreyAkinshin assigned #1316
  • Dec 06 21:15
    AndreyAkinshin labeled #1315
  • Dec 06 15:35
    AndreyAkinshin commented #1323
  • Dec 06 15:32
    AndreyAkinshin milestoned #1323
  • Dec 06 15:32
    AndreyAkinshin labeled #1323
  • Dec 06 15:32
    AndreyAkinshin labeled #1323
  • Dec 06 15:32
    AndreyAkinshin closed #1323
  • Dec 06 15:32

    AndreyAkinshin on master

    Support empty lines in disassem… (compare)

  • Dec 06 15:32
    AndreyAkinshin assigned #1323
  • Dec 06 15:09
    AndreyAkinshin commented #1323
  • Dec 06 13:51
    Vermistuk commented #1323
  • Dec 06 13:45
    Vermistuk commented #1323
  • Dec 06 13:43
    Vermistuk commented #1323
bitbonk
@bitbonk
Looks like the answer is yes!
TeBeCo
@tebeco
async Task or async void
:D
bitbonk
@bitbonk
@tebeco I tested async Task
async void is probably not such a good idea
TeBeCo
@tebeco
yes sorry was just trolling around, i should stop doing that
Adam Sitnik
@adamsitnik
@bitbonk as of today it's impossible to mark one of the parameters as baseline dotnet/BenchmarkDotNet#881
async setup, cleanup and benchmarks are supported
async void is not supported by design and throws an exception ;)
bitbonk
@bitbonk
The docs about the EtwProfiler seem to be misleading: https://benchmarkdotnet.org/articles/features/etwprofiler.html

If do it as suggested:

private static void Main(string[] args)
     => BenchmarkSwitcher
         .FromAssembly(typeof(Program).Assembly)
         .Run(
            args,
            DefaultConfig.Instance
                .With(new EtwProfiler()));

I get the message

You should select the target benchmark(s). Please, print a number of a benchmark (e.g. '0') or a contained benchmark caption (e.g. 'SerializationBenchmarks')

bitbonk
@bitbonk

It looks like you don't have to

Extend the DefaultConfig.Instance with new instance of EtwProfiler

because that's what the [EtwProfiler] Attribute already did

eschneider999
@eschneider999
getting this error: Most probably the name of output exe is different than the name of the .(c/f)sproj
project name a output is the same name though
Adam Sitnik
@adamsitnik
@bitbonk this is expected. BenchmarkSwitcher prints the list of benchmarks and allows you to choose, by design. You can pass --filter $glob to filter the benchmarks and then it's not going to ask you
the alternative is to use BenchmarkRunner, which does not ask anything but simply runs benchmarks from given type/assembly
@eschneider999 are you customizing the output path? basically BDN searches the folder tree until it finds an .sln file and then it's looking in given folder recursively for a project file of the .dll that defines the benchmarks
Do you have a solution file
Richard Tasker
@ritasker
Afternoon All,
I am adding benchmarks to the CommunityCarter/Carter repo. We want to see what perf differences we get from migrating to AspNetCore 3.0 are.
The issue is I am getting a timeout on one of the tests after 5mins. I would love a 2nd pair of eyes to see what I can do differently.
My draft PR is here https://github.com/CarterCommunity/Carter/pull/210/files
Specifically the Carter.Benchmarks.Harness.UseCarterBenchmark are failing.
Richard Tasker
@ritasker
My Appvoyer build timed out last night after an hour. This was the output from the benchmark. Are 83 iterations ok?
Johan Larsson
@JohanLarsson
You can use gist for those
Thanks @JohanLarsson
Richard Tasker
@ritasker
@adamsitnik Any advice would be appreciated.
Adam Sitnik
@adamsitnik
Johan Larsson
@JohanLarsson
Is there something for getting the file name of for example the github.md report givne a Summary?
I copy benchmark results next to the benchmark code like this: https://github.com/DotNetAnalyzers/WpfAnalyzers/tree/master/WpfAnalyzers.Benchmarks/Benchmarks
And there has been some churn in the copy code due to files changing names
Not a huge issue, more nice to have.
Naram Qashat
@CyberBotX
Hi there. So, I wasn't sure if this was something in the docs that I missed or if it isn't in the docs at all, but is it possible to pre-build the benchmark EXEs to run them on another machine that doesn't have a development environment on it?
Adam Sitnik
@adamsitnik
@CyberBotX You can use InProcessToolchain https://benchmarkdotnet.org/articles/samples/IntroInProcess.html
Naram Qashat
@CyberBotX
Ah, I guess I did miss that when I looked at that earlier.
Naram Qashat
@CyberBotX
One thing I just ran into with using InProcess, though, on one of the computers I'm running the benchmarks on, it takes an obscenely long time, so much so that it causes it to throw an exception about execution time. Any way around that without not using InProcess?
Naram Qashat
@CyberBotX
I dug around a bit and found my answer, instead of using the attribute, I needed to add to the default job in the config, used .With(Job.Default.With(new InProcessEmitToolchain(TimeSpan.FromDays(10), true))). (Yes, 10 days is overkill, but it works for my purposes.)
Johannes Rössel [yWorks]
@jroessel

Good morning. I'm currently trying to use BenchmarkDotNet for performance regression analysis of a library we're authoring. There are basically two things I hope to achieve here:

  • Some tests are intended to demonstrate linear scaling of certain API usages (i.e. test doing something with one item or a hundred and not get a 10000× slowdown). (scaling tests)
  • Other tests are intended to compare API performance compared with an older version of the library to see whether we've done something very wrong with a change during development. (regression tests)

I've got something working-ish, right now, but encountered a few things:

  • I currently create jobs explicitly since testing against different library versions requires custom MSBuild parameters. Creating a config and jobs explicitly seems to be at odds with using attributes to do the same (it appears the attributes get ignored in this case?).
  • I do actually need two (maybe even three) different configurations and two subsets of tests (regression tests where the baseline is a number of benchmarks in a completely different job, and scaling tests where the baseline is different for each benchmark). Is it a good idea to put them all into the same project? Should each individual benchmark be its own project instead of its own class? I could probably separate benchmark classes by category well enough, I'm just wondering whether it's a good idea.
  • Basically, I'm trying to put this into a CI job that runs daily-ish and use BDN mainly to not worry about statistics. I'd also want to reduce the overhead in creating new tests so that new ones can be added as needed. I do realize that this is probably far from pin-point performance optimization and micro-benchmarks, so perhaps I'm also trying to use the wrong tool here.
Adam Sitnik
@adamsitnik
Hello @jroessel ! You can group the benchmarks into categories and then use categories filter
and have a dedicated config with per category
sth like
[BenchmarkCategory("SomeCategoryName")]
public class SomeBenchmarks

[BenchmarkCategory("OtherCategoryName")]
public class OtherBenchmarks

IConfig commonSettings = DefaultConfig.Instance; // can be extended using .With

IConfig first = commonSettings
    .With(Job.Defeault.WithBuildConfiguration("SOME"))
    .With(new AllCategoriesFilter("SomeCategoryName"));

IConfig second = commonSettings
    .With(new AllCategoriesFilter("OtherCategoryName"));

BenchmarkRunner.Run(typeof(Program).Assembly, first);
BenchmarkRunner.Run(typeof(Program).Assembly, second);
Johannes Rössel [yWorks]
@jroessel
@adamsitnik Thanks, I think I've got something useful now.
Johannes Rössel [yWorks]
@jroessel
There seems to be a difference in how projects get built at runtime by BDN and the temporary projects (along with the batch file) that are created. Right now if I'm mistakenly specify a dependency version that doesn't exist in the file system (which should fail the build and test), building and benchmarking works from BDN, but running the batch file individually doesn't work (as expected).
If that can't be solved, I thought I could look at the desired dependency version and the actual assembly version somehow, but couldn't figure out how (or whether) I can access the current job configuration from within a benchmark. I'd find it likely that it isn't supported, though. This would be my pre-benchmark sanity checks, though, which check whether the benchmarks have a hope of yielding useful results.
Fabian Yzerman
@yzrmn
Hi guys, I'm a little confused on how to use custom columns with data from a benchmark. I guess it has something to do with the BenchmarkCase parameter in IColumn.GetValue(), but how do I access e.g. a property of the benchmark which was set during a run?
Clinton Ingram
@saucecontrol
Is the property you're looking for not present on the BenchmarkCase ?
Fabian Yzerman
@yzrmn
The property is not from BDN, it's a custom one
I fear that the thing I want to do is not possible at the moment
However, I found a (theoretical) workaround by writing some data at Cleanup to a file and then read it again in the column code
Clinton Ingram
@saucecontrol
ah, gotcha