Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Repo info
Activity
  • Nov 10 2016 13:26
    martinwoodward unassigned #296
  • Nov 10 2016 13:26
    martinwoodward unassigned #295
  • Nov 10 2016 13:26
    martinwoodward unassigned #293
  • Nov 10 2016 13:26
    martinwoodward unassigned #294
  • Nov 10 2016 13:26
    martinwoodward unassigned #292
  • Nov 10 2016 13:26
    martinwoodward unassigned #291
  • Nov 10 2016 13:26
    martinwoodward unassigned #288
  • Nov 10 2016 13:26
    martinwoodward unassigned #286
  • Nov 10 2016 13:26
    martinwoodward unassigned #287
  • Nov 10 2016 13:26
    martinwoodward unassigned #285
  • Nov 10 2016 13:26
    martinwoodward unassigned #281
  • Nov 10 2016 13:26
    martinwoodward unassigned #280
  • Nov 10 2016 13:26
    martinwoodward unassigned #278
  • Nov 10 2016 13:26
    martinwoodward unassigned #273
  • Nov 10 2016 13:26
    martinwoodward unassigned #271
  • Nov 10 2016 13:26
    martinwoodward unassigned #276
  • Nov 10 2016 13:26
    martinwoodward unassigned #272
  • Nov 10 2016 13:26
    martinwoodward unassigned #275
  • Nov 10 2016 13:26
    martinwoodward unassigned #268
  • Nov 10 2016 13:26
    martinwoodward unassigned #265
Andrey Akinshin
@AndreyAkinshin
warnings.png
Also I have implemented new kind of plugins: analysers, see PerfDotNet/BenchmarkDotNet@527df65
@smudge202 @mattwarren what do you think?
Matt Warren
@mattwarren

Guys, now we have big amount of upcoming features, idea and plans, discussions. What do you think about a wiki page with a roadmap?

Yeah, we need to put this somewhere, even if it's only a GitHub discussion. For instnace I've got several ideas for new Diagnostics that can go on the list

Matt Warren
@mattwarren

Also I have implemented new kind of plugins: analysers, see PerfDotNet/BenchmarkDotNet@527df65

I really like the new plugin system, especially the analysers, they're a really nice idea

Andrey Akinshin
@AndreyAkinshin
@mattwarren, thanks!
I created a wiki page: https://github.com/PerfDotNet/BenchmarkDotNet/wiki Feel free to make any changes.
Tommy Long
@smudge202
Good discussions in the repo today.
Nikita
@Metallium

what is the way to configure benchmark runner to make it less precise but run much faster?

it happens you often need to make raw A/B testing on some things

Andrey Akinshin
@AndreyAkinshin
@Metallium, right now you can reduce amount of processes (the first parameter of BenchmarkTask) , amount of warmup and target iterations (the two last parameters). E.g., processCount=1, warmupIterationCount=3, targetIterationCount=5. I am trying to implement smart logic that will be adjust these parameters based on you benchmark: if you really don't need super precision, benchmarking will perform quickly. Hopefully, new logic will be published in December.
Tommy Long
@smudge202
^ This would be great @AndreyAkinshin .
Btw, I took a look at the public API last night. I actually only found that one usage of Exec, everything else looked fine.
Andrey Akinshin
@AndreyAkinshin
This message was deleted
@smudge202, cool. Waiting for your PR.
Nikita
@Metallium
it would be great to flexible setup
already tried to lower iteration counts - benchmark still takes a lot)
Matt Warren
@mattwarren
You can also do this:
[BenchmarkTask(mode: BenchmarkMode.SingleRun, processCount: 1, warmupIterationCount: 1, targetIterationCount: 1)] i.e. "SingleRun"
Tommy Long
@smudge202
I haven't checked, but is it possible to inherit from BenchmarkTaskAttribute and assign default properties through the base ctor?
For example, allowing @Metallium to create a QuickBenchmarkTaskAttribute with settings as you've described.
Matt Warren
@mattwarren
good question, I'm not sure either?
Tommy Long
@smudge202
Pretty sure it isn't sealed. Will have to check the discoverer
Matt Warren
@mattwarren
well this compiles for me:
public class QuickBenchmarkTaskAttribute : BenchmarkTaskAttribute
    {
        public QuickBenchmarkTaskAttribute() 
            : base(mode: BenchmarkMode.SingleRun, processCount: 1, warmupIterationCount: 1, targetIterationCount: 1)
        { }
    }
Tommy Long
@smudge202
I don't think it will be discovered
Because the GetCustomAttributes call linked above states false for allow inheritance
Matt Warren
@mattwarren
and
    [QuickBenchmarkTask]
    public class PerformanceUnitTest
Tommy Long
@smudge202
https://msdn.microsoft.com/en-us/library/ms130871(v=vs.110).aspx <= we need to set inherit: true to discover derived attributes I believe
Matt Warren
@mattwarren
that's a good point about GetCustomAttributes, I haven't actually run the test yet¬
Tommy Long
@smudge202
I can't see a reason not to set it to true to be honest
I believe the code that's there will still work
Installing VS2015 U1 so can't check in VS, and too lazy to update VSCode... sorry. :)
Tommy Long
@smudge202
I'm going to create an associated bug. Either BenchmarkAttribute should be sealed or the GetCustomAttributes call should allow inheritance. The combination we have right now I consider to be a bug. Obviously, changing the call instead of sealing the attribute would be a better fix.
Matt Warren
@mattwarren
sounds good
Tommy Long
@smudge202
PerfDotNet/BenchmarkDotNet#57
Matt Warren
@mattwarren

wierdly enough

[QuickBenchmarkTask]
public class PerformanceUnitTest()

works for me, but I'll look into it a bit more when I get round to the issue (there's a bit of a backlog building up :-))

Tommy Long
@smudge202
Oh... was I wrong then? It does discover inherited attributes?
Matt Warren
@mattwarren
not really sure at the moment, maybe, maybe not!
Tommy Long
@smudge202
lol
Andrey Akinshin
@AndreyAkinshin
@smudge202, @mattwarren, do you have any comment about new plugin system and API? I want to make v0.8.0 on the weekend and publish it on NuGet. @smudge202, I am waiting four your PR with executables.
Andrey Akinshin
@AndreyAkinshin
Guys? Any feedback?
Matt Warren
@mattwarren
@AndreyAkinshin sorry for the delay, finally got time to look thru the plugin system, looks good to me
release it!
Andrey Akinshin
@AndreyAkinshin
@mattwarren, @smudge202, ok, v0.8.0 is in the NuGet. Now I want to keep the master branch in conformity with the latest NuGet package. Let's use the develop branch for developing.
Andrey Akinshin
@AndreyAkinshin
Guys, check it out: new benchmark framework https://petabridge.com/blog/introduction-to-nbench/ A funny thing: NBench uses BenchmarkDotNet for own infrastructure benchmarking: https://github.com/petabridge/NBench/blob/dev/benchmarks/NBench.Microbenchmarks/Program.cs
Matt Warren
@mattwarren
Cool, I saw NBench show up in my twitter feed, but didn't have time to look at it. That's really funny that NBench actually uses BenchmarkDotNet to test itself! (although I do wonder why he wrote a new library and didn't just use BenchmarkDotNet itself, maybe we weren't extensible enough? It wouldn't be that hard to add the NBench functionality to BenchmarkDotNet, now that we have the new plugin system)
Matt Warren
@mattwarren
Just found this comment from the author of NBench, it pretty much answers my previous question
Basicially this:
PerfDotNet only does throughput and micro-benchmarking, which is super helpful unto itself and it excels at doing that. But I needed to be able to also make assertions against Memory consumption and GC, so it didn't cover the spectrum of needs there. Plus I wanted unit test-style pass/fail semantics, which it also does not do.
Andrey Akinshin
@AndreyAkinshin
I published v0.8.1 on NuGet.
@mattwarren, please, use the develop branch for your commits. I want to keep the master branch and the NuGet package relevant.
Matt Warren
@mattwarren
yeah sorry about that, I just read the notice on the wiki page. I'll use develop in the future
what do you think about the NBench stuff, is it worth chatting to the author to see if we can persuade him to use BenchmarkDotNet, or do you think it's too late for that?