Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Repo info
Activity
  • Nov 10 2016 13:26
    martinwoodward unassigned #296
  • Nov 10 2016 13:26
    martinwoodward unassigned #295
  • Nov 10 2016 13:26
    martinwoodward unassigned #293
  • Nov 10 2016 13:26
    martinwoodward unassigned #294
  • Nov 10 2016 13:26
    martinwoodward unassigned #292
  • Nov 10 2016 13:26
    martinwoodward unassigned #291
  • Nov 10 2016 13:26
    martinwoodward unassigned #288
  • Nov 10 2016 13:26
    martinwoodward unassigned #286
  • Nov 10 2016 13:26
    martinwoodward unassigned #287
  • Nov 10 2016 13:26
    martinwoodward unassigned #285
  • Nov 10 2016 13:26
    martinwoodward unassigned #281
  • Nov 10 2016 13:26
    martinwoodward unassigned #280
  • Nov 10 2016 13:26
    martinwoodward unassigned #278
  • Nov 10 2016 13:26
    martinwoodward unassigned #273
  • Nov 10 2016 13:26
    martinwoodward unassigned #271
  • Nov 10 2016 13:26
    martinwoodward unassigned #276
  • Nov 10 2016 13:26
    martinwoodward unassigned #272
  • Nov 10 2016 13:26
    martinwoodward unassigned #275
  • Nov 10 2016 13:26
    martinwoodward unassigned #268
  • Nov 10 2016 13:26
    martinwoodward unassigned #265
Matt Warren
@mattwarren
fair enough BDN is definitely targeted at micro-benchmarks, but at the end of the day you can put whatever code you want inside a [Benchmark] method.
Tommy Long
@smudge202
i.e. How exactly I can push a new implementation of InterfaceX into the pipeline?
Matt Warren
@mattwarren

Yeh, I think I need to spend more time looking at what is/is not extensible. Have you got any examples of extensibility outside the core repo?

At the moment, unless I've forgotten about something, there's no extensibility outside of the core repo. But it's a good question, maybe raise an issue and we can see what @AndreyAkinshin reckons

Tommy Long
@smudge202
Or perhaps there's a sample in the repo I've missed that shows extensibility?
In the same way games are often more popular when they have good modding support, I find the same for libraries. The more extensible a package is, the less code required in the core repo and the more tinkering outsiders can do for their peculiar scenarios. :)
Matt Warren
@mattwarren
yeah it's definitely a good idea,
I have to log off now, glad you are liking BenchmarkDotNet!
Tommy Long
@smudge202
No prob. Thanks for the help.
Andrey Akinshin
@AndreyAkinshin
Extensibility is a really good idea. I want to implement a system of plugins such that our users can form a desired plugin set from the core, configure it, or write own plugins. My nearest problem is the result export logic, next I want to implement a warning system (messages like Hey, your StdDev is too big, you should pay attention to it) and toolchain system (custom Generator/Builder/Executor configurations). What do you think?
Tommy Long
@smudge202
Yes, yes, and more yes.
But how to go about it?
I toyed with creating an issue for extensibility, but I think I'll leave it to you fellas. :)
Andrey Akinshin
@AndreyAkinshin
Ok, I will implement some basic logic on the weekend.
Tommy Long
@smudge202
Let me know if I can do anything to help.
Andrey Akinshin
@AndreyAkinshin
Ok, thanks!
Andrey Akinshin
@AndreyAkinshin
Guys, now we have big amount of upcoming features, idea and plans, discussions. What do you think about a wiki page with a roadmap?
Andrey Akinshin
@AndreyAkinshin
@mattwarren, @smudge202, please, review a new plugin system: PerfDotNet/BenchmarkDotNet@7eb70a1
Tommy Long
@smudge202
Just scanning over that commit now, @AndreyAkinshin
Tommy Long
@smudge202
That commit doesn't paint the whole picture; I assume there was previous work/commits for a plugin mechanism.
It would be great to have a sample showing off this plugin stuff so we can see the API you intend and then follow that through to check out implementation.
Andrey Akinshin
@AndreyAkinshin
@smudge202, thank you for the review. Check out my last commits: I have updated our integration tests with help of new plugin system. In particular, I defined own logger that accumulate logs in StringBuilder instead of System.Console.
Andrey Akinshin
@AndreyAkinshin
warnings.png
Also I have implemented new kind of plugins: analysers, see PerfDotNet/BenchmarkDotNet@527df65
@smudge202 @mattwarren what do you think?
Matt Warren
@mattwarren

Guys, now we have big amount of upcoming features, idea and plans, discussions. What do you think about a wiki page with a roadmap?

Yeah, we need to put this somewhere, even if it's only a GitHub discussion. For instnace I've got several ideas for new Diagnostics that can go on the list

Matt Warren
@mattwarren

Also I have implemented new kind of plugins: analysers, see PerfDotNet/BenchmarkDotNet@527df65

I really like the new plugin system, especially the analysers, they're a really nice idea

Andrey Akinshin
@AndreyAkinshin
@mattwarren, thanks!
I created a wiki page: https://github.com/PerfDotNet/BenchmarkDotNet/wiki Feel free to make any changes.
Tommy Long
@smudge202
Good discussions in the repo today.
Nikita
@Metallium

what is the way to configure benchmark runner to make it less precise but run much faster?

it happens you often need to make raw A/B testing on some things

Andrey Akinshin
@AndreyAkinshin
@Metallium, right now you can reduce amount of processes (the first parameter of BenchmarkTask) , amount of warmup and target iterations (the two last parameters). E.g., processCount=1, warmupIterationCount=3, targetIterationCount=5. I am trying to implement smart logic that will be adjust these parameters based on you benchmark: if you really don't need super precision, benchmarking will perform quickly. Hopefully, new logic will be published in December.
Tommy Long
@smudge202
^ This would be great @AndreyAkinshin .
Btw, I took a look at the public API last night. I actually only found that one usage of Exec, everything else looked fine.
Andrey Akinshin
@AndreyAkinshin
This message was deleted
@smudge202, cool. Waiting for your PR.
Nikita
@Metallium
it would be great to flexible setup
already tried to lower iteration counts - benchmark still takes a lot)
Matt Warren
@mattwarren
You can also do this:
[BenchmarkTask(mode: BenchmarkMode.SingleRun, processCount: 1, warmupIterationCount: 1, targetIterationCount: 1)] i.e. "SingleRun"
Tommy Long
@smudge202
I haven't checked, but is it possible to inherit from BenchmarkTaskAttribute and assign default properties through the base ctor?
For example, allowing @Metallium to create a QuickBenchmarkTaskAttribute with settings as you've described.
Matt Warren
@mattwarren
good question, I'm not sure either?
Tommy Long
@smudge202
Pretty sure it isn't sealed. Will have to check the discoverer
Matt Warren
@mattwarren
well this compiles for me:
public class QuickBenchmarkTaskAttribute : BenchmarkTaskAttribute
    {
        public QuickBenchmarkTaskAttribute() 
            : base(mode: BenchmarkMode.SingleRun, processCount: 1, warmupIterationCount: 1, targetIterationCount: 1)
        { }
    }
Tommy Long
@smudge202
I don't think it will be discovered
Because the GetCustomAttributes call linked above states false for allow inheritance
Matt Warren
@mattwarren
and
    [QuickBenchmarkTask]
    public class PerformanceUnitTest
Tommy Long
@smudge202
https://msdn.microsoft.com/en-us/library/ms130871(v=vs.110).aspx <= we need to set inherit: true to discover derived attributes I believe
Matt Warren
@mattwarren
that's a good point about GetCustomAttributes, I haven't actually run the test yet¬
Tommy Long
@smudge202
I can't see a reason not to set it to true to be honest
I believe the code that's there will still work