Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Repo info
Activity
  • Nov 10 2016 13:26
    martinwoodward unassigned #296
  • Nov 10 2016 13:26
    martinwoodward unassigned #295
  • Nov 10 2016 13:26
    martinwoodward unassigned #293
  • Nov 10 2016 13:26
    martinwoodward unassigned #294
  • Nov 10 2016 13:26
    martinwoodward unassigned #292
  • Nov 10 2016 13:26
    martinwoodward unassigned #291
  • Nov 10 2016 13:26
    martinwoodward unassigned #288
  • Nov 10 2016 13:26
    martinwoodward unassigned #286
  • Nov 10 2016 13:26
    martinwoodward unassigned #287
  • Nov 10 2016 13:26
    martinwoodward unassigned #285
  • Nov 10 2016 13:26
    martinwoodward unassigned #281
  • Nov 10 2016 13:26
    martinwoodward unassigned #280
  • Nov 10 2016 13:26
    martinwoodward unassigned #278
  • Nov 10 2016 13:26
    martinwoodward unassigned #273
  • Nov 10 2016 13:26
    martinwoodward unassigned #271
  • Nov 10 2016 13:26
    martinwoodward unassigned #276
  • Nov 10 2016 13:26
    martinwoodward unassigned #272
  • Nov 10 2016 13:26
    martinwoodward unassigned #275
  • Nov 10 2016 13:26
    martinwoodward unassigned #268
  • Nov 10 2016 13:26
    martinwoodward unassigned #265
Andrey Akinshin
@AndreyAkinshin
Ok, I will implement some basic logic on the weekend.
Tommy Long
@smudge202
Let me know if I can do anything to help.
Andrey Akinshin
@AndreyAkinshin
Ok, thanks!
Andrey Akinshin
@AndreyAkinshin
Guys, now we have big amount of upcoming features, idea and plans, discussions. What do you think about a wiki page with a roadmap?
Andrey Akinshin
@AndreyAkinshin
@mattwarren, @smudge202, please, review a new plugin system: PerfDotNet/BenchmarkDotNet@7eb70a1
Tommy Long
@smudge202
Just scanning over that commit now, @AndreyAkinshin
Tommy Long
@smudge202
That commit doesn't paint the whole picture; I assume there was previous work/commits for a plugin mechanism.
It would be great to have a sample showing off this plugin stuff so we can see the API you intend and then follow that through to check out implementation.
Andrey Akinshin
@AndreyAkinshin
@smudge202, thank you for the review. Check out my last commits: I have updated our integration tests with help of new plugin system. In particular, I defined own logger that accumulate logs in StringBuilder instead of System.Console.
Andrey Akinshin
@AndreyAkinshin
warnings.png
Also I have implemented new kind of plugins: analysers, see PerfDotNet/BenchmarkDotNet@527df65
@smudge202 @mattwarren what do you think?
Matt Warren
@mattwarren

Guys, now we have big amount of upcoming features, idea and plans, discussions. What do you think about a wiki page with a roadmap?

Yeah, we need to put this somewhere, even if it's only a GitHub discussion. For instnace I've got several ideas for new Diagnostics that can go on the list

Matt Warren
@mattwarren

Also I have implemented new kind of plugins: analysers, see PerfDotNet/BenchmarkDotNet@527df65

I really like the new plugin system, especially the analysers, they're a really nice idea

Andrey Akinshin
@AndreyAkinshin
@mattwarren, thanks!
I created a wiki page: https://github.com/PerfDotNet/BenchmarkDotNet/wiki Feel free to make any changes.
Tommy Long
@smudge202
Good discussions in the repo today.
Nikita
@Metallium

what is the way to configure benchmark runner to make it less precise but run much faster?

it happens you often need to make raw A/B testing on some things

Andrey Akinshin
@AndreyAkinshin
@Metallium, right now you can reduce amount of processes (the first parameter of BenchmarkTask) , amount of warmup and target iterations (the two last parameters). E.g., processCount=1, warmupIterationCount=3, targetIterationCount=5. I am trying to implement smart logic that will be adjust these parameters based on you benchmark: if you really don't need super precision, benchmarking will perform quickly. Hopefully, new logic will be published in December.
Tommy Long
@smudge202
^ This would be great @AndreyAkinshin .
Btw, I took a look at the public API last night. I actually only found that one usage of Exec, everything else looked fine.
Andrey Akinshin
@AndreyAkinshin
This message was deleted
@smudge202, cool. Waiting for your PR.
Nikita
@Metallium
it would be great to flexible setup
already tried to lower iteration counts - benchmark still takes a lot)
Matt Warren
@mattwarren
You can also do this:
[BenchmarkTask(mode: BenchmarkMode.SingleRun, processCount: 1, warmupIterationCount: 1, targetIterationCount: 1)] i.e. "SingleRun"
Tommy Long
@smudge202
I haven't checked, but is it possible to inherit from BenchmarkTaskAttribute and assign default properties through the base ctor?
For example, allowing @Metallium to create a QuickBenchmarkTaskAttribute with settings as you've described.
Matt Warren
@mattwarren
good question, I'm not sure either?
Tommy Long
@smudge202
Pretty sure it isn't sealed. Will have to check the discoverer
Matt Warren
@mattwarren
well this compiles for me:
public class QuickBenchmarkTaskAttribute : BenchmarkTaskAttribute
    {
        public QuickBenchmarkTaskAttribute() 
            : base(mode: BenchmarkMode.SingleRun, processCount: 1, warmupIterationCount: 1, targetIterationCount: 1)
        { }
    }
Tommy Long
@smudge202
I don't think it will be discovered
Because the GetCustomAttributes call linked above states false for allow inheritance
Matt Warren
@mattwarren
and
    [QuickBenchmarkTask]
    public class PerformanceUnitTest
Tommy Long
@smudge202
https://msdn.microsoft.com/en-us/library/ms130871(v=vs.110).aspx <= we need to set inherit: true to discover derived attributes I believe
Matt Warren
@mattwarren
that's a good point about GetCustomAttributes, I haven't actually run the test yet¬
Tommy Long
@smudge202
I can't see a reason not to set it to true to be honest
I believe the code that's there will still work
Installing VS2015 U1 so can't check in VS, and too lazy to update VSCode... sorry. :)
Tommy Long
@smudge202
I'm going to create an associated bug. Either BenchmarkAttribute should be sealed or the GetCustomAttributes call should allow inheritance. The combination we have right now I consider to be a bug. Obviously, changing the call instead of sealing the attribute would be a better fix.
Matt Warren
@mattwarren
sounds good
Tommy Long
@smudge202
PerfDotNet/BenchmarkDotNet#57
Matt Warren
@mattwarren

wierdly enough

[QuickBenchmarkTask]
public class PerformanceUnitTest()

works for me, but I'll look into it a bit more when I get round to the issue (there's a bit of a backlog building up :-))

Tommy Long
@smudge202
Oh... was I wrong then? It does discover inherited attributes?
Matt Warren
@mattwarren
not really sure at the moment, maybe, maybe not!
Tommy Long
@smudge202
lol
Andrey Akinshin
@AndreyAkinshin
@smudge202, @mattwarren, do you have any comment about new plugin system and API? I want to make v0.8.0 on the weekend and publish it on NuGet. @smudge202, I am waiting four your PR with executables.
Andrey Akinshin
@AndreyAkinshin
Guys? Any feedback?
Matt Warren
@mattwarren
@AndreyAkinshin sorry for the delay, finally got time to look thru the plugin system, looks good to me
release it!
Andrey Akinshin
@AndreyAkinshin
@mattwarren, @smudge202, ok, v0.8.0 is in the NuGet. Now I want to keep the master branch in conformity with the latest NuGet package. Let's use the develop branch for developing.