Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Repo info
Activity
  • Nov 10 2016 13:26
    martinwoodward unassigned #296
  • Nov 10 2016 13:26
    martinwoodward unassigned #295
  • Nov 10 2016 13:26
    martinwoodward unassigned #293
  • Nov 10 2016 13:26
    martinwoodward unassigned #294
  • Nov 10 2016 13:26
    martinwoodward unassigned #292
  • Nov 10 2016 13:26
    martinwoodward unassigned #291
  • Nov 10 2016 13:26
    martinwoodward unassigned #288
  • Nov 10 2016 13:26
    martinwoodward unassigned #286
  • Nov 10 2016 13:26
    martinwoodward unassigned #287
  • Nov 10 2016 13:26
    martinwoodward unassigned #285
  • Nov 10 2016 13:26
    martinwoodward unassigned #281
  • Nov 10 2016 13:26
    martinwoodward unassigned #280
  • Nov 10 2016 13:26
    martinwoodward unassigned #278
  • Nov 10 2016 13:26
    martinwoodward unassigned #273
  • Nov 10 2016 13:26
    martinwoodward unassigned #271
  • Nov 10 2016 13:26
    martinwoodward unassigned #276
  • Nov 10 2016 13:26
    martinwoodward unassigned #272
  • Nov 10 2016 13:26
    martinwoodward unassigned #275
  • Nov 10 2016 13:26
    martinwoodward unassigned #268
  • Nov 10 2016 13:26
    martinwoodward unassigned #265
Tommy Long
@smudge202
lol
Andrey Akinshin
@AndreyAkinshin
@smudge202, @mattwarren, do you have any comment about new plugin system and API? I want to make v0.8.0 on the weekend and publish it on NuGet. @smudge202, I am waiting four your PR with executables.
Andrey Akinshin
@AndreyAkinshin
Guys? Any feedback?
Matt Warren
@mattwarren
@AndreyAkinshin sorry for the delay, finally got time to look thru the plugin system, looks good to me
release it!
Andrey Akinshin
@AndreyAkinshin
@mattwarren, @smudge202, ok, v0.8.0 is in the NuGet. Now I want to keep the master branch in conformity with the latest NuGet package. Let's use the develop branch for developing.
Andrey Akinshin
@AndreyAkinshin
Guys, check it out: new benchmark framework https://petabridge.com/blog/introduction-to-nbench/ A funny thing: NBench uses BenchmarkDotNet for own infrastructure benchmarking: https://github.com/petabridge/NBench/blob/dev/benchmarks/NBench.Microbenchmarks/Program.cs
Matt Warren
@mattwarren
Cool, I saw NBench show up in my twitter feed, but didn't have time to look at it. That's really funny that NBench actually uses BenchmarkDotNet to test itself! (although I do wonder why he wrote a new library and didn't just use BenchmarkDotNet itself, maybe we weren't extensible enough? It wouldn't be that hard to add the NBench functionality to BenchmarkDotNet, now that we have the new plugin system)
Matt Warren
@mattwarren
Just found this comment from the author of NBench, it pretty much answers my previous question
Basicially this:
PerfDotNet only does throughput and micro-benchmarking, which is super helpful unto itself and it excels at doing that. But I needed to be able to also make assertions against Memory consumption and GC, so it didn't cover the spectrum of needs there. Plus I wanted unit test-style pass/fail semantics, which it also does not do.
Andrey Akinshin
@AndreyAkinshin
I published v0.8.1 on NuGet.
@mattwarren, please, use the develop branch for your commits. I want to keep the master branch and the NuGet package relevant.
Matt Warren
@mattwarren
yeah sorry about that, I just read the notice on the wiki page. I'll use develop in the future
what do you think about the NBench stuff, is it worth chatting to the author to see if we can persuade him to use BenchmarkDotNet, or do you think it's too late for that?
Andrey Akinshin
@AndreyAkinshin
Of course, we can discuss it. I think, if people really want to make performance unit tests, we can implement this feature in BenchmarkDotNet. However, we should put some warnings that they should do it carefully, use a separate machine for such tests, etc. For now, NBench has poor accuracy, our library has a lot of features that NBench doesn't have. It is cheaper to add unit testing for BenchmarkDotNet instead of implementing another full-featured benchmarking library.
By the way, right now I am implementing new algorithms that will choose wise amount of iteration. It will allow to run benchmarks really quickly and still have good accuracy. I'm almost finished, but such things takes a lot of time for testing.
Matt Warren
@mattwarren

It is cheaper to add unit testing for BenchmarkDotNet instead of implementing another full-featured benchmarking library.

Yeah good point, I guess we just add what we can to BenchmarkDotNet and see what happens. It's already getting pretty good traction in the CoreCLR/CoreFX GitHub issues.

By the way, right now I am implementing new algorithms that will choose wise amount of iteration. It will allow to run benchmarks really quickly and still have good accuracy. I'm almost finished, but such things takes a lot of time for testing.

That's cool, it'll stop things like this happening! https://twitter.com/ben_a_adams/status/678447298142199808

Jared Hester
@cloudRoutine
I've been using BenchmarkDotNet for a few weeks now and love it, PerfUtil used to be my goto until I found your project. Thanks for all your hard work! I've been trying to evangelize it to the F# community, and I think they're slowly warming to it. Maybe an FSharp.Charting plugin/helper module would make it more enticing....
Andrew Cherry
@kolektiv
@cloudRoutine - you've got it working with f#? i'm trying, but i'm finding it doesn't pick up an f# core. the assemblies and associated .config it generates doesn't have the assembly binding redirects to pick up FSharp.Core - have you found that? or a way round it?
Jared Hester
@cloudRoutine
@kolektiv I didn't run into that issue, got a repo I can inspect?
Jared Hester
@cloudRoutine
@kolektiv this trival example should run without issue
module BDNetFSharp.Program
open BenchmarkDotNet 
open System.Reflection

type ArrayVsList () =
    [<Params (10, 100)>] 
    member val public ColSize = 0 with get, set
    [<Benchmark>]
    member self.GetSquareArray () =
        let value = self.ColSize / 2 in [| for _ in 0 .. self.ColSize -> value |].[value]
    [<Benchmark>]
    member self.GetSquareList () =
        let value = self.ColSize / 2 in [for _ in 0 .. self.ColSize -> value ].[value]

let [<EntryPoint>] main args =
    let competitionSwitch =    
        Assembly.GetExecutingAssembly().GetTypes()
        |> Array.filter (fun t ->
            t.GetMethods (BindingFlags.Instance ||| BindingFlags.Public )
            |> Array.exists (fun m -> m.GetCustomAttributes (typeof<BenchmarkAttribute>, false) <> [||]))
        |> BenchmarkSwitcher
    competitionSwitch.Run args 
    0
Andrey Akinshin
@AndreyAkinshin
@cloudRoutine, I want to add special plugin type for plotting on the next week. It would be cool, if you can create PR with FSharp.Charting support.
Andrey Akinshin
@AndreyAkinshin
analysers.png
@mattwarren, check it out, I added BenchmarkEnvironmentAnalyser: PerfDotNet/BenchmarkDotNet@de01401
Andrew Cherry
@kolektiv
@cloudRoutine - see issue #67 :)
Andrey Akinshin
@AndreyAkinshin
Another project that uses BenchmarkDotNet: http://www.mapperator.net/Benchmark
Andrey Akinshin
@AndreyAkinshin
@mattwarren, I have implemented basic plot support (based on R script) in the refactoring branch.
Algo_MostSignificantBit-barplot.png
Matt Warren
@mattwarren
Looks cool
Tommy Long
@smudge202
I've been assigned some issues? ^^
Andrey Akinshin
@AndreyAkinshin
@smudge202, I thought that you want to implement CoreCLR support (you even have a local branch with some commits: https://github.com/smudge202/BenchmarkDotNet/commits/features/33-dnx-compatibility) Am I wrong?
@mattwarren, I am ready to publish v0.8.2, there is ChangeLog: https://github.com/PerfDotNet/BenchmarkDotNet/wiki/ChangeLog It includes your recent Baseline feature. Do you want to commit something before publishing?
Tommy Long
@smudge202
I'll take a look when I get the chance, but it's not very high on a fairly long list I'm afraid. We're not using Benchmarks as we couldn't find a good solution for our use cases.
Andrey Akinshin
@AndreyAkinshin
Ok, then I do it myself.
Matt Warren
@mattwarren
@AndreyAkinshin I just pushed one extra change (PerfDotNet/BenchmarkDotNet@9e16e8c) to make the Delta Result Extender a bit more robust. If that could go into 0.8.2 as well that would be cool.
BTW I think that the ChangeLog is a really nice idea. It also shows that BenchmarkDotNet is becoming a more mature OSS project, good job!
Andrey Akinshin
@AndreyAkinshin
@mattwarren, Ok, no problem. I will test the last develop version tonight and publish it.
Andrey Akinshin
@AndreyAkinshin
@mattwarren, your last merge contains errors, I can't build our solution in the develop branch.
You have removed my BenchmarkStatResultExtenderTests and spoil BaselineDeltaResultExtenderTest, PerformanceUnitTest, see diff with parent 140deccc.
Andrey Akinshin
@AndreyAkinshin
I made a rollback. Please, review it.
Matt Warren
@mattwarren
Uggh, I'm sorry about that. It's weird I had a problem with the merge, but I thought that I double-checked it by downloading the latest code and building it from scratch
Did you manage to fix it, or do you need me to do something?
Andrey Akinshin
@AndreyAkinshin
I already fixed it.
Matt Warren
@mattwarren
thanks
Andrey Akinshin
@AndreyAkinshin
It would be cool, if you will check that everything is fine in the develop branch.
Matt Warren
@mattwarren
BTW can you wait to release till tomorrow night? I just noticed that "Baseline" doesn't work when you run a benchmark as new BenchmarkRunner().Run<BenchmarkProgram>();, it only works via the BenchmarkSwitcher
Yeah, I'll take a look at the develop branch as well
Andrey Akinshin
@AndreyAkinshin

can you wait to release till tomorrow night

Of course.

Matt Warren
@mattwarren
thanks
BTW this (me messing up the merge) makes me think that we need some sort of C.I build