Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Activity
    Martin Ankerl
    @martinus
    Ask any question about ankerl::nanobench here!
    Clément Grégoire
    @Lectem
    Hello, what do you think of allowing to set the output format on top of the output stream ? That way it would be easier to always ouptut results in a given format (json, cpp) without having to remember to call Bench::render in the correct place
    it could also make it easier to switch based on command line options, since if you reuse a same bench with a different title it will clear the current results ( and so you need to remember to call render before each call to title or a change in config)
    Martin Ankerl
    @martinus
    Hi, that's already possible. You can use output(nullptr) to disable console ouput, then use the render functionality to render to std::cout
    Clément Grégoire
    @Lectem
    Yes but you need to call render yourself
    so if I have something like:
    
        int benchsIterations[] = {2, 8, 16, 32, 64, 256, 1024, 1024 * 1024};
        for (int nbValues : benchsIterations)
        {
            cfg.title("Summation " + std::to_string(nbValues)).relative(true).minEpochIterations(50).minEpochTime(10ms);
            nanobench::Rng rng;
            std::vector<Scalar> values(nbValues);
            std::generate(values.begin(), values.end(), [&] { return Scalar(rng.uniform01()); });
    
            cfg.run("Naive raw sum", [&] {
                const Scalar* beg = values.data();
                Scalar acc        = naive_sum(beg, beg + nbValues);
                nanobench::doNotOptimizeAway(acc);
            });
    
    // More benchs
    
    // I need to call render here, because if put out of the loop it will only render for the last loop
    
        }
    Martin Ankerl
    @martinus
    In version 4.0 I'll release some big changes to the rendering, then it will behave a bit differently. You still need to call render yourself though. You might want to create an issue for this though. Due to the corona lockdown I've practically no time for this so I would definitely forget without an issue
    Clément Grégoire
    @Lectem
    I might try to draft something first, see how usable it is :)
    Clément Grégoire
    @Lectem
    Hi
    I saw this change
    https://github.com/martinus/nanobench/commit/6f3b8f2adce0160d8e09f0802250bc80cf2e04b7#diff-6d37775f1ac5713cba7f653f48ed3b48L1987-L1996
    Does this mean I should not be using the current master / mustache anymore ?
    Clément Grégoire
    @Lectem
            case Node::Type::tag:
    -           if (n == "unit") {
    -               out << bench.unit();
    -           } else if (n == "title") {
    -               out << bench.title();
    -           } else if (n == "batch") {
    -               out << bench.batch();
    -           } else {
    -               throw std::runtime_error("unknown tag '" + std::string(n.begin, n.end) + "'");
    -           }
    +           throw std::runtime_error("unknown tag '" + std::string(n.begin, n.end) + "'");
                break;
            }
        }
    Clément Grégoire
    @Lectem
    Oh... the title is only available in the Result now ?
    This seems a bit unpractical that generate now takes only a std::vector<Result> and not Bench as param
    Clément Grégoire
    @Lectem
    Could be useful if the list of results was not cleared when title/unit changes but right now I think it only duplicates data
    Clément Grégoire
    @Lectem
    just in case, I opened a PR martinus/nanobench#29
    elbaro
    @elbaro
    @martinus can you make a new release?
    Martin Ankerl
    @martinus
    Hi all! my plan is to update the documentation, potentially make a few small fixes, and then have a new release. @Lectem , you mentioned in onqtam/doctest#251 that you have a few changes to nanobench, do you plan/think it would be worthwhile to have these changes in nanobench? If so than I'd like to merge them into nanobench before updating the documentation. I don't want to rewrite it twice
    Clément Grégoire
    @Lectem
    Glad to hear you're back/safe :)
    I do have a few changes but they are not quite ready to be merged I think, but I'd love to discuss those with you
    Basicly it's linked to reporters support, I hacked in a thing so that one can switch the reporter globally / replace the markdown one
    The main objective was to be able to output json easily during CI
    I also have a few other changes related to the iteration logic, they make it a bit longer to run but are IMHO more correct.
    Clément Grégoire
    @Lectem
    I actually had to do it to have a more decent reproductibility of my results, even with very simple tests and cpu scaling disabled, isolated CPUs etc
    It +/- all boils down to the fact nanobench currently tries to reach not 1000x the clock resolution, but in fact only 2/3*1000
    Martin Ankerl
    @martinus
    actually it should be somewhere between 1000x and 1200x of the clock resolution, the 2/3 is to find out when to stop upscaling the iterations and switch to measurement mode. It seems to be correct for one of my examples
    Clément Grégoire
    @Lectem
    ah right, somehow it gave me slightly better results, maybe it depends on the resolution of the clock and duration of tests ?
    speaking of which, the comment says and adds a 10% noise while it's a 20% noise :)
    Martin Ankerl
    @martinus
    It's actually random between 0% and 20%, so on average it should be 10% :)
    Clément Grégoire
    @Lectem
    well, that's true I suppose, though it depends on the quality of the PRNG :D
    Martin Ankerl
    @martinus
    Good news, everyone! I've just released nanobench 4.0.0, which is a major update from 3.1.0. Lots of updates, and a new beautiful documentation: https://nanobench.ankerl.com/
    Clément Grégoire
    @Lectem
    I'm gonna have to take a look^^
    btw, I was reading a bit about cpu counters, it seems that if you want accurate results on linux, you must have 4 enabled at most at the same time (for x64 CPUs), otherwise it's using multiplexing
    which means you need the benchmark to last long enough otherwise results are interpolated
    Clément Grégoire
    @Lectem
    Oh, and it might be nice to have a documentation badge at the top of the readme :)
    Clément Grégoire
    @Lectem
    hello, just to let you know, I'm having issues building with the latest emscripten, not sure if its because of one of their changes or because of upstream clang
    ./external/nanobench/include\nanobench.h:444:19: error: couldn't allocate input reg for constraint 'r'
    asm volatile("" : : "r"(val) : "memory");
    ^
    looks like asm volatile("" : : "r,m"(val) : "memory"); works though
    Martin Ankerl
    @martinus
    Hi, that's good to know. I guess I'll switch to what google benchmark is doing, which does exactly what you propose. It seems to be the most well tested. Currently I'm doing the same as facebook's folly is doing.