by

Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Repo info
Activity
    James Kleeh
    @jameskleeh
    Wouldn’t that always be the value passed to Refill.intervally in that example?
    Vladimir Bukhtoyarov
    @vladimir-bukhtoyarov
    @jameskleeh refill describes the speed of tokens regeneration.
    // means that 10 tokens are added to bucket each second
    Refill.intervally(10, Duration.ofSeconds(1));
    James Kleeh
    @jameskleeh
    Right so wouldn’t the bucket only ever hold 10 tokens?
    Vladimir Bukhtoyarov
    @vladimir-bukhtoyarov

    Wouldn’t that always be the value passed to Refill.intervally in that example?

    No. Capacity of the bucket is an independent entity. With the token-bucket algorithm, you have 3 configuration points. The token bucket algorithm can be conceptually understood as follows:

    • N tokens are added to the bucket every M seconds.
    • The bucket can hold at the most C tokens. If a token arrives when the bucket is full, it is discarded.

    So acording to your question:

    • N = 10 tokens
    • M = 1 second
    • C = 600 tokens
    James Kleeh
    @jameskleeh
    So if the bucket can hold 600 tokens, that means 600 requests could be made within a short amount of time without violating the rate limit?
    Vladimir Bukhtoyarov
    @vladimir-bukhtoyarov
    yes
    James Kleeh
    @jameskleeh
    So its basically a limit on rollover
    ie your tokens roll over from previous periods until they reach X
    OK thanks for the explanation. I think the docs could be improved in this area
    Vladimir Bukhtoyarov
    @vladimir-bukhtoyarov
    eventually
    Vladimir Bukhtoyarov
    @vladimir-bukhtoyarov

    @jameskleeh there is a variant of API usage when N always equals C:

    Bandwidth.simple(10, Duaration.ofSeconds(1))

    In this case both N and C set to 10

    James Kleeh
    @jameskleeh
    Yeah I needed to use the classic variant and struggled to understand why the additional configuration was needed
    Seems to me you could forgo the Refill class altogether and just have a builder for Bandwith
    Vladimir Bukhtoyarov
    @vladimir-bukhtoyarov
    yes, it is used just in Builder API
    Rasmus Lund
    @rasmuslund
    Any update on the support for using Redis?
    Vladimir Bukhtoyarov
    @vladimir-bukhtoyarov
    @rasmuslund direct support for Redis based on Lua stored procedures has been postponed and will not be included to the upcoming 5.0 release. But you can build something useful from sources based on this example. It is example of usage RMapBasedRedissonBackend.
    Rasmus Lund
    @rasmuslund
    @vladimir-bukhtoyarov Thank you :-)
    deepthipr7
    @deepthipr7
    Is there a limit on how many buckets can be created with bucket4j?
    Vladimir Bukhtoyarov
    @vladimir-bukhtoyarov
    @deepthipr7 there are no limits
    Vladimir Bukhtoyarov
    @vladimir-bukhtoyarov
    @deepthipr7 but you have to have a clear understanding of eviction policy if you are using in-memory grids https://github.com/vladimir-bukhtoyarov/bucket4j/blob/master/doc-pages/production-jcache-checklist.md#retention-tuning-is-your-responsibility
    deepthipr7
    @deepthipr7
    @vladimir-bukhtoyarov thanks for the clarification.
    sjoshi-liveperson
    @sjoshi-liveperson

    Hello,

    We have to implement rate limiter in distributed environment.

    Requirement:
    In order to make these rate limit determinations with minimal latency, it’s necessary to make checks locally in memory. This can be done by relaxing the rate check conditions and using an eventually consistent model. For example, each node can create a data sync cycle that will synchronize with the centralized data store. Each node periodically pushes a counter increment for each consumer and window it saw to the datastore, which will atomically update the values. The node can then retrieve the updated values to update it’s in-memory version. This cycle of converge → diverge → reconverge among nodes in the cluster is eventually consistent.

    AFAIK: All the caching libraries linked here like Ignite, Hazelcast etc. They provide distributed caching where cache key map is distributed across all the nodes in the cluster. With this, a request to consume token coming to different node other than the node on which that particular token is stored using distributed cache, will still require comunicating to the "primary node" of that key and updating the value.

    Is there any mechanism with Bucket4j where above requirement can be solved? I want to avoid every time reaching out to the "primary node" of that key to know if the bucket has token.

    Reference: https://konghq.com/blog/how-to-design-a-scalable-rate-limiting-algorithm/

    Vladimir Bukhtoyarov
    @vladimir-bukhtoyarov

    Hello @sjoshi-liveperson

    Bucket4j will solve the over-communication problem in another way: for upcoming 5.0 release, I have implemented request batching.
    The algorithm is simple:

    • First request is imideatelly routed to primary node.
    • Subsequent requests are accumulated in the queue.
    • When first request done, then all accumulated requests combined in the MultiCommand and sent to primary node as single request.

    The algorithm above does not solve latency problem, but significantly increases the output. It is easy to achieve milions operation per second on single bucket

    But there is a limitation - batching is applied per bucket basis. If you have a huge amount of buckets with a low request rate on each bucket then batching with not work well.
    Vladimir Bukhtoyarov
    @vladimir-bukhtoyarov
    The next work will be batching requests from the different bucket, but I do not know when I will have time to do this, because it requires to use vendor-specific API for each grid provider.
    Vladimir Bukhtoyarov
    @vladimir-bukhtoyarov
    @sjoshi-liveperson If it is not enough, let's discuss your use case more deeply, I am sure that requirements can be meat by building something on top Bucket4j without introducing new functionality to the core
    sjoshi-liveperson
    @sjoshi-liveperson

    Thank you @vladimir-bukhtoyarov , if I understand correctly this is what you are saying:

    When using bucket4j with distributed cache. let us say there are 20 nodes and 40 buckets, then each node will primarily host roughly around 2 buckets as part of distributed caching. suppose node1 hosts bucket1 and bucket2. suppose Node 11 to 20 all get request involving bucket1, in that case All those 10 nodes will reach out to node1 to get token and here you are trying to increasing the efficiency to reduce bucket count efficently without adding much locking.

    But the fact that all 10 nodes talking to primary node1 of that bucket will still happen and add to latency correct?

    Vladimir Bukhtoyarov
    @vladimir-bukhtoyarov
    yes, the problem with latency still exists
    sjoshi-liveperson
    @sjoshi-liveperson
    @vladimir-bukhtoyarov thank you very much for the clarification.
    Moises Gamio
    @mgamio
    Hi, is it possible to implement it in SOAP web service?
    Vladimir Bukhtoyarov
    @vladimir-bukhtoyarov

    @mgamio hello.

    I did not understand your question. Bucket4j is library and since it is not a framework it could be used anywhere you wish.

    sjoshi-liveperson
    @sjoshi-liveperson
    @vladimir-bukhtoyarov In addition to the throttle bucket algortihm, if we want to implement a throttling when say host CPU is say above 90%, Are there any recommendations/examples to implement the same?
    Vladimir Bukhtoyarov
    @vladimir-bukhtoyarov
    I i were you i would introduce global volatile variable for holding current cpu utulization. This variable should be updated by background timer. Then just check value of variable on each request,
    and compare with threshold. Also it is obviously that CPU checking filter should be placed before other throttlers.
    sjoshi-liveperson
    @sjoshi-liveperson
    Thank you
    Srikkanth
    @srionline_gitlab
    Hi @vladimir-bukhtoyarov , Is there a sample/example for integrating embedded Apache Ignite with bucket4j in a Spring boot application? Also, Could you elaborate on the asynchronous processing that we get when using bucket4j-ignite vs bucket4j-jcache. Thanks
    Vladimir Bukhtoyarov
    @vladimir-bukhtoyarov
    Hello @srionline_gitlab try to find examples there https://github.com/MarcGiffing/bucket4j-spring-boot-starter
    Vladimir Bukhtoyarov
    @vladimir-bukhtoyarov
    About async, it is simple. You is unable to use async operations with bucket4j-jcache because JSR 107 does not specify the async API. With bucket4j-ignite you can use async API if you want. The entry point for async API is bucket.asAsync(). The full list of async methods could be discovered there
    Srikkanth
    @srionline_gitlab
    Thanks @vladimir-bukhtoyarov.
    max904-github
    @max904-github
    Hi Vladimir,
    Please add Armeria to the list of Bucket4j third-party integrations. "armeria-bucket4j" module available since Armeria version 0.99.
    Armeria (https://line.github.io/armeria/) is a pupular asynchronous RPC/REST library built on top of Java 8, Netty, HTTP/2, Thrift and gRPC. Project's Github - https://github.com/line/armeria. It uses Bucket4j to extend its throttling capabilities for HTTP and RPC protocols.
    Vladimir Bukhtoyarov
    @vladimir-bukhtoyarov
    Will do
    Vladimir Bukhtoyarov
    @vladimir-bukhtoyarov
    @max904-github btw, if I understand correctly Armeria just uses Bucket4j, so Armeria does not add any new value to Bucket4j like for example bucket4j-springboot-starter does. So right pkace should be "who uses"
    max904-github
    @max904-github
    @vladimir-bukhtoyarov Sorry for the confusion. Correct. Armeria just uses Bucket4j for advanced throttling.
    kommrad homer
    @kommradHomer
    @vladimier-bukhtoyarov Hey there , I just read your answer about deleting a bucket . I'm using ignite for cache. Do I need to connect to Ignite and delete the bucket ? nothing via the Bucket4j ? I just couldn't figure out which ID we are talking about
    Vladimir Bukhtoyarov
    @vladimir-bukhtoyarov

    @kommradHomer hello

    Bucket4j does not provide deleting functionality. I told about ID which was used during bucket creation. Bucket4j does not provide deleting functionality but it is easy to achieve via Cache API

    Bucket bucket = Bucket4j.extension(io.github.bucket4j.grid.ignite.Ignite.class).builder()
                       .addLimit(Bandwidth.simple(1_000, Duration.ofMinutes(1)))
                       .build(cache, key, RecoveryStrategy.RECONSTRUCT);
    
    // to delete just remove the item for cache
    cache.remove(key);

    But there is a tricky moment. If you will continue to use bucket then it will be resurrected on next iteration with bucket because of RecoveryStrategy.RECONSTRUCT

    kommrad homer
    @kommradHomer
    @vladimir-bukhtoyarov thanks a lot for the quick response as always !!! I just figured it out 15 minutes ago , perhaps i had to refresh the knowledge . Perhaps I was confused about "ID" , as I would call that a "key". Hopefully It must be valuable to have the answer listed here too. But I couldn't find how to search through all the discussion here :/
    Ehsan Salmani
    @isanuric
    Hi Vladimir, is it possible to set BucketState.getLastRefillTimeNanos(int bandwidth) to public? I need the last update time of bucket and I want to avoid parsing the GetGridBucketState(key).getState().toString(). Thanks.
    Vladimir Bukhtoyarov
    @vladimir-bukhtoyarov
    @isanuric it is possible. Create the issue on github. Btw I am interested why do you need in this functionality
    abhishek2bommakanti
    @abhishek2bommakanti
    @vladimir-bukhtoyarov Hello Vladimir - how do I configure a bucket to have no limits? Basically a bucket that does not restrict any calls coming in?
    Vladimir Bukhtoyarov
    @vladimir-bukhtoyarov
    @abhishek2bommakanti hello. There are no such feature. But it can be emulated by providing high-limit, for example 1_000_000_000 tokens per one second.