Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Repo info
Activity
    Vladimir Bukhtoyarov
    @vladimir-bukhtoyarov
    Hello @akshayjain328 maybe in the next year
    akshayjain328
    @akshayjain328
    Thanks
    max904-github
    @max904-github
    Hi Vladimir,
    I have the following use case. A client sends messages to the server. The traffic is not limited in any way by either party, but at some point the server may instruct the client to reduce its rate by 20% (out of the traffic the client currently generates). How do you think bucket4j can help implementing such logic on the client side? The client has to constantly track its rate somehow, and when required, set the limit corresponding to the current rate minus 20%. Is there a way to obtain the current rate from the bucket and use it to reconfigure the limit?
    Vladimir Bukhtoyarov
    @vladimir-bukhtoyarov

    @max904-github possibility of Bucket4j usage in this case highly depends from the answer to the following question:
    what is rate and how it should be calculated. Because it is obvious that you can not reduce thing that can not be measured and understood.

    If rate can be undestood as X request per Y time-window that it is possible to use Bucket4j in this case in the following way:

    • You need to begin a measurement of request rate. SmoothlyDecayingRollingCounter from
      Rolling-Metrics will be a good solution for calculation the rate at specific time window.
    • In time when client got response which contains requirement for rate reducing you need to get sum from counter(let it be Z) and create bucket with parameters Z tokens/Y time-window and use this bucket for request throttling.
    max904-github
    @max904-github
    Thank you, @vladimir-bukhtoyarov! Yes, indeed, the rate in my case is X requests per Y time-window. I will check out your Rolling-Metrics project.
    mvanmeerbeck
    @mvanmeerbeck
    Hello there!
    Is it a good idea to use keycloak with bucket4j to limit my api per client ? Any suggestions ? Thanks guys!
    Vladimir Bukhtoyarov
    @vladimir-bukhtoyarov

    hello @mvanmeerbeck

    Bucket4j is just a library, you are free to use it together with everything you want.

    mvanmeerbeck
    @mvanmeerbeck
    no, it's not just a library, it's an awesome library
    Vladimir Bukhtoyarov
    @vladimir-bukhtoyarov
    Definitely no. The upcoming 5.0 version will be awesome :-)
    Miloš Andrić
    @milos01
    Hello @vladimir-bukhtoyarov, I'm interested in one specific use case of Bucket4J library. The behavior that I want to achieve is to have limits based on concurrency (not space-time wise). I want to throttle some API with an initial size of 10 and every time that request comes in to consume 1 token and after its finished to return that 1 token to the bucket but without automatic refill by some refill rate or mechanism. So this behavior enables me to have max 10 concurrent requests going on at the time and if the 11th one will be blocked until I return one of tokens to the bucket. Thanks in advance!
    Vladimir Bukhtoyarov
    @vladimir-bukhtoyarov

    Hello @milos01 ,

    You can achieve this by:

    Doing both steps above allows achieving explicitly that you want. But I am not sure that you need to go in this way. If I have understood correctly you requirements the semaphore will be fully enough for your use case https://docs.oracle.com/javase/7/docs/api/java/util/concurrent/Semaphore.html

    Miloš Andrić
    @milos01
    @vladimir-bukhtoyarov Perfect, I would like to use more generic approach with your library because this is not the only use case that I have. I would like to have also the default use case with limiting time-wise so using just Semaphone alone won't get me all, or you had something other on mind?
    Vladimir Bukhtoyarov
    @vladimir-bukhtoyarov
    with infinite refill, only methods declared in the basic interface Bucket will work correctly. Extensions like BlockingBucket and AsyncScheduledBucket will not work correctly.
    Miloš Andrić
    @milos01
    Hmm, Im using asScheduler() method from Bucket interface, but as I can see its instance of BlockingBucket
    Vladimir Bukhtoyarov
    @vladimir-bukhtoyarov
    asScheduler returns BlockingBucket and this will not work. So I highly recommend to use semaphore if you need to reduce concurrency level instead of consumption rate. Bucket4j as well as token-bucket is designed to limit consumption rate.
    Miloš Andrić
    @milos01
    Understood, thanks for your help!
    pandhariwal-1
    @pandhariwal-1
    hey,
    How to get a specific header value in SPEL in webflux based application?
    Vladimir Bukhtoyarov
    @vladimir-bukhtoyarov

    Hello @pandhariwal-1

    It is better to ask on SpringFramework forum

    pandhariwal-1
    @pandhariwal-1
    okay.
    Joy
    @Joy94257280_twitter
    Hi All,
    I'm looking into using Bucket4J over Hazelcast in memory cluster. Before doing so I would like to know that consuming from a bucket and updating it won't have performance impact on my application or increased latency on responses due to communication between the participating nodes of the Hazelcast cluster while a read or write operation is taking place. Let's assume that I'll need to enforce a limit per user and that each user can perform between a few to 10's operations (equivalent to bucket try consume operation) per second and that I'll have 1000's of users accessing my system.
    Joy
    @Joy94257280_twitter
    :point_up: October 29, 2018 11:20 PM
    I have just found this suggestion for a case similar to mine from about two years ago.
    I wonder if something has evolved in the past two years and perhaps Bucket4J has a builtin mechanism similar to the suggested one or another technique is recommended these days
    Joy
    @Joy94257280_twitter
    @vladimir-bukhtoyarov can you perhaps shed a bit light on the above?
    Vladimir Bukhtoyarov
    @vladimir-bukhtoyarov

    Hello @Joy94257280_twitter

    I do not think that we need to do something if the requests are well distributed over keys, in that case, we can let IDMG do its job. The worst case is the case when all requests are associated with a single(or very few amount of keys), for that case I've implemented several optimizations in the 5.0 branch, but I do not think that you really need to configure this optimization because as you described above there is no single bucket that will act as a bottleneck.

    Joy
    @Joy94257280_twitter
    Thank you
    @vladimir-bukhtoyarov just to have a whole understanding around this issue can you elaborate on these optimizations or refer me to the code where they are implemented?
    Vladimir Bukhtoyarov
    @vladimir-bukhtoyarov

    @Joy94257280_twitter

    Optimizations were implemented in the 5.0 branch which was not officially released yet. Source code can be found there https://github.com/vladimir-bukhtoyarov/bucket4j/tree/5.0/bucket4j-core/src/main/java/io/github/bucket4j/distributed/proxy/optimization

    There are three types of optimizations:

    • Batching - just groups parallel requests to the same bucket into the batches. It is not your case, because you do not expect the huge rate on same bucket.
    • Delayed -may skip synchronization with shared storage according to configured delay parameters. Brings the risk overconsumption in case of aggressive setting.
    • Predictive acts in the same way as Delayed, but additionally tries to predict consumption rate in the cluster. Brings the risk of underconsumption in case of wrong predictions.

    General facts about optimizations:

    • All optimizations applied on per bucket level. So optimized bucket proxy must be treated as a heavy-weight object and must be cached between user requests, in opposite with non-optimized proxy which is cheap and does not need to be stored anywhere.
    • There are no background thread which does the synchronization of bucket state with centralized storage. Synchronization is performed in the scope of Bucket API call if the optimizer decides that it is time to do it.

    Availability:

    Joy
    @Joy94257280_twitter
    This sounds amazing. I’ll keep following. Thanks
    Vladimir Bukhtoyarov
    @vladimir-bukhtoyarov
    examples of usage can be found in unit tests
    Joy
    @Joy94257280_twitter
    In your tests and during development were you able to come up with some statistics on what can be the expected delay imposed on a system as a function of the number of keys used? Assuming enough shards such that the probability of making a network call is high.
    Also interesting to know is what the affect on such a sytem if one or two keys of it are accessed much more excessively than all the rest
    Vladimir Bukhtoyarov
    @vladimir-bukhtoyarov

    In your tests and during development were you able to come up with some statistics on what can be the expected delay imposed on a system as a function of the number of keys used? Assuming enough shards such that the probability of making a network call is high.

    It depends on the particular IDMG provider and the capacity of the cluster. For example, on my production, I have an average latency close to 1,5ms with a request rate clost to 15000 requests per second(but requests are well distributed).

    Also interesting to know is what the affect on such a sytem if one or two keys of it are accessed much more excessively than all the rest

    It is obvious that high contentions on the same key lead to serious performance degradation. In my experience all IDMG are not able to process more then 1000 requests per second on the same key.

    Joy
    @Joy94257280_twitter
    which IMDG are you using in your production?
    Vladimir Bukhtoyarov
    @vladimir-bukhtoyarov
    GridGain. It is commercial version of Apache Ignite.
    Joy
    @Joy94257280_twitter
    Any reason related to performance to prefer it over Hazelcast or the other way round?
    Vladimir Bukhtoyarov
    @vladimir-bukhtoyarov
    I suppose that it must not be significant differences in the performance GridGaing versus Hazelcast. We just already had GridGain licenses when were implementing the throttling. So it was no real comparison on our side.
    Joy
    @Joy94257280_twitter
    Part of the problem I’m trying to solve is “concurrent” rate limit. In my case queries to a database. I guess that concurrent can be imposed by ignoring the fact whether a query has already ended and setting the time frame of a bucket to a very short period, say 1 second. And also using a composite bucket that has rate limits for the number of queries per minute and per second where the per second is the limit on the number of concurrent queries. However this is not ideal as the enforced concurrent limit will be more strict than it could be if a lot of queries finish really quickly. Also, using such a short period of time, 1 second, will not allow me to use the above mentioned optimization. I understand that these last two considerations are contradicting almost by nature by I wanted to lay them down just for the completenes of the discussion. Have you encountered a similar problem during your work on limits?
    Vladimir Bukhtoyarov
    @vladimir-bukhtoyarov

    I think that it is better to divide the problem into two independent parts:

    • Parallel processing limitation - use semaphore there.
    • Request rate limitation - use token-bucket there.

    Token-bucket can not be used as a semaphore

    Joy
    @Joy94257280_twitter
    I’ve started with a similar idea based on distributed counters but the thought that a worker might die without decrementing the counter sent me to think on other ideas. Now I see that indeed Hazelcast has a semaphore that is automatically being released if a node dies. Thanks.
    Joy
    @Joy94257280_twitter
    @vladimir-bukhtoyarov what happens if my IMDG cluster looses a node? Or connectivity between nodes? How would rate limitting behave in such a case?
    Vladimir Bukhtoyarov
    @vladimir-bukhtoyarov
    Dermot Haughey
    @hderms
    does anyone know of any places to see the architecture of bucket4j? In particular I'm interested in how it handles distributed rate limiting
    Vladimir Bukhtoyarov
    @vladimir-bukhtoyarov

    @hderms in few words: Bucket4j uses EntryProcessor behind. One call to bucket equals to one execution of EntryProcessor. You can read about entry processors there https://hazelcast.com/blog/an-easy-performance-improvement-with-entryprocessor/

    Also following links can be useful:

    Alberto Acevedo
    @alberto-acevedo
    Is there a way to auto tune the bucket and refill rate on the client side based on how long it takes a server to respond to a client request? The idea is to automatically detect and reduce the tokens availability if the server is too slow of it can not handle the current rate. Thanks.
    Vladimir Bukhtoyarov
    @vladimir-bukhtoyarov

    @alberto-acevedo hello,

    No, there are no such feature.

    dream-y
    @dream-y
    Is there any update for redis support?
    Vladimir Bukhtoyarov
    @vladimir-bukhtoyarov
    no updates
    limjy
    @limJingYun
    Is there a warmup function for Bucket4J like Guava RateLimiter? I could not find any examples in the Basic & Advanced Usage documentation on the Bucket4J repo.
    Vladimir Bukhtoyarov
    @vladimir-bukhtoyarov
    No updates
    There is no such functionality.
    Vladimir Bukhtoyarov
    @vladimir-bukhtoyarov
    This room has been obsolete because of migration to Github discussions https://github.com/vladimir-bukhtoyarov/bucket4j/discussions