I have the following use case. A client sends messages to the server. The traffic is not limited in any way by either party, but at some point the server may instruct the client to reduce its rate by 20% (out of the traffic the client currently generates). How do you think bucket4j can help implementing such logic on the client side? The client has to constantly track its rate somehow, and when required, set the limit corresponding to the current rate minus 20%. Is there a way to obtain the current rate from the bucket and use it to reconfigure the limit?
@max904-github possibility of Bucket4j usage in this case highly depends from the answer to the following question:
what is rate and how it should be calculated. Because it is obvious that you can not reduce thing that can not be measured and understood.
If rate can be undestood as X request per Y time-window that it is possible to use Bucket4j in this case in the following way:
- You need to begin a measurement of request rate. SmoothlyDecayingRollingCounter from
Rolling-Metrics will be a good solution for calculation the rate at specific time window. - In time when client got response which contains requirement for rate reducing you need to get sum from counter(let it be Z) and create bucket with parameters Z tokens/Y time-window and use this bucket for request throttling.
Hello @milos01 ,
You can achieve this by:
- Providing slow refill, for example, one token per Long.MAX_VALUE nanoseconds, which can be treated as zero refill.
- Returning tokens manually via addTokens method https://github.com/vladimir-bukhtoyarov/bucket4j/blob/master/bucket4j-core/src/main/java/io/github/bucket4j/Bucket.java#L170
Doing both steps above allows achieving explicitly that you want. But I am not sure that you need to go in this way. If I have understood correctly you requirements the semaphore will be fully enough for your use case https://docs.oracle.com/javase/7/docs/api/java/util/concurrent/Semaphore.html
I'm looking into using Bucket4J over Hazelcast in memory cluster. Before doing so I would like to know that consuming from a bucket and updating it won't have performance impact on my application or increased latency on responses due to communication between the participating nodes of the Hazelcast cluster while a read or write operation is taking place. Let's assume that I'll need to enforce a limit per user and that each user can perform between a few to 10's operations (equivalent to bucket try consume operation) per second and that I'll have 1000's of users accessing my system.
I have just found this suggestion for a case similar to mine from about two years ago.
I wonder if something has evolved in the past two years and perhaps Bucket4J has a builtin mechanism similar to the suggested one or another technique is recommended these days
Hello @Joy94257280_twitter
I do not think that we need to do something if the requests are well distributed over keys, in that case, we can let IDMG do its job. The worst case is the case when all requests are associated with a single(or very few amount of keys), for that case I've implemented several optimizations in the 5.0 branch, but I do not think that you really need to configure this optimization because as you described above there is no single bucket that will act as a bottleneck.
@Joy94257280_twitter
Optimizations were implemented in the 5.0 branch which was not officially released yet. Source code can be found there https://github.com/vladimir-bukhtoyarov/bucket4j/tree/5.0/bucket4j-core/src/main/java/io/github/bucket4j/distributed/proxy/optimization
There are three types of optimizations:
- Batching - just groups parallel requests to the same bucket into the batches. It is not your case, because you do not expect the huge rate on same bucket.
- Delayed -may skip synchronization with shared storage according to configured delay parameters. Brings the risk overconsumption in case of aggressive setting.
- Predictive acts in the same way as Delayed, but additionally tries to predict consumption rate in the cluster. Brings the risk of underconsumption in case of wrong predictions.
General facts about optimizations:
- All optimizations applied on per bucket level. So optimized bucket proxy must be treated as a heavy-weight object and must be cached between user requests, in opposite with non-optimized proxy which is cheap and does not need to be stored anywhere.
- There are no background thread which does the synchronization of bucket state with centralized storage. Synchronization is performed in the scope of Bucket API call if the optimizer decides that it is time to do it.
Availability:
- There is no official release. But well tested
5.0.0-beta.01-09-2020has been deployed to the maven central https://mvnrepository.com/artifact/com.github.vladimir-bukhtoyarov/bucket4j-core/5.0.0-beta.01-09-2020 - There is no javadocs or wikipages that are addressed to
5.0branch, because documentation is in progress.
Also the unit tests by itself can be a good explanation of optimization loggic:
- https://github.com/vladimir-bukhtoyarov/bucket4j/blob/5.0/bucket4j-core/src/test/java/io/github/bucket4j/distributed/proxy/optimization/delay/DelayedCommandExecutorSpecification.groovy
- https://github.com/vladimir-bukhtoyarov/bucket4j/blob/5.0/bucket4j-core/src/test/java/io/github/bucket4j/distributed/proxy/optimization/predictive/PredictiveCommandExecutorSpecification.groovy
And do not forget to configure the listener in order to get proofs whether the optimization is helpful or not for your case
Also interesting to know is what the affect on such a sytem if one or two keys of it are accessed much more excessively than all the rest
In your tests and during development were you able to come up with some statistics on what can be the expected delay imposed on a system as a function of the number of keys used? Assuming enough shards such that the probability of making a network call is high.
It depends on the particular IDMG provider and the capacity of the cluster. For example, on my production, I have an average latency close to 1,5ms with a request rate clost to 15000 requests per second(but requests are well distributed).
Also interesting to know is what the affect on such a sytem if one or two keys of it are accessed much more excessively than all the rest
It is obvious that high contentions on the same key lead to serious performance degradation. In my experience all IDMG are not able to process more then 1000 requests per second on the same key.
@hderms in few words: Bucket4j uses EntryProcessor behind. One call to bucket equals to one execution of EntryProcessor. You can read about entry processors there https://hazelcast.com/blog/an-easy-performance-improvement-with-entryprocessor/
Also following links can be useful:
@vladimir-bukhtoyarov:
is there a way to deserialise a JSON string of Bucket to the Bucket object in java
Here is the Json string
{"timeMeter":{"wallClockBased":true},"configuration":{"bandwidths":[{"capacity":2,"initialTokens":2,"refillPeriodNanos":100000000000,"refillTokens":2,"refillIntervally":true,"timeOfFirstRefillMillis":-9223372036854776000,"useAdaptiveInitialTokens":false,"id":null,"gready":false,"intervallyAligned":false}]},"availableTokens":1}
A little bit of context of what I am trying to do
- I have created a bucket object for rate-limiting and currently saving in redis
- But not able to retrive this from redis as i need to create a custom deserialiser as im using jackson lib for deserialising in my project .
- is there any custom deserializer in the project to take the input json and return the bucket object ?