These are chat archives for allegro/hermes

8th
Jun 2016
Stevo Slavić
@sslavic
Jun 08 2016 11:18
FYI it was our bug that was causing stuck consumers, custom filter (0.8.6 based, so not using generic filtering support in hermes 0.8.7) was not releasing inflight semaphore on filtered out messages
you mentioned that response timeout is now configurable per subscription, in 0.8.8 - have you thought about making inflight (besides rate) configurable per subscription (with global/default max inflight)?
Stevo Slavić
@sslavic
Jun 08 2016 11:23
configuring rate and not inflight limit, especially not validating rate against inflight is potential source of misconfiguration - e.g. trying to increase rate, while too low inflight will be misleading, will not increase actual throughput limits
Adam Dubiel
@adamdubiel
Jun 08 2016 11:29
inflight is configurable as well
Stevo Slavić
@sslavic
Jun 08 2016 11:29
great
Adam Dubiel
@adamdubiel
Jun 08 2016 11:29
i know it is crucial to configure inflight & timeout together, for the reasons above
we run it on prod, it works well, next iteration we will add it to hermes console for all clients to see
but inflight size will be probably deducted from timeout there, because it takes better understanding of hermes to decide how to set proper inflight :)
Stevo Slavić
@sslavic
Jun 08 2016 11:34
thanks!
btw I find it bit misleading, backoff and rate configuration
backoff seems to be constant
but it's not applied when errors occur, different, rate mechanism is in place to slow down sending messages
I expected backoff not to be constant, to be more complex (linear or exponential)
and rate control to be simple, just max rate limiting
Stevo Slavić
@sslavic
Jun 08 2016 11:46
but (I think) I understand how it works now, and its fine as long as it works well :)
Adam Dubiel
@adamdubiel
Jun 08 2016 11:51
these are a bit separate concerns here. before we introduced backoff we had some problems with clients, who would be receiving same msgs again and again until rate control kicked in
so to mitigate this, we added backoff - msgs are now repeated in a certain time span, although rate control is still in hands of rate limiter
maybe it is time to rethink how backoff interleaves with rate control. we see that current rate control alg works very good for our clients, so it would be probably some kind of evolution to make backoff feel more natural to grasp
Stevo Slavić
@sslavic
Jun 08 2016 12:02
I understand, thanks for heads up
Adam Dubiel
@adamdubiel
Jun 08 2016 12:03
btw we used to have this simple algorithm you described (flat rate control + exponential backoff) in first versions of Hermes
but clients were not sure how to configure it to be safe. giving them single parameter (message ttl) which basically tells them for how long can the subscriber malfunction before discarding first message proved to be much better
Stevo Slavić
@sslavic
Jun 08 2016 12:06
makes sense