These are chat archives for RBMHTechnology/eventuate

5th
Apr 2016
nonysingh79
@nonysingh79
Apr 05 2016 08:21
@krasserm : We have a case where we have created huge number of events in one of the event log and now trying to add another location, while trying to add new location , replication fails due to huge payload size : akka.remote.OversizedPayloadException: Discarding oversized payload sent to Actor[akka.tcp://location@172.XX.XX.X:2554/temp/$1]: max allowed size 1024000 bytes, actual size of encoded class com.rbmhtechnology.eventuate.ReplicationProtocol$ReplicationReadSuccess was 6979033 bytes.. We have tried reducing batch writer size to 8 and increasing the tcp frame zsize till 1024000b. when i run the app with replication enabled from beginning there is no issue with message size. what is the recommendation here? increase message size? is ther a way to replicate all events to another location through some scripts?
Martin Krasser
@krasserm
Apr 05 2016 08:47
@nonysingh79 try to iteratively increase akka.remote.netty.tcp.maximum-frame-size by a factor of 2 and decrease eventuate.log.write-batch-size by a factor of 2 until it works (see also http://rbmhtechnology.github.io/eventuate/reference/event-sourcing.html#batch-replication-failure-handling). We plan to automate that in a later Eventuate release (i.e. have an internal mechanism to adjust batch/message sizes automatically).
nonysingh79
@nonysingh79
Apr 05 2016 08:51
thanks @krasserm but in genral if we have such a scenario where we want to add a new location in future after lot of events has been added on location one would there be a migration script or we still leave it to replication?
Martin Krasser
@krasserm
Apr 05 2016 08:58
@nonysingh79 we thought about internally optimizing replication for that purpose (as we ran into somewhat related issues like #236 when adding a new location). However making/installing a storage backend copy + modification script might make sense too. We need to further analyze that. Feel free to open a ticket describing your needs (or event better, make a contribution ;)). Thanks!
@nonysingh79 leaving it to replication has the advantage to remain storage-backend-independent. When two locations use the same storage backend, a more special procedure might be a much faster bootstrap.
nonysingh79
@nonysingh79
Apr 05 2016 09:04
@krasserm : yes you are right , i will look at #236 and will also see if I can make some contribution
thanks
Martin Krasser
@krasserm
Apr 05 2016 09:05
@nonysingh79 that would be great :)