@Horusiath I am overloading the sql-server (localdb) and the CircuteBreaker opens. On the Dev machine the db-files are storged on a SSD and the serverside not and this makes allready the difference. How to increase the timeout times or trothle the quries of the journal?
@schepersk concerning lack of rebalancing in your example - I think, that this may be caused by the default threshold required between two nodes, which is 10 by default (so node with the highest number of shards needs to have at least 10 more shards than the node with the lowest number of them). You can change this setting with akka.cluster.sharding.least-shard-allocation-strategy.rebalance-threshold = X - I've made that on the example in core repo (lowered threshold to 3), and everything works fine.
@Horusiath from the CircuteBreaker or connectionTimeout. but i think i increased the connectionTimeout allready to 45sec. What is the default timeout from the CircuteBreaker ? i could activate the backpresure-mode in my coordinator.
i think the scheduler is breaking, i am suddenly dont get messages that are scheduled for 1sec to itself
"Context.System.Scheduler.ScheduleTellOnce(TimeSpan.FromSeconds(1), Coordinator, c, Self);" and now its stopped working, an other schedule dont fire too. And some timeouts dont fire
@Horusiath In the Journal table of one db, where soem empty pid's, the clustered prim key was not created and that created the behaivior and sql-server perf. problems. I needed to extend my backpressure algo. in my coordinator, to trigger beside MaxChildreanCount on MaxRecoveringCount too. Recovering to many PR's at the same time will overwhelm the sql-connection pool.
I know that this is relatated to a the uninteresting "sql-persistent-plugin" but what is missing is some throttle mechanismuss in the Journal itself.