I found in our logs the following warning:
Configured Total of Connection timeout (15 seconds) and Command timeout (30 seconds) is less than or equal to Circuit breaker timeout (10 seconds). This may cause unintended write failures
I think two things need to be corrected here.
- The warning should say "greater than Circuit breaker timeout"
- Since it is recommended to set Circuit breaker timeout to the sum of those two, the condition in the source code should trigger the warning only if the sum is greater, not greater or equal.
@Aaronontheweb This is a minor detail but can be confusing. If you agree with my interpretation, I can send a PR.
I have a question about akka monitoring and application insight. I have followed the wiki page, installed the required packages, added some code on my actors to send metrics to AI and everything works fine, I can see my data on the metrics of Application Insight. The main issue is that the throughput of the application is more than halved! In the same application I have also Serilog sending logs to the same instance of Application Insight. Could it be a problem? I have tried to search for any issue but I did not find anything. Can somebody help me ? Thank you!
@object I peeked at some of your issues akkadotnet/akka.net#4265 and akkadotnet/Akka.Persistence.SqlServer#190
- You may or may not want to try logging some threadpool metrics to see whether there's some form of contention going on. This helped us trobleshoot issues where a misconfigured grafana dependency was causing pool explosions when actors would start up/shut down
ThreadPool.GetAvailableThreads(out int avWt, out int avIot); ThreadPool.GetMaxThreads(out int maxWt, out int maxIot); var (workerThreads,ioThreads)= (maxWt - avWt,maxIot-avIot); //log this every second
If you can enable MVCC (i.e. Enable snapshot Isolation and default isolation level READ_COMMITTED_SNAPSHOT) that could help
Might be worth logging the SQL executed in the chunk
Mayyyybe try adding some code in the batching process so that if there's only READ commands in the batch, a transaction isn't used.
If you're feeling adventurous, you may want to try (it is technically prerelease state) the Linq2db plugin which is based on akka-persistence-jdbc. Code is the branch in akkadotnet/Akka.Persistence.Linq2Db#10 and I have it as a nuget package here. Configuring for 'compatibility mode' (Where it should be both backward and forward compatible with Existing Journal/snapshot) is exampled here. I've been running the journal in not-quite-prod (i.e. syncing data with old system) for a few months now with great success
scheduler {
implementation = "Akka.TestKit.TestScheduler, Akka.TestKit"
}
and yes we are using a standard scheduler within one our of Actors that due to the new configuration stop sending scheduled messages.
The tests that fail are testing the entire app. IOW they are simulating the execution kicked in the main but using a Testing System created via TestKit.
TestScheduler has to be manually advanced
I'm getting the error "When using remember-entities the shard id extractor must handle ShardRegion.StartEntity(id)" even though the StartEntity is handled in the message extractor. Am I missing something? Hope someone will have time to answer me ;-)
public class CustomMessageExtractor : HashCodeMessageExtractor
{
public CustomMessageExtractor() : base(maxNumberOfShards: 100) { }
public override string EntityId(object message)
{
return message switch
{
NotificationActor.Subscribe sub => sub.ConnectionId,
ShardRegion.StartEntity start => start.EntityId,
_ => null
};
}
public override string ShardId(object message)
{
if (message is NotificationActor.Subscribe sub)
{
return sub.EventName;
}
return null;
}
}
I think the example in "Remembering entities" is wrong
https://getakka.net/articles/clustering/cluster-sharding.html
string ShardId(object message) from that class. I wonder if it should be sealed...
StartEntity being handled, so users can see the difference
Is it possible to broadcast a message to all entities in the same shard? For instance, I want to send one message to all entities that live in the shard
Ummm I feel like there's a way to do that, let me think