@object I peeked at some of your issues akkadotnet/akka.net#4265 and akkadotnet/Akka.Persistence.SqlServer#190
- You may or may not want to try logging some threadpool metrics to see whether there's some form of contention going on. This helped us trobleshoot issues where a misconfigured grafana dependency was causing pool explosions when actors would start up/shut down
ThreadPool.GetAvailableThreads(out int avWt, out int avIot); ThreadPool.GetMaxThreads(out int maxWt, out int maxIot); var (workerThreads,ioThreads)= (maxWt - avWt,maxIot-avIot); //log this every second
If you can enable MVCC (i.e. Enable snapshot Isolation and default isolation level READ_COMMITTED_SNAPSHOT) that could help
Might be worth logging the SQL executed in the chunk
Mayyyybe try adding some code in the batching process so that if there's only READ commands in the batch, a transaction isn't used.
If you're feeling adventurous, you may want to try (it is technically prerelease state) the Linq2db plugin which is based on akka-persistence-jdbc. Code is the branch in akkadotnet/Akka.Persistence.Linq2Db#10 and I have it as a nuget package here. Configuring for 'compatibility mode' (Where it should be both backward and forward compatible with Existing Journal/snapshot) is exampled here. I've been running the journal in not-quite-prod (i.e. syncing data with old system) for a few months now with great success
scheduler {
implementation = "Akka.TestKit.TestScheduler, Akka.TestKit"
}
and yes we are using a standard scheduler within one our of Actors that due to the new configuration stop sending scheduled messages.
The tests that fail are testing the entire app. IOW they are simulating the execution kicked in the main but using a Testing System created via TestKit.
TestScheduler has to be manually advanced
I'm getting the error "When using remember-entities the shard id extractor must handle ShardRegion.StartEntity(id)" even though the StartEntity is handled in the message extractor. Am I missing something? Hope someone will have time to answer me ;-)
public class CustomMessageExtractor : HashCodeMessageExtractor
{
public CustomMessageExtractor() : base(maxNumberOfShards: 100) { }
public override string EntityId(object message)
{
return message switch
{
NotificationActor.Subscribe sub => sub.ConnectionId,
ShardRegion.StartEntity start => start.EntityId,
_ => null
};
}
public override string ShardId(object message)
{
if (message is NotificationActor.Subscribe sub)
{
return sub.EventName;
}
return null;
}
}
I think the example in "Remembering entities" is wrong
https://getakka.net/articles/clustering/cluster-sharding.html
string ShardId(object message) from that class. I wonder if it should be sealed...
StartEntity being handled, so users can see the difference
Is it possible to broadcast a message to all entities in the same shard? For instance, I want to send one message to all entities that live in the shard
Ummm I feel like there's a way to do that, let me think
.... I mean, I have no clue whether this works or would do what you need, but you could try an
ActorSelectionlike/system/sharding/customer/customerupdated/*<-- Maybe it should be /user/ but I think it's /system/
Thanks a lot @to11mtm
It is no longer needed because I went in a different direction but it is worth to try :-)
One more question, is it possible to configure the shard actor's supervision or even better tell akka sharding to use a custom shard actor?
Hey everyone. I started up a project using Akka.NET after a few years of doing the tutorial and some small projects. Right now, it's mostly just pulling stock data and inserting into a Timescale database. Probably a pretty common use case :)
Anyways, I was wondering what the recommended approach might be to creating many singleton actors. A common case will be a simple ETL style pipeline. So, some sort of coordinator/conductor actors specifying what to download and when, then a extractor actor, possibly an actor to handle api rate limiting, then a loader actor to insert the data into the database.
I have a few of these created, but right now I just have everything initialized in the main method and passed through the constructors, which is pretty ugly, so I want to fix these before I create too many more of these pipelines.
A few (well, lots of) questions:
- It seems like some DI packages specifically for Akka.NET recently came out. Guessing I should be using this?
- Should I create a conductor (Is there a better name for this?) actor for each pipeline? And should the messages flow through the pipeline actors directly, or should they all be relayed through the conductor? It seems like could possibly make error handling easier, but message routing might be a little more awkward.
- Should I define the pipelines in code, or in the database and create them at runtime?
- For situations where I am loading data into a table, I will have a loader for each table working serially. Should I use async/await on the database call, or PipeTo? Since I want to do this serially (at least for now, since I am just using a single staging table truncated between requests and repopulated with postgres copy command), I'm thinking I should use async/await.
- How much do I have to worry about message size if I am using a single machine on my home server? (Ryzen 3600 6 cores, 12 threads, 32 GB ECC ram, SDDs). Right now, I am putting entire files/streams in messages to load with the copy command, which is a efficient way to load data into postgres.
Sorry for the wall of text of questions.
Akka.FSharp still a thing?