For more context, I want to deploy several modules on an akka cluster. Some of them use aecor and some don't, and I'm considering using domain events for inter-service integration. The published events should be persistent for a reasonal amount of time for failover. Kinesis is the most viable option for me because I can't afford MSK and don't want to manage a Kafka cluster manually.
@notxcain Hi! I spent three more months with aecor and now I got more understanding about it. It was a lot of fun :)
Today's question: How long does it take usually for you to deploy a new processs for DistributedProcessing? We're deploying a 5-node akka cluster on Fargate, and unfortunately it takes almost 20 minutes to finish the rolling update despite of its small size. It goes like this:
- New node submits the newly added process to the ShardCoordinator
- The ShardCoordinator attempts to register the process with one of the existing nodes but fails
- The whole node goes down
- Repeat 1~3 until enough new nodes join to the cluster
Could you give me some advice on redeploying an aecor cluster?
@falconepl There are many entries like this from
akka.cluster.sharding.ShardRegion:FeeInquiryExpirationProcess: Trying to register to coordinator at [ActorSelection[Anchor(akka://diamond@10.40.20.22:2552/), Path(/system/sharding/FeeInquiryExpirationProcessCoordinator/singleton/coordinator)]], but no acknowledgement. Total [108] buffered messages. [Coordinator [Member(address = akka://diamond@10.40.20.22:2552, status = Up)] is reachable.]
I'm not an expert, but what's your current sharding configuration?
trying out aecor, based on https://github.com/vpavkin/ticket-booking-aecor, and for some reason i'm not seeing anything being written to the db
tables are created
can curl a new entry and I get an appropriate response
but i see nothing written to the db
no errors in the logs either, unless i deliberately send malformed json
both show the tables as empty
@falconepl yeah the issue is with my codebase but its based on the ticket booking demo and really only involves so far some minor renaming. when i get a chance i'm going to re-test with the ticket booking codebase itself. i thought that maybe i had messed up the protobuf serialization but a unit test showed that it was working as intended. my guess is that something i did is generating an exception that is being swallowed.
Were you able to observe those missing writes also with vanilla ticket-booking-aecor demo or only with your own project?
the ticket booking demo works just fine, btw
here is the code in question: ```
def reserve(client: ClientId, records: NonEmptyList[LnsRecord], expiresAt: Instant): I[Unit] = {
for {
now <- currentTime
x <- OptionT(read).cata(
if (records.distinct =!= records) reject(DuplicateRecords)
else if (records.size > 4) reject(TooManyRecords)
else if (expiresAt.isBefore(now)) reject(InvalidExpirationDate)
else append(PointCodeReserved(client, records, expiresAt)),
_ => reject(PointCodeAlreadyExists),
)
} yield x
}
this is my version of
EventsourcedBooking.place
starting out i'm oddly able to re-use nearly everything in the ticket booking example, so i've had to make very few changes
the only real difference is that i now need the current time
the issue appears to be related to
OptionT(read).cata
i'm not sure why, but if i remove that part, and just pattern match on the result then i get everything working as normal
if not, then i get nothing written in the database
my guess is that the Functor for
I that cata needs is some how messed up
mapping when i should have been flatMapping, which was hidden by the any to Unit + turning off the value discard warning
plus,
cata can't do what i want
figured the issue out by unrolling what
cata does
Yeah,
-Ywarn-value-discard is super helpful, especially when used along with -Xfatal-warnings
Saved me many times. Debugging value discard related bugs can get pretty nasty sometimes
in general i like the auto-unit conversion that scala does
i do wonder though for when you want effects if it wouldn't make sense to have a separate placeholder, like say Akka does with
NotUsed I think it is, for when you really intend to have no action
an ergonomics + bike shedding issue for sure
scala-steward is great, but the amount of spam it generates is unbelievable 🙂
Oh, and by the way. I'm currently working on some basic saga support for Aecor and it seems that I will need to make saga's algebra shape as
Saga[F[_], D] - and that's because it should provide some create(data: D): F[Unit] capability (every user's concrete saga implementation may operate on a different data type passed around)