Thanks, I will go through the comments over the weekend. Biggest question is if it's not better to get rid of the old data fields, and don't serialize the PersistentRepr - as that's same approach as in akka-persistence-cassandra is going for. And add couple of more fields into the record, support for metadata.
Got rid of PersistenRepr serialization and stored persistenceId, sequenceNumber, writerUUID and persistentRepr.manifest as separate fields. Sender is currently set to null. Is sender even used? Haven't seen it to be explicitly set in the cassandra plugin, neither seen a field for it in cassandra schema. The persistenceId and sequenceNr could be derived from the records key but I went for separate field instead.
I'll try these changes and will post any issues.
On a aside: I should really look on how to do an integration test with the GDPR plugin.. hmmm
@coreyoconnor Patrick commented on the PR, that's not used. I don't think that a integration test is required. I have added a dummy async serializer which is basically a Java serializer wrapped inside a future. That should replicate the required behaviour enough. I'm currently testing the latest changes in the implementation. Once I will have some time I will look into the performance suite and try to recover the bits which are gone because of the persistent view deprecation. Next on my list is the AWS SDK 2.0 / akka-http (alpakka-client) and persistence query but that's a bit tastier changes.
Good to know about the integration test. I agree that dummy async serializer is sufficient.
Has an appropriate version bump been considered? Only a minor version bump?
Sorry for not picking this up earlier
@teroxik Sorry, the notification for this got lost in the flood. You wouldn’t want #82 to go in a release as well?
Oh, #80 was also not merged yet, I’d guess that is the reason you want a new release, right?
I recall encountering a null pointer exception when it attempted to recover from a journal using the old schema
I just added a comment to the PR with what I recall. Unfortunately I can't find my notes.
I do run it in production, didn't notice any null pointers, but currently there is not a massive throughput through the system and there wasn't many new / old persistent entitities. So any feedback welcome. The test for backwards compatibility is there, maybe extend that one a bit. We could release it as RC at least.
I agree with releasing as an RC first. Seems like a good practice regardless.
in which case
version.sbt should be updated to 1.2.0-RC1?
Kato's implementation is at 1.0.7 and using "dynamo-db-plugin". Which was adds some complexity to all this but what's done is done .
feel free to ping me at coreyoconnor@gmail.com if I'm blocking and not on gitter :)
Hi, I have a question about atomicity. As far as I understand from the PersistentActor documentation, the
persistAll method is supposed to be atomic for the whole batch. Currently, the journal implementation works around the fact that a batch write is not atomic in DynamoDB with the idx and cnt attributes. Has anyone looked into using TransactWriteItems to guarantee this atomicity?
Hi, I am going over the docs atm but it's not obvious to me how to do a recovery based on an offset i.e I keep an offset in dynamo and only replay the messages where offset >= sequenceId, can somebody point me in the right direction? Is this even supported by the plugin or I need to implement it myself? Thanks!
At first glance... That seems like something that would be generic to akka persistence