akka/akka-persistence-dynamodb

DynamoDBJournal for Akka Persistence

Michal Janousek

@teroxik

yeah makes sense, the naming is a bit silly

Michal Janousek

@teroxik

Nobody is really picking up in the akka/dev, so I guess we would have to wait for Johan being back to from holiday. The 1.2.0-RC1 seems good.

Corey O'Connor

@coreyoconnor

ok. I'll create a PR for the version bump if you'd like?

Johan Andrén

@johanandren

I can release a 1.2.0-RC1 now, sounds good? @teroxik @coeryoconnor

Michal Janousek

@teroxik

Yep, sounds great

Johan Andrén

@johanandren

Artifacts on their way to maven central now. I have to run now, feel free to announce over in the Akka discuss forums if you want.

Michal Janousek

@teroxik

Will do today / tomorrow.

Michal Janousek

@teroxik

Thanks for finally helping out publishing it.

Johan Andrén

@johanandren

Sorry for the long delay!

Michal Janousek

@teroxik

https://discuss.lightbend.com/t/akka-persistence-dynamodb-1-2-0-rc1-release/4841

Corey O'Connor

@coreyoconnor

thanks!

feel free to ping me at coreyoconnor@gmail.com if I'm blocking and not on gitter :)

Vladimir

@vladimir-lu

Hi, I have a question about atomicity. As far as I understand from the PersistentActor documentation, the persistAll method is supposed to be atomic for the whole batch. Currently, the journal implementation works around the fact that a batch write is not atomic in DynamoDB with the idx and cnt attributes. Has anyone looked into using TransactWriteItems to guarantee this atomicity?

reference: https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/transaction-apis.html

Corey O'Connor

@coreyoconnor

I don't think anybody has looked into that. Interesting idea tho. IIRC TransactWriteItems has a limit on the number of items. What would be reasonable behavior if persistAll batch exceeded that limit?

Todor Kolev

@todor-kolev

Hi, I am going over the docs atm but it's not obvious to me how to do a recovery based on an offset i.e I keep an offset in dynamo and only replay the messages where offset >= sequenceId, can somebody point me in the right direction? Is this even supported by the plugin or I need to implement it myself? Thanks!

Corey O'Connor

@coreyoconnor

Do you have an example from another storage plugin of this?

At first glance... That seems like something that would be generic to akka persistence

Michal Janousek

@teroxik

We should probably try to update on akka 2.6 at least. We can look into using transactions for simplification of the implementation of atomicity as well.

Johan Andrén

@johanandren

One thing to consider wrt upgrading Akka dependency: Since Akka is binary backwards compatible keeping it depending on 2.5 will allow it to be used with 2.6 as well, while the opposite might not be true (you could depend on some feature added in 2.6 not available in 2.5).

Michal Janousek

@teroxik

Makes sense to keep it still on 2.5, you are right.

Michal Janousek

@teroxik

Started the refactor for AWS V2, going to have first stab soon. The configuration changes are a bit painful so leaving that for later, currently updating the tests.

Corey O'Connor

@coreyoconnor

thanks!

Michal Janousek

@teroxik

Opened the WIP, so it's visible it's an WIP, will see if I wll get to it, over the weekend

Daniel Vigovszky

@vigoo

Hi! It seems like the library already supports Scala 2.13.x and cross-compilation is set up for it, but there are no published 2.13 artifacts. Is there any reason for that? Would it be possible to publish a 2.13 version of the last stable build?

Corey O'Connor

@coreyoconnor

I suspect there is no blocker for that. I'll check

@vigoo opened akka/akka-persistence-dynamodb#94

Daniel Vigovszky

@vigoo

thanks!

Corey O'Connor

@coreyoconnor

@vigoo The 1.2.0-RC2 release has published 2.13 artifacts :)

Daniel Vigovszky

@vigoo

@coreyoconnor I see, thanks!

Saumil Kapadia

@saumilsdk

Hello Everyone, I just joined this akka dynamodb gitter. Looking forward to get help from all of you :)

Saumil Kapadia

@saumilsdk

I am getting this error "java.net.URISyntaxException: Expected authority at index 8: https://"

any idea?

akka.actor.ActorInitializationException: akka://DigitalTwinSystem/system/dynamodb-journal: exception during creation

Saumil Kapadia

@saumilsdk

ignore above error. I figured it out :)

Damien RAYMOND

@Dnomyar

Hi folks,
I have been exploring a solution to have dynamo items TTL configured at the table level (journal and snapshot). #99
I read the contributing guide afterwards and realised that a discussing is advised before pull request. :grimacing:

In a nutshell, we would have some optional configuration for journal and snapshot tables to specify the TTL:

my-dynamodb-journal { // also available for snapshots tables
  dynamodb-item-ttl-config {
    field-name = "expiresAt" // in the AWS UI, we need to set the field name https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/time-to-live-ttl-how-to.html
    ttl = 30d
  }
}

With this configuration, it would add a field "expiresAt" for each journal item inserted with value now + 30d.
What do you think about this approach?

kali786516

@kali786516

Guys I am new to this github repo I have several questions is there an end to end example scala repo how to use this library

also do I need to create the tables manually in dynamodb before starting the scala akka job

Michal Janousek

@teroxik

Hi, you have to create the tables manually you can see https://github.com/akka/akka-persistence-dynamodb/tree/master/scripts. Not sure about some public project using this plugin.

kali786516

@kali786516

@teroxik I figured out

I have a question guys https://stackoverflow.com/questions/73560217/convert-akka-journal-event-columns-string-value-to-java-object , as there is no dyn read journal I need small help converting event string value from dyn table to human readble former

kali786516

@kali786516

how to read journal of data written to dynamodb I think this lightbend library doest have read journal correct ?

the data written to dyn journal is in base 64 event data format how to parse it do we have examples ?

rO0ABXNyAD9jb20uY2Fwb25lLmJhbmsuYWN0b3JzLlBlcnNpc3RlbnRCYW5rQWNjb3VudCRCYW5rQWNjb3VudENyZWF0ZWQrGoMniq0AywIAAUwAC2JhbmtBY2NvdW50dAA6TGNvbS9jYXBvbmUvYmFuay9hY3RvcnMvUGVyc2lzdGVudEJhbmtBY2NvdW50JEJhbmtBY2NvdW50O3hwc3IAOGNvbS5jYXBvbmUuYmFuay5hY3RvcnMuUGVyc2lzdGVudEJhbmtBY2NvdW50JEJhbmtBY2NvdW5011CikshX3ysCAAREAAdiYWxhbmNlTAAIY3VycmVuY3l0ABJMamF2YS9sYW5nL1N0cmluZztMAAJpZHEAfgAETAAEdXNlcnEAfgAEeHBAj0AAAAAAAHQAA0VVUnQAJDM5M2M2NmRiLTJhYmItNDEwNS04NWUyLWMwZjc3MzExMDNlM3QAB3JjYXJkaW4=

kali786516

@kali786516

ok I found a way to deserialize the object

https://stackoverflow.com/questions/73560217/convert-akka-journal-event-columns-string-value-to-java-object/73584622#73584622

work around

Dondre Lockett

@dondre

Can someone help me design DynamoDb table?

kali786516

@kali786516

Wrong place to ask but is anyone hiring for scala developers or spark scala developers?

kali786516

@kali786516

Hi All, is anyone looking for a spark scala contract role inside the USA? A company called Maxonic has an open spark scala contract position (100% remote) inside the USA if anyone is interested, please send your CV to kali.tummala@gmail.com

Where communities thrive

People

Repo info

Activity