These are chat archives for atomix/atomix

7th
Nov 2017
Johno Crawford
@johnou
Nov 07 2017 19:22
@kuujo gz, what was the problem in the end that you were seeing in prod?
noticed that primitives are in master too :smile:
Johno Crawford
@johnou
Nov 07 2017 19:32
i'm also curious as to why you implemented the memory management instead of using say, netty-buffer
Jordan Halterman
@kuujo
Nov 07 2017 20:24
Yeah, most of the primitives are now in master. Partitioning, cluster management, messaging will all be done today. And then I'll be working on an HTTP API as well. Most of this work is being moved over from ONOS, but the HTTP API is a new addition. It will take some time.
Johno Crawford
@johnou
Nov 07 2017 20:25
http api?
Jordan Halterman
@kuujo
Nov 07 2017 20:27
The recent changes to the Raft implementation are all mostly around compacting logs. They stemmed originally from seeing significant drops in throughput when taking snapshots, but testing changes led to discovering related bugs in log compaction. The log compaction algorithm is the newest part of the Raft implementation, so it's taken some time to stabilize. I think there's still one more round of testing PRs to go before it's stable. That will be done in the next couple weeks.
HTTP API for primitives.... doesn't exist yet but it will soon.
Johno Crawford
@johnou
Nov 07 2017 20:30
but in what sense? /endpoint/foo/incrementAndGet?
Jordan Halterman
@kuujo
Nov 07 2017 20:31
The reason Atomix has its own Buffer abstraction is just because it's an abstraction over disk, memory mapped files, and memory. It's mostly just used to make the three interchangeable in Raft logs/snapshots/etc. Those utilities were previously in Catalyst and were written years ago, and the memory ones now mostly just wrap ByteBuffer
Yeah
Johno Crawford
@johnou
Nov 07 2017 20:31
ah okay, makes sense
Jordan Halterman
@kuujo
Nov 07 2017 20:32
An HTTP API that allows Atomix to be deployed as a standalone external cluster with a high level HTTP primitive API - locks, counters, leader elections, maps, sets, etc but via HTTP.
Also makes it easier to write external utilities that interact with an Atomix cluster whether it's embedded or not
Johno Crawford
@johnou
Nov 07 2017 20:39
keen as mustard in seeing how messaging is implemented and seeing how that performs in comparison to jgroups for Orbit
looking at the atomix api though it's almost like it could be used as a drop in replacement for Orbit with a bit of work
Johno Crawford
@johnou
Nov 07 2017 20:44
you essentially have a virtual actor that any "client" jvm can invoke, it is instantiated somewhere in the cluster and invocations may return a completablefuture with an arbitary serializable object
i don't think the atomix model supports reentrant invocations though right? (different requests may be freely interleaved)
Jordan Halterman
@kuujo
Nov 07 2017 20:48
Right
Jonathan Halterman
@jhalterman
Nov 07 2017 21:59
This would have been transactions?
Workaround is define your state machine and operations such that they don't need to be combined
Johno Crawford
@johnou
Nov 07 2017 22:30
@jhalterman not quite, consider i return a completablefuture from workqueue with object result using method foo
while foo is being invoked, it cannot be invoked a second time until the first invocation is complete
but if interleaving was enabled, while calculating that result in another future foo can be invoked a second time
probably better explained here