These are chat archives for atomix/atomix

5th
Mar 2016
Richard Pijnenburg
@electrical
Mar 05 2016 18:33
@jhalterman pretty busy i see ;-)
Jonathan Halterman
@jhalterman
Mar 05 2016 20:03
yea - it's a rainy day and i have a bit of free time at the moment
mostly trying to get docs completed for monday is making me push through a few little changes here and there
Richard Pijnenburg
@electrical
Mar 05 2016 20:04
hehe okay
Richard Pijnenburg
@electrical
Mar 05 2016 20:38
argh. i hate java.
have a class for some jvm info stuff. copied it and modified it for OS info. implemented in the same class where i call jvminfo and barfs
so annoying
Jonathan Halterman
@jhalterman
Mar 05 2016 20:39
the biggest benefit over the dynamics like ruby/python, which you'll come to appreciate, is that you can pretty much always follow code paths when walking through code
Richard Pijnenburg
@electrical
Mar 05 2016 20:40
ah. found it
uhg
Jonathan Halterman
@jhalterman
Mar 05 2016 20:40
whereas with python/ruby/etc, you often find yourself looking at a funciton call wondering - where does this take me
Richard Pijnenburg
@electrical
Mar 05 2016 20:40
hehe yeah indeed
Jonathan Halterman
@jhalterman
Mar 05 2016 20:40
and sometimes it's really hard to find out the answer to such a basic question, which sucks
...and is a waste of time
Richard Pijnenburg
@electrical
Mar 05 2016 20:41
very true.
been digging in some ruby code guys at my new job made
headache :-)
Richard Pijnenburg
@electrical
Mar 05 2016 21:01
@jhalterman kinda hoped there was an existing module / class that could do all the yaml file parsing and exposing functions for calling the values etc.
Also for my project i want a kind of end-to-end persistence. so i was thinking of storing the original event in raft and then push it in the pipeline. only when its been sent out successfully then it gets removed out of raft.. this way if something messes up in the pipeline ( crash or whatever ) the original message can be replayed. what do you guys think of that?
Jonathan Halterman
@jhalterman
Mar 05 2016 21:04
i just left a comment on that github issue that has a more stripped down example of how to parse YAML to an object
my preference for internal config classes is to just have public fields and not even bother with getters/setters. if it's a config class that's part of a public API, then you probably want get/set methods.
Richard Pijnenburg
@electrical
Mar 05 2016 21:05
ah i see. makes sense yeah
Jonathan Halterman
@jhalterman
Mar 05 2016 21:05
so you'd use raft as a temporary storage mechanism?
Richard Pijnenburg
@electrical
Mar 05 2016 21:06
yeah
Jonathan Halterman
@jhalterman
Mar 05 2016 21:07
it's probably not a typical use, but as long as the volume of events isn't crazy it should be fine. so basically using it like a really safe, durable, persistent queue
Richard Pijnenburg
@electrical
Mar 05 2016 21:07
Yeah. thing is i want to ensure that received events are not lost by crashes or anything
Jonathan Halterman
@jhalterman
Mar 05 2016 21:08
it's actually very appropriate, but the obvious tradeoff for that safety is the limitation on throughput. several thousand ops/second should be no problem though.
Richard Pijnenburg
@electrical
Mar 05 2016 21:08
not sure if you followed my earlier convos with @kuujo a bit but i’m trying to build a replacement for logstash :-)
Jonathan Halterman
@jhalterman
Mar 05 2016 21:08
ah i missed those. logstash stored in elasticsearch?
Richard Pijnenburg
@electrical
Mar 05 2016 21:09
well. at the moment logstash it self is a single isolated thing.. no concept of clustering or anything..Now they started their ‘clustering’ project which is a glorified storing of the config in ES so multiple nodes can read it
sort of config management replacement
which is stupid :-)
so i want to show them with some effort a real logstash cluster can be done with Raft.
Jonathan Halterman
@jhalterman
Mar 05 2016 21:10
oh yea you can ship entries to elasticsearch
that is very interesting idea
Richard Pijnenburg
@electrical
Mar 05 2016 21:11
raft is just to manage the cluster. and ensure its all stable and consistent before things are sent out
also some nodes can handle certain things so need to find something for sending data from node to node very fast
and everything has to be reliable as well :p
tried the task queue that @kuujo built but was to slow
i can give you access to the repo if you like. I’m very much a Java novice though
Jonathan Halterman
@jhalterman
Mar 05 2016 21:15
ah i'll have enough to do with atomix for the near future to have time to spend on it really :)
Richard Pijnenburg
@electrical
Mar 05 2016 21:16
hehe fair enough :-)
Jordan Halterman
@kuujo
Mar 05 2016 21:21
@electrical we have a better way to go about the logstash thing now. I'll talk to you about it later. Out with my family ATM
Richard Pijnenburg
@electrical
Mar 05 2016 21:21
@kuujo cool! talk to you later. enjoy :-)
Richard Pijnenburg
@electrical
Mar 05 2016 21:34
@jhalterman that example you commented i don’t fully understand ( lack of java know-how ) what is that MyConfig.class thing?
Richard Pijnenburg
@electrical
Mar 05 2016 21:46
Elasticsearch has a fairly complicated config part.
at least complicated for me to understand :-)
Jonathan Halterman
@jhalterman
Mar 05 2016 22:01
@kuujo probably isn't even out. he's probably hacking together his idea for the logstash problem
Richard Pijnenburg
@electrical
Mar 05 2016 22:01
haha