Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Repo info
Activity
  • Oct 05 2017 15:32
    pawelpacana closed #8
  • Oct 05 2017 15:30
    pawelpacana closed #10
  • Oct 05 2017 15:29
    pawelpacana commented #10
  • Sep 01 2017 09:10

    pawelpacana on redesign

    (compare)

  • Aug 30 2017 17:41

    pawelpacana on aggregate-root-handle-concurrent-writes

    (compare)

  • Aug 30 2017 17:41

    pawelpacana on locking_friendly

    (compare)

  • Aug 30 2017 17:33

    pawelpacana on master

    Indicate this repository has be… (compare)

  • Aug 30 2017 07:59
    paneq commented #8
  • Aug 29 2017 16:51
    paneq commented #8
  • Aug 29 2017 16:49
    paneq closed #11
  • Aug 29 2017 16:49
    paneq commented #11
  • Aug 29 2017 16:31
    paneq commented #8
  • Aug 29 2017 14:13
    mlomnicki commented #8
  • Aug 29 2017 14:02
    paneq commented #8
  • Aug 29 2017 13:23
    mlomnicki commented #8
  • Aug 29 2017 13:06
    andrzejsliwa commented #8
  • Aug 29 2017 13:00
    paneq commented #8
  • Aug 29 2017 12:59
    paneq commented #8
  • Aug 29 2017 12:58
    paneq commented #8
  • Aug 28 2017 14:00
    paneq synchronize #11
Markus Schirp
@mbj
Just asking because its sitting there w/o comments for a while.
Andrzej Krzywda
@andrzejkrzywda
@mbj merged and thanks! sorry it took so long
Markus Schirp
@mbj
@andrzejkrzywda NP, was just curious if such PRs are okay. I try to keep the "known mutant users" in the loop. To get better feedback.
@andrzejkrzywda BTW mutant-0.8 was released yesterday. You might want to "update again" ;)
Right now the set of "known mutant users" is small enough to explicitly ping people.
Andrzej Krzywda
@andrzejkrzywda
@lessless yes, all kinds of feeds may be a good fit for the event-driven approaches. you may want to consider also the concept of read models for easier displaying of the feed - http://blog.arkency.com/2015/05/introducing-read-models-in-your-legacy-application/
Andrzej Krzywda
@andrzejkrzywda
@mbj the level of "customer support" mutant represents is fucking awesome! thanks :)
Markus Schirp
@mbj
@andrzejkrzywda Its just to optimize my own work. The more feedback with good signal / noise ratio I get the better I can do that tool for my own commercial activities.
@andrzejkrzywda That basically means your signal / noise ratio is positive ;)
Yevhenii Kurtov
@lessless
@andrzejkrzywda your blog is awesome
I think that Poland has most prominent DDD-adapts out there
Andrzej Krzywda
@andrzejkrzywda
@lessless thanks :)
Pierre-Louis Gottfrois
@gottfrois
@andrzejkrzywda would be great to see a blog post about sagas and some ruby code example. Basically I need to call other commands (to generate new events) once my TaskCompleted event has been applied to my aggregate. But I read that aggregates should basically not talk to the outside world nor generates new events, therefor sagas if I understood their role correctly. Would love to have your thoughts on this.
Andrzej Krzywda
@andrzejkrzywda
@gottfrois what is your requirement example exactly? What happens after TaskCompleted?
Pierre-Louis Gottfrois
@gottfrois
@andrzejkrzywda so when a task is completed, I need to do some business checks in order to determine if a new task should be automatically created or not. Basically it's a graph of tasks. I start with one node and when it's completed, I ask the system whether or not it should stop their or create a new task. So my first guess is to add a subscriber that would watch for TaskCompleted events, do the business check and then publish a new TaskCreated event if necessary. And from http://www.cqrs.nu/Faq/sagas this sounds like the definition of Sagas "An independent component that reacts to domain events in a cross-aggregate, eventually consistent manner."
Andrzej Krzywda
@andrzejkrzywda
yeah, it may sound like a proper place for the Saga pattern - @mpraglowski may be able to help here
Mirosław Pragłowski
@mpraglowski

@gottfrois my understanding of CQRS & Event Sourcing is:

  • there are a few sources of commands (the message that initiate some process): user (UI), actor (external system), saga (internal process)
  • aggregates must protect its invariants, therefore they should not talk to other aggregates or other systems - instead they publish domain events (and BTW any change in aggregate state is done ONLY via applying domain event)
  • the only sources of domain events are: aggregates & adapters for external systems
  • saga is a process (with its internal state but the state here is only a process state not the state of any aggregate) that reacts to domain events and when some state of a process will be reached it sends a command to the system initiating some action (handled by some aggregate). BTW another name of saga is "process manager" (and this seems to fit your example)

let's try to apply this to your example:
We have a graph of tasks, after each task completion the Task publishes domain event TaskCompleted with some attributes that allow to decide if next task should be run or not.
Everything is started by first command (initiated by a user?) StartProcess what starts the first task in a graph.
If you decide to model a graph of tasks as saga it should be created when first task of graph is started, then it could respond to events TaskCompleted and if validation will be successful send a command RunTask(X) where X is next task from graph to run. Of course, there could be more events that might have an impact on the saga (graph) state like TaskFailed, TaskTimeouted etc.

Especially TaskTimeouted might be useful. Saga could send it to itself with some defined delay Timeout when it sends command RunTask. If it gets TaskCompleted or TaskFailed before TaskTimeouted it should act as usual (no timeout), if TaskTimeouted is first saga should react to this and all next (i.e. TaskCompleted) events should be ignored.

I hope that this will clarify some things, ping me if I could help more :)

Pierre-Louis Gottfrois
@gottfrois

@mpraglowski thanks for this great explanation. So far I see CQRS & Event Sourcing the same way as you do (awesome). Some of my commands are triggered by user actions and some of them will be triggered by internal processes (saga).

The idea behind my "graph of tasks" is to explicitly create the first one (the root node) by some command trigged by a user action (UI), execute it and if the task get "completed" call an internal process to build the next task if necessary. So in my usecase, the saga is a "task manager" that determine if it reached the end of the chain or if it should append a new task. But I think you got the basic idea right and the timeout event is a good idea.

Now what I am not sure of, is how I should implement the saga. What does it looks like in terms of ruby code. As far as I understand this, I think it should be a state machine that can transition from one state to another depending on some conditions. Each transitions will trigger a specific command that will be run against an aggregate. But since all this CQRS & Event Sourcing journey is new to me and you guys seems to have more experience with it than me, it would be awesome to see a basic implementation of a saga on one of your blog posts :)

In the meantime, i'll give this a try and will come back to you.

Mirosław Pragłowski
@mpraglowski
@gottfrois do you have some sample code to show how you are using RES ? I will be happy to see how others are using RES. Or maybe would you like to prepare better sample application that the one I've published (mainly as a sample for a blog post)?
Pierre-Louis Gottfrois
@gottfrois

Yes sure, i'll be happy to put something together that could represent a subset of what i am doing. From what I have seen already, my domain model is pretty big :-/ I am not sure if it's me that is missing an intermediate object or if it's ok to have quite big domain models.

Since events need to be stored at the aggregate root level, all my user actions translate to a method in the domain model in order to trigger an event. Correct me if i'm wrong, but that can lead to having a huge domain model (in terms of number of lines). Mine is already ~300 lines long.

Also the second pain points that I have is that I had to create a lot of custom exception classes. I use them to perform business validations in my domain model. Since the domain model internals are private, it makes it impossible to extract business validations in a PolicyObject for example :-/ So I endup with code liek this:

    def assign_author_task!(assignor_id, assignee_id)
      fail AuthorTaskNotCreated if author_task.draft?
      fail AuthorTaskAlreadyCompleted if author_task.completed?
      fail AuthorTaskAlreadyAssigned if author_task.assigned?

      apply Events::Document::AuthorTask::Assigned.create(id, assignor_id, assignee_id)
    end

Whereas I would have prefer to do:

    def assign_author_task!(assignor_id, assignee_id)
      fail 'some custom class' PolicyObjects::AuthorTaskAssignable.new(self).valid?

      apply Events::Document::AuthorTask::Assigned.create(id, assignor_id, assignee_id)
    end

But then I loose the specific custom exception classes which was nice I guess. If you guys have thoughts and feedbacks on this it would be welcomed :)

Mirosław Pragłowski
@mpraglowski

I am not sure if it's me that is missing an intermediate object or if it's ok to have quite big domain models

It always depends on the domain :) maybe you should have identified some bounded contexts and split model into several ones?

Pierre-Louis Gottfrois
@gottfrois

Well the issue with splitting is that generated events will not be stored under the same uuid anymore. What I want in the end, is to be able to look at all events applied to my domain model so i can understand what happened to it, what user changed during its life cycle. So with that in mind, the only way I see to achieve this is to put everything into that domain model.

However they are things that in regular OO programing is generally a code smell which makes me having serious doubt. For example here is a subset of my "Document" domain model:

def create
  # apply event here
end
def create_author_task!
  # apply event here
end
def assign_author_task!
  # apply event here
end
def reassign_author_task!
  # apply event here
end
def complete_author_task!
  # apply event here
end
def create_qc_task!
  # apply event here
end
def assign_qc_task!
  # apply event here
end
def reassign_qc_task!
  # apply event here
end
def approve_qc_task!
  # apply event here
end

Usually when you have methods with same prefix/suffix it's a hint that you have a hidden object right? But here I really do want the applied event to be applied on the same aggregate root. Because from the business perspective, we want to follow the life cycle of that "Document", not only a particular "author_task" or "qc_task".

Mirosław Pragłowski
@mpraglowski
I do not know your domain, but this seems you want everything in one domain object to easily read event stream for it from RES. But the event stream seems to be used as an audit log. And here is a catch. I would divide it into Document & Task (at least) aggregates - probably more (BTW what is Task in the context of a document?). Then I will create a read model that will generate audit log (in any form you want it to be read).
Pierre-Louis Gottfrois
@gottfrois
Yes this is exactly it. Hum I have to admit I did not think of building the audit log using a read model. I'll give some thoughts to this. So a Document is a piece of text that needs to be translated from one language to another. The translate action is the "author_task", then admin can review the translation quality, this is the "qc_task" and finally the translation needs to be reviewed by the client who asked it, this is the "client_review_task". As you can see, each tasks can be assigned to someone (author, admin, or client depending on the task), it has a "completion" status (approved, rejected, completed). I'm going to try splitting each type of tasks in their own aggregate root to see how it fits. Thanks for the suggestion !
Mirosław Pragłowski
@mpraglowski
Ok, so for me this looks like you should have here separate aggregate for a Document, then each Task could be implemented as a Saga (Process Manager) that should handle all assignments, approvals, rejections etc and generate ApplyTranslation command when completed successfully. I don't see here a need for an QC Task - it's still part of the same process of doing translation.
Pierre-Louis Gottfrois
@gottfrois

Assignment, approvals and rejections are user actions. That's why I want to manage them using events. A QcTask is just an admin task to review if the translation is correct and fix it otherwise before sending the translation to the client for final review. While I understand that the author task and the qc task are fundamentally the same, they have different business rules and side effects that would be easy to manage if they are 2 different tasks I think.

Right now I am moving task's events outside the document domain model into their own domain models. Common task features (creation, assignation, completion) will trigger common events (TaskCreated, TaskAssigned, TaskCompleted). Specific task feature will trigger their own set of events (there are special events for the author task that does not exists on the other tasks).

I'll let you know if this solution fits better :) Thanks for the feedback by the way

Kenneth Kalmer
@kennethkalmer
Thanks for the article on Testing an Event Sourced application, was secretly hoping there would be one on implementing a saga as well by now
I'm trying to build my first saga on the ideas in http://blog.jonathanoliver.com/cqrs-sagas-with-event-sourcing-part-i-of-ii/
long story short is that I'm using a separate ES client with a separate repository for the saga's state, and saga-only events to tick the state machine along, but it doesn't feel as good as I expected
Kenneth Kalmer
@kennethkalmer
apart from it being very verbose, it appears that the state machine doesn't tick along properly, all the events are only being applied at the end of the saga so some of the guards aren't working correctly...
apologies if this is a bit vague and/or noisy, just trying to lay out my thoughts and start a discussion on how to build out sagas
szan
@szan

Hi! Thanks for great resources on Event Sourcing! I'm trying to get a grip on the concept and I was looking at your gem with sample app (https://github.com/mpraglowski/cqrs-es-sample-with-res).
I have a question:
Is there a reason why loading events by EventRepository returns RailsEventStore::EventEntity instances instead of user-created events? Those are later passed to domain objects apply_<event_name> methods and in the sample app an assumption is made that event argument here:

def apply_events_order_created(event)
      @customer_id = event.customer_id
      @number = event.order_number
      @state = :created
    end

is an Events::OrderCreated instance, as it uses its accessors.

Mirosław Pragłowski
@mpraglowski

@szan

Is there a reason why loading events by EventRepository returns RailsEventStore::EventEntity instances instead of user-created events?

because RES does not know about event classes, and I need to check it - IMHO it should return RailsEventStore::Event instead of RailsEventStore::EventEntity

Pierre-Louis Gottfrois
@gottfrois
hey @mpraglowski I rich a point where I need to deal with concurrency issues. Since we can have multiple instances of the event store (running on multiple machines for scalability), we need a way to handle 2 identical commands. One would hit event_store_service_1 and the other one would hit event_store_service_2. Since we load past events in the aggregate root to build domain's current state we might apply 2 times the event generated by the command. How is this supposed to be handled? Simply by optimistic event versions? Would love to see a concrete example on your great blog at some point. Thx!
Owen Tran
@owentran
Hi, was looking to use the event_stores_events and was thinking of leveraging a jsonb column for the metadata and data field. Did you decide to use text because it's supported more widely across databases?
Second question, was there ever a second blog post in regards to handling the events in an asynchronous manner? I imagine you would to start thinking about leveraging sidekiq/resque at that point.
And since I'm on a roll, are there any examples of using projections to keep a snapshot of the aggregate instead of building it each time?
Thanks for contributing, very clean code and helping me get a grasp of using an event store.
Mirosław Pragłowski
@mpraglowski
@owentran on "leveraging a jsonb column" - yep, we use text because of all db supports that, but it would be great to you could create PR that will allow to setup jsonb column - we could start discussion there
@owentran "regards to handling the events in an asynchronous manner" - we did that in our previous version (internal to some project) of EventStore - and since I'm still not fully happy with that solution we don't have it yet in RES. Also we wanted to start small and simple - with all things inside the same transaction. Going async is leading to some interesting issues but for sure it should be a way to go. Again I will be happy to discuss that ;)
"are there any examples of using projections to keep a snapshot of the aggregate instead of building it each time?" not yet, not supported for now
Mirosław Pragłowski
@mpraglowski

@gottfrois "Since we can have multiple instances of the event store (running on multiple machines for scalability)" - don't know if RES is the best thing to run on several machines ;) I would rather go with Greg Young's EventStore (http://geteventstore.com) and use our HttpEventStore client to connect to it (https://github.com/arkency/http_eventstore/). Also I'm not sure if I understand your problem here.

You said you have 2 commands - but this are 2 identical commands? or this is the same command send to 2 instances of ... what? the command should be send to your write model - RES has nothing to do with handling commands (yep I know AggregateRoot and all that stuff should really be moved out of RES).

Don't know if this will help but there are several concepts described in competing consumers section of GetEventStore docs http://docs.geteventstore.com/introduction/competing-consumers/

could you write more how you use and why you need 2 instances of RES ?

Owen Tran
@owentran
Thanks, @mpraglowski. Appreciate the response. I'm still figuring out how we'll leverage the event store, since full CQRS is a mighty leap for us. Keep up the great work!
Pierre-Louis Gottfrois
@gottfrois

@mpraglowski thanks for the answer. Talking about a multi-threaded (or simply load balanced) rails app running RES. Somehow 2 identical commands are issued (poor user interface that allow a user to double click on the "create order" button for example).

  • Thread 1 receive command 1. Apply command on the aggregate root.
    • Aggregate build its internals using already stored events.
    • Checks business rules (already created? no)
    • Finally ask event store to store new event.
  • Thread 2 receive command 2. Apply command on the aggregate root.
    • Aggregate build its internals using already stored events.
    • Checks business rules (already created? no)
    • Finally ask event store to store new event.

You end-up with 2 events stored instead of one thread throwing the "AlreadyCreated" domain exception defined here.

A gist to illustrate https://gist.github.com/gottfrois/98acaa73e71c616c38b6.

That's why i'm asking about the optimistic locking. Currently I don't know how you can know the previous event id when you create your command and use that previous event id to perform the optimistic locking. Any guide lines? I'm talking about implementation details, like how/where can we get the previous event id? Am I missing something? Thanks.

Pierre-Louis Gottfrois
@gottfrois
hey @mpraglowski any chance we can discuss my previous comment about concurrency anytime? :) Thanks!
Mirosław Pragłowski
@mpraglowski
@gottfrois I'm at BuildStuff - will try to answer in the evening
Pierre-Louis Gottfrois
@gottfrois
sounds great
Mirosław Pragłowski
@mpraglowski

@gottfrois it is missing a current version (last replayed event) in aggregate. It should be added to publish_event when storing aggregate state. Then you will get WrongExpectedEventVersion when trying to store event to the stream (we have a stream per aggregate) that has been changed in meantime.

It should be easy to implement - send us a PR ;) (or wait till next week when I will be back home).

Pierre-Louis Gottfrois
@gottfrois

@mpraglowski I have already tried to add the last replayed event to the gem. It seems to work at first glance but after several concurrent tests (using threads like code above) it turns out that 2 threads can "read" from the database at the exact same time and figure that the expected version they have is correct, ordering the database to "write" their events. Which cause the issue of having twice the same event stored.

Anyway, the gem is missing that last event, i'll be working on it first thing on Monday and submit a PR so that we can discuss this further with concrete code and specs :) Thanks man, enjoy your weekend.

Mirosław Pragłowski
@mpraglowski
there is a slack http://railsddd.slack.com/ some discussions on RailsEventStore are happening there, ping me here or by email to get an invite (we need it until Slack will have better invitation process)