Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Activity
  • Jan 30 14:49
    slashdotdash edited #169
  • Jan 30 14:48
    Freyskeyd commented #249
  • Jan 30 14:48
    Freyskeyd closed #249
  • Jan 30 10:28
    slashdotdash commented #249
  • Jan 30 09:08
    Freyskeyd commented #249
  • Jan 29 11:37
    Freyskeyd commented #187
  • Jan 29 11:34
    Freyskeyd synchronize #249
  • Jan 29 10:50
    Freyskeyd synchronize #249
  • Jan 29 10:16
    slashdotdash closed #231
  • Jan 29 10:16
    slashdotdash commented #231
  • Jan 29 09:48
    Freyskeyd commented #231
  • Jan 29 09:10
    Freyskeyd synchronize #249
  • Jan 28 15:49
    Freyskeyd edited #249
  • Jan 28 15:32
    Freyskeyd synchronize #249
  • Jan 28 15:04
    Freyskeyd opened #249
  • Jan 28 11:20
    imetallica closed #119
  • Jan 28 11:20
    imetallica commented #119
  • Jan 28 10:40
    Freyskeyd commented #119
  • Jan 28 10:40
    slashdotdash commented #184
  • Jan 28 10:39
    Freyskeyd commented #52
Ben Smith
@slashdotdash
You need to manually deserialise types such as DateTime from the persisted JSON representation. There’s an example in the docs for what you’ll need to do:
This is needed because there’s no type information stored in the event JSON.
Paolo Laurenti
@PaoloLaurenti
OK, thanks @slashdotdash
Francesco Belladonna
@Fire-Dragon-DoL

hey @slashdotdash any chance you are aware why nested maps are json-deserialized into maps of ATOMS?

I have an event which internally has a map field (strings => string). For whatever reason when decoded, the map turns into atom => string (which is a big problem since the map is made of user-input values froma csv)

    defimpl Commanded.Serialization.JsonDecoder do
      def decode(%@for{} = event) do
        %{event | prospect_id: PR.Iris.Prospect.ID.new(event.prospect_id)}
        |> IO.inspect(label: "fields")
      end
    end

# This one outputs a lot of fields, but the main problem is:
#  custom: %{"Programming Language": "Elixir", Spoken: "English"},
Francesco Belladonna
@Fire-Dragon-DoL

actually @slashdotdash I think I found the problem:

https://github.com/commanded/commanded/blob/v0.19.1/lib/commanded/serialization/json_serializer.ex#L24

This says: keys: :atoms. This has the downside of decoding nested keys to atoms. This is extremely dangerous because event data can easily contain user-provided values (in our case a mapping of csv headers to some specific values)

Instead of that approach, keys should default to strings and the "top level" keys can be decoded to atoms for map creation, but not nested keys (that's in the hands of the developer)

Francesco Belladonna
@Fire-Dragon-DoL
I filed an issue commanded/commanded#301
Ben Smith
@slashdotdash
Issue #299 covers how to pass config options to the configured serializer which would allow you to specify how keys in maps are deserialised. If you don’t feel like tackling that then I’d suggest you copy & paste the JSON serializer from Commanded into your own app and change the map key handling option to keys: :strings
Changing the default map key handling to strings would be a breaking change. I’m ok with making that change as long as it’s possible to easily override (which can’t be done without #299).
Ben Smith
@slashdotdash
In your example could you whitelist the custom fields be from the user input before copying into the event data to prevent possible atom exhaustion?
Ruben Wagner
@ouven
Hi, I'm new to commanded (and Elixir) coming from Scala/Akka. I was wondering, if I have a chance to get the current state of an aggregate without using a projection. It feels a bit elaborated, to build the same state twice if I need to connect some logics to it in the same (DDD) domain (like authorizing with a UserAccount aggregate). Is there a concept, that I was missing out? Cheers Ruben
Ben Smith
@slashdotdash
@ouven You can use the undocumented Commanded.Aggregates.Aggregate.aggregate_state/3 function to directly access the state of an aggregate.
Ruben Wagner
@ouven
@slashdotdash thank you! I really missed it out, when I was digging through the sourcees :D
Francesco Belladonna
@Fire-Dragon-DoL
@slashdotdash no, I can't. The user is free to insert any kind of string values in that map (csv headers).
Does the suggestion I gave work? A configurable JSON serializer/deserializer. That should address both the configuration capabilities (provide a module which uses Jason with different options), as well as my issue.
As I described in my comment, using keys: :strings would break decoding for all events and aggregates for any person using commanded. In the sense that it wouldn't be possible to decode any event and aggregate into a struct
Francesco Belladonna
@Fire-Dragon-DoL
and @slashdotdash the ability to configure Jason would still not solve my problem, because if I use strings, I wouldn't be able to decode any struct unless I'm able to provide the Any implementation for JasonDecoder
Ben Smith
@slashdotdash

@Fire-Dragon-DoL We want to provide a way of allowing the user to specify whether keys are converted to atoms recursively or not.

Currently we use Jason’s keys: :atoms option which converts atoms to keys in all maps. We always need the top level keys to be converted to atoms (to ensure the event can be created from the struct) but we optionally want to choose whether to apply this to children.

Maybe we could support JSON serialization options like: keys: [:atoms, recursive: true] (default) and keys: [:atoms, recursive: false] (to opt-out of converting children)
Francesco Belladonna
@Fire-Dragon-DoL
yes exactly @slashdotdash . That recursive option is not natively supported by Jason though, right?
yeah checked the doc, not natively supported
Ben Smith
@slashdotdash
No we would need to support it ourselves
Jacob Foster
@shadyendless
Hey everyone! I have a quick question. Is it possible to run tests with async: true using Commanded and its various friends (ecto projections)?
Ben Smith
@slashdotdash
It depends whether you’re testing individual modules directly (unit test) or attempting to do more of an integration test covering how components integrate.
  • Unit tests, such as when testing aggregates, event handlers, and PMs by directly calling their functions, can be run async. You might need to mock / stub any components which cause side affects (e.g. email sending from an event handler).
  • Integration test might be possible to be run async but your tests need to be written such that they don’t cause issues due to concurrency. It’s often easier to run them sync to have isolation.
Ben Smith
@slashdotdash
I use the AggregateCase ExUnit case template for testing aggregates directly and it supports async.
https://gist.github.com/slashdotdash/097661df7e04ddfc11dbcb9a14d97124
Jacob Foster
@shadyendless
Alright, that was what I was thinking. I have a similar AggregateCase setup from working through the Conduit series on LeanPub, though it looks like the one you have now is different.
Dave Vallance
@dvallance
@slashdotdash quick question.
Is there a way to shutdown a process manager instance from the error handling in the process manager error/3. It's not alway convenient for me to create an event just to be able to use it in the interested/? call to then have a {:stop, process_id} returned.
Jacob Foster
@shadyendless
I have a general architectural question. If I want to support both strong and eventual consistency, how would that look? I tried defining both strong/eventual consistency projectors for all actions but that doesn't appear to be the correct approach.
Jacob Foster
@shadyendless
My current approach is to have two separate Projectors with unique IDs listening to it and then just ignoring if there's an %Ecto.Changeset{} error, but that seems wrong since I get errors in my console all of the time from it.
Jacob Foster
@shadyendless
Nevermind, my current approach doesn't appear to work. Hmm.
Marco Milanesi
@kpanic
hi @slashdotdash ! do you know if it's possible to stream all events from the event store again, directed to one particular projection? Think about warming up a search index with a command. Thanks!
Ruben Wagner
@ouven
@kpanic there is a mix task, that resets a stream. If I got you right, this might help you.
Marco Milanesi
@kpanic
@ouven looks interesting, thanks! I want to replay the same events happened to a specific projection
Ruben Wagner
@ouven
@kpanic somehow, the mix task does not work for me, but if I call it in iex dirctly, it works:
with pid <- Commanded.Registration.whereis_name({Commanded.Event.Handler, projection_name}) do send(pid, :reset)
Then all the events for te one projection will be replayed. Hope it helps.
Marco Milanesi
@kpanic
@ouven thanks for following up! in the end I did a "trick" ;) something like EventStore.stream_forward(stream_id) |> Enum.map(&MyProjection.handle(&1))
Which suits my needs.
Sorry for the late reply!
For the record, I have an older version of commanded which does not have :reset
Jacob Foster
@shadyendless
@slashdotdash is there a way to have an error from a process manager bubble back up to the Router?
I have middleware in place that is reporting the errors back to the original caller, but I can't seem to get the errors from the process manager to be returned.
It appears as though it is stuck trying to process the job indefinitely. I can tell it to skip and drop the pending events, but that doesn't seem like what I want... Maybe I am misinterpreting how the process manager is supposed to work.
My initial assumption was that it could be used as a sort of "transaction", to ensure that all events either happened or didn't, but maybe that was a wrong assumption.
Ben Smith
@slashdotdash
A process manager is a specialized type of event handler. It has some state and its purpose is to dispatch commands when handling events.
It doesn’t provide any transactional guarantees except for “at-least-once” processing of every event.
You can use a PM to implement a saga (an eventually consistent transaction)
Errors from a PM won't be returned to the original command dispatcher.
Jacob Foster
@shadyendless

Is there a better structure I should be using in this case? My use case is as follows:

Users are allowed to register on the website with their email/password
Users can link a social account to their account after they are registered

I am wanting to combine the above two flows into one to allow users to register with a social account, meaning it would do it as a "process" which would essentially wrap the two.

Ben Smith
@slashdotdash
You could use a PM to do the above process. To report back the status to the user you could subscribe to the relevant event stream(s) looking for success/failure events.
Modeling the registration process as an aggregate might be worthwhile too.
Jacob Foster
@shadyendless
Alright, I'll explore those options. Thanks! :D
Dave Vallance
@dvallance

@slashdotdash

Is there a way to shutdown a process manager instance from the error handling in the process manager error/3. It's not alway convenient for me to create an event just to be able to use it in the interested/? call to then have a {:stop, process_id} returned.

Just re-posting this question. You might have missed it.