Support chat for http://getprooph.org/ - official announcement: https://www.sasaprolic.com/2018/08/the-future-of-prooph-components.html
๐ The new #proophboard wiki is online ๐
๐ https://wiki.prooph-board.com
๐๐ต๐ฎ๐ป๐ด๐ฒ๐น๐ผ๐ด
๐New layout supporting light and dark mode as well as different font sizes
๐งญ๏ธ New sidebar with better structure and much easier navigation
๐กMore content incl. #eventstorming basics + guideline
๐ Please have a look and tell me what you think. Feedback is very much appreciated.
2 replies
Good afternoon, our application is currently implemented using the PostgresSingleStreamStrategy.
So far we have around 15k players and in the Player stream we have around 8 million events (related to the 15k players)
When we have a huge amount of events at the same time (sometimes because of huge backlog), we find that the query that loads the actual player aggregate is taking much more time (around 1/2 second) that in normal scenarios.
SELECT * FROM "_889c6853a117aca83ef9d6523335dc065213ae86" WHERE metadata->>? = $1 AND metadata->>? = $2 AND CAST(metadata->>? AS INT) > $3 AND no >= $4 ORDER BY no ASC LIMIT $5Technically its not an issue per-se for our product as its still fast, but we were wondering if moving to a different strategy might actually solve it and make it more future proof
So we opted to see how it would behave by moving to AggregateStreamStrategy by changing the oneStreamPerAggregate to true in the player AggregateRepository. We also did a projection to migrate the events on the into the different streams (tables) therefore ended up with 15k tables and in itself it helps now with querying the aggregate repository because each stream is much smaller since its per player
Having said that projections deteriorate drastically since they have to consume over 15k tables to be updated. We know that this could be possible as it is stated in the documentation of the AggregateStreamStrategy (http://docs.getprooph.org/event-store/implementations/pdo_event_store/variants.html#3-2-1-2-5-1)
PS we did update the projections to use fromCategory instead of fromStream
anyone has managed to overcome such scenario
@andrew.magri_gitlab Did you consider using snapshots to speed up aggregate loading?
8 million events in a single stream shouldn't be a problem.
Regarding AggregateStreamStrategy:
The idea would be to write an event store plugin that hooks into the transaction to write events (or references) also into a category stream and let projections only subscribe to the category and not the aggregate streams.
Another option is to switch to prooph/event-store v8 and use EventStoreDB ;) That would be the safest path for the future even with a lot more players and events in the system.
yes we are using snapshots. I guess on a daily basis its around 50ms to load but when we have this issue that the source is halted and we get huge batches of data it struggles a bit. kind off we find that sometimes takes half a seconds... with data of around 1 week sent together
so its reasonable but still
Another option is to switch to prooph/event-store v8 and use EventStoreDB ;) That would be the safest path for the future even with a lot more players and events in the system
so with EventStoreDB I would have better performance for AggregateStreamStrategy
we are currently using prooph/event-store v7.5.6
Every service has it's own local event store and then there is a shared event store with each service having a public stream in that shared store. There was some trouble with projections consuming too much resources but this was mainly caused by the set up of the docker containers and K8s cluster.
For prooph board we use AWS SQS as an event-driven transport.
DS-UUID1[xxxx]
Alternatively you can query the event store directly from your service too
For our client this was more of a problem and one reason they decided to use persistent public streams. A team can set up a new service without coordinating that activity with other teams. They just consume all public streams they need and that's it. Definitely a huge advantage, but it's not for free.
Also all teams use prooph board to design their services and document the events available in the public streams.
They have bi-weekly architecture meetings with all team leads on prooph board to discuss public event changes in the streams: like new event versions, new available event types, new consumers and sometimes even new stream versions.
They found a very good way (with our help ;)) to handle their architecture, but without proper collaboration and documentation you can end up in a mess.
https://github.com/prooph/pdo-event-store/releases/tag/v1.14.0
1 reply
Hey, I'm thinking about a revival of prooph/event-sourcing but in a totally new way and it would be nice to get some feedback regarding the idea:
@sandrokeil had the idea to create a plugin for Cody (https://youtu.be/0FAgsPNqUV4) that generates framework agnostic domain model code: aggregates, value objects, messages, handlers, ... from a prooph board event map.
One reason to abandon prooph/event-sourcing was that it creates a framework dependency in your business logic. Something you don't really want. So instead you should use the package as inspiration only. But what if you could have a code generator, that generates all the glue code for you in the namespace of your choice?
- Value Objects
- Event Sourced Aggregates
- Commands
- Events
- Repository Interface
One can then connect the result with the framework of their choice. We think about prooph/event-store and Ecotone packages that can be used in combination to also generate framework related glue code (e.g. repository implementation, PHP8 attributes for Ecotone, ...)
Would you be interested in such a package?
Let's try it like this:
Usually developers are lazy. They try to automate repeating tasks and make their life easier. That's the key idea behind Cody as well.
Once you've found rules and conventions how to organize your code base, you'll do the same coding over and over again:
Create this class here, configure message routing there, set up an API endpoint, map data to VOs, define a contract (e.g. OpenAPI/AsyncAPI files) and so on. Isn't that boring?
Why not automate the boring parts? Writing the automation logic is a purely technical and depending on how crazy you want it to be also a challenging task. Something developers usually like. If you write and maintain the logic yourself would you still say it's magic?
@darrylhein good point. The context is super important. For companies with multiple teams all working with the same basic set up and maintaining software over a long period of time, framework decoupling is much more important than for smaller single team projects. A framework often influence the way how problems are solved in code. what works good today, can become a huge problem tomorrow when you want to scale (technically and in teams size)
Another good reason for decoupling is much cleaner business logic that is easier to understand and change over time.
But a framework agnostic domain model means:
more glue code required to make it work with your current framework of choice. But if you automate glue code generation as well as parts of the domain model code itself then you have a win-win situation.
if it is possible to have a generator that can update the code with my changes mixed in later, I'd probably be more interested
that's actually the case. It works like rector or a code sniffer. In rector you have those rules to upgrade parts of the code to a new PHP version. It also modifies your code without destroying it. Cody can do the same. If a VO exists already, Cody does not touch it! But if you want to add a new interface and/or method to all VOs, Cody can easily do that while keeping the original implementation in place.
we are maintaining a single low-time project (ever since starting to use prooph :)), so no need to upgrade multiple things, just making sure things keep on running / evolve with least effort :) ( Also, no hostility meant, I am being honest but not judging, there absolutely is a value of what you suggest)
when I talk about magic, I mean magic in code. I like stupid code. I like obvious code. I spent way too much time of my life debugging / figuring out why something magically does not work (any kind of magic routing / loading / ... ). I prefer repetitive over magic.
When we (team) implement new features, the basic setup is the stuff that takes the least time. Setting up read models / projections is what takes the most time (although repetitive but way more time consuming). That's the place where I personally would like to cut effort (esp since we use json functions and these are a PITA).