Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Repo info
Activity
  • 03:56
    jeremydmiller synchronize #2053
  • 03:56
    jeremydmiller synchronize #2053
  • 03:56

    jeremydmiller on multi-tenancy

    Exposing IMartenSession.TenantI… (compare)

  • 03:56

    jeremydmiller on multi-tenancy

    Exposing IMartenSession.TenantI… (compare)

  • Jan 19 16:01
    jeremydmiller synchronize #2053
  • Jan 19 16:01
    jeremydmiller synchronize #2053
  • Jan 19 16:01

    jeremydmiller on multi-tenancy

    Bumped to 5.0.0-alpha.2 Updated to Weasel 3.0, preparat… (compare)

  • Jan 19 16:01

    jeremydmiller on multi-tenancy

    Bumped to 5.0.0-alpha.2 Updated to Weasel 3.0, preparat… (compare)

  • Jan 19 15:34
    jeremydmiller synchronize #2053
  • Jan 19 15:34
    jeremydmiller synchronize #2053
  • Jan 19 15:34

    jeremydmiller on multi-tenancy

    removing more duplicate tests w… (compare)

  • Jan 19 15:34

    jeremydmiller on multi-tenancy

    removing more duplicate tests w… (compare)

  • Jan 19 15:14
    jeremydmiller synchronize #2053
  • Jan 19 15:14
    jeremydmiller synchronize #2053
  • Jan 19 15:14

    jeremydmiller on multi-tenancy

    disabling test-codegen in CI (compare)

  • Jan 19 15:14

    jeremydmiller on multi-tenancy

    disabling test-codegen in CI (compare)

  • Jan 19 14:03
    jeremydmiller synchronize #2053
  • Jan 19 14:03
    jeremydmiller synchronize #2053
  • Jan 19 14:02

    jeremydmiller on multi-tenancy

    eliminating tests that are dupl… (compare)

  • Jan 19 14:02

    jeremydmiller on multi-tenancy

    eliminating tests that are dupl… (compare)

Oskar Dudycz
@oskardudycz
In V3 ViewProjection supported such a case where you could run a query to find the multiple ids of documents you want to update (e.g. All ids of orders with selected tax id). In V4, in theory, FanOut should handle that, but that I wasn’t able to find the way to do that. Jeremy would need to answer if he wants to support that.
Jedidja
@jedidja
Ah ok, thanks.
Jedidja
@jedidja

On a slightly related note, then. If I wanted to create the CustomerDashboard "projection" manually by applying events from a Customer stream and a Tax stream (vastly simplifying things here), is there anything in Marten I can leverage?

Currently, it seems I would

  • Call FetchStream for both streams and combine them into a list of IEvent. Apply each event to CustomerDashboard
  • The Apply(IEvent @event) method in CustomerDashboard would look at the generic type argument , and cast @event.Data appropriately, then call Apply(TaxChanged @event) for example.

It seems like that second bullet point is probably already taken care of somewhere in Marten, and I think this paragraph hints at it - is that where I should continue exploring? (from https://martendb.io/documentation/events/projections/)

Aggregates Across Multiple Streams
Example coming soon, and check Jeremy's blog for a sample soon.

It's possible currently by using either a custom IProjection or using the existing aggregation capabilities with a custom IAggregateFinder<T>, where T is the projected view document type.

I'm not worried (yet) about storing the projection if that makes any difference.

Jeremy D. Miller
@jeremydmiller
@jedidja @oskardudycz In V3 you had to tell the ViewProjection how to determine the view identity in every single call to ProjectEvent(). In V4 you accomplish that through the SetIdentity() method instead, with the assumption/guidance being that you’d make that much easier by using some kind of common interface on the events to expose the view id and you’d only have to do that once as opposed to doing that event type by event type in the V3 and before ViewProjection.
@jedidja You’ve also got the “live aggregation” option through AggregateStreamAsync() that just applies the projection in memory against the current events
Jedidja
@jedidja
@jeremydmiller Got it. So the identity was still a requirement.
Oskar Dudycz
@oskardudycz
@jeremydmiller can Identity take a property with the list of ids?
Jedidja
@jedidja
AggregateStreamAsync still only uses the events from one particular stream right?
My scenario is probably wrong which is why it doesn't fit into what's available, I'm guessing :)
Jeremy D. Miller
@jeremydmiller
Oh shoot, sorry, I didn’t read things quite closely enough up there. @jedidja Today you’d have to do what you said to do the projection.
Yeah.
Oskar Dudycz
@oskardudycz
I might be wrong, but it seemed to me that it handles only single Id
Jedidja
@jedidja
Thanks for the confirmation @jeremydmiller.
Jeremy D. Miller
@jeremydmiller
What you’re describing though would work out nicely if #1680 were done. We’ve got a story in the backlog to run an aggregated projection in memory against an arbitrary query of events as a custom Linq operation, but yours truly cut that out of the V4 timeline just to get V4 done faster.
#1680 wouldn’t be a big story to do if you’re interested.
@oskardudycz Yeah, only a single id. Why would you want to use multiple ids?
The identity is just used to sort events into the right aggregates in the “slicing” part of the projection
Jedidja
@jedidja
Totally understand about wanting to get v4 out the door :) I will take a look at #1680 and see if I can help out .. happy to learn more about the internals of Marten.
Jeremy D. Miller
@jeremydmiller
@oskardudycz @jedidja The current ViewProjection does assume that whatever information you need to determine the identity of the aggregate is available in only the event itself. It admittedly doesn’t support any kind of scenario where you’d have to query into the database or another service to find the actual view identity. We could try to add that to ViewProjection, but it’d be hard to do that in a way that was actually effecient to avoid hammering the database with little calls. That was actually the goal of #1628, to allow the async projections to look ahead through events coming in and make batched queries to fetch related data instead of doing an N+1 loop event by event the way it admittedly has to today.
Or we push users for now to go down the path of a custom IProjection for scenarios like @jedidja ’s where they can easily write more efficient code for their one off scenario.
Oskar Dudycz
@oskardudycz
I think that having the ability to provide the list of ids, without being able to do custom query would be enough.
Jeremy D. Miller
@jeremydmiller
How would that be different than what it already does though? The current GetIdentity() approach is good enough to describe the id per event. Unless we allow users to supply their own IEventSlicer (but change the name to something like “grouper” maybe).
Jeremy D. Miller
@jeremydmiller
Or I guess what I’m asking is what problem are we trying to solve? I’m not arguing, just not understanding the use case yet.
Oskar Dudycz
@oskardudycz
I’ll try to prepare some sample to make it more explicit
Jeremy D. Miller
@jeremydmiller
:thumbsup:
Oskar Dudycz
@oskardudycz
I will do that with v3 so we can discuss if we can have a mapping of this case somehow
Jeremy D. Miller
@jeremydmiller
Better yet.
Onder
@ondertipioglu
@jeremydmiller For example, for products in a basket object, I have seen that we can create a snapshot table only for a basket object, am I wrong?
Jeremy D. Miller
@jeremydmiller
You can happily create a Basket document that refers to Product children if you want, but I’d think that Product would probably be its own entity and you’d model that as a Basket that has items of some sort that refer to a known Product
Onder
@ondertipioglu
In Exercise08Snapshots.cs in the EventStoreBasics.Tests project, can we add a one-to-many list for the user object (eg user languages) and keep the final version as a snapshot table? As far as I can see now, we only take snapshots for the user's table, but what if we want to get the latest versions of the user's languages?
var userSnapshot = new SnapshotToTable<User>(
databaseConnection,
@"INSERT INTO users (id, name, version) VALUES (@Id, @Name, @Version)
ON CONFLICT (id)
DO UPDATE SET name = @Name, version = @Version");
Jeremy D. Miller
@jeremydmiller
Are you sure you’re in the right Gitter room? That’s nothing related to Marten
Onder
@ondertipioglu
@jeremydmiller thank you, I found the right room thanks to you :)
Jedidja
@jedidja

Looking at #1680, just so I don't start down the wrong path, here are bunch of total newbie questions (apologies in advance) based on a random-ish walk through EventStore and friends.

  1. I see that EventStore has QueryRawEventDataOnly<T> and QueryAllRawEvents. Is QueryAllRawEventData() a new method to add?

  2. The work item title says "...against any IQueryable<IEvent>". Is that really IMartenQueryable<IEvent>?

  3. If the answer to 2 is yes, then would the implementation for this go in MartinLinqQueryable?

Happy to work on a PR for this if anyone can give a quick high-level overview of which area of the code I should poke at :) Or, if questions like this are better asked in the work item comments, I can move the discussion there. I know it's not a "big story" for @jeremydmiller but all brand-new stuff for me heh.

Jeremy D. Miller
@jeremydmiller
Let’s see:
1.) I’ve slept since then, I’d have to go rmeember the difference:)
2.) Yeah, IMartenQueryable<IEvent>
3.) I’d put it in an extension method in a new class in the Marten.Events namespace
Dariusz Firek
@DariuszFirek

Hi, I've started transition to v4 in our production project. As for now almost everything is working. I just miss one feature we had in the past and used extensively. I saw similar discussion recently here.
ViewProjection<TDoc, TId> used to have an option to apply one event to multiple documents via ProjectEvent(Func<IDocumentSession, TEvent, IList<Guid>> idSelector, (...)) or something similar. Now it's missing, and all I have is Identity<TEvent>(Func<TEvent, TId> identityFunc) which only allows to select one Id to be updated.

Our use is similar to: https://0bin.net/paste/pEbbcvtw#m6caJOCjZbQf+-UT+5iSzF3xtDe5rY8KIkmbEmYYhc/ (sorry for raw form)

How can I replicate that behaviour in v4? I wouldn't like to migrate any data since it's already on production and spans trough gigabytes of data... Just by looking at the code and JasperFx/marten#1610 (and linked issues), I see some options (I don't know if they are valid):

  1. override Task ApplyAsync(IDocumentSession session, EventPage page, CancellationToken token); and other methods from base class and provide own implementations (I see this as dangerous)
  2. Maybe something like FanOut to <Touple<Guid, TEvent>>, where Guid would be modelId but what to do later?
  3. Ditch ViewProjection as base class and go with EventProjection, like Oskar did here: https://github.com/oskardudycz/EventSourcing.NetCore/pull/38/files#diff-5246f5500eeddbee0e524da196022d2e669c5961f26f464565812515d9bbd504 but this still requires some more work to have something that was already provided by marten in the past.
  4. Create my own IProjection implementation with all the features I need
Btw old/red code in Oskar's repository is very similar to the one we have already working.
Jeremy D. Miller
@jeremydmiller
@DariuszFirek:
Jeremy D. Miller
@jeremydmiller
  1. This would fail spectacularly, so let’s just not do that:)
  2. FanOut() effectively makes a collection of objects within a single event look like events to the ViewProjection. FanOut() here would work if it had an option to pass in an IQuerySession to do your query. What it would effectively do is “explode” the incoming event into a new virtual event for each manufacturer. After that, the ViewProjection mechanics are unchanged and you’d get all the optimizations built into the V4 aggregations
  3. If you need this done fast, I’d say this is your best option
  4. Same as 3., but you’d potentially lose some of the optimizations we’ve built into the built in projections if you’re running these in the async daemon
The good news is that doing the improved FanOut() wouldn’t take very long.
Jeremy D. Miller
@jeremydmiller
@DariuszFirek See the new issue tracking this: JasperFx/marten#1784
Jedidja
@jedidja
@jeremydmiller When you get a chance, could you confirm the answer to question 1 please? :) Thank you and if you have any other pointers on #1680 I'd love to hear them.
Jeremy D. Miller
@jeremydmiller
@jedidja Gimme about an hour and I’ll write it up on #1680?
Jedidja
@jedidja
@jeremydmiller Sure, no rush - when you get a chance. Thank you! My plan is to spend a bit of time on it each day along with the prototypes I'm working on.
Jeremy D. Miller
@jeremydmiller
@jedidja Sorry, I got sucked into Zoom meetings and lost track of this. I’m going into it now...
Jeremy D. Miller
@jeremydmiller
@jedidja See if any of this helps, and don’t hesitate to just keep asking questions: JasperFx/marten#1680
Thanks for taking this on!
Jedidja
@jedidja
@jeremydmiller Will take a look now and ... yes .. there will probably be more questions :) Thanks for the initial notes!
Jeremy D. Miller
@jeremydmiller
No, thank you!