Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Repo info
Activity
  • 09:50
    simbadltd commented #1879
  • 09:50
    simbadltd commented #1879
  • Nov 26 16:20
    DJBMASTER commented #1926
  • Nov 26 16:20
    DJBMASTER commented #1926
  • Nov 26 12:04
    Leh2 commented #1879
  • Nov 26 12:04
    Leh2 commented #1879
  • Nov 25 14:09
    mronnblom-cint opened #1994
  • Nov 25 14:09
    mronnblom-cint opened #1994
  • Nov 24 23:18
    jeremydmiller commented #1989
  • Nov 24 23:18
    jeremydmiller commented #1989
  • Nov 24 23:08
    jeremydmiller commented #1993
  • Nov 24 23:08
    jeremydmiller commented #1993
  • Nov 24 23:04
    BusschaertTanguy commented #1993
  • Nov 24 23:04
    BusschaertTanguy commented #1993
  • Nov 24 23:02
    jeremydmiller commented #1993
  • Nov 24 23:02
    jeremydmiller commented #1993
  • Nov 24 22:58
    BusschaertTanguy commented #1993
  • Nov 24 22:58
    BusschaertTanguy commented #1993
  • Nov 24 22:53
    jeremydmiller commented #1993
  • Nov 24 22:53
    jeremydmiller commented #1993
Jeremy D. Miller
@jeremydmiller
Yeah.
Oskar Dudycz
@oskardudycz
I might be wrong, but it seemed to me that it handles only single Id
Jedidja
@jedidja
Thanks for the confirmation @jeremydmiller.
Jeremy D. Miller
@jeremydmiller
What you’re describing though would work out nicely if #1680 were done. We’ve got a story in the backlog to run an aggregated projection in memory against an arbitrary query of events as a custom Linq operation, but yours truly cut that out of the V4 timeline just to get V4 done faster.
#1680 wouldn’t be a big story to do if you’re interested.
@oskardudycz Yeah, only a single id. Why would you want to use multiple ids?
The identity is just used to sort events into the right aggregates in the “slicing” part of the projection
Jedidja
@jedidja
Totally understand about wanting to get v4 out the door :) I will take a look at #1680 and see if I can help out .. happy to learn more about the internals of Marten.
Jeremy D. Miller
@jeremydmiller
@oskardudycz @jedidja The current ViewProjection does assume that whatever information you need to determine the identity of the aggregate is available in only the event itself. It admittedly doesn’t support any kind of scenario where you’d have to query into the database or another service to find the actual view identity. We could try to add that to ViewProjection, but it’d be hard to do that in a way that was actually effecient to avoid hammering the database with little calls. That was actually the goal of #1628, to allow the async projections to look ahead through events coming in and make batched queries to fetch related data instead of doing an N+1 loop event by event the way it admittedly has to today.
Or we push users for now to go down the path of a custom IProjection for scenarios like @jedidja ’s where they can easily write more efficient code for their one off scenario.
Oskar Dudycz
@oskardudycz
I think that having the ability to provide the list of ids, without being able to do custom query would be enough.
Jeremy D. Miller
@jeremydmiller
How would that be different than what it already does though? The current GetIdentity() approach is good enough to describe the id per event. Unless we allow users to supply their own IEventSlicer (but change the name to something like “grouper” maybe).
Jeremy D. Miller
@jeremydmiller
Or I guess what I’m asking is what problem are we trying to solve? I’m not arguing, just not understanding the use case yet.
Oskar Dudycz
@oskardudycz
I’ll try to prepare some sample to make it more explicit
Jeremy D. Miller
@jeremydmiller
:thumbsup:
Oskar Dudycz
@oskardudycz
I will do that with v3 so we can discuss if we can have a mapping of this case somehow
Jeremy D. Miller
@jeremydmiller
Better yet.
Onder
@ondertipioglu
@jeremydmiller For example, for products in a basket object, I have seen that we can create a snapshot table only for a basket object, am I wrong?
Jeremy D. Miller
@jeremydmiller
You can happily create a Basket document that refers to Product children if you want, but I’d think that Product would probably be its own entity and you’d model that as a Basket that has items of some sort that refer to a known Product
Onder
@ondertipioglu
In Exercise08Snapshots.cs in the EventStoreBasics.Tests project, can we add a one-to-many list for the user object (eg user languages) and keep the final version as a snapshot table? As far as I can see now, we only take snapshots for the user's table, but what if we want to get the latest versions of the user's languages?
var userSnapshot = new SnapshotToTable<User>(
databaseConnection,
@"INSERT INTO users (id, name, version) VALUES (@Id, @Name, @Version)
ON CONFLICT (id)
DO UPDATE SET name = @Name, version = @Version");
Jeremy D. Miller
@jeremydmiller
Are you sure you’re in the right Gitter room? That’s nothing related to Marten
Onder
@ondertipioglu
@jeremydmiller thank you, I found the right room thanks to you :)
Jedidja
@jedidja

Looking at #1680, just so I don't start down the wrong path, here are bunch of total newbie questions (apologies in advance) based on a random-ish walk through EventStore and friends.

  1. I see that EventStore has QueryRawEventDataOnly<T> and QueryAllRawEvents. Is QueryAllRawEventData() a new method to add?

  2. The work item title says "...against any IQueryable<IEvent>". Is that really IMartenQueryable<IEvent>?

  3. If the answer to 2 is yes, then would the implementation for this go in MartinLinqQueryable?

Happy to work on a PR for this if anyone can give a quick high-level overview of which area of the code I should poke at :) Or, if questions like this are better asked in the work item comments, I can move the discussion there. I know it's not a "big story" for @jeremydmiller but all brand-new stuff for me heh.

Jeremy D. Miller
@jeremydmiller
Let’s see:
1.) I’ve slept since then, I’d have to go rmeember the difference:)
2.) Yeah, IMartenQueryable<IEvent>
3.) I’d put it in an extension method in a new class in the Marten.Events namespace
Dariusz Firek
@DariuszFirek

Hi, I've started transition to v4 in our production project. As for now almost everything is working. I just miss one feature we had in the past and used extensively. I saw similar discussion recently here.
ViewProjection<TDoc, TId> used to have an option to apply one event to multiple documents via ProjectEvent(Func<IDocumentSession, TEvent, IList<Guid>> idSelector, (...)) or something similar. Now it's missing, and all I have is Identity<TEvent>(Func<TEvent, TId> identityFunc) which only allows to select one Id to be updated.

Our use is similar to: https://0bin.net/paste/pEbbcvtw#m6caJOCjZbQf+-UT+5iSzF3xtDe5rY8KIkmbEmYYhc/ (sorry for raw form)

How can I replicate that behaviour in v4? I wouldn't like to migrate any data since it's already on production and spans trough gigabytes of data... Just by looking at the code and JasperFx/marten#1610 (and linked issues), I see some options (I don't know if they are valid):

  1. override Task ApplyAsync(IDocumentSession session, EventPage page, CancellationToken token); and other methods from base class and provide own implementations (I see this as dangerous)
  2. Maybe something like FanOut to <Touple<Guid, TEvent>>, where Guid would be modelId but what to do later?
  3. Ditch ViewProjection as base class and go with EventProjection, like Oskar did here: https://github.com/oskardudycz/EventSourcing.NetCore/pull/38/files#diff-5246f5500eeddbee0e524da196022d2e669c5961f26f464565812515d9bbd504 but this still requires some more work to have something that was already provided by marten in the past.
  4. Create my own IProjection implementation with all the features I need
Btw old/red code in Oskar's repository is very similar to the one we have already working.
Jeremy D. Miller
@jeremydmiller
@DariuszFirek:
Jeremy D. Miller
@jeremydmiller
  1. This would fail spectacularly, so let’s just not do that:)
  2. FanOut() effectively makes a collection of objects within a single event look like events to the ViewProjection. FanOut() here would work if it had an option to pass in an IQuerySession to do your query. What it would effectively do is “explode” the incoming event into a new virtual event for each manufacturer. After that, the ViewProjection mechanics are unchanged and you’d get all the optimizations built into the V4 aggregations
  3. If you need this done fast, I’d say this is your best option
  4. Same as 3., but you’d potentially lose some of the optimizations we’ve built into the built in projections if you’re running these in the async daemon
The good news is that doing the improved FanOut() wouldn’t take very long.
Jeremy D. Miller
@jeremydmiller
@DariuszFirek See the new issue tracking this: JasperFx/marten#1784
Jedidja
@jedidja
@jeremydmiller When you get a chance, could you confirm the answer to question 1 please? :) Thank you and if you have any other pointers on #1680 I'd love to hear them.
Jeremy D. Miller
@jeremydmiller
@jedidja Gimme about an hour and I’ll write it up on #1680?
Jedidja
@jedidja
@jeremydmiller Sure, no rush - when you get a chance. Thank you! My plan is to spend a bit of time on it each day along with the prototypes I'm working on.
Jeremy D. Miller
@jeremydmiller
@jedidja Sorry, I got sucked into Zoom meetings and lost track of this. I’m going into it now...
Jeremy D. Miller
@jeremydmiller
@jedidja See if any of this helps, and don’t hesitate to just keep asking questions: JasperFx/marten#1680
Thanks for taking this on!
Jedidja
@jedidja
@jeremydmiller Will take a look now and ... yes .. there will probably be more questions :) Thanks for the initial notes!
Jeremy D. Miller
@jeremydmiller
No, thank you!
Jedidja
@jedidja
:)
Jedidja
@jedidja

@jeremydmiller I think I understand what AggregateStreamAsync() is doing, and it seems somewhat straightforward to create the AggregateTo method if it were in EventStore. But as an extension method, I'm a bit lost.

Also, I may have gone down a rabbit hole via ProjectionCollection to AggregateProjection.Codegen.cs ... eep. Pretty sure you weren't suggesting I duplicate any of that :)

[1] As a starting point, is this vaguely what you were suggesting? Questions in comments as well.

public static class EventStoreExtensions
{
    public static T AggregateTo<T>(this IMartenQueryable<IEvent> queryable) where T : class
    {
        var events = queryable.ToList();

        // Should I be getting this from ProjectionCollection instead somehow?
        var projection = new AggregateProjection<T>();

        // Seems like I can create an aggregator in a couple ways ...
        var aggregatorOne = projection.Build(new StoreOptions());
        var aggregatorTwo = projection.BuildLiveAggregator();

        // ... but there's no session available for Build()
        // and I think we want to reuse this to call Apply.
        var aggregate = aggregatorOne.Build(events, session, null);

        return aggregate;
    }
}

[2] Where would be the appropriate place to add the first test? Much happier when I can see tests fail and then work on the code :)

Jeremy D. Miller
@jeremydmiller
Yeah, no:) You’ll need to “find” the pre-registered AggregateProjection<T> the exact same way that AggregateStreamAsync() does. You need the dynamic code stuff to be memoized because it’s expensive to build. I think the tests would go in their own code file just under /Events. Just give it a descriptive name like `AggregateTo_Linq_Operator_Tests” or some such thing.
And if it comes down to it, just make it work w/ backing tests and it’s no big deal for me to deal w/ optimizations later
And yeah, that’s very close. Didn’t mean to make it sound like you were too far off other than don’t build up the AggregateProjection<T> yourself and do the memoization.
Jedidja
@jedidja

You’ll need to “find” the pre-registered AggregateProjection<T> the exact same way that AggregateStreamAsync() does

I'm definitely missing something. Presumably there's no "global" way to get to the ProjectionCollection that holds the pre-registered projections? It appears to live in EventGraph but I'm not sure how to access that in an extension method. AggregateStreamAsync has access to DocumentStorewhich leads to Events which leads to Projections but all I've got is an enumeration of IEvent :)

(From AggregateStreamAsync)
            var aggregator = _store.Events.Projections.AggregatorFor<T>();
            var aggregate = await aggregator.BuildAsync(events, _session, state, token);
Jeremy D. Miller
@jeremydmiller
You’ve got an IMartenQueryable<IEvent> when you trigger this off. One second while I load the code
Jedidja
@jedidja
Hmm. Sorry I must have overlooked something in the interface.
Jeremy D. Miller
@jeremydmiller
Okay, this is gonna be a little fugly. We can add an internal getter property for the current IMartenSession in MartenQueryable<T>. In your extension method, upcast to the concrete type, grab its IMartenSession, which gets you Options.Events.Projections.AggregatorFor<T>() and pretend for now that neither of us has ever heard of the Law of Demeter.