Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Repo info
Activity
  • 17:24
    oskardudycz commented #1854
  • 17:24
    oskardudycz commented #1854
  • 15:22
    amferguson commented #1854
  • 15:22
    amferguson commented #1854
  • 14:45
    oskardudycz edited #1854
  • 14:45
    oskardudycz edited #1854
  • 14:24

    jeremydmiller on master

    removed Envelope.SentAttempts t… little cleanup on Envelope Using the DAtabaseConstants thr… and 11 more (compare)

  • 14:07
    tonykaralis commented #1854
  • 14:07
    tonykaralis commented #1854
  • 13:41

    jeremydmiller on optimization

    removed Envelope.SentAttempts t… little cleanup on Envelope Using the DAtabaseConstants thr… and 20 more (compare)

  • 13:41
    oskardudycz commented #1854
  • 13:41
    oskardudycz commented #1854
  • 13:16
    willhausman opened #1939
  • 13:16
    willhausman opened #1939
  • 13:13
    oskardudycz commented #1854
  • 13:13
    oskardudycz commented #1854
  • 13:12

    oskardudycz on pk_migration

    (compare)

  • 13:12

    oskardudycz on pk_migration

    (compare)

  • 13:12

    oskardudycz on master

    Used Weasel 1.0.3 with a fix fo… (compare)

  • 13:12

    oskardudycz on master

    Used Weasel 1.0.3 with a fix fo… (compare)

Jeremy D. Miller
@jeremydmiller
Let’s see:
1.) I’ve slept since then, I’d have to go rmeember the difference:)
2.) Yeah, IMartenQueryable<IEvent>
3.) I’d put it in an extension method in a new class in the Marten.Events namespace
Dariusz Firek
@DariuszFirek

Hi, I've started transition to v4 in our production project. As for now almost everything is working. I just miss one feature we had in the past and used extensively. I saw similar discussion recently here.
ViewProjection<TDoc, TId> used to have an option to apply one event to multiple documents via ProjectEvent(Func<IDocumentSession, TEvent, IList<Guid>> idSelector, (...)) or something similar. Now it's missing, and all I have is Identity<TEvent>(Func<TEvent, TId> identityFunc) which only allows to select one Id to be updated.

Our use is similar to: https://0bin.net/paste/pEbbcvtw#m6caJOCjZbQf+-UT+5iSzF3xtDe5rY8KIkmbEmYYhc/ (sorry for raw form)

How can I replicate that behaviour in v4? I wouldn't like to migrate any data since it's already on production and spans trough gigabytes of data... Just by looking at the code and JasperFx/marten#1610 (and linked issues), I see some options (I don't know if they are valid):

  1. override Task ApplyAsync(IDocumentSession session, EventPage page, CancellationToken token); and other methods from base class and provide own implementations (I see this as dangerous)
  2. Maybe something like FanOut to <Touple<Guid, TEvent>>, where Guid would be modelId but what to do later?
  3. Ditch ViewProjection as base class and go with EventProjection, like Oskar did here: https://github.com/oskardudycz/EventSourcing.NetCore/pull/38/files#diff-5246f5500eeddbee0e524da196022d2e669c5961f26f464565812515d9bbd504 but this still requires some more work to have something that was already provided by marten in the past.
  4. Create my own IProjection implementation with all the features I need
Btw old/red code in Oskar's repository is very similar to the one we have already working.
Jeremy D. Miller
@jeremydmiller
@DariuszFirek:
Jeremy D. Miller
@jeremydmiller
  1. This would fail spectacularly, so let’s just not do that:)
  2. FanOut() effectively makes a collection of objects within a single event look like events to the ViewProjection. FanOut() here would work if it had an option to pass in an IQuerySession to do your query. What it would effectively do is “explode” the incoming event into a new virtual event for each manufacturer. After that, the ViewProjection mechanics are unchanged and you’d get all the optimizations built into the V4 aggregations
  3. If you need this done fast, I’d say this is your best option
  4. Same as 3., but you’d potentially lose some of the optimizations we’ve built into the built in projections if you’re running these in the async daemon
The good news is that doing the improved FanOut() wouldn’t take very long.
Jeremy D. Miller
@jeremydmiller
@DariuszFirek See the new issue tracking this: JasperFx/marten#1784
Jedidja
@jedidja
@jeremydmiller When you get a chance, could you confirm the answer to question 1 please? :) Thank you and if you have any other pointers on #1680 I'd love to hear them.
Jeremy D. Miller
@jeremydmiller
@jedidja Gimme about an hour and I’ll write it up on #1680?
Jedidja
@jedidja
@jeremydmiller Sure, no rush - when you get a chance. Thank you! My plan is to spend a bit of time on it each day along with the prototypes I'm working on.
Jeremy D. Miller
@jeremydmiller
@jedidja Sorry, I got sucked into Zoom meetings and lost track of this. I’m going into it now...
Jeremy D. Miller
@jeremydmiller
@jedidja See if any of this helps, and don’t hesitate to just keep asking questions: JasperFx/marten#1680
Thanks for taking this on!
Jedidja
@jedidja
@jeremydmiller Will take a look now and ... yes .. there will probably be more questions :) Thanks for the initial notes!
Jeremy D. Miller
@jeremydmiller
No, thank you!
Jedidja
@jedidja
:)
Jedidja
@jedidja

@jeremydmiller I think I understand what AggregateStreamAsync() is doing, and it seems somewhat straightforward to create the AggregateTo method if it were in EventStore. But as an extension method, I'm a bit lost.

Also, I may have gone down a rabbit hole via ProjectionCollection to AggregateProjection.Codegen.cs ... eep. Pretty sure you weren't suggesting I duplicate any of that :)

[1] As a starting point, is this vaguely what you were suggesting? Questions in comments as well.

public static class EventStoreExtensions
{
    public static T AggregateTo<T>(this IMartenQueryable<IEvent> queryable) where T : class
    {
        var events = queryable.ToList();

        // Should I be getting this from ProjectionCollection instead somehow?
        var projection = new AggregateProjection<T>();

        // Seems like I can create an aggregator in a couple ways ...
        var aggregatorOne = projection.Build(new StoreOptions());
        var aggregatorTwo = projection.BuildLiveAggregator();

        // ... but there's no session available for Build()
        // and I think we want to reuse this to call Apply.
        var aggregate = aggregatorOne.Build(events, session, null);

        return aggregate;
    }
}

[2] Where would be the appropriate place to add the first test? Much happier when I can see tests fail and then work on the code :)

Jeremy D. Miller
@jeremydmiller
Yeah, no:) You’ll need to “find” the pre-registered AggregateProjection<T> the exact same way that AggregateStreamAsync() does. You need the dynamic code stuff to be memoized because it’s expensive to build. I think the tests would go in their own code file just under /Events. Just give it a descriptive name like `AggregateTo_Linq_Operator_Tests” or some such thing.
And if it comes down to it, just make it work w/ backing tests and it’s no big deal for me to deal w/ optimizations later
And yeah, that’s very close. Didn’t mean to make it sound like you were too far off other than don’t build up the AggregateProjection<T> yourself and do the memoization.
Jedidja
@jedidja

You’ll need to “find” the pre-registered AggregateProjection<T> the exact same way that AggregateStreamAsync() does

I'm definitely missing something. Presumably there's no "global" way to get to the ProjectionCollection that holds the pre-registered projections? It appears to live in EventGraph but I'm not sure how to access that in an extension method. AggregateStreamAsync has access to DocumentStorewhich leads to Events which leads to Projections but all I've got is an enumeration of IEvent :)

(From AggregateStreamAsync)
            var aggregator = _store.Events.Projections.AggregatorFor<T>();
            var aggregate = await aggregator.BuildAsync(events, _session, state, token);
Jeremy D. Miller
@jeremydmiller
You’ve got an IMartenQueryable<IEvent> when you trigger this off. One second while I load the code
Jedidja
@jedidja
Hmm. Sorry I must have overlooked something in the interface.
Jeremy D. Miller
@jeremydmiller
Okay, this is gonna be a little fugly. We can add an internal getter property for the current IMartenSession in MartenQueryable<T>. In your extension method, upcast to the concrete type, grab its IMartenSession, which gets you Options.Events.Projections.AggregatorFor<T>() and pretend for now that neither of us has ever heard of the Law of Demeter.
Don’t worry too much about making things clean upfront is my vote. Get the tests passing, then we worry more about the internal structures and who exposes who.
Jedidja
@jedidja
Ok :) Will do.
Thanks again. Nothing like day one of working in a new codebase heh.
Jeremy D. Miller
@jeremydmiller
And Marten isn’t typical just due to what it is.
Jedidja
@jedidja
Just adds to the fun :D
Jedidja
@jedidja
@jeremydmiller Put together a tiny PR in my own fork for when you get a chance to comment. jedidja/marten#1 ... no rush (signing off for the day now)
4 replies
Michael Rosario
@michaelprosario
To Jeremy Miller and team.. I wanted to thank you for all your hard work on this project. It's been fun to leverage Marten for my side projects. Excited to share the tech with my user group.
Oskar Dudycz
@oskardudycz
@michaelprosario thank you for the kind words :) Send the feedback after you share it :)
Alex Frie
@agf2413_gitlab
I am having some problems storing and reading primitive types in marten.
If I store a DateTimeOffset or int as a type object, marten will read these as DateTime and Long.
Unlike custom objects Marten don't store any informatin about the $type, and therefore don't have much to go on since the property is of type object.
Is there a way to tell marten to store type information for primitive types aswell, or do I need to wrap all primitive objects in a custom class for it to work with type object?
Oskar Dudycz
@oskardudycz
Do you have issue with other primitives or only with DateTime related?
Alex Frie
@agf2413_gitlab
I only tested with int, DateTimeOffset and string.
String is fine, but int(int32) is read as long(int64) and DateTimeOffset is read as DateTime.
Oskar Dudycz
@oskardudycz
Yes, the DateTime is special, as it's not precisely primitive
As we're using Npgsql as the provider we're following the recommendation from them: https://www.npgsql.org/doc/types/datetime.html
I don't think that we're doing much more customisation on what's described there
Btw. we also support NodaTime (at least for Newtonsoft JSON.NET)
This can help on time handling
Jeremy D. Miller
@jeremydmiller
@agf2413_gitlab Marten isn’t really meant to be used that way. If you’re just storing primitives, why not just use Postgresql as an RDBMS?
Alex Frie
@agf2413_gitlab
I am storing custom objects. I am just also storing some primitives.
Taking a simple example:
// This works
class1 {
 public int IntProperty {get; set;}
}
//This does not work
class2 {
  public object IntProperty {get; set;}
}
//This works
class3 {
  public object LongProperty {get; set;}
}
Jeremy D. Miller
@jeremydmiller
Well, yeah:) That’s just not an idiomatic usage of Marten or the JSON serializers for that matter.
Alex Frie
@agf2413_gitlab

I get this isn't a normal way of using Marten.
A workaround would be to create a wrapper class like this:

class CustomInt {
  public int Value {get; set;}
}

class class1 {
  public CustomInt IntProperty {get; set;
}

Now marten will store type information for CustomInt, and have a property of int to deserialize to. It just seems like a fun workaround in order to force marten to write a type definition for simple types.
So while I get this is certainly not the intended use case, I was wondering if there was some sort of support to force type definitions on all types, including primitive types.

Oskar Dudycz
@oskardudycz
I think that there is no issue with primitives
the issue is that you're trying to use them as objects.
You can safely store primitives types, as you showed in your samples.