Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Repo info
Activity
  • Oct 03 19:40
    smbecker edited #2362
  • Oct 03 19:40
    smbecker edited #2362
  • Oct 03 19:38
    smbecker opened #2362
  • Oct 03 19:38
    smbecker opened #2362
  • Oct 03 00:35
    dnfadmin commented #1843
  • Oct 03 00:35
    dnfadmin commented #1843
  • Oct 01 13:40
    dnfadmin commented #1746
  • Oct 01 13:40
    dnfadmin commented #1746
  • Oct 01 05:23
    dnfadmin commented #1846
  • Oct 01 05:23
    dnfadmin commented #1846
  • Sep 30 01:33
    dnfadmin commented #1706
  • Sep 30 01:33
    dnfadmin commented #1706
  • Sep 29 16:22
    dnfadmin commented #2149
  • Sep 29 16:22
    dnfadmin commented #2149
  • Sep 29 09:31
    NikiforovAll commented #2360
  • Sep 29 09:31
    NikiforovAll commented #2360
  • Sep 28 21:38
    dnfadmin commented #1728
  • Sep 28 21:38
    dnfadmin commented #1728
  • Sep 28 02:11
    kuanpak opened #70
  • Sep 27 10:46
    dnfadmin commented #1580
JT
@Hawxy
image.png
I assume the rebuild process was built with the idea that the daemon has always been previously running and thus no at-least-once detection of the high water mark occurs?
dionfoster
@dionfoster
I think i've managed to reproduce an issue with inline projections halting at gaps, I've thrown this PR up for review/discussion:
JasperFx/marten#2313
jeez, going back to a conversation we were having on the 26th July, where'd that time go?
redwraith
@redwraith:matrix.org
[m]
hi all 👋 going through the docs I noticed Marten doesn't support Intersect Linq operator. Is there any alternative way to achieve this functionality? I'm trying to check if two arrays intersect in Marten query.
4 replies
Oskar Dudycz
@oskardudycz

@/all I started to work on the Marten API for Upcasting events data. I'd like to add a built-in tooling for simple payload transformations. You could use that if your event schema has changed and you'd like to use new event types in your aggregates and projections. Those transformations will be run during the serialisation phase, so upcasting on the fly old event data into the new one.

I added two types of upcasters:

  • CLR types - when you're fine with keeping old event payload then it'll at first deserialise it, and you can map to the new one,
  • Raw json transformations, more flexible, less allocation, but a bit more tedious.

I have an early draft, of it, I'd be grateful for your feedback, especially about the public API (e.g. accessibility, naming, etc.).

https://github.com/JasperFx/marten/pull/2314/files#diff-530572fa09e2864ed0df01ade900998ed7c26a12d7dc2ace6666d7be76214dc6R22

Technically, this is what I described in my blog article: https://event-driven.io/en/simple_events_versioning_patterns/

Thoughts?

Oskar Dudycz
@oskardudycz
https://twitter.com/oskar_at_net/status/1555854977700184064 - would be greateful for RT to get more feedback :)
Oskar Dudycz
@oskardudycz
I did another roundtrip and trimmed implementation more.
philiplorenz-code
@philiplorenz-code
Is the dataflow between marten framework and postgres encrypted in any way? Which possibilities do I have therefor?
Jeremy D. Miller
@jeremydmiller
That's all Postgresql & Npgsql. Start here maybe: https://www.npgsql.org/doc/security.html?tabs=tabid-1
philiplorenz-code
@philiplorenz-code
So when enabling the SSL-feature on Postgres it will automatically be encrypted?
Jeremy D. Miller
@jeremydmiller
I'm going to have to direct you to those docs, I haven't used it personally
Joe Nathan Abellard
@jabellard
I have a question regarding the Marten Linq Provider. How come it can understand a query such as querySession.Query<Book>().Where(b => true).ToList();
But can not understand a query such as querySession.Query<Book>().Where(b => b != null).ToList();
In the end, these queries don't really filter anything out , but I'm curious to know why the first one is executed successfully where the second one keeps triggering an exception.
Jeremy D. Miller
@jeremydmiller
Do you understand how Linq works by chance?
Joe Nathan Abellard
@jabellard
Some knowledge, but don't know too much about the inner workings.
Jeremy D. Miller
@jeremydmiller
Okay, so the Linq provider never executes your code, rather it's an interpreter. Meaning that we have to anticipate and in many cases explicitly code for every single possible usage in order for that to work. The well lit paths are going to be fine. Oddball things that we didn't anticipate anybody every using like Where(x => x == null), are probably not going to have explicit support.
So the next question is, why in the world would you possibly do that?
Joe Nathan Abellard
@jabellard
Good question. I'm not wrtiting these queries manually myself. I'm currently trying to integrate the HotChocolate GraphQl library with Marten. It comes with out of the box support with some nice features like liftering (https://chillicream.com/docs/hotchocolate/fetching-data/filtering), which works by converting the graphql arguments into an expression tree that it then will apply to the IMartenQueryable. Let's say I have a Book object which as a Rating property. If I want to filter by that property, It will generate an expression that looks like $_s0 != null && $_s0.Rating == 10 instead of just $_s0.Rating == 10. That first part of the expression is what is causing the problem.
Jeremy D. Miller
@jeremydmiller
Blech, my first guess was that it was something like that
And that's such a horrendously inefficient way to do that
Gotta go, I'll be back. You submitted an issue for this anyway, righ?
Joe Nathan Abellard
@jabellard
Not yet. I'll submit one. I can also work on putting an enhancement that will fix this if you can point me in the right direction on how to go about resolving it.
Jeremy D. Miller
@jeremydmiller
Start in LinqHandlerBuilder and look for where the Where clauses are interpreted. Somehow gonna have to catch and interpret this as no where clause. Don't go in expecting this to be a simple if/then statement anywhere:(
Joe Nathan Abellard
@jabellard
Challenge accepted.
Jannik Buschke
@jannikbuschke
Is it reasonable to set a UserId Header on any session like this? Do people something similar or is putting a user id on metadata not something you usually want to do?
public class ScopedSessionFactory: ISessionFactory
    {
        private readonly IDocumentStore store;
        private readonly IUserService userService;

        public ScopedSessionFactory(IDocumentStore store, IUserService userService)
        {
            this.store = store;
            this.userService = userService;
        }

        public IQuerySession QuerySession()
        {
            return store.QuerySession();
        }

        public IDocumentSession OpenSession()
        {
            var session = store.LightweightSession();

            var userId = userService.GetUserId();
            session.SetHeader("UserId", userId);
            return session;
        }
    }
Oskar Dudycz
@oskardudycz
I think that's reasonable, just beware about GDPR :)
5 replies
Jannik Buschke
@jannikbuschke

Are these diagnostic messages normal/expected? I get them several times per second
`
[14:31:51 INF] High Water agent is stale after threshold of 3 seconds, skipping gap to events marked after 08/08/2022 12:30:20 +00:00

[14:31:51 INF] Daemon projection high water detection skipping a gap in event sequence, determined that the 'safe harbor' sequence is at null
`

Jeremy D. Miller
@jeremydmiller
@jannikbuschke It’s not the sign of a healthy running system. Is that happening at the same time you’re getting a lot of system failures?
You can bump up that “stale time” if you have long running transactions in your system that make it look like it’s stale
Jannik Buschke
@jannikbuschke
Ok, seems this happens when I have issues saving/applying events (using system.text.json + custom converter)
Jeremy D. Miller
@jeremydmiller
I'm shocked that STJ could ever be a problem. Shocked I tell you!
Marten should be putting tombstone events in place of the failed events, but maybe we've never tested that workflow in the case of serialization failures earlier in the SaveChanges() call.
Oskar Dudycz
@oskardudycz
@jeremydmiller, you're trying to push me from the 1st place for the biggestSTJ hater? :D
Jeremy D. Miller
@jeremydmiller
Have I ever mentioned how much I appreciate you & @phillip-haydon taking the lead on STJ issues? :-)
Oskar Dudycz
@oskardudycz
You did :P
Jannik Buschke
@jannikbuschke
Im using System.text.json + https://github.com/Tarmil/FSharp.SystemTextJson and I actually like it. It gives some errors as kind of expected but actually now I have a model that I am happy with. I.e. strings are now not nullable but need to be "string option" (FsharpOption) and null will error on de/serializing
Oskar Dudycz
@oskardudycz
Sorry to be that guy, but did you consider using a battle tested Newtonsoft?
To balance that, F# user experience will be my next goal after merging Upcasters PR. If you could write up all of your challenges in the GH issue, then I could try to include that.
Still, I need to learn F# language and idiomatic approach better, so it’ll take some time
Ben Edwards
@elexisvenator
Speaking of STJ, anyone ever had the problem when calling GetHeader() on an event will sometimes return a STJ object instead of the header string, even when marten is not configured to use STJ?
It's not something that always happens and I haven't been able to reliably recreate it, but ive taken to putting an extra ToString on all my header calls
Oskar Dudycz
@oskardudycz
I’m not aware of such an issue. But if you were able to reproduce it, we’ll give it a look. Are you sure that’s STJ JsonDocument, and not Newtonsoft JObject?
JObject could be returned if there was an unknown type and it did dynamic deserialisation. It can be cast to the specific type.
Jannik Buschke
@jannikbuschke
Tested with Newtonsoft and fsharp types are serialized okish, however the deserialization format for options and discriminated unions is imho not ideal. For both types are different json representations possible, Ill try write a blogpost about my experiences but not yet sure when...
Oskar Dudycz
@oskardudycz
Do you think that being able to do custom serialization like in my PR would help? JasperFx/marten#2314
2 replies
In theory, it’s more for event schema migration, but I was thinking of making it more generic also for documents as the next step
Phillip Haydon
@phillip-haydon

Have I ever mentioned how much I appreciate you & @phillip-haydon taking the lead on STJ issues? :-)

not to me :D still waiting on a reply to know if we can set casing to be insensitive by default to match newtonsoft and fix my projection issue :D

Jeremy D. Miller
@jeremydmiller
Okay, that's a "no", that needs to be opt in
Phillip Haydon
@phillip-haydon
:+1: no worries, i shall add some docs somewhere so if i didn't already add it so it's not forgotten
Egor Pavlikhin
@EgorPavlikhin_twitter
What is the default for AutoCreateSchemaObjects if it's not specified? I have queries periodically failing for a service running in Google Cloud Run with timeouts like so. Just wondering what could be the reason.
---> System.TimeoutException: The operation has timed out.
   at Weasel.Core.Migrations.TimedLock.Lock(TimeSpan timeout)
   at Weasel.Core.Migrations.DatabaseBase`1.generateOrUpdateFeature(Type featureType, IFeatureSchema feature, CancellationToken token)
   at Weasel.Core.Migrations.DatabaseBase`1.ensureStorageExists(IList`1 types, Type featureType, CancellationToken token)
   at Marten.Linq.MartenLinqQueryProvider.ensureStorageExistsAsync(LinqHandlerBuilder builder, CancellationToken cancellationToken)
   at Marten.Linq.MartenLinqQueryProvider.ExecuteAsync[TResult](Expression expression, CancellationToken token)