JasperFx/jasper

Next generation application development framework for .Net

Mark Warpool

@CodingGorilla

Second, I changed the JasperOptions.ScheduledJobs.FirstExecution and JasperOptions.ScheduledJobs.PollingTime, which seems to control the durability agent's queue processing. I set them both to 2 minutes (default of 0 seconds and 5 seconds respectively), and the problem seems to go away.

Mark Warpool

@CodingGorilla

But along with the problem going away, I also notice that the message is no longer being picked up by the RecoverIncomingMessages queue from the durability agent. But I still cannot figure out where the actual execution comes from.

I'm sure I'll be back, banging my head on this wall in the morning, I hate a puzzle I can't solve =D

Mark Warpool

@CodingGorilla

Oh, I think I figured it out, if I understand incoming messages structure properly, an OwnerId of 0 means "AnyNode". EnqueueDurably() calls down into persistOrSend, which since there isn't an ongoing transaction, calls Envelope.Send(). This in turn calls the DurabilityLoopbackSendingAgent.StoreAndForward().

Mark Warpool

@CodingGorilla

This is where the problem is. .StoreAndForward() initially saves the message as an incoming message with an OwnerId of 0 (= AnyNode), and then immediately throws it into it's WorkerQueue, which will start running the handler regardless of the "owning node". Then, since the RecoverIncomingMessages fires every 5 seconds by default, assuming the handler takes longer than 5 seconds to run, it will see the message sitting in the incoming messages table, and "recover it" and run it again.

BOOM, problem identified. Gotta sleep now. =)

Jeremy D. Miller

@jeremydmiller

Yup, that would do that. Awfully good catch.

Jeremy D. Miller

@jeremydmiller

And a one line code fix:/

Mark Warpool

@CodingGorilla

Is the fix to just make.StoreAndForward() save it with the current node id?

Jeremy D. Miller

@jeremydmiller

Yes

Mark Warpool

@CodingGorilla

Makes sense...

Mark Warpool

@CodingGorilla

Would you like me to prepare a pull request? Or have you got it?

Jeremy D. Miller

@jeremydmiller

I’ll take a PR, or I’m going to be back in Jasper for other reasons in the next week I think

Mark Warpool

@CodingGorilla

#532 Is ready to go

Jeremy D. Miller

@jeremydmiller

:thumbsup:

Got it

or will very soon

Mark Warpool

@CodingGorilla

Thanks, we have a big release of a different product next week, so I have a little breathing room on this particular issue, so no rush.

That and I put a bunch of locking on the handler to make sure it can't run twice at the same time for the short term =)

Svein Ellingsen

@lovmoen

Is there a way to perform a remote async "request/response" in the same way as Invoke<TResponse>()?

Jeremy D. Miller

@jeremydmiller

I don’t remember if that got yanked out or not, but there used to be an Request() method for that. I think I eliminated that in the thinking that you’d just use HTTP. That was some significant complexity

Svein Ellingsen

@lovmoen

OK, thanks for your quick response! I'll just have to figure a way around that then.

Svein Ellingsen

@lovmoen

On another note: The documentation doesn't really explain well the difference between SendAndExpectResponseFor, Send and Publish. Reading the section on "Send Message with Eventual Reply" it just shows a call to SendAndExpectResponseFor which looks just like a call to Send or Publish. A section explaining how these behave differently would be very helpful.

Svein Ellingsen

@lovmoen

@jeremydmiller I refer you to our discussions back in june; I had some trouble with scheduled messages. Since then I have been on vacation and later busy with other projects, but now I have this project on the top of my priority list again. I was wondering if you'd had any more time to look into the issues I raised wrt ScheduleSend and durable/non-durable queues. Also, I have now come a bit further in my project, and have implemented a saga whose Starts() method returns messages for future processing. I return an IEnumerable<object> of ScheduledResponse objects which should be sent out at a future time. But the behaviour I'm getting is immediate processing of these messages.

Svein Ellingsen

@lovmoen

Would I be better off using IMessageContext.ScheduleSend directly instead, injecting the IMessageContext into my saga object?

Svein Ellingsen

@lovmoen

I tried that approach, but ScheduleSend with a time in the future still sends the messages immediately.

Svein Ellingsen

@lovmoen

Sorry, not so! When converting correctly between local and UTC, Jasper does exactly what it says on the tin! My bad. Will check and report back on whether all my woes with scheduled messages have been due to poor timezone management on my part.

Svein Ellingsen

@lovmoen

Yep. That makes it work equally well when returning ScehduledResponse objects from the saga start handler.

Svein Ellingsen

@lovmoen

The problem I mentioned back in june remains, however: If I do Schedule to a queue, nothing happens. If I do ScheduleSend, it gets processed immediately if the queue is durable, and at the expected time if ithe queue is non-durable.

The behaviour can be reproduced with the example code I put on github: https://github.com/lovmoen/myjaspertest/tree/schedule-durable-queue

Jeremy D. Miller

@jeremydmiller

There was a bug about the scheduled messages going to a loopback queue, that was real.

I remember this conversation from back then. Are you on the latest now?

Svein Ellingsen

@lovmoen

Just tested on 0.9.13.

Jeremy D. Miller

@jeremydmiller

Nevermind that, I don’t see anything in the commit history for that.

I’ll try to get into that this weekend. I was just about to do some other work on Jasper anyway

Svein Ellingsen

@lovmoen

That's great! I just pushed my updated code to the above git repo if that's of any help.

Jeremy D. Miller

@jeremydmiller

Okay, thanks. I’ll definitely try to take a look soon

ddivita

@ddivita

What does the roadmap look like for .net Core 3?

Jeremy D. Miller

@jeremydmiller

Wait for aspnetcore 3.0 to come out, convert over to that, and finally kick out a 1.0

Mark Warpool

@CodingGorilla

@jeremydmiller Any word on #532 ?

Jeremy D. Miller

@jeremydmiller

Sigh, no. I could I guess actually sneak in some OSS time this afternoon though

Or tomorrow maybe

Svein Ellingsen

@lovmoen

Has there been a recent change in how Jasper uses Marten/database? When I started my project back in May, I set up marten backed persistence and everything worked falwlessly until I recently deployed my project with a fresh/empty database. Now I keep getting these exeptions: '[ERR] Running Recover persisted outgoing messages
Npgsql.PostgresException (0x80004005): 42P01: relation "public.jasper_outgoing_envelopes" does not exist' and 'Npgsql.PostgresException (0x80004005): 42P01: relation "public.jasper_incoming_envelopes" does not exist'. I set the marten option AutoCreateSchemaObjects to AutoCreate.All. Are there other (new?) initialization procedures that need to be performed?

Svein Ellingsen

@lovmoen

If I manually restore the tables from my old database, the exceptions go away.

Jeremy D. Miller

@jeremydmiller

Yeah, there has. I did a lot of work this spring to put the Sql Server backed persistence and Postgresql backed persistence on the same common core, so it got much less Marten-centric. Slightly different table structure now

The docs are up to date on how to seed your database, but I didn’t do anything for conversion:(

Svein Ellingsen

@lovmoen

Should have seen that Thanks! :)

Svein Ellingsen

@lovmoen

Any idea what could cause this on startup:

[11:09:25 FTL] An error occurred stopping the application
System.AggregateException: One or more errors occurred. (A task was canceled.) ---> System.Threading.Tasks.TaskCanceledException: A task was canceled.
at Jasper.Messaging.Durability.DurabilityAgent.StopAsync(CancellationToken cancellationToken)
at Microsoft.AspNetCore.Hosting.Internal.HostedServiceExecutor.ExecuteAsync(Func

2 callback)                                                       
   --- End of inner exception stack trace ---                                                                                                         
   at Microsoft.AspNetCore.Hosting.Internal.HostedServiceExecutor.ExecuteAsync(Func

2 callback)
at Microsoft.AspNetCore.Hosting.Internal.HostedServiceExecutor.StopAsync(CancellationToken token)
---> (Inner Exception #0) System.Threading.Tasks.TaskCanceledException: A task was canceled.
at Jasper.Messaging.Durability.DurabilityAgent.StopAsync(CancellationToken cancellationToken)
at Microsoft.AspNetCore.Hosting.Internal.HostedServiceExecutor.ExecuteAsync(Func`2 callback)<---

[11:09:25 FTL] An error occurred stopping the application                                                                                             
System.AggregateException: One or more errors occurred. (A task was canceled.) ---> System.Threading.Tasks.TaskCanceledException: A task was canceled.
   at Jasper.Messaging.Durability.DurabilityAgent.StopAsync(CancellationToken cancellationToken)                                                      
   at Microsoft.AspNetCore.Hosting.Internal.HostedServiceExecutor.ExecuteAsync(Func`2 callback)                                                       
   --- End of inner exception stack trace ---                                                                                                         
   at Microsoft.AspNetCore.Hosting.Internal.HostedServiceExecutor.ExecuteAsync(Func`2 callback)                                                       
   at Microsoft.AspNetCore.Hosting.Internal.HostedServiceExecutor.StopAsync(CancellationToken token)                                                  
---> (Inner Exception #0) System.Threading.Tasks.TaskCanceledException: A task was canceled.                                                          
   at Jasper.Messaging.Durability.DurabilityAgent.StopAsync(CancellationToken cancellationToken)                                                      
   at Microsoft.AspNetCore.Hosting.Internal.HostedServiceExecutor.ExecuteAsync(Func`2 callback)<---

Where communities thrive

People

Repo info

Activity