Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Repo info
Activity
  • 18:45
    Build #2171 passed
  • 18:45
    Build #2171 passed
  • Sep 17 14:46
    Build #2170 passed
  • Sep 17 14:46
    Build #2170 passed
  • Sep 16 00:06
    Build #2169 passed
  • Sep 16 00:06
    Build #2169 passed
  • Sep 14 03:42
    Build #2168 passed
  • Sep 14 03:42
    Build #2168 passed
  • Sep 13 22:44
    Build #2167 passed
  • Sep 13 22:44
    Build #2167 passed
  • Sep 10 20:10
    Build #2164 passed
  • Sep 10 20:10
    Build #2164 passed
  • Sep 09 18:16
    Build #2162 passed
  • Sep 09 18:16
    Build #2162 passed
  • Sep 09 17:24
    Build #2161 passed
  • Sep 09 17:24
    Build #2161 passed
  • Sep 08 03:21
    Build #2157 passed
  • Sep 08 03:21
    Build #2157 passed
  • Sep 08 02:55
    Build #2156 passed
  • Sep 08 02:55
    Build #2156 passed
Alexey Lebedev
@odysseus1973
@ricardoromaobr Yes
Marko Lahma
@lahma
@nizmow Quartz reads jobs (triggers) to run and pushes them to execution, so single consumer to pushes to job executor queue (threads)
Neil Houghton
@nizmow
I see, so ramping up thread count isn't necessarily going to kill my database
Can we control how many triggers it gathers in a sweep? Ideally for many small jobs I want fewer sweeps, but with a large amount of triggers acquired
Marko Lahma
@lahma
there should be a setting for that too
you might want to consider ditching the LockHandler definition, it will trigger some "smarts": https://github.com/quartznet/quartznet/blob/master/src/Quartz/Impl/AdoJobStore/JobStoreSupport.cs#L491
Neil Houghton
@nizmow
There is a setting, actually -- I've configured it to threads, so very very low
Neil Houghton
@nizmow
@lahma that's possibly good advice
Neil Houghton
@nizmow
@lahma thanks for advice, churning through now
set the trigger acquisition to threads * 4, threads to 100
In addition, I pulled one machine out of the cluster -- I think I might try for a configuration where we have 2 machines, one for MT to consume from the queue and the other to run the jobs
at the moment when job load is heavy the consumer rate goes way down, so if we're under load the queue fills up, which can be very bad news if it gets out of hand
Chris Patterson
@phatboyg
we use like 100 threads for our scheduler.
And we get way more than 5/s - we get like 400
Marko Lahma
@lahma
I think that default of 10 originates from some 20 years back. Quartz.NET inherited it from Java version and the .NET version is older than 10 years..
Unfortunately there's no right answer either
maldworth
@maldworth

@lahma I was just looking at the acquisition config (from java here). It mentions

property is set to > 1, and JDBC JobStore is used, then the property "org.quartz.jobStore.acquireTriggersWithinLock" must be set to "true" to avoid data corruption.

so if using this in .net world, should one be setting "quartz.jobStore.acquireTriggersWithinLock" to true?

I know we aren't using JDBC store, but instead ADO for SqlServer
but I assume locking still applies with a relational DB
@phatboyg are you using this acquisition config that nizmow is using? first time I've really heard about it, and want to know if I should add it to my MT Quartz config.
one of them is quartz.scheduler.batchTriggerAcquisitionMaxCount
Marko Lahma
@lahma
if I remember right this is more mandatory for SQLite use
Chris Patterson
@phatboyg
Also, make sure if you're running multiple nodes you have set it up in cluster mode - different style of locking.
Marko Lahma
@lahma
yep, cluster pretty much changes the game, then the locking will handle a lot things - database row lock acquired to get consistency between nodes
Chris Patterson
@phatboyg
Whew, that's one big commit: phatboyg/MassTransit@f91e216
Marko Lahma
@lahma
"no breaking changes" :wink:
Chris Patterson
@phatboyg
Well, mostly there aren't.
I didn't change a single line of unit test code.
Marko Lahma
@lahma
If I didn't know that you have great coverage, I could have thrown in the card about no tests - no breakage
kudos for keeping the code base so clean
Chris Patterson
@phatboyg
A big change like this definitely gives me a chance to see a huge swathe of the codebase
Marko Lahma
@lahma
can you maintain and add features on company time (adds value) or do you need to find spare time for things? I'm quite baffled about the things you manage to do (size/impact/new features)
Chris Patterson
@phatboyg
Well, this commit is probably a couple of evenings of coding, maybe 10 hours?
I've put a lot of time on v6 over the past 9 days.
The design/thoughts/notes go back months though.
Marko Lahma
@lahma
OK, so you still have the energy and can squeeze some time, nice
Yep, you can save a lot of work with pre-planning and thinking things through a couple of times
It just keeps amazing me, I've studies some of the AppInsights, ASP.NET Core DI code and preparing to add same kind to things to Quartz, a lot of work
Chris Patterson
@phatboyg
Oh, yeah, Quartz. That little thing :)
Thanks for dropping in and helping with the Quartz tuning questions. Any suggestions for adjusting the default configuration settings are welcome and appreciated.
Marko Lahma
@lahma
No problem, I like to watch this room because of MT insights, we do run it in production :slight_smile:
Chris Patterson
@phatboyg
Haha, nice. We run Quartz too :)
maldworth
@maldworth
yep, +1 to that, thanks for the tidbits of quartz wisdom. it's somewhat of a black box to me
just twisting knobs, and turning dials to get it to run nicely
Marko Lahma
@lahma
I admit the documentation isn't the greatest, some things I know by heart because I've been testing and porting the code. But some things I don't hit running in production because they are just there.
I do have new documentation engine in the backlog that should make thins easier to maintain and to the eye, but the time...
I've just been struggling with finding the energy and keeping up with Quartz, fortunately the library is quite mature, but stalled in technological sense. Next steps would be getting proper ASP.NET Core DI and AppInsights, I actually would like to use the features at work too.
And the long overdue fluent configuration interface, saying .UseSqlServer and no tinkering about the locking SQL and other strategies
Version 3.1 will bring initial fluent interface and the DI/AppInsights stuff hopefully
Ricardo
@ricardoromaobr
@odysseus1973 Ok, I'll try again. I made a worker service and running it as a console app, but when I try to start a service it does'nt start with a timout error to start.