Distributed Works/dkron

Filipe Pina

@fopina

@victorcoder seems to work now cheers! Haven’t managed to deploy it yet but the local reference setup seems good :+1:

Victor Castell

@victorcoder

great

qpippop

@qpippop

Hi, @victorcoder
According to docs, cron doesn't take job execution time into account. I want to know how much effort will it take to add such feature? Sorry, i'm not familar with golang so i need someone's estimate before diving deeper. Thanks

Filipe Pina

@fopina

@qpippop I don’t there’s any cron out there that works that way.. you’d probably be better off running some long running process with sleep() after execution..

qpippop

@qpippop

yeah, it's special requirement for scheduling system..
but in other aspects dkron is perfect for me
not sure about sleep(), because i dont need to wait for a fixed time

Victor Castell

@victorcoder

Hi @qpippop, what do you mean, you need to limit job execution time?

qpippop

@qpippop

Basically, i need just wait for job execution and only then calculate next execution time

Victor Castell

@victorcoder

and what if the job execution fail?

qpippop

@qpippop

retry if specified, otherwise - calculate next time

Victor Castell

@victorcoder

and the next execution time will be a fixed interval? ie. 2h?

I can think of a couple of ways to do this

using a fixed time job @at that autoupdates on finish

or creating a new Processor that can do exactly that

qpippop

@qpippop

Sounds nice that there's no need to modify cron lib and it can be achieved by already implemented api
Thanks for your response!

Filipe Pina

@fopina

can processors modify job configs (internally, without calling HTTP API) @victorcoder ? and, out of topic, is yvan no longer contributing actively to the project?

Victor Castell

@victorcoder

no @fopina the can modify just executions

I don't know about yvanoers, no news, but definitely not as active

Filipe Pina

@fopina

hey @victorcoder , here https://dkron.io/basics/getting-started/#usage you mention the ports required

    8946 for serf layer between agents
    8080 for HTTP for the API and Dashboard
    6868 for gRPC and raft layer comunication between agents.

but no protocol nor differentiation between server/agents nor egress/ingress

my assumption/understanding so far:
8080/tcp -> not used by dkron itself, exposed to users/apps using dkron
8946/tcp -> ingress for servers, egress for servers/agents (?)
8946/udp -> ingress and egress for servers and agents (?)
6868/udp -> ingress and egress for servers and agents (now with dkron3)

is this correct?

Victor Castell

@victorcoder

It's not specified because, ideally, ingress/egress for all nodes all those ports is required, for example with the latest changes 6868 ingress was added to non-server nodes, it's better to have them already open

consider all of them bidirectional, exclude 8080 if you want

and consider 8946 is udp+tcp 6868 is tcp only

Filipe Pina

@fopina

yes, that's what I have at the moment :)

finally (re)upgrading cluster to dkron3 again (after rolling back last time).
there was a lot of missed job runs with dkron2 by the way (criteria: searching for server "run:job" and matching against agents "agent: Starting job")

about 10% I believe. Hopefully dkron3 persistent connection will solve this (or flag it better). do you want me to open an issue with details for tracking? (and then update it, in a few days, if it doesn't happen with dkron3 anymore)

Victor Castell

@victorcoder

Yeah sure, do it if you want, at this point there's no missing executions due to Serf weirdness, it could be by other causes tho, but it will be reflected in your logs

Filipe Pina

@fopina

so missing executions were expected in dkron < 3 ?

Victor Castell

@victorcoder

it seems, under certain conditions, serf was silently loosing ~1-2% of the executions distribworks/dkron#744 specially with high volumes of simultaneous starts

Filipe Pina

@fopina

ah so definitely that, I won't open new ticket then :)

I guess that jenkins HASH schedule would have helped us prevent this

Filipe Pina

@fopina

@victorcoder what do you think about an "evacuate" mode for agents? something like there is cloud infrastructure or docker swarm: mark an agent so it doesn't start more jobs without killing (so it finishes running what is there)?

haven't looked into those bits, not sure how easy that would be (or if it makes sense)

an easy hack would be to be able to remove the selection labels (if they are used, such as in my use case). that would actually be something more generic: update node labels in runtime

Filipe Pina

@fopina

and off-topic, with v3 there isn't a starting job anymore apparently, can't easily take the jobs that were actually executed unless they have output :(

Victor Castell

@victorcoder

you mean in the logs?

Filipe Pina

@fopina

yes

nothing in the agent log (without debug) when picking a up a job I think

Victor Castell

@victorcoder

maybe it got lost with all the refactor, would be nice to re-add it

Victor Castell

@victorcoder

but you can use the grpc.agent_run statsd metric to get the same info

Filipe Pina

@fopina

with dkron2 I was taking lists from the log aggregation to track which jobs were missed, not only how many. But I'll take a look at the statsd metric, if it matches, I don't need to know which :)

distribworks/dkron@b83bde1 :bow:

my 2 PRs should be clean enough, if you want to take a look :P

Victor Castell

@victorcoder

done @fopina

Shiva Prasad G

@shivaprasad573

Hello, I am trying to run a job on every first monday of the month. I have generated a cron expression as 0 0 12 ? 1/1 MON#1 * but this is not working in DKron. Please can anyone help to achieve this?

Victor Castell

@victorcoder

I think you can not do that in Dkron cron spec syntax, try with this solution https://ngelinux.com/how-to-schedule-a-cron-to-execute-on-first-monday-of-every-month/

Filipe Pina

@fopina

Hey @victorcoder have you considered moving the plugins to normal go plugins (now that they exist)?
As the gRPC is not really used for remote, it could probably simplify the code a lot :)

Victor Castell

@victorcoder

Sadly @fopina native go plugins do not work in all platforms, Windows, and I have the impression that they doesn't offer same functionality

Though I'm always trying to find an alternative for the current grpc plugins

I will replace them sooner or later

Filipe Pina

@fopina

Makes sense, sorry. I’ll ping if i come across anything

Where communities thrive

People

Repo info

Activity