Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Activity
  • Feb 08 19:06
    virtualdvid commented #412
  • Feb 08 18:58
    virtualdvid commented #412
  • Feb 08 18:56
    virtualdvid commented #412
  • Feb 08 18:56
    virtualdvid commented #412
  • Feb 08 18:34
    aidiss opened #421
  • Feb 05 10:20
    0x587 closed #420
  • Feb 05 10:20
    0x587 commented #420
  • Feb 05 10:16
    agronholm commented #420
  • Feb 05 10:06
    0x587 commented #420
  • Feb 05 09:12
    agronholm commented #420
  • Feb 05 06:28
    0x587 commented #420
  • Feb 04 16:36
    0x587 commented #420
  • Feb 04 15:49
    agronholm commented #420
  • Feb 04 15:47
    0x587 opened #420
  • Feb 04 14:51
    srPuebla commented #419
  • Feb 04 13:27
    agronholm commented #419
  • Feb 04 12:06
    srPuebla opened #419
  • Feb 03 03:18
    xinqinglxl closed #409
  • Feb 03 03:18
    xinqinglxl commented #409
  • Jan 12 13:54
    agronholm commented #418
Omer Katz
@thedrow
@agronholm I have another feature request for Celery 5. One of our goals is to be more obsevable. We've selected eliot as our logging library for that purpose.
Would you mind using it as well instead of the logging module?
Alex Grönholm
@agronholm
I will take a look at it but if it's not compatible with stdlib logging then I will probably not use it
Omer Katz
@thedrow
It's not but you can provide a lot more context this way
Alex Grönholm
@agronholm
celery can use eliot because it's a service, not a library to be used inside an application
apscheduler is different in that way
Nicolas Marcq
@Sispheor
found my issue. As the BackgroundScheduler start a thread, the asyncio loop need to be initialized in the the main thread
Omer Katz
@thedrow
@agronholm Is there new code for me to take a look into?
Alex Grönholm
@agronholm
I can pastebin the WIP code for the postgresql data store
Omer Katz
@thedrow
I'd like to experiment with implementations but it isn't easy if I don't have the an implementation with asyncio
That'd be wonderful
I have some ideas I'd like to try
Alex Grönholm
@agronholm
and about eliot: I would really like to increase the observability of APScheduler, but I feel eliot's functionality overlaps quite a bit with OpenTelemetry
and I'm not sure what the proper course of action is for me
Omer Katz
@thedrow
Isn't eliot an implementation of OpenTelemetry?
Right now I'm thinking about writing a retrying library using APScheduler which would be extremely useful for Celery
Alex Grönholm
@agronholm
is it?
I didn't see any mention of that in the readme
I haven't looked for existing retrying libraries for Python but I'd be mildly surprised if there wasn't any existing one
eliot's entire documentation does not contain the word opentelemetry
Omer Katz
@thedrow
Maybe I was thinking about another thing
Is OpenTelemetry a competing standard of OpenTracing?
Alex Grönholm
@agronholm
OpenTelemetry a broader standard that will replace OpenTracing
but its reference implementations are still in alpha/beta
Omer Katz
@thedrow
Oh
That makes me reconsider using eliot as well
but I don't see logging there as part of the solution
Alex Grönholm
@agronholm
eliot seems concerned about not just logging but tracing as well
Omer Katz
@thedrow
Kind of
It's a structured log
Alex Grönholm
@agronholm
and the stdlib logger can already accept extra information that custom loggers can use as they please
loggers can even have proxies that feed predefined extras to all log entries emitted through them so every logger call does not have to specify those extras
Omer Katz
@thedrow
Yes but eliot makes all that easy
I've tried it
Alex Grönholm
@agronholm
it's also not compatible with stdlib logging which is a showstopper problem for a generic library
Omer Katz
@thedrow
Yes, I'll try to think how to resolve that with Itamar
Alex Grönholm
@agronholm
alright
Omer Katz
@thedrow
I get your design
It's elegant.
Alex Grönholm
@agronholm
thanks.
a corresponding mysql data store would have to rely on polling or an external notification mechanism
Omer Katz
@thedrow
How do you feel about using Arrow or another library for handling datetimes?
Alex Grönholm
@agronholm
I have a strong dislike for Arrow
I'm not sure what would even be accomplished by that
Alex Grönholm
@agronholm
that's just one line
and I don't see that as complicated or difficult in the least
Omer Katz
@thedrow
What if we'll make the datastore an async iterator?
async for pending_jobs in datastore.clock():
   pass # schedule the jobs onto workers