Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Repo info
Activity
Nicolas Sebrecht
@nicolas33
I don't want this anymore.
Jens Neuhalfen
@neuhalje
That depends on the requirements: Sharing object instances across different processes: Nope
Singleton invariant across processes: Nope
Multiprocess makes many things easier by making a few things way harder :-)
di_impl.py
Nicolas Sebrecht
@nicolas33
I do want message by passing to allow multiprocessing and avoid most of locks or sync state issues.
Thanks, I'll look at that.
Jens Neuhalfen
@neuhalje
Message passing seems like a good idea
How would you synchronize resource access?
Nicolas Sebrecht
@nicolas33
There no need to sync resources.
Each worker will loop and noop if no request.
Jens Neuhalfen
@neuhalje

With message reordering I would assume that deadlocks could be tricky.

Ah, ok. But how about e.g. manipulating the IMAP/Maildir?

… the cats demand their food :-) … afk …
Nicolas Sebrecht
@nicolas33
Do you have an example?
IMAP worker started
Maildir worker started
Engine started
State worker started
engine requests news from state to current state to maildir
engine requests news from state to current state to imap
engine receives the updates
engine applies the sync logic
engine send the updates to either side (imap and/or maildir)
engine send the applied updates to the state
No sync required (except if we consider the "updates received from the repositories").
Nicolas Sebrecht
@nicolas33
No need for message reordering.
"state" is the repository. It's the common ancestor. How the previous sync finished.
(syncing is a 3-way merge)
Jens Neuhalfen
@neuhalje
@nicolas33 is there a possibility that e.g. two processes manipulate the same folder?
Nicolas Sebrecht
@nicolas33
Sure.
I'd say this is the reponsability of the app owner.
Nicolas Sebrecht
@nicolas33
Oh, there's another concurrency issue you're pointing out.
I don't see how a repository could be requested non-compatible commands.
I think it's fine to start the naive approach about concurrency and see how it goes.
Jens Neuhalfen
@neuhalje
All “1-1” connections between folders "on the left" and folders “on the right” should be (relatively) painless.
Nicolas Sebrecht
@nicolas33
yes
(afk)
Nicolas Sebrecht
@nicolas33
I'll continue the code up to tomorrow.
Nicolas Sebrecht
@nicolas33
Hmm, there's no choice but pubsub at some point.
This is because Python queues can't pass queues.
That sucks.
So, everywhere we want to start workers dynamically, we must have a pubsub routeur with the queues already set and available.
Python is brillant sometimes....
Nicolas Sebrecht
@nicolas33
Good news. I'm making big steps in the global design. It's time to turn this into code, now...
Jens Neuhalfen
@neuhalje
that’s good news!
Jens Neuhalfen
@neuhalje
any news :)?
Nicolas Sebrecht
@nicolas33
I didn't have the time, yet. I'll let you know.
Nicolas Sebrecht
@nicolas33
@neuhalje Here is the rewrite: https://github.com/nicolas33/imapfw-rewrite
It's very early stage.
I don't expect contributions at this stage. However, it's made public so this can be reviewed.
Nicolas Sebrecht
@nicolas33
I've added jpg and a readme.