Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Repo info
Activity
  • Feb 06 20:52
    pre-commit-ci[bot] synchronize #522
  • Feb 06 20:52
    pre-commit-ci[bot] edited #522
  • Feb 06 15:41
    davidbrochart commented #251
  • Feb 04 14:08
    gwk opened #528
  • Feb 02 14:01
    ggardet commented #508
  • Jan 30 20:35
    pre-commit-ci[bot] synchronize #522
  • Jan 30 20:35
    pre-commit-ci[bot] edited #522
  • Jan 25 10:39
    gschaffner commented #526
  • Jan 25 09:11
    gschaffner commented #525
  • Jan 24 17:07
    uSpike closed #526
  • Jan 24 17:07
    uSpike commented #526
  • Jan 24 16:34
    agronholm commented #526
  • Jan 24 16:33
    agronholm commented #526
  • Jan 24 16:29
    uSpike commented #526
  • Jan 24 15:28
    agronholm commented #374
  • Jan 24 15:26
    reidmeyer commented #374
  • Jan 24 15:22
    agronholm commented #374
  • Jan 24 15:15
    smurfix commented #374
  • Jan 24 15:09
    reidmeyer commented #374
  • Jan 24 15:04
    agronholm commented #374
graingert
@graingert:matrix.org
[m]
You only need to add one extra async and await up the call stack
Michael Adkins
@madkinsz
You can run the sync code in a worker thread then send the async function back to the event loop thread, it gets complicated fast though.
graingert
@graingert:matrix.org
[m]
But there's no need for that here just for a list comprehension
Nicolas Epstein
@nilueps

i'm using an event to trigger the cancellation of a nested scope, but it's not behaving as i would expect. Looks something like this:

cancel_event = Event()
async with create_task_group() as tg:
    tg.start_soon(long_running_task, cancel_event)

def long_running_task(cancel_event):
    try:
        async with create_task_group() as tg:
            tg.start_soon(some_task)
            await cancel_event.wait()
            tg.cancel_scope.cancel()
    except get_cancelled_exc_class():
        # cleanup stuff...
        raise

set()-ing the event triggerr the cancellation, but the exception never catches anything and the cleanup logic doesn't run. am I missing something?

Michael Adkins
@madkinsz
I believe cancellation only raises that exception in tasks within the cancel scope, so outside of the task group there won't be an error.
Nicolas Epstein
@nilueps
right, but the following syntax fares no better:
async def long_running_task(cancel_event):
    async with create_task_group() as tg:
         try
            tg.start_soon(some_task)
            await cancel_event.wait()
            tg.cancel_scope.cancel()
        except get_cancelled_exc_class():
            # cleanup stuff...
            raise
Nicolas Epstein
@nilueps
a runnable example (i'm sure i must be missing something obvious):
from anyio import Event, create_task_group, sleep, get_cancelled_exc_class, run


async def task(cancel_event: Event):
    async def long_task():
        while True:
            await sleep(1)

    async with create_task_group() as tg:
        try:
            tg.start_soon(long_task)
            await cancel_event.wait()
            tg.cancel_scope.cancel()
        except get_cancelled_exc_class():
            print("Never printed")
            raise


async def main():
    cancel_event = Event()
    async with create_task_group() as tg:
        tg.start_soon(task, cancel_event)
        await sleep(2)
        cancel_event.set()


run(main)
14 replies
Michael Adkins
@madkinsz
I've replied with a working example in the thread.
graingert
@graingert:matrix.org
[m]
Does it behave the same on trio?
Really the cleanup should handle being awaited in a cancellation
You should be able to refactor your code to aclose forcefully without shield at all
(I think shield is an antipatten)
Michael Adkins
@madkinsz
Just wondering, how do you avoid a shield when making async calls in a cancelled scope like that?
graingert
@graingert:matrix.org
[m]
There's two kinds of cleanup typically, returning a connection to a pool or terminating a connection
When terminating a connection you let aclose_forcefully Peirce your protocol stack to a socket.close call and let the kernel handle waiting on the connection
When returning a connection to a pool your pool maintains a task group that handles graceful termination
Michael Adkins
@madkinsz
I see. For my use-case, I need to report that the task was cancelled via an HTTP API call.
graingert
@graingert:matrix.org
[m]
Yeah so that's sorta like the pool case
You'd create a nursery responsible for reporting errors start reporting tasks in that
So you'd be like
async with reporter() as r:
    async def some_thing()
        try:
            await foo()
        except BaseException as e:
            r.report(e)
    ...
Michael Adkins
@madkinsz
Ah I see, requires a reporter to be passed around. Right now we have a async with report_crashes(): ... which captures and reports on a wide range of errors. It's actually outside the our cancel scope so we don't normally need shielding but we don't have control over how the user is running their event loop so there may be a cancel scope outside of our code and we need to report that failure still.
graingert
@graingert:matrix.org
[m]
That's the cleanest way of doing that
But you can use a contextvar to pass stuff through code that doesn't know about your reporter tool
If the user cares about error reporting they should open an async context manager for you to run a nursery to handle that in
graingert
@graingert:matrix.org
[m]
Yeah there should always be a place where a user of @madkinsz: library can introduce an async with above their cancellation
Michael Adkins
@madkinsz
That makes sense. Our async users are most likely to be the ones that are capable of doing so as well. What's the downside to the shielded cancel?
Alex Grönholm
@agronholm
it could indefinitely delay the enforcement of the cancellation
I prefer setting a reasonable timeout with move_on_after(..., shield=True):
Alex Grönholm
@agronholm
just spent another night fighting that last failing cancellation test
no success :(
Nicolas Epstein
@nilueps
i'm currently porting an asyncio program to anyio, is there a wait to start an async task in a sync scope in a similar fashion as with asyncio.create_task(coro)?
graingert
@graingert:matrix.org
[m]
Yeah you use TaskGroup.start_soon
Nicolas Epstein
@nilueps
TaskGroup itself or a TaskGroup() instance?
graingert
@graingert:matrix.org
[m]
an instance
Nicolas Epstein
@nilueps
ok that's what i figured... for deeply nested calls, is there a way to recover the encapsulating tg, or is it necessary to drill down through the call stack passing along the instance?
Alex Grönholm
@agronholm
@nilueps you could use a context variable, but I'm not sure that's a great idea from a code readability standpoint
Nicolas Epstein
@nilueps
I agree... I think I'm going to have to rethink the program architecture a bit. Thanks for the help
graingert
@graingert:matrix.org
[m]
@nilueps: out of interest what's the program?
it's a mess, don't judge xD
Tobias Alex-Petersen
@tapetersen
Is there a reason TaskGroup.start_soon is annotated to require a Coroutine and not accept a general Awaitable (asking because I want to schedule an async generators aclose method which is an awaitable and get type complaints (it seems to work in this case and I can cast it to fix the "error")
Or rather it needs a Callable returning a Coroutine but couldn't that still be loosened to an Awaitable?
Alex Grönholm
@agronholm
@tapetersen this has been changed in the upcoming v4.0
8 replies
in master we accept any awaitables
Nicolas Epstein
@nilueps
Why can't I pass keyword arguments to start/start_soon?
graingert
@graingert:matrix.org
[m]
I think it predates positional only arguments
Nicolas Epstein
@nilueps
predates?
Michael Adkins
@madkinsz
That's what they do in the Python stdlib — keyword arguments are reserved for settings that are passed to the event loop (or similar) itself.
With positional only arguments, you could do a more elegant interface, but those were added later. https://peps.python.org/pep-0570/
Nicolas Epstein
@nilueps
interesting, ok. thanks for the info!