python-trio/general

Discussion of Trio, a friendly Python library for async concurrency and I/O

Charles

@norwood867

right

NoskLo

@nosklo_gitlab

Another option is to make a scoped object, using metaclasses

async with MyClass(...) as myinstance:
     await myinstance.do_stuff()

Charles

@norwood867

ok, that was where my thinking was going

should i hand off the open channel to the nursery running in main? or just leave it in the object? i’ll have a dozen or so of these running at a time.

NoskLo

@nosklo_gitlab

carefully designing where to open the nursery is where you really start reaping the benefits of using structured concurrency in first place

the nursery is a tool to prevent background tasks from outliving it - it will wait all tasks inside it to finish when the async with block is over

if the background tasks spawned by the object won't outlive the object itself, it makes sense to bind the nursery to the lifetime of the object (using one of the asynccontextmanager methods discussed above)

creating more nurseries has virtually no cost so you should use it as a tool to enforce your concurrency structure

Charles

@norwood867

ok, that is what i was wondering about, the cost

and the back task and object are linked until death so that will work.

thank you. trio and its structure seems to stretch the limits of my brain.

but it is so powerful and has made a number of things i’ve built so much easier. :)

NoskLo

@nosklo_gitlab

the nursery at first seems a constraint, but after using it for a while you can see that it makes easy to reason about your code's concurrency model

code using it correctly becomes more readable with time

NoskLo

@nosklo_gitlab

@norwood867 the tricycle helper module has a BackgroundObject class you can inherit that automatically creates a nursery bound to the lifetime of the object

you can even make it auto-kill the background tasks when the object scope ends

András Mózes

@mozesa

@oremanj Thanks

now it is even clearer :smile:

Charles

@norwood867

@nosklo_gitlab , thanks. i’ll try it after i get this object working :|

András Mózes

@mozesa

I have checked the metaclass as well and yes there is the yield self

jonwang72

@jonwang72

Anyone know the cause of trio.Cancelled: Cancelled? Don't really understand the explanation here

Nathaniel J. Smith

@njsmith

@jonwang72 well, it means your code was cancelled :-) there are three ways this can happen: (1) a timeout, (2) calling .cancel() on a cancel scope, (3) if an unhandled exception hits a nursery, then all the other tasks inside the nursery are cancelled

goodboy

@goodboy

so what's the replacement for FdStream if I didn't want to check the changelog ;P

Joshua Oreman

@oremanj

FdStream has never existed on Windows

There is no public exposure for “stream over Windows named pipe” as of yet

goodboy

@goodboy

ahhh

so then trio-run-in-process doesn't have windows support

@oremanj is there an issue I can link to on this?

Joshua Oreman

@oremanj

There might be something private you can import from trio._windows_pipes, but that requires the file to be opened in overlapped mode, and I think normally your standard I/O streams won’t be, even if backed by a named pipe

trio #174 I think?

yup: python-trio/trio#174

goodboy

@goodboy

awesome thx

Nathaniel J. Smith

@njsmith

one of the requirements to make something a valid stdio stream is that it can't be opened in overlapped mode

because........ because.

goodboy

@goodboy

lol

i linked to the swaths of discussion on it in the trip issue i just opened

jonwang72

@jonwang72

@njsmith Thanks

jonwang72

@jonwang72

I seem to be having trouble running pytest with trio functions. When I run something like the below test:

async def test_hello(nursery):
    await nursery.start(hello)

async def hello(task_status=trio.TASK_STATUS_IGNORED):
    task_status.started()
    await trio.run_process(['echo', 'hi'])

I'm getting a Cancellation

Nathaniel J. Smith

@njsmith

@jonwang72 oh, the pytest-trio special nursery fixture is automatically cancelled when the test itself finishes

The idea is that you normally put your actual test code inside the test function, and you use the nursery to start up background services that your test code needs

So once the test code in the main function is finished, the background services aren't needed anymore, so they get cancelled

Nathaniel J. Smith

@njsmith

You might be better off making your own nursery instead of using the fixture – the point of the fixture is mostly just the auto cancellation

Peter Sutton

@dj-foxxy

What's the best way to receive all data that's immediately readable from a socket?

Something like this:

async def recv_readable(socket: SocketType, size: int = 2 ** 8) -> bytes:
    data_parts = []
    while socket.is_readable:
        data = await socket.recv(size)
        data_parts.append(data)
        if len(data) < size:
            break
    return b''.join(data_parts)

Matthias Urlichs

@smurfix

I'd use a larger buffer (one that's larger than a typical message you'd read from the socket), but that looks reasonable.

Peter Sutton

@dj-foxxy

@smurfix Thanks for reviewing :). I'll be ok here typical message size is 1 byte :).

Quentin Pradet

@pquentin

@dj-foxxy I think you want is_readable() instead of is_readable

Quentin Pradet

@pquentin

or just use while True:, is_readable() is only needed in special cases

Joshua Oreman

@oremanj

@pquentin I think “all data that’s immediately readable” is one of those special cases :-)

Where communities thrive

People

Repo info

Activity