For trio that is, as I just never managed to find asyncio efficient in the same way trio is. That's why I think I'll stick to watching out for future httpx and asks commits rather than trying to move everything to aiohttp
Asks is great a part from a few issues such as proxy support and handling redirects, while httpx looks promising with it's active maintenance
But I have yet to make httpx work with nurseries and the library itself has yet to implement proper parallel requests
I wouldn't worry about that "parallel requests" thing. I think most people don't really want their HTTP library implementing its own concurrency. You want your HTTP library to give you primitives to work with single requests, and then you want to use those from inside a more general concurrency library.
IIUC the "parallel requests" mentioned in their docs is an idea for implementing some basic concurrency in the library itself for users who for some reason can't use a "real" concurrency framework, but nobody else has anything like that and it's fine.
Florimond and he showed me that there was a request pool limit that wasn't explicitly explained in the docs, so I kept getting blank exceptions because I was trying to make too many requests at once with a session
But it's all good now
question, I'm sitting and reading the docs, and it says to not, as a general rule, pass a timeout parameter. Is there an idiomatic way of applying a timeout to the entry of a context manager block rather than all operations within it. I'm thinking specifically of something like
with connect_to_something() as conn:
await do_many_things_with_conn()Its unclear to me how I would apply a cancellation scope to the conn context manager from calling code without passing in a timeout.
but yeah, if you specifically want timeouts on some sub-part of a larger operation and that needs to be integrated into the operation's logic, then passing in a timeout argument can make sense
nurseries have the basic "wait for a bunch of stuff to finish" part built in, and then for collecting up the results it depends on what you want
(there are a bunch of details that vary depending on what you're doing... how do you want to keep track of which result goes with which operation? do you want them incrementally as they complete? if one of the worker tasks has an unhandled exception, what do you want to do?)
we probably will add some helpers eventually, maybe using https://github.com/python-trio/trimeter, but that hasn't happened yet
@rzeigler It's also possible to do the following:
-create a 'timeout' send/receive channel pair
-launch a task that sleeps for N seconds, then send a message to the timeout send channel
-inside the nursery, wait for messages on the timeout receive channel and call cancel_scope on the nursery when a message arrives
Is anyone up for replying to https://trio.discourse.group/t/develope-assembly-line/235 ? I've been meaning to but haven't had enough brain
Hey posting here 'cause I'm not sure where else to ask. I've been off-and-on working on running wxPython on top of trio, and would like some input. Mostly just about if I'm abusing trio nurseries, but also other criticism is welcome. Current work is here.
"IIUC the "parallel requests" mentioned in their docs is an idea for implementing some basic concurrency in the library itself for users who for some reason can't use a "real" concurrency framework, but nobody else has anything like that and it's fine." - This is worth digging into, yeah...
The parallel requests proposal in httpx (which might well just get dropped) was there for two reasons...
- To allow existing threaded-concurrency frameworks to be able to make large numbers of requests concurrently. (Because it's using async under the hood, rather than spinning up a thread-per-request.)
- Because asyncio at 3.8 is a bit naff, and still doesn't give you a nursery primitive, so providing folks with a sensible "we'll keep HTTP request tasks properly scoped for you and deal with failure cases in dependable ways" seemed worthwhile.
The first one of those two was the main driver for the proposal. As it is, I'd expect it's probably something we'll just drop - it's not really needed. (And it's better that asyncio focuses on getting the basics in place, rather than us having to work around that on behalf of our users.)
@ItsLiamCo_twitter - "I have yet to make httpx work with nurseries" - That's surprising. Any chance you'd be able to reduce that down to a simple reproducible case, and open an issue around that?
@lojack5 hello, I have a
@lojack5 Have you seen PySimpleGui?
It has WX part as well. Earlier I used trio with PySimpleGui and it was really easy to work with.
It's somewhat moot for httpx, since we'd already done the "make this work on trio and asyncio" by the time we knew about that. (We could perfectly well switch over to anyio, but it doesn't buy us a lot.) But the task groups specificially I'm interested in since eg. if I'm talking folks how to spawn tasks in an asyncio context, I really want to be able to point them at a sensible primitive like that.
I believe that Yury Selivanov has an asyncio TaskGroup implementation essentially complete, but it's not available yet.
Btw folks, we have several PRs ready for review, both simple and not-so-simple :-)