Do you happen to know how most people get the length of a regular TCP stream? is there something in trio or another library, or do I need to be struct.unpack-ing it manually?
I've used approaches ranging from a stateful parser ("sans I/O" is a relevant term here) to just a dumb function that tries to decode an entire bytestring in one shot (and returns
None if it's incomplete)
the latter is simple to implement but accidentally quadratic
That's why you need a higher level protocol on top to do the framing
That's how early versions of HTTP worked
But it's not a very popular approach, because usually you want to send multiple messages and you don't want to have to set up a new connection for each one
And because the recipient might not be able to tell the difference between a successful transmission, versus one where they only received half the message and then the connection broke due to an error
for busy folks I've indexed the interesting bits of my trio talk
https://trio.discourse.group/t/talk-productive-concurrency-with-trio-pycon-jp/210/2
https://trio.discourse.group/t/talk-productive-concurrency-with-trio-pycon-jp/210/2
@belm0 btw next time you need a copy of the trio logo, there are vector versions in the repo
I’ve been considering SCTP support in the stdlib (socket module up through asyncio). Is there any precedent for a langauge runner having an implementation for an IP protocol the OS/clib is missing? e.g. if on Windows or macOS,
IPPROTO_SCTP isn’t available, the stdlib could support socket creation with IPPROTO_SCTP, but behind the scenes the OS call is for SOCK_RAW/IPPROTO_RAW and the socket module actually provides the SCTP implementation. Does that sound terrible? Would there be precedent for something like Trio to supply that implementation if the OS and stdlib don’t?
@JustinTArthur the
socket module is a pretty direct exposure of the underlying system's socket API, so I don't think there's precedent for something like that at that level. Also I'm guessing the cpython team might be reluctant to take responsibility for shipping a SCTP implementation inside python.
For Trio we have a bit more flexibility; we could either support it directly in the
trio package, or as a third-party library sort of like trio-websocket. It's more of a judgement call on what we think will be best for users.
my first thought is that a separate library might actually be better, since SCTP is a somewhat specialized thing, so that way it gets more TLC from the folks who are experts in SCTP?
JustinTArthur @JustinTArthur nods
definitely the
trio package should give you everything you need to implement SCTP-on-trio, and if it doesn't then we should fix that
maybe the
trio.socket API is already what UDP users want? UDP is a pretty specific and low-level thing (compared to say TCP, which is just one example out of many streaming byte transports)
so the answer is, it just depends on whether someone comes up with a compelling higher-level API
trio.socket may truly be enough. The only thing I can think of is that the standard socket API might seem intimidating to someone who doesn’t know how to interact with it. Looking at asyncio’s datagram support, it’s a pretty thin wrapper on the socket API, so maybe there’s not much that needs to be done.
(Linux also has
SOCK_SEQPACKET for unix-domain sockets, for in-order delivery with framing, which can be super useful if you're OK with it being linux-specific. The key feature is that each send is atomic, so you can have multiple processes acting as producers and consumers on the same socket.)
but yeah,
SOCK_SEQPACKET is a Channel[bytes] (or whatever we end up calling it...), and if you're using a UDP socket for point-to-point communication only, you could model that as a Channel[bytes], though more generally I guess it's a Channel[Tuple[address, bytes]]
(for point-to-point UDP, I'm imagining a class where you specify who you're talking to when its instantiated, and then it automatically addresses outgoing packets to them, and automatically discards incoming packets from anyone else.)
and I also don't know how often people use UDP without also wanting to do other messy low-level socket stuff like setting socket options or passing flags to
recvmsg
(video conferencing)
does your networking code get to assume 1-1 communication, or does it have to be built to handle multi-party calls for generality even though they only make up a small fraction of all calls?
or hmm, I guess if you're using a hub-and-spoke topology with muxing through some central server then all the clients can assume 1-1 communication
@belm0 also fyi I showed your slides to spouse and some friends and now they're all saying things like "Squeeeee!!!!!! OMG" and looking at the different outfits
