These are chat archives for ProtoDef-io/node-protodef

5th
Apr 2017
Hans Elias J.
@hansihe
Apr 05 2017 00:01
unsure what you mean by that
oh
i see
well, that requires code generation
William Gaylord
@wgaylord
Apr 05 2017 00:01
So why are containers that are virtual not virtual_containers?
Hans Elias J.
@hansihe
Apr 05 2017 00:01
so it the compiler has to be aware of it
chibill: that mainly grew out of the way the IR in my compiler is structured. there is only one internal variant
we might want to change that
William Gaylord
@wgaylord
Apr 05 2017 00:03
so virtual_fields are virtual and different then a field but a container can be virtual or not..
seems odd but okay.
Hans Elias J.
@hansihe
Apr 05 2017 00:04
we might want to change that
I still have a couple of arguments defending it
if you want to hear them
William Gaylord
@wgaylord
Apr 05 2017 00:04
that post was after mine.
Hans Elias J.
@hansihe
Apr 05 2017 00:05
not on my side, but sure
Robin Lambertz
@roblabla
Apr 05 2017 00:07

TBF, you could just do

{
  "types": {
     "virtual_container": ["container", {
       virtual: true,
       fields: "$0"
    }]
  }
}

Or something like that anyway ? I think.

At least in node I think that works sorta :P
(Dirty hack though)
Anyway, g2g to sleep.
Hans Elias J.
@hansihe
Apr 05 2017 00:09
me too
talk to you later
Robin Lambertz
@roblabla
Apr 05 2017 09:38
So, some thought about protodef in general. Currently, minecraft protocol.json doesn't handle framing, compression, encryption, etc...
This is kinda sad, and should be remedied.
Which brings me to two fun topics. Protocol types, and recursive serialization.
There are essentially two protocol types : stream and framed protocols. In a stream protocol, we don't know the end of a "thing" until we've attempted reading it. In a framed protocol, we know the size beforehand.
Now, it is possible to encapsulate one type of protocol inside of another. For instance, TCP is a stream protocol, but it encapsulates the minecraft protocol, which (after 1.7) is framed.
Similarely, there are stream-based protocols built on top of UDP.
Protodef currently only handles one or the other really well, but it's not really possible to "go" from one type to the other. For instance, in NMP, we handle framing separately so that the protocol.json can assume that we live in a framed protocol.
There are other concerns : how do we bring back compression and encryption inside the protocol.json, where it belongs.
Robin Lambertz
@roblabla
Apr 05 2017 09:44
This brings me to kaitai struct. I love that project too much. In kaitai struct, they have a process keyword. https://github.com/kaitai-io/kaitai_struct/wiki/Processing-binary-data
Basically, it operates on a buffer, and it does shiz to it to transform it
The other small problem is what I used to refer as "pipeline management". Basically, previous packets in the protocol can activate or deactivate some features.
Robin Lambertz
@roblabla
Apr 05 2017 09:50
in minecraft, it goes like serialization -> framing -> compression (don't touch the length) -> encryption, but the last two start deactivated, and activates based on packets.
Hans Elias J.
@hansihe
Apr 05 2017 09:58
I kinda disagree with a lot of that, but we can have that discussion later when the issue is more relevant :)
Robin Lambertz
@roblabla
Apr 05 2017 09:58
I'm trying to fit this thing in the new format.
In a way that doesn't suck
:D
mhsjlw
@mhsjlw
Apr 05 2017 10:32
Do you guys think that Crystal has the potential to be used for creating games?
It has seamless C bindings with no C code, and it has ruby-like syntax with OOP
(and it compiles to native binaries with llvm so you can take it anywhere)
I'll come back to Crystal in two years when it becomes production ready
William Gaylord
@wgaylord
Apr 05 2017 13:32
Just going to say I thought ProtoDef was just for serialization.
mhsjlw
@mhsjlw
Apr 05 2017 13:44
No, that'd be useless
It's both ways
William Gaylord
@wgaylord
Apr 05 2017 13:46
I meant like it did not do networking.....
mhsjlw
@mhsjlw
Apr 05 2017 13:46
serialization != deserialization automatically
and it doesn't do networking
it can be used with networking libraries
and it is
but it can also be used for reading/writing binary files
William Gaylord
@wgaylord
Apr 05 2017 13:47
So why is ronlabla taking about networking....
mhsjlw
@mhsjlw
Apr 05 2017 13:48
compression and encryption are not networking
William Gaylord
@wgaylord
Apr 05 2017 13:48
He's talking about tcp and udp
mhsjlw
@mhsjlw
Apr 05 2017 13:49
Now, it is possible to encapsulate one type of protocol inside of another. For instance, TCP is a stream protocol, but it encapsulates the minecraft protocol, which (after 1.7) is framed.
unrelated to ProtoDef
the bottom line is, ProtoDef is not a networking library in any shape or form
Robin Lambertz
@roblabla
Apr 05 2017 14:05
I'd like it to be a networking library. But I'm mostly talking about future plans here @chibill. Things I'd like it to do, to make it more useful.
(Well, depends on what you mean by a networking library)
XD
I'd like the protodef format to be an actual, thorough description of all you need to read/write a protocol.
This means knowing on top of what it was built (tcp/udp/unix socket/whatever), how packets are formed, etc...
William Gaylord
@wgaylord
Apr 05 2017 14:22
Now that i think about it you could define TCP and UDP as protocols because thats really all they are... And its still not networking because the raw socket that sends them is the real network part.
mhsjlw
@mhsjlw
Apr 05 2017 14:24
can anyone else access digitalocean ?
i can't log in
{"id":"service_unavailable","message":"Failed to forward the request you made, please try again."}
does that happen for you guys ?
William Gaylord
@wgaylord
Apr 05 2017 14:25
same on that ling
link*
Robin Lambertz
@roblabla
Apr 05 2017 14:25
yup
mhsjlw
@mhsjlw
Apr 05 2017 14:25
hmm weird
Robin Lambertz
@roblabla
Apr 05 2017 14:25
that's a very pretty error screen
William Gaylord
@wgaylord
Apr 05 2017 14:25
maybe there Nginx/Apahce is broken?
mhsjlw
@mhsjlw
Apr 05 2017 14:25
yeah
i'm not sure what is up
status says everything is fine :P
All Systems Operational
Robin Lambertz
@roblabla
Apr 05 2017 14:26
oh wait, wow
It's firefox that's making the page all pretty :O
mhsjlw
@mhsjlw
Apr 05 2017 14:26
I can't put in a ticket... because I can't log in :D
Robin Lambertz
@roblabla
Apr 05 2017 14:26
It detects that it's JSON, and it brings the devtools up
mhsjlw
@mhsjlw
Apr 05 2017 14:26
good thinking
Robin Lambertz
@roblabla
Apr 05 2017 14:26
that's pretty amazing.
mhsjlw
@mhsjlw
Apr 05 2017 14:26
@roblabla :D
yeah
it's firefox 53
i think
Robin Lambertz
@roblabla
Apr 05 2017 14:27
IDK, I'm on dev edition :P
(Because dark skin XD)
mhsjlw
@mhsjlw
Apr 05 2017 14:27
oh ?
i need that
Robin Lambertz
@roblabla
Apr 05 2017 14:27
it's possible to get the dark skin on normal firefox
But it's a pain
so I just use deved :P
William Gaylord
@wgaylord
Apr 05 2017 14:28
Also roblabla you could make the packet stuff all the way down to the Link layer even... Because all of that is just fancy packaging around the data...
Also I found that scapy Might be neat to use for mc. :P http://scapy.readthedocs.io/en/latest/build_dissect.html I can build my own little layer.
Robin Lambertz
@roblabla
Apr 05 2017 14:31
@chibill yeah but there's not much point to that XD. I don't want to have to redefine MAC, IP and TCP just to define minecraft. However, I do want to document that minecraft is based on TCP
and I want protodef to be smart enough to understand "hey, it's TCP, so it's a stream by default"
William Gaylord
@wgaylord
Apr 05 2017 14:32
okay. I was just saying..
Robin Lambertz
@roblabla
Apr 05 2017 14:38
Yupyup