These are chat archives for ProtoDef-io/node-protodef

27th
Feb 2017
Hans Elias J.
@hansihe
Feb 27 2017 14:16
rom1504: what language were you thinking for the compiler?
I agree it should be some language other then elixir
but any language with some form of pattern matching would make things a lot easier
Robin Lambertz
@roblabla
Feb 27 2017 14:17
TBH, I'm fine with keeping it in elixir so long as we can easily distribute binaries from it
If we're going to rewrite it though, rust and ocaml are both viable options IMO. Maybe haskell, too, but I don't know it :P
Romain Beaumont
@rom1504
Feb 27 2017 14:18
oh I just meant I wanted to finish my js-specific to-js compiler
about a generic compiler, no idea, whoever feels like doing it can do it
(and can choose I mean)
Hans Elias J.
@hansihe
Feb 27 2017 14:19
I would probably do it in rust
just because I know it better than haskell, I don't know ocaml at all
Romain Beaumont
@rom1504
Feb 27 2017 14:19
@roblabla started a compiler in rust
I don't think it's online anywhere yet though, is it ?
Robin Lambertz
@roblabla
Feb 27 2017 14:21
Nope. And it's as rust-specific as you can get anyway
Hans Elias J.
@hansihe
Feb 27 2017 14:21
my current small project is almost done, I might pick this up as after
Robin Lambertz
@roblabla
Feb 27 2017 14:21
my plan was to use it to figure out the edge cases, more as a quick-n-dirty thing.
and then fold it back into the elixir compiler
But if we're going to change the language the compiler is written in then yeah :P
Hans Elias J.
@hansihe
Feb 27 2017 14:22
the elixir compiler is actually not all that big/complex. I think I could move it to some other language in a few days at most
having proper static typing would make it a lot more pleasant to work with as well, so it might be worth it
there is no easy way I know of to distribute elixir stuff as standalone programs, so that's a big minus for using it for this
Robin Lambertz
@roblabla
Feb 27 2017 14:24
Rust also supports cross-compilation, that's a huge plus. Being able to generate darwin or windows executables without dual boot is so great
Robin Lambertz
@roblabla
Feb 27 2017 17:10
Whoever thought it'd be a good idea to make rust's str clonable needs to die.
I just spent an hour looking for an error in my protodef compiler with this line :
let parent_type = get_type_name(&self.1["types"][get_type_name(ty)]).clone()
the code later complains about me borrowing self mutably, even though it's already immutably borrowed at this line
It turns out get_type_name returns an &str, and even though it is cloned, it is still dependant on the &self.1 borrow...
Rust has a few very silly edge cases TBH >_>.
Robin Lambertz
@roblabla
Feb 27 2017 17:24
Great news. I'm making progress in rust-protodef again.
I managed to properly separate the TypeGenerator from the Type (in a very ugly way though)
Hans Elias J.
@hansihe
Feb 27 2017 17:52
feel free to ping me if you have any problems, I think I am fairly well indoctrinated in borrowchk at this point :)
and the way &str.clone works kinda makes sense
Robin Lambertz
@roblabla
Feb 27 2017 17:53
how so ? I don't even understand what it could concievably do :|
Hans Elias J.
@hansihe
Feb 27 2017 17:53
if you have struct MyStruct<'a> { string: &'a str } and call clone on it
what else would you have it do?
Robin Lambertz
@roblabla
Feb 27 2017 17:54
meh. So is clone() on a &str a noop ?
impl<'a, T> Clone for &'a T where T: ?Sized
Hans Elias J.
@hansihe
Feb 27 2017 17:54
it does this for every reference
yeah
it's pretty much the only way it can work
Robin Lambertz
@roblabla
Feb 27 2017 17:55
Yeah I get that
but I find it to be a bit of a footgun tbh. Because I expect clone() to be a dirty way to fix my brwchk problems, but then if I get a ref for some reason it fails :P
Hans Elias J.
@hansihe
Feb 27 2017 17:57
yeah, I can see what you mean :)
Robin Lambertz
@roblabla
Feb 27 2017 17:57
(And the message is extra confusing too)
though I guess that could be fixed.
Hans Elias J.
@hansihe
Feb 27 2017 17:57
it could definitely give you a hint or something, yeah
Robin Lambertz
@roblabla
Feb 27 2017 18:08
@rom1504 just realized there's something missing to our current pstring/cstring definition in protodef. Something pretty crucial at that
The encoding of the string ^^.
We sort of just assume utf-8, which is arguably the right thing to do in "most" protocol, and is fortunately ASCII-compatible.
Hans Elias J.
@hansihe
Feb 27 2017 18:15
yeah, that's pretty critical
Robin Lambertz
@roblabla
Feb 27 2017 18:16
making an issue
Robin Lambertz
@roblabla
Feb 27 2017 18:22
ProtoDef-io/ProtoDef#16 open
Robin Lambertz
@roblabla
Feb 27 2017 18:29
well, only array, magic, restBuffer, ipAddress, ltriad and endOfArray left @mhsjlw :D
(and making sure the generated stuff, y'know, compiles 'n stuff :P)
Romain Beaumont
@rom1504
Feb 27 2017 18:31
Heh I guess didn't pull weird encodings on us yet that's why :p
Robin Lambertz
@roblabla
Feb 27 2017 18:31
inb4 "next mc version is latin1 only"
at least this time we'll be prepared mwahaha
Romain Beaumont
@rom1504
Feb 27 2017 18:34
*Mojang
Where did that word go
Haha
Robin Lambertz
@roblabla
Feb 27 2017 18:35
trying to get my compiled stuff to... well, compile is fun too.
Romain Beaumont
@rom1504
Feb 27 2017 18:35
Anyway it should be pretty trivial to add an encoding property in pstring
Optional and defaulting to utf8
Robin Lambertz
@roblabla
Feb 27 2017 18:35
Declaration-wise, it's easy.
Implementation-wise, there's a few question marks
How standard are encoding names, for one.
Romain Beaumont
@rom1504
Feb 27 2017 18:36
Well in node-protodef I think there's an "utf8" thing somewhere
Oh
Robin Lambertz
@roblabla
Feb 27 2017 18:36
And in nodejs' case, is there even a way to get a string in another encoding ?
I guess you have to go through buffer
now hmm
Robin Lambertz
@roblabla
Feb 27 2017 18:38
so let's say I specify iso-8859-1 encoding. In nodejs you'll want to transform it to utf8 on reading
and back on writing
Robin Lambertz
@roblabla
Feb 27 2017 18:38
woah, they actually have latin1 now
that's pretty cool
Romain Beaumont
@rom1504
Feb 27 2017 18:40
'hex' that gives me bad ideas
Robin Lambertz
@roblabla
Feb 27 2017 18:40
haha
Romain Beaumont
@rom1504
Feb 27 2017 18:41
let's use it to encode packet ids
errrrr....
:d
Robin Lambertz
@roblabla
Feb 27 2017 18:41
let's not :P
Romain Beaumont
@rom1504
Feb 27 2017 18:43
so I guess we'd need to specify which encoding must be supported by protodef implementation ?
Robin Lambertz
@roblabla
Feb 27 2017 18:43
well, I like everything, we should specify base expectations
specific impls can always support more
Romain Beaumont
@rom1504
Feb 27 2017 18:43
yeah
Robin Lambertz
@roblabla
Feb 27 2017 18:44
I think utf8 and ascii should be specified
close contenders are WTF8, UTF16 and latin1.
utf16 is "interesting" due to being multibyte
Romain Beaumont
@rom1504
Feb 27 2017 18:45
interesting in the sense it'll break everything sure
Robin Lambertz
@roblabla
Feb 27 2017 18:45
that's why I put scare quotes :P
Robin Lambertz
@roblabla
Feb 27 2017 18:59

Why we all love rust :

/// A bytecount of a structural datatype
enum Count {
    /// The structural datatype should read the inner datatype to know its size
    Type(Rc<Datatype>),
    /// The structural datatype is fixed in size
    Fixed(u64),
    /// The structural datatype should use the given JSON Pointer to find
    /// its size
    Ref(String)
}

Good luck being expressive like this in any other language.

Robin Lambertz
@roblabla
Feb 27 2017 19:05
Actually, you know what, let's actually make a FieldRef type :P
I'm almost there
Romain Beaumont
@rom1504
Feb 27 2017 19:36
nice
Robin Lambertz
@roblabla
Feb 27 2017 19:37
The guts of my compiler are in a severe refactoring need though. Everything's currently in one file, and fairly disorganized :P
Robin Lambertz
@roblabla
Feb 27 2017 19:56
and finally, it compiles !
(if it works is another matter)
Romain Beaumont
@rom1504
Feb 27 2017 19:57
and the most annoying matter is whether it's fast
well
it's annoying in node :p
in rust I think it should be okay right ?
Robin Lambertz
@roblabla
Feb 27 2017 19:57
Well, we're missing quite a few optimisations we did in nmp
Like, right now it deserializes everything, there's no way to selectively say "don't deserialize this packet, I don't need it"
Romain Beaumont
@rom1504
Feb 27 2017 19:59
yeah well
Romain Beaumont
@rom1504
Feb 27 2017 19:59
current nmp doesn't actually do that
and
Robin Lambertz
@roblabla
Feb 27 2017 19:59
this is the new protocol that compiles
Romain Beaumont
@rom1504
Feb 27 2017 19:59
current nmp is 100x slower than it could be
Robin Lambertz
@roblabla
Feb 27 2017 19:59
if you want to take it for a test drive
Obviously it's missing a few things though :P
Romain Beaumont
@rom1504
Feb 27 2017 19:59
could be = manually written code = elixir generated code
(= future js compiler code hopefully :p)
Robin Lambertz
@roblabla
Feb 27 2017 20:00
Also, my impl isn't currently zero copy
Which sucks because it could be :P
rofl
let mut v = String::with_capacity(read_int::<BigEndian>(read,2usize) ? as i16 as usize) ;
isize as i16 as usize
Hello casting my old friend. I've come to talk with you again.
mhsjlw
@mhsjlw
Feb 27 2017 23:57
ocaml would be perfect lmao
the haxe of protodef
inb4 writes protodef in haxe
still wouldn't be useful since you can't interop with the code effectively
ocaml would be interesting , writing a single toolchain that compiles to whatever language targets we choose