Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Activity
  • 05:52
    andrewchambers closed #223
  • 05:52
    andrewchambers closed #223
  • 05:49
    andrewchambers edited #225
  • 05:49
    andrewchambers edited #225
  • 05:49
    andrewchambers opened #225
  • 05:49
    andrewchambers opened #225
  • 05:48
    andrewchambers opened #224
  • 05:48
    andrewchambers opened #224
  • 04:17

    bakpakin on master

    New unmarshal proposal. Gives … (compare)

  • 04:17

    bakpakin on master

    New unmarshal proposal. Gives … (compare)

  • 01:55

    bakpakin on master

    Address #215 Also update docs … (compare)

  • 01:55

    bakpakin on master

    Address #215 Also update docs … (compare)

  • 01:47
    andrewchambers opened #223
  • 01:47
    andrewchambers opened #223
  • 01:21
    bakpakin closed #211
  • 01:21
    bakpakin closed #211
  • 01:21
    bakpakin commented #211
  • 01:21
    bakpakin commented #211
  • 01:19
    bakpakin closed #218
  • 01:19
    bakpakin closed #218
andrewchambers
@andrewchambers
not sure what I've done wrong.
I'll push a draft
Calvin Rose
@bakpakin
ok, ill take a look
andrewchambers
@andrewchambers
I seem to have corrupted something but was sort of blankly staring at the code to work out what lol
I am giving up temporarily for now, if you can think of a better way to do it too, be my guest :P the PR will act as a reminder anyway.
Calvin Rose
@bakpakin
Ah, order is important with when marshalling things, especially when it comes to references. Right now, every time a non-trivial Janet is marshalled, it is added to a table as a key, and the value is an incremented id. If we are marshalling a value already in this table as a key, we instead marshal a reference to the id value we get. This is so we can do circular references. Durin unmarshal, every time a non-trivial value is unmarshalled, it is append to an array. A reference is then just indexing into this array as we go.
The issue from CI is because unless values are added to the unmarshal array in the EXACT same order as they are added to the marshal table, the references are off
The position of the MARK_SEEN() macros are important
MARK_SEEN adds a value to the unmarshal table, and then increments the id
Oh, and if you try to change the order too much, you can create situations in which cyclic references become incorrect, or worse, loop forever
Calvin Rose
@bakpakin
Basically, cyclic references are the bane of marsh.c
andrewchambers
@andrewchambers
haha I see
I was wondering about that, I think I remember reading a blog post from Go about how they did the encoding/gob package
and it had some issues about some infinite cycles they were patching
i may be totally misremembering though
Calvin Rose
@bakpakin
There are some actually some pretty weird stuff to consider, like tuples and structs (immutable), need to be handled differently than mutable types
andrewchambers
@andrewchambers
I think if we can add an allocation limit flag
it will make the fuzzing work better
since we can mark too much allocation as not a bug
though maybe we want that :P
i might try add an AFL script for the unmarshaller
I guess i can do a patch just for this purpose anyway.
andrewchambers
@andrewchambers
I'm also quite liking the workflow
of using janet as the place to write unit tests for c code
Calvin Rose
@bakpakin
Haha
yeah the c functions aren't too hard to write, builds quickly, and you can repl things out
andrewchambers
@andrewchambers
yeah
I am using tweetnacl for encryption (for now)
and I can just generate keys and throw them around like they are nothing
so easy :P
I think technically I could have done this before with python or lua
but i feel more confident now with janet's code
can sort of understand everything
amalgamation is also awesome
You can rebuild with different C flags and use valgrind etc
Calvin Rose
@bakpakin
So I got a working version of unmarshal that lets you write abstract types that are segfault proof, as well as DOS proof. We do this by moving the allocation of abstract types inside the unmarshal function pointer.
@andrewchambers You also need to marshal the abstract header with janet_marshal_abstract, and then unmarshal the abstract type with janet_unmarshal_abstract. This let's us solve the cyclic reference problem.
andrewchambers
@andrewchambers
awesome :D
lots of fixes coming in the next release :)
I'll close my PR, its not needed anymore
andrewchambers
@andrewchambers
so my little backup tool is currently ~2 megs unstripped
janet + sqlite + some C code.
(statically linked)
it doesnt do much yet lol
andrewchambers
@andrewchambers
just generate a master key and client keys
andrewchambers
@andrewchambers
Anyone wanting some fun
try converting from a janet tuple based ir to this
and generate cfuns at runtime :D