Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Repo info
Activity
  • Jan 30 2019 00:41
    gammazero commented #346
  • Jan 27 2019 00:59
    fuath starred wamp-proto/wamp-proto
  • Jan 26 2019 13:51
    deepaksood619 starred wamp-proto/wamp-proto
  • Jan 25 2019 12:53
    oberstet labeled #346
  • Jan 25 2019 12:53
    oberstet labeled #346
  • Jan 25 2019 12:53
    oberstet labeled #346
  • Jan 25 2019 12:53
    oberstet opened #346
  • Jan 21 2019 08:03
    Niputi starred wamp-proto/wamp-proto
  • Jan 17 2019 21:54
    karenpeng commented #345
  • Jan 17 2019 07:22
    oberstet commented #345
  • Jan 17 2019 07:21
    oberstet commented #345
  • Jan 17 2019 07:07
    oberstet commented #345
  • Jan 17 2019 01:44
    karenpeng starred wamp-proto/wamp-proto
  • Jan 16 2019 21:11
    karenpeng commented #345
  • Jan 16 2019 20:11
    karenpeng commented #345
  • Jan 16 2019 16:44
    meejah commented #345
  • Jan 16 2019 13:05
    KSDaemon commented #344
  • Jan 16 2019 12:20
    goeddea commented #344
  • Jan 16 2019 09:36
    KSDaemon commented #345
  • Jan 16 2019 09:26
    oberstet labeled #345
Entropealabs
@entropealabs_gitlab
looks like CI failed, but not positive it's related to my change
Justin Turner Arthur
@JustinTArthur
Do any of you know if parsing JSON strings as bytes based on the null-prefix is required for all fields of a WAMP protcol message, or only args and kwargs?
Entropealabs
@entropealabs_gitlab
What JSON parser are you using?
Justin Turner Arthur
@JustinTArthur
@entropealabs_gitlab does it matter? I thought the null-prefix was outside of the JSON RFCs
In my case, I’m letting the user choose between Python stdlib json, ultrajson, or rapidjson.
Entropealabs
@entropealabs_gitlab
so are you making a custom parser, I guess I'm not sure of the issue. if you are using JSON as the serialization, you just serialize and deserialize the whole array/message as JSON
the individual elements shouldn't be serialized independently
can you point to something specific in the protocol you are referring to?
Entropealabs
@entropealabs_gitlab
here's the JSON serializer for my Elixir implementation, https://gitlab.com/entropealabs/wampex/-/blob/master/lib/serializers/json.ex
super simple
I think strings have to be utf-8 per the spec
When using JSON as WAMP serialization format, URIs (as other strings) are transmitted in UTF-8 [@!RFC3629] encoding.
Justin Turner Arthur
@JustinTArthur
@entropealabs_gitlab, per the spec, bytes are packed as null-prefixed strings when using JSON serialization. I didn’t think Wampex had implemented that yet. https://wamp-proto.org/_static/gen/wamp_latest.html#binary-conversion-of-json-strings
But to me the spec is unclear for which parts of the message that rule applies to, if it’s not all parts
Entropealabs
@entropealabs_gitlab
ah so specifically sending binary data in JSON serialization
it would only apply to actual binary data that you want to send using JSON serialization
so like a raw jpeg image or something
Justin Turner Arthur
@JustinTArthur
@entropealabs_gitlab does that include the details and options message parts, or only args and kwargs?
Entropealabs
@entropealabs_gitlab
neither, if you have a key in your kwargs called raw_image the value would use the method you posted
and then you would send the serialized structure over the wire, your peer that receives the message wouldhave to know how to unencode that specific key
so that is best left up to the implementer, and not the library, although I guess you could create a binary type, that your client would know how to handle, but it's specifically mentioned that clients/router shouldn't have to look at the arguments passed
msgpack doesn't have that issue, as it can handle binary data natively
so you can just send the plain old binary data
Justin Turner Arthur
@JustinTArthur
@entropealabs_gitlab, are you sure? This is why I’m bringing it up, as the spec isn’t clear. It’s referenced in section 5.2.1 pertaining to serialization itself, which makes me think it’s a requirement for JSON serialization, i.e. up to library maintainers like us to include it
Entropealabs
@entropealabs_gitlab
I'm pretty positive, json is not a binary protocol, it's just mentioning how you would encode binary data in your JSON if you need to
Justin Turner Arthur
@JustinTArthur
I figured it was specifically for that reason that it’s a requirement in a WAMP implementation, to workaround lack of low-level support.
Entropealabs
@entropealabs_gitlab
how would you know that the value is binary data, and not just some other random data?
you could encode that in an object, within your arguments, but again, that would be up to the user, not the library
Justin Turner Arthur
@JustinTArthur
Because it’s prefixed with a null character. And if that is indeed how, then presumably we are not allowed to use null-prefixed strings for any character string purposes
Entropealabs
@entropealabs_gitlab
I don't think you want to scan every key and value in your arguments
Justin Turner Arthur
@JustinTArthur
It would be neat if it was up to the application layer, but if so, why mention a format for it in the spec? Someone with their kwargs[“raw_image”] can pack bytes however they want to.
@entropealabs_gitlab I definitely don’t, which is why I’m asking :)
Entropealabs
@entropealabs_gitlab
right, I think it's just a suggestion on best practices for doing to
"Binary data follows a convention for conversion to JSON strings"... follows a convention
Justin Turner Arthur
@JustinTArthur
@oberstet or @goeddea can you weigh in? Is it a best practice suggestion for application developers or a specification for WAMP implementors?
Entropealabs
@entropealabs_gitlab
Good call, I'm definitely not positive, but based on other statements in the spec about not looking at the arguments, I think it's at the application level
Justin Turner Arthur
@JustinTArthur
@entropealabs_gitlab for reference, autobahn-python does just that as part of the serialization/deserialization process.
Entropealabs
@entropealabs_gitlab
it looks for binary data?
Justin Turner Arthur
@JustinTArthur
yep
Entropealabs
@entropealabs_gitlab
interesting
Justin Turner Arthur
@JustinTArthur
It does it for every message part, including details, options, and string-packed URIs.
Entropealabs
@entropealabs_gitlab
it looks for binary data in the URIs?
Justin Turner Arthur
@JustinTArthur
Yes, as they are transported as character strings. Serializer code/Deserializer code
Entropealabs
@entropealabs_gitlab
well now I'm intrigued
Justin Turner Arthur
@JustinTArthur
It’s easy in the stdlib Python JSON mechanics to make it part of the parsing/unparsing process so you don’t iterate over your keys/values again following standard JSON parsing. As you can see in their implementation though, they won’t bother with the second-visit approach if you chose a C-based JSON parser; I may end up doing the same.
Entropealabs
@entropealabs_gitlab
seems like a nightmare, that would be a fun bug to find a few years down the road
somehow you have a different json parser
Justin Turner Arthur
@JustinTArthur
Jason for Elixer doesn’t look tooo difficult to extend in your case at least
Entropealabs
@entropealabs_gitlab
yeah, I don't think it would be a huge deal