Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Repo info
Activity
    Jeremy Whitlock
    @whitlockjc
    Long story short, json-refs is an NPM module for supporting JSON Pointers and JSON References. It was originally part of an OpenAPI/Swagger library but broken out into its own project for easier reuse.
    Paul Bryan
    @pbryan
    Alright, makes sense.
    Jeremy Whitlock
    @whitlockjc
    While working on OpenAPI tooling and even some direct consumers, it has been requested the ability to merge values resolved via $ref. Simple merging would mean allowing neighboring properties beside the $ref and merging the properties of the JSON Reference object and the resolved object. That use case seems simple, of course some ruling for conflicts where both have the same property.
    But I could see more complex use cases where the JSON Reference object had some excludes/includes capability and/or other things giving the author more control over the resulting object.
    Instead of doing this in a vacuum, I figured I'd reach out.
    Paul Bryan
    @pbryan
    Hmm, seems like semantically, would be better to have a "merge" operation take a JSON document as an argument (or $ref if external) and apply a merge.
    So, something like JSON Patch, but with somewhat different rules, such as how to handle conflicts, for example.
    Jeremy Whitlock
    @whitlockjc
    That's what I was thinking but couldn't the input be the $ref container itself? That's the use case first requested by customers.
    Paul Bryan
    @pbryan
    $ref was always intended to be, well, just a reference to some (often external) JSON document. The semantics were pretty much: treat this $ref as the referenced document itself.
    Jeremy Whitlock
    @whitlockjc
    Yep. That's how json-refs does it.
    Paul Bryan
    @pbryan
    The idea of merging then assumes you want to do something with this document (or referenced document). In this case, you would want to combine it with something else.
    I don't know that trying to "reference" merging two (or more?) documents makes sense without some structure outside of JSON Reference itself.
    I like the idea of a JSON Merge document, whose purpose is to express the requirements around merging two documents.
    It could then presumably reference the documents to be merged.
    Another thought would be to say, reference such a merge document, with arguments containing documents (or references to documents) to be merged.
    Paul Bryan
    @pbryan
    Something like... {"$merge": [{"$ref": merge_doc}, {"$ref": doc1}, {"$ref": doc2}]}
    Or, even more general purpose, like document composition, where merge is but one feature.
    $compose
    (I'm just spit-balling here, off the top of my head...)
    Jeremy Whitlock
    @whitlockjc
    Any reason why the node containing the $ref couldn't be treated as an input?
    Paul Bryan
    @pbryan
    I'd argue that semantically, $ref is pretty much set. It means "replace with doc".
    Jeremy Whitlock
    @whitlockjc
    Fair enough. It's in the spec so that's how I've treated it, just wondering.
    Paul Bryan
    @pbryan
    I imagine in your merge case, you'd need maybe a list of rules regarding how to resolve merge conflicts, for example. This would likely result in a pretty sizable JSON object structure, so a document in an of itself.
    Different rules for different properties within the documents to be merged.
    Jeremy Whitlock
    @whitlockjc
    Sorry, I got pulled into a meeting.
    I could see users finding it intuitive to overload the $ref container for merging, as that's how they did it by mistake in the past. I don't disagree with you that it's semantically "replace the container with the referenced document/fragment" but any reasoning behind why this use case isn't ideal?

    test.json

    {
      "name": "Jeremy",
      "$ref": "./jeremy.json"
    }

    jeremy.json

    {
      "age": 37
    }

    resolved.json (merge)

    {
      "age": 37,
      "name": "Jeremy"
    }
    That's how some people expected it to work, but instead they get a warning now.
    I do think that it's confusing that in some cases neighboring properties are warnings or not...
    So...if you replaced $ref with $merge or $compose or something else, then you could handle this as well without the ambiguity. Thoughts?
    Jeremy Whitlock
    @whitlockjc
    That's the simple case. For the complex case, maybe $merge/$compose would allow an object that allowed you to specify the URI and the rules. Maybe like this:
    {
      "$ref": {
        "exclude": [
          "#/address"
        ],
        "include": [
          "#/age"
        ],
        "uri": "./jeremy.json"
      }
    }
    That's how I saw reusing/overrideing $ref working. If $ref was a string, it's a replacement. If it's an object, it's a merge.
    Shooting from the hip here. I just liked the idea that $ref itself doesn't change (replace the container with the resolved value) but you give the author control as to how the resolution happens.
    Thoughts?
    Jeremy Whitlock
    @whitlockjc
    Maybe figure out some way to take an existing thing (JSON Merge, JSON Patch, ...) and turn that into a $ref-like definition.
    Paul Bryan
    @pbryan
    I get that because a JSON Reference object is expected to be replaced, it could potentially be merged. But it raises all kinds of weird cases. What if the referenced document is not an object? As you point out, what are the rules for actually merging? (Which I project could get very complicated in some cases.)
    Jeremy Whitlock
    @whitlockjc
    Those would be errors/warnings? Seems like if you're suggesting a merge and the input isn't mergeable, that's a failure case.
    But I do agree, it's likely not to be simple. I know JSON Schema had a heck of a time with this.
    Paul Bryan
    @pbryan
    Yeah, it seems worthy of a document in an of itself.
    Hence, why defining a document format separately makes the most sense to me.
    pbryan @pbryan has to run away for a while...
    Jeremy Whitlock
    @whitlockjc
    No worries. Thanks for your time.
    James Parry
    @aurigajamesparry
    Hi @whitlockjc , Ih have a question if I may? I have have a yaml file which contains two $ref to the same local file at different locations. However i notice for the second reference it adds a "*ref_0" pointer rather than including the content. Is there a any way to always include the reference multiple times. If you need to see an example of what i mean, just let me know.
    to clarify i'm trying to use json-refs to reduce duplication of the same content in the yaml file for a openapi definition spread across multiple files.
    James Parry
    @aurigajamesparry

    for Example

    openapi: "3.0.1"
    info:
      $ref: ./info/index.yaml
    servers:
      $ref: ./servers/index.yaml
    paths:
      $ref: ./paths/index.yaml
    components: 
      $ref: ./servers/index.yaml # has been set as /servers/ just to prove the concept of importing same file twice in same yaml.

    running this through resolveRefs results in:

    ....
    servers: &ref_1
      - url: 'https://api.example.com/{basePath}'
        variables:
          basePath:
            default: /v1
    ......
    components: *ref_1

    Is there anyway to get the following output instead? (Ignoring the fact that of course components does not conform to the the standard, its just an example of the behaviour i'm seeing. with the pointers.

    ....
    servers: 
      - url: 'https://api.example.com/{basePath}'
        variables:
          basePath:
            default: /v1
    ......
    components: 
      - url: 'https://api.example.com/{basePath}'
        variables:
          basePath:
            default: /v1
    Jeremy Whitlock
    @whitlockjc
    Do you have a reproduction recipe you can share?
    Jeremy Whitlock
    @whitlockjc
    Oh...you want to duplicate the referenced thing, instead of each duplicate reference being a pointer to the first referenced thing.
    I could make a configuration for this but I can tell you it will likely have a runtime cost, especially for large documents.
    On the one hand, I think this is a consumer issue If you want #/components to be a deep copy of #/servers, that's your needs but from a resolution perspective, they come from same source and are not copied. On the other, I can see this being a configuration item to do it for you.
    Jeremy Whitlock
    @whitlockjc
    Submit a feature request please.