Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Activity
    Florent Gallaire
    @fgallaire
    I have the first request
    *.state
    But I don't succeed to construct the second request with this data structure
    any help ?
    glenveegee
    @glenveegee
    okay will take a look
    Florent Gallaire
    @fgallaire
    I'm afraid it's not possible
    but I prefer to ask experienced people
    glenveegee
    @glenveegee
    I've tried variations on this a few times and I always seem to think I can somehow do it. Unfortunately I don't believe it's possible because jmespath cannot set keys of object projections using values from the current context (as far as I know... someone please correct me if I'm wrong)
    What language are you hoping to use jmespath in because there are spec compliant libraries out there that do let you do this fairly easily
    I know of a few javascript/typescript libraries out there that extend the jmespath spec to include this functionality
    Florent Gallaire
    @fgallaire
    thanks a lot for your tries, yes I think jmespath needs well chosen lists to be efficient, but I prefer to avoid lists for some reasons (less repetitions and best UI to edit dicts)
    I use Python
    JensOAndersen
    @JensOAndersen

    Hey people, i hope this is the correct place to ask, i'm trying my hand with jmespath, and i've ran into some null value issues, i've tried looking around on stackoverflow and the documentation, but i havnt been able to find any solution to my issue:
    I've got the following:

    {
        "items": [{
            "val1": "some",
            "val2": null
        }, {
            "val1": "other",
            "val2": "value"
        }]
    }

    To which i apply the transformation:
    items[*].{newValue:join(' - ',[val1, val2])}
    However, it doesnt work, as items[0].val2 has the value 'null', is there some way to look through properties and replace null values with a default value, emptystring in this case.

    varadasandeep
    @varadasandeep
    trying to filter output of az pipelines variable-group variable list but failing in all ways

    Name Is Secret Value


    bearer_token False
    org_id True
    org_id1 False dhsakjfhakjshfkjdahf
    owner_id True
    owner_id1 False mdfjsadhfkjsdahfkjsdhf
    password True
    password1 True
    projectid False
    username False testtest
    username1 False test

    need the Name column
    Jan Walzer
    @jwalzer

    Hi! I need some help with a json query. My data (simplified and to yaml) is

    network:
      ipfabric:
        lab:
          links:
            - nodes:
                "h01":
                  interface: "swp1"
                  ip: 192.168.253.180
                "h02":
                  interface: "eth0"
                  ip: 192.168.253.250
              networks:
                ip4:
                  cidr: 192.168.253.0/24
              bfd: yes
              bgp: yes
            - nodes:
                "h01":
                  interface: "swp2"
                  ip: 192.168.23.180
                "h04":
                  interface: "eth2"
                  ip: 192.168.23.250
              networks:
                ip4:
                  cidr: 192.168.253.0/24
              bgp: yes

    I'm querying for all interfaces on "h01" with ipfabric.*.links[*].nodes.h01.interface[] which works fine ... But:

    I want now to query for "all interfaces on h01 where bfd is enabled"
    ... I tried with different variations on ipfabric.*.links[? bfd == `yes`].nodes.h01.interface[]
    can somehone point me to the direction?
    Oryulin
    @oryulin

    Based on your sample data structure above, I "think" this is what you were hoping for:

    ipfabric.lab.links[?bfd == `yes`].nodes.h01.interface

    Since I do most of my testing in Ansible, one note I will add is that yes translates to True, so I had to wrap the yes in single quotes to stop it from processing as a boolean value. If you are using the same, you can change it to this to search for the Boolean True instead of the string yes:

    ipfabric.lab.links[?bfd == `true`].nodes.h01.interface
    Jan Walzer
    @jwalzer
    Yes, I'm also using this in ansible and I thought already of the yes->true transition, but I'm missing practice on json_query and so I wanted some opinions and help. I'll give it a try
    the '*' instead of 'lab' is correct, because there are other structures overlapping with different names
    Oryulin
    @oryulin
    Since you are using Ansible as well, here is my "testing" task:
    - name: JMESPath Testing
      vars:
        jmespath_query: >-
          ipfabric.lab.links[?bfd == `true`].nodes.h01.interface
      debug:
        var: network | json_query(jmespath_query)
    Oryulin
    @oryulin

    Re-reading what you wrote, I think I am gathering the picture. lab also has other similar environments that need to be searched for thus needs to be wildcard'ed if I am understanding right...trying this:

    ipfabric.*[].links[?bfd == `true`].nodes.h01.interface | []

    The thing about using the dictionary wildcard ipfabric.* converts the what you would expect to be dict results, actually converts it to a list. You can tell by doing this:

    ipfabric.*

    You will notice the top results is a list now, instead of a dictionary. To get past this conversion I simply flatten the list result from the asterisk and keep going. At the end you will notice that there is a list within a list, because of the flattening...so I pipe to another flatten and thus I think is the end goal you were hoping?

    Jan Walzer
    @jwalzer
    ah ... your explaination makes sense ... I'm trying this
    Jan Walzer
    @jwalzer
    BTW: I'm currently testing quite the same way as you do in ansible, so I'm probably not completely on the wrong track ;)
    so my current reasoning is: I'll try to match the "true" and I get an empty list. I'll try to match the negative (bfd != true) and I also get an empty list. So somehow it doesn't return anything after the filtering. I cut down from the end to see if I miss something.
    and it seems, one of the culprits is the convertion between "yes" and any kind of "true" - For testing purposes I changed my input data to literal true - and then I can in jmespath compare to true - but the same does not work for yes
    Oryulin
    @oryulin

    It seems like you found your issue but I had already typed this out, and it is a helpful gem for new users and testing so I sent it anyways:

    For JMESPath testing, I always start at the beginning and make my way down the rabbit hole, so to speak.
    Start by commenting out the | json_query(jmespath_query)

    Then if that works, uncomment it and start cutting down the jmespath_query filter one step at a time...like so:

    ipfabric.
    ipfabric.*
    ipfabric.*[]
    ipfabric.*[].links
    ipfabric.*[].links[?bfd == `true`]
    ipfabric.*[].links[?bfd == `true`].nodes
    ipfabric.*[].links[?bfd == `true`].nodes.h01
    ipfabric.*[].links[?bfd == `true`].nodes.h01.interface
    ipfabric.*[].links[?bfd == `true`].nodes.h01.interface | []

    If you don't see results from the first comment, then you know the variable network is not the right one you should use in yours...look at where your data is coming from and change accordingly. If its in the jmespath filter, you will see it break (empty results) eventually and know where your query needs to be updated.

    Jan Walzer
    @jwalzer
    and I just now found out, that: If its a boolean (yes) then I don't need to compare to anything, but can just do an ipfabric.*[].links[?bfd].....
    @oryulin yep, thats how I normally construct this stuff to. This time I had to adjust the existing(and working) query for that new parameter, and thought, it could be so easy. But it was an insightful chat, nevertheless ... thank you
    Oryulin
    @oryulin
    Absolutely @jwalzer, glad I could be of assistance before my work day started.
    Jan Walzer
    @jwalzer
    hehe .. here in Berlin the workday is about to end soon ;) -- but to summarize my understanding: Because of the flattening (that I'm doing nevertheless at the end) my old query worked without the filtering. But because of the .. thing flattening also the filter didn't work, and I had to change to .[]. to avoid that flattening. Right?
    Oryulin
    @oryulin

    Your original query was accurate in my opinion...I think the only problem you had was due to ipfabric.*.links as that would never yield results because the asterisk converts the dict to a list of matching dicts... so we had to deal with that by adding the flattener right after the asterisk:

    ipfabric.*[].links

    The rest was on par I believe (minus of course Ansible's fun yes/no to boolean good times). :)

    You could also pipe it again if that makes it look more visually appealing too, semantics to me.
    ipfabric.* | [].links[?bfd].nodes.h01.interface | []
    Jan Walzer
    @jwalzer
    I have to admit, I'm not yet so natural with the syntax of jmespath and I'll need more practice on that before the semantics are intuitive for me. But at least I could solve that problem, and I'm sure I'll get to the power of jmespath in the future again, before trying uggly things with loops in ansible/jinja
    Oryulin
    @oryulin
    Not that this is the write chat for this statement, but JMESPath > Ansible loops in the sense of speed and efficiency. If you are only data processing/manipulation, I would stay FAR away from Ansible loop as its resource costly/timely. JMESPath can filter hundreds of thousands of list/items in seconds whereas that same processing in Ansible loop...you'll be sitting a while. If you are doing actual work like API calls, then you sort of have to do Ansible loop.
    Jan Walzer
    @jwalzer
    Jep, thats no question at all -- I've seen lots of uggly ansible code with recursive loops down the hierarchy, just to find some objects or keys in data structures, that are possible with a oneliner, exactly like the one above. Luckily, Infrastructure managed with ansible isn't that ofthen so large, that the timing issue really matters, but of course, it's also about complexity and maintainability of the code, and a single, readable oneliner is much better in that metric to
    Oryulin
    @oryulin

    @jwalzer -
    Be sure to use this little nugget with JMESPath and Ansible if you ever attempt to use functions in JMESPath with Ansible such as map, ends_with, et...:

        - name: JMESPath Testing
          vars:
            jmespath_query: >-
              ipfabric.* | [].links[*].nodes.h01.interface | [] | [?ends_with(@,'1')]
          debug:
            var: network | json_query(jmespath_query)

    By all accounts this should work but you find that you get a "type" error:
    In function ends_with(), invalid type for value: swp1, expected one of: ['string'], received: \"unknown

    This is due to Ansible's data types not matching the JSON spec is what I believe is the root cause. To fix it simply add the to_json and from_json syntax before the json_query and profit.

        - name: JMESPath Testing
          vars:
            jmespath_query: >-
              ipfabric.* | [].links[*].nodes.h01.interface | [] | [?ends_with(@,'1')]
          debug:
            var: network | to_json | from_json | json_query(jmespath_query)
    Jan Walzer
    @jwalzer
    this must be quite a "LolWTF!?!?!"-Moment then *g
    and these are the people, telling me, that perl code is uggly
    Christoph
    @cruepprich
    Hi all. I have a question regarding querying time stamps. In the example below I would like to filter the items that are within the last 10 days of the current date. Is there a JMESPath function/feature that allows me to do that?
    So when I run the query on September 16th it should return the first item only (backup b3).
    [{
        "backup": "b3",
        "time_ended": "2020-09-10T14:55:39.840000+00:00"
    }, {
        "backup": "b2",
        "time_ended": "2020-08-25T15:40:48.740000+00:00"
    }, {
        "backup": "b1",
        "time_ended": "2020-07-10T15:40:48.740000+00:00"
    }]
    alonsocamaro
    @alonsocamaro
    Hi everybody
    where can find a jmespath online expression evaluator
    ?
    I find jsonpath ones but not jmespath ones
    more precisely, the one in jmespath.org doesn't always update when fields are updated
    so I'm looking for another one
    Oryulin
    @oryulin

    @alonsocamaro - THe homepage for JMESPath can be used as an online evaluator if desired. Its a real-time test and you can manipulate the search and json structure that will be searched.

    https://jmespath.org/

    I have not ran into an issue where it has "failed" for me thus far unless there is an error in my syntax which is most likely the cause of your not updating. That is one area where it lacks is it doesn't let you know when the syntax has an error so its likely assumed to be a web-based error. If you have a specific scenario/filter feel free to type it here and if I am paying attention will attempt to help.

    Charles Albrecht
    @karillo_gitlab

    I asked this on stackoverflow as well, but thought I'd ask it here to try and get some traction.

    AWS's list-secrets has an odd way of listing version strings as keys:

    {
      "SecretList": [
        {
          "Name": "sandbox_config",
          "Description": "sandbox config file",
          "SecretVersionsToStages": {
            "2895f1a4-bcae-46b2-9a45-4f06f490d8ed": [
              "AWSCURRENT"
            ],
            "336a5030-cd81-4626-a100-1aa68b70b5b1": [
              "AWSPREVIOUS"
            ]
          }
        }
      ]
    }

    I'm trying to figure out a query that will let me pull out the versionkey where AWSCURRENT is in the value array.

    This seems like it's close:

    SecretList[].{Version:SecretVersionsToStages.*[?@==`AWSCURRENT`]|[]|[0] }

    But I'm clearly missing something, since that just spits back AWSCURRENT to me.

    PeiSong
    @naivefun
    Hi, how can I do simple math with jmespath? like length(values(@)) * 2 ?