Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Repo info
Activity
    Sven Erik Jeroschewski
    @eriksven
    However, I am getting the response "405 Not Allowed". But I did not finde more insights in the Swagger Documentation that this is an expected Response Code. I used the same devops credentials that work fine for other calls. Can you maybe point me in a direction what is not alllowed? Probably, I am just missing something here. Thank You!
    Thomas Jaeckle
    @thjaeckle
    hi Erik - you are using /api/2/policies/{policyId} ? so is the /api/2 included?
    Sven Erik Jeroschewski
    @eriksven
    No :|, that was already helpful as I now get a 401 response which makes more sense to me.
    Thomas Jaeckle
    @thjaeckle
    Sorry for calling you Erik 😉 for 401 you were probably not authenticated. Did you provide credentials? User ditto password ditto for example..
    Atul Kumar
    @writetoatul_twitter
    hi, do ditto stores the data on mongodb on its own, or do we need to write a service for it?
    and how can i access that database?
    Thomas Jaeckle
    @thjaeckle
    hi @writetoatul_twitter - yes, Ditto stores data on MongoDB, you don't have to do anything on your own
    what do you mean by "how can i access that database?"
    Julian
    @JulianFeinauer
    @thjaeckle are there any known implementations of "history" services or something?
    To store traces for the different values
    Thomas Jaeckle
    @thjaeckle
    @JulianFeinauer you mean implementations for Ditto?
    Martin Hoppen
    @mhoppen_gitlab

    Hi. tl;dr: How can I address entries from JSON lists using attribute paths?

    In our information models, we have hierarchically structured twins with generic one-to-many associations containing objects of different specializations. Simplified Example: Class "Car" has 1-n-association (composite aggregation) "parts" to base class "Part"; "Engine" and "Chassis" are derived from "Part". In our Ditto / JSON implementation, we put instances of this hierarchical structure below the "attributes" key and realize associations as JSON lists. Simplified Example:

    {
        "thingID": "...",
        "policyID": "...",
        "attributes": {
            "class": "Car",
            "parts": [
                {
                    "class": "Engine",
                    "rpm": 1000
                },
                {
                    "class": "Chassis",
                    "id": "4711"
                }
            ]
        }
    }

    Question: How can I access "rpm" of the Engine using a Ditto attribute path?
    Thanks :-)

    Thomas Jaeckle
    @thjaeckle
    hi @mhoppen_gitlab - unfortunately accessing single array entries via the HTTP API via "attribute paths" is not supported yet - it is only possible to work on the complete "parts" array (e.g. retrieve it in total and modify it in total)
    what we suggest in such cases is to use a map structure and make the "class" the key, so:
    "parts": {
      "Engine": {
        "rpm": 1000
      }, 
       "Chassis": {
    ...
    }
    as searching in arrays is also not supported in Ditto, this would also be the better way in order to e.g. search for "existance" of a parts/Engine attribute
    Martin Hoppen
    @mhoppen_gitlab
    Hi @thjaeckle, I had seen that coming ;-) The problem with that approach (class as key) is that we can have multiple parts of the same type, e.g.
    {
        "thingID": "...",
        "policyID": "...",
        "attributes": {
            "class": "Car",
            "parts": [
                {
                    "class": "Engine",
                    "rpm": 1000
                },
                {
                    "class": "Wheel",
                    "id": "4711"
                },
                {
                    "class": "Wheel",
                    "id": "4712"
                },
                {
                    "class": "Wheel",
                    "id": "4713"
                },
                {
                    "class": "Wheel",
                    "id": "4714"
                }
            ]
        }
    }
    BTW I forgot to say thank you for your instant reply. So, thank you :-)
    Thomas Jaeckle
    @thjaeckle
    ah, yes ..
    we did not find a good solution yet for automatically indexing array entries in our MongoDB based search index, so as long as we don't have that solved, adding better support for working with arrays is not a high priority ;)
    Martin Hoppen
    @mhoppen_gitlab
    Do you happen to have a syntax in mind for such a feature (even if you did not implement it yet)? We have the same problem somewhere else (not Ditto) where we can parse the path ourselves. I thought about XPath-like addressing (although preferring indices from 0),
    e.g.
    parts[0]/rpm
    parts[id='4714']/class
    parts[class='Engine']/rpm
    Thomas Jaeckle
    @thjaeckle
    puh, that would for HTTP URLs be too fancy I think
    we orient that to the JsonPointer spec which only supports using integers for accessing arrays, so e.g.: parts/0/rpm
    see: https://tools.ietf.org/html/rfc6901
    Martin Hoppen
    @mhoppen_gitlab
    thanks. I will think about that.
    Julian
    @JulianFeinauer
    @thjaeckle yes
    Thomas Jaeckle
    @thjaeckle
    no, then I don't know of existing implementations
    Musti
    @mustii82
    Hi Guys I have some questions regarding the MQTT Connection and Ditto
    Julian
    @JulianFeinauer
    Hi @mustii82 😎
    ottlukas
    @ottlukas
    :wave: MQTT Mosquitto + Ditto = :heartpulse:
    Julian
    @JulianFeinauer
    Do you have any practical experiences, @ottlukas?
    Musti
    @mustii82
    @JulianFeinauer it was clear that you would be found here too :P
    Florian Fendt
    @ffendt
    Hi @mustii82, well then go for it and ask your questions ;-)
    ottlukas
    @ottlukas
    @JulianFeinauer Only a small test-setup with Mosquitto and Ditto on Minikube
    Julian
    @JulianFeinauer
    I never used Minikube
    Alexander Wellbrock
    @lionax_gitlab
    So today my whole ditto cluster broke down around the concierge complaining about unreachable thingProxy and I'm still figuring out what's going on. I'd like to consult the experts here, since you might have a clue on what's happening, but then again I don't want to spam this chat with error logs and troubleshooting. What do you think?
    Thomas Jaeckle
    @thjaeckle
    hi @lionax_gitlab - I'm on a run currently .. so just in short:
    my suggestion would be to open a GH issue for that in order to trackdown/trace it there
    a first suggestion/question: Did you set memory limits for the Ditto docker containers? because otherwise they might indefinitely grow and eventually crash, maybe event the guest OS
    so: would be good to have your configuration in that GH issue :)
    logs will of course also help
    Musti
    @mustii82
    Currently im trying to connect my Digital Twin (created on my local ditto cluster) to a device via MQTT
    im using this piggybackCommand which is mentioned at @thjaeckle Tutorial at ditto-examples https://github.com/eclipse/ditto-examples/tree/master/mqtt-bidirectional
    {
    "targetActorSelection": "/system/sharding/connection",
    "headers": {
        "aggregate": false
    },
    "piggybackCommand": {
        "type": "connectivity.commands:createConnection",
        "connection": {
            "id": "mqtt-example-connection-123",
            "connectionType": "mqtt",
            "connectionStatus": "open",
            "failoverEnabled": true,
            "uri": "tcp://test.mosquitto.org:1883",
            "sources": [{
                "addresses": ["ditto-tutorial/#"],
                "authorizationContext": ["nginx:ditto"],
                "qos": 0,
                "filters": []
            }],
            "targets": [{
                "address": "ditto-tutorial/{{ thing:id }}",
                "topics": [
                    "_/_/things/twin/events",
                    "_/_/things/live/messages"
                ],
                "authorizationContext": ["nginx:ditto"],
                "qos": 0
            }],
            "mappingContext": {
                "mappingEngine": "JavaScript",
                "options": {
                    "incomingScript": "function mapToDittoProtocolMsg(headers, textPayload, bytePayload, contentType) {const jsonString = String.fromCharCode.apply(null, new Uint8Array(bytePayload));const jsonData = JSON.parse(jsonString); const thingId = jsonData.thingId; const value = { temp_sensor: { properties: { value: jsonData.temp } },altitude:        {            properties:            {                value: jsonData.alt            }        }    };    return Ditto.buildDittoProtocolMsg('my.test', thingId, 'things', 'twin', 'commands', 'modify', '/features', headers, value);}"
                }
            }
        }
    }
    }
    firstly the command isn't working for me at all because I get the error that my "incomingScript" in the mappingContext is invalid
    Musti
    @mustii82
    so I used this piggybackCommand without mapping at all, which should work with the full JSON becauseMapping is optional
    according to the logs its connecting to the topic /ditto-tutorial/#
    but when I publish the json to topic: ditto-tutorial/# or ditto-tutorial/my.test:octopus the digital twin does not change
    Thomas Jaeckle
    @thjaeckle
    hi @mustii82
    the "incomingScript" is probably wrongly escaped - as it uses single quotes inside and I guess that you put it in a cURL command having single quotes around the payload
    when creating the connection without the script: in which format do you send payload?
    When you send in Ditto Protocol JSON, your twin should be updated. When you however followed the tutorial, that can however not work without the "incomingScript" which converts your messages to Ditto Protocol JSON.
    Here is an Ditto Protocol JSON example how to modify a complete twin: https://www.eclipse.org/ditto/protocol-examples-modifything.html
    Julian
    @JulianFeinauer
    General question:is the incoming script something which is executed once for each payload?
    Florian Fendt
    @ffendt
    Yes, that's right.
    Alexander Wellbrock
    @lionax_gitlab
    Does ditto devops support hashed passwords for devops and status auth config options? It's not mentioned in the config file or the docs. Right now it's possible to query these passwords in plaintext via rest calls if you have access to devops section
    Thomas Jaeckle
    @thjaeckle
    no, it does not
    if you have access to /devops section, you already have the devops password anyhow ;)
    Alexander Wellbrock
    @lionax_gitlab
    That's true, but not necessarily the status password - I got access to credentials I'm not necessarily meant to have... but it's really a minor issue that just caught my attention :D
    Thomas Jaeckle
    @thjaeckle
    ok - the devops password is like the super-duper-admin password of Ditto - if you have that, you can do anything anyways using the "piggyback command" API :)
    Alexander Wellbrock
    @lionax_gitlab
    Is it possible to configure the buffer size of the ditto message serializer? It's currently set to 262144. I've a micro-service hooked into ditto via websockets and using ditto as auth/policy-proxy to provide capabilities via ditto messages. Unfortunately the service may respond with payloads over 1MB which first caused the limits to kick in. I've sorted that out and set the limits for messages to 8MB. Now the buffer of the gateway's serializer is overflowing.
    Thomas Jaeckle
    @thjaeckle
    sure, that is configured here: https://github.com/eclipse/ditto/blob/master/services/utils/config/src/main/resources/ditto-akka-config.conf#L169
    So in order to configure it differently, adjust REMOTE_MAX_FRAMESIZE for all Ditto microservices
    Alexander Wellbrock
    @lionax_gitlab
    Nice, thank you Thomas!
    Alexander Wellbrock
    @lionax_gitlab

    I've set maximum frame size to 8 MiB and message size limit to 8000k. Now it does not throw any errors but also does not forward the response to the original call (which was http rest). For smaller requests it's not a problem, but these bigger ones won't work and I can't figure out why. Any other idea or hint which might drop or block the response in the cluster? Since it caused errors in the gateway service, the message seems to arrive there. Now with the new settings it's just silently dismissed somewhere.

    I've tracked the message and it's response via it's correlation id and found the following: after the message reaches the gateway, it takes around 3 seconds for the micro-service via websocket to gather 1.3MiB of data and preparing the response message on the web socket. The gateway however receives the response only around 45 seconds later (45 seconds from when the micro-service logs a successful ws send). 14 seconds later the 60 seconds timeout window is reached and the HttpActor closes the connection.