Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Repo info
Activity
    Alessandro Becci
    @aleshark87
    Maybe a custom Java Mapper could be the solution ?
    Alessandro Becci
    @aleshark87
    @thjaeckle you tried http://plugfest.thingweb.io/webui/ ?
    I tried uploading a Ditto generated TD in a github gist, but it can't be parsed the right way. Looking at their implementation i think maybe it could be a issue with the fact that Ditto generates "nested" properties (Features in another TD).
    Alessandro Becci
    @aleshark87

    I tried uploading a Ditto generated TD in a github gist, but it can't be parsed the right way. Looking at their implementation i think maybe it could be a issue with the fact that Ditto generates "nested" properties (Features in another TD).

    speaked to the node-wot team, and i'm raising a github issue.

    2 replies
    Tobias Månsson
    @tobias-zeptio
    I'm having trouble understanding command-response over AMQP 1.0. I'm publishing a message to a incoming connection using payload mapping, and from what I can see it's successful. But where is the response sent? I can't set a target topic to "_/_/things/twin/commands".
    What is the connection configuration to be able to map command responses?
    For modify command we listen to twin events for change propagation.
    Tobias Månsson
    @tobias-zeptio
    Digging into the debug logs I see this Signal dropped, target address unresolved: <{{header:reply-to}}>
    Tobias Månsson
    @tobias-zeptio
    Wow, setting a my outgoing connection target address worked. But then the message is sent as a ditto protocol, how can I add a payload mapper to command responses?
    Thomas Jaeckle
    @thjaeckle
    @tobias-zeptio the configured "payloadMapper" of the "Source" should also be used for command responses - there is currently no way to configure another payload mapper for the responses differently
    the "outgoingScript" of the configured JS payload mapper is then executed for the command responses (which also are Ditto Protocol messages, so the signature to implement is the same)
    Tobias Månsson
    @tobias-zeptio
    Ah, I see. Thank you. I'll try that out.
    Alessandro Becci
    @aleshark87
    I have to send a ditto message when a certain mqtt message is sent. DItto Protocol is the way to go ?
    (In the Javascript Mapper)
    Thomas Jaeckle
    @thjaeckle
    sure, Ditto in the end only "understands" Ditto Protocol messages (produced by the JS mapper)
    Pranshu-G
    @Pranshu-G
    Hi @thjaeckle , I imported ditto into my eclipse workspace to go through the code, and was facing certain dependency issues (image1: https://imgur.com/F6zWgAT, image 2:https://imgur.com/DyvEsZx) Do I manually get the plugins and add them to respective pom.xml's? I do not want to break the build therefore I wanted to know if that is right.
    4 replies
    Thomas Jaeckle
    @thjaeckle
    @Pranshu-G seems like you need a Scala plugin in your Eclipse IDE ..
    we can't help with Eclipse IDE related issues, as we don't use the Eclipse IDE ourselves - the project compiles via "mvn clean install", so all needed maven plugins are in the poms
    Pranshu-G
    @Pranshu-G
    Right, what IDE do you use? Incase I'm unable to resolve the issues. I'll check for scala plug-in.
    Thomas Jaeckle
    @thjaeckle
    IntelliJ IDE
    Pranshu-G
    @Pranshu-G
    Thank you
    Pranshu-G
    @Pranshu-G
    Importing it in intelliJ shows 2 errors which seem internal dependency issues Cannot resolve symbol 'RqlPredicateParser$ and Cannot resolve symbol 'internal'
    Thomas Jaeckle
    @thjaeckle
    in IntelliJ, you also need to install the Scala plugin
    Pranshu-G
    @Pranshu-G
    The plugin somewhat helped, but it still has a lot of errors which cannot be explained since the mvn clean install and the ditto build in general runs flawlessly on my machine. It's just the IDE that gives error. Should I try to resolve them(if I am to contribute I might need to test the code).
    Thomas Jaeckle
    @thjaeckle
    sure, you should get your IDE "green" before starting to contribute in order to run unit tests and be able to debug, etc.
    Kim
    @kim-ae
    Hi ppl,
    Firstly I'd like to thank all the hard work put into ditto, it's a pretty amazing piece of software!
    Said that, I'm currently trying to do some load test on it, here where I work we intend to use it to manage around 300k devices and would like to have something around 6k~7k device sincronizations/s. We use k8s currently and I'm doing all my tests using helm chart for ditto and percona mongodb.
    I'm having a bit of strugle to achieve 1k msgs/s. My test is basically create 100k devices and start something around 30 pods sending around 500 msgs/s.
    Currently I'm using the following configurations:
    nginx: 15 pods, cpu: 80m, memory: 10Mi
    Gateway: 6 pods, cpu: 800m, memory: 4000Mi
    concierge: 7 pods, cpu: 600m, memory: 4000Mi
    policies: 1 pods, cpu: 400m, memory: 4000Mi
    things: 5 pods, cpu: 800m, memory: 4000Mi
    thingsSearch: 10 pods, cpu: 800m, memory: 4250Mi
    connectivity: 5 pods, cpu: 1000m, memory: 4000Mi
    Would anyone have some idea on were I could increase the resourses, or maybe an article or documentation to help me to make this decision in order to achieve the desired throughput?
    Thomas Jaeckle
    @thjaeckle
    @kim-ae cool, sounds like fun. Do you send the messgess via http or connectivity? Both are scaled similar. In general I would use more Cpu (2000m at least, up to 4000m) per pod and less pods if overall resources are limited.
    Do you have insights via metrics via Prometheus and Grafana? That would be the easiest way to detect bottlenecks..
    I noticed you scaled the search the most. We are currently working on a optimized search index which will drastically improve performance. The current search index is not optimized for complex thing structured and puts the Mongodb under a lot of CPU pressure.
    Kim
    @kim-ae
    @thjaeckle thanks for the insights,
    So I'm testing sending the messages through the /cloudevents endpoint.
    My idea is to publish the events into a rabbitmq using the connectivity.
    I increased the search because I saw that it was one of the modules that more throttled. Right now my tests only change a value for a prop on a specific feature, suing the topic "things/twin/commands/modify". It's not that complex.
    I have some metrics for k8s resources and found out some metrics from each pod as well, I'm trying to guide my adjusts through this metrics either.
    I'll try to increase the cpu for the modules and see if I'm able to achieve a higher throughput. I'll let you know the results thanks :)
    Thomas Jaeckle
    @thjaeckle
    You could also scale search to 0 if searching is not required for the scenario and spend the resources on the other services
    13 replies
    ctron
    @ctron:dentrassi.de
    [m]
    but @aleshark87 you are referring to a Gradle project, which uses Eclipse Ditto … which is different to working on the Eclipse Ditto code itself, which doesn't use Gradle, right?
    Alessandro Becci
    @aleshark87
    Yes, don't know nothing about that part.
    Alessandro Becci
    @aleshark87
    I'm using without any issue Server Sent Events to get notified about properties changes.
    Now i'm trying to set that up for receiving Messages, but i can't get any Event. I have a webapp(with sse working for properties update) and i'm trying to send messages via the ditto api page.
    new EventSource('http://localhost:8080/api/2/things/com.project.thesis:greenhouse01/outbox/messages/subject', {withCredentials: true}).onmessage = function (event){ if(event.data != ''){ console.log(event); } console.log(event); //$('#eventParagraph').text(event); }
    and this is the curl request that the ditto api does: curl -X 'POST' \ 'http://localhost:8080/api/2/things/com.project.thesis%3Agreenhouse01/outbox/messages/subject?timeout=0' \ -H 'accept: */*' \ -H 'Authorization: Basic ZGl0dG86ZGl0dG8=' \ -H 'Content-Type: application/json' \ -d '"test"'
    Alessandro Becci
    @aleshark87
    I've tried now to use a WebSocket connection for receiving messages and it works, but i would like to use SSE to comply with WoT forms.
    Alessandro Becci
    @aleshark87
    I tried also creating a EventSource attached to http://localhost:8080/api/2/things where, if i understand the ditto SSE mechanism, i should receive all events. I'm quite sure that the way i send the messages is right, since i receive that via WebSocket. There is something i can do for debugging ?
    3 replies
    Dirk Van Haerenborgh
    @vhdirk:matrix.org
    [m]
    @thjaeckle: We're planning on taking this up again: eclipse/ditto#993
    Would you be available sometime next week to discuss the current state (conflicts with main branch)?
    2 replies
    Tobias Månsson
    @tobias-zeptio
    I was expecting the merge of my PR eclipse/packages#371 to generate the release of a new chart version, but the latest chart version is still 2.5.0 in helm. Is the release a manual process, or what remains to be done?
    3 replies
    Johan
    @JohanBlue
    Hello, I just learnt that Hono doesn't support writing telemetry messages to a Mongo DB out-of-the-box, but can Ditto do that? i mean through a Ditto connection (with a sql command?) maybe?
    Thomas Jaeckle
    @thjaeckle
    @JohanBlue Ditto does per default persist every update to a twin to a MongoDB (which is the backing database of Ditto)
    I wonder how you want to use SQL for writing messages to a MongoDB .. what is your goal you are trying to achieve?
    Johan
    @JohanBlue
    i want to save every values that the twin had. Wait... so i can actually connect to the MongoDB and read everything as a user?
    Thomas Jaeckle
    @thjaeckle
    if your "as a user" has access to the database - yes
    Ditto uses Event sourcing which means that event (change) is persisted to the DB ..
    by default Ditto deletes stale events in the background, but this can be deactivated via configuration
    Johan
    @JohanBlue
    oh ok, so what kind of command do i use to access to these data? is there a curl command or an API?
    Thomas Jaeckle
    @thjaeckle
    no, you do database queries against MongoDB
    Johan
    @JohanBlue
    ok i ll try that ^^ thank you for these informations
    My goal is to create a tiny IoT AWS. I m using a cloud2edge cluster (locally for now) and i want to stay as close as possible to the functions of AWS.
    Kim
    @kim-ae
    I was unable to find in the documentation, How I create a custom JS function to map outgoing messages using the connectivity module?
    I didn't found the endpoint where to send the function or the data model to use to be able to create a custom mapper.
    Thanks in advance.
    2 replies