Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Repo info
Activity
    yasarkurt
    @yasarkurt
    do you think
    Don't you need to review?
    Thomas Jaeckle
    @thjaeckle
    Reviewing only makes sense for Ditto committers who know the architecture and the specialities about the Akka toolkit. Did you fully understand what Ditto does and do you use it? Do you know the Akka toolkit? Then it would of course be great to get development help. On what issue do you want to help? We also have some documentation issues which could be a good start..
    Thomas Jaeckle
    @thjaeckle
    glennergeerts-aloxy
    @glennergeerts-aloxy
    Hi, I just found the java client from the ditto-clients repository, which looks very useful. I was wondering if it is also possible to send raw ditto protocol messages over the websocket connection next to the Things CRUD API?
    Thomas Jaeckle
    @thjaeckle
    Hi @glennergeerts-aloxy .. Not yet, no - that would be a good enhancement to the client I think
    but the java client is also able to send/receive Ditto messages and it is capable of receiving change events .. so it's not only for CRUD ;)
    glennergeerts-aloxy
    @glennergeerts-aloxy
    yes I know of the change events. If I understand correctly the Ditto messages are used for sending directly to or from a device, while I want to modify the digital twin.
    To give a bit more context, I'm playing a bit with Vorto and i want to send the ditto protocol message as generated by Vorto's TwinPayloadFactory.toDittoProtocol() to Ditto, and I was wondering if I could use the java client for this as well
    Thomas Jaeckle
    @thjaeckle
    adding this would be quite easy .. I will create a PR ;)
    Thomas Jaeckle
    @thjaeckle
    @glennergeerts-aloxy eclipse/ditto-clients#3
    glennergeerts-aloxy
    @glennergeerts-aloxy
    @thjaeckle even already merged in the mean time I see, that was fast :) thanks alot! I will try it out later today or tomorrow
    glennergeerts-aloxy
    @glennergeerts-aloxy
    Small question on the java ditto client behavior (or possibly ditto in general?): I noticed that when a client registers for changes on a thing or feature it will not receive these changes after modifying the twin using the same client instance. Other client instances do receive the changes however. I was wondering if this is expected behavior? And, if yes, is this behavior of the client or ditto server in general? It does not sound illogical, but I'm a bit confused since the DittoClientUsageExamples.useTwinCommandsAndEvents() register a lot of callbacks on the client instance which is used for modifying as well, and thus will never trigger in this test.
    Thomas Jaeckle
    @thjaeckle
    @glennergeerts-aloxy yes, this is an intended behavior - the Ditto backend filters does not send events issued by a websocket session back to that same websocket session
    the DittoClientUsageExamples uses 2 different client instances, so opens 2 websocket sessions, that way this works of course :)
    could be that useTwinCommandsAndEvents uses a wrong client instance if this is not working
    glennergeerts-aloxy
    @glennergeerts-aloxy
    ok thanks. Is this the case for websocket only or for other connection protocols like AMQP as well?
    Thomas Jaeckle
    @thjaeckle
    yes, same for the other connection protocols like AMQP, etc.
    glennergeerts-aloxy
    @glennergeerts-aloxy
    It is working in useTwinCommandsAndEvents() in the sense that instance 2 receives the changes but what confused me is thtat there are a lot of callbacks registered for instance 1 as well (which does not recieve the changes)
    Thomas Jaeckle
    @thjaeckle
    you will however get response messages on that connection/websocket session when e.g. a modify command was successful
    glennergeerts-aloxy
    @glennergeerts-aloxy
    ok, thanks for this clarification
    I might have missed it in the documentation
    but it is good to know that is behavior of ditto backend in general
    Alexander Wellbrock
    @lionax_gitlab
    Hey there! Your blog post about OpenID Connect does not mention, since when (which version) this feature is supported. I assume it being since 0.9.0?
    Thomas Jaeckle
    @thjaeckle
    hi @lionax_gitlab .. no, it's not yet released - it's already merged on the master branch and will be released in an upcoming 1.0.0-M1 release
    we still gather some other bugfixes but will probably release within the next weeks
    Alexander Wellbrock
    @lionax_gitlab
    Ok thanks! In future postings I'd suggest adding information about that. I also think that a blog is actually a user-view component, which means that IMO saying "Eclipse Ditto now supports..." should be a reserved wording for features actually released ;) (I got totally excited - I'm still, but now damped)
    Thomas Jaeckle
    @thjaeckle
    well, you can always build it for yourself :D
    but agreed, would have been a good information to add
    Alexander Wellbrock
    @lionax_gitlab

    Hmmmmm. Of course you are right, but actually, well I'd rather not build my own build-process in my production environment since unfortunately I've limited resources to maintain that alone. But now that you mention it you are raising a good point here. My company is currently basically relaying on your releases in production. Would you say that's a bad idea and I should get my own build-process?

    Following up on that I still need to get keycloak running with ditto. I read an issue from you about it but I'm still confused if it's possible to configure keycloak in 0.9.0 without building the project myself but instead using the provided docker containers.

    Bob Claerhout
    @BobClaerhout
    Hi @lionax_gitlab, I had the same issue as you a couple of months ago and I was able to solve this myself: https://stackoverflow.com/questions/55027785/authenticate-websocket-with-keycloak-through-openresty
    Basically, you can configure nginx/openresty to communicate with keycloak by itself. You can then use the returned user to pass to ditto.
    On July 22 of this year someone else asked questions about this as well. You can find it in this gitter chat if you want
    Thomas Jaeckle
    @thjaeckle
    as bob said .. with 0.9.0 you could do it via nginx or other reverse proxies
    if you don't mind waiting from time to time for the newest features it is totally ok to rely on official Ditto releases
    as however the Eclipse releasing process is not the lightest one, doing non-milestone releases is quite an effort for us maintainers
    the official 1.0.0 should hopefully be ready end of this year
    Alexander Wellbrock
    @lionax_gitlab
    Ok, thank you both!
    Alexander Wellbrock
    @lionax_gitlab

    I've a question on the integration on vorto. I'm trying to build a backend infrastructure with ditto+vorto and a requirement is currently to use a vorto-infomodel to create a thing in ditto.

    So at the moment I manually create a vorto model with several functionblocks and an infomodel, then I write a json-file for the thing in my editor, following the schema (IM and FBs) and adding an attribute vortoIM to the thing for later reference. This json file is then used in my client code to bring together measured raw data with ditto thing as javascript representation.

    At first I thought, there'd be a nice and easy way to download/generate such a template via the vorto generators json/ditto. Actually they generate loose schemas for every functionblock and I need a json-fake-gen eating the schemas and generating a fake-thing which I can use as template.

    So what I'd like to have is a way to download a ditto template out of the vorto model (which also takes the infomodel into account). This is of course a feature request to the vorto team, I'll post that there if necessary. I'm asking you guys because I want to know if I'm just missing something here and having a wired view on the use-case or if you have an idea or a hint how you'd expect someone to use ditto with some kind of templating. Maybe vorto is just not the right tool for the job of getting ready to use templates for my 50 device types :D

    Thomas Jaeckle
    @thjaeckle

    I think Vorto might be just the right tool for what you want to do.
    Btw.: we also plan to add a definition field on Thing level, see: eclipse/ditto#247
    And our idea in utilizing Vorto models would be similar:
    When creating a Thing with a Vorto "InfoModel", all the Thing's features with their feature names derived from the infomodel could be created as "skeleton".
    I think Vorto allows default values - so that values could be the initial ones. When no defaults are given I would rather not choose a json-fake-data generator, but use e.g. 0 for ints, empty string for strings, etc..

    But I think this would best be done outside of Ditto (e.g. in a API facade wrapping the creation of Things) - as we probably don't want a hard dependency to a Vorto repository during runtime.
    It would be great if the generator for Ditto JsonSchema in Vorto could be enhanced with such a feature :+1:

    Alexander Wellbrock
    @lionax_gitlab

    To give you an update on how I solved the authentication keycloak topic for ditto 0.9.0: Since I'm in a complete docker environment getting nginx to understand keycloak openid connect with openresty like @BobClaerhout suggested would have required me to build a new image with the plugins installed and configured, setup CI/CD etc. So instead I used the keycloak/keycloak-gatekeeper docker image from docker-hub and set this up in front of the nginx container.

    The following tutorial was extremely helpful:
    https://github.com/ibuetler/docker-keycloak-traefik-workshop
    And of course keycloak-gatekeeper docs:
    https://www.keycloak.org/docs/latest/securing_apps/index.html#_keycloak_generic_adapter

    I then had to change disable nginx basic auth and set the headers to the following:

    proxy_set_header              X-Forwarded-User    $http_x_auth_preferred_username;
    proxy_set_header              x-ditto-dummy-auth  "nginx:${http_x_auth_preferred_username}";

    And for this case in keycloak-gatekeeper.conf:

    add-claims:
    - preferred_username
    Bob Claerhout
    @BobClaerhout
    nice!
    Alexander Wellbrock
    @lionax_gitlab

    I've one more question on authorization:

    I'm not getting how the creation / the management of a thing itself is authorized in ditto. The policy does give control for subjects on a thing and it's resources. But how do I control on top of that who is able to create things. Is it that everyone in principle may create things? So should I use a auth proxy for the respective API calls that I want to restrict access to? Like creating or deleting a thing? That would secure the REST API, but what with Websockets and MQTT messages?

    Thomas Jaeckle
    @thjaeckle
    Yes, in principle every authenticated user may create policies and things in Ditto.
    As you correctly stated even when you add a http proxy / API gateway, other APIs like Websocket, MQTT could still create things.
    In our commercial service, built on top of Ditto, we solve that by identifying the tenant making an API call: each tenant may register namespaces (part of the thing- and policy-ids) - and only if the tenant could be identified (e.g. via an "Api Token" at HTTP level), a Thing may be created in the the tenant's namespace.
    So the bad news: multi-tenancy is not part of the OSS Eclipse Ditto.
    The good news: as we build our commercial service on top of Ditto, there are extension points in order to plug-in such additional checks (as Java-code), e.g.: org.eclipse.ditto.services.concierge.starter.proxy.DefaultEnforcerActorFactory is the default Ditto implementation of the service making its authorization checks - that one may be swapped out for a custom implementation doing additional checks
    glennergeerts-aloxy
    @glennergeerts-aloxy
    @thjaeckle we are also looking into further integrating Vorto. The top level definition field is certainly interesting for creating the skeleton as you mentioned and describe the thing's capabilities to outside users (which is the main advantage of Vorto imo). In the issue you mention (#247) there are a few open points to clarify wrt to how strict Ditto should enforce this InformationModel. If I understand you correctly you do not want a hard dependency on the Vorto repository and instead implement this logic (creating a skeleton with correct definitions according to the InformationModel) to another component? After creation of this skeleton Ditto only enforces that features with a specified definition field do not change structure, in order to enforce compatibility with the InformationModel used during skeleton generation. Is this understanding correct?
    Alexander Wellbrock
    @lionax_gitlab

    Thanks @thjaeckle for your explanations. That's a starting point - let's see what I can do with this :D

    Another question on the policies: is it possible to search for policies and see what policies do exist? I did not find anything about that in the docs and I suppose that managing policies on a higher level than things is not expected / has no user-story? I'm not quite getting how policies are meant to be managed - without e.g. a search on policies I need to manage my policies outside of ditto to keep track of what I have and what the current state is. Is this expected or just not yet implemented or in other words would such a feature be desired?

    Do you mind if I put your elaborations on multi-tenancy, authentication and authorization into PRs for the docs? (Of course you could simply reject them, but I'm lazy :D)

    Thomas Jaeckle
    @thjaeckle

    hi @glennergeerts-aloxy

    #247 is not about enforcing the InformationModel, but just about providing a point where to put it ;)

    actually, we currently are thinking about if this makes sense at all to enforce/validate the Vorto based structure
    this would be very helpful during development of course - but for production when serving thousands of requests per second (and also thousands of updates per second), validating each update will cause a massive performance overhead
    we don't work on that currently, but I think that when we start, we will first and only provide the definition field without validating any structure during runtime.

    Thomas Jaeckle
    @thjaeckle

    @lionax_gitlab a Policy search is currently not available - we also have thought about that but don't see an urgent need currently
    Policies are often used on a 1-1 relationship for Things - in that case the policyId is often the same as the thingId
    But yes: it is not easy to find out which policies I "have" without a search.
    Of course you could define several policies on a tenant level which may be used for all the Things this tenant creates - that way they would be "well known" and if one of those policy is changed, all authorization logic for all the Things using that single policy is immediately changed as well - there are use cases which need that.
    Such a feature (search for policies) would be desired, yes - we simply have other priorities currently

    Would be great to see a PR updating the docs :+1: - please think about creating an Eclipse account first and signing the "ECA" (Eclipse committer aggreement) prior to the PR and sign your commits, as for PRs it is checked that all commits are signed by eclipse users with valid ECA

    Alexander Wellbrock
    @lionax_gitlab

    Thanks @thjaeckle I'll look into it.

    Bother sharing how you use it productively? 1-1 or "well-known"? Would be interesting to know how you tackle it or think about it ;) I'm actually right now doing it the "well-known" way because, well, it's easy right? :D

    Thomas Jaeckle
    @thjaeckle
    we have for different tenants different approaches .. although most of them use 1-1 ;)
    as most of the times an end-user has to be inserted into the policy ..
    the policy-id could of course also contain a user-id if the policy should be used for all things "belonging" to a user
    LiuHu
    @liuhu
    In ThingPersistenceActor I can't find any strategy to process CleanupPersistenceResponse command message. Maybe is MatchAnyDuringInitializeStrategy? I'm not sure.
    Dominik Guggemos
    @dguggemos
    As the ThingPersistenceActor handles the CleanupPersistence command itself, it does not handle the respective response. The handling of the CleanupPersistence is in the abstract class AbstractPersistentActorWithTimersAndCleanup.
    Thomas Jaeckle
    @thjaeckle
    @lionax_gitlab Ditto 1.0.0-M1a was released which now contains the OpenID connect support ;)
    Alexander Wellbrock
    @lionax_gitlab

    Oh nice! Thanks a lot!

    In the meantime I've also created a ditto skeleton generator running for vorto 0.11.x (project on github) which is also available on docker hub
    I'm currently resolving a bug with extending function blocks. When this is resolved I'll PR this to the official ditto plugin as configurable option. I think this'll be a nice addition for the vorto-ditto integration :)

    Thomas Jaeckle
    @thjaeckle
    ah, cool .. FMPOV this could also be a default for the Ditto generator plugin - no need to make it configurable from the way I see it
    (and I was the author of that plugin) ;)