Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Repo info
Activity
    Kai Hudalla
    @sophokles73
    @robmontalvojr would you mind sharing what kind of adjustments you made or need to make? It might be helpful to others running into similar problems ...
    Kai Hudalla
    @sophokles73
    @damian-gallo Hi Damian, you can set the adapters' client configuration using the adapters.amqpMessagingNetworkSpec, adapters.tenantSpec, adapters.deviceRegistrationSpec, adapters.credentialsSpec and adapters.deviceConnectionSpec properties in the values.yaml file. You need to provide the full set of properties for the particular client, i.e. not just the properties that you want to change.
    Damián Gallo
    @damian-gallo
    Great, thanks, I'll give it a try
    robmontalvojr
    @robmontalvojr
    @sophokles73 Kai, here it goes.. In hopes others can benefit from my mistakes... When I first started my exploration journey, I attempted to do so in a severely constrained, old hardware platform. On top of this, I took my chances by bringing up minikube with only 2vCPUs, and about 6GB memory. Because of the issues I was having on that physical platform, I decided to move the experiment to the cloud, and there I was able to get things to come up and stabilize in a few minutes. It was then I figured I needed to upgrade my box. I added memory, switched the old HDD for an SSD, and re-did my minikube cluster to match the stated requirements in the package. After effecting these changes, the environment became functional and the experience was comparable to what I had observed with my cloud environment.
    Kai Hudalla
    @sophokles73
    thanks for sharing @robmontalvojr :-)
    pravussum
    @pravussum

    Hi there... I don't know if this is the right place to ask my question, please feel free to redirect me otherwise...
    I set up the cloud2edge package and I'm now struggeling to send a custom "event" from a device connected to Hono via Ditto to a queue connected to Ditto.
    In Ditto-speak that would be the "smokeDetected" Message mentioned here: https://www.eclipse.org/ditto/basic-messages.html
    My setup is as follows:
    MQTT client (representing the device) <-> Hono <-> Ditto <-> RabbitMQ (via Ditto connection service)

    If I publish a MQTT message to the "event topic" I get the following error logged in the ditto-connectivity container:

    2020-11-09 09:51:36,503 INFO  [] o.e.d.s.c.m.MessageMappingProcessorActor akka://ditto-cluster/system/sharding/connection/5/hono-connection-for-idealcustomer/pa/$v/c1/messageMappingProcessor - Got DittoRuntimeException 'json.format.invalid' when ExternalMessage was processed: The headers did not contain a value for mandatory header with key <ditto-message-direction>! -
    2020-11-09 09:51:36,514 INFO  [] o.e.d.s.c.m.a.AmqpConsumerActor akka://ditto-cluster/system/sharding/connection/5/hono-connection-for-idealcustomer/pa/$v/c1/amqpConsumerActor-0-event%2Fidealcustomer-03 - Acking <ID:AMQP_NO_PREFIX:EventSenderImpl-16> with original external message headers=<{orig_adapter=hono-mqtt, device_id=org.anization:thething, qos=1, creation-time=1604911896475, message-id=ID:AMQP_NO_PREFIX:EventSenderImpl-16, content-type=application/octet-stream, to=event/idealcustomer, orig_address=event}>, isSuccess=<true>, ackType=<1 accepted>

    I tried to add the "ditto-message-direction" to the MQTT payload in a "headers" member and I tried configure it in the tenants Hono-Ditto-Connection configuration as a fix Header mapping, but still no luck:

              "headerMapping": {
                "hono-device-id": "{{ header:device_id }}",
                "content-type": "{{ header:content-type }}",
                "ditto-message-direction": "FROM"
              },

    I'm not quite sure where the problem (or my lack of understanding) is located, my current assumption is:

    • MQTT doesn't support headers, so Hono is not forwarding any to Ditto
    • Ditto checks the headers of the incoming message before it looks at the payload and complains about the missing header
    • the Header mapping is somehow not used / working

    What is the way this is supposed to work? Is it maybe something I need to configure in Hono? Please let me know, if you need any further information (e. g. the connection configuration or the MQTT message payload, I left it out for brevity)

    Thomas Jaeckle
    @thjaeckle
    @pravussum could you provide the Ditto Protocol payload you send? I guess that in the path the addressing of inbox or outbox is missing which would deyemne
    Which would determine the direction. As this is a Ditto internal header, setting it via header mapping won't work.
    pravussum
    @pravussum
    That would be the following. I added the path attribute only because I got an error otherwise that it's missing. Since I didn't want to relate it to a specific feature/attribute, I used the "empty" path "/".
    {
      "topic": "org.anization/thething/things/live/messages",
      "headers": {
        "ditto-message-direction":"FROM"
      },
      "path": "/"
    }
    pravussum
    @pravussum

    OK, thanks for the hint regarding the path! That brought me to read up on the Ditto protocol details for messages, so I got the payload straight:

    {
      "topic": "org.anization/thething/things/live/messages/mytestsubject",
      "headers": {  },
      "path": "/inbox/messages/mytestsubject"
    }

    And that worked!

    Thomas Jaeckle
    @thjaeckle
    :+1:
    Dejan Bosanac
    @dejanb
    Hey folks, I added a passcode to the community zoom meeting so we don't have to login with eclipse credentials and manually allow people to the meeting
    Please join the meeting today with the following url https://eclipse.zoom.us/j/317801130?pwd=UGpXL29PejRrZWIzT2pKRFJGVUNqUT09
    I'll update the calendar and site later today with Frederic
    Kai Hudalla
    @sophokles73
    Thanks for the update, @dejanb :-)
    Carsten Lohmann
    @calohmn
    The new Hono chart version for the changes committed today isn't online yet. An issue has been created to resolve the corresponding CI build errors.
    Tobias Månsson
    @tobias-zeptio
    I'm running ditto on a kubernetes node with lots of memory, and ditto is configured to use a percentage of available memory. This end up in high memory usage, that doesn't seem neccesary. Have anyone run a memory profiler on the different ditto containers, and see actual heap and stack sizes? So that we can maybe adjust the charts default from percentage to fixed. And now all containers use the same parameters, which doesn't seem likely to be optimal.
    Thomas Jaeckle
    @thjaeckle
    @tobias-zeptio as the helm chart in the packages is not intended for productive use without configuring it you'll have to find reasonable values yourself for your setup
    this vastly depends on e.g.
    a) how many things should be managed (only thousands or millions, or more)
    b) how many instances of each service is running (horizontally scaling the Ditto services)
    c) how many connections to foreign systems exist (only some or hundrets/thousands)
    ...
    in principle each Ditto service is able to be operated with as few as 500MB of RAM - if you manage however a million things I your things service would need more
    Tobias Månsson
    @tobias-zeptio
    @thjaeckle Thanks. I've been doing some testing and also found 500 MB to be a minimum. I was just asking because I would expect gateway and concierge taking much less memory than connectivity and things, for example. We also want to run several instances of some, for redundancy.
    Thomas Jaeckle
    @thjaeckle
    @tobias-zeptio yes, gateway+concierge should require the fewest memory .. concierge however depends also on the amount of used (cached) policies in concierge (required to do the authorization of requests) - so when having millions of policies, concierge will also need more memory :)
    Tobias Månsson
    @tobias-zeptio
    @thjaeckle Ah, I would expect the policies to be cached in "policies". :P My thinking is that for a use-case with manly amqp access for thousands of devices, connectivity and things would be the most used services. Is that not the case then?
    Thomas Jaeckle
    @thjaeckle
    policies service caches policies as well (being the persistence service for policies)
    however concierge does the (auth) enforcement based on policies and for efficiency reasons also caches policies
    for your use case connectivity-concierge-things (and things-search as well, but with delay) services are "affected" by each consumed message via AMQP
    Tobias Månsson
    @tobias-zeptio
    Ah, that makes sense. Thanks for the info!
    Dirk Van Haerenborgh
    @vhdirk:matrix.org
    [m]
    Hi! We know we shouldn't run Hono with the demo certificates in production. But as far as I can tell, there's no easy way to replace them with our own certificates without just cloning the chart and overwriting them.
    https://github.com/eclipse/packages/blob/master/charts/hono/templates/hono-adapter-amqp/hono-adapter-amqp-vertx-secret.yaml
    This always pulls file from the local dir, but there seems to be no way to either overrule these file paths, nor is there a way to set other certificate paths in the config itself: https://github.com/eclipse/packages/blob/master/charts/hono/templates/_helpers.tpl#L213
    Kai Hudalla
    @sophokles73
    That should actually be pretty easy.
    1. Create a secret containing your private key and certificate. Let's assume the secret is called my-keys and the files in it are my-key.pem and my-cert.pem.
    2. Use the adapters.amqp.extraSecretMounts property in the Values.yaml to specify your secret, e.g.
      extraSecretMounts:
      keys:
       secretName: "my-keys"
       mountPath: "/etc/hono-keys"
    3. Use the adapters.amqp.hono.amqp property in the Values.yaml to specify the properties of your adapter, e.g.
      amqp:
      bindAddress: "0.0.0.0"
      keyPath: "/etc/hono-keys/my-key.pem"
      certPath: "/etc/hono-keys/my-cert.pem"
    Dirk Van Haerenborgh
    @vhdirk:matrix.org
    [m]
    Thanks @sophokles73. What about the keys for the device registry? https://github.com/eclipse/packages/blob/master/charts/hono/templates/_helpers.tpl#L213
    Kai Hudalla
    @sophokles73
    same approach, just use properties deviceRegistryExample.extraSecretMounts and deviceRegistryExample.mongoDBBasedDeviceRegistry.hono.registry.amqp ...
    Dirk Van Haerenborgh
    @vhdirk:matrix.org
    [m]
    ok, cool, thanks!
    We'll give that a go
    Kai Hudalla
    @sophokles73
    if you are using a cert in the registry which has not been signed by a well-known CA, then you will need to make sure to add the CA (or intermediate CA) to the adapters' trust store as well ...
    Dirk Van Haerenborgh
    @vhdirk:matrix.org
    [m]
    Yes, I figured as much. Thanks for the heads-up
    Dirk Van Haerenborgh
    @vhdirk:matrix.org
    [m]
    Hey @sophokles73 , I don't quite understand yet how this is supposed to work for the device registry
    For any of the adapters, we ultimately always end up here: https://github.com/eclipse/packages/blob/master/charts/hono/templates/_helpers.tpl#L236
    With the helm chart in its current form (unless I'm mistaken of course), there seems to be no way to specify a custom truststore (or credentials) for an adapter, unless we would deploy a device registry manually outside of this chart
    Kai Hudalla
    @sophokles73
    I see your point. We didn't talk about using the adapter's custom key/cert in the registry service clients ...
    You should be able to use the adapters.tenantSpec, adapters.deviceRegistrationSpec, adapters.credentialsSpec, adapters.deviceConnectionSpec properties of values.yaml to configure the registry service clients accordingly.
    Dirk Van Haerenborgh
    @vhdirk:matrix.org
    [m]
    But in order for that to work, you'd need to set <root>.deviceRegistryExample.enabled=false? As seen here https://github.com/eclipse/packages/blob/master/charts/hono/templates/_helpers.tpl#L220
    Dirk Van Haerenborgh
    @vhdirk:matrix.org
    [m]
    I can make a PR to make those work similar to the command router? https://github.com/eclipse/packages/blob/master/charts/hono/templates/_helpers.tpl#L243
    Kai Hudalla
    @sophokles73
    Yes, that seems to have been written with some other assumptions in mind. A PR for fixing it would be highly appreciated. Could you also open a corresponding issue for properly tracking it?
    Dirk Van Haerenborgh
    @vhdirk:matrix.org
    [m]
    Additionally, it seems the only way to make this work is to put every component's keys into a separate secret, since templates like hono.amqpMessagingNetworkClientConfig are shared across components
    Sven Erik Jeroschewski
    @eriksven
    Hello IoT Packages community,
    I am considering to submit a proposal for a talk on using Eclipse IoT Packages for Edge and IoT scenarios with Kubernets at the "Kubernetes on Edge Day" which is co-located with the next KubeCon + CloudNativeCon Europe in May 2021. Please let me know if you have a similar idea or any objections to this. I am asking because I was not a core developer in IoT Packages to this point and also want to avoid duplicate proposals.
    Dejan Bosanac
    @dejanb
    @eriksven I think that would be awesome and that you should go for it
    Kai Hudalla
    @sophokles73
    @eriksven Great idea! Good luck with the proposal being accepted :-)
    Kai Hudalla
    @sophokles73
    @/all I would like to drop the checks for Kubernetes 1.15 and 1.16 from the CI workflows and instead add 1.21. Consequently, we would then also raise the minimum supported Kubernetes version to 1.17 on the Packages project homepage. Azure Kubernetes Service supports 1.18 since Aug 2020. Amazon's EKS supports 1.16 - 1.20, so I do not think we are missing out on anything when dropping support for 1.15 and 1.16. Any thoughts?
    Thomas Jaeckle
    @thjaeckle
    sounds good to me :+1:
    Carsten Lohmann
    @calohmn
    +1
    Kai Hudalla
    @sophokles73
    @calohmn would you mind taking a look at #242 ?
    Carsten Lohmann
    @calohmn
    @ctron Could you have a look at eclipse/packages#258