Hi, I have a problem with rabbitMQ, Running 2 service on my local machine with local dapr, all have a different port (grpc, metrics, etc) one service published event then subscriber triggering and returning 500 (for testing pub/sub mechanism) after that I restarted and didn't notice any triggering for my subscriber, I edited yaml component like this
apiVersion: dapr.io/v1alpha1
kind: Component
metadata:
name: messagebus
namespace: default
spec:
type: pubsub.rabbitmq
metadata:
- name: host
value: amqp://localhost:5672 # Required. Example: "amqp://rabbitmq.default.svc.cluster.local:5672", "amqp://localhost:5672"
- name: consumerID
value: rabbitMqConsumer # Required. Any unique ID. Example: "myConsumerID"
# - name: durable
# value: "true" # Optional. Default: "false"
# # - name: deletedWhenUnused
# # value: <REPLACE-WITH-DELETE-WHEN-UNUSED> # Optional. Default: "false"
- name: autoAck
value: "true" # Optional. Default: "false"
# # - name: deliveryMode
# # value: "0" # Optional. Default: "0". Values between 0 - 2.
# - name: requeueInFailure
# value: "true" # Optional. Default: "false".
@Admiralkheir Watch out for dapr logs. If a component is not being initialized properly it's ought to say so
with redis component it worked but with rabbit this ack mechanism is not working
@Admiralkheir Try cloning my repo https://github.com/amitHansda/store-manager
It uses tye to start all service together. Also in components folder you may need to change the connection string. Mine uses possibly a different username password than yours
It uses tye to start all service together. Also in components folder you may need to change the connection string. Mine uses possibly a different username password than yours
3 replies
@Admiralkheir 
Hah, I think our YAML set correctly. so if there is a problem, it's on rabbitMQ component realization.
Hah, I think our YAML set correctly. so if there is a problem, it's on rabbitMQ component realization.
@Admiralkheir I did the test again and it looks like the autoAck mechanism didn't work.
Unfortunately, no matter how many times I started a subscriber, he wouldn't have received a return of 500. The next step I think is to debug the dapr runtime. check if the autoAck parameters have been read according to our YAML.
Unfortunately, no matter how many times I started a subscriber, he wouldn't have received a return of 500. The next step I think is to debug the dapr runtime. check if the autoAck parameters have been read according to our YAML.
apiVersion: dapr.io/v1alpha1
kind: Component
metadata:
name: messagebus
namespace: default
spec:
type: pubsub.rabbitmq
metadata:
- name: host
value: amqp://user:GSlU5LpIa1:32421 # Required. Example: "amqp://rabbitmq.default.svc.cluster.local:5672", "amqp://localhost:5672"
- name: consumerID
value: rabbitMqConsumer # Required. Any unique ID. Example: "myConsumerID"
# - name: durable
# value: "true" # Optional. Default: "false"
# # - name: deletedWhenUnused
# # value: <REPLACE-WITH-DELETE-WHEN-UNUSED> # Optional. Default: "false"
- name: autoAck
value: true # Optional. Default: "false"
# # - name: deliveryMode
# # value: "0" # Optional. Default: "0". Values between 0 - 2.
# - name: requeueInFailure
# value: "true" # Optional. Default: "false".
and I even get the log but the at-least-once semantics not work.
== APP == 2020/09/02 00:27:58 event - PubsubName: messagebus, Topic: neworder, ID: 5dc38ceb-9a1a-4d71-9c8e-0d44f98f4845, Data: ping
== DAPR == time="2020-09-02T00:27:58.172224+08:00" level=error msg="rabbitmq pub/sub: error handling message from topic 'neworder', error returned from app while processing pub/sub event: . status code returned: 500" app_id=faildsub1 instance=ArideMacBook-Air.local scope=dapr.contrib type=log ver=0.10.0
hey everyone, community call starting now. password: eWRhSklVTjJjSnhTaURDcFZaU2ZzQT09. url: https://us02web.zoom.us/j/85305980190
DAPR won’t distribute message from input binding to multiple instances of consumer?
I have the following scenario and issue:
1) External system publish 500 TODO job message to RabbitMQ
2) Consumer Pod use input binding (AMQP protocol) to receive messages from RabbitMQ, and Consumer Pod is running on demand (if no message, then no instances running)
2) KEDA scale the Consumer Pod from 0 to 5 based on custom metrics of RabbitMQ queueLength
3) Kubernetes create 5 instances of Consumer Pod based on scale rule (incoming 500 messages)
4) The daprd sidecar of first instances of Consumer Pod will get all 500 messages and processed them one by one, each message will use 20 seconds to complete, at meantime, the status of all messages in RabbitMQ changed from Ready to Unacked - UnAcknoledged
5) The other 4 instances of consumer Pod did NOT get any un-processed messages from RabbitMQ and free all time
I need all 5 instances of Consumer Pod parallelly process all non-complete messages in RabbitMQ, What configuration I missed or did wrong?
13 replies
In my opinion, Yes, input binding only sequencingly sends to one app. @Li-Yanzhi . And it seems that use Pub/Sub and queueSubscribe meet your needs.
Thanks @abserari , is this the only way to fix issue? it seems I have a lot code to refactor. I originally thought input binding can fully support the scenario of "competing consumer" mode, on the other hand, is such greedy algorithm of daprd sidecar safe enough? I means that if daprd sidecar get all current messages from queue (all message's status changed from Ready to Unack), when main container crashed for some reason, then all left messages cannot be picked up and processed by other pod?
@aronchick if using the Dapr CLI, see this: https://github.com/dapr/cli#running-sidecar-only
4 replies
Hi, could someone explain how HTTP requests in dapr are handled and what exactly happens under the hood? As an example I have a dapr service that exposes an HTTP API, implemented with the .NET core framework. So what happens exactly if I call the API from another dapr service with the
InvokeMethod method of the dapr client? Am I right that the DaprClient makes a gRPC request to the sidecar, then the sidecar searches for the service and makes a gRPC request to the sidecar of the API service. And then the sidecar makes an HTTP call to the API? If so, can someone point me to the right code in GitHub?
There are some messages talking to integrate dapr with istio. Considering there will be two sidecar in the system, what is the benefit of using istio? I am trying to understand what istio does and dapr doesn't in service mesh? I have a look at https://istio.io/latest/docs/concepts/what-is-istio/ and didn't see a big difference between dapr consepts and istio consepts. Considering i will start to build a new design and all i need service to service communication, should i place istio in a daprized environment?
1 reply
Hi, after a while we seem to lose connection to pubsub (using redis) or state store (using sql). Our application will work for a while then we start to see ERR_PUBSUB_NOT_FOUND or ERR_STATE_STORE_NOT_CONFIGURED errors in the logs. Restarting the pods will get it working again, usually. Any ideas what is causing this? We are on version 0.10.0 and also using Istio
13 replies
Hi guys, I would like to clarify the best approach for a certain use case. We have a front end react app which publishes events as messages to a Google Cloud PubSub topic. We would like to pull these messages into our dapr-enabled back-end and then process them amongst back-end Microservices using a standard Dapr redis.pubsub component. With the new ability of Dapr to connect to multiple pubsub components (in our case Google Cloud PubSub for messaging coming from outside the Dapr-system environment and Redis PubSub for messages inside Dapr) what is die best approach. Should we use Dapr Pub Sub to get messages from Google PubSub or should we use the Dapr Binding which also supports Google PubSub? Is Dapr PubSub only for communication between Dapr enabled Microservices or is it feasible to use it to get messages from an external topic source as described. What will the pros and cons be between using Dapr PubSub with Google PubSub or Dapr Bindings with Google PubSub. We are discussing architecture and this question came up without clear answers. Any insight will be very valuable - thank you!
1 reply
with different IPs and all?
@yaron2 I'm seeing the sidecar injector fail with this message:
It goes away for a while after restarting the deployment and then returns. Any idea what the problem could be? I saw the workaround here dapr/dapr#1621 (which is what was followed), but there was no link in that issue to the corresponding change in documentation. This is a version 0.8 installation.
2020/09/04 00:02:52 http: TLS handshake error from 100.96.2.0:41578: remote error: tls: bad certificateIt goes away for a while after restarting the deployment and then returns. Any idea what the problem could be? I saw the workaround here dapr/dapr#1621 (which is what was followed), but there was no link in that issue to the corresponding change in documentation. This is a version 0.8 installation.
2 replies
Hi.. I'm researching on Dapr and I have 2 questions that I haven't found answers for yet. Could anyone help?:
1) With Dapr.Workflows is it possible to manage the workflow (LogicApp) in Azure? How do we go about getting usage details and princing?
2) For Redis state store, does the current component support failover and/or Redis Sentinel? I saw that the go-redis library does.
1) With Dapr.Workflows is it possible to manage the workflow (LogicApp) in Azure? How do we go about getting usage details and princing?
2) For Redis state store, does the current component support failover and/or Redis Sentinel? I saw that the go-redis library does.
14 replies
dapr/docs#774 Request a review about doc update. The update is about to remove the nats.md and add the nats-streaming.md in how to setup pub-sub message broker.
Hi I'm having trouble getting service invocation from C# sdk to work.
Even tried running the samples at https://github.com/dapr/dotnet-sdk/tree/master/samples/Client/DaprClient.
I have "routing" up and running, and can invoke it successfully via dapr invokeGet -a routing -m 17 .
However, when running the sample Client code (with the GET invocation uncommented), i get the following error:== APP == Unhandled exception. Grpc.Core.RpcException: Status(StatusCode=Unknown, Detail="context deadline exceeded")
I get the same error when I try to invoke my own service code from another of my own services, which is what started me down this road. After getting the excact same error from sample code i figured it's time to ask for help :P Any ideas? Running on OSX if that matters.
8 replies
Asked a question about using Dapr and CAP together here: dotnetcore/CAP#662 . CAP automates outbox pattern for transactional messaging but talks directly to underlying bus, hence seems incompatible with Dapr pubsub. Comments are welcome.
5 replies
I just moved from 0.9.0 -> 0.10.0 with the C# sdk and it broke my pub/sub. Previously it took one string parameter and now it takes two (pubsubName and topicName). I don't know which one was there first and am struggling to find the documentation to describing how they are used. Can someone point me to the documentation on the change?
20 replies
Hi, I have a question about pubsub component's metadata name, Is it a disadvantage that the same Pubsub component uses the same metadata name for many microservices? Is it affect performance etc. For example, I have 3 microservices they have 3 different methods and they are listening to 3 different topics(the same pubsub metadata name) and i used the same pubsub component(nats.streaming) YAML each of them. What is the best practices for this mechanism
5 replies
@piotrmsc Hi Piotr, Did you produce any documentation on how to get Istio working with Dapr, we are bumping into issues ourselves but getting into trouble with Redis and Dapr and the EnvoyProxy (Istio manages) seems to require more configuration. Any help or pointers would be appreciated, thanks , Anthony.