Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Repo info
Activity
Akara Sucharitakul
@akara
@arjunknemc
@ganeshjanu Sorry for the delay on the response. I don't think you have to tune anything to make things execute concurrently. However, your forkjoin executor with parallelism-min at 1024 is going to kill the system. The actual parallelism should be optimally 1xCPU cores. It could go up to 3xCPU cores. I'm sure you're not testing it on a 1000 core system. The fjpool will kill you at that rate.
It is more important to see what you're doing in your request handling code to cause it to run in serial.
If you can please share that, I'd be happy to take a look.
ganeshjanu
@ganeshjanu
@akara
Thank you for sharing the system. My server is a 2 core system.
i need to handle parallel request for some orchestration process which will talk to multiple processes on different systems.
what would be the recommended Configuration in my case? Please let me know. Thank you in advance
Akara Sucharitakul
@akara
@ganeshjanu your configuration except for the parallelism seems all good. The fact that you're handling the requests in serial seems to point to something blocking in your request handling code. Do you make a call to a blocking API directly or indirectly? Can it be a non-blocking API? A non-blocking API is far recommended. If not, we need to make sure the blocking API is called on a blocking dispatcher which can have hundreds of threads. You must not use fork-join for that dispatcher. Generally, we use a standard ThreadPoolExecutor for such cases. There is already a blocking-io-dispatcher available in Akka. Just need to change the parallelism of that.
For blocking dispatcher, 100s is quite fine.
ganeshjanu
@ganeshjanu
@akara blocking dispatcher is the default one that contains threadpool executor in squbs unicomplex reference.conf. To override, have defined fork join that replaces thread pool executor. Why the squbs has default with blocking dispatcher?
how can we achieve parallelism in such case?
Akara Sucharitakul
@akara
Nope, you should never use the FJ pool for blocking. Leave it as is. Just make sure any blocking code runs on the blocking dispatcher.
Also, that blocking dispatcher was defined before Akka had a blocking-io-dispatcher in their reference.conf. At this time it is probably better to use the Akka one.
How you get your blocking code to run on the blocking dispatcher depends how your code is structured. For instance, you can configure a particular actor to run in the blocking-io-dispatcher.
Or if your blocking code is in a Future, you can set the execution context to the blocking-io-dispatcher programmatically before firing up that future.
That's why I said, if you can show code or at least explain what your code does, structurally (i.e. svc a calls actor B which makes a blocking DB call), we can help figure the best way to make sure you're not blocked.
ganeshjanu
@ganeshjanu
I not using blocking dispatcher with any of my actors. My doubts whether the squbs routee using this dispatcher by default.
Because am creating the RESTful services based on Squbs routee
@akara
Akara Sucharitakul
@akara
Nope it doesn't. The fact that you do not use any blocking dispatcher could be the problem, especially if you have blocking code.
So the set of questions to ask is 1) does my request handler call any blocking APIs directly? If so, that should be encapsulated in preferably an Actor. A Future will work but you got to be careful that all blocking calls run on the blocking dispatcher. 2) Then, does my actor call any blocking APIs? If so, that actor MUST BE configured to run on the blocking dispatcher.
If you have blocking calls and do not run it on the blocking dispatcher, you'll get exactly the behavior you're observing.
ganeshjanu
@ganeshjanu
My project is a simple architecture. HttpService actor has defined the routee and accept the request. Then it forwards all the to an actor called Dispatchor Actor. It has created a controller actor(router) and forwards all the request to the router. Also,am not defining any dispatcher for this router and using the balancing pool only.The controller actor have the business logic and execute the blocking calls sequentially using ask method. Do I need to use the blocking dispatcher when creating the controller router?
ganeshjanu
@ganeshjanu
@akara
Please see my answers above
Akara Sucharitakul
@akara
The controller actor is the one executing the blocking calls. So the controller actor MUST run on the blocking dispatcher. As far as I can read, the controller actor forwards the request to the router. Unless the actors behind the router make blocking calls, they do not need to be on the blocking dispatcher. The router itself should never be on the blocking dispatcher.
Assuming the blocking call is done only in the controller actor, you need to 1) make sure the controller actor uses the blocking dispatcher and 2) you have multiple instances of it for running parallel blocking request. This can be achieved by adding a router in front of it or by spawning a new controller actor for every request.
ganeshjanu
@ganeshjanu
@akara In my case, Do you want me to setup the blocking dispatcher with multiple instance for HttpService Actor? Or which actor?
Akara Sucharitakul
@akara
Controller actor, that does the blocking call.
ganeshjanu
@ganeshjanu
In my case, controller actor has the router.
Akara Sucharitakul
@akara
In front of it?
Or behind it? That's what I could not figure.
ganeshjanu
@ganeshjanu
httpservice->Dispatcher->Controller
@akara
Akara Sucharitakul
@akara
Where's the router?
Also, httpservice -> Dispatcher is an ask, right?
What directive are you using to respond upon ask completion?
Assuming you're using the route API, of course.
ganeshjanu
@ganeshjanu
HttpService -> Dispatcher is a look up call via the
Squbs routee
dispatcher -> controller is a forward call. This controller actor is a router
ganeshjanu
@ganeshjanu
controller-> worker actor is a ask call. This is a separate actor for
Each routee
@akara
Varun Kumar
@vk9
Hi all, Is it possible to support multiple version of scala, currently only scala 2.12 is being supported ? Currently I am working on some application which restricts upgrade from 2.11 to 2.12
Akara Sucharitakul
@akara
We can do so. There is even an issue for this. It may need some minor work on the API. The issue is the bandwidth to implement dual publish. But if you'd be happy to file a PR for this, we're more than open.
I think there is a demand.
Varun Kumar
@vk9
Can you please share the issue id, couldn't find it in open issues.
Will try to contribute and create a PR
ganeshjanu
@ganeshjanu
@akara It was a great session with you. Thank you for all your valuable inputs and suggestions with my use case.