Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Repo info
Activity
Andrea Peruffo
@andreaTP
I have integrated Source and Sink on top of the PersistentQueue and I will like to contribute them back
At the moment my problem is that I need to publish the persistent buffer implementation for both akka 2.4 and akka 2.5 do you mind if I publish from my organization account?
it is maybe useful to mark also the akka dependency as provided ... and I need to publish as well for scala 2.11 and 2.12
Arjun KN
@arjunknemc
We wanted to implement integrate test case for my application, can anyone provide me any github or any links to start with the implementations
Sherif Ebady
@sebady
@arjunknemc There are starter projects for squbs here https://github.com/paypal/squbs-scala-seed.g8 and https://github.com/paypal/squbs-java-seed.g8 (for java). These use squbs CustomTeskit which may be helpful with testing. You can find out more about testing with squbs testkit here https://github.com/paypal/squbs/blob/master/docs/testing.md
ganeshjanu
@ganeshjanu
Hi Friends, I am new to this room. We have implemented a project in SQUBS for RESTful services. It's queuing the incoming request and not working concurrently.
blocking-dispatcher {
type = Dispatcher
executor = fork-join-executor
fork-join-executor {
parallelism-min = 1024
parallelism-factor = 100.0
parallelism-max = 2048
}
throughput = 5
}
Even the throughput is "1", same results
http {
client {
idle-timeout = 500s
}
server {
idle-timeout = 310s
request-timeout = 55s
pipelining-limit = 1
}
host-connection-pool {
max-connections = 100
min-connections = 25
max-retries = 0
idle-timeout = infinite
pool-implementation = new
max-open-requests = 256
}
}
Have sent with "5" parallel request to the Squbs server
Arjun KN
@arjunknemc
Hi, i want to shutdown the squbs in runtime, in squbs website its provided as " squbs runtime can be properly shutdown by sending the Unicomplex() a GracefulStop message". Do i need to extend the Unicomplex class and then do the shutdown or how?. Please let me know
Akara Sucharitakul
@akara
Really sorry about the delay. No, you do not need to extend the Unicomplex class. You can just call Unicomplex(system).uniActor ! GracefulStop.
@arjunknemc
@ganeshjanu Sorry for the delay on the response. I don't think you have to tune anything to make things execute concurrently. However, your forkjoin executor with parallelism-min at 1024 is going to kill the system. The actual parallelism should be optimally 1xCPU cores. It could go up to 3xCPU cores. I'm sure you're not testing it on a 1000 core system. The fjpool will kill you at that rate.
It is more important to see what you're doing in your request handling code to cause it to run in serial.
If you can please share that, I'd be happy to take a look.
ganeshjanu
@ganeshjanu
@akara
Thank you for sharing the system. My server is a 2 core system.
i need to handle parallel request for some orchestration process which will talk to multiple processes on different systems.
what would be the recommended Configuration in my case? Please let me know. Thank you in advance
Akara Sucharitakul
@akara
@ganeshjanu your configuration except for the parallelism seems all good. The fact that you're handling the requests in serial seems to point to something blocking in your request handling code. Do you make a call to a blocking API directly or indirectly? Can it be a non-blocking API? A non-blocking API is far recommended. If not, we need to make sure the blocking API is called on a blocking dispatcher which can have hundreds of threads. You must not use fork-join for that dispatcher. Generally, we use a standard ThreadPoolExecutor for such cases. There is already a blocking-io-dispatcher available in Akka. Just need to change the parallelism of that.
For blocking dispatcher, 100s is quite fine.
ganeshjanu
@ganeshjanu
@akara blocking dispatcher is the default one that contains threadpool executor in squbs unicomplex reference.conf. To override, have defined fork join that replaces thread pool executor. Why the squbs has default with blocking dispatcher?
how can we achieve parallelism in such case?
Akara Sucharitakul
@akara
Nope, you should never use the FJ pool for blocking. Leave it as is. Just make sure any blocking code runs on the blocking dispatcher.
Also, that blocking dispatcher was defined before Akka had a blocking-io-dispatcher in their reference.conf. At this time it is probably better to use the Akka one.
How you get your blocking code to run on the blocking dispatcher depends how your code is structured. For instance, you can configure a particular actor to run in the blocking-io-dispatcher.
Or if your blocking code is in a Future, you can set the execution context to the blocking-io-dispatcher programmatically before firing up that future.
That's why I said, if you can show code or at least explain what your code does, structurally (i.e. svc a calls actor B which makes a blocking DB call), we can help figure the best way to make sure you're not blocked.
ganeshjanu
@ganeshjanu
I not using blocking dispatcher with any of my actors. My doubts whether the squbs routee using this dispatcher by default.
Because am creating the RESTful services based on Squbs routee
@akara
Akara Sucharitakul
@akara
Nope it doesn't. The fact that you do not use any blocking dispatcher could be the problem, especially if you have blocking code.
So the set of questions to ask is 1) does my request handler call any blocking APIs directly? If so, that should be encapsulated in preferably an Actor. A Future will work but you got to be careful that all blocking calls run on the blocking dispatcher. 2) Then, does my actor call any blocking APIs? If so, that actor MUST BE configured to run on the blocking dispatcher.
If you have blocking calls and do not run it on the blocking dispatcher, you'll get exactly the behavior you're observing.
ganeshjanu
@ganeshjanu
My project is a simple architecture. HttpService actor has defined the routee and accept the request. Then it forwards all the to an actor called Dispatchor Actor. It has created a controller actor(router) and forwards all the request to the router. Also,am not defining any dispatcher for this router and using the balancing pool only.The controller actor have the business logic and execute the blocking calls sequentially using ask method. Do I need to use the blocking dispatcher when creating the controller router?
ganeshjanu
@ganeshjanu
@akara
Please see my answers above
Akara Sucharitakul
@akara
The controller actor is the one executing the blocking calls. So the controller actor MUST run on the blocking dispatcher. As far as I can read, the controller actor forwards the request to the router. Unless the actors behind the router make blocking calls, they do not need to be on the blocking dispatcher. The router itself should never be on the blocking dispatcher.
Assuming the blocking call is done only in the controller actor, you need to 1) make sure the controller actor uses the blocking dispatcher and 2) you have multiple instances of it for running parallel blocking request. This can be achieved by adding a router in front of it or by spawning a new controller actor for every request.
ganeshjanu
@ganeshjanu
@akara In my case, Do you want me to setup the blocking dispatcher with multiple instance for HttpService Actor? Or which actor?
Akara Sucharitakul
@akara
Controller actor, that does the blocking call.
ganeshjanu
@ganeshjanu
In my case, controller actor has the router.
Akara Sucharitakul
@akara
In front of it?
Or behind it? That's what I could not figure.
ganeshjanu
@ganeshjanu
httpservice->Dispatcher->Controller
@akara
Akara Sucharitakul
@akara
Where's the router?
Also, httpservice -> Dispatcher is an ask, right?
What directive are you using to respond upon ask completion?