These are chat archives for fanout/pushpin

16th
Apr 2016
Daniel Baskaran
@danielpradeep
Apr 16 2016 00:02
@jkarneges Ok. How about this? Behind the ELB, there are 2 instances running pushpin at port 80 and apache at port 9000 (configure pushpin route all the requests to localhost:9000). I haven't used haproxy yet, so I don't know how it would not be a single point of failure.
Justin Karneges
@jkarneges
Apr 16 2016 00:05
yep that sounds fine to me
Daniel Baskaran
@danielpradeep
Apr 16 2016 00:15
@jkarneges I just saw this https://www.digitalocean.com/community/tutorials/an-introduction-to-haproxy-and-load-balancing-concepts, is this the other option swapping out ELB with HAPROXY?
Justin Karneges
@jkarneges
Apr 16 2016 00:17
yes. you could run haproxy on your own ec2 instances for example. to avoid single point of failure you need multiple instances and dns round robin, or an active/standby style setup where an elastic IP can migrate to a standby instance if the active fails
Daniel Baskaran
@danielpradeep
Apr 16 2016 00:18
@jkarneges Ok, I would have to try it out.
@jkarneges Not all the API calls need LONG POLLING or WEB SOCKET connection, only few does. Therefore, in that context, for those LONG POLLING API calls, the actual response is sent by another instance, typically an APP server.
@jkarneges Does the ELB option would help me here?
Justin Karneges
@jkarneges
Apr 16 2016 00:19
yes you could use ELB to serve your normal endpoints and haproxy to serve the realtime ones
you might also be able to use ELB for everything just fine. all I can say is I've not done much testing with this
Daniel Baskaran
@danielpradeep
Apr 16 2016 00:21
@jkarneges Ok...
@jkarneges I would setup an ELB based infra to test this out and see how it goes.
Justin Karneges
@jkarneges
Apr 16 2016 00:22
yes, I'd say it's good to start with that, and I think the only issue will be connection timeouts which you can avoid with keep alives
for long polling that's easy, just have a short enough timeout (e.g. 55 seconds)
for websockets you'd need the client to send a packet periodically to the server
Daniel Baskaran
@danielpradeep
Apr 16 2016 00:23
@jkarneges Other question I have is on the APP server responding back to pushpin - it really does know which pushpin server to respond to (if there are two pushpin instances). This was not a problem in the DEV env as there are just one instances each.
@jkarneges Yes, 55 secs is more than enough for long polling
Justin Karneges
@jkarneges
Apr 16 2016 00:25
backend should send data to all pushpin servers
if a pushpin has no connections for a channel, it will simply drop the data
Daniel Baskaran
@danielpradeep
Apr 16 2016 00:25
@jkarneges I do have a requirement to have a websocket connection open for each client, but I guess ws and stream are pretty much similar in sending packets
@jkarneges Got it, the backend server should send a batch response to all the pushpin servers
Justin Karneges
@jkarneges
Apr 16 2016 00:27
if you are pushing a lot of data and really only want the data to go to the pushpin servers with listeners then this is possible too but may require some extra work (like sending to pushpin's zmq port)
Daniel Baskaran
@danielpradeep
Apr 16 2016 00:29
@jkarneges Ok, the size of the data is around 256 k for each response, pretty much a JSON string
@jkarneges Would you still recommend to use zmq publish?
@jkarneges Infact that would be easier too, because, the infra also got RabbitMQ to connect the APP servers
Justin Karneges
@jkarneges
Apr 16 2016 00:35
that's a big packet. I'd say it comes down to how often you push
Daniel Baskaran
@danielpradeep
Apr 16 2016 00:35
@jkarneges would chat feature require that?
Justin Karneges
@jkarneges
Apr 16 2016 00:36
if you're using rabbit already, then you could try making a worker that bridges between pushpin and rabbit. then app servers push to rabbit once
Daniel Baskaran
@danielpradeep
Apr 16 2016 00:37
@jkarneges Yes that could solve most problems around this, will try to do it
Daniel Baskaran
@danielpradeep
Apr 16 2016 00:45
@jkarneges that’s a lot of information, thanks a ton! It helps.
Justin Karneges
@jkarneges
Apr 16 2016 00:46
sure thing. if you do write a worker, you might look at tools/edgebroker.py
it bridges zmq<->zmq but you could think about how to do it with rabbit
Daniel Baskaran
@danielpradeep
Apr 16 2016 00:47
@jkarneges Sure, great. Let me take a look at it.
Daniel Baskaran
@danielpradeep
Apr 16 2016 01:40
@jkarneges Here is an old blog on R0MQ https://www.rabbitmq.com/blog/tag/zeromq/ and I guess even if that does not work out, I could write a worker to publish to 0MQ socket from a RMQ worker (like this http://zguide.zeromq.org/php:taskwork). Let me know your thoughts.
Justin Karneges
@jkarneges
Apr 16 2016 01:44
I think you'll probably need to write some worker code, at least to do message conversion
Daniel Baskaran
@danielpradeep
Apr 16 2016 01:45
@jkarneges Does 0MQ accepts JSON string? I would have the JSON string in that work
Justin Karneges
@jkarneges
Apr 16 2016 01:46
pushpin has two zmq inputs: PULL and SUB. both require a tnetstring formatted message rather than json
PULL accepts a single item as the message. SUB is a zmq multi part message, where the first part is the channel and the second part is a single item
PULL is easiest to start with. but SUB is what you'd use if you want to messages to only go to certain instances
Daniel Baskaran
@danielpradeep
Apr 16 2016 01:49
@jkarneges So PULL should work from all the pushpin instances
Justin Karneges
@jkarneges
Apr 16 2016 01:49
yes
Daniel Baskaran
@danielpradeep
Apr 16 2016 01:51
@jkarneges Ok… I would perhaps avoid running workers in web servers. Does SUB also work the same?
@jkarneges Ok, I guess if the worker can still run on the app server and can connect to PULL socket, it will be great
Justin Karneges
@jkarneges
Apr 16 2016 01:55
the workers can run wherever you want
Daniel Baskaran
@danielpradeep
Apr 16 2016 01:55
@jkarneges makes sense… thanks
Justin Karneges
@jkarneges
Apr 16 2016 01:55
pushpin's zmq sockets can be accessible from a remote machine, and if SUB is used then messages will only travel if there is a listener
Daniel Baskaran
@danielpradeep
Apr 16 2016 01:58
@jkarneges I guess as you recommended, PULL should be the first go-to. Based on the results, would think about SUB, though it will be tough to know which pushpin-apache combo has sent the request
Justin Karneges
@jkarneges
Apr 16 2016 02:00
the SUB socket will tell a worker whenever a channel has subscribers or not, which the worker could use to change the rabbit subscription
so it should not be necessary for the publisher to know where anything is
but yes an easier first step is to subscribe to everything from rabbit, and send everything to pushpin's PULL socket
and consider optimizing to SUB someday
Daniel Baskaran
@danielpradeep
Apr 16 2016 02:02
@jkarneges sure, I would have to digest the SUB and upgrade to it. But, I would get there
@jkarneges Would share the results… thanks a lot!