Hi @jkarneges I work with @niranjan-kmit. To answer your question, pushpin is fronted by AWS ALB which is doing the SSL termination and forwarding the request to pushpin on http or ws. What could be the possible issue here? This was previously working and just started getting this issue last week.
you could try connecting a websocket directly to pushpin to see if that still works
I've got a basic node express server working with pushpin using websockets over http. I'm using serve-grip & grip and I really like how things are progressing. The question that I have is how do we setup ping/pong heartbeats to check for disconnected clients? I have scoured the docs and while I found reference to being able to send ping/pong, I have found nothing showing how this might be accomplished. It has also been mentioned in this room, but again with no explanation as to how. Can you point me to any examples? Thanks.
to send a control message using WebSocketContext from serve-grip, do it like
ws.sendControl({"type": "keep-alive", "message-type": "ping", "content": "", "timeout": 30})
Thanks. Can I recommend adding that to the JS docs? Small things like that are useful. I read up on the keep-alives last night, but then reading further it was a bit confusing and seemed like that was not the way to go. As someone new in this area, there's terminology and concepts that are not self explanatory, so examples are pretty key.
The client socket is returning analytics (ip, location, etc) on the init. After that it’s just sending back info like if the page is currently active or not. That info is like a ping/pong but I’m not sure what I want the interval to be so I was also checking out the idea of a heartbeat. It may be redundant in this case, but i want to understamd how it all works.
So in this case, do I just close() or disconnect(), or both?
on the server side I'd simply ack closes (if
ws.recv() returns null, call ws.close()) and ignore disconnects
heartbeats from server->client can be useful even if the server isn't tracking connections though. for example, the server could send a message every 30 seconds, and if the client receives nothing in the last 60 seconds then the client can assume its connection is dead. in this case, it may be better for the server to use a TEXT frame for the heartbeat (instead of PING) since a reply would not be required
ws.sendControl({"type": "keep-alive", "content": "still alive", "timeout": 30})
Hi @jkarneges is there a limit on how many channels a single pushpin instance could handle? Are channels expensive in Pushpin?
Also found this in the pushpin config "connection_subscription_max=20". Is this the max number of channels per client connection? Can this be increased in a much larger number? And will be the drawback?
Thanks.
Also found this in the pushpin config "connection_subscription_max=20". Is this the max number of channels per client connection? Can this be increased in a much larger number? And will be the drawback?
Thanks.
Hi @jkarneges
I am trying to send messages to pushping using 0mq but I am unable to do it, I am using the following code:
ZMQ.Context context = ZMQ.context(1);
ZMQ.Socket publisher = context.socket(ZMQ.PUB);
publisher.connect("tcp://" + ip + ":5562");
publisher.send("J{\"id\": \"an-item-id\", \"channel\": \"test\", \"formats\": {\"http-stream\": {\"content\": \"a chunk of data\n\"}}}",0);
publisher.close();
context.term();
I am also using the following versions of 0MQ,
zeromq-4.1.4
zeromq/jzmq
The code does not display an error but messages are not appearing in pushpin logs.
Any suggestion related to this? might I be missing something?
some other things to consider. there must be at least one subscriber, otherwise your publisher won't actually send anything to pushpin (and nothing will get logged). and it can take a moment for subscription information to be sent to the publisher. I suggest adding a 500ms sleep between the connect() and send()
Hi @jkarneges ,
We are trying to publish messages through ZeroMQ to pushpin and something that we noticed was, if the message has new line characters (\n) the message breaks
For eg if we are trying to publish a json to pushpin it will not be a json as new line characters are parsed.
Can you suggest how can we make ZMQ escape the new line characters in the original message?
Thanks
Hi @jkarneges,
Here is our code
Here is our code
ZMQ.Context context = ZMQ.context(1);
publisher = context.socket(ZMQ.PUB);
publisher.connect("tcp://localhost:5562");
Thread.sleep(700);
String channelId = "test-channelId";
String messageId = data.getId();
String serializedWebSocketEvent = gson.toJson(data);
ZMsg zMsg = new ZMsg();
zMsg.addString(channelId);
zMsg.addString(getBodyMessage(channelId,serializedWebSocketEvent));
zMsg.send(publisherSubSocket);
private String getBodyMessage(String messageId,String channel,String content){
StringBuilder bodyMessage = new StringBuilder();
messageId = Strings.isNullOrEmpty(messageId) ? UUID.randomUUID().toString().replace("-","") : messageId;
bodyMessage.append("J{\"id\": \"")
.append(messageId)
.append("\",")
.append("\"channel\": \"")
.append(channel)
.append("\",")
.append("\"formats\": {\"")
.append("ws-message")
.append("\":{\"content\":\"")
// Note: Having new line characters messes the message in zeromq
.append(content.replace("\\n","").replace("\"","\\\"") )
.append("\"}}}");
return bodyMessage.toString();
}
@ashmeet-kandhari ah, you need to escape the content. I suggest either generating the published item using a json encoder rather than concatenating strings (in which case the
content field would get escaped as part of that process), or json-encode just the content value and use that in your string concatenations
hi @ricopollantecs , the subscription control messages come from your backend server, so this is up to you. for example you could require the client to provide authentication information in the query string when connecting, or in a message after connecting, and your backend would decide whether to apply a subscription to the connection
hi @vinayvbn , that somewhat depends on the http library you use.
GripControl.createGripChannelHeader creates a single header value. if your http server library supports setting multiple values for the same header name, then you can just call that method multiple times and set the result. if not, you'll need to join the values with a semicolon ; and set as a single value
my previous answer was for the HTTP transport, where channels are specified in HTTP headers that could be combined or not. however, for the WebSocket transport, channels are always specified as individual control messages
Has anyone experienced stream timeouts after 15 minutes? We have a system that uses pushpin to stream data, and after 15 minutes the connection gets dropped. When the client tries to reconnect it receives err_connection_timed_out messages for a number of attempts, and then the connection can be re-established? There's no significant idle time on the connection as there is always data flowing. Grip Keep-Alives have been been implemented and they don't help. There's no network reason for the dropped connection (no firewall timing out the connection, nor another proxy). There is only 1 client accessing the stream. We can replicate the connection drop using Curl, not jsut JS in Chrome. We are running pushpin through a docker container. Could docker be terminating the connection after 15 minutes? Can anyone suggest something to look into, or have experience dealing with strange connection interruptions?
I'm looking into config settings on the netscaler now, and looking into docker networking to see where the connection interruption could be happening. I came here to explore any pushpin behaviors I'm not aware of, or if anyone had experienced something similar.