by

Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Repo info
Activity
  • Aug 13 18:17

    jkarneges on develop

    handler: make http publish maxi… Merge pull request #47691 from … (compare)

  • Jul 29 18:41

    jkarneges on v1.30.0

    (compare)

  • Jul 29 18:40

    jkarneges on master

    limit body bytes to internal bu… runner: support condure service use client_maxconn and condure_… and 1 more (compare)

  • Jul 29 18:40

    jkarneges on develop

    update version (compare)

  • Jul 28 02:16

    jkarneges on develop

    use client_maxconn and condure_… (compare)

  • Jul 28 02:12

    jkarneges on develop

    runner: support condure service (compare)

  • Jul 22 01:42

    jkarneges on develop

    limit body bytes to internal bu… (compare)

  • Jul 15 18:45

    jkarneges on v1.29.0

    (compare)

  • Jul 15 18:45

    jkarneges on master

    fix crash when parsing accept h… provide credits on websocket ac… fix crash when response hold ti… and 10 more (compare)

  • Jul 15 18:45

    jkarneges on develop

    update version (compare)

  • Jul 13 21:36

    jkarneges on develop

    init field (compare)

  • Jul 13 18:59

    jkarneges on develop

    split websocket frames to fit i… (compare)

  • Jul 09 23:07

    jkarneges on develop

    update qzmq handler: only process body when… handler: don't allow pausing tw… (compare)

  • Jul 07 15:42

    jkarneges on develop

    update qzmq (compare)

  • Jun 26 20:54

    jkarneges on develop

    update submodules fix warnings (compare)

  • Jun 21 16:50

    jkarneges on develop

    fix crash when response hold ti… don't re-enable keep-alive if m… (compare)

  • May 22 03:40

    jkarneges on develop

    provide credits on websocket ac… (compare)

  • May 13 16:28

    jkarneges on develop

    fix crash when parsing accept h… (compare)

  • Apr 08 15:46

    jkarneges on master

    websocket-over-http: add option… update version (compare)

  • Apr 08 15:46

    jkarneges on v1.28.0

    (compare)

Justin Karneges
@jkarneges
backend can validate using the same key
NIRANJAN
@niranjan-kmit
@jkarneges !GripControl.validateSig(session.getHeaders().get("grip-sig"), "<key>") above mentioned keys, from where grip-sig andand key comes?
Justin Karneges
@jkarneges
@niranjan-kmit Pushpin adds Grip-Sig to the proxied request. key is whatever is specified in pushpin.conf
NIRANJAN
@niranjan-kmit
if I want authorize valid user or request then how can do with the pushpin?
NIRANJAN
@niranjan-kmit
@jkarneges if I got the JWT Token from UI Service then how can I validate that token while making q request to the pushpin proxy?
@jkarneges Basically I want to implement Authentication/Autherization with Pushpin Proxy?
Justin Karneges
@jkarneges
@niranjan-kmit pushpin relays all headers sent by the client to the backend. you can then auth the client from the backend
ccbeloy
@ccbeloy
Hi @jkarneges ...is there an event pushpin raise when removing a channel? And how to subscribe to that event?v
Justin Karneges
@jkarneges
@ccbeloy see the zmq SUB socket. some info here: https://pushpin.org/docs/advanced/#subscription-forwarding
the code sample at the end of that section
NIRANJAN
@niranjan-kmit
@jkarneges How do I know that my channel X having N number subscriptions? can you please provide any reference since i am a beiginner for pushpin
Justin Karneges
@jkarneges
@niranjan-kmit you can find that on the subscription stats https://pushpin.org/docs/advanced/#stats-socket
NIRANJAN
@niranjan-kmit
Hi@jkarneges can we send json response with custome message using java grip library.
as of now i see newFixedLengthResponse in the code
Justin Karneges
@jkarneges
@niranjan-kmit you can send any data format you want
NIRANJAN
@niranjan-kmit
@jkarneges Json formate with status code with message
like {200: "deployed succesfully"}
Justin Karneges
@jkarneges
@niranjan-kmit pushpin sends bytes. so if you want to send json, just encode the json to bytes and send the bytes :)
NIRANJAN
@niranjan-kmit
Hi @jkarneges my backend service running to validate token.after validation of token ineed to send response standard way with response code.
Justin Karneges
@jkarneges
@niranjan-kmit if you just want to respond normally then you can simply do that. if no grip instructions are provided, the proxying is a passthrough
NIRANJAN
@niranjan-kmit
Hi @jkarneges how can we implement heart beat service for pushpin? i want know the active connections
NIRANJAN
@niranjan-kmit
Hello . Can any one guide me to the implementation of heart beat endpoint with puhpin?
Justin Karneges
@jkarneges
@niranjan-kmit there are a couple of ways to do this. one is to subscribe every connection to a unique channel, and monitor the subscriptions to see who is connected. however, TCP connections that go away may linger for a long time, so a more accurate way to determine active connections is for the clients to send heartbeats, which pushpin will then pass along to your backend
NIRANJAN
@niranjan-kmit
@jkarneges any ref documentation we have for this?
Justin Karneges
@jkarneges
@niranjan-kmit for the first approach, see the code sample at the end of this section for monitoring subscriptions https://pushpin.org/docs/advanced/#subscription-forwarding
the second approach is just generic proxying, so there's nothing specific written about it
NIRANJAN
@niranjan-kmit
Hi @jkarneges I want to include the timestamp to message before pushpin send the message to client? so that i can find out the latency from pushpin to client
Sasha K
@akovalev

Hey,

thanks for this project and writing very detailed documentation. I really enjoyed reading it.

At the moment I'm trying to understand the difference between Pusher and Fanout/Pushpin capabilities, especially with regards to possibility to subscribing to a given channel on-demand.

If I understood correctly, Pushpin doesn't provide any option to specify which channel a client should be subscribed to when using HTTP/Server-Sent Events as a realtime transport, except for the initial "negotiating" phase which takes place when a client sends HTTP request to the location of the remote resource serving the events/messages, e.g. a client might encode a channel it's interested in, say, as a query parameter of the remote resource URL. In other words, it looks like everything depends on the instructions sent back from origin server to Pushpin Proxy via Grip-Channel HTTP Response header.

At the same time, if one decides to use WebSocket/WebSocket-over-HTTP as a real transport, it seems that it's possible to communicate client's intention to subscribe to a given channel by sending a WebSocket frame from the client app and expect that frame to be handled the origin server.

To provide you with more context, I can describe the potential use-case in which on-demand subscription to channels might be useful: for example, we have an offline-friendly web app that manages/syncs the content of collaborative documents with the server. e.g. we persist local copies of the documents in the browser, say, in IndexedDB and periodically sync these document with server. Each user might have access to multiple documents, but we don't want to sync all documents in the background at the same time, but rather "postpone" the synchronisation until the moment when user explicitly opens it. Moreover, even if later on we decide to sync multiple documents (like last N recently opened documents) at the same time, we would like to do it efficiently via 1 HTTP/multiplexed connection rather than using 1 connection per document as there's a limit for 6 simultaneous TCP connections to a given hostname in modern browsers unless HTTP2 is used which is not the case with Pushpin if I understand correctly.

That being said, what would you recommend if it's needed to subscribe/unsubscribe to channels on-demand based on certain event occurring in the runtime of a client app? Is it feasible to achieve by choosing WebSocket as realtime transport and implementing ?

1 reply
Justin Karneges
@jkarneges
hi @akovalev , welcome to the room. indeed, the WebSocket transport would be more appropriate for managing multiple subscriptions dynamically. note that it is possible to change subscriptions with the HTTP streaming transport as well (by using reliable mode and triggering the change out-of-band), so for example a client could POST to some other endpoint to change the channels of a connection they have open, but this is too stateful of an API design for my taste
pushpin doesn't support HTTP/2 yet, but you could put another proxy in front of pushpin that does, such haproxy or nginx. that way the client could potentially use only one connection for multiple HTTP streams. I'd probably prefer that approach over trying to change the channels of an existing HTTP connection
Justin Karneges
@jkarneges
bidirectional WebSocket communication is absolutely an appropriate use-case for pushpin. in some sense, pushpin is more of a connection manager for stateless backends than purely a push solution. though, I think anyone using pushpin would also be pushing with it :)
Sasha K
@akovalev

thanks @jkarneges

I was also thinking about notifying an origin server via a separate endpoint and also didn't like this idea too much as it implies keeping a certain state on the server. Another option that I was considering is to to subscribe a given user to all documents they have access to, but I'm afraid it will result in an excessive payload on the channel since as I've said before client becomes interested in document updates only under certain circumstances, i.e. we are going to push updates from origin server to the Pushpin proxy even when the client isn't interested in them.

as for HTTP/2 proxy in front of pushpin -- having asked my initial question I found this thread 1 where you recommended pretty much the same and it turned out to be working pretty well.

Also, it's nice to hear that bi-directional WebSocket communication is a valid usage of Pushpin.

Brian Zier
@bzier
The Java GripControl library does not properly decode events containing unicode. I've submitted a pull request to the repo here:
fanout/java-gripcontrol#1
Please let me know if there is anything I need to do in order to get the change merged and a new version created. Thanks.
12 replies
torellrobinson
@torellrobinson
HI everyone, there were a comment on the "SSL handshake failed:" error messages that are present in the pushpin containers logs. Was anybody able to resolve the issue?
[ERR] 2020-07-17 20:50:52.804 [m2 https:9443] (src/io.c:195: errno: None) SSL handshake failed: -1
[ERR] 2020-07-17 20:50:57.805 [m2 https:9443] (src/io.c:150: errno: Resource temporarily unavailable) Handshake failed with error code: -29312
Justin Karneges
@jkarneges
according to mbedtls/ssl.h, error code -0x7280 is MBEDTLS_ERR_SSL_CONN_EOF. this means the client closed the connection early and should be harmless
@torellrobinson
torellrobinson
@torellrobinson
@jkarneges Thanks for your response. These 2 lines are generated every second in my log file. Could this affect performance since I am seeing random latency between 20ms-200ms?
Justin Karneges
@jkarneges
@torellrobinson sounds like a health check? one connection open/closed per second shouldn't affect performance
jaligamap
@jaligamap
@jkarneges we are using the real-time publishing of messages to the subscribers from the backend via Pushpin. What happens if we have multiple clients subscribing to the same channel, for load-balancing and fail-over purposes? we see that all the instances are currently receiving those messages and will process them, which is not what we want. Is there any way to do a round-robin kind of a thing among those subscribers?
Justin Karneges
@jkarneges
@jaligamap you could set up routing rules in your load balancer to send clients to specific pushpin instances. note that this would require the URL/headers to be informative enough to figure out which pushpin to send to
jaligamap
@jaligamap
@jkarneges we only have one pushpin instance but our customers are running multiple instances of the client..
jaligamap
@jaligamap
i am asking if pushpin can do a round robin between the subscribers
Justin Karneges
@jkarneges
@jaligamap what is the use case? I'm trying to imagine an application that a user would want to run more than once, but not receive updates on all of them
jaligamap
@jaligamap
@jkarneges - use case is, customers can subscribe to changes on our side..so they run subscribers on their side to receive the updates pushed by pushpin..if they run two instances and subscribe to the same channel, pushpin will push the messages to both subscribers
and customers are running multiple instances of subscribers for fault-tolerance
Justin Karneges
@jkarneges
@jaligamap in that case I'd suggest ensuring your messages can be handled idempotently. so both instances receive the message and process, but the end result is the same as if only one received and processed
jaligamap
@jaligamap
I dont think its an optimal solution..lot of wasted resources processing duplicate requests..i am surprised you did not encounter this situation before @jkarneges
Justin Karneges
@jkarneges
@jaligamap well the other idea is to use a unique channel ID for each subscription, or potentially a set of IDs with minimal sharing
jaligamap
@jaligamap
@jkarneges the reason why we have multiple subscribers is for load balancing/fault tolerance purposes..so if we use unique channel ids we wont satisfy that requirement..in this case, pushpin knows exactly how many receivers are there for each channel, so theoretically, it can balance the load and make sure that its sending the message to only one of those subscribers