by

Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Repo info
Activity
  • Aug 13 18:17

    jkarneges on develop

    handler: make http publish maxi… Merge pull request #47691 from … (compare)

  • Jul 29 18:41

    jkarneges on v1.30.0

    (compare)

  • Jul 29 18:40

    jkarneges on master

    limit body bytes to internal bu… runner: support condure service use client_maxconn and condure_… and 1 more (compare)

  • Jul 29 18:40

    jkarneges on develop

    update version (compare)

  • Jul 28 02:16

    jkarneges on develop

    use client_maxconn and condure_… (compare)

  • Jul 28 02:12

    jkarneges on develop

    runner: support condure service (compare)

  • Jul 22 01:42

    jkarneges on develop

    limit body bytes to internal bu… (compare)

  • Jul 15 18:45

    jkarneges on v1.29.0

    (compare)

  • Jul 15 18:45

    jkarneges on master

    fix crash when parsing accept h… provide credits on websocket ac… fix crash when response hold ti… and 10 more (compare)

  • Jul 15 18:45

    jkarneges on develop

    update version (compare)

  • Jul 13 21:36

    jkarneges on develop

    init field (compare)

  • Jul 13 18:59

    jkarneges on develop

    split websocket frames to fit i… (compare)

  • Jul 09 23:07

    jkarneges on develop

    update qzmq handler: only process body when… handler: don't allow pausing tw… (compare)

  • Jul 07 15:42

    jkarneges on develop

    update qzmq (compare)

  • Jun 26 20:54

    jkarneges on develop

    update submodules fix warnings (compare)

  • Jun 21 16:50

    jkarneges on develop

    fix crash when response hold ti… don't re-enable keep-alive if m… (compare)

  • May 22 03:40

    jkarneges on develop

    provide credits on websocket ac… (compare)

  • May 13 16:28

    jkarneges on develop

    fix crash when parsing accept h… (compare)

  • Apr 08 15:46

    jkarneges on master

    websocket-over-http: add option… update version (compare)

  • Apr 08 15:46

    jkarneges on v1.28.0

    (compare)

NIRANJAN
@niranjan-kmit
@jkarneges why i am getting "Error while proxying to origin." while openning the streaming connection to pushpin. I have changed pushpin conf file like * localhost:8000
Justin Karneges
@jkarneges

you mean the routes file? try setting the content to

*,debug localhost:8000

and then make a request

Justin Karneges
@jkarneges
@jeremyll for zmq with django grip, you can try something like:
GRIP_PROXIES = [{
    'control_zmq_uri': 'tcp://localhost:5563',
    'require_subscribers': True
}]
NIRANJAN
@niranjan-kmit
@jkarneges how do we know that specific channel doesn't have any subscribers ,the chanel which is created by backed service previously?
Justin Karneges
@jkarneges
@niranjan-kmit you can monitor for subscriptions via zmq, either the sub socket used for publishing or the stats socket. for what purpose?
NIRANJAN
@niranjan-kmit
Hello @jkarneges if any no clients subscribed to particular channel then i need to delete the channel?
Justin Karneges
@jkarneges
@niranjan-kmit there's no need to delete channels. they are created automatically and destroyed automatically
NIRANJAN
@niranjan-kmit
i mean i need to delete orphan channels
@jkarneges is there any particular time we have to destroy that orphan channel automatically?
Justin Karneges
@jkarneges
are you asking how long does it take pushpin to destroy a channel with no subscribers?
NIRANJAN
@niranjan-kmit
@jkarneges Yes
Justin Karneges
@jkarneges
for http stream and websocket transports it is immediately when the last subscriber going away. for http response (used for long polling) the channel lingers for about a minute in case a subscriber comes back
NIRANJAN
@niranjan-kmit
Thank you very much @jkarneges
NIRANJAN
@niranjan-kmit
@jkarneges Can you please provide some resource for getting insite about JWT validation in puspin proxy
Justin Karneges
@jkarneges
@niranjan-kmit validation of the Grip-Sig header? or something else?
NIRANJAN
@niranjan-kmit
@jkarneges Grip-Sig header
NIRANJAN
@niranjan-kmit
@jkarneges how can we validate token which is cumming from fronted using pushpin?
Justin Karneges
@jkarneges
@niranjan-kmit the signing key is specified in pushpin.conf sig_key
backend can validate using the same key
NIRANJAN
@niranjan-kmit
@jkarneges !GripControl.validateSig(session.getHeaders().get("grip-sig"), "<key>") above mentioned keys, from where grip-sig andand key comes?
Justin Karneges
@jkarneges
@niranjan-kmit Pushpin adds Grip-Sig to the proxied request. key is whatever is specified in pushpin.conf
NIRANJAN
@niranjan-kmit
if I want authorize valid user or request then how can do with the pushpin?
NIRANJAN
@niranjan-kmit
@jkarneges if I got the JWT Token from UI Service then how can I validate that token while making q request to the pushpin proxy?
@jkarneges Basically I want to implement Authentication/Autherization with Pushpin Proxy?
Justin Karneges
@jkarneges
@niranjan-kmit pushpin relays all headers sent by the client to the backend. you can then auth the client from the backend
ccbeloy
@ccbeloy
Hi @jkarneges ...is there an event pushpin raise when removing a channel? And how to subscribe to that event?v
Justin Karneges
@jkarneges
@ccbeloy see the zmq SUB socket. some info here: https://pushpin.org/docs/advanced/#subscription-forwarding
the code sample at the end of that section
NIRANJAN
@niranjan-kmit
@jkarneges How do I know that my channel X having N number subscriptions? can you please provide any reference since i am a beiginner for pushpin
Justin Karneges
@jkarneges
@niranjan-kmit you can find that on the subscription stats https://pushpin.org/docs/advanced/#stats-socket
NIRANJAN
@niranjan-kmit
Hi@jkarneges can we send json response with custome message using java grip library.
as of now i see newFixedLengthResponse in the code
Justin Karneges
@jkarneges
@niranjan-kmit you can send any data format you want
NIRANJAN
@niranjan-kmit
@jkarneges Json formate with status code with message
like {200: "deployed succesfully"}
Justin Karneges
@jkarneges
@niranjan-kmit pushpin sends bytes. so if you want to send json, just encode the json to bytes and send the bytes :)
NIRANJAN
@niranjan-kmit
Hi @jkarneges my backend service running to validate token.after validation of token ineed to send response standard way with response code.
Justin Karneges
@jkarneges
@niranjan-kmit if you just want to respond normally then you can simply do that. if no grip instructions are provided, the proxying is a passthrough
NIRANJAN
@niranjan-kmit
Hi @jkarneges how can we implement heart beat service for pushpin? i want know the active connections
NIRANJAN
@niranjan-kmit
Hello . Can any one guide me to the implementation of heart beat endpoint with puhpin?
Justin Karneges
@jkarneges
@niranjan-kmit there are a couple of ways to do this. one is to subscribe every connection to a unique channel, and monitor the subscriptions to see who is connected. however, TCP connections that go away may linger for a long time, so a more accurate way to determine active connections is for the clients to send heartbeats, which pushpin will then pass along to your backend
NIRANJAN
@niranjan-kmit
@jkarneges any ref documentation we have for this?
Justin Karneges
@jkarneges
@niranjan-kmit for the first approach, see the code sample at the end of this section for monitoring subscriptions https://pushpin.org/docs/advanced/#subscription-forwarding
the second approach is just generic proxying, so there's nothing specific written about it
NIRANJAN
@niranjan-kmit
Hi @jkarneges I want to include the timestamp to message before pushpin send the message to client? so that i can find out the latency from pushpin to client
Sasha K
@akovalev

Hey,

thanks for this project and writing very detailed documentation. I really enjoyed reading it.

At the moment I'm trying to understand the difference between Pusher and Fanout/Pushpin capabilities, especially with regards to possibility to subscribing to a given channel on-demand.

If I understood correctly, Pushpin doesn't provide any option to specify which channel a client should be subscribed to when using HTTP/Server-Sent Events as a realtime transport, except for the initial "negotiating" phase which takes place when a client sends HTTP request to the location of the remote resource serving the events/messages, e.g. a client might encode a channel it's interested in, say, as a query parameter of the remote resource URL. In other words, it looks like everything depends on the instructions sent back from origin server to Pushpin Proxy via Grip-Channel HTTP Response header.

At the same time, if one decides to use WebSocket/WebSocket-over-HTTP as a real transport, it seems that it's possible to communicate client's intention to subscribe to a given channel by sending a WebSocket frame from the client app and expect that frame to be handled the origin server.

To provide you with more context, I can describe the potential use-case in which on-demand subscription to channels might be useful: for example, we have an offline-friendly web app that manages/syncs the content of collaborative documents with the server. e.g. we persist local copies of the documents in the browser, say, in IndexedDB and periodically sync these document with server. Each user might have access to multiple documents, but we don't want to sync all documents in the background at the same time, but rather "postpone" the synchronisation until the moment when user explicitly opens it. Moreover, even if later on we decide to sync multiple documents (like last N recently opened documents) at the same time, we would like to do it efficiently via 1 HTTP/multiplexed connection rather than using 1 connection per document as there's a limit for 6 simultaneous TCP connections to a given hostname in modern browsers unless HTTP2 is used which is not the case with Pushpin if I understand correctly.

That being said, what would you recommend if it's needed to subscribe/unsubscribe to channels on-demand based on certain event occurring in the runtime of a client app? Is it feasible to achieve by choosing WebSocket as realtime transport and implementing ?

1 reply
Justin Karneges
@jkarneges
hi @akovalev , welcome to the room. indeed, the WebSocket transport would be more appropriate for managing multiple subscriptions dynamically. note that it is possible to change subscriptions with the HTTP streaming transport as well (by using reliable mode and triggering the change out-of-band), so for example a client could POST to some other endpoint to change the channels of a connection they have open, but this is too stateful of an API design for my taste
pushpin doesn't support HTTP/2 yet, but you could put another proxy in front of pushpin that does, such haproxy or nginx. that way the client could potentially use only one connection for multiple HTTP streams. I'd probably prefer that approach over trying to change the channels of an existing HTTP connection
Justin Karneges
@jkarneges
bidirectional WebSocket communication is absolutely an appropriate use-case for pushpin. in some sense, pushpin is more of a connection manager for stateless backends than purely a push solution. though, I think anyone using pushpin would also be pushing with it :)
Sasha K
@akovalev

thanks @jkarneges

I was also thinking about notifying an origin server via a separate endpoint and also didn't like this idea too much as it implies keeping a certain state on the server. Another option that I was considering is to to subscribe a given user to all documents they have access to, but I'm afraid it will result in an excessive payload on the channel since as I've said before client becomes interested in document updates only under certain circumstances, i.e. we are going to push updates from origin server to the Pushpin proxy even when the client isn't interested in them.

as for HTTP/2 proxy in front of pushpin -- having asked my initial question I found this thread 1 where you recommended pretty much the same and it turned out to be working pretty well.

Also, it's nice to hear that bi-directional WebSocket communication is a valid usage of Pushpin.