These are chat archives for akkadotnet/AkkaStreams
@amri depending on the database i.e. Redis has support for pub/sub, and MongoDB has tailable cursors, so they can be made push-based (and to support backpressure, they need to expose api to limit upstream). But usually most databases offer only pull model (so you'd need to call a database to retrieve data). I've created an example publisher working with Github HTTP API .
Basically when the consumer request for a data, I'm downloading a page of issues from github and buffer it. If the page is bigger than the request demands, I serve events to the stream from in-memory buffer, and make another HTTP request only when the demand if bigger than number of buffered events. This way I can prefetch some data inside the buffer, so I don't need make a HTTP request every time consumer asks for more, but I also don't need to download a whole result set into memory.