Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Repo info
Activity
Julien Truffaut
@julien-truffaut
basically I want send A request to a tcp server and I am expecting n B in response
Daniel Spiewak
@djspiewak
I think that the problem might be the queue bound that you're imposing on mergeN. Try giving it a higher value. Either that or try draining the results of the for-comprehension
Julien Truffaut
@julien-truffaut
I tried to increase the value of mergeN but it hadnt an impact
Daniel Spiewak
@djspiewak
hrm. try draining the results of the for-comprehension
you're not using the output anyway
Julien Truffaut
@julien-truffaut
ok I will try that tomorrow
can you explain what does drain do?
and why shall it change the behaviour of the server
Daniel Spiewak
@djspiewak
drain basically removes all of the emits in a resulting process. it converts a process that produces data into a process which never emits and only performs effects. since your server processes are just performing effects anyway, it shouldn't make a difference directly
however emits do slightly alter the behavior of various combinators, like mergeN and wye, since they have to deal with the fact that data is getting shuffled around
Julien Truffaut
@julien-truffaut
ok I will tell you how it goes, thanks
Daniel Spiewak
@djspiewak
gl!
Julien Truffaut
@julien-truffaut
I am trying to upgrade scalaz-stream but I am getting the following error, any idea what is the source?
[error] Could not run test scalaz.netty.NettySpecs: java.lang.NoSuchMethodError: scalaz.stream.Process$.Process0Syntax(Lscalaz/stream/Process;)Lscalaz/stream/Process$Process0Syntax;
Daniel Spiewak
@djspiewak
@julien-truffaut It's specs2's fault. I actually have a branch locally where I did the work and hit that exact spot
We basically just need Eric to release a new version that depends on 0.7a, and a lot of the ecosystem is backed up on that
Julien Truffaut
@julien-truffaut
specs2 depends on scalaz -stream?
Daniel Spiewak
@djspiewak
ye[
*yep
Julien Truffaut
@julien-truffaut
oh I didnt know
Daniel Spiewak
@djspiewak
yeah, so it overrides the dependency in src/test
(silently too!)
so that screws up the classpath and you end up with the error you pasted
see also my twitter rant from ten minutes ago :-S
Julien Truffaut
@julien-truffaut
I created etorreborre/specs2#347
Daniel Spiewak
@djspiewak
Yeah, the upgrade is done, it just hasn't been published yet.
Julien Truffaut
@julien-truffaut
@djspiewak drain does not help
so it only encoded a single value ...
Steve Buzzard
@sbuzzard
\
Julien Truffaut
@julien-truffaut
how can you perform a side effect once the tcp server has been started?
Julien Truffaut
@julien-truffaut
is it possible to get notified when a server close a connection? It seems that if I kill the process that run the server, the client is only failing when I start to send something to the server
Julien Truffaut
@julien-truffaut
@djspiewak @alissapajer do you know when will you release the next version?
Daniel Spiewak
@djspiewak
@/all so sorry for the delays on things here! been really busy with other stuff. v0.1.9 is now up, with updated dependencies and landed PRs. I'll be pushing a new release with 2.12.0-M1 cross building as soon as possible (blocked on specs right now)
Julien Truffaut
@julien-truffaut
amazing, thanks!
Erlend Hamnaberg
@hamnis
Does this work with udp sockets?
Alissa Pajer
@alissapajer
@/all version 0.1.10 is now up. This version fixes a bug introduced in 0.1.9, so if you upgraded to 0.1.9, upgrading again is strongly recommended :)
Daniel Spiewak
@djspiewak
@hamnis it's just TCP right now. UDP seems very doable, but it would require some new APIs. pull requests welcome!
Gabor Aranyossy
@gabor-aranyossy
Daniel Spiewak
@djspiewak
@gabor-aranyossy Because there's never a need for more than one thread. This is basically a thread which hands off incoming connection notifications (not even data!) from Netty to scalaz-stream. It doesn't do any actual work, it just feeds a queue. Parallelism would just cause unnecessary contention and context shifts, in addition to creating an extra GC root
Gabor Aranyossy
@gabor-aranyossy
Thanks for the quick response. Ok, I think I get it. You don't need more worker threads because you just feed an async.boundedQueue. This queue serves as a gateway to scalaz-stream turning the push based api to a pull based one. The result of this is that the processing speed depends on how fast my stream can consume this queue, which will run on the implicit execution context threads. Is this correct?
Gabor Aranyossy
@gabor-aranyossy

@djspiewak Another thing related to this. I see that the backpressure is implemented here: https://github.com/RichRelevance/scalaz-netty/blob/master/src/main/scala/scalaz/netty/Client.scala#L82

Should we have many clients, this will block the netty-worker thread as soon as a client's queue is full and by doing that it will also block the other clients. Should not the back-pressure be separated for the different clients?

Daniel Spiewak
@djspiewak
@gabor-aranyossy Arguably yes. For my use cases, backpressure is uniform across all clients by definition, so it doesn't make a difference, but I can see there are other use cases. I would accept a pull request that separates the BP! :-)
Gabor Aranyossy
@gabor-aranyossy
@djspiewak I'll see what I can do. :-)
Gabor Aranyossy
@gabor-aranyossy
@djspiewak Hi! I wanted to write a test-case to exercise backpressuring. I am not sure if the test-case is correct, but I noticed that it may or may not succeed depending on how big the limit is. Could you please validate the test case?
Daniel Spiewak
@djspiewak
@gabor-aranyossy I think it's a good test in principle, but I'm not sure I see how it's actually testing that backpressure propagates properly? If we broke backpressure (e.g. by swapping run for runAsync in the client handler), would this test break?
Gabor Aranyossy
@gabor-aranyossy
@djspiewak The naming was unfortunate. I wanted to exercise the BP and not test it. The point is however that this fails. I noticed a couple of things. Executing the read strictly after the write will actually make the test fail. If you replace it with Nondeterminisb.both it works. See here. Another thing is that using the same worker thread for the client and the server is probably not a good idea in such use cases as their back pressure will clash. Give me some time and I will figure this out. :-)
Daniel Spiewak
@djspiewak
v0.2 is up, with an important bug fix (data loss!) and some substantial performance tweaking thanks to @gabor-aranyossy.
Luciano Leggieri
@lukiano
Hi, was looking at the source code, why is it that you copy between bytevector and bytebuf using a byte array, instead of just sharing the underlying bytebuffer ?