Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Repo info
Activity
  • Jan 20 17:02
    fzakaria commented #840
  • Jan 20 01:20
    fzakaria review_requested #840
  • Jan 20 01:17
    fzakaria synchronize #840
  • Jan 19 21:21
    pitr-ch commented #840
  • Jan 19 21:21
    pitr-ch synchronize #840
  • Jan 19 21:11

    pitr-ch on master

    Add names to global pools formatting Add missing require and 3 more (compare)

  • Jan 19 21:11
    pitr-ch closed #812
  • Jan 19 17:55
    pitr-ch synchronize #812
  • Jan 19 17:45

    pitr-ch on master

    Adds WrappingExecutor class Th… Generates docs for WrappingExec… Update documentation and 4 more (compare)

  • Jan 19 17:45
    pitr-ch closed #830
  • Jan 19 17:45
    pitr-ch synchronize #830
  • Jan 19 16:13
    pitr-ch synchronize #830
  • Jan 19 15:56
    pitr-ch synchronize #812
  • Jan 19 15:27
    pitr-ch commented #808
  • Jan 19 15:09
    pitr-ch closed #831
  • Jan 19 15:09
    pitr-ch commented #831
  • Jan 19 11:51
    pitr-ch commented #840
  • Jan 16 16:54
    fzakaria commented #840
  • Jan 16 16:53
    fzakaria synchronize #840
  • Jan 16 13:40
    jdantonio commented #840
Eloy Espinaco
@eloyesp
@allcentury ^
Anthony Ross
@allcentury
@eloyesp cheers sir! I will give this a try in the morning, thanks again
Chris TenHarmsel
@epchris
Hi all, I have this strange case where a method is returning Concurrent::Promises.fulfilled_future(true) and this breaks later on with this error: TypeError: returned value true is not a Future. However, if I do something like: f = Concurrent::Promises.fulfilled_future(true); logger.info("HELLO WORLD"); return f then everything works fine....
Any ideas why this might be happening?
Chris TenHarmsel
@epchris
Actually, I think it's because I was later calling flat_future on that future, because in some cases I was anticipating that I might have nested futures
this still leaves me puzzled
up the stack I have .then_on(pool, upload, &method(:method_that_returns_a_future)), and when that looks like this: .then_on(pool, upload, &method(:method_that_returns_a_future)).flat_future I get that error
Chris TenHarmsel
@epchris
This is surprising to me since the then_on documentation states that the block return value: "will become result of the returned Future. Its returned value becomes #value fulfilling it,
so I expected a Future fulfilled with another future
Charles Oliver Nutter
@headius
@epchris I have not used future in concurrent-ruby but can you gist a full backtrace?
Charles Oliver Nutter
@headius
@pitr-ch Are you still helping to maintain concurrent-ruby? I can merge ruby-concurrency/concurrent-ruby#840 but I'm not sure about those failures
I don't think I have push rights for the gem either, and not sure about current dev status
Farid Zakaria
@fzakaria
headius: hey -- took me a while to download the IRC app
Charles Oliver Nutter
@headius
Gitter app I assume
hello!
Farid Zakaria
@fzakaria
Nah -- i'm using Textual IRC
Charles Oliver Nutter
@headius
oh nice, IRC bridge to gitter
Farid Zakaria
@fzakaria
yea -- not bad https://irc.gitter.im/
i remember the gitter app being terrible
Charles Oliver Nutter
@headius
it's passable but we moved JRuby to Matrix because Gitter seems unmaintained
still have to use Gitter for several projects though
Farid Zakaria
@fzakaria
Ah -- Slack is a no go ? As much as I dislike how it's taken over at least it works well on my phone :)
(I was just chatting to a co-worker)
So what do you think about that PR & I'
I'd love to resume the chat about reworking the AtExit;
Jerry D'Antonio
@jdantonio
@headius Yes, @pitr-ch is still the maintainer, but he's many timezones ahead of us in the US so he may not be online right now. Re Gitter, I set this up a long time ago but I haven't done any open source work in a few years so I never moved to something else. Since I only lurk these days it would be up to @pitr-ch to switch to a different tool. @fzakaria WRT this specific PR, I'd love to see it merged since I'm the one who wrote the original, dirty hack. Unfortunately, I don't understand the build failure, either. It looks like actors are failing and that's Pitr's thing.
Farid Zakaria
@fzakaria
Sounds good.
fzakaria @fzakaria I'll wait
Farid Zakaria
@fzakaria
i'll wait to hear back from him. Exposing a ruby-shim for a thread-factory would be a cleaner solution than the AtExit; the simplicity is punting the cleanup to the user
Jerry D'Antonio
@jdantonio
@fzakaria The failing test has a comment FIXME this leads to weird message processing ordering and is one of the "edge" features. So this may just be a buggy test and probably shouldn't prevent us from merging. Can you please comment out the test on line 191 of spec/concurrent/actor_spec.rb and see if it passes CI? If that works we may just temporarily disable the test and merge your change.
Charles Oliver Nutter
@headius
I'm going to try re-running that Java 11 failure on master
it passed in the previous build and failed before that so it seems like something is hanging intermittently
yeah it's failing like one out of every five
I'll run it a bunch of times locally and see if I can get it to hang here
Charles Oliver Nutter
@headius
blasted thing refuses to hang locally
Farid Zakaria
@fzakaria
jdantonio: I had to skip another test in that actor spec
Charles Oliver Nutter
@headius
@fzakaria @jdantonio I'm adding concurrent-ruby to JRuby's travis so we can make sure it stays green... I also moved jruby-head out of allowed-failures on concurrent-ruby, so if it fails someone will let us know
the hangs in 9.2.8 were due to a mutex issue fixed in 9.2.9, but unfortunately 9.2.9 introduced a subtle Java integration bug that's fixed in 9.2.10
here's the first concurrent-ruby run on JRuby travis: https://travis-ci.org/jruby/jruby/jobs/637495980
Charles Oliver Nutter
@headius
@fzakaria I think you should move your spec skips to a separate PR...it's not relevant to your daemon PR
I will take a little time this morning to see if I can get the specs to be more stable, but jruby-head seems very solid right now so the failures on JRuby 9.2.9 may be something we can ignore
Farid Zakaria
@fzakaria
headius: gave it another pass
Charles Oliver Nutter
@headius
@fzakaria all checks passed!
Chalupa Petr
@pitr-ch
@epchris could you open an github issue with the whole snippet for me to look at, thanks
@headius yes I am maintaining the gem, my time is quite limited though so I rely a lot on external contributions
Chalupa Petr
@pitr-ch
Yes there are intermittent failures in CI. Disabling the actor test does not help though. It is a place where it often manifests for some reason, but running just the actor tests for a whole night works fine.
Pablo Herrero
@pabloh

Hi, I'm looking for a advice about how to implement the following schema:

I want to share a thread pool among several workers (separate threads), to parallelize I/O bounded tasks using Futures, I would like to have each worker to use 10 threads at most, and no more that 50 in total for the whole process, I wanted to know if there's some recommend way to configure a thread pool for such use case?.

I would like to keep threads on the pool alive for a while (I'm using 2 minutes although I'm not sure if that's the best option), since the previous approach I had (parallel gem) was creating a new Thread for each new task and the memory usage kept going up fast, since Ruby doesn't seem to return the resources to the OS fast enough.

The best solution I could come up with is this: https://gist.github.com/pabloh/94c5dd1ead2ed8bc26dfbba600691f89, that's close to what I want but I don't know if that idle time is a good choice, or how to limit the max number of threads used per worker to just 10.

BTW I'm on CRuby 2.6.5 and sadly I can't use an async solution for the time being since the whole codebase I have is already implemented as blocking.

Charles Oliver Nutter
@headius
@pabloh I don't know of a way to configure an executor to do something like "chunked" thread groups (e.g. your ten-per-worker thing) but your workers could manage that by feeding work into a 10-deep blocking queue
that's just one possible way, but suffice to say there's nothing I know of in the executor stuff to prevent any worker from feeding enough jobs in to saturate all 50 threads
@pitr-ch I will try to help keep an eye on issues and PRs and make sure contributions are moving along