Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Activity
  • Sep 19 23:17
    jClaireCodesStuff synchronize #698
  • Sep 18 23:31
    jClaireCodesStuff opened #698
  • Sep 18 00:47
    jClaireCodesStuff opened #697
  • Sep 15 18:18
    jondot opened #696
  • Sep 14 19:35
    nikomatsakis synchronize #695
  • Sep 14 19:33
    nikomatsakis opened #695
  • Sep 13 14:14
    magnusmanske opened #694
  • Sep 09 23:08
    tiby312 edited #693
  • Sep 09 23:07
    tiby312 edited #693
  • Sep 09 23:04
    tiby312 edited #693
  • Sep 09 23:04
    tiby312 opened #693
  • Sep 06 21:01
    bors[bot] closed #692
  • Sep 06 21:01
    bors[bot] closed #687
  • Sep 06 21:01

    bors[bot] on master

    Link the iterator traits from t… Typo in docs Co-Authored-By: J… Merge #692 692: Link the itera… (compare)

  • Sep 06 19:33

    bors[bot] on staging.tmp

    (compare)

  • Sep 06 19:33

    bors[bot] on staging

    Merge #692 692: Link the itera… (compare)

  • Sep 06 19:33

    bors[bot] on staging.tmp

    Link the iterator traits from t… Typo in docs Co-Authored-By: J… [ci skip][skip ci][skip netlify… (compare)

  • Sep 06 19:33

    bors[bot] on staging.tmp

    [ci skip][skip ci][skip netlify] (compare)

  • Sep 06 19:32
    vorner synchronize #692
  • Sep 06 19:25

    bors[bot] on staging.tmp

    (compare)

Josh Stone
@cuviper
the rest is stale
Niko Matsakis
@nikomatsakis
ok
I'd like to check out the future thing
sorry, got distracted, I suck this week :)
probably we'll really get back going post rustconf
Josh Stone
@cuviper
sure :)
and yeah, meeting up wednesday would be nice
Josh Stone
@cuviper
@stjepang I saw the announcement of async-std, congrats!
now my question, would my rayon spawn_future be usable with this?
or would you just task::spawn(async { /* something using rayon */ })?
Stuart Axelbrooke
@soaxelbrooke
Howdy! I am seeing a globally built thread pool only spawn 1 thread (should be 4) - any tips on how to debug this? I observed this via a work queue where workers lock one of 4 resources (1 per thread) to do their work, but only 1 lock is ever being attempted at a time. The thread pool is used via something.into_par_iter().for_each(...).
Josh Stone
@cuviper
@soaxelbrooke what does rayon::current_num_threads() say?
Stuart Axelbrooke
@soaxelbrooke
I'll check, brb!
It's reporting 4 threads
Josh Stone
@cuviper
well at least that's what you expected :)
Stuart Axelbrooke
@soaxelbrooke
:P
Josh Stone
@cuviper
what type is something, and how many items does it have?
Stuart Axelbrooke
@soaxelbrooke
its a vector of work items, ~1000 items long
owned by the function that into_par_iter() is being invoked in
Josh Stone
@cuviper
how are you selecting which to lock among your 4 resources?
Stuart Axelbrooke
@soaxelbrooke
it iterates over the vec of workers calling try_lock() until one is available. It logs before attempting try_lock (just in case it blocked for some reason), but only one thread ever logs an attempt.
Josh Stone
@cuviper
you could just pick workers[rayon::current_thread_index()] if the lengths match
Stuart Axelbrooke
@soaxelbrooke
that's good to know!
it was definitely a hack that has stuck around
Josh Stone
@cuviper
once you get this working, you might try for_each_init where the init function is the one that locks your worker locally, returning the MutexGuard
but backing up, it's strange that you don't even get your logged attempts
Stuart Axelbrooke
@soaxelbrooke
I added the thread index to the logging, and it looks like different threads are active, just never at the same time...
Josh Stone
@cuviper
ok, I was about to suggest that
weird
Stuart Axelbrooke
@soaxelbrooke
for context, this is a web scraper, and the work items are different URLs to scrape, though I don't think that would change anything
Josh Stone
@cuviper
that will definitely block the thread, as this isn't an async library, but other threads should still progress
if you attach a debugger, you should be able to get a backtrace of each thread, and see where they're blocked
Stuart Axelbrooke
@soaxelbrooke
oh god, I'm sorry, it was a shared num_processed variable they were all trying to lock at the same time
threading, how do you even
Josh Stone
@cuviper
whew
rustc will make sure your threading is safe, but not necessarily effective
Stuart Axelbrooke
@soaxelbrooke
you can only protect people from themselves so much :P
Niko Matsakis
@nikomatsakis
So @cuviper I left a comment on #679 -- basically I think that the signature of spawn_future is maybe not quite what I expected
Josh Stone
@cuviper
OK, I hadn't thought of it that way
I guess we would need to implement a Context and Waker then
which is probably doable
Josh Stone
@cuviper
interesting, async-task does look appropriate
Niko Matsakis
@nikomatsakis
@cuviper can't make sync today; first day of school and I want to take "DD" out to ice cream :)
but after digging a bit more into async-task, it did seem like a good fit for what we need -- haven't checked if there are more comments on #679 yet though
Josh Stone
@cuviper
no worries, kids are synchronous
Josh Stone
@cuviper
@nikomatsakis any concerns before I publish 1.2? #686
nothing major in there, but a few folks were wanting the updated crossbeam-deque
Emilio Cobos Álvarez
@emilio
Is there any way (even with some runtime overhead) to wait on a thread-pool to shut down? I basically want a sync-drop, to ensure everything is finished by some point, for leak-checking purposes
If there is none, would there be any objection to adding such a switch? @cuviper?
Josh Stone
@cuviper
the global pool doesn't shut down at all