Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Activity
  • Jan 22 01:17
    cuviper labeled #719
  • Jan 22 01:17
    cuviper labeled #719
  • Jan 21 17:43
    ian-p-cooke opened #719
  • Jan 04 05:04
    dbregman opened #718
  • Dec 23 2019 14:51
    bors[bot] closed #717
  • Dec 23 2019 14:51

    bors[bot] on master

    Change wording, less "High leve… Merge #717 717: Change wording… (compare)

  • Dec 23 2019 14:33

    bors[bot] on staging.tmp

    (compare)

  • Dec 23 2019 14:33

    bors[bot] on staging

    Change wording, less "High leve… Merge #717 717: Change wording… (compare)

  • Dec 23 2019 14:33

    bors[bot] on staging.tmp

    Change wording, less "High leve… [ci skip][skip ci][skip netlify… (compare)

  • Dec 23 2019 14:33

    bors[bot] on staging.tmp

    [ci skip][skip ci][skip netlify] (compare)

  • Dec 22 2019 11:35
    matthiasbeyer opened #717
  • Dec 22 2019 00:58
    cuviper closed #697
  • Dec 22 2019 00:57
    cuviper closed #698
  • Dec 22 2019 00:54

    cuviper on v1.3.0

    (compare)

  • Dec 22 2019 00:54

    cuviper on rayon-core-v1.7.0

    (compare)

  • Dec 22 2019 00:54

    cuviper on rayon-futures-v0.1.1

    (compare)

  • Dec 21 2019 15:59
    bors[bot] closed #716
  • Dec 21 2019 15:59

    bors[bot] on master

    Remove rayon-futures Remove cfg(rayon_unstable) Update ci/compat-Cargo.lock and 7 more (compare)

  • Dec 21 2019 15:20

    bors[bot] on staging.tmp

    (compare)

  • Dec 21 2019 15:20

    bors[bot] on staging

    Remove rayon-futures Remove cfg(rayon_unstable) Update ci/compat-Cargo.lock and 7 more (compare)

Josh Stone
@cuviper
or Arc-wrapped
Emilio Cobos Álvarez
@emilio
Right, but I cannot access any Arc-wrapped thing in an exit_handler, afaict...
Oh, nvm, it is a closure, I thought it was just a plain fn
Josh Stone
@cuviper
@emilio please file an issue so this doesn't get lost, including some description of the workflow you want
I think we can do it properly in a new API, just need to design that
Emilio Cobos Álvarez
@emilio
Josh Stone
@cuviper
thanks
Emilio Cobos Álvarez
@emilio
np, thank you :)
Josh Stone
@cuviper
@nikomatsakis sync?
Niko Matsakis
@nikomatsakis
@cuviper :wave:
Josh Stone
@cuviper
hey
spawn_future is probably the most interesting thing we have to discuss :)
Niko Matsakis
@nikomatsakis
yeah, probably
sorry, was just skimming a bit
I'm catching up on that PR now
Josh Stone
@cuviper
ISTM that accepting a Future puts us in quite a different position than just returning one
Niko Matsakis
@nikomatsakis
I don't really understand Alex's comment
I guess I would assume he didn't read the PR very closely
anyway, I agree it is rather different
which is why I was pressing for it :)
as it seems significantly more enabling
Josh Stone
@cuviper
my loose understanding is that it would be more difficult for a rayon thread in WASM to wait for an async event from the outside environment (JS)
but it's not really clear to me who would get suspended where
async is still a bit magic to me
Niko Matsakis
@nikomatsakis
I guess I don't really understand what the WASM integration looks like
if we ignore that for a second,
you wrote this on the PR
I guess if the async work leading up to our part isn't ready, we'd just return NotReady too and let some outer executor deal with it?
I believe what would happen is roughly like this:
  • when you invoke spawn_future(F: impl Future), we would schedule a job that calls F::poll, indeed
  • we would give it a Waker that is tied to rayon, I think
  • that job would return NotReady (due to some I/O event), and it would have clone'd our Waker to hold on to it
  • when the I/O event occurs, it would invoke the wake method; this would cause us to add the job back to the (Rayon) thread-pool, at which point we go back to step 1
  • when I did this before, the "unsafety" bit was exactly around this Waker step. If you spawned a future into a scope, I wanted to guarantee that until that future was complete, the scope would not terminate. But the compiler couldn't know that.
  • it worked by having the future hold a ref on the scope, basically, that wasn't discharged until the future was enqueued etc
I also set things up to have exactly one Arc; I think it served as the waker + the "task" that we enqueued + the future that we returned to the user
anyway, I guess the question at hand is sort of ... does it make sense to have a "create future" API that just takes a closure? it does suffice for taking some heavy CPU computation that uses Rayon
Niko Matsakis
@nikomatsakis
what it doesn't do is let us have DAGs of computations or anything like that; but I don't know that anybody is asking for it -- and, if they did, I don't know that the Future API would be the way to set it up, or at least only for some sorts of cases
Josh Stone
@cuviper
suppose we don't do all that, and just take FnOnce as my PR does now. I think this means that async users would just do something like other_future.and_then(|| rayon::spawn_future(...))
Niko Matsakis
@nikomatsakis
still, it feels like the "right" setup to me, even if it mostly only ever gets used to spawn an async block that is effectively a closure
Josh Stone
@cuviper
so the blocking on other async events would be completely external from us
Niko Matsakis
@nikomatsakis
well it kind of defeats the point of async
which is to let you write .await and not convert to CPS form
also, that .. wouldn't exactly suffice I don't think
because calling and_then just creates a future
it doesn't "schedule" it for execution
you'd have to enqueue that future somewhere else
(which can't be rayon, because we wouldn't offer that API)
Josh Stone
@cuviper
then this?
let x = other_future.await;
rayon::spawn_future(|| foo(x)).await;
// etc.
Niko Matsakis
@nikomatsakis
right, you could do that, you just can't do the await inside of the "rayon code"