by

Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Activity
  • Jun 23 14:25
    SuperFluffy opened #774
  • Jun 21 11:51
    Jasper-Bekkers closed #576
  • Jun 20 18:57
    bergkvist edited #773
  • Jun 20 18:56
    bergkvist opened #773
  • Jun 19 23:17
    bergkvist edited #772
  • Jun 19 23:16
    bergkvist opened #772
  • Jun 19 01:15
    CAD97 synchronize #771
  • Jun 19 01:14
    CAD97 synchronize #771
  • Jun 19 01:12
    CAD97 synchronize #771
  • Jun 18 22:24
    CAD97 synchronize #771
  • Jun 18 21:51
    CAD97 synchronize #771
  • Jun 18 21:41
    CAD97 synchronize #771
  • Jun 18 21:35
    CAD97 ready_for_review #771
  • Jun 18 21:34
    CAD97 edited #771
  • Jun 18 21:34
    CAD97 synchronize #771
  • Jun 18 21:30
    CAD97 edited #771
  • Jun 18 21:29
    CAD97 synchronize #771
  • Jun 18 21:27
    CAD97 edited #771
  • Jun 18 21:23
    CAD97 synchronize #771
  • Jun 18 05:16
    CAD97 converted_to_draft #771
Josh Stone
@cuviper
on 1.31.0
needs 1.34, but do all 32-bit targets have AtomicU64 anyway?
I thought we talked about using AtomicUsize instead
Niko Matsakis
@nikomatsakis
I think we probably did
I have a vague memory of that
Josh Stone
@cuviper
and how that would limit the number of threads, but <256 threads for 32-bit is probably fine
though we need to enforce that
frankly, over 256 threads is probably past diminishing returns even on 64-bit targets
Niko Matsakis
@nikomatsakis
ah yeah I remember that I wanted to document those flags
and convince myself that rollover was good
let's start (on the PR?) a to-do list of things to track
Niko Matsakis
@nikomatsakis
Ok, I did that, feel free @cuviper to edit and add add'l items
should've done this a while back
Niko Matsakis
@nikomatsakis
heh, I found the dropbox paper that we created during that video, it has some of these work items too
Zhengyi Yang
@zhengyi-yang
Is there a par_drain method in rayon?
Josh Stone
@cuviper
@zhengyi-yang there's not, but I think we could support that fairly easily. could you file an enhancement issue?
easy for Vec, at least -- I don't think we could do that for other collections without access to more internals
Zhengyi Yang
@zhengyi-yang
@cuviper No problem. Thanks Josh.
Niko Matsakis
@nikomatsakis
hey @cuviper not sure I can make the sync today, but I have nothing to add beyond the work I did on the PR recently -- I think the next step I had in mind is to update RFC a bit and to run some measurements, but I was expecting you would need time to review the code in more detail in any case
I see there's a lot of notifications though (been trying to stay on top of that...) and I might try to go through some of them over next few days
I'm expecting to be on "stay-cation" at home, but then again I tend to count time working on rayon as fun vacation anyway :)
Josh Stone
@cuviper
@nikomatsakis ok, no worries, I still owe you a new review on that
I'll always welcome your input on general issues and such too, when you have the time :)
Niko Matsakis
@nikomatsakis
:heart:
Zhengyi Yang
@zhengyi-yang
Is it possible to implement rayon's par_iter and into_par_iter for crossbeam::queue?
Josh Stone
@cuviper
hmm, maybe -- we'd have to do it ourselves, not in crossbeam, to keep the dependency order straight
it would be an unindexed producer
@zhengyi-yang they don't implement plain Iterator either
Zhengyi Yang
@zhengyi-yang
@cuviper Thanks Josh. I see. So for now if I want to do some parallel to a queue, is it efficient to use something like (0..queue.len()).into_par_iter().map(|_| {queue.deque()})?
Josh Stone
@cuviper
I guess you mean queue.pop()? that would be efficient enough, sure, although it will just fail, not block, if the queue is empty
if you know nothing else will be pushing/popping, that should be fine
they won't be in order, of course
Zhengyi Yang
@zhengyi-yang
Okay. Got it. Thanks so much!
Niko Matsakis
@nikomatsakis
@cuviper btw we're going to have to find a better time for the sync, this particular time doesn't work for me anymore
Josh Stone
@cuviper
@nikomatsakis I don't have many scheduled meetings, so please suggest times that work for you. maybe a doodle?
Niko Matsakis
@nikomatsakis
@cuviper ok
Zhengyi Yang
@zhengyi-yang
Hello guys, does someone know if the order will be reserved if I call vec.into_par_iter().flat_map().collect::<Vec<_>>()
And if the order doesn't matter, can I do collect faster?
Josh Stone
@cuviper
@zhengyi-yang collect does preserve order, at least for ordered collections like Vec
if you don't care about order, maybe some kind of concurrent vector would be faster, which you could push in a for_each, but I don't have specific recommendations
flat_map is kind of awkwardly handled in rayon, since we split that in parallel too -- it may over-split your data and make it move data a lot to collect it back
sometimes you may do better with a custom fold/reduce
Zhengyi Yang
@zhengyi-yang
@cuviper Got it, thanks Josh
Niko Matsakis
@nikomatsakis
Man I'm so sad I apparently never pushed to https://github.com/nikomatsakis/rayon-plot
I wonder if some old computer of mine might have a checkout
I was looking @cuviper just a bit into the sort performance but nothing jumped out at me yet
Christopher N. Hesse
@raymanfx
Hi everyone. I'm looking for feedback similar to the case of rayon-rs/rayon#699, but didn't want to necro that old "issue". Basically, I'd like to parallelize the following (outer) loop: https://github.com/raymanfx/ffimage/blob/next/src/packed/convert.rs#L18. Now I'm not sure how to do that. I implemented iterators that hand out mutable pixel references, so I could iterate through the target image buffer. But then I don't have the pixel coordinates anymore which I need to transform the source pixel. I tried zipping the mutable target and the immutable source pixel iterators, but did not succeed that way. I'd appreciate if someone could give me a hint here.
  • adding to the above: I'd like to perform the conversion in memory, so don't want to collect the converted pixels in a Vec and then overwrite the image pixels with the ones from that vec
Niko Matsakis
@nikomatsakis
@cuviper so do you have any sense for how we ought to proceed with "the branch"?
I'm feeling a bit stuck :/