by

Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Activity
  • Jul 10 19:01
    XAMPPRocky edited #591
  • Jul 08 16:03
    linweijun opened #775
  • Jul 05 19:04
    bors[bot] closed #774
  • Jul 05 19:04

    bors[bot] on master

    Note that collect_into_vec is i… Merge #774 774: Note that coll… (compare)

  • Jul 05 18:57

    bors[bot] on staging.tmp

    (compare)

  • Jul 05 18:57

    bors[bot] on staging

    Note that collect_into_vec is i… Merge #774 774: Note that coll… (compare)

  • Jul 05 18:57

    bors[bot] on staging.tmp

    Note that collect_into_vec is i… [ci skip][skip ci][skip netlify… (compare)

  • Jul 05 18:57

    bors[bot] on staging.tmp

    [ci skip][skip ci][skip netlify] (compare)

  • Jun 23 14:25
    SuperFluffy opened #774
  • Jun 21 11:51
    Jasper-Bekkers closed #576
  • Jun 20 18:57
    bergkvist edited #773
  • Jun 20 18:56
    bergkvist opened #773
  • Jun 19 23:17
    bergkvist edited #772
  • Jun 19 23:16
    bergkvist opened #772
  • Jun 19 01:15
    CAD97 synchronize #771
  • Jun 19 01:14
    CAD97 synchronize #771
  • Jun 19 01:12
    CAD97 synchronize #771
  • Jun 18 22:24
    CAD97 synchronize #771
  • Jun 18 21:51
    CAD97 synchronize #771
  • Jun 18 21:41
    CAD97 synchronize #771
Josh Stone
@cuviper
I'll always welcome your input on general issues and such too, when you have the time :)
Niko Matsakis
@nikomatsakis
:heart:
Zhengyi Yang
@zhengyi-yang
Is it possible to implement rayon's par_iter and into_par_iter for crossbeam::queue?
Josh Stone
@cuviper
hmm, maybe -- we'd have to do it ourselves, not in crossbeam, to keep the dependency order straight
it would be an unindexed producer
@zhengyi-yang they don't implement plain Iterator either
Zhengyi Yang
@zhengyi-yang
@cuviper Thanks Josh. I see. So for now if I want to do some parallel to a queue, is it efficient to use something like (0..queue.len()).into_par_iter().map(|_| {queue.deque()})?
Josh Stone
@cuviper
I guess you mean queue.pop()? that would be efficient enough, sure, although it will just fail, not block, if the queue is empty
if you know nothing else will be pushing/popping, that should be fine
they won't be in order, of course
Zhengyi Yang
@zhengyi-yang
Okay. Got it. Thanks so much!
Niko Matsakis
@nikomatsakis
@cuviper btw we're going to have to find a better time for the sync, this particular time doesn't work for me anymore
Josh Stone
@cuviper
@nikomatsakis I don't have many scheduled meetings, so please suggest times that work for you. maybe a doodle?
Niko Matsakis
@nikomatsakis
@cuviper ok
Zhengyi Yang
@zhengyi-yang
Hello guys, does someone know if the order will be reserved if I call vec.into_par_iter().flat_map().collect::<Vec<_>>()
And if the order doesn't matter, can I do collect faster?
Josh Stone
@cuviper
@zhengyi-yang collect does preserve order, at least for ordered collections like Vec
if you don't care about order, maybe some kind of concurrent vector would be faster, which you could push in a for_each, but I don't have specific recommendations
flat_map is kind of awkwardly handled in rayon, since we split that in parallel too -- it may over-split your data and make it move data a lot to collect it back
sometimes you may do better with a custom fold/reduce
Zhengyi Yang
@zhengyi-yang
@cuviper Got it, thanks Josh
Niko Matsakis
@nikomatsakis
Man I'm so sad I apparently never pushed to https://github.com/nikomatsakis/rayon-plot
I wonder if some old computer of mine might have a checkout
I was looking @cuviper just a bit into the sort performance but nothing jumped out at me yet
Christopher N. Hesse
@raymanfx
Hi everyone. I'm looking for feedback similar to the case of rayon-rs/rayon#699, but didn't want to necro that old "issue". Basically, I'd like to parallelize the following (outer) loop: https://github.com/raymanfx/ffimage/blob/next/src/packed/convert.rs#L18. Now I'm not sure how to do that. I implemented iterators that hand out mutable pixel references, so I could iterate through the target image buffer. But then I don't have the pixel coordinates anymore which I need to transform the source pixel. I tried zipping the mutable target and the immutable source pixel iterators, but did not succeed that way. I'd appreciate if someone could give me a hint here.
  • adding to the above: I'd like to perform the conversion in memory, so don't want to collect the converted pixels in a Vec and then overwrite the image pixels with the ones from that vec
Niko Matsakis
@nikomatsakis
@cuviper so do you have any sense for how we ought to proceed with "the branch"?
I'm feeling a bit stuck :/
I dont' feel great about the perf results, it'd probably be good to drill a bit more into which ones we care about
I guess if you're interested, I might be game to schedule some time to screen share and do a bit of joint investigation :)
not sure if that's something that "pairs well"
also would Tuesdays at 15:30 Eastern time (12:30 Pacific) work for you as a sync slot?
Josh Stone
@cuviper
@nikomatsakis sorry I missed this. Tuesdays at 12:30 Pacific will be fine for me
for the branch, I don't know -- I'm almost inclined to just SHIP IT :rocket:
but we should probably make sure that it at least still improves the idle case it's targeting
(make sure the redesigns haven't lost that property)
Niko Matsakis
@nikomatsakis
@cuviper ok let me add something to the calendar. Regarding the branch, I kind of feel the same way.
Like, let's land it, maybe do an alpha release and encourage people to test
but yeah let's verify that we're solving the problem we set out to solve first I guess
I definitely feel like it'd be awesome if we could draw some eyes, maybe some poeple can come up in and optimize things
living on a PR is no way to live
Luca Barbato
@lu-zero
I can test it today
btw is there a way to move from a parallel iterator to a serial iterator without having a collect() in between?
Josh Stone
@cuviper
@lu-zero there's no builtin way to go parallel to serial. A channel might work, if you don't care about order.
Luca Barbato
@lu-zero
I care about order thus why I wanted that kind of adaptor ^^
meanwhile I have some data on ARM, our usual test encode is just slightly faster there
Josh Stone
@cuviper
well the issue for serial conversion is #210
faster is good :)
Luca Barbato
@lu-zero
:)
it seems so