Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Activity
  • Jan 22 22:18

    cuviper on rayon-core-v1.10.2

    (compare)

  • Jan 22 22:17
    bors[bot] closed #1008
  • Jan 22 22:17
    bors[bot] closed #952
  • Jan 22 22:17
    bors[bot] closed #1013
  • Jan 22 22:17

    bors[bot] on master

    Release rayon-core 1.10.2 Merge #1013 1013: Release rayo… (compare)

  • Jan 22 22:10

    bors[bot] on staging.tmp

    (compare)

  • Jan 22 22:10

    bors[bot] on staging

    Release rayon-core 1.10.2 Merge #1013 1013: Release rayo… (compare)

  • Jan 22 22:10

    bors[bot] on staging.tmp

    Release rayon-core 1.10.2 [ci skip][skip ci][skip netlify… (compare)

  • Jan 22 22:10

    bors[bot] on staging.tmp

    [ci skip][skip ci][skip netlify] (compare)

  • Jan 22 22:07
    cuviper opened #1013
  • Jan 21 23:59
    bors[bot] closed #1011
  • Jan 21 23:59

    bors[bot] on master

    Use pointers instead of `&self`… Add a virtual wrapper for &Latch Merge #1011 1011: Use pointers… (compare)

  • Jan 21 23:53

    bors[bot] on staging.tmp

    (compare)

  • Jan 21 23:53

    bors[bot] on staging

    Use pointers instead of `&self`… Add a virtual wrapper for &Latch Merge #1011 1011: Use pointers… (compare)

  • Jan 21 23:53

    bors[bot] on staging.tmp

    [ci skip][skip ci][skip netlify] (compare)

  • Jan 21 23:53

    bors[bot] on staging.tmp

    Use pointers instead of `&self`… Add a virtual wrapper for &Latch [ci skip][skip ci][skip netlify… (compare)

  • Jan 20 20:48
    cuviper ready_for_review #1011
  • Jan 20 16:51
    cuviper synchronize #1011
  • Jan 19 10:15
    yu-re-ka synchronize #1012
  • Jan 19 07:37
    yu-re-ka opened #1012
xmac94x
@xmac94x:matrix.org
[m]
Hi everyone :) I just noticed some weird behavior on our game velorem where we are waiting for some threads to finish in places where we wouldn't have thought they would. I opened a issue with the details: rayon-rs/rayon#969 . Not sure if this is a bug, but we noticed that a threadpool.spawn inside a parallel iterator will let the parallel iterator wait until the spawn completed. Which is not always what we want. Do you know any good workaround for that problem ? Maybe already others experienced it (:
Josh Stone
@jistone:fedora.im
[m]
xmac94x: I think this is the nature of work-stealing, but see my reply on the issue for gory details
(also, don't take my word as gospel!)
xmac94x
@xmac94x:matrix.org
[m]
I understand that rayon is using work-stealing, but not why its not having a check to not steal unreleated work inside a par_iter ? Or even steal work from other scopes
Josh Stone
@jistone:fedora.im
[m]
it has no concept of what is "related"
there's one global queue for each pool, and each thread within the pool has a local deque
Miguel Raz Guzmán Macedo
@miguelraz
Hello!
I'm looking for a good collection of serial vs Rayon's parallelization benchmarks snippets
Thinking of stuff like
Dot product calculation,
Independently processing many strings in a big file
or anything numeric/related to linear algebra
Josh Stone
@jistone:fedora.im
[m]
rayon-demo (in the rayon repo) has some, not sure if you'd call them "good" though ;)
Miguel Raz Guzmán Macedo
@miguelraz
@jistone:fedora.im yup, those seem just like what I wanted. What's the command to run them though? cargo bench in the rayon/rayon-demo dir?
:sweat:
whoknow
@whoknow:matrix.org
[m]
Hi, somewhere in my code I need to check a big vector (also some times it can be small) for finding some data in it (Im using data.iter().find(...) in old method) its work good, I didn't notice any kind of performance lost in it, but I was thinking changing it so it search in parallel using rayon make sense? I mean there is any cost for rayon to spawn the threads everytime Im calling it? (as I said it would so many times...)
at the end is it worth it for use it in this kind of situations?
Josh Stone
@jistone:fedora.im
[m]
rayon keeps a threadpool, so after the first time it won't be spawning any new threads
3 replies
(unless you explicitly create your own ThreadPool each time)
whoknow
@whoknow:matrix.org
[m]
also one other thing, is it possible to use rayon inside normal loops?
2 replies
WGH
@WGH:torlan.ru
[m]
No clue about Windows, never really used it for any development work
On Linux, I'd use perf, if that helps
No, you can't
Josh Stone
@jistone:fedora.im
[m]
the language for loops are hard-coded to the regular Iterator trait
but if you can rewrite that with Iterator::for_each, then this can translate to ParallelIterator::for_each
I have a crate that hacks that transformation with a procedural macro: https://crates.io/crates/rayon-macro
Miguel Raz Guzmán Macedo
@miguelraz
Let’s say I have a dir with many nested dirs (at different levels of , all containing .txt files that can be processed independently.
What’s the Rayon idiomatic way of processing them? (for argument’s sake, a wc on each file will suffice)
WcaleNieWolny
@WcaleNieWolny
pub fn fast_split(data: &[i8], cache: Vec<usize>) -> Vec<i8>{
        let mut out = Vec::<i8>::with_capacity(data.len());
    //TODO: resize

        for i in 0..data.len(){
            out[cache[i]] = data[i]
        }

        out
    }
how would I make this run in parallel? Like using rayon or something like that
Josh Stone
@jistone:fedora.im
[m]
@WcaleNieWolny: arbitrary parallel writes are difficult in Rust, but the best way is probably to temporarily cast that out vector to &mut [AtomicI8].
or with your own unsafe pointer cast the same way, https://doc.rust-lang.org/src/core/sync/atomic.rs.html#2064
then in rayon, (0..data.len()).into_par_iter().for_each(|i| /* atomic store */)
it may not parallelize well though -- memory bound and probably cache-unfriendly
mr.rusty
@mr.rusty:matrix.org
[m]
Hi everyone! Just wanted to know if things like the JoinHandle are possible with Rayon. Or are there alternatives to wait for a task to finish?
Josh Stone
@jistone:fedora.im
[m]
mr.rusty: most of the rayon api blocks for completion, but for spawn you can use a scope, as that waits for its spawns to complete before returning.
mr.rusty
@mr.rusty:matrix.org
[m]
Thank your, i think that'll help me :)
mr.rusty
@mr.rusty:matrix.org
[m]
:point_up: Edit: Thank you, i think that'll help me :)
Miguel Raz Guzmán Macedo
@miguelraz
Hello @jistone:fedora.im !
I'm thinking of using some of my Xmas break to implement this - rayon-rs/rayon#329
What's a good way to start?
Josh Stone
@jistone:fedora.im
[m]
I don't know how Niko thought that would work, and he never got back to that issue with a better write-up...
@miguelraz: you might have to do your own research on that one
maybe looking into what @wagnerf42 was doing
wagnerf42
@wagnerf42
yeah i do have a scan
each task has its own state
wagnerf42
@wagnerf42
i use it in several places but i think it's always in use with a par_windows
Miguel Raz Guzmán Macedo
@miguelraz
Oh sweet, thanks @wagnerf42!
What would you say are the big blockers before shaping that up into something that could land in Rayon?
Also - I'm a huge fan of your course. I've been scouring for HPC courses that exploited Rust and yours is the only one I've been able to find so far!
<3
JT
@jntrnr
I might be missing something - but if you do .par_bridge() on an iterator that's potentially infinite, is there a way to get the values back out without collecting? Can you chunk it? If not, is there another way to work with a potentially unbound iterator?
wagnerf42
@wagnerf42
@miguelraz hey thanks a lot. thanks to let me know. i have a lot of material i should upload
wagnerf42
@wagnerf42