Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Activity
  • Jan 06 12:28
    CHCP edited #821
  • Jan 06 12:28
    CHCP edited #821
  • Jan 06 12:26
    CHCP edited #821
  • Jan 06 12:26
    CHCP edited #821
  • Jan 06 12:25
    CHCP opened #821
  • Dec 21 2020 22:09
    bluss closed #614
  • Dec 10 2020 19:06
    nikomatsakis closed #816
  • Dec 10 2020 19:05
    nikomatsakis closed #819
  • Dec 08 2020 09:15
    errrken closed #818
  • Dec 02 2020 23:08
    jpalus edited #820
  • Dec 02 2020 13:18
    jpalus edited #820
  • Dec 02 2020 13:15
    jpalus opened #820
  • Dec 02 2020 04:02
    tyan-boot opened #819
  • Nov 28 2020 17:54
    errrken edited #818
  • Nov 28 2020 17:49
    errrken opened #818
  • Nov 27 2020 11:39
    JackThomson2 edited #817
  • Nov 27 2020 11:38
    JackThomson2 synchronize #817
  • Nov 27 2020 11:14
    JackThomson2 opened #817
  • Nov 23 2020 20:00
    vtavernier opened #816
  • Nov 19 2020 19:31
    bors[bot] closed #815
Dirkjan Ochtman
@djc
T1 is a String in my case
matt rice
@ratmice
like https://docs.rs/im/15.0.0/im/struct.OrdMap.html I guess is what I was thinking, it has aParallelIterator impl IIRC, similar to BTreeMap
Dirkjan Ochtman
@djc
it's interesting that it has a union_with() method
matt rice
@ratmice
Maybe i'm misremembering it having ParallelIterator, but you don't actually need that to construct one I guess
Dirkjan Ochtman
@djc
yeah, looks like only Vector impls ParallelIterator
wagnerf42
@wagnerf42
@djc reducing like you do has a high cost because the same data gets fused and fused all over again
fusing ordered trees would be bad
wagnerf42
@wagnerf42
you can maybe try a concurrent hashmap
also, this seem very memory intensive, you can maybe get better performances with less threads
Josh Stone
@cuviper
I was also going to suggest a concurrent data structure -- maybe like the sharded dashmap
Dirkjan Ochtman
@djc
I've changed the code to use ordered Vecs instead, which seems to help at least some
Niko Matsakis
@nikomatsakis
@cuviper :wave:
Josh Stone
@cuviper
hey @nikomatsakis
Niko Matsakis
@nikomatsakis
how goes
...I'm curious about PRs...
Josh Stone
@cuviper
it goes -- apparently my rayon backlog has deadlocked
Niko Matsakis
@nikomatsakis
oh?
Josh Stone
@cuviper
just that I keep not getting around to it
nikomatsakis @nikomatsakis knows the feeling
Niko Matsakis
@nikomatsakis
we don't really have many problems with new scheduler, right?
well I see rayon-rs/rayon#820 :)
it seems like we can close rayon-rs/rayon#819 ?
Josh Stone
@cuviper
Niko Matsakis
@nikomatsakis
and probably rayon-rs/rayon#816 too
Josh Stone
@cuviper
816 and 819 to close, sure
at least I don't think we have any action
Niko Matsakis
@nikomatsakis
yep
nice diagnoses :)
anything else going on ?
I don't see many other new issues
plenty of old ones
it might be fun to spend some time (not today) doing a triage effort
we only have 109 issues, we could actually get through them all without too much effort
Josh Stone
@cuviper
I don't think there's anything else new
want to plan a rayon triage blitz, maybe next week?
Niko Matsakis
@nikomatsakis
I do but not next week
:)
Josh Stone
@cuviper
the next two are xmas eve and new year's eve
Niko Matsakis
@nikomatsakis
yeah, I was thinking more in january
Josh Stone
@cuviper
Thursdays, that is
ok, sure
Niko Matsakis
@nikomatsakis
actually I might be inclined to skip syncs until January
Josh Stone
@cuviper
yeah
Niko Matsakis
@nikomatsakis
my .. life is very full over the next few weeks
Bryn Keller
@xoltar
Hi folks, in a Rayon for_each that pulls from an iterator bridge, is it normal to have perf report show the top two lines taking so much time? Things that I recognize as my code don't show up until the third or fourth line, at around 4.7%. Does this indicate that the source iterator is slow somehow?
  51.03%  sift     sift                [.] rayon::iter::plumbing::bridge_producer_consumer::helper                                             
  13.66%  sift     [kernel.kallsyms]   [k] native_queued_spin_lock_slowpath
Josh Stone
@cuviper
@xoltar it's likely that a lot of your code is inlined in this helper.
that's a recursive function, so it tends to be the outermost layer
if you enable debuginfo in your release build, you can get perf to annotate that function with source
or get perf to record callgraphs, which will include inlining, or use another tool that does that like flamegraphs
Bryn Keller
@xoltar
I see, yes, there's a lot of my code inlined there, thanks!