These are chat archives for rust-lang/rust

2nd
Nov 2017
Sean Perry
@shaleh
Nov 02 2017 05:44
what is the canonical way to walk a string looking for a sub string repeatedly? I know that find will return the pos of the first location. But how to find the second? find does not accept a position and I do not see another method that does. My use case right now is finding the contents of a string. I just found the first double quote. How do I locate the next one? I do not have to worry about nested quotes so easy brute force is fine.
Aleksey Kladov
@matklad
Nov 02 2017 06:38
@shaleh you can slice a string. Like, let pos1 = s.find("foo").unwrap(); let pos2 = s[pos1..].find("foo").
Sean Perry
@shaleh
Nov 02 2017 06:43
ah, the find also works on the slice. Got it.
Paul Masurel
@fulmicoton
Nov 02 2017 07:32
I think doubling the vector capacity is actually a bad strategy
Denis Lisov
@tanriol
Nov 02 2017 07:34
@fulmicoton Why and what strategy would you prefer?
Paul Masurel
@fulmicoton
Nov 02 2017 07:34
trying to find the doc about it.
the key idea was, let's assume you grow your vector by pushing item back to it.
you start with a capacity of 1 to simplify
you will trigger a resize to 2
then to 4 , 8, 16 etc.
every single time you grow your capacity, it is for a capacity that is 2^k = 1 + sum 2^i for i going from 0 to k-1
so it is always slightly greater than the sum of what you have been allocated so far.
this is not playing nice with your allocator, because that means it cannot ever reuse the space it has allocated so far.
i forgot what their alternative strategy was
Paul Masurel
@fulmicoton
Nov 02 2017 07:40
I assume something exponential but with a speed lesser than 2
maybe fibonnaciesque ?
Aleksey Kladov
@matklad
Nov 02 2017 07:42
Wow, an interesting reead @fulmicoton! And std::Vec uses 2! I wonder if there's an issue on the Rust repository about changing this to 1.5...
Paul Masurel
@fulmicoton
Nov 02 2017 07:42
yes I saw that we use 2 in rust
folly does actually way more than that apparently
like it use realloc when possible
and it can also ask jemalloc what a convenient size would be
Aleksey Kladov
@matklad
Nov 02 2017 07:46
It's interesting that Rust plans to switch away from jemalloc soon by default.
Regarding relocatable objects, copy of vector in Rust is always memcpy, which I think is super-nice!
Aleksey Kladov
@matklad
Nov 02 2017 07:49
rust-lang/rust#45434 :)
Paul Masurel
@fulmicoton
Nov 02 2017 07:52
We don't initialize "capacity" do we?
If so I guess for large vectors, the better strategy would be to ask for a super large capacity upfront. Most allocators will call mmap for those. As long as we don't touch them, the OS will not have physical memory to back those.
@matklad haha, I hadn't seen this PR.
Paul Masurel
@fulmicoton
Nov 02 2017 08:01
32 pages is also the MMAP threshold
Aleksey Kladov
@matklad
Nov 02 2017 08:02

If so I guess for large vectors, the better strategy would be to ask for a super large capacity upfront. Most allocators will call mmap for those. As long as we don't touch them, the OS will not have physical memory to back those.

Hm, I wonder if you need to relocate vectors at all if you are allocating in pages...

Like, in theory OS can just fiddle with virtual address table, so that no actual physical copy of RAM happens...
Paul Masurel
@fulmicoton
Nov 02 2017 08:04
yes as long as you have the virtual memory reserved right?
If your vector start at addrand you have a page allocated at addr+16 you'll have to reallocate.
which mean you want to somehow make sure to reserve virtual memory. After all it is supposed to be free...
Aleksey Kladov
@matklad
Nov 02 2017 08:05

yes as long as you have the virtual memory reserved right?

I think even that is not required!

Paul Masurel
@fulmicoton
Nov 02 2017 08:06
How come?
Aleksey Kladov
@matklad
Nov 02 2017 08:06
Like, you need to allocate new contiguous range of virtual addresses
and then copy old pages onto the new ones
Paul Masurel
@fulmicoton
Nov 02 2017 08:06
oh I understand
yes.
I think there is no API to remap a given mapped frame with an anonymous mapping to another page.
Paul Masurel
@fulmicoton
Nov 02 2017 08:11
That's ... weird and sad. If it's file-backed mmap, then that's absolutely feasible.
oh wait it is feasible with syscall I have never seen before :D
i looked at 3 virtual ring buffer implementation... the first two used a temp file, but the third one uses shget & shmat to get that done.
Moggers
@Moggers
Nov 02 2017 17:41
when would you use and_then and or_else over a match for an Option?
Denis Lisov
@tanriol
Nov 02 2017 18:12
Explicit match is three lines, so it's not that difficult to be more readable :-)
Ashley Mannix
@KodrAus
Nov 02 2017 21:15
If you've got a few operations to chain, like traversing a json value, then combinators can keep things terse and easy to follow. But I tend to use them on Result a lot more than Option. If I am using combinators on Option I think it'll usually be part of an if let block