These are chat archives for rust-lang/rust

13th
May 2019
Omer Katz
@thedrow
May 13 09:29
@tanriol No because PyO3 converts Python objects to these types automagically.
@krnik Strange I can't iterate tuples...
How do I convert a slice to an array?
I'm trying to do this:
Omer Katz
@thedrow
May 13 09:34
                let b = b.to_object(py).cast_as::<PyBytes>(py)?.as_bytes();
                let mut a: [u8; 16] = Default::default();
                a.copy_from_slice(&b[0..16]);
                handle = Uuid::from_bytes(a);
I'm getting the following error:
error[E0716]: temporary value dropped while borrowed
  |
  = note: consider using a `let` binding to create a longer lived value
Strange, I had to break it down to multiple lets...
Denis Lisov
@tanriol
May 13 10:14
@thedrow I'd actually write a stupid 5-line match to check them and convert into an enum at the same time.
Omer Katz
@thedrow
May 13 10:15
Got an example?
Denis Lisov
@tanriol
May 13 10:22

Something like the following

let data = match (hex, bytes, bytes_le, fields, int) {
    (Some(hex), None, None, None, None) => Data::Hex(hex),
    (None, Some(bytes), None, None, None) => Data::Bytes(bytes),
    (None, None, Some(bytes_le), None, None) => Data::BytesLe(bytes_le),
    (None, None, None, Some(fields), None) => Data::Fields(fields),
    (None, None, None, None, Some(int)) => Data::Int(int),
    _ => return TypeError::into(
        "exactly one of the hex, bytes, bytes_le, fields, or int arguments must be given",
    ),
}

or maybe even fold some branches into the same variant (decode hex, reverse bytes_le, something like that)

Denis Lisov
@tanriol
May 13 10:30
The basic idea is that, instead of "check parameters and then separately use them", you may want to check the parameters just as you merge them into the final representation used for processing.
Omer Katz
@thedrow
May 13 11:07
Oh this is elegant
Cool
Ingvar Stepanyan
@RReverser
May 13 13:12
Alternatively, you can use series of map + or like:
hex.map(Data::Hex)
.or(bytes.map(Data::Bytes))
.or(bytes_le.map(Data::BytesLe))
.or(fields.map(Data::Fields))
.or(int.map(Data::Int))
.ok_or_else(|| TypeError::into("exactly one of the hex, bytes, bytes_le, fields, or int arguments must be given"))
Ingvar Stepanyan
@RReverser
May 13 14:08
Upd - nvm, I've realised that it doesn't check strict number of params.
Robyn Speer
@rspeer
May 13 15:07
So I'm back to having rand-related questions that are actually basic Rust questions.
I want to be able to control the seeding of a PRNG, as before. I can currently do this with a constant [u8; 16] hard-coded in, as in:
let mut rng = Pcg32::from_seed([0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15]);
How would I turn some dynamic value into a [u8; 16]?
I can make lots of things into a &[u8; 16] but I don't know how to make a [u8; 16] without the reference.
(I guess I meant just &[u8] for that first one)
Ingvar Stepanyan
@RReverser
May 13 15:15
you can use try_from
there is blanket implementation for impl<'a, T> TryFrom<&'a [T]> for &'a [T; 16] which will check the length and give you &[u8; 16] out of &[u8] if it matches
Robyn Speer
@rspeer
May 13 15:21
Oh nice
Thanks
Ichoran
@Ichoran
May 13 16:06
@rspeer - You can also let mut seed: [u8; 16] = [0; 16]; and then manually put the bytes where you want them. (This is rather error-prone, however.)
Denis Lisov
@tanriol
May 13 16:22
@rspeer You may want to use SeedableRnd::seed_from_u64 instead, it's easier to use for non-crypto purposes :-)
Robyn Speer
@rspeer
May 13 19:13
Oh thanks Denis, that's helpful to know about
Siddhant Sanyam
@siddhant3s
May 13 20:53

Hi. I'm wondering if lambdas in rust can be inlined
What I would like to be able to write is :

let v: Vec<u32> = ...;
let sum = v.iter.fold(0, |x, y| x + y);

And have it be compiled to nearly the same code as

let v: Vec<u32> = ...;
let mut sum = 0;
let mut i = v.len();
while i {
  sum = sum + i;
  i = i - 1;
}

Seems like Rust's compiler might be able to (and it probably does) compile the two version to same machine code, but is there any guarantee from the language that they are equivalent? This is a trivial example so that we can discuss the main point: which is the cost of constructing and destructing an Iterator objects. Is there really a cost like such in case of Rust?

matrixbot
@matrixbot
May 13 20:58
bspeice Constructing/destructing iterators is a stack allocation (unlike Java, where it's a heap allocation); they behave very closely to C++.
Denis Lisov
@tanriol
May 13 20:58
There are no language-level guarantees that they will be inlined, but normally they are.
Your second version may (after fixing it to use v[i-1] instead of i) actually be slower.
And they're not strictly equivalent because the iteration order is different, so debug panics on overflow would happen in a different place of the array :-)
matrixbot
@matrixbot
May 13 21:03
bspeice And I don't think there's any way within Rust to forcibly inline closures, but you can force inlining functions through #[inline(always)]
Siddhant Sanyam
@siddhant3s
May 13 21:04
@tanriol , I would love if it was actually slower. But I want to know why so that I can defend the zero cost of abstraction in Rust.
Here is what worries me about the first version if it's anything close to Scala (which is what I've been using until now). What the cost of calling iter()? Seems like it returns an object of type Iter which is a struct with two pointers. What is the cost of creating that closure. What is the cost of calling map on the Iter. All these would be explicit costs in Scala for example (since lambdas are basically objects) and calling methods is not free either.
I understand that these cost would not matter until this piece of code is being called many times but for the sake of this discussion let's talk as if it was being called multiple times.
Denis Lisov
@tanriol
May 13 21:04
IIRC, a closure can also have #[inline(always)]
Siddhant Sanyam
@siddhant3s
May 13 21:04
Also, yes I meant to do sum + v[i]
Denis Lisov
@tanriol
May 13 21:06
v[i] means bounds checking in every iteration unless LLVM is able to elide them, which is... not always :-)
matrixbot
@matrixbot
May 13 21:07
bspeice Denis Lisov (Gitter): Interestingly enough, you can inline closures. Learned something new today: https://godbolt.org/z/4ByyNq
bspeice Siddhant Sanyam (Gitter): Iterators are created as objects, yes, but they don't go to the allocator like Java.
bspeice "Creating" the iterator is free, as the memory needed to hold the pointer is allocated as part of creating the function stack frame.
Siddhant Sanyam
@siddhant3s
May 13 21:16
@tanriol Thanks for your comments. Could you also comment on the cost of calling and creating of the closure and also on the fact if a for x in v would've yielded the same performance (or better) than the first version.
Denis Lisov
@tanriol
May 13 21:17
The cost of creating or calling a closure is roughly zero. In this case there are no captures, so it does not require any data memory and the code gets inlined.
The for loops work with iterators in Rust, so I don't expect a for loop to be faster :-)
Siddhant Sanyam
@siddhant3s
May 13 21:19

In this case there are no captures, so it does not require any data memory and the code gets inlined.

I'm curious as to why would captures ruin inlinability ?

Denis Lisov
@tanriol
May 13 21:20
They do not ruin it, they'd require a bit of stack, but that's still almost zero cost :-)
By the way, I've played with your examples a bit and your second example actually has good chances to be significantly slower :-) because of reverse iteration order LLVM does not autovectorize it, while it does autovectorize the iterator-based one (and a manually written one with normal iteration order).
Siddhant Sanyam
@siddhant3s
May 13 21:22
I'm still not able to understand this. Let me try explaining my confusion. If there are captures, and especially mutable captures, I would assume that an inline procedure would help mitigate the extra stack space needed for these captures. Woudln't it make more sense for the compiler to inline the closure (if they are small enough specially) to avoid having extra stack space?
@tanriol I'm extremely sorry to cause confusion. My intent wasn't to discuss about the affect of traversing in reverse.
I'll edit my example (now that I know that you can edit your chat messages).
Oh well, seems like I can only edit recent messages. (Makes sense)
Denis Lisov
@tanriol
May 13 21:24
I may be thinking too deep... assuming the closure is inlined (and thus optimized together with the function), it's most likely zero cost.
Siddhant Sanyam
@siddhant3s
May 13 21:25
I see. Basically the place where I'm coming from is that I want to use more functional aspects of Rust and them be nearly as good as the iterative versions.
So all this is good news.
Ichoran
@Ichoran
May 13 21:26
@siddhant3s - A lot of the answers boil down to asking about the sophistication of the optimizing heuristics in llvm. The Rust code compiles to something that looks very much like reasonably sophisticated C or some flavor of C++; the cost of that isn't zero but it's equivalent to a zero-cost version which the optimizer can often produce.
Denis Lisov
@tanriol
May 13 21:29
The costs are low enough that avoiding high-level features sounds like premature optimization everywhere or almost everywhere.
Ichoran
@Ichoran
May 13 21:29
The JIT compiler in the JVM also tries to do that, but (1) it has to do it at runtime, which limits the sophistication of the transforms it can apply, and (2) the memory model, safety guarantees at runtime, and possibility of virtual inheritance make it more difficult to actually abstract away everyting
Siddhant Sanyam
@siddhant3s
May 13 21:29
@Ichoran Yeah that makes sense. I wish I knew how to reason about these things more clearly. Like being able to look at the IR which Rust produces. I'm sure there exist tools to do that. I'd search when I'll really need to optimize these things.
Ichoran
@Ichoran
May 13 21:30
And I agree with @tanriol that the costs when not zero tend to be so close to zero as to not care about except in the very most demanding applications (where it's worth checking).
matrixbot
@matrixbot
May 13 21:30
bspeice Siddhant Sanyam (Gitter): Godbolt is typically a great resource for understanding what's going on. Interestingly enough, it was actually developed specifically for checking out differences between iterator patterns.
bspeice *originally developed for
Siddhant Sanyam
@siddhant3s
May 13 21:31
I think what I wasn't happy about in JVM model is that the primary abstraction which both languages (Scala and Java) exposes is Objects. And using many many many objects is costly because of heap allocation and then garbage collection. Specially when dealing with large amount of data and process which runs for days, seeing GC stalls because someone used a recursive functional approach which end up producing many intermediate objects, was painful.
Ichoran
@Ichoran
May 13 21:32
Rust's main weakness, if it has one, is when you necessarily have to allocate large numbers of smallish items of variable lifetime. This plagues C and C++ also (with some strategies to ameliorate it that are harder in Rust, like free lists and memory arenas), and is where the JVM really shines.
But Rust is better than anything at avoiding heap allocation when you don't absolutely need it.
Siddhant Sanyam
@siddhant3s
May 13 21:32
If the primary abstraction which the language exposes hurts you when used extensively, it is a bit unsettling.
Although in all fairness, the GC is pretty sophosticated.
@matrixbot Thanks! I'll check it out. Good to know.
Ichoran
@Ichoran
May 13 21:33
@siddhant3s - Yeah, it's an awkward thing. It's awesome at it, but it takes the awesomeness so much for granted that you can easily overtax it.
Siddhant Sanyam
@siddhant3s
May 13 21:47
@matrixbot Godbolt is awesome!. I tried two versions (one with for-expression and one with a Iterator::for_each and they resulted in the same code after passing -O. Awesome.