These are chat archives for rust-lang/rust

26th
Mar 2019
Brian Knapp
@knappador
Mar 26 03:04
https://crates.io/crates/e-nguyen Working on a music visualization library / program. Joiners welcome. Could really use Windows & OSX audio. I can probably get around to doing Android myself.
Iulian Rotaru
@mortimr
Mar 26 05:18

Hi everyone, I started using Rust for a week now and I am trully struggling with a library that takes a reference as argument, and stores a reference from it in its return type. How would I be able to return this value ? Will my best option be to completely convert this value into a self contained variable that do not rely on any other reference ? The lifetime of the return type is bound to the lifetime of my input and I just can't understand why it has been built this way ? It's the pest library, its parser has a parse method taking an & std::string::String and returns a structure of this type

#[derive(Clone)]
pub struct Pairs<'i, R> {
    queue: Rc<Vec<QueueableToken<R>>>,
    input: &'i str,
    start: usize,
    end: usize,
}

I never saw any example of code where the result is used outside of the function calling the parse.

I guess that this input field lifetime is bound to the lifetime of my input ?
Denis Lisov
@tanriol
Mar 26 10:03
You need to either derive the reference you pass to it from your arguments (so that its lifetime is bound by them), or make it self contained.
Iulian Rotaru
@mortimr
Mar 26 14:45
What do you mean by “make it self contained” ?
Iulian Rotaru
@mortimr
Mar 26 14:56
It's a manual process depending on the specific case ? Or is there a way to bind this return and my input together somehow ?
Denis Lisov
@tanriol
Mar 26 14:57
Could you show what you're trying to do?
Iulian Rotaru
@mortimr
Mar 26 15:54

Ok
So I'm using the pest parser, which provides an attribute to create a parser instance from a grammar. This parser instance has only one method called fn parse(rule: R, input: &str) -> Result<Pairs<R>, Error<R>>. The returned Pairs is the following structure

pub struct Pairs<'i, R> {
    queue: Rc<Vec<QueueableToken<R>>>,
    input: &'i str,
    start: usize,
    end: usize,
}

where every node keeps a reference on my input.
What I'm trying to do is make a function that would take a path as argument, read the content of the file, parse the content and return the path, content and ast in a structure

pub struct File<'a> {
    path: std::string::String,
    source: std::string::String,
    ast: pest::iterators::Pairs<'a, Rule>
}

This is what I'm doing for the moment, and I cannot get this ast out with the content inside my File structure.

#[derive(Parser)]
#[grammar = "lib/parser/grammar.pest"]
struct CircuitParser;

pub fn parse_source(sources: & std::string::String, maybe_rule: Option<Rule>) -> std::result::Result<pest::iterators::Pairs<Rule>, pest::error::Error<Rule>> {
    if let Some(rule) = maybe_rule {
        CircuitParser::parse(rule, sources)
    } else {
        CircuitParser::parse(Rule::Circuit, sources)
    }
}

pub fn parse_file(path: & str) -> File {

    let source = std::fs::read_to_string(&path).expect("Unable to read file");

    let ast = parse_source(&source, None).expect("Error while parsing");

    File {
        path: path.to_string(),
        source,
        ast
    }

}

This is what I get

error[E0597]: `source` does not live long enough
  --> src/lib/parser/ast.rs:35:29
   |
35 |     let ast = parse_source(&source, None).expect("Error while parsing");
   |                             ^^^^^^ borrowed value does not live long enough
...
43 | }
   | - borrowed value only lives until here
   |
note: borrowed value must be valid for the anonymous lifetime #1 defined on the function body at 31:1...
  --> src/lib/parser/ast.rs:31:1
   |
31 | / pub fn parse_file(path: & str) -> File {
32 | |
33 | |     let source = std::fs::read_to_string(&path).expect("Unable to read file");
34 | |
...  |
42 | |
43 | | }
   | |_^

And if I understand correctly, the return value is bound to the lifetime of the borrowing , and as it dies at the end of the parse_file scope, it gives me this error ? I'm sure there is an easy answer but the problem seems so hard to explain in a succint manner that I haven't found a proper solution online

Ichoran
@Ichoran
Mar 26 16:00
The AST is full of references to your string. You need to preserve the string for as long as you preserve the AST. So, for instance, you need to keep it in your struct.
Denis Lisov
@tanriol
Mar 26 16:00
There's no easy and obvious way out.
Ichoran
@Ichoran
Mar 26 16:01
Or you need to rebuild the AST into a different structure that no longer refers to the original string, and return that structure (probably an AST of your own devising) instead.
Iulian Rotaru
@mortimr
Mar 26 16:01
But the source is stored with the ast in the returned File structure
That's the final option, I want to be sure I'm not missing some Rust feature that would allow me to have this ast + the source together in the struct before trying to reformat all the ast.
Denis Lisov
@tanriol
Mar 26 16:01
The "keep it in your struct" option does exist, but requires support for self-referential structs like rental provides.
Ichoran
@Ichoran
Mar 26 16:03
Oh, right. I should try things out before commenting. My lifetime intuition isn't sufficient yet :P
(And I should read the code more carefully, sorry--I somehow skipped the source line and only saw the path.)
Iulian Rotaru
@mortimr
Mar 26 16:04
Hmm, this would prevent errors of the type but data from 'y' flows into 'x' here to happen ? I tried moving a mutable File and assign the ast on the fly, after building it from the source that is already stored in it and got an error of this type.
Denis Lisov
@tanriol
Mar 26 16:05
It can (probably) be done with rental. It cannot be done without libraries and without unsafe.
The other option is to refactor your API because... are you sure the Pest's AST is what you want to keep for a long time?
Iulian Rotaru
@mortimr
Mar 26 16:09
I'm using it to parse a language where include statements might occur, and I need to store the AST/file to prevent loading the same file multiple times and also detect cyclic includes
Iulian Rotaru
@mortimr
Mar 26 17:01

rental did the trick

rental! {

    mod rental_ready {
        use super::*;

        #[rental(debug)]
        pub struct File {

            path: std::string::String,
            source: std::string::String,
            ast: pest::iterators::Pairs<'source, Rule>

        }

    }

}

And this is how the File is built now

let file = rental_ready::File::new(
        path.to_string(),
        | path | {
            std::fs::read_to_string(&path).expect("Unable to read file")
        },
        | source, path | {
            CircuitParser::parse(Rule::Circuit, source).expect("Parsing error")
        }
    );

Thanks @tanriol and @Ichoran for your answers !

kylegoetz
@kylegoetz
Mar 26 17:23

I have a function with the header

pub fn add_entry<T: AsRef<str>>(&self, checksum: T, path: T) -> Option<bool>

and some code calling it:

db.add_entry(cs, img.get_path().to_str().unwrap())

I get the compiler error mismatched types: expected struct std::string::String, found &str

Why am I getting this when add_entry takes anything that implements AsRef<str> and to_str().unwrap() should be &str?

Denis Lisov
@tanriol
Mar 26 17:24
What's the type of cs?
kylegoetz
@kylegoetz
Mar 26 17:25
String
Denis Lisov
@tanriol
Mar 26 17:26
Your signature requires both arguments to be of the same type T, but they're not (one is String, the other one is &str)
kylegoetz
@kylegoetz
Mar 26 17:27

Oh, interesting. So if I didn't want to impose this, would I have to redefine add_entry as

pub fn add_entry<S, T: AsRef<str>>(&self, checksum: S, path: T) -> Option<bool>

(I guess I'd have to double check the syntax to see exactly how you have two type symbols of the same type

Denis Lisov
@tanriol
Mar 26 17:28
<S: AsRef<str>, T: AsRef<str>>
kylegoetz
@kylegoetz
Mar 26 17:28
Gotcha. Coolio. Thanks. Still a noob to Rust, at least I'm not getting borrow/own errors constantly anymore :)
thojest
@thojest
Mar 26 20:01
hey guys I could need your advice
concerning error handling :) if I have a custom error which is an enum but I want to assign custom messages at runtime e.g.
enum MyError {
ParseError(String)
}
1) do I have to use Result<(), Box<dyn Error>> in the signature of a function? i.e. boxing the Error
2) is this considered bad style?