These are chat archives for rust-lang/rust

9th
Nov 2018
Ichoran @Ichoran takes notes.
Ichoran
@Ichoran
Nov 09 2018 00:08
Hopefully I have a more accurate idea of what impl Trait means now; that was an interesting problem.
Denis Lisov
@tanriol
Nov 09 2018 00:12
impl Trait in all cases means "some type that implements Trait (and that's the only thing we know about it)"
Sackery
@sackery
Nov 09 2018 00:18
vec<i32> to vec<i64> how to in handy?
Denis Lisov
@tanriol
Nov 09 2018 00:21
Something like my_vec.iter().map(|&v| v as i64).collect()
Sackery
@sackery
Nov 09 2018 00:21
@tanriol into_iter?
Denis Lisov
@tanriol
Nov 09 2018 00:22
Depends on whether you want to keep the original vector.
Sackery
@sackery
Nov 09 2018 00:23
ok
can use transmute?
Ichoran
@Ichoran
Nov 09 2018 00:24
@tanriol - The key thing about impl Trait which I hadn't adequately appreciated is that it's existential instead of universal.
Unlike dyn Trait which is universal.
Denis Lisov
@tanriol
Nov 09 2018 00:25
@sackery First, no, you can't, these two have different representations. Second, don't use unsafe unless you really need to - you'd most likely screw up and won't have any serious wins anyway :-)
Sackery
@sackery
Nov 09 2018 00:26
@tanriol get it~
Denis Lisov
@tanriol
Nov 09 2018 00:27
@Ichoran Not sure about the terminology, but people often mention that impl Trait is existential as a return type, but universal as an argument type.
Chuck Musser
@cmusser
Nov 09 2018 00:27
@tanriol and @lchoran, that was an interesting problem for sure.
Ichoran
@Ichoran
Nov 09 2018 00:30
@tanriol - Yeah.
aohan237
@aohan237
Nov 09 2018 02:30
if unwrap used too much ,how to deal with the panic?
red75prime
@red75prime
Nov 09 2018 03:21

@Ichoran > They look the same to me. This compiles:

impl Trait in function arguments means that function is impicitly generic and accepts any type that implements Trait. impl Trait in return position means that function returns some concrete type that implement Trait.

I was wrong about the need to box incoming argument. fn foo(in: impl Trait) -> impl Trait {in} is implicitly generic and so it can return in as is, without boxing. Anyway, I doubt that allowing impl Trait in input position was a good idea.

aohan237
@aohan237
Nov 09 2018 03:51
@aohan237
how to combine futures ? say you have a db.query futures and only when you get the result ,you then start a http request future
use and_then?
Ichoran
@Ichoran
Nov 09 2018 04:56
@red75prime - Thanks, yeah, I figured that out after @tanriol explained what was going on.
aohan237
@aohan237
Nov 09 2018 08:34
how to do with chain futures?
melbourne2991
@melbourne2991
Nov 09 2018 09:06
I am trying to store an iterator on my struct but having some difficulties with typing it - below is my attempt but I am still getting expected trait std::ops::FnMut, found closure - struggling to find examples of how to tackle this
pub struct Lexer<'a, R> {
  lines: FlatMap<R, String, (FnMut(Result<String, Error>) -> Chars<'a>)>,
}

impl<'a, R: BufRead> Lexer<'a, R> {
  pub fn new(reader: R) -> Lexer<'a, R> {
    let lexer = Lexer {
      lines: reader.lines().flat_map(|line: Result<String, Error>| -> Chars<'a> { line.expect("Fail").chars()}),
    };
    lexer
  }
}
Dmitriy
@dpogretskiy
Nov 09 2018 09:12
pub struct Lexer<'a, T, R: Iterator<Item = T>> { lines: R }
something like this
you can't type Fn's since they are all different types
Tim Robinson
@1tgr
Nov 09 2018 09:14
There’s fn, for functions that don’t close over anything in their environment https://doc.rust-lang.org/std/primitive.fn.html
lines: R is the way to go here though
melbourne2991
@melbourne2991
Nov 09 2018 09:16
Ah sorry, the field in that struct is poorly named - it should really be chars
I want to store an iterator which iterates over the chars
melbourne2991
@melbourne2991
Nov 09 2018 09:21
With @dpogretskiy 's suggestion doesn't that imply the iterator is passed in...
Tim Robinson
@1tgr
Nov 09 2018 09:23
Sure, it’s passed in by the Lexer itself in fn new
(There’s also Box<Iterator> if you’re stuck and don’t mind the extra allocation)
Dmitriy
@dpogretskiy
Nov 09 2018 09:25
tho whole thing won't work, since you BufRead will be dropped along with String's you read from it after you leave new function
Tim Robinson
@1tgr
Nov 09 2018 09:28
Seems ok? The iterator returned by BufRead::lines owns reader
Dmitriy
@dpogretskiy
Nov 09 2018 09:29
use std::io::{BufRead, Error};
use std::str::Chars;

pub struct Lexer<T, R: Iterator<Item = T>> {
    read: Box<BufRead>,
    pub lines: R,
}

impl<AT1, AT2> Lexer<AT1, AT2> {
    pub fn new<T: BufRead>(reader: Box<T>) -> Lexer<char, impl Iterator<Item = char>> {
        let lexer = Lexer {
            lines: reader
                .lines()
                .flat_map(|line: Result<String, Error>| -> Chars<'a> {
                    line.expect("Fail").chars()
                }),
            read: reader,
        };
        lexer
    }
}
that's where i got so far
'a is non existent in this case
so gotta find something out
maybe i'm wrong tho, it doesn't compile yet
Tim Robinson
@1tgr
Nov 09 2018 09:32
Doesn’t need read: Box<BufRead>, and indeed read: reader won’t work as reader is moved on the line above
melbourne2991
@melbourne2991
Nov 09 2018 09:34
Just so you know guys I'm paying attention to the back and forth here, and appreciate it - that said I don't have much to contribute as this is literally my first day of using Rust. Comforting to see it's not a super straight forward problem though
Dmitriy
@dpogretskiy
Nov 09 2018 09:35
I'm just in rush, no time to finish, went towards something working, but didn't have enough time
melbourne2991
@melbourne2991
Nov 09 2018 09:35
Cool, i'll try pick up where you left it :D
Dmitriy
@dpogretskiy
Nov 09 2018 09:35
that's somewhere around what you want, not close enough :)
melbourne2991
@melbourne2991
Nov 09 2018 09:36
cheers!
Dmitriy
@dpogretskiy
Nov 09 2018 09:36
BR: BufRead instead of Box
and so on
Tim Robinson
@1tgr
Nov 09 2018 09:36
I think it needs fn into_chars(String) -> Chars, which is not something that exists
Something that owns the String while the Chars is being iterated over
Iterating over the bytes would be easier
melbourne2991
@melbourne2991
Nov 09 2018 09:38
Yeah and I was tempted to just iterate over bytes which would probably be fine for what i'm looking to do - but when I run into these kind of issues I get hung up on finding the solution, particularly if it's a new language i'm learning
Tim Robinson
@1tgr
Nov 09 2018 09:40
Iterating over the chars of a string is complicated, as it’s giving you individual code points
fn main() {
    assert_eq!(1, "😀".chars().count());
    assert_eq!(4, "😀".bytes().len());
}
melbourne2991
@melbourne2991
Nov 09 2018 09:42
Yep, and I probably only need to accept ascii which is 1 byte per char I believe
but was curious if I could solve it anyhow
Your code snippet above is giving me:
no method named `expect` found for type `std::result::Result<std::string::String, dyn std::error::Error>` in the current scope

note: the method `expect` exists but the following trait bounds were not satisfied:
      `dyn std::error::Error : std::marker::Sized`
Tim Robinson
@1tgr
Nov 09 2018 09:44
Should be referring to io::Error which is Sized
melbourne2991
@melbourne2991
Nov 09 2018 09:44
Ohh... I see
Tim Robinson
@1tgr
Nov 09 2018 09:44
io::Error is what BufRead returns in the event of a problem
melbourne2991
@melbourne2991
Nov 09 2018 09:45
Awesome thanks
I was tackling this same error before... good to know
Tim Robinson
@1tgr
Nov 09 2018 09:45
You can use error::Error provided you box it: Box<error::Error> is Sized whereas plain error::Error is not
I tend to use Box<error::Error> as a general purpose error type, as the Error types in the other modules (io::Error and others) can be automatically converted to Box<error::Error>
melbourne2991
@melbourne2991
Nov 09 2018 09:49
Ok i'll keep that in mind
Dmitriy
@dpogretskiy
Nov 09 2018 10:27
@melbourne2991 so i'm back, and best i could get to is this
use std::io::BufRead;

pub struct Lexer<R: Iterator<Item = char>> {
    pub lines: R,
}

impl<AT2: Iterator<Item = char>> Lexer<AT2> {
    pub fn new<T: BufRead>(reader: T) -> Lexer<impl Iterator<Item = char>> {
        let lexer = Lexer {
            lines: reader
                .lines()
                .map(|line| line.expect("Fail"))
                .flat_map(|l| l.chars()),
        };
        lexer
    }
}
melbourne2991
@melbourne2991
Nov 09 2018 10:29
Oh cool that looks very similar to what I converged on based on Tim's version:
pub struct Lexer<R: Iterator<Item = char>> {
  chars: R,
}

impl<R: Iterator<Item = char>> Lexer<R> {
  pub fn new<T: BufRead>(reader: T) -> Lexer<impl Iterator<Item = char>> {
    let lexer = Lexer {
      chars: reader
        .lines()
        .flat_map(|line: Result<String, Error>| line.expect("Fail").chars()),
    };
    lexer
  }
}
Dmitriy
@dpogretskiy
Nov 09 2018 10:29
there is an error in my version
which is basically line of type String does not live long enough
chars() is heavy duty stuff
melbourne2991
@melbourne2991
Nov 09 2018 10:30
my ide isn't complaining but haven't tried a build yet... yours looks identitcal to mine I think so I may run into the same issue
yeah I might just iterate over bytes..
Dmitriy
@dpogretskiy
Nov 09 2018 10:31
nah it won't work either
the thing is you need to iterate over chars in place
melbourne2991
@melbourne2991
Nov 09 2018 10:32
mm I'm a little surprised that iterating over characters in a string seems to be so non-trivial
Dmitriy
@dpogretskiy
Nov 09 2018 10:32
so if you have concrete idea about what you wanna do with those chars you should probably do it right there
it's just there is no GC in rust
you can't flat_map endlessly, every closure with drop all the items it owns upon completion
in this case it's a String
melbourne2991
@melbourne2991
Nov 09 2018 10:33
Yeah i'm coming from higher level languages so i'm not so familiar with these kind of challenges
Dmitriy
@dpogretskiy
Nov 09 2018 10:33
so then you have iterator that points nowhere
Tim Robinson
@1tgr
Nov 09 2018 10:34
Many languages represent strings as Unicode code points, so getting characters is easy, and getting bytes is hard
melbourne2991
@melbourne2991
Nov 09 2018 10:34
Ah, and it rust it's the opposite
That's interesting
Tim Robinson
@1tgr
Nov 09 2018 10:35
I don’t think I’ve ever needed to get the Chars from a string in Rust, but I’m constantly making use of the fact that &str is really the same as &[u8]
Probably because I’m doing a lot of byte-level parsing and I don’t have any emoji or even £ symbols in my data
Dmitriy
@dpogretskiy
Nov 09 2018 10:37
the problem are not chars, but just intermediate data that gets dropped all over the place
melbourne2991
@melbourne2991
Nov 09 2018 10:37
So if I was just using a loop and doing everything in that loop it would be fine?
Dmitriy
@dpogretskiy
Nov 09 2018 10:37
yup
chars() function is a bit weird
it'd suggest you use it right where you have an idea of what to do with those chars
Roman Proskuryakov
@kpp
Nov 09 2018 10:39

I’m constantly making use of the fact that &str is really the same as &[u8]

Really?

Tim Robinson
@1tgr
Nov 09 2018 10:39
@dpogretskiy yeah, it seems like there should be String::into_chars the same as you have String::into_bytes
Dmitriy
@dpogretskiy
Nov 09 2018 10:40
String is chars itself :)
melbourne2991
@melbourne2991
Nov 09 2018 10:40
@dpogretskiy Yeah for my purposes I can do that just have to architect it a bit differently to what I had in mind
Roman Proskuryakov
@kpp
Nov 09 2018 10:40
That is true only if there is ASCII inside String or &str
Tim Robinson
@1tgr
Nov 09 2018 10:40
@kpp yep, for writing parsers it’s great
Denis Lisov
@tanriol
Nov 09 2018 10:42
@melbourne2991 How does the high-level API look like? Do you expect to store a Lexer somewhere and then access it from different areas of your code?
Tim Robinson
@1tgr
Nov 09 2018 10:42
The &[u8] returned by str::as_bytes gives you the UTF-8 data, which for ASCII is one byte per char
Roman Proskuryakov
@kpp
Nov 09 2018 10:43
That is true only if there is ASCII inside String or &str
melbourne2991
@melbourne2991
Nov 09 2018 10:43
@tanriol my original plan was for lexer to implement the iterator trait and pull tokens from it in the parser with next()
It doesn't need to be an iterator though
This is mainly a toy project to learn some rust so I wanted to experiment with some of the features
Tim Robinson
@1tgr
Nov 09 2018 10:46
You could implement Iterator yourself instead of going via flat_map
Dmitriy
@dpogretskiy
Nov 09 2018 10:46
yeah, that's a better option
Tim Robinson
@1tgr
Nov 09 2018 10:46
That way you can deal with the ownership, chars etc explicitly in your code while still exposing Iterator::next
melbourne2991
@melbourne2991
Nov 09 2018 10:46
What would the implementation look like though, roughly
Those lines replaced some code based on filter_map
If you imagine writing your code as a for loop then you’d use local variables to keep track of your state
Iterators are the same except the next method is like the body of your for loop, and you use struct fields to keep track of your state not local variables
melbourne2991
@melbourne2991
Nov 09 2018 10:51
Yep but I still need to pull char by char (or if we assume ascii then byte by byte) from a bufread
It looks straightforward enough to read in the whole file and then do what I like with it
Tim Robinson
@1tgr
Nov 09 2018 10:55
This stuff is a lot easier when you have a String containing the whole file, yes
melbourne2991
@melbourne2991
Nov 09 2018 10:55
Yeah ok
Or if I wanted to do it incrementally I guess I create a fixed sized buffer then read, clear, read clear
Dmitriy
@dpogretskiy
Nov 09 2018 10:57
reading from a file incrementally might not be the best idea tho
Tim Robinson
@1tgr
Nov 09 2018 10:58
Yes you can read blocks incrementally from the file, but you’ve still got to keep track of your state between one block and the next
If you’re reading say 10MB+ files then it could be worth it
melbourne2991
@melbourne2991
Nov 09 2018 10:58
Yeah for reading files it's probably not necessary, not expecting any insanely large files
Yeah, which is unlikely tbh
Tim Robinson
@1tgr
Nov 09 2018 11:00
It’s probably even faster to do it in one go due to hand wavy explanation involving caching
melbourne2991
@melbourne2991
Nov 09 2018 11:01
:D
I think my original thinking was, I wanted to lexer not to assume anything about the source of the data
Dmitriy
@dpogretskiy
Nov 09 2018 11:04
fn br<BR: BufRead>(reader: BR) -> String {
    let mut acc = String::with_capacity(500);
    for l in reader.lines() {
        match l {
            Ok(stringy) => acc.extend(stringy.chars()),
            Err(_io_error) => panic!("oh noes."),
        }
    }
    acc
}
reading file into string is pretty easy, not sure if lines() -> chars() will contain \n's
Tim Robinson
@1tgr
Nov 09 2018 11:05
From memory I think lines gives you the \n at the end of every String
Dmitriy
@dpogretskiy
Nov 09 2018 11:05
then you can chars() and iterate all you want on THE string
:)
melbourne2991
@melbourne2991
Nov 09 2018 11:06
Yeah I might have to go with that
Tim Robinson
@1tgr
Nov 09 2018 11:06
Yes get somebody else to give you Iterator<Item=char> and go from there
melbourne2991
@melbourne2991
Nov 09 2018 11:06
hm yeah good idea.
That makes a lot more sense really xD
Dmitriy
@dpogretskiy
Nov 09 2018 11:07
@1tgr worth trying out anyways, adding \n is not that hard
@melbourne2991 so the problem with original code sample was that you get your string somewhere in the middle of the iteration and nobody wants to hold it
but in this case you do pretty much same thing
it's just you take small strings and push them into big one
thing about rust is that it's equally useful to know what you can't do along with what you can
then it all kinda makes sense
Tim Robinson
@1tgr
Nov 09 2018 11:12
When I debug stuff like this, the question I keep asking myself is, “who owns this value”?
If you own it that’s cool
If all you have is a reference (like the case of Chars) then who owns the data behind that reference is important
Dmitriy
@dpogretskiy
Nov 09 2018 11:13
more then that, i don't thing there is a way to return Charsfrom anywhere really, unless you given String by reference with a lifetime
i would just stick to using Chars in place since it gets complex real fast
melbourne2991
@melbourne2991
Nov 09 2018 11:15
I think I will ditch chars altogether and stick with bytes
I know i'm parsing ascii anyhow
Dmitriy
@dpogretskiy
Nov 09 2018 11:46
@melbourne2991 then don't use lines
since String is always utf-8
read into arrays or Vecs
btw if you feel like shooting legs you can do something like that
use std::io::BufRead;
use std::str::Chars;

struct BRIter<BR> {
    br: BR,
    current_string: String,
    current_chars: Chars<'static>,
}

impl<R: BufRead> BRIter<std::io::Lines<R>> {
    fn new(reader: R) -> Self {
        let new_string = String::new();
        let chars = unsafe { std::mem::transmute::<Chars<'_>, Chars<'static>>(new_string.chars()) };
        Self {
            br: reader.lines(),
            current_string: new_string,
            current_chars: chars,
        }
    }
}

impl<BR: Iterator<Item = Result<String, std::io::Error>>> Iterator for BRIter<BR> {
    type Item = char;
    fn next(&mut self) -> Option<char> {
        match self.current_chars.next() {
            Some(c) => Some(c),
            None => {
                match self.br.next() {
                    Some(Ok(s)) => {
                        self.current_string = s;
                    }
                    Some(Err(_io_error)) => panic!("oops"),
                    None => return None,
                };
                unsafe {
                    self.current_chars = std::mem::transmute::<Chars<'_>, Chars<'static>>(
                        self.current_string.chars(),
                    );
                }
                self.next()
            }
        }
    }
}
not saying it works
didn't try
but it compiles with a bit of unsafe magic
don't do this at home
use std::env;
use std::fs::File;
use std::io::prelude::*;

fn main() {
    // --snip--
    println!("In file {}", filename);

    let mut f = File::open(filename).expect("file not found");

    let mut contents = String::new();
    f.read_to_string(&mut contents)
        .expect("something went wrong reading the file");

    println!("With text:\n{}", contents);
}
this is from rust book btw
probably it's more efficient then what i suggested
Dmitriy
@dpogretskiy
Nov 09 2018 11:54
use std::io;

fn main() {
    let stdin = io::stdin();
    let index_of_some_char = BRIter::new(stdin.lock());

    for c in index_of_some_char {
        println!("{}", c);
    }
}
unsafe thing actually works, but don't use it :)
Tim Robinson
@1tgr
Nov 09 2018 11:55
haha
'static is only needed because a reference can’t refer to a field declared in the same struct
Denis Lisov
@tanriol
Nov 09 2018 11:57
How about using some library for writing parsers instead of doing everything by hand?
Dmitriy
@dpogretskiy
Nov 09 2018 11:57
parsers are fun
Tim Robinson
@1tgr
Nov 09 2018 11:58
Yeah, why let somebody else have all the fun?
melbourne2991
@melbourne2991
Nov 09 2018 12:37
@tanriol yeah I've used parser generators in the past in langs i'm familiar with and had a look at [nom] which I might come back to later,
(https://github.com/Geal/nom) but given i'm just starting with rust it had too many new concepts for me to grok all at once. Plus this is just a toy project i'm doing to learn the language.
This is what i've ended up in the end:
pub struct Lexer<R> {
  chars: R,
}

impl<R: Iterator<Item = Result<u8, Error>>> Lexer<R> {
  pub fn new(it: R) -> Lexer<R> {
    let lexer = Lexer { chars: it };

    lexer
  }

  pub fn to_token(&mut self, b: u8) -> Option<Token> {
    match b as char {
      '(' => Some(Token::LeftParen),
      _ => Some(Token::Whitespace),
    }
  }
}

impl<R: Iterator<Item = Result<u8, Error>>> Iterator for Lexer<R> {
  type Item = Token;

  fn next(&mut self) -> Option<Token> {
    let val = self.chars.next();

    match val {
      Some(val) => match val {
        Ok(val) => self.to_token(val),
        Err(_e) => panic!("Problem reading file"),
      },
      _ => None,
    }
  }
}
And calling it like this:
    let f = File::open(filename).expect("file not found!");
    let mut fbytes = f.bytes();

    let lexer = lexer::Lexer::new(fbytes.by_ref());

    for token in lexer {
        println!("{:?}", token);
    }
Denis Lisov
@tanriol
Nov 09 2018 12:43
Erm... so you're not using a buffering reader, are you?
melbourne2991
@melbourne2991
Nov 09 2018 12:44
No
But the lexer leaves that option available
it's not coupled to a particular reader
Denis Lisov
@tanriol
Nov 09 2018 12:46
Well, if being ASCII-only is fine for your use case, this implementation does make sense.
melbourne2991
@melbourne2991
Nov 09 2018 12:48
Yeah I mean, I was curious to see if I could get the non ascii use case working - or at least figure out how I would get it to work. But I didn't need it and given the headaches involved I'll wait until it's something I actually do need to figure it out
Denis Lisov
@tanriol
Nov 09 2018 13:02
Did you want to also keep the ability to parse without reading everything into memory at once?
melbourne2991
@melbourne2991
Nov 09 2018 13:07
Mm yes and no, I want that to be an option. But I don't see an immediate need for it given i'm just reading in files that I expect are not going to be enormous in size. I think if the lexer takes an iterator as opposed to say a string - then that leaves the option on the table - accepting an iterator keeps the data source separate (whether it's a string or incrementally pulling values out of a buffer).
Also, f.bytes returns an iterator (not sure what the underlying implementation is) but it could well be streaming the data in
I don't know wether it reads the whole file in at once or not
Denis Lisov
@tanriol
Nov 09 2018 13:10
There used to be an unstable Read::chars method, which was deprecated in 1.27 with motivation like "if one needs decoding to be incremental, they probably also need to manually control buffering anyway"
I'm afraid that in your code the bytes iterator will read every single byte separately...
melbourne2991
@melbourne2991
Nov 09 2018 13:13
Yeah I did see some thread about it being a performance issue:
rust-lang/rust#28073
So I guess I could just read in the whole file and pass in an iterator that iterates over the string
Or handle the buffering manually but probably overkill for now