These are chat archives for rust-lang/rust

9th
Mar 2019
Sam Johnson
@sam0x17
Mar 09 01:02
@tanriol thanks for the advice -- interestingly I did not have the lifetime issue you mentioned -- I was able both to return an arbitrary new slice and return a modified version of the old slice without it complaining
currently struggling to add a lifetime parameter to this so it will compile :(
trait BaseTrait {
    fn target(&mut self) -> &mut BaseTrait;
}

struct Implementer {
    target: &mut BaseTrait
}

impl BaseTrait for Implementer {
    fn target(&mut self) -> &mut BaseTrait {
        self.target
    }
}
Riley Cat
@WreckedAvent
Mar 09 02:23
whenever you're fighting the lifetime or borrow system a lot it's generally a sign you're doing something unidiomatic
in this case you have a struct with a reference but there's no way to know how long that reference is supposed to live
you could force it to compile by adding a lifetime parameter to implementer and then the implementation would define it in terms of 'static' or something but
I would suggest more changing the design
the easiest way would be by using some kind of smart pointer that keeps track of that lifetime for you:
trait BaseTrait {
    fn target(self) -> Box<BaseTrait>;
}

struct Implementer {
    target: Box<BaseTrait>
}

impl BaseTrait for Implementer {
    fn target(self) -> Box<BaseTrait> {
        self.target
    }
}
but note the ownership change from &self to self
Riley Cat
@WreckedAvent
Mar 09 02:33
if you want to keep it &self or similar you can do something like this:
trait BaseTrait {
    fn target(&self) -> &BaseTrait;
}

struct Implementer {
    target: Box<BaseTrait>
}

impl BaseTrait for Implementer {
    fn target(&self) -> &BaseTrait {
        &*self.target
    }
}
&* looks a little strange but it just means to remove the thing from our smarter pointer and create a dumb pointer and since this now follows that rule I talked about earlier with auto-inserting then the lifetime is clearly tied to &self
(more generally though traits are meant for behavior and not fields)
Riley Cat
@WreckedAvent
Mar 09 02:43
so it's hard to suggest a more idiomatic way to write it without slightly more details so that the trait encapsulates only the behavior and not the data with it
but that's how you can get that to compile anyway
Riley Cat
@WreckedAvent
Mar 09 02:48
^ @sam0x17
Sam Johnson
@sam0x17
Mar 09 03:46
@WreckedAvent thanks a bunch -- I will try with box and see if that works in other cases as that sounds like what I need
Riley Cat
@WreckedAvent
Mar 09 03:47
you generally want to use the pointer types like Box when something "owns" that reference
when I adjusted my thinking to understanding that &T is closer to a slice, or a temporary view of T, it made me fight with the borrower checker a lot less
with that thinking it doesn't really make sense to have a temporary view of something in a struct, because it's temporary and might be gone right after the object was created
but it does make perfect sense for functions, since you know that temporary view is still valid for at least the run time of the function
Sam Johnson
@sam0x17
Mar 09 03:52
yeah this is a situation where one stream feeds into the next, so instance A needs a reference to instance B , and instance B needs a reference to instance C etc
and I'm going to have to modify it so it's an Option instead, which is going to be painful, but I wanted to fix the lifetimes first before fighting with that
what is the difference though between having a method that is &self and self? I didn't know plain old self was a thing in that context so I have no idea
Riley Cat
@WreckedAvent
Mar 09 04:02
depending on the type involved it can be cheaper to just copy self instead of make a (still cheap) dumb reference to it
but there's also semantic meaning
unless the type is copy then self without the reference indicates you are moving the ownership into the method
this is typically what you see with into methods that "consume" self
(i.e you can't use it after the function is called)
Sam Johnson
@sam0x17
Mar 09 04:05
ahh, ok that makes sense
if you were making a linked list, what type of reference would you use for node links ... Box?
Riley Cat
@WreckedAvent
Mar 09 04:08
why don't you look at the reference material for box and see :slight_smile:
but yes Box is good for most recursive data structures
Sam Johnson
@sam0x17
Mar 09 04:29
ok great thanks
Riley Cat
@WreckedAvent
Mar 09 04:47
as for "one stream feeds into the next", you might want to look into combinators
// the trait definition for parse is straight forward
// because we've defined this as a trait, basically any type
// can define some way to parse things if appropriate
trait Parser<I, O> {

  /// Performs an incremental parse
  fn parse(&self, input: I) -> O;
}

// but this is really just a wrapper around calling a function
// so let's make it easy for functions to become parsers

// we provide a general implementation
impl<I, O, F> Parser<I, O> for F

  // specifically for functions
  where F: Fn(I) -> O {

  // which just defines parse for us
  fn parse(&self, input: I) -> O {
    self(input)
  }
}

// the type signature here is really gnarly but that's mostly to do with generics
fn combine<P1, P2, I1, O1, O2>

  // combine takes two parsers and returns a closure that takes i1 and returns o2
  (p1: P1, p2: P2) -> impl Fn(I1) -> O2

  // and both parsers need to have compatible signatures so we can compose them
  where P1: Parser<I1, O1>,
        P2: Parser<O1, O2>, // notice o1 goes into p2

{
  // and our implementation is straight-forward
  move |input| {
    let o1 = p1.parse(input);
    p2.parse(o1)
  }
}

// some dummy parsers
fn p1(input: isize) -> bool { input >= 1 }
fn p2(input: bool) -> isize { if input { 100 } else { 0 }}

/// and our test suite
/// ``
/// use bach::test;
///
/// assert_eq!(test::main(1), 100);
/// assert_eq!(test::main(0), 0);
/// ``
pub fn main(input: isize) -> isize {
  let p3 = combine(p1, p2);

  p3(input)
}
rust's type system is strong enough that we can have type-safe combinators of pretty much any type which is pretty neat
Riley Cat
@WreckedAvent
Mar 09 04:53
you can go one step further and make combine return a parser or know how to piece together results or options but that's the general idea
(and the type signature was crazy enough)
Sam Johnson
@sam0x17
Mar 09 05:20
that's crazy cool, but yeah the stream infrastructure is a lot more complicated than that in this case -- but thanks!!
Riley Cat
@WreckedAvent
Mar 09 05:22
:+1:
Sam Johnson
@sam0x17
Mar 09 06:48
@WreckedAvent would there be a way to do this with a mutable reference though?
trait Stream {
    fn target(&self) -> &Stream;
}

struct PassthroughStream {
    target: Box<Stream>
}

impl Stream for PassthroughStream {
    fn target(&self) -> &Stream {
        &*self.target
    }
}
(BaseTrait => Stream, Implementer => PassthroughStream)
Riley Cat
@WreckedAvent
Mar 09 06:51
afaik if you just add mut everywhere it'll compile
as in
trait Stream {
    fn target(&mut self) -> &mut Stream;
}

struct PassthroughStream {
    target: Box<Stream>
}

impl Stream for PassthroughStream {
    fn target(&mut self) -> &mut Stream {
        &mut *self.target
    }
}
Sam Johnson
@sam0x17
Mar 09 06:52
oh, my bad, I missed one and it gives crazy weird errors
thanks!!
Riley Cat
@WreckedAvent
Mar 09 06:54
I would try to limit how often you have mutable references but yeah should compile :+1:
Sam Johnson
@sam0x17
Mar 09 06:54
why doesnt the target: Box<Stream> line need a mut?
Riley Cat
@WreckedAvent
Mar 09 06:56
that's a pretty good question but I assume it has something to do with ergonomics
in that rust isn't a purely immutable language and structs are still inherently mutable but you just need the right ownership/reference to mutate them
Sam Johnson
@sam0x17
Mar 09 06:58
gotcha, yeah I thought I had to do target: Box<&mut Stream> which is where I was going wrong -- didn't understand whether its a reference just by saing Box<SomeType> or if you have to do Box<&SomeType>
*saying
Riley Cat
@WreckedAvent
Mar 09 06:58
Box is a pointer type
so you don't want a reference in it
Sam Johnson
@sam0x17
Mar 09 06:58
gotcha, that was my main mistake
Riley Cat
@WreckedAvent
Mar 09 06:58
and pointer types typically "own" what's inside so you don't need mut since they own it
Sam Johnson
@sam0x17
Mar 09 06:58
ahhhhh kk
Denis Lisov
@tanriol
Mar 09 09:29
Are the streams you're working on something like Iterator or the futures streams? If not, what's the key difference?
Sam Johnson
@sam0x17
Mar 09 15:46
@tanriol key difference is there are triggers that fire when the data reaches a certain number of bytes, so you end up with a construction like this:
pub trait Stream {
    fn limit(&mut self) -> &mut usize;
    fn remaining_bytes(&mut self) -> &mut usize;
    fn total_bytes_in(&mut self) -> &mut usize;
    fn total_bytes_out(&mut self) -> &mut usize;
    fn target(&mut self) -> &mut Stream;
    fn on_limit(&mut self) {}
    fn transform<'a>(&self, data: &'a [u8]) -> &'a [u8];

    fn send_input_data(&mut self, data: &[u8]) {
        if data.len() == 0 { return; }
        *self.total_bytes_in() += data.len();
        self.send_output_data(self.transform(data));
    }

    fn send_output_data(&mut self, data: &[u8]) {
        if data.len() == 0 { return; }
        *self.total_bytes_out() += data.len();
        self.target().write(data);
    }

    fn write(&mut self, data: &[u8]) {
        let mut cursor = data;
        loop {
            if *self.limit() > 0 && cursor.len() >= *self.remaining_bytes() {
                let deferred_data = &cursor[*self.remaining_bytes()..cursor.len()];
                let remaining_bytes = *self.remaining_bytes();
                self.send_input_data(&cursor[0..remaining_bytes]);
                *self.remaining_bytes() = *self.limit();
                self.on_limit();
                if deferred_data.len() == 0 { break; }
                cursor = deferred_data;
            } else {
                self.send_input_data(cursor);
                *self.remaining_bytes() -= cursor.len();
                break;
            }
        }
    }
}
Sam Johnson
@sam0x17
Mar 09 15:55
(once I get things with targets sorted out, this will also have pipe / unpipe and that sort of thing --- implementers of this just have to define the transform method and the methods at the top which are all just returns... if Rust had better abstract class support it would just be defining one method but alas no)
Denis Lisov
@tanriol
Mar 09 15:55
This looks more like a type than a trait, especially the getters. Why does it have to be a trait?
Sam Johnson
@sam0x17
Mar 09 15:56
it probably doesnt -- I have no idea what I"m doing
everything that is simple in languages with OOP support is complicated AF in rust lol
in "normal" languages, Stream would be an abstract class with a default implementation for transform that can be overridden
so what is a Type, then? I thought there were just traits and structs?
Denis Lisov
@tanriol
Mar 09 16:00
What are you actually trying to do? I can make some proof-of-concept of "how I would do a limiter", but better understanding leads to better implementation :-)
Sam Johnson
@sam0x17
Mar 09 16:16
so I'm porting over code that I have working perfectly in crystal, D, and javascript -- basically a Stream base class, that allows me to create subclasses for encryption/decryption/compression/decompression/stats counting/file splitting/file joining -- everything has to be chainable (stream1.pipe(stream2).pipe(stream3).pipe(stream4)), and ideally it uses closures for on_data and on_limit however borrowing rules have made that pretty much impossible --- in other languages I am able to do things like stream1.on_data { |data| <-- do stuff here --> };
Zakarum
@omni-viral
Mar 09 16:16

everything that is simple in languages with OOP support is complicated AF in rust lol

This is usually what you get trying do somthing dangerous, unsafe or wrong, that OOP language happily allows you

Sam Johnson
@sam0x17
Mar 09 16:18
I mean the public interface is super simple -- there should be a way to get it to work in Rust
Zakarum
@omni-viral
Mar 09 16:18

stream1.on_data { |data| <-- do stuff here --> };

Try stream1.on_data(move |data| <-- do stuff here --> );

Denis Lisov
@tanriol
Mar 09 16:18
In Rust pipelines like this work better if built in compile time with generics, not in runtime.
Sam Johnson
@sam0x17
Mar 09 16:19
@omni-viral the problem I usually run into isn't with the data variable but with trying to mutate the stream or other objects in the closure
because typically in the on_limit or on_data callback you want external side effects
Zakarum
@omni-viral
Mar 09 16:20
Then callback should receive mutable reference to that object
Not capture it
Sam Johnson
@sam0x17
Mar 09 16:20
been there, you end up with all these lifetimes and RefCell's and its a mess
Zakarum
@omni-viral
Mar 09 16:20
RefCell should not be used lightly
95% you want RefCell you basically doing something wrong
Denis Lisov
@tanriol
Mar 09 16:21
What kind of changes do you usually want in runtime after you've first built and started the pipeline?
Zakarum
@omni-viral
Mar 09 16:21
Rust doesn't allow you to share objects mutably. But usually you actually don't need to
Sam Johnson
@sam0x17
Mar 09 16:23
@tanriol could be anything -- updating something in the GUI, changing the limit on the current stream, doing some IO stuff, updating external data structures -- etc
Denis Lisov
@tanriol
Mar 09 16:26
...but usually not changing the pipeline itself (removing stages, inserting ones, etc.)?
Sam Johnson
@sam0x17
Mar 09 16:27
typically not
actually that's not true, sometimes you have to unpipe and pipe in the middle of streaming a file, even
with my use case
that's why this interface is so handy
Denis Lisov
@tanriol
Mar 09 16:29
And we're talking about streaming, so you almost certainly want integration with the futures/async ecosystem, correct?
Sam Johnson
@sam0x17
Mar 09 16:30
eventually yeah
but I'm willing to do it all synchronously if it's too much of a pain
here is a working D implementation -- there are 53 specs that cover every aspect of it that I don't include but I assure you it's correct:
https://gist.github.com/sam0x17/561de58e69dbe979ba4b1c5042b6c153
it's very compact
in the crystal implementation, I'm also able to stream to/from any IO directly thanks to abstract classes, but I don't expect rust to let me do that
Denis Lisov
@tanriol
Mar 09 16:37
There's one significant impedance mismatch between your interface and the ones normally used in Rust. Your code is push-based (writes data into the next stage), while the de-facto standard interfaces in Rust are pull-based (poll the previous stage for data).
However, your code can still be translated much more idiomatically that the trait above using a generic wrapper.
Sam Johnson
@sam0x17
Mar 09 16:43
ok so what should I research to learn how to do that?
I am not grasping how generics would factor into this :(
and yeah, pull-based is the standard in every language, but in classic OOP languages it's usually easy to write a wrapper that inverts that
AFAIK
Denis Lisov
@tanriol
Mar 09 16:49
I've heard that push-based logic becomes lots of fun when you try to integrate it with async, but haven't tried personally.
Sam Johnson
@sam0x17
Mar 09 16:50
yes, this is all inspired by node.js's streams interface, which I have seen nothing like in any other language
it was also a pain to work with eventually for different reasons, but those are solved by this whole "limiting" concept I develop
in node.js, it was impossible to pause the stream and resume based on certain amounts of data flowing through, and you have no way of knowing, if you have a chain of streams piping into each other, whether your X bytes of data that just went in have fully cleared / passed through which streams in the chain at any point in time (especially when you are doing things that change the size of the data, like compression/encryption)
so it became this awkward system of using a coordinating class to figure out ok X bytes went into compression, came out size Y, so when we see Y bytes come through the next stream, that is actually our file, etc
you can also use special headers to separate data, but that is a nightmare as well - - crystal's channels make that really easy though but that's the only language that has been easy in
I've done this in Java, C++, D, Crystal, and JavaScript at this point XD
Sam Johnson
@sam0x17
Mar 09 16:57
javascript was bad because of performance and callback hell (and I fucking hate promises and think they are the most unintuitive looking thing ever invented, so not doing that), crystal is bad because windows support and parallelism not done and I need those for my use case, D is bad because it's dying and all the packages haven't been updated in 5 years (otherwise was perfect and lowest friction of all things tried), Java was bad because Java, and so far Rust has been bad because even after 4 months of tweaking I can't get the basic streams interface working whereas I learned D in a day and had it working the next day :/
Riley Cat
@WreckedAvent
Mar 09 16:58
you hate promises? they single handedly ended call back hell
Sam Johnson
@sam0x17
Mar 09 16:58
and replaced it with something I literally can't understand when I look at it lol
Denis Lisov
@tanriol
Mar 09 16:59
I'll try to translate your D into Rust, but I'd really suggest that you may want to try the futures::Stream.
Sam Johnson
@sam0x17
Mar 09 16:59
omg thank you so much and yes I'll take a look at that
Riley Cat
@WreckedAvent
Mar 09 17:03
well, a future is another name for a promise, and if they dislike promises...
Denis Lisov
@tanriol
Mar 09 17:03
No, it's not the same.
Sam Johnson
@sam0x17
Mar 09 17:14
yeah to clarify, I can work with promises, but I dislike them enough to prefer callback hell, so take that for what it's worth XD
I always name each callback, which makes things more manageable
Martijn Bakker
@Gladdy
Mar 09 17:16
speaking of promises vs callback hell, does anyone know whether there has been any exploration of the difference in performance between the state machine generated by async/await and 0.1-style futures? I wasn't able to find anything.
Denis Lisov
@tanriol
Mar 09 17:19
@sam0x17 Sorry, additional clarification: do you expect any callback to be able to, for example, send headers on some different object?
Sam Johnson
@sam0x17
Mar 09 17:23
the send_input_data / send_output_data stuff is used internally by that stream and I'm ok with those not being callable externally -- send_output_data (_on_data in the D implementation) is basically a way to send some headers to the next stream, skipping the limit interface of this stream
so with compression/decompression that is necessary
and its also used as a convenience method when writing the implementation for write
basically used for anything that is triggered off of a certain amount of data, so compression headers, http headers, footers in some exotic cases I'm sure, or some situation where like between each file you need some piece of data to separate them ... that sort of thing uses send_output_data
also if it wasn't clear, total_bytes_in and total_bytes_out are tracked separately because in many cases 20 bytes in could mean 15 bytes out or vice versa
Sam Johnson
@sam0x17
Mar 09 17:28
depending on what the stream is doing (how transform is defined)
  • transform is defined
Riley Cat
@WreckedAvent
Mar 09 17:47
is there semantic difference between a future and a promise in rust? they refer to the same concept of monadic single-threaded async, usually done with some form of state machine or polling
@tanriol
(I haven't been keeping up with all the async/await hooplah)
Denis Lisov
@tanriol
Mar 09 17:48
The callback hell/promise discussion was in JavaScript context, not Rust, and futures are definitely not JS promises :-)
And yes, async/await will change the situation.
Sam Johnson
@sam0x17
Mar 09 18:16
@tanriol here are some excerpts from the D language unit tests that show some typical usage:
unittest { // header/footer sending
  DuplicatorStream stream1 = new DuplicatorStream();
  ubyte[] output;
  stream1.setLimit(5);
  stream1.dataCallback = delegate void(const ubyte[] data) { output ~= data; };
  stream1.limitCallback = delegate void() {
    stream1.sendOutputData([0, 0, 1, 1]); // [0, 0, 1, 1] is data footer
  };
  stream1.write([0, 1, 2, 3, 4]);
  assert(output == [0, 1, 2, 3, 4, 0, 1, 2, 3, 4, 0, 0, 1, 1]);
  assert(stream1.getTotalBytesIn() == 5);
  assert(stream1.getTotalBytesOut() == 14);
}
it is also very typical to call stream.setLimit from within a limit callback
DuplicatorStream (as its name implies) will duplicate any writes that are sent to it, so .write([0, 1, 2]) is the equivalent of doing .write([0, 1, 2, 0, 1, 2]) on a standard PassthroughStream -- I just use it for testing purposes as it is a basic example of a stream where input size != output size
Denis Lisov
@tanriol
Mar 09 18:25
Okay, the basics seem to work
Sam Johnson
@sam0x17
Mar 09 18:26
aha, I never could have come up with this lol thank you SOO MUCH
very cool, a lot of elements end up being very similar to my original implementation, only it seems like you got around the problems I ran into
like I had limit_callback and data_callback defined exactly like that at one point
Denis Lisov
@tanriol
Mar 09 18:29
Quite possible that trait Transform should be actually just an FnMut(Bytes) -> io::Result<Bytes>
Sam Johnson
@sam0x17
Mar 09 18:29
I see
well glad to see that closures do work with this
I'm a bit closure happy as I come from a ruby background
(if that wasn't already painfully obvious)
well anyway I really really appreciate this I'll reach out of I have any more questions but definitely going to include a special thanks in the final project to you @tanriol
Denis Lisov
@tanriol
Mar 09 18:33
The fine point is that, IIUC, the main purpose of the limiting subsystem is to control buffering and avoid problems with infinite buffering...
...while the pull-based model does that automagically because the pipeline won't poll for new data until everything that could be done to the data already received is done.
Sam Johnson
@sam0x17
Mar 09 18:35
yeah, in normal circumstances pull-based is easier hence everyone using it
I have all this working asynchronously in crystal using channels, which were delightfully push-based as well
(and with fibers)
channels in particular seem like something that would be awesome to have in rust because they would abstract away a lot of ownership problems imo
Denis Lisov
@tanriol
Mar 09 18:38
What are the features that rust channels are missing?
Sam Johnson
@sam0x17
Mar 09 18:38
oh I didn't know rust had them, that answers my other question
lol
looking at the spec, I would say that in crystal they aren't explicitly cross-thread -- you can use them for basic data movement as well within the same thread
rust's channels seem to be only for cross-thread stuff (I could be wrong though)
in crystal they are like "oh you need data piping of some kind, here are some channels for you"
Denis Lisov
@tanriol
Mar 09 18:45
Rust's channels can work across threads or in a single thread with no problems.
I'm afraid one thing you may be missing in Rust is a select on channels.
Sam Johnson
@sam0x17
Mar 09 18:48
oh? what would that do?
in crystal the entire API is basically channel = Channel.new(DataType).... received_data = channel.receive and you can do this across Fibers etc
so its entirely object based
Denis Lisov
@tanriol
Mar 09 18:49
Some people coming from Go are missing the ability to wait on multiple channels at the same time in the same thread.
Sam Johnson
@sam0x17
Mar 09 18:50
ah yeah I can see that being a pain
as long as you can peek you can sort of cheat with that, but it's awkward
like a busy loop that peaks each of the n channels
*peek
Sam Johnson
@sam0x17
Mar 09 19:39
is Bytes a create, @tanriol ?
*crate
or maybe I need to upgrade my rust
Denis Lisov
@tanriol
Mar 09 19:40
Crate, bytes
Sam Johnson
@sam0x17
Mar 09 19:40
thx
Sam Johnson
@sam0x17
Mar 09 19:49
@tanriol I actually love how you separated the Limiter functionality out from everything else
Denis Lisov
@tanriol
Mar 09 19:59
Well, that was the purpose :-) looking at it once again, I'm not sure why the limiter functionality is required at every node at all... maybe it could just be a separate node on its own?
Sam Johnson
@sam0x17
Mar 09 20:04
it could be, but in my use case you need to limit practically everything, so I prefer it like this
also the limiter itself makes testing a lot easier
kylegoetz
@kylegoetz
Mar 09 20:18
I have a file, scannable_file.rs with pub trait ScannableFile { fn is_valid(...) -> bool } in it. I have a file in the same directory called scannable_video.rs. How do I use that trait in the scannable_video.rs file? I've read the Rust book online but they only have one example using separate files but it doesn't help.
My understanding is that I don't need to put any sort of pub mod { ... } wrapper around the trait because scannable_file.rs will act as a module by default. But when I try to import that module in scannable_video.rs I get messages that indicate the compiler is looking for a folder called scannable_video containing a file called scannable_file. But they're in the same path, /src. I've tried use crate::scannable_file but it tells me there's no scannable_file in the root. But in src/, which is the root, scannable_file.rs is right there.
mod scannable_file tells me file not found for module 'scannable_file'.use scannable_file` tells me there's no scannable_file module in the root.
kylegoetz
@kylegoetz
Mar 09 20:24

The actual output when I put mod scannable_file is this: src/scannable_file.rs
file not found for modulescannable_file``

So the compiler appears to be looking for src/scannable_file.rs, which exists, but it doesn't see it.

Denis Lisov
@tanriol
Mar 09 20:25
What mod statements do you have in your lib.rs or main.rs?
kylegoetz
@kylegoetz
Mar 09 20:26
mod scannable_image;
mod scannable_video;
Both are eventually going to have structs that impl ScannableFile, whcih is in scannable_file.rs
Denis Lisov
@tanriol
Mar 09 20:27
Add mod scannable_file; there and use crate::scannable_file::ScannableFile; in the scannable_video module (file)
kylegoetz
@kylegoetz
Mar 09 20:27
once I can get them to successfully import scannable_file::ScannableFile
why does my main have to import something it doesn't use?
Denis Lisov
@tanriol
Mar 09 20:29
use statements are just references in your project or outside it, they don't actually import modules. You really want to import scannable_file in just one place and with your current module structure it should be main
kylegoetz
@kylegoetz
Mar 09 20:30
So if I were to have a bunch of modules in separate files, they all need to be referenced in both the entrypoint, which never actually uses it, on top of the files that actually do use it?
Denis Lisov
@tanriol
Mar 09 20:33
They all need to be included with mod in their parent module. That may be the main.rs file, or you may introduce some different structure.
For example, you may do something like
// src/main.rs
mod scannable_file;
// src/scannable_file.rs
mod scannable_video;
mod scannable_image;
pub trait ScannableFile { /* ... */ }
// src/scannable_file/scannable_image.rs
use super::ScannableFile;
// src/scannable_file/scannable_video.rs
use super::ScannableFile;
kylegoetz
@kylegoetz
Mar 09 20:36
OK thanks. I am going to look into submitting PRs for the Rust docs to add an example of three files with a -> b -> c references that shows a must still reference c, because that wasn't clear from the section on modules unless I missed something. Thank you very much!
Denis Lisov
@tanriol
Mar 09 20:49
The point you may be missing here is that there are two independent parts in this process. use statements work with modules, not files, so you need to make a module for the file first. The mod statement makes that module and normally you don't want to have more than one for the same file because then these will be different modules made for the same file.
kylegoetz
@kylegoetz
Mar 09 22:22

I don't have two modules in the same file. And the Rust book/docs says that you don't need to put anything about mod {} because the file itself operates as a mod. So I have one trait in a file and want a second file to use the trait. I don't know Rust terminology yet, but think of one as an abstract class and the other is a class that implements it. The thing that confuses me is why the main file has to import the class and also the abstract class even though main never uses the abstract class. It only uses the implementation.

It's still hard for me to talk about this because I come from Scala, Python, JavaScript. Scala doesn't require you to import both abstract class and implementation, just the implementation, the concept of an abstract class doesn't exist in Python (but if you wrote an A that extends B, you'd only have to import A in your main, not A and B), and I think the idea of an abstract class in JS is a new feature that isn't even standardized yet IIRC. So apologies if I'm not using the terms correctly even within the bounds of OOP.

Main uses A, and A implements B. I am struggling to understand why main has to reference B and A even though it only needs A. Apparently this is the case, but I don't understand why it has to be that way. But the code works, so I can understand the why another day. :)
actually now that I think about it, JS has compositional inheritance and it works like Scala and Python, not like Rust.
Denis Lisov
@tanriol
Mar 09 22:25
The question is much simpler that you're making it sound :-)
Somewhere in your project there is a trait ScannableFile that you want to use. Just like a file on a computer is located in some folder, the trait has to be located in some module. There has to be some path that you can use to refer to it.
It can be crate::ScannableFile (if it lives in main.rs), or crate::scannable_file::ScannableFile, or even crate::scannable_image::scannable_file::ScannableFile.
But it cannot be both crate::scannable_image::scannable_file::ScannableFile and crate::scannable_video::scannable_file::ScannableFile at the same time because there's only one canonical path to it.
Depending on what name you prefer, you have to put the corresponding mod statements.
Denis Lisov
@tanriol
Mar 09 22:31
The use statements create shortcuts, but to create a shortcut you need to create the file first.
Riley Cat
@WreckedAvent
Mar 09 23:45

@kylegoetz it's actually pretty simple. compilation has to work somehow, so in the top level module (either in lib.rs or bin.rs) you mod in the files that you want compiled. If a.rs depends on b.rs but b.rs is being loaded in by the main, you don't need the mod in a to bring it in for compilation.

and then whence a file has been included for compilation, it'll have a name relative to its module. that can get long/annoying, so we have use to create short paths in scope.

you can have c.rs be loaded in for compilation by another module that is not bin/lib (like a.rs) but it needs to be in an according folder.
but the top level does not necessarily need to import c.rs just because a.rs or b.rs has it as a dependency