These are chat archives for rust-lang/rust

20th
Aug 2017
Maik Klein
@MaikKlein
Aug 20 2017 11:10
Isn't it called placeholder?
Restioson
@Restioson
Aug 20 2017 13:16
You know how thread::spawn has that thing where you can't use variables in scope unless you move?
Is it possible to make that the same for one of my methods
Ideally outright can't use, not even move
Michal 'vorner' Vaner
@vorner
Aug 20 2017 13:51
Well, the „unless you move“ is mandating 'static lifetime on the closure.
Not even move ‒ you can accept just functions, not closures.
but that's probably inconvenient to use
Restioson
@Restioson
Aug 20 2017 13:52
And globals?
Michal 'vorner' Vaner
@vorner
Aug 20 2017 13:56
what about globals?
Restioson
@Restioson
Aug 20 2017 13:56
Can't functions access those?
Essentially what I'm doing is sending bytecode for instructions to another processor (via SPI) so I don't want any of the environment to be allowed to be used
Michal 'vorner' Vaner
@vorner
Aug 20 2017 13:57
Yes, and you can't really prevent that.
Restioson
@Restioson
Aug 20 2017 13:57
rip
Michal 'vorner' Vaner
@vorner
Aug 20 2017 13:58
Uh, what bytecode? If you send bytecode, where do rust functions/closures come into that?
Restioson
@Restioson
Aug 20 2017 13:58
AVR bytecode
it gets flashed to onboard flash, and then is run
Bytecode of the closure
Michal 'vorner' Vaner
@vorner
Aug 20 2017 14:00
I don't really get what you try to do… so you take the closure, cast it to a pointer and try to send that as a byte array, or what?
Restioson
@Restioson
Aug 20 2017 14:00
mhm
Michal 'vorner' Vaner
@vorner
Aug 20 2017 14:00
because I don't think that has any chance of working reliable. Rust is not really byte code.
Restioson
@Restioson
Aug 20 2017 14:01
It is?
Michal 'vorner' Vaner
@vorner
Aug 20 2017 14:01
It is compiled to native.
Restioson
@Restioson
Aug 20 2017 14:01
yep, native bytecode
Michal 'vorner' Vaner
@vorner
Aug 20 2017 14:02
and I guess nobody ever promised that you can just take the code from one process to another, for example because of different process can have the libraries linked at different addresses, etc.
Restioson
@Restioson
Aug 20 2017 14:03
Tbh i don't think it's that complex for a little guy like this https://ktechnics.com/wp-content/uploads/2015/12/atmega328p-pu.png
the same libraries will be linked
@vorner the processors are all the same btw
Michal 'vorner' Vaner
@vorner
Aug 20 2017 14:06
Yes, still I don't think it is a good idea to start with. It might work „by accident“, but in general it seems like something Rust never attempted to allow or guarantee. If you have a python bytecode, for example, it's standardized how that looks and there are methods to inject the bytecode into the interpreter. But there are no such guarantees for Rust.
I'd say what you try to do is beyond the unsafeties of mem::transmute
And, anyway, you can't really make functions not be able to access globals. You probably could send whole libraries with some interface, though.
Restioson
@Restioson
Aug 20 2017 14:08
Oh yeah, this is definitely very unsafe x)
The global thing - I guess I could just warn against it
@vorner I am decently sure that if you compile
fn hi() -> u8 {
    10
}
Michal 'vorner' Vaner
@vorner
Aug 20 2017 14:09
You could put it into the documentation of your „send the closure out“ function and mark it as unsafe.
Restioson
@Restioson
Aug 20 2017 14:09
with the same compiler, for the same architecture, with the same libraries linked, etc, the bytecode will be the same?
tbh literally everything is "unsafe" on this platform
even multiplication and division
although its not marked as such
interfacing with anything requires write_volatile
although I guess this is quite unsafe
Michal 'vorner' Vaner
@vorner
Aug 20 2017 14:11
Well, the unsafe mark on function basically means „you need to be double careful and really read the docs“.
Restioson
@Restioson
Aug 20 2017 14:11
volatile_store*
Michal 'vorner' Vaner
@vorner
Aug 20 2017 14:12
Anyway, even function like the one you show ‒ how do you know how many bytes to send, where the function ends? And what other functions it might call, that are not available on the target? And that half of the function didn't get inlined onto the caller side?
I guess there isn't something like a dynamic linker on that platform? So you could send a stand-alone library and dlopen that (or the equivalent)? Because that approach does guarantee that you have all your functions you call and all the globals you need bundled with it.
Restioson
@Restioson
Aug 20 2017 14:15
It will end at the asm ret symbol
Most of this will be up to the user
Nope, there is no dynamic linking. There's only static
Eventually I will detect calls and send those too, but for now they will be isolated fragments with only access to core
(and whatever you link)
It's designed for quite literally programming another chip with
Michal 'vorner' Vaner
@vorner
Aug 20 2017 14:17
Hmm, I wonder if you could actually make something like dynamic linking. If you send a whole .o file, that one also has some kind of defined interface and properties.
And you probably need to detect jmp as well as call.
Restioson
@Restioson
Aug 20 2017 14:18
?
I don't think the rust compiler for this would use jmp for calling?
Michal 'vorner' Vaner
@vorner
Aug 20 2017 14:19
I don't think there's any promise that a function will live in one continuous block of memory. And you probably don't want to know what everything the compiler does during its optimisation phase. I don't say it does it, I just say it might.
Restioson
@Restioson
Aug 20 2017 14:19
True
Michal 'vorner' Vaner
@vorner
Aug 20 2017 14:19
And, generally, static strings (for example) live somewhere else than code.
Restioson
@Restioson
Aug 20 2017 14:19
I am talking this over with the developer of the avr backend though
they seem to think its ok?
where do they live?
Michal 'vorner' Vaner
@vorner
Aug 20 2017 14:21
At least in linux, there are different sections of the executable. Some of them have read only executable code, some of them are read only but not executable (its where the strings live), some of them have read-write data, some of them contain debug symbols. Maybe AVR just inlines them, because there's no memory protection to begin with, I guess.
But I can also imagine if you have the same string constant at multiple places that it gets to the executable just once and is referenced.
I mean, it might be OK in some cases, but the general approach feels like asking for trouble in general. Maybe I'm just too scared of these things and too old to go for that kind of thing, though. Some people tell me that I code too defensively sometimes.
Restioson
@Restioson
Aug 20 2017 14:24
In AVR your program lives in flash, too
oh yeah, it's definitely very unsafe x)
my fallback option is to just compile separate routines beforehand
Michal 'vorner' Vaner
@vorner
Aug 20 2017 14:40
I still keep coming up with more and more challenges. Basically, you need to be able to relocate a code that was not build to be relocable to begin with ‒ because, in the target memory it will have to live in different location (the original one might be already taken). Furthermore, the source and target programs will have the core library at different address and probably called just by address, not by name. Just imagine the program in target never used Result::unwrap but the source one did. So the core in the source program has one more function in it, so everything after that is in a different location.

And, you can even have multiple ret instructions in the function, eg the equivalent of this:

fn is_prime(n: prime) -> bool {
  for i in 2..n {
    if n % i == 0 {
      return false;
    }
  }
  true
}

If you stop sending at the first ret, you miss the other half of the function.

And all this seems to be unsolvable without full understanding of the AVR code and its actual meaning.
But I was wondering, could you manage to hook into the rust compiler somehow and let it compile just the fragment for you? Because rustc already understands the semantics and LLVM knows the AVR code.
Michal 'vorner' Vaner
@vorner
Aug 20 2017 14:48
Hmm. It might be a real fun to find where the function ends in case LLVM decides to use a jump to a computed address.
Restioson
@Restioson
Aug 20 2017 14:50
@vorner but yeah, this is essentially a "god I really hope this works please please please don't blow up in my face" kinda solution
I might inject some kind of symbol (if thats possible?)
Still, please do tell me the flaws/scenarios where it doesn't work!
Michal 'vorner' Vaner
@vorner
Aug 20 2017 14:51
I think you won't be compiling on that chip itself. So you probably could use debug info (if it's possible to generate) to carve the function out. Something like use the syntax crate to map line numbers to functions and the debug info to map instructions to lines in the code.
Restioson
@Restioson
Aug 20 2017 14:54
https://i.imgur.com/p4RYnyb.png currently have this
Michal 'vorner' Vaner
@vorner
Aug 20 2017 14:56
Right, probably add full understanding of the byte code ‒ you need to distinguish between what is an instruction and what is just an operand, or you get confused by things like let x: u8 = OPCODE_OF_RET;.
And you need to predict where a jump on a computed address will land, because you need to DFS through the whole function (all paths must lead to either an infinite cycle, invalid instruction or a RET)
But I think the problem is in making the payload, not loading it (at least, not as big problem) and you need to compile on a „real“ machine anyway, so you probably can use any kind of heavy tools for that (like the debug symbols, or asking LLVM to do something special)
Do you actually need to augment the code on the chip, or is it OK to completely reload the whole program?
Restioson
@Restioson
Aug 20 2017 15:07
Wdym?
Oh, overwrite it?
I think there's a way you can protect a certain place in flash from being overwirtten
essentially the chips will be running custom bootloaders
I'll look at how arduino's Optiboot does it
Michal 'vorner' Vaner
@vorner
Aug 20 2017 15:18
No, I don't mean overwriting it by accident. I mean like if you could lower your needs a bit and instead of sending messages „Please, plug in this new function, but otherwise keep the old code“ you could say „hey, reboot to this completely new program, but keep the state“ or something. Because the later seems much easier to do.
Restioson
@Restioson
Aug 20 2017 15:30
hm, true
Flashing is slow though, so I am probably going to do it ahead of time, too
I don't see how that's any easier?
Michal 'vorner' Vaner
@vorner
Aug 20 2017 15:34
Because, presumably, rustc can already produce a whole working program, but extracting a function and all its deps and splicing it into another program is much harder.
Restioson
@Restioson
Aug 20 2017 15:34
Oh, you dont need to keep the state either
Actually a way simpler option
I flash the entire program to the chip
the same one as on main
but with a slightly differing main method?
i.e instead of actual main code, just wait for instructions?
that way each chip can be used for any routine...
Michal 'vorner' Vaner
@vorner
Aug 20 2017 15:36
Yes, then you already have all the functions in there, so you just need a way to tell it which one to run.
Restioson
@Restioson
Aug 20 2017 15:37
This has the advantage that it can be done at compile time -- no runtime flashing needed...
CC @dylanmckay
Michal 'vorner' Vaner
@vorner
Aug 20 2017 15:38
(and instead of different main, just have some kind of boot parameter/jumper to read/one byte on a concrete address)
Restioson
@Restioson
Aug 20 2017 15:38
And it'd be a whole lot easier and safer
Could do that
have the bootloader take a boolean for isMain
which is tied to a specific address
then the code would be
fn main() {
    if (volatile_load(IS_MAIN) == 0xFF) {
        volatile_store(IS_MAIN, 0x00);
        main_routine();
    }
    else {
        slave_loop();
    }
}
At boot of the controller (acts as a multiplexer between slaves and main chip) it could flash the program
Restioson
@Restioson
Aug 20 2017 15:43
and, as bonus, if its already flashed nothing happens
For bonus bonus bonus points, the actual program could just be stored on an sd card in hex format, so you dont have to include!() the code (into PROGMEM) of the controller...
maybe with two copies, one as main, one as slave?
Restioson
@Restioson
Aug 20 2017 15:48
like #if defs in c
one with main true, one false
I'd still have to do the no statics thing
or locals
Restioson
@Restioson
Aug 20 2017 15:53
I guess I could do it with functions though?
Michal 'vorner' Vaner
@vorner
Aug 20 2017 15:56
Well, you wouldn't need to send function pointers. You could send messages (eg. an enum with data). Or is it inflexible, because of fixed set of options in it? If it was inside the same program, then you wouldn't need to add them dynamically.
Restioson
@Restioson
Aug 20 2017 15:58
Was thinking of a macro to define a routine
It could (somehow, I don't know) add to a global list of name: function pointer, and then define the function
sry, define first, then add to global map/dict
Restioson
@Restioson
Aug 20 2017 16:06
I'd rather have it less explicit, but enum could work
Michal 'vorner' Vaner
@vorner
Aug 20 2017 16:39
How often do the events like rustfest happen? I would like to visit one some day (preferably sooner than later), but I'm unable to make it to the one in Zurich 30.9. :-(
Restioson
@Restioson
Aug 20 2017 17:12
Rip, there's probably never going to be one near me :(
Michal 'vorner' Vaner
@vorner
Aug 20 2017 17:14
Do you live in some remote part of the world? I guess living in Europe does have it advantages.
Restioson
@Restioson
Aug 20 2017 17:14
South Africa
Michal 'vorner' Vaner
@vorner
Aug 20 2017 17:15
Yes, I see, that's a bit far away from both Europe and USA, which is the places I know had some of these conferences :-|
Restioson
@Restioson
Aug 20 2017 17:15
:l
Michal 'vorner' Vaner
@vorner
Aug 20 2017 17:16
And I was thinking 10 hours long drive was long
Restioson
@Restioson
Aug 20 2017 17:16
10 hour flight, haha
14h*
up to 16h
Wow, even up to 18h
Depends how many stops, and where
Flying isn't as bad as driving though
Douglas Campos
@qmx
Aug 20 2017 19:36
at least for the drive you can make it a road trip, go slow, visit small cities on the way, etc