People
Activity
    paulmeyer
    @paulmeyer

    My current problem is tough to debug. If I use bump allocator, I can read the root directory (assignment 2, phase 3b). However, if I use my 'bin' allocator, I get a hang. Tracked it down to this line of code:

                kprintln!(" bb ");
                let    a = 0 as *mut u8;
                kprintln!(" bbb {:?}",a);
                kprintln!(" bbb {:?}",(*block).next);
                (*block).next  = a;

    which prints

     bb 
     bbb 0x0
     bbb

    'block' here is a local struct representing a bin block. My first hang was creating a 'std::ptr::null_mut::<u8>', so I replaced that with casting zero as an experiment. No panic. I may need to install aarch64 exception handlers to see if I'm triggering an invalid op or something.

    Olav Junker Kjær
    @esx
    If you get a hang it must be due to an "unsafe" operation, right? I had a bug where I only allocated say 2 bytes if the size was 2. But since the pointer written in the released blocks is isize it would overwrite the released block. So it needs to allocate at least 8 bytes even if you only need one byte. The pointer write happens in an unsafe block in LinkedList.
    paulmeyer
    @paulmeyer
    This is in an unsafe block (see the 0 as *mut u8) where I am setting up my 'binblock'. My next step is to start printing out everything and/or dumping the contents of memory to figure out where I derailed.
    paulmeyer
    @paulmeyer

    Pretty basic: my "bin allocator" was creating blocks based on minimum required alloc size, aligned as required, and then padding it with room for my header. That passed all the "alloc" unit tests just fine. However, on the Pi it keels over quickly because I can have cases where my pointers are not 8 byte aligned. First deref of an unaligned pointer will kick in an aarch64 unaligned fault, kicking us to the nonexistant fault handler. Death.

    Easy enough to fix! Onward.

    paulmeyer
    @paulmeyer
    Extension of the same thing: I was creating ebpb and mbr structs that simply overlaid the raw data structure from the disk. The raw structure has u16 and u32 values that are not aligned to their own size. This causes issues in some cases when those fields are used (hangs printing ebpb.bbytes_per_sector, etc.). Going back to the constructor to have a 'raw' struct, and a 'safe' struct, with the safe struct being delivered back.
    Gregory Katz
    @gregkatz
    @paulmeyer I definitely hit that alignment issue on the raw data structures. I ended up using byteorder to read in the u16s and u32s as u8 arrays
    paulmeyer
    @paulmeyer

    For people working on Filesystem stuff:
    Anyplace where you create a struct (MBR, EBPB, etc) by casting it from an array of bytes, realize that those bytes can/will have any alignment. Example: bytes_per_sector in EBPB is a u16 with offset of 11 bytes from beginning of struct. Giving that struct member type u16 seemed to work fine on x86 (passed all tests), but fails hard on aarch64 (kprintf!("{}",ebpb.bytes_per_sector hangs, likely heading off to the ARM abort handler with an alignment fault).

    You could have a "raw" version of the struct, and copy over to a 'safe' version (no #reprc) (similar to how ATAGs was handled), but that is kind of a pain, and will fail the unit tests (which check the exact size of the EBPB/MBR structs). What I did: everything in those structs is either a single u8 or an array of u8s. No alignment required. All fields are 'private', and any field that is used outside has a 'getter' function that does (a[0] as u16 + (a[1] as u16)<<8) (or u32 equivalent) to turn the data into the expected type. Works, relatively simple, only needs to be implemented for the fields you actually use.

    You should be watching for this with any struct that has #[repr(C, packed)].

    RichRandall
    @RichRandall
    Wow, just found this group! I've just come back to looking at this stuff and I'm stuck on assignment-1 bootloader. My ttywrite and xmodem implementations pass all the tests, but I'm getting BrokenPipe "bad transmit" on sending. The bootloader itself just continues to get timeouts on receive (revealed by LED debugging!). I'm at a bit of a loss at the moment
    paulmeyer
    @paulmeyer
    We'll figure it out. I'll have to dig back into my code. Are you getting the 'broken pipe' error on the host side? Mac, linux, or windows? I assume shell worked fine?
    BTW, assume the bug is in the xmodem code. The tests don't really test it effectively as far as I can tell.
    RichRandall
    @RichRandall
    Yeah, ttywrite is chucking out BrokenPipe. OK, good to know where to concentrate my efforts
    paulmeyer
    @paulmeyer

    One thing that really helped me, I created a new version of 'transmit_with_progress' called 'transmit_with_debug'. It took an extra argument: a function that took a fn(&str, int), but otherwise it looked like the progressfn. I could then send debug messages with a number back to the ttywrite program to print out. On the Pi I did the same thing with receive_with_debug(), and I had 8 LEDs wired up on the Pi so I could send byte values and byte messages out on the pi side. Enough to let me know how far I was getting in the code before it died.

    the basic xxx_with_progress() function just called the xxx_with_debug() function, using a noop debug function.

    At least for me, if I don't have some equivalent to 'printf debugging' deep in the code, I'm lost.

    RichRandall
    @RichRandall
    Thanks paul, that sounds like a promising strategy. I'll give it a go tomorrow and let you know how I get on.
    RichRandall
    @RichRandall
    Apologies, just realised I didn't answer all your questions! I'm communicating with my pi from a linux machine, and the shell was working great.
    Olav Junker Kjær
    @esx
    @paulmeyer : Thanks for this find, but does this mean reading integers from a "C" struct is inherently unsafe?
    paulmeyer
    @paulmeyer
    @esx I'm not positive here, but that seems to be the case. It sure seemed to work on x86 (or none of the tests stressed it). However, on the Pi, whenever I tried to do any deref of a pointer that was unaligned, or kprintln("{}",a.x);, where x was a u16 or u32 that was unaligned, I got a silent fail. I'm assuming I got an alignment fault, but not postive. If I replaced the direct struct read with a getter that returned a safe result, no more hang.
    RichRandall
    @RichRandall
    Hmm, so my bootloader seems to read 13 or 14 bytes of a packet, and then I think timeout reading the next byte. After the timeout it sends a NAK (a new Xmodem instance), which the sender interprets as a checksum mismatch, retrying the packet... Haven't quite figured it out yet, but I'm gonna keep thinking about it
    paulmeyer
    @paulmeyer
    If you are getting that many bytes into the packet, then I don't think your protocol is messed up. (you might verify that both sides think those 13/14 bytes are packet bytes. If the receiver thinks they are packet, but the sender is actually lost and starting over or something, it's a different story). If both sides are really 13 bytes into a packet and you lose it, I'd double check your physical port settings, especially the baud rate multiplier. Something weird is going on.
    RichRandall
    @RichRandall
    Seems you are right Paul. I tried reduced the baudrate, and I noticed I was getting further into each packet before falling over (obviously matching the rate hard-coded in miniuart to the one I used with ttywrite). I'm currently set at 1200 and the bootloader is working (albeit rather slowly). So it could be a hardware thing I suppose? Would anyone be willing to give me an implementation they know works on their hardware so that I can check?
    RichRandall
    @RichRandall
    Pretty sure my baud rate divisors are correct. I was using 270 for 115200 and am currently on 26041 for 1200
    paulmeyer
    @paulmeyer
    I am using 270 as well, so that is likely not it. Are you sure you are using 115200 on the host side? Another possibility would be if you were running at a different frequency on the pi (if you had stale files on the sd card, sometimes pi installations overclock things)
    Jeff McGlynn
    @jwmcglynn
    Yeah, I think that the base frequency depends on the clock rate. Someone else had a bootloader on their SD card that loaded before the other code, maybe that's the case here?
    RichRandall
    @RichRandall
    Interesting, but I've checked and all that's on the SD card in addition to bootloader (kernel8.img) are the bootcode.bin, config.txt, start.elf as per the assignment
    It's weird, because I tried about 4 different baud rates, and the lower it went the more bytes were successfully received
    paulmeyer
    @paulmeyer
    If you've used a clean SD card, it should be fine. The issue before was people that used an SD card with Raspbian on it and hadn't deleted all files. It could also be frequency off on the host side. You are specifying 115200 on ttywrite, correct? Try playing with the baud rate on the host side (much quicker than recompiling the pi side) and see.
    RichRandall
    @RichRandall
    Ok, well I've got a working bootloader now. I changed my implementation of uart.wait_for_byte(). It had previously been polling has_byte() in a loop with a 10ms spin_sleep, and counting how many waits had been performed until the specified timeout was reached. I think for some reason I was worrying unnecessarily about the timer overflowing/wrapping to zero. I now poll has_byte() in a hot loop until the current time is greater than the start time + the timeout. I was surprised my original code was causing issues, but guess I misunderstood how serial works!
    paulmeyer
    @paulmeyer
    The only thing I can think: with the 10ms sleep loop you were giving the buffer time to overflow. Wouldn't affect shell (unless you were a really fast typer), but at 115kbps you will get ~10k B/s, or 100 bytes in 10ms. Enough that you can't afford to wait between draining the buffer or you will overflow and lose data. Your buffer was regularly pitching bits, and by the time all bytes were sent you had picked up 10% of them. Reducing baud rate helped, you pitched fewer and made it further. Glad you figured it out, that's a tough one, especially since the inner routines (read_byte(), etc) all were 'tested' as they worked in shell at 60 bytes/second.
    RichRandall
    @RichRandall
    That sounds very plausible. I think I had I assumed that somehow UART would guarantee that the sender's TxAvailable bit of the LSR register would not be set if sending more bytes would result the receiver losing data. But on further thought it's probably naive to expect the device to provide that kind of flow control - that's presumably one of the main reasons to implement the xmodem protocol.
    Yerkebulan Tulibergenov
    @yerke
    Can someone help me with bootloader part? I am trying to read from Xmodem, but I don't really understand where we should write to. I want to write to [u8] array, but arrays don't implement std:io::Write trait, only Vec<u8> do. Do I need an additional structure on top of array to implement that trait?
    RichRandall
    @RichRandall
    Hi. So the aim is to write the data directly into memory at address 0x80000 (BINARY_START_ADDR). If you look at the hints in the Phase 4 notes there's a standard library function that will convert a raw address into something implementing Write.
    Yerkebulan Tulibergenov
    @yerke
    Thanks @RichRandall. That helped. Don't know how I forgot to check the hints.
    RichRandall
    @RichRandall
    Glad you got it working @yerke
    Nathan Ringo
    @remexre
    Are there known issues with using an RPi 3B+? I can boot Raspbian, but I'm not getting anything over serial during the blink example.
    I do get a blink pattern, but not a regular one; it's 4 "long" blinks, 4 "short" blinks, and a pause
    Nipun Pruthi
    @nipunpruthi
    Hello everyone, I just found out about stanford's amazing course and got here from reddit. Its summer vacations here(India) and I will do this course during free time. @tejom Can you please add me to code sharing room you created. Thanks
    Drapsag74
    @Drapsag74
    Hello everybody, I'm actually looking at the stanford course, would be interested if you could add me to code sharing room if there's one.
    Thanks
    Lequn Chen
    @abcdabcd987
    Hi everyone, just in case of someone like me uses the latest nightly compiler, I changed os.git so that the assignment 2 can run. I've put the patch here on the gist. Feel free to take it :)
    Yerkebulan Tulibergenov
    @yerke
    Hi. Did anyone try to run this project on RPi 3B+ instead of 3B? If yes, did you have any problems?
    isaak yansane
    @donkey-hotei
    Not sure you'll be able to get the expected behavior if you follow the course as-is. You should check the datasheet.
    Tomonori Murakami
    @tomosm
    Hi everyone. @tejom could you please add me to the code sharing room?
    Matthew Tejo
    @tejom
    @tomosm added you. though its been unused since the class ended. i should probabaly open it up
    Tomonori Murakami
    @tomosm
    @tejom Thanks!
    isaak yansane
    @donkey-hotei
    @tejom Would you mind opening it up?
    Also, not sure if anyone else ran into this problem: https://www.reddit.com/r/cs140e/comments/9xocfx/garbage_bytes_received_over_uart/
    Feeling pretty stuck on it at the moment.
    Matthew Tejo
    @tejom
    Yeah ill try to remember to when I get back from my trip. I need to find out how. I think your problem might be having two separate sessions. I think You need one program that reads and writes. I'm familiar with xxd though
    I thought you only need to use screen to send and receive stuff
    isaak yansane
    @donkey-hotei
    Thanks @tejom! Though, the problem is not that I have two separate sessions. I am using xxd s.t. I can debug an already existing problem.
    isaak yansane
    @donkey-hotei
    It's possible to have two file descriptors for the same file, xxd creates one when it calls open on /dev/ttyUSB0 with the O_RDONLY flag while screen opens the file with write access.
    isaak yansane
    @donkey-hotei
    Okay, now I'm face palming hard. For whatever reason I'd decided to OR the bits into the LCR, BAUD, IIR, and CNTL registers in the GPIO intialization step instead of just writing the bits to them.