Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Activity
  • Dec 24 2021 07:00

    nuta on main

    Drop support for ARMv6-M I hav… (compare)

  • Dec 20 2021 14:03
    nuta commented #54
  • Dec 20 2021 13:46
    krishpranav commented #54
  • Dec 20 2021 13:45
    krishpranav commented #54
  • Dec 13 2021 12:12
    nuta commented #56
  • Dec 13 2021 12:04
    AlessandroSangiuliano opened #56
  • Nov 12 2021 10:34
    nuta commented #54
  • Nov 11 2021 17:43
    Sologix commented #54
  • Nov 11 2021 13:28
    nuta commented #54
  • Nov 11 2021 13:24
    Sologix commented #52
  • Nov 11 2021 13:22
    Sologix commented #52
  • Nov 11 2021 12:17
    Sologix opened #54
  • Nov 07 2021 16:40
    Sologix commented #52
  • Oct 26 2021 22:55
    nuta commented #48
  • Oct 26 2021 15:01
    nagesh-c commented #48
  • Sep 02 2021 14:10

    nuta on move-cursor

    Update Update Update and 1 more (compare)

  • Sep 02 2021 10:03

    nuta on move-cursor

    Update (compare)

  • Sep 02 2021 09:54

    nuta on move-cursor

    Update (compare)

  • Sep 01 2021 13:23

    nuta on move-cursor

    Update Update Update (compare)

  • Sep 01 2021 10:01

    nuta on move-cursor

    Update (compare)

Seiya Nuta
@nuta
@PrayagS Good point. Currently, I'm working on the virtio-gpu driver and plan to improve the virtio library in couple of days. In the improvement, I'll try addressing the issue you described. FWIW, you can investigate why your virtio driver doesn't work by inserting some printf functions into QEMU (hw/virtio/virtio.c).

@nuta I was thinking I can take up one more task in parallel with KASAN, now that my exams are over and I have some time (and curiosity xD). Are we considering a slab allocation scheme in malloc?

A slab allocator sounds good idea :)

Yashraj Kakkad
@yashrajkakkad

A slab allocator sounds good idea :)

Great! Can you please create an issue for the same? I will start working on it!

Seiya Nuta
@nuta
@yashrajkakkad Sure! I've created an issue for the slab allocator. By the way, if you have feature requests for proposal, please feel free to create an issue by yourself :) nuta/resea#39
Yashraj Kakkad
@yashrajkakkad
Sure, thank you very much @nuta ! :D
Yashraj Kakkad
@yashrajkakkad

Hi @nuta, our 6 weeks are about to get over. We have our project presentation scheduled this Sunday evening. where we have to explain what we did and show our work (Fingers crossed haha :D). But that is just for our University and we'd love to continue working on the features we're implementing. Meantime, we'd be really grateful if you can share what you feel about our work, which we can use as a testimony in the concluding part of our presentation.

It's the first time we worked on an open-source at this scale and you've been a true guide to us in our journey! Thanks a ton :heart:

Prayag Savsani
@PrayagS
@nuta I totally agree with @yashrajkakkad and would love to continue contributing to this project. It has been a great learning experience working with you. I would really appreciate it if you can share a few words regarding the work we done. And thanks again for being such a great mentor to us.
Seiya Nuta
@nuta
@yashrajkakkad @PrayagS I'm glad to hear that! Especially because, I originally started writing Resea for educational purposes so that university students can understand and develop microkernel-based operating system without pains. I'm impressed that you folks have successfully submitted good pull requests (99% of my review comments were trivial nitpicks) within your deadline despite Resea lacks documentation. You should be proud of your software engineering and discussion skills. I'm looking forward you contributing famous open-source projects :)

@PrayagS Good point. Currently, I'm working on the virtio-gpu driver and plan to improve the virtio library in couple of days. In the improvement, I'll try addressing the issue you described. FWIW, you can investigate why your virtio driver doesn't work by inserting some printf functions into QEMU (hw/virtio/virtio.c).

I've pushed improvements on virtio library to allow constructing a descriptor chain. Can you try it out?

Prayag Savsani
@PrayagS
@nuta Thanks for your kind words. I completely agree with the "without pains" part as the way you've designed the OS did make it quite comfortable for us to work on it. Kudos to you for that!
And I'll be sure to take a look at the changes you made in the virtio library.
Yashraj Kakkad
@yashrajkakkad
Thanks a lot for your kind words, @nuta. We are honoured indeed! We really look forward to interacting with you and contributing more to Resea :smiley:
Yashraj Kakkad
@yashrajkakkad
Hi @nuta. I am stuck with that page fault thing in KASAN for quite a while now. Can you please suggest something?
Seiya Nuta
@nuta
@yashrajkakkad Sure. Can you share your git branch for KASAN?
Yashraj Kakkad
@yashrajkakkad

@yashrajkakkad Sure. Can you share your git branch for KASAN?

https://github.com/yashrajkakkad/resea/tree/asan

Seiya Nuta
@nuta

@yashrajkakkad I tried your code and it failed with general protection fault.

[kernel] WARN: Exception #13

[kernel] RIP = 0000000001005cc5 CS  = 000000000000002b  RFL = 0000000000000246
[kernel] SS  = 0000000000000023 RSP = 000000000300ad70  RBP = 000000000300adb0
[kernel] RAX = 0a110ced0a110ced RBX = 0000000000000004  RCX = 0000000000000001
[kernel] RDX = 0000000004124120 RSI = 00000000000000aa  RDI = f8f8f8f8f8f8f910
[kernel] R8  = 0000000000000003 R9  = 0000000000000000  R10 = 000000000300abd0
[kernel] R11 = 0000000000000246 R12 = f8f8f8f8f8f8f910  R13 = 0000000000000010
[kernel] R14 = 0000000000000020 R15 = f8f8f8f8f8f8f8f8  ERR = 0000000000000000
[vm] WARN: test: exception occurred, killing the task...
[kernel] destroying test...
QEMU 4.2.1 monitor - type 'help' for more information
(qemu) xp /i 0x1005cc5
0x01005cc5:  00 00                    addb     %al, (%rax)

I think this is caused by accessing a non-canonical address in RAX by ADD instruction.

I'm not sure but I guess some part of the memory (e.g. return address in the stack) is accidentally overwritten because RIP = 0x1005cc5 seems to be incorrect:

$ objdump -d build/vm.debug.elf|egrep '1005cc[0-9a-z]' 
 1005930:       0f 85 8d 03 00 00       jne    1005cc3 <vaddr2paddr+0x3e3>
 1005cc3:       48 bf c0 19 00 03 00    movabs $0x30019c0,%rdi
 1005cca:       00 00 00        <-- instruction at 1005cc5 does not exist
 1005ccd:       48 89 de                mov    %rbx,%rsi

Sadly, fixing your bug might take a time...

Yashraj Kakkad
@yashrajkakkad
@nuta Is the problem caused by something in my code? I was seeing such a problem even when my check_address function had nothing but a DBG statement.
Seiya Nuta
@nuta
I'm not sure but enabling KASan itself might cause the problem (unintended infinite recursion in check_address, etc.).
Yashraj Kakkad
@yashrajkakkad
@nuta I think that's the case as putting anything at all in check_address creates a problem. Any way I can help fix this?
Seiya Nuta
@nuta

@nuta I think that's the case as putting anything at all in check_address creates a problem. Any way I can help fix this?

Sure! I'll try fixing it in this weekend.

Seiya Nuta
@nuta
@yashrajkakkad OK I figured out the bug. It seems __shadow is too small and shadow_free accidentally destroyed other memory area. Thus it reproduces even if KASAN is disabled. Try the following patch:
diff --git a/libs/resea/malloc.c b/libs/resea/malloc.c
index e8040b25..5f78190c 100644
--- a/libs/resea/malloc.c
+++ b/libs/resea/malloc.c
@@ -345,6 +345,14 @@ void shadow_free(struct malloc_chunk *chunk)
     reladdr = ptr_cur - (paddr_t)__heap;
     reladdr >>= 3;
     DBG("%u", reladdr);
+
+    size_t len = chunk->capacity + MALLOC_REDZONE_LEN;
+    if ((vaddr_t) &__shadow[reladdr] + len >= (vaddr_t) __shadow_end) {
+        WARN_DBG("__shadow is too small: addr=%p, len=%d, reladdr=%p",
+                 &__shadow[reladdr], len, reladdr);
+        return;
+    }
+
     for(size_t i = 0; i < (chunk->capacity + MALLOC_REDZONE_LEN); i++)
     {
         __shadow[reladdr++] = SHADOW_FREED;
Yashraj Kakkad
@yashrajkakkad
Hi @nuta. Thanks for your time. I will work further on it this weekend
Manav-Kumar
@Manav-Kumar
Hello All, I'm new to resea and wanted to actively contribute in it. I'm a final year CSE btech student enthusiast about the OS concepts, tools etc. Can somebody give me a quick head start for the current active projects and whats the goal briefly. I'll be glad.
Seiya Nuta
@nuta

Hi, @Manav-Kumar. We have already implemented basic features like the microkernel, FAT file system, TCP/IP, and some device drivers. We're working on some improvements and new features:

Since I'm working on another OS project in Rust in very recent weeks, the progress is slow. That said, if you'd like to contribute to Resea, I can support you :)

Aside from those projects, we welcome your ideas and would like you to work on a project whatever you want. For example, integrating Linux Kernel Library would be an interesting topic.

Manav-Kumar
@Manav-Kumar
Yeah, it would be fun to implement it. Can you please share some material around it and how do we plan for it.
Does resea supports multitasking features in current version. Can we plan to improve it in any way if it does.
Manav-Kumar
@Manav-Kumar
Memory Management, vm, paging, interruptions, scheduling.. this might already be implemented in basic kernel features, are there any scopes of improvement.
Seiya Nuta
@nuta

Does resea supports multitasking features in current version. Can we plan to improve it in any way if it does.

Yes it supports. In a nutshell, Resea kernel implements a multi-tasking concept called task (so-called process). Each task has its own virtual memory space and can communicate with other tasks over message passing IPC. Threads are not supported intentionally because all applications in Resea are implemented in a single-threaded, event-driven way to make it easier to debug (we can still implement sort of threads in the userland by mapping same memory pages through vm_map syscalls btw).

Memory Management, vm, paging, interruptions, scheduling.. this might already be implemented in basic kernel features, are there any scopes of improvement.

We only have rudimental implementation so I think there's room for further improvements. Among the mentioned areas, user-level scheduling (nuta/resea#16) might be interesting to work on. A MINIX wiki page is informative on this topic.

I've implemented task priority in the kernel (task_schedule syscall) but the feature is unused at all.
Seiya Nuta
@nuta
Regarding Linux Kernel Library, this recent paper and its references might be helpful. Porting LKL into Resea would be an advanced topic, but it should be fun.
Manav-Kumar
@Manav-Kumar
Hello Sorry for the late response, got busy with my full time job. I'm planning to start working upon it immediately. Can you still please provide support for it. Really interested to implement linux kernel library or if possible something around adding more tricks for resource management like Copy On Write, Zero filled on demand, demand paging etc if it's not implemented yet.
Seiya Nuta
@nuta
You don't have to be sorry! Let's enjoy hacking only when you have time :) Linux Kernel Library should be fun but I'm not sure it's difficulty. For the first step, It might be good to work on smaller improvement on VM server, the core of memory management such as physical page allocation and page fault handling for normal apps. I guess you're interested in improving the server.
Seiya Nuta
@nuta
Here's my some comments on improvements you mentioned:
  • copy-on-write: Not implemented. Mapping zero-filled pages (e.g .bss section) using CoW might improve the startup time of a userland application.
  • zero filled on demand: I don't know what this feature does. BTW, preparing zero-filled pages in advance during an idle state would be good for performance.
  • demand paging: Implemented. In fact, every page is mapped by the pager task on the first access.
Manav-Kumar
@Manav-Kumar
Thanks for the pointers, i'll start working upon improving small things. Yes, you are right about the interest and particularly about core of memory management.
Bigpilot
@bigpilot
I was thinking about whether it would be difficult to add hard real-time capability to Resea. In my opinion only one Task could ever have hard real-time capability, all other Tasks can only be soft real-time at best.
Seiya Nuta
@nuta
I think the most difficult part is WCET analysis. I have no experience with a hard real-time systems though.
Bigpilot
@bigpilot
My thought was that the real-time task would always be executed immediately after a predefined interrupt occurs. Also, it's quantum would be much larger than that of "normal" Tasks since it has to handle the interrupt without, well, interruption.
Can you tell something about the Linux ABI emulation? I assume it can only run a limited number of Linux binaries since there are few device drivers written for Resea.
Seiya Nuta
@nuta

always be executed immediately after a predefined interrupt occurs

We need to figure out how long does it would take until an interrupt is delivered in the worst case. Regarding the quantum, Resea already has a priority-based scheduler and it keeps running the task with the highest priority.

Can you tell something about the Linux ABI emulation? I assume it can only run a limited number of Linux binaries since there are few device drivers written for Resea.

IIRC, it can run only some very basic commands on Busybox's shell.

Bigpilot
@bigpilot
Why would we need to find out how long it takes? It would depend on the processor and architecture in any case. You could easily measure it with an oscilloscope I suppose.
Seiya Nuta
@nuta
Because that's what hard real-time guarantees to meet critical deadlines. You might be interested in this paper: http://apsys11.ucsd.edu/papers/apsys11-blackham.pdf
Bigpilot
@bigpilot
Thanks for that. I was hoping it would be somewhat easier. The hard real time execution path could maybe be constructed in such a way that the rest of the operating system doesn't really get into play. It should go from the interrupt straight to the hard real-time Task. Most hard real-time OS'es do this by having a seperate real-time kernel.
Seiya Nuta
@nuta
I hope so too :) What makes things complicated is that interrupts are not always enabled and its delivery can be delayed (unless it's a NMI).
elkaamee326
@elkaamee326
Hello Seiya Nuta, I hope everything’s well. I’ve been trying to develop an HD audio driver for an OS written in C# using something called Mosa-Core, which translates .net IL code into assembly bootable code. I’ve taken a look at your HD Audio article, which helped me to understand HD audio a bit better along with the Intel specification, (which could be confusing). In the project I’m working on, I used most of your code and some parts from other implementations, since they were the only implementations that I could understand clearly (most of the other ones were way too tricky). I’m still struggling to communicate with the codec using CORB and RIRB. I’ve been trying to develop an HDA driver for about 5 months and I just get 0’s / incorrect values everytime I try to send a command to the codec (I’m using Virtualbox as my VM). Could you please help me figure out/identify what I’m doing wrong? Also, the code that I’m using only uses physical memory and not virtual. Any help would be very appreciated :) Have a great day :)
Here is the link to the source:
elkaamee326
@elkaamee326
I also used a Virtualbox VM running Linux Mint as a debugger environment, so that I can compare and analyze values written to resea's debugger and the driver I'm working on. I tested resea's HD Audio driver and it plays sound but there is noticiable glitching and popping whilst playback.
Seiya Nuta
@nuta
@elkaamee326 How about using QEMU? When I got stuck, I usually take a look at the device emulation in QEMU to investigate what's going on. For example, I added some printf in hw/audio/intel-hda.c to debug a HDA driver.
Seiya Nuta
@nuta
BTW, here's for talking about Resea, not your OS. Please ask for help in other communities such as OSDev.org and StackOverflow.
PeterOSdevel
@PeterOSdevel
hello. is resea running on UEFI?
Seiya Nuta
@nuta
no not yet