Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Activity
  • Nov 12 10:34
    nuta commented #54
  • Nov 11 17:43
    Sologix commented #54
  • Nov 11 13:28
    nuta commented #54
  • Nov 11 13:24
    Sologix commented #52
  • Nov 11 13:22
    Sologix commented #52
  • Nov 11 12:17
    Sologix opened #54
  • Nov 07 16:40
    Sologix commented #52
  • Oct 26 22:55
    nuta commented #48
  • Oct 26 15:01
    nagesh-c commented #48
  • Sep 02 14:10

    nuta on move-cursor

    Update Update Update and 1 more (compare)

  • Sep 02 10:03

    nuta on move-cursor

    Update (compare)

  • Sep 02 09:54

    nuta on move-cursor

    Update (compare)

  • Sep 01 13:23

    nuta on move-cursor

    Update Update Update (compare)

  • Sep 01 10:01

    nuta on move-cursor

    Update (compare)

  • Aug 28 13:18
    nuta edited #41
  • Aug 28 13:18
    nuta edited #41
  • Aug 26 15:10

    nuta on move-cursor

    Update Update Update (compare)

  • Aug 26 09:48
    nuta edited #41
  • Aug 26 09:47
    nuta edited #41
  • Aug 26 09:47
    nuta edited #41
Yashraj Kakkad
@yashrajkakkad
Thanks @nuta, it does help!
Seiya Nuta
@nuta
@PrayagS I skimmed through your code and am sure that you're on the right track! I've added a doc about timer_set. I hope it helps.
The datetime server would set a timer periodically so that it can update its internal wall clock.
Regarding the datetime server's interface, I think it should provide a single API like gettimeofday(2) on Linux (i.e. unixtime): rpc gettimeofday() -> (unixtime: uint64).
Seiya Nuta
@nuta
I'm not sure and have been thinking how we should handle timezones btw.
Also, we need to convert the date (e.g. 2020-11-13) from the RTC into unixtime, but it seems to be complicated. I need to study the algorithm first...
First of all, I recommend you to send a PR for your RTC driver.
Prayag Savsani
@PrayagS
@nuta Thanks for taking the time to go through the code. That doc on timer_set surely helps. I'll share my progress on that soon. And I agree with dealing with time values as Linux does. I have seen implementations before and I should be able to get back with something on it soon.

I've sent the PR for RTC driver (nuta/resea#30).

I haven't thought about timezones yet. Regarding apps that make use of this API, I had thought of implementing a simplified version of date(1). I also thought maybe we could show the current time on the QEMU screen. Please let me know your ideas/thoughts on this.

Seiya Nuta
@nuta
Thank you for the PR, @PrayagS! I replied review comments on GitHub.
As the first step, let's forget about the timezone for now. I don't recommend implementing date(1)-like interface because sending a string over IPC is not efficient (we need some memory allocations and additional IPC calls). Instead, consider returning a tuple of integers (year, month, day, ...).
Seiya Nuta
@nuta
Since handling unixtime is complicated, tuple of integers (year, month, ...) seems to be a realistic approach to me. That is, datetime calls the rtc driver at startup, update the datetime internally using timer_set, and reply the tuple to clients.
Prayag Savsani
@PrayagS
Thanks for laying out the complete picture again @nuta! Working with a tuple, in this case, is indeed easier. I'll go ahead with that and still maybe take a look at how other OSes handle unixtime. Might just implement it if the conditions are in our favor. I'll be sure to discuss that first.
Prayag Savsani
@PrayagS
So I did up going with unixtime since it would be easier to handle the time internally in that case. So the datetime server gets the time from RTC (in terms of year, month, etc.) which is then converted to unixtime. The timer keeps on incrementing and sends the timestamp if gettimeofday message is requested.
I'm confused regarding the location of the helper functions since we're sending the timestamp and applications might need to convert it back to the datetime struct. For now, they're defined in datetime/datetime.h
Seiya Nuta
@nuta
How about placing them in the standard library? (libs/resea and libs/resea/include/resea).
Prayag Savsani
@PrayagS
@nuta Done. Anything else you suggest?
Seiya Nuta
@nuta
Your design looks good to me! Could you create a PR to review?
Prayag Savsani
@PrayagS
The PR is up. nuta/resea#31
Arpit Vaghela
@arpitvaghela
@nuta Won't the test for nuta/resea#28 be similar to usage of shm in linux.Also, I believe shm.c to be a part of resea/include/ or will it be in the kernel/ as well ?
Seiya Nuta
@nuta

@nuta Won't the test for nuta/resea#28 be similar to usage of shm in linux.

I think so. Specifically, we need at least two tasks (apps/test and a new app like app/shm_test). app/test creates a shared memory and tell it app/shm_test. It maps the shm and writes something to the shm. Lastly, apps/test checks if the written data is visible.

Also, I believe shm.c to be a part of resea/include/ or will it be in the kernel/ as well ?

Do you mean API wrappers like libs/resea/include/async.h? If so, I think it should be resea/include/experimental/shm.h since its not yet matured feature.

Prayag Savsani
@PrayagS
Congratulations on the new release @nuta! I'm glad that our team could be part of it.
Seiya Nuta
@nuta
Thank you a lot for your contribution @PrayagS! :D
Yashraj Kakkad
@yashrajkakkad
Hi @nuta I apologise for being inactive. I am not able to contribute due to my upcoming examinations. I will be up soon.
Seiya Nuta
@nuta
@yashrajkakkad That's absolutely no problem and don't have to be sorry! You're totally right and preparing for examinations is more and more important. Please work on only when you have time to spare :)
malbx
@malbx
@nuta I have started working on the random number server. Can you take a look and let me know if this approach is what you had in mind? Also, do you have any thoughts on how you would like to handle the seed?
malbx/resea@803d14e
Seiya Nuta
@nuta

@malbx Your design looks good to me! Could you create a PR? Let's improve it incrementally.

Also, do you have any thoughts on how you would like to handle the seed?

Towards a cryptographic-secure RNG, we might need to add kernel support and a system call to get the seed from the hardware. I'm not familiar with random number generators but AFAIK Linux extracts randomness from interrupt timings and RDRAND instruction on x64.

Yashraj Kakkad
@yashrajkakkad
Thank you @nuta! See you soon hopefully :D
Seiya Nuta
@nuta
good luck 👍
Prayag Savsani
@PrayagS
@nuta Our presentations are due soon and we've around a week until that. I was thinking of another project to work on and saw virtio-blk and Userspace debugging using LLDB on the board. Which one do you think is more suitable given the short timeframe? Does the LLDB one involve that much work? Please let me know your thoughts.
Or if you have any ideas on something completely else that fits the timeframe, please feel free to share those as well.
Prayag Savsani
@PrayagS

@nuta So I started some work on the virtio-blk driver and went through the spec and some existing implementations (1, 2).

I noticed how they're sending three different buffers (one for header, one for data and one for device writeable status byte) and chaining them using the next field. resea's virtio_modern implementation doesn't have that but does mention descriptor chaining in the code. So can you please explain how that would work in resea?

I think I should be able to make it work like those existing ones using the legacy interface but I'm confused regarding how chaining works in the modern implementation.

Seiya Nuta
@nuta

@PrayagS I think implementing virtio-blk driver (for legacy virtio device) is more realistic.

You may already know, Resea's virtio_modern implementation uses Packed Virtqueue. See 2.7.6 Next Flag: Descriptor Chaining in the spec (the specification describes both legacy and modern devices in the single page and it makes a bit confusing...).

I haven't tried yet but according to the QEMU's implemenetation (in virtqueue_packed_read_next_desc()), we don't use next field in the packed virtqueue. Instead, we need to use continuous descriptors as a chain and set VIRTIO_DESC_F_NEXT to the descriptor flags (except the last one).

The current virtio-net driver does not use that simply because it's not needed. Thus, you may also need to improve also the Resea's virtio_modern to handle the descriptor chaining.

Prayag Savsani
@PrayagS
@nuta Thanks for clearing this out. So if I'm getting it correctly, since we're using packed virtqueues, we don't need to set the next field and just set the flags accordingly. Please let me know if I wrongly interpreted this. I'll still implement it for the legacy device first and then proceed with the modern implementation.
Seiya Nuta
@nuta

since we're using packed virtqueues, we don't need to set the next field and just set the flags accordingly

i'm not 100% sure but I think so.

I'll still implement it for the legacy device first and then proceed with the modern implementation.

Ah I got it wrong. It looks that Resea's virtio library abstracts modern/legacy interfaces so you won't have to care the differences.

Yashraj Kakkad
@yashrajkakkad
@nuta Should I continue with '>>3' in shadow? The paper describes a way of assigning "states" to our 8 bits.
Seiya Nuta
@nuta
I think you don't have to. The original ASan uses single bit state for each memory address to reduce the memory footprint. However, I think we don't need to do that since we don't have large scale apps.
Yashraj Kakkad
@yashrajkakkad
I can surely do that, but then we'll not be able to mark different regions with different states right? It'll be either addressable or unaddressable. Wouldn't the scheme mentioned in the paper you shared be better?
Seiya Nuta
@nuta

we'll not be able to mark different regions with different states right?

Can you provide a concrete example? It seems I misunderstood something.

Yashraj Kakkad
@yashrajkakkad

Relevant section from the paper -

We use the following encoding for each shadow byte:
0 means that all 8 bytes of the corresponding application
memory region are addressable; k (1 ≤ k ≤ 7) means that
the first k bytes are addressible; any negative value indi-
cates that the entire 8-byte word is unaddressable. We
use different negative values to distinguish between dif-
ferent kinds of unaddressable memory (heap redzones,
stack redzones, global redzones, freed memory).

Seiya Nuta
@nuta
Thanks for the sharing! It makes sense. I misunderstood how ASan records the shadow memory. Give me a time to skim through the paper...
Yashraj Kakkad
@yashrajkakkad
Thank you! I'll be waiting for your nod :)
Seiya Nuta
@nuta
@yashrajkakkad As I commented in #29, please continue using >> 3.
Prayag Savsani
@PrayagS
So I noticed that virtq_allocate_buffers() allocates vq->num_descs number of buffers of both the same size and r/w access (either all are device-writeable for reading or all are device-readable for transmission).
In the case of virtio-blk, as I had mentioned before, the buffer containing the header needs to be device-readable. The payload buffer's r/w access depends on the operation. And the status byte needs to be device-writeable.
What I did was create three packets for writing and then three for reading. I chained them using the VIRTIO_DESC_F_NEXT flag. Most of what I did is according to the examples given in the spec (2.7.21.3 and 2.7.22).
The missing headers error doesn't come up now but the device isn't responding at the read request for some reason. You can take a look at my code here.
I think it's either the way I'm handling the buffer IDs (the spec doesn't go in much detail but looking at their example, it seemed like it is important) or it is the buffer allocation process since I'm using virtq_allocate_buffers() to allocate buffers of the same size regardless of their type.
Any help would be appreciated. Especially if you can elaborate on the id part since all the elements of a block request are referred to as elements of a buffer whose ID is stored in the last descriptor of the chain.
Yashraj Kakkad
@yashrajkakkad
@nuta I was thinking I can take up one more task in parallel with KASAN, now that my exams are over and I have some time (and curiosity xD). Are we considering a slab allocation scheme in malloc?
Seiya Nuta
@nuta
@PrayagS Good point. Currently, I'm working on the virtio-gpu driver and plan to improve the virtio library in couple of days. In the improvement, I'll try addressing the issue you described. FWIW, you can investigate why your virtio driver doesn't work by inserting some printf functions into QEMU (hw/virtio/virtio.c).

@nuta I was thinking I can take up one more task in parallel with KASAN, now that my exams are over and I have some time (and curiosity xD). Are we considering a slab allocation scheme in malloc?

A slab allocator sounds good idea :)

Yashraj Kakkad
@yashrajkakkad

A slab allocator sounds good idea :)

Great! Can you please create an issue for the same? I will start working on it!

Seiya Nuta
@nuta
@yashrajkakkad Sure! I've created an issue for the slab allocator. By the way, if you have feature requests for proposal, please feel free to create an issue by yourself :) nuta/resea#39