Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Activity
    softraid(4)
    @softraid:matrix.org
    [m]
    @dqminh: thanks for that gist btw
    Ian Lewis
    @ianlewis
    Waypipe might work. Easiest thing to do is connect over a local network socket.
    prattmic
    @prattmic:matrix.org
    [m]
    From https://github.com/google/gvisor/pull/6199/files, it looks like this fs only supports mmap and not read? If so, you could still call Translate and then memmap.File.MapInternal and read from the returned BlockSeq. Though that is getting a bit awkward and you may want to add a special interface specific to the fs
    Skallwar (Esteban Blanc)
    @skallwar:matrix.org
    [m]

    Translate and then memmap.File.MapInternal and read from the returned BlockSeq

    This is what I'm trying to do here: google/gvisor#6397

    Zach Eddy
    @ZachEddy

    Hello! I’m struggling to get gVisor to work in Kubernetes v1.21 after I upgraded containerd to v1.5.5 and gVisor to 20210720 on my worker nodes. Looking at the containerd logs, I see the following message:

    level=error msg="RunPodSandbox for &PodSandboxMetadata{...} failed, error" error="failed to create containerd task: failed to create shim: type with url runtimeoptions.v1.Options: not found: unknown"
    level=error msg="failed to delete" cmd="/usr/bin/containerd-shim-runsc-v1 ..."

    I can use ctr to manually run a container with runsc, so I think gVisor is installed correctly. Any idea what the issue could be? Thanks!

    7 replies
    Yong He
    @zhuangel
    Hello, does anyone know why FPState size for KVM platform is fixed 4096 bytes (refer to https://github.com/google/gvisor/blob/master/pkg/sentry/arch/fpu/fpu_amd64.go#L36), actually ExtendedStateSize has already initialed as "maximum size if all valid features" (refer to https://github.com/google/gvisor/blob/master/pkg/cpuid/cpuid_x86.go#L724)
    Daniel, Dao Quang Minh
    @dqminh
    @ianlewis do you know any blockers that would prevent gvisor from using containerd package >v1.3 as dependency ? I made google/gvisor#6485 which seems to pass the tests locally, but not sure if i missed anything
    Guoqing Li
    @gqlo

    Hello there. I am trying to use cyclictest to benchmark the CPU latency in gVisor containers:
    docker run --privileged -it --runtime=runsc --cpus=4 --memory="8192m" leap-runsc /bin/bash

    I am getting:
    Unable to change scheduling policy!
    either run as root or join realtime group

    Apparently I am running a privileged container, logged into the shell with root user. There are no issues with runc, kata-runtime etc. Is there a way to run cyclictest in gVisor?

    Jianfeng Tan
    @tanjianfeng
    Unable to change scheduling policy!
    either run as root or join realtime group
    In gVisor, sentry intercepts all app syscalls, including all sched_* syscalls., see https://github.com/google/gvisor/blob/master/pkg/sentry/syscalls/linux/sys_sched.go. And sentry reuses the scheduler in go runtime (which is a work-steal, FCFS scheduler). So basically, there's no special sched features, like policy, priority.
    Ian Lewis
    @ianlewis
    @dqminh it's mostly that we have to support older versions of containerd and upgrading breaks compatibility with 1.2
    @gqlo gVisor doesn't support changing scheduling policy partly because we use the go runtime's goroutine scheduler and that doesn't provide any ways to tweak it. Can you maybe create an issue for running cyclictest so we can track?
    Daniel, Dao Quang Minh
    @dqminh
    @ianlewis did you mean compatibility with 1.3 ? I think the client <-> containerd API is stable for the parts gvisor used ( shim etc ), see https://github.com/containerd/containerd/blob/main/RELEASES.md#public-api-stability. So if we can build the client with newer version of containerd, it should work with at least 1.3. At least google/gvisor#6485 is passing containerd tests on my laptop.
    Guoqing Li
    @gqlo
    @tanjianfeng Thank you for the code snippet!
    Ian Lewis
    @ianlewis
    @dqminh I mean the newly built shim with containerd 1.3 code needs to work with containerd 1.2
    Unfortunately containerd 1.3 code needs an environment var that 1.2 doesn't set
    The API is "stable" if you build the shim with a version of containerd that is older than the version of containerd you're using
    Unfortunately, compatibility occasionally breaks there too.
    But the stability containerd provides is backwards compatibility, not forwards compatibility.
    Ian Lewis
    @ianlewis
    The requirement that it work with 1.2 is because we still have k8s clusters using it we need to support. Cloud has longer deprecation policies these days :-/
    I'd like to explore ways we could support newer versions of containerd better but none of the options seem good maintenance wise.
    @dqminh is there a reason you need it to be built with 1.3?
    Daniel, Dao Quang Minh
    @dqminh
    @ianlewis i think we are confusing the version number :D gvisor currently is built with containerd 1.3, and we are testing with both containerd 1.3 and 1.4. The PR is upgrading the containerd dependency to 1.4 ( which i need to have some cgroupv2 related functions ), which should still work with containerd >= 1.3
    Shivansh Sukhija
    @Decodx09
    new member here , been working on back-end dev and game dev for a while if someone could walk me through the steps to contribute in gVisor
    that would be helpful
    i have not contributed ever in any open source
    Alok Hegde
    @alok1929
    Hey I'm new here, been working on web dev for a while.If someone could guide me in gVisor contribution, that would be great
    Ian Lewis
    @ianlewis
    @dqminh yeah. I was maybe getting the version mixed up. If I'm not mistaken upgrading the dependency to 1.4 means that it will not work with 1.3 because the 1.4 dependency code requires an environment variable to be set that isn't set by 1.3. We wanted to upgrade ages ago but haven't been able to because we need to support containerd 1.3.
    @alok1929 @Decodx09 there is some info in the CONTRIBUTING doc and on the website https://gvisor.dev/contributing/
    You can navigate issues on GitHub using labels. Some issues are labeled as good first issue.
    Another label to look at is the help wanted label
    Fadhil Kurnia
    @fadhilkurnia

    Hi guys, I don't know much about Linux syscall. I am trying to intercept any disk io syscall from a container to OS kernel, and I found out about this awesome gVisor project. After reading its architecture for a while, I am wondering why we need Gofer for handling disk i/o, why Sentry can not handle it?

    Any explanation would be appreciated!

    Jianfeng Tan
    @tanjianfeng

    Hi guys, I don't know much about Linux syscall. I am trying to intercept any disk io syscall from a container to OS kernel, and I found out about this awesome gVisor project. After reading its architecture for a while, I am wondering why we need Gofer for handling disk i/o, why Sentry can not handle it?

    Any explanation would be appreciated!

    The reason lies in the design of gVisor. gVisor sandbox process shall be reduced in small amount of syscalls (into host kernel); and some file-related syscalls are dangerous but indispensable. So gVisor uses the way of defense-in-depth. Put those syscalls into gofer process, and let sandbox process to talk with gofer processes in a protocol (p9 in current implementation). Actually, not all file-related syscalls are put into gofer, as this path is too slow. So gofer process may donate FDs into sandbox process, and it calls pread/pwrite to make sure some data paths are faster.

    Fadhil Kurnia
    @fadhilkurnia

    Thanks for the reply @tanjianfeng !

    I have more questions. I tried to intercept some write(2) system calls issued by a running container. For a starter, I added fmt.Println("log") in https://github.com/google/gvisor/blob/master/pkg/sentry/syscalls/linux/sys_write.go#L46 to make sure I intercept at the right location, as what Ian Lewis said in https://groups.google.com/g/gvisor-users/c/r4xrbpP9-pQ. It works well, whenever there is any write system call, my write log is triggered.

    However, when I tried to do more complex logic (I tried to make an HTTP request when Sentry detects writes syscall to a specific file, I tried to log the write requests in a file using os.WriteFile), the running container stop immediately.

    Is this because of the limited syscall that Sentry can use? so I can't create an HTTP request or write a file from Sentry? Is there any better way to intercept file IO syscall in Sentry?

    Thank you for the guidance!

    Jianfeng Tan
    @tanjianfeng
    Yes, most likely, the container was terminated because it violates seccomp rules. os.WriteFile accepts a parameter of file path, which means it opens that file, and open() is not allowed in sandbox process.
    Jianfeng Tan
    @tanjianfeng

    You already can intercept the write syscall, and the problem you face now is how to redirect the write content to "somewhere".

    • When you use fmt.Println("log"), the content is rewritten into stdout or stderr of sandbox process (when you enable debug option, it's in runsc log file like *.boot or a coalescent log file).

    • If you want to rewrite the content to some specified file, you need to borrow a "fd" from gofer. The background behind this, sentry does not open file, it lets gofer to open file, and donate fd back to gofer. To rewrite to the given fd, you can refer this function: https://github.com/google/gvisor/blob/master/pkg/sentry/fsimpl/gofer/handle.go#L107

    Mahmoud
    @i3abghany
    Hello all :-)
    I'm trying to build from source and I've run make runsc
    However, I do not get a directory pointed to by bazel-bin.
    $ stat bazel-bin
      File: bazel-bin -> /root/.cache/bazel/_bazel_root/06dac7d55a426239d70c12d704d4fc29/execroot/__main__/bazel-out/k8-opt/bin
      Size: 102           Blocks: 8          IO Block: 4096   symbolic link
    Device: 807h/2055d    Inode: 562602      Links: 1
    Access: (0777/lrwxrwxrwx)  Uid: (    0/    root)   Gid: (    0/    root)
    Access: 2021-09-09 22:17:37.958164085 +0200
    Modify: 2021-09-09 22:16:33.471035217 +0200
    Change: 2021-09-09 22:16:33.471035217 +0200
     Birth: -
    but I cannot execute the sudo cp ./bazel-bin/runsc/linux_amd64_pure_stripped/runsc /usr/local/bin command because it cannot access the path of bazel-bin/...
    Mahmoud
    @i3abghany
    .
    $ ​sudo cp ./bazel-bin/runsc/linux_amd64_pure_stripped/runsc /usr/local/bin
    cp: cannot stat './bazel-bin/runsc/linux_amd64_pure_stripped/runsc': No such file or directory
    Bhasker Hariharan
    @hbhasker
    @Decodx09 @alok1929 I would see our open issues and our Contributing guide to pick out some small issues to get familiar with the code.
    Mahmoud
    @i3abghany
    ^2 I found the runsc binary in ./bazel-out/k8-opt-ST-4c64f0b3d5c7/bin/runsc/runsc_/runsc. That path was echoed in the make logs
    Bhasker Hariharan
    @hbhasker
    Hi, I am currently in the process of gathering pain points about gVisor/Netstack in general and networking features that folks find missing/incomplete etc. I have opened an issue to gather requirements from our Open source community
    I should clarify I am currently focussing on Networking related features, but if folks have specific other feature requests, feel free to add them and I will try and route them to proper owners.
    Alok Hegde
    @alok1929
    @hbhasker i have asked about being assigned to an issue but haven't gotten any response. Issue number is #5745. Still waiting 😃
    Bhasker Hariharan
    @hbhasker
    @alok1929 I would start by reading the code in https://github.com/google/gvisor/blob/108410638aa8480e82933870ba8279133f543d2b/pkg/sentry/fsimpl/proc/task_net.go to understand how to add a new proc pseudo file.
    Fadhil Kurnia
    @fadhilkurnia

    You already can intercept the write syscall, and the problem you face now is how to redirect the write content to "somewhere".

    • When you use fmt.Println("log"), the content is rewritten into stdout or stderr of sandbox process (when you enable debug option, it's in runsc log file like *.boot or a coalescent log file).

    • If you want to rewrite the content to some specified file, you need to borrow a "fd" from gofer. The background behind this, sentry does not open file, it lets gofer to open file, and donate fd back to gofer. To rewrite to the given fd, you can refer this function: https://github.com/google/gvisor/blob/master/pkg/sentry/fsimpl/gofer/handle.go#L107

    Thanks for the recommendations!

    I am stumbling upon the performance overhead of gVisor, if we use ptrace to intercept any syscalls, the syscall becomes ~20x slower (src: https://gvisor.dev/performance/syscall.csv), however why gVisor can still achieve low bandwidth overhead for file I/O as presented in https://gvisor.dev/performance/fio.csv?

    Could somebody explain the reasoning behind this?
    Thanks!