Where communities thrive

Join over 1.5M+ people
Join over 100K+ communities
Free without limits
Create your own community

Explore more communities

gvisor/community

People

Repo info

Activity

Fabricio Voznika

@fvoznika

We should make things like runsc list a bit more resilient in case old state files were left behind or got corrupted, but there is no guarantee that list will be able to list old containers for example.

Marek

@majek

How to profile runsc or guest pids, under kvm? We got perf to do nice flame graphs for pthread, but not sure how to handle kvm.

Does gvisor (kvm/pthread) support ASLR?

prattmic

@prattmic:matrix.org

[m]

To the first question, you have to enable guest profiling for kvm (perf record -e cycles:hg), however I don't recall if perf handles the guest symbolization well. I think you might just get a ton of "unknowns" for the guest.

To the second question, we support ASLR for applications (actually we don't support turning it off...), but we don't have (binary) ASLR for runsc at the moment due to limitations with Go support for PIE binaries

Marek

@majek

I'm doing some networking benchmarks for gvisor, maybe you'll find this interesting. I have an echo server and echo client. The echo client is sending 40 times a bulk transfer of 4MiB and waiting for it to come back

first scenario, server on host, client in guest (gvisor)

the whiskers show min/max in us (per transfer of 4MiB block), the block is supposed to be avg +- deviation.

this is second scenario, where both tcp server and tcp echo client are inside gvisor

Marek

@majek

There are two takeaways:

if you have a server on host and a guest in gvisor, then passing --network=host to runsc gives you 2x perf boost
- if you have unix domain sockets inside runsc then the performance is tragic. 10x worse than anything else

is the tragiacal performance of unix domain sockets inside runsc (both server and client, using /tmp/pipe endpoint path) known?

am I expected to raise a bug?

Bhasker Hariharan

@hbhasker

Unix domain sockets are slow becausr the cureent default buffer size is 32kb

Could you run the test after bumping that up

For host yes it is faster simply because linux can move skbs around. Whereas netstack has to copy bytes back and forth from the host packet socket.

There are othe things too like our buffer management etc

Btw i believe you will see similar crappy perfomance for udp as well since it defaults to really small buffer sizes

Bhasker Hariharan

@hbhasker

There are two takeaways:

if you have a server on host and a guest in gvisor, then passing --network=host to runsc gives you 2x perf boost

if you have unix domain sockets inside runsc then the performance is tragic. 10x worse than anything else

dammit wanted to reply inline

FWIW --network=host also loosens your security of the sandbox because now you are using the host's networking stack

so its a trade off

Marek

@majek

I'm doing unix domain sockets on /tmp/pipe (within runsc container), am I expected to use another path for better performance?

Bhasker Hariharan

@hbhasker

Uds is inside runsc is probably poor because it defaults to a 32kb buffer. Could you increase the buffer sizes and try

Bhasker Hariharan

@hbhasker

@majek Could you share more data on how you ran these tests. I would like to run them locally to see if I can understand why UDS would perform badly

Marek

@majek

@hbhasker at your service google/gvisor#5132

https://gist.github.com/majek/a2cea2d116fa04aaf167f45f357b4311

Bhasker Hariharan

@hbhasker

Thanks! I will take a look and see if i can figure out why uds is performing much worse than tcp.

stormgbs

@stormgbs

Hi, everybody, what kinds of workloads you are running with gVisor ? :)

Robin Luk

@lubinsz

Hi, everybody, what kinds of workloads you are running with gVisor ? :)

I think java applications are often used in gVisor.

Pi dy/dx

@pidydx

Hello! I'm having sudden issues with gvisor using a configuration that I have had success with so I'm not sure what changed that is causing the problems. When I use crictl runp --runtime runsc sandbox.json as in the quickstart example I get "FATA[0000] run pod sandbox: rpc error: code = Unknown desc = failed to create containerd task: dial unix /run/containerd/s/2935bdefdde6ca60c8f96d66dbc70beb464c3ca85ac649fcb98193a3b1e8d189: connect: connection refused: unknown"

Travis DePrato

@travigd

Does gvisor disallow setuid? I have a legacy application that uses sudo that I'd like to run under gvisor, but I'm running into issues...

Adin Scannell

@amscanne

@pidydx containerd 1.3.9 included a fix for a security issue that changed the paths for unix sockets: https://github.com/containerd/containerd/security/advisories/GHSA-36xw-fx78-c5r4 .. A compatible fix was included with 9eb77281c4fe1c1f252a0df67ca2c1fee8867b80

We should send an announcement and update our containerd docs with this error (likely next week, or PRs accepted :)

Adin Scannell

@amscanne

@travigd I believe that setuid is still on the backlog (bit ignored), if you file a bug it would be good to understand what requires sudo (and why) to inform the priority

Pi dy/dx

@pidydx

@amscanne I am doing my testing with test-kitchen using ubuntu20.04 (which seems to have containerd 1.3.3) and I was having the problem with both 20201208.0 and 20201012.0 versions. Is that still likely related?

Pi dy/dx

@pidydx

Huh, welp good news I guess. I swapped out to using containerd.io 1.4.* from docker repo and it all magically works as expected again

Pi dy/dx

@pidydx

It is definitely a bit concerning that two different 1.3.3 packages in ubuntu20.04 appear to behave differently

Hasan Turken

@turkenh

Hi everyone,
Having an issue with gvisor on GKE.
Running a pod with two containers and need them to communicate over Unix Domain Socket (a socket file on a shared emptydir volume). This works on non gvisor nodes but getting following on gvisor:

"failed to create connection to unix socket: /tmp/socketfile.sock, error: dial unix /tmp/socketfile.sock: connect: connection refused"

Any idea on how to make this work?

Fabricio Voznika

@fvoznika

Shared volumes are mounted independently inside containers and Unix Domain Socket from one container will not be visible in the other container. Are you able to use abstract UDS?

If not, it might work if the shared volume is tmpfs (medium: Memory). In this case, there will be a single volume mounted in the pod that is shared between containers.

Hasan Turken

@turkenh

I did try medium: Memory but no luck.
I don't know what abstract UDS is (and how to use it in a pod between two containers), but will check and try.

Fabricio Voznika

@fvoznika

Abstract sockets uses a name that starts with "\0" followed by the socket name. The socket can be addressed by other processes using the same name, but it's not stored in the file system, thus it can be shared between containers of the same pod.

Also, I'm almost sure that "medium: Memory" with VFS2. You can find instructions to enable VFS2 in GKE here: https://docs.google.com/document/d/1_qMNPzXyDwtGnZaTeQLMmZ2JhKwiTfjwtGG2Z5zA030/edit

I can give it a try later today to confirm...

Hasan Turken

@turkenh

Thanks @fvoznika
I see VFS2 requires 1.18.9-gke.1501 or greater which is not possible for us ATM. But will keep this in mind for the future once we are able to move to that version. Is it planned to have it enabled by default on GKE at some point?

Hasan Turken

@turkenh

wow, abstract sockets just worked!
@fvoznika thanks a million!!

Fabricio Voznika

@fvoznika

I'm glad it worked. As for VFS2, yes it will become default as soon as we finish validating it.