A quick example might be the dirtySet. Currently, the information is stored in CachingInodeOperations, and the pgalloc might need to release a page that belongs to dirty set. So pgalloc must be able to track the page to inode, and finds a way to flush it, and the flushing involves the dirty sets in inode. So if we want to do that in pgalloc, we should allow pgalloc to have such information.
Besides, if pgalloc frees page, but inode.cache still has it in the cacheSet
Then further access to this page that causes a "cache hit" in inode level, is actually not a real cache hit, because we need to read the page back into cache from backing file
So after looking at Evict interface again, I think I understand the interaction here. To periodically flush dirty memory, the encouraged way in gvisor is to pass something like DirtyRange to pgalloc, and start per-memoryfile flusher goroutine. But is it a good idea to have more than 1 flush routines?
Apart from the implementation choice, do you have comments on the idea itself? Do you want lru as an eviction strategy in gvisor?
(I will be back shortly)
OK
there's already a goroutine for each pgalloc MemoryFile which is the reclaimer
for evictable memory at the moment it essentially evicts everything when needed (https://github.com/google/gvisor/blob/master/pkg/sentry/pgalloc/pgalloc.go#L1147)
The existing EvictableMemoryUser interfaces are probably sufficient, and you'll actually want to do a few things:
- Implement the LRU mechanism for pgalloc on the basis when the range was called with MarkEvictable. (You could probably add an additional Touch method that brings things to the front of the list; that would be called by e.g. write and read).
- Introduce additional behavior to the reclaimer goroutine such that it can enforce the provided options. Essentially you would have a startEvictionsLocked evict a specific amount instead of just "everything".
- With this is place, it might also be feasible to extend the EvictableMemoryUser to support a "Flush(...)" interface in addition to the "Evict(...)" call, which would just flush dirty pages but not evict. You could add a separate pgalloc goroutine that does appropriate flushing based on the provided targets. I think this is a somewhat distinct independent feature though.
Anyways, I definitely want @nixprime's opinion here, but that's my view. I think that essentially pgalloc has a super coarse eviction policy now, and making that smarter with the existing eviction interfaces is the way you want to go
Hi Folks, I am trying out gvisor on GKE and see that pods on two different nodes (in gvisor node pool) can't communicate with each other over ClusterIP. Is this expected behavior on GKE or a limitation of gvisor? I have been able to get pods to communicate running in gVisor on other cluster (GCE VM's) that was brought up manually using kubeadm. Thanks.
@fvoznika, thanks for checking. i have a simple grpc server/client setup and all it does is send messages back and forth. I tried running server as a deployment and exposed it as ClusterIP and client runs as a Job, both in gvisor. Server deployment comes up fine, however, client is unable to talk to server. I also tried the same using runc and it is complaining that udp connection was refused. It seems maybe some problem with my GKE networking setup.
hi folks, something is still not quite right. i ran the same yaml specs on GKE non-sandboxed cluster (using runc) and things worked. client was able to talk to the server. I did not have to modify any firewall rules either. i'll spin up a sandboxed cluster again and try to capture more info.
@sdeoras if you could come up with repro steps it would help immensely. You can also file a bug in https://github.com/google/gvisor/issues/new
hi @fvoznika here is a quick doc outlining steps to repro the issue. Thanks again for taking a look. https://gist.github.com/sdeoras/42542c1911a93e8d696f9ddad94887e6
Let continue the discussion here: google/gvisor#807
I think that it will be a few months before everything is fully landed
The hook is called from outside the sandbox, so it does see all files in the host.
Community meeting tonight @ 5PM PST (tomorrow morning in Asia). Check the calendar for details: https://gvisor.dev/docs/community/
downloading from the release page (https://gvisor.dev/docs/user_guide/docker/#install-gvisor) , i get
$ sha512sum runsc
dc75a46e2d02809839a87bae0c0bc2a392743990b426b1bfb6987cd6eeef718fe0deb3c8dfc1b5dd5db6ab22d2191e0c1f6e5cdfb7eb0306e192c08a65fd284a runsc
$ cat runsc.sha512
23fdd694970e7a51693ca1c28000cb7394779b42a0432fdc5dc022e17312e04c11091bcfe8eda6eb0d9758de40876ceb28f2759ecac4f2455483925d9965bbc9 runsc
I'll try to see what's up with that.