Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Activity
    Adin Scannell
    @amscanne
    (It might be something simple, but it will help me to understand)
    FullPlus
    @WhoisZihan
    A quick example might be the dirtySet. Currently, the information is stored in CachingInodeOperations, and the pgalloc might need to release a page that belongs to dirty set. So pgalloc must be able to track the page to inode, and finds a way to flush it, and the flushing involves the dirty sets in inode. So if we want to do that in pgalloc, we should allow pgalloc to have such information.
    Besides, if pgalloc frees page, but inode.cache still has it in the cacheSet
    Then further access to this page that causes a "cache hit" in inode level, is actually not a real cache hit, because we need to read the page back into cache from backing file
    Adin Scannell
    @amscanne
    So pgalloc calls the Evict function via the EvictableMemoryUser interface .. I’m just refreshing my full understanding of this interaction..
    FullPlus
    @WhoisZihan
    Ok, i wasn't quite relying on the the evict interface. Then it seems implementation in pgalloc is doable, will change it in next version.
    FullPlus
    @WhoisZihan
    So after looking at Evict interface again, I think I understand the interaction here. To periodically flush dirty memory, the encouraged way in gvisor is to pass something like DirtyRange to pgalloc, and start per-memoryfile flusher goroutine. But is it a good idea to have more than 1 flush routines?
    Apart from the implementation choice, do you have comments on the idea itself? Do you want lru as an eviction strategy in gvisor?
    Adin Scannell
    @amscanne
    Hm, there’s really just one pgalloc.MemoryFile, why does it imply more than one routine?
    (I will be back shortly)
    FullPlus
    @WhoisZihan
    Will there be only one MemoryFile in all situations? In my experiments there is only one, but I'm not sure if it stays the same in all conditions. If yes, then it's good.
    OK
    Adin Scannell
    @amscanne
    Well, I don't think you can assume there's one, because the design should be modular and capable of supporting multiple instances (e.g. it might be the case for tests), but in practice I think there would be a single instance
    there's already a goroutine for each pgalloc MemoryFile which is the reclaimer
    for evictable memory at the moment it essentially evicts everything when needed (https://github.com/google/gvisor/blob/master/pkg/sentry/pgalloc/pgalloc.go#L1147)
    The existing EvictableMemoryUser interfaces are probably sufficient, and you'll actually want to do a few things:
    Adin Scannell
    @amscanne
    • Extend the pgalloc options to accept some alternate parameters that align with your needs, e.g. an evictable memory target (as % of committed?), maximum, etc.
    • Implement the LRU mechanism for pgalloc on the basis when the range was called with MarkEvictable. (You could probably add an additional Touch method that brings things to the front of the list; that would be called by e.g. write and read).
    • Introduce additional behavior to the reclaimer goroutine such that it can enforce the provided options. Essentially you would have a startEvictionsLocked evict a specific amount instead of just "everything".
    Adin Scannell
    @amscanne
    • With this is place, it might also be feasible to extend the EvictableMemoryUser to support a "Flush(...)" interface in addition to the "Evict(...)" call, which would just flush dirty pages but not evict. You could add a separate pgalloc goroutine that does appropriate flushing based on the provided targets. I think this is a somewhat distinct independent feature though.
    Anyways, I definitely want @nixprime's opinion here, but that's my view. I think that essentially pgalloc has a super coarse eviction policy now, and making that smarter with the existing eviction interfaces is the way you want to go
    FullPlus
    @WhoisZihan
    Thanks for your suggestion, very clear and practical.
    FullPlus
    @WhoisZihan
    Is there any new message about VFS2 and ext4? Will there be cache design in VFS2?
    Adin Scannell
    @amscanne
    I don't believe the cache implementation will be changed with VFS2 directly, I suspect that we will update/migrate the CachingInode implementation when the vfs2 switch occurs and use that to wrap ext but not sure (CC @nlacasse @kevinGC)
    Saurabh Deoras
    @sdeoras
    Hi Folks, I am trying out gvisor on GKE and see that pods on two different nodes (in gvisor node pool) can't communicate with each other over ClusterIP. Is this expected behavior on GKE or a limitation of gvisor? I have been able to get pods to communicate running in gVisor on other cluster (GCE VM's) that was brought up manually using kubeadm. Thanks.
    Adin Scannell
    @amscanne
    @fvoznika is just looking into this now, seeing if he can reproduce the behavior
    Fabricio Voznika
    @fvoznika
    I just a quick test using httpd and I was able to reach pods both ways (from gVisor pool to default pool and vice-versa). Can you share a bit more about your configuration?
    Saurabh Deoras
    @sdeoras
    @fvoznika, thanks for checking. i have a simple grpc server/client setup and all it does is send messages back and forth. I tried running server as a deployment and exposed it as ClusterIP and client runs as a Job, both in gvisor. Server deployment comes up fine, however, client is unable to talk to server. I also tried the same using runc and it is complaining that udp connection was refused. It seems maybe some problem with my GKE networking setup.
    Adin Scannell
    @amscanne
    @fvoznika Isn’t it two nodes in the gVisor pool? I’m not sure the repro was the same as the described problem
    Fabricio Voznika
    @fvoznika
    Ah, connection between pods in different nodes of the same node pool certainly works.
    Saurabh Deoras
    @sdeoras
    hi folks, something is still not quite right. i ran the same yaml specs on GKE non-sandboxed cluster (using runc) and things worked. client was able to talk to the server. I did not have to modify any firewall rules either. i'll spin up a sandboxed cluster again and try to capture more info.
    Fabricio Voznika
    @fvoznika
    @sdeoras if you could come up with repro steps it would help immensely. You can also file a bug in https://github.com/google/gvisor/issues/new
    Saurabh Deoras
    @sdeoras
    @fvoznika let me put together repro steps later today. thanks
    Saurabh Deoras
    @sdeoras
    hi @fvoznika here is a quick doc outlining steps to repro the issue. Thanks again for taking a look. https://gist.github.com/sdeoras/42542c1911a93e8d696f9ddad94887e6
    Fabricio Voznika
    @fvoznika
    Thanks for the detailed steps. It worked for my on my first try, but I used an old cluster I had. I'll try again with a new cluster using the same config as yours.
    Let continue the discussion here: google/gvisor#807
    tanjianfeng
    @tanjianfeng
    Hi folks, is there a timeline for vfs2 (Which Jamie Liu, etc, works on)?
    Adin Scannell
    @amscanne
    The core is in, we’ve started work on the filesystems (In fs/impl) and then will transition
    I think that it will be a few months before everything is fully landed
    tanjianfeng
    @tanjianfeng
    Good to know that, thanks a lot, @amscanne
    tanjianfeng
    @tanjianfeng
    Hi folks, anyone uses the prestart hooks in gVisor? Refer to the implementation: runsc/container/hook.go:executeHooks(), it seems that it shall not work, as it uses standard go library to fork process for commands and sandbox process shall not see any files.
    Fabricio Voznika
    @fvoznika
    Yes, docker uses prestart hooks to setup the network namespace for the container.
    The hook is called from outside the sandbox, so it does see all files in the host.
    Zach Koopmans
    @madscout12
    Community meeting tonight @ 5PM PST (tomorrow morning in Asia). Check the calendar for details: https://gvisor.dev/docs/community/
    tanjianfeng
    @tanjianfeng
    @fvoznika Ah yes, after re-read the code, the hook is indeed called from outside the sandbox.
    Brian King
    @kingishb
    :wave: Can someone double check if the sha512sum and the release are matching?
    downloading from the release page (https://gvisor.dev/docs/user_guide/docker/#install-gvisor) , i get
    $ sha512sum runsc
    dc75a46e2d02809839a87bae0c0bc2a392743990b426b1bfb6987cd6eeef718fe0deb3c8dfc1b5dd5db6ab22d2191e0c1f6e5cdfb7eb0306e192c08a65fd284a  runsc
    
    $ cat runsc.sha512
    23fdd694970e7a51693ca1c28000cb7394779b42a0432fdc5dc022e17312e04c11091bcfe8eda6eb0d9758de40876ceb28f2759ecac4f2455483925d9965bbc9  runsc
    Michael Pratt
    @prattmic
    @kingishb you're right, I get the same result as you.
    I'll try to see what's up with that.