Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Activity
    Vít Novotný
    @witiko:matrix.org
    [m]
    Those are good points, especially the one about private details about runners that we may not want to have public. GitLab issues seem as a good place to have well thought-out discussions for archival after brainstormong on Matrix.
    Vít Novotný
    @witiko:matrix.org
    [m]
    Oh, and I do appreciate your patience with me in our exchanges. 🙂
    Vít Novotný
    @witiko:matrix.org
    [m]
    While drawing, I realized that there is just no way. We can have layered tree- images for the schemes, but this layering would be lost the moment we copy the files over from the tree- images to the latest-* images.
    1 reply
    Not all is lost.
    We could still layer the schemes.
    Ben Frank
    @benfrank:matrix.org
    [m]
    But for what benefit?
    Vít Novotný
    @witiko:matrix.org
    [m]
    For none in the way I drew in the above figure.
    However, if we dispense with the tree- image and keep just the latest- image, which is the way Dockerfile works now, we could layer the schemes and get more layer-friendly images I think. Back to the drawing board.
    Ben Frank
    @benfrank:matrix.org
    [m]
    Yep, that's basically what Henri suggested 😄
    But the benefits of layer-friendliness is lost as soon as updates come into play.
    Vít Novotný
    @witiko:matrix.org
    [m]
    I'd like to show that they are not.
    Vít Novotný
    @witiko:matrix.org
    [m]
    Here, you get layer friendliness across both axes.
    Ben Frank
    @benfrank:matrix.org
    [m]
    Nope, you don't. After one weekly update the basic layer is not related to the minimal layer anymore.
    Vít Novotný
    @witiko:matrix.org
    [m]
    Okay, so the week 0 latest-minimal image has the base and tlmgr install scheme-minimal layers, right?
    1 reply
    Then the week 1 latest-minimal image has the base, tlmgr install scheme-minimal, and tlmgr update layers.
    1 reply
    You go up and then right.
    1 reply
    Each week, the image grows one layer to the right.
    Right, we don't update the base image in what I drew.
    Ben Frank
    @benfrank:matrix.org
    [m]
    It's not about the base image!
    Vít Novotný
    @witiko:matrix.org
    [m]
    Ah, yes. Since week 1, the updates for the different schemes are unrelated, you are right.
    But here is the kicker:
    Ben Frank
    @benfrank:matrix.org
    [m]
    Your whole up-arrow is only valid in the first week. And otherwise you only get the right-arrows. Hence, we can save us the trouble of doing step-wise installation in week 0 in the first place.
    Vít Novotný
    @witiko:matrix.org
    [m]
    The deduplication saves our ass precisely in week 0. The size of updates over the year is negligible compared to the first installation.
    1 reply
    So perhaps we don't care so much that the updates to the different schemes are unrelated after week 1, inclusive.
    1 reply
    It won't necessarily be easy to set up, but I assure you that the updates over the year are more than a factor of ten smaller compared to the initial download of TeX Live in week 0, which is precisely why it's crucial to deduplicate in week 0 by layering the schemes but not necessarily in the following weeks. It's easy to demonstrate that by downloading a TL 2021 ISO and then updating it.
    2 replies
    Vít Novotný
    @witiko:matrix.org
    [m]
    Yeah, we would probably need a separate Dockerfile for the week 0 seed image.
    And there are other concerns, too. It's not ideal to use a year-old base image, so perhaps we want to at least apply security updates once every N weeks.
    3 replies
    Ben Frank
    @benfrank:matrix.org
    [m]
    Actually, we don't. We take the current Dockerfile. We need a new Dockerfile.update which basically does:
    ARG image
    FROM $image
    
    RUN tlmgr update --self --all --reinstall-previously-removed
    Vít Novotný
    @witiko:matrix.org
    [m]
    Right, the current Dockerfile would be the separate image.
    I can run the numbers and we can decide afterwards if we think it is worth it.
    1 reply
    :point_up: Edit: The deduplication saves us precisely in week 0. The size of updates over the year is negligible compared to the first installation.
    We would lose the historical tree images but get historical schemes.
    3 replies
    Vít Novotný
    @witiko:matrix.org
    [m]
    They don't run on rsync, though.
    1 reply
    Or do they?
    They do.
    And yet, we still have #19 open.
    That confused me. 😉
    Ben Frank
    @benfrank:matrix.org
    [m]
    Oh, I thought I marked it to be closed by !23. Seems I did not. I'll close it.
    Vít Novotný
    @witiko:matrix.org
    [m]
    Feels good to close issues. 👍️
    1 reply
    No, see multi-arch builds are done in a single command. 🙂
    1 reply
    It would slow things down, but they are a single command that produces a single fatter image.
    Ben Frank
    @benfrank:matrix.org
    [m]
    TL installs are specific to the architecture so it will actually blow up size quite a bit.
    Vít Novotný
    @witiko:matrix.org
    [m]
    Just the binaries, but it would blow up the CI time quite a lot.
    The .gitlab-ci.yml file would still be nice and simple, though.
    Ben Frank
    @benfrank:matrix.org
    [m]
    Yes, of course.
    Vít Novotný
    @witiko:matrix.org
    [m]
    However, #15 has another big issue; specifically, some binaries such as biber are not available for ARM.
    Forget about Docker, I can't get biber working on my Raspberry Pi without compiling it myself.
    Ben Frank
    @benfrank:matrix.org
    [m]
    Right. But situation is improving for linuxmusl (i.e. alpine) iirc. So that could be a viable alternative because MUSL is self-contained and should also be able to be run on ARM.