The only other solution I can think of would be to install TeX Live outside of the Docker container, then start
FROM scratch and copy the TeX Live tree inside. Then layer the Debian image on top of that.
Also users can just run
tlmgr update inside of their local copy.
1 reply
--- a/Dockerfile.latest
+++ b/Dockerfile.latest
@@ -45,7 +45,7 @@ RUN echo "Building with documentation: $DOCFILES" && \
# actually install TeX Live
cd install-tl* && \
# choose complete installation
- echo "selected_scheme scheme-full" > install.profile && \
+ echo "selected_scheme scheme-infraonly" > install.profile && \
# … but disable documentation and source files when asked to stay slim
if [ "$DOCFILES" = "no" ]; then echo "tlpdbopt_install_docfiles 0" >> install.profile && \
echo "BUILD: Disabling documentation files"; fi && \
@@ -60,6 +60,12 @@ RUN echo "Building with documentation: $DOCFILES" && \
# add all relevant binaries to the PATH
$(find /usr/local/texlive -name tlmgr) path add
+RUN tlmgr install scheme-minimal
+RUN tlmgr install scheme-basic
+RUN tlmgr install scheme-small
+RUN tlmgr install scheme-medium
+RUN tlmgr install scheme-full
+
RUN \
# test the installation
latex --version && printf '\n' && \
This is what I have in mind.
1 reply
The schemes
context, gust, and tetex can be ignored I think.
The
scheme-gust is specific to GUST I guess and users of ConTeXt are anyway much better off using the ConTeXt beta images.
tetex doesn't exist anymore (it was the predecessor of TeX Live) but if necessary this scheme would go between medium and full.
Hello. When can we expect TL2021-historic images on GitLab and DockerHub?
1 reply
I see. That's an annoyance to be sure! Have you tried passing --force-rm and --no-build-cache to docker build?
1 reply
It makes
docker build more aggressive about the layers it keeps when building. Please, let me know if it helps. 🙂
Unfortunately, it does not work. GL runners are simply not suited anymore. I now built the image locally and will push it from here. But only for
TL2021-historic (no doc, no src).
I would be happy to discuss why it does not work and how we could make it work, but perhaps in a separate thread, so that we don't spam others?
1 reply
It's only on DockerHub, though. Pushing to GitLab failed.
That is unfortunate, but also much appreciated. I will create a TL2021 release of witiko/markdown right away.
Hi friends, I just want to say thank you to Ben and Vít for taking the time to look at the Docker images! ❤️
:point_up: Edit: That is unfortunate, but also much appreciated. I will create a TL2021 release of witiko/markdown right away.
https://github.com/Witiko/markdown/actions/runs/2289199925
https://github.com/Witiko/markdown/actions/runs/2289199925
Hi guys, do you have someone to host gitlab runner already or not? If you don't, I could provide a VM on my desktop (VM params: 4 cores, 16GB ram, 60gb nvme) right now to build what is needed. I know it is a bad solution (since it would be up only 18 hours per day), but this is the quickest help I can offer...
1 reply
I've recently read on the TeX Live list that you are having problems with being time-limited by GitLab CI for building the images. Have you considered inverting the build for the historic images? Since the entire TeX Live tree is frozen for them it would make sense to “install” TeX Live (potentially in portable mode) outside of Docker into a directory, then start
FROM scratch, copy that directory into the image and only as a last step layer the operating system on top. The initial layer containing nothing but the historic TeX Live will be perfectly deterministic (by virtue of being frozen) so you get perfect layer reuse and OS updates on top can be done in a matter of seconds.
1 reply
At the moment, we have a couple of runners and mounting a mirror of CTAN is indeed one of the options on the table.
However, it's less reproducible than doing it inside docker, so that's that. Also does not scale as well to more runners.
If anyone is interested: https://hub.docker.com/r/texlive/texlive/tags, the
TL<YEAR>-historic-tree* tags are those that contain only the TeX Live trees (for Linux x86-64). We have not publicly announced the names yet but probably they will stay as is.
We discussed it with Ben Frank two weeks ago: https://gitlab.com/islandoftex/images/texlive/-/issues/5#note_959539914
Apparently, we have everything in place. With !22 merged, it's just a matter of carving out a couple of hours to make an MR.
1 reply
It still seems to me that incremental updates are a more pressing issue from the standpoint of a user.
Yes, but incremental updates combined with incremental split into schemes would be very bad layering resulting in explosion of pulled image size.
I guess incremental updates are quite a strong argument for the independent scheme images. Probably users will not switch that often between schemes so it probably makes sense.
It would seem that way. We cannot be incremental both across time and across schemes.
And it seems to me that most users will be pulling a single scheme repeatedly.
So each scheme would be its own set of layers corresponding to incremental updates in time, independent on other schemes.
What we could do is this: we could build the root image for each year from the smaller schemes.
3 replies
But they would be separate from then on.
This would likely solve most of the duplication issues, since the diff over a year will be a drop in a bucked t compared to what we already have.
Not sure what we would name them, probably tree-something, but there would be several, one for each scheme. They could share layers.
3 replies
On top of them would be layers with weekly updates.
islandoftex/images/texlive#11 is what I had in mind after all. Simply build the images per scheme which would be one
matrix entry more in the CI file. And as I said, the update strategy as per the other issue would be orthogonal to that split.
And the tree-* images, which we would build at the beginning of TL's annual cycle could be build on top of each other (such as tree-full using tree-minimal).
The latest-* images would not care and could be just a matrix entry as you say.
This way, we get rid of most of the duplication, get incremental updates in time and we get Debian's security updates.
Definitely, let's break these changes into small bits. It's the only way any of this gets done. 👍️
1 reply
Not a problem. Can make a tikz version too if we'd like to have it in the TUG 2022 paper.
Oh, you should ask Paulo Cereda about that one, he is our chief of papers and presentations. But maybe he'd like it in the presentation as well 😄
1 reply
What I don't entirely understand is this, because I never cared much about schemes: is there a nice division of packages into schemes without much duplication? Because is seems to me that schemes overlap and for example Debian packagers have to completely reorder things to get rid of duplication.
1 reply
Ah.
Would it make sense to provide images for collections rather than schemes?