opam 2.4.0 was released on 18th July followed by opam 2.4.1 a few days later. This update needs to be propagated through the CI infrastructure. The first step is to update the base images for each OS.
Linux
ocurrent/docker-base-images
The Linux base images are created using the Docker base image builder, which uses ocurrent/ocaml-dockerfile to know which versions of opam are available. Kate submitted PR#235 with the necessary changes to ocurrent/ocaml-dockerfile. This was released as v8.2.9 under PR#28251.
With v8.2.9 released, PR#327 can be opened to update the pipeline to build images which include opam 2.4. Rebuilding the base images takes a good deal of time, particularly as it’s marked as a low-priority task on the cluster.
macOS
ocurrent/macos-infra
Including opam 2.4 in the macOS required PR#56, which adds 2.4.1 to the list of opam packages to download. There are Ansible playbooks that build the macOS base images and recursively remove the old images and their (ZFS) clones. They take about half an hour per machine. I run the Intel and Apple Silicon updates in parallel, but process each pool one at a time.
The Ansible command is:
ansible-playbook update-ocluster.yml
FreeBSD (rosemary.caelum.ci.dev)
ocurrent/freebsd-infra
The FreeBSD update parallels the macOS update, requiring that 2.4.1 be added to the loop of available versions. PR#15.
The Ansible playbook for updating the machine is named update.yml. However, we have been suffering from some reliability issues with the FreeBSD worker, see issue#449, so I took the opportunity to rebuild the worker from scratch.
The OS reinstallation is documented in this post, and it’s definitely worth reading the README.md in the repo for the post-installation steps.
Windows (thyme.caelum.ci.dev)
ocurrent/obuilder
The Windows base images are built using a Makefile which runs unattended builds of Windows using QEMU virtual machines. The Makefile required PR#198 to The command is make windows.
Once the new images have been built, stop ocluster worker and move the new base images into place.
The next is to remove results/* as these layers will link to the old base images, and remove state/* so obuilder will create a new empty database on startup. Avoid removing cache/* as this is the download cache for opam objects.
The unattended installation can be monitored via VNC by connecting to localhost:5900.
OpenBSD (oregano.caelum.ci.dev)
ocurrent/obuilder
The OpenBSD base images are built using the same Makefile used for Windows. There is a seperate commit in PR#198 for the changes needed for OpenBSD, which include moving from OpenBSD 7.6 to 7.7. Run make openbsd.
Once the new images have been built, stop ocluster worker and move the new base images into place.
The next is to remove results/* as these layers will link to the old base images, and remove state/* so obuilder will create a new empty database on startup. Avoid removing cache/* as this is the download cache for opam objects.
As with Windows, the unattended installation can be monitored via VNC by connecting to localhost:5900.
OCaml-CI
OCaml-CI uses ocurrent/ocaml-dockerfile as a submodule, so the module needs to be updated to the released version. Edits are needed to lib/opam_version.ml to include V2_4, then the pipeline needs to be updated in service/conf.ml to use version 2.4 rather than 2.3 for all the different operating systems. Linux is rather more automated than the others
Lastly, since we now have OpenBSD 7.7, I have also updated references to OpenBSD 7.6. PR#1020.
opam-repo-ci
opam-repo-ci tests using the latest tagged version of opam, which is called opam-dev within the base images. It also explicitly tests against the latest release in each of the 2.x series. With 2.4 being tagged, this will automatically become the used dev version once the base images are updated, but over time, 2.4 and the latest tagged version will diverge, so PR#448 is needed to ensure we continue to test with the released version of 2.4.