Meet cl-lxc for container management!

Forum Announcements
,
1

LXC containers have been with us for a while now: indeed, we have dozens of them at Calculate Ltd. Containers can be quickly and easily deployed and configured, but further maintenance may become a challenge. Updating the services over time requires newer versions of libraries, which in turn means upgrading obsolete systems. And that is when, of course, one realizes that the container has not been updated for quite a long period! Doing it after all this time may disrupt normal service operation in the container, or else cause unpredictable results, as all packages are to be updated simultaneously. Another problem is the disk space, considering that each container stores a copy of Portage and downloadable packages.

We disregarded this at first, since LXC uses its native tools for deployment and configuration, until the problem became time consuming. And that’s really not what we are striving for. Therefore, here is the new cl-lxc utility. It is written in BASH and provided by sys-apps/calculate-toolkit since version 0.3.0.

Features

  • Update from an image file
  • Less disk memory usage
  • Btrfs support

cl-lxc updates containers in the same way we do it elsewhere on Calculate Linux, that is, from an image file. /var/calculate must be a separately mounted partition to make this possible. First it prepares the image file: deploys and updates the container, adds the packages that were installed in the system to be updated, etc. A restart follows, that replaces the root of the system. All data remains in /var/calculate, while all settings are applied by templates when booting for the first time into the new system.

Cl-lxc uses cache, thus allowing you to deploy multiple containers in no time. The image to be used for update is cached and can be used for further updates as well. The containers are configured to use Portage as on the host system. As a result, a Calculate Container Scratch takes only 1.2 G. If Btrfs is used, the tool will run reflink when copying and create a subvolume for new containers. Unlike lxc-create, cl-lxc creates a subvolume for the container directory, not for rootfs. It allows to keep multiple root system images.

Installation

Make sure that you have installed sys-apps/calculate-toolkit 0.3.0 or higher.

Install the LXC tools:

emerge -a app-containers/lxc

Host configuration

Containers configured with cl-lxc mounts Portage and the overlays from the host, but the default Container overlay is absent from the host server. For an overlay to be updated in the container, it must be added to the update list. To do so, edit /var/calculate/calculate.env:

[update]
cl_update_rep_name = gentoo,calculate,distros,container
cl_update_rep_url = https://git.calculate-linux.org/calculate/gentoo-overlay,https://git.calculate-linux.org/calculate/calculate-overlay,https://git.calculate-linux.org/calculate/distros-overlay,https://git.calculate-linux.org/calculate/container-overlay

To make sure that the Container overlay is active, synchronize Portage by running cl-update --sync-only:

Screenshot_2022-03-21_18-13-46
Screenshot_2022-03-21_18-13-46742×458 28.7 KB

Creating a container

Run cl-lxc --create test to create a ‘test’ container:

Снимок экрана_2022-03-21_18-29-45
Снимок экрана_2022-03-21_18-29-45742×459 27.4 KB

The /var/calculate/lxc path will be created, containing the directory provided by the test container.
You can now start the container:
lxc-start test

Connect to the container:
lxc-attach test

Screenshot_2022-03-21_18-32-51
Screenshot_2022-03-21_18-32-51742×459 32.6 KB

Note that the root prompt appears in a different color in the container.

If no network bridge has been configured on the host, the network settings will be inherited by the container.

Update

Besides the usual cl-update or emerge method of update, you can update from an image file. When preparing a new image, cl-lxc takes up the ‘custom’ set (/etc/portage/sets/custom) from the container. Therefore, put packages in this set when you install them. In addition to convenient porting, it also helps knowing which packages were installed and why.

For example, let’s install Vim:

Screenshot_2022-03-21_18-34-21
Screenshot_2022-03-21_18-34-21742×584 96.5 KB

Now you need to prepare the image file to update from. Please run cl-lxc --prepare test on the host.

Screenshot_2022-03-21_18-41-11
Screenshot_2022-03-21_18-41-11742×637 80.7 KB

A temporary container will be created, for the custom set to be moved into, and an update performed. Then it will be cached in /var/calculate/lxc/.cache/rootfs-test and removed.

All you need to do now to update your container is running cl-lxc --upgrade test:

Screenshot_2022-03-21_18-54-46
Screenshot_2022-03-21_18-54-46742×422 26.3 KB

It only takes seconds. /var/calculate/lxc/test contains the config file, storing the container settings, the calculate directory mounted at /var/calculate in the container and two system images with the rootfs and rootfs1 root directories respectively. Once it has been tested, rootfs can be deleted.

Note that the network, language and locale settings, as well as the package list, were migrated from the updated system to the new one at the update time. Later on, settings can be migrated using templates, while the actual data can be stored in /var/calculate.

Templates

As you can see in the examples, setting up a container, preparing and updating the system image is fast and simple. You will still have your data in a dedicated mountable directory, /var/calculate. Now what about templates, so that you can migrate settings between systems?

When preparing the container, basic templates for binary update server configuration are generated automatically at calculate/templates. Let’s take a closer look at them.

b0f696dfb9f65b94eafbbcb3ae85bafd63dbfca0
b0f696dfb9f65b94eafbbcb3ae85bafd63dbfca0742×459 14.3 KB

For convenience, all basic configuration templates are stored in the ‘default’ directory. Its properties are described in the .calculate_directory file:

# Calculate env=install ac_install_live==on append=skip

where:

  • env=install describes the variable scope
  • ac_install_live==on checks for package installation event. Here’s to learn more about this parameter: /var/db/repos/calculate/profiles/templates/3.6/3_ac_install_live/README-en.txt.
  • append=skip skips a directory when migrating to the system that is being configured

The portage.binhost template looks somewhat like this:

# Calculate mergepkg(sys-apps/portage)!= path=/etc/portage/make.conf name=binhost protected comment=#
PORTAGE_BINHOST="https://mirror.calculate-linux.org/grp/x86_64"

where:

  • mergepkg(sys-apps/portage)!= tells to check for sys-apps/portage
  • path=/etc/portage/make.conf name=binhost is the path and name of the file to be edited
  • protected tells not to link the generated file to sys-apps/portage
  • comment=# is the comment separator

The runlevel.eth0 template is created if a bridge has been setup on the host. Here are its contents:

# Calculate mergepkg(sys-apps/openrc)!= path=/etc/runlevels/default name=net.eth0 protected link=/etc/init.d/net.lo symbolic

where:
link=/etc/init.d/net.lo symbolic stands for creating a symlink pointing to /etc/init.d/net.lo.

Conclusions

I give no examples of customization by templates in the article. Maybe later, if you’re interested? Setting up a container with templates seems easier because there is no risk of corrupting the underlying system files, and the container filesystem can be quickly restored from the cache.