Kernel Module Management Application¶

The app-kernel-module-management application enables dynamic loading of out-of-tree kernel modules at runtime, with modules distributed independently of the platform. Kernel modules can be optionally built on demand into container images or provided as pre-built container images. These images are used to deploy and load the required modules onto target Kubernetes nodes. Custom Kubernetes resources are used to define which kernel modules to load, from which container images, and the specific nodes on which the modules should be deployed.

Install App-kernel-module-management Application¶

Perform the following steps to install the app-kernel-module-management application.

Procedure

Upload the application package by running the following command:

~(keystone_admin)]$ system application-upload /usr/local/share/applications/helm/kernel-module-management-*.tgz

Create and apply the Helm overrides.

The kernel-module-management application requires a Docker registry to manage the module images it builds. The application will use this registry to pull and push the images.

The registry may be an external or a local registry, as shown in the following example:

Set the Docker credentials.

USERNAME="sysinv"
PASSWORD=$(keyring get sysinv services)
DOCKER_CREDENTIALS=$(echo -n "${USERNAME}":"${PASSWORD}" | base64)

Create docker-config.json.

cat >docker-config.json<<EOF
{
  "auths": {
      "https://registry.local:9001": {
         "auth": "$DOCKER_CREDENTIALS"
      }
  }
}
EOF

dconfigjson=$(cat docker-config.json | base64 -w 0)

Create the override file with registry information.

cat >kmm-app-override.yaml<<EOF
dockerRegistrySecretName: "kmm-registry-secret"
dockerConfigJson: "$dconfigjson"
EOF

Apply the overrides.

~(keystone_admin)]$ system helm-override-update kernel-module-management kernel-module-management kernel-module-management --values kmm-app-override.yaml

After uploading the application and applying the overrides, apply the application using the following command.

~(keystone_admin)]$ system application-apply kernel-module-management

After the application is successfully applied, the two pods will be running as shown below:

(keystone_admin)]$ kubectl get pods -n kernel-module-management
NAME                                       READY   STATUS    RESTARTS   AGE
kmm-operator-controller-86dfc6bff8-swvf7   1/1     Running   0          97s
kmm-operator-webhook-788f76bd7c-64t6t      1/1     Running   0          97s

Building and Loading Kernel Modules¶

Create Kernel Module file¶

To build and load a kernel module, you need to have a ConfigMap and a CRD module.

Create the ConfigMap module.

cat << 'EOF' > hello_world_cm.yaml
apiVersion: v1
kind: ConfigMap
metadata:
  name: kmm-hello-world-cm
  namespace: kernel-module-management
data:
  dockerfile: |
    FROM docker.io/starlingx/kmm-builder:stx.12.0-v1.0.0 as builder
    ARG KERNEL_FULL_VERSION
    RUN make KERNEL_DIR=/lib/modules/${KERNEL_FULL_VERSION}/build && \
        mkdir -p /opt/lib/modules/${KERNEL_FULL_VERSION} && \
        cp -v ./* /opt/lib/modules/${KERNEL_FULL_VERSION}
    RUN depmod -b /opt ${KERNEL_FULL_VERSION}
EOF

Create the CRD module.

cat << 'EOF' > hello_world_mod.yaml
apiVersion: kmm.sigs.x-k8s.io/v1beta1
kind: Module
metadata:
  name: kmm-hello-world
  namespace: kernel-module-management
spec:
  moduleLoader:
    container:
      modprobe:
        moduleName: hello_world_dmesg
      kernelMappings:
        - literal: "6.12.0-1-amd64"
          containerImage: registry.local:9001/kmm/kmm-hello-world:amd64-hw
          build:
            buildArgs:
              - name: KERNEL_FULL_VERSION
                value: "6.12.0-1-amd64"
            baseImageRegistryTLS:
              insecure: false
              insecureSkipTLSVerify: true
            dockerfileConfigMap:
              name: kmm-hello-world-cm
          registryTLS:
            insecure: false
            insecureSkipTLSVerify: true

        - literal: "6.12.0-1-rt-amd64"
          containerImage: registry.local:9001/kmm/kmm-hello-world:rt-amd64-hw
          build:
            buildArgs:
              - name: KERNEL_FULL_VERSION
                value: "6.12.0-1-rt-amd64"
            baseImageRegistryTLS:
              insecure: false
              insecureSkipTLSVerify: true
            dockerfileConfigMap:
              name: kmm-hello-world-cm
          registryTLS:
            insecure: false
            insecureSkipTLSVerify: true

  imageRepoSecret:
    name: "kmm-registry-secret"

  selector:
    kubernetes.io/os: linux
    kubernetes.io/hostname: controller-0

  tolerations:
    - key: "services"
      operator: "Equal"
      value: "disabled"
      effect: "NoExecute"

EOF

Build and Load Modules¶

To run the kernel module in the same path where both the yaml files were created, run the following command:

~(keystone_admin)]$ kubectl apply -f hello_world_cm.yaml -f hello_world_mod.yaml

The kernel module build progress can be tracked using the kubectl logs command.

~(keystone_admin)]$ kubectl logs -n kernel-module-management kmm-hello-world-build-<POD_ID> -f

After successfully building the kernel module, confirm that the module is correctly loaded by running the following dmesg:

sudo dmesg

[34352.320647] systemd-sysv-generator[2076803]: Overwriting existing symlink /run/systemd/generator.late/radosgw.service with real service.
[102022.711368] e1000: enp0s3 NIC Link is Down
[102028.836500] e1000: enp0s3 NIC Link is Up 1000 Mbps Full Duplex, Flow Control: RX
[102422.922297] e1000: enp0s3 NIC Link is Down
[102422.922706] e1000 0000:00:03.0 enp0s3: Reset adapter
[102425.122913] e1000: enp0s3 NIC Link is Up 1000 Mbps Full Duplex, Flow Control: RX
[102582.800513] e1000: enp0s3 NIC Link is Down
[102586.903818] e1000: enp0s3 NIC Link is Up 1000 Mbps Full Duplex, Flow Control: RX
[103642.175972] Hello, world!

Unload Kernel Module¶

To unload the kernel module, run the following command in the same path where the previous module files were created:

~(keystone_admin)]$ kubectl delete -f hello_world_cm.yaml -f hello_world_mod.yaml

Confirm that the kernel module was successfully unloaded by running dmesg again and checking for the Goodbye message logged.

sudo dmesg

[102425.122913] e1000: enp0s3 NIC Link is Up 1000 Mbps Full Duplex, Flow Control: RX
[102582.800513] e1000: enp0s3 NIC Link is Down
[102586.903818] e1000: enp0s3 NIC Link is Up 1000 Mbps Full Duplex, Flow Control: RX
[103642.175972] Hello, world!
[104469.925530] Goodbye, world!

Uninstall App-kernel-module-management Application¶

To uninstall the app-kernel-module-management application, run the following command:

~(keystone_admin)]$ system application-remove kernel-module-management

Delete the uploaded application by running the following command:

~(keystone_admin)]$ system application-delete kernel-module-management