Building Multiple Debian Releases from Master Branch

StarlingX supports building different Debian releases (Bullseye and Trixie) from the master branch. This allows development for newer Debian releases without requiring separate feature branches.

Overview

The build system has been enhanced to support concurrent development of multiple Debian releases:

  • Bullseye (Debian 11) - Current stable release

  • Trixie (Debian 13) - Next-generation release under development

Key benefits:

  • No need for separate feature branches

  • Minimal code duplication

  • Shared packaging where possible

  • Independent build environments for each release

Configuration Changes

stx.conf Updates

The configuration file has been updated with new parameters to support multiple OS releases:

Old Configuration:

dist=bullseye
stx_mirror_url=http://mirror.example.com/debian

New Configuration:

# Project settings
debian_distribution = bullseye    # or trixie
debian_version = 11.3             # or 13.0 for trixie

# Debian snapshot configuration
debian_snapshot_base = https://mirror.starlingx.windriver.com/mirror/debian/debian/snapshot.debian.org/archive/debian
debian_security_snapshot_base = https://mirror.starlingx.windriver.com/mirror/debian/debian/snapshot.debian.org/archive/debian-security
debian_snapshot_timestamp = 20220331T000000Z

# Builder settings
os_id = debian
os_codename = bullseye            # or trixie
os_arch = amd64                   # or arm64

# Mirror configuration (more flexible)
os_mirror_url = https://mirror.starlingx.windriver.com/mirror/
os_mirror_dist_path = debian/debian/
os_mirror_dl_path = debian/
lat_mirror_url = https://mirror.starlingx.windriver.com/mirror/
lat_mirror_lat_path = lat-sdk/

# Repository manager settings
stx_mirror_strategy = stx_mirror_first

Automatic Migration:

Pre-existing stx.conf files are automatically adapted to the new structure with default values. No manual intervention is required if you have been using default values.

Complete stx.conf.sample Structure

The complete stx.conf.sample file is organized into sections:

[project] Section:

buildbranch = master
debian_distribution = bullseye
debian_version = 11.3
debian_snapshot_base = https://mirror.starlingx.windriver.com/mirror/debian/debian/snapshot.debian.org/archive/debian
debian_security_snapshot_base = https://mirror.starlingx.windriver.com/mirror/debian/debian/snapshot.debian.org/archive/debian-security
debian_snapshot_timestamp = 20220331T000000Z
gitemail = stx.builder@opendev.org
gituser = STX Builder
manifest = default.xml
name = stx
ostree_osname = debian

[builder] Section:

os_id = debian
os_codename = bullseye
os_arch = amd64
myuname = builder
uid = 1000
debemail = stx.builder@opendev.org
debfullname = STX Builder
release = 6.0
stx_pkg_ext = .stx

[repomgr] Section:

os_mirror_url = https://mirror.starlingx.windriver.com/mirror/
os_mirror_dist_path = debian/debian/
os_mirror_dl_path = debian/
lat_mirror_url = https://mirror.starlingx.windriver.com/mirror/
lat_mirror_lat_path = lat-sdk/
stx_mirror_strategy = stx_mirror_first
origin = starlingx
type = aptly

See stx-tools/stx.conf.sample for the complete template with all available options.

File Structure and Naming Changes

Package Directory Lists

Package directory list files have been renamed to include the OS codename.

Old Names:

<git>/debian_pkg_dirs
<git>/debian_pkg_dirs_rt
<git>/debian_pkg_dirs_sign

New Names:

<git>/debian_bullseye_pkg_dirs_std
<git>/debian_bullseye_pkg_dirs_rt
<git>/debian_bullseye_pkg_dirs_sign
<git>/debian_trixie_pkg_dirs_std
<git>/debian_trixie_pkg_dirs_rt
<git>/debian_trixie_pkg_dirs_sign

Note

Legacy debian_pkg_dirs, debian_pkg_dirs_rt, and debian_pkg_dirs_sign locations are still supported for Bullseye builds.

Note

For details on build types (_std, _rt, _sign), layer configuration, and build order, see Debian Packaging Reference.

ISO Image Configuration

ISO image configuration files now include the OS codename:

Old:

cgcs-root/build-tools/stx/debian-bullseye-image.inc

New:

cgcs-root/build-tools/stx/debian-bullseye-image.inc  (unchanged for Bullseye)
cgcs-root/build-tools/stx/debian-trixie-image.inc    (new for Trixie)

Build Tools Directory Structure

Build tool files have been reorganized by OS codename:

Dockerfiles:

Old: stx-tools/stx/dockerfiles/bullseye/
New: stx-tools/stx/debian/bullseye/dockerfiles/
     stx-tools/stx/debian/trixie/dockerfiles/

Common Files:

Old: stx-tools/stx/toCOPY/
New: stx-tools/stx/debian/bullseye/toCOPY/
     stx-tools/stx/debian/trixie/toCOPY/

Mirror Configuration:

Old: stx-tools/debian-mirror-tools/config/debian/
New: stx-tools/debian-mirror-tools/config/debian/bullseye/
     stx-tools/debian-mirror-tools/config/debian/trixie/

Package Lists:

Old: stx-tools/debian-mirror-tools/config/debian/*/*bullseye*
New: stx-tools/debian-mirror-tools/config/debian/bullseye/*/*bullseye*
     stx-tools/debian-mirror-tools/config/debian/trixie/*/*trixie*

Package Debian Directory Structure

The packaging metadata directory structure has been enhanced to support release-specific and shared packaging.

Scenario 2: Release-Specific Packaging

When packaging differs between releases but source code is the same or has minimal differences:

<pkg>/
├── debian/
│   ├── bullseye/
│   │   ├── control
│   │   ├── rules
│   │   └── changelog
│   └── trixie/
│       ├── control
│       ├── rules
│       └── changelog
└── src/

Use conditional compilation or execution blocks for minor source code differences (see examples below).

Scenario 3: Fully Separated Packaging and Source

When both packaging and source code differ significantly:

Option A:

<pkg>/
├── debian/
│   ├── bullseye/
│   └── trixie/
└── src/
    ├── bullseye/
    └── trixie/

Option B:

<pkg>/
└── debian/
    ├── bullseye/
    │   ├── control
    │   ├── rules
    │   └── src/
    └── trixie/
        ├── control
        ├── rules
        └── src/

Warning

Avoid this scenario when possible. Significant duplication imposes long-term maintenance costs.

Special Case: Forward-Compatible Changes

If changes required for Trixie are also compatible with Bullseye, use Scenario 1 (shared packaging) even though changes are needed. This is common when:

  • New language features are introduced gradually across Debian releases

  • The package was coded to an older standard

  • Updates work on both releases

Detailed Packaging Guidance

This section provides detailed guidance on choosing the appropriate packaging approach based on the type and extent of changes between Debian releases.

Decision Tree

Use this decision tree to determine the appropriate packaging approach:

  1. No source code differences between releases?

    • Yes → Use Approach 3 (Scenario 1: Shared Packaging)

    • No → Continue to step 2

  2. Are changes minimal with no code duplication?

    • Yes → Use Approach 2 with conditional logic (Scenario 2)

    • No → Continue to step 3

  3. Can changes be isolated to patches?

    • Yes → Use Approach 2 with build-time patches

    • No → Continue to step 4

  4. Are changes extensive affecting most/all source files?

    • Yes → Use Approach 1 (Scenario 3: Fully Separated)

Approach 1: Fully Separated (Extensive Changes)

Use when changes are extensive and impact most or all source files (e.g., stx-puppet).

Directory Structure:

package_name/
└── debian/
    ├── bullseye/
    │   ├── deb_folder/
    │   ├── metadata.yaml
    │   └── src/
    └── trixie/
        ├── deb_folder/
        ├── metadata.yaml
        └── src/

When to Use:

  • Major refactoring between releases

  • Significant API changes

  • Different dependency trees

  • Extensive file-level changes

Limitations:

  • High maintenance overhead

  • Risk of divergence between releases

  • Duplicate bug fixes required

  • Increased repository size

Example Repositories:

  • stx-puppet (merged)

Approach 2: Shared Source with Release-Specific Packaging

Use when packaging differs but source code is the same or has minimal differences.

Directory Structure:

package_name/
├── debian/
│   ├── bullseye/
│   │   ├── deb_folder/
│   │   └── metadata.yaml
│   └── trixie/
│       ├── deb_folder/
│       └── metadata.yaml
└── src/

When to Use:

  • Different build dependencies

  • Different runtime dependencies

  • Minor source code differences

  • Release-specific build flags

Implementation Options:

Option 2a: Conditional Logic

For very minimal changes with no code duplication. Changes implemented using conditional logic.

Example - debian/rules:

#!/usr/bin/make -f

DEB_CODENAME := $(shell lsb_release -cs)

ifneq ($(filter trixie,$(DEB_CODENAME)),)
override_dh_auto_configure:
    ./configure --enable-trixie
else
override_dh_auto_configure:
    ./configure
endif

%:
    dh $@

Example - Makefile:

DEB_CODENAME := $(shell lsb_release -cs)

ifneq ($(filter trixie,$(DEB_CODENAME)),)
override_dh_auto_configure:
    ./configure --enable-trixie
else
override_dh_auto_configure:
    ./configure
endif

Example - Python:

def get_debian_codename():
    """
    Returns the Debian codename, e.g., 'bookworm', 'trixie', 'bullseye'.
    """
    # Preferred way: parse /etc/os-release
    try:
        with open("/etc/os-release") as f:
            lines = f.readlines()
            for line in lines:
                if line.startswith("VERSION_CODENAME="):
                    return line.strip().split("=")[1]
    except FileNotFoundError:
        return None
    return None

codename = get_debian_codename()
if codename == "trixie":
    # Trixie-specific implementation
    pass

Limitations:

  • Runtime overhead from conditional checks

  • Code complexity increases

  • Harder to test both paths

Option 2b: Build-Time Patches

For minimal changes that don’t impact the entire source code. Patches are created and applied during build time.

Example:

See: https://review.opendev.org/c/starlingx/update/+/975125

Limitations:

  • Patch maintenance overhead

  • Time-consuming to create and test patches

  • Not recommended for frequently updated repositories

  • Patches may conflict with upstream changes

Example Repositories:

  • config (sysinv) - Approach 2 (merged)

  • metal (rest) - Approach 2 (merged)

  • ha (service-mgmt-api) - Approach 2 (merged)

  • update - Approach 2 with patches (merged)

Migration Considerations

When migrating packages to support multiple releases:

Python Version Differences:

  • Bullseye (25.09): Python 3.9

  • Trixie (26.09): Python 3.12

Consider compatibility when backporting changes.

Ansible Syntax Changes:

Some Ansible playbook syntax changes were made due to version updates. These changes could be applied to Bullseye, but require testing.

Testing Requirements:

  • Test on both releases before merging

  • Verify conditional logic paths

  • Check for runtime performance impact

  • Validate patch application

Review Process:

All multi-OS packaging changes should be reviewed with the topic:

  • Upstream: topic:"stx-trixie-dir-change"

  • WRS: topic:"wrs-trixie-dir-change"

Detecting Debian Version in Build Code

When conditional logic is needed, detect the Debian version using lsb_release -cs.

In debian/rules

#!/usr/bin/make -f

DEB_CODENAME := $(shell lsb_release -cs)

ifneq ($(filter trixie,$(DEB_CODENAME)),)
override_dh_auto_configure:
    ./configure --enable-trixie-features
else
override_dh_auto_configure:
    ./configure
endif

%:
    dh $@

In Makefiles

DEBIAN_VERSION := $(shell lsb_release -cs)

ifeq ($(DEBIAN_VERSION),trixie)
    CFLAGS += -DDEBIAN_TRIXIE
else ifeq ($(DEBIAN_VERSION),bullseye)
    CFLAGS += -DDEBIAN_BULLSEYE
endif

all:
    gcc $(CFLAGS) main.c -o main

In Python Code

def get_debian_codename():
    """
    Returns the Debian codename, e.g., 'bullseye', 'trixie'.
    """
    try:
        with open("/etc/os-release") as f:
            for line in f:
                if line.startswith("VERSION_CODENAME="):
                    return line.strip().split("=")[1]
    except FileNotFoundError:
        pass
    return None

codename = get_debian_codename()

if codename == "trixie":
    # Trixie-specific code
    use_new_api()
elif codename == "bullseye":
    # Bullseye-specific code
    use_legacy_api()

In Shell Scripts

#!/bin/bash

DEBIAN_CODENAME=$(lsb_release -cs)

if [ "$DEBIAN_CODENAME" = "trixie" ]; then
    echo "Building for Trixie"
    # Trixie-specific commands
else
    echo "Building for Bullseye"
    # Bullseye-specific commands
fi

Developer Workflow

For Bullseye Developers

Minimal changes are required:

  1. Rebuild build containers after repo sync:

    cd stx-tools
./stx-init-env --rebuild

  2. Verify stx.conf (should be automatically updated):

    cat stx.conf | grep os_codename
# Should show: os_codename=bullseye

  3. Continue normal development workflow - no new command-line arguments required

  4. Legacy packaging locations (debian_pkg_dirs, <pkg>/debian/) continue to work

For Trixie Developers

Create a new, independent build environment:

  1. Create a new workspace directory:

    mkdir ~/starlingx-trixie
cd ~/starlingx-trixie

  2. Initialize repo:

    repo init -u https://opendev.org/starlingx/manifest -b master
repo sync

  3. Configure for Trixie in stx.conf:

    cd stx-tools
source import-stx
stx config --add os_codename trixie

  4. Initialize build environment:

    ./stx-init-env

  5. Follow normal build steps - no extra command-line arguments required

For Developers Working on Both

Maintain two completely independent build environments:

~/starlingx-bullseye/    # Bullseye workspace
~/starlingx-trixie/      # Trixie workspace

Warning

DO NOT try to switch back and forth between releases in the same workspace. The build containers, downloaded packages, and build artifacts are release-specific.

Recommended directory structure:

~/starlingx/
├── bullseye/
│   ├── stx-tools/
│   ├── cgcs-root/
│   └── ...
└── trixie/
    ├── stx-tools/
    ├── cgcs-root/
    └── ...

Downloading Dependencies

The downloader tool automatically uses the correct package lists based on os_codename.

Download Source Packages

downloader -s -B std,rt

Scans for package directories:

  • debian_${os_codename}_pkg_dirs_std

  • debian_${os_codename}_pkg_dirs_rt

  • debian_${os_codename}_pkg_dirs_sign

Download Binary Packages

downloader -b -B std,rt

Downloads binary packages from mirror lists. See Debian Packaging Reference for details on package list organization (base-*.lst, os-std.lst, os-rt.lst).

Download Both

downloader -B std,rt

Best Practices

Minimize Code Duplication

  • Use Scenario 1 (shared debian/all/) whenever possible

  • Only create release-specific directories when truly necessary

  • Consider if Trixie changes can be made backward-compatible with Bullseye

Use Conditional Logic Sparingly

  • Prefer shared code over conditional compilation

  • When conditionals are needed, make them clear and well-documented

  • Test both code paths

Package Naming Consistency

  • Use consistent naming: debian_${codename}_pkg_dirs_${build_type}

  • Follow established patterns for new packages

  • Document any deviations from standard structure

Testing

  • Test packages on both Bullseye and Trixie when using shared packaging

  • Verify conditional logic works correctly on both releases

  • Check that legacy package locations still work for Bullseye

Documentation

  • Document release-specific requirements in package README

  • Note any conditional compilation flags

  • Explain why release-specific packaging is needed (if used)

Migration Guide

Migrating Existing Packages

For packages that work on both releases:

  1. Move debian/ to debian/all/:

    cd <package>
git mv debian debian_tmp
mkdir -p debian
git mv debian_tmp debian/all

  2. Test on both Bullseye and Trixie

  3. Commit changes

For packages needing release-specific changes:

  1. Create release-specific directories:

    cd <package>
git mv debian debian_tmp
mkdir -p debian/bullseye debian/trixie
cp -r debian_tmp/* debian/bullseye/
cp -r debian_tmp/* debian/trixie/
rm -rf debian_tmp

  2. Make necessary changes to debian/trixie/

  3. Add conditional logic if source changes are needed

  4. Test both releases

  5. Commit changes

Troubleshooting

Build Container Issues

If containers fail to start after switching releases:

stx control stop
stx control start

If problems persist, rebuild containers:

./stx-init-env --rebuild

Wrong Packages Downloaded

Verify os_codename in stx.conf:

grep os_codename stx.conf

Clear download cache and re-download:

rm -rf $STX_MIRROR/*
downloader -B std,rt

Package Not Found

Check that package is listed in correct file:

grep <package> stx-tools/debian-mirror-tools/config/debian/${os_codename}/<layer>/*.lst

Verify package directory list includes the package:

find . -name "debian_${os_codename}_pkg_dirs_*" -exec grep -l <package> {} \;

Reference

Key Configuration Variables

Project Settings:

  • debian_distribution - Debian release codename (bullseye, trixie)

  • debian_version - Debian version number (11.3, 13.0)

Debian Snapshot Configuration:

  • debian_snapshot_base - Base URL for Debian snapshot archive

  • debian_security_snapshot_base - Base URL for Debian security snapshot archive

  • debian_snapshot_timestamp - Snapshot timestamp (format: YYYYMMDDTHHMMSSZ)

Builder Settings:

  • os_id - Operating system identifier (debian)

  • os_codename - Debian release codename (bullseye, trixie)

  • os_arch - Architecture (amd64, arm64)

Mirror Configuration:

  • os_mirror_url - Base URL for OS packages

  • os_mirror_dist_path - Path to distribution on mirror (e.g., debian/debian/)

  • os_mirror_dl_path - Path for downloads on mirror (e.g., debian/)

  • lat_mirror_url - Base URL for LAT packages

  • lat_mirror_lat_path - Path to LAT packages on mirror (e.g., lat-sdk/)

Repository Manager:

  • stx_mirror_strategy - Mirror strategy (stx_mirror_first, upstream_first)

Important Files

  • stx.conf - Build configuration

  • stx.conf.sample - Configuration template

  • debian_${codename}_pkg_dirs_std - Standard package list

  • debian_${codename}_pkg_dirs_rt - Real-time package list

  • debian-${codename}-image.inc - ISO image configuration