Containerized Yocto Builds with KAS: Why, How, and When It Breaks

The Yocto Project builds embedded Linux distributions from source — a compiler toolchain, C library, hundreds of packages, and a root filesystem. A clean build takes 2-4 hours on fast hardware. The build host needs specific versions of Python, Git, tar, and a dozen other tools. Without containerization, a developer on Fedora 40 and another on Ubuntu 24.04 will produce different binaries from the same recipes.

KAS 1 wraps Yocto builds in a Docker or Podman container with a known-good toolchain, parses YAML configuration files to manage layers and settings, and provides a single command to rebuild an entire distribution from any Linux host. This article walks through what I figured out about KAS: how it works, how to structure a multi-variant Yocto project with KAS overlays, how the shared state cache affects build times, and the failure modes I ran into when builds stopped working.

Why containerized Yocto builds matter

The Yocto build host requirements are documented but fragile. BitBake depends on specific versions of Python, GNU Make, and shell utilities. The buildtools-tarball installer mitigates some of this, but it does not address the entire host environment — filesystem features, kernel version, and available system libraries all influence build output 2.

A container image freezes the entire build environment. When you use kas-container build, KAS pulls a known container image (e.g., ghcr.io/siemens/kas/kas:5.2), mounts the project directory, and runs BitBake inside that container. The host only needs Docker or Podman. Nothing else.

The reproducibility benefit is immediate. A build that works on CI works on a developer's machine. A build that worked six months ago still works today, because the container image is pinned by tag. There is no "it worked yesterday" mystery that ends with discovering a system python3 upgrade broke BitBake.

The KAS YAML layering model

KAS configuration files use YAML to define the entire build: target machine, distribution, layer repositories (with pinned commits), local configuration (local.conf equivalents), and the build target. Configurations are layered — a base config defines defaults, and overlay configs modify or extend them.

Base configuration

A minimal KAS config for an x86-64 Yocto build looks like this 3. For background on the Yocto layer model and how BitBake resolves layers, see the Yocto overview manual 8 and the layers guide 9:

header:
  version: 14
 
machine: genericx86-64
 
repos:
  openembedded-core:
    url: https://git.openembedded.org/openembedded-core
    branch: scarthgap
    commit: 530fb9ea
    layers:
      meta:
 
  meta-custom:
    path: meta-custom
    layers:
      meta-custom:
 
local_conf_header:
  standard: |
    PACKAGE_CLASSES = "package_ipk"
 
target: core-image-minimal

The key design decisions are visible here:

• header.version is the KAS config format version. Check the KAS documentation for the version that matches your KAS release; KAS validates the config against this value.
• Commit pinning (commit: 530fb9ea) ensures every build uses the exact same layer source, regardless of what has been pushed to the branch since.
• Local layers (path: meta-custom) reference directories within the project repository. Replace this with your own layer path. These do not need commit pins — they move with the project's git history.
• local_conf_header is equivalent to local.conf snippets. Variables set here override the layer defaults.

Overlay configs

An overlay config modifies the base config without duplicating it. Overlays are applied using colon notation:

kas-container build kas-config.yml:kas-config-dev.yml

A development overlay might look like this:

header:
  version: 14
 
target: core-image-dev
 
local_conf_header:
  dev_image: |
    IMAGE_FEATURES += "tools-debug debug-tweaks"
    EXTRA_IMAGE_FEATURES += "ssh-server-openssh"

The overlay overrides the target and adds local_conf_header entries. The base config's repos, machine, and distro settings are inherited unchanged.

This layering model supports multiple variants from a single base:

Each variant shares the same layer commits, kernel config, and base packages. Differences are isolated to the overlay file 4.

Commit pinning and reproducible builds

The Yocto Project's layer model is a double-edged sword. Upstream layers update continuously — a security fix in meta-openembedded today, a new package version tomorrow. If your config references branches without commit pins, every build can produce different output.

KAS enforces commit pinning for remote repositories. The commit: field locks a layer to a specific git SHA. When you are ready to update, you change the SHA and rebuild. The diff between the old and new SHAs is your audit trail.

The update workflow is:

# Check for new commits on tracked branches
./check-kas-updates.sh
 
# If updates are available, review the diff
git log <old-commit>..<new-commit> -- <layer-path>
 
# Update pins and rebuild
# Edit kas-config.yml with new commit values
kas-container build kas-config.yml

A script like this checks whether pinned commits have moved:

#!/bin/bash
# check-kas-updates.sh — compare pinned commits against remote HEAD
for repo in openembedded-core meta-yocto meta-openembedded; do
  pinned=$(grep -A1 "$repo" kas-config.yml | grep commit | awk '{print $2}')
  branch=$(grep -A2 "$repo" kas-config.yml | grep branch | awk '{print $2}')
  remote=$(grep -B1 "$repo" kas-config.yml | grep url | awk '{print $2}')
  remote_head=$(git ls-remote "$remote" "refs/heads/$branch" | awk '{print $1}')
  if [ "$pinned" != "$remote_head" ]; then
    echo "$repo: pinned=$pinned remote=$remote_head (out of date)"
  fi
done

This turns the impossible task of tracking a dozen upstream layers into a periodic five-minute check. This approach works for straightforward KAS configs where each repo block has a consistent field order. For complex configs with nested includes or variable interpolation, parse the YAML properly with yq or a Python script rather than relying on grep offset patterns.

Sstate-cache economics

The shared state cache is the difference between a 2-hour clean build and a 5-minute incremental build. It stores the output of every BitBake task, keyed by a hash of the task's inputs (recipe, source, configuration, and dependencies). When you rebuild, BitBake skips any task whose inputs have not changed and reuses the cached output 5.

Build times

The large jump from 10 to 30+ minutes on a layer update reflects the fact that any task that depends on the updated layer must be rebuilt. Updating openembedded-core invalidates nearly every task. Updating a leaf recipe invalidates only that recipe and its dependents.

Where the cache lives

In a typical KAS project, the sstate cache is in build/sstate-cache/ inside the project directory. This is fine for a single developer. For teams, a shared sstate mirror (HTTP server or NFS) allows one developer's build outputs to accelerate everyone else's.

KAS does not manage sstate mirrors natively, but you can configure one in the base config:

local_conf_header:
  sstate_mirror: |
    SSTATE_MIRRORS = "file://.* https://sstate.example.com/PATH;downloadfilename=PATH"

Caveat: Do not share a single sstate cache directory across concurrent builds from different branches. BitBake does not lock the cache, and concurrent writes can corrupt task outputs. Isolate caches per branch or use a read-only shared mirror with a local write-through cache 6.

The build-all.sh wrapper pattern

While kas-container build is the canonical command, real projects accumulate flags: build variants, clean builds, USB flashing, skip-if-cached, logging. A thin shell wrapper keeps these manageable:

#!/bin/bash
# build-all.sh — build all variants with sensible defaults
set -e
 
KAS="./kas-container"
BASE="kas-config.yml"
LOG_DIR="logs"
 
mkdir -p "$LOG_DIR"
 
build_variant() {
  local name="$1"
  local overlay="$2"
  local config="${BASE}:${overlay}"
 
  echo "=== Building ${name} ==="
  "$KAS" build "$config" 2>&1 | tee "${LOG_DIR}/${name}.log"
}
 
case "${1:-}" in
  --dev-only)
    build_variant "dev" "kas-config-dev.yml"
    ;;
  --prod-only)
    build_variant "prod" "kas-config-prod.yml"
    ;;
  *)
    build_variant "prod" "kas-config-prod.yml"
    build_variant "prod-flasher" "kas-config-prod.yml:kas-config-flasher.yml"
    ;;
esac

This wrapper handles the common patterns — log tee, sequential variant builds, clean vs incremental — without embedding build logic. The KAS configs remain the single source of truth for what each variant contains 7.

When containerized builds break

Containerization removes most host-dependency problems, but introduces new failure modes. Here are the ones you will hit.

Container runtime not found

KAS auto-detects a container runtime, preferring Podman over Docker if both are installed. You can override this with KAS_CONTAINER_ENGINE:

export KAS_CONTAINER_ENGINE=docker
kas-container build kas-config.yml

If neither runtime is installed, or the user lacks permissions for the detected runtime, kas-container fails immediately.

# Check runtime
podman --version || docker --version
 
# Check rootless mode (Podman)
podman info | grep rootless
 
# Ensure the user is in the docker group if using Docker
groups | grep docker

On systems with Podman in rootless mode, some mounts may fail due to user namespace restrictions. The symptom is a permission error during the build container startup. The fix is to configure subordinate UID/GID ranges or fall back to Docker.

Disk space exhaustion

A single Yocto build needs 100 GB. A full build with multiple variants, sstate cache, and downloads can consume 150-200 GB.

# Monitor disk usage during build
watch -n 30 'df -h build/'
 
# If space runs out, clean tmp but keep downloads and sstate
rm -rf build/tmp

The build/tmp/ directory is the largest consumer (working directories for every recipe). It can be deleted safely — BitBake will rebuild from scratch using the sstate cache and downloads cache.

Git fetch failures inside the container

The KAS container has its own network stack. If the container cannot reach the layer repositories (corporate proxy, VPN, air-gapped network), do_fetch fails.

Git and HTTP proxy settings must be passed into the container. KAS supports this via environment variables:

export KAS_CONTAINER_EXTRA_ARGS="--env http_proxy=http://proxy:8080 --env https_proxy=http://proxy:8080 --env no_proxy=localhost,.internal"
kas-container build kas-config.yml

Kernel module build failures in the container

Out-of-tree kernel modules that use the host kernel headers (instead of the target kernel headers) will fail inside the container because the container kernel and host kernel may differ. This is not a KAS-specific problem, but the container obscures the symptom — the error message references paths that only exist inside the container.

The fix is always in the recipe: ensure inherit module is present and KERNEL_DIR points to ${STAGING_KERNEL_DIR}.

Layer compatibility drift

KAS pins layer commits, but the container image itself is updated periodically. A KAS 5.2 container includes specific versions of BitBake, Python, and host tools. If you update the container image without updating layer commits, the new tools may reject old layer metadata. If you update layer commits without updating the container, the new metadata may require features not present in the old tools.

What I do now: update container image and layer commits together. I test the combination before committing either change in isolation.

Summary

Using KAS for containerized builds removed the "works on my machine" problem from my Yocto workflow. The YAML layering model keeps my configuration DRY across build variants. Commit pinning means I can reproduce any build I made months ago. The sstate cache cuts my rebuilds from hours to minutes.

The failure modes I ran into — disk space exhaustion, container networking, layer compatibility drift — were frustrating but fixable. None of them were unique to KAS. They were Yocto problems that KAS made visible instead of silent and mysterious.

References

[1] Siemens, "KAS — Setup Tool for BitBake Based Projects," https://kas.readthedocs.io/en/latest/intro.html, accessed June 2026.

[2] Yocto Project, "Required Packages for the Build Host," https://docs.yoctoproject.org/ref-manual/system-requirements.html#required-packages-for-the-build-host, accessed June 2026.

[3] Siemens, "KAS Project Configuration," https://kas.readthedocs.io/en/latest/userguide/project-configuration.html, accessed June 2026.

[4] Siemens, "KAS User Guide — Sub-commands (Plugins)," https://kas.readthedocs.io/en/latest/userguide/plugins.html, accessed June 2026.

[5] Yocto Project, "Shared State Cache," https://docs.yoctoproject.org/overview-manual/concepts.html#shared-state-cache, accessed June 2026.

[6] Yocto Project, "Invalidating Shared State to Force a Task to Run," https://docs.yoctoproject.org/dev/dev-manual/debugging.html#invalidating-shared-state-to-force-a-task-to-run, accessed June 2026.

[7] Siemens, "KAS — Building in a Container," https://kas.readthedocs.io/en/latest/userguide/kas-container.html, accessed June 2026.

[8] Yocto Project, "Yocto Project Overview and Concepts Manual," https://docs.yoctoproject.org/overview-manual/index.html, accessed June 2026.

[9] Yocto Project, "Understanding and Creating Layers," https://docs.yoctoproject.org/dev/dev-manual/layers.html, accessed June 2026.