Looking at differences when having a custom image

When there is an update I can run rpm-ostree db diff and see what has changed.

Now I have a custom image and rpm-ostree db diff shows my changes.

But in a situation where the base image has changed my custom image will be rebuilt which is great. However, then I don’t see what has changed in the base image.

Is there a way to get this information?

Others might have a better/simpler answer. If you are on your own custom image, rpm-ostree db diff will show you the daily changes both in your overlay packages and in the base image. As far as I know, the rpm-ostree command will just give you the total list and not categorize them for you as to which changes are due to your overlay changes and which ones are base image package changes. To do that you would have to write a custom script that is based on your knowledge of what you’ve done in your build.sh. In my case it’s a little more complicated because I’m not just adding a few packages, I’m also swapping out packages from the base image to the KDE Plasma (and Gear) Git repos (using a COPR source). So my (overly) complicated script looks like this (you can modify based on what you are doing….yours could be quite a bit simpler):

#!/bin/bash

# Script to categorize rpm-ostree db diff output between base image changes and overlay changes
# Based on whelanh/aurora-kdegit-dx custom image
# Accounts for KDE package swaps from solopasha COPRs

set -euo pipefail

# Colors for output
RED='\033[0;31m'
GREEN='\033[0;32m'
BLUE='\033[0;34m'
YELLOW='\033[0;33m'
PURPLE='\033[0;35m'
NC='\033[0m' # No Color

# Known overlay packages from your custom image
OVERLAY_PACKAGES=(
    # R and RStudio (from iucar/rstudio COPR)
    "r"
    "R-devel" 
    "rstudio"
    "gcc-gfortran"
    
    # KDE build dependencies
    "git"
    "python3-dbus"
    "python3-pyyaml" 
    "python3-setproctitle"
    "clang-devel"
    
    # Development tools (commented out in your script but might be installed)
    "neovim"
    "zsh"
    "flatpak-builder"
    "kdevelop"
    "kdevelop-devel"
    "kdevelop-libs"
)

# Function to check if a package is a KDE Plasma/Gear package that gets swapped
is_kde_swap_package() {
    local pkg="$1"
    local base_pkg="${pkg%%-[0-9]*}" # Remove version info
    
    # KDE Plasma packages
    if [[ "$base_pkg" =~ ^(plasma-|kwin|kscreenlocker|powerdevil|systemsettings|kde-cli-tools|khotkeys|kmenuedit|kscreen|ksshaskpass|kwallet|kwayland|kwrited|libkscreen|libksysguard|milou|oxygen|breeze|sddm-kcm|xdg-desktop-portal-kde|drkonqi|kactivitymanagerd|kdecoration|kgamma|kinit|kio-extras|kpipewire|ksystemstats|layer-shell-qt|libkworkspace|libplasma|libqalculate|ocean-sound-theme|phonon-qt|plasma5support|polkit-kde|qqc2-breeze-style|xwaylandvideobridge).* ]]; then
        return 0
    fi
    
    # KDE Gear/Applications packages  
    if [[ "$base_pkg" =~ ^(kate|konsole|dolphin|gwenview|okular|ark|spectacle|kcalc|kcharselect|kcolorchooser|kdf|kfind|kgpg|khelpcenter|kinfocenter|kmag|kmousetool|kmouth|kruler|ksystemlog|ktimer|kwalletmanager|filelight|kbackup|kgpg|kleopatra|kmix|krdc|krfb|ksnapshot|kuser|partitionmanager|sweeper|yakuake|elisa|juk|kamoso|dragon|ffmpegthumbs|kdenlive|kwave|amarok).* ]]; then
        return 0
    fi
    
    # KDE Frameworks packages (kf5/kf6 prefix) - these get updated via solopasha COPRs
    if [[ "$base_pkg" =~ ^k[f56]-.* ]]; then
        return 0
    fi
    
    # Additional KDE packages that get swapped
    if [[ "$base_pkg" =~ ^(sddm-wayland-plasma|libkgapi|krdp).* ]]; then
        return 0
    fi
    
    # Qt packages that might get swapped
    if [[ "$base_pkg" =~ ^qt[56]-.* ]]; then
        return 0
    fi
    
    return 1
}

# Function to check if a package is likely from overlay (non-swap additions)
is_overlay_addition() {
    local pkg="$1"
    local base_pkg="${pkg%%-[0-9]*}" # Remove version info
    
    # Check if it's in our known overlay packages
    for overlay_pkg in "${OVERLAY_PACKAGES[@]}"; do
        if [[ "$base_pkg" == *"$overlay_pkg"* ]] || [[ "$overlay_pkg" == *"$base_pkg"* ]]; then
            return 0
        fi
    done
    
    # Check for R-related packages (likely from R COPR)
    if [[ "$base_pkg" =~ ^[Rr]-.*|.*-[Rr]-.*|.*[Rr]studio.* ]]; then
        return 0
    fi
    
    # Check for additional development packages
    if [[ "$base_pkg" =~ .*-devel$|.*-dev$|.*-headers$|.*-static$ ]] && ! is_kde_swap_package "$base_pkg"; then
        return 0
    fi
    
    return 1
}

# Function to check if a package is a base system package
is_base_system_package() {
    local pkg="$1"
    local base_pkg="${pkg%%-[0-9]*}" # Remove version info
    
    # Core system packages
    if [[ "$base_pkg" =~ ^(kernel|systemd|glibc|bash|coreutils|util-linux|filesystem|setup|tzdata|ca-certificates|fedora-release|rpm|dnf|ostree|bootc).* ]]; then
        return 0
    fi
    
    # Graphics and multimedia (base system components)
    if [[ "$base_pkg" =~ ^(mesa|xorg|wayland|pipewire|pulseaudio|gstreamer|ffmpeg|alsa|vulkan).* ]]; then
        return 0
    fi
    
    # Python packaging system
    if [[ "$base_pkg" =~ ^(python3-pip|pyproject-srpm-macros).* ]]; then
        return 0
    fi
    
    # Networking and hardware
    if [[ "$base_pkg" =~ ^(NetworkManager|bluez|cups|udev|dbus|avahi).* ]]; then
        return 0
    fi
    
    # Base desktop components (non-KDE)
    if [[ "$base_pkg" =~ ^(gnome|gtk|cairo|pango|fontconfig|freetype).* ]]; then
        return 0
    fi
    
    # Package that are clearly not KDE and not from your overlay
    if [[ "$base_pkg" =~ ^(firefox|chromium|libreoffice|flatpak|podman|toolbox|distrobox|vim|nano|wget|curl|rsync|git-core|openssh|sudo|polkit|accountsservice).* ]]; then
        return 0
    fi
    
    return 1
}

# Main function
main() {
    echo -e "${BLUE}🔍 Categorizing rpm-ostree db diff output for aurora-kdegit-dx...${NC}\n"
    
    # Get the diff output
    local diff_output
    if ! diff_output=$(rpm-ostree db diff --format=diff 2>/dev/null); then
        echo -e "${RED}Error: Failed to get rpm-ostree db diff${NC}" >&2
        exit 1
    fi
    
    if [[ -z "$diff_output" ]]; then
        echo -e "${YELLOW}No package changes found.${NC}"
        exit 0
    fi
    
    # Arrays to store categorized packages
    local base_changes=()
    local overlay_additions=()
    local kde_swaps=()
    local unknown_changes=()
    
    # Process each line of diff output
    while IFS= read -r line; do
        # Skip ostree diff header lines
        if [[ "$line" =~ ^ostree\ diff\ commit ]]; then
            continue
        fi
        
        # Extract package name (remove leading +, -, !, = symbols)
        local pkg_line="${line#[+\-!=]}"
        local pkg_name="${pkg_line%%-[0-9]*}" # Remove version info
        pkg_name="${pkg_name%%.fc*}" # Remove fedora version info
        pkg_name="${pkg_name%%.x86_64}" # Remove architecture
        pkg_name="${pkg_name%%.noarch}" # Remove architecture
        
        if [[ -n "$pkg_name" ]]; then
            if is_kde_swap_package "$pkg_name"; then
                kde_swaps+=("$line")
            elif is_overlay_addition "$pkg_name"; then
                overlay_additions+=("$line")
            elif is_base_system_package "$pkg_name"; then
                base_changes+=("$line")
            else
                unknown_changes+=("$line")
            fi
        fi
    done <<< "$diff_output"
    
    # Display results
    echo -e "${GREEN}📦 BASE IMAGE CHANGES (from upstream Aurora):${NC}"
    if [[ ${#base_changes[@]} -eq 0 ]]; then
        echo "  No base image changes detected"
    else
        printf '  %s\n' "${base_changes[@]}"
    fi
    
    echo -e "\n${PURPLE}🔄 KDE PACKAGE SWAPS (solopasha/plasma-unstable & kde-gear-unstable):${NC}"
    if [[ ${#kde_swaps[@]} -eq 0 ]]; then
        echo "  No KDE package swaps detected"
    else
        printf '  %s\n' "${kde_swaps[@]}"
    fi
    
    echo -e "\n${YELLOW}➕ OVERLAY ADDITIONS (your custom packages):${NC}"
    if [[ ${#overlay_additions[@]} -eq 0 ]]; then
        echo "  No overlay additions detected"
    else
        printf '  %s\n' "${overlay_additions[@]}"
    fi
    
    echo -e "\n${BLUE}❓ UNKNOWN/UNCLEAR ORIGIN:${NC}"
    if [[ ${#unknown_changes[@]} -eq 0 ]]; then
        echo "  No unclear changes detected"
    else
        printf '  %s\n' "${unknown_changes[@]}"
    fi
    
    # Summary
    echo -e "\n${NC}📊 SUMMARY:"
    echo "  Base image changes: ${#base_changes[@]}"
    echo "  KDE package swaps: ${#kde_swaps[@]}"
    echo "  Overlay additions: ${#overlay_additions[@]}"
    echo "  Unknown origin: ${#unknown_changes[@]}"
    echo "  Total changes: $((${#base_changes[@]} + ${#kde_swaps[@]} + ${#overlay_additions[@]} + ${#unknown_changes[@]}))"
    
    echo -e "\n${PURPLE}💡 Note: KDE swaps show packages replaced with solopasha COPR versions${NC}"
}

# Show help
show_help() {
    cat << EOF
Usage: $0 [OPTIONS]

Categorizes rpm-ostree db diff output to show which changes come from:
- Base Aurora image updates (upstream changes)
- KDE package swaps (solopasha/plasma-unstable & kde-gear-unstable COPRs)
- Overlay additions (R/RStudio, dev tools, etc.)
- Unknown/unclear origin

This script is specifically tailored for whelanh/aurora-kdegit-dx custom image
and accounts for the KDE package swapping behavior in your build script.

OPTIONS:
    -h, --help      Show this help message

EXAMPLES:
    $0              # Run categorization on current diff
    
The script understands your image's package management:
- KDE Plasma/Gear packages swapped with solopasha COPR versions
- R/RStudio from iucar/rstudio COPR  
- Development tools and dependencies
- Flatpak applications (system-wide)

EOF
}

# Parse arguments
while [[ $# -gt 0 ]]; do
    case $1 in
        -h|--help)
            show_help
            exit 0
            ;;
        *)
            echo "Unknown option: $1" >&2
            echo "Use -h or --help for usage information" >&2
            exit 1
            ;;
    esac
done

# Run main function
main%    

Thanks for your reply. I actually don’t need to distinguish between what I added to the base image and the base image diffs.

Yesterday I got an update of my custom image which was because the base image has changed (at least I haven’t done anything on my side). I have switched to my newly built custom image and now rpm-ostree db diff says

ostree diff commit from: rollback deployment (512533e34a125836869eef4b931e7b1948750caae5d348382551786bf442e628)
ostree diff commit to: booted deployment (65a9c107e8e5f3a4a38d50faa2add47d74de42291fa8df4e76dfac94f155a65c)

That’s why I concluded that I don’t see what happens in the base image. But perhaps my thinking is wrong. :thinking:

Ah….I understand now. If those were the only two lines you got, I think then nothing changed between your two nightly builds and your system updated to the new base image simply because it was the latest available from your git builds. For comparison, this is what I got from my overnight build (my base is aurora-dx):

ostree diff commit from: rollback deployment (8807f35ad092037f74dab213a414e9ce7c17c92a967640ed9bd121f54af21e0f)
ostree diff commit to:   booted deployment (a79a971c3746f05482c4dbfd5df20eb528bf21070164ea25ab8e36cded165f4c)
Upgraded:
  dolphin 25.11.70~25.gitdf57589-1.fc42 -> 25.11.70~26.git407ce07-1.fc42
  dolphin-libs 25.11.70~25.gitdf57589-1.fc42 -> 25.11.70~26.git407ce07-1.fc42
  flatpak-kcm 6.4.80~13.gitb5d6aaf-1.fc42 -> 6.4.80~14.gitaef712f-1.fc42
  kactivitymanagerd 6.4.80~11.git821612d-1.fc42 -> 6.4.80~12.git7fa75bd-1.fc42
  kate 25.11.70~37.gitdd79fc1-1.fc42 -> 25.11.70~38.git3beb4fa-1.fc42
  kate-libs 25.11.70~37.gitdd79fc1-1.fc42 -> 25.11.70~38.git3beb4fa-1.fc42
  kate-plugins 25.11.70~37.gitdd79fc1-1.fc42 -> 25.11.70~38.git3beb4fa-1.fc42
  kf6-kfilemetadata 6.19.0~1.git253b740-1.fc42 -> 6.19.0~2.git38bbefc-1.fc42
  kf6-kimageformats 6.19.0~6.git14286a6-1.fc42 -> 6.19.0~7.gita4e1873-1.fc42
  kf6-ksvg 6.19.0~3.gitd3d163a-1.fc42 -> 6.19.0~4.git741de4c-1.fc42
  kf6-kwindowsystem 6.19.0~2.git40e4a4e-1.fc42 -> 6.19.0~3.gitac28bf9-1.fc42
  krdp 6.4.80~19.git318cc28-1.fc42 -> 6.4.80~20.git86f4b28-1.fc42
  krdp-libs 6.4.80~19.git318cc28-1.fc42 -> 6.4.80~20.git86f4b28-1.fc42
  kscreen 1:6.4.80~30.gitae4640e-1.fc42 -> 1:6.4.80~31.git9e27b87-1.fc42
  kwin 6.4.80~104.git21d2b8a-1.fc42 -> 6.4.80~105.gitf07b9cb-1.fc42
  kwrite 25.11.70~37.gitdd79fc1-1.fc42 -> 25.11.70~38.git3beb4fa-1.fc42
  libkgapi 25.11.70~4.git5bbf7b4-1.fc42 -> 25.11.70~5.git905fe58-1.fc42
  libkworkspace6 6.4.80~103.git8211d59-1.fc42 -> 6.4.80~104.gita3f1f5d-1.fc42
  mesa-dri-drivers 1:25.1.7-1.fc42 -> 1:25.1.9-1.fc42
  mesa-filesystem 1:25.1.7-1.fc42 -> 1:25.1.9-1.fc42
  mesa-libEGL 1:25.1.7-1.fc42 -> 1:25.1.9-1.fc42
  mesa-libGL 1:25.1.7-1.fc42 -> 1:25.1.9-1.fc42
  mesa-libgbm 1:25.1.7-1.fc42 -> 1:25.1.9-1.fc42
  mesa-libxatracker 1:25.1.7-1.fc42 -> 1:25.1.9-1.fc42
  mesa-va-drivers 1:25.1.7-1.fc42 -> 1:25.1.9-1.fc42
  mesa-vulkan-drivers 1:25.1.7-1.fc42 -> 1:25.1.9-1.fc42
  pipewire-libs-extra 1.4.7-1.fc42 -> 1.4.8-1.fc42
  plasma-discover 6.4.80~34.git1a52766-1.fc42 -> 6.4.80~35.gitbc8a822-1.fc42
  plasma-discover-flatpak 6.4.80~34.git1a52766-1.fc42 -> 6.4.80~35.gitbc8a822-1.fc42
  plasma-discover-kns 6.4.80~34.git1a52766-1.fc42 -> 6.4.80~35.gitbc8a822-1.fc42
  plasma-discover-libs 6.4.80~34.git1a52766-1.fc42 -> 6.4.80~35.gitbc8a822-1.fc42
  plasma-discover-notifier 6.4.80~34.git1a52766-1.fc42 -> 6.4.80~35.gitbc8a822-1.fc42
  plasma-drkonqi 6.4.80~17.gita80fd4b-1.fc42 -> 6.4.80~18.git13c4098-1.fc42
  plasma-lookandfeel-fedora 6.4.80~103.git8211d59-1.fc42 -> 6.4.80~104.gita3f1f5d-1.fc42
  plasma-nm 6.4.80~34.git6b02459-1.fc42 -> 6.4.80~35.gitcdb4d28-1.fc42
  plasma-nm-openconnect 6.4.80~34.git6b02459-1.fc42 -> 6.4.80~35.gitcdb4d28-1.fc42
  plasma-nm-openvpn 6.4.80~34.git6b02459-1.fc42 -> 6.4.80~35.gitcdb4d28-1.fc42
  plasma-nm-vpnc 6.4.80~34.git6b02459-1.fc42 -> 6.4.80~35.gitcdb4d28-1.fc42
  plasma-workspace 6.4.80~103.git8211d59-1.fc42 -> 6.4.80~104.gita3f1f5d-1.fc42
  plasma-workspace-common 6.4.80~103.git8211d59-1.fc42 -> 6.4.80~104.gita3f1f5d-1.fc42
  plasma-workspace-libs 6.4.80~103.git8211d59-1.fc42 -> 6.4.80~104.gita3f1f5d-1.fc42
  powerdevil 6.4.80~29.git00dabc2-1.fc42 -> 6.4.80~31.git09b8b6b-1.fc42
  pyproject-srpm-macros 1.18.3-1.fc42 -> 1.18.4-1.fc42
  python3-pip 24.3.1-2.fc42 -> 24.3.1-5.fc42
  sddm-wayland-plasma 6.4.80~103.git8211d59-1.fc42 -> 6.4.80~104.gita3f1f5d-1.fc42
  xdg-desktop-portal-kde 6.4.80~20.git1629d7a-1.fc42 -> 6.4.80~21.gitf37d516-1.fc42

1 Like

Yeah, it could be that I simply misinterpreted things… My base image is: ghcr.io/ublue-os/aurora-dx-nvidia-open:stable

I still don’t understand things very good but I see that an image gets built on a daily basis. Looking here Workflow runs · manfredlotz/aurora-dx-nvidia-open-asus-aso · GitHub at the latest Build container image I see in View workflow file that there is a daily schedule

- cron: '05 10 * * *' # 10:05am UTC everyday

So, it seems that no matter if the aurora base image has changed a new image will be built.

This output simply seems to tell me that (at least from an rpm point of view) nothing has changed.

❯ rpm-ostree db diff
ostree diff commit from: booted deployment (0703a0b2e5e8ded3e18580ac9eeaf2b79514c2ebc2bd2e0ba6a3aaa77f9953c4)
ostree diff commit to: pending deployment (e3063aefcafc4cd0a47834dce2fe99c8ebcf3e541975fa623266cee63cfe873f)

I only can say that I did not do anything deliberately regarding schedule of images,

Your understanding is correct: your workflow is scheduled to run every morning regardless of whether there are any changes in the base system or in the overlay packages you’ve added in build.sh. Your build logs show your workflow is running correctly (it also runs every time you make a code change). I think the fact that your base image is the :stable one helps explain why you aren’t seeing very many changes. You could experiment with using :latest to confirm.

Thanks for the confirmation. I guess I will simply be patient as I know that usually on a weekly basis changes in the aurora base image will happen.

Currently I will see New deployment staged in the shell daily. I am thinking about changing this check a bit so that I will see if also rpm packages are involved. :thinking:

I found the following on base image update frequency:

Stable versus other release channels

The frequency of updates varies depending on the release channel you choose:

  • Stable: This channel receives a new image once per week, though it checks for updates daily. It is intended for enthusiasts who want a modern Fedora experience with a slightly more conservative update pace than the latest channel. Stable releases use a “gated” kernel from Fedora CoreOS, meaning kernel upgrades arrive 2–3 weeks after a new version is released upstream.

  • Stable-daily: If you want daily system updates but prefer the more controlled, “gated” kernel of the stable channel, this is available for images like Bluefin and Aurora.

I’m not sure if stable-daily still exists. Choices now might be stable or latest.

stable-daily still exists too

Actually, I am happy with Stable as I don’t want to reboot each day.

When I didn’t use a custom image and got the New deployment staged message in the starship prompt I knew that

  • either there were changes in rpm packages which I could see when running rpm-ostree db diff and perhaps also other changes
  • or there were only other changes when rpm-ostree db diff didn’t show anything.

Now, with the daily build of my custom image I get New deployment staged each day. I can find out if there are rpm changes but I have no idea if there are other changes in the aurora base image.

As said before I could check for rpm changes but it would be nice if I would have a possibility to detect if the aurora base image has been changed as well. Is there any possibility doing this?

For my images rather than build them on a schedule I use Renovate to only build when there is a new upstream image.

The way it works is it will pin the sha256 fingerprint of the current stable (or whatever label you use) in the FROM line in your Containerfile then watch the upstream repo for the stable image to change, at which point it can automatically open a PR to update the fingerprint and automatically merge it if the test build passes. Then you get a fresh new image only when there’s something to update!

1 Like

Thanks @ssweeny Sounds promising. I will take a look.