Hi there ublue community! 
First of all, thank you to everyone involved in this amazing project!
Been running ublue, specifically Aurora, for about a month now and it seems to be exactly what I was looking for!
However, I am currently facing an issue when trying to update via rpm-ostree. It seems to crash right after staging the deployment. I first encountered this yesterday and while I was able to update yesterday by booting into a previous deployment and update from there, it still seems to be broken.
❯ rpm-ostree status
State: idle
AutomaticUpdates: stage; rpm-ostreed-automatic.timer: no runs since boot
Deployments:
● ostree-image-signed:docker://ghcr.io/nihaiden/aurora-dx:39
Digest: sha256:d9293379177796ee0d8e2d96681d037cfa582fbd2fceb7725c63aa4cfb57594b
Version: 39.20240403.0 (2024-04-03T22:57:24Z)
LayeredPackages: yakuake
Pinned: yes
ostree-image-signed:docker://ghcr.io/nihaiden/aurora-dx:39
Digest: sha256:9230201a51c19466fbbc8351bee56969aae566a967ea5514ea136a00b8238e60
Version: 39.20240309.0 (2024-03-09T16:52:52Z)
LayeredPackages: yakuake
Pinned: yes
❯ rpm-ostree upgrade
note: automatic updates (stage) are enabled
Pulling manifest: ostree-image-signed:docker://ghcr.io/nihaiden/aurora-dx:39
Importing: ostree-image-signed:docker://ghcr.io/nihaiden/aurora-dx:39 (digest: sha256:31217b7474311aef96ad1aa1f14bd16c23121bc41c0cfc30b9d8da2301b8e935)
ostree chunk layers already present: 60
ostree chunk layers needed: 5 (328,1 MB)
custom layers needed: 3 (1,7 GB)
Checking out tree 2b108be... done
Enabled rpm-md repositories: copr:copr.fedorainfracloud.org:ublue-os:akmods updates fedora rpmfusion-free-updates-testing rpmfusion-free-updates rpmfusion-free rpmfusion-nonfree-updates-testing rpmfusion-nonfree-updates rpmfusion-nonfree updates-archive
Updating metadata for 'updates'... done
Updating metadata for 'updates-archive'... done
Importing rpm-md... done
rpm-md repo 'copr:copr.fedorainfracloud.org:ublue-os:akmods' (cached); generated: 2024-03-17T06:02:48Z solvables: 216
rpm-md repo 'updates'; generated: 2024-04-05T02:28:11Z solvables: 25499
rpm-md repo 'fedora' (cached); generated: 2023-11-01T00:12:39Z solvables: 70825
rpm-md repo 'rpmfusion-free-updates-testing' (cached); generated: 2024-03-31T11:47:48Z solvables: 6
rpm-md repo 'rpmfusion-free-updates' (cached); generated: 2024-03-31T11:47:41Z solvables: 170
rpm-md repo 'rpmfusion-free' (cached); generated: 2023-11-04T16:49:08Z solvables: 445
rpm-md repo 'rpmfusion-nonfree-updates-testing' (cached); generated: 2024-03-31T12:15:07Z solvables: 2
rpm-md repo 'rpmfusion-nonfree-updates' (cached); generated: 2024-03-31T12:14:57Z solvables: 74
rpm-md repo 'rpmfusion-nonfree' (cached); generated: 2023-11-04T17:26:32Z solvables: 208
rpm-md repo 'updates-archive'; generated: 2024-04-04T01:11:00Z solvables: 40351
Resolving dependencies... done
Checking out packages... done
Running pre scripts... done
Running post scripts... done
Running posttrans scripts... done
Writing rpmdb... done
Writing OSTree commit... done
Staging deployment... done
error: Bus owner changed, aborting. This likely means the daemon crashed; check logs with `journalctl -xe`.
Looking in journalctl as suggested, I can find the following stacktrace:
Apr 05 13:15:16 framework-jan systemd-coredump[7959]: Process 3737 (rpm-ostree) of user 0 dumped core.
Module libnss_resolve.so.2 from rpm systemd-254.10-1.fc39.x86_64
Module libnss_mdns4_minimal.so.2 from rpm nss-mdns-0.15.1-9.fc39.x86_64
Module libsasldb.so from rpm cyrus-sasl-2.1.28-11.fc39.x86_64
Module libplain.so from rpm cyrus-sasl-2.1.28-11.fc39.x86_64
Module liblogin.so from rpm cyrus-sasl-2.1.28-11.fc39.x86_64
Module libkdexoauth2.so from rpm libkgapi-23.08.5-1.fc39.x86_64
Module libgssapiv2.so from rpm cyrus-sasl-2.1.28-11.fc39.x86_64
Module libanonymous.so from rpm cyrus-sasl-2.1.28-11.fc39.x86_64
Module libnss_myhostname.so.2 from rpm systemd-254.10-1.fc39.x86_64
Module libnss_systemd.so.2 from rpm systemd-254.10-1.fc39.x86_64
Module libnss_sss.so.2 from rpm sssd-2.9.4-1.fc39.x86_64
Module libnss_altfiles.so.2 from rpm nss-altfiles-2.23.0-1.fc39.x86_64
Module libcrypt.so.2 from rpm libxcrypt-4.4.36-2.fc39.x86_64
Module libbrotlicommon.so.1 from rpm brotli-1.1.0-1.fc39.x86_64
Module libsasl2.so.3 from rpm cyrus-sasl-2.1.28-11.fc39.x86_64
Module libevent-2.1.so.7 from rpm libevent-2.1.12-9.fc39.x86_64
Module libkeyutils.so.1 from rpm keyutils-1.6.3-1.fc39.x86_64
Module libkrb5support.so.0 from rpm krb5-1.21.2-3.fc39.x86_64
Module libcom_err.so.2 from rpm e2fsprogs-1.47.0-2.fc39.x86_64
Module libk5crypto.so.3 from rpm krb5-1.21.2-3.fc39.x86_64
Module libkrb5.so.3 from rpm krb5-1.21.2-3.fc39.x86_64
Module libunistring.so.5 from rpm libunistring-1.1-5.fc39.x86_64
Module libbrotlidec.so.1 from rpm brotli-1.1.0-1.fc39.x86_64
Module libgssapi_krb5.so.2 from rpm krb5-1.21.2-3.fc39.x86_64
Module libpsl.so.5 from rpm libpsl-0.21.2-4.fc39.x86_64
Module libssh.so.4 from rpm libssh-0.10.6-2.fc39.x86_64
Module libidn2.so.0 from rpm libidn2-2.3.7-1.fc39.x86_64
Module libnghttp2.so.14 from rpm nghttp2-1.55.1-4.fc39.x86_64
Module libblkid.so.1 from rpm util-linux-2.39.3-6.fc39.x86_64
Module libattr.so.1 from rpm attr-2.5.1-8.fc39.x86_64
Module libyaml-0.so.2 from rpm libyaml-0.2.5-12.fc39.x86_64
Module libzck.so.1 from rpm zchunk-1.4.0-1.fc39.x86_64
Module libcurl.so.4 from rpm curl-8.2.1-4.fc39.x86_64
Module librpm_sequoia.so.1 from rpm rust-rpm-sequoia-1.6.0-1.fc39.x86_64
Module liblua-5.4.so from rpm lua-5.4.6-3.fc39.x86_64
Module libpopt.so.0 from rpm popt-1.19-3.fc39.x86_64
Module libpcre2-8.so.0 from rpm pcre2-10.42-1.fc39.2.x86_64
Module libselinux.so.1 from rpm libselinux-3.5-5.fc39.x86_64
Module libmount.so.1 from rpm util-linux-2.39.3-6.fc39.x86_64
Module libgmodule-2.0.so.0 from rpm glib2-2.78.3-1.fc39.x86_64
Module libffi.so.8 from rpm libffi-3.4.4-4.fc39.x86_64
Module libxml2.so.2 from rpm libxml2-2.10.4-3.fc39.x86_64
Module libbz2.so.1 from rpm bzip2-1.0.8-16.fc39.x86_64
Module liblz4.so.1 from rpm lz4-1.9.4-4.fc39.x86_64
Module libzstd.so.1 from rpm zstd-1.5.6-1.fc39.x86_64
Module liblzma.so.5 from rpm xz-5.4.4-1.fc39.x86_64
Module libacl.so.1 from rpm acl-2.3.1-9.fc39.x86_64
Module libostree-1.so.1 from rpm ostree-2024.5-1.fc39.x86_64
Module libz.so.1 from rpm zlib-1.2.13-4.fc39.x86_64
Module libcrypto.so.3 from rpm openssl-3.1.1-4.fc39.x86_64
Module libssl.so.3 from rpm openssl-3.1.1-4.fc39.x86_64
Module libmodulemd.so.2 from rpm libmodulemd-2.15.0-5.fc39.x86_64
Module libsolv.so.1 from rpm libsolv-0.7.28-1.fc39.x86_64
Module libsolvext.so.1 from rpm libsolv-0.7.28-1.fc39.x86_64
Module librepo.so.0 from rpm librepo-1.17.1-1.fc39.x86_64
Module libjson-c.so.5 from rpm json-c-0.17-1.fc39.x86_64
Module libsystemd.so.0 from rpm systemd-254.10-1.fc39.x86_64
Module librpmio.so.10 from rpm rpm-4.19.1.1-1.fc39.x86_64
Module librpm.so.10 from rpm rpm-4.19.1.1-1.fc39.x86_64
Module libglib-2.0.so.0 from rpm glib2-2.78.3-1.fc39.x86_64
Module libgio-2.0.so.0 from rpm glib2-2.78.3-1.fc39.x86_64
Module libpolkit-gobject-1.so.0 from rpm polkit-123-1.fc39.1.x86_64
Module libgobject-2.0.so.0 from rpm glib2-2.78.3-1.fc39.x86_64
Module libarchive.so.13 from rpm libarchive-3.7.1-1.fc39.x86_64
Module libjson-glib-1.0.so.0 from rpm json-glib-1.8.0-1.fc39.x86_64
Module libcap.so.2 from rpm libcap-2.48-9.fc39.x86_64
Module rpm-ostree from rpm rpm-ostree-2024.4-3.fc39.x86_64
Stack trace of thread 6559:
#0 0x00007f9378c4f834 __pthread_kill_implementation (libc.so.6 + 0x90834)
#1 0x00007f9378bfd8ee raise (libc.so.6 + 0x3e8ee)
#2 0x00007f9378be58ff abort (libc.so.6 + 0x268ff)
#3 0x00007f9379a0e056 g_assertion_message.cold (libglib-2.0.so.0 + 0x20056)
#4 0x00007f9378e42558 split_kernel_args (libostree-1.so.1 + 0xa1558)
#5 0x00007f9378e46f80 ostree_kernel_args_append (libostree-1.so.1 + 0xa5f80)
#6 0x00007f9378e47147 ostree_kernel_args_append_argv_filtered (libostree-1.so.1 + 0xa6147)
#7 0x00007f9378e29e60 _ostree_deployment_set_bootconfig_from_kargs (libostree-1.so.1 + 0x88e60)
#8 0x00007f9378e601bc sysroot_initialize_deployment.constprop.0 (libostree-1.so.1 + 0xbf1bc)
#9 0x00007f9378e2c033 ostree_sysroot_stage_tree_with_options (libostree-1.so.1 + 0x8b033)
#10 0x0000564f4966ccf7 rpmostree_sysroot_upgrader_deploy (rpm-ostree + 0x293cf7)
#11 0x0000564f4962a63b _ZL26deploy_transaction_executeP22_RpmostreedTransactionP13_GCancellablePP7_GError (rpm-ostree + 0x25163b)
#12 0x0000564f4961bf90 _ZL26transaction_execute_threadP6_GTaskPvS1_P13_GCancellable (rpm-ostree + 0x242f90)
#13 0x00007f9379bef774 g_task_thread_pool_thread (libgio-2.0.so.0 + 0xb7774)
#14 0x00007f9379a7edb2 g_thread_pool_thread_proxy.lto_priv.0 (libglib-2.0.so.0 + 0x90db2)
#15 0x00007f9379a7a523 g_thread_proxy (libglib-2.0.so.0 + 0x8c523)
#16 0x00007f9378c4d897 start_thread (libc.so.6 + 0x8e897)
#17 0x00007f9378cd4a5c __clone3 (libc.so.6 + 0x115a5c)
Stack trace of thread 3811:
#0 0x00007f9378cc6e3d __poll (libc.so.6 + 0x107e3d)
#1 0x00007f9379aa4eb4 g_main_context_iterate_unlocked.isra.0 (libglib-2.0.so.0 + 0xb6eb4)
#2 0x00007f9379a47ad3 g_main_context_iteration (libglib-2.0.so.0 + 0x59ad3)
#3 0x0000564f4960fcdc rpmostree_builtin_start_daemon (rpm-ostree + 0x236cdc)
#4 0x0000564f4960a3de _ZN12rpmostreecxx14rpmostree_mainEN4rust10cxxbridge15SliceIKNS1_3StrEEE (rpm-ostree + 0x2313de)
#5 0x0000564f49603404 rpmostreecxx$cxxbridge1$rpmostree_main (rpm-ostree + 0x22a404)
#6 0x0000564f49904c7c _ZN14rpmostree_rust3ffi14rpmostree_main17hbb3cd952e5524884E (rpm-ostree + 0x52bc7c)
#7 0x0000564f495c8d98 _ZN5tokio7runtime4task3raw4poll17h23652abd376cffd4E (rpm-ostree + 0x1efd98)
#8 0x0000564f49cb3663 _ZN3std10sys_common9backtrace28__rust_begin_short_backtrace17h7326b76817e1f983E (rpm-ostree + 0x8da663)
#9 0x0000564f49cb47bd _ZN4core3ops8function6FnOnce40call_once$u7b$$u7b$vtable.shim$u7d$$u7d$17h7a74a56e731c2ff6E (rpm-ostree + 0x8db7bd)
#10 0x0000564f49d19795 _ZN3std3sys4unix6thread6Thread3new12thread_start17h067379cbc031a159E (rpm-ostree + 0x940795)
#11 0x00007f9378c4d897 start_thread (libc.so.6 + 0x8e897)
#12 0x00007f9378cd4a5c __clone3 (libc.so.6 + 0x115a5c)
Stack trace of thread 3815:
#0 0x00007f9378cc6e3d __poll (libc.so.6 + 0x107e3d)
#1 0x00007f9379aa4eb4 g_main_context_iterate_unlocked.isra.0 (libglib-2.0.so.0 + 0xb6eb4)
#2 0x00007f9379a4b447 g_main_loop_run (libglib-2.0.so.0 + 0x5d447)
#3 0x00007f9379c54592 gdbus_shared_thread_func.lto_priv.0 (libgio-2.0.so.0 + 0x11c592)
#4 0x00007f9379a7a523 g_thread_proxy (libglib-2.0.so.0 + 0x8c523)
#5 0x00007f9378c4d897 start_thread (libc.so.6 + 0x8e897)
#6 0x00007f9378cd4a5c __clone3 (libc.so.6 + 0x115a5c)
Stack trace of thread 3813:
#0 0x00007f9378cc6e3d __poll (libc.so.6 + 0x107e3d)
#1 0x00007f9379aa4eb4 g_main_context_iterate_unlocked.isra.0 (libglib-2.0.so.0 + 0xb6eb4)
#2 0x00007f9379a47ad3 g_main_context_iteration (libglib-2.0.so.0 + 0x59ad3)
#3 0x00007f9379a47b29 glib_worker_main (libglib-2.0.so.0 + 0x59b29)
#4 0x00007f9379a7a523 g_thread_proxy (libglib-2.0.so.0 + 0x8c523)
#5 0x00007f9378c4d897 start_thread (libc.so.6 + 0x8e897)
#6 0x00007f9378cd4a5c __clone3 (libc.so.6 + 0x115a5c)
Stack trace of thread 3737:
#0 0x00007f9378cd4e62 epoll_wait (libc.so.6 + 0x115e62)
#1 0x0000564f49cce172 _ZN3mio4poll4Poll4poll17h51f7985e6c22c33cE (rpm-ostree + 0x8f5172)
#2 0x0000564f49cbecb7 _ZN5tokio7runtime2io6driver6Driver4turn17h0c24b7b5e974ec18E (rpm-ostree + 0x8e5cb7)
#3 0x0000564f49cc0b26 _ZN5tokio7runtime4time6Driver13park_internal17h0ad0738478a5c1a8E (rpm-ostree + 0x8e7b26)
#4 0x0000564f49cbdf91 _ZN5tokio7runtime9scheduler14current_thread7Context4park17h0b0740b25f3c759eE (rpm-ostree + 0x8e4f91)
#5 0x0000564f49601e88 _ZN10rpm_ostree4main17h5b28a65e60637af5E (rpm-ostree + 0x228e88)
#6 0x0000564f4959e306 _ZN3std10sys_common9backtrace28__rust_begin_short_backtrace17ha279d27be3640aecE (rpm-ostree + 0x1c5306)
#7 0x0000564f495a29ec _ZN3std2rt10lang_start28_$u7b$$u7b$closure$u7d$$u7d$17h77ecae052b4313d1E (rpm-ostree + 0x1c99ec)
#8 0x0000564f49d009c1 _ZN3std2rt19lang_start_internal17hd4969f33a050e23bE (rpm-ostree + 0x9279c1)
#9 0x0000564f49602417 main (rpm-ostree + 0x229417)
#10 0x00007f9378be714a __libc_start_call_main (libc.so.6 + 0x2814a)
#11 0x00007f9378be720b __libc_start_main@@GLIBC_2.34 (libc.so.6 + 0x2820b)
#12 0x0000564f49551935 _start (rpm-ostree + 0x178935)
Stack trace of thread 3814:
#0 0x00007f9378cd285d syscall (libc.so.6 + 0x11385d)
#1 0x00007f9379aa1b2d g_cond_wait (libglib-2.0.so.0 + 0xb3b2d)
#2 0x00007f9379a1522b g_async_queue_pop_intern_unlocked (libglib-2.0.so.0 + 0x2722b)
#3 0x00007f9379a7e393 g_thread_pool_spawn_thread (libglib-2.0.so.0 + 0x90393)
#4 0x00007f9379a7a523 g_thread_proxy (libglib-2.0.so.0 + 0x8c523)
#5 0x00007f9378c4d897 start_thread (libc.so.6 + 0x8e897)
#6 0x00007f9378cd4a5c __clone3 (libc.so.6 + 0x115a5c)
ELF object binary architecture: AMD x86-64
Running ostree’s fsck finds an error, though I am not sure how to go about fixing this or if it is even related. Could I simply run sudo ostree fsck -a --delete?
❯ sudo ostree fsck -a
Validating refs...
Validating refs in collections...
Enumerating commits...
Verifying content integrity of 130 commit objects...
fsck objects (145317/205920) [========= ] 70%In commits b664e2e5d6a71764762e1fc8f3811a1f3d505d8994951bb936fbb135fa7426fc, 52ac9beeb873e3577b3be8aeeed66b94452f83b004ec190a114b8fec0bf58df9, a96124a5494aa0225c630e32de6c84aed8fdb8cf3e272fac897d9fabd58e4efa, 9f5452967134d193c66e96d63d76e7bf1c99b5f7c503e752d091534fe8270d5c: fsck content object e2b13cdbe0ef6facc26c5938f51bcb18d5fc293d65ab846d19e3ce8c63df38f5: Corrupted file object; checksum expected='e2b13cdbe0ef6facc26c5938f51bcb18d5fc293d65ab846d19e3ce8c63df38f5' actual='933b8b10b69525ffe6c613f3a30e71b2a610432b40f29df5f9c60db06900b02b'
fsck objects (205920/205920) [=============] 100%
error: Repository corruption encountered
Apologies if this is the wrong place to ask, any help is highly appreciated!
