WTF is Cloud Native and what is all this

KubeCon US 2024 Project Report

(I have set this post as a wiki, I am tired and need help linking to stuff, thanks!)

Now that bootc is going into the CNCF that means we’re now a cloud native community. Welcome everyone, I’ve been here since 2015 or so, if you’re new to this way of Linuxing then don’t worry. I’m going to go into detail why this is a monumental change for Universal Blue.

If you’ve been as excited as I am about getting this thing done and as invisible as possible … we just gained a few million new developer friends. This is crucial as the Linux desktop has been underserved, and that’s a bunch of great people who know how to computer.

This is one of the reasons we say that why Universal Blue is not a distribution, we’re here to solve that exact problem, remove it from our lives. We want to concentrate on the workload, and make the OS invisible. :smiling_imp:

What is this report?

One of the benefits of being in cloud native is that center of gravity pulses hard around the KubeCon+KCCNs around the world. This year we’ve had Paris, Salt Lake City, and Hong Kong. And the first one in India is coming up! That means that our project now has a cadence that matches the rest of cloud native. We align with the industry naturally.

It also means that we’ll try to make updates around that time. Since this was a monumental one for us I thought I’d write a lengthy report. Please come ready with questions, no one’s going to yell at you, the idea is to get a goodm, written down status of where we stand in our quest.

Events

When I worked on Ubuntu at Canonical the pinnacle event was the Ubuntu Developer Summit. It’s where I met people whom I would become friends with for over 20 years now, many of whom do really awesome stuff in this industry. Community thrives when we’re together. Here’s the latest Ubuntu Summit vids.

By being in cloud native we can use those events as our first developer summits. We automatically have a worldwide network of places where other cloud native developers meet.

There were alot of Frameworks at KubeCon, and helm did a release on a Bluefin laptop! There’s definitely a growing set of people who love theirs, and they all run all sorts of Linuxes, it’s a diverse community. Just getting those enthusiasts together will help accelerate adoption.

The Curse of Enterprise

I know many of you have concerns about how your layered apps will live in the bootc world, so let me tldr it. But before that let me state this:

We’re not going to put you in a position where you are doomed. We will always support local layering. But we will default an image intended to run without local packages as the intended experience.

You may need to manually set a config in a file. But you do it once and move on with it, you just don’t mind the occasional bit of maintenance.

And also, I have no choice, many of you know I hate local layering with the fury of 1,000 suns. But in the real world, there’s tons of things in production that will need to be supported. That bodes well for Red Hat’s investment in the tech. That’s the classic open source model (there are others), but this is a good example that we should point out.

Enterprises pay companies for this kind of stuff. If there’s anything I learned from my time at VMware, enterprise software is so awful that the bootc team will be chasing after every edge case at some point haha.

What you’ll actually see

But I want Bluefin to be the one with the least amount of compromises, so we’ll strive for that pure image mode, I ain’t got time for this. Also we’re talking about the layer we’re at, X11 is gone, that’s got nothing to do with us, sorry folks. But I can see people are confused so …

As far as user experience goes, rpm-ostree is being split into two commands, bootc, and dnf. That’s it. :smile:

As a user you will see this (note that you need to use sudo with bootc currently but let’s show what it will look like):

bootc

bootc manages the image stuff, that’s it:

  • rpm-ostree statusbootc status
  • rpm-ostree updatebootc update
  • rpm-ostree rebase horrible-urlbootc switch ghcr.io/ublue-os/bluefin:stable
  • rpm-ostree rollbackbootc rollback

If you use ujust update or ujust rebase-helper you’ll just see nicer progress bars.

dnf

dnf will do your local package stuff:

  • rpm-ostree install vpn.rpmdnf install vpn.rpm
  • and then whatever dnf thing for third party repos, etc, it’s a dnf plugin. So whatever you have now

But also this doesn’t exist yet! So you’ll continue to use rpm-ostree for this. This will probably be sorted next release of Fedora but who knows.

Setting Expectations

I get the feeling that many of you think that dnf locally will give you the traditional linux experience but with the reliability of a bootc image.

You’re probably not going to get that. Manually maintaining individual packages always adds entropy we can’t account for. If you turn on local layering, then you’re saddled with the Curse of the Enterprise. I realize many of you will wear this as a badge of honor. :smile: But hey, I ain’t going back to that life haha. (Also it’s no big deal)

dnf5 will make a huge difference for how we build our images, we’ll be able to just cut out most of this stuff. We can replace all that rpm-ostree gunk with clean dnf installs existing Fedora users will be immediately able to recognize. Death to technical debt!

I hope that clears things up. Please ask questions.

What’s up with zstd:chunked?

Our last main boss is download efficiency. This is a high priority upstream, probably around this spring, so one more cycle.

sysext

And then we have the systexts. The Final Shape is here. @tulilirockz @gerblesh and zeglius have been working on sysexts, and if one of them could reply with some of the cool videos you’ve been showing off in the discord that would be awesome.

And since we’re a cloud native project, we share a common friend in Flatcar Linux (my preferred server distro, btw). They are heavily investing in sysexts for their distro, and we intend to utilize it as another powerful tool to get what we want.

I was able to introduce the podman team to the Flatcar team, and there’s tons of places where we can get big wins, like with composefs, and Fedora’s Timothee Ravier also has been doing a ton of work in that area. Seems like everyone wants this because it helps get rid of that local layer gunk. This is why we hate the term “immutable distro”, that makes no sense, people have been doing read only /usr since the UNIX days. It’s not about that, it’s about aligning ourselves with the model that can help make the linux desktop better, by a significant margin!

So wtf is cloud native?

We’ve gotten dunked on for the term “cloud native” because many of you had never heard of that term. But as I said in my video, this community moves quickly, and we’re now a part of it. Of course parts of the disk are immutable, it’s basic reliability. I expect my car to come with a seatbelt, but we’d upset people if we called traditional distros “Unreliable” or “Legacy”. In cloud native we call that a “pet”. You want one, you take care of it, I have a real beagle named Oscar which is better in every possible way:

We’ve taken the Linux desktop into an entirely new model, that’s why we don’t call it a distro. Distros just kind of fade into the background. People call this “distroless”. But I think most of you are realizing that making the linux desktop the same as the rest of linux is really nice!

This is the simplest definition of cloud native: One common way to linux, based around container technology. Server on any cloud provider, bare metal, a desktop, an HTPC, a handheld, and your gaming rig. It’s all the same thing, Linux.

I can’t even begin write that community aspects, so we’ll have to figure that out together. That’s the true win, the cloud native community itself is huge for us. I cannot WAIT to introduce you to them.

If I missed anything please ask a question, it will be useful to come back to this at KubeCon+KCCN London and see where we were at this point in the project’s life.

You can help by donating to the open source projects you love! Here’s our starter list.

33 Likes

This year marks my first KubeCon, and I cannot overstate how thankful I am for both Universal Blue and Bazzite, especially our teams and our ever-growing roster of users. You all made this dream come true for me, and I will never forget it

We’re all sold the narrative early on that we need to go to college to pursue successful careers. For various reasons, college didn’t work out for me. I had hope I could skip a step and started looking at entry level jobs. While hunting for work, I also filled my time working on personal projects, pursuing certs, and playing video games. One of those projects being a tool that automated the setup for my various OpenSUSE installations; something Universal Blue thankfully made obsolete. Thanks to what I’ve learned and the connections I’ve made working on Ublue, I get to wake up and do something I love every single day (package software and build images)

Looking back, it’s surreal to think this all started because I was simply trying to replace SteamOS on my Steam Deck with something better :joy:

When the Steam Deck was announced in 2021, I was in awe. The idea of playing games like The Elder Scrolls: Oblivion and Grand Theft Auto IV on the go was unreal. However, SteamOS left much to be desired. It’s PC, so I had PC expectations that SteamOS couldn’t accommodate. I wanted the latest kernel, BTRFS for everything, disk encryption (even if the touchscreen prompt came a bit later), and up-to-date packages. I wanted the flexibility I’d enjoyed using Fedora and OpenSUSE. That’s when I found Bazzite, and damn, could it print! (courtesy of cups)

Bazzite is not just a SteamOS replacement—it is something much bigger. It is a love letter to Linux gamers who have struggled with the endless cycle of maintenance. Thanks to Universal Blue, Bazzite is at the forefront of Cloud Native, and that’s something KubeCon really opened my eyes to

I visited Red Hat’s booth, where I had the opportunity of speaking with Dan Walsh and Collin Walters. Dan shared his belief that Bazzite is at the tip of the spear when it comes to bootc; he also often references Bazzite as an example when discussing bootc with others. Hearing that from him was very humbling. It’s clear that Bazzite is not only empowering Linux gamers, but it, along with Universal Blue, is also playing a crucial role in shaping Cloud Native technologies

KubeCon was something to behold. I had the pleasure of meeting Jorge, Marco, and Wayne the first day of the event. I also caught various talks covering everything from using Linux namespaces in place of pods in functional tests to the nuances of Kubernetes platform and API upgrades to the use of WASM in AI workflows. (Additionally, I spread the word of Wolfi over at Chainguard’s booth to countless other attendees)

I’m very excited for our future, especially now as more projects are making their way under the CNCF’s wing, strengthening the Cloud Native community and encouraging more collaboration than ever

Once again, thank you all! I’m so proud of what we’ve been able to achieve together, and I look forward to what we’re able to accomplish in the future :fire:

12 Likes

Hello from New Delhi. I got to know from this post that there is a Kubecon happening here next month. :sweat_smile:
So, here’s a PSA for those of you who’re traveling: air pollution is at a catastrophic level in Delhi right now. Please bring a few N95 or other masks to protect yourself from the air. Also, if possible, call your hotels/accommodation ahead of time and request an air purifier for your room. Hope you have a great time here. :grinning:

Edit-1: Honestly, they should mention the air pollution on this page: Health + Safety | LF Events

Edit-2: If anyone is interested in knowing the severity of the situation then this thread is an example of where the discourse around air is at at the moment: x.com

7 Likes

Hello everyone! About the Systemd-Sysext support, we’ve been working on it! We’ve sent a patch upstream that addresses the current issues that systemd-sysext has with SELinux, and, with initial testing, it seems to be working pretty nicely! Of couse there are a few things we still need to fix, but still, amazing progress that @gerblesh made so far with this issue!

Here are a few screenshots of me using some sysexts made by Timothee on my device.

They all seem to run pretty much as if they were installed through rpm-ostree as a layer through sysext pretty much fawlessly! The Chromium/VSCodium sysexts have hardware acceleration, too! All that on a system with absolutely no packages layered on top of Bluefin-dx

~> sudo bootc status                          
No staged image present                                                                               
Current booted image: ghcr.io/ublue-os/bluefin-dx:latest                                              
    Image version: latest-41.20241117.6 (2024-11-17 21:18:30 UTC)                                     
    Image digest: sha256:47f4c91dda60b6cfe90e14b39a5539d9ffbb72304776ac64a919e6186de9c1c5             
Current rollback image: ghcr.io/ublue-os/bluefin-dx:latest                                            
    Image version: latest-41.20241117.5 (2024-11-17 17:14:40 UTC)                                     
    Image digest: sha256:6378d29dbce3f88e93181e2b517c467b32de6eef20ae3dbd4b772e4d3f870859 

If you guys want to know what the hell are these sysext-things, please check out a few of these presentations:

Pottering and the GNOME-OS team have great write-ups about those:

https://blogs.gnome.org/alatiera/2023/08/04/developing-gnome-os-systemd-sysext/

Hopefully, all this work will lead to a better and more cohesive system for us all!

19 Likes

Is it possible to install kernel modules in a sysext?

I believe so! You probably wouldn’t be able to replace things like the init system or the kernel, but kernel modules that get loaded after boot probably could work. I’ve heard of someone on the GNOME OS team running the Nvidia proprietary drivers as a sysext.

1 Like

Pardon if this has been addressed elsewhere, but do we have some ideas on the limitations of sysexts? For example:

  1. Can personal customizations go into a sysext along with a specified runtime?
  2. Given the video above and the brief discussion of working on incompatibilities between different sysexts, are sysexts capable of specifying multiple runtimes that could alleviate this potential incompatibility issue somewhat?

Thanks in advance for any insight. I’m trying to envision the possibilities of what sysexts can do, so it’s helpful to have a ceiling on what it cannot do.

They are supposed to be additive to your system.

If they require a dependency on your host and your host doesn’t have it… They won’t work.

There is some basic checking possible by using items from your /etc/os-release.

Right now the recommendation is either provide static binaries or match the os-release to the system that built it.

We are still looking at how to tie versions to your system.

1 Like

Hello everyone, the patch that makes sysexts work properly on Fedora got merged! You all should be able to use them most likely whenever Fedora 42 gets released! We will integrate them a bit more into the system and add some documentation til then. (do not try them right now since they WILL break your installation if you dont know how to get them fixed again)

12 Likes