Bluefin-dx - The Bluefin Developer Experience

Bluefin Developer Experience (bluefin-dx) is a dedicated developer image with bundled tools. Unlike traditional Linux systems, the operating system and developer environment are explicitly and purposely separated.

This means that tooling is not installed on the host, and is instead containerized, in a virtual machine, or scoped to the user’s home directory. It is designed to meet the following use cases:

  • Endeavors to be the world’s most powerful cloud native developer environment
  • Full virtualization support
  • Provide fleet management of Bluefin and other Linux systems via Cockpit (Incomplete, needs a volunteer!)

The Cloud Native Approach

Bluefin goes “all in” on cloud native development and is used differently than a traditional distribution such as Ubuntu:

  • GUI apps are installed via Flatpak
  • Development is done in devcontainers
  • Command line applications are installed using homebrew
  • Preconfigured ad-hoc containers for Ubuntu, Fedora, Wolfi, and a custom bluefin-cli container. Use whichever distribution you want.

We strive to remove host level package management completely and consider these components as seperate layers. Communication between components is typically done via Flatpak Portals or via the container runtime.

This differs from apt, snap, and dnf where one packaging system handles both the graphical applications AND the command line applications. This decoupling is what provides greater system reliability, near unlimited choice of software, and “distributed by default” development.

In short, we picked it because this pattern is how servers are deployed in modern infrastructure. And the developers deploying those systems are already using homebrew and we want to leverage as much of that success as possible for the Linux desktop.

Introduction

The pattern in bluefin-dx is centered around container focused development using devcontainers. Since development is not dependent on the operating system image, you can use whatever you want, you do not need to use everything in here in order to be productive – think of them as prepaved paths that you can choose to use.

Dev Containers were chosen to facilitate distributed development, each project has a declarative environment that is intended to be start the user with a “best practice” cloud-native workflow out of the box. The Ultimate Guide to Dev Containers has a good write up of the advantages of using devcontainers.

Homebrew can also be used for installation of development tools.

Note: This is an opinionated developer workflow that differs from Fedora’s use of toolbox. Toolbox is included for people who prefer the upstream approach.

Instructions

Any Bluefin machine can turn on DX mode, ISOs are also provided for new installations.

Clean Installation

bluefin-dx-gts.iso (checksum)

Recommended for developers with AMD and Intel based GPUs (3.7GB, based on Fedora 38)

bluefin-dx-nvidia-gts.iso (checksum)

Recommended for developers with Nvidia GPUs (5.4GB, based on Fedora 38)

Enabling Developer Mode on an existing Bluefin installation

You can rebase to bluefin-dx by using the following command:

Step 1 of 2 - ujust devmode to enable or disable the dx mode, then reboot:

image

Step 2 of 2 - ujust dx-group - to add your user account to the right groups. Then log out and back in. This step only needs to be done once.

Like all Universal Blue images, switching is atomic, allowing for clean switching between modes depending on the use case.

Features

Quality of Life Improvements

  • Cockpit for local and remote management
  • A collection of well curated monospace fonts
  • Tailscale for VPN
  • Just task runner for post-install automation tasks. Check out our documentation for more information on using and customizing just.
  • fish and zsh available as optional shells, use ujust fish or ujust zsh and follow the prompts to configure them

Visual Studio Code with Docker

Visual Studio Code is included on the image as the default IDE. It comes with the devcontainers extension already installed. It’s the recommended developer experience, start here if you’re new to containerized development!

The most current Docker Engine is included by default and is set up to be the default container runtime for vscode. DevPod is included to use devcontainer functionality across clouds and self-hosted setups.

Command Line Experience

Manage your containers with Ptyxis’s built in container support:

image

Ptyxis also includes a host terminal so that you can quickly switch between containers and the host.

Other Tooling

JetBrains

ujust jetbrains-toolbox will fetch and install the JetBrains Toolbox application, which will manage the installation of the JetBrains set of tools. This application will handle installation, removal, and upgrade of the JetBrains products, and is handled completely in your home directory, independent of the operating system image.

Check the Jetbrains documentation for integrating those tools with the podman runtime. Also check out how to setup Jetbrains with devcontainers

Kubernetes and other Cloud Native Tooling

  • kind - Run a Kubernetes cluster on your machine. Run kind create cluster on the host to get started!
    • kubectl - Administer Kubernetes Clusters
    • helm, ko, flux, minio-client – if it’s an incubated project we intend to add it where appropriate

Virtualization and Container Runtimes

  • virt-manager and associated tooling (KVM, qemu)
  • Incus provides system containers
    • LXC and LXD are also provided for compatability reasons

Podman Development

All the upstream podman tools are included. This is the default system container runtime, and is the recommended developer configuration that Fedora ships with.

Though bluefin-dx defaults to docker and vscode for development, all of the Fedora upstream tools are included for those that prefer that experience.

6 Likes

How can I remove Jetbrains toolbox once installed ?

thanks @j0rge, thought I could have done it with the app manager or something…
All good!

On a fresh bluefin install, there is no devmode recipe. Instead, I found devmode-on and devmode-off. After running devmode-on I get:

Pulling manifest: ostree-unverified-image:oci:/var/ublue-os/image
error: Creating importer: Failed to invoke skopeo proxy method OpenImage: remote error: lstat /var/ublue-os: no such file or directory
error: Recipe `devmode-on` failed with exit code 1

What am I doing wrong?

Looks like you’re on a fresh install, this part in the status shows that you’re still on the image from initial installation: ostree-unverified-image:oci:/var/ublue-os/image

Do a just update, that will rebase you to a signed image and then reboot to apply all the updates. Then the command will be there. Sorry about the installer jank, hopefully we won’t need to deal with this for much longer. :smile:

1 Like

I just tried to rebase to bluefin from silverblue. No overlayed packages (I already did reset and cleaned everything), but I get the following error:

fdr@fdr-framework:~$ sudo rpm-ostree rebase ostree-image-signed:docker://ghcr.io/ublue-os/bluefin-dx-framework:39
Place your left index finger on the fingerprint reader
Pulling manifest: ostree-image-signed:docker://ghcr.io/ublue-os/bluefin-dx-framework:39
error: Preparing import: Fetching manifest: containers-policy.json specifies a default of `insecureAcceptAnything`; refusing usage

I’m out of lock trying to reason about this one :confused:
Any ideas?

Edit: some extra info: /etc/containers/policy.json is not edited, so it has the default values from /usr/etc

You can’t rebase to a signed image from Fedora (they don’t have our signing key), so you need to rebase to an unsigned image first, follow these instructions:

1 Like

With the recent Debian kernel drama, I’m all the more looking forward to the day when I can use bluefin-dx-nvidia as my daily driver!

2 Likes

Why is not all the commands listen when you just type just? For example there is no mention of ˙just devmode˙, and i think there is no mention of it in the docs, either unless I missed it.

just is outputting the commands here:

You can check out this section in the just docs to make sure your ~/.justfile has the right imports in it.

If you’re not getting results it’s likely that file isn’t set up right. However just upstream has merged in the import feature into the stable branch, which will require some adjustments on our part, so heads up on that: fix: Use import for just by KyleGospo · Pull Request #179 · ublue-os/config · GitHub

The devmode section is at the top of the document.

I’m getting this error when trying to use a dev container in vscode on a fresh installation

[1519 ms] Command failed: docker ps -q -a --filter label=vsc.devcontainer.volume.name=anaconda --filter label=vsc.devcontainer.volume.folder=anaconda
[1558 ms] Exit code 1
[1563 ms] Command failed: node /root/.vscode-remote-containers/dist/dev-containers-cli-0.327.0/dist/spec-node/devContainersSpecCLI.js read-configuration --workspace-folder /workspaces/anaconda --id-label vsc.devcontainer.volume.name=anaconda --id-label vsc.devcontainer.volume.folder=anaconda --log-level debug --log-format json --mount-workspace-git-root
[1564 ms] Exit code 1

Thanks for trying it! How fresh is your installation? We just landed the devcontainer support last night. Which container are you trying to launch?

I found a couple of potential issues with the benchmark code:

git clone https://github.com/aime-team/pytorch-benchmarks.git
cd pytorch-benchmarks
python3 main.py --num_gpus 2 --compile --model bert-large-uncased --data_name squad --global_batch_size 24

First, the code assumes two GPUs (--num_gpus 2).
Second, when I ran the code on my RTX A4500, it ran out of memory. I had about 15GB of the 20GB free at the time.

Both of these could limit who can successfully run the benchmark.

I had just installed it a few minutes before posting, but I’ll try again! It was just the default python container, but all of them were giving me this error.

Wow, this is a new world for me. First learning about immutable os , than i found this.
Lot of things to read :slight_smile:

1 Like

FYI, running just by itself isn’t printing the available commands as the discussion above indicates it should. Instead, I get the message error: Justfile contains no default recipe.

My ~/.justfile just imports /usr/share/ublue-os/justfile, and that in turn imports /usr/share/ublue-os/just/00-default.just, which contains a recipe called _default that does what j0rge descibes above, but that recipe doesn’t get executed as the default recipe in the installed version of just. After reviewing the just docs, it’s ambiguous to me whether this is correct behavior or a bug… the docs say that the default recipe is the first receipe, and bluefin’s _default is the first after processing imports, but perhaps the just dev meant the first recipe in the main justfile.

Edit: Oh, I see that there’s also a ujust command that behaves as described… this issue was discussed in this post. That makes sense, but given that the Bluefin announcment blog post uses plain just in its examples, this could continue confusing new users, at least until more getting started documentation is written. FWIW, I’d vote for sticking with the plain just command to avoid confusion, and just moving the default recipe into ~/.justfile if needed.

1 Like

I’ll edit the blog post to fix it up, sorry we’re getting a bunch of new users during the just transition and it’s taking a while to fix up the docs.

1 Like

Do you expect any issues to appear when running both docker containers and podman containers at the same time? For example, vscode open and attached to some work in a docker devcontainer, plus a prompt terminal attached to a distrobox (podman) wolfi container.

I note that the CoreOS docs advise against it:
Fedora CoreOS Frequently Asked Questions :: Fedora Docs (fedoraproject.org)

I’m wondering how relatable the problems are here.

@siosm pinged us and let us know but I can’t seem to find the github issue where we discussed that.

Here’s a bunch of related back story though: Recommend docker as the default for vscode/devcontainers · Issue #726 · ublue-os/bluefin · GitHub

I run with this setup all the time and haven’t had issues since we switched but I do keep an eye out for it. So far so good unless someone has had a bad experience that wants to share?