Bluefin-dx - The Bluefin Developer Experience

Bluefin
1

Bluefin Developer Experience (bluefin-dx) is a dedicated developer image with bundled tools. Unlike traditional Linux systems, the operating system and developer environment are explicitly and purposely decoupled.

This means that tooling is not installed on the host, and is instead containerized, in a virtual machine, or scoped to the user’s home directory.
It is designed to meet the following use cases:

Instructions

You can rebase to bluefin-dx by using the following command:

  • just devmode to enable or disable the dx mode:

image
image707×277 21.1 KB

Like all Universal Blue images, switching is atomic, allowing for clean switching between modes depending on the use case.

Features

Bluefin offers four “default” developer workflows. Since development is not dependant on the operating system image, you can use whatever you want.

Visual Studio Code

Visual Studio Code is included on the image to facilitate local development. It comes configured for usage with devcontainers and Podman via a small default configuration file.

Homebrew

just distrobox-brew will install the upstream container as a distrobox on your system.

Thanks to Osama Albahrani for implementing support in Homebrew.

Devpod

DevPod is included to provide infrastructure-independent and client-only reproducible developer environments, powered by devcontainers

Devbox

Nix-powered Development Experience powered by Devbox and Fleek

  • just nix-devbox to get started
  • just nix-devbox-global to install a global profile

Containerized Development with Distrobox

  • Built-in Ubuntu user space
  • Ctrl-Alt-u - will launch an Ubuntu image inside a terminal via Distrobox and your home directory will be transparently mounted for the Ubuntu image to access
  • A BlackBox terminal is used just for this configuration
  • Use this container for your typical CLI needs or to install software that is not available via Flatpak or Fedora
  • Optional ubuntu-toolbox image with Python, and other convenience development tools. just distrobox-bluefin to get started. To configure just follow the guide.
  • Optional universal image with Python, Node.js, JavaScript, TypeScript, C++, Java, C#, F#, .NET Core, PHP, Go, Ruby, and and Conda. just distrobox-universal to get started
  • just assemble shortcut to declaratively build distroboxes defined in /etc/distrobox/distrobox.ini
  • Refer to the Distrobox documentation for more information on using and configuring custom images
  • GNOME Terminal - Ctrl-Alt-t - will launch a host-level GNOME Terminal if you need to do host-level things in Fedora (you shouldn’t need to do much).

Other Tooling

JetBrains

just jetbrains-toolbox will fetch and install the JetBrains Toolbox application, which will manage the installation of the JetBrains set of tools. This application will handle installation, removal, and upgrade of the JetBrains products, and is handled completely in your home directory, independent of the operating system image.

Check the Jetbrains documentation for integrating those tools with the podman runtime. Also check out how to setup Jetbrains with devcontainers

Kubernetes and other Cloud Native Tooling

  • kind - Run a Kubernetes cluster on your machine. Run kind create cluster on the host to get started!
    • kubectl - Administer Kubernetes Clusters
    • helm, ko, flux, minio-client – if it’s an incubated project we intend to add it where appropriate

Docker Engine

The most current Docker Engine is included by default. Use just docker to turn it on, this will turn on the service and add your users to the docker group. You will then need to log out and back into your desktop.

To configure Visual Studio Code to use Docker go to the extensions section, and click the gear on the bottom right of the remote containers extension to configure it:

image
image725×247 29.1 KB

Then change the following defaults to point to docker-compose and docker:

image
image839×605 59.7 KB

Virtualization and Container Runtimes

  • virt-manager and associated tooling (KVM, qemu)
  • Incus provides system containers
    • LXC and LXD are also provided for compatability reasons

Machine Learning

Bluefin includes a NGC container that includes the latest stable PyTorch from Nvidia on top of Ubuntu LTS. It includes python3, pip3 , pandas, matplotlib, and scikit-learn.

!!! note
The additional pip packages are commonly used but not comprehensive. Only pip is used and mamba or conda are not currently tested.

Pre-requisites

You must already be using bluefin-dx-nvidia as it’s meant for those GPUs and has nvidia container toolkit support. If you want to test before downloading the larger nvidia container run nvidia-smi
inside a regular Ubuntu box (distrobox create --nvidia -i ubuntu) to check that everything works correctly.

To create the box use the following command. Note that this container is large (20+ GB!):

just distrobox-mlbox

To enter the working environment:

distrobox enter mlbox

Then the init-hooks will run once. After which, you should be able to run:

nvidia-smi

To check if GPUs are seen and enter the python repo run:

import torch;torch.cuda.is_available()

Various tests can be run to test a transformers inference or training job, or to git clone a pytorch benchmarks repo and run single or multi gpu commands: E.g. to test multi-gpu setup on two 3090s:

git clone https://github.com/aime-team/pytorch-benchmarks.git
cd pytorch-benchmarks
python3 main.py --num_gpus 2 --compile --model bert-large-uncased --data_name squad --global_batch_size 24

On other operating systems, use this .ini file and run:

distrobox assemble create --file /path/to/your/mlbox.ini

Quality of Life Improvements

  • Cockpit for local and remote management
  • A collection of well curated monospace fonts
  • systemd shutdown timers adjusted to 15 seconds
  • Tailscale for VPN
  • Just task runner for post-install automation tasks. Check out our documentation for more information on using and customizing just.
  • fish and zsh available as optional shells, use just fish or just zsh and follow the prompts to configure them