I'm confused and new on some things, IE several applications for AI,

Bazzite
1

I’m trying to see if I can get Bazzite running with local LLM or Ai models for local text or text to image. I have LM studio installed, but alot of the models keep crashing (Z-image and several other image based llm). I also have Jan installed from the Bazaar, but can’t make it past the initial download of the 2.7gb Jan model download on the startup page.

If it helps, I’m running a mini AMD pc with Ryzen 7 8845hs, Radeon 780m (With 12gb of total Vram allocated), 20 out of 32gb of ddr5 ram available, and a 1tb nvme. Is there something I’m missing, like outdated drivers? Outdated firmware? Missing files or folders? Installation for some of these programs are corrupt or not directed to the right place (This one, I experimented with a WebGUI local Image generator. But, failed to have it load back up)?

I also tried messing around with ROCm, but can’t get passed Pytorch installation. And, I can’t seem to get ComfyUi installed correctly on my system (Maybe it’s an incompatibility issue?). Sorry, I’m still new to Fedora based Distros.

2

I think there are 2 problems.

  1. In order to install most AI software you need to use tools like docker, AppImage and distrobox to run things on Bazzite. I highly recommed rebasing to Bazzite-DX as it has additional tooling to help you do this. If software wants you to install dependencies to your base system, you have to layer them in with rpm-ostree but it makes updates slower and prone to break.
  2. I run LM-Studio and invoke-AI without issues on my system. I have a 7900XTX that runs through rocm without much tweaks and a 6600M with 8GB on a laptop for simpler AI models. Non supported GPUs for ROCM often require flags. See Does ROCm 5.7 support Radeon 780M (gfx1103)? · ROCm/ROCm · Discussion #2631 · GitHub for stuff specific to your GPU. As for running models, it’s often best done on dedicated GPU cards as there can be issues running on iGPUs (exception Strix Halo). Don’t know what’s going wrong in your LM studio but I would look through logs and see if it’s that your model size is too large or the iGPU doesn’t have the right compute.