Unsloth Studio Installation Summary (on Debian)

Final Working Method: Docker (Recommended)

Why Docker? The local uv / Python installation was very unstable due to compatibility issues between Unsloth 2026.4.4, PyTorch 2.6, and torchao. Docker provides a stable, pre-configured environment.

Step 1: Install NVIDIA Container Toolkit

Bash

curl -fsSL https://nvidia.github.io/libnvidia-container/gpgkey | sudo gpg --dearmor -o /usr/share/keyrings/nvidia-container-toolkit-keyring.gpg

curl -s -L https://nvidia.github.io/libnvidia-container/stable/deb/nvidia-container-toolkit.list | \
  sed 's#deb https://#deb [signed-by=/usr/share/keyrings/nvidia-container-toolkit-keyring.gpg] https://#g' | \
  sudo tee /etc/apt/sources.list.d/nvidia-container-toolkit.list

sudo apt-get update
sudo apt-get install -y nvidia-container-toolkit

sudo nvidia-ctk runtime configure --runtime=docker
sudo systemctl restart docker

Step 2: Test GPU Access in Docker

Bash

docker run --rm --gpus all nvidia/cuda:12.4.1-base-ubuntu22.04 nvidia-smi

(You should see your GT 1030 listed.)

Step 3: Launch Unsloth Studio

Bash

# Remove old container if exists
docker rm -f unsloth-studio 2>/dev/null || true

# Start Unsloth Studio (using port 8001 to avoid conflict with Portainer)
docker run -d \
  --name unsloth-studio \
  --gpus all \
  -p 8888:8888 \
  -p 8001:8000 \
  -v ~/.unsloth:/root/.unsloth \
  -v $(pwd)/unsloth_data:/workspace/data \
  --restart unless-stopped \
  unsloth/unsloth:latest

Access Unsloth Studio

(Optional) Add password on first run:

Bash

-e JUPYTER_PASSWORD="your_strong_password"

Hardware Notes (GT 1030 – 2GB VRAM)

  • Use small models only (1B–3B range)
  • Prefer GGUF format in Q4_K_M or Q5_K_M
  • Good starter models:
    • unsloth/gemma-2-2b-it-GGUF → Q5_K_M or Q4_K_M
    • bartowski/gemma-2-2b-it-GGUF → Q5_K_M
    • phi-3-mini-4k-instruct-GGUF

Avoid: 7B+ models, Q6_K or higher quants.

Quick Commands Summary

  • Check running containers: docker ps
  • View logs: docker logs unsloth-studio –tail 100
  • Stop Studio: docker stop unsloth-studio
  • Restart Studio: docker restart unsloth-studio
  • Remove Studio: docker rm -f unsloth-studio