09 July 2026

Fine-tuning LLMs locally with Unsloth Studio — NFS storage, boot-on-start, and a self-inflicted security hole

I've wanted a proper local fine-tuning setup for a while. Cloud GPUs are fine until the bill arrives, and there's something satisfying about training a model on hardware sitting in the next room. So when a spare box on one of my Proxmox nodes ended up with four RTX 5060 Ti 16GB cards in it, I installed Unsloth Studio — the web UI from Unsloth (the open-source fine-tuning project built by Australian brothers Daniel and Michael Han — see their about page) — and set about making it behave the way I wanted.

The aim:

  • Unsloth Studio running in an unprivileged LXC container on Proxmox (container 117, the four GPUs passed through).
  • Its HuggingFace model cache pointed at my TrueNAS NFS share — the container only has a 64GB root disk, and a single bnb-4bit model is 6–13GB, so that fills up fast.
  • The studio starting on boot (it's a thing you launch by hand from a shell, which is no good for a box you want to just work).
  • Reachable at https://unsloth.mattsouthgate.co.uk, behind the same Cloudflare Access gate as the rest of my home lab.

It mostly went well. A couple of things bit back, and one of them — entirely my own doing — was a security hole I'll come to. Here's the whole thing, dead ends included.

Where the models live

The setting in the UI is Storage → Models Folder, and out of the box it points at /root/.cache/huggingface/hub — the standard HuggingFace hub cache. That's what I wanted on the NAS.

My first instinct was to mount the NFS share inside the container. Unfortunately that doesn't work in an unprivileged LXC:

# inside the container
mount -t nfs -o ro 10.140.3.200:/mnt/zrust/ml-models /mnt/nfs-test
mount: /mnt/nfs-test: permission denied.

The kernel blocks NFS mounts inside a user namespace, even with AppArmor unconfined. I briefly considered converting the container to privileged — which would let it mount NFS — but in a privileged container, container-root is host-root, and this is a box that downloads and runs arbitrary model code from the internet. That's exactly the boundary an unprivileged container exists to keep. Not worth it.

The right way is the boring way: mount the share on the Proxmox host, then bind-mount it into the container. The host already had it in /etc/fstab, so I only needed the bind and to set the container to start on boot:

# on the Proxmox host
pct set 117 -mp0 /mnt/ml-models,mp=/mnt/ml-models
pct set 117 -onboot 1
pct reboot 117

The unprivileged ownership dance

Here's the bit that catches people. An unprivileged LXC maps container-root (uid 0) to host uid 100000 (this is documented in the Proxmox container docs). A directory created on the share as host-root shows up as nobody inside the container, and the container can't write to it. So the cache directory has to be owned by 100000 on the host:

# on the host
mkdir -p /mnt/ml-models/llm/hf-cache /mnt/ml-models/llm/hf-xet
chown 100000:100000 /mnt/ml-models/llm/hf-cache /mnt/ml-models/llm/hf-xet

That chown only works if the NFS export isn't root-squashed — mine is no_root_squash, so it took. A check from inside the container confirmed it had landed:

# inside the container, post-reboot
mount | grep ml-models
# 10.140.3.200:/mnt/zrust/ml-models on /mnt/ml-models type nfs (rw,...)

echo hi > /mnt/ml-models/llm/hf-cache/.wtest && echo WRITE-OK
# WRITE-OK
ls -lan /mnt/ml-models/llm/ | grep hf-cache
# drwxr-xr-x 2 0 0 ... hf-cache    <- uid 0 (root) inside; 100000 on the host

What the host sees as 100000 the container sees as 0. Symlinks work over the mount too, which matters — the HuggingFace cache is full of them (snapshots are symlinks into a blobs store).

Telling the studio (and only the studio) to use it

I dug through the Unsloth Studio source to see how it decides where the cache goes. The relevant function is _setup_cache_env() in backend/utils/paths/storage_roots.py, and the key line is this:

for key, value in defaults.items():
    if key not in os.environ:
        os.environ[key] = value

It only sets the cache variables if you haven't already. So explicit environment variables win. That let me do something neat: point the hub cache at the NAS while leaving HF_HOME at its local default — which keeps the auth token ($HF_HOME/token) on local disk and off the shared NAS, where it has no business being.

HF_HUB_CACHE=/mnt/ml-models/llm/hf-cache
HF_XET_CACHE=/mnt/ml-models/llm/hf-xet
# HF_HOME left alone -> token stays at /root/.cache/huggingface/token

A trap worth flagging: that same function does a best-effort mkdir on the cache path. If you start the studio with the cache pointed at /mnt/ml-models/... before the bind-mount exists, it'll cheerfully create that directory on the local disk and then shadow the mount when it appears. Mount first, start second.

The token, which did not go quietly

Logging the HuggingFace token in should have been one command. It wasn't:

hf auth login --token hf_xxxx…
...
ValueError: Token claude-hub not found in /root/.cache/huggingface/stored_tokens

The newer hf CLI (from the huggingface_hub library) stores tokens under a name in a stored_tokens file and then tries to "activate" them — and that machinery tripped over itself, writing an empty entry and declaring the token missing. Rather than fight it, I wrote the plain token file directly, which huggingface_hub reads perfectly well:

printf '%s' 'hf_xxxx…' > /root/.cache/huggingface/token
chmod 600 /root/.cache/huggingface/token
hf auth whoami
# user: IdeasFixer

whoami hits the API, so that's the token genuinely validated, not merely written.

Starting on boot

Unsloth Studio is launched by hand (unsloth studio -H 0.0.0.0 -p 8888) — no service file. For a box I want to just work, that won't do, so it gets a systemd unit. The cache variables go in Environment= lines, because systemd doesn't read your shell profile:

# /etc/systemd/system/unsloth-studio.service
[Unit]
Description=Unsloth Studio (LAN on :8888, HF cache on TrueNAS NFS)
After=network-online.target
Wants=network-online.target
RequiresMountsFor=/mnt/ml-models

[Service]
Type=simple
User=root
Environment=HF_HUB_CACHE=/mnt/ml-models/llm/hf-cache
Environment=HF_XET_CACHE=/mnt/ml-models/llm/hf-xet
ExecStartPre=/usr/bin/test -d /mnt/ml-models/llm/hf-cache
ExecStart=/root/.local/bin/unsloth studio -H 0.0.0.0 -p 8888 --no-cloudflare
Restart=on-failure
RestartSec=5

[Install]
WantedBy=multi-user.target

(That --no-cloudflare flag is important — more on why in a moment.) onboot: 1 on the container handles the other half. I tested the whole chain by rebooting the container and watching it come back unattended:

Check Result
Container running after reboot ✅
NFS bind-mount present ✅
unsloth-studio auto-started + enabled ✅ active / enabled
Listening on :8888 ✅
HF_HUB_CACHE resolves to NFS, models present ✅ 3 models

A tidy hostname, via the existing Caddy

I didn't want a fresh tunnel for this. I already run Caddy (thanks to Matt Holt and the Caddy authors) as a reverse proxy behind my main Cloudflare Tunnel, fronting every other *.mattsouthgate.co.uk service. Adding one more is three small steps.

A Caddy vhost — and mind the http:// prefix, because the tunnel delivers on plain port 80 and a bare hostname triggers Caddy's automatic HTTPS, which sends a 308 redirect that loops straight back through the tunnel (ask me how I know):

# /etc/caddy/conf.d/unsloth.mattsouthgate.co.uk.caddy
http://unsloth.mattsouthgate.co.uk {
    reverse_proxy http://10.140.0.61:8888
}

Then the tunnel ingress (unsloth.mattsouthgate.co.uk → http://10.140.3.156, the Caddy box) and a proxied DNS CNAME to the tunnel. A curl confirmed it both works and is gated:

curl -sI https://unsloth.mattsouthgate.co.uk/
# HTTP/2 302
# location: https://mattsouthgate.cloudflareaccess.com/cdn-cgi/access/login/...

That 302 to the Cloudflare Access login is exactly what I wanted. Anyone reaching the studio has to pass Access and the studio's own password. Which brings me to the bit I got wrong.

The hole I dug for myself

While auditing what I'd built, I noticed something I should have caught immediately. Unsloth Studio, very helpfully, auto-creates its own free Cloudflare tunnel on startup so you can share it without any setup. Lovely feature. Except it means that alongside my carefully Access-gated hostname, the studio was also publishing a second, public *.trycloudflare.com URL — with no Access gate in front of it. The only thing standing between the open internet and a box with four GPUs and my HuggingFace token was the studio's login password (which, for the record, is a deliberately throwaway "get it working first" password I hadn't yet replaced — so, not much).

There it was in the process list and the logs:

ps aux | grep cloudflared
# /root/.unsloth/studio/bin/cloudflared tunnel --url http://localhost:8888 --no-autoupdate
journalctl -u unsloth-studio | grep trycloudflare
# Unsloth Studio running on https://wine-graduates-evaluated-herbal.trycloudflare.com

The fix is one flag — --no-cloudflare — which I'd already slipped into the unit file above. After a restart, the ungated door is gone:

# the old public URL — now dead
curl -s -o /dev/null -w "%{http_code}\n" https://wine-graduates-evaluated-herbal.trycloudflare.com/
# 530      (Cloudflare: origin tunnel is gone)

# my gated hostname — still serving
curl -s -o /dev/null -w "%{http_code}\n" https://unsloth.mattsouthgate.co.uk/
# 302      (redirect to Cloudflare Access login)

# and no tunnel process left
pgrep -a cloudflared
# NO cloudflared process — confirmed

Lesson, cheerfully relearned: when a tool offers to expose itself to the internet "for free and with zero config", check whether it's still doing that after you've built your own properly-gated route. A fuller security audit of the whole setup is a follow-up post of its own.

Which models?

These are 16GB cards, and Unsloth's open-source path fine-tunes on a single GPU per run (four cards means four parallel jobs, not one big 64GB job). For QLoRA — the 4-bit fine-tuning method from Tim Dettmers and colleagues, arXiv:2305.14314, which underpins the bnb-4bit format via the bitsandbytes library — that comfortably fits up to about 14B in 4-bit; 24B and up don't. So I pulled the Unsloth bnb-4bit dynamic quants (the training-ready format, not the GGUFs, which are for llama.cpp inference):

Model Base model author Size on disk
unsloth/Meta-Llama-3.1-8B-Instruct-unsloth-bnb-4bit Meta 6.0 GB
unsloth/phi-4-unsloth-bnb-4bit (14B) Microsoft 10.4 GB
unsloth/gemma-3-12b-it-unsloth-bnb-4bit Google DeepMind 12.8 GB
unsloth/gemma-3-4b-it-unsloth-bnb-4bit Google DeepMind 4.6 GB

gemma-3-27b-class and Mistral-Small-3.2-24B (14.6GB) were left out — their 4-bit weights alone meet or exceed 16GB before you've fit a single activation.

The proof it all hangs together — a fresh download lands on the NAS and not the local disk:

hf download hf-internal-testing/tiny-random-bert
ls -d /mnt/ml-models/llm/hf-cache/models--hf-internal-testing--tiny-random-bert  # present
ls -d /root/.cache/huggingface/hub/models--hf-internal-testing--tiny-random-bert # absent

And migrating the pre-existing 21GB cache off the root filesystem freed exactly what you'd hope:

# df -h /  (container root)
# before: 46G used / 63G  (77%)
# after:  26G used / 63G  (43%)

Summary

  • Unsloth Studio fine-tunes locally on four 16GB GPUs in an unprivileged Proxmox LXC.
  • Models live on TrueNAS over NFS (host mount + bind-mount + a chown 100000 so the unprivileged container can write); the auth token stays on local disk.
  • It starts on boot via systemd, verified across a full reboot.
  • It's reachable at a clean HTTPS hostname through the existing Caddy + Cloudflare setup, gated by Cloudflare Access — and the studio's own ungated tunnel is now disabled.

Future work

  • Replace the placeholder password — pass was only ever a get-it-running stand-in; it wants a real one now the routing is sorted.
  • Move the fine-tuned outputs to the NAS too. I redirected the download cache, but the outputs/exports/datasets directories are still on the local disk. They want symlinking onto the share — carefully, because the studio's SQLite databases (studio.db, auth.db) must stay local (SQLite and NFS don't get along).
  • Back up those config databases, which currently live only on the container's root disk.
  • A proper security audit of the whole home-lab edge — the trycloudflare surprise above earned it. That's the next post.

References and credits

With thanks to the people whose work this is built on:

I hope this saves someone the permission-denied head-scratching — and prompts you to check whether your tools have quietly opened a door you didn't ask for. Enjoy!

Matt

Posted by Fixer at 10:58

No comments:

Post a Comment

Subscribe to: Post Comments (Atom)