03 August 2026

Building a Multi-Model Audio Transcription Pipeline on PVE2: Part 1 — Choosing the Models

I take a recording device to client meetings. The recordings come back to the laptop sounding like someone had a conversation in a server room while a hoover ran in the background — which, in a couple of cases, is exactly what happened. Single-model Whisper does its best and produces something coherent, but "something coherent" and "accurate verbatim transcript with speaker attribution" are rather different things, and when I'm trying to reconstruct who said what about a project deadline, the distinction matters.

The current setup — one faster-whisper CPU container on the laptop, single model, no diarisation, no word-level timestamps — was always a temporary measure. It was time to do it properly: GPU-accelerated, multiple independent models cross-checking each other, speaker diarisation for attribution, word-level timestamps, and a local LLM to reconcile the disagreements and produce a consultancy summary. All of it running in a Proxmox LXC on PVE2 (four idle RTX 5060 Ti cards, ~16 GB VRAM each), audio stored on TrueNAS.

Before writing a single line of pipeline code, I spent a session researching the current state of the art. This post documents what I found, including the wrong turns. Part 2 goes deeper on how those benchmarks are actually constructed and what we found when we looked harder. Part 3 has the results of running the ensemble on real recordings.

A quick glossary

This post uses some ASR-specific jargon. For those not already immersed in it:

WER (Word Error Rate) — the standard accuracy metric: the percentage of words that were substituted, deleted, or inserted by the model compared to the correct transcript. Lower is better. 5% WER means 5 words in every 100 were wrong.

RTFx (Real-Time Factor, multiplied) — how many times faster than real-time the model processes audio. RTFx 3,380 means a one-hour recording finishes in about one second once the model is warm.

CTC (Connectionist Temporal Classification) — a training objective that lets a model align audio frames to text without needing pre-labelled boundaries. Produces fast, accurate models without an autoregressive decoder.

TDT (Token-and-Duration Transducer) — a variant of the RNN-Transducer architecture that predicts both the output token and how many frames to skip simultaneously, giving very high throughput while preserving timestamp accuracy.

GER (Generative Error Correction) — using a large language model to post-process ASR output, fixing errors by leveraging the LLM's understanding of language rather than acoustic signal.

ROVER (Recogniser Output Voting Error Reduction) — a 1997 algorithm that combines multiple ASR hypotheses by aligning them word-by-word and picking the most-voted word at each position.

Diarisation — segmenting audio into speaker turns and labelling them ("who spoke when"). Anonymous labels only (Speaker 1, Speaker 2); speaker identification by name is a separate problem.

VAD (Voice Activity Detection) — detecting which portions of audio contain speech vs silence or noise, so ASR engines don't waste compute on empty segments.

Forced alignment — matching an already-known transcript back to the audio to produce word-level timestamps. Used when a model doesn't natively output timestamps.

NeMo — NVIDIA's open-source toolkit for training and serving speech and language models. Most NVIDIA ASR models (Parakeet, Canary) are NeMo-native.

Why multiple models?

The core idea is simple: each ASR engine makes different mistakes. A Conformer-based model misreads differently to a Transformer-based one; a model fine-tuned on read speech falls apart on spontaneous conversation differently to one trained on telephone calls. If you run three or four architecturally distinct engines on the same audio and then ask a local LLM to reconcile the differences — rather than doing a naive majority vote — you get something closer to what was actually said than any single engine produces alone.

The alternative, ROVER, aligns N-best hypotheses and picks the most frequent word at each position. It's fast and deterministic. It's also twenty-five years old and, as the research notes, often performs worse than the ASR baseline when the hypotheses aren't already good — the bad guesses vote each other up rather than out. LLM-based generative error correction (GER) is the more interesting approach: rather than voting on words, you feed all the hypotheses to a language model and ask it to infer what was most likely said, with the constraint that it shouldn't invent content. The GenSEC challenge at IEEE SLT 2024 (Yang et al., arXiv:2409.09785) puts this on a proper footing as a benchmark task.

The catch: the LLM has to be constrained. A zero-shot prompt-only LLM will hallucinate fluent text that sounds plausible but was never spoken — which is worse than any of the raw transcripts. Low temperature, an explicit "do not generate content not present in at least one hypothesis" instruction, and [inaudible] for segments where no engine agrees, helps keep it honest.

Finding the leaderboard

My first stop was the Open ASR Leaderboard — a collaboration between Hugging Face, NVIDIA, Mistral AI, and the University of Cambridge that evaluates 86 open-source and commercial models across 12 datasets. If a model is genuinely competitive for English speech recognition, it's on here. Part 2 goes into how the leaderboard is constructed and what its numbers actually mean for noisy meeting recordings.

The dedicated ASR family

These are models built specifically for speech recognition, without the overhead of a general-purpose language model backbone.

NVIDIA NeMo Canary-Qwen-2.5B

HuggingFace repo: nvidia/canary-qwen-2.5b
Leaderboard: #1 average WER, 5.63% on English

A Conformer encoder paired with a Qwen 2.5B LLM decoder — an interesting architecture because you get a language model on the output side without running a separate reconciliation step. CC-BY-4.0 licence, English-focused on the published benchmarks. It needs forced alignment for word-level timestamps, since the LLM decoder doesn't output them natively.

NVIDIA NeMo Parakeet-TDT-0.6b-v2

HuggingFace repo: nvidia/parakeet-tdt-0.6b-v2
WER: 6.05% average; LibriSpeech test-clean 1.69%

A Conformer-TDT hybrid at 600M parameters producing native word-level timestamps — no forced alignment step needed. RTFx 3,380 with batch size 128: in practice a one-hour recording processes in about one second once the model is warm. English-only, CC-BY-4.0, around 2 GB VRAM. There's now a v3 (nvidia/parakeet-tdt-0.6b-v3) covering 25 European languages at 6.32% WER, worth considering if recordings include anything other than English.

The model card confirms it's optimised for NVIDIA Ampere, Blackwell, Hopper, and Volta architectures — so the 5060 Ti (sm_120) should be fine, though I'll verify that before trusting it.

WhisperX with openai/whisper-large-v3

HuggingFace repo: openai/whisper-large-v3
WER: ~7.4% average English

WhisperX (github.com/m-bain/whisperX, v3.8.6) wraps faster-whisper with wav2vec2 forced alignment for word-level timestamps and pyannote-audio speaker diarisation. Despite not being the accuracy leader, it stays in the ensemble for two reasons: (1) Whisper large-v3 was trained on a vast, diverse dataset and handles UK accents and background noise better than cleaner-trained models; (2) it's the only engine here that provides speaker diarisation natively, making it the timing and attribution backbone. WhisperX's assign_word_speakers attaches speaker labels to every word; the other engines' text hypotheses anchor to that skeleton.

Pyannote requires accepting a gated model licence on HuggingFace — the HF token in ~/.cache/huggingface/token needs to be present before first run.

CrisperWhisper

HuggingFace repo: nyrahealth/CrisperWhisper
WER: ~6.67% average

A fine-tuned Whisper variant optimised for verbatim transcription, including disfluencies like "um", "uh", and false starts that standard Whisper smooths over. Timestamp-specialised models tend to be more useful on messy real-world audio than their leaderboard numbers suggest — meetings have rather a lot of filler words.

IBM Granite Speech 3.3

HuggingFace repo: ibm-granite/granite-speech-3.3-8b
WER: ~5.74% (8B), ~6.00% (2B variant)
Apache-2.0 licence

IBM's entry, sitting near the top of the leaderboard. Apache-2.0 is a more permissive licence than CC-BY for anything commercial. The 2B variant fits alongside the rest of the ASR pack on a single GPU; the 8B needs its own card.

The voice-LLM family

Large language models that accept audio as a direct input modality. Architecturally very different to the dedicated ASR engines — they make different mistakes — which is precisely why they're useful in an ensemble.

Mistral Voxtral-Mini-3B-2507

HuggingFace repo: mistralai/Voxtral-Mini-3B-2507
Released: 15 July 2025
Apache-2.0 licence
Max audio: 30 minutes per call

The most practical voice-LLM for this pipeline. Runs in vLLM with the Mistral tokenizer mode and exposes an OpenAI-compatible /v1/audio/transcriptions endpoint natively:

vllm serve mistralai/Voxtral-Mini-3B-2507 \
  --tokenizer_mode mistral \
  --config_format mistral \
  --load_format mistral

The pipeline can POST audio files to it exactly as it would to OpenAI Whisper's API — no bespoke client code. Outperforms Whisper large-v3 on transcription benchmarks, runs in ~9.5 GB VRAM in bf16/fp16. A 90-minute meeting needs chunking into three overlapping segments; the preprocessing step handles this.

Microsoft Phi-4-multimodal-instruct

HuggingFace repo: microsoft/Phi-4-multimodal-instruct
WER: 6.14%
MIT licence
Max audio per ASR call: 40 seconds

A 5.6B-parameter multimodal model with a 128K token context window. The 40-second ASR audio cap means chunking is required for long meetings — similar to Gemma 4's 30-second limit, which I'll come back to. MIT licence, architecturally completely different from the dedicated ASR engines. Fits on a single 16 GB card.

Gemma 4 E4B (experimental)

HuggingFace repo: google/gemma-4-E4B-it
Released: 2 April 2026
Apache-2.0 licence
Max audio: 30 seconds

I want to address the naming confusion here because it took a while to resolve. There are two different Google model families that sound similar:

  • Gemma 3n (google/gemma-3n-E4B-it) — the predecessor, ASR-capable
  • Gemma 4 (google/gemma-4-E4B-it) — released April 2026, genuinely separate architecture, also ASR-capable

Google has a history of reusing model family names in confusing ways (the Gemma 1/2/3/3n/4 progression is not linear). Gemma 4 does exist as a distinct series. Its 30-second audio window is designed for on-device/phone deployment; for a 90-minute meeting that means 25-second chunks with overlapping boundaries. Whether the architectural diversity it adds to the ensemble justifies the chunking complexity is a build-time decision. I'm including it as optional.

Kyutai STT-2.6b-en

HuggingFace repo: kyutai/stt-2.6b-en
WER: 6.4% on Open ASR Leaderboard
CC-BY-4.0 licence

A streaming decoder-only Transformer from Kyutai using Mimi audio tokenization. 2.6B parameters, handles audio up to 2 hours in a single pass with a 2.5-second streaming delay. Word-level timestamps require a 2.5-second offset adjustment. Small enough to co-reside with other models. A cheap extra vote if VRAM headroom allows.

The models I chose not to include

Ultravox — audio understanding and instruction-following, not verbatim transcription. Different job.

GLM-4-Voice, Moshi, Step-Audio — speech output / dialogue systems. They produce speech, not text.

Kimi-Audio — Moonshot AI's audio model, competitive on Chinese benchmarks, questionable data practices for anything touching client-confidential material.

Qwen3-Omni-30B-A3B — genuinely impressive (LibriSpeech test-clean 1.22%), but needs two GPUs and doesn't have native word timestamps. Too complex for the return at this stage.

I'll admit Part 2 has one addition I missed here — there's a model that belongs in this ensemble and wasn't on my initial radar. More on that there.

The Blackwell trap: CTranslate2 and sm_120

PVE2's GPUs are RTX 5060 Ti — Blackwell architecture, compute capability sm_120. This was the most important discovery of the research phase, because it affects every model using CTranslate2 as its inference backend (which includes faster-whisper, and therefore WhisperX).

The error when running INT8 compute on sm_120:

cuBLAS failed with status CUBLAS_STATUS_NOT_SUPPORTED

Blackwell's INT8 tensor cores require specific padding that the current cuBLAS API version used by CTranslate2 doesn't supply. The fix: use float16 instead of INT8. Straightforward to configure, but it means VRAM estimates for faster-whisper models are higher than documentation suggests — whisper-large-v3 in float16 is roughly twice the VRAM of int8. There's a fix merged (CTranslate2 issue #1865, PR #1937), but I'll test before assuming it's resolved.

For PyTorch itself, PyTorch 2.7 is the first stable release with sm_120 kernel support, via the cu128 wheel index:

uv pip install torch==2.7.0 --index-url https://download.pytorch.org/whl/cu128

Any model that pulls in an older torch without specifying the index will fail at runtime with "no kernel image available for device 0". The vLLM stack (Voxtral, summary LLM) and the ASR stack (WhisperX/CTranslate2) have conflicting torch pins — they go into two separate isolated venvs. Phase 3 of the build clears this gate before touching pipeline code.

SubtitleEdit users hit this first and documented it well (issue #10180).

The session rate limit, and why I mention it

At one point during the research phase, the agent session hit the hourly rate limit and had to pause until it reset. Both research agents — one covering dedicated ASR, one covering voice-capable LLMs — completed their full reports after the limit cleared. I'm mentioning it partly because the blog post is meant to be honest about the process, and partly because it had a consequence: one significant model was entirely absent from the initial draft and only surfaced in the follow-up review. That's in Part 2.

The initial ensemble

GPU layout, models loaded per-job rather than all resident at once:

GPU Role Models
GPU 0 Voice LLM transcription Voxtral-Mini-3B-2507 (~9.5 GB bf16)
GPU 1 Voice LLM transcription Phi-4-multimodal-instruct (~12 GB)
GPU 2 Dedicated ASR pack WhisperX (large-v3) + Parakeet-TDT-0.6b-v2 + whisper-large-v3-turbo
GPU 3 Reconciliation + summary Qwen3-14B-AWQ via vLLM

WhisperX provides diarisation and the timing backbone. Parakeet and Whisper turbo add two more dedicated-ASR hypotheses. Voxtral and Phi-4 contribute two architecturally distinct voice-LLM hypotheses. All five go to the reconciliation LLM on GPU 3.

Outputs per job: transcript.json (words + timestamps + speaker labels + all per-segment hypotheses for auditing), transcript.md (human-readable, speaker turns with [mm:ss] markers), summary.md (exec summary, decisions, action items by speaker). Both .md files mirrored into the KnowledgeBase for RAG.

Part 2 covers how the ensemble got revised once I looked at the benchmarks more carefully. Part 3 has what it actually produced on real recordings.

References

Ta ta for now — Part 2 looks at what the benchmarks actually measure, and where the initial model list fell short.

Posted by Fixer at 17:13 0 comments

19 July 2026

Building a Multi-Model Audio Transcription Pipeline on PVE2: Part 2 — What the Benchmarks Actually Measure

This is Part 2 of a series. Part 1 covered the initial model selection and design decisions. Part 3 will have the actual test results.

Part 1 ended with an ensemble of five models and a GPU layout. This part is a step back: before running anything, it's worth understanding what the benchmark numbers actually mean, whether the leaderboard rankings translate to real meeting recordings, and — most honestly — which model I'd missed entirely the first time around.

How the Open ASR Leaderboard works

The Open ASR Leaderboard is a collaboration between Hugging Face, NVIDIA, Mistral AI, and the University of Cambridge. It evaluates 86 models (open-source and commercial APIs) across 12 standardised datasets, producing reproducible results using open-source toolkits (ESPNet, NeMo, SpeechBrain, HuggingFace Transformers). The accompanying paper (arXiv:2510.06961) describes the methodology in detail.

The 12 datasets span several distinct conditions:

Dataset What it tests
LibriSpeech test-clean Read speech from audiobooks. Studio-quality, single speaker, no noise.
LibriSpeech test-other Same audiobook data but harder accents and recording conditions.
TED-LIUM 3 Conference talks. Scripted or semi-scripted, moderate noise.
AMI meeting corpus Real multi-speaker meetings. Overlapping speech, room acoustics, variable mic quality.
GigaSpeech Web audio: podcasts, YouTube, diverse conditions.
Earnings-22 Financial earnings calls. Domain-specific vocabulary, telephone audio quality.
CallHome Conversational telephone calls between family members.
VoxPopuli European Parliament speech (multilingual track).
Common Voice Crowdsourced speech across many languages.
FLEURS Read sentences across 102 languages.
MLS (Multilingual LibriSpeech) Audiobook data for European languages.
CV 16 Common Voice, specific to 16 languages.

The English average WER that headlines the leaderboard is a mean across a subset of these. LibriSpeech, TED-LIUM, AMI, GigaSpeech, Earnings-22, CallHome, and Common Voice all feed into it. A model's headline figure is therefore a blend of clean and noisy conditions — but the blend is dominated by the cleaner datasets, which makes the headline look better than meeting-only performance would suggest.

The gap between benchmarks and meeting recordings

This is the most important thing to understand before picking models for this use case.

LibriSpeech test-clean is read speech from audiobooks, recorded in quiet conditions, by a single speaker per file, with a professional microphone. It is the easiest English ASR benchmark by a considerable margin. NVIDIA Parakeet-TDT-0.6b-v2 achieves 1.69% WER on LibriSpeech test-clean. On the AMI meeting corpus — which captures real multi-speaker meetings in rooms — the same model scores 11.16% WER. That's a 6.6× degradation.

Whisper large-v3 degrades less sharply (around 3.5×) on noisy conditions because it was trained on noisier, more diverse data. But it starts from a higher WER on clean benchmarks, so it ends up in a similar place in practice. IBM Granite Speech 3.3 reports only 7.54% relative degradation from clean to noisy conditions — unusually low, suggesting deliberate noise-robustness training.

The upshot: for selecting models for poor-quality office recordings, the AMI score and any available noise-condition figures are more relevant than the LibriSpeech headline. The leaderboard presents both where available; it's worth checking rather than reading only the top-line average.

A more complete model comparison

Here's the picture when you include all the serious contenders, with the metrics that matter most for this use case:

Model HF repo Avg WER AMI WER Noise notes Timestamps Licence VRAM (float16)
Canary-Qwen-2.5B nvidia/canary-qwen-2.5b 5.63% — 2.41% at 10 dB SNR forced align CC-BY-4.0 ~8 GB
IBM Granite 3.3 8B ibm-granite/granite-speech-3.3-8b 5.74% — 7.54% relative degradation forced align Apache-2.0 ~18 GB
IBM Granite 3.3 2B ibm-granite/granite-speech-3.3-2b ~6.00% — similar noise profile forced align Apache-2.0 ~5 GB
Qwen3-ASR-1.7B Qwen/Qwen3-ASR-1.7B ~6% — 16.17% vs Whisper's 63.17% under extreme noise native Apache-2.0 ~4 GB
Phi-4-multimodal microsoft/Phi-4-multimodal-instruct 6.14% — — no native (40-sec cap) MIT ~12 GB
Parakeet-TDT-0.6b-v2 nvidia/parakeet-tdt-0.6b-v2 6.05% 11.16% — native CC-BY-4.0 ~2 GB
CrisperWhisper nyrahealth/CrisperWhisper 6.67% — disfluency-aware specialised — ~6 GB
Kyutai STT-2.6b-en kyutai/stt-2.6b-en 6.4% — 2-hr files, streaming token-level CC-BY-4.0 ~6 GB
Voxtral-Mini-3B mistralai/Voxtral-Mini-3B-2507 — — beats Whisper-v3 no native (30-min cap) Apache-2.0 ~9.5 GB
Whisper large-v3 (WhisperX) openai/whisper-large-v3 7.4% ~9% diverse training data wav2vec2 align MIT ~6 GB
Gemma 4 E4B google/gemma-4-E4B-it — — — no native (30-sec cap) Apache-2.0 ~8 GB

The commercial-API leaders (ElevenLabs on long-form, RevAI, Speechmatics) don't appear here because there's no self-hosted option. For transcribing client-confidential recordings, sending audio to a third-party API isn't on the table regardless of the accuracy numbers.

The model I missed: Qwen3-ASR

Qwen3-ASR (GitHub: QwenLM/Qwen3-ASR) was released by Alibaba Cloud's Qwen team at the end of January 2026, under Apache-2.0, covering 52 languages. The technical report is arXiv:2601.21337. It has over 700 million downloads on HuggingFace, which suggests it's found a real audience.

The number that makes it relevant here: under extreme noise conditions, Qwen3-ASR-1.7B achieves 16.17% WER compared to Whisper large-v3's 63.17% — a 3.9× advantage. In moderate noise the gap narrows, but the model was specifically designed to handle acoustic environments that simpler models fall apart on: significant ambient noise, background music, overlapping voices, regional dialects.

For recordings made in server rooms, open-plan offices, and cafes — which describes several of the recordings I want to transcribe — this is the most relevant differentiator on the whole leaderboard.

Two variants: Qwen3-ASR-1.7B (flagship, maximum accuracy) and Qwen3-ASR-0.6B (lightweight, ~600M parameters). The 1.7B fits at ~4 GB in float16, small enough to join the dedicated ASR pack on GPU 2.

The reason it wasn't in Part 1: it surfaced only in the follow-up review pass, after the initial research agents had already completed. That's now corrected.

Some honest caveats about the plan

On the LLM reconciler. The GenSEC paper (arXiv:2409.09785) is unambiguous that zero-shot prompt-only LLMs frequently make ASR output worse — they're fluent but they hallucinate. The literature's wins on GER come from fine-tuned models (Whispering LLaMA, arXiv:2310.06434, being the canonical example). I'm using a general-purpose Qwen3-14B-AWQ with a carefully constrained prompt. That's a reasonable starting point but not the same thing, and I'll be watching the per-segment hypotheses in the JSON output early on to see whether the LLM is actually helping or just confidently rearranging the furniture.

On pyannote diarisation. Pyannote works well on clean two-speaker audio with good microphone separation. Meeting recordings — overlapping speech, varying acoustic distance, sometimes three or four speakers — are harder. Diarisation error rates on real meetings can be 20–30% even with good tooling. The timestamps will be accurate (wav2vec2 forced alignment is robust); the speaker attribution will be best-effort. The auditable transcript.json with per-segment hypotheses lets you check and correct speaker assignments manually if needed.

On the short audio caps. Phi-4-multimodal handles 40 seconds per ASR call; Voxtral handles 30 minutes; Gemma 4 handles 30 seconds. All three need the audio chunked with overlapping boundaries before submission. The preprocessing step handles this — but it's worth noting that the "just POST the file" simplicity only applies to Parakeet, Canary-Qwen, WhisperX, and Qwen3-ASR.

The revised ensemble

With Qwen3-ASR added and whisper-large-v3-turbo (which was in the GPU table in Part 1 but never properly introduced) replaced:

GPU Role Models
GPU 0 Voice LLM transcription Voxtral-Mini-3B-2507 (~9.5 GB bf16)
GPU 1 Voice LLM transcription Phi-4-multimodal-instruct (~12 GB)
GPU 2 Dedicated ASR pack WhisperX (large-v3) + Parakeet-TDT-0.6b-v2 + Qwen3-ASR-1.7B
GPU 3 Reconciliation + summary Qwen3-14B-AWQ via vLLM

WhisperX remains the diarisation and timing backbone. Parakeet contributes native word timestamps and extraordinary throughput. Qwen3-ASR adds the noise-robustness the ensemble was previously missing. Voxtral and Phi-4 give two architecturally distinct voice-LLM hypotheses. Kyutai STT, Canary-Qwen, and IBM Granite 3.3 2B are on deck as optional additions if GPU 2 VRAM headroom allows — at build time I'll measure rather than guess.

What we're about to test

The recordings I'm working with include:

  • 60–90 minute meetings in university offices (reasonable quiet, 2–3 speakers, formal vocabulary)
  • Site visits at client premises (background machinery, 2–4 speakers, more variable)
  • A couple of recordings from what I can only describe as "enthusiastically air-conditioned corridors"

For each, the pipeline will produce five hypotheses per segment. Part 3 will show where the models agreed, where they disagreed, how the LLM reconciler resolved it, and whether the final transcript reads better or worse than any individual engine's output. I'll also have the diarisation accuracy to report — at least for the recordings where I know who was speaking when.

The architecture is built. Time to find out if the benchmarks mean anything on real audio.

References

Part 3 will have the actual numbers. Et volia — or not. We'll see.

Posted by Fixer at 05:18 0 comments

17 July 2026

Caddy and dnsmasq: HTTPS .lan Hostnames for Every Home Lab Service

The .lan suffix is a perfectly reasonable convention for home lab services, but getting it to work cleanly — proper HTTPS, no certificate warnings, short names that resolve everywhere on the machine — takes a few moving parts. This is the setup I've landed on (Ubuntu 24.04, as of June 2026), plus the DNS priority conflict I ran into when trying to add services that the router already knew about.

The pieces

Three components work together:

  1. dnsmasq at 127.0.0.2 — resolves every *.lan name to 127.0.0.1 via a single wildcard rule
  2. systemd-resolved — routes .lan queries to dnsmasq, everything else upstream
  3. Caddy v2 — listens on 127.0.0.1:443, terminates TLS, proxies to backends

The result: any .lan name resolves locally, Caddy picks it up and proxies to the right container or port, and the browser sees a valid HTTPS certificate (from Caddy's built-in CA).

dnsmasq

Install and configure:

sudo apt install dnsmasq

# /etc/dnsmasq.d/lan.conf
address=/.lan/127.0.0.1
listen-address=127.0.0.2

The listen-address=127.0.0.2 keeps dnsmasq off port 53 on the main loopback (which systemd-resolved's stub resolver already owns at 127.0.0.53). The wildcard address=/.lan/127.0.0.1 answers every *.lan query with 127.0.0.1 — no per-service DNS entries needed.

sudo systemctl enable --now dnsmasq

systemd-resolved routing domain

# /etc/systemd/resolved.conf.d/lan.conf
[Resolve]
DNS=127.0.0.2
Domains=~lan

The ~ prefix makes lan a routing domain: systemd-resolved sends .lan queries to 127.0.0.2 (dnsmasq) and nothing else. Queries for everything else go to whichever upstream DNS the network provides.

sudo systemctl restart systemd-resolved
resolvectl status | grep -A3 "Global"

You should see 127.0.0.2 listed and ~lan as the domain.

Caddy

Install from the official Cloudsmith apt repo rather than the Ubuntu package — the Ubuntu version lags significantly:

curl -1sLf 'https://dl.cloudsmith.io/public/caddy/stable/gpg.key' \
  | sudo gpg --dearmor -o /usr/share/keyrings/caddy-stable-archive-keyring.gpg
curl -1sLf 'https://dl.cloudsmith.io/public/caddy/stable/debian.deb.txt' \
  | sudo tee /etc/apt/sources.list.d/caddy-stable.list
sudo apt update && sudo apt install caddy

Trust Caddy's internal CA so the browser accepts .lan certs without a warning:

sudo caddy trust

(You'll need to import the CA into Firefox manually if you use it — the caddy trust command handles Chrome/system trust stores.)

Organising the config

Rather than one large Caddyfile, a snippet import keeps things tidy:

# /etc/caddy/Caddyfile
import /etc/caddy/sites-enabled/*.caddy

Each service gets its own file:

# /etc/caddy/sites-enabled/open-webui.caddy
open-webui.lan {
    tls internal
    reverse_proxy 10.140.20.61:8080
}

For a service with a self-signed upstream certificate (Proxmox web UI being the obvious example):

# /etc/caddy/sites-enabled/xenon.caddy
xenon.lan {
    tls internal
    reverse_proxy https://10.140.3.82:8006 {
        transport http {
            tls_insecure_skip_verify
        }
    }
}

Add a service, reload:

sudo caddy validate --config /etc/caddy/Caddyfile && sudo systemctl reload caddy

The DNS priority conflict

This works perfectly for services the router doesn't know about — typically anything on a local container subnet (10.140.20.x in my setup, managed by the Incus bridge's DHCP). For services elsewhere on the network whose hostnames appear in the router's DHCP lease table, there's a conflict.

systemd-resolved receives .lan queries and has two servers claiming authority: the global ~lan → 127.0.0.2 entry from resolved.conf, and the per-link entry pushed by DHCP (domain=lan, DNS=<router IP>). Per-link entries win over global entries in systemd-resolved's priority ordering. So for a hostname the router knows — say gitea.lan for a container it gave a lease to — the router answers first with the real IP, Caddy is bypassed, and you get a connection refused on port 443 (or wherever Caddy is listening).

The fix in NetworkManager is to set a positive dns-priority on the affected connections, which makes them lower priority than the global server:

sudo nmcli con mod "Wired connection 3" ipv4.dns-priority 100
sudo nmcli con mod "Wired connection 3" ipv6.dns-priority 100
sudo nmcli con up "Wired connection 3"

In my session this didn't propagate immediately (the priority change may need a full reconnect or systemctl restart systemd-resolved). The pragmatic workaround is an /etc/hosts entry for any hostname the router already knows:

echo "127.0.0.1 gitea.lan" | sudo tee -a /etc/hosts
echo "127.0.0.1 xenon.lan" | sudo tee -a /etc/hosts

Not elegant, but unambiguous — /etc/hosts is checked before DNS by glibc, so it wins regardless of what the router says.

Current service map

For reference, the full set of .lan entries currently active on this machine:

Hostname Backend Notes
open-webui.lan 10.140.20.61:8080 Open WebUI (Incus container)
litellm.lan 10.140.20.63:4000 LiteLLM proxy
searxng.lan 10.140.20.15:8080 SearXNG search
ollama.lan localhost:11434 Ollama LLM inference
comfyui.lan localhost:8188 ComfyUI image generation
gitea.lan 10.140.3.116:3000 Gitea (via /etc/hosts)
xenon.lan 10.140.3.82:8006 Proxmox UI, TLS skip (via /etc/hosts)
vaultwarden.lan 10.140.20.16:8080 Vaultwarden
wake.lan 10.140.20.45:8080 Wake-on-LAN service
notes.lan /home/user/claude/notes Static file server (Caddy file_server)
cockpit.lan https://localhost:9090 Cockpit, TLS skip verify

Adding a new service is one file and a reload — the DNS side takes care of itself.

References

  • Caddy v2 — reverse proxy with automatic TLS; tls internal generates certs from Caddy's built-in CA
  • Caddy Cloudsmith apt repo — official packages, more current than Ubuntu's
  • dnsmasq — address=/ directive for wildcard DNS answers
  • systemd-resolved — stub resolver; Domains=~lan routing domain syntax
  • resolvectl(1) — resolvectl query <name> shows which DNS server answered and from which link
  • NetworkManager / nmcli — ipv4.dns-priority for per-link DNS priority (positive value = lower priority)
  • Incus — LXC/VM container manager; containers on 10.140.20.0/24 in this setup
  • Proxmox VE — hypervisor; web UI on port 8006 with self-signed cert

I hope you find this helpful.

Posted by Fixer at 00:21 0 comments

15 July 2026

Giving a Multi-Agent Framework Shell Access — What We Fixed and Why It Matters

Background

jiuwenclaw is a Chinese-origin multi-agent framework built on top of the openjiuwen agent-core. It supports a "team mode" where a leader agent coordinates a group of specialist teammates, each with their own skills, tools, and workspace.

This post documents three connected bugs we found and fixed, all triggered by a single user request: "Can the agent run shell commands the way Gemini CLI does?"

Problem 1: The React Loop (Agent Generates Text Instead of Tool Calls)

Symptom

After asking a question that required inspecting the machine (e.g. "what WiFi networks can you see?"), the agent entered an infinite loop. Each iteration the LLM produced a natural-language response describing how it would run a command — but never actually ran one. The runner log showed finish_reason: stop on every turn, meaning no tool was ever called.

Root Cause

The react loop in the openjiuwen harness re-invokes the LLM when the response has no tool call (finish_reason != tool_calls). The LLM was generating text because no shell tool existed in the agent's tool card list.

The mcp_exec_command tool was already implemented in command_tools.py and wired into the SkillDevService, but the team/agent mode uses a different tool registration path. In interface_deep.py, _get_tool_cards() built the list of available tools — and mcp_exec_command was never added to it.

Fix

Register mcp_exec_command in _get_tool_cards():

# jiuwenclaw/agentserver/deep_agent/interface_deep.py
from jiuwenclaw.agentserver.tools.command_tools import mcp_exec_command

def _get_tool_cards(self, ...) -> list[ToolCard]:
    ...
    # After wiki tools:
    if not Runner.resource_mgr.get_tool(mcp_exec_command.card.id):
        Runner.resource_mgr.add_tool(mcp_exec_command)
    tool_cards.append(mcp_exec_command.card)
    return tool_cards

This is the pattern used by all other tools in that method. Once registered, the LLM can actually call the tool instead of narrating about it.

Problem 2: The Sandbox Restriction Blocked Useful Commands

Symptom

After fixing the registration, agents could call mcp_exec_command — but any workdir outside the agent workspace raised a ValueError, producing [ERROR]: workdir is outside project workspace. This prevented running commands like ip addr in /tmp or inspecting files anywhere outside the agent's sandbox.

Root Cause

_resolve_command_workdir unconditionally called:

candidate.relative_to(project_root)  # raises ValueError if outside workspace

This was appropriate for a sandboxed skill dev environment but too restrictive for an agent with system-level access.

Fix

Introduce a MCP_EXEC_COMMAND_SANDBOX environment variable (default false) that opts into the restriction rather than enforcing it always:

def _is_workdir_sandbox_enabled() -> bool:
    raw = os.getenv("MCP_EXEC_COMMAND_SANDBOX", "false").strip().lower()
    return raw in ("1", "true", "yes", "on")

def _resolve_command_workdir(workdir: str) -> Path:
    project_root = get_agent_workspace_dir()
    candidate = Path(workdir) if workdir else project_root
    if not candidate.is_absolute():
        candidate = project_root / candidate
    candidate = candidate.resolve()
    if _is_workdir_sandbox_enabled():
        candidate.relative_to(project_root)  # raises ValueError if outside workspace
    return candidate

The switch is documented in .env:

# mcp_exec_command: restrict workdir to agent workspace (true) or allow any path (false)
MCP_EXEC_COMMAND_SANDBOX=false

Problem 3: Workspace Files Written in Chinese Despite preferred_language: en

Symptom

After the agent's workspace was initialised, the generated files (AGENT.md, SOUL.md, HEARTBEAT.md, IDENTITY.md) were written in Chinese, even though the config contained preferred_language: en.

Root Cause — Tracing the Data Flow

The language setting travels through three layers before it reaches the template selector:

  1. Config → preferred_language: en
  2. load_team_spec_dict → builds per-agent WorkspaceSpec dicts
  3. WorkspaceSpec.language → passed to get_workspace_schema(language) → selects EN or CN template

The bug was in step 2. _DEFAULT_AGENT_WORKSPACE was defined as:

_DEFAULT_AGENT_WORKSPACE = {"stable_base": True}

No language key. When this dict was used to construct each agent's workspace spec via merged.setdefault("workspace", deepcopy(default_workspace)), the resulting spec had no language, so WorkspaceSpec.language fell back to its model default — "cn".

The preferred_language value was read in load_team_spec_dict but only assigned to spec_dict["language"] (the top-level field) — it was never propagated into the per-agent workspace dicts.

Fix

Thread preferred_language as a language parameter through _build_agents_config, then inject it into every workspace spec via setdefault (respecting any explicit per-agent override):

def _build_agents_config(
    team_raw: dict[str, Any], config_base: dict[str, Any], language: str = "zh"
) -> dict[str, Any]:
    default_model = _build_default_model_dict(config_base)
    default_workspace, max_iterations, completion_timeout = _build_agent_defaults()
    default_workspace.setdefault("language", language)  # ← inject here

    for agent_key, raw_agent_config in agents_raw.items():
        agent_config = dict(raw_agent_config) if isinstance(raw_agent_config, dict) else {}
        if "workspace" in agent_config and isinstance(agent_config["workspace"], dict):
            agent_config["workspace"].setdefault("language", language)  # ← and per-agent
        ...

Also propagate to the team-level workspace spec:

# in load_team_spec_dict:
resolved_lang = str(config_base.get("preferred_language", "zh")).strip().lower()
agents = _build_agents_config(team_raw, config_base, language=resolved_lang)
...
workspace_spec = _build_workspace_spec(team_raw)
if workspace_spec is not None:
    workspace_spec.setdefault("language", resolved_lang)
    spec_dict["workspace"] = workspace_spec

Using setdefault throughout means an agent that explicitly configures workspace: {language: zh} won't have it overridden.

Skill File

To guide the agent on what commands are available and how to use them, a skill was created at:

~/.jiuwenclaw/agent/jiuwenclaw_workspace/skills/system-commands/SKILL.md

The allowed_tools frontmatter field in jiuwenclaw's skill system is advisory — it hints to the LLM which tools the skill uses. Actual tool availability is determined by what's registered in _get_tool_cards(). The skill body includes common command patterns for networking, hardware inspection, process listing, and log tailing.

Unit Tests

test_command_tools.py — 19 tests

TestSandboxSwitch (3 tests): verifies _is_workdir_sandbox_enabled() reads the env var correctly, including case-insensitive truthy/falsy parsing for true/True/TRUE/1/yes/on and false/False/FALSE/0/no/off.

TestResolveCommandWorkdir (6 tests): verifies that:
- Empty workdir falls back to agent workspace
- Relative paths are resolved under the workspace
- Sandbox off: absolute paths outside workspace are allowed
- Sandbox on: absolute paths outside workspace raise ValueError
- Sandbox on: paths inside workspace (including root) are allowed

test_team_config_loader.py — 8 new tests in TestLanguagePropagation

  • preferred_language: en sets spec["language"] == "en" at the top level
  • Absent preferred_language defaults to "zh"
  • "en" propagates to each agent's workspace spec
  • "en" propagates to all agents when multiple agents are configured
  • "en" is injected into explicit per-agent workspace dicts (that set other keys)
  • An explicit workspace: {language: zh} on an agent is not overridden
  • "en" propagates to the team-level workspace spec
  • An explicit language: zh in the team workspace spec is not overridden

Key Architectural Learnings

  1. _get_tool_cards() is the source of truth for tool availability in team/agent mode — not get_mcp_tools(), not skill allowed_tools. Any tool that should be callable must be explicitly added here.

  2. The react harness loops on finish_reason != tool_calls — if the model can't call a tool (because none are registered for the task), it will narrate endlessly. The fix is always to provide the right tool, not to adjust the loop logic.

  3. setdefault is the right pattern for propagating config defaults — it lets outer layers provide defaults while inner (explicit) config takes precedence.

  4. MCP_EXEC_COMMAND_SANDBOX follows the same bool-string convention as other env vars — 1/true/yes/on enable, everything else (including absence) disables.

Files Changed

File Change
jiuwenclaw/agentserver/deep_agent/interface_deep.py Register mcp_exec_command in _get_tool_cards()
jiuwenclaw/agentserver/tools/command_tools.py Add _is_workdir_sandbox_enabled(), make sandbox conditional
jiuwenclaw/agentserver/team/config_loader.py Propagate preferred_language into all workspace specs
tests/unit_tests/agentserver/test_command_tools.py New — 19 tests for sandbox switch and workdir resolution
tests/unit_tests/agentserver/test_team_config_loader.py 8 new TestLanguagePropagation tests + updated 1 existing assertion
~/.jiuwenclaw/agent/.../skills/system-commands/SKILL.md New skill providing shell command guidance
~/.jiuwenclaw/config/.env Document MCP_EXEC_COMMAND_SANDBOX=false
Posted by Fixer at 23:35 0 comments
Subscribe to: Posts (Atom)