26 August 2026

Never Remember an IP Address Again: A Proxmox LXC Naming Convention

If you run a home lab with more than a handful of containers, you've probably had the experience of SSH-ing into what you thought was your monitoring stack and finding yourself staring at someone's media server. IP addresses are fine for machines; they're not fine for humans.

I've been running a Proxmox cluster (three nodes — Xenon, PVE8, PVE1, plus a couple of remote machines) for a couple of years now, and the container count has crept up to around 80. At some point I got tired of either memorising subnets or running nmap every time I wanted to find something. So I settled on a convention, and more recently wrote a script to enforce it.

The Convention

For containers on Xenon that need to be on the home LAN (routable across the network, rather than isolated in an Incus bridge subnet), the rule is:

LXC ID = X, IP = 10.140.3.X, where 100 < X < 254

That's it. Container 110 lives at 10.140.3.110. Container 169 lives at 10.140.3.169. If you know the ID, you know the IP. If you know the IP, you know the ID.

This sounds obvious, but there are two ways it breaks down in practice. First, Proxmox LXC IDs are cluster-wide — so ID 108 might already be taken by a container on a different node entirely (I discovered this the hard way while trying to create a new container and getting "CT 108 already exists on node 'pve8'"). Second, older containers on Xenon use a 7XXX ID scheme and their IPs are already assigned in the 10.140.3.x range. LXC 7128 lives at 10.140.3.128, so you can't pick X=128 without checking both tables.

The Script

So I wrote agents/pve_free_slots.py, which does three things:

  1. SSHes to Xenon and queries the Proxmox cluster API (pvesh get /cluster/resources) to pull every VM and LXC ID across all nodes.
  2. SSHes to the OpenWRT router to pull DHCP leases and static reservations in the 10.140.3.x subnet.
  3. Accounts for 7XXX and 9XXX LXCs — ID 7128 occupies IP 10.140.3.128, so X=128 is blocked even though LXC ID 128 is technically free.

A slot is only free if both the ID is unused cluster-wide and the IP is unoccupied.

#!/usr/bin/env python3
"""
Find available Proxmox LXC ID / IP slots following the convention:
  LXC ID = X,  IP = 10.140.3.X,  100 <= X <= 253

A slot is free when both:
  - X is not in use as a cluster-wide VM/LXC ID (any node)
  - 10.140.3.X is not in any DHCP lease or static reservation

Sources:
  - Proxmox cluster:  pvesh on Xenon (covers all nodes via cluster API)
  - DHCP leases:      OpenWRT router at 10.140.2.6 (/tmp/dhcp.leases + uci static)
  - 7XXX LXC pattern: IDs 7100-7253 on Xenon map to IPs 10.140.3.100-253

Usage:
    python3 pve_free_slots.py                # show available slots
    python3 pve_free_slots.py --taken        # also list what's occupied
    python3 pve_free_slots.py --min 150      # restrict range floor

# Author: Matthew / Claude
"""

import argparse, json, subprocess, sys

RANGE_MIN, RANGE_MAX = 100, 253
XENON_DIRECT = "root@10.140.3.82"
XENON_CF = "root@xenon-ssh.mattsouthgate.co.uk"
XENON_CF_HOSTNAME = "xenon-ssh.mattsouthgate.co.uk"
ROUTER = "root@10.140.2.6"


def ssh(target, cmd, *, proxy_hostname=None, timeout=15):
    args = ["ssh", "-o", "StrictHostKeyChecking=accept-new",
            "-o", "BatchMode=yes", "-o", f"ConnectTimeout={timeout}"]
    if proxy_hostname:
        args += ["-o", f"ProxyCommand=cloudflared access ssh --hostname {proxy_hostname}"]
    args += [target, cmd]
    try:
        r = subprocess.run(args, capture_output=True, text=True, timeout=timeout + 5)
        return r.stdout, r.returncode == 0
    except subprocess.TimeoutExpired:
        return "", False


def xenon_ssh(cmd, timeout=15):
    out, ok = ssh(XENON_DIRECT, cmd, timeout=timeout)
    if ok:
        return out, True
    return ssh(XENON_CF, cmd, proxy_hostname=XENON_CF_HOSTNAME, timeout=timeout + 10)


def get_cluster_ids():
    out, ok = xenon_ssh("pvesh get /cluster/resources --type vm --output-format json")
    if not ok or not out.strip():
        return None, "Could not reach Xenon cluster API"
    try:
        return {int(r["vmid"]): f"{r.get('name','?')} ({r['node']}, {r['type']})"
                for r in json.loads(out)}, None
    except (json.JSONDecodeError, KeyError) as e:
        return None, f"Parse error: {e}"


def get_dhcp_taken():
    taken = {}
    out, ok = ssh(ROUTER, "cat /tmp/dhcp.leases", timeout=8)
    if ok:
        for line in out.splitlines():
            parts = line.split()
            if len(parts) >= 3 and parts[2].startswith("10.140.3."):
                x = int(parts[2].split(".")[-1])
                taken[x] = f"DHCP lease ({parts[3] if len(parts) > 3 else '?'})"
    out, ok = ssh(ROUTER,
        "uci show dhcp 2>/dev/null | grep '\\.ip=' | grep '10\\.140\\.3\\.'", timeout=8)
    if ok:
        for line in out.splitlines():
            if "10.140.3." in line:
                ip = line.split("=")[-1].strip("'\"")
                if ip.startswith("10.140.3."):
                    x = int(ip.split(".")[-1])
                    taken[x] = f"DHCP static ({ip})"
    return taken, ok

(The full script with argument parsing and range notation is at agents/pve_free_slots.py.)

Test Run

Running it against the cluster today (26 June 2026, from the university — so the router was unreachable and DHCP data wasn't available, but the cluster query still worked):

Querying Proxmox cluster IDs...
  81 resources across cluster
Querying OpenWRT DHCP leases (10.140.3.x)...
  Router unreachable — DHCP data unavailable (may be off home LAN)

==========================================================
  Free slots  (ID=X, IP=10.140.3.X)  range 100–253
==========================================================
  110
  118
  122
  125
  129
  131
  133
  136
  139–142
  145
  147
  151–164
  166–168
  171–173
  175–179
  181–185
  188–198
  202–253

  105 free,  49 occupied

With --taken, the occupied section shows why each slot is blocked — whether it's an ID conflict on a specific node, or an IP conflict from a 7XXX container:

Occupied — ID conflict:
  100  pialert (pve8, lxc)
  101  openwrt (pve1, qemu)
  102  claude-cli (pve8, lxc)
  103  caddy (pve8, lxc)
  ...
  108  smokeping (pve8, lxc)   ← this is the one that bit me
  ...

Occupied — IP conflict only (ID is free):
  114  IP taken by 7114 (paperless-ngx (xenon, lxc))
  117  IP taken by 7117 (cockpit (xenon, lxc))
  119  IP taken by 7119 (rtsptoweb (xenon, lxc))
  ...

The net result: I ran the script, picked ID 110 (first clean slot after 108), created the container (pct clone ... 110), and set the IP to 10.140.3.110. Done. No nmap, no guessing, no collisions.

The .lan Side

The IP convention handles the "where is it" problem. The hostname convention handles the "what do I call it" problem. All routable containers get a .lan DNS entry through Caddy (CT103 on PVE8), so litellm.lan resolves to 10.140.3.110. When a container moves or gets replaced, you update the Caddyfile, not every script that talks to it. The OpenWRT router handles .lan resolution for the rest of the network via dnsmasq.

Gaps and Future Work

A few things I haven't sorted yet:

  • DHCP reservations aren't automated. New containers still get a DHCP lease by default — I assign the static reservation separately in OpenWRT. The script catches any conflicts, but doesn't create the reservation when a container is provisioned.
  • Some containers have DHCP IPs. The Xenon vLLM container (LXC 8003) is currently at 10.140.1.166, which is fine until the lease rotates. There's an open todo to pin it.
  • The 7XXX naming scheme is a historical accident and I haven't migrated those containers. They work, so the motivation to touch them is low. The script handles them correctly, they just look a bit odd in the occupied list.
  • PVE8 and PVE1 containers predate the convention — CT103 is Caddy at 10.140.3.156, which has nothing to do with ID 103. The convention only applies to new Xenon containers going forward.

Not a perfect system, but it does mean that when an AI agent or a slightly-sleepy human needs to create a new container, the available slots are a single command away.

I hope you find this helpful. Ta ta for now,
Matt

Automating a Cashback Offer Alert on a Cloudflare + Cognito Protected Site

I wanted a daily alert when a particular AliExpress bonus cashback offer reappears on Quidco — a UK cashback site. The offer shows up in a carousel on the logged-in homepage and disappears within a day or two. Catching it manually is unreliable. This is a write-up of building a fully automated checker that runs at 1 AM every night.

The Target

The Quidco homepage carousel shows rotating offers like "Bonus Cashback — AliExpress: Get a £7.50 Bonus when you opt in and spend £15 or more." I wanted to be notified the moment one of these appears, without having to check manually.

The page is:
- JavaScript-rendered (React/Next.js)
- Protected by Cloudflare bot detection
- Authenticated via AWS Cognito (short-lived JWTs, 1-hour TTL)

A simple curl or requests fetch gets a 403 immediately. So we need a real browser.

Tool: Playwright Firefox

Playwright is a browser automation library that drives real browser engines headlessly. The first instinct is Chromium — it's the default — but Cloudflare's cf_clearance cookie is bound to the TLS fingerprint (JA3 hash) of the browser that solved the challenge. My Firefox session's cf_clearance won't work in Chromium because the two engines produce different TLS ClientHello signatures.

Solution: use playwright's Firefox engine, which is close enough in fingerprint to the real Firefox that the cf_clearance transfers across.

with sync_playwright() as p:
    browser = p.firefox.launch(headless=True)

Problem 1: Cookies Weren't Being Sent

Firefox stores cookies in an SQLite database at snap/firefox/common/.mozilla/firefox/<profile>/cookies.sqlite. I read them and injected them into the playwright context — but the page kept redirecting to login.

Tracing the actual HTTP requests showed cf_clearance and session_id were missing from the Cookie header on requests to www.quidco.com, even though I'd injected them.

The bug: Firefox's SQLite host column uses a leading dot (.quidco.com) to signal subdomain-matching cookies, mirroring the Set-Cookie: Domain= attribute in RFC 6265. I was stripping that dot:

# Wrong — tells playwright "exact host only"
cookies.append({"domain": host.lstrip('.'), ...})

# Right — keep the dot so playwright sends it to www.quidco.com too
cookies.append({"domain": host, ...})

After that fix, all the right cookies arrived at the server and the page loaded.

Problem 2: Cognito Redirect Loop

The Cognito access token (stored as cognito_token cookie) has a 1-hour TTL. At 1 AM, if the user hasn't visited Quidco recently, it'll be stale. Sending a stale token caused an infinite redirect loop:

GET /home/           → 302 /?auth=login   (Cognito middleware: token expired)
GET /?auth=login     → 302 /home/         (session_id is valid, go home)
GET /home/           → 302 /?auth=login   (token still expired)
...

The fix is counterintuitive: don't send the expired token at all. When the token is missing rather than expired, the server's Cognito middleware steps aside and lets the client-side Amplify.js handle authentication instead.

if name == "cognito_token":
    if exp > now:
        cognito_expired = False
    else:
        continue  # omit it — sending it causes a redirect loop

Problem 3: Token Refresh via Amplify.js

With no cognito_token but a valid cognito_refresh_token (6-month TTL), the Quidco page's embedded AWS Amplify SDK detects the missing token on load and silently fetches a new one from Cognito using the refresh token. It then redirects the client to /home/ — entirely client-side, no server round-trip.

def refresh_cognito(ctx, page) -> bool:
    # Navigate to root (not /home/) — server accepts it without Cognito check
    page.goto("https://www.quidco.com/", wait_until="domcontentloaded")
    try:
        # Amplify.js fires, refreshes the token, redirects client to /home/
        page.wait_for_url("**/home/**", timeout=20_000)
        return True
    except TimeoutError:
        return False

I tested this with a genuinely expired token (12 minutes past expiry). The root page loaded, Amplify.js ran, a new token was silently obtained, and the browser landed on /home/ — all in the first page load.

Parsing the Carousel

The carousel cards are rendered as div.main elements with a div.main-title inside. BeautifulSoup makes extraction straightforward:

from bs4 import BeautifulSoup

soup = BeautifulSoup(html, "html.parser")
offers = []
seen = set()
for card in soup.find_all("div", class_="main"):
    title_el = card.find("div", class_="main-title")
    if not title_el:
        continue
    title = title_el.get_text(strip=True)
    if title in seen:
        continue
    seen.add(title)
    desc_el = card.find("div", class_="main-description")
    offers.append({
        "title": title,
        "description": desc_el.get_text(strip=True) if desc_el else ""
    })

Importantly, I check the carousel titles specifically rather than searching full-page body text. Quidco also shows AliExpress in a "Your Favourites" section — a body text search would give false positives.

Alerts via Claude Push Notifications

For the 1 AM alert, I use the claude CLI in non-interactive mode to push a notification to the Android Claude app:

claude -p "Send a push notification: Quidco AliExpress offer on carousel — £7.50 bonus" \
    --allowedTools PushNotification

This spawns a lightweight Claude Code session that calls the PushNotification tool, which routes through to Claude's mobile app via Remote Control. No email credentials, no third-party push service.

The Cron Job

# crontab -l
0 0 * * * /home/user/claude/quidco/alert.sh

Midnight UTC = 1 AM BST. The carousel rolls over at midnight, so this catches whatever's new for the day.

Full Flow

cron (00:00 UTC)
  └─ alert.sh
       └─ quidco_check.py
            ├─ read Firefox cookies.sqlite
            ├─ [if token expired] playwright Firefox → quidco.com root
            │    └─ Amplify.js refreshes token → redirects to /home/
            ├─ [if token fresh] playwright Firefox → /home/ directly
            ├─ parse div.main carousel cards
            └─ return {"found": bool, "carousel": [...]}
  └─ [if found] claude -p "push notification"
  └─ [if found] notify-send (best-effort desktop)
  └─ log to check.log

Results

Current carousel on a typical day: Boots, Temu, IHG Hotels, Goldsmiths, Shepherds Friendly ISA, Antler, Opodo, LG Electronics, Quidco Gift Cards, Very, Quidco In-Store, Pooch and Mutt, Lovehoney, Virgin Experience Days.

When AliExpress appeared earlier today (£7.50 bonus, "Ends Today"), the script correctly detected it. After midnight when the offer expired, it correctly returned found: false.

The main fragility is the cognito_refresh_token — it has a ~6 month lifetime. When it expires, a fresh Firefox login to Quidco is all that's needed to re-establish the session.

Code

agents/quidco_check.py — about 100 lines of Python. Dependencies: playwright, beautifulsoup4 (both already available in the project venv).

Orphaned Processes and Port Conflicts: Finding, Killing, and Preventing Them

You start an app, something goes wrong, you hit Ctrl+C — and now the app won't start again because the port is already in use. Sound familiar? This is the orphaned process problem, and it's more common than it should be.

What Happened

I was running JiuwenClaw, a Python-based AI agent platform. It launches several sub-processes on startup: an agent server and a gateway, each binding to their own port. After terminating the parent with Ctrl+C, the app refused to restart:

Port 19001 already in use
Port 18092 already in use

Why Ctrl+C Doesn't Always Clean Up

When you press Ctrl+C in a terminal, the shell sends SIGINT to the foreground process group. If the parent process spawned children via Python's subprocess module without explicitly adding them to the same process group — or without a signal handler that forwards the signal — those children keep running after the parent dies.

They become orphans: no parent, no controlling terminal, but still holding their network ports open.

Finding the Culprits

Two commands are all you need.

Check which ports are in use:

ss -tlnp | grep -E '19001|18092'

Output:

LISTEN  127.0.0.1:19001  users:(("python3",pid=80615,...))
LISTEN  127.0.0.1:18092  users:(("python3",pid=80613,...))

Identify the processes by PID:

lsof -i :19001 -i :18092

Output:

python3  80613  user  ...  TCP  localhost:18092 (LISTEN)
python3  80615  user  ...  TCP  localhost:19001 (LISTEN)

Confirm what they are:

ps -p 80613 -o pid,ppid,cmd --no-headers
ps -p 80615 -o pid,ppid,cmd --no-headers

Output:

80613  2602  python3 -m jiuwenclaw.app_agentserver
80615  2602  python3 -m jiuwenclaw.app_gateway

There they are — orphaned sub-processes from the previous run, still alive and holding the ports.

Killing Them

Once you have the PIDs:

kill 80613 80615

Verify the ports are clear:

ss -tlnp | grep -E '19001|18092'

No output means the ports are free. You can now restart the app normally.

If a process ignores SIGTERM, escalate with SIGKILL:

kill -9 80613 80615

One-liner: kill by port

If you just want to nuke whatever is on a port without finding the PID first:

fuser -k 19001/tcp 18092/tcp

Or using ss and kill together:

ss -tlnp | grep 19001 | grep -oP 'pid=\K[0-9]+' | xargs kill

Mitigations

1. Fix the application's signal handling (ideal)

The root cause is that the app doesn't forward signals to its children. A well-behaved Python launcher should use a process group and kill the whole group on exit:

import os
import signal
import subprocess

proc = subprocess.Popen(["python3", "-m", "myapp.server"], start_new_session=True)

def cleanup(sig, frame):
    os.killpg(os.getpgid(proc.pid), signal.SIGTERM)

signal.signal(signal.SIGINT, cleanup)
signal.signal(signal.SIGTERM, cleanup)

Using start_new_session=True puts the child in its own process group, and os.killpg kills the whole group cleanly.

2. Use a stop script or PID file

Many services write a PID file on startup and provide a stop command. If the app supports it, prefer jiuwenclaw-stop over Ctrl+C — a proper stop command can clean up child processes before exiting.

3. Wrap the launcher in a shell trap

If you're starting the app via a shell script, add a trap:

#!/bin/bash
jiuwenclaw-start &
PARENT_PID=$!

cleanup() {
    kill -- -$(ps -o pgid= $PARENT_PID | tr -d ' ')
}

trap cleanup EXIT INT TERM
wait $PARENT_PID

The kill -- -<pgid> sends the signal to the entire process group.

4. Run inside a process supervisor

Tools like systemd, supervisord, or s6 track all child processes and kill them as a unit when the service stops. If you're running something in production or semi-permanent, this is the right answer — it also handles restarts, logging, and health checks.

5. Check before you start

Add a pre-flight check to your workflow. Before starting any service, quickly verify the ports it needs are free:

ss -tlnp | grep -E '19001|18092' && echo "Port conflict!" || echo "Ports clear"

Summary

Step Command
Find listening ports ss -tlnp \| grep <port>
Find PID by port lsof -i :<port>
Confirm process identity ps -p <pid> -o pid,cmd
Kill by PID kill <pid>
Kill by port directly fuser -k <port>/tcp
Verify ports are free ss -tlnp \| grep <port>

The quick fix is always kill. The real fix is making sure the app cleans up after itself — or wrapping it in something that does.

Proxmox on Android Through Cloudflare Access — Getting Native Apps Past the Browser Challenge

I have Proxmox Virtual Environment exposed externally via a Cloudflare tunnel. Access is locked down with Cloudflare Zero Trust — a wildcard application on *.mattsouthgate.co.uk that requires either a WARP-enrolled device or email authentication before you get anywhere near the server.

That works perfectly in a browser. It does not work in the Proxmox VE Android app.

This is the story of figuring out why, and the fix that keeps the servers protected while letting the app through.


The Problem

The Proxmox VE app on Android connects directly to the server's HTTPS API at port 8006. Through a Cloudflare tunnel, that becomes https://pve8.mattsouthgate.co.uk on port 443 — Cloudflare handles the external TLS and forwards traffic to the server internally.

With Cloudflare Access in front, the first thing any unauthenticated connection receives is a challenge page. In a browser, you click through, authenticate, get a session cookie, and proceed. The Proxmox app isn't a browser. It expects a JSON API response. What it gets instead is:

Connection error. Could not establish connection.
Format exception, unexpected character (at character 1) <!DOCTYPE html>
^

The <!DOCTYPE html> at position 1 is Cloudflare's Access login page. The app has no way to handle it — it's looking for { not <.


What Doesn't Work

WARP / Cloudflare One on the phone. The instinct here is correct — enroll the device in Zero Trust, and Cloudflare should recognise it as trusted without a browser challenge. The Cloudflare One app (the replacement for 1.1.1.1 after May 2026) handles this enrollment.

The problem is in how Access policies evaluate WARP. An Allow policy with a WARP posture check doesn't skip authentication — it requires the device to be WARP-connected and complete an Access login session. The login session is the browser step. The app still can't do that.

Service tokens. Cloudflare Access supports service tokens — a client ID and secret sent as HTTP headers (CF-Access-Client-Id, CF-Access-Client-Secret) — designed exactly for non-browser clients. The Proxmox Android app has no mechanism to set custom HTTP headers on its connections. That option is closed off.

WARP posture check in a Bypass policy. A Bypass policy skips the Access login entirely for traffic matching its rules. Combining Bypass with a WARP Include rule sounds like it would work — bypass only for enrolled devices, everyone else still hits the email flow. In practice, the Cloudflare dashboard warns you when you select Bypass that it only reliably supports IP-based and group-based rules for its conditions. Device posture checks, including WARP, don't function reliably in Bypass policies.


What Does Work

The solution is a second, more specific Access application scoped only to pve8.mattsouthgate.co.uk with a Bypass + Everyone policy.

In Access controls → Applications, create a new Self-hosted application:

  • Subdomain: pve8
  • Domain: mattsouthgate.co.uk
  • Policy action: Bypass
  • Include: Everyone

Cloudflare evaluates Access applications by specificity — the most specific hostname match wins. pve8.mattsouthgate.co.uk is more specific than *.mattsouthgate.co.uk, so traffic to pve8 hits the Bypass policy first and passes straight through to the Proxmox login page. Every other subdomain continues to be handled by the wildcard application with its WARP and email policies intact.

The Proxmox app connects immediately. No HTML. No challenge. Just the API.


What About Security?

Bypassing Cloudflare Access on pve8 means anyone who knows the URL can reach the Proxmox login page. That's worth being clear-eyed about.

What they reach is Proxmox's own authentication — username, password, and optionally two-factor. Proxmox's login is not bypassed, only Cloudflare's pre-authentication layer. The server is not open; it's just relying on its own credentials rather than having an additional Cloudflare gate in front.

For the other subdomains — services that may not have strong authentication of their own — the wildcard application continues to enforce Zero Trust access. Nothing about that changes.

This is a reasonable split: Proxmox has solid built-in auth and the app has no way to satisfy Cloudflare's browser flow, so Cloudflare steps aside and lets Proxmox handle it. Services that are less hardened stay behind the full Zero Trust wall.


Tunnel Configuration

One separate thing worth checking if the app connects but behaves oddly: the tunnel's origin settings for pve8 should have both noTLSVerify and disableChunkedEncoding enabled.

Proxmox uses a self-signed certificate internally, so noTLSVerify tells Cloudflare not to reject it on the internal leg. disableChunkedEncoding matters because Cloudflare tunnels don't support HTTP chunked transfer encoding, which Proxmox uses — without disabling it you can get HTTP 501 errors on certain operations.

Both are set under Networks → Tunnels → [tunnel] → Public Hostnames → [hostname] → Additional application settings → Origin.


App Settings

With the above in place, the Proxmox VE Android app connects with:

  • Host: pve8.mattsouthgate.co.uk
  • Port: 443

Port 443, not 8006 — Cloudflare terminates on 443 externally. The :8006 only appears on the internal leg between Cloudflare and the server, which the tunnel configuration already handles.