fix(nzbget): wait for gluetun tunnel before starting (prevents queue cancellation)

On pod start the nzbget container raced gluetun: /etc/resolv.conf points at
10.128.0.1 (reachable only via the WireGuard tunnel), so for the ~20s gluetun
needs to establish the tunnel every DNS lookup from nzbget returned EAI_AGAIN.
Any in-queue download that had articles fetched during that window dropped
below the HealthCheck threshold (~97.9%) and was auto-cancelled — even items
that would otherwise complete (saw 97.6-97.8% health = "very nearly fine").

Override the nzbget container's entrypoint to poll DNS resolution and only
exec /init once it succeeds. That's the direct test of "tunnel is up + DNS
works", which is what nzbget needs.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

TODO-cloudnativepg-migration.md

@@ -0,0 +1,174 @@
+# TODO: Migrate PostgreSQL to CloudNativePG
+
+**Status:** planned, not started
+**Target:** [CloudNativePG](https://cloudnative-pg.io/) (CNPG operator)
+**Created:** 2026-05-31
+
+## Why
+
+Both Postgres instances run **Bitnami** images, which Bitnami froze in
+Aug 2025 (free images moved to the `bitnamilegacy` archive and stopped
+getting updates / security patches):
+
+| Instance | Namespace | Version | Image | How deployed | Consumers |
+|---|---|---|---|---|---|
+| `pgsql` | `default` | PG **16.2** | `docker.io/bitnami/postgresql:16.2.0-debian-12-r8` (non-legacy, tag effectively gone from the registry) | standalone helm release `pgsql` (chart `postgresql-15.1.2`) | lidarr, radarr, sonarr, prowlarr, alhfmf |
+| `gitea-postgresql` | `gitea` | PG **17.6** | `bitnamilegacy/postgresql:17.6.0-debian-12-r4` | **bundled subchart** inside the `gitea` helm release | gitea only |
+
+CloudNativePG is the chosen replacement: actively maintained, operator-managed
+HA/backups, first-class `pg_dump`-based import for migration, and not tied to
+Bitnami.
+
+> ⚠️ `pgsql` is on the **non-legacy** `bitnami/` path whose `16.2.0` tag is no
+> longer pullable — if that pod is ever rescheduled to a node without the image
+> cached, it will **fail to start**. Treat instance A as the higher priority.
+
+## Databases to migrate
+
+- **pgsql (PG16):** `alhfmf`, `lidarr_db`, `lidarr_db_log`, `radarr_db`,
+  `radarr_db_log`, `sonarr_db`, `sonarr_db_log`, `prowlarr_db`, `prowlarr_db_log`
+  (confirm full list at cutover with `\l`).
+- **gitea (PG17):** `gitea` (owner role `gitea`).
+
+---
+
+## 0. Prerequisites
+
+- [ ] Install the CNPG operator (pin a recent stable version):
+  ```bash
+  helm repo add cnpg https://cloudnative-pg.github.io/charts
+  helm repo update cnpg
+  helm upgrade --install cnpg cnpg/cloudnative-pg \
+    -n cnpg-system --create-namespace --wait
+  kubectl get deploy -n cnpg-system   # operator Running
+  ```
+- [ ] Decide on storage: reuse `nfs-client` StorageClass, **or** prefer local
+      storage for the DB PVCs (Postgres on NFS is workable but not ideal;
+      CNPG defaults to RWO). Pick a `storageClass` + size per cluster below.
+- [ ] Decide CNPG topology: homelab can run `instances: 1` (no HA) to keep it
+      light on the Pi nodes; bump to 2–3 later if desired.
+- [ ] Take a manual backup of both instances first (safety net):
+  ```bash
+  kubectl exec -n default pgsql-postgresql-0 -- \
+    bash -c 'PGPASSWORD=$POSTGRES_PASSWORD pg_dumpall -U postgres' > pgsql-all.sql
+  kubectl exec -n gitea gitea-postgresql-0 -- \
+    bash -c 'PGPASSWORD=$POSTGRES_PASSWORD pg_dumpall -U postgres' > gitea-all.sql
+  ```
+
+---
+
+## Instance A — `pgsql` (media *arr + alhfmf), PG16
+
+CNPG can pull data straight from the old server at bootstrap via
+`initdb.import` (it runs `pg_dump`/`pg_restore` for you). Use **monolith** type
+to copy all listed databases + roles in one shot.
+
+- [ ] Keep the old `pgsql` release **running** during migration (read source).
+- [ ] Create a secret with the old server's superuser creds for the importer:
+  ```bash
+  # password: kubectl get secret pgsql-postgresql -n default -o jsonpath='{.data.postgres-password}' | base64 -d
+  kubectl create secret generic pg16-source-superuser -n default \
+    --from-literal=username=postgres --from-literal=password='<OLD_PW>'
+  ```
+- [ ] Apply a CNPG `Cluster` with import bootstrap (sketch — tune names/size):
+  ```yaml
+  apiVersion: postgresql.cnpg.io/v1
+  kind: Cluster
+  metadata:
+    name: pg16
+    namespace: default
+  spec:
+    instances: 1
+    imageName: ghcr.io/cloudnative-pg/postgresql:16   # match major 16
+    storage:
+      size: 10Gi
+      storageClass: nfs-client            # or local SC — see prereqs
+    bootstrap:
+      initdb:
+        import:
+          type: monolith
+          databases: ["alhfmf","lidarr_db","lidarr_db_log","radarr_db","radarr_db_log","sonarr_db","sonarr_db_log","prowlarr_db","prowlarr_db_log"]
+          roles: ["*"]
+          source:
+            externalCluster: old-pgsql
+    externalClusters:
+      - name: old-pgsql
+        connectionParameters:
+          host: pgsql-postgresql.default.svc.cluster.local
+          user: postgres
+          dbname: postgres
+        password:
+          name: pg16-source-superuser
+          key: password
+  ```
+- [ ] Wait for import to finish: `kubectl get cluster pg16 -n default -w`
+      (phase → `Cluster in healthy state`). Check logs of the bootstrap job.
+- [ ] Verify row counts / `\dt` in a couple of DBs vs. the old server.
+- [ ] **Cutover:** the new service is `pg16-rw.default.svc.cluster.local:5432`.
+      Update each consumer's DB host:
+  - [ ] lidarr  — `custom_helm_charts/lidarr` / `helm-values/lidarr_values.yaml`
+  - [ ] radarr  — `helm-values/radarr_values.yaml`
+  - [ ] sonarr  — `helm-values/sonarr_values.yaml`
+  - [ ] prowlarr — `custom_helm_charts/prowlarr` / `helm-values/prowlarr_values.yml`
+  - [ ] alhfmf  — `alhfmf` release (find its DB env/secret)
+      (the *arr apps store DB host/creds in their `config.xml`/Postgres env —
+      confirm where each one is configured before flipping.)
+      Commit + push so Argo redeploys; verify each app reconnects.
+- [ ] Soak for a few days.
+- [ ] Decommission old: `helm uninstall pgsql -n default`, delete its PVC
+      (`data-pgsql-postgresql-0`) and `resources/pgsql_persistent_volume.yml`,
+      remove `non_argo_values/pgsql_values.yaml`, drop `pg16-source-superuser`.
+
+---
+
+## Instance B — `gitea` DB, PG17
+
+Trickier because Postgres is a **subchart of the gitea release**, so it must be
+externalized from the gitea chart.
+
+- [ ] Stand up a CNPG `Cluster` `pg17` in the `gitea` ns, PG major **17**
+      (`imageName: ghcr.io/cloudnative-pg/postgresql:17`), same import approach:
+      `databases: ["gitea"]`, `roles: ["*"]`, source =
+      `gitea-postgresql.gitea.svc.cluster.local`, superuser secret from
+      `kubectl get secret gitea-postgresql -n gitea -o jsonpath='{.data.postgres-password}'`.
+- [ ] **Quiesce gitea during final cutover** (scale gitea deploy to 0) so the
+      source DB is static, then run/refresh the import (or do a final `pg_dump`
+      of `gitea` and restore into `pg17`) to avoid losing writes.
+- [ ] Edit `non_argo_values/gitea_values.yaml`:
+  - set `postgresql.enabled: false` (drop the bundled subchart)
+  - point `gitea.config.database` at the CNPG service:
+    ```yaml
+    gitea:
+      config:
+        database:
+          DB_TYPE: postgres
+          HOST: pg17-rw.gitea.svc.cluster.local:5432
+          NAME: gitea
+          USER: gitea
+          PASSWD: <from CNPG-managed secret pg17-app>
+    ```
+    (CNPG creates an app user/secret; either reuse the migrated `gitea` role or
+    wire gitea to the `pg17-app` secret.)
+- [ ] `helm upgrade gitea ...` (Recreate strategy already set). Bring gitea back
+      up; verify login + that the GitOps repo (`admin/turingpi.git`) is intact —
+      **Argo depends on this**, so validate before moving on.
+- [ ] Decommission: once `postgresql.enabled: false` is live, the old
+      `gitea-postgresql` StatefulSet is removed by the chart; delete leftover
+      PVC `data-gitea-postgresql-0` after confirming the new DB is good.
+
+---
+
+## Rollback
+
+- Old instances stay intact until the explicit decommission steps — to roll
+  back, just point the consumer/gitea config back at the original
+  `*-postgresql` service. Keep the `pg_dumpall` files until both soaks pass.
+
+## Open questions / decisions
+
+- [ ] NFS vs local storage for DB PVCs (perf + RWO behavior on node loss).
+- [ ] HA (`instances: 1` vs `3`) given Pi resource limits.
+- [ ] Backups: configure CNPG scheduled backups (Barman to NFS or object store)
+      as part of this — replaces whatever (if any) backup the Bitnami charts did.
+- [ ] Confirm exact per-app DB connection config for the 5 consumers of `pgsql`
+      before cutover (where each *arr stores its Postgres host/creds).

applications/disk-usage-alert.yaml

@@ -1,19 +1,16 @@
 apiVersion: argoproj.io/v1alpha1
 kind: Application
 metadata:
-  name: gluetun
+  name: disk-usage-alert
   namespace: argocd
-  annotations:
 spec:
   project: default
   source:
     repoURL: http://gitea-http.gitea.svc.cluster.local:3000/admin/turingpi.git
     targetRevision: HEAD
-    path: custom_helm_charts/gluetun
+    path: manifests/disk-usage-alert
-    helm:
+    directory:
-      releaseName: gluetun
+      recurse: false
-      valueFiles:
-        - ../../helm-values/gluetun_values.yaml
   destination:
     server: https://kubernetes.default.svc
     namespace: default

applications/maintainerr.yaml

@@ -0,0 +1,23 @@
+apiVersion: argoproj.io/v1alpha1
+kind: Application
+metadata:
+  name: maintainerr
+  namespace: argocd
+spec:
+  project: default
+  source:
+    repoURL: http://gitea-http.gitea.svc.cluster.local:3000/admin/turingpi.git
+    targetRevision: HEAD
+    path: manifests/maintainerr
+    directory:
+      recurse: false
+  destination:
+    server: https://kubernetes.default.svc
+    namespace: default
+  syncPolicy:
+    automated:
+      prune: true
+      selfHeal: true
+    syncOptions:
+    - CreateNamespace=true
+    - ServerSideApply=true

custom_helm_charts/gluetun/.helmignore

@@ -1,21 +0,0 @@
-# Patterns to ignore when building packages.
-# This supports shell glob matching, relative path matching, and
-# negation (prefixed with !). Only one pattern per line.
-.DS_Store
-# Common VCS dirs
-.git/
-.gitignore
-.bzr/
-.bzrignore
-.hg/
-.hgignore
-.svn/
-# Common backup files
-*.swp
-*.bak
-*.tmp
-*~
-# Various IDEs
-.project
-.idea/
-*.tmproj

custom_helm_charts/gluetun/Chart.yaml

@@ -1,6 +0,0 @@
-apiVersion: v1
-appVersion: "1.0"
-description: Gluetun - VPN client with HTTP proxy
-name: gluetun
-version: 0.1.0
-icon: https://raw.githubusercontent.com/qdm12/gluetun/master/.github/logo.png

custom_helm_charts/gluetun/README.md

@@ -1,22 +0,0 @@
-# Usenet stack (Gluetun + NZBGet)
-
-Service endpoints:
-- NZBGet UI: nzbget.default.svc.cluster.local:6789
-- Gluetun HTTP proxy: gluetun.default.svc.cluster.local:8888
-
-AirVPN WireGuard values:
-- Update `helm-values/gluetun_values.yaml`:
-  - `env.WIREGUARD_ADDRESSES` -> WireGuard tunnel address(es) (IPv4 /32 and optional IPv6)
-  - `env.SERVER_COUNTRIES` -> recommended AirVPN server selection (e.g. Netherlands)
-  - If your cluster does not support IPv6, remove the IPv6 address from `env.WIREGUARD_ADDRESSES`.
-- Create a Secret named `gluetun-wireguard` with key `WIREGUARD_PRIVATE_KEY` from your AirVPN WireGuard config (do not commit the key).
-  - Add `WIREGUARD_PRESHARED_KEY` from the same AirVPN WireGuard config.
-  - `helm-values/gluetun_values.yaml` sets `secret.create: false` so the chart does not create a placeholder secret.
-
-Validation:
-- ArgoCD health: `argocd app get gluetun` and `argocd app get nzbget`
-- WireGuard up: `kubectl -n default logs deploy/gluetun | rg -i "wireguard|tunnel"`
-- VPN egress from NZBGet: `kubectl -n default exec deploy/nzbget -- curl -s ifconfig.me`
-- NZBGet UI: `kubectl -n default port-forward deploy/nzbget 6789:6789`
-
-Note: NZBGet is configured to use the HTTP proxy via `HTTP_PROXY`/`HTTPS_PROXY`. If your NNTP traffic does not honor proxy settings, consider using a proxy-aware downloader or running the downloader in the same pod as Gluetun.

custom_helm_charts/gluetun/templates/NOTES.txt

@@ -1,19 +0,0 @@
-1. Get the application URL by running these commands:
-{{- if .Values.ingress.enabled }}
-{{- range .Values.ingress.hosts }}
-  http{{ if $.Values.ingress.tls }}s{{ end }}://{{ . }}{{ $.Values.ingress.path }}
-{{- end }}
-{{- else if contains "NodePort" .Values.service.type }}
-  export NODE_PORT=$(kubectl get --namespace {{ .Release.Namespace }} -o jsonpath="{.spec.ports[0].nodePort}" services {{ template "gluetun.fullname" . }})
-  export NODE_IP=$(kubectl get nodes --namespace {{ .Release.Namespace }} -o jsonpath="{.items[0].status.addresses[0].address}")
-  echo http://$NODE_IP:$NODE_PORT
-{{- else if contains "LoadBalancer" .Values.service.type }}
-     NOTE: It may take a few minutes for the LoadBalancer IP to be available.
-           You can watch the status of by running 'kubectl get svc -w {{ template "gluetun.fullname" . }}'
-  export SERVICE_IP=$(kubectl get svc --namespace {{ .Release.Namespace }} {{ template "gluetun.fullname" . }} -o jsonpath='{.status.loadBalancer.ingress[0].ip}')
-  echo http://$SERVICE_IP:{{ .Values.service.port }}
-{{- else if contains "ClusterIP" .Values.service.type }}
-  export POD_NAME=$(kubectl get pods --namespace {{ .Release.Namespace }} -l "app={{ template "gluetun.name" . }},release={{ .Release.Name }}" -o jsonpath="{.items[0].metadata.name}")
-  echo "Visit http://127.0.0.1:8888 to use your application"
-  kubectl port-forward $POD_NAME 8888:{{ .Values.service.port }}
-{{- end }}

custom_helm_charts/gluetun/templates/_helpers.tpl

@@ -1,32 +0,0 @@
-{{/* vim: set filetype=mustache: */}}
-{{/*
-Expand the name of the chart.
-*/}}
-{{- define "gluetun.name" -}}
-{{- default .Chart.Name .Values.nameOverride | trunc 63 | trimSuffix "-" -}}
-{{- end -}}
-
-{{/*
-Create a default fully qualified app name.
-We truncate at 63 chars because some Kubernetes name fields are limited to this (by the DNS naming spec).
-If release name contains chart name it will be used as a full name.
-*/}}
-{{- define "gluetun.fullname" -}}
-{{- if .Values.fullnameOverride -}}
-{{- .Values.fullnameOverride | trunc 63 | trimSuffix "-" -}}
-{{- else -}}
-{{- $name := default .Chart.Name .Values.nameOverride -}}
-{{- if contains $name .Release.Name -}}
-{{- .Release.Name | trunc 63 | trimSuffix "-" -}}
-{{- else -}}
-{{- printf "%s-%s" .Release.Name $name | trunc 63 | trimSuffix "-" -}}
-{{- end -}}
-{{- end -}}
-{{- end -}}
-
-{{/*
-Create chart name and version as used by the chart label.
-*/}}
-{{- define "gluetun.chart" -}}
-{{- printf "%s-%s" .Chart.Name .Chart.Version | replace "+" "_" | trunc 63 | trimSuffix "-" -}}
-{{- end -}}

custom_helm_charts/gluetun/templates/deployment.yaml

@@ -1,55 +0,0 @@
-apiVersion: apps/v1
-kind: Deployment
-metadata:
-  name: {{ template "gluetun.fullname" . }}
-  labels:
-    app: {{ template "gluetun.name" . }}
-    chart: {{ template "gluetun.chart" . }}
-    release: {{ .Release.Name }}
-    heritage: {{ .Release.Service }}
-spec:
-  replicas: {{ .Values.replicaCount }}
-  selector:
-    matchLabels:
-      app: {{ template "gluetun.name" . }}
-      release: {{ .Release.Name }}
-  template:
-    metadata:
-      labels:
-        app: {{ template "gluetun.name" . }}
-        release: {{ .Release.Name }}
-    spec:
-      volumes:
-{{ toYaml .Values.volumes | indent 6 }}
-      containers:
-        - name: {{ .Chart.Name }}
-          image: "{{ .Values.image.repository }}:{{ .Values.image.tag }}"
-          imagePullPolicy: {{ .Values.image.pullPolicy }}
-          securityContext:
-{{ toYaml .Values.securityContext | indent 12 }}
-          env:
-{{ toYaml .Values.env | indent 12 }}
-          ports:
-            - name: http-proxy
-              containerPort: {{ .Values.service.port }}
-              protocol: TCP
-          livenessProbe:
-{{ toYaml .Values.livenessProbe | indent 12 }}
-          readinessProbe:
-{{ toYaml .Values.readinessProbe | indent 12 }}
-          volumeMounts:
-{{ toYaml .Values.volumeMounts | indent 12 }}
-          resources:
-{{ toYaml .Values.resources | indent 12 }}
-    {{- with .Values.nodeSelector }}
-      nodeSelector:
-{{ toYaml . | indent 8 }}
-    {{- end }}
-    {{- with .Values.affinity }}
-      affinity:
-{{ toYaml . | indent 8 }}
-    {{- end }}
-    {{- with .Values.tolerations }}
-      tolerations:
-{{ toYaml . | indent 8 }}
-    {{- end }}

custom_helm_charts/gluetun/templates/ingress.yaml

@@ -1,38 +0,0 @@
-{{- if .Values.ingress.enabled -}}
-{{- $fullName := include "gluetun.fullname" . -}}
-{{- $ingressPath := .Values.ingress.path -}}
-apiVersion: extensions/v1beta1
-kind: Ingress
-metadata:
-  name: {{ $fullName }}
-  labels:
-    app: {{ template "gluetun.name" . }}
-    chart: {{ template "gluetun.chart" . }}
-    release: {{ .Release.Name }}
-    heritage: {{ .Release.Service }}
-{{- with .Values.ingress.annotations }}
-  annotations:
-{{ toYaml . | indent 4 }}
-{{- end }}
-spec:
-{{- if .Values.ingress.tls }}
-  tls:
-  {{- range .Values.ingress.tls }}
-    - hosts:
-      {{- range .hosts }}
-        - {{ . }}
-      {{- end }}
-      secretName: {{ .secretName }}
-  {{- end }}
-{{- end }}
-  rules:
-  {{- range .Values.ingress.hosts }}
-    - host: {{ . }}
-      http:
-        paths:
-          - path: {{ $ingressPath }}
-            backend:
-              serviceName: {{ $fullName }}
-              servicePort: http
-  {{- end }}
-{{- end }}

custom_helm_charts/gluetun/templates/secret.yaml

@@ -1,15 +0,0 @@
-{{- if .Values.secret.create -}}
-apiVersion: v1
-kind: Secret
-metadata:
-  name: {{ .Values.secret.name }}
-  labels:
-    app: {{ template "gluetun.name" . }}
-    chart: {{ template "gluetun.chart" . }}
-    release: {{ .Release.Name }}
-    heritage: {{ .Release.Service }}
-type: Opaque
-stringData:
-  WIREGUARD_PRIVATE_KEY: {{ .Values.secret.privateKey | quote }}
-  WIREGUARD_PRESHARED_KEY: {{ .Values.secret.presharedKey | quote }}
-{{- end }}

custom_helm_charts/gluetun/templates/service.yaml

@@ -1,19 +0,0 @@
-apiVersion: v1
-kind: Service
-metadata:
-  name: {{ template "gluetun.fullname" . }}
-  labels:
-    app: {{ template "gluetun.name" . }}
-    chart: {{ template "gluetun.chart" . }}
-    release: {{ .Release.Name }}
-    heritage: {{ .Release.Service }}
-spec:
-  type: {{ .Values.service.type }}
-  ports:
-    - port: {{ .Values.service.port }}
-      targetPort: {{ .Values.service.port }}
-      protocol: TCP
-      name: http-proxy
-  selector:
-    app: {{ template "gluetun.name" . }}
-    release: {{ .Release.Name }}

custom_helm_charts/gluetun/values.yaml

@@ -1,102 +0,0 @@
-replicaCount: 1
-
-image:
-  repository: qmcgaw/gluetun
-  tag: latest
-  pullPolicy: IfNotPresent
-
-env:
-  - name: VPN_SERVICE_PROVIDER
-    value: "airvpn"
-  - name: VPN_TYPE
-    value: "wireguard"
-  - name: WIREGUARD_PRIVATE_KEY
-    valueFrom:
-      secretKeyRef:
-        name: gluetun-wireguard
-        key: WIREGUARD_PRIVATE_KEY
-  - name: WIREGUARD_PRESHARED_KEY
-    valueFrom:
-      secretKeyRef:
-        name: gluetun-wireguard
-        key: WIREGUARD_PRESHARED_KEY
-  - name: WIREGUARD_ADDRESSES
-    value: "REPLACE_ME"
-  - name: SERVER_COUNTRIES
-    value: "REPLACE_ME"
-  - name: HTTPPROXY
-    value: "on"
-  - name: HTTPPROXY_LOG
-    value: "off"
-  - name: FIREWALL_INPUT_PORTS
-    value: "8888"
-  - name: TZ
-    value: "Europe/Amsterdam"
-
-secret:
-  create: true
-  name: gluetun-wireguard
-  privateKey: "REPLACE_ME"
-  presharedKey: "REPLACE_ME"
-
-service:
-  type: ClusterIP
-  port: 8888
-
-ingress:
-  enabled: false
-  annotations:
-    kubernetes.io/ingress.class: nginx
-    kubernetes.io/tls-acme: "true"
-  path: /
-  hosts:
-    - gluetun.example.org
-  tls:
-    - secretName: gluetun-example-org
-      hosts:
-        - gluetun.example.org
-
-volumes:
-  - name: dev-tun
-    hostPath:
-      path: /dev/net/tun
-
-volumeMounts:
-  - name: dev-tun
-    mountPath: "/dev/net/tun"
-
-securityContext:
-  allowPrivilegeEscalation: false
-  capabilities:
-    add:
-      - NET_ADMIN
-
-livenessProbe:
-  tcpSocket:
-    port: 8888
-  initialDelaySeconds: 10
-  periodSeconds: 20
-  timeoutSeconds: 2
-  failureThreshold: 3
-
-readinessProbe:
-  tcpSocket:
-    port: 8888
-  initialDelaySeconds: 5
-  periodSeconds: 10
-  timeoutSeconds: 2
-  failureThreshold: 3
-
-resources:
-  requests:
-    memory: 128Mi
-    cpu: 100m
-  limits:
-    memory: 512Mi
-    cpu: 500m
-
-nodeSelector: {}
-
-tolerations: []
-
-affinity: {}

custom_helm_charts/nzbget/templates/deployment.yaml

@@ -21,10 +21,33 @@ spec:
     spec:
       volumes:
 {{ toYaml .Values.volumes | indent 6 }}
+    {{- with .Values.initContainers }}
+      initContainers:
+{{ toYaml . | indent 8 }}
+    {{- end }}
       containers:
+        - name: gluetun
+          image: "{{ .Values.gluetun.image.repository }}:{{ .Values.gluetun.image.tag }}"
+          imagePullPolicy: {{ .Values.gluetun.image.pullPolicy }}
+          securityContext:
+{{ toYaml .Values.gluetun.securityContext | indent 12 }}
+          env:
+{{ toYaml .Values.gluetun.env | indent 12 }}
+          livenessProbe:
+{{ toYaml .Values.gluetun.livenessProbe | indent 12 }}
+          readinessProbe:
+{{ toYaml .Values.gluetun.readinessProbe | indent 12 }}
+          volumeMounts:
+{{ toYaml .Values.gluetun.volumeMounts | indent 12 }}
+          resources:
+{{ toYaml .Values.gluetun.resources | indent 12 }}
         - name: {{ .Chart.Name }}
           image: "{{ .Values.image.repository }}:{{ .Values.image.tag }}"
           imagePullPolicy: {{ .Values.image.pullPolicy }}
+          {{- with .Values.command }}
+          command:
+{{ toYaml . | indent 12 }}
+          {{- end }}
           env:
 {{ toYaml .Values.env | indent 12 }}
           ports:

helm-values/gluetun_values.yaml

@@ -1,88 +0,0 @@
-replicaCount: 1
-
-image:
-  repository: qmcgaw/gluetun
-  tag: "latest"
-  pullPolicy: Always
-
-env:
-  - name: VPN_SERVICE_PROVIDER
-    value: "airvpn"
-  - name: VPN_TYPE
-    value: "wireguard"
-  - name: WIREGUARD_PRIVATE_KEY
-    valueFrom:
-      secretKeyRef:
-        name: gluetun-wireguard
-        key: WIREGUARD_PRIVATE_KEY
-  - name: WIREGUARD_PRESHARED_KEY
-    valueFrom:
-      secretKeyRef:
-        name: gluetun-wireguard
-        key: WIREGUARD_PRESHARED_KEY
-  - name: WIREGUARD_ADDRESSES
-    value: "10.160.17.207/32,fd7d:76ee:e68f:a993:61d7:a5fe:f834:90e1/128"
-  - name: SERVER_COUNTRIES
-    value: "Netherlands"
-  - name: HTTPPROXY
-    value: "on"
-  - name: HTTPPROXY_LOG
-    value: "off"
-  - name: FIREWALL_INPUT_PORTS
-    value: "8888"
-  - name: TZ
-    value: "Europe/Amsterdam"
-
-secret:
-  create: false
-  name: gluetun-wireguard
-  privateKey: "REPLACE_ME"
-
-service:
-  type: ClusterIP
-  port: 8888
-
-volumes:
-  - name: dev-tun
-    hostPath:
-      path: /dev/net/tun
-
-volumeMounts:
-  - name: dev-tun
-    mountPath: "/dev/net/tun"
-
-securityContext:
-  allowPrivilegeEscalation: false
-  capabilities:
-    add:
-      - NET_ADMIN
-
-livenessProbe:
-  tcpSocket:
-    port: 8888
-  initialDelaySeconds: 10
-  periodSeconds: 20
-  timeoutSeconds: 2
-  failureThreshold: 3
-
-readinessProbe:
-  tcpSocket:
-    port: 8888
-  initialDelaySeconds: 5
-  periodSeconds: 10
-  timeoutSeconds: 2
-  failureThreshold: 3
-
-resources:
-  requests:
-    memory: 128Mi
-    cpu: 100m
-  limits:
-    memory: 512Mi
-    cpu: 500m
-
-nodeSelector: {}
-
-tolerations: []
-
-affinity: {}

helm-values/nzbget_values.yaml

@@ -5,6 +5,25 @@ image:
   tag: "latest"
   pullPolicy: Always
 
+# Wait for the gluetun sidecar's tunnel to come up before starting nzbget.
+# Without this, nzbget races gluetun on pod start: /etc/resolv.conf points at
+# 10.128.0.1 (reachable only via the tunnel), so during the ~20s gluetun takes
+# to establish WireGuard, every DNS lookup from nzbget fails with EAI_AGAIN.
+# Any items in the queue at that moment hit nzbget's per-file HealthCheck
+# threshold (~97.9%) and get auto-cancelled — even ones that would otherwise
+# complete fine. Polling DNS resolution is the direct test: if it resolves,
+# the tunnel is up and DNS works.
+command:
+  - sh
+  - -c
+  - |
+    until nslookup news.newshosting.com >/dev/null 2>&1; do
+      echo "waiting for gluetun tunnel + DNS..."
+      sleep 2
+    done
+    echo "tunnel up, starting nzbget"
+    exec /init
+
 env:
   - name: PUID
     value: "1000"
@@ -12,16 +31,121 @@ env:
     value: "1000"
   - name: TZ
     value: "Europe/Amsterdam"
-  - name: HTTP_PROXY
+
-    value: "http://gluetun.default.svc.cluster.local:8888"
+# gluetun runs as a sidecar in this pod (same pattern as qbittorrent): it shares
-  - name: http_proxy
+# the pod network namespace and installs the WireGuard tunnel + a kill-switch, so
-    value: "http://gluetun.default.svc.cluster.local:8888"
+# ALL of nzbget's traffic — including NNTP (port 563) to newshosting — egresses
-  - name: HTTPS_PROXY
+# through the VPN. (An HTTP proxy can't cover NNTP, which is why the old
-    value: "http://gluetun.default.svc.cluster.local:8888"
+# HTTP_PROXY-to-standalone-gluetun approach left usenet downloads going direct.)
-  - name: https_proxy
+# Uses its own AirVPN device/secret (gluetun-wireguard-nzbget) to avoid sharing a
-    value: "http://gluetun.default.svc.cluster.local:8888"
+# WireGuard IP with the qbittorrent tunnel. Keep DOT=off + DNS_ADDRESS — see the
-  - name: NO_PROXY
+# AirVPN-blocks-DoT gotcha in CLAUDE.md.
-    value: "localhost,127.0.0.1,.svc,.cluster.local"
+gluetun:
+  image:
+    repository: qmcgaw/gluetun
+    tag: v3.41.1
+    pullPolicy: IfNotPresent
+  env:
+    - name: VPN_SERVICE_PROVIDER
+      value: "airvpn"
+    - name: VPN_TYPE
+      value: "wireguard"
+    - name: WIREGUARD_PRIVATE_KEY
+      valueFrom:
+        secretKeyRef:
+          name: gluetun-wireguard-nzbget
+          key: WIREGUARD_PRIVATE_KEY
+    - name: WIREGUARD_PRESHARED_KEY
+      valueFrom:
+        secretKeyRef:
+          name: gluetun-wireguard-nzbget
+          key: WIREGUARD_PRESHARED_KEY
+    - name: WIREGUARD_ADDRESSES
+      value: "10.166.207.220/32,fd7d:76ee:e68f:a993:bd5e:ddfc:ad2c:d30c/128"
+    - name: SERVER_COUNTRIES
+      value: "Netherlands"
+    - name: DOT
+      value: "off"
+    - name: DNS_ADDRESS
+      value: "10.128.0.1"
+    - name: FIREWALL_INPUT_PORTS
+      value: "6789"
+    - name: TZ
+      value: "Europe/Amsterdam"
+  securityContext:
+    allowPrivilegeEscalation: false
+    capabilities:
+      add:
+        - NET_ADMIN
+  livenessProbe:
+    tcpSocket:
+      port: 8000
+    initialDelaySeconds: 10
+    periodSeconds: 20
+    timeoutSeconds: 2
+    failureThreshold: 3
+  readinessProbe:
+    tcpSocket:
+      port: 8000
+    initialDelaySeconds: 5
+    periodSeconds: 10
+    timeoutSeconds: 2
+    failureThreshold: 3
+  resources:
+    requests:
+      memory: 128Mi
+      cpu: 100m
+    limits:
+      memory: 512Mi
+      cpu: 500m
+  volumeMounts:
+    - name: dev-tun
+      mountPath: "/dev/net/tun"
+
+# nzbget cannot read server credentials from environment variables (its
+# ${...} config syntax only references other nzbget options, not env). So an
+# init container renders the Server1 (newshosting) block into nzbget.conf on
+# every start: the non-secret settings live here in git, while the username
+# and password come from the out-of-band `usenet-creds` Secret (same pattern
+# as gluetun-wireguard — secret not committed). Rotating the secret + a pod
+# restart re-renders the creds. No provider password is ever stored in git.
+initContainers:
+  - name: render-newshosting
+    image: lscr.io/linuxserver/nzbget:latest
+    command:
+      - sh
+      - -c
+      - |
+        f=/config/nzbget.conf
+        [ -f "$f" ] || { echo "nzbget.conf absent; main container will seed defaults"; exit 0; }
+        sed -i \
+          -e "s|^Server1.Active=.*|Server1.Active=yes|" \
+          -e "s|^Server1.Name=.*|Server1.Name=newshosting|" \
+          -e "s|^Server1.Host=.*|Server1.Host=news.newshosting.com|" \
+          -e "s|^Server1.Port=.*|Server1.Port=563|" \
+          -e "s|^Server1.Encryption=.*|Server1.Encryption=yes|" \
+          -e "s|^Server1.Connections=.*|Server1.Connections=50|" \
+          -e "s|^ArticleCache=.*|ArticleCache=700|" \
+          -e "s|^WriteBuffer=.*|WriteBuffer=1024|" \
+          -e "s|^Server1.Username=.*|Server1.Username=${NEWSHOSTING_USER}|" \
+          -e "s|^Server1.Password=.*|Server1.Password=${NEWSHOSTING_PASS}|" \
+          "$f"
+        echo "rendered newshosting Server1 block into nzbget.conf"
+    env:
+      - name: NEWSHOSTING_USER
+        valueFrom:
+          secretKeyRef:
+            name: usenet-creds
+            key: NEWSHOSTING_USER
+      - name: NEWSHOSTING_PASS
+        valueFrom:
+          secretKeyRef:
+            name: usenet-creds
+            key: NEWSHOSTING_PASS
+    volumeMounts:
+      - name: plex-data
+        mountPath: /config
+        subPath: configs/nzbget
 
 service:
   type: ClusterIP
@@ -31,6 +155,9 @@ volumes:
   - name: plex-data
     persistentVolumeClaim:
       claimName: "plex-data"
+  - name: dev-tun
+    hostPath:
+      path: /dev/net/tun
 
 volumeMounts:
   - name: plex-data

manifests/disk-usage-alert/cronjob.yaml

@@ -0,0 +1,81 @@
+# SSD disk-usage alert for turing3 /mnt/ssd (shared by Postgres + Plex + media).
+# A full SSD crashes Postgres and corrupts Plex's SQLite DB, so we warn early.
+#
+# NOTE: the Telegram bot token + chat id live in the `telegram-disk-alert` Secret,
+# created out-of-band (NOT in git) so the token stays out of history:
+#   kubectl -n default create secret generic telegram-disk-alert \
+#     --from-literal=botToken='<token>' --from-literal=chatId='<chatId>'
+#
+# Mounts the existing RWX plex-data PVC purely to read `df` of the underlying SSD.
+apiVersion: batch/v1
+kind: CronJob
+metadata:
+  name: disk-usage-alert
+  namespace: default
+spec:
+  schedule: "*/15 * * * *"
+  concurrencyPolicy: Forbid
+  successfulJobsHistoryLimit: 1
+  failedJobsHistoryLimit: 3
+  jobTemplate:
+    spec:
+      backoffLimit: 0
+      activeDeadlineSeconds: 120
+      template:
+        spec:
+          restartPolicy: Never
+          containers:
+            - name: check
+              image: curlimages/curl:8.11.1
+              imagePullPolicy: IfNotPresent
+              env:
+                - name: THRESHOLD          # alert at/above this % used
+                  value: "90"
+                - name: COOLDOWN_SEC       # min seconds between alerts (3h)
+                  value: "10800"
+                - name: MOUNT
+                  value: "/data"
+                - name: BOT_TOKEN
+                  valueFrom:
+                    secretKeyRef: { name: telegram-disk-alert, key: botToken }
+                - name: CHAT_ID
+                  valueFrom:
+                    secretKeyRef: { name: telegram-disk-alert, key: chatId }
+              command: ["/bin/sh", "-c"]
+              args:
+                - |
+                  set -u
+                  pct=$(df -P "$MOUNT" | awk 'END{gsub("%","",$5); print $5}')
+                  avail=$(df -Ph "$MOUNT" | awk 'END{print $4}')
+                  now=$(date +%s)
+                  marker="$MOUNT/.disk-alert-last"
+                  send() {
+                    curl -s -m 15 "https://api.telegram.org/bot${BOT_TOKEN}/sendMessage" \
+                      --data-urlencode "chat_id=${CHAT_ID}" \
+                      --data-urlencode "text=$1" -d "parse_mode=HTML" >/dev/null
+                  }
+                  if [ "${TEST:-0}" = "1" ]; then
+                    send "✅ <b>turingpi disk-alert test</b> — SSD at ${pct}% (free ${avail}). Alerting works."
+                    echo "TEST sent (${pct}% used)"; exit 0
+                  fi
+                  if [ "$pct" -ge "$THRESHOLD" ]; then
+                    last=0; [ -f "$marker" ] && last=$(cat "$marker" 2>/dev/null || echo 0)
+                    if [ $((now - last)) -ge "$COOLDOWN_SEC" ]; then
+                      send "⚠️ <b>turingpi SSD ${pct}% full</b> — only ${avail} free on /mnt/ssd. Postgres + Plex crash at 100%. Prune content / check maintainerr."
+                      echo "$now" > "$marker"
+                      echo "ALERT sent (${pct}% >= ${THRESHOLD}%)"
+                    else
+                      echo "over threshold (${pct}%) but within cooldown; skipping"
+                    fi
+                  else
+                    echo "ok: ${pct}% used, ${avail} free (threshold ${THRESHOLD}%)"
+                  fi
+              resources:
+                requests: { cpu: 10m, memory: 16Mi }
+                limits: { memory: 64Mi }
+              volumeMounts:
+                - { name: data, mountPath: /data }
+          volumes:
+            - name: data
+              persistentVolumeClaim:
+                claimName: plex-data

manifests/maintainerr/deployment.yaml

@@ -0,0 +1,63 @@
+---
+apiVersion: apps/v1
+kind: Deployment
+metadata:
+  name: maintainerr
+  namespace: default
+  labels:
+    app: maintainerr
+spec:
+  replicas: 1
+  # SQLite on the shared NFS PVC: never run two writers at once.
+  strategy:
+    type: Recreate
+  selector:
+    matchLabels:
+      app: maintainerr
+  template:
+    metadata:
+      labels:
+        app: maintainerr
+    spec:
+      securityContext:
+        runAsUser: 1000
+        runAsGroup: 1000
+        fsGroup: 1000
+      containers:
+        - name: maintainerr
+          image: ghcr.io/maintainerr/maintainerr:latest
+          imagePullPolicy: Always
+          ports:
+            - name: http
+              containerPort: 6246
+              protocol: TCP
+          env:
+            - name: TZ
+              value: "Europe/Amsterdam"
+          volumeMounts:
+            - name: plex-data
+              mountPath: /opt/data
+              subPath: configs/maintainerr
+          readinessProbe:
+            httpGet:
+              path: /api/app/status
+              port: http
+            initialDelaySeconds: 20
+            periodSeconds: 15
+          livenessProbe:
+            httpGet:
+              path: /api/app/status
+              port: http
+            initialDelaySeconds: 60
+            periodSeconds: 30
+          resources:
+            requests:
+              memory: "256Mi"
+              cpu: "100m"
+            limits:
+              memory: "512Mi"
+              cpu: "1000m"
+      volumes:
+        - name: plex-data
+          persistentVolumeClaim:
+            claimName: plex-data

manifests/maintainerr/service.yaml

@@ -0,0 +1,17 @@
+---
+apiVersion: v1
+kind: Service
+metadata:
+  name: maintainerr
+  namespace: default
+  labels:
+    app: maintainerr
+spec:
+  type: ClusterIP
+  ports:
+    - port: 6246
+      targetPort: http
+      protocol: TCP
+      name: http
+  selector:
+    app: maintainerr

non_argo_values/metallb_values.yaml

@@ -0,0 +1,13 @@
+# MetalLB configuration for TuringPi K3s cluster
+#
+# L2 (ARP) mode only: the cluster advertises LoadBalancer IPs via the default
+# IPAddressPool + L2Advertisement (resources/metallb.yml). There are no BGP
+# peers, so FRR is not needed. Disable both the legacy embedded FRR and the
+# newer frr-k8s daemonset -- their images cause DiskPressure on the small Pi
+# nodes and add no value without BGP.
+speaker:
+  frr:
+    enabled: false
+
+frrk8s:
+  enabled: false