Reorganize the brownfield repository, remove retired and generated artifacts, harden ignore rules, and record the GitOps/IaC redesign.
119 lines
5.7 KiB
Markdown
119 lines
5.7 KiB
Markdown
# Envoy Gateway — LAN ingress + Authelia forward-auth
|
|
|
|
The cluster's HTTP entry point for services on `ad.ddupan.top`, and the enforcement
|
|
point for Authelia authentication. Envoy's LoadBalancer holds **192.168.10.127** (k3s
|
|
ServiceLB); routing is by `Host` header.
|
|
|
|
- Chart: `oci://docker.io/envoyproxy/gateway-helm` **v1.5.6**, ns `envoy-gateway-system`
|
|
- `gateway.yaml` — `GatewayClass eg` + `Gateway eg` (`:80` plaintext, `:443` wildcard TLS)
|
|
- Gateway API **v1.3.0**
|
|
- Replaced Contour on 2026-07-25 (see [Why not Contour](#why-not-contour))
|
|
|
|
## Adding a service
|
|
|
|
1. an `HTTPRoute` with `parentRefs` → `eg` / `envoy-gateway-system`, `sectionName: https`
|
|
2. an A record in `samba_ad_extra_a_records` → `192.168.10.127` (`../../infrastructure/samba-ad`), then
|
|
`ansible-playbook provision-dc.yml --tags dns`
|
|
3. **optionally** a `SecurityPolicy` for Authelia forward-auth — see `../../apps/netbox`
|
|
|
|
No certificate work: the `:443` listener already serves the `*.ad.ddupan.top` wildcard
|
|
from `../cert-manager`. Worked example: `../../apps/netbox`.
|
|
|
|
## Why not Contour
|
|
|
|
Contour worked as an ingress, but supports **only the gRPC** Envoy ext_authz protocol —
|
|
*"Only the Envoy GRPC authorization protocol will be supported"* (Contour 1.33 docs).
|
|
Authelia implements the **HTTP** ExtAuthz filter. The two cannot meet, so Authelia
|
|
forward-auth was impossible.
|
|
|
|
That mattered because **NetBox has no SSO group→role mapping** — its group/superuser
|
|
mapping is LDAP-only, and the OIDC pipeline can only assign one static group. Without
|
|
forward-auth the options were "promote every user by hand" or "drop SSO and use LDAP,
|
|
losing 2FA". Envoy Gateway's `SecurityPolicy.extAuth.http` removes the dilemma.
|
|
|
|
Envoy Gateway was chosen over Traefik because everything already built — Gateway,
|
|
HTTPRoute, wildcard cert, DNS — is Gateway API, so only the GatewayClass and controller
|
|
changed. Traefik would have meant reverting to `IngressRoute` + `Middleware`.
|
|
|
|
## Forward-auth: how it fits together
|
|
|
|
```
|
|
browser ──▶ Envoy ──(ext_authz HTTP)──▶ Authelia ──▶ 200 + Remote-* headers
|
|
│ └──▶ 401 ⇒ 302 to auth.ddupan.top
|
|
└──▶ upstream app (headers attached)
|
|
```
|
|
|
|
Pieces, each in the directory that owns it:
|
|
|
|
| where | what |
|
|
|---|---|
|
|
| `../../apps/netbox/securitypolicy.yaml` | `SecurityPolicy` targeting the app's HTTPRoute |
|
|
| `../../apps/authelia/referencegrant-extauth.yaml` | lets a SecurityPolicy in another namespace reference the Authelia Service |
|
|
| `../../apps/authelia/values.yaml` | `server.endpoints.authz.ext-authz` + an `access_control` rule |
|
|
| `../../apps/netbox/networkpolicy.yaml` | stops anything bypassing Envoy to reach the app directly |
|
|
|
|
### Three details that cost time
|
|
|
|
- **`headersToBackend` lives under `extAuth.http`, not `extAuth`.** One level up the API
|
|
rejects it: `unknown field "spec.extAuth.headersToBackend"`.
|
|
- **`backendRefs.port` is the SERVICE port, not the container port.** The Authelia chart
|
|
publishes `80 → targetPort http (9091)`; using `9091` fails with
|
|
`TCP Port 9091 not found on service authelia/authelia`.
|
|
- **Authelia's session cookie is scoped to `ddupan.top`**, which already covers
|
|
`*.ad.ddupan.top` — so SSO works across both without touching the cookie config.
|
|
|
|
### ⚠ Trust boundary
|
|
|
|
Apps behind forward-auth trust `Remote-*` headers. Two things make that safe and **both**
|
|
must remain true:
|
|
|
|
1. `headersToBackend` **overrides** any client-supplied value ("coexisting headers will be
|
|
overridden"), so a spoofed `Remote-User` cannot survive the hop through Envoy.
|
|
2. A `NetworkPolicy` per app restricts pod ingress to `envoy-gateway-system`, so nothing
|
|
in-cluster can bypass Envoy. k3s enforces NetworkPolicy (kube-router), so this is real.
|
|
|
|
Verified by test: a pod in `default` sending `Remote-User: admin` to the app Service gets
|
|
**connection refused**, while the same request from `envoy-gateway-system` is served.
|
|
Also set `failOpen: false` — if Authelia is down, refuse traffic rather than admit
|
|
unauthenticated requests to an app whose auth model is "trust the header".
|
|
|
|
## Install
|
|
|
|
```bash
|
|
# The Gateway API CRDs may already be owned by another tool's field manager (Contour's
|
|
# quickstart used client-side apply), which makes Helm fail with
|
|
# "conflict with kubectl-client-side-apply: .spec.versions".
|
|
# Transfer ownership WITHOUT deleting the CRDs (deleting them would delete every
|
|
# Gateway and HTTPRoute):
|
|
helm pull oci://docker.io/envoyproxy/gateway-helm --version v1.5.6 --untar
|
|
kubectl apply --server-side --force-conflicts --field-manager=helm \
|
|
-f gateway-helm/crds/gatewayapi-crds.yaml
|
|
|
|
helm upgrade --install envoy-gateway oci://docker.io/envoyproxy/gateway-helm \
|
|
--version v1.5.6 -n envoy-gateway-system --create-namespace
|
|
|
|
kubectl apply -f envoy-gateway/gateway.yaml
|
|
```
|
|
|
|
Only one LoadBalancer can hold `:80/:443` — k3s ServiceLB uses hostPorts, so a second
|
|
one's `svclb-*` DaemonSet sits at `0/1` and the Gateway stays `PROGRAMMED: False` until
|
|
the previous controller's Service is gone.
|
|
|
|
## Verify
|
|
|
|
```bash
|
|
kubectl -n envoy-gateway-system get gateway eg # PROGRAMMED=True, ADDRESS 192.168.10.127
|
|
kubectl -n envoy-gateway-system get pods # envoy-... 2/2
|
|
kubectl -n <ns> get securitypolicy # Accepted=True
|
|
curl -s -o /dev/null -w '%{http_code}\n' http://192.168.10.127/ # 404 = serving, no route matched
|
|
```
|
|
|
|
`ss -lntp` shows **nothing** on `:80/:443` even when healthy — ServiceLB forwards via
|
|
iptables/NodePort rather than binding. Test with `curl`, not `ss`.
|
|
|
|
## Outstanding
|
|
|
|
Contour source manifests were removed from the clean baseline. Live leftovers may
|
|
still exist and must be inventoried before deletion; do not infer live state from
|
|
the archive cleanup. The legacy Git history retains the retired manifests.
|