{
    "product": "Cenvero Stratum",
    "generated_at": "2026-08-03T07:17:34+00:00",
    "format": "cenvero-docs-v1",
    "document_count": 1,
    "documents": [
        {
            "slug": "networking/load-balancer",
            "title": "Load Balancer",
            "category": "Networking",
            "url": "https://stratum.cenvero.com/docs/networking/load-balancer",
            "headings": [
                {
                    "level": 1,
                    "text": "Load Balancer"
                },
                {
                    "level": 2,
                    "text": "Two load balancers, and which one you want"
                },
                {
                    "level": 3,
                    "text": "What the L4 balancer requires: its own interface"
                },
                {
                    "level": 3,
                    "text": "Switching a node's mode"
                },
                {
                    "level": 2,
                    "text": "How the L4 balancer moves a packet"
                },
                {
                    "level": 3,
                    "text": "One VIP is one address, one port, one protocol"
                },
                {
                    "level": 3,
                    "text": "TLS at layer 4"
                },
                {
                    "level": 3,
                    "text": "Direct server return"
                },
                {
                    "level": 2,
                    "text": "Creating a VIP"
                },
                {
                    "level": 2,
                    "text": "Algorithms"
                },
                {
                    "level": 3,
                    "text": "Setting weights"
                },
                {
                    "level": 2,
                    "text": "Health checks"
                },
                {
                    "level": 3,
                    "text": "Overriding a backend's health manually"
                },
                {
                    "level": 2,
                    "text": "Adding and removing backends live"
                },
                {
                    "level": 3,
                    "text": "Retiring a backend without dropping traffic"
                },
                {
                    "level": 2,
                    "text": "North-south VIPs on Gateway nodes"
                },
                {
                    "level": 2,
                    "text": "Deleting a VIP"
                },
                {
                    "level": 2,
                    "text": "Layer-7 HTTP load balancer"
                },
                {
                    "level": 3,
                    "text": "How a request travels"
                },
                {
                    "level": 3,
                    "text": "Terminating TLS"
                },
                {
                    "level": 3,
                    "text": "Serving several ports"
                },
                {
                    "level": 3,
                    "text": "Backend pools"
                },
                {
                    "level": 3,
                    "text": "TLS-terminating frontends"
                },
                {
                    "level": 2,
                    "text": "See also"
                }
            ],
            "word_count": 3037,
            "markdown": "# Load Balancer\n\nStratum ships an L4 load balancer designed to run in the in-kernel data plane: it distributes TCP and UDP flows across a set of backends using a per-VIP algorithm, maintains a stateful connection table so established flows stay pinned to their backend, and performs health checks to take unhealthy backends out of rotation automatically.\n\n> **Status:** the L4 load balancer's control surface (`lb` commands, VIP definitions, backend pools, health checks) and its in-kernel data plane are both in place, and the data plane loads and attaches on a real Linux kernel. Validate the L4 balancer against your own workload profile before production rollout. The Layer-7 HTTP proxy described below is a separate userspace load balancer and does not depend on it.\n\nA VIP can serve traffic from outside the fabric or stay internal for service-mesh style balancing between endpoints — that is a matter of how you route and announce it, not of what kind of node you are on.\n\n## Two load balancers, and which one you want\n\nStratum has two, and they solve different problems. Choosing the wrong one is the\nmost common mistake, so start here.\n\n| | **L4 (`lb`)** | **L7 HTTP (`l7lb`)** |\n|---|---|---|\n| Works on | TCP and UDP — any protocol | HTTP and HTTPS only |\n| Decides using | Addresses and ports | Hostname, URL path, headers |\n| Handles TLS | No — traffic passes through untouched | Yes — terminates it for you |\n| Use it for | Databases, game servers, SMTP, gRPC, anything not HTTP, or HTTPS you want passed straight through | Websites and APIs where you want one address serving several apps, or TLS handled centrally |\n\nThey are independent. You can run both — an L4 VIP in front of a database and an\nL7 frontend for your web tier — and neither depends on the other.\n\n### What the L4 balancer requires: its own interface\n\nThis is the requirement people actually hit, and it has nothing to do with node\nmode.\n\nThe L4 balancer must be given a **dedicated interface** to receive VIP traffic on\n— one it does not share with your workload bridge or with a Gateway node's WAN or\nLAN interface. Each interface can carry only one of these fast-path attachments,\nso they cannot be stacked on the same NIC.\n\n> **Until that interface is configured, VIPs are configuration only.** Every `lb`\n> command works, your VIPs and backends are stored and listed back correctly, and\n> health checks report — but no traffic is balanced, because nothing is attached\n> to receive it. If your VIP looks perfectly configured and simply does not pass\n> traffic, this is the first thing to check.\n\nThe interface is named in the node's configuration and applied at startup — see\n[Configuration](/docs/configuration). The **L7** balancer has no such\nrequirement: it is an ordinary listener, so it needs a free port and nothing else.\n\n### Switching a node's mode\n\nMode is not a runtime switch. It is set in the node's **signed configuration** —\nat install, or delivered from the panel — and applied when the agent starts.\nThere is deliberately no command that flips it live, because changing which\ninterfaces forward traffic mid-flight would cut the traffic already crossing the\nnode.\n\nTo change it: update the node's configuration, let the node pick it up, and\nrestart the agent. To read the current mode:\n\n```bash\ncenvero-str-ctl node info\n```\n\nEvery node can host workloads *and* forward traffic — one node doing both is a\nnormal setup. What cannot be shared is an *interface*, not a role.\n\n## How the L4 balancer moves a packet\n\nUnderstanding this matters because one deployment detail depends on it.\n\n1. **A client connects to the VIP.** As far as the client is concerned it is\n   talking to one address, and it will keep talking to that address for the\n   lifetime of the connection.\n2. **A backend is chosen**, once, using the VIP's algorithm.\n3. **The destination address is rewritten** to that backend, and the packet is\n   forwarded on. The backend can be anywhere the node can route to — it does not\n   have to be on the same subnet or the same machine.\n4. **The connection is remembered**, so every later packet in that conversation\n   goes to the same backend. The choice is made once, not per packet.\n5. **The reply comes back through the node**, which puts the VIP address back on\n   it before it reaches the client — so the client only ever sees the address it\n   connected to.\n\nTwo consequences follow, and they surprise people:\n\n> **Your backends see the real client address.** Only the destination is\n> rewritten, never the source. Your application logs show actual client IPs with\n> no extra configuration and no `X-Forwarded-For` to parse.\n\n> **Replies must come back through the balancing node.** Step 5 is where the VIP\n> address is restored. If a backend answers the client directly — because it has\n> its own route out — the client receives a reply from an address it never\n> contacted and discards it. The usual symptom is connections that hang instead\n> of failing. Make sure your backends route back through the node holding the\n> VIP.\n\n### One VIP is one address, one port, one protocol\n\nA VIP carries a single `frontend_port` and a single `protocol`. To serve a\nservice on two ports, create two VIPs:\n\n```bash\n# HTTP and HTTPS for the same service = two VIPs, same backends\nsudo cenvero-str-ctl lb create '{\"id\":\"web-80\",\"frontend_ip\":\"10.0.0.100\",\"frontend_port\":80,\"protocol\":\"tcp\",\"algorithm\":\"round-robin\"}'\nsudo cenvero-str-ctl lb create '{\"id\":\"web-443\",\"frontend_ip\":\"10.0.0.100\",\"frontend_port\":443,\"protocol\":\"tcp\",\"algorithm\":\"round-robin\"}'\n```\n\nEach VIP has its own backends, its own algorithm and its own health checks, so\nthey can differ where that is useful — for example checking a different health\npath per port.\n\n### TLS at layer 4\n\nThe L4 balancer does not read or terminate TLS. Encrypted traffic passes through\nexactly as it arrives, and each backend must present its own certificate. That is\nthe right choice when you want end-to-end encryption with no middle box, or when\nthe protocol is not HTTP at all.\n\nIf you want certificates handled in one place instead, that is the L7 balancer —\nsee [Terminating TLS](#terminating-tls) below.\n\n### Direct server return\n\n`dsr_enabled` is accepted on a VIP but **is not yet in effect**: a VIP with it set\nis served through the normal path described above, with replies returning through\nthe node. Plan capacity on that basis.\n\n## Creating a VIP\n\n`lb create` takes **one argument**: a JSON object describing the virtual IP. A VIP and its backends are created separately — first the VIP, then each backend with `lb add-backend`.\n\n| Field | Type | Description |\n|-------|------|-------------|\n| `id` | string | **Required.** The VIP's name, used by every other `lb` command. |\n| `frontend_ip` | string | **Required.** The virtual IP address clients connect to. |\n| `algorithm` | string | **Required.** One of the algorithms below. |\n| `frontend_port` | number | The port clients connect to. |\n| `protocol` | string | `tcp` or `udp`. |\n| `dsr_enabled` | boolean | Enable direct server return. |\n| `health_check` | object | Optional active health check — see **Health checks** below. |\n\n```bash\nsudo cenvero-str-ctl lb create '{\"id\":\"web-lb\",\"frontend_ip\":\"10.20.0.10\",\"frontend_port\":80,\"protocol\":\"tcp\",\"algorithm\":\"least-conn\"}'\n```\n\nList your VIPs:\n\n```bash\ncenvero-str-ctl lb list\n```\n\nShow detail for one VIP, including per-backend health and connection counts:\n\n```bash\ncenvero-str-ctl lb show web-lb\n```\n\n```text\n{\n  \"data\": {\n    \"load_balancer\": {\n      \"id\": \"web-lb\",\n      \"frontend_ip\": \"10.20.0.10\",\n      \"frontend_port\": 80,\n      \"protocol\": \"tcp\",\n      \"algorithm\": \"least-conn\",\n      \"dsr_enabled\": false,\n      \"backends\": [\n        {\n          \"id\": \"web-1\",\n          \"ip\": \"10.20.0.50\",\n          \"port\": 80,\n          \"weight\": 1,\n          \"healthy\": true,\n          \"active_conns\": 0\n        }\n      ]\n    }\n  },\n  \"status\": \"ok\"\n}\n```\n\n## Algorithms\n\nSet one of these as the `algorithm` field. An unrecognized value is rejected when the VIP is created.\n\n| Algorithm | Value | Behaviour |\n|-----------|-------|-----------|\n| Round-robin | `round-robin` | Distributes new connections evenly in turn. Good default for stateless services. |\n| Least connections | `least-conn` | Sends each new connection to the backend with the fewest active connections. Handles variable-cost requests well. |\n| Weighted round-robin | `weighted` | Like round-robin but each backend has a relative weight (see below). Use when backends have unequal capacity. |\n| Source-hash persistence | `source-hash` | Hashes the client source IP to a backend. The same client always reaches the same backend as long as the backend is healthy — useful for session-affinity without shared state. |\n| Maglev | `maglev` | Consistent hashing with Maglev's lookup table, which keeps flow-to-backend mapping stable when the backend set changes. |\n| Consistent hash | `consistent-hash` | Hash-ring backend selection, minimizing remapping when backends are added or removed. |\n\n### Setting weights\n\nWeight is a property of each backend, so choose the `weighted` algorithm on the VIP and give each backend its relative weight as you add it:\n\n```bash\nsudo cenvero-str-ctl lb create '{\"id\":\"api-lb\",\"frontend_ip\":\"10.20.0.20\",\"frontend_port\":443,\"protocol\":\"tcp\",\"algorithm\":\"weighted\"}'\n\nsudo cenvero-str-ctl lb add-backend '{\"vip_id\":\"api-lb\",\"id\":\"api-1\",\"ip\":\"10.20.0.50\",\"port\":443,\"weight\":3}'\nsudo cenvero-str-ctl lb add-backend '{\"vip_id\":\"api-lb\",\"id\":\"api-2\",\"ip\":\"10.20.0.51\",\"port\":443,\"weight\":1}'\n```\n\nThe weights `3` and `1` are relative — `api-1` receives three times the new connections of `api-2`.\n\n## Health checks\n\nThe agent probes backends at a configurable interval. A backend that fails enough consecutive checks is marked unhealthy and removed from the connection pool. It re-enters the pool automatically when it passes the same number of consecutive checks.\n\nA health check is configured as part of the VIP, in the optional `health_check` object passed to `lb create`:\n\n```bash\nsudo cenvero-str-ctl lb create '{\"id\":\"api-lb\",\"frontend_ip\":\"10.20.0.20\",\"frontend_port\":443,\"protocol\":\"tcp\",\"algorithm\":\"round-robin\",\"health_check\":{\"type\":\"http\",\"interval_sec\":5,\"timeout_sec\":2,\"threshold\":3,\"http_path\":\"/healthz\"}}'\n```\n\n| Field | Description | Notes |\n|-------|-------------|-------|\n| `type` | `tcp` (connection probe) or `http` (expects a 2xx response) | Omit the whole object, or leave `type` empty, for no active check |\n| `interval_sec` | Seconds between probes | |\n| `timeout_sec` | Probe timeout in seconds | |\n| `threshold` | Consecutive checks before flipping a backend's state | One value covers both directions |\n| `http_path` | Path to request for an `http` check, e.g. `/healthz` | Ignored for `tcp` checks |\n\n### Overriding a backend's health manually\n\n`lb set-health` is a manual override, not the check configurator — it forces one backend up or down, which is useful for draining a backend before maintenance. It takes three positional arguments: the VIP id, the backend id, and `up` or `down`.\n\n```bash\n# Take a backend out of rotation\nsudo cenvero-str-ctl lb set-health web-lb web-1 down\n\n# Put it back\nsudo cenvero-str-ctl lb set-health web-lb web-1 up\n```\n\nIf an active health check is configured for the VIP, the next probe result will overwrite a manual override.\n\n## Adding and removing backends live\n\nBackend changes take effect immediately without dropping established connections. The connection table keeps existing flows on their current backend until the flow closes naturally.\n\n`lb add-backend` takes a JSON object identifying the VIP (`vip_id`) plus the backend's own `id`, `ip`, `port`, and `weight`. `vip_id`, `id`, and `ip` are required.\n\n```bash\n# Add a backend\nsudo cenvero-str-ctl lb add-backend '{\"vip_id\":\"web-lb\",\"id\":\"web-3\",\"ip\":\"10.20.0.52\",\"port\":80,\"weight\":1}'\n```\n\n### Retiring a backend without dropping traffic\n\n`lb remove-backend` takes effect immediately, which cuts the connections currently\non that backend. To retire one cleanly, drain it first: the load balancer stops\nsending it **new** connections while the existing ones finish, and reports how many\nare still active.\n\n```bash\n# Stop new connections; existing ones keep being served\nsudo cenvero-str-ctl lb drain web-lb web-1\n\n# Re-run to watch the count fall, then remove it once it reaches zero\nsudo cenvero-str-ctl lb remove-backend web-lb web-1\n```\n\nDraining the only healthy backend of a load balancer is refused, since that would\ntake the service down.\n\n`lb remove-backend` takes two positional arguments — the VIP id and the backend id:\n\n```bash\n# Remove a backend\nsudo cenvero-str-ctl lb remove-backend web-lb web-1\n```\n\nTo drain a backend gracefully, mark it down first and let its existing flows finish before removing it:\n\n```bash\nsudo cenvero-str-ctl lb set-health web-lb web-1 down\n# ...wait for active_conns on that backend to reach 0 in `lb show web-lb`...\nsudo cenvero-str-ctl lb remove-backend web-lb web-1\n```\n\nList a VIP's backends at any time:\n\n```bash\ncenvero-str-ctl lb backends web-lb\n```\n\n## North-south VIPs on Gateway nodes\n\nOn a Gateway node, VIPs in the fabric subnet are reachable from outside the cluster automatically because the Gateway node announces those routes via BGP. If you want a VIP to have a publicly routable address, assign it from a prefix your BGP session announces:\n\n```bash\nsudo cenvero-str-ctl lb create '{\"id\":\"external-web\",\"frontend_ip\":\"203.0.113.10\",\"frontend_port\":443,\"protocol\":\"tcp\",\"algorithm\":\"round-robin\"}'\n\nsudo cenvero-str-ctl lb add-backend '{\"vip_id\":\"external-web\",\"id\":\"web-1\",\"ip\":\"10.20.0.50\",\"port\":443,\"weight\":1}'\nsudo cenvero-str-ctl lb add-backend '{\"vip_id\":\"external-web\",\"id\":\"web-2\",\"ip\":\"10.20.0.51\",\"port\":443,\"weight\":1}'\n```\n\nSee [BGP Edge Routing](/docs/networking/bgp) for how prefixes are advertised.\n\n## Deleting a VIP\n\n```bash\nsudo cenvero-str-ctl lb delete web-lb\n```\n\nActive connections are torn down immediately. Drain traffic to backends first if zero disruption is required.\n\n## Layer-7 HTTP load balancer\n\nSeparately from the in-kernel L4 balancer above, Stratum ships a **userspace Layer-7 (HTTP/HTTPS) reverse proxy**. Where the L4 balancer distributes raw TCP/UDP flows, the L7 proxy understands HTTP: it terminates TLS, routes by `Host` header and URL path prefix, and load-balances across backend pools. The two are orthogonal — reach for the L4 balancer for raw flow distribution and the L7 proxy when you need HTTP-aware routing or TLS termination. The L7 proxy stays **off until you configure a frontend**.\n\nIts two building blocks are:\n\n- **Pools** — a named set of HTTP backends with a selection algorithm (`round-robin` or `least-connections`) and health checks.\n- **Frontends** — a listener, optionally terminating TLS, that routes each request to a pool by `Host` and path prefix, with a default pool as fallback.\n\n### How a request travels\n\n1. **A request arrives at a frontend** — a listener on one address and port.\n2. **If the frontend has TLS, it is decrypted here.** From this point on the\n   proxy is working with a plain HTTP request.\n3. **The routes are checked in order and the first match wins.** A route matches\n   on hostname, on a path prefix, or both. Anything matching no route goes to the\n   frontend's `default_pool`; a frontend with no default and no match returns an\n   error rather than guessing.\n4. **The pool picks a backend** — round-robin, or the one with the fewest active\n   requests.\n5. **The request is forwarded over plain HTTP** and the response passed back.\n\nBecause the proxy is the one talking to the client, it adds `X-Forwarded-For`\n(the real client address), `X-Forwarded-Proto` and `X-Forwarded-Host` so your\napplication can still see who asked and how. WebSocket and other upgrade\nconnections pass through unchanged.\n\nRouting is per **request**, not per connection — two requests on the same\nkeep-alive connection can land on different backends, and on different pools.\n\n### Terminating TLS\n\nThis is where most questions come up, so plainly:\n\n> **The certificate goes on the frontend, once. Your backends do not need one.**\n\nA frontend with a `tls` block accepts HTTPS from clients, decrypts, and then\nspeaks **plain HTTP to the backends**. So:\n\n- **You need one certificate per frontend**, not one per backend and not one per\n  port. Ten backends behind an HTTPS frontend need zero certificates between\n  them.\n- **The backend link is unencrypted.** That is fine when the backends are on a\n  network you control — the usual case, since they are your own endpoints on your\n  own fabric. If you need encryption all the way to the application, do not\n  terminate here: use an L4 VIP and let each backend hold its own certificate.\n- **A frontend with no `tls` block serves plain HTTP.** Mixing is normal — run a\n  plain frontend on `:80` and a TLS one on `:443`.\n\nThe certificate itself is either a PEM certificate and key you supply, or the\nnode's own managed certificate. The minimum accepted version is TLS 1.2.\n\n### Serving several ports\n\nUnlike an L4 VIP, one frontend is one listener — but you can add as many\nfrontends as you need, and they can share pools:\n\n```bash\n# Plain HTTP on 80 and TLS on 443, both serving the same backends\nsudo cenvero-str-ctl l7lb frontend add '{\"name\":\"http\",\"listen\":\":80\",\"default_pool\":\"web\"}'\nsudo cenvero-str-ctl l7lb frontend add '{\"name\":\"https\",\"listen\":\":443\",\"tls\":{\"use_managed\":true},\"default_pool\":\"web\"}'\n```\n\nOne frontend can also serve **many sites** on a single port by routing on\nhostname — that is the usual reason to choose L7 over L4:\n\n```bash\nsudo cenvero-str-ctl l7lb frontend add '{\"name\":\"edge\",\"listen\":\":443\",\"tls\":{\"use_managed\":true},\n  \"routes\":[\n    {\"host\":\"api.example.com\",\"pool\":\"api\"},\n    {\"host\":\"www.example.com\",\"path_prefix\":\"/static\",\"pool\":\"assets\"},\n    {\"host\":\"www.example.com\",\"pool\":\"web\"}\n  ],\"default_pool\":\"web\"}'\n```\n\nOrder matters: the `/static` rule sits before the general `www` rule, because the\nfirst match wins.\n\n### Backend pools\n\nA pool is created from a single JSON object:\n\n```bash\nsudo cenvero-str-ctl l7lb pool add '{\"name\":\"web\",\"algorithm\":\"round-robin\",\"backends\":[\"10.20.0.50:8080\",\"10.20.0.51:8080\"]}'\n```\n\nList pools, and add or remove a pool's backends live:\n\n```bash\ncenvero-str-ctl l7lb pool list\ncenvero-str-ctl l7lb backend list web\nsudo cenvero-str-ctl l7lb backend add web 10.20.0.52:8080\nsudo cenvero-str-ctl l7lb backend remove web 10.20.0.50:8080\n```\n\nThe proxy runs active health checks (periodic HTTP probes with configurable healthy and unhealthy thresholds) and also ejects a backend passively when it returns transport errors while proxying, re-admitting it once it recovers.\n\n### TLS-terminating frontends\n\nA frontend binds a listener and routes to pools. This one terminates TLS with an operator-provided certificate and routes one host and path prefix to the `web` pool, falling back to it by default:\n\n```bash\nsudo cenvero-str-ctl l7lb frontend add '{\"name\":\"https\",\"listen\":\":8443\",\"tls\":{\"cert_file\":\"/etc/cenvero-str/tls/lb.crt\",\"key_file\":\"/etc/cenvero-str/tls/lb.key\"},\"default_pool\":\"web\",\"routes\":[{\"host\":\"api.example.com\",\"path_prefix\":\"/v1\",\"pool\":\"web\"}]}'\n```\n\nTLS termination accepts either an operator-supplied PEM certificate and key (as above) or the node's own managed certificate; the minimum TLS version is 1.2. A frontend with no `tls` block serves plain HTTP. The proxy sets the `X-Forwarded-For`, `X-Forwarded-Proto`, and `X-Forwarded-Host` headers on proxied requests and passes WebSocket and other upgrade connections through unchanged.\n\nList or remove frontends, and show the full picture — every frontend and pool with per-backend health and active connection counts:\n\n```bash\ncenvero-str-ctl l7lb frontend list\nsudo cenvero-str-ctl l7lb frontend remove https\ncenvero-str-ctl l7lb status\n```\n\nWhen a pool has no healthy backend the proxy returns `503`; when a chosen backend fails mid-request it returns `502` without leaking internal error detail.\n\n## See also\n\n- [Networking Overview](/docs/networking/overview) — where the load balancer sits in the data plane.\n- [Zero-Trust Firewall](/docs/networking/firewall) — VIP addresses also need firewall allow rules for external access.\n- [BGP Edge Routing](/docs/networking/bgp) — advertising VIP prefixes upstream.\n- [Quick Start](/docs/quickstart) — end-to-end example including an L4 VIP.\n"
        }
    ]
}