BGP Edge Routing
Every node runs an integrated BGP speaker that connects your fabric to the outside world. You configure peer sessions, announce your managed-network prefixes upstream, optionally import external routes, and filter routes with prefix-lists and route-maps — all through cenvero-str-ctl.
How it fits together
A node at the edge of the fabric speaks BGP upstream. It speaks eBGP to one or more upstream routers or route reflectors, announcing the subnets of your internal networks so external hosts can reach them. Inbound prefixes learned from peers are installed into the kernel routing table and become reachable from within the fabric, except a default route and any route over the server's own network, gateway or addresses (see Checking the routing tables).
What travels in each direction
Two separate flows, and confusing them is the usual cause of "the session is up but nothing works":
Out — what you announce. Nothing leaves automatically. A prefix is advertised only once you announce it, so bringing a session up publishes nothing on its own. Until you announce, your peer has no idea your networks exist and traffic will not come to you.
In — what you learn. Prefixes your peers advertise are accepted, subject to any filters you set, and installed as routes so workloads inside the fabric can reach them. A peer that sends you nothing leaves you with a working session and no new routes, which is normal if it is only meant to receive.
A session being Established means the two speakers are talking. It does not mean any route has been exchanged in either direction — check what is actually being carried rather than inferring it from the session state.
What a restart keeps
Everything you configure on this page is kept by the node: peers, announced prefixes, prefix-lists, route-maps and the policies that bind them, route-origin validation and EVPN. When the agent restarts — for an update, or after a reboot — the node has all of it back before it contacts a peer, so your sessions come up again on their own and your prefixes are announced again, filtered as before. A peer you removed or a prefix you withdrew stays gone, and a change the node cannot keep is refused with an error and changes nothing. The sessions themselves do restart: while the agent is down your peers see them close, and routes they learned from this node are withdrawn or held briefly, depending on the peer, until the node is back.
How a session comes up
A peer moves through a sequence of states, and knowing them makes the difference between waiting and debugging:
| State | Meaning |
|---|---|
| Idle | Not trying yet, or backing off after a failure before the next attempt. |
| Connect / Active | Trying to reach the peer. Sitting here means the peer is not answering — check reachability and that it is expecting you. |
| OpenSent / OpenConfirm | Connected; the two sides are agreeing terms. Sitting here usually means a mismatch, most often the AS number each side expects of the other. |
| Established | Working. Routes can now be exchanged. |
Both ends may try to connect to each other at the same time — normal when two Stratum nodes peer directly. Two connections briefly exist and one is discarded automatically, which is expected and needs no configuration.
Once established, the two exchange keepalives. If nothing is heard for the hold
time (90 seconds by default) the session is considered dead and torn down, and
the routes learned through it are withdrawn. The session uses the lower of the
two ends' hold times. This node sends a keepalive every keepalive_interval
seconds, at most a third of the hold time; left out, it is a third of the hold
time (every 30 seconds with the default), and it is sent more often when the
peer agrees a shorter hold time. Lower the hold time if you need failure
detected faster, at the cost of more chatter.
BFD is not available: a lost neighbour is noticed when its hold time runs out.
Peers that keep returning to Idle are usually being rejected rather than failing to connect. Check the AS numbers on both sides first; that is the most common cause.
For redundancy with two nodes, both can establish their own peer sessions and announce the same prefixes, so your upstream keeps a path while either is up. A shared VIP that moves between a pair of nodes is not available to set up yourself yet — see Gateway High Availability.
Configuring a peer session
sudo cenvero-str-ctl bgp peer add '{"peer_addr":"198.51.100.1","peer_as":65001,"local_as":65100,"md5_key":"mysecretpass"}'
The neighbour is a single JSON object. peer_addr, peer_as, and local_as are required; the rest are optional:
| Field | Description |
|---|---|
peer_addr | Upstream router's IP address |
peer_as | Upstream router's AS number |
local_as | This node's AS number |
md5_key | TCP MD5 key that signs the session (optional but recommended; at most 80 bytes; never returned in reads) |
hold_time | Hold time in seconds, 3 to 65535 (default 90) |
keepalive_interval | Keepalive interval in seconds, 1 up to a third of the hold time (default: a third of the hold time) |
A neighbour's TCP MD5 key signs every segment of its session, on the connection this node opens and on the one it accepts. An unsigned or wrongly signed connection is not let in, a neighbour whose key cannot be put in place gets no session, and the key is never shown again. On a system without TCP MD5 protection, a key is refused. Adding or removing a neighbour with a key briefly reopens the node's listening socket: a peer connecting at that moment connects again, and established sessions are untouched.
BFD settings (bfd_enabled, bfd_interval_ms, bfd_multiplier) are refused. A
neighbour saved with them before keeps its session and lists them under
not_enforced.
List configured peers and their session state:
cenvero-str-ctl bgp peer list
PEER PEER-ASN LOCAL-ASN STATE PREFIXES-RX PREFIXES-TX
198.51.100.1 65001 65100 Established 12 3
198.51.100.2 65002 65100 Established 8 3
Remove a peer session:
sudo cenvero-str-ctl bgp peer remove 198.51.100.1
Removing a neighbour also removes its import and export policies.
Announcing prefixes
Tell Stratum which internal prefixes to advertise to peers:
sudo cenvero-str-ctl bgp announce 10.20.0.0/24
sudo cenvero-str-ctl bgp announce 10.30.0.0/24
Announcements take effect as soon as they are added, and reach peers whose sessions are already up at once. Sessions carry IPv4 unicast routes (and EVPN routes when it is on), so an IPv6 prefix is refused. An optional second argument gives the next hop; without one, each peer is sent this node's own address on that session. You can also announce a summary prefix that covers multiple networks:
sudo cenvero-str-ctl bgp announce 10.20.0.0/20
List what you are announcing:
cenvero-str-ctl bgp announcements
Withdraw a prefix (peers are notified immediately):
sudo cenvero-str-ctl bgp withdraw 10.20.0.0/24
Importing external prefixes
By default, prefixes learned from peers are installed in the kernel routing table and are reachable from within the fabric, without ever replacing a route the server has. You can filter what gets imported.
Filtering is expressed as prefix-lists and route-maps, then bound to a neighbour as an import (or export) policy — each is a single JSON object. First define a prefix-list of the CIDRs you care about (entries are matched in order; each has an action and optional ge/le length bounds):
sudo cenvero-str-ctl bgp prefix-list add '{"name":"customer-routes","entries":[{"prefix":"172.16.0.0/12","action":"allow","le":24},{"prefix":"0.0.0.0/0","action":"deny"}]}'
An action is allow (or permit) or deny (or block), in any letter case; an entry with no action is an allow. Any other value is refused — so a typo can never turn a deny entry into a permit.
Reference that prefix-list from a route-map, which decides what to do with a matching route and can set attributes such as local-preference:
sudo cenvero-str-ctl bgp route-map add '{"name":"import-filter","entries":[{"seq":10,"action":"allow","match_prefix":"customer-routes","set_local_pref":200}]}'
Bind the route-map to a neighbour as its import policy (use export for the outbound direction):
sudo cenvero-str-ctl bgp policy import 198.51.100.1 import-filter
A change to a prefix-list, a route-map or a policy applies to the routes the node already holds, not only to the next ones a peer sends, and to what established peers are sent.
Over the API, GET /api/v1/bgp/routes takes ?family=ipv4 (the default) or ?family=ipv6; any other value is refused with 400 rather than quietly answered with the IPv4 table. An announcement over the API (POST /api/v1/bgp/announce) can also carry communities, each asn:value (both 0–65535) or one of the well-known names no-export, no-advertise, no-export-subconfed and no-peer; they are sent to peers. See API Reference.
Checking the routing tables
Inspect the BGP routing information base:
cenvero-str-ctl bgp rib
PREFIX NEXT-HOP AS-PATH MED LOCAL-PREF SOURCE
10.20.0.0/24 0.0.0.0 - - - local
0.0.0.0/0 198.51.100.1 65001 0 100 ebgp
172.16.0.0/12 198.51.100.1 65001 0 100 ebgp
Routes learned over BGP never replace or override a route the server has. They are installed with protocol bgp at metric 4200000000, so at an equal prefix the server's own route always wins. A learned default route is never installed, and one over the server's own network, gateway or addresses is not installed, nor one over another cluster member's address that leaves another way than the server reaches it now. A withdrawal removes only the learned route.
Show the routes the node has installed in its own routing table. A route it wanted to install and could not is listed under not_installed, with the reason:
cenvero-str-ctl bgp fib
Over the API, GET /api/v1/bgp/rib answers one of the three tables: ?table=in (what each peer sent that the node accepted), local (the routes it selected, its own included; the default) or out (what it announces; with ?neighbor=<address>, what that peer's session has actually been sent). ?prefix= narrows it to the routes equal to or inside a prefix, or to those covering an address. It answers at most 1,000 routes, and truncated says there were more.
RPKI route-origin validation
Route-Origin Validation (ROV, RFC 6811) checks each prefix a peer advertises against Resource Public Key Infrastructure (RPKI) data, so you can tell whether the AS that originated a route is actually authorized to announce it. Stratum learns the set of validated prefix-to-origin records (the VRP set) from an external RPKI-to-Router (RTR, RFC 8210) cache — such as Routinator, rpki-client, or StayRTR — and classifies every received route as:
- valid — a matching record authorizes this prefix from this origin AS.
- invalid — a record covers the prefix but the origin AS or prefix length does not match, a likely mis-origination or hijack.
- not-found — no record covers the prefix.
ROV is off by default and fail-safe. With no RTR configured, a cache that has not answered, or data that has expired, the VRP set is empty, every route resolves to not-found, and BGP behaves exactly as it would without ROV — an RTR outage can never cause a route to be rejected. If the cache stops answering, the data it last sent is used until it expires. By default ROV only marks each route with its validation state and does not change route selection; the opt-in reject-invalid policy leaves invalid routes out.
Every route the node holds is checked again whenever the cache's data changes or expires, or validation or its policy changes. Under reject-invalid, a route that becomes invalid is withdrawn, from the node's routing table too, and one that stops being invalid comes back — so routes learned before the cache's first answer are judged as well.
Point Stratum at your RTR cache and start validating:
sudo cenvero-str-ctl bgp rpki configure 192.0.2.10:323 --reject-invalid
| Argument / flag | Description |
|---|---|
<rtr-addr> | Address (host:port) of the RTR cache to connect to |
--reject-invalid | Drop RPKI-invalid routes instead of only marking them (optional) |
Check the RTR session state (inactive, connecting, connected or error), the number of records in effect, and the routes held now, counted by their state:
cenvero-str-ctl bgp rpki status
Test how a given prefix and origin AS would classify against the current VRP set:
cenvero-str-ctl bgp rpki validate 192.0.2.0/24 64500
Start or stop ROV using the configured RTR address; disabling clears the VRP set and reverts to plain BGP:
sudo cenvero-str-ctl bgp rpki enable
sudo cenvero-str-ctl bgp rpki disable
EVPN address family
Stratum's BGP speaker can carry the EVPN (Ethernet VPN, RFC 7432) address family — L2VPN/EVPN over MP-BGP — integrated with the VXLAN overlay (EVPN-VXLAN, RFC 8365). It exchanges two route types with EVPN-capable peers:
- Type 2 (MAC/IP) — advertises the MAC (and optional IP) of a local overlay endpoint, so peers learn which VTEP a MAC lives behind.
- Type 3 (Inclusive Multicast) — advertises the local VTEP so peers know where to send broadcast, unknown-unicast, and multicast traffic for a VNI.
Each route carries a Route Distinguisher and Route Target, and the 24-bit VXLAN VNI is encoded in the MPLS label field.
Scope — control plane only. EVPN in Stratum is a route-exchange feature: it learns remote MAC-to-VTEP and VTEP bindings over BGP and maintains the EVPN route tables, but it does not program those learned routes into the overlay forwarding path. Use it to exchange and inspect MAC/VTEP reachability, and configure VXLAN peers explicitly for forwarding (see Networking Overview).
EVPN is off by default and backward-compatible: while disabled, the EVPN address family is not advertised and BGP behaves exactly like the plain IPv4-unicast speaker.
Turning EVPN on or off reaches a peering that is already up only once that peering is established again. Over the API, GET /api/v1/bgp/evpn reports "forwarding": false: learned routes are not used to forward traffic.
Enable EVPN, advertising a local VTEP and route distinguisher:
sudo cenvero-str-ctl bgp evpn enable --vtep 10.0.0.1 --rd 65000:100
| Flag | Description |
|---|---|
--vtep | Local VTEP source IP advertised as the EVPN next hop |
--rd | Route distinguisher, e.g. 65000:100 or 10.0.0.1:100 |
Register a local MAC (and optional IP) under a VNI so it is advertised as a Type 2 route:
sudo cenvero-str-ctl bgp evpn local-mac add 10100 aa:bb:cc:dd:ee:ff 192.0.2.10
Inspect EVPN state and the local plus learned routes:
cenvero-str-ctl bgp evpn status
cenvero-str-ctl bgp evpn show
Disable EVPN — it stops advertising the address family and withdraws the routes it announced:
sudo cenvero-str-ctl bgp evpn disable
See also
- Gateway High Availability — a redundant pair of nodes and VIP failover.
- Load Balancer — combine BGP with L4 VIPs for published external services.
- Firewall — north-south traffic from BGP peers passes through the firewall.