Tenants & Bandwidth
A tenant is a named owner you attach resources to — networks, addresses, API keys and a bandwidth allowance. If you run workloads for several customers, teams or environments on the same hardware, tenants are how you keep them apart and how you cap what each one can consume.
If everything on your nodes belongs to you, you can ignore tenants entirely. Nothing requires one.
What a tenant actually separates
A tenant is an ownership label plus five things that are genuinely enforced:
| What you get | What it does |
|---|---|
| Ownership of resources | Networks and addresses are recorded against the tenant, so you can see and delete everything one customer owns as a unit. |
| Separation from other tenants | Traffic between the private networks of two different tenants is refused in the packet path — no firewall rule needed. Virtual machines and containers that Stratum runs for a tenant are also separated on the shared segment itself, at their own port. |
| A scoped API key | A key that can act only for that tenant — it cannot read or change another tenant's resources. |
| A bandwidth cap | A real rate limit, applied in the traffic path. See below. |
| Caps on machines, addresses and volumes | A machine, an address or a volume past the tenant's limit is refused when it is created or attached. Addresses are capped on each node; machines and volumes are capped across the cluster once every member runs a version that supports it. See The tenant quota. |
The separation applies to networks created for a tenant — network create --tenant <id>. It is put back in place when the node restarts, and it cannot be routed around, because private addresses are not reachable from outside either. It rests on each network's addresses, so two networks on one node never share an address: a second tenant asking for a range another network already uses is refused, without being told whose it is.
Separation on the shared segment
Every tenant network on a node shares the same workload bridge. Refusing routed traffic between tenants is therefore not enough on its own: two workloads on one bridge can also talk to each other directly, frame to frame, without anything being routed. Stratum closes that path for the workloads it runs itself — its virtual machines and the containers you attach with container attach:
- Nothing one tenant's workload sends reaches another tenant's workload directly — not unicast, not broadcast, not multicast, whatever the protocol: IPv4, IPv6 (link-local addresses included), ARP, neighbour discovery, DHCP, or anything else carried in a frame.
- A tenant's workload cannot look up another tenant's addresses. An ARP request for an address in another tenant's network — including that network's gateway on the node — is dropped where it enters.
- The sender is identified by the port its traffic came in on, which a guest cannot fake. A workload that forges another tenant's hardware address is still recognised as itself.
- Same-tenant traffic, the node itself and the uplink are unaffected. Workloads of one tenant reach each other as before, the node reaches every workload (so DHCP, DNS and the gateway keep working), and traffic to and from the outside is handled exactly as before.
- It is in place before the workload first runs. A tenant's virtual machine or container is not started (or attached) on a port that cannot be separated; you get an error saying why. The node checks every 30 seconds that the separation is still in place and restores it, with a security event, if something removed it outside Stratum. It stays in force while the agent restarts, on every supported kernel. After an upgrade it is put back in place as the new version starts, so there is a brief moment during the upgrade without it.
- Only tenant workloads are held apart. A virtual machine or container without a tenant, and anything you plug into the workload bridge yourself, is not a tenant's: it is not separated at its port, and what it sends is not held back from tenant workloads either. Put every customer's workloads under a tenant.
Three things the separation deliberately does not cover:
- Two networks of the same tenant can reach each other. Whether they should is a firewall decision — set the default action to deny and write the allows you want. See Firewall.
- Networks created without a tenant are not separated from anything.
- Public addresses stay reachable, by design. Two tenants that each hold a public address can reach each other on it, exactly as anyone on the internet could — when both are on the same node, that traffic is simply turned around inside the node. See Gateway NAT.
A tenant's virtual machine can also hold one of the public addresses your provider routes to you (see Public Addresses). That address counts as the tenant's: it cannot be used to reach another tenant's private networks, it is covered by the tenant's bandwidth cap and suspension, and it counts towards the tenant's addresses. The cap and the suspension stay with the tenant: once the machine is deleted they no longer apply to the address, and the next machine to take it does not inherit them. Like any public address, it stays reachable from anywhere.
A workload sends only from its own addresses
A tenant's virtual machine or container may only send IPv4 traffic from the addresses Stratum gave it: the address it holds on its network, or its public address. A packet from any other address — another address of its network, another customer's, or one it made up — is dropped at its port, before it reaches another machine, the node or the internet. So is an ARP message announcing an address that is not the workload's own. This matters most for public addresses: providers act against servers that send traffic with forged source addresses.
What keeps working:
- Getting an address by DHCP. A DHCP request is sent before the machine has an address (from 0.0.0.0); it passes.
- Duplicate-address checks (ARP probes, which also carry no address).
- IPv6 link-local traffic, neighbour discovery included. IPv6 is not checked: the node does not give workloads IPv6 addresses yet.
- VLANs a guest builds on top of its own interface. Frames a workload tags with a VLAN number are not checked: the node routes none of them, and the separation above still keeps them away from other tenants' workloads (they reach the tenant's own machines, and anything you plugged into the workload bridge yourself). A frame whose only tag is a priority tag (VLAN 0) counts as untagged and is checked like one.
What no longer works: a second address configured by hand inside a guest, containers inside a virtual machine that are bridged onto its interface with addresses of their own, or an address that floats between two machines. Traffic from such an address is dropped. Give each machine one address and let containers inside it share that address.
As with the separation above, only tenant workloads are held to their addresses; a virtual machine or container without a tenant is not. The check is in place before the workload first runs and is restored within 30 seconds, with a security event, if something removed it outside Stratum.
Suspending a tenant
A suspended tenant's workloads send and receive nothing: not to the internet, not to other tenants, not to the node, and not to each other — whatever the number of machines, networks or addresses it has. Its virtual machines are also stopped and cannot be started until it is resumed; resuming lifts the cut-off and starts again the machines that were running. Suspension is usually driven by your billing system; see Billing Integration. Its customers' portal accounts keep signing in and reading the portal, but cannot act on machines (see Customer portal accounts).
Creating and deleting tenants
sudo cenvero-str-ctl tenant create --name acme
cenvero-str-ctl tenant list
Deleting a tenant is a cascade — it removes that tenant's API keys, quota, networks and address allocations together:
sudo cenvero-str-ctl tenant delete <tenant-id>
Tenants in a cluster
In a cluster, a tenant belongs to the whole cluster: create, rename, suspend, resume, limit or delete it on any member, and every member applies the change. Deleting it also revokes its API keys on every member, and is refused while any member still holds the tenant's machines, networks or volumes, or cannot be asked. A member that cannot reach the rest of the cluster refuses tenant changes with 503 and changes nothing, because the change has to reach the cluster as a whole: make it on a member that can reach the others, or restore the network between them (see A node cut off from the others). Reading tenants keeps working everywhere. Of a tenant's quotas, the bandwidth cap is shared, and so are its machine and volume limits once every member runs a version that supports it. Its address limit is set on each node, for that node, and a rule limit cannot be set at all (see The tenant quota).
A tenant's keys, and its portal, on every member. Once every member runs
this release, a tenant's API keys belong to the whole cluster: a key minted on
any member works on every member, and revoking it on any member stops it on all
of them within seconds. Minting and revoking are then changes to the cluster, refused
with 503 on a member that cannot reach the others, like the tenant changes
above. The customer can sign in to the portal
at any member's address and sees their machines, networks, addresses, tasks and
activity on every member; a machine on another member is started, stopped or
opened in a console through the member they signed in to, and they never need
to know which server holds what. If that server is not answering, they are told
so in plain words, and their machine keeps running. Until every member runs this
release, a key works only on the node that issued it. Customer portal accounts follow the same rule: they work on every member once every member runs this release, and until then the account routes, and a customer's sign-in, answer 409 (see Customer portal accounts).
Scoped API keys
A tenant key lets you hand out API access that is confined to one tenant, instead of sharing an operator key that can change everything. It is for your customer; for your own staff and scripts, use accounts and API tokens, which you can also confine to one tenant (a role on /tenants/<id>).
# Mint a key (optionally labelled, optionally time-limited)
sudo cenvero-str-ctl tenant key-generate <tenant-id> --name "acme-portal" --ttl 720h
cenvero-str-ctl tenant key-list <tenant-id>
sudo cenvero-str-ctl tenant key-revoke <key-id>
The key is shown once, when it is minted. Give it a TTL if it is going into a system you do not fully control — a key that expires on its own is one less thing to remember to revoke.
If the node cannot save a change to a key, the command fails and says so; it never reports success. A key that could not be saved when it was minted is not created. A key whose revocation could not be saved stops working at once, but could work again after the agent restarts: run key-revoke again once the cause in the agent's log is fixed. The same holds for apikeys revoke and for deleting a tenant.
What a tenant key can and cannot do. A tenant key is safe to hand to the customer it belongs to:
- It can read its own tenant, its quota (caps and usage) and its billing state, and list, mint and revoke its own tenant's keys. A key it mints belongs to the same tenant and never outlives the key that minted it.
- It can see its own machines, networks and public addresses; start, stop and restart its own machines, open their consoles and reset a password through the guest agent. It signs in to the web console's customer portal, which shows exactly this. While the tenant is suspended it can still look, but not act.
- It cannot change anything you control: creating or deleting machines, networks or volumes, its quota, its bandwidth limit, suspending or resuming it, renaming it, changing its status or deleting it. Those calls are refused (
403) and need your node token, an operator API key, or an account or API token whose role allows them. A suspended customer cannot resume itself, and a capped one cannot lift its cap. - It cannot see or touch another tenant, or anything else on the node. Asking for another customer's machine answers exactly as for one that does not exist, so a "not found" may simply mean the object is not theirs.
- It never sees your tags or notes. The tags and notes you keep on a tenant and on its machines, networks and other objects are yours: the customer portal never shows them, a tenant's own views never carry them, and a tenant key is refused (
403) on every tags-and-notes route. Write what you like about a customer there; it never reaches them.
Who on your side may manage a tenant's keys. Revoking one needs Tenant.Keys on the tenant. Minting one also needs everything the key lets its holder do there — see the tenant, its machines and networks, power the machines, open their consoles, use the guest agent — so an Operator or Administrator on the tenant can, a TenantManager cannot (see Roles).
Customer portal accounts
A customer signs in to the web console's customer portal with an account of their own, or with a tenant key. An account is a login for one customer. It sees that tenant's machines, networks, public addresses, tasks and usage, acts on the tenant's machines, and does nothing else. It holds no role, no group and no API token, and it cannot make or revoke tenant keys. A tenant key works as it did before.
Who may manage accounts. Reading them needs Tenant.Audit on the tenant. Making an account, changing it other than by disabling it, and resetting its password or second factor need Tenant.Keys on the tenant and every power a tenant key has there: Tenant.Audit, VM.Audit, VM.PowerMgmt, VM.Console, VM.GuestAgent and Network.Audit. Disabling an account and deleting one need Tenant.Keys alone, so the holder of that privilege can stop a compromised account at once. Administrators and Operators hold all of these. A TenantManager holds Tenant.Keys, so it can disable and delete an account. It cannot make one, enable one, or reset its password or second factor, because it lacks the powers a tenant key has there (see Roles).
Making one. In the console, open the tenant under Datacenter, then its Portal tab: Portal accounts lists the accounts, and Add an account makes one. Give the account a login, which is the customer's email address, and a display name if you want one. Set its password yourself, at least 12 characters and not a common one, or ask for a generated one. You can also give it an expiry date:
curl -k -X POST "$NODE/api/v1/access/tenants/t-9f3a21/portal-users" \
-H "Authorization: Bearer $TOKEN" -H "Content-Type: application/json" \
-d '{"login": "[email protected]", "display_name": "Acme owner", "generate": true, "expires_at": "2027-01-31T00:00:00Z"}'
A generated password is in the answer once, as temporary_password. The node keeps only a one-way check of it, so pass it on at once. Whether you set the password or it was generated, the customer must choose a new password at their first sign-in.
Every account has a second factor, and the portal asks for one. At the first sign-in the customer sets up an authenticator app before the portal opens, after the new password if one is due. The set-up screen shows a QR code to scan, and the key to type in. They are then given ten recovery codes. Each works once, and they are shown once, so they must save them then. A recovery code works in place of a code from the app, at sign-in and for a change that asks for a code, and using it spends it. A customer can make new recovery codes at any time from Your account, with the current password and a code from the app; the old codes stop working, and the new ones are shown once.
Limits. A tenant holds at most 50 accounts. A cluster holds at most 5,000, and a single server counts as its own cluster. A create that would go past a limit is refused with one of these sentences:
the cap of customer accounts for this tenant has been reached; remove an account firstthe cap of customer accounts on this server has been reached
A login that another account already has is refused with a customer account with that email address already exists.
Changing an account. Every change names the account's revision, if_rev, as you last read it. A change made on an older copy is refused with this account changed since you loaded it; reload and try again.
- Disable (
"enabled": false). The account can no longer sign in, and its open sessions end at their next request. A customer who types the right password is toldthis account can no longer sign in; contact your provider. A wrong password gets the same refusal as any wrong password. - Enable, or change the expiry (
expires_at, an RFC 3339 time in UTC;nullremoves it). An account past its expiry cannot sign in, the same as a disabled one. - Reset the password. The customer's sessions end, and they must choose a new password at their next sign-in.
- Reset the second factor. The customer's sessions end, and they set up an authenticator app again at their next sign-in.
- Delete. The account goes with its second factor and its sessions. Deleting the tenant deletes its accounts too. If a removal is refused, the node finishes it in a later sweep, which runs soon after the refusal and otherwise every five minutes.
What a customer cannot do. A customer who asks for something only the provider can do, such as making a key or changing the tenant's limits, is answered this is not part of your portal. Another customer's machines answer as if they did not exist.
Suspension and closure. A suspended tenant's accounts keep signing in and reading the portal, but cannot act on machines (see Suspending a tenant). For a closed tenant, a correct password is answered this account is closed, and the sessions end.
In a cluster. Accounts work across the cluster once every member runs this release. Until then every account route answers 409, and a customer's sign-in does the same, with the customer portal needs every server in this cluster to be updated first; try again later, or contact your provider. A member on an older release is left out of a customer's lists, which then say the answer is partial, and a machine held there cannot be reached from another member.
The portal name
Each tenant can have one portal name, the host name its customers reach the portal at, such as portal.example.net. The node reads the name from the host of each request. A request for a portal name this node holds is answered for that tenant's customers only (see What staff cannot do under a portal name). In the console, the name is set on the tenant's Portal tab, under Portal name.
curl -k -X PUT "$NODE/api/v1/tenant/t-9f3a21/portal-host" \
-H "Authorization: Bearer $TOKEN" -H "Content-Type: application/json" \
-d '{"host": "portal.example.net"}'
- The rules. A name has at least two parts separated by dots. Each part is letters, digits and hyphens, at most 63 characters, and it neither starts nor ends with a hyphen. The whole name is at most 253 characters. The last part is not all digits. Addresses,
localhost, and names ending in.localor.localhostare refused, and so are wildcard names. Upper case is accepted and kept in lower case, and one trailing dot is dropped. - One tenant per name. A name another tenant holds on this node is refused with
that address already belongs to another customer on this server. A name that breaks the rules is refused withenter a host name made of letters, digits and hyphens, with dots between the parts; wildcard names and addresses are not accepted. - Not the server's own name. A name staff reach the server by is refused with
that name is one of this server's own names, and a portal there would keep staff from reaching the server by it; choose a name only customers use: the server's host name, a name its certificate carries, and on a cluster a member's name or the host of its API address. A name set to one of these before this rule existed is not treated as a portal name, so staff can still reach the server there and clear it. - Per node. Each node keeps its own names, and they are not shared across a cluster. Set the name on every node your front sends the tenant's customers to.
- Routing only. A name never appears on the node's certificate. No certificate is issued for it, and no approval is asked. A customer's browser trusts the portal only through the front described below.
- Removing the tenant clears its name.
- Setting or clearing a name is a change to the tenant. A frozen license refuses it, but reading the name is never refused. Setting or clearing needs
Tenant.Modify, and reading needsTenant.Audit.
Putting the portal behind your own address
Give customers an address in your own name, served by an HTTPS front that you run. The front holds a certificate for that name which the customer's browser trusts. It forwards only the portal's paths to the node's API port (7070 by default), and it checks the node's own certificate against your account root. The node's certificate is never shown to the customer.
- Set the portal name on each node that holds the tenant's machines (see The portal name).
- Point the name's DNS record at the front.
- Give the front a publicly trusted certificate for the name. Let it obtain one (ACME), or install your own.
- Run the front with one of the examples below. Put in the node's name, the one on its certificate, and the account root. Download the root from Certificates › Trust your certificate authority (see Trusting your account root).
- List the front's address in
portal_trusted_frontson each node (see Listing the front).
nginx
server {
listen 443 ssl;
server_name portal.example.net;
# Your publicly trusted certificate for the portal name.
ssl_certificate /etc/nginx/tls/portal.example.net.crt;
ssl_certificate_key /etc/nginx/tls/portal.example.net.key;
# The node's API port, and its certificate, checked against the account root.
# proxy_ssl_name is the name on the node's certificate.
proxy_ssl_server_name on;
proxy_ssl_name node1.example.net;
proxy_ssl_verify on;
proxy_ssl_trusted_certificate /etc/nginx/tls/stratum-account-root.pem;
location = / {
proxy_pass https://192.0.2.10:7070;
proxy_set_header Host $host;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
}
location = /console {
proxy_pass https://192.0.2.10:7070;
proxy_set_header Host $host;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
}
location /console/ {
proxy_pass https://192.0.2.10:7070;
proxy_set_header Host $host;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
}
# The console's machine sockets (VNC and serial) upgrade to WebSocket.
location ~ ^/api/v1/vms/[^/]+/console/(vnc|serial)$ {
proxy_pass https://192.0.2.10:7070;
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "upgrade";
proxy_set_header Host $host;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_read_timeout 1h;
}
# Everything else is not the portal.
location / {
return 404;
}
}
Caddy (2.6 or later)
(stratum_node) {
reverse_proxy https://192.0.2.10:7070 {
header_up Host {host}
transport http {
tls_server_name node1.example.net
tls_trusted_ca_certs /etc/caddy/tls/stratum-account-root.pem
}
}
}
# Caddy obtains a publicly trusted certificate for the name itself.
portal.example.net {
@portal path / /console /console/*
@console_socket path_regexp ^/api/v1/vms/[^/]+/console/(vnc|serial)$
handle @portal {
import stratum_node
}
handle @console_socket {
import stratum_node
}
handle {
respond 404
}
}
What the examples do:
- They forward only the portal's paths.
/,/consoleand everything under/console/go to the node, and so do the two console sockets, for VNC and for serial. Anything else is answered404at the front. The node refuses the same paths under a portal name, so the front's list is a second line, not the only one. - They keep the
Hostheader. The node takes the portal from the name the customer typed, so the front must pass that name on unchanged. The customer's browser session and its origin checks depend on it too. - They verify the node's certificate. The front checks the node's certificate against the account root, for the name you give. Leave verification on. A front that does not check accepts any server that answers.
- They append the customer's address to
X-Forwarded-For. nginx's$proxy_add_x_forwarded_fordoes that, and Caddy does it by default. The node reads only the last entry, and only from a listed front, so a front must append to the header, never pass a customer's own header through. - They upgrade the console sockets. nginx needs the
UpgradeandConnectionheaders, which the example sets, and it has a read timeout long enough for a console. Caddy passes the WebSocket headers without any setting. - If you offer customers API access with a tenant key, also forward
/api/v1/tenant/<tenant-id>/, with the same lines as the portal's locations. Leave it out otherwise.
Listing the front
Without a listed front, the node sees every customer as coming from the front. Its limits on sign-in attempts and its shared lock-out then count all customers together, so one customer's failed sign-ins can lock out the others. List the front's address, and the node reads each customer's own address from the front's X-Forwarded-For entry:
sudo cenvero-str-ctl config set portal_trusted_fronts 203.0.113.10
- List the address the front connects from. Several go in one comma-separated value, up to 64 in all:
203.0.113.10,198.51.100.0/24. Each entry is an IP address or a CIDR range, and a range may be no broader than a/24for IPv4 or a/64for IPv6. Any other value is refused witheach trusted front must be an IP address or a range in CIDR form, and there may be at most 64. A range broader than that is refused witheach trusted front must be a single address or a narrow range: a range broader than /24 for IPv4 or /64 for IPv6 is refused. - Only a request that comes from a listed front is read this way. Any other request is judged by its own address.
- A listed front that sends no usable address, because the last entry is missing or is not an IP address, is judged by its own address. A broken front is therefore limited like any other.
- The list is a setting of this node. The configuration your account delivers to the node does not include it, so set it on each node.
- It takes effect at the next start:
sudo systemctl restart cenvero-stratum. - To clear it, set an empty value:
sudo cenvero-str-ctl config set portal_trusted_fronts "".
Direct access
If the portal name's record points straight at the node, the customer reaches the node itself, and the browser shows the node's own certificate. A browser does not trust that certificate until the account root is installed on the customer's machine, so give customers the address of a front whose certificate their browser trusts.
What staff cannot do under a portal name
Under a portal name the node serves the tenant's customers and nobody else:
- Its paths. It answers the console, the portal's own routes and the console's machine sockets. Every other path gets the same answer as a path that does not exist, before any sign-in is looked at. That includes the operator's API, the cluster's routes and
/metrics. A path in an unusual form, with a double slash, a.or..segment, or percent-encoding, gets the same answer. Do your own work on the node's own address, not on a portal name. - Operator credentials. An operator's key, token or console session is refused on every route that takes it, with
403and the sentenceoperator accounts cannot sign in here; use the server's own address. - People. A person's sign-in, by password, second factor or identity provider, is refused with that same sentence.
- Other customers. A key or an account of another tenant is refused with
this address belongs to another customer; sign in at your own address. - Machine consoles. A console socket opened with an operator's token, or with an operator's ticket, is refused with the same operator sentence.
- The sign-in page offers only the customer portal.
Usage figures
The usage figures show the network traffic a tenant's machines carried. Download is what the machines received and upload is what they sent, counted on their network interfaces as the host counts them. The server's own note on the figures says what they are: the usage figures are the network traffic each machine's interfaces carried, as the server counted it; they are not a bill. They are not the monthly usage quota below, which counts hardware addresses.
- Only running machines add to them. A machine's figures are sampled every 15 seconds while it runs. A stopped machine adds nothing, and a time when none of a tenant's machines runs is a gap. A gap shows as no figure, never as zero.
- The totals are bytes. A window's totals are the bytes its machines carried, whatever the graph shows: a peak view draws the peaks, and its totals are the same as the average view's.
- A restart loses a little. A machine's counters start again at zero when it restarts, and the node starts counting afresh when it restarts. The first reading after either adds nothing, so about one reading's worth of traffic is not counted.
- A deleted machine keeps its traffic. It stays in its tenant's totals. Its graphs are no longer shown, because the machine is gone. The node keeps its history on disk for seven days after the deletion, without showing it, and then removes it.
- The windows. The Usage page covers the last hour, day, week, month or year, and Live, which shows the newest samples as they are taken and refreshes every ten seconds. The node keeps minute figures for a day, 10-minute figures for a week, hourly figures for a month and figures every 12 hours for a year, so a year's view reaches back 366 days.
- A tenant's history is removed after it has gone 400 days without a figure.
- In a cluster, the figures of every member for the tenant are added together. A member that cannot answer is left out, and the answer says so.
The tenant quota
A tenant's quota sets what the tenant may use. Each limit counts per node: a
node counts and caps only what the tenant holds on it, except the machine and
volume limits, which count across the cluster once every member runs a version that supports it. 0 means unlimited.
| Limit | What it caps | When it is enforced |
|---|---|---|
Bandwidth cap (--max-bandwidth-bps) | The rate the tenant's traffic may use, in bits per second. It is shared across a cluster. | On the way out of the node, in the traffic path. |
Machines (--max-vms) | The virtual machines the tenant may have: on the node, or across the cluster, once every member runs a version that supports it. | When a machine is created. Lowering it never stops a machine that already exists. |
Addresses (--max-ips) | The addresses the tenant may hold on the node: its private network addresses and its public addresses. | When an address is attached to a machine or a container, or a public address is given to a machine. Lowering it never releases an address the tenant already holds. |
Volumes (--max-volumes, --max-volume-gib) | The number of volumes, across the cluster, once every member runs a version that supports it, and their total size on each node. | When a volume is created, restored or grown. |
Across a cluster. Once every member runs a version that supports it, a machine or a volume counts against the
tenant's limit wherever it runs. A create or a volume that would pass the limit is
refused, even when two members take one at the same moment: the slot is taken
through the cluster's leader, so only one of them can have the last one. Until every member runs a version that supports it, each member applies a limit on its own. cenvero-str-ctl cluster
usage <tenant> shows the machines and volumes the tenant has across the members.
sudo cenvero-str-ctl tenant quota-set <tenant-id> --max-bandwidth-bps 1000000000
sudo cenvero-str-ctl tenant quota-set <tenant-id> --max-ips 20
cenvero-str-ctl tenant quota <tenant-id>
quota-set changes only what you give it; every other quota field keeps its current value. It refuses a call with nothing to change, a negative value, and a tenant that does not exist. The API behaves the same way (PUT /api/v1/tenant/{id}/quota).
A rule limit cannot be set. A tenant owns no firewall rules, so there is nothing to count a rule limit against, and no limit on rules is enforced. Setting one to any value other than 0 is refused with "the rule limit cannot be enforced: a tenant owns no firewall rules, so there is nothing to count it against; set it to 0 (unlimited)". A rule limit stored before this was refused is kept as it was, is listed under not_enforced in the quota, and changes nothing. Set it to 0 to clear it.
The quota also reports what the tenant uses now on the node: its addresses (used_ips), its machines (used_vms) and its volumes. Each figure is read from the node when you ask, so it cannot fall behind what is really attached. Traffic is reported separately, in the usage figures above.
Your plan's speed ceiling
Above every limit you set yourself sits one you do not: your license carries the maximum speed the node may use, and it is applied to the node's uplink. The ceiling for each plan is listed on the pricing page; your own is whatever your license carries, which you can read with cenvero-str-ctl license status.
Five things worth knowing:
- It is per node, not per account. Each licensed node has its own ceiling, matching how the plans are priced.
- It is the node's total, shared across all its network cards. A licensed node behaves as though its cards have exactly that much capacity between them: a 2 Gbps license on a single 10 Gbps card passes 2 Gbps, and two cards share the same 2 Gbps — you can distribute traffic across them however you like, but the total is still 2 Gbps. Adding a card does not give you more headroom. (Your management interface is deliberately left alone — throttling the control plane would take the node out of its cluster.)
- Speeds are bits per second, the way link speed and the plans are quoted — not bytes.
- It applies on top of anything you configure. A tenant limit below the ceiling gives that tenant its limit; a tenant limit above it does not lift the node past its ceiling.
- Changing plan takes effect without a restart. The node re-reads its license periodically, so an upgrade raises the ceiling within about a minute of the new license landing.
Going over the ceiling does not fail — it throttles. Traffic above the rate is held to it, exactly as described for bandwidth limits below, so a transfer slows rather than breaking.
The ceiling behaves like any other cap here — a credit allowance refilling at the licensed rate, with burst headroom so ordinary traffic is not penalised. See what that means for TCP and UDP below.
Bandwidth limits
Bandwidth is the one resource control that is real, and it is applied on the way out of a node — egress shaping. A limit is a rate in bits per second, and 0 always means unlimited.
There are two ways to apply one, and they answer different questions.
Per tenant — "this customer gets 1 Gbps in total":
sudo cenvero-str-ctl tenant quota-set <tenant-id> --max-bandwidth-bps 1000000000
Per target — "this particular workload gets 100 Mbps", regardless of who owns it:
cenvero-str-ctl bandwidth list
sudo cenvero-str-ctl bandwidth delete <id>
A per-target limit is submitted as a JSON object describing what to limit and at what rate — see the API Reference for the field names, and CLI Reference for the command form. A burst left out is one second of the rate. Several limits on one hardware address all hold: the lowest rate in each direction applies, and removing one leaves the others. A limit is a ceiling only: no share of bandwidth can be guaranteed (guaranteed_bps is refused).
Pools
A pool is a shared allowance that several targets draw from together, rather than each holding its own independent cap. Use it when a group should be limited collectively — "these five workloads share 500 Mbps between them" — instead of five separate 100 Mbps limits that cannot lend capacity to one another.
Pools are created and filled over the API (POST /api/v1/bandwidth/pools, then POST /api/v1/bandwidth/pools/members per hardware address — see the API Reference); cenvero-str-ctl bandwidth list shows them with your limits. A few rules worth knowing:
- Adding a member that is already in the pool replaces its rate rather than adding a second allocation.
- A member whose rate would take the pool past its total is refused.
- A member's own rate caps it, with one second of that rate as its burst, and an address is in one pool at a time.
- Removing a member takes it off the pool's rate; any limit of its own still holds. Removing an address that is not a member is refused, and an unknown pool answers "not found".
What a limit does to traffic
This is the part to understand before you set one, because the behaviour is not what "shaping" usually implies.
A limit is a credit allowance that refills at the rate you set. Every packet spends credit equal to its size. While there is credit, traffic passes untouched. When the credit runs out, traffic over the rate is dropped — not queued and delivered late.
That distinction matters:
- TCP copes well. Loss is exactly the signal TCP uses to slow down, so a TCP transfer finds the rate and settles there. This is the normal case and it works.
- UDP does not. Nothing tells a UDP sender to slow down, so anything over the rate is simply lost. Video, voice, game traffic and metrics streams degrade rather than slow down. Size the limit for the peak these need, not the average.
Unused credit accumulates up to a ceiling, so a burst after a quiet period passes at full speed and only sustained traffic is held to the rate. A short spike is not punished.
Two more consequences worth planning around:
- The cap is enforced per node. Each node shapes the tenant's workloads that live on it to the cap — it is not a single allowance summed across your nodes. On independent nodes, set it on every node where the tenant has workloads; it applies to that node only. In a cluster the cap is set once, on any member, and every member applies it to the workloads it holds — still per node, not summed.
- It applies on the way out. Limiting what a workload can receive means limiting whoever is sending to it, which you only control if the sender is also yours.
Monthly usage quotas
Separately from rate limiting, a node can track how much a workload transfers per calendar month and act when it crosses a threshold. This is a volume cap, not a speed cap — the two are independent, and you can use either or both.
cenvero-str-ctl quota list
cenvero-str-ctl quota get <mac>
A quota counts its hardware address's traffic whether or not the address has a bandwidth limit, and its usage is brought up to date every few minutes. Counters reset at the start of each month, in UTC. A quota only alerts unless it is enforced: an enforced quota, whose action must then be throttle, holds the address to almost nothing until the month resets or the quota is raised, and its own limits apply again afterwards. Use rate limits to control how fast a tenant can go, and monthly quotas to control how much they can move in total.
See also
- Firewall — the policy that actually separates tenants from each other.
- Networking Overview — networks, endpoints, and how a workload joins one.
- Monitoring & Observability — reading real throughput per workload.
- API Reference — tenant, quota and bandwidth endpoints.