Exclusive Access · Invitation Only

Storage Pools

A pool is a place on the node where volumes and backups are kept. Every node has a default pool, which holds the machines' disks and every volume and backup made before pools existed. You can add more: a directory pool on the node's own disk, or a block pool on storage the host already has.

Part of Compute, in early access. Changing pools needs a licence that includes Storage (a Compute licence includes it) and is not frozen. Reading them always works.

The kinds of pool

KindWhere its volumes are keptWhat the host needs first
DefaultThe node's storage area, which every node hasNothing
Directory (dir)A directory the node makes inside its storage areaNothing. To add space, mount a disk on the directory
Block, thin-provisioned (thin-provisioned)Thin volumes in a thin pool of an LVM volume groupThe thin pool, with the LVM tools
Block, ZFS (zfs)Volumes in a ZFS pool, or in a dataset of oneThe ZFS pool or dataset, with the ZFS tools

Directory pools

storage pool create --name fast makes a directory pool called fast. The name is 1 to 63 lowercase letters, digits and hyphens, and it cannot start or end with a hyphen. --location names the directory when it should differ from the name; it is one name, with no slashes.

The directory must be missing or empty when the pool is made, so a pool never takes over files it did not make. To add space to a directory pool, mount a filesystem on its directory.

A directory pool on a shared filesystem, such as an NFS export, can be declared shared between the members of a cluster. See Shared Storage Pools.

Block pools

A block pool keeps its volumes on storage the host already has. The node asks the host's own tools for each volume. You set that storage up first, and the node checks it before it records the pool. The node never creates it.

What to set up on the host

  • Thin-provisioned. A thin pool inside an LVM volume group. The pool must be active and healthy, with room left in it. The LVM tools must be installed (the lvm2 package on Debian and Ubuntu). Its location is the group and the pool, written group/pool.
# A thin pool called pool, in the volume group vg0, that takes all of the group's free space
sudo lvcreate --type thin-pool --extents 100%FREE --name pool vg0
  • ZFS. A ZFS pool, or a dataset in one. The pool must be healthy and have free space. The ZFS tools must be installed (zfsutils-linux on Debian and Ubuntu; on Debian the ZFS module comes from the contrib component, which must be enabled). Its location is the pool's name, or the pool and the dataset written pool/dataset.
# A dataset called vm in the existing ZFS pool tank, for the node's volumes
sudo zfs create -p tank/vm

Creating a block pool

cenvero-str-ctl storage pool create --name thin-pool --backend thin-provisioned --location vg0/pool
cenvero-str-ctl storage pool create --name zfs-pool --backend zfs --location tank/vm

Over the API, POST /api/v1/storage/pools takes the same three fields, and the API page gives the answers. The node refuses a location it cannot use, and says why:

  • A location in the wrong form: "a thin-provisioned pool's location is its group and pool name, written as group/pool; a ZFS pool's location is its pool name, or pool/dataset".
  • A location the host does not have: "the storage location of this pool does not exist on this node: check its name, or create it first".
  • A location that is not healthy: "the storage location of this pool is not healthy on this node, so it cannot be used: check it, then try again".

Which kinds this node offers

A block pool is in service only while the node has the tools for its kind. storage pool list shows each kind and whether this node can use it now, under backends. A tool installed later is used at once, without a restart.

On a node without the tools, creating a pool of that kind answers "this kind of pool is not available on this node" (501 over the API). A block pool recorded earlier, whose tools have since gone, shows as unreadable, and changes to it are refused until the tools are back.

What works on a block pool

  • Volumes. Create one with volume create --pool, attach and detach it, and grow it with volume resize. A volume cannot shrink.
  • Snapshots. Take and delete snapshots as on any volume. Going back to a snapshot discards what was written since, and the volume keeps its size, which a revert could otherwise shrink. A machine that uses the volume must be stopped first, or the volume detached.
  • Backups, in full. A backup of a volume on a block pool is a full backup, and it is written to the default pool. That pool must have room for the whole volume, and the node checks the room before the backup starts. Snapshots, backups and schedules describes the rest.
  • Restores. A restore into a new volume makes the volume the size of the backup. An in-place restore needs the machine stopped or the volume detached, and the backup must be exactly the size of the volume. Otherwise the node says: "the backup is not the size of this volume, so it cannot be restored in place: restore it into a new volume".
  • Templates. A template is copied to a block pool once, the first time a machine is made there from it. Every machine made from that template on the pool after that is a linked clone of the copy, so it starts at once and takes little space at first. A template with machines on a pool cannot be deleted. Templates and clones describes the rest.
  • Full clones. A stopped machine's boot disk can be copied in every direction: from a directory pool to a block pool, from a block pool to a directory pool, or from one block pool to another. Over the API, the clone's pool names the pool its boot disk is kept on. On the command line, vm clone --pool keeps the clone's boot disk on that pool, and the default pool is used when it is left out.
  • Machines. vm create --pool makes a machine whose boot disk is a volume on the pool, from an image or from a template. Removing a machine whose disk is on a block pool needs the storage feature in the licence.
  • Templates from machines. A stopped machine whose boot disk is on a block pool can be made into a template as usual. The template is a copy in the node's image store, so it can then be placed on any pool.

What does not work on a block pool

  • Incremental backups. A block volume takes only full backups, and an automatic backup of one is a full backup too. An incremental backup, or a schedule of incremental backups, is refused with "an incremental backup is not available on a thin-provisioned or ZFS pool yet: take a full backup, or keep the volume on a directory pool".

Room and quotas

  • A new ZFS volume takes its full size from its pool when it is made, so a ZFS pool needs that much free space. A thin-provisioned volume takes space as it is written.
  • Set a pool_used alert on every thin-provisioned pool, for example at 80%: the pool can fill as its machines write, and a full pool pauses the machines that are writing to it.
  • Before a create, a copy, a restore or a backup starts, the node checks that the pool has the room the work needs. A full pool refuses the work with "the storage pool is full: free some space, or add a pool".
  • A restore of copies read back from a backup target needs room for those copies as well as for the new volume. The node holds the room the copies take in the pool each one is written to, from the start of the fetch until it ends. Another create, copy, restore or backup that does not fit beside that room is refused until the fetch ends.
  • A machine's boot disk is not counted against a tenant's volume quota, on any pool. Data volumes are counted. Current limits lists the quotas.

Deleting a pool, and watching it

  • A pool that still holds volumes cannot be deleted, and the default pool cannot be deleted at all. The node answers "this pool still holds volumes: move or delete them first".
  • Deleting a block pool removes the node's record of it. The thin pool or ZFS dataset on the host is left as it was.
  • The node reads each block pool's free space on every poll. The pool_used condition in Monitoring watches the share of a pool in use, for directory and block pools alike.

In the console

On the node's Storage page, the Pools tab lists every pool with its kind, state, size and use. Create a pool says which kinds this server cannot use.

Installing the pools with the installer

On a new server, the installer can check or create a thin-provisioned pool and a ZFS pool for you. See Storage pools at install.

See also

↓ This page as JSON ↓ All documentation as JSON