Exclusive Access · Invitation Only

Shared Storage Pools

A directory pool can be declared shared between the members of a cluster when every member mounts the same storage under the pool's directory. The declaration is what a later release relies on to move a machine while it runs (Moving Machines Between Nodes). Declaring a pool shared does not change any disk, and it does not move any machine.

Part of Clustering and Storage Pools, in early access. Declaring a pool shared needs a licence that includes Storage, and every member of the cluster to run a version that supports it.

What a shared pool is

  • Only a directory pool can be shared. A thin-provisioned pool or a ZFS pool cannot, and the node answers "Only a directory pool can be shared between members."
  • The pool's directory must sit on a filesystem that the virtual machine software treats as shared. The list is below.
  • The declaration is per member. Each member declares its own mount of the pool, and takes its own declaration back. A pool that one member declares shared is not shared with the others until they declare it too.

Which filesystems qualify

The node reads the filesystem of the pool's directory each time you declare the pool. These types qualify:

  • NFS
  • GFS2
  • OCFS2
  • AFS
  • SMB and CIFS
  • CephFS
  • GPFS
  • ACFS
  • BeeGFS
  • GlusterFS, mounted with the type fuse.glusterfs
  • Quobyte, mounted with the type fuse.quobyte

A FUSE mount counts by its mount type, so a FUSE mount of any other type does not qualify. If the pool's directory is a symbolic link, the node judges the directory the link leads to. Lustre is not on the list, so a directory on Lustre is refused.

Declaring a pool shared

Before you declare a pool, mount the same storage under its directory on every member that will share it. The node checks the type of the filesystem under the directory. It cannot check that the other members mount the same storage, so make sure they do.

In the console, open the node's Storage page, choose the Pools tab, and open the pool. In the Shared between members card, press Share with the other members, then confirm with Share the pool. The card then says Shared with the other members. To stop sharing, press Stop sharing and confirm with Stop sharing. Only an account with the Storage.Allocate privilege on the pool can change it. The card offers the declaration only on a pool whose directory is on one of the filesystems listed below. On any other pool it says why, and offers no declaration: the card shows the sentence for the reason in the table below, so a directory the node could not check is not reported as one on an ordinary filesystem. The filesystem makes no difference to stopping sharing.

On the command line, there is no command to declare a pool shared yet. The storage pool commands list, show, create and delete pools. Declare the pool in the console or over the API.

Over the API, send PUT /api/v1/storage/pools/{id}/shared with one field. Use {"shared": true} to declare the pool, or {"shared": false} to stop sharing it:

{"shared": true}

The answer is 200 with the pool as it now stands, its shared field set. The call needs the Storage.Allocate privilege on the pool, and a licence that includes Storage.

When a declaration is refused

A refused declaration changes nothing. A pool that is already shared stays shared until you stop sharing it. The node answers with these sentences:

The node saysWhat it means, and what to do
"Only a directory pool can be shared between members."The pool is thin-provisioned or ZFS, and such a pool is never shared. Use a directory pool if you want it shared.
"This pool's directory is not on a filesystem that can be shared between members. Mount a shared network or cluster filesystem there first."The directory is on an ordinary local filesystem. Mount one of the filesystems above on it, then declare the pool again.
"The pool's filesystem could not be checked right now. Try again when the node answers."The node could not read the filesystem under the directory. Try again. The cause goes to the node's log.
"Every member must run a version that supports this."Not every member runs a version that supports sharing. Update the other members first (see Upgrades). The console and the API say this until they do.
"This needs a cluster of servers: create one or join one first."The node is in no cluster. Create one, or join one, first.
"The pool's sharing must be an object that says whether the pool is shared."An API call sent a body other than {"shared": true} or {"shared": false}. Send one of those two bodies.

A licence that does not include Storage refuses the declaration, and an account without the Storage.Allocate privilege on the pool is refused for the privilege.

Stopping sharing

Stopping sharing takes the declaration back. Over the API it works whether or not every member of the cluster runs a version that supports sharing, and outside a cluster too, because taking a declaration back only removes a capability: the node does not ask about the other members, and a licence freeze does not stop it. The call needs the Storage.Allocate privilege on the pool, and a licence that includes Storage, as declaring does. The console offers Stop sharing whenever the pool is shared, at any level, outside a cluster and under a frozen licence. It keeps the button off, with the reason, for an account without Storage.Allocate on the pool or a licence without Storage.

A take-back is refused only while a machine is moving live on this member, as its source or its destination. It is also refused when the node cannot tell whether one is, so a take-back is never let through on a guess. A refused take-back changes nothing, and the pool stays shared. The node answers with these sentences:

The node saysWhat it means, and what to do
"A machine is moving while it runs on this pool, so the pool must stay shared until that move ends."A machine is moving live on this member. Wait until the move ends, then stop sharing.
"Whether a machine is moving on this pool could not be checked right now. Try again when the node answers."The node could not read its own move records. Try again. The cause goes to the node's log.

What sharing does, and what it does not do yet

  • Declaring a pool shared changes nothing on its disks, and it moves no machine.
  • Moving a running machine is not available yet. The node answers "Moving a running machine is not available yet. Choose an offline move." Until that move is offered, every move stops the machine, copies its disks, and starts it on the new member, whether or not its pool is shared. See Moving Machines Between Nodes.

Limits

  • The node checks the filesystem type, not what the other members mount. Declare a pool only where every member mounts the same storage under its directory.
  • A declaration can outlive its mount. If the directory later stops being on a shared filesystem, a new declaration is refused, but the declaration already held stays until you stop sharing it.

See also

↓ This page as JSON ↓ All documentation as JSON