CephFS driver¶
The CephFS driver enables manila to export shared filesystems backed by Ceph’s File System (CephFS) using either the Ceph network protocol or NFS protocol. Guests require a native Ceph client or an NFS client in order to mount the filesystem.
When guests access CephFS using the native Ceph protocol, access is controlled via Ceph’s cephx authentication system. If a user requests share access for an ID, Ceph creates a corresponding Ceph auth ID and a secret key if they do not already exist, and authorizes the ID to access the share. The client can then mount the share using the ID and the secret key. To learn more about configuring Ceph clients to access the shares created using this driver, please see the Ceph documentation
And when guests access CephFS through NFS, an NFS-Ganesha server (or CephFS NFS service) mediates access to CephFS. The driver enables access control by managing the NFS-Ganesha server’s exports.
Supported Operations¶
The following operations are supported with CephFS backend:
Create, delete, update and list share
Allow/deny access to share
Only
cephx
access type is supported for CephFS native protocol.Only
ip
access type is supported for NFS protocol.read-only
andread-write
access levels are supported.
Extend/shrink share
Manage/unmanage shares
Manage/unmanage share snapshots
Create, delete, update and list snapshot
Create, delete, update and list share groups
Delete and list share group snapshots
Important
Share group snapshot creation is no longer supported in mainline CephFS. This feature has been removed from manila W release.
Prerequisites¶
Important
A manila share backed by CephFS is only as good as the underlying filesystem. Take care when configuring your Ceph cluster, and consult the latest guidance on the use of CephFS in the Ceph documentation.
Ceph testing matrix¶
As Ceph and Manila continue to grow, it is essential to test and support combinations of releases supported by both projects. However, there is little community bandwidth to cover all of them. For simplicity sake, we are focused on testing (and therefore supporting) the current Ceph active releases. Check out the list of Ceph active releases here.
Below is the current state of testing for Ceph releases with this project. Adjacent components such as devstack-plugin-ceph and tripleo are added to the table below. Contributors to those projects can determine what versions of ceph are tested and supported with manila by those components; however, their state is presented here for ease of access.
OpenStack release |
Manila |
devstack-plugin-ceph |
---|---|---|
Wallaby |
Pacific |
Pacific |
Xena |
Pacific |
Quincy |
Yoga |
Quincy |
Quincy |
Zed |
Quincy |
Quincy |
2023.1 (“Antelope”) |
Quincy |
Quincy |
2023.2 (“Bobcat”) |
Quincy |
Reef |
2024.1 (“Caracal”) |
Reef |
Reef |
2024.2 (“Dalmatian”) |
Reef |
Reef |
Additionally, it is expected that the version of the Ceph client available to manila is aligned with the Ceph server version. Mixing server and client versions is strongly unadvised.
In case of using the NFS Ganesha driver, it’s also a good practice to use the versions that align with the Ceph version of choice.
Common Prerequisites¶
A Ceph cluster with a filesystem configured (See Create ceph filesystem on how to create a filesystem.)
python3-rados
andpython3-ceph-argparse
packages installed in the servers running the manila-share service.Network connectivity between your Ceph cluster’s public network and the servers running the manila-share service.
Enabling snapshot support in Ceph backend¶
From Ceph Nautilus, all new filesystems created on Ceph have snapshots enabled by default. If you’ve upgraded your ceph cluster and want to enable snapshots on a pre-existing filesystem, you can do so:
ceph fs set {fs_name} allow_new_snaps true
Configuring CephFS backend in manila.conf¶
Space considerations¶
The CephFS driver reports total and free capacity available across the Ceph
cluster to manila to allow provisioning. All CephFS shares are thinly
provisioned, i.e., empty shares do not consume any significant space
on the cluster. The CephFS driver does not allow controlling oversubscription
via manila. So, as long as there is free space, provisioning will continue,
and eventually this may cause your Ceph cluster to be over provisioned and
you may run out of space if shares are being filled to capacity. It is advised
that you use Ceph’s monitoring tools to monitor space usage and add more
storage when required in order to honor space requirements for provisioned
manila shares. You may use the driver configuration option
reserved_share_percentage
to prevent manila from filling up your Ceph
cluster, and allow existing shares to grow.
Known restrictions¶
A CephFS driver instance, represented as a backend driver section in manila.conf, requires a Ceph auth ID unique to the backend Ceph Filesystem. Using a non-unique Ceph auth ID will result in the driver unintentionally evicting other CephFS clients using the same Ceph auth ID to connect to the backend.
Snapshots are read-only. A user can read a snapshot’s contents from the
.snap/{manila-snapshot-id}_{unknown-id}
folder within the mounted share.
Security¶
Each share’s data is mapped to a distinct Ceph RADOS namespace. A guest is restricted to access only that particular RADOS namespace. https://docs.ceph.com/en/latest/cephfs/file-layouts/
Configuration Reference¶
Configuration option = Default value |
Description |
---|---|
[DEFAULT] |
|
|
(String) The name of the ceph auth identity to use. |
|
(String) The name of the cluster in use, if it is not the default (‘ceph’). |
|
(String) Fully qualified path to the ceph.conf file. |
|
(String) The type of protocol helper to use. Default is CEPHFS. |
|
(Boolean) Whether the NFS-Ganesha server is remote to the driver. |
|
(String) The IP address of the NFS-Ganesha server. |
|
(String) The type of protocol helper to use. Default is CEPHFS. |
|
(String) The username to authenticate as in the remote NFS-Ganesha server host. |
|
(String) The path of the driver host’s private SSH key file. |
|
(String) The password to authenticate as the user in the remote Ganesha server host. This is not required if ‘cephfs_ganesha_path_to_private_key’ is configured. |
|
(String) List of IPs to export shares. If not supplied, then the value of ‘cephfs_ganesha_server_ip’ will be used to construct share export locations. |
|
(String) The read/write/execute permissions mode for CephFS volumes, snapshots, and snapshot groups expressed in Octal as with linux ‘chmod’ or ‘umask’ commands. |
|
(String) The name of the filesystem to use, if there are multiple filesystems in the cluster.” |
|
(String) Provide a unique string value to make the driver ensure all of the shares it has created during startup. Ensuring would re-export shares and this action isn’t always required, unless something has been administratively modified on CephFS. |
|
(String) The ID of the NFS cluster to use. |