HPE 3PAR Driver for OpenStack Manila

The HPE 3PAR driver provides NFS and CIFS shared file systems to OpenStack using HPE 3PAR’s File Persona capabilities.

For information on HPE 3PAR Driver for OpenStack Manila, refer to content kit page.

HPE 3PAR File Persona Software Suite concepts and terminology

The software suite comprises the following managed objects:

  • File Provisioning Groups (FPGs)

  • Virtual File Servers (VFSs)

  • File Stores

  • File Shares

The File Persona Software Suite is built upon the resilient mesh-active architecture of HPE 3PAR StoreServ and benefits from HPE 3PAR storage foundation of wide-striped logical disks and autonomic Common Provisioning Groups (CPGs). A CPG can be shared between file and block to create the File Shares or the logical unit numbers (LUNs) to provide true convergence.

A File Provisioning Group (FPG) is an instance of the HPE intellectual property Adaptive File System. It controls how files are stored and retrieved. Each FPG is transparently constructed from one or multiple Virtual Volumes (VVs) and is the unit for replication and disaster recovery for File Persona Software Suite. There are up to 16 FPGs supported on a node pair.

A Virtual File Server (VFS) is conceptually like a server. As such, it presents virtual IP addresses to clients, participates in user authentication services, and can have properties for such things as user/group quota management and antivirus policies. Up to 16 VFSs are supported on a node pair, one per FPG.

File Stores are the slice of a VFS and FPG at which snapshots are taken, capacity quota management can be performed, and antivirus scan service policies customized. There are up to 256 File Stores supported on a node pair, 16 File Stores per VFS.

File Shares are what provide data access to clients via SMB, NFS, and the Object Access API, subject to the share permissions applied to them. Multiple File Shares can be created for a File Store and at different directory levels within a File Store.

Supported shared filesystems

The driver supports CIFS and NFS shares.

Operations supported

  • Create a share.

    – Share is not accessible until access rules allow access.

  • Delete a share.

  • Allow share access.

    Note the following limitations:

    – IP access rules are required for NFS share access.

    – User access rules are not allowed for NFS shares.

    – User access rules are required for SMB share access.

    – User access requires a File Persona local user for SMB shares.

    – Shares are read/write (and subject to ACLs).

  • Deny share access.

  • Create a snapshot.

  • Delete a snapshot.

  • Create a share from a snapshot.

  • Extend a share.

  • Shrink a share.

  • Share networks.

HPE 3PAR File Persona driver can be configured to work with or without share networks. When using share networks, the HPE 3PAR driver allocates an FSIP on the back end FPG (VFS) to match the share network’s subnet and segmentation ID. Security groups associated with share networks are ignored.

Operations not supported

  • Manage and unmanage

  • Manila Experimental APIs (consistency groups, replication, and migration) were added in Mitaka but have not yet been implemented by the HPE 3PAR File Persona driver.

Requirements

On the OpenStack host running the Manila share service:

  • python-3parclient version 4.2.0 or newer from PyPI.

On the HPE 3PAR array:

  • HPE 3PAR Operating System software version 3.2.1 MU3 or higher.

  • The array class and hardware configuration must support File Persona.

Pre-configuration on the HPE 3PAR StoreServ

The following HPE 3PAR CLI commands show how to set up the HPE 3PAR StoreServ to use File Persona with OpenStack Manila. HPE 3PAR File Persona must be initialized, and started on the HPE 3PAR storage.

cli% startfs 0:2:1 1:2:1
cli% setfs nodeip -ipaddress 10.10.10.11 -subnet 255.255.240.0 0
cli% setfs nodeip -ipaddress 10.10.10.12 -subnet 255.255.240.0 1
cli% setfs dns 192.168.8.80,127.127.5.50 foo.com,bar.com
cli% setfs gw 10.10.10.10

  • A File Provisioning Group (FPG) must be created for use with the Shared File Systems service.

    cli% createfpg examplecpg examplefpg 18T

  • A Virtual File Server (VFS) must be created on the FPG.

  • The VFS must be configured with an appropriate share export IP address.

    cli% createvfs -fpg examplefpg 10.10.10.101 255.255.0.0 examplevfs

  • A local user in the Administrators group is needed for CIFS (SMB) shares.

    cli% createfsgroup fsusers
cli% createfsuser –passwd <password> -enable true -grplist
Users,Administrators –primarygroup fsusers fsadmin

  • The WSAPI with HTTP and/or HTTPS must be enabled and started.

    cli% setwsapi -https enable
cli% startwsapi

HPE 3PAR shared file system driver configuration

  • Install the python-3parclient python package on the OpenStack Block Storage system:

    $ pip install 'python-3parclient>=4.0,<5.0'

  • Manila configuration file

    The Manila configuration file (typically /etc/manila/manila.conf) defines and configures the Manila drivers and backends. After updating the configuration file, the Manila share service must be restarted for changes to take effect.

  • Enable share protocols

    To enable share protocols, an optional list of supported protocols can be specified using the enabled_share_protocols setting in the DEFAULT section of the manila.conf file. The default is NFS, CIFS which allows both protocols supported by HPE 3PAR (NFS and SMB). Where Manila uses the term CIFS, HPE 3PAR uses the term SMB. Use the enabled_share_protocols option if you want to only provide one type of share (for example, only NFS) or if you want to explicitly avoid the introduction of other protocols that can be added for other drivers in the future.

  • Enable share back ends

    In the [DEFAULT] section of the Manila configuration file, use the enabled_share_backends option to specify the name of one or more back-end configuration sections to be enabled. To enable multiple back ends, use a comma-separated list.

    Note

    The name of the backend’s configuration section is used (which may be different from the share_backend_name value)

  • Configure each back end

    For each back end, a configuration section defines the driver and back end options. These include common Manila options, as well as driver-specific options. The following Driver options section describes the parameters that need to be configured in the Manila configuration file for the HPE 3PAR driver.

Driver options

The following table contains the configuration options specific to the share driver.

Description of HPE 3PAR share driver configuration options

Configuration option = Default value

Description

[DEFAULT]

hpe3par_api_url =

(String) 3PAR WSAPI Server Url like https://<3par ip>:8080/api/v1

hpe3par_cifs_admin_access_domain = LOCAL_CLUSTER

(String) File system domain for the CIFS admin user.

hpe3par_cifs_admin_access_password =

(String) File system admin password for CIFS.

hpe3par_cifs_admin_access_username =

(String) File system admin user name for CIFS.

hpe3par_debug = False

(Boolean) Enable HTTP debugging to 3PAR

hpe3par_fpg = None

(Unknown) The File Provisioning Group (FPG) to use

hpe3par_fstore_per_share = False

(Boolean) Use one filestore per share

hpe3par_password =

(String) 3PAR password for the user specified in hpe3par_username

hpe3par_require_cifs_ip = False

(Boolean) Require IP access rules for CIFS (in addition to user)

hpe3par_san_ip =

(String) IP address of SAN controller

hpe3par_san_login =

(String) Username for SAN controller

hpe3par_san_password =

(String) Password for SAN controller

hpe3par_san_ssh_port = 22

(Port number) SSH port to use with SAN

hpe3par_share_mount_path = /mnt/

(String) The path where shares will be mounted when deleting nested file trees.

hpe3par_username =

(String) 3PAR username with the ‘edit’ role

HPE 3PAR Manila driver configuration example

The following parameters shows a sample subset of the manila.conf file, which configures two backends and the relevant [DEFAULT] options. A real configuration would include additional [DEFAULT] options and additional sections that are not discussed in this document. In this example, the backends are using different FPGs on the same array:

[DEFAULT]
enabled_share_backends = HPE1,HPE2
enabled_share_protocols = NFS,CIFS
default_share_type = default
[HPE1]
share_backend_name = HPE3PAR1
share_driver = manila.share.drivers.hpe.hpe_3par_driver.HPE3ParShareDriver
driver_handles_share_servers = False
max_over_subscription_ratio = 1
hpe3par_fpg = examplefpg,10.10.10.101
hpe3par_san_ip = 10.20.30.40
hpe3par_api_url = https://10.20.30.40:8080/api/v1
hpe3par_username = <username>
hpe3par_password = <password>
hpe3par_san_login = <san_username>
hpe3par_san_password = <san_password>
hpe3par_debug = False
hpe3par_cifs_admin_access_username = <fs_admin>
hpe3par_cifs_admin_access_password = <fs_password>
[HPE2]
share_backend_name = HPE3PAR2
share_driver = manila.share.drivers.hpe.hpe_3par_driver.HPE3ParShareDriver
driver_handles_share_servers = False
max_over_subscription_ratio = 1
hpe3par_fpg = examplefpg2,10.10.10.102
hpe3par_san_ip = 10.20.30.40
hpe3par_api_url = https://10.20.30.40:8080/api/v1
hpe3par_username = <username>
hpe3par_password = <password>
hpe3par_san_login = <san_username>
hpe3par_san_password = <san_password>
hpe3par_debug = False
hpe3par_cifs_admin_access_username = <fs_admin>
hpe3par_cifs_admin_access_password = <password>

Network approach

Network connectivity between the storage array (SSH/CLI and WSAPI) and the Manila host is required for share management. Network connectivity between the clients and the VFS is required for mounting and using the shares. This includes:

  • Routing from the client to the external network.

  • Assigning the client an external IP address, for example a floating IP.

  • Configuring the Shared File Systems service host networking properly for IP forwarding.

  • Configuring the VFS networking properly for client subnets.

  • Configuring network segmentation, if applicable.

In the OpenStack Kilo release, the HPE 3PAR driver did not support share networks. Share access from clients to HPE 3PAR shares required external network access (external to OpenStack) and was set up and configured outside of Manila.

In the OpenStack Liberty release, the HPE 3PAR driver could run with or without share networks. The configuration option driver_handles_share_servers``( ``True or False ) indicated whether share networks could be used. When set to False, the HPE 3PAR driver behaved as described earlier for Kilo. When set to True, the share network’s subnet, segmentation ID and IP address range were used to allocate an FSIP on the HPE 3PAR. There is a limit of four FSIPs per VFS. For clients to communicate with shares via this FSIP, the client must have access to the external network using the subnet and segmentation ID of the share network.

For example, the client must be routed to the neutron provider network with external access. The Manila host networking configuration and network switches must support the subnet routing. If the VLAN segmentation ID is used, communication with the share will use the FSIP IP address. Neutron networking is required for HPE 3PAR share network support. Flat and VLAN provider networks are supported, but the HPE 3PAR driver does not support share network security groups.

Share access

A share that is mounted before access is allowed can appear to be an empty read-only share. After granting access, the share must be remounted.

  • IP access rules are required for NFS.

  • SMB shares require user access rules.

With the proper access rules, share access is not limited to the OpenStack environment. Access rules added via Manila or directly in HPE 3PAR CLI can be used to allow access to clients outside of the stack. The HPE 3PAR VFS/FSIP settings determine the subnets visible for HPE 3PAR share access.

  • IP access rules

    To allow IP access to a share in the horizon UI, find the share in the Project|Manage Compute|Shares view. Use the Manage Rules action to add a rule. Select IP as the access type, and enter the external IP address (for example, the floating IP) of the client in the Access to box.

    You can also use the command line to allow IP access to a share in the horizon UI with the command:

    $ manila access-allow <share-id> ip <ip-address>

  • User access rules

    To allow user access to a share in the horizon UI, find the share in the Project|Manage Compute|Shares view. Use the Manage Rules action to add a rule. Select user as the access type and enter user name in the Access to box.

    You can also use the command line to allow user access to a share in the horizon UI with the command:

    $ manila access-allow <share-id> user <user name>

    The user name must be an HPE 3PAR user.

    Share access is different from file system permissions, for example, ACLs on files and folders. If a user wants to read a file, the user must have at least read permissions on the share and an ACL that grants him read permissions on the file or folder. Even with full control share access, it does not mean every user can do everything due to the additional restrictions of the folder ACLs.

    To modify the file or folder ACLs, allow access to an HPE 3PAR File Persona local user that is in the administrator’s group and connect to the share using that user’s credentials. Then, use the appropriate mechanism to modify the ACL or permissions to allow different access than what is provided by default.

Share types

When creating a share, a share type can be specified to determine where and how the share will be created. If a share type is not specified, the default_share_type set in the Shared File Systems service configuration file is used.

Manila share types are a type or label that can be selected at share creation time in OpenStack. These types can be created either in the Admin horizon UI or using the command line, as follows:

$ manila --os-username admin --os-tenant-name demo type-create
–is_public false <name> false

The <name> is the name of the new share type. False at the end specifies driver_handles_share_servers=False. The driver_handles_share_servers setting in the share type needs to match the setting configured for the back end in the manila.conf file.

is_public is used to indicate whether this share type is applicable to all tenants or will be assigned to specific tenants.

--os-username admin --os-tenant-name demo are only needed if your environment variables do not specify the desired user and tenant.

For share types that are not public, use Manila type-access-add to assign the share type to a tenant.

  • Using share types to require share networks

    The Shared File Systems service requires that the share type include the driver_handles_share_servers extra-spec. This ensures that the share is created on a back end that supports the requested driver_handles_share_servers (share networks) capability. From the Liberty release forward, both True and False are supported.

    The driver_handles_share_servers setting in the share type must match the setting in the back end configuration.

  • Using share types to select backends by name

    Administrators can optionally specify that a particular share type be explicitly associated with a single back end (or group of backends) by including the extra spec share_backend_name to match the name specified within the share_backend_name option in the back end configuration.

    When a share type is not selected during share creation, the default share type is used. To prevent creating these shares on any back end, the default share type needs to be specific enough to find appropriate default backends (or to find none if the default should not be used). The following example shows how to set share_backend_name for a share type.

    $ manila --os-username admin --os-tenant-name demo type-key <share-type>
set share_backend_name=HPE3PAR2

  • Using share types to select backends with capabilities

    The HPE 3PAR driver automatically reports capabilities based on the FPG used for each back end. An administrator can create share types with extra specs, which controls share types that can use FPGs with or without specific capabilities.

    With the OpenStack Liberty release or later, below section shows the extra specs used with the capabilities filter and the HPE 3PAR driver:

    hpe3par_flash_cache

    When the value is set to <is> True (or <is> False), shares of this type are only created on a back end that uses HPE 3PAR Adaptive Flash Cache. For Adaptive Flash Cache, the HPE 3PAR StoreServ Storage array must meet the following requirements:

    • Adaptive Flash Cache enabled

    • Available SSDs

    • Adaptive Flash Cache must be enabled on the HPE 3PAR StoreServ Storage array. This is done with the following CLI command:

      cli% createflashcache <size>

      <size> must be in 16 GB increments. For example, the below command creates 128 GB of Flash Cache for each node pair in the array.

      cli% createflashcache 128g

    • Adaptive Flash Cache must be enabled for the VV set used by an FPG. For example, setflashcache vvset:<fpgname>. The VV set name is the same as the FPG name.

      Note

      This setting affects all shares in that FPG (on that back end).

    Dedupe

    When the value is set to <is> True (or <is> False), shares of this type are only created on a back end that uses deduplication. For HPE 3PAR File Persona, the provisioning type is determined when the FPG is created. Using the createfpg –tdvv option creates an FPG that supports both dedupe and thin provisioning. The thin deduplication must be enabled to use the tdvv option.

    thin_provisioning

    When the value is set to <is> True (or <is> False), shares of this type are only created on a back end that uses thin (or full) provisioning. For HPE 3PAR File Persona, the provisioning type is determined when the FPG is created. By default, FPGs are created with thin provisioning. The capacity filter uses the total provisioned space and configured max_oversubscription_ratio when filtering and weighing backends that use thin provisioning.

  • Using share types to influence share creation options

    Scoped extra-specs are used to influence vendor-specific implementation details. Scoped extra-specs use a prefix followed by a colon. For HPE 3PAR, these extra specs have a prefix of hpe3par.

    The following HPE 3PAR extra-specs are used when creating CIFS (SMB) shares:

    hpe3par:smb_access_based_enum

    smb_access_based_enum (Access Based Enumeration) specifies if users can see only the files and directories to which they have been allowed access on the shares. Valid values are True or False. The default is False.

    hpe3par:smb_continuous_avail

    smb_continuous_avail (Continuous Availability) specifies if continuous availability features of SMB3 should be enabled for this share. Valid values are True or False. The default is True.

    hpe3par:smb_cache

    smb_cache specifies client-side caching for offline files. The default value is manual. Valid values are:

    • off — the client must not cache any files from this share. The share is configured to disallow caching.

    • manual — the client must allow only manual caching for the files open from this share.

    • optimized — the client may cache every file that it opens from this share. Also, the client may satisfy the file requests from its local cache. The share is configured to allow automatic caching of programs and documents.

    • auto — the client may cache every file that it opens from this share. The share is configured to allow automatic caching of documents.

    When creating NFS shares, the following HPE 3PAR extra-specs are used:

    hpe3par:nfs_options

    Comma separated list of NFS export options.

    The NFS export options have the following limitations:

    ro and rw are not allowed (will be determined by the driver)

    no_subtree_check and fsid are not allowed per HPE 3PAR CLI support

    (in)secure and (no_)root_squash are not allowed because the HPE 3PAR driver controls those settings

    All other NFS options are forwarded to the HPE 3PAR as part of share creation. The HPE 3PAR performs additional validation at share creation time. For details, see the HPE 3PAR CLI help.

Implementation characteristics

  • Shares from snapshots

    • When a share is created from a snapshot, the share must be deleted before the snapshot can be deleted. This is enforced by the driver.

    • A snapshot of an empty share will appear to work correctly, but attempting to create a share from an empty share snapshot may fail with an NFS Create export error.

    • HPE 3PAR File Persona snapshots are for an entire File Store. In Manila, they appear as snapshots of shares. A share sub-directory is used to give the appearance of a share snapshot when using create share from snapshot .

  • Snapshots

    • For HPE 3PAR File Persona, snapshots are per File Store and not per share. So, the HPE 3PAR limit of 1024 snapshots per File Store results in a Manila limit of 1024 snapshots per tenant on each back end FPG.

    • Before deleting a share, you must delete its snapshots. This is enforced by Manila. For HPE 3PAR File Persona, this also kicks off a snapshot reclamation.

  • Size enforcement

    Manila users create shares with size limits. HPE 3PAR enforces size limits by using File Store quotas. When using hpe3par_fstore_per_share``= ``True``(the non-default setting) there is only one share per File Store, so the size enforcement acts as expected. When using ``hpe3par_fstore_per_share = False (the default), the HPE 3PAR Manila driver uses one File Store for multiple shares. In this case, the size of the File Store limit is set to the cumulative limit of its Manila share sizes. This can allow one tenant share to exceed the limit and affect the space available for the same tenant’s other shares. One tenant cannot use another tenant’s File Store.

  • File removal

    When shares are removed and the hpe3par_fstore_per_share``=``False setting is used (the default), files may be left behind in the File Store. Prior to Mitaka, removal of obsolete share directories and files that have been stranded would require tools outside of OpenStack/Manila. In Mitaka and later, the driver mounts the File Store to remove the deleted share’s subdirectory and files. For SMB/CIFS share, it requires the hpe3par_cifs_admin_access_username and hpe3par_cifs_admin_access_password configuration. If the mount and delete cannot be performed, an error is logged and the share is deleted in Manila. Due to the potential space held by leftover files, File Store quotas are not reduced when shares are removed.

  • Multi-tenancy

    • Network

      The driver_handles_share_servers configuration setting determines whether share networks are supported. When driver_handles_share_servers is set to True, a share network is required to create a share. The administrator creates share networks with the desired network, subnet, IP range, and segmentation ID. The HPE 3PAR is configured with an FSIP using the same subnet and segmentation ID and an IP address allocated from the neutron network. Using share network-specific IP addresses, subnets, and segmentation IDs give the appearance of better tenant isolation. Shares on an FPG, however, are accessible via any of the FSIPs (subject to access rules). Back end filtering should be used for further separation.

    • Back end filtering

      A Manila HPE 3PAR back end configuration refers to a specific array and a specific FPG. With multiple backends and multiple tenants, the scheduler determines where shares will be created. In a scenario where an array or back end needs to be restricted to one or more specific tenants, share types can be used to influence the selection of a back end. For more information on using share types, see Share types .

    • Tenant limit

      The HPE 3PAR driver uses one File Store per tenant per protocol in each configured FPG. When only one back end is configured, this results in a limit of eight tenants (16 if only using one protocol). Use multiple back end configurations to introduce additional FPGs on the same array to increase the tenant limit.

      When using share networks, an FSIP is created for each share network (when its first share is created on the back end). The HPE 3PAR supports 4 FSIPs per FPG (VFS). One of those 4 FSIPs is reserved for the initial VFS IP, so the share network limit is 48 share networks per node pair.