HDFS native driver
HDFS native driver is a plugin based on the OpenStack manila service, which uses
Hadoop distributed file system (HDFS), a distributed file system designed to hold
very large amounts of data, and provide high-throughput access to the data.
A manila share in this driver is a subdirectory in hdfs root directory. Instances
talk directly to the HDFS storage backend with ‘hdfs’ protocol. And access to each
share is allowed by user based access type, which is aligned with HDFS ACLs to
support access control of multiple users and groups.
Network configuration
The storage backend and manila hosts should be in a flat network, otherwise, the L3
connectivity between them should exist.
Supported shared filesystems
- HDFS (authentication by user)
Supported Operations
- Create HDFS share
- Delete HDFS share
- Allow HDFS Share access
* Only support user access type
* Support level of access (ro/rw)
- Deny HDFS Share access
- Create snapshot
- Delete snapshot
- Create share from snapshot
- Extend share
Requirements
- Install HDFS package, version >= 2.4.x, on the storage backend
- To enable access control, the HDFS file system must have ACLs enabled
- Establish network connection between the manila host and storage backend
Manila driver configuration
share_driver = manila.share.drivers.hdfs.hdfs_native.HDFSNativeShareDriver
- hdfs_namenode_ip = the IP address of the HDFS namenode, and only single
namenode is supported now
hdfs_namenode_port = the port of the HDFS namenode service
hdfs_ssh_port = HDFS namenode SSH port
hdfs_ssh_name = HDFS namenode SSH login name
- hdfs_ssh_pw = HDFS namenode SSH login password, this parameter is not
necessary, if the following hdfs_ssh_private_key is configured
hdfs_ssh_private_key = Path to the HDFS namenode private key to ssh login
Known Restrictions
- This driver does not support network segmented multi-tenancy model. Instead
multi-tenancy is supported by the tenant specific user authentication
- Only support for single HDFS namenode in Kilo release
The manila.share.drivers.hdfs.hdfs_native Module
HDFS native protocol (hdfs) driver for manila shares.
Manila share is a directory in HDFS. And this share does not use
service VM instance (share server). The instance directly talks
to the HDFS cluster.
The initial version only supports single namenode and flat network.
- Configuration Requirements:
- To enable access control, HDFS file system must have ACLs enabled.
-
class HDFSNativeShareDriver(*args, **kwargs)
Bases: manila.share.driver.ExecuteMixin, manila.share.driver.ShareDriver
HDFS Share Driver.
Executes commands relating to shares.
API version history:
1.0 - Initial Version
-
allow_access(context, share, access, share_server=None)
Allows access to the share for a given user.
-
check_for_setup_error()
Return an error if the prerequisites are met.
-
create_share(context, share, share_server=None)
Create a HDFS directory which acted as a share.
-
create_share_from_snapshot(context, share, snapshot, share_server=None)
Creates a snapshot.
-
create_snapshot(context, snapshot, share_server=None)
Creates a snapshot.
-
delete_share(context, share, share_server=None)
Deletes share storage.
-
delete_snapshot(context, snapshot, share_server=None)
Deletes a snapshot.
-
deny_access(context, share, access, share_server=None)
Denies the access to the share for a given user.
-
do_setup(context)
Do initialization while the share driver starts.
-
ensure_share(context, share, share_server=None)
Ensure the storage are exported.
-
extend_share(share, new_size, share_server=None)
Extend share storage.
-
get_network_allocations_number()