HDFS native driver

updated: 2020-03-12 01:42

HDFS native driver¶

The HDFS native driver is a plug-in for the Shared File Systems service. It uses Hadoop distributed file system (HDFS), a distributed file system designed to hold very large amounts of data, and provide high-throughput access to the data.

A Shared File Systems service share in this driver is a subdirectory in the hdfs root directory. Instances talk directly to the HDFS storage back end using the hdfs protocol. Access to each share is allowed by user based access type, which is aligned with HDFS ACLs to support access control of multiple users and groups.

Network configuration¶

The storage back end and Shared File Systems service hosts should be in a flat network, otherwise L3 connectivity between them should exist.

Supported shared filesystems and operations¶

The driver supports HDFS shares.

The following operations are supported:

Create a share.
Delete a share.
Allow share access.

Note the following limitations:
- Only user access type is supported.
Deny share access.
Create a snapshot.
Delete a snapshot.
Create a share from a snapshot.

Requirements¶

Install HDFS package, version >= 2.4.x, on the storage back end.
To enable access control, the HDFS file system must have ACLs enabled.
Establish network connection between the Shared File Systems service host and storage back end.

Shared File Systems service driver configuration¶

To enable the driver, set the share_driver option in file manila.conf and add other options as appropriate.

share_driver = manila.share.drivers.hdfs.hdfs_native.HDFSNativeShareDriver

Known restrictions¶

This driver does not support network segmented multi-tenancy model. Instead multi-tenancy is supported by the tenant specific user authentication.
Only support for single HDFS namenode in Kilo release.

Driver options¶

The following table contains the configuration options specific to the share driver.

Description of HDFS share driver configuration options¶
Configuration option = Default value	Description
[DEFAULT]
`hdfs_namenode_ip` = `None`	(String) The IP of the HDFS namenode.
`hdfs_namenode_port` = `9000`	(Port number) The port of HDFS namenode service.
`hdfs_ssh_name` = `None`	(String) HDFS namenode ssh login name.
`hdfs_ssh_port` = `22`	(Port number) HDFS namenode SSH port.
`hdfs_ssh_private_key` = `None`	(String) Path to HDFS namenode SSH private key for login.
`hdfs_ssh_pw` = `None`	(String) HDFS namenode SSH login password, This parameter is not necessary, if ‘hdfs_ssh_private_key’ is configured.

updated: 2020-03-12 01:42

Except where otherwise noted, this document is licensed under Creative Commons Attribution 3.0 License. See all OpenStack Legal Documents.