Python Sahara client¶
Overview¶
Sahara Client provides a list of Python interfaces to communicate with the Sahara REST API. Sahara Client enables users to perform most of the existing operations like retrieving template lists, creating Clusters, submitting EDP Jobs, etc.
Instantiating a Client¶
To start using the Sahara Client users have to create an instance of the Client class. The client constructor has a list of parameters to authenticate and locate Sahara endpoint.
- class saharaclient.api.client.Client(session=None, sahara_url=None, endpoint_type='publicURL', service_type='data-processing', region_name=None, **kwargs)[source]¶
- Important!
It is not a mandatory rule to provide all of the parameters above. The minimum number should be enough to determine Sahara endpoint, check user authentication and tenant to operate in.
Authentication check¶
Passing authentication parameters to Sahara Client is deprecated. Keystone Session object should be used for this purpose. For example:
from keystoneauth1.identity import v2
from keystoneauth1 import session
from saharaclient import client
auth = v2.Password(auth_url=AUTH_URL,
username=USERNAME,
password=PASSWORD,
tenant_name=PROJECT_ID)
ses = session.Session(auth=auth)
sahara = client.Client('1.1', session=ses)
For more information about Keystone Sessions, see Using Sessions.
Sahara endpoint discovery¶
If user has a direct URL pointing to Sahara REST API, it may be specified as sahara_url. If this parameter is missing, Sahara client will use Keystone Service Catalog to find the endpoint. There are two parameters: service_type and endpoint_type to configure endpoint search. Both parameters have default values.
from keystoneauth1.identity import v2
from keystoneauth1 import session
from saharaclient import client
auth = v2.Password(auth_url=AUTH_URL,
username=USERNAME,
password=PASSWORD,
tenant_name=PROJECT_ID)
ses = session.Session(auth=auth)
sahara = client.Client('1.1', session=ses,
service_type="non-default-service-type",
endpoint_type="internalURL")
Object managers¶
Sahara Client has a list of fields to operate with:
plugins
clusters
cluster_templates
node_group_templates
images
data_sources
job_binaries
job_binary_internals
job_executions
job_types
Each of this fields is a reference to a Manager for a corresponding group of REST calls.
Supported operations¶
Plugin ops¶
- class saharaclient.api.plugins.PluginManagerV1(api)[source]¶
- convert_to_cluster_template(plugin_name, hadoop_version, template_name, filecontent)[source]¶
Convert to cluster template
Create Cluster Template directly, avoiding Cluster Template mechanism.
- get(plugin_name)¶
Get information about a Plugin.
- get_version_details(plugin_name, hadoop_version)¶
Get version details
Get the list of Services and Service Parameters for a specified Plugin and Plugin Version.
- list(search_opts=None)¶
Get a list of Plugins.
- update(plugin_name, values)¶
Update plugin and then return updated result to user
Image Registry ops¶
- class saharaclient.api.images.ImageManagerV1(api)[source]¶
- get(id)¶
Get information about an image
- list(search_opts=None)¶
Get a list of registered images.
- unregister_image(image_id)¶
Remove an Image from Sahara Image Registry.
- update_image(image_id, user_name, desc=None)¶
Create or update an Image in Image Registry.
Node Group Template ops¶
- class saharaclient.api.node_group_templates.NodeGroupTemplateManagerV1(api)[source]¶
- create(name, plugin_name, hadoop_version, flavor_id, description=None, volumes_per_node=None, volumes_size=None, node_processes=None, node_configs=None, floating_ip_pool=None, security_groups=None, auto_security_group=None, availability_zone=None, volumes_availability_zone=None, volume_type=None, image_id=None, is_proxy_gateway=None, volume_local_to_instance=None, use_autoconfig=None, shares=None, is_public=None, is_protected=None, volume_mount_prefix=None)[source]¶
Create a Node Group Template.
- list(search_opts=None, marker=None, limit=None, sort_by=None, reverse=None)[source]¶
Get a list of Node Group Templates.
- update(ng_template_id, name=NotUpdated, plugin_name=NotUpdated, hadoop_version=NotUpdated, flavor_id=NotUpdated, description=NotUpdated, volumes_per_node=NotUpdated, volumes_size=NotUpdated, node_processes=NotUpdated, node_configs=NotUpdated, floating_ip_pool=NotUpdated, security_groups=NotUpdated, auto_security_group=NotUpdated, availability_zone=NotUpdated, volumes_availability_zone=NotUpdated, volume_type=NotUpdated, image_id=NotUpdated, is_proxy_gateway=NotUpdated, volume_local_to_instance=NotUpdated, use_autoconfig=NotUpdated, shares=NotUpdated, is_public=NotUpdated, is_protected=NotUpdated, volume_mount_prefix=NotUpdated)[source]¶
Update a Node Group Template.
Cluster Template ops¶
- class saharaclient.api.cluster_templates.ClusterTemplateManagerV1(api)[source]¶
- create(name, plugin_name, hadoop_version, description=None, cluster_configs=None, node_groups=None, anti_affinity=None, net_id=None, default_image_id=None, use_autoconfig=None, shares=None, is_public=None, is_protected=None, domain_name=None)[source]¶
Create a Cluster Template.
- list(search_opts=None, marker=None, limit=None, sort_by=None, reverse=None)[source]¶
Get list of Cluster Templates.
- update(cluster_template_id, name=NotUpdated, plugin_name=NotUpdated, hadoop_version=NotUpdated, description=NotUpdated, cluster_configs=NotUpdated, node_groups=NotUpdated, anti_affinity=NotUpdated, net_id=NotUpdated, default_image_id=NotUpdated, use_autoconfig=NotUpdated, shares=NotUpdated, is_public=NotUpdated, is_protected=NotUpdated, domain_name=NotUpdated)[source]¶
Update a Cluster Template.
Cluster ops¶
- class saharaclient.api.clusters.ClusterManagerV1(api)[source]¶
- create(name, plugin_name, hadoop_version, cluster_template_id=None, default_image_id=None, is_transient=None, description=None, cluster_configs=None, node_groups=None, user_keypair_id=None, anti_affinity=None, net_id=None, count=None, use_autoconfig=None, shares=None, is_public=None, is_protected=None)[source]¶
Launch a Cluster.
- list(search_opts=None, limit=None, marker=None, sort_by=None, reverse=None)[source]¶
Get a list of Clusters.
- scale(cluster_id, scale_object)[source]¶
Scale an existing Cluster.
- Parameters
scale_object – dict that describes scaling operation
- Example
The following scale_object can be used to change the number of instances in the node group and add instances of new node group to existing cluster:
{ "add_node_groups": [ { "count": 3, "name": "new_ng", "node_group_template_id": "ngt_id" } ], "resize_node_groups": [ { "count": 2, "name": "old_ng" } ] }
Data Source ops¶
- class saharaclient.api.data_sources.DataSourceManagerV1(api)[source]¶
- create(name, description, data_source_type, url, credential_user=None, credential_pass=None, is_public=None, is_protected=None, s3_credentials=None)[source]¶
Create a Data Source.
- list(search_opts=None, limit=None, marker=None, sort_by=None, reverse=None)[source]¶
Get a list of Data Sources.
- update(data_source_id, update_data)[source]¶
Update a Data Source.
- Parameters
update_data (dict) – dict that contains fields that should be updated with new values.
Fields that can be updated:
name
description
type
url
is_public
is_protected
credentials - dict with the keys user and password for data source in Swift, or with the keys accesskey, secretkey, endpoint, ssl, and bucket_in_path for data source in S3
Job Binary Internal ops¶
Job Binary ops¶
- class saharaclient.api.job_binaries.JobBinariesManagerV1(api)[source]¶
- create(name, url, description=None, extra=None, is_public=None, is_protected=None)[source]¶
Create a Job Binary.
- Parameters
extra (dict) – authentication info needed for some job binaries, containing the keys user and password for job binary in Swift or the keys accesskey, secretkey, and endpoint for job binary in S3
- list(search_opts=None, limit=None, marker=None, sort_by=None, reverse=None)[source]¶
Get a list of Job Binaries.
- update(job_binary_id, data)[source]¶
Update Job Binary.
- Parameters
data (dict) – dict that contains fields that should be updated with new values.
Fields that can be updated:
name
description
url
is_public
is_protected
extra - dict with the keys user and password for job binary in Swift, or with the keys accesskey, secretkey, and endpoint for job binary in S3
Job ops¶
Job Execution ops¶
- class saharaclient.api.job_executions.JobExecutionsManager(api)[source]¶
- create(job_id, cluster_id, input_id=None, output_id=None, configs=None, interface=None, is_public=None, is_protected=None)[source]¶
Launch a Job.