org.apache.oozie.action.hadoop

Class SparkActionExecutor

java.lang.Object

org.apache.oozie.action.ActionExecutor

org.apache.oozie.action.hadoop.JavaActionExecutor

org.apache.oozie.action.hadoop.SparkActionExecutor

public class SparkActionExecutor
extends JavaActionExecutor

Nested Class Summary

Nested classes/interfaces inherited from class org.apache.oozie.action.ActionExecutor

ActionExecutor.Context, ActionExecutor.RETRYPOLICY

Field Summary

Fields

Modifier and Type	Field and Description
static String	MAPRED_CHILD_ENV
static String	SPARK_CLASS
static String	SPARK_DEFAULT_OPTS
static String	SPARK_JAR
static String	SPARK_JOB_NAME
static String	SPARK_MAIN_CLASS_NAME
static String	SPARK_MASTER
static String	SPARK_MODE
static String	SPARK_OPTS

Fields inherited from class org.apache.oozie.action.hadoop.JavaActionExecutor

ACL_MODIFY_JOB, ACL_VIEW_JOB, DEFAULT_LAUNCHER_MAX_ATTEMPS, DEFAULT_LAUNCHER_MEMORY_MB, DEFAULT_LAUNCHER_PRIORITY, DEFAULT_LAUNCHER_QUEUE, DEFAULT_LAUNCHER_VCORES, FAILED, FAILED_KILLED, HADOOP_CHILD_JAVA_ENV, HADOOP_CHILD_JAVA_OPTS, HADOOP_JOB_CLASSLOADER, HADOOP_MAP_JAVA_ENV, HADOOP_MAP_JAVA_OPTS, HADOOP_MAP_MEMORY_MB, HADOOP_NAME_NODE, HADOOP_REDUCE_JAVA_OPTS, HADOOP_USER, HADOOP_USER_CLASSPATH_FIRST, HADOOP_YARN_KILL_CHILD_JOBS_ON_AMRESTART, HADOOP_YARN_RM, HADOOP_YARN_TIMELINE_SERVICE_ENABLED, HADOOP_YARN_UBER_MODE, KILLED, LAUNCER_MODIFY_ACL, LAUNCER_VIEW_ACL, LOG, MAPREDUCE_TO_CLASSPATH, MAX_EXTERNAL_STATS_SIZE, OOZIE_ACTION_LAUNCHER_PREFIX, OOZIE_COMMON_LIBDIR, OOZIE_CREDENTIALS_SKIP, OOZIE_LAUNCHER_ADD_MAPREDUCE_TO_CLASSPATH_PROPERTY, RUNNING, SUCCEEDED, workflowConf, YARN_AM_COMMAND_OPTS, YARN_AM_ENV, YARN_AM_RESOURCE_MB, YARN_MEMORY_MB_MIN

Fields inherited from class org.apache.oozie.action.ActionExecutor

ACTION_RETRY_INTERVAL, ACTION_RETRY_POLICY, CONF_PREFIX, ERROR_OTHER, MAX_RETRIES, OOZIE_ACTION_YARN_TAG, RETRY_INTERVAL

Constructor Summary

Constructors

Constructor and Description
SparkActionExecutor()

Method Summary

Modifier and Type	Method and Description
protected void	addActionSpecificEnvVars(Map<String,String> env) Adds action-specific environment variables.
protected String	getDefaultShareLibName(org.jdom.Element actionXml) Return the sharelib name for the action.
List<Class<?>>	getLauncherClasses()
protected String	getLauncherMain(org.apache.hadoop.conf.Configuration launcherConf, org.jdom.Element actionXml)
String[]	getShareLibFilesForActionConf()

Methods inherited from class org.apache.oozie.action.hadoop.JavaActionExecutor

addActionLibs, addAppNameContext, addShareLib, addSystemShareLibForAction, check, createBaseHadoopConf, createBaseHadoopConf, createJobClient, createYarnClient, end, getActionCredentialsProperties, getActionData, getActualExternalId, getAppName, getCaptureOutput, getCommonLauncherClasses, getCredProperties, getMaxExternalStatsSize, getMaxOutputData, getShareLibNames, getTrimmedEncodedPath, initActionType, injectCallback, isCompleted, kill, loadHadoopDefaultResources, parseJobXmlAndConfiguration, parseJobXmlAndConfiguration, prepareActionDir, requiresNameNodeJobTracker, setActionCompletionData, setActionTokenProperties, setCredentialTokens, setLibFilesArchives, start, submitLauncher, supportsConfigurationJobXML

Methods inherited from class org.apache.oozie.action.ActionExecutor

convertException, disableInit, enableInit, getActionDir, getActionDirPath, getActionSignal, getActionYarnTag, getMaxRetries, getOozieConf, getOozieRuntimeDir, getOozieSystemId, getRetryInterval, getRetryPolicy, getType, registerError, resetInitInfo, setMaxRetries, setRetryInterval, setRetryPolicy

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

SPARK_MAIN_CLASS_NAME

public static final String SPARK_MAIN_CLASS_NAME

See Also:

Constant Field Values

SPARK_MASTER

public static final String SPARK_MASTER

See Also:

Constant Field Values

SPARK_MODE

public static final String SPARK_MODE

See Also:

Constant Field Values

SPARK_OPTS

public static final String SPARK_OPTS

See Also:

Constant Field Values

SPARK_DEFAULT_OPTS

public static final String SPARK_DEFAULT_OPTS

See Also:

Constant Field Values

SPARK_JOB_NAME

public static final String SPARK_JOB_NAME

See Also:

Constant Field Values

SPARK_CLASS

public static final String SPARK_CLASS

See Also:

Constant Field Values

SPARK_JAR

public static final String SPARK_JAR

See Also:

Constant Field Values

MAPRED_CHILD_ENV

public static final String MAPRED_CHILD_ENV

See Also:

Constant Field Values

Constructor Detail

SparkActionExecutor

public SparkActionExecutor()

Method Detail

getLauncherClasses

public List<Class<?>> getLauncherClasses()

Overrides:

getLauncherClasses in class JavaActionExecutor

getDefaultShareLibName

protected String getDefaultShareLibName(org.jdom.Element actionXml)

Return the sharelib name for the action.

Overrides:

getDefaultShareLibName in class JavaActionExecutor

Parameters:

actionXml -

Returns:

returns spark.

addActionSpecificEnvVars

protected void addActionSpecificEnvVars(Map<String,String> env)

Description copied from class: JavaActionExecutor

Adds action-specific environment variables. Default implementation is no-op. Subclasses should override this method if necessary.

Overrides:

addActionSpecificEnvVars in class JavaActionExecutor

Parameters:

env - action specific environment variables stored as Map of key and value.

getLauncherMain

protected String getLauncherMain(org.apache.hadoop.conf.Configuration launcherConf,
                                 org.jdom.Element actionXml)

Overrides:

getLauncherMain in class JavaActionExecutor

getShareLibFilesForActionConf

public String[] getShareLibFilesForActionConf()

Overrides:

getShareLibFilesForActionConf in class JavaActionExecutor