org.ow2.proactive.resourcemanager.nodesource.infrastructure

Class BatchJobInfrastructure

java.lang.Object

org.ow2.proactive.resourcemanager.nodesource.infrastructure.InfrastructureManager

org.ow2.proactive.resourcemanager.nodesource.infrastructure.BatchJobInfrastructure

All Implemented Interfaces:

Serializable, NodeSourcePlugin

Direct Known Subclasses:

GenericBatchJobInfrastructure, LSFInfrastructure, PBSInfrastructure

public abstract class BatchJobInfrastructure
extends InfrastructureManager

This class implements the basics common operations that can be performed on a resource manager (i.e batching job system like PBS, Torque, LSF...). It ensures that the internal number of nodes is coherent regarding acquisition a removal requests after timeouts occur. For instance, even if a getDeleteJobCommand() fails, the node is removed from the core anyway, if the SSH command to the frontend or getSubmitJobCommand() fail, this IM ensure that after the nodeTimeOut occurs no more nodes will be registered (this IM maintains an internal "black list").

Service Providers have to implements 3 methods:

• getBatchinJobSystemName(): The name of the target resource manager, PBS, Torque, LSF are such examples. The returned string is not really significant, it is only used to build meaningful nodes' name or for logging.
• getDeleteJobCommand(): The command required to delete a job on the target resource manager.
• getSubmitJobCommand(): The command required to submit a new job.

See Also:

Serialized Form

Nested Class Summary

Nested classes/interfaces inherited from class org.ow2.proactive.resourcemanager.nodesource.infrastructure.InfrastructureManager

InfrastructureManager.PersistedInfraVariablesHandler<T>, InfrastructureManager.RMDeployingNodeAccessor

Field Summary

Fields

Modifier and Type	Field and Description
protected String	javaOptions Additional java options to append to the command executed on the remote host
protected String	javaPath Path to the Java executable on the remote hosts
protected int	maxNodes maximum number of nodes this infrastructure can ask simultaneously to the Job Batching system
protected int	nodeTimeOut time out after which one nodes are not expected to register anymore.
protected File	rmCredentialsPath Path to the credentials file user for RM authentication
protected String	schedulingPath Path to the Scheduling installation on the remote hosts
protected String	serverName name of the server on which the job batching software is running.
protected String	sshOptions ShhClient options (@see SSHClient)
protected String	submitJobOpt options for the submit job command executed on serverName

Fields inherited from class org.ow2.proactive.resourcemanager.nodesource.infrastructure.InfrastructureManager

ELASTIC, logger, nodeSource, persistedInfraVariables, readLock, RM_URL_KEY, writeLock

Constructor Summary

Constructors

Constructor and Description
BatchJobInfrastructure()

Method Summary

Modifier and Type	Method and Description
void	acquireAllNodes() Acquires as much nodes as possible, making one distinct reservation per node
void	acquireNode() Acquires a single node through pbs
void	configure(Object... parameters) Configures this infrastructure manager parameters[0] = java path parameters[1] = ssh options parameters[2] = scheduling path parameters[3] = java options parameters[4] = max node parameters[5] = node timeout parameters[6] = scheduler server name parameters[7] = PA scheduler credentials parameters[8] = submit job options
protected abstract String	extractSubmitOutput(String output) Parses the submit (getSubmitJobCommand()) command output to extract job's ID.
protected abstract String	getBatchinJobSystemName() Return a string to identify the type of the target Batching Job System.
protected abstract String	getDeleteJobCommand() To be able to get from implementations the command that will be used to delete a job
String	getDescription()
Map<Integer,String>	getSectionDescriptions()
protected abstract String	getSubmitJobCommand() To be able to get from implementations the command that will be used to submit a new Job
protected void	initializePersistedInfraVariables() This method should initialize a value in the runtime variables map for all the runtime variables that will be used in the class.
void	notifyAcquiredNode(org.objectweb.proactive.core.node.Node node) Notifies the implementation of the infrastructure manager that a new node has been registered.
protected void	notifyDeployingNodeLost(String pnURL) This method is called by Infrastructure Manager in case of a pending node removal.
void	notifyDownNode(String nodeName, String nodeUrl, org.objectweb.proactive.core.node.Node node) Define what needs to be done in an infrastructure implementation when a node is detected as down.
void	removeNode(org.objectweb.proactive.core.node.Node node) Removes the node from the resource manager.
void	shutDown() Notify this infrastructure it is going to be shut down along with its nodesource.
String	toString()

Methods inherited from class org.ow2.proactive.resourcemanager.nodesource.infrastructure.InfrastructureManager

acquireAllNodes, acquireNodes, acquireNodes, addDeployingNode, addDeployingNodeWithLockAndPersist, addLostNodeWithLockAndPersist, addMultipleDeployingNodes, buildDeployingNodeURL, checkAllNodesAreAcquiredAndDo, checkNodeIsAcquiredAndDo, declareDeployingNodeLost, getDefaultCommandLineBuilder, getDeployingAndLostNodes, getDeployingNodesWithLock, getDeployingOrLostNode, getEmptyCommandLineBuilder, getLostNodesWithLock, getMeta, getPersistedDeployingNodesUrl, getPersistedInfraVariable, getPersistedLostNodesUrl, getRmUrl, internalConfigure, internalNotifyDownNode, internalRegisterAcquiredNode, internalRemoveDeployingNode, internalRemoveNode, internalShutDown, isShutDown, isUsingDeployingNode, multipleDeclareDeployingNodeLost, onDownNodeReconnection, persistInfrastructureVariables, reconfigure, recoverPersistedInfraVariables, removeDownNodePriorToNotify, searchForNotAcquiredRmNode, setNodeSource, setPersistedInfraVariable, setPersistedNodeSourceData, setRmDbManager, setRmUrl, update, updateDeployingNodeDescription

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Field Detail

javaPath

protected String javaPath

Path to the Java executable on the remote hosts

sshOptions

protected String sshOptions

ShhClient options (@see SSHClient)

schedulingPath

protected String schedulingPath

Path to the Scheduling installation on the remote hosts

javaOptions

protected String javaOptions

Additional java options to append to the command executed on the remote host

maxNodes

protected int maxNodes

maximum number of nodes this infrastructure can ask simultaneously to the Job Batching system

nodeTimeOut

protected int nodeTimeOut

time out after which one nodes are not expected to register anymore. When this time expires, nodes which register are in reponse of the previous "acquireNode" request are discarded. This time out is also used to time out submit job and delete job command's exit status

serverName

protected String serverName

name of the server on which the job batching software is running. will be contacted using ssh

rmCredentialsPath

protected File rmCredentialsPath

Path to the credentials file user for RM authentication

submitJobOpt

protected String submitJobOpt

options for the submit job command executed on serverName

Constructor Detail

BatchJobInfrastructure

public BatchJobInfrastructure()

Method Detail

acquireAllNodes

public void acquireAllNodes()

Acquires as much nodes as possible, making one distinct reservation per node

Specified by:

acquireAllNodes in class InfrastructureManager

acquireNode

public void acquireNode()

Acquires a single node through pbs

Specified by:

acquireNode in class InfrastructureManager

configure

public void configure(Object... parameters)

Configures this infrastructure manager parameters[0] = java path parameters[1] = ssh options parameters[2] = scheduling path parameters[3] = java options parameters[4] = max node parameters[5] = node timeout parameters[6] = scheduler server name parameters[7] = PA scheduler credentials parameters[8] = submit job options

Specified by:

configure in class InfrastructureManager

Parameters:

parameters - of the infrastructure manager

notifyAcquiredNode

public void notifyAcquiredNode(org.objectweb.proactive.core.node.Node node)
                        throws RMException

Notifies the implementation of the infrastructure manager that a new node has been registered. If this method throws an exception, the node registration will be discarded. This method is always called if implementor doesn't use deploying nodes (no call to InfrastructureManager.addDeployingNode(String, String, String, long) was made), and is called only for deploying nodes for which one no timeout occurred.

Specified by:

notifyAcquiredNode in class InfrastructureManager

Parameters:

node - the newly registered node

Throws:

RMException - if the implementation does not approve the node acquisition request

notifyDeployingNodeLost

protected void notifyDeployingNodeLost(String pnURL)

This method is called by Infrastructure Manager in case of a pending node removal. We take advantage of it to specify to the remote process control loop of the removal. This one will then exit.

Overrides:

notifyDeployingNodeLost in class InfrastructureManager

Parameters:

pnURL - the deploying node's URL for which one the timeout occurred.

notifyDownNode

public void notifyDownNode(String nodeName,
                           String nodeUrl,
                           org.objectweb.proactive.core.node.Node node)
                    throws RMException

Description copied from class: InfrastructureManager

Define what needs to be done in an infrastructure implementation when a node is detected as down.

Specified by:

notifyDownNode in class InfrastructureManager

node - the ProActive Programming Node that is down.

Throws:

RMException - if any problems occurred.

removeNode

public void removeNode(org.objectweb.proactive.core.node.Node node)
                throws RMException

Removes the node from the resource manager.

Specified by:

removeNode in class InfrastructureManager

Parameters:

node - the node to release.

Throws:

RMException - if any problems occurred.

getDescription

public String getDescription()

Specified by:

getDescription in class InfrastructureManager

Returns:

short description of the IM

toString

public String toString()

Overrides:

toString in class Object

shutDown

public void shutDown()

Description copied from class: InfrastructureManager

Notify this infrastructure it is going to be shut down along with its nodesource. All necessary cleanup should be done here.

Overrides:

shutDown in class InfrastructureManager

getSubmitJobCommand

protected abstract String getSubmitJobCommand()

To be able to get from implementations the command that will be used to submit a new Job

Returns:

submit job command on the target Batching Job System.

getDeleteJobCommand

protected abstract String getDeleteJobCommand()

To be able to get from implementations the command that will be used to delete a job

Returns:

delete job command on the target Batching Job System.

getBatchinJobSystemName

protected abstract String getBatchinJobSystemName()

Return a string to identify the type of the target Batching Job System. Return's content is not really significant, it is only used to build nodes name and for logging...

Returns:

target Batching Job System's name

extractSubmitOutput

protected abstract String extractSubmitOutput(String output)

Parses the submit (getSubmitJobCommand()) command output to extract job's ID.

Parameters:

output - the submit command output

Returns:

the job's ID in case of success, empty string or null if the method is unable to compute the job's ID.

initializePersistedInfraVariables

protected void initializePersistedInfraVariables()

Description copied from class: InfrastructureManager

This method should initialize a value in the runtime variables map for all the runtime variables that will be used in the class. It is called at initialization time of the infrastructure, just before the InfrastructureManager.configure(Object...) method. This method runs with the write lock acquired.

Specified by:

initializePersistedInfraVariables in class InfrastructureManager

getSectionDescriptions

public Map<Integer,String> getSectionDescriptions()

Specified by:

getSectionDescriptions in interface NodeSourcePlugin

Overrides:

getSectionDescriptions in class InfrastructureManager