ProActive Workflows & Scheduling

The following terms are used throughout the documentation:

In order to use Scheduler for executing various computations, one needs to write the execution definition also known as the Workflow definition. A workflow definition is an XML file that adheres to XML schema for ProActive Workflows.

It specifies a number of XML tags for specifying execution steps, their sequence and dependencies. Each execution step corresponds to a task which is the smallest unit of execution that can be performed on a computation resources. There are several types of tasks which caters different use cases.

ProActive Scheduler currently supports three main types of tasks:

For instance, a script task can be used to execute an inline script definition or a script file as an execution step whereas a native task can be used to execute a native executable file.

One can use ProActive Workflow Studio to create and submit workflows graphically . You can simply drag-and-drop various task constructs and draw their dependencies to form complex jobs. It also provides various flow control widgets such as conditional branch, loop, replicate etc to construct complex workflows.

In this tasks graph, we see that task 4 is preceded by task 1, that means the Scheduler waits the end of task 1 execution before launching task 4. In a more concrete way, task 1 could be the calculation of a part of the problem to solve, and task 4 takes the result provided by task 1 and compute another step of the calculation. We introduce here the concept of result passing between tasks. This relation is called a Dependency, and we say that task 4 depends on task 1.

We see that task 1, 2 and 3 are not linked, so these three tasks can be executed in parallel, because there are independent from each other.

The task graph is defined by the user at the time of workflow creation, but can also be modified during the job execution by control flow actions such as Replicate.

A finished job contains the results and logs of each task. Upon failure, a task can be restarted automatically or cancelled.

The quickstart tutorial on try.activeeon.com shows you how to build a simple workflow using script tasks.

In order for Docker tasks to work, the Node must have Docker and Docker Compose installed, please refer to the official Docker documentation to see how to install Docker and Docker Compose.

A Docker task expects the content of a Docker Compose file inside the Script section. You can find out how to write Docker Compose files with the official Docker Compose documentation.

To get started: a simple Docker Compose example is explained below.

The content of the Script section (equal to the content of a Docker Compose file) is: --- helloworld: image: busybox command: echo "Hello ProActive" --- The above example describes a container which is called 'helloworld'. That container is created from a busybox image, which will run the command 'echo "Hello ProActive"'

Using native tasks you can easily reuse existing applications and embed them in a workflow. The Scheduler lets you define a native task that takes the name of the executable and list of parameters. Once the executable is wrapped as a task you can easily leverage some of the workflow constructs to run your executable in parallel.

You can find an example of such integration in this XML workflow or you can also build one yourself using the Workflow Studio.

Native application by nature can be tied to a given operating system so if you have different nodes at your disposal, you might need to select a suitable node to run your native task. This can be achieved using selection script.

Proactive Scheduler supports tasks written in languages other than Java. The currently supported dynamic languages are Groovy, Python, Ruby and Javascript.

Native scripts can also be executed using a script task. Currently Bash and CMD scripts are supported, simply set the language attribute to bash or cmd and type your script in the workflow.

You can easily embed small scripts in your workflow. The nice thing is that the workflow will be self-contained and will not require to compile your code before executing. However, we recommend that you keep these scripts smalls to keep the workflow easy to understand.

Please note that the script execution relies on Java implementation of the cited languages thus it may come with some limitations.

You can find an example of a script task in this XML workflow or you can also build one yourself using the Workflow Studio. The quickstart tutorial relies on script task and is a nice introduction to workflows.

Scripts can also be used to decorate tasks with specific actions, we support pre, post, clean and selection scripts.

MPI is often used in the area of parallel computing. The Scheduler integrates with MPI with the concept of multi node tasks. This particular task will acquire several nodes and will expose these nodes to the MPI environment.

Applications built with MPI are often executed using the mpirun command. It accepts several parameters to choose how many CPUs and how many machines will be used. One particular parameter is the machine file that is built by the Scheduler when using multi node tasks. Here is how an MPI application invocation would look like when executed with the Scheduler:

The variable $variables_PA_NODESFILE contains the path to a machine file created by the Scheduler that will be similar to:

Meaning that myApplication will use 2 CPUs on compute-host and 1 CPU on another-host. You can also use $variables_PA_NODESNUMBER to retrieve the number of acquired nodes.

You can find an example of a native task in this XML workflow or you can also build one yourself using the Workflow Studio.

To achieve the best performance for MPI applications, nodes can be selected taking into account their network topology. One might want to select nodes that are as close to each other as possible to obtain better network performance or nodes on different hosts to split the I/O load. Refer to Topology Types for more details.

To run a workflow, the user submits it to ProActive Scheduler. A verification is performed to ensure the well-formedness of the workflow. Next, a job is created and inserted into the pending queue and waits until free resources become available. Once the required resources are provided by the ProActive Resource Manager, the job is started. Finally, once the job is finished, it goes to the queue of finished jobs where its result can be retrieved.

You can submit a workflow to the Scheduler using the Workflow Studio, the Scheduler Web Interface or command line tools. For advanced users we also expose REST and JAVA APIs.

Variables used for data access and propagation exist in three flavors:

Their usage depends of the context (see Variables quick reference).

Once a Job is terminated, it is possible to get its result. You can only get the result of the job that you own. A Job result is the collection of the tasks' results and you can retrieve it from either Java API or command line (it will be converted to a string).

Job results are also available in the Scheduler Web Interface in the Preview tab.

Dependencies can be set between tasks in a TaskFlow job. It provides a way to execute your tasks in a specified order, but also to forward result of an ancestor task to its children as parameter. Dependency between tasks is then both a temporal dependency and a data dependency.

Dependencies between tasks can be added either in ProActive Workflow Studio or simply in workflow XML as shown below:

The replication allows the execution of multiple tasks in parallel when only one task is defined and the number of tasks to run could change.

The branch construct provides the ability to choose between two alternative task flows, with the possibility to merge back to a common flow.

The loop provides the ability to repeat a set of tasks.

Workflows often relies on task blocks. Task blocks are defined by pairs of start and end tags.

Task blocks are very similar to the parenthesis of most programming languages: anonymous and nested start/end tags. The only difference is that a parenthesis is a syntactical information, whereas task blocks are semantic.

The role of task blocks is to restrain the expressiveness of the system so that a workflow can be statically checked and validated. A treatment that can be looped or iterated will be isolated in a well-defined task block.

To perform a control flow action such as if, replicate or loop, a Control Flow Script is executed on the ProActive node. This script takes the result of the task as input; meaning a Java object if it was a Java or Script task, or nothing if it was a native task.

The script is executed on the ProActive node, just after the task’s executable. If the executable is a Java Executable and returns a result, the variable result will be set in the script’s environment so that dynamic decisions can be taken with the task’s result as input. Native Java objects can be used in a Groovy script.

Similarly to how parameters are passed through the result variable to the script, the script needs to define variables specific to each action to determine what action the script will lead to.

The assigned value needs be a strictly positive integer.

The assigned value needs to be the string value if or else.

The assigned value needs to be a boolean.

Failure to set the required variables or to provide a valid control flow script will not be treated gracefully and will result in the failure of the task.

When Control Flow actions such as replicate or loop are performed, some tasks are replicated. To be able to identify replicated tasks uniquely, each replicated task has an iteration index, replication index, and a tag. In addition to help to identify uniquely, these tags are useful to filter tasks by iterations for example.

An Embarrassingly Parallel problem is a problem that is easy to split into smaller independent tasks. With ProActive Scheduler you can tackle this type of problem with the Replicate construct.

The Advanced workflows is an example of an embarrassingly parallel problem where the computation is easily distributed across ProActive Nodes.

A selection script provides an ability for the Scheduler to execute tasks on particular ProActive nodes. E.g. you can specify that a task must be executed on a Unix/Linux system.

A selection script is always executed before the task itself on any candidate node: the execution of a selection script must set the boolean variable selected, that indicates if the candidate node is suitable for the execution of the associated task.

A Java helper org.ow2.proactive.scripting.helper.selection.SelectionUtils is provided for allowing user to simply make some kind of selections. Some script samples are available in PROACTIVE_HOME/samples/scripts/selection.

The following example selects only nodes running on Windows:

Another functionality is the possibility to define pre and post scripts. For a given task (Java, Script or Native task), it is possible to launch a script before and after its execution. This possibility can be useful to copy files to a node, or clean a directory before or after task execution, for example.

This is a way to separate from business code the preparation of execution environment and its cleaning. A good example is a script that removes files from a specific directory once the task is finished.

It is possible to start a task under the job owner if the system is configured for that purpose. There are 2 possible ways to run a task under user account (in any case, the administrator should have set computing hosts to authorize one of the 2 methods) :

User must first create a credential containing this key :

This command will create a new credentials with username as login, userpwd as password, using Scheduler public key at config/authentication/keys/pub.key for credentials encryption and using the private SSH key at path/to/private/sshkey provided by administrator. The new credential will be stored in myCredentials.cred

Once created, user must connect the Scheduler using this credential. Then, in order to execute your task under your account set runAsMe=true in the task.

To create a multi-nodes task often used for MPI applications you need to add a parallel environment to the task. Parallel environment describes how many nodes are needed for a task as well as where these nodes should be located. For example if you’d like to run 4 processes in parallel it a scope of one task you should specify it in your task

It’s possible to have errors when your workflow is executed. ProActive Scheduler provides several mechanisms to deal with exceptional situations in your workflows.

To understand if a task is successfully finished or not the Scheduler checks its exit code (for native tasks) or exceptions (for Java/script tasks). Then, it decides what to do next according to the following parameters:

ProActive Scheduler supports the execution of time-based tasks or cron tasks. The users can assign a cron expression to the loop variable of a Loop in a ProActive Job. All tasks which belong that Loop will be executed iteratively and indefinitely. The start time of the next iteration is the next point of time specified by the cron expression.

Using the Scheduler Web Interface, it is possible to access the display of the ProActive Node running a given task. First remote visualization must be activated via the Scheduler configuration files (set novnc.enabled to true).

The task that you want to visualize must output a special string using the following format: PA_REMOTE_CONNECTION;JOB_ID;TASK_ID;vnc;HOSTNAME:PORT where JOB_ID must be the current job id, TASK_ID the current task id and HOSTNAME:PORT the hostname and port where the VNC server runs.

It is the task’s responsibility to print the string. Here is a sample script that starts a Xvnc server and runs a graphical application. It can be used in a native task for instance.

In the Scheduler Web Interface, select the running job and use the Preview tab to enable remote visualization.

A Bucket is a collection of Workflows that can be shared between users. The default settings provide the following Buckets:

Listing the catalog for existing Buckets is done using the following HTTP request:

It is also possible to list all Workflows inside a particular Bucket using its bucket_id:

Then, we can fetch a particular Workflow based on its workflow_id:

The Workflow Catalog provides versioning of workflows. You can list all the revisions of a particular Workflow using the HTTP request:

You can fetch a Workflow detailed information using its workflow_id and revision_id:

A Workflow Revision is created from a ProActive XML Workflow definition. Beside from the id, name and project_name of the Workflow, a few more information are indexed from the XML content such as Generic Information and Variables.

More detailed information on how to create or modify Workflows and Buckets are available in the Admin guide.

You can search or filter Workflows and Workflow Revisions by means of Workflow Catalog Query Language (WCQL).

Search on the latest versions of Workflows contained in a particular Bucket:

Search through all versions of a particular Workflow:

Tasks executed by ProActive Scheduler might need to use a password or another kind of credentials to acces third-party services or applications. For example a user might wish to pass his MySQL password to Native Task which runs a mysql command to dump a database.

In such cases it is often desireable to store the credentials separately from the Workflow definition. ProActive Scheduler provides such facility.

Third-party credentials in ProActive Scheduler are user-defined key-value pairs of strings stored on the server side in encrypted form. The credentals are accessible in the tasks executed by the user via API (Script and Java Tasks) or variable substitution (arguments of Native Tasks and parameters of Java Tasks).

Methods to add or remove the credentials and to list the stored credential keys are exposed in both Java and REST APIs.

End users can manage their credentials using the Scheduler Web Interface (Portal → Manage third-party credentials) or the Command Line Interface.

A Docker fork execution environment executes a JVM inside a Docker container. Hence the task which is executes in the Docker fork execution environment can be customized by the JVM settings and by the tool-set and isolation provided by Docker containers.

Workflows are written in XML and we provide a XSD grammar to help you write and validate them. The Workflows XSD is available here and you can also find the documentation here.

This section describes useful variables, their scope, and their usage.

Note that for script tasks, syntax for accessing maps (like variables and credentials) is language dependent.

For Groovy:

For Python:

For Bash:

ProActive Scheduler supports the following nodes topologies for multi-nodes tasks.

For this task the scheduler will try to find a host with 4 nodes. If there is no such a host another one will be used with a bigger capacity (if exists). In this case extra nodes on this host will be also occupied by the task but will not be used for the execution.

For this task the scheduler will try to find 4 nodes on a set of hosts. If there is no such a set which gives you exactly 4 nodes another one will be used with a bigger capacity (if exists). In this case extra nodes on this host will be also occupied by the task but will not be used for the execution.

For this task the scheduler will try to find 4 hosts with one node on each. If number of "single node" hosts are not enough hosts with bigger capacity will be selected. In this case extra nodes on this host will be also occupied by the task but will not be used for the execution.

The ProActive client allows to interact with the Scheduler and Resource Manager. The client has an interactive mode started if you do not provided any command.

The client usage is also available using the -h parameter as shown below: