qcg.pilotjob.executionschema module

class qcg.pilotjob.executionschema.ExecutionSchema(resources, config)

Bases: object

Method of executing job. Currently two methods are supported:

SlurmExecuition - jobs are run via ‘srun’ command DirectExecution - jobs are run as a normal processes

resources

available resources and their origin

Type

Resources

config

QCG-PilotJob configuration

Type

dict

Initialize instance.

Parameters

• resources (Resources) – available resources

• config (dict) – QCG-PilotJob configuration

classmethod get_schema(resources, config)

Create and return suitable instance of execution schema.

Currently decision about type of execution schema is taken based on origin of resources - if QCG-PilotJob manager is run inside Slurm allocation, the SlurmExecution is selected. In other cases the DirectExecution schema is instantiated.

Parameters

• resources (Resources) – available resources

• config (dict) – QCG-PilotJob configuration

Returns

instance of execution schema

Return type

ExecutionSchema

preprocess(ex_job)

Preprocess job iteration description before launching. This method might be implemented in child classes.

Args

ex_job (ExecutionJob): execution job iteration data

get_env_opts()

Return options for environment instances. This method might be implemented in child classes

class qcg.pilotjob.executionschema.SlurmExecution(resources, config)

Bases: ExecutionSchema

The Slurm execution schema. The jobs are launched with srun command.

Initialize instance.

Parameters

• resources (Resources) – available resources

• config (dict) – QCG-PilotJob configuration

EXEC_NAME = 'slurm'

JOB_MODELS = {'default': '_preprocess_default', 'intelmpi': '_preprocess_intelmpi', 'openmpi': '_preprocess_openmpi', 'srunmpi': '_preprocess_srunmpi', 'threads': '_preprocess_threads'}

preprocess(ex_job)

“Preprocess job iteration description before launching. Prepare job iteration execution arguments.

Args

ex_job (ExecutionJob): execution job iteration data

get_env_opts()

Return options for environment instances.

Set environments to not create ‘hostfile’

class qcg.pilotjob.executionschema.DirectExecution(resources, config)

Bases: ExecutionSchema

Directly execute job iteration without any proxy commands.

Initialize instance.

Parameters

• resources (Resources) – available resources

• config (dict) – QCG-PilotJob configuration

EXEC_NAME = 'direct'

preprocess(ex_job)

“Preprocess job iteration description before launching. Prepare job iteration execution arguments.

Args

ex_job (ExecutionJob): execution job iteration data

get_env_opts()

Return options for environment instances.

Set environments to not create ‘hostfile’