qcg.pilotjob.executionschema module¶

class qcg.pilotjob.executionschema.ExecutionSchema(resources, config)¶

Bases: object

Method of executing job. Currently two methods are supported:

SlurmExecuition - jobs are run via ‘srun’ command DirectExecution - jobs are run as a normal processes

resources¶

available resources and their origin

Type:	Resources

config¶

QCG-PilotJob configuration

Type:	dict

Initialize instance.

Parameters:	resources (Resources) – available resources config (dict) – QCG-PilotJob configuration

classmethod get_schema(resources, config)¶

Create and return suitable instance of execution schema.

Currently decision about type of execution schema is taken based on origin of resources - if QCG-PilotJob manager is run inside Slurm allocation, the SlurmExecution is selected. In other cases the DirectExecution schema is instantiated.

Parameters:	resources (Resources) – available resources config (dict) – QCG-PilotJob configuration
Returns:	instance of execution schema
Return type:	ExecutionSchema

preprocess(ex_job)¶

Preprocess job iteration description before launching. This method might be implemented in child classes.

Args: ex_job (ExecutionJob): execution job iteration data

get_env_opts()¶: Return options for environment instances. This method might be implemented in child classes

class qcg.pilotjob.executionschema.SlurmExecution(resources, config)¶

Bases: qcg.pilotjob.executionschema.ExecutionSchema

The Slurm execution schema. The jobs are launched with srun command.

Initialize instance.

Parameters:	resources (Resources) – available resources config (dict) – QCG-PilotJob configuration

EXEC_NAME = 'slurm'¶

JOB_MODELS = {'default': '_preprocess_default', 'intelmpi': '_preprocess_intelmpi', 'openmpi': '_preprocess_openmpi', 'srunmpi': '_preprocess_srunmpi', 'threads': '_preprocess_threads'}¶

preprocess(ex_job)¶

“Preprocess job iteration description before launching. Prepare job iteration execution arguments.

Args: ex_job (ExecutionJob): execution job iteration data

get_env_opts()¶

Return options for environment instances.

Set environments to not create ‘hostfile’

class qcg.pilotjob.executionschema.DirectExecution(resources, config)¶

Bases: qcg.pilotjob.executionschema.ExecutionSchema

Directly execute job iteration without any proxy commands.

Initialize instance.

Parameters:	resources (Resources) – available resources config (dict) – QCG-PilotJob configuration

EXEC_NAME = 'direct'¶

preprocess(ex_job)¶

“Preprocess job iteration description before launching. Prepare job iteration execution arguments.

Args: ex_job (ExecutionJob): execution job iteration data

get_env_opts()¶

Return options for environment instances.

Set environments to not create ‘hostfile’