qcg.pilotjob.iterscheduler module

The definition of iteration resources schedulers.

The role of iteration resources scheduler is to based on single iteration resource requirements described as a minimum number of resources and number of available resources in allocation, assign exact number of resources in order to optimize resources usage. Therefore the job’s resource requirements do not have to be changed for different allocations. The resource requirements can apply to both: number of cores and number of nodes specifications.

class qcg.pilotjob.iterscheduler.IterScheduler

Bases: object

Iteration resources schedulers utility class.

classmethod get_scheduler(name)

Return scheduler with given name.

Parameters

name (str) – scheduler name

Returns

scheduler instance with given name, or if no such scheduler is available the default implementation

(DefaultScheduler)

static get_exact_iter_plan(iter_plan, exact)

Replace range style resource requirements with the exact one.

Remove max, min, scheduler from the resource requirements and place as exact value the given value

Parameters

  • iter_plan (joblist.ResourceSize) – the instance of resource requirements to modify

  • exact (int) – the exact value for resource requirements

Returns

the cloned and modified version of resource requirements

class qcg.pilotjob.iterscheduler.MaximumIters(job_resources, iterations, avail_resources, **params)

Bases: object

The iteration resource scheduler for maximizing resource usage.

The maximum-iters iteration resource scheduler is trying to launch as many iterations in the same time on all available resources. In case where number of iterations exceeds the number of available resources, the ‘maximum-iters’ schedulers splits iterations into ‘steps’ minimizing this number, and allocates as many resources as possible for each iteration inside ‘step’.

Create maximum-iters iteration resource scheduler instance.

Parameters

  • job_resources (joblist.ResourceSize) – job’s resource requirements

  • iterations (int) – number of iterations

  • avail_resources (int) – number of available resources

  • params (dict) – additional scheduler parameters

SCHED_NAME = 'maximum-iters'
generate()

Generate exact job’s resource requirements for next iteration.

Yields

exact resource requirements for following iterations

Raises

InvalidRequest – when parameter max is used in resource description

class qcg.pilotjob.iterscheduler.SplitInto(job_resources, iterations, avail_resources, **params)

Bases: object

The iteration resource scheduler for partitioning available resources.

This simple iteration resource scheduler splits all available resources in given partitions, and each iteration will be executed inside whole single partition.

Create split-into iteration resource scheduler instance.

The number of partitions is taken as value of parts key of params dictionary if exists, and number of iterations in other case.

Parameters

  • job_resources (joblist.ResourceSize) – job’s resource requirements

  • iterations (int) – number of iterations

  • avail_resources (int) – number of available resources

  • params (dict) – additional scheduler parameters

SCHED_NAME = 'split-into'
generate()

Generate exact job’s resource requirements for next iteration.

Yields

exact resource requirements for following iterations

Raises

InvalidRequest – when parameter max is used in resource description

qcg.pilotjob.iterscheduler.DefaultScheduler

alias of MaximumIters