qcg.pilotjob.utils.proc_traces module

class qcg.pilotjob.utils.proc_traces.ProcTraces(paths, ignore_errors=True)

Bases: object

Analyze process traces.

Atributes:

paths (list(str)) - paths with traces files ignore_errors (bool) - raise exception when error occur nodes_procs (dict(str,dict)) - a dictionary with node names as keys and process data as values

each process data is dictionary with pid (as string) as a key and dictionary of attributes as value

read()

Read process traces from log files.

get_process(job_pid, node_name=None)

Find process data with given pid.

If node_name is not specified, and there are more than single process with given pid on all nodes the first encountered process is returned.

Parameters

  • job_pid (str,int) –

  • node_name (str) – are searched

Returns

process data

Return type

dict

childs_on_other_nodes(process, slurm_step_id=None)

Find child process on other nodes not explicitely linked. For example when launching openmpi application where some of the instances will be launched on other nodes, mpirun should launch ‘orted’ deamon (via slurm) with identifier. When we find that such process has been created, we can look for ‘orted’ processes on other nodes with the same identifier.

Parameters

  • process (dict) –

  • slurm_step_id (str) - a slurm's step identifier (optional) –

Returns

list of process identifiers that has been run on other nodes

Return type

list(str)

process_iterator(pid, node_name=None)

Generator which iterates on process and it’s childs.

Parameters

  • pid (str,int) –

  • node_name (str) –

Returns

a pair with process data and level of nesting in tree

Return type

dict, int