qcg.pilotjob.utils.proc_traces module

class qcg.pilotjob.utils.proc_traces.ProcTraces(paths, ignore_errors=True)

Bases: object

Analyze process traces.

Atributes:

paths (list(str)) - paths with traces files ignore_errors (bool) - raise exception when error occur nodes_procs (dict(str,dict)) - a dictionary with node names as keys and process data as values

each process data is dictionary with pid (as string) as a key and dictionary of attributes as value
read()

Read process traces from log files.

get_process(job_pid, node_name=None)

Find process data with given pid.

If node_name is not specified, and there are more than single process with given pid on all nodes the first encountered process is returned.

Parameters:
  • job_pid (str,int) –
  • node_name (str) – are searched
Returns:

process data

Return type:

dict

childs_on_other_nodes(process, slurm_step_id=None)

Find child process on other nodes not explicitely linked. For example when launching openmpi application where some of the instances will be launched on other nodes, mpirun should launch ‘orted’ deamon (via slurm) with identifier. When we find that such process has been created, we can look for ‘orted’ processes on other nodes with the same identifier.

Parameters:
  • process (dict) –
  • slurm_step_id (str) - a slurm's step identifier (optional) –
Returns:

list of process identifiers that has been run on other nodes

Return type:

list(str)

process_iterator(pid, node_name=None)

Generator which iterates on process and it’s childs.

Parameters:
  • pid (str,int) –
  • node_name (str) –
Returns:

a pair with process data and level of nesting in tree

Return type:

dict, int