Subprocess

DIRAC Wrapper to execute python and system commands with a wrapper, that might set a timeout. 3 FUNCTIONS are provided:

  • shellCall( iTimeOut, cmdSeq, callbackFunction = None, env = None ): it uses subprocess.Popen class with “shell = True”. If cmdSeq is a string, it specifies the command string to execute through the shell. If cmdSeq is a sequence, the first item specifies the command string, and any additional items will be treated as additional shell arguments.

  • systemCall( iTimeOut, cmdSeq, callbackFunction = None, env = None ): it uses subprocess.Popen class with “shell = False”. cmdSeq should be a string, or a sequence of program arguments.

    stderr and stdout are piped. callbackFunction( pipeId, line ) can be defined to process the stdout (pipeId = 0) and stderr (pipeId = 1) as they are produced

    They return a DIRAC.ReturnValue dictionary with a tuple in Value ( returncode, stdout, stderr ) the tuple will also be available upon timeout error or buffer overflow error.

  • pythonCall( iTimeOut, function, *stArgs, **stKeyArgs ) calls function with given arguments within a timeout Wrapper should be used to wrap third party python functions

class DIRAC.Core.Utilities.Subprocess.Subprocess(timeout=False, bufferLimit=52428800)

Bases: object

class Subprocess
__init__(timeout=False, bufferLimit=52428800)

c’tor

Parameters:

  • timeout (int) – timeout in seconds

  • bufferLimit (int) – buffer size, default 5MB

changeTimeout(timeout)

set the time out limit to :timeout: seconds

Parameters:

timeout (int) – time out in seconds

getChildPID()

child pid getter

killChild(recursive=True)

kill child process

FIXME: this can easily be rewritten just using this recipe: https://psutil.readthedocs.io/en/latest/index.html#kill-process-tree

Parameters:

recursive (boolean) – flag to kill all descendants

pythonCall(function, *stArgs, **stKeyArgs)

call python function :function: with :stArgs: and :stKeyArgs:

systemCall(cmdSeq, callbackFunction=None, shell=False, env=None)

system call (no shell) - execute :cmdSeq:

class DIRAC.Core.Utilities.Subprocess.Watchdog(func, args=None, kwargs=None)

Bases: object

timeout watchdog decorator

__init__(func, args=None, kwargs=None)

c’tor

run_func(s_ok_error)

subprocess target

Parameters:

pipe (Pipe) – pipe used for communication

watchdog()

watchdog thread target

DIRAC.Core.Utilities.Subprocess.getChildrenPIDs(ppid, foreachFunc=None)
Get all children recursively for a given ppid.

Optional foreachFunc will be executed for each children pid

DIRAC.Core.Utilities.Subprocess.pythonCall(timeout, function, *stArgs, **stKeyArgs)

Use SubprocessExecutor class to execute function with provided arguments, with a timeout wrapper.

DIRAC.Core.Utilities.Subprocess.shellCall(timeout, cmdSeq, callbackFunction=None, env=None, bufferLimit=52428800)

Use SubprocessExecutor class to execute cmdSeq (it can be a string or a sequence) with a timeout wrapper, cmdSeq it is invoque by /bin/sh

DIRAC.Core.Utilities.Subprocess.systemCall(timeout, cmdSeq, callbackFunction=None, env=None, bufferLimit=52428800)

Use SubprocessExecutor class to execute cmdSeq (it can be a string or a sequence) with a timeout wrapper, it is executed directly without calling a shell