PoolComputingElement

The Computing Element to run several jobs simultaneously in separate processes managed by a ProcessPool

class DIRAC.Resources.Computing.PoolComputingElement.PoolComputingElement(ceUniqueID)

Bases: DIRAC.Resources.Computing.ComputingElement.ComputingElement

__init__(ceUniqueID)

Standard constructor.

available(jobIDList=None)

This method returns the number of available slots in the target CE. The CE instance polls for waiting and running jobs and compares to the limits in the CE parameters.

Parameters:jobIDList (list) – list of already existing job IDs to be checked against
finalizeJob(taskID, result)

Finalize the job by updating the process utilisation counters

Parameters:
  • taskID (int) – local PoolCE task ID
  • result (dict) – result of the job execution
getCEStatus(jobIDList=None)

Method to return information on running and pending jobs.

Returns:dictionary of numbers of jobs per status
getDescription()

Get CE description as a dictionary

getParameterDict()

Get the CE complete parameter dictionary

getProcessorsInUse()

Get the number of currently allocated processor cores

Returns:number of processor cores
initializeParameters()

Initialize the CE parameters after they are collected from various sources

isProxyValid(valid=1000)

Check if the stored proxy is valid

isValid()

Check the sanity of the Computing Element definition

loadBatchSystem()

Instantiate object representing the backend batch system

mandatoryParameters = []
monitorProxy(pilotProxy, payloadProxy)

Monitor the payload proxy and renew as necessary.

Parameters:
  • pilotProxy (str) – location of the pilotProxy
  • payloadProxy (str) – location of the payloadProxy
sendOutput(stdid, line)

Callback function such that the results from the CE may be returned.

setCPUTimeLeft(cpuTimeLeft=None)

Update the CPUTime parameter of the CE classAd, necessary for running in filling mode

setParameters(ceOptions)

Add parameters from the given dictionary overriding the previous values

Parameters:ceOptions (dict) – CE parameters dictionary to update already defined ones
setProxy(proxy, valid=0)

Set proxy for this instance

submitJob(executableFile, proxy, **kwargs)

Method to submit job.

Parameters:
  • executableFile (str) – location of the executable file
  • proxy (str) – payload proxy
Returns:

S_OK/S_ERROR of the result of the job submission

writeProxyToFile(proxy)

CE helper function to write a CE proxy string to a file.

DIRAC.Resources.Computing.PoolComputingElement.executeJob(executableFile, proxy, taskID, **kwargs)

wrapper around ce.submitJob: decides which CE to use (Sudo or InProcess)

Parameters:
  • executableFile (str) – location of the executable file
  • proxy (str) – proxy file location to be used for job submission
  • taskID (int) – local task ID of the PoolCE
Returns: