ComputingElement

The Computing Element class is a base class for all the various types CEs. It serves several purposes:

  • collects general CE related parameters to generate CE description for the job matching

  • provides logic for evaluation of the number of available CPU slots

  • provides logic for the proxy renewal while executing jobs

The CE parameters are collected from the following sources, in hierarchy descending order:

  • parameters provided through setParameters() method of the class

  • parameters in /LocalSite configuration section

  • parameters in /LocalSite/<ceName>/ResourceDict configuration section

  • parameters in /LocalSite/ResourceDict configuration section

  • parameters in /LocalSite/<ceName> configuration section

  • parameters in /Resources/Computing/<ceName> configuration section

  • parameters in /Resources/Computing/CEDefaults configuration section

The ComputingElement objects are usually instantiated with the help of ComputingElementFactory.

The ComputingElement class can be considered abstract. 3 kinds of abstract ComputingElements can be distinguished from it:

  • Remote ComputingElement: includes methods to interact with a remote ComputingElement (e.g. HtCondorCEComputingElement, AREXComputingElement).

  • Inner ComputingElement: includes methods to locally interact with an underlying worker node. It is worth noting that an Inner ComputingElement provides synchronous submission (the submission of a job is blocking the execution until its completion). It deals with one job at a time.

  • Inner Pool ComputingElement: includes methods to locally interact with Inner ComputingElements asynchronously. It can manage a pool of jobs running simultaneously.

To configure the use of Tokens for CEs:

  • the CE is able to receive any token. Validation: ‘Tag = Token’ should be included in the CE parameters.

  • the CE is able to receive VO-specifc tokens. Validation: ‘Tag = Token:<VO>’ should be included in the CE parameters.

class DIRAC.Resources.Computing.ComputingElement.ComputingElement(ceName)

Bases: object

ComputingElement base class

__init__(ceName)

Standard constructor

available()

This method returns the number of available slots in the target CE. The CE instance polls for waiting and running jobs and compares to the limits in the CE parameters.

getCEStatus()

Method to get dynamic job information, can be overridden in sub-class.

getDescription()

Get CE description as a dictionary.

This is called by the JobAgent for the case of “inner” CEs.

initializeParameters()

Initialize the CE parameters after they are collected from various sources

isValid()

Check the sanity of the Computing Element definition

loadBatchSystem(batchSystemName)

Instantiate object representing the backend batch system

Parameters:

batchSystemName (str) – name of the batch system

sendOutput(stdid, line)

Callback function such that the results from the CE may be returned.

setCPUTimeLeft(cpuTimeLeft=None)

Update the CPUTime parameter of the CE classAd, necessary for running in filling mode

setParameters(ceOptions)

Add parameters from the given dictionary overriding the previous values

Parameters:

ceOptions (dict) – CE parameters dictionary to update already defined ones

setProxy(proxy)

Set proxy for this instance

setToken(token)
shutdown()

Optional method to shutdown the (Inner) Computing Element

submitJob(executableFile, proxy, **kwargs)

Method to submit job, should be overridden in sub-class.

writeProxyToFile(proxy)

CE helper function to write a CE proxy string to a file.

DIRAC.Resources.Computing.ComputingElement.getCEConfigDict(section: str) → dict

Look into section for configuration Parameters for this CE

Parameters:

section – name of the CFG section to exploit