AREXComputingElement

AREX Computing Element (ARC REST interface)

Allows interacting with ARC AREX services via a REST interface.

Configuration Parameters

Configuration for the AREXComputingElement submission can be done via the configuration system. It inherits from the ARCComputingElement configuration parameters. Below, you can find a list of parameters specific to the AREX CE.

ARCRESTTimeout:

Duration in seconds before declaring a timeout exception.

Port:

Port added to the CE host name to interact with AREX services.

ProxyTimeLeftBeforeRenewal:

Time in seconds before the AREXCE renews proxy of submitted payloads.

RESTVersion:

Version of the REST interface to use.

Code Documentation

class DIRAC.Resources.Computing.AREXComputingElement.AREXComputingElement(ceUniqueID)

Bases: ARCComputingElement

__init__(ceUniqueID)

Standard constructor.

available(jobIDList=None)

This method returns the number of available slots in the target CE. The CE instance polls for waiting and running jobs and compares to the limits in the CE parameters.

Parameters:

jobIDList (list) – list of already existing job IDs to be checked against

cleanJob(jobIDList)

Clean files related to the specified jobs

Parameters:

jobIDList (list) – list of job references

getCEStatus()

Method to return information on running and pending jobs. Query the CE directly to get the number of waiting and running jobs for the given VO and queue. The specification is apparently in glue2 and if you do a raw print out of the information it goes on and on as it dumps the full configuration of the CE for all VOs, queues, states and parameters. Hopefully this function weeds out everything except the relevant queue.

getDescription()

Get CE description as a dictionary.

This is called by the JobAgent for the case of “inner” CEs.

getJobLog(jobID)

Get job logging info

Parameters:

jobID (str) – Job reference followed by the DIRAC stamp.

Returns:

string representing the logging info of a given jobID

getJobOutput(jobID, workingDirectory=None)

Get the outputs of the given job reference.

Outputs and stored in workingDirectory if present, else in a new directory named <ARC JobID>.

Parameters:
  • jobID (str) – job reference followed by the DIRAC stamp.

  • workingDirectory (str) – name of the directory containing the retrieved outputs.

Returns:

content of stdout and stderr

getJobStatus(jobIDList)

Get the status information for the given list of jobs.

Parameters:

jobIDList (list) – list of job references, followed by the DIRAC stamp.

initializeParameters()

Initialize the CE parameters after they are collected from various sources

isProxyValid(valid=1000)

Check if the stored proxy is valid

isValid()

Check the sanity of the Computing Element definition

killJob(jobIDList)

Kill the specified jobs

Parameters:

jobIDList (list) – list of Job references

loadBatchSystem(batchSystemName)

Instantiate object representing the backend batch system

Parameters:

batchSystemName (str) – name of the batch system

sendOutput(stdid, line)

Callback function such that the results from the CE may be returned.

setCPUTimeLeft(cpuTimeLeft=None)

Update the CPUTime parameter of the CE classAd, necessary for running in filling mode

setParameters(ceOptions)

Add parameters from the given dictionary overriding the previous values

Parameters:

ceOptions (dict) – CE parameters dictionary to update already defined ones

setProxy(proxy, valid=0)

Set proxy for this instance

setToken(token)

Set the token and update the headers

Parameters:
  • token – OAuth2Token object or dictionary containing token structure

  • valid (int) – validity period in seconds

shutdown()

Optional method to shutdown the (Inner) Computing Element

submitJob(executableFile, proxy, numberOfJobs=1, inputs=None, outputs=None)

Method to submit job Assume that the ARC queues are always of the format nordugrid-<batchSystem>-<queue> And none of our supported batch systems have a “-” in their name

writeProxyToFile(proxy)

CE helper function to write a CE proxy string to a file.