AREXComputingElement

AREX Computing Element (ARC REST interface)

Allows interacting with ARC AREX services via a REST interface.

Configuration Parameters

Configuration for the AREXComputingElement submission can be done via the configuration system. Below, you can find a list of parameters specific to the AREX CE.

Preamble:

Line that should be executed just before the executable file.

Timeout:

Duration in seconds before declaring a timeout exception.

Port:

Port added to the CE host name to interact with AREX services.

ProxyTimeLeftBeforeRenewal:

Time in seconds before the AREXCE renews proxy of submitted payloads.

RESTVersion:

Version of the REST interface to use.

XRSLExtraString:

Default additional string for ARC submit files. Should be written in the following format:

(key = "value")

Times (wall/CPU time) should be given in seconds (see https://www.nordugrid.org/arc/arc6/users/xrsl.html?#cputime).

XRSLMPExtraString:

Default additional string for ARC submit files for multi-processor jobs. Should be written in the following format:

(key = "value")

The XRSLExtraString note about times also applies to this configuration option.

Code Documentation

class DIRAC.Resources.Computing.AREXComputingElement.AREXComputingElement(ceUniqueID)

Bases: ComputingElement

__init__(ceUniqueID)

Standard constructor.

available()

This method returns the number of available slots in the target CE. The CE instance polls for waiting and running jobs and compares to the limits in the CE parameters.

cleanJob(jobIDList)

Clean files related to the specified jobs

Parameters:

jobIDList (list) – list of job references

getCEStatus()

Method to return information on running and pending jobs. Query the CE directly to get the number of waiting and running jobs for the given VO and queue. The specification is apparently in glue2 and if you do a raw print out of the information it goes on and on as it dumps the full configuration of the CE for all VOs, queues, states and parameters. Hopefully this function weeds out everything except the relevant queue.

getDescription()

Get CE description as a dictionary.

This is called by the JobAgent for the case of “inner” CEs.

getJobLog(jobID)

Get job logging info

Parameters:

jobID (str) – Job reference followed by the DIRAC stamp.

Returns:

string representing the logging info of a given jobID

getJobOutput(jobID, workingDirectory=None)

Get the outputs of the given job reference.

Outputs and stored in workingDirectory if present, else in a new directory named <ARC JobID>.

Parameters:
  • jobID (str) – job reference followed by the DIRAC stamp.

  • workingDirectory (str) – name of the directory containing the retrieved outputs.

Returns:

content of stdout and stderr

getJobStatus(jobIDList)

Get the status information for the given list of jobs.

Parameters:

jobIDList (list) – list of job references, followed by the DIRAC stamp.

initializeParameters()

Initialize the CE parameters after they are collected from various sources

isValid()

Check the sanity of the Computing Element definition

killJob(jobIDList)

Kill the specified jobs

Parameters:

jobIDList (list) – list of Job references

loadBatchSystem(batchSystemName)

Instantiate object representing the backend batch system

Parameters:

batchSystemName (str) – name of the batch system

sendOutput(stdid, line)

Callback function such that the results from the CE may be returned.

setCPUTimeLeft(cpuTimeLeft=None)

Update the CPUTime parameter of the CE classAd, necessary for running in filling mode

setParameters(ceOptions)

Add parameters from the given dictionary overriding the previous values

Parameters:

ceOptions (dict) – CE parameters dictionary to update already defined ones

setProxy(proxy)

Set proxy for this instance

setToken(token)

Set the token and update the headers

Parameters:

token – OAuth2Token object or dictionary containing token structure

shutdown()

Optional method to shutdown the (Inner) Computing Element

submitJob(executableFile, proxy, numberOfJobs=1, inputs=None, outputs=None)

Method to submit job Assume that the ARC queues are always of the format nordugrid-<batchSystem>-<queue> And none of our supported batch systems have a “-” in their name

writeProxyToFile(proxy)

CE helper function to write a CE proxy string to a file.