AREXComputingElement
AREX Computing Element (ARC REST interface)
Allows interacting with ARC AREX services via a REST interface.
Configuration Parameters
Configuration for the AREXComputingElement submission can be done via the configuration system.
It inherits from the ARCComputingElement configuration parameters.
Below, you can find a list of parameters specific to the AREX CE.
- ARCRESTTimeout:
Duration in seconds before declaring a timeout exception.
- Port:
Port added to the CE host name to interact with AREX services.
- ProxyTimeLeftBeforeRenewal:
Time in seconds before the AREXCE renews proxy of submitted payloads.
- RESTVersion:
Version of the REST interface to use.
- AlwaysIncludeProxy:
A boolean, set to true to include the proxy in job submission even in cases where tokens are the primary authentication method. (Recommended for ARC6 tokens, deprecated for ARC7)
Code Documentation
- class DIRAC.Resources.Computing.AREXComputingElement.AREXComputingElement(ceUniqueID)
Bases:
ARCComputingElement- __init__(ceUniqueID)
Standard constructor.
- available(jobIDList=None)
This method returns the number of available slots in the target CE. The CE instance polls for waiting and running jobs and compares to the limits in the CE parameters.
- Parameters:
jobIDList (list) – list of already existing job IDs to be checked against
- cleanJob(jobIDList)
Clean files related to the specified jobs
- Parameters:
jobIDList (list) – list of job references
- getCEStatus()
Method to return information on running and pending jobs. Query the CE directly to get the number of waiting and running jobs for the given VO and queue. The specification is apparently in glue2 and if you do a raw print out of the information it goes on and on as it dumps the full configuration of the CE for all VOs, queues, states and parameters. Hopefully this function weeds out everything except the relevant queue.
- getDescription()
Get CE description as a dictionary.
This is called by the JobAgent for the case of “inner” CEs.
- getJobLog(jobID)
Get job logging info
- Parameters:
jobID (str) – Job reference followed by the DIRAC stamp.
- Returns:
string representing the logging info of a given jobID
- getJobOutput(jobID, workingDirectory=None)
Get the outputs of the given job reference.
Outputs and stored in workingDirectory if present, else in a new directory named <ARC JobID>.
- getJobStatus(jobIDList)
Get the status information for the given list of jobs.
- Parameters:
jobIDList (list) – list of job references, followed by the DIRAC stamp.
- initializeParameters()
Initialize the CE parameters after they are collected from various sources
- isProxyValid(valid=1000)
Check if the stored proxy is valid
- isValid()
Check the sanity of the Computing Element definition
- loadBatchSystem(batchSystemName)
Instantiate object representing the backend batch system
- Parameters:
batchSystemName (str) – name of the batch system
- sendOutput(stdid, line)
Callback function such that the results from the CE may be returned.
- setCPUTimeLeft(cpuTimeLeft=None)
Update the CPUTime parameter of the CE classAd, necessary for running in filling mode
- setParameters(ceOptions)
Add parameters from the given dictionary overriding the previous values
- Parameters:
ceOptions (dict) – CE parameters dictionary to update already defined ones
- setProxy(proxy, valid=0)
Set proxy for this instance
- setToken(token)
Set the token and update the headers
- Parameters:
token – OAuth2Token object or dictionary containing token structure
valid (int) – validity period in seconds
- shutdown()
Optional method to shutdown the (Inner) Computing Element
- submitJob(executableFile, proxy, numberOfJobs=1, inputs=None, outputs=None)
Method to submit job Assume that the ARC queues are always of the format nordugrid-<batchSystem>-<queue> And none of our supported batch systems have a “-” in their name
- writeProxyToFile(proxy)
CE helper function to write a CE proxy string to a file.