Systems APIs
Currently, client-server interaction with DIRAC systems is provided by the HTTP services that are coded using the TornadoService module.
This module implements a fixed interface via HTTP POST requests meant to implement RPC calls, see HTTPS Services with Tornado.
Starting with 8.0, an additional TornadoREST module was created, which expands the possibilities for writing access interfaces to DIRAC systems by implementing a REST interface.
TornadoService and TornadoREST inherit from BaseRequestHandler class the basic logic for authorizing the request, search and run the target method in the thread.
From the scheme it is intuitively clear that on the basis of BaseRequestHandler it is possible to implement other frameworks.
What is the purpose of this?
The purpose of this component is to build DIRAC system status access interfaces based on HTTP requests outside the fixed RPC request interface described in TornadoServices.
The main reason for the implementation of this feature is the need to implementation of API for DIRAC Authorization Server based on OAuth2 framework,
which in turn is an integral part of the implementation of OAuth2 DIRAC authorization.
So currently, there is only one API implementation for the Framework system, which is AuthHandler.
For the same reason, the ability to authorize with an access token (see https://datatracker.ietf.org/doc/html/rfc6750#section-2) was added to the authorization steps.
How to write APIs
You need to choose the system for which you want to write an API, then create a file with the name of your API in the following path /src/DIRAC/<System name>/API/<Your API name>Handler.py
In order to create a handler for your service, it has to follow a certain skeleton.
Simple example:
.. code-block:: python
from DIRAC.Core.Tornado.Server.TornadoREST import *
class yourEndpointHandler(TornadoREST):
def get_hello(self, *args, **kwargs):
''' Usage:
requests.get(url + '/hello/pos_arg1', params=params).json()['args]
['pos_arg1']
'''
return {'args': args, 'kwargs': kwargs}
.. code-block:: python
from diraccfg import CFG
from DIRAC.Core.Utilities.JDL import loadJDLAsCFG, dumpCFGAsJDL
from DIRAC.Core.Tornado.Server.TornadoREST import *
from DIRAC.WorkloadManagementSystem.Client.JobManagerClient import JobManagerClient
from DIRAC.WorkloadManagementSystem.Client.JobMonitoringClient import JobMonitoringClient
class yourEndpointHandler(TornadoREST):
# Specify the default permission for the handler
DEFAULT_AUTHORIZATION = ['authenticated']
# Base URL
DEFAULT_LOCATION = "/"
@classmethod
def initializeHandler(cls, infosDict):
''' Initialization '''
cls.my_requests = 0
cls.j_manager = JobManagerClient()
cls.j_monitor = JobMonitoringClient()
def initializeRequest(self):
''' Called at the beginning of each request '''
self.my_requests += 1
# In the annotation, you can specify the expected value type of the argument
def get_job(self, jobID:int, category=None):
'''Usage:
requests.get(f'https://myserver/job/{jobID}', cert=cert)
requests.get(f'https://myserver/job/{jobID}/owner', cert=cert)
requests.get(f'https://myserver/job/{jobID}/site', cert=cert)
'''
if not category:
return self.j_monitor.getJobStatus(jobID)
if category == 'owner':
return self.j_monitor.getJobOwner(jobID)
if category == 'owner':
return self.j_monitor.getJobSite(jobID)
else:
# TornadoResponse allows you to call tornadoes methods, thread-safe
return TornadoResponse().redirect(f'/job/{jobID}')
def get_jobs(self, owner=None, *, jobGroup=None, jobName=None):
'''Usage:
requests.get(f'https://myserver/jobs', cert=cert)
requests.get(f'https://myserver/jobs/{owner}?jobGroup=job_group?jobName=job_name', cert=cert)
'''
conditions = {"Owner": owner or self.getRemoteCredentials}
if jobGroup:
conditions["JobGroup"] = jobGroup
if jobName:
conditions["JobName"] = jobName
return self.j_monitor.getJobs(conditions, date)
def post_job(self, manifest):
'''Usage:
requests.post(f'https://myserver/job', cert=cert, json=[{Executable: "/bin/ls"}])
'''
jdl = dumpCFGAsJDL(CFG.CFG().loadFromDict(manifest))
return self.j_manager.submitJob(str(jdl))
def delete_job(self, jobIDs):
'''Usage:
requests.delete(f'https://myserver/job', cert=cert, json=[123, 124])
'''
return self.j_manager.deleteJob(jobIDs)
@authentication(["VISITOR"])
@authorization(["all"])
def options_job(self):
'''Usage:
requests.options(f'https://myserver/job')
'''
return "You use OPTIONS method to access job manager API."
.. note:: This example aims to show how access interfaces can be implemented and no more
This class can read the method annotation to understand what type of argument expects to get the method,
see :py:meth:`_getMethodArgs`.
Note that because we inherit from :py:class:`tornado.web.RequestHandler`
and we are running using executors, the methods you export cannot write
back directly to the client. Please see inline comments in
:py:class:`BaseRequestHandler <DIRAC.Core.Tornado.Server.private.BaseRequestHandler.BaseRequestHandler>` for more details.
Note
If you need to implement the interface on a standard 443 https port, you will need to use a balancer, such as nginx
The example described is likely to be sufficient for most writing cases. But here are some additional features, see BaseRequestHandler:
USE_AUTHZ_GRANTSset the list and order of steps to authorize the request. For example, setUSE_AUTHZ_GRANTS = ["JWT"]to allow access to your endpoint only with a valid access token.
DEFAULT_AUTHORIZATIONset the authorization requirements. For example,DEFAULT_AUTHORIZATION = ['authenticated']will allow access only to authenticated users.in addition to standard S_OK/S_ERROR you can return text, whether the dictionary for example or nothing, the result will be sent with a 200 status.
If your API is complex enough and may include, for example, redirection or additional headers, you can use
TornadoResponseto add all these necessary things, which is thread-safe because TornadoResponse will call your actions outside the thread in which this method is executed:from DIRAC.Core.Tornado.Server.private.BaseRequestHandler import TornadoResponse class MyClass(TornadoREST): # Describe the class as in the example. def web_myMethod(self, my_option): # Do some thing response = TornadoResponse(data) response.set_status(201) return response
The framework takes into account the target method annotations when preparing arguments using the inspect module, see https://docs.python.org/3/library/inspect.html#inspect.signature.
This means that if you specify an argument type in the annotation, the framework will try to convert the received argument from the request to the specified type.
Note
This component is still quite poor and can be improved by subsequent PRs.