Systems APIs

Currently, client-server interaction with DIRAC systems is provided by the HTTP services that are coded using the TornadoService module. This module implements a fixed interface via HTTP POST requests meant to implement RPC calls, see HTTPS Services with Tornado. Starting with 8.0, an additional TornadoREST module was created, which expands the possibilities for writing access interfaces to DIRAC systems by implementing a REST interface.

TornadoService and TornadoREST inherit from BaseRequestHandler class the basic logic for authorizing the request, search and run the target method in the thread.

Inheritance of classes.

From the scheme it is intuitively clear that on the basis of BaseRequestHandler it is possible to implement other frameworks.

What is the purpose of this?

The purpose of this component is to build DIRAC system status access interfaces based on HTTP requests outside the fixed RPC request interface described in TornadoServices.

The main reason for the implementation of this feature is the need to implementation of API for DIRAC Authorization Server based on OAuth2 framework, which in turn is an integral part of the implementation of OAuth2 DIRAC authorization. So currently, there is only one API implementation for the Framework system, which is AuthHandler. For the same reason, the ability to authorize with an access token (see https://datatracker.ietf.org/doc/html/rfc6750#section-2) was added to the authorization steps.

How to write APIs

You need to choose the system for which you want to write an API, then create a file with the name of your API in the following path /src/DIRAC/<System name>/API/<Your API name>Handler.py

Example of the simple system API handler
    In order to create a handler for your service, it has to follow a certain skeleton.

    Simple example:

    .. code-block:: python

        from DIRAC.Core.Tornado.Server.TornadoREST import *

        class yourEndpointHandler(TornadoREST):

            def get_hello(self, *args, **kwargs):
                ''' Usage:

                        requests.get(url + '/hello/pos_arg1', params=params).json()['args]
                        ['pos_arg1']
                '''
                return {'args': args, 'kwargs': kwargs}

    .. code-block:: python

        from diraccfg import CFG
        from DIRAC.Core.Utilities.JDL import loadJDLAsCFG, dumpCFGAsJDL
        from DIRAC.Core.Tornado.Server.TornadoREST import *
        from DIRAC.WorkloadManagementSystem.Client.JobManagerClient import JobManagerClient
        from DIRAC.WorkloadManagementSystem.Client.JobMonitoringClient import JobMonitoringClient

        class yourEndpointHandler(TornadoREST):

            # Specify the default permission for the handler
            DEFAULT_AUTHORIZATION = ['authenticated']
            # Base URL
            DEFAULT_LOCATION = "/"

            @classmethod
            def initializeHandler(cls, infosDict):
                ''' Initialization '''
                cls.my_requests = 0
                cls.j_manager = JobManagerClient()
                cls.j_monitor = JobMonitoringClient()

            def initializeRequest(self):
                ''' Called at the beginning of each request '''
                self.my_requests += 1

            # In the annotation, you can specify the expected value type of the argument
            def get_job(self, jobID:int, category=None):
                '''Usage:

                    requests.get(f'https://myserver/job/{jobID}', cert=cert)
                    requests.get(f'https://myserver/job/{jobID}/owner', cert=cert)
                    requests.get(f'https://myserver/job/{jobID}/site', cert=cert)
                '''
                if not category:
                    return self.j_monitor.getJobStatus(jobID)
                if category == 'owner':
                    return self.j_monitor.getJobOwner(jobID)
                if category == 'owner':
                    return self.j_monitor.getJobSite(jobID)
                else:
                    # TornadoResponse allows you to call tornadoes methods, thread-safe
                    return TornadoResponse().redirect(f'/job/{jobID}')

            def get_jobs(self, owner=None, *, jobGroup=None, jobName=None):
                '''Usage:

                    requests.get(f'https://myserver/jobs', cert=cert)
                    requests.get(f'https://myserver/jobs/{owner}?jobGroup=job_group?jobName=job_name', cert=cert)
                '''
                conditions = {"Owner": owner or self.getRemoteCredentials}
                if jobGroup:
                    conditions["JobGroup"] = jobGroup
                if jobName:
                    conditions["JobName"] = jobName
                return self.j_monitor.getJobs(conditions, date)

            def post_job(self, manifest):
                '''Usage:

                    requests.post(f'https://myserver/job', cert=cert, json=[{Executable: "/bin/ls"}])
                '''
                jdl = dumpCFGAsJDL(CFG.CFG().loadFromDict(manifest))
                return self.j_manager.submitJob(str(jdl))

            def delete_job(self, jobIDs):
                '''Usage:

                    requests.delete(f'https://myserver/job', cert=cert, json=[123, 124])
                '''
                return self.j_manager.deleteJob(jobIDs)

            @authentication(["VISITOR"])
            @authorization(["all"])
            def options_job(self):
                '''Usage:

                    requests.options(f'https://myserver/job')
                '''
                return "You use OPTIONS method to access job manager API."

    .. note:: This example aims to show how access interfaces can be implemented and no more

    This class can read the method annotation to understand what type of argument expects to get the method,
    see :py:meth:`_getMethodArgs`.

    Note that because we inherit from :py:class:`tornado.web.RequestHandler`
    and we are running using executors, the methods you export cannot write
    back directly to the client. Please see inline comments in
    :py:class:`BaseRequestHandler <DIRAC.Core.Tornado.Server.private.BaseRequestHandler.BaseRequestHandler>` for more details.

Note

If you need to implement the interface on a standard 443 https port, you will need to use a balancer, such as nginx

The example described is likely to be sufficient for most writing cases. But here are some additional features, see BaseRequestHandler:

  • USE_AUTHZ_GRANTS set the list and order of steps to authorize the request. For example, set USE_AUTHZ_GRANTS = ["JWT"] to allow access to your endpoint only with a valid access token.

  • DEFAULT_AUTHORIZATION set the authorization requirements. For example, DEFAULT_AUTHORIZATION = ['authenticated'] will allow access only to authenticated users.

  • in addition to standard S_OK/S_ERROR you can return text, whether the dictionary for example or nothing, the result will be sent with a 200 status.

  • If your API is complex enough and may include, for example, redirection or additional headers, you can use TornadoResponse to add all these necessary things, which is thread-safe because TornadoResponse will call your actions outside the thread in which this method is executed:

    from DIRAC.Core.Tornado.Server.private.BaseRequestHandler import TornadoResponse
    
    class MyClass(TornadoREST):
    
        # Describe the class as in the example.
    
        def web_myMethod(self, my_option):
            # Do some thing
            response = TornadoResponse(data)
            response.set_status(201)
            return response
    

The framework takes into account the target method annotations when preparing arguments using the inspect module, see https://docs.python.org/3/library/inspect.html#inspect.signature. This means that if you specify an argument type in the annotation, the framework will try to convert the received argument from the request to the specified type.

Note

This component is still quite poor and can be improved by subsequent PRs.