Installing HTTPs services in DIRAC

Background

For many years, DIRAC services have been exposed through the historical DISET (dips://) protocol. Few years ago DIRAC developers started exposing DIRAC services using the HTTPs protocol. This page explains how to migrate existing DIPs protocol to HTTPs. For a developer view, refer to HTTPS Services with Tornado.

For a summary presentation you can check this.

NB: not all DIPs services can be exposed through HTTPs. For a comprehensve list, please refer to Dirac Service Limitations.

General principle

Contrary to the DISET protocol where each service would have its own process and port opened, HTTPs services are served by a unique process and port, based on Tornado.

There are exceptions to that, due to the use of global variables in some parts of DIRAC. Namely:

  • Master CS

All other services can follow the standard procedure described below.

First, the following configuration subsections have to be added to CS:

# "Main" section
DIRAC
{
  Setups
  {
    ...
    Tornado = Production
  }
}

# Add Tornado to Systems section
Systems
{
  ...
  Tornado
  {
    Production
    {
      Port = 443
    }
  }
}

Installation of an HTTPs based service

This procedure is to be used if you are want to serve a new service with HTTPs.

Case 1: you do NOT run the equivalent DISET service

This is the most trivial case. Just run dirac-install-tornado-service with the service you are interested in. This will install an runit component running tornado-start-all.

Case 2: you run the equivalent DISET service

Because the CS already contains the handler definition for DISET, dirac-install-tornado-service will not modify it. Thus, you have to update it yourself, before running the command, otherwise tornado-start-all will not find any service to run, and the installation will be shown as failed.

Procedure:

  1. Update by hand the CS of the desired service:

  • Remove the port definition

  • Modify the handler to point to the Tornado handler

  • add Protocol=https

The example the follows is for the “DIRAC File Catalog” (DFC) service. This would normally be in CS as:

Systems
{
  ...
  DataManagement
  {
    Production
    {
      URLs
      {
        ...
        FileCatalog = dips://my.server.org:9197/DataManagement/FileCatalog
      }
      Services
      {
        ...
        FileCatalog
        {
          ...
          Port = 9197
          Protocol = dips
          ...
        }
      }
      ...
    }
  }
}

And you need to change it to:

Systems
{
  ...
  DataManagement
  {
    Production
    {
      URLs
      {
        ...
        FileCatalog = https://my.server.org:8443/DataManagement/FileCatalog
      }
      Services
      {
        ...
        FileCatalog
        {
          ...
          Protocol = https
          HandlerPath = DIRAC/DataManagementSystem/Service/TornadoFileCatalogHandler.py
          ...
        }
      }
      ...
    }
  }
}

  1. Run dirac-install-tornado-service or restart the tornado component if already running.

Note

This means that from now on, the DISET service cannot be restarted anymore, as its configuration would be wrong. So, go ahead and remove (by hand, for now) the runit-related directories of the DISET service.

Example of configuration before/after:

JobMonitoring configuration for DISET
JobMonitoring
{
  Port = 9130
  Authorization
  {
    Default = authenticated
  }
}
JobMonitoring configuration for HTTPs
TornadoJobMonitoring
{
  Protocol = https
  Authorization
  {
    Default = authenticated
  }
}

In any case, do not forget to update the URL of the service you just installed, such that other services can reach it.

Adding more tornado instances on a different machine

Simply use dirac-install-tornado-service with no arguments on the new machine.

MasterCS special case

The master Configuration Server is different and needs to be run in a separate process. In order to do so:

  • Do NOT specify Protocol=https in the service description, otherwise it will be ran with all the other Tornado services.

  • If you run on the same machine as other TornadoService, specify a Port in the service description (you can keep the existing 9135, if already there).

  • Modify the content of the Configuration so that URLs are updated to use HTTPs instead of DIPs.

  • Add HandlerPath = DIRAC/ConfigurationSystem/Service/TornadoConfigurationHandler.py in the etc/dirac.cfg configuration file of the machine where the master CS is running (needed for bootstrap).

Finally, there is no automatic installations script. So just install a CS as you normally would do, and then edit the run file like that:

#!/bin/bash
  rcfile=/opt/dirac/bashrc
  [ -e $rcfile ] && source $rcfile
  #
  export DIRAC_USE_TORNADO_IOLOOP=Yes
  exec 2>&1
  #
  [ "service" = "agent" ] && renice 20 -p $$
  #
  #
  exec tornado-start-CS -ddd