Developing Commands

Warning

This instructions here demonstrate how to support both the legacy (Python 2) and future (Python3) installations of DIRAC. If only having Python 3 support is acceptable, the requirement for scripts to be in the the scripts directory of their parent system will be removed and the only requirement will be for the function to be decorated with the @Script() decorator.

Commands are one of the main interface tools for the users. Commands are also called scripts in DIRAC lingo.

Where to place scripts

All scripts should live in the scripts directory of their parent system. For instance, the command:

dirac-wms-job-submit

will live in:

DIRAC/WorkloadManagementSystem/scripts/dirac-wms-job-submit.py

The command script name is the same as the command name itself with the .py suffix appended. When DIRAC client software is installed, all scripts will be placed in the installation scripts directory and stripped of the .py extension. This is done by the dirac-deploy-scripts command that you should have already done when you installed. This way users can see all the scripts in a single place and it makes easy to include all the scripts in the system PATH variable.

Coding commands

All the commands should be coded following a common recipe and having several mandatory parts. The instructions below must be applied as close as possible although some variation are allowed according to developer’s habits.

1. All scripts must start with a Shebang line like the following:

#!/usr/bin/env python

which will set the interpreter directive to the python on the environment.

2. The next is the documentation line which is describing the command. This same documentation line will be used also the command help information available with the -h command switch.

3. The majority of the code should be contained with a function, often called main though this is not required. This function should be wrapped with the @Script() decorator to allow the DIRAC plugin mechanism to override the script with the function from the highest priority extension.

#Import the required DIRAC modules
from DIRAC.Core.Utilities.DIRACScript import DIRACScript as Script
from DIRAC.Interfaces.API.DIRAC import DIRAC
from DIRAC import gLogger

@Script()
def main():
  # Do stuff

if __name__ == "__main__":
  main()

4. Next the function must be registered as a console_scripts entrypoint in the setuptools metadata (more details). This is done by adding a line to the console_scripts list in setup.cfg like below, where the first string is the name for the script you want to create, the left hand side of : is the module that contains your function and the right hand side is the object you want to invoke (e.g. a function).

console_scripts =
    dirac-info = DIRAC.Core.scripts.dirac_info:main
    dirac-proxy-info = DIRAC.FrameworkSystem.scripts.dirac_proxy_info:main

5. Users need to specify parameters to scripts to define what they want to do. To do so, they pass arguments when calling the script. The first thing any script has to do is define what options and arguments the script accepts. Once the valid arguments are defined, the script can parse the command line. An example follows which is a typical command description part

#!/usr/bin/env python
"""
Ping a list of services and show the result

Example:
  $ dirac-ping-info MySystem
  Ping MySystem!
"""
from __future__ import absolute_import
from __future__ import division
from __future__ import print_function

__RCSID__ = "$Id$"

import sys

from DIRAC import S_OK, S_ERROR, gLogger, exit as DIRACExit
from DIRAC.Core.Utilities.DIRACScript import DIRACScript as Script


# Define a simple class to hold the script parameters
class Params(object):
    def __init__(self):
        self.raw = False
        self.pingsToDo = 1

    def setRawResult(self, value):
        self.raw = True
        return S_OK()

    def setNumOfPingsToDo(self, value):
        try:
            self.pingsToDo = max(1, int(value))
        except ValueError:
            return S_ERROR("Number of pings to do has to be a number")
        return S_OK()


@Script()
def main():
    # Instantiate the params class
    cliParams = Params()

    # Register accepted switches and their callbacks
    Script.registerSwitch("r", "showRaw", "show raw result from the query", cliParams.setRawResult)
    Script.registerSwitch("p:", "numPings=", "Number of pings to do (by default 1)", cliParams.setNumOfPingsToDo)
    Script.registerArgument(["System: system names"])

    # Parse the command line and initialize DIRAC
    switches, servicesList = Script.parseCommandLine(ignoreErrors=False)

    # Get the list of services
    servicesList = Script.getPositionalArgs()

    # Do something!
    gLogger.notice("Ping %s!" % ", ".join(servicesList))


if __name__ == "__main__":
    main()

Let’s follow the example step by step. First, we import the required modules from DIRAC. S_OK and S_ERROR are the default way DIRAC modules return values or errors. The Script module is the initialization and command line parser that scripts use to initialize themselves. No other DIRAC module should be imported here.

Once the required modules are imported, a Params class is defined. This class holds the values for all the command switches together with all their default values. When the class is instantiated, the parameters get the default values in the constructor function. It also has a set of functions that will be called for each switch that is specified in the command line. We’ll come back to that later.

Then the list of valid switches and what to do in case they are called is defined using registerSwtch() method of the Scripts module. Each switch definition has 4 parameters:

  1. Short switch form. It has to be one letter. Optionally it can have ‘:’ after the letter. If the switch has ‘:’ it requires one parameter with the switch. A valid combination for the previous example would be ‘-r -p 2’. That means show raw results and make 2 pings.

  2. Long switch form. ‘=’ is the equivalent of ‘:’ for the short form. The same combination of command switches in a long form will look like ‘–showRaw –numPings 2’.

  3. Definition of the switch. This text will appear in the script help.

  4. Function to call if the user uses the switch in order to process the switch value

There are several reserved switches that DIRAC uses by default and cannot be overwritten by the script. Those are:

  • -h and –help show the script help

  • -d and –debug enables debug level for the script. Note that the forms -dd and -ddd are accepted resulting in increasingly higher verbosity level

  • -s and –section changes the default section in the configuration for the script

  • -o and –option set the value of an option in the configuration

  • -c and –cert use certificates to connect to services

All the command line arguments that are not corresponding to the explicitly defined switches are returned by the getPositionalArguments() function.

After defining the switches, the parseCommandLine() function has to be called. This method not only parses the command line options but also initializes DIRAC collecting all the configuration data. It is absolutely important to call this function before importing any other DIRAC module. The callbacks defined for the switches will be called when parsing the command line if necessary. Even if the switch is not supposed to receive a parameter, the callback has to receive a value. Switches without callbacks defined can be obtained with getUnprocessedSwitches() function.

5. Once the command line has been parsed and DIRAC is properly initialized, the rest of the required DIRAC modules can be imported and the script logic can take place.

Having understood the logic of the script, there are few good practices that must be followed:

  • Use DIRAC.exit( exitCode ) instead of sys.exit( exitCode )

  • Encapsulate the command code into functions / classes so that it can be easily tested

  • Usage of gLogger instead of print is mandatory. The information in the normal command execution must be printed out in the NOTICE logging level.

Example command

Applying all the above recommendations, the command implementation can look like this yet another example:

#!/usr/bin/env python
"""
This script prints out how great is it, shows raw queries and sets the
number of pings.

Example:
  $ dirac-my-great-script detail Bob MyService
  Your name is: Bob
  This is the servicesList: MyService
  We are done with detail report.
"""
from __future__ import absolute_import
from __future__ import division
from __future__ import print_function

__RCSID__ = "$Id$"

from DIRAC import S_OK, S_ERROR, gLogger, exit as DIRACExit
from DIRAC.Core.Utilities.DIRACScript import DIRACScript as Script


class Params(object):
    """
    Class holding the parameters raw and pingsToDo, and callbacks for their respective switches.
    """

    def __init__(self):
        """C'or"""
        self.raw = False
        self.pingsToDo = 1
        # Defined all switches that can be used while calling the script from the command line interface.
        self.switches = [
            ("", "text=", "Text to be printed"),
            ("u", "upper", "Print text on upper case"),
            ("r", "showRaw", "Show raw result from the query", self.setRawResult),
            ("p:", "numPings=", "Number of pings to do (by default 1)", self.setNumOfPingsToDo),
        ]

    def setRawResult(self, _):
        """ShowRaw option callback function, no option argument.

        :return: S_OK()
        """
        self.raw = True
        return S_OK()

    def setNumOfPingsToDo(self, value):
        """NumPings option callback function

        :param value: option argument

        :return: S_OK()/S_ERROR()
        """
        try:
            self.pingsToDo = max(1, int(value))
        except ValueError:
            return S_ERROR("Number of pings to do has to be a number")
        return S_OK()


def registerArguments():
    """
    Registers a positional arguments that can be used while calling the script from the command line interface.
    """

    # it is important to add a colon after the name of the argument in the description
    Script.registerArgument(" ReportType: report type", values=["short", "detail"])
    Script.registerArgument(("Name:  user name", "DN: user DN"))
    Script.registerArgument(["Service: list of services"], default="no elements", mandatory=False)


def parseSwitchesAndPositionalArguments():
    """
    Parse switches and positional arguments given to the script
    """

    # Parse the command line and initialize DIRAC
    Script.parseCommandLine(ignoreErrors=False)

    # Get arguments
    allArgs = Script.getPositionalArgs()
    gLogger.debug("All arguments: %s" % ", ".join(allArgs))

    # Get unprocessed switches
    switches = dict(Script.getUnprocessedSwitches())

    gLogger.debug("The switches used are:")
    map(gLogger.debug, switches.iteritems())

    # Get grouped positional arguments
    repType, user, services = Script.getPositionalArgs(group=True)
    gLogger.debug("The positional arguments are:")
    gLogger.debug("Report type:", repType)
    gLogger.debug("Name or DN:", user)
    gLogger.debug("Services:", services)

    return switches, repType, user, services


# IMPORTANT: Make sure to add the console-scripts entry to setup.cfg as well!
@Script()
def main():
    """
    This is the script main method, which will hold all the logic.
    """
    params = Params()

    # Script initialization
    Script.registerSwitches(params.switches)
    registerArguments()
    switchDict, repType, user, services = parseSwitchesAndPositionalArguments()

    # Import the required DIRAC modules
    from DIRAC.Interfaces.API.Dirac import Dirac

    # let's do something
    if services == "no elements":
        gLogger.error("No services defined")
        DIRACExit(1)
    gLogger.notice("Your %s is:" % ("DN" if user.startswith("/") else "name"), user)
    gLogger.notice("This is the servicesList:", ", ".join(services))
    gLogger.notice("We are done with %s report." % repType)

    DIRACExit(0)


if __name__ == "__main__":
    main()