Developing Commands

Commands are one of the main interface tools for the users. Commands are also called scripts in DIRAC lingo.

Where to place scripts

All scripts should live in the scripts directory of their parent system. For instance, the command:

$ dirac-wms-job-submit

will live in src/DIRAC/WorkloadManagementSystem/scripts/dirac_wms_job_submit.py.

Scripts become command line scripts when DIRAC is pip-installed, using the console_scripts entry point, meaning that new scripts should be added to the list in setup.cfg file.

Coding commands

All the commands should be coded following a common recipe and having several mandatory parts. The instructions below must be applied as close as possible although some variation are allowed according to developer’s habits.

1. All scripts must start with a Shebang line like the following:

#!/usr/bin/env python

which will set the interpreter directive to the python on the environment.

2. The next is the documentation line which is describing the command. This same documentation line will be used also the command help information available with the -h command switch.

3. The majority of the code should be contained with a function, often called main though this is not required. This function should be wrapped with the @Script() decorator to allow the DIRAC plugin mechanism to override the script with the function from the highest priority extension.

#Import the required DIRAC modules
from DIRAC.Core.Base.Script import Script
from DIRAC.Interfaces.API.DIRAC import DIRAC
from DIRAC import gLogger

@Script()
def main():
    # Do stuff

if __name__ == "__main__":
    main()

4. Next the function must be registered as a console_scripts entrypoint in the setuptools metadata (more details). This is done by adding a line to the console_scripts list in setup.cfg like below, where the first string is the name for the script you want to create, the left hand side of : is the module that contains your function and the right hand side is the object you want to invoke (e.g. a function).

console_scripts =
    dirac-info = DIRAC.Core.scripts.dirac_info:main
    dirac-proxy-info = DIRAC.FrameworkSystem.scripts.dirac_proxy_info:main

5. Users need to specify parameters to scripts to define what they want to do. To do so, they pass arguments when calling the script. The first thing any script has to do is define what options and arguments the script accepts. Once the valid arguments are defined, the script can parse the command line. An example follows which is a typical command description part

 1#!/usr/bin/env python
 2"""
 3Ping a list of services and show the result
 4
 5Example:
 6  $ dirac-ping-info MySystem
 7  Ping MySystem!
 8"""
 9from DIRAC import S_OK, S_ERROR, gLogger
10from DIRAC.Core.Base.Script import Script
11
12
13# Define a simple class to hold the script parameters
14class Params:
15    def __init__(self):
16        self.raw = False
17        self.pingsToDo = 1
18
19    def setRawResult(self, value):
20        self.raw = True
21        return S_OK()
22
23    def setNumOfPingsToDo(self, value):
24        try:
25            self.pingsToDo = max(1, int(value))
26        except ValueError:
27            return S_ERROR("Number of pings to do has to be a number")
28        return S_OK()
29
30
31@Script()
32def main():
33    # Instantiate the params class
34    cliParams = Params()
35
36    # Register accepted switches and their callbacks
37    Script.registerSwitch("r", "showRaw", "show raw result from the query", cliParams.setRawResult)
38    Script.registerSwitch("p:", "numPings=", "Number of pings to do (by default 1)", cliParams.setNumOfPingsToDo)
39    Script.registerArgument(["System: system names"])
40
41    # Parse the command line and initialize DIRAC
42    switches, servicesList = Script.parseCommandLine(ignoreErrors=False)
43
44    # Get the list of services
45    servicesList = Script.getPositionalArgs()
46
47    # Do something!
48    gLogger.notice("Ping", ", ".join(servicesList))
49
50
51if __name__ == "__main__":
52    main()

Let’s follow the example step by step. First, we import the required modules from DIRAC. S_OK and S_ERROR are the default way DIRAC modules return values or errors. The Script module is the initialization and command line parser that scripts use to initialize themselves. No other DIRAC module should be imported here.

Once the required modules are imported, a Params class is defined. This class holds the values for all the command switches together with all their default values. When the class is instantiated, the parameters get the default values in the constructor function. It also has a set of functions that will be called for each switch that is specified in the command line. We’ll come back to that later.

Then the list of valid switches and what to do in case they are called is defined using registerSwtch() method of the Scripts module. Each switch definition has 4 parameters:

  1. Short switch form. It has to be one letter. Optionally it can have ‘:’ after the letter. If the switch has ‘:’ it requires one parameter with the switch. A valid combination for the previous example would be ‘-r -p 2’. That means show raw results and make 2 pings.

  2. Long switch form. ‘=’ is the equivalent of ‘:’ for the short form. The same combination of command switches in a long form will look like ‘–showRaw –numPings 2’.

  3. Definition of the switch. This text will appear in the script help.

  4. Function to call if the user uses the switch in order to process the switch value

There are several reserved switches that DIRAC uses by default and cannot be overwritten by the script. Those are:

  • -h and –help show the script help

  • -d and –debug enables debug level for the script. Note that the forms -dd and -ddd are accepted resulting in increasingly higher verbosity level

  • -s and –section changes the default section in the configuration for the script

  • -o and –option set the value of an option in the configuration

  • -c and –cert use certificates to connect to services

All the command line arguments that are not corresponding to the explicitly defined switches are returned by the getPositionalArguments() function.

After defining the switches, the parseCommandLine() function has to be called. This method not only parses the command line options but also initializes DIRAC collecting all the configuration data. It is absolutely important to call this function before importing any other DIRAC module. The callbacks defined for the switches will be called when parsing the command line if necessary. Even if the switch is not supposed to receive a parameter, the callback has to receive a value. Switches without callbacks defined can be obtained with getUnprocessedSwitches() function.

5. Once the command line has been parsed and DIRAC is properly initialized, the rest of the required DIRAC modules can be imported and the script logic can take place.

Having understood the logic of the script, there are few good practices that must be followed:

  • Use DIRAC.exit( exitCode ) instead of sys.exit( exitCode )

  • Encapsulate the command code into functions / classes so that it can be easily tested

  • Usage of gLogger instead of print is mandatory. The information in the normal command execution must be printed out in the NOTICE logging level.

Example command

Applying all the above recommendations, the command implementation can look like this yet another example:

  1#!/usr/bin/env python
  2"""
  3This script prints out how great is it, shows raw queries and sets the
  4number of pings.
  5
  6Example:
  7  $ dirac-my-great-script detail Bob MyService
  8  Your name is: Bob
  9  This is the servicesList: MyService
 10  We are done with detail report.
 11"""
 12from DIRAC import S_OK, S_ERROR, gLogger, exit as DIRACExit
 13from DIRAC.Core.Base.Script import Script
 14
 15
 16class Params:
 17    """
 18    Class holding the parameters raw and pingsToDo, and callbacks for their respective switches.
 19    """
 20
 21    def __init__(self):
 22        """C'or"""
 23        self.raw = False
 24        self.pingsToDo = 1
 25        # Defined all switches that can be used while calling the script from the command line interface.
 26        self.switches = [
 27            ("", "text=", "Text to be printed"),
 28            ("u", "upper", "Print text on upper case"),
 29            ("r", "showRaw", "Show raw result from the query", self.setRawResult),
 30            ("p:", "numPings=", "Number of pings to do (by default 1)", self.setNumOfPingsToDo),
 31        ]
 32
 33    def setRawResult(self, _):
 34        """ShowRaw option callback function, no option argument.
 35
 36        :return: S_OK()
 37        """
 38        self.raw = True
 39        return S_OK()
 40
 41    def setNumOfPingsToDo(self, value):
 42        """NumPings option callback function
 43
 44        :param value: option argument
 45
 46        :return: S_OK()/S_ERROR()
 47        """
 48        try:
 49            self.pingsToDo = max(1, int(value))
 50        except ValueError:
 51            return S_ERROR("Number of pings to do has to be a number")
 52        return S_OK()
 53
 54
 55def registerArguments():
 56    """
 57    Registers a positional arguments that can be used while calling the script from the command line interface.
 58    """
 59
 60    # it is important to add a colon after the name of the argument in the description
 61    Script.registerArgument(" ReportType: report type", values=["short", "detail"])
 62    Script.registerArgument(("Name:  user name", "DN: user DN"))
 63    Script.registerArgument(["Service: list of services"], default="no elements", mandatory=False)
 64
 65
 66def parseSwitchesAndPositionalArguments():
 67    """
 68    Parse switches and positional arguments given to the script
 69    """
 70
 71    # Parse the command line and initialize DIRAC
 72    Script.parseCommandLine(ignoreErrors=False)
 73
 74    # Get arguments
 75    allArgs = Script.getPositionalArgs()
 76    gLogger.debug(f"All arguments: {', '.join(allArgs)}")
 77
 78    # Get unprocessed switches
 79    switches = dict(Script.getUnprocessedSwitches())
 80
 81    gLogger.debug("The switches used are:")
 82    map(gLogger.debug, switches.iteritems())
 83
 84    # Get grouped positional arguments
 85    repType, user, services = Script.getPositionalArgs(group=True)
 86    gLogger.debug("The positional arguments are:")
 87    gLogger.debug("Report type:", repType)
 88    gLogger.debug("Name or DN:", user)
 89    gLogger.debug("Services:", services)
 90
 91    return switches, repType, user, services
 92
 93
 94# IMPORTANT: Make sure to add the console-scripts entry to setup.cfg as well!
 95@Script()
 96def main():
 97    """
 98    This is the script main method, which will hold all the logic.
 99    """
100    params = Params()
101
102    # Script initialization
103    Script.registerSwitches(params.switches)
104    registerArguments()
105    switchDict, repType, user, services = parseSwitchesAndPositionalArguments()
106
107    # Import the required DIRAC modules
108    from DIRAC.Interfaces.API.Dirac import Dirac
109
110    # let's do something
111    if services == "no elements":
112        gLogger.error("No services defined")
113        DIRACExit(1)
114    gLogger.notice(f"Your {'DN' if user.startswith('/') else 'name'} is:", user)
115    gLogger.notice("This is the servicesList:", ", ".join(services))
116    gLogger.notice(f"We are done with {repType} report.")
117
118    DIRACExit(0)
119
120
121if __name__ == "__main__":
122    main()