Developing Commands¶
Warning
This instructions here demonstrate how to support both the legacy (Python 2) and future (Python3) installations of DIRAC.
If only having Python 3 support is acceptable, the requirement for scripts to be in the the scripts directory of their parent system will be removed and the only requirement will be for the function to be decorated with the @DIRACScript()
decorator.
Commands are one of the main interface tools for the users. Commands are also called scripts in DIRAC lingo.
Where to place scripts¶
All scripts should live in the scripts directory of their parent system. For instance, the command:
dirac-wms-job-submit
will live in:
DIRAC/WorkloadManagementSystem/scripts/dirac-wms-job-submit.py
The command script name is the same as the command name itself with the .py suffix appended. When DIRAC client software is installed, all scripts will be placed in the installation scripts directory and stripped of the .py extension. This is done by the dirac-deploy-scripts command that you should have already done when you installed. This way users can see all the scripts in a single place and it makes easy to include all the scripts in the system PATH variable.
Coding commands¶
All the commands should be coded following a common recipe and having several mandatory parts. The instructions below must be applied as close as possible although some variation are allowed according to developer’s habits.
1. All scripts must start with a Shebang line like the following:
#!/usr/bin/env python
which will set the interpreter directive to the python on the environment.
2. The next is the documentation line which is describing the command. This same documentation line will be used also the command help information available with the -h command switch.
3. The majority of the code should be contained with a function, often called main
though this is not required. This function should be wrapped with the @DIRACScript()
decorator to allow the DIRAC plugin mechanism to override the script with the function from the highest priority extension.
#Import the required DIRAC modules
from DIRAC.Core.Utilities.DIRACScript import DIRACScript
from DIRAC.Interfaces.API.DIRAC import DIRAC
from DIRAC import gLogger
@DIRACScript()
def main():
# Do stuff
if __name__ == "__main__":
main()
4. Next the function must be registered as a console_scripts
entrypoint
in the setuptools
metadata (more details). This is done by adding a line to the console_scripts
list in setup.cfg
like below, where the first string is the name for the script you want to create, the left hand side of :
is the module that contains your function and the right hand side is the object you want to invoke (e.g. a function).
console_scripts =
dirac-info = DIRAC.Core.scripts.dirac_info:main
dirac-proxy-info = DIRAC.FrameworkSystem.scripts.dirac_proxy_info:main
5. Users need to specify parameters to scripts to define what they want to do. To do so, they pass arguments when calling the script. The first thing any script has to do is define what options and arguments the script accepts. Once the valid arguments are defined, the script can parse the command line. An example follows which is a typical command description part
#!/usr/bin/env python """ Ping a list of services and show the result Usage: dirac-ping-info [options] ... System ... Arguments: System: system name(mandatory) """ from __future__ import absolute_import from __future__ import division from __future__ import print_function __RCSID__ = "$Id$" import sys from DIRAC import exit as DIRACExit from DIRAC import S_OK, S_ERROR from DIRAC.Core.Base import Script from DIRAC.Core.Utilities.DIRACScript import DIRACScript # Define a simple class to hold the script parameters class Params(object): def __init__(self): self.raw = False self.pingsToDo = 1 def setRawResult(self, value): self.raw = True return S_OK() def setNumOfPingsToDo(self, value): try: self.pingsToDo = max(1, int(value)) except ValueError: return S_ERROR("Number of pings to do has to be a number") return S_OK() @DIRACScript() def main(): # Instantiate the params class cliParams = Params() # Register accepted switches and their callbacks Script.registerSwitch("r", "showRaw", "show raw result from the query", cliParams.setRawResult) Script.registerSwitch("p:", "numPings=", "Number of pings to do (by default 1)", cliParams.setNumOfPingsToDo) # Parse the command line and initialize DIRAC Script.parseCommandLine(ignoreErrors=False) # Get the list of services servicesList = Script.getPositionalArgs() # Check and process the command line switches and options if not servicesList: Script.showHelp(exitCode=1) # Do something! if __name__ == "__main__": main()
Let’s follow the example step by step. First, we import the required modules from DIRAC. S_OK and S_ERROR are the default way DIRAC modules return values or errors. The Script module is the initialization and command line parser that scripts use to initialize themselves. No other DIRAC module should be imported here.
Once the required modules are imported, a Params class is defined. This class holds the values for all the command switches together with all their default values. When the class is instantiated, the parameters get the default values in the constructor function. It also has a set of functions that will be called for each switch that is specified in the command line. We’ll come back to that later.
Then the list of valid switches and what to do in case they are called is defined using registerSwtch() method of the Scripts module. Each switch definition has 4 parameters:
Short switch form. It has to be one letter. Optionally it can have ‘:’ after the letter. If the switch has ‘:’ it requires one parameter with the switch. A valid combination for the previous example would be ‘-r -p 2’. That means show raw results and make 2 pings.
Long switch form. ‘=’ is the equivalent of ‘:’ for the short form. The same combination of command switches in a long form will look like ‘–showRaw –numPings 2’.
Definition of the switch. This text will appear in the script help.
Function to call if the user uses the switch in order to process the switch value
There are several reserved switches that DIRAC uses by default and cannot be overwritten by the script. Those are:
-h and –help show the script help
-d and –debug enables debug level for the script. Note that the forms -dd and -ddd are accepted resulting in increasingly higher verbosity level
-s and –section changes the default section in the configuration for the script
-o and –option set the value of an option in the configuration
-c and –cert use certificates to connect to services
All the command line arguments that are not corresponding to the explicitly defined switches are returned by the getPositionalArguments() function.
After defining the switches, the parseCommandLine() function has to be called. This method not only parses the command line options but also initializes DIRAC collecting all the configuration data. It is absolutely important to call this function before importing any other DIRAC module. The callbacks defined for the switches will be called when parsing the command line if necessary. Even if the switch is not supposed to receive a parameter, the callback has to receive a value. Switches without callbacks defined can be obtained with getUnprocessedSwitches() function.
5. Once the command line has been parsed and DIRAC is properly initialized, the rest of the required DIRAC modules can be imported and the script logic can take place.
Having understood the logic of the script, there are few good practices that must be followed:
Use DIRAC.exit( exitCode ) instead of sys.exit( exitCode )
Encapsulate the command code into functions / classes so that it can be easily tested
Usage of gLogger instead of print is mandatory. The information in the normal command execution must be printed out in the NOTICE logging level.
Example command¶
Applying all the above recommendations, the command implementation can look like this yet another example:
#!/usr/bin/env python
"""
This script prints out how great is it, shows raw queries and sets the
number of pings.
Usage:
dirac-my-great-script [options] <Arguments>
Arguments:
<service1> [<service2> ...]
"""
from __future__ import absolute_import
from __future__ import division
from __future__ import print_function
__RCSID__ = "$Id$"
from DIRAC import S_OK, S_ERROR, gLogger, exit as DIRACExit
from DIRAC.Core.Utilities.DIRACScript import DIRACScript
from DIRAC.Core.Base import Script
class Params(object):
"""
Class holding the parameters raw and pingsToDo, and callbacks for their
respective switches.
"""
def __init__(self):
self.raw = False
self.pingsToDo = 1
def setRawResult(self, value):
self.raw = True
return S_OK()
def setNumOfPingsToDo(self, value):
try:
self.pingsToDo = max(1, int(value))
except ValueError:
return S_ERROR("Number of pings to do has to be a number")
return S_OK()
def registerSwitches():
"""
Registers all switches that can be used while calling the script from the
command line interface.
"""
# Some of the switches have associated a callback, defined on Params class.
cliParams = Params()
switches = [
("", "text=", "Text to be printed"),
("u", "upper", "Print text on upper case"),
("r", "showRaw", "Show raw result from the query", cliParams.setRawResult),
("p:", "numPings=", "Number of pings to do (by default 1)", cliParams.setNumOfPingsToDo),
]
# Register switches
for switch in switches:
Script.registerSwitch(*switch)
def parseSwitches():
"""
Parse switches and positional arguments given to the script
"""
# Parse the command line and initialize DIRAC
Script.parseCommandLine(ignoreErrors=False)
# Get the list of services
servicesList = Script.getPositionalArgs()
gLogger.info("This is the servicesList %s:" % servicesList)
# Gets the rest of the
switches = dict(Script.getUnprocessedSwitches())
gLogger.debug("The switches used are:")
map(gLogger.debug, switches.iteritems())
switches["servicesList"] = servicesList
return switches
# IMPORTANT: Make sure to add the console-scripts entry to setup.cfg as well!
@DIRACScript()
def main():
"""
This is the script main method, which will hold all the logic.
"""
# Script initialization
registerSwitches()
switchDict = parseSwitches()
# Import the required DIRAC modules
from DIRAC.Interfaces.API.Dirac import Dirac
# let's do something
if not len(switchDict["servicesList"]):
gLogger.error("No services defined")
DIRACExit(1)
gLogger.notice("We are done")
DIRACExit(0)
if __name__ == "__main__":
main()