Job

Job Base Class

This class provides generic job definition functionality suitable for any VO.

Helper functions are documented with example usage for the DIRAC API. An example script (for a simple executable) would be:

from DIRAC.Interfaces.API.Dirac import Dirac
from DIRAC.Interfaces.API.Job import Job

j = Job()
j.setCPUTime(500)
j.setExecutable('/bin/echo hello')
j.setExecutable('yourPythonScript.py')
j.setExecutable('/bin/echo hello again')
j.setName('MyJobName')

dirac = Dirac()
jobID = dirac.submitJob(j)
print 'Submission Result: ',jobID

Note that several executables can be provided and wil be executed sequentially.

class DIRAC.Interfaces.API.Job.Job(script=None, stdout='std.out', stderr='std.err')

Bases: DIRAC.Core.Base.API.API

DIRAC jobs

__init__(script=None, stdout='std.out', stderr='std.err')

Instantiates the Workflow object and some default parameters.

execute()

Developer function. Executes the job locally.

runLocal(dirac=None)

The dirac (API) object is for local submission.

setBannedSites(sites)

Helper function.

Can specify a desired destination site for job. This can be useful for debugging purposes but often limits the possible candidate sites and overall system response time.

Example usage:

>>> job = Job()
>>> job.setBannedSites(['LCG.GRIDKA.de','LCG.CNAF.it'])
Parameters:sites (str or list) – single site string or list
setCPUTime(timeInSecs)

Helper function.

Under Development. Specify CPU time requirement in DIRAC units.

Example usage:

>>> job = Job()
>>> job.setCPUTime(5000)
Parameters:timeInSecs (Int) – CPU time
setConfigArgs(cfgString)

Developer function. Allow to pass arbitrary settings to the payload configuration service environment.

setDestination(destination)

Helper function.

Can specify a desired destination site or sites for job. This can be useful for debugging purposes but often limits the possible candidate sites and overall system response time.

Example usage:

>>> job = Job()
>>> job.setDestination('LCG.CERN.ch')
Parameters:destination (str or list) – site string
setDestinationCE(ceName, diracSite=None)

Developer function.

Allows to direct a job to a particular Grid CE.

setExecutable(executable, arguments='', logFile='', modulesList=None, parameters=None, paramValues=None)

Helper function.

Specify executable script to run with optional arguments and log file for standard output.

These can be either:

  • Submission of a python or shell script to DIRAC
    • Can be inline scripts e.g. C{‘/bin/ls’}
    • Scripts as executables e.g. python or shell script file

Example usage:

>>> job = Job()
>>> job.setExecutable('myScript.py')
Parameters:
  • executable (string) – Executable
  • arguments (string) – Optional arguments to executable
  • logFile (string) – Optional log file name
  • modulesList (list) – Optional list of modules (to be used mostly when extending this method)
  • parameters (list of tuples) – Optional list of parameters (to be used mostly when extending this method)
  • paramValues – Optional list of parameters values (to be used mostly when extending this method)
setExecutionEnv(environmentDict)

Helper function.

Optionally specify a dictionary of key, value pairs to be set before the job executes e.g. {‘MYVAR’:3}

The standard application environment variables are always set so this is intended for user variables only.

Example usage:

>>> job = Job()
>>> job.setExecutionEnv({'<MYVARIABLE>':'<VALUE>'})
Parameters:environmentDict (dictionary) – Environment variables
setInputData(lfns)

Helper function.

Specify input data by Logical File Name (LFN).

Example usage:

>>> job = Job()
>>> job.setInputData(['/lhcb/production/DC04/v2/DST/00000742_00003493_10.dst'])
Parameters:lfns (Single LFN string or list of LFNs) – Logical File Names
setInputDataPolicy(policy, dataScheduling=True)

Helper function.

Specify a job input data policy, this takes precedence over any site specific or global settings.

Possible values for policy are ‘Download’ or ‘Protocol’ (case-insensitive). This requires that the module locations are defined for the VO in the CS.

Example usage:

>>> job = Job()
>>> job.setInputDataPolicy('download')
setInputSandbox(files)

Helper function.

Specify input sandbox files less than 10MB in size. If over 10MB, files or a directory may be uploaded to Grid storage, see C{dirac.uploadSandbox()}.

Paths to the options file and (if required) ‘lib/’ directory of the DLLs are specified here. Default is local directory. CMT requirements files or executables may be placed in the lib/ directory if desired. The lib/ directory is transferred to the Grid Worker Node before the job executes.

Files / directories can be specified using the * character e.g. *.txt these are resolved correctly before job execution on the WN.

Example usage:

>>> job = Job()
>>> job.setInputSandbox(['DaVinci.opts'])
Parameters:files (Single string or list of strings ['','']) – Input sandbox files, can specify full path
setJobGroup(jobGroup)

Helper function.

Allows to group certain jobs according to an ID.

Example usage:

>>> job = Job()
>>> job.setJobGroup('Bs2JPsiPhi')
Parameters:jobGroup (string) – JobGroup name
setLogLevel(logLevel)

Helper function.

Optionally specify a DIRAC logging level for the job, e.g. ALWAYS, INFO, VERBOSE, WARN, DEBUG by default this is set to the info level.

Example usage:

>>> job = Job()
>>> job.setLogLevel('debug')
Parameters:logLevel (string) – Logging level
setName(jobName)

Helper function.

A name for the job can be specified if desired. This will appear in the JobName field of the monitoring webpage. If nothing is specified a default value will appear.

Example usage:

>>> job=Job()
>>> job.setName("myJobName")
Parameters:jobName (string) – Name of job
setOutputData(lfns, outputSE=None, outputPath='')

Helper function.

For specifying output data to be registered in Grid storage. If a list of OutputSEs are specified the job wrapper will try each in turn until successful. If the OutputPath is specified this will appear only after / <VO> / user / <initial> / <username> directory.

The output data can be LFNs or local file names. If they are LFNs they should be pre-prended by “LFN:”, otherwise they will be interpreted as local files to be found. If local files are specified, then specifying the outputPath may become necessary, because if it’s not specified then it will be constructed starting from the user name.

Example usage:

>>> job = Job()
>>> job.setOutputData(['DVNtuple.root'])
Parameters:
  • lfns (Single string or list of strings ['','']) – Output data file or files.
  • outputSE (string or list) – Optional parameter to specify the Storage Element to store data or files, e.g. CERN-tape
  • outputPath (string) – Optional parameter to specify part of the path in the storage (see above)
setOutputSandbox(files)

Helper function.

Specify output sandbox files. If specified files are over 10MB, these may be uploaded to Grid storage with a notification returned in the output sandbox.

Example usage:

>>> job = Job()
>>> job.setOutputSandbox(['DaVinci_v19r12.log','DVNTuples.root'])
Parameters:files (Single string or list of strings ['','']) – Output sandbox files
setOwner(ownerProvided)

Developer function.

Normally users should always specify their immutable DIRAC nickname.

setOwnerDN(ownerDN)

Developer function.

Allows to force expected owner DN of proxy.

setOwnerGroup(ownerGroup)

Developer function.

Allows to force expected owner group of proxy.

setParameterSequence(name, parameterList, addToWorkflow=False)

Function to define a sequence of values for parametric jobs.

Parameters:
  • name (str) – sequence parameter name
  • parameterList (list) – list of parameter values
  • addToWorkflow (bool) – flag to add parameter to the workflow on the fly, if str, then use as the workflow parameter
Returns:

setParametricInputData(**kwargs)

Helper function.

Deprecated since version v6r15: Use setParameterSequence() instead.

Specify input data by Logical File Name (LFN) to be used as a parameter in a parametric job

Example usage:

>>> job = Job()
>>> job.setParametricInputData(['/lhcb/production/DC04/v2/DST/00000742_00003493_10.dst'])
Parameters:lfns (list of LFNs or list of lists of LFNs) – Logical File Names
setParametricInputSandbox(**kwargs)

Helper function.

Deprecated since version v6r15: Use setParameterSequence() instead.

Specify input sandbox files to be used as parameters in the Parametric jobs. The possibilities are identical to the setInputSandbox.

Example usage:

>>> job = Job()
>>> job.setParametricInputSandbox(['LFN:/Some_file','LFN:/Some_other_file'])
Parameters:files (Single LFN string or list of LFNs) – Logical File Names
setPlatform(platform)

Developer function: sets the target platform, e.g. Linux_x86_64_glibc-2.5. This platform is in the form of what it is returned by the dirac-platform script (or dirac-architecture if your extension provides it)

setSubmitPool(backend)

Developer function.

Choose submission pool on which job is executed. Default in place for users.

setTag(tags)

Set the Tags job requirements

Example usage:

>>> job = Job()
>>> job.setTag( ['WholeNode','8GBMemory'] )
Parameters:tags (str or list) – single tag string or a list of tags
setType(jobType)

Developer function.

Specify job type for testing purposes.