JobSanity

The Job Sanity Agent accepts all jobs from the Job receiver and screens them for the following problems:

  • Output data already exists
  • Problematic JDL
  • Jobs with too much input data e.g. > 100 files
  • Jobs with input data incorrectly specified e.g. castor:/
  • Input sandbox not correctly uploaded.
class DIRAC.WorkloadManagementSystem.Executor.JobSanity.JobSanity

Bases: DIRAC.WorkloadManagementSystem.Executor.Base.OptimizerExecutor.OptimizerExecutor

The specific Optimizer must provide the following methods:
  • optimizeJob() - the main method called for each job
and it can provide:
  • initializeOptimizer() before each execution cycle
class JobLog(log, jid)
class LogWrap(log, jid, funcName)
__init__(log, jid, funcName)
__init__(log, jid)
__init__

x.__init__(…) initializes x; see help(type(x)) for signature

checkInputData(jobState, jobType, voName)

This method checks both the amount of input datasets for the job and whether the LFN conventions are correct.

checkInputSandbox(jobState, manifest)

The number of input sandbox files, as specified in the job JDL are checked in the JobDB.

checkOutputDataExists(jobState)

If the job output data is already in the LFC, this method will fail the job for the attention of the data manager. To be tidied for DIRAC3…

deserializeTask(taskStub)
disableFastTrackForTask()
classmethod ex_enabled()
classmethod ex_getExtraArguments()
classmethod ex_getMind()
classmethod ex_getOption(optName, defaultValue=None)
classmethod ex_getProperty(optName)
classmethod ex_optimizerName()
classmethod ex_setMind(mindName, **extraArgs)
classmethod ex_setOption(optName, value)
classmethod ex_setProperty(optName, value)
fastTrackDispatch(jid, jobState)
freezeTask(freezeTime)
classmethod initialize()
classmethod initializeOptimizer()

Initialize specific parameters for JobSanityAgent.

isTaskFrozen()
jobLog
optimizeJob(jid, jobState)

This method controls the order and presence of each sanity check for submitted jobs. This should be easily extended in the future to accommodate any other potential checks.

processTask(jid, jobState)
retrieveOptimizerParam(name)
serializeTask(cjs)
setNextOptimizer(jobState=None)
storeOptimizerParam(name, value)