TransformationClient

Class that contains client access to the transformation DB handler.

class DIRAC.TransformationSystem.Client.TransformationClient.TransformationClient(**kwargs)

Bases: Client

Exposes the functionality available in the DIRAC/TransformationManagerHandler

This inherits the DIRAC base Client for direct execution of server functionality. The following methods are available (although not visible here).

Transformation (table) manipulation

deleteTransformation(transName) getTransformationParameters(transName,paramNames) getTransformationWithStatus(status) setTransformationParameter(transName,paramName,paramValue) deleteTransformationParameter(transName,paramName)

TransformationFiles table manipulation

addFilesToTransformation(transName,lfns) addTaskForTransformation(transName,lfns=[],se=’Unknown’) getTransformationStats(transName)

TransformationTasks table manipulation

setTaskStatus(transName, taskID, status) setTaskStatusAndWmsID(transName, taskID, status, taskWmsID) getTransformationTaskStats(transName) deleteTasks(transName, taskMin, taskMax) extendTransformation( transName, nTasks) getTasksToSubmit(transName,numTasks,site=’’)

TransformationLogging table manipulation

getTransformationLogging(transName)

File/directory manipulation methods (the remainder of the interface can be found below)

getFileSummary(lfns) exists(lfns)

Web monitoring tools

getDistinctAttributeValues(attribute, selectDict) getTransformationStatusCounters() getTransformationSummary() getTransformationSummaryWeb(selectDict, sortList, startItem, maxItems)

__init__(**kwargs)

Simple constructor

addDirectory(path, force=False)

addFile(cls, fileDicts, force, **kwargs)

Interface provides { LFN1 : { PFN1, SE1, … }, LFN2 : { PFN2, SE2, … } }

Automatically created for the service function export_addFile()

Parameters:

• fileDicts – fileDicts

• force – force

addFilesToTransformation(self, transName, lfns, **kwargs)

Automatically created for the service function export_addFilesToTransformation()

Parameters:

• transName – transName

• lfns – lfns

addTaskForTransformation(self, transName, lfns, se, **kwargs)

Automatically created for the service function export_addTaskForTransformation()

Parameters:

• transName – transName

• lfns – lfns

• se – se

addTransformation(transName, description, longDescription, transType, plugin, agentType, fileMask, transformationGroup='General', groupSize=1, inheritedFrom=0, body='', maxTasks=0, eventsPerTask=0, addFiles=True, inputMetaQuery=None, outputMetaQuery=None, timeout=1800)

add a new transformation

property call

To be removed once we’re sure self.call has been removed

Type:

Deprecated

cleanTransformation(transID)

Clean the transformation, and set the status parameter (doing it here, for easier extensibility)

completeTransformation(transID)

Complete the transformation, and set the status parameter (doing it here, for easier extensibility)

createTransformationMetaQuery(self, transName, queryDict, queryType, **kwargs)

Automatically created for the service function export_createTransformationMetaQuery()

Parameters:

• transName – transName

• queryDict – queryDict

• queryType – queryType

deleteTasks(self, transName, taskMin, taskMax, **kwargs)

Automatically created for the service function export_deleteTasks()

Parameters:

• transName – transName

• taskMin – taskMin

• taskMax – taskMax

deleteTransformation(self, transName, **kwargs)

Automatically created for the service function export_deleteTransformation()

Parameters:

transName – transName

deleteTransformationMetaQuery(self, transName, queryType, **kwargs)

Automatically created for the service function export_deleteTransformationMetaQuery()

Parameters:

• transName – transName

• queryType – queryType

deleteTransformationParameter(self, transName, paramName, **kwargs)

Automatically created for the service function export_deleteTransformationParameter()

Parameters:

• transName – transName

• paramName – paramName

executeRPC(*parms, **kws)

This method extracts some parameters from kwargs that are used as parameter of the constructor or RPCClient. Unfortunately, only a few of all the available parameters of BaseClient are exposed.

Parameters:

• rpc – if an RPC client is passed, use that one

• timeout – we can change the timeout on a per call bases. Default is self.timeout

• url – We can specify which url to use

exists(cls, lfns, **kwargs)

Automatically created for the service function export_exists()

Parameters:

lfns – lfns

extendTransformation(self, transName, nTasks, **kwargs)

Automatically created for the service function export_extendTransformation()

Parameters:

• transName – transName

• nTasks – nTasks

getAdditionalParameters(self, transName, **kwargs)

Automatically created for the service function export_getAdditionalParameters()

Parameters:

transName – transName

getClientKWArgs()

Returns a copy of the connection arguments

getCounters(table, attrList, condDict, older=None, newer=None, timeStamp=None)

getDistinctAttributeValues(cls, attribute, selectDict, **kwargs)

Automatically created for the service function export_getDistinctAttributeValues()

Parameters:

• attribute – attribute

• selectDict – selectDict

getFileSummary(cls, lfns, **kwargs)

Automatically created for the service function export_getFileSummary()

Parameters:

lfns – lfns

getServer()

Getter for the server url. Useful ?

getTabbedSummaryWeb(self, table, requestedTables, selectDict, sortList, startItem, maxItems, **kwargs)

Automatically created for the service function export_getTabbedSummaryWeb()

Parameters:

• table – table

• requestedTables – requestedTables

• selectDict – selectDict

• sortList – sortList

• startItem – startItem

• maxItems – maxItems

getTableDistinctAttributeValues(cls, table, attributes, selectDict, **kwargs)

Automatically created for the service function export_getTableDistinctAttributeValues()

Parameters:

• table – table

• attributes – attributes

• selectDict – selectDict

getTasksToSubmit(self, transName, numTasks, site, **kwargs)

Retrieve the necessary information for the submission of a specified number of tasks for a given transformation. This includes reserving tasks to avoid race conditions.

Parameters:

• transName (int | str) – Name of the transformation

• numTasks (int) – Number of tasks to retrieve for submission

• site (str) – Optional site specification

Returns:

S_OK Dictionary containing transformation and task submission details

Automatically created for the service function export_getTasksToSubmit()

getTransformation(transName, extraParams=False)

getTransformationFiles(condDict=None, older=None, newer=None, timeStamp=None, orderAttribute=None, limit=None, timeout=1800, offset=0, maxfiles=None, columns=None)

gets all the transformation files for a transformation, incrementally. “limit” here is just used to determine the offset.

getTransformationFilesAsJsonString(self, **kwargs)

DEncode cannot cope with nested structures of multiple millions items. Encode everything as a json string, that DEncode can then transmit faster.

This will be the default as of v9.0

Automatically created for the service function export_getTransformationFilesAsJsonString()

getTransformationFilesCount(self, transName, field, selection, **kwargs)

Automatically created for the service function export_getTransformationFilesCount()

Parameters:

• transName – transName

• field – field

• selection – selection

getTransformationFilesSummaryWeb(self, selectDict, sortList, startItem, maxItems, **kwargs)

Automatically created for the service function export_getTransformationFilesSummaryWeb()

Parameters:

• selectDict – selectDict

• sortList – sortList

• startItem – startItem

• maxItems – maxItems

getTransformationLogging(self, transName, **kwargs)

Automatically created for the service function export_getTransformationLogging()

Parameters:

transName – transName

getTransformationMetaQuery(self, transName, queryType, **kwargs)

Automatically created for the service function export_getTransformationMetaQuery()

Parameters:

• transName – transName

• queryType – queryType

getTransformationParameters(self, transName, parameters, **kwargs)

Automatically created for the service function export_getTransformationParameters()

Parameters:

• transName – transName

• parameters – parameters

getTransformationStats(self, transName, **kwargs)

Automatically created for the service function export_getTransformationStats()

Parameters:

transName – transName

getTransformationStatusCounters(cls, **kwargs)

Automatically created for the service function export_getTransformationStatusCounters()

getTransformationSummary(self, **kwargs)

Get the summary of the currently existing transformations

Automatically created for the service function export_getTransformationSummary()

getTransformationSummaryWeb(self, selectDict, sortList, startItem, maxItems, **kwargs)

Get the summary of the transformation information for a given page in the generic format

Automatically created for the service function export_getTransformationSummaryWeb()

Parameters:

• selectDict – selectDict

• sortList – sortList

• startItem – startItem

• maxItems – maxItems

getTransformationTaskStats(self, transName, **kwargs)

Automatically created for the service function export_getTransformationTaskStats()

Parameters:

transName – transName

getTransformationTasks(condDict=None, older=None, newer=None, timeStamp=None, orderAttribute=None, limit=10000, inputVector=False)

gets all the transformation tasks for a transformation, incrementally. “limit” here is just used to determine the offset.

getTransformationTasksSummaryWeb(self, selectDict, sortList, startItem, maxItems, **kwargs)

Automatically created for the service function export_getTransformationTasksSummaryWeb()

Parameters:

• selectDict – selectDict

• sortList – sortList

• startItem – startItem

• maxItems – maxItems

getTransformationWithStatus(self, status, **kwargs)

Automatically created for the service function export_getTransformationWithStatus()

Parameters:

status – status

getTransformations(condDict=None, older=None, newer=None, timeStamp=None, orderAttribute=None, limit=100, extraParams=False, columns=None)

gets all the transformations in the system, incrementally. “limit” here is just used to determine the offset.

getTransformationsSummaryWeb(self, selectDict, sortList, startItem, maxItems, **kwargs)

Automatically created for the service function export_getTransformationsSummaryWeb()

Parameters:

• selectDict – selectDict

• sortList – sortList

• startItem – startItem

• maxItems – maxItems

httpsClient

alias of TornadoClient

isOK()

moveFilesToDerivedTransformation(transDict, resetUnused=True)

move files input to a transformation, to the derived one

removeFile(cls, lfns, **kwargs)

Interface provides [ LFN1, LFN2, … ]

Automatically created for the service function export_removeFile()

Parameters:

lfns – lfns

setFileStatusForTransformation(transName, newLFNsStatus=None, lfns=None, force=False)

Sets the file status for LFNs of a transformation

For backward compatibility purposes, the status and LFNs can be passed in 2 ways:

• newLFNsStatus is a dictionary with the form: {‘/this/is/an/lfn1.txt’: ‘StatusA’, ‘/this/is/an/lfn2.txt’: ‘StatusB’, … } and at this point lfns is not considered

• newLFNStatus is a string, that applies to all the LFNs in lfns

setMetadata(cls, path, querydict, **kwargs)

Set metadata to a file or to a directory (path)

Automatically created for the service function export_setMetadata()

Parameters:

• path – path

• querydict – querydict

setServer(url)

Set the server URL used by default

Parameters:

url – url of the service

setTaskStatus(self, transName, taskID, status, **kwargs)

Automatically created for the service function export_setTaskStatus()

Parameters:

• transName – transName

• taskID – taskID

• status – status

setTaskStatusAndWmsID(self, transName, taskID, status, taskWmsID, **kwargs)

Automatically created for the service function export_setTaskStatusAndWmsID()

Parameters:

• transName – transName

• taskID – taskID

• status – status

• taskWmsID – taskWmsID

setTransformationParameter(transID, paramName, paramValue, force=False, currentStatus=None)

Sets a transformation parameter. There’s a special case when coming to setting the status of a transformation.

Parameters:

currentStatus – if set, make sure the status did not change in the DB before setting it

startTransformation(transID)

Start the transformation

stopTransformation(transID)

Stop the transformation