TransformationManagerHandler

Service for interacting with TransformationDB

class DIRAC.TransformationSystem.Service.TransformationManagerHandler.TransformationManagerHandler(handlerInitDict, trid)

Bases: TransformationManagerHandlerMixin, RequestHandler

__init__(handlerInitDict, trid)

Constructor

Parameters:

• handlerInitDict (dictionary) – Information vars for the service

• trid (object) – Transport to use

auth_ping = ['all']

auth_refreshConfiguration = ['CSAdministrator']

auth_whoami = ['all']

checkPermissions(transName: str)

checks if remote user has permission to access to a given transformation

Parameters:

transName (str) – Name of the transformation to check

Returns:

S_ERROR if user does not have permission or if transformation does not exist S_OK otherwise

classmethod export_addDirectory(path, force=False)

classmethod export_addFile(fileDicts, force=False)

Interface provides { LFN1 : { PFN1, SE1, … }, LFN2 : { PFN2, SE2, … } }

export_addFilesToTransformation(transName, lfns)

export_addTaskForTransformation(transName, lfns=[], se='Unknown')

export_addTransformation(transName, description, longDescription, transType, plugin, agentType, fileMask, transformationGroup='General', groupSize=1, inheritedFrom=0, body='', maxTasks=0, eventsPerTask=0, addFiles=True, inputMetaQuery=None, outputMetaQuery=None)

export_cleanTransformation(transName)

export_completeTransformation(transName)

export_createTransformationMetaQuery(transName, queryDict, queryType)

export_deleteTasks(transName, taskMin, taskMax)

export_deleteTransformation(transName)

export_deleteTransformationMetaQuery(transName, queryType)

export_deleteTransformationParameter(transName, paramName)

static export_echo(data)

This method is used for testing performance of the service

Parameters:

data (str) – data to be sent back to the caller

Returns:

S_OK, Value is the input data

classmethod export_exists(lfns)

export_extendTransformation(transName, nTasks)

export_getAdditionalParameters(transName)

classmethod export_getCounters(table, attrList, condDict, older=None, newer=None, timeStamp=None)

classmethod export_getDistinctAttributeValues(attribute, selectDict)

classmethod export_getFileSummary(lfns)

export_getTabbedSummaryWeb(table, requestedTables, selectDict, sortList, startItem, maxItems)

classmethod export_getTableDistinctAttributeValues(table, attributes, selectDict)

export_getTasksToSubmit(transName, numTasks, site='')

Retrieve the necessary information for the submission of a specified number of tasks for a given transformation. This includes reserving tasks to avoid race conditions.

Parameters:

• transName (int | str) – Name of the transformation

• numTasks (int) – Number of tasks to retrieve for submission

• site (str) – Optional site specification

Returns:

S_OK Dictionary containing transformation and task submission details

export_getTransformation(transName, extraParams=False)

export_getTransformationFiles(condDict=None, older=None, newer=None, timeStamp='LastUpdate', orderAttribute=None, limit=None, offset=None, columns=None, include_web_records=True)

export_getTransformationFilesAsJsonString(*args, **kwargs)

DEncode cannot cope with nested structures of multiple millions items. Encode everything as a json string, that DEncode can then transmit faster.

This will be the default as of v9.0

export_getTransformationFilesCount(transName, field, selection={})

export_getTransformationFilesSummaryWeb(selectDict, sortList, startItem, maxItems)

export_getTransformationLogging(transName)

export_getTransformationMetaQuery(transName, queryType)

export_getTransformationParameters(transName, parameters)

export_getTransformationStats(transName)

classmethod export_getTransformationStatusCounters()

export_getTransformationSummary()

Get the summary of the currently existing transformations

export_getTransformationSummaryWeb(selectDict, sortList, startItem, maxItems)

Get the summary of the transformation information for a given page in the generic format

export_getTransformationTaskStats(transName)

classmethod export_getTransformationTasks(condDict=None, older=None, newer=None, timeStamp='CreationTime', orderAttribute=None, limit=None, inputVector=False, offset=None)

export_getTransformationTasksSummaryWeb(selectDict, sortList, startItem, maxItems)

export_getTransformationWithStatus(status)

classmethod export_getTransformations(condDict=None, older=None, newer=None, timeStamp='CreationDate', orderAttribute=None, limit=None, extraParams=False, offset=None, columns=None)

export_getTransformationsSummaryWeb(selectDict, sortList, startItem, maxItems)

export_ping()

static export_refreshConfiguration(fromMaster)

Force refreshing the configuration data

Parameters:

fromMaster (bool) – flag to refresh from the controller configuration service

classmethod export_removeFile(lfns)

Interface provides [ LFN1, LFN2, … ]

export_setFileStatusForTransformation(transName, dictOfNewFilesStatus)

Sets the file status for the transformation.

The dictOfNewFilesStatus is a dictionary with the form: {12345: (‘StatusA’, errorA), 6789: (‘StatusB’,errorB), … } where the keys are fileIDs The tuple may be a string with only the status if the client was from an older version

classmethod export_setMetadata(path, querydict)

Set metadata to a file or to a directory (path)

export_setTaskStatus(transName, taskID, status)

export_setTaskStatusAndWmsID(transName, taskID, status, taskWmsID)

export_setTransformationParameter(transName, paramName, paramValue)

export_whoami()

A simple whoami, returns all credential dictionary, except certificate chain object.

classmethod getCSOption(optionName, defaultValue=False)

Get an option from the CS section of the services

Returns:

Value for serviceSection/optionName in the CS being defaultValue the default

getRemoteAddress()

Get the address of the remote peer.

Returns:

Address of remote peer.

getRemoteCredentials()

Get the credentials of the remote peer.

Returns:

Credentials dictionary of remote peer.

initialize()

Initialize this instance of the handler (to be overwritten)

classmethod initializeHandler(serviceInfoDict)

Initialization of DB object

srv_disconnect(trid=None)

classmethod srv_disconnectClient(trid)

srv_getActionTuple()

classmethod srv_getCSOption(optionName, defaultValue=False)

Get an option from the CS section of the services

Returns:

Value for serviceSection/optionName in the CS being defaultValue the default

srv_getClientSetup()

srv_getClientVO()

srv_getClientVersion()

srv_getFormattedRemoteCredentials()

classmethod srv_getMonitor()

srv_getRemoteAddress()

Get the address of the remote peer.

Returns:

Address of remote peer.

srv_getRemoteCredentials()

Get the credentials of the remote peer.

Returns:

Credentials dictionary of remote peer.

classmethod srv_getServiceName()

srv_getTransportID()

classmethod srv_getURL()

classmethod srv_msgCreate(msgName)

srv_msgReply(msgObj)

classmethod srv_msgSend(trid, msgObj)

transfer_bulkFromClient(bulkId, token, bulkSize, fileHelper)

transfer_bulkToClient(bulkId, token, fileHelper)

transfer_fromClient(fileId, token, fileSize, fileHelper)

transfer_listBulk(bulkId, token, fileHelper)

transfer_toClient(fileId, token, fileHelper)

types_addDirectory = [<class 'str'>]

types_addFile = [[<class 'list'>, <class 'dict'>, <class 'str'>]]

types_addFilesToTransformation = [[<class 'int'>, <class 'str'>], [<class 'list'>, <class 'tuple'>]]

types_addTaskForTransformation = [[<class 'int'>, <class 'str'>]]

types_addTransformation = [<class 'str'>, <class 'str'>, <class 'str'>, <class 'str'>, <class 'str'>, <class 'str'>, <class 'str'>]

types_cleanTransformation = [[<class 'int'>, <class 'str'>]]

types_completeTransformation = [[<class 'int'>, <class 'str'>]]

types_createTransformationMetaQuery = [[<class 'int'>, <class 'str'>], <class 'dict'>, <class 'str'>]

types_deleteTasks = [[<class 'int'>, <class 'str'>], <class 'int'>, <class 'int'>]

types_deleteTransformation = [[<class 'int'>, <class 'str'>]]

types_deleteTransformationMetaQuery = [[<class 'int'>, <class 'str'>], <class 'str'>]

types_deleteTransformationParameter = [[<class 'int'>, <class 'str'>], <class 'str'>]

types_echo = [<class 'str'>]

types_exists = [<class 'list'>]

types_extendTransformation = [[<class 'int'>, <class 'str'>], <class 'int'>]

types_getAdditionalParameters = [[<class 'int'>, <class 'str'>]]

types_getCounters = [<class 'str'>, <class 'list'>, <class 'dict'>]

types_getDistinctAttributeValues = [<class 'str'>, <class 'dict'>]

types_getFileSummary = [<class 'list'>]

types_getTabbedSummaryWeb = [<class 'str'>, <class 'dict'>, <class 'dict'>, <class 'list'>, <class 'int'>, <class 'int'>]

types_getTableDistinctAttributeValues = [<class 'str'>, <class 'list'>, <class 'dict'>]

types_getTasksToSubmit = [[<class 'int'>, <class 'str'>], <class 'int'>]

types_getTransformation = [[<class 'int'>, <class 'str'>]]

types_getTransformationFiles = []

types_getTransformationFilesAsJsonString = []

types_getTransformationFilesCount = [[<class 'int'>, <class 'str'>], <class 'str'>]

types_getTransformationFilesSummaryWeb = [<class 'dict'>, <class 'list'>, <class 'int'>, <class 'int'>]

types_getTransformationLogging = [[<class 'int'>, <class 'str'>]]

types_getTransformationMetaQuery = [[<class 'int'>, <class 'str'>], <class 'str'>]

types_getTransformationParameters = [[<class 'int'>, <class 'str'>], [<class 'str'>, <class 'list'>]]

types_getTransformationStats = [[<class 'int'>, <class 'str'>]]

types_getTransformationStatusCounters = []

types_getTransformationSummary = []

types_getTransformationSummaryWeb = [<class 'dict'>, <class 'list'>, <class 'int'>, <class 'int'>]

types_getTransformationTaskStats = [[<class 'int'>, <class 'str'>]]

types_getTransformationTasks = []

types_getTransformationTasksSummaryWeb = [<class 'dict'>, <class 'list'>, <class 'int'>, <class 'int'>]

types_getTransformationWithStatus = [[<class 'str'>, <class 'list'>, <class 'tuple'>]]

types_getTransformations = []

types_getTransformationsSummaryWeb = [<class 'dict'>, <class 'list'>, <class 'int'>, <class 'int'>]

types_ping = []

types_refreshConfiguration = [<class 'bool'>]

types_removeFile = [[<class 'list'>, <class 'dict'>]]

types_setFileStatusForTransformation = [[<class 'int'>, <class 'str'>], <class 'dict'>]

types_setMetadata = [<class 'str'>, <class 'dict'>]

types_setTaskStatus = [[<class 'int'>, <class 'str'>], [<class 'list'>, <class 'int'>], <class 'str'>]

types_setTaskStatusAndWmsID = [[<class 'int'>, <class 'str'>], <class 'int'>, <class 'str'>, <class 'str'>]

types_setTransformationParameter = [[<class 'int'>, <class 'str'>], <class 'str'>]

types_whoami = []

class DIRAC.TransformationSystem.Service.TransformationManagerHandler.TransformationManagerHandlerMixin

Bases: object

checkPermissions(transName: str)

checks if remote user has permission to access to a given transformation

Parameters:

transName (str) – Name of the transformation to check

Returns:

S_ERROR if user does not have permission or if transformation does not exist S_OK otherwise

classmethod export_addDirectory(path, force=False)

classmethod export_addFile(fileDicts, force=False)

Interface provides { LFN1 : { PFN1, SE1, … }, LFN2 : { PFN2, SE2, … } }

export_addFilesToTransformation(transName, lfns)

export_addTaskForTransformation(transName, lfns=[], se='Unknown')

export_addTransformation(transName, description, longDescription, transType, plugin, agentType, fileMask, transformationGroup='General', groupSize=1, inheritedFrom=0, body='', maxTasks=0, eventsPerTask=0, addFiles=True, inputMetaQuery=None, outputMetaQuery=None)

export_cleanTransformation(transName)

export_completeTransformation(transName)

export_createTransformationMetaQuery(transName, queryDict, queryType)

export_deleteTasks(transName, taskMin, taskMax)

export_deleteTransformation(transName)

export_deleteTransformationMetaQuery(transName, queryType)

export_deleteTransformationParameter(transName, paramName)

classmethod export_exists(lfns)

export_extendTransformation(transName, nTasks)

export_getAdditionalParameters(transName)

classmethod export_getCounters(table, attrList, condDict, older=None, newer=None, timeStamp=None)

classmethod export_getDistinctAttributeValues(attribute, selectDict)

classmethod export_getFileSummary(lfns)

export_getTabbedSummaryWeb(table, requestedTables, selectDict, sortList, startItem, maxItems)

classmethod export_getTableDistinctAttributeValues(table, attributes, selectDict)

export_getTasksToSubmit(transName, numTasks, site='')

Retrieve the necessary information for the submission of a specified number of tasks for a given transformation. This includes reserving tasks to avoid race conditions.

Parameters:

• transName (int | str) – Name of the transformation

• numTasks (int) – Number of tasks to retrieve for submission

• site (str) – Optional site specification

Returns:

S_OK Dictionary containing transformation and task submission details

export_getTransformation(transName, extraParams=False)

export_getTransformationFiles(condDict=None, older=None, newer=None, timeStamp='LastUpdate', orderAttribute=None, limit=None, offset=None, columns=None, include_web_records=True)

export_getTransformationFilesAsJsonString(*args, **kwargs)

DEncode cannot cope with nested structures of multiple millions items. Encode everything as a json string, that DEncode can then transmit faster.

This will be the default as of v9.0

export_getTransformationFilesCount(transName, field, selection={})

export_getTransformationFilesSummaryWeb(selectDict, sortList, startItem, maxItems)

export_getTransformationLogging(transName)

export_getTransformationMetaQuery(transName, queryType)

export_getTransformationParameters(transName, parameters)

export_getTransformationStats(transName)

classmethod export_getTransformationStatusCounters()

export_getTransformationSummary()

Get the summary of the currently existing transformations

export_getTransformationSummaryWeb(selectDict, sortList, startItem, maxItems)

Get the summary of the transformation information for a given page in the generic format

export_getTransformationTaskStats(transName)

classmethod export_getTransformationTasks(condDict=None, older=None, newer=None, timeStamp='CreationTime', orderAttribute=None, limit=None, inputVector=False, offset=None)

export_getTransformationTasksSummaryWeb(selectDict, sortList, startItem, maxItems)

export_getTransformationWithStatus(status)

classmethod export_getTransformations(condDict=None, older=None, newer=None, timeStamp='CreationDate', orderAttribute=None, limit=None, extraParams=False, offset=None, columns=None)

export_getTransformationsSummaryWeb(selectDict, sortList, startItem, maxItems)

classmethod export_removeFile(lfns)

Interface provides [ LFN1, LFN2, … ]

export_setFileStatusForTransformation(transName, dictOfNewFilesStatus)

Sets the file status for the transformation.

The dictOfNewFilesStatus is a dictionary with the form: {12345: (‘StatusA’, errorA), 6789: (‘StatusB’,errorB), … } where the keys are fileIDs The tuple may be a string with only the status if the client was from an older version

classmethod export_setMetadata(path, querydict)

Set metadata to a file or to a directory (path)

export_setTaskStatus(transName, taskID, status)

export_setTaskStatusAndWmsID(transName, taskID, status, taskWmsID)

export_setTransformationParameter(transName, paramName, paramValue)

classmethod initializeHandler(serviceInfoDict)

Initialization of DB object

types_addDirectory = [<class 'str'>]

types_addFile = [[<class 'list'>, <class 'dict'>, <class 'str'>]]

types_addFilesToTransformation = [[<class 'int'>, <class 'str'>], [<class 'list'>, <class 'tuple'>]]

types_addTaskForTransformation = [[<class 'int'>, <class 'str'>]]

types_addTransformation = [<class 'str'>, <class 'str'>, <class 'str'>, <class 'str'>, <class 'str'>, <class 'str'>, <class 'str'>]

types_cleanTransformation = [[<class 'int'>, <class 'str'>]]

types_completeTransformation = [[<class 'int'>, <class 'str'>]]

types_createTransformationMetaQuery = [[<class 'int'>, <class 'str'>], <class 'dict'>, <class 'str'>]

types_deleteTasks = [[<class 'int'>, <class 'str'>], <class 'int'>, <class 'int'>]

types_deleteTransformation = [[<class 'int'>, <class 'str'>]]

types_deleteTransformationMetaQuery = [[<class 'int'>, <class 'str'>], <class 'str'>]

types_deleteTransformationParameter = [[<class 'int'>, <class 'str'>], <class 'str'>]

types_exists = [<class 'list'>]

types_extendTransformation = [[<class 'int'>, <class 'str'>], <class 'int'>]

types_getAdditionalParameters = [[<class 'int'>, <class 'str'>]]

types_getCounters = [<class 'str'>, <class 'list'>, <class 'dict'>]

types_getDistinctAttributeValues = [<class 'str'>, <class 'dict'>]

types_getFileSummary = [<class 'list'>]

types_getTabbedSummaryWeb = [<class 'str'>, <class 'dict'>, <class 'dict'>, <class 'list'>, <class 'int'>, <class 'int'>]

types_getTableDistinctAttributeValues = [<class 'str'>, <class 'list'>, <class 'dict'>]

types_getTasksToSubmit = [[<class 'int'>, <class 'str'>], <class 'int'>]

types_getTransformation = [[<class 'int'>, <class 'str'>]]

types_getTransformationFiles = []

types_getTransformationFilesAsJsonString = []

types_getTransformationFilesCount = [[<class 'int'>, <class 'str'>], <class 'str'>]

types_getTransformationFilesSummaryWeb = [<class 'dict'>, <class 'list'>, <class 'int'>, <class 'int'>]

types_getTransformationLogging = [[<class 'int'>, <class 'str'>]]

types_getTransformationMetaQuery = [[<class 'int'>, <class 'str'>], <class 'str'>]

types_getTransformationParameters = [[<class 'int'>, <class 'str'>], [<class 'str'>, <class 'list'>]]

types_getTransformationStats = [[<class 'int'>, <class 'str'>]]

types_getTransformationStatusCounters = []

types_getTransformationSummary = []

types_getTransformationSummaryWeb = [<class 'dict'>, <class 'list'>, <class 'int'>, <class 'int'>]

types_getTransformationTaskStats = [[<class 'int'>, <class 'str'>]]

types_getTransformationTasks = []

types_getTransformationTasksSummaryWeb = [<class 'dict'>, <class 'list'>, <class 'int'>, <class 'int'>]

types_getTransformationWithStatus = [[<class 'str'>, <class 'list'>, <class 'tuple'>]]

types_getTransformations = []

types_getTransformationsSummaryWeb = [<class 'dict'>, <class 'list'>, <class 'int'>, <class 'int'>]

types_removeFile = [[<class 'list'>, <class 'dict'>]]

types_setFileStatusForTransformation = [[<class 'int'>, <class 'str'>], <class 'dict'>]

types_setMetadata = [<class 'str'>, <class 'dict'>]

types_setTaskStatus = [[<class 'int'>, <class 'str'>], [<class 'list'>, <class 'int'>], <class 'str'>]

types_setTaskStatusAndWmsID = [[<class 'int'>, <class 'str'>], <class 'int'>, <class 'str'>, <class 'str'>]

types_setTransformationParameter = [[<class 'int'>, <class 'str'>], <class 'str'>]