StorageManagerHandler

StorageManagerHandler is the implementation of the StorageManagementDB in the DISET framework

class DIRAC.StorageManagementSystem.Service.StorageManagerHandler.StorageManagerHandler(handlerInitDict, trid)

Bases: StorageManagerHandlerMixin, RequestHandler

__init__(handlerInitDict, trid)

Constructor

Parameters:

• handlerInitDict (dictionary) – Information vars for the service

• trid (object) – Transport to use

auth_ping = ['all']

auth_refreshConfiguration = ['CSAdministrator']

auth_whoami = ['all']

static export_echo(data)

This method is used for testing performance of the service

Parameters:

data (str) – data to be sent back to the caller

Returns:

S_OK, Value is the input data

classmethod export_getAssociatedReplicas(replicaIDs)

Retrieve the list of Replicas that belong to the same Tasks as the provided list

classmethod export_getCacheReplicas(condDict, older=None, newer=None, timeStamp='LastUpdate', orderAttribute=None, limit=None)

Get the replcias known to the DB.

classmethod export_getCacheReplicasSummary()

Reports breakdown of file number/size in different staging states across storage elements

classmethod export_getOfflineReplicas()

This method obtains the replicas for which all replicas in the task are Offline

classmethod export_getReplicasWithStatus(status)

This method allows to retrieve replicas with the supplied status

classmethod export_getStageRequests(condDict, older=None, newer=None, timeStamp='StageRequestSubmitTime', orderAttribute=None, limit=None)

Get the replcias known to the DB.

classmethod export_getStageSubmittedReplicas()

This method obtains the replica metadata and the stage requestID for the replicas in StageSubmitted status

classmethod export_getStagedReplicas()

This method obtains the replicas for which all replicas in the task are Staged/StageSubmitted

classmethod export_getSubmittedStagePins()

This method obtains the number of files and size of the requests submitted for each storage element

classmethod export_getTaskInfo(taskID)

Obtain the metadata of the stage task from the DB.

classmethod export_getTaskStatus(taskID)

Obtain the status of the stage task from the DB.

classmethod export_getTaskSummary(taskID)

Obtain the summary of the stage task from the DB.

classmethod export_getTasks(condDict, older=None, newer=None, timeStamp='LastUpdate', orderAttribute=None, limit=None)

Get the replicas known to the DB.

classmethod export_getTasksWithStatus(status)

This method allows to retrieve Tasks with the supplied status

classmethod export_getWaitingReplicas()

This method obtains the replicas for which all replicas in the task are Waiting

classmethod export_insertStageRequest(requestReplicas, pinLifetime)

This method inserts the stage request ID assocaited to supplied replicaIDs

classmethod export_killTasksBySourceTaskID(sourceTaskIDs)

Given SourceTaskIDs (jobIDs), this will cancel further staging of files for the corresponding tasks

export_ping()

static export_refreshConfiguration(fromMaster)

Force refreshing the configuration data

Parameters:

fromMaster (bool) – flag to refresh from the master configuration service

classmethod export_removeStageRequests(replicaIDs)

classmethod export_removeTasks(taskIDs)

This method removes the entries from TaskReplicas and Tasks with the supplied task IDs

classmethod export_removeUnlinkedReplicas()

This method removes Replicas which have no associated Tasks

classmethod export_setOldTasksAsFailed(daysOld)

Set Tasks older than “daysOld” number of days to Failed These tasks have already been retried every day for staging

classmethod export_setRequest(lfnDict, source, callbackMethod, taskID)

This method allows stage requests to be set into the StagerDB

classmethod export_setStageComplete(replicaIDs)

This method updates the status of the stage request for the supplied replica IDs

classmethod export_setTasksDone(taskIDs)

This method sets the status in the Tasks table to Done for the list of supplied task IDs

classmethod export_updateReplicaFailure(replicaFailures)

This method sets the status of the replica to failed with the supplied reason

classmethod export_updateReplicaInformation(replicaTuples)

This method sets the pfn and size for the supplied replicas

classmethod export_updateReplicaStatus(replicaIDs, newReplicaStatus)

This allows to update the status of replicas

classmethod export_updateTaskStatus(sourceID, status, successful=[], failed=[])

An example to show the usage of the callbacks.

classmethod export_wakeupOldRequests(oldRequests, retryInterval)

get only StageRequests with StageRequestSubmitTime older than 1 day AND are still not staged delete these requests reset Replicas with corresponding ReplicaIDs to Status=’New’

export_whoami()

A simple whoami, returns all credential dictionary, except certificate chain object.

classmethod getCSOption(optionName, defaultValue=False)

Get an option from the CS section of the services

Returns:

Value for serviceSection/optionName in the CS being defaultValue the default

getRemoteAddress()

Get the address of the remote peer.

Returns:

Address of remote peer.

getRemoteCredentials()

Get the credentials of the remote peer.

Returns:

Credentials dictionary of remote peer.

initialize()

Initialize this instance of the handler (to be overwritten)

classmethod initializeHandler(serviceInfoDict)

Initialization of DB object

srv_disconnect(trid=None)

classmethod srv_disconnectClient(trid)

srv_getActionTuple()

classmethod srv_getCSOption(optionName, defaultValue=False)

Get an option from the CS section of the services

Returns:

Value for serviceSection/optionName in the CS being defaultValue the default

srv_getClientSetup()

srv_getClientVO()

srv_getClientVersion()

srv_getFormattedRemoteCredentials()

classmethod srv_getMonitor()

srv_getRemoteAddress()

Get the address of the remote peer.

Returns:

Address of remote peer.

srv_getRemoteCredentials()

Get the credentials of the remote peer.

Returns:

Credentials dictionary of remote peer.

classmethod srv_getServiceName()

srv_getTransportID()

classmethod srv_getURL()

classmethod srv_msgCreate(msgName)

srv_msgReply(msgObj)

classmethod srv_msgSend(trid, msgObj)

transfer_bulkFromClient(bulkId, token, bulkSize, fileHelper)

transfer_bulkToClient(bulkId, token, fileHelper)

transfer_fromClient(fileId, token, fileSize, fileHelper)

transfer_listBulk(bulkId, token, fileHelper)

transfer_toClient(fileId, token, fileHelper)

types_echo = [<class 'str'>]

types_getAssociatedReplicas = [<class 'list'>]

types_getCacheReplicas = [<class 'dict'>]

types_getCacheReplicasSummary = []

types_getOfflineReplicas = []

types_getReplicasWithStatus = [(<class 'str'>,)]

types_getStageRequests = [<class 'dict'>]

types_getStageSubmittedReplicas = []

types_getStagedReplicas = []

types_getSubmittedStagePins = []

types_getTaskInfo = [(<class 'int'>,)]

types_getTaskStatus = [(<class 'int'>,)]

types_getTaskSummary = [(<class 'int'>,)]

types_getTasks = [<class 'dict'>]

types_getTasksWithStatus = [(<class 'str'>,)]

types_getWaitingReplicas = []

types_insertStageRequest = [<class 'dict'>, [(<class 'int'>,), (<class 'int'>,)]]

types_killTasksBySourceTaskID = [<class 'list'>]

types_ping = []

types_refreshConfiguration = [<class 'bool'>]

types_removeStageRequests = [<class 'list'>]

types_removeTasks = [<class 'list'>]

types_removeUnlinkedReplicas = []

types_setOldTasksAsFailed = [(<class 'int'>,)]

types_setRequest = [<class 'dict'>, (<class 'str'>,), (<class 'str'>,), (<class 'int'>,)]

types_setStageComplete = [<class 'list'>]

types_setTasksDone = [<class 'list'>]

types_updateReplicaFailure = [<class 'dict'>]

types_updateReplicaInformation = [<class 'list'>]

types_updateReplicaStatus = [<class 'list'>, (<class 'str'>,)]

types_updateTaskStatus = []

types_wakeupOldRequests = [<class 'list'>, (<class 'int'>,)]

types_whoami = []

class DIRAC.StorageManagementSystem.Service.StorageManagerHandler.StorageManagerHandlerMixin

Bases: object

classmethod export_getAssociatedReplicas(replicaIDs)

Retrieve the list of Replicas that belong to the same Tasks as the provided list

classmethod export_getCacheReplicas(condDict, older=None, newer=None, timeStamp='LastUpdate', orderAttribute=None, limit=None)

Get the replcias known to the DB.

classmethod export_getCacheReplicasSummary()

Reports breakdown of file number/size in different staging states across storage elements

classmethod export_getOfflineReplicas()

This method obtains the replicas for which all replicas in the task are Offline

classmethod export_getReplicasWithStatus(status)

This method allows to retrieve replicas with the supplied status

classmethod export_getStageRequests(condDict, older=None, newer=None, timeStamp='StageRequestSubmitTime', orderAttribute=None, limit=None)

Get the replcias known to the DB.

classmethod export_getStageSubmittedReplicas()

This method obtains the replica metadata and the stage requestID for the replicas in StageSubmitted status

classmethod export_getStagedReplicas()

This method obtains the replicas for which all replicas in the task are Staged/StageSubmitted

classmethod export_getSubmittedStagePins()

This method obtains the number of files and size of the requests submitted for each storage element

classmethod export_getTaskInfo(taskID)

Obtain the metadata of the stage task from the DB.

classmethod export_getTaskStatus(taskID)

Obtain the status of the stage task from the DB.

classmethod export_getTaskSummary(taskID)

Obtain the summary of the stage task from the DB.

classmethod export_getTasks(condDict, older=None, newer=None, timeStamp='LastUpdate', orderAttribute=None, limit=None)

Get the replicas known to the DB.

classmethod export_getTasksWithStatus(status)

This method allows to retrieve Tasks with the supplied status

classmethod export_getWaitingReplicas()

This method obtains the replicas for which all replicas in the task are Waiting

classmethod export_insertStageRequest(requestReplicas, pinLifetime)

This method inserts the stage request ID assocaited to supplied replicaIDs

classmethod export_killTasksBySourceTaskID(sourceTaskIDs)

Given SourceTaskIDs (jobIDs), this will cancel further staging of files for the corresponding tasks

classmethod export_removeStageRequests(replicaIDs)

classmethod export_removeTasks(taskIDs)

This method removes the entries from TaskReplicas and Tasks with the supplied task IDs

classmethod export_removeUnlinkedReplicas()

This method removes Replicas which have no associated Tasks

classmethod export_setOldTasksAsFailed(daysOld)

Set Tasks older than “daysOld” number of days to Failed These tasks have already been retried every day for staging

classmethod export_setRequest(lfnDict, source, callbackMethod, taskID)

This method allows stage requests to be set into the StagerDB

classmethod export_setStageComplete(replicaIDs)

This method updates the status of the stage request for the supplied replica IDs

classmethod export_setTasksDone(taskIDs)

This method sets the status in the Tasks table to Done for the list of supplied task IDs

classmethod export_updateReplicaFailure(replicaFailures)

This method sets the status of the replica to failed with the supplied reason

classmethod export_updateReplicaInformation(replicaTuples)

This method sets the pfn and size for the supplied replicas

classmethod export_updateReplicaStatus(replicaIDs, newReplicaStatus)

This allows to update the status of replicas

classmethod export_updateTaskStatus(sourceID, status, successful=[], failed=[])

An example to show the usage of the callbacks.

classmethod export_wakeupOldRequests(oldRequests, retryInterval)

get only StageRequests with StageRequestSubmitTime older than 1 day AND are still not staged delete these requests reset Replicas with corresponding ReplicaIDs to Status=’New’

classmethod initializeHandler(serviceInfoDict)

Initialization of DB object

types_getAssociatedReplicas = [<class 'list'>]

types_getCacheReplicas = [<class 'dict'>]

types_getCacheReplicasSummary = []

types_getOfflineReplicas = []

types_getReplicasWithStatus = [(<class 'str'>,)]

types_getStageRequests = [<class 'dict'>]

types_getStageSubmittedReplicas = []

types_getStagedReplicas = []

types_getSubmittedStagePins = []

types_getTaskInfo = [(<class 'int'>,)]

types_getTaskStatus = [(<class 'int'>,)]

types_getTaskSummary = [(<class 'int'>,)]

types_getTasks = [<class 'dict'>]

types_getTasksWithStatus = [(<class 'str'>,)]

types_getWaitingReplicas = []

types_insertStageRequest = [<class 'dict'>, [(<class 'int'>,), (<class 'int'>,)]]

types_killTasksBySourceTaskID = [<class 'list'>]

types_removeStageRequests = [<class 'list'>]

types_removeTasks = [<class 'list'>]

types_removeUnlinkedReplicas = []

types_setOldTasksAsFailed = [(<class 'int'>,)]

types_setRequest = [<class 'dict'>, (<class 'str'>,), (<class 'str'>,), (<class 'int'>,)]

types_setStageComplete = [<class 'list'>]

types_setTasksDone = [<class 'list'>]

types_updateReplicaFailure = [<class 'dict'>]

types_updateReplicaInformation = [<class 'list'>]

types_updateReplicaStatus = [<class 'list'>, (<class 'str'>,)]

types_updateTaskStatus = []

types_wakeupOldRequests = [<class 'list'>, (<class 'int'>,)]