FileCatalogHandler

mod:FileCatalogHandler
class DIRAC.DataManagementSystem.Service.FileCatalogHandler.FileCatalogHandler(handlerInitDict, trid)

Bases: DIRAC.Core.DISET.RequestHandler.RequestHandler

..class:: FileCatalogHandler

A simple Replica and Metadata Catalog service.

exception ConnectionError(msg)

Bases: exceptions.Exception

__init__(msg)

x.__init__(…) initializes x; see help(type(x)) for signature

args
message
__init__(handlerInitDict, trid)

Constructor

Parameters:
  • handlerInitDict (dictionary) – Information vars for the service
  • trid (object) – Transport to use
auth_ping = ['all']
export_addDataset(datasets)

Add a new dynamic dataset defined by its meta query

export_addDatasetAnnotation(datasetDict)

Add annotation to an already created dataset

export_addFile(lfns)

Register supplied files

export_addFileAncestors(lfns)

Add file ancestor information for the given list of LFNs

export_addGroup(groupName)

Add a new group to the File Catalog

export_addMetadataField(fieldName, fieldType, metaType='-d')

Add a new metadata field of the given type

export_addMetadataSet(setName, setDict)

Add a new metadata set

export_addReplica(lfns)

Register supplied replicas

export_addUser(userName)

Add a new user to the File Catalog

export_changePathGroup(lfns, recursive=False)

Get replica info for the given list of LFNs

export_changePathMode(lfns, recursive=False)

Get replica info for the given list of LFNs

export_changePathOwner(lfns, recursive=False)

Get replica info for the given list of LFNs

export_checkDataset(datasets)

Check the given dynamic dataset for changes since its definition

export_createDirectory(lfns)

Create the supplied directories

export_deleteGroup(groupName)

Delete group from the File Catalog

export_deleteMetadataField(fieldName)

Delete the metadata field

export_deleteUser(userName)

Delete user from the File Catalog

static export_echo(data)

This method used for testing the performance of a service

export_exists(lfns)

Check whether the supplied paths exists

export_findDirectoriesByMetadata(metaDict, path='/')

Find all the directories satisfying the given metadata set

export_findFilesByMetadata(metaDict, path='/')

Find all the files satisfying the given metadata set

export_findFilesByMetadataDetailed(metaDict, path='/')

Find all the files satisfying the given metadata set

export_findFilesByMetadataWeb(metaDict, path, startItem, maxItems)

Find files satisfying the given metadata set

export_freezeDataset(datasets)

Freeze the contents of the dataset making it effectively static

export_getCatalogCounters()

Get the number of registered directories, files and replicas in various tables

export_getCompatibleMetadata(metaDict, path='/')

Get metadata values compatible with the given metadata subset

export_getDatasetAnnotation(datasets)

Get annotation of the given datasets

export_getDatasetFiles(datasets)

Get lfns in the given dataset

export_getDatasetParameters(datasets)

Get parameters of the given dynamic dataset as they are stored in the database

export_getDatasets(datasets)

Get parameters of the given dynamic dataset as they are stored in the database

export_getDirectoryMetadata(lfns)

Get the size of the supplied directory

export_getDirectoryReplicas(lfns, allStatus=False)

Get replicas for files in the supplied directory

export_getDirectorySize(lfns, longOut=False, fromFiles=False)

Get the size of the supplied directory

export_getDirectoryUserMetadata(path)

Get all the metadata valid for the given directory path

export_getFileAncestors(lfns, depths)

Get the status for the supplied replicas

export_getFileDescendents(lfns, depths)

Get the status for the supplied replicas

export_getFileMetadata(lfns)

Get the metadata associated to supplied lfns

export_getFileSize(lfns)

Get the size associated to supplied lfns

export_getFileUserMetadata(path)

Get all the metadata valid for the given file

export_getGroups()

Get all the groups defined in the File Catalog

export_getLFNForGUID(guids)

Get the matching lfns for given guids

export_getMetadataFields()

Get all the metadata fields

export_getMetadataSet(setName, expandFlag)

Add a new metadata set

export_getPathPermissions(lfns)

Determine the ACL information for a supplied path

export_getReplicaStatus(lfns)

Get the status for the supplied replicas

export_getReplicas(lfns, allStatus=False)

Get replicas for supplied lfns

export_getReplicasByMetadata(metaDict, path='/', allStatus=False)

Find all the files satisfying the given metadata set

export_getUsers()

Get all the users defined in the File Catalog

export_hasAccess(paths, opType)

Determine if the given op can be performed on the paths The OpType is all the operations exported

The reason for the param types is backward compatibility. Between v6r14 and v6r15, the signature of hasAccess has changed, and the two parameters were swapped.

export_isDirectory(lfns)

Determine whether supplied path is a directory

export_isFile(lfns)

Check whether the supplied lfns are files

static export_isOK()

returns S_OK if DB is connected

export_listDirectory(lfns, verbose)

List the contents of supplied directories

export_ping()
static export_rebuildDirectoryUsage()

Rebuild DirectoryUsage table from scratch

export_releaseDataset(datasets)

Release the contents of the frozen dataset allowing changes in its contents

export_removeDataset(datasets)

Check the given dynamic dataset for changes since its definition

export_removeDirectory(lfns)

Remove the supplied directories

export_removeFile(lfns)

Remove the supplied lfns

export_removeMetadata(pathMetadataDict)

Remove the specified metadata for the given path

export_removeReplica(lfns)

Remove the supplied replicas

export_repairCatalog()

Repair the catalog inconsistencies

export_setFileStatus(lfns)

Remove the supplied lfns

export_setMetadata(path, metadatadict)

Set metadata parameter for the given path

export_setMetadataBulk(pathMetadataDict)

Set metadata parameter for the given path

export_setReplicaHost(lfns)

Change the registered SE for the supplied replicas

export_setReplicaStatus(lfns)

Set the status for the supplied replicas

export_updateDataset(datasets)

Update the given dynamic dataset for changes since its definition

findFilesByMetadataWeb(metaDict, path, startItem, maxItems)

Find all the files satisfying the given metadata set

classmethod getCSOption(optionName, defaultValue=False)

Get an option from the CS section of the services

Returns:Value for serviceSection/optionName in the CS being defaultValue the default
getRemoteAddress()

Get the address of the remote peer.

Returns:Address of remote peer.
getRemoteCredentials()

Get the credentials of the remote peer.

Returns:Credentials dictionary of remote peer.
getSEDump(seName)
Return all the files at a given SE, together with checksum and size
Parameters:seName – name of the StorageElement
Returns:S_OK with list of tuples (lfn, checksum, size)
initialize()

Initialize this instance of the handler (to be overwritten)

srv_disconnect(trid=None)
classmethod srv_disconnectClient(trid)
srv_getActionTuple()
classmethod srv_getCSOption(optionName, defaultValue=False)

Get an option from the CS section of the services

Returns:Value for serviceSection/optionName in the CS being defaultValue the default
srv_getClientSetup()
srv_getClientVO()
srv_getFormattedRemoteCredentials()
classmethod srv_getMonitor()
srv_getRemoteAddress()

Get the address of the remote peer.

Returns:Address of remote peer.
srv_getRemoteCredentials()

Get the credentials of the remote peer.

Returns:Credentials dictionary of remote peer.
classmethod srv_getServiceName()
srv_getTransportID()
classmethod srv_getURL()
classmethod srv_msgCreate(msgName)
srv_msgReply(msgObj)
classmethod srv_msgSend(trid, msgObj)
transfer_bulkFromClient(bulkId, token, bulkSize, fileHelper)
transfer_bulkToClient(bulkId, token, fileHelper)
transfer_fromClient(fileId, token, fileSize, fileHelper)
transfer_listBulk(bulkId, token, fileHelper)
transfer_toClient(seName, token, fileHelper)

This method used to transfer the SEDump to the client, formated as CSV with ‘|’ separation

Parameters:seName – name of the se to dump
Returns:the result of the FileHelper
types_addDataset = [<type 'dict'>]
types_addDatasetAnnotation = [<type 'dict'>]
types_addFile = [[<type 'list'>, <type 'dict'>, <type 'str'>, <type 'unicode'>]]
types_addFileAncestors = [<type 'dict'>]
types_addGroup = [(<type 'str'>, <type 'unicode'>)]
types_addMetadataField = [(<type 'str'>, <type 'unicode'>), (<type 'str'>, <type 'unicode'>)]
types_addMetadataSet = [(<type 'str'>, <type 'unicode'>), <type 'dict'>]
types_addReplica = [[<type 'list'>, <type 'dict'>, <type 'str'>, <type 'unicode'>]]
types_addUser = [(<type 'str'>, <type 'unicode'>)]
types_changePathGroup = [[<type 'list'>, <type 'dict'>, <type 'str'>, <type 'unicode'>]]
types_changePathMode = [[<type 'list'>, <type 'dict'>, <type 'str'>, <type 'unicode'>]]
types_changePathOwner = [[<type 'list'>, <type 'dict'>, <type 'str'>, <type 'unicode'>]]
types_checkDataset = [<type 'dict'>]
types_createDirectory = [[<type 'list'>, <type 'dict'>, <type 'str'>, <type 'unicode'>]]
types_deleteGroup = [(<type 'str'>, <type 'unicode'>)]
types_deleteMetadataField = [(<type 'str'>, <type 'unicode'>)]
types_deleteUser = [(<type 'str'>, <type 'unicode'>)]
types_echo = [<type 'basestring'>]
types_exists = [[<type 'list'>, <type 'dict'>, <type 'str'>, <type 'unicode'>]]
types_findDirectoriesByMetadata = [<type 'dict'>]
types_findFilesByMetadata = [<type 'dict'>, (<type 'str'>, <type 'unicode'>)]
types_findFilesByMetadataDetailed = [<type 'dict'>, (<type 'str'>, <type 'unicode'>)]
types_findFilesByMetadataWeb = [<type 'dict'>, (<type 'str'>, <type 'unicode'>), [<type 'int'>, <type 'long'>], [<type 'int'>, <type 'long'>]]
types_freezeDataset = [<type 'dict'>]
types_getCatalogCounters = []
types_getCompatibleMetadata = [<type 'dict'>, (<type 'str'>, <type 'unicode'>)]
types_getDatasetAnnotation = [<type 'dict'>]
types_getDatasetFiles = [<type 'dict'>]
types_getDatasetParameters = [<type 'dict'>]
types_getDatasets = [<type 'dict'>]
types_getDirectoryMetadata = [[<type 'list'>, <type 'dict'>, <type 'str'>, <type 'unicode'>]]
types_getDirectoryReplicas = [[<type 'list'>, <type 'dict'>, <type 'str'>, <type 'unicode'>], <type 'bool'>]
types_getDirectorySize = [[<type 'list'>, <type 'dict'>, <type 'str'>, <type 'unicode'>]]
types_getDirectoryUserMetadata = [(<type 'str'>, <type 'unicode'>)]
types_getFileAncestors = [[<type 'list'>, <type 'dict'>], [<type 'list'>, <type 'int'>, <type 'long'>]]
types_getFileDescendents = [[<type 'list'>, <type 'dict'>], [<type 'list'>, <type 'int'>, <type 'long'>]]
types_getFileMetadata = [[<type 'list'>, <type 'dict'>, <type 'str'>, <type 'unicode'>]]
types_getFileSize = [[<type 'list'>, <type 'dict'>, <type 'str'>, <type 'unicode'>]]
types_getFileUserMetadata = [(<type 'str'>, <type 'unicode'>)]
types_getGroups = []
types_getLFNForGUID = [[<type 'list'>, <type 'dict'>, <type 'str'>, <type 'unicode'>]]
types_getMetadataFields = []
types_getMetadataSet = [(<type 'str'>, <type 'unicode'>), <type 'bool'>]
types_getPathPermissions = [[<type 'list'>, <type 'dict'>, <type 'str'>, <type 'unicode'>]]
types_getReplicaStatus = [[<type 'list'>, <type 'dict'>, <type 'str'>, <type 'unicode'>]]
types_getReplicas = [[<type 'list'>, <type 'dict'>, <type 'str'>, <type 'unicode'>], <type 'bool'>]
types_getReplicasByMetadata = [<type 'dict'>, (<type 'str'>, <type 'unicode'>), <type 'bool'>]
types_getUsers = []
types_hasAccess = [[<type 'basestring'>, <type 'dict'>], [<type 'basestring'>, <type 'list'>, <type 'dict'>]]
types_isDirectory = [[<type 'list'>, <type 'dict'>, <type 'str'>, <type 'unicode'>]]
types_isFile = [[<type 'list'>, <type 'dict'>, <type 'str'>, <type 'unicode'>]]
types_isOK = []
types_listDirectory = [[<type 'list'>, <type 'dict'>, <type 'str'>, <type 'unicode'>], <type 'bool'>]
types_ping = []
types_rebuildDirectoryUsage = []
types_releaseDataset = [<type 'dict'>]
types_removeDataset = [<type 'dict'>]
types_removeDirectory = [[<type 'list'>, <type 'dict'>, <type 'str'>, <type 'unicode'>]]
types_removeFile = [[<type 'list'>, <type 'dict'>, <type 'str'>, <type 'unicode'>]]
types_removeMetadata = [<type 'dict'>]
types_removeReplica = [[<type 'list'>, <type 'dict'>, <type 'str'>, <type 'unicode'>]]
types_repairCatalog = []
types_setFileStatus = [<type 'dict'>]
types_setMetadata = [(<type 'str'>, <type 'unicode'>), <type 'dict'>]
types_setMetadataBulk = [<type 'dict'>]
types_setReplicaHost = [[<type 'list'>, <type 'dict'>, <type 'str'>, <type 'unicode'>]]
types_setReplicaStatus = [[<type 'list'>, <type 'dict'>, <type 'str'>, <type 'unicode'>]]
types_updateDataset = [<type 'dict'>]
DIRAC.DataManagementSystem.Service.FileCatalogHandler.initializeFileCatalogHandler(serviceInfo)

handler initialisation