FileCatalogClient

The FileCatalogClient is a class representing the client of the DIRAC File Catalog

class DIRAC.Resources.Catalog.FileCatalogClient.FileCatalogClient(url=None, **kwargs)

Bases: DIRAC.Resources.Catalog.FileCatalogClientBase.FileCatalogClientBase

Client code to the DIRAC File Catalogue

ADMIN_METHODS = ['addUser', 'deleteUser', 'addGroup', 'deleteGroup', 'getUsers', 'getGroups', 'getCatalogCounters', 'repairCatalog', 'rebuildDirectoryUsage']
NO_LFN_METHODS = ['findFilesByMetadata', 'addMetadataField', 'deleteMetadataField', 'getMetadataFields', 'setMetadata', 'setMetadataBulk', 'removeMetadata', 'getDirectoryUserMetadata', 'findDirectoriesByMetadata', 'getReplicasByMetadata', 'findFilesByMetadataDetailed', 'findFilesByMetadataWeb', 'getCompatibleMetadata', 'addMetadataSet', 'getMetadataSet', 'getFileUserMetadata', 'getLFNForGUID', 'addUser', 'deleteUser', 'addGroup', 'deleteGroup', 'repairCatalog', 'rebuildDirectoryUsage']
READ_METHODS = ['hasAccess', 'exists', 'getPathPermissions', 'isFile', 'getFileMetadata', 'getReplicas', 'getReplicaStatus', 'getFileSize', 'isDirectory', 'getDirectoryReplicas', 'listDirectory', 'getDirectoryMetadata', 'getDirectorySize', 'getDirectoryContents', 'getLFNForPFN', 'getLFNForGUID', 'findFilesByMetadata', 'getMetadataFields', 'findDirectoriesByMetadata', 'getReplicasByMetadata', 'findFilesByMetadataDetailed', 'findFilesByMetadataWeb', 'getCompatibleMetadata', 'getMetadataSet', 'getDatasets', 'getFileDescendents', 'getFileAncestors', 'getDirectoryUserMetadata', 'getFileUserMetadata', 'checkDataset', 'getDatasetParameters', 'getDatasetFiles', 'getDatasetAnnotation']
WRITE_METHODS = ['createLink', 'removeLink', 'addFile', 'setFileStatus', 'addReplica', 'removeReplica', 'removeFile', 'setReplicaStatus', 'setReplicaHost', 'setReplicaProblematic', 'createDirectory', 'setDirectoryStatus', 'removeDirectory', 'changePathMode', 'changePathOwner', 'changePathGroup', 'addMetadataField', 'deleteMetadataField', 'setMetadata', 'setMetadataBulk', 'removeMetadata', 'addMetadataSet', 'addDataset', 'addDatasetAnnotation', 'removeDataset', 'updateDataset', 'freezeDataset', 'releaseDataset', 'addUser', 'deleteUser', 'addGroup', 'deleteGroup', 'repairCatalog', 'rebuildDirectoryUsage']
__init__(url=None, **kwargs)

Constructor function.

addDataset(**kwargs)

Add a new dynamic dataset defined by its meta query

addDatasetAnnotation(**kwargs)

Add annotation to an already created dataset

addFile(**kwargs)

Register supplied files

addFileAncestors(**kwargs)

Add file ancestor information for the given list of LFNs

addGroup(groupName, timeout=120)

Add a new group to the File Catalog

addMetadataField(fieldName, fieldType, metaType='-d', timeout=120)

Add a new metadata field of the given type

addMetadataSet(setName, setDict, timeout=120)

Add a new metadata set

addReplica(**kwargs)

Register supplied replicas

addUser(userName, timeout=120)

Add a new user to the File Catalog

changePathGroup(**kwargs)

Get replica info for the given list of LFNs

changePathMode(**kwargs)

Get replica info for the given list of LFNs

changePathOwner(**kwargs)

Get replica info for the given list of LFNs

checkDataset(**kwargs)

Check the given dynamic dataset for changes since its definition

createDirectory(**kwargs)

Create the supplied directories

deleteGroup(groupName, timeout=120)

Delete group from the File Catalog

deleteMetadataField(fieldName, timeout=120)

Delete the metadata field

deleteUser(userName, timeout=120)

Delete user from the File Catalog

executeRPC(*parms, **kws)

This method extracts some parameters from kwargs that are used as parameter of the constructor or RPCClient. Unfortunately, only a few of all the available parameters of BaseClient are exposed.

Parameters:
  • rpc – if an RPC client is passed, use that one
  • timeout – we can change the timeout on a per call bases. Default 120 s
  • url – We can specify which url to use
exists(**kwargs)

Check whether the supplied paths exists

findDirectoriesByMetadata(metaDict, path='/', timeout=120)

Find all the directories satisfying the given metadata set

findFilesByMetadata(metaDict, path='/', timeout=120)

Find files given the meta data query and the path

findFilesByMetadataDetailed(metaDict, path='/', timeout=120)

Find all the files satisfying the given metadata set

findFilesByMetadataWeb(metaDict, path, startItem, maxItems, timeout=120)

Find files satisfying the given metadata set

freezeDataset(**kwargs)

Freeze the contents of the dataset making it effectively static

getCatalogCounters(timeout=120)

Get the number of registered directories, files and replicas in various tables

getCompatibleMetadata(metaDict, path='/', timeout=120)

Get metadata values compatible with the given metadata subset

getDatasetAnnotation(**kwargs)

Get annotation of the given datasets

getDatasetFiles(**kwargs)

Get lfns in the given dataset two lines !

getDatasetParameters(**kwargs)

Get parameters of the given dynamic dataset as they are stored in the database

getDatasets(**kwargs)

Get parameters of the given dynamic dataset as they are stored in the database

getDirectoryMetadata(**kwargs)

Get standard directory metadata

getDirectoryReplicas(**kwargs)

Find all the given directories’ replicas

getDirectorySize(**kwargs)

Get the size of the supplied directory

getDirectoryUserMetadata(path, timeout=120)

Get all the metadata valid for the given directory path

getFileAncestors(**kwargs)

Get the status for the supplied replicas

getFileDescendents(**kwargs)

Get the status for the supplied replicas

getFileMetadata(**kwargs)

Get the metadata associated to supplied lfns

getFileSize(**kwargs)

Get the size associated to supplied lfns

getFileUserMetadata(path, timeout=120)

Get the meta data attached to a file, but also to the its corresponding directory

getGroups(timeout=120)

Get all the groups defined in the File Catalog

classmethod getInterfaceMethods()

Get the methods implemented by the File Catalog client

Return tuple:( read_methods_list, write_methods_list, nolfn_methods_list )
getLFNForGUID(guids, timeout=120)

Get the matching lfns for given guids

getMetadataFields(timeout=120)

Get all the metadata fields

getMetadataSet(setName, expandFlag, timeout=120)

Add a new metadata set

getPathPermissions(**kwargs)

Determine the ACL information for a supplied path

getReplicaStatus(**kwargs)

Get the status for the supplied replicas

getReplicas(**kwargs)

Get the replicas of the given files

getReplicasByMetadata(metaDict, path='/', allStatus=False, timeout=120)

Find all the files satisfying the given metadata set

getSEDump(seName, outputFilename)

Dump the content of an SE in the given file. The file contains a list of [lfn,checksum,size] dumped as csv, separated by ‘|’

Parameters:
  • seName – name of the StorageElement
  • outputFilename – path to the file where to dump it
Returns:

result from the TransferClient

getServer()

Getter for the server url. Useful ?

getUsers(timeout=120)

Get all the users defined in the File Catalog

hasAccess(**kwargs)

Determine if the given op can be performed on the paths The OpType is all the operations exported

classmethod hasCatalogMethod(methodName)

Check of a method with the given name is implemented :param str methodName: the name of the method to check :return: boolean Flag True if the method is implemented

isDirectory(**kwargs)

Determine whether supplied path is a directory

isFile(**kwargs)

Check whether the supplied lfns are files

isOK(timeout=120)

Check that the service is OK

listDirectory(**kwargs)

List the given directory’s contents

rebuildDirectoryUsage(timeout=120)

Rebuild DirectoryUsage table from scratch

releaseDataset(**kwargs)

Release the contents of the frozen dataset allowing changes in its contents

removeDataset(**kwargs)

Check the given dynamic dataset for changes since its definition

removeDirectory(**kwargs)

Remove the directory from the File Catalog. The recursive keyword is for the ineterface.

removeFile(**kwargs)

Remove the supplied lfns

removeMetadata(pathMetadataDict, timeout=120)

Remove the specified metadata for the given path

removeReplica(**kwargs)

Remove the supplied replicas

repairCatalog(timeout=120)

Repair the catalog inconsistencies

setFileStatus(**kwargs)

Remove the supplied lfns

setMetadata(path, metadatadict, timeout=120)

Set metadata parameter for the given path

setMetadataBulk(pathMetadataDict, timeout=120)

Set metadata parameter for the given path

setReplicaHost(**kwargs)

Change the registered SE for the supplied replicas

setReplicaProblematic(**kwargs)

Set replicas to problematic.

Parameters:
  • lfns (lfn) – has to be formated this way : { lfn : { se1 : pfn1, se2 : pfn2, …}, …}
  • revert – If True, remove the problematic flag
Returns:

{ successful : { lfn : [ ses ] } : failed : { lfn : { se : msg } } }

setReplicaStatus(**kwargs)

Set the status for the supplied replicas

setServer(url)

Set the server URL used by default

Parameters:url – url of the service
setTimeout(timeout)

Specify the timeout of the call. Forwarded to RPCClient

Parameters:timeout – guess…
updateDataset(**kwargs)

Update the given dynamic dataset for changes since its definition