FileCatalogClient

The FileCatalogClient is a class representing the client of the DIRAC File Catalog

class DIRAC.Resources.Catalog.FileCatalogClient.FileCatalogClient(url=None, **kwargs)

Bases: FileCatalogClientBase

Client code to the DIRAC File Catalogue

ADMIN_METHODS = ['addUser', 'deleteUser', 'addGroup', 'deleteGroup', 'getUsers', 'getGroups', 'getCatalogCounters', 'repairCatalog', 'rebuildDirectoryUsage']

NO_LFN_METHODS = ['findFilesByMetadata', 'addMetadataField', 'deleteMetadataField', 'getMetadataFields', 'setMetadata', 'setMetadataBulk', 'removeMetadata', 'getDirectoryUserMetadata', 'findDirectoriesByMetadata', 'getReplicasByMetadata', 'findFilesByMetadataDetailed', 'findFilesByMetadataWeb', 'getCompatibleMetadata', 'addMetadataSet', 'getMetadataSet', 'getFileUserMetadata', 'getLFNForGUID', 'addUser', 'deleteUser', 'addGroup', 'deleteGroup', 'repairCatalog', 'rebuildDirectoryUsage']

READ_METHODS = ['hasAccess', 'exists', 'getPathPermissions', 'isFile', 'getFileMetadata', 'getFileDetails', 'getReplicas', 'getReplicaStatus', 'getFileSize', 'isDirectory', 'getDirectoryReplicas', 'listDirectory', 'getDirectoryMetadata', 'getDirectorySize', 'getDirectoryContents', 'getLFNForPFN', 'getLFNForGUID', 'findFilesByMetadata', 'getMetadataFields', 'findDirectoriesByMetadata', 'getReplicasByMetadata', 'findFilesByMetadataDetailed', 'findFilesByMetadataWeb', 'getCompatibleMetadata', 'getMetadataSet', 'getDatasets', 'getFileDescendents', 'getFileAncestors', 'getDirectoryUserMetadata', 'getFileUserMetadata', 'checkDataset', 'getDatasetParameters', 'getDatasetFiles', 'getDatasetAnnotation', 'getSEDump', 'getDirectoryDump']

WRITE_METHODS = ['createLink', 'removeLink', 'addFile', 'addFileAncestors', 'setFileStatus', 'addReplica', 'removeReplica', 'removeFile', 'setReplicaStatus', 'setReplicaHost', 'setReplicaProblematic', 'createDirectory', 'setDirectoryStatus', 'removeDirectory', 'changePathMode', 'changePathOwner', 'changePathGroup', 'addMetadataField', 'deleteMetadataField', 'setMetadata', 'setMetadataBulk', 'removeMetadata', 'addMetadataSet', 'addDataset', 'addDatasetAnnotation', 'removeDataset', 'updateDataset', 'freezeDataset', 'releaseDataset', 'addUser', 'deleteUser', 'addGroup', 'deleteGroup', 'repairCatalog', 'rebuildDirectoryUsage']

__init__(url=None, **kwargs)

Constructor function.

addDataset(datasets, timeout=120)

Add a new dynamic dataset defined by its meta query

addDatasetAnnotation(datasetDict, timeout=120)

Add annotation to an already created dataset

addFile(lfns, timeout=120)

Register supplied files

addFileAncestors(lfns, timeout=120)

Add file ancestor information for the given dict of LFNs.

Parameters:

lfns (dict) – {lfn1: {‘Ancestor’: [ancestorLFNs]}, lfn2: {‘Ancestors’: …}}

addGroup(groupName, timeout=120)

Add a new group to the File Catalog

addMetadataField(fieldName, fieldType, metaType='-d', timeout=120)

Add a new metadata field of the given type

addMetadataSet(setName, setDict, timeout=120)

Add a new metadata set

addReplica(lfns, timeout=120)

Register supplied replicas

addUser(userName, timeout=120)

Add a new user to the File Catalog

property call

To be removed once we’re sure self.call has been removed

Type:

Deprecated

changePathGroup(lfns, recursive=False, timeout=120)

Get replica info for the given list of LFNs

changePathMode(lfns, recursive=False, timeout=120)

Get replica info for the given list of LFNs

changePathOwner(lfns, recursive=False, timeout=120)

Get replica info for the given list of LFNs

checkDataset(datasets, timeout=120)

Check the given dynamic dataset for changes since its definition

createDirectory(lfns, timeout=120)

Create the supplied directories

deleteGroup(groupName, timeout=120)

Delete group from the File Catalog

deleteMetadataField(fieldName, timeout=120)

Delete the metadata field

deleteUser(userName, timeout=120)

Delete user from the File Catalog

executeRPC(*parms, **kws)

This method extracts some parameters from kwargs that are used as parameter of the constructor or RPCClient. Unfortunately, only a few of all the available parameters of BaseClient are exposed.

Parameters:

• rpc – if an RPC client is passed, use that one

• timeout – we can change the timeout on a per call bases. Default is self.timeout

• url – We can specify which url to use

exists(lfns, timeout=120)

Check whether the supplied paths exists

findDirectoriesByMetadata(metaDict, path='/', timeout=120)

Find all the directories satisfying the given metadata set

findFilesByMetadata(metaDict, path='/', timeout=120)

Find files given the meta data query and the path

findFilesByMetadataDetailed(metaDict, path='/', timeout=120)

Find all the files satisfying the given metadata set

findFilesByMetadataWeb(metaDict, path, startItem, maxItems, timeout=120)

Find files satisfying the given metadata set

freezeDataset(datasets, timeout=120)

Freeze the contents of the dataset making it effectively static

getCatalogCounters(timeout=120)

Get the number of registered directories, files and replicas in various tables

getClientKWArgs()

Returns a copy of the connection arguments

getCompatibleMetadata(metaDict, path='/', timeout=120)

Get metadata values compatible with the given metadata subset

getDatasetAnnotation(datasets, timeout=120)

Get annotation of the given datasets

getDatasetFiles(datasets, timeout=120)

Get lfns in the given dataset two lines !

getDatasetParameters(datasets, timeout=120)

Get parameters of the given dynamic dataset as they are stored in the database

getDatasets(datasets, timeout=120)

Get parameters of the given dynamic dataset as they are stored in the database

getDirectoryDump(lfns, timeout=120)

Get the content of a directory recursively

getDirectoryMetadata(lfns, timeout=120)

Get standard directory metadata

getDirectoryReplicas(lfns, allStatus=False, timeout=120)

Find all the given directories’ replicas

getDirectorySize(lfns, longOut=False, fromFiles=False, timeout=120, recursiveSum=True)

Get the size of the supplied directory

getDirectoryUserMetadata(path, timeout=120)

Get all the metadata valid for the given directory path

getFileAncestors(lfns, depths, timeout=120)

Get the status for the supplied replicas

getFileDescendents(lfns, depths, timeout=120)

Get the status for the supplied replicas

getFileDetails(lfns, timeout=120)

Get the (user) metadata associated to supplied lfns

getFileMetadata(lfns, timeout=120)

Get the metadata associated to supplied lfns

getFileSize(lfns, timeout=120)

Get the size associated to supplied lfns

getFileUserMetadata(path, timeout=120)

Get the meta data attached to a file, but also to the its corresponding directory

getGroups(timeout=120)

Get all the groups defined in the File Catalog

classmethod getInterfaceMethods()

Get the methods implemented by the File Catalog client

Return tuple:

( read_methods_list, write_methods_list, nolfn_methods_list )

getLFNForGUID(guids, timeout=120)

Get the matching lfns for given guids

getMetadataFields(timeout=120)

Get all the metadata fields

getMetadataSet(setName, expandFlag, timeout=120)

Add a new metadata set

getPathPermissions(lfns, timeout=120)

Determine the ACL information for a supplied path

getReplicaStatus(lfns, timeout=120)

Get the status for the supplied replicas

getReplicas(lfns, allStatus=False, timeout=120)

Get the replicas of the given files

getReplicasByMetadata(metaDict, path='/', allStatus=False, timeout=120)

Find all the files satisfying the given metadata set

getSEDump(seNames, outputFilename)

Dump the content of SEs in the given file. The file contains a list of [SEName, lfn,checksum,size] dumped as csv, separated by ‘|’

Parameters:

• seName – list of StorageElement names

• outputFilename – path to the file where to dump it

Returns:

result from the TransferClient

getServer()

Getter for the server url. Useful ?

getUsers(timeout=120)

Get all the users defined in the File Catalog

hasAccess(paths, opType, timeout=120)

Determine if the given op can be performed on the paths The OpType is all the operations exported

classmethod hasCatalogMethod(methodName)

Check of a method with the given name is implemented

Parameters:

methodName (str) – the name of the method to check

Returns:

boolean Flag True if the method is implemented

httpsClient

alias of TornadoClient

isDirectory(lfns, timeout=120)

Determine whether supplied path is a directory

isFile(lfns, timeout=120)

Check whether the supplied lfns are files

isOK(timeout=120)

Check that the service is OK

listDirectory(lfn, verbose=False, timeout=120)

List the given directory’s contents

rebuildDirectoryUsage(timeout=120)

Rebuild DirectoryUsage table from scratch

releaseDataset(datasets, timeout=120)

Release the contents of the frozen dataset allowing changes in its contents

removeDataset(datasets, timeout=120)

Check the given dynamic dataset for changes since its definition

removeDirectory(lfn, recursive=False, timeout=120)

Remove the directory from the File Catalog. The recursive keyword is for the ineterface.

removeFile(lfns, timeout=120)

Remove the supplied lfns

removeMetadata(pathMetadataDict, timeout=120)

Remove the specified metadata for the given path

removeReplica(lfns, timeout=120)

Remove the supplied replicas

repairCatalog(timeout=120)

Repair the catalog inconsistencies

setFileStatus(lfns, timeout=120)

Remove the supplied lfns

setMetadata(path, metadatadict, timeout=120)

Set metadata parameter for the given path

setMetadataBulk(pathMetadataDict, timeout=120)

Set metadata parameter for the given path

setReplicaHost(lfns, timeout=120)

Change the registered SE for the supplied replicas

setReplicaProblematic(lfns, revert=False)

Set replicas to problematic.

Parameters:

• lfns (lfn) – has to be formated this way : { lfn : { se1 : pfn1, se2 : pfn2, …}, …}

• revert – If True, remove the problematic flag

Returns:

{ successful : { lfn : [ ses ] } : failed : { lfn : { se : msg } } }

setReplicaStatus(lfns, timeout=120)

Set the status for the supplied replicas

setServer(url)

Set the server URL used by default

Parameters:

url – url of the service

updateDataset(datasets, timeout=120)

Update the given dynamic dataset for changes since its definition