RucioFileCatalogClient

The RucioFileCatalogClient is a class that allows to interface Dirac with Rucio

class DIRAC.Resources.Catalog.RucioFileCatalogClient.RucioFileCatalogClient(**options)

Bases: FileCatalogClientBase

ADMIN_METHODS = ['getUserDirectory', 'createUserDirectory', 'createUserMapping', 'removeUserDirectory']

NO_LFN_METHODS = ['getUserDirectory', 'createUserDirectory', 'createUserMapping', 'removeUserDirectory']

READ_METHODS = ['hasAccess', 'exists', 'getPathPermissions', 'isLink', 'readLink', 'isFile', 'getFileMetadata', 'getReplicas', 'getReplicaStatus', 'getFileSize', 'isDirectory', 'getDirectoryReplicas', 'listDirectory', 'getDirectoryMetadata', 'getDirectorySize', 'getDirectoryContents', 'resolveDataset', 'getLFNForPFN', 'getUserDirectory']

WRITE_METHODS = ['createLink', 'removeLink', 'addFile', 'addReplica', 'removeReplica', 'removeFile', 'setReplicaStatus', 'setReplicaHost', 'createDirectory', 'removeDirectory', 'removeDataset', 'removeFileFromDataset', 'createDataset', 'changePathOwner', 'changePathMode']

__init__(**options)

Constructor. Takes options defined in Resources and Operations for this client.

Parameters:

options – options dict

addDataset(self, datasets, **kwargs)

Add a new dynamic dataset defined by its meta query

Automatically created for the service function export_addDataset()

Parameters:

datasets – datasets

addDatasetAnnotation(self, datasetDict, **kwargs)

Add annotation to an already created dataset

Automatically created for the service function export_addDatasetAnnotation()

Parameters:

datasetDict – datasetDict

addFile(lfns)

Register supplied files

addFileAncestors(self, lfns, **kwargs)

Add file ancestor information for the given list of LFNs

Automatically created for the service function export_addFileAncestors()

Parameters:

lfns – lfns

addGroup(self, groupName, **kwargs)

Add a new group to the File Catalog

Automatically created for the service function export_addGroup()

Parameters:

groupName – groupName

addMetadataField(self, fieldName, fieldType, metaType, **kwargs)

Add a new metadata field of the given type

Automatically created for the service function export_addMetadataField()

Parameters:

• fieldName – fieldName

• fieldType – fieldType

• metaType – metaType

addMetadataSet(self, setName, setDict, **kwargs)

Add a new metadata set

Automatically created for the service function export_addMetadataSet()

Parameters:

• setName – setName

• setDict – setDict

addReplica(lfns)

This adds a replica to the catalogue.

addUser(self, userName, **kwargs)

Add a new user to the File Catalog

Automatically created for the service function export_addUser()

Parameters:

userName – userName

property call

To be removed once we’re sure self.call has been removed

Type:

Deprecated

changePathGroup(self, lfns, recursive, **kwargs)

Get replica info for the given list of LFNs

Automatically created for the service function export_changePathGroup()

Parameters:

• lfns – lfns

• recursive – recursive

changePathMode(self, lfns, recursive, **kwargs)

Get replica info for the given list of LFNs

Automatically created for the service function export_changePathMode()

Parameters:

• lfns – lfns

• recursive – recursive

changePathOwner(self, lfns, recursive, **kwargs)

Get replica info for the given list of LFNs

Automatically created for the service function export_changePathOwner()

Parameters:

• lfns – lfns

• recursive – recursive

checkDataset(self, datasets, **kwargs)

Check the given dynamic dataset for changes since its definition

Automatically created for the service function export_checkDataset()

Parameters:

datasets – datasets

property client

Check if the session to the server is still active and return an instance of RucioClient

createDirectory(self, lfns, **kwargs)

Create the supplied directories

Automatically created for the service function export_createDirectory()

Parameters:

lfns – lfns

deleteGroup(self, groupName, **kwargs)

Delete group from the File Catalog

Automatically created for the service function export_deleteGroup()

Parameters:

groupName – groupName

deleteMetadataField(self, fieldName, **kwargs)

Delete the metadata field

Automatically created for the service function export_deleteMetadataField()

Parameters:

fieldName – fieldName

deleteUser(self, userName, **kwargs)

Delete user from the File Catalog

Automatically created for the service function export_deleteUser()

Parameters:

userName – userName

executeRPC(*parms, **kws)

This method extracts some parameters from kwargs that are used as parameter of the constructor or RPCClient. Unfortunately, only a few of all the available parameters of BaseClient are exposed.

Parameters:

• rpc – if an RPC client is passed, use that one

• timeout – we can change the timeout on a per call bases. Default is self.timeout

• url – We can specify which url to use

exists(lfns)

Check if the path exists

findDirectoriesByMetadata(self, metaDict, path, **kwargs)

Find all the directories satisfying the given metadata set

Automatically created for the service function export_findDirectoriesByMetadata()

Parameters:

• metaDict – metaDict

• path – path

findFilesByMetadata(self, metaDict, path, **kwargs)

Find all the files satisfying the given metadata set

Automatically created for the service function export_findFilesByMetadata()

Parameters:

• metaDict – metaDict

• path – path

findFilesByMetadataDetailed(self, metaDict, path, **kwargs)

Find all the files satisfying the given metadata set

Automatically created for the service function export_findFilesByMetadataDetailed()

Parameters:

• metaDict – metaDict

• path – path

findFilesByMetadataWeb(self, metaDict, path, startItem, maxItems, **kwargs)

Find files satisfying the given metadata set

Automatically created for the service function export_findFilesByMetadataWeb()

Parameters:

• metaDict – metaDict

• path – path

• startItem – startItem

• maxItems – maxItems

freezeDataset(self, datasets, **kwargs)

Freeze the contents of the dataset making it effectively static

Automatically created for the service function export_freezeDataset()

Parameters:

datasets – datasets

getCatalogCounters(self, **kwargs)

Get the number of registered directories, files and replicas in various tables

Automatically created for the service function export_getCatalogCounters()

getClientKWArgs()

Returns a copy of the connection arguments

getCompatibleMetadata(queryDict, path, credDict)

Get distinct metadata values compatible with the given already defined metadata

getDatasetAnnotation(self, datasets, **kwargs)

Get annotation of the given datasets

Automatically created for the service function export_getDatasetAnnotation()

Parameters:

datasets – datasets

getDatasetFiles(self, datasets, **kwargs)

Get lfns in the given dataset

Automatically created for the service function export_getDatasetFiles()

Parameters:

datasets – datasets

getDatasetParameters(self, datasets, **kwargs)

Get parameters of the given dynamic dataset as they are stored in the database

Automatically created for the service function export_getDatasetParameters()

Parameters:

datasets – datasets

getDatasets(self, datasets, **kwargs)

Get parameters of the given dynamic dataset as they are stored in the database

Automatically created for the service function export_getDatasets()

Parameters:

datasets – datasets

getDirectoryDump(self, lfns, **kwargs)

Recursively list the contents of supplied directories

Automatically created for the service function export_getDirectoryDump()

Parameters:

lfns – lfns

getDirectoryMetadata(self, lfns, **kwargs)

Get the metadata of the supplied directory

Automatically created for the service function export_getDirectoryMetadata()

Parameters:

lfns – lfns

getDirectoryReplicas(self, lfns, allStatus, **kwargs)

Get replicas for files in the supplied directory

Automatically created for the service function export_getDirectoryReplicas()

Parameters:

• lfns – lfns

• allStatus – allStatus

getDirectorySize(lfns, longOutput=False, rawFiles=False)

Get the directory size

getDirectoryUserMetadata(self, path, **kwargs)

Get all the metadata valid for the given directory path

Automatically created for the service function export_getDirectoryUserMetadata()

Parameters:

path – path

getFileAncestors(self, lfns, depths, **kwargs)

Get the status for the supplied replicas

Automatically created for the service function export_getFileAncestors()

Parameters:

• lfns – lfns

• depths – depths

getFileDescendents(self, lfns, depths, **kwargs)

Get the status for the supplied replicas

Automatically created for the service function export_getFileDescendents()

Parameters:

• lfns – lfns

• depths – depths

getFileDetails(self, lfns, **kwargs)

Get all the metadata associated to supplied lfns, including user metadata

Automatically created for the service function export_getFileDetails()

Parameters:

lfns – lfns

getFileMetadata(lfns, ownership=False)

Returns the file metadata associated to a supplied LFN

getFileSize(lfns)

Get the size of a supplied file

getFileUserMetadata(self, path, **kwargs)

Get all the metadata valid for the given file

Automatically created for the service function export_getFileUserMetadata()

Parameters:

path – path

getGroups(self, **kwargs)

Get all the groups defined in the File Catalog

Automatically created for the service function export_getGroups()

classmethod getInterfaceMethods()

Get the methods implemented by the File Catalog client

Return tuple:

( read_methods_list, write_methods_list, nolfn_methods_list )

getLFNForGUID(self, guids, **kwargs)

Get the matching lfns for given guids

Automatically created for the service function export_getLFNForGUID()

Parameters:

guids – guids

getMetadataFields(self, **kwargs)

Get all the metadata fields

Automatically created for the service function export_getMetadataFields()

getMetadataSet(self, setName, expandFlag, **kwargs)

Add a new metadata set

Automatically created for the service function export_getMetadataSet()

Parameters:

• setName – setName

• expandFlag – expandFlag

getPathPermissions(lfns)

Default method: returns Read & Write permission for all the paths

Parameters:

paths (lfn) – has to be formatted this way : { lfn : { se1 : pfn1, se2 : pfn2, …}, …}

Returns:

{ successful : { lfn : [ ses ] } : failed : { lfn : { se : msg } } }

getReplicaStatus(self, lfns, **kwargs)

Get the status for the supplied replicas

Automatically created for the service function export_getReplicaStatus()

Parameters:

lfns – lfns

getReplicas(lfns, allStatus=False)

Returns replicas for an LFN or list of LFNs

getReplicasByMetadata(self, metaDict, path, allStatus, **kwargs)

Find all the files satisfying the given metadata set

Automatically created for the service function export_getReplicasByMetadata()

Parameters:

• metaDict – metaDict

• path – path

• allStatus – allStatus

getServer()

Getter for the server url. Useful ?

getUsers(self, **kwargs)

Get all the users defined in the File Catalog

Automatically created for the service function export_getUsers()

hasAccess(paths, _opType)

Default method: returns True for all paths and all actions

Parameters:

• paths (lfn) – has to be formatted this way : { lfn : { se1 : pfn1, se2 : pfn2, …}, …}

• _opType (str) – dummy string

Returns:

{ successful : { lfn : [ ses ] } : failed : { lfn : { se : msg } } }

classmethod hasCatalogMethod(methodName)

Check of a method with the given name is implemented

Parameters:

methodName (str) – the name of the method to check

Returns:

boolean Flag True if the method is implemented

httpsClient

alias of TornadoClient

isDirectory(lfns)

Determine whether the path is a directory

isFile(lfns)

Determine whether the path is a file

isOK(timeout=120)

Check that the service is OK

listDirectory(lfns, verbose=False)

Returns the result of __getDirectoryContents for multiple supplied paths.

Parameters:

• lfns (list) – a list of logical filenames

• verbose (bool) – verbose flag.

Returns:

S_OK with a Value of successful and failed directory tree

rebuildDirectoryUsage(self, **kwargs)

Rebuild DirectoryUsage table from scratch

Automatically created for the service function export_rebuildDirectoryUsage()

releaseDataset(self, datasets, **kwargs)

Release the contents of the frozen dataset allowing changes in its contents

Automatically created for the service function export_releaseDataset()

Parameters:

datasets – datasets

removeDataset(self, datasets, **kwargs)

Check the given dynamic dataset for changes since its definition

Automatically created for the service function export_removeDataset()

Parameters:

datasets – datasets

removeDirectory(lfns)

Remove the supplied directory

removeFile(lfns)

Remove the supplied path

removeMetadata(self, pathMetadataDict, **kwargs)

Remove the specified metadata for the given path

Automatically created for the service function export_removeMetadata()

Parameters:

pathMetadataDict – pathMetadataDict

removeReplica(lfns)

Remove the supplied replicas

repairCatalog(self, **kwargs)

Repair the catalog inconsistencies

Automatically created for the service function export_repairCatalog()

setFileStatus(self, lfns, **kwargs)

Remove the supplied lfns

Automatically created for the service function export_setFileStatus()

Parameters:

lfns – lfns

setMetadata(self, path, metadatadict, **kwargs)

Set metadata parameter for the given path

Automatically created for the service function export_setMetadata()

Parameters:

• path – path

• metadatadict – metadatadict

setMetadataBulk(self, pathMetadataDict, **kwargs)

Set metadata parameter for the given path

Automatically created for the service function export_setMetadataBulk()

Parameters:

pathMetadataDict – pathMetadataDict

setReplicaHost(self, lfns, **kwargs)

Change the registered SE for the supplied replicas

Automatically created for the service function export_setReplicaHost()

Parameters:

lfns – lfns

setReplicaStatus(self, lfns, **kwargs)

Set the status for the supplied replicas

Automatically created for the service function export_setReplicaStatus()

Parameters:

lfns – lfns

setServer(url)

Set the server URL used by default

Parameters:

url – url of the service

updateDataset(self, datasets, **kwargs)

Update the given dynamic dataset for changes since its definition

Automatically created for the service function export_updateDataset()

Parameters:

datasets – datasets

DIRAC.Resources.Catalog.RucioFileCatalogClient.get_scope(lfn, scopes=None, diracAlgorithm='dirac')

Helper function that extracts the scope from the LFN.

Parameters:

• lfn (str) – Logical file name

• scopes (list) – list of scopes

• diracAlgorithm (str) – only used by extract_scope if there is no config file with an algorithm listed. Otherwise use the algorithm listed in the config file.

Returns:

scope name