LcgFileCatalogClient

Class for the LCG File Catalog Client

class DIRAC.Resources.Catalog.LcgFileCatalogClient.LcgFileCatalogClient(**options)

Bases: DIRAC.Resources.Catalog.FileCatalogClientBase.FileCatalogClientBase

ADMIN_METHODS = ['getUserDirectory', 'createUserDirectory', 'createUserMapping', 'removeUserDirectory']
NO_LFN_METHODS = ['getUserDirectory', 'createUserDirectory', 'createUserMapping', 'removeUserDirectory']
READ_METHODS = ['hasAccess', 'exists', 'getPathPermissions', 'isLink', 'readLink', 'isFile', 'getFileMetadata', 'getReplicas', 'getReplicaStatus', 'getFileSize', 'isDirectory', 'getDirectoryReplicas', 'listDirectory', 'getDirectoryMetadata', 'getDirectorySize', 'getDirectoryContents', 'resolveDataset', 'getLFNForPFN', 'getUserDirectory']
WRITE_METHODS = ['createLink', 'removeLink', 'addFile', 'addReplica', 'removeReplica', 'removeFile', 'setReplicaStatus', 'setReplicaHost', 'createDirectory', 'removeDirectory', 'removeDataset', 'removeFileFromDataset', 'createDataset', 'changePathOwner', 'changePathMode']
__init__(**options)

Constructor function.

addDataset(self, datasets, **kwargs)

Add a new dynamic dataset defined by its meta query

Automatically created for the service function export_addDataset()

Parameters:datasets – datasets
addDatasetAnnotation(self, datasetDict, **kwargs)

Add annotation to an already created dataset

Automatically created for the service function export_addDatasetAnnotation()

Parameters:datasetDict – datasetDict
addFile(self, lfns, **kwargs)

Register supplied files

Automatically created for the service function export_addFile()

Parameters:lfns – lfns
addFileAncestors(self, lfns, **kwargs)

Add file ancestor information for the given list of LFNs

Automatically created for the service function export_addFileAncestors()

Parameters:lfns – lfns
addGroup(self, groupName, **kwargs)

Add a new group to the File Catalog

Automatically created for the service function export_addGroup()

Parameters:groupName – groupName
addMetadataField(self, fieldName, fieldType, metaType, **kwargs)

Add a new metadata field of the given type

Automatically created for the service function export_addMetadataField()

Parameters:
  • fieldName – fieldName
  • fieldType – fieldType
  • metaType – metaType
addMetadataSet(self, setName, setDict, **kwargs)

Add a new metadata set

Automatically created for the service function export_addMetadataSet()

Parameters:
  • setName – setName
  • setDict – setDict
addReplica(**kwargs)

This adds a replica to the catalogue.

addUser(self, userName, **kwargs)

Add a new user to the File Catalog

Automatically created for the service function export_addUser()

Parameters:userName – userName
changePathGroup(self, lfns, recursive, **kwargs)

Get replica info for the given list of LFNs

Automatically created for the service function export_changePathGroup()

Parameters:
  • lfns – lfns
  • recursive – recursive
changePathMode(**kwargs)

Change the ownership of the directory to the user associated to the supplied DN

changePathOwner(**kwargs)

Change the ownership of the directory to the user associated to the supplied DN

checkDataset(self, datasets, **kwargs)

Check the given dynamic dataset for changes since its definition

Automatically created for the service function export_checkDataset()

Parameters:datasets – datasets
createDataset(**kwargs)
createDirectory(self, lfns, **kwargs)

Create the supplied directories

Automatically created for the service function export_createDirectory()

Parameters:lfns – lfns
createUserDirectory(usernames)

Creates the user directory

createUserMapping(userDNs)

Create a user with the supplied DN and return the userID

deleteGroup(self, groupName, **kwargs)

Delete group from the File Catalog

Automatically created for the service function export_deleteGroup()

Parameters:groupName – groupName
deleteMetadataField(self, fieldName, **kwargs)

Delete the metadata field

Automatically created for the service function export_deleteMetadataField()

Parameters:fieldName – fieldName
deleteUser(self, userName, **kwargs)

Delete user from the File Catalog

Automatically created for the service function export_deleteUser()

Parameters:userName – userName
executeRPC(*parms, **kws)

This method extracts some parameters from kwargs that are used as parameter of the constructor or RPCClient. Unfortunately, only a few of all the available parameters of BaseClient are exposed.

Parameters:
  • rpc – if an RPC client is passed, use that one
  • timeout – we can change the timeout on a per call bases. Default is self.timeout
  • url – We can specify which url to use
exists(**kwargs)

Check if the path exists

findDirectoriesByMetadata(self, metaDict, path, **kwargs)

Find all the directories satisfying the given metadata set

Automatically created for the service function export_findDirectoriesByMetadata()

Parameters:
  • metaDict – metaDict
  • path – path
findFilesByMetadata(self, metaDict, path, **kwargs)

Find all the files satisfying the given metadata set

Automatically created for the service function export_findFilesByMetadata()

Parameters:
  • metaDict – metaDict
  • path – path
findFilesByMetadataDetailed(self, metaDict, path, **kwargs)

Find all the files satisfying the given metadata set

Automatically created for the service function export_findFilesByMetadataDetailed()

Parameters:
  • metaDict – metaDict
  • path – path
findFilesByMetadataWeb(self, metaDict, path, startItem, maxItems, **kwargs)

Find files satisfying the given metadata set

Automatically created for the service function export_findFilesByMetadataWeb()

Parameters:
  • metaDict – metaDict
  • path – path
  • startItem – startItem
  • maxItems – maxItems
freezeDataset(self, datasets, **kwargs)

Freeze the contents of the dataset making it effectively static

Automatically created for the service function export_freezeDataset()

Parameters:datasets – datasets
getCatalogCounters(self, **kwargs)

Get the number of registered directories, files and replicas in various tables

Automatically created for the service function export_getCatalogCounters()

getCompatibleMetadata(self, metaDict, path, **kwargs)

Get metadata values compatible with the given metadata subset

Automatically created for the service function export_getCompatibleMetadata()

Parameters:
  • metaDict – metaDict
  • path – path
getDatasetAnnotation(self, datasets, **kwargs)

Get annotation of the given datasets

Automatically created for the service function export_getDatasetAnnotation()

Parameters:datasets – datasets
getDatasetFiles(self, datasets, **kwargs)

Get lfns in the given dataset

Automatically created for the service function export_getDatasetFiles()

Parameters:datasets – datasets
getDatasetParameters(self, datasets, **kwargs)

Get parameters of the given dynamic dataset as they are stored in the database

Automatically created for the service function export_getDatasetParameters()

Parameters:datasets – datasets
getDatasets(self, datasets, **kwargs)

Get parameters of the given dynamic dataset as they are stored in the database

Automatically created for the service function export_getDatasets()

Parameters:datasets – datasets
getDirectoryMetadata(self, lfns, **kwargs)

Get the size of the supplied directory

Automatically created for the service function export_getDirectoryMetadata()

Parameters:lfns – lfns
getDirectoryReplicas(**kwargs)

This method gets all of the pfns in the directory

getDirectorySize(self, lfns, longOut, fromFiles, **kwargs)

Get the size of the supplied directory

Automatically created for the service function export_getDirectorySize()

Parameters:
  • lfns – lfns
  • longOut – longOut
  • fromFiles – fromFiles
getDirectoryUserMetadata(self, path, **kwargs)

Get all the metadata valid for the given directory path

Automatically created for the service function export_getDirectoryUserMetadata()

Parameters:path – path
getFileAncestors(self, lfns, depths, **kwargs)

Get the status for the supplied replicas

Automatically created for the service function export_getFileAncestors()

Parameters:
  • lfns – lfns
  • depths – depths
getFileDescendents(self, lfns, depths, **kwargs)

Get the status for the supplied replicas

Automatically created for the service function export_getFileDescendents()

Parameters:
  • lfns – lfns
  • depths – depths
getFileMetadata(**kwargs)

Returns the file metadata associated to a supplied LFN

getFileSize(**kwargs)

Get the size of a supplied file

getFileUserMetadata(self, path, **kwargs)

Get all the metadata valid for the given file

Automatically created for the service function export_getFileUserMetadata()

Parameters:path – path
getGroups(self, **kwargs)

Get all the groups defined in the File Catalog

Automatically created for the service function export_getGroups()

classmethod getInterfaceMethods()

Get the methods implemented by the File Catalog client

Return tuple:( read_methods_list, write_methods_list, nolfn_methods_list )
getLFNForGUID(self, guids, **kwargs)

Get the matching lfns for given guids

Automatically created for the service function export_getLFNForGUID()

Parameters:guids – guids
getLFNForPFN(**kwargs)
getMetadataFields(self, **kwargs)

Get all the metadata fields

Automatically created for the service function export_getMetadataFields()

getMetadataSet(self, setName, expandFlag, **kwargs)

Add a new metadata set

Automatically created for the service function export_getMetadataSet()

Parameters:
  • setName – setName
  • expandFlag – expandFlag
getName()
getPathPermissions(**kwargs)

Determine the VOMs based ACL information for a supplied path

getReplicaStatus(self, lfns, **kwargs)

Get the status for the supplied replicas

Automatically created for the service function export_getReplicaStatus()

Parameters:lfns – lfns
getReplicas(**kwargs)

Returns replicas for an LFN or list of LFNs

getReplicasByMetadata(self, metaDict, path, allStatus, **kwargs)

Find all the files satisfying the given metadata set

Automatically created for the service function export_getReplicasByMetadata()

Parameters:
  • metaDict – metaDict
  • path – path
  • allStatus – allStatus
getServer()

Getter for the server url. Useful ?

getUserDirectory(usernames)

Takes a list of users and determines whether their directories already exist

getUsers(self, **kwargs)

Get all the users defined in the File Catalog

Automatically created for the service function export_getUsers()

hasAccess(**kwargs)

Default method: returns True for all paths and all actions

Parameters:
  • paths (lfn) – has to be formatted this way : { lfn : { se1 : pfn1, se2 : pfn2, …}, …}
  • _opType (str) – dummy string
Returns:

{ successful : { lfn : [ ses ] } : failed : { lfn : { se : msg } } }

classmethod hasCatalogMethod(methodName)

Check of a method with the given name is implemented

Parameters:methodName (str) – the name of the method to check
Returns:boolean Flag True if the method is implemented
isDirectory(**kwargs)

Determine whether the path is a directory

isFile(self, lfns, **kwargs)

Check whether the supplied lfns are files

Automatically created for the service function export_isFile()

Parameters:lfns – lfns
isOK()

Check that the service is OK

listDirectory(**kwargs)

Returns the result of __getDirectoryContents for multiple supplied paths

rebuildDirectoryUsage(**kwargs)

Rebuild DirectoryUsage table from scratch

Automatically created for the service function export_rebuildDirectoryUsage()

releaseDataset(self, datasets, **kwargs)

Release the contents of the frozen dataset allowing changes in its contents

Automatically created for the service function export_releaseDataset()

Parameters:datasets – datasets
removeDataset(self, datasets, **kwargs)

Check the given dynamic dataset for changes since its definition

Automatically created for the service function export_removeDataset()

Parameters:datasets – datasets
removeDirectory(self, lfns, **kwargs)

Remove the supplied directories

Automatically created for the service function export_removeDirectory()

Parameters:lfns – lfns
removeFile(**kwargs)

Remove the supplied path

removeFileFromDataset(**kwargs)
removeMetadata(self, pathMetadataDict, **kwargs)

Remove the specified metadata for the given path

Automatically created for the service function export_removeMetadata()

Parameters:pathMetadataDict – pathMetadataDict
removeReplica(self, lfns, **kwargs)

Remove the supplied replicas

Automatically created for the service function export_removeReplica()

Parameters:lfns – lfns
removeUserDirectory(username)

Remove the user directory and remove the user mapping

repairCatalog(self, **kwargs)

Repair the catalog inconsistencies

Automatically created for the service function export_repairCatalog()

resolveDataset(**kwargs)
setAuthorizationId(dn)

Set authorization id for the proxy-less LFC communication

setFileStatus(self, lfns, **kwargs)

Remove the supplied lfns

Automatically created for the service function export_setFileStatus()

Parameters:lfns – lfns
setMetadata(self, path, metadatadict, **kwargs)

Set metadata parameter for the given path

Automatically created for the service function export_setMetadata()

Parameters:
  • path – path
  • metadatadict – metadatadict
setMetadataBulk(self, pathMetadataDict, **kwargs)

Set metadata parameter for the given path

Automatically created for the service function export_setMetadataBulk()

Parameters:pathMetadataDict – pathMetadataDict
setReplicaHost(**kwargs)

This modifies the replica metadata for the SE.

setReplicaProblematic(**kwargs)

Set replicas to problematic.

Parameters:
  • lfns (lfn) – has to be formated this way : { lfn : { se1 : pfn1, se2 : pfn2, …}, …}
  • revert – If True, remove the problematic flag
Returns:

{ Successful : { lfn : [ ses ] }, Failed : { lfn : { se : msg } } }

setReplicaStatus(self, lfns, **kwargs)

Set the status for the supplied replicas

Automatically created for the service function export_setReplicaStatus()

Parameters:lfns – lfns
setServer(url)

Set the server URL used by default

Parameters:url – url of the service
setTimeout(**kwargs)

Specify the timeout of the call. Forwarded to RPCClient

param int timeout:
 timeout for the RPC calls

Deprecated: Please use self._getRPC(timeout) or self.timeout=<value>

updateDataset(self, datasets, **kwargs)

Update the given dynamic dataset for changes since its definition

Automatically created for the service function export_updateDataset()

Parameters:datasets – datasets
DIRAC.Resources.Catalog.LcgFileCatalogClient.addReplica(guid, pfn, se, master)
DIRAC.Resources.Catalog.LcgFileCatalogClient.addUserDN(userID, dn)
DIRAC.Resources.Catalog.LcgFileCatalogClient.closeDirectory(oDirectory)
DIRAC.Resources.Catalog.LcgFileCatalogClient.existsGuid(guid)

Check if the guid exists

DIRAC.Resources.Catalog.LcgFileCatalogClient.getClientCertInfo()
DIRAC.Resources.Catalog.LcgFileCatalogClient.getDNFromUID(userID)
DIRAC.Resources.Catalog.LcgFileCatalogClient.getDNUserID(dn)
DIRAC.Resources.Catalog.LcgFileCatalogClient.getRoleFromGID(groupID, path=None)
DIRAC.Resources.Catalog.LcgFileCatalogClient.modReplica(pfn, newse)
DIRAC.Resources.Catalog.LcgFileCatalogClient.removeReplica(pfn)
DIRAC.Resources.Catalog.LcgFileCatalogClient.returnCode(error, value='', errMsg='')
DIRAC.Resources.Catalog.LcgFileCatalogClient.setLfnReplicas(lfn, replicas, successful, failed)
DIRAC.Resources.Catalog.LcgFileCatalogClient.setReplicaStatus(pfn, status)