DirectoryClosure

DIRAC FileCatalog component representing a directory tree with a closure table

General warning: when we return the number of affected row, if the values did not change

then they are not taken into account, so we might return “Dir does not exist” while it does…. the timestamp update should prevent this to happen, however if you do it several times within 1 second, then there will be no changed, and affected = 0

class DIRAC.DataManagementSystem.DB.FileCatalogComponents.DirectoryManager.DirectoryClosure.DirectoryClosure(database=None)

Bases: DIRAC.DataManagementSystem.DB.FileCatalogComponents.DirectoryManager.DirectoryTreeBase.DirectoryTreeBase

Class managing Directory Tree with a closure table http://technobytz.com/closure_table_store_hierarchical_data.html http://fungus.teststation.com/~jon/treehandling/TreeHandling.htm http://www.slideshare.net/billkarwin/sql-antipatterns-strike-back http://dirtsimple.org/2010/11/simplest-way-to-do-tree-based-queries.html

__init__(database=None)
changeDirectoryGroup(paths, recursive=False)

Bulk setting of the directory group

Parameters

paths (dictionary) – dictionary < lfn : group >

changeDirectoryMode(paths, recursive=False)

Bulk setting of the directory mode

Parameters

paths (dictionary) – dictionary < lfn : mode >

changeDirectoryOwner(paths, recursive=False)

Bulk setting of the directory owner

Parameters

paths (dictionary) – dictionary < lfn : owner >

countSubdirectories(dirId, includeParent=True)

Count the number of subdirectories

Parameters

  • dirID – id of the directory

  • includeParent – count itself

Returns

S_OK(value)

createDirectory(dirs, credDict)

Checking for existence of directories

exists(lfns)
existsDir(path)

Check the existence of a directory at the specified path

Parameters

path – directory path

Returns

S_OK( { ‘Exists’ : False } ) if the directory does not exist S_OK( { ‘Exists’ : True, ‘DirID’ : directory id } ) if the directory exists

findDir(path, connection=False)

Find directory ID for the given path

Parameters

path – path of the directory

Returns

S_OK(id) and res[‘Level’] as the depth

findDirs(paths, connection=False)

Find DirIDs for the given path list

Parameters

paths – list of path

Returns

S_OK( { path : ID} )

getAllSubdirectoriesByID(dirIdList)

Get IDs of all the subdirectories of directories in a given list

Parameters

dirList – list of dir Ids

Returns

S_OK([ unordered dir ids ])

getChildren(path, connection=False)

Get child directory IDs for the given directory

getDirectoryCounters(connection=False)

Get the total number of directories

getDirectoryDump(lfns)

Get the dump of the directories in lfns

getDirectoryParameters(pathOrDirId)

Get parameters of the given directory

Parameters

pathOrDirID – the path or the id of the directory

Returns

S_OK(dict), where dict has the following keys: “DirID”, “UID”, “Owner”, “GID”, “OwnerGroup”, “Status”, “Mode”, “CreationDate”, “ModificationDate”

getDirectoryPath(dirID)

Get directory name by directory ID

Parameters

dirID – directory ID

Returns

S_OK(dir name), or S_ERROR if it does not exist

getDirectoryPaths(dirIDList)

Get directory names by directory ID list

Parameters

dirIDList – list of dirIds

Returns

S_OK( { dirID : dirName} )

getDirectoryPermissions(path, credDict)

Get permissions for the given user/group to manipulate the given directory

getDirectoryReplicas(lfns, allStatus=False)

Get replicas for files in the given directories

getDirectorySize(lfns, longOutput=False, rawFileTables=False, recursiveSum=True)

Get the total size of the requested directories. If longOutput flag is True, get also physical size per Storage Element

Parameters

  • longOutput (bool) – if True, also fetches the physical size per SE

  • rawFileTables (bool) – if True, uses the File table instead of the pre-computed values

  • recursiveSum (bool) – if True (default), takes into account subdirectories

getFileIDsInDirectoryWithLimits(dirID, credDict, startItem=1, maxItems=25)

Get file IDs for the given directory

getFileLFNsInDirectory(dirID, credDict)

Get file lfns for the given directory or directory list

getFileLFNsInDirectoryByDirectory(dirIDList, credDict)

Get file LFNs and IDs for the given directory or directory list

Parameters

  • dirIDList (list) – List of directory IDs

  • credDict (dict) – dictionary of user credentials

Returns

S_OK/S_ERROR with Value dictionary {“DirLFNDict”: dirLfnDict, “IDLFNDict”: idLfnDict} where dirLfnDict has the structure <directory_name>:<list of contained file names>, idLfnDict has structure <fileID>:<LFN>

getPathIDs(path)

Get IDs of all the directories in the parent hierarchy for a directory specified by its path, including itself

Parameters

path – path of the directory

Returns

S_OK( list of ids ), S_ERROR if not found

getPathIDsByID(dirID)

Get IDs of all the directories in the parent hierarchy for a directory specified by its ID, including itself

Parameters

dirID – id of the dictionary

Returns

S_OK( list of ids )

getPathPermissions(lfns, credDict)

Get permissions for the given user/group to manipulate the given lfns

getSubdirectories(path)

Get subdirectories of the given directory

Parameters

path – path of the directory

Returns

S_OK ( { dirID, depth } )

getSubdirectoriesByID(dirID, requestString=False, includeParent=False)

Get all the subdirectories of the given directory at a given level

Parameters

  • dirID – id of the directory

  • requestString – if true, returns an sql query to get the information

  • includeParent – if true, the parent (dirID) will be included

Returns

S_OK ( { dirID, depth } ) if requestString is False S_OK(request) if requestString is True

getTreeTable()

Get the string of the Directory Tree type

isDirectory(paths)

Checking for existence of directories

isEmpty(path)

Find out if the given directory is empty

Rem: the speed could be enhanced if we were joining the FC_Files and FC_Directory* in the query.

For the time being, it can stay like this

Parameters

path – path of the directory

Returns

S_OK(true) if there are no file nor directorie, S_OK(False) otherwise

listDirectory(lfns, verbose=False)

Get the directory listing

makeDir(path)
makeDirectories(path, credDict)

Make all the directories recursively in the path. The return value is the dictionary containing all the parameters of the newly created directory

makeDirectory(path, credDict, status=1)

Create a directory

Parameters

  • path – has to be an absolute path. The parent dir has to exist

  • credDict – credential dict of the owner of the directory

Param

status ????

Returns

S_OK (dirID) with a flag res[‘NewDirectory’] to True or False S_ERROR if there is a problem, or if there is no parent

removeDir(path)

Remove directory

Removing a non existing directory is successful. In that case, DirID is 0

Parameters

path – path of the dir

Returns

S_OK() and res[‘DirID’] the id of the directory removed

removeDirectory(dirs, force=False)

Remove an empty directory from the catalog

setDatabase(database)
setDirectoryStatus(path, status)

set the directory status