XROOTStorage

This is the XROOTD StorageClass

class DIRAC.Resources.Storage.XROOTStorage.XROOTStorage(**kwargs)

Bases: DIRAC.Resources.Storage.StorageBase.StorageBase

class XROOTStorage

Xroot interface to StorageElement using pyxrootd

DYNAMIC_OPTIONS = {'SpaceToken': 'svcClass'}
PROTOCOL_PARAMETERS = ['Protocol', 'Host', 'Path', 'Port', 'SpaceToken', 'WSUrl']
__init__(**kwargs)

c’tor

Parameters:
  • self – self reference
  • storageName (str) – SE name
  • protocol (str) – protocol to use
  • rootdir (str) – base path for vo files
  • host (str) – SE host
  • port (int) – port to use to communicate with :host:
  • spaceToken (str) – space token
  • wspath (str) – location of SRM on :host:
changeDirectory(directory)

Change the directory to the supplied directory

constructURLFromLFN(lfn, withWSUrl=False)

Overwrite to add the double slash

createDirectory(path)
Make a/several new directory on the physical storage
This method creates all the intermediate directory
Parameters:
  • self – self reference
  • path (str) – path (or list of path) on storage (pfn : root://…)
Returns:

Successful dict {path : True} Failed dict {path : error message }

exists(path)

Check if the given path exists. The ‘path’ variable can be a string or a list of strings.

Parameters:
  • self – self reference
  • path – path (or list of path) on storage (it’s a pfn root://blablabla)
Returns Failed dictionary:
 

{pfn : errorMsg} Successful dictionary: {pfn : bool}

getCurrentDirectory()

Get the current directory

getCurrentURL(fileName)

Overwrite to add the double slash

getDirectory(path, localPath=False)

Get locally a directory from the physical storage together with all its files and subdirectories.

Param:path: path (or list of path) on storage (pfn : root://…)
Param:localPath: local path where to store what is downloaded
Returns:successful and failed dictionaries. The keys are the pathes, the values are dictionary {‘Files’: amount of files downloaded, ‘Size’: amount of data downloaded}
getDirectoryMetadata(path)

Get metadata associated to the directory(ies)

Parameters:
  • self – self reference
  • path – path (or list of path) on storage (pfn : root://…)
Returns:

Successful dict {path : metadata} Failed dict {path : error message }

getDirectorySize(path)

Get the size of the directory on the storage

Warning

the size is not recursive, and does not go into subfolders

Parameters:
  • self – self reference
  • path – path (or list of path) on storage (pfn : root://…)
Returns:

list of successfull and failed dictionnary, both indexed by the path

  • In the failed, the value is the error message

  • In the successful the values are dictionnaries :

    • Files : amount of files in the directory
    • Size : summed up size of files
    • subDirs : amount of sub directories

getEndpoint()
This will get endpoint of the storage. It basically is the same as getURLBase()
but without the basePath
Returns:‘proto://hostname<:port>’
getFile(path, localPath=False)

make a local copy of a storage :path:

Parameters:
  • self – self reference
  • path (str) – path (pfn root://) on storage
  • localPath (mixed) – if not specified, self.cwd
Returns:

Successful dict {path : size} Failed dict {path : error message }

getFileMetadata(path)

Get metadata associated to the file(s)

Parameters:
  • self – self reference
  • path – path (or list of path) on storage (pfn : root://…)
Returns:

Successful dict {path : metadata} Failed dict {path : error message }

getFileSize(path)

Get the physical size of the given file

Parameters:
  • self – self reference
  • path – path (or list of path) on storage (pfn : root://…)
Returns:

Successful dict {path : size} Failed dict {path : error message }

getName()

The name with which the storage was instantiated

getOccupancy(**kwargs)

Get the StorageElement occupancy info in MB.

This generic implementation download a json file supposed to contain the necessary info.

Parameters:occupancyLFN – (mandatory named argument) LFN of the json file.
Returns:S_OK/S_ERROR dictionary. The S_OK value should contain a dictionary with Total and Free space in MB
getParameters()

Get the parameters with which the storage was instantiated

getTransportURL(path, protocols=False)

obtain the tURLs for the supplied path and protocols

Parameters:
  • self – self reference
  • path (str) – path on storage (pfn : root://…)
  • protocols (mixed) – protocols to use (must be or include ‘root’)
Returns:

Successful dict {path : path} Failed dict {path : error message }

getURLBase(withWSUrl=False)

Overwrite to add the double slash

isDirectory(path)

Check if the given path exists and it is a directory

Parameters:
  • self – self reference
  • path – path (or list of path) on storage (pfn : root://…)
Returns:

Successful dict {path : boolean} Failed dict {path : error message }

isFile(path)

Check if the given path exists and it is a file

Parameters:
  • self – self reference
  • path – path (or list of path) on storage
Returns:

Successful dict {path : boolean} Failed dict {path : error message }

isNativeURL(url)

Check if URL :url: is valid for :self.protocol:

Parameters:
  • self – self reference
  • url (str) – URL
isOK()
isURL(path)

Guess if the path looks like a URL

Parameters:
  • self – self reference
  • path (string) – input file LFN or URL
Returns boolean:
 

True if URL, False otherwise

listDirectory(path)

List the supplied path

Warning

It is not recursive!

Parameters:path – single or list of path (root://..)
Returns:successful and failed dictionaries. The keys are the pathes, the values are dictionary ‘SubDirs’ and ‘Files’. Each are dictionaries with path as key and metadata as values (for Files only, SubDirs has just True as value)
pinFile(*parms, **kws)

Pin the file on the destination storage element

prestageFile(*parms, **kws)

Issue prestage request for file

prestageFileStatus(*parms, **kws)

Obtain the status of the prestage request

putDirectory(path)

puts a or several local directory to the physical storage together with all its files and subdirectories

Parameters:
  • self – self reference
  • path (str) – dictionnary {pfn (root://…) : local dir}
Returns:

successful and failed dictionaries. The keys are the pathes, the values are dictionary {‘Files’: amount of files uploaded, ‘Size’: amount of data uploaded}

putFile(path, sourceSize=0)

Put a copy of the local file to the current directory on the physical storage

Parameters:
  • path – dictionnary {pfn (root://…) : localFile}
  • sourceSize – size in B (NOT USED)
Returns:

Successful dict {path : size} Failed dict {path : error message } S_ERROR(errMsg) in case of arguments problems

releaseFile(*parms, **kws)

Release the file on the destination storage element

removeDirectory(path, recursive=False)

Remove a directory on the physical storage together with all its files and subdirectories.

Parameters:
  • path – single or list of path (root://..)
  • recursive – if True, we recursively delete the subdir
Returns:

successful and failed dictionaries. The keys are the pathes, the values are dictionary {‘Files’: amount of files deleted, ‘Size’: amount of data deleted}

removeFile(path)

Remove physically the file specified by its path

A non existing file will be considered as successfully removed.

Parameters:path – path (or list of path) on storage (pfn : root://…)
Returns:Successful dict {path : True} Failed dict {path : error message }
resetCurrentDirectory()

Reset the working directory to the base dir

setParameters(parameterDict)

Set standard parameters, method can be overriden in subclasses to process specific parameters

setStorageElement(se)
updateURL(url, withWSUrl=False)

Update the URL according to the current SE parameters