StorageBase

Base Storage Class provides the base interface for all storage plug-ins

exists()

These are the methods for manipulating files:

isFile() getFile() putFile() removeFile() getFileMetadata() getFileSize() prestageFile() getTransportURL()

These are the methods for manipulating directories:

isDirectory() getDirectory() putDirectory() createDirectory() removeDirectory() listDirectory() getDirectoryMetadata() getDirectorySize()

These are the methods for manipulating the client:

changeDirectory() getCurrentDirectory() getName() getParameters() getCurrentURL()

These are the methods for getting information about the Storage:

getOccupancy()

class DIRAC.Resources.Storage.StorageBase.StorageBase(name, parameterDict)

Bases: object

class StorageBase

DYNAMIC_OPTIONS = {}

PROTOCOL_PARAMETERS = ['Protocol', 'Host', 'Path', 'Port', 'SpaceToken', 'WSUrl']

__init__(name, parameterDict)

changeDirectory(directory)

Change the directory to the supplied directory

constructURLFromLFN(lfn, withWSUrl=False)

Construct URL from the given LFN according to the VO convention for the primary protocol of the storage plagin

Parameters:

• lfn (str) – file LFN

• withWSUrl (boolean) – flag to include the web service part into the resulting URL

Return result:

result[‘Value’] - resulting URL

createDirectory(*parms, **kws)

Make a new directory on the physical storage

exists(*parms, **kws)

Check if the given path exists

getCurrentDirectory()

Get the current directory

getCurrentURL(fileName)

Obtain the current file URL from the current working directory and the filename

Parameters:

• self – self reference

• fileName (str) – path on storage

getDirectory(*parms, **kws)

Get locally a directory from the physical storage together with all its files and subdirectories.

getDirectoryMetadata(*parms, **kws)

Get the metadata for the directory

getDirectorySize(*parms, **kws)

Get the size of the directory on the storage

getEndpoint()

This will get endpoint of the storage. It basically is the same as getURLBase()

but without the basePath

Returns:

‘proto://hostname<:port>’

getFile(*parms, **kws)

Get a local copy of the file specified by its path

getFileMetadata(*parms, **kws)

Get metadata associated to the file

getFileSize(*parms, **kws)

Get the physical size of the given file

getName()

The name with which the storage was instantiated

getOccupancy(**kwargs)

Get the StorageElement occupancy info in MB.

This generic implementation download a json file supposed to contain the necessary info.

Parameters:

occupancyLFN – (mandatory named argument) LFN of the json file.

Returns:

S_OK/S_ERROR dictionary. The S_OK value should contain a dictionary with Total and Free space in MB

getParameters()

Get the parameters with which the storage was instantiated

getTransportURL(pathDict, protocols)

Get a transport URL for a given URL. For a simple storage plugin it is just returning input URL if the plugin protocol is one of the requested protocols

Parameters:

• pathDict (dict) – URL obtained from File Catalog or constructed according to convention

• protocols (python:list) – a list of acceptable transport protocols in priority order

getURLBase(withWSUrl=False)

This will get the URL base. This is then appended with the LFN in DIRAC convention.

Parameters:

• self – self reference

• withWSUrl (bool) – flag to include Web Service part of the url

Returns:

URL

getWLCGTokenPath(lfn: str, wlcgTokenBasePath: str) → str

Returns the path expected to be in a WLCG token It basically consists of basepath - tokenBasePath  + LFN The tokenBasePath is a configuration on the storage side.

isDirectory(*parms, **kws)

Check if the given path exists and it is a directory

isFile(*parms, **kws)

Check if the given path exists and it is a file

isNativeURL(url)

Check if URL :url: is valid for :self.protocol:

Parameters:

• self – self reference

• url (str) – URL

isURL(path)

Guess if the path looks like a URL

Parameters:

• self – self reference

• path (string) – input file LFN or URL

Returns boolean:

True if URL, False otherwise

listDirectory(*parms, **kws)

List the supplied path

prestageFile(*parms, **kws)

Issue prestage request for file

prestageFileStatus(*parms, **kws)

Obtain the status of the prestage request

putDirectory(*parms, **kws)

Put a local directory to the physical storage together with all its files and subdirectories.

putFile(*parms, **kws)

Put a copy of the local file to the current directory on the physical storage

releaseFile(*parms, **kws)

Release the file on the destination storage element

removeDirectory(*parms, **kws)

Remove a directory on the physical storage together with all its files and subdirectories.

removeFile(*parms, **kws)

Remove physically the file specified by its path

resetCurrentDirectory()

Reset the working directory to the base dir

setParameters(parameterDict)

Set standard parameters, method can be overriden in subclasses to process specific parameters

setStorageElement(se)

updateURL(url, withWSUrl=False)

Update the URL according to the current SE parameters