StorageElement

This is the StorageElement class.

class DIRAC.Resources.Storage.StorageElement.StorageElementCache

Bases: object

class DIRAC.Resources.Storage.StorageElement.StorageElementItem(name, plugins=None, vo=None, hideExceptions=False)

Bases: object

class StorageElement

common interface to the grid storage element

self.name is the resolved name of the StorageElement i.e CERN-tape self.options is dictionary containing the general options defined in the CS e.g. self.options[‘Backend] = ‘Castor2’ self.storages is a list of the stub objects created by StorageFactory for the protocols found in the CS. self.localPlugins is a list of the local protocols that were created by StorageFactory self.remotePlugins is a list of the remote protocols that were created by StorageFactory self.protocolOptions is a list of dictionaries containing the options found in the CS. (should be removed)

dynamic method : retransferOnlineFile( lfn ) exists( lfn ) isFile( lfn ) getFile( lfn, localPath = False ) putFile( lfnLocal, sourceSize = 0 ) : {lfn:local} replicateFile( lfn, sourceSize = 0 ) getFileMetadata( lfn ) getFileSize( lfn ) removeFile( lfn ) prestageFile( lfn, lifetime = 86400 ) prestageFileStatus( lfn ) pinFile( lfn, lifetime = 60 * 60 * 24 ) releaseFile( lfn ) isDirectory( lfn ) getDirectoryMetadata( lfn ) getDirectorySize( lfn ) listDirectory( lfn ) removeDirectory( lfn, recursive = False ) createDirectory( lfn ) putDirectory( lfn ) getDirectory( lfn, localPath = False )

addAccountingOperation(lfns, startDate, elapsedTime, storageParameters, callRes)

Generates a DataOperation accounting if needs to be, and adds it to the DataStore client cache

:param lfns : list of lfns on which we attempted the operation :param startDate : datetime, start of the operation :param elapsedTime : time (seconds) the operation took :param storageParameters : the parameters of the plugins used to perform the operation :param callRes : the return of the method call, S_OK or S_ERROR

The operation is generated with the OperationType “se.methodName” The TransferSize and TransferTotal for directory methods actually take into account the files inside the directory, and not the amount of directory given as parameter

checksumType()

get specific /Resources/StorageElements/<SEName>/ChecksumType option if defined, otherwise global /Resources/StorageElements/ChecksumType

dump()

Dump to the logger a summary of the StorageElement items.

generateTransferURLsBetweenSEs(lfns, sourceSE, protocols=None)

This negociate the URLs to be used for third party copy. This is mostly useful for FTS. If protocols is given, it restricts the list of plugins to use

Parameters:
  • lfns – list/dict of lfns to generate the URLs
  • sourceSE – storageElement instance of the sourceSE
  • protocols – ordered protocol restriction list

:return:dictionnary Successful/Failed with pair (src, dest) urls

getChecksumType()

Checksum type getter for backward compatibility

getLFNFromURL(urls)

Get the LFN from the PFNS . :param lfn : input lfn or lfns (list/dict)

getLocalPlugins()

Get the list of all the local access protocols defined for this Storage Element

getPlugins()

Get the list of all the plugins defined for this Storage Element

getRemotePlugins()

Get the list of all the remote access protocols defined for this Storage Element

getStatus()

Return Status of the SE only if the SE is valid It returns an S_OK/S_ERROR structure

getStorageElementName()

SE name getter for backward compatibility

getStorageParameters(plugin=None, protocol=None)

Get plugin specific options :param plugin : plugin we are interested in :param protocol: protocol we are interested in

Either plugin or protocol can be defined, not both, but at least one of them

getURL(lfn, protocol=False, replicaDict=None)

execute ‘getTransportURL’ operation. :param str lfn: string, list or dictionary of lfns :param protocol: if no protocol is specified, we will request self.turlProtocols :param replicaDict: optional results from the File Catalog replica query

isValid(operation=None)

check CS/RSS statuses for :operation:

Parameters:operation (str) – operation name
negociateProtocolWithOtherSE(sourceSE, protocols=None)

Negotiate what protocol could be used for a third party transfer between the sourceSE and ourselves. If protocols is given, the chosen protocol has to be among those

:param sourceSE : storageElement instance of the sourceSE :param protocols: ordered protocol restriction list

Returns:a list protocols that fits the needs, or None
status()

Return Status of the SE, a dictionary with:

  • Read: True (is allowed), False (it is not allowed)

  • Write: True (is allowed), False (it is not allowed)

  • Remove: True (is allowed), False (it is not allowed)

  • Check: True (is allowed), False (it is not allowed).

    Note

    Check is always allowed IF Read is allowed (regardless of what set in the Check option of the configuration)

  • DiskSE: True if TXDY with Y > 0 (defaults to True)

  • TapeSE: True if TXDY with X > 0 (defaults to False)

  • TotalCapacityTB: float (-1 if not defined)

  • DiskCacheTB: float (-1 if not defined)

It returns directly the dictionary

storageElementName()

SE name getter