FailoverTransfer

Failover Transfer

The failover transfer client exposes the following methods: - transferAndRegisterFile() - transferAndRegisterFileFailover()

Initially these methods were developed inside workflow modules but have evolved to a generic ‘transfer file with failover’ client.

The transferAndRegisterFile() method will correctly set registration requests in case of failure.

The transferAndRegisterFileFailover() method will attempt to upload a file to a list of alternative SEs and set appropriate replication to the original target SE as well as the removal request for the temporary replica.

class DIRAC.DataManagementSystem.Client.FailoverTransfer.FailoverTransfer(requestObject=None, log=None, defaultChecksumType='ADLER32')

Bases: object

class FailoverTransfer
__init__(requestObject=None, log=None, defaultChecksumType='ADLER32')

Constructor function, can specify request object to instantiate FailoverTransfer or a new request object is created.

commitRequest()

Send request to the Request Management Service

getRequest()

get the accumulated request object

transferAndRegisterFile(fileName, localPath, lfn, destinationSEList, fileMetaDict, fileCatalog=None, masterCatalogOnly=False, retryUpload=False)

Performs the transfer and register operation with failover.

Parameters:

  • filename – of absolute no use except for printing logs.

  • localPath – path to the file locally

  • lfn – LFN

  • destinationSEList – list of possible destination for the file. Loop over it until one succeeds or we reach the end of it.

  • fileMetaDict – file metadata for registration

  • fileCatalog – list of catalogs to use (see DIRAC.DataManagementSystem.Client.DataManager)

  • masterCatalogOnly – use only master catalog (see DIRAC.DataManagementSystem.Client.DataManager)

  • retryUpload – if set to True, and there is only one output SE in destinationSEList, retry several times.

transferAndRegisterFileFailover(fileName, localPath, lfn, targetSE, failoverSEList, fileMetaDict, fileCatalog=None, masterCatalogOnly=False)

Performs the transfer and register operation to failover storage and sets the necessary replication and removal requests to recover.