FTS3Operation

class DIRAC.DataManagementSystem.Client.FTS3Operation.FTS3Operation(ftsFiles=None, username=None, userGroup=None, rmsReqID=-1, rmsOpID=0, sourceSEs=None, activity=None, priority=None)

Bases: JSerializable

Abstract class to represent an operation to be executed by FTS. It is a container for FTSFiles, as well as for FTSJobs.

There can be a mapping between one FTS3Operation and one RMS Operation.

The FTS3Operation takes care of generating the appropriate FTSJobs, and to perform a callback when the work with FTS is over. The actual generation and callback depends on the subclass.

This class should not be instantiated directly, but rather one of its subclass

ALL_STATES = ['Active', 'Processed', 'Finished', 'Canceled', 'Failed']
FINAL_STATES = ['Finished', 'Canceled', 'Failed']
INIT_STATE = 'Active'
__init__(ftsFiles=None, username=None, userGroup=None, rmsReqID=-1, rmsOpID=0, sourceSEs=None, activity=None, priority=None)
Parameters:

  • ftsFiles – list of FTS3Files object that belongs to the operation

  • username – username whose proxy should be used

  • userGroup – group that should be used with username

  • rmsReqID – ID of the Request in the RMS system

  • rmsOpID – ID of the Operation in the RMS system

  • sourceSEs – list of SE to be used as source (if applicable)

  • activity – FTS activity to use

  • priority – FTS priority to use

activity
assignment
callback()

Trigger the callback once all the FTS interactions are done and update the status of the Operation to ‘Finished’ if successful

creationTime
error
classmethod fromRMSObjects(rmsReq, rmsOp, username)

Construct an FTS3Operation object from the RMS Request and Operation corresponding. The attributes taken are the OwnerGroup, Request and Operation IDS, sourceSE, and activity and priority if they are defined in the Argument field of the operation

Parameters:

  • rmsReq – RMS Request object

  • rmsOp – RMS Operation object

  • username – username to which associate the FTS3Operation (normally comes from the Req OwnerDN)

Returns:

FTS3Operation object

ftsFiles
ftsJobs
init_on_load()

This method initializes some attributes. It is called by sqlalchemy (which does not call __init__)

isTotallyProcessed()

Returns True if and only if there is nothing else to be done by FTS for this operation. All files are successful or definitely failed

lastUpdate
operationID
prepareNewJobs(maxFilesPerJob=100, maxAttemptsPerFile=10)

Prepare the new jobs that have to be submitted

Parameters:

  • maxFilesPerJob – maximum number of files assigned to a job

  • maxAttemptsPerFile – maximum number of retry after an fts failure

Returns:

list of jobs

priority
rmsOpID
rmsReqID
sourceSEs
status
type
userGroup
username
property vo

return vo of the usergroup

Type:

returns

class DIRAC.DataManagementSystem.Client.FTS3Operation.FTS3StagingOperation(ftsFiles=None, username=None, userGroup=None, rmsReqID=-1, rmsOpID=0, sourceSEs=None, activity=None, priority=None)

Bases: FTS3Operation

Class to be used for a Staging operation

ALL_STATES = ['Active', 'Processed', 'Finished', 'Canceled', 'Failed']
FINAL_STATES = ['Finished', 'Canceled', 'Failed']
INIT_STATE = 'Active'
__init__(ftsFiles=None, username=None, userGroup=None, rmsReqID=-1, rmsOpID=0, sourceSEs=None, activity=None, priority=None)
Parameters:

  • ftsFiles – list of FTS3Files object that belongs to the operation

  • username – username whose proxy should be used

  • userGroup – group that should be used with username

  • rmsReqID – ID of the Request in the RMS system

  • rmsOpID – ID of the Operation in the RMS system

  • sourceSEs – list of SE to be used as source (if applicable)

  • activity – FTS activity to use

  • priority – FTS priority to use

activity
assignment
callback()

Trigger the callback once all the FTS interactions are done and update the status of the Operation to ‘Finished’ if successful

creationTime
error
classmethod fromRMSObjects(rmsReq, rmsOp, username)

Construct an FTS3Operation object from the RMS Request and Operation corresponding. The attributes taken are the OwnerGroup, Request and Operation IDS, sourceSE, and activity and priority if they are defined in the Argument field of the operation

Parameters:

  • rmsReq – RMS Request object

  • rmsOp – RMS Operation object

  • username – username to which associate the FTS3Operation (normally comes from the Req OwnerDN)

Returns:

FTS3Operation object

ftsFiles
ftsJobs
init_on_load()

This method initializes some attributes. It is called by sqlalchemy (which does not call __init__)

isTotallyProcessed()

Returns True if and only if there is nothing else to be done by FTS for this operation. All files are successful or definitely failed

lastUpdate
operationID
prepareNewJobs(maxFilesPerJob=100, maxAttemptsPerFile=10)

Prepare the new jobs that have to be submitted

Parameters:

  • maxFilesPerJob – maximum number of files assigned to a job

  • maxAttemptsPerFile – maximum number of retry after an fts failure

Returns:

list of jobs

priority
rmsOpID
rmsReqID
sourceSEs
status
type
userGroup
username
property vo

return vo of the usergroup

Type:

returns

class DIRAC.DataManagementSystem.Client.FTS3Operation.FTS3TransferOperation(ftsFiles=None, username=None, userGroup=None, rmsReqID=-1, rmsOpID=0, sourceSEs=None, activity=None, priority=None)

Bases: FTS3Operation

Class to be used for a Replication operation

ALL_STATES = ['Active', 'Processed', 'Finished', 'Canceled', 'Failed']
FINAL_STATES = ['Finished', 'Canceled', 'Failed']
INIT_STATE = 'Active'
__init__(ftsFiles=None, username=None, userGroup=None, rmsReqID=-1, rmsOpID=0, sourceSEs=None, activity=None, priority=None)
Parameters:

  • ftsFiles – list of FTS3Files object that belongs to the operation

  • username – username whose proxy should be used

  • userGroup – group that should be used with username

  • rmsReqID – ID of the Request in the RMS system

  • rmsOpID – ID of the Operation in the RMS system

  • sourceSEs – list of SE to be used as source (if applicable)

  • activity – FTS activity to use

  • priority – FTS priority to use

activity
assignment
callback()

Trigger the callback once all the FTS interactions are done and update the status of the Operation to ‘Finished’ if successful

creationTime
error
classmethod fromRMSObjects(rmsReq, rmsOp, username)

Construct an FTS3Operation object from the RMS Request and Operation corresponding. The attributes taken are the OwnerGroup, Request and Operation IDS, sourceSE, and activity and priority if they are defined in the Argument field of the operation

Parameters:

  • rmsReq – RMS Request object

  • rmsOp – RMS Operation object

  • username – username to which associate the FTS3Operation (normally comes from the Req OwnerDN)

Returns:

FTS3Operation object

ftsFiles
ftsJobs
init_on_load()

This method initializes some attributes. It is called by sqlalchemy (which does not call __init__)

isTotallyProcessed()

Returns True if and only if there is nothing else to be done by FTS for this operation. All files are successful or definitely failed

lastUpdate
operationID
prepareNewJobs(maxFilesPerJob=100, maxAttemptsPerFile=10)

Prepare the new jobs that have to be submitted

Parameters:

  • maxFilesPerJob – maximum number of files assigned to a job

  • maxAttemptsPerFile – maximum number of retry after an fts failure

Returns:

list of jobs

priority
rmsOpID
rmsReqID
sourceSEs
status
type
userGroup
username
property vo

return vo of the usergroup

Type:

returns