FTS3DB
Frontend to FTS3 MySQL DB. Written using sqlalchemy
- class DIRAC.DataManagementSystem.DB.FTS3DB.FTS3DB(pool_size=15, url=None, parentLogger=None)
Bases:
object- class RequestDB
db holding requests
- __init__(pool_size=15, url=None, parentLogger=None)
c’tor
- Parameters:
self – self reference
pool_size – size of the connection pool to the DB
- cancelNonExistingJob(operationID, ftsGUID)
Cancel an FTS3Job with the associated FTS3Files. This is to be used when the job is not found on the server when monitoring.
The status of the job and files will be ‘Canceled’. The error is specifying that the job is not found. The ftsGUID of the file is released as to be able to pick it up again
- Parameters:
operationID – guess
ftsGUID – guess
- Returns:
S_OK() if successful, S_ERROR otherwise
- createTables()
create tables
- deleteFinalOperations(limit=20, deleteDelay=180)
deletes operation in final state that are older than given time
- getActiveJobs(limit=20, lastMonitor=None, jobAssignmentTag='Assigned')
- Get the FTSJobs that are not in a final state, and are not assigned for monitoring
or has its operation being treated
- By assigning the job to the DB:
it cannot be monitored by another agent
the operation to which it belongs cannot be treated
- Parameters:
limit – max number of Jobs to retrieve
lastMonitor – jobs monitored earlier than the given date
jobAssignmentTag – if not None, block the Job for other queries, and use it as a prefix for the value in the operation table
- Returns:
list of FTS3Jobs
- getNonFinishedOperations(limit=20, operationAssignmentTag='Assigned')
Get all the non assigned FTS3Operations that are not yet finished, so either Active or Processed. An operation won’t be picked if it is already assigned, or one of its job is.
- Parameters:
limit – max number of operations to retrieve
operationAssignmentTag – if not None, block the operations for other queries, and use it as a prefix for the value in the operation table
- Returns:
list of Operations
- getOperation(operationID)
read request
This does not set the assignment flag
- Parameters:
operationID – ID of the FTS3Operation
- getOperationsFromRMSOpID(rmsOpID)
Returns the FTS3Operations matching a given RMS OperationID
This does not set the assignment flag
- Parameters:
rmsOpID – ID of the RMS Operation
- kickStuckJobs(limit=20, kickDelay=2)
- finds jobs that have not been updated for more than a given
time but are still assigned and resets the assignment
- kickStuckOperations(limit=20, kickDelay=2)
- finds operations that have not been updated for more than a given
time but are still assigned and resets the assignment
- persistOperation(operation)
- update or insert request into db
Also release the assignment tag
- Parameters:
operation – FTS3Operation instance
- updateFileStatus(fileStatusDict, ftsGUID=None)
- Update the file ftsStatus and error
The update is only done if the file is not in a final state (To avoid bringing back to life a file by consuming MQ a posteriori)
TODO: maybe it should query first the status and filter the rows I want to update !
- Parameters:
fileStatusDict – { fileID : { status , error, ftsGUID } }
ftsGUID – If specified, only update the rows where the ftsGUID matches this value. This avoids two jobs handling the same file one after another to step on each other foot. Note that for the moment it is an optional parameter, but it may turn mandatory soon.
- updateJobStatus(jobStatusDict)
- Update the job Status and error
The update is only done if the job is not in a final state The assignment flag is released
- Parameters:
jobStatusDict – { jobID : { status , error, completeness } }