FTSDB

mod:FTSDB

synopsis:FTS DB

FTS DB

class DIRAC.DataManagementSystem.DB.FTSDB.FTSDB(systemInstance='Default')

Bases: DIRAC.Core.Base.DB.DB

class FTSDB

database holding FTS jobs and their files

class ConnectionPool(host, user, passwd, port=3306, graceTime=600)

Bases: object

Management of connections per thread

__init__(host, user, passwd, port=3306, graceTime=600)

x.__init__(…) initializes x; see help(type(x)) for signature

clean(now=False)
get(dbName, retries=10)
transactionCommit(dbName)
transactionRollback(dbName)
transactionStart(dbName)
__init__(systemInstance='Default')

c’tor

Parameters:
  • self – self reference
  • systemInstance (str) –

    ???

  • maxQueueSize (int) – size of queries queue
buildCondition(condDict=None, older=None, newer=None, timeStamp=None, orderAttribute=None, limit=False, greater=None, smaller=None, offset=None)

Build SQL condition statement from provided condDict and other extra check on a specified time stamp. The conditions dictionary specifies for each attribute one or a List of possible values greater and smaller are dictionaries in which the keys are the names of the fields, that are requested to be >= or < than the corresponding value. For compatibility with current usage it uses Exceptions to exit in case of invalid arguments

cleanUpFTSFiles(requestID, fileIDs)

delete FTSFiles for given :requestID: and list of :fileIDs:

Parameters:
  • requestID (int) – ReqDB.Request.RequestID
  • fileIDs (list) – [ ReqDB.File.FileID, … ]
countEntries(table, condDict, older=None, newer=None, timeStamp=None, connection=False, greater=None, smaller=None)

Count the number of entries wit the given conditions

createTables(toCreate=None, force=False)

create tables

createViews(force=False)

create views

deleteEntries(tableName, condDict=None, limit=False, conn=None, older=None, newer=None, timeStamp=None, orderAttribute=None, greater=None, smaller=None)

Delete rows from “tableName” with N records can match the condition if limit is not False, the given limit is set String type values will be appropriately escaped, they can be single values or lists of values.

deleteFTSFiles(operationID, opFileIDList=None)

delete FTSFiles for reschedule

Parameters:
  • operationID (int) – ReqDB.Operation.OperationID
  • opFileIDList (list) – [ ReqDB.File.FileID, … ]
deleteFTSJob(ftsJobID)

delete FTSJob given ftsJobID

dictCursor(conn=None)

get dict cursor for connection :conn:

Returns:S_OK( { “cursor”: cursors.DictCursor, “connection” : connection } ) or S_ERROR
executeStoredProcedure(packageName, parameters, outputIds)
executeStoredProcedureWithCursor(packageName, parameters)
getAllFTSFilesForRequest(requestID)

get FTSFiles with status in :statusList: for request given its :requestID:

getCSOption(optionName, defaultValue=None)
getCounters(table, attrList, condDict, older=None, newer=None, timeStamp=None, connection=False, greater=None, smaller=None)

Count the number of records on each distinct combination of AttrList, selected with condition defined by condDict and time stamps

getDBSummary()

get DB summary

getDistinctAttributeValues(table, attribute, condDict=None, older=None, newer=None, timeStamp=None, connection=False, greater=None, smaller=None)

Get distinct values of a table attribute under specified conditions

getFTSFile(ftsFileID)

read FTSFile from db given FTSFileID

getFTSFileIDs(statusList=None)

select FTSFileIDs for a given status list

getFTSFileList(statusList=None, limit=1000)

get at most :limit: FTSFiles with status in :statusList:

Parameters:
  • statusList (list) – list with FTSFiles statuses
  • limit (int) – select query limit
getFTSFilesForRequest(requestID, statusList=None)

get FTSFiles with status in :statusList: for request given its :requestID:

getFTSHistory()

query FTSHistoryView, return list of FTSHistoryViews

getFTSJob(ftsJobID=None)

get FTSJob given FTSJobID

getFTSJobIDs(statusList=['Submitted', 'Active', 'Ready'])

get FTSJobIDs for a given status list

getFTSJobList(statusList=None, limit=500)

select FTS jobs with statuses in :statusList:

getFTSJobsForRequest(requestID, statusList=None)

get list of FTSJobs with status in :statusList: for request given its requestID

TODO: should be more smart, i.e. one query to select all ftsfiles

getFields(tableName, outFields=None, condDict=None, limit=False, conn=None, older=None, newer=None, timeStamp=None, orderAttribute=None, greater=None, smaller=None)

Select “outFields” from “tableName” with condDict N records can match the condition return S_OK( tuple(Field,Value) ) if outFields is None all fields in “tableName” are returned if limit is not False, the given limit is set inValues are properly escaped using the _escape_string method, they can be single values or lists of values.

static getTableMeta()

get db schema in a dict format

getTables()

get tables

static getViewMeta()

return db views in dict format

at the moment only one view - FTSHistoryView

insertFields(tableName, inFields=None, inValues=None, conn=None, inDict=None)

Insert a new row in “tableName” assigning the values “inValues” to the fields “inFields”. String type values will be appropriately escaped.

peekFTSFile(ftsFileID)

peek FTSFile given FTSFileID

putFTSFile(ftsFile)

put FTSFile into fts db

putFTSFileList(ftsFileList)

bulk put of FSTFiles

Parameters:ftsFileList (list) – list with FTSFile instances
putFTSJob(ftsJob)

put FTSJob to the db (INSERT or UPDATE)

Parameters:ftsJob (FTSJob) – FTSJob instance
setFTSFilesWaiting(operationID, sourceSE, opFileIDList=None)

propagate states for descendants in replication tree

Parameters:
  • operationID (int) – ReqDB.Operation.OperationID
  • sourceSE (str) – waiting source SE
  • opFileIDList (list) – [ ReqDB.File.FileID, … ]
setFTSJobStatus(ftsJobID, status)

Set the status of an FTS job

transactionCommit()
transactionRollback()
transactionStart()
updateFields(tableName, updateFields=None, updateValues=None, condDict=None, limit=False, conn=None, updateDict=None, older=None, newer=None, timeStamp=None, orderAttribute=None, greater=None, smaller=None)

Update “updateFields” from “tableName” with “updateValues”. updateDict alternative way to provide the updateFields and updateValues N records can match the condition return S_OK( number of updated rows ) if limit is not False, the given limit is set String type values will be appropriately escaped.