DataIntegrityDB

DataIntegrityDB class is a front-end to the Data Integrity Database.

class DIRAC.DataManagementSystem.DB.DataIntegrityDB.DataIntegrityDB(parentLogger=None)

Bases: DB

Only 1 table, that can be created with:

CREATE TABLE Problematics(
  FileID INTEGER NOT NULL AUTO_INCREMENT,
  Prognosis VARCHAR(32) NOT NULL,
  LFN VARCHAR(255) NOT NULL,
  PFN VARCHAR(255),
  Size BIGINT(20),
  SE VARCHAR(32),
  GUID VARCHAR(255),
  Status VARCHAR(32) DEFAULT 'New',
  Retries INTEGER DEFAULT 0,
  InsertDate DATETIME NOT NULL,
  LastUpdate DATETIME NOT NULL,
  Source VARCHAR(127) NOT NULL DEFAULT 'Unknown',
  PRIMARY KEY(FileID),
  INDEX (Prognosis,Status)
);

__init__(parentLogger=None): Standard Constructor

buildCondition(condDict=None, older=None, newer=None, timeStamp=None, orderAttribute=None, limit=False, greater=None, smaller=None, offset=None, useLikeQuery=False): Build SQL condition statement from provided condDict and other extra check on a specified time stamp. The conditions dictionary specifies for each attribute one or a List of possible values greater and smaller are dictionaries in which the keys are the names of the fields, that are requested to be >= or < than the corresponding value. For compatibility with current usage it uses Exceptions to exit in case of invalid arguments For performing LIKE queries use the parameter useLikeQuery=True

changeProblematicPrognosis(fileID, newPrognosis): Change prognisis for file by FileID

countEntries(table, condDict, older=None, newer=None, timeStamp=None, connection=False, greater=None, smaller=None): Count the number of entries wit the given conditions

deleteEntries(tableName, condDict=None, limit=False, conn=None, older=None, newer=None, timeStamp=None, orderAttribute=None, greater=None, smaller=None): Delete rows from “tableName” with N records can match the condition if limit is not False, the given limit is set String type values will be appropriately escaped, they can be single values or lists of values.

executeStoredProcedure(packageName, parameters, outputIds, *, conn=None)

executeStoredProcedureWithCursor(packageName, parameters, *, conn=None)

getCSOption(optionName, defaultValue=None)

getCounters(table, attrList, condDict, older=None, newer=None, timeStamp=None, connection=False, greater=None, smaller=None): Count the number of records on each distinct combination of AttrList, selected with condition defined by condDict and time stamps

getDistinctAttributeValues(table, attribute, condDict=None, older=None, newer=None, timeStamp=None, connection=False, greater=None, smaller=None): Get distinct values of a table attribute under specified conditions

getDistinctPrognosis(): Get a list of all the current problematic types

getFields(tableName, outFields=None, condDict=None, limit=False, conn=None, older=None, newer=None, timeStamp=None, orderAttribute=None, greater=None, smaller=None, useLikeQuery=False): Select “outFields” from “tableName” with condDict N records can match the condition return S_OK(tuple(Field, Value)) if outFields is None all fields in “tableName” are returned if limit is not False, the given limit is set inValues are properly escaped using the _escape_string method, they can be single values or lists of values. if useLikeQuery=True, then conDict can return matched rows if “%” is defined inside conDict.

getProblematic(): Get the next file to resolve

getProblematicsSummary(): Get a summary of the current problematics table

getPrognosisProblematics(prognosis): Get all the active files with the given problematic

getTransformationProblematics(transID): Get problematic files matching a given production

incrementProblematicRetry(fileID): Increment retry count

insertFields(tableName, inFields=None, inValues=None, conn=None, inDict=None): Insert a new row in “tableName” assigning the values “inValues” to the fields “inFields”. String type values will be appropriately escaped.

insertProblematic(source, fileMetadata): Insert the supplied file metadata into the problematics table

removeProblematic(fileID): Remove Problematic file by FileID

setProblematicStatus(fileID, status): Set Status for problematic file by FileID

transactionCommit()

transactionRollback()

transactionStart()

updateFields(tableName, updateFields=None, updateValues=None, condDict=None, limit=False, conn=None, updateDict=None, older=None, newer=None, timeStamp=None, orderAttribute=None, greater=None, smaller=None): Update “updateFields” from “tableName” with “updateValues”. updateDict alternative way to provide the updateFields and updateValues N records can match the condition return S_OK( number of updated rows ) if limit is not False, the given limit is set String type values will be appropriately escaped.