ElasticJobParametersDB¶
Module containing a front-end to the ElasticSearch-based ElasticJobParametersDB. This module interacts with one ES index: “ElasticJobParametersDB”, which is a drop-in replacement for MySQL-based table JobDB.JobParameters. While JobDB.JobParameters in MySQL is defined as:
CREATE TABLE `JobParameters` (
`JobID` INT(11) UNSIGNED NOT NULL,
`Name` VARCHAR(100) NOT NULL,
`Value` BLOB NOT NULL,
PRIMARY KEY (`JobID`,`Name`),
FOREIGN KEY (`JobID`) REFERENCES `Jobs`(`JobID`)
) ENGINE=InnoDB DEFAULT CHARSET=latin1;
Here we define a mapping as:
"JobID": {"type": "long"},
"Name": {"type": "keyword"},
"Value": {"type": "text"}
which is an “equivalent” representation.
The reason for switching to a ES-based JobParameters lies in the extended searching capabilities of ES (ES will analyze+index text fields, while MySQL won’t do that on BLOB types). This results in higher traceability for DIRAC jobs.
- The following class methods are provided for public usage
getJobParameters()
setJobParameter()
deleteJobParameters()
- class DIRAC.WorkloadManagementSystem.DB.ElasticJobParametersDB.ElasticJobParametersDB¶
Bases:
ElasticDB
- RESULT_SIZE = 10000¶
- __init__()¶
Standard Constructor
- bulk_index(indexPrefix, data=None, mapping=None, period='day', withTimeStamp=True)¶
- Parameters
- Returns
S_OK/S_ERROR
- clusterName = ''¶
- createIndex(indexPrefix, mapping=None, period='day')¶
- deleteByQuery(indexName, query)¶
Delete data by query (careful!)
- deleteJobParameters(jobID, paramList=None)¶
- delete Job Parameters defined for jobID.
Returns a dictionary with the Job Parameters. If paramList is empty - all the parameters for the job are removed
- existingIndex(indexName)¶
Checks the existance of an index, by its name
- Parameters
indexName (str) – the name of the index
- Returns
S_OK/S_ERROR if the request is successful
- static generateFullIndexName(indexName, period)¶
Given an index prefix we create the actual index name.
- getCSOption(optionName, defaultValue=None)¶
- getDocTypes(indexName)¶
Returns mappings, by index.
- Parameters
indexName (str) – is the name of the index…
- Returns
S_OK or S_ERROR
- getIndexPrefix()¶
It returns the DIRAC setup.
- getIndexes(indexName=None)¶
It returns the available indexes…
- getJobParameters(jobID, paramList=None)¶
- Get Job Parameters defined for jobID.
Returns a dictionary with the Job Parameters. If paramList is empty - all the parameters are returned.
- getUniqueValue(indexName, key, orderBy=False)¶
- index(indexName, body=None, docID=None, op_type='index')¶
- pingDB()¶
Try to connect to the database
- Returns
S_OK(TRUE/FALSE)
- query(index, query)¶
Executes a query and returns its result (uses ES DSL language).
- setJobParameter(jobID, key, value)¶
Inserts data into ElasticJobParametersDB index
- setJobParameters(jobID, parameters)¶
Inserts data into ElasticJobParametersDB index using bulk indexing
- update(index, query=None, updateByQuery=True, id=None)¶
Executes an update of a document, and returns S_OK/S_ERROR