ElasticDB

ElasticDB is a base class used to connect an Elasticsearch database and manages queries.

class DIRAC.Core.Base.ElasticDB.ElasticDB(dbname, fullName, indexPrefix='', parentLogger=None)

Bases: DIRACDB, ElasticSearchDB

Class for interfacing DIRAC ES DB definitions to ES clusters

RESULT_SIZE = 10000
__init__(dbname, fullName, indexPrefix='', parentLogger=None)

c’tor

Parameters:

  • self – self reference

  • dbName (str) – DIRAC name of the database for example: ‘MonitoringDB’

  • fullName (str) – The DIRAC full name of the database for example: ‘Monitoring/MonitoringDB’

  • indexPrefix (str) – it is the indexPrefix used to load all indexes

  • parentLogger – logger to use as parentLogger

bulk_index(indexPrefix, data=None, mapping=None, period='day', withTimeStamp=True)
Parameters:

  • indexPrefix (str) – index name.

  • data (list) – contains a list of dictionary

  • mapping (dict) – the mapping used by elasticsearch

  • period (str) – Accepts ‘day’ and ‘month’. We can specify which kind of indexes will be created.

  • withTimeStamp (bool) – add timestamp to data, if not there already.

Returns:

S_OK/S_ERROR

clusterName = ''
createIndex(indexPrefix, mapping=None, period='day')
Parameters:

  • indexPrefix (str) – it is the index name.

  • mapping (dict) – the configuration of the index.

  • period (str) – We can specify, which kind of index will be created. Currently only daily and monthly indexes are supported.

deleteByQuery(indexName, query)

Delete data by query (careful!)

Parameters:

  • indexName (str) – the name of the index

  • query (str) – the JSON-formatted query for which we want to issue the delete

deleteDoc(index: str, docID: str)

Deletes a document in an index.

Parameters:

  • index – name of the index

  • docID – document ID

deleteIndex(indexName)
Parameters:

indexName (str) – the name of the index to be deleted…

existingIndex(indexName)

Checks the existance of an index, by its name

Parameters:

indexName (str) – the name of the index

Returns:

S_OK/S_ERROR if the request is successful

existsDoc(index: str, docID: str) → bool

Returns information about whether a document exists in an index.

Parameters:

  • index – name of the index

  • docID – document ID

static generateFullIndexName(indexName, period)

Given an index prefix we create the actual index name.

Parameters:

  • indexName (str) – it is the name of the index

  • period (str) – We can specify which kind of indexes will be created (day, week, month, year, null).

Returns:

string with full index name

getCSOption(optionName, defaultValue=None)
getDoc(index: str, docID: str) → dict

Retrieves a document in an index.

Parameters:

  • index – name of the index

  • docID – document ID

getDocTypes(indexName)

Returns mappings, by index.

Parameters:

indexName (str) – is the name of the index…

Returns:

S_OK or S_ERROR

getDocs(indexFunc, docIDs: list[str]) → list[dict]

Efficiently retrieve many documents from an index.

Parameters:

  • index – name of the index

  • docIDs – document IDs

getIndexPrefix()

It returns the DIRAC setup.

getIndexes(indexName=None)

It returns the available indexes…

getUniqueValue(indexName, key, orderBy=False)
Parameters:

  • indexName (str) – the name of the index which will be used for the query

  • orderBy (dict) – it is a dictionary in case we want to order the result {key:’desc’} or {key:’asc’}

Returns:

a list of unique value for a certain key from the dictionary.

index(indexName, body=None, docID=None, op_type='index')
Parameters:

  • indexName (str) – the name of the index to be used

  • body (dict) – the data which will be indexed (basically the JSON)

  • id (int) – optional document id

  • op_type (str) – Explicit operation type. (options: ‘index’ (default) or ‘create’)

Returns:

the index name in case of success.

pingDB()

Try to connect to the database

Returns:

S_OK(TRUE/FALSE)

query(index: str, query)

Executes a query and returns its result (uses ES DSL language).

Parameters:

  • self – self reference

  • index (str) – index name

  • query (dict) – It is the query in ElasticSearch DSL language

update(index: str, query=None, updateByQuery: bool = True, docID: str = None)

Executes an update of a document, and returns S_OK/S_ERROR

Parameters:

  • index – index name

  • query – It is the query in ElasticSearch DSL language

  • updateByQuery – A bool to determine update by query or index values using index function.

  • docID – ID for the document to be created.

updateDoc(index: str, docID: str, body) → dict

Update an existing document with a script or partial document

Parameters:

  • index – name of the index

  • docID – document ID

  • body – The request definition requires either script or partial doc