VirtualMachineDB

VirtualMachineDB class is a front-end to the virtual machines DB

Life cycle of VMs Images in DB

  • New: Inserted by Director (Name - Status = New ) if not existing when launching a new instance

  • Validated: Declared by VMMonitoring Server when an Instance reports back correctly

  • Error: Declared by VMMonitoring Server when an Instance reports back wrong requirements

Life cycle of VMs Instances in DB

  • New: Inserted by Director before launching a new instance, to check if image is valid

  • Submitted: Inserted by Director (adding UniqueID) when launches a new instance

  • Wait_ssh_context: Declared by Director for submitted instance wich need later contextualization using ssh

    (VirtualMachineContextualization will check)

  • Contextualizing: on the waith_ssh_context path is the next status before Running

  • Running: Declared by VMMonitoring Server when an Instance reports back correctly

    (add LastUpdate, publicIP and privateIP)

  • Stopping: Declared by VMManager Server when an Instance has been deleted outside of the VM

    (f.e “Delete” button on Browse Instances)

  • Halted: Declared by VMMonitoring Server when an Instance reports halting

  • Stalled: Declared by VMManager Server when detects Instance no more running

  • Error: Declared by VMMonitoring Server when an Instance reports back wrong requirements

    or reports as running when Halted

New Instances can be launched by Director if VMImage is not in Error Status.

Instance UniqueID: for KVM it could be the MAC, for Amazon the returned InstanceID(i-5dec3236), or Occi returned the VMID

class DIRAC.WorkloadManagementSystem.DB.VirtualMachineDB.VirtualMachineDB(maxQueueSize=10, parentLogger=None)

Bases: DB

__init__(maxQueueSize=10, parentLogger=None)

c’tor

Parameters:

self – self reference

allowedTransitions = {'Image': {'Validated': ['New', 'Validated']}, 'Instance': {'Halted': ['New', 'Running', 'Stopping', 'Stalled', 'Halted'], 'Running': ['Submitted', 'Running', 'Stalled', 'New'], 'Stalled': ['New', 'Submitted', 'Running'], 'Stopping': ['Running', 'Stalled'], 'Submitted': ['New']}}
buildCondition(condDict=None, older=None, newer=None, timeStamp=None, orderAttribute=None, limit=False, greater=None, smaller=None, offset=None, useLikeQuery=False)

Build SQL condition statement from provided condDict and other extra check on a specified time stamp. The conditions dictionary specifies for each attribute one or a List of possible values greater and smaller are dictionaries in which the keys are the names of the fields, that are requested to be >= or < than the corresponding value. For compatibility with current usage it uses Exceptions to exit in case of invalid arguments For performing LIKE queries use the parameter useLikeQuery=True

checkImageStatus(imageName)

Check Status of a given image Will insert a new Image in the DB if it does not exits returns: S_OK(Status) if Status is valid and not Error S_ERROR(ErrorMessage) otherwise

countEntries(table, condDict, older=None, newer=None, timeStamp=None, connection=False, greater=None, smaller=None)

Count the number of entries wit the given conditions

declareInstanceHalting(uniqueID, load)

Insert the heart beat info from a halting instance Declares “Halted” the instance and the image It returns S_ERROR if the status is not OK

declareInstanceRunning(uniqueID, publicIP, privateIP='')

Declares an instance Running and sets its associated info (uniqueID, publicIP, privateIP) Returns S_ERROR if: - instanceName does not have a “Submitted” or “Contextualizing” entry - uniqueID is not unique

declareInstanceStopping(instanceID)

Mark a VM instance to be stopped.

Next time the instance’s VirtualMachineMonitor checks in for an update it will be told to halt.

Returns:

S_OK if instance updated, S_ERROR otherwise.

declareInstanceSubmitted(uniqueID)

After submission of the instance the Director should declare the submitted Status

declareStalledInstances()

Check last Heart Beat for all Running instances and declare them Stalled if older than interval

deleteEntries(tableName, condDict=None, limit=False, conn=None, older=None, newer=None, timeStamp=None, orderAttribute=None, greater=None, smaller=None)

Delete rows from “tableName” with N records can match the condition if limit is not False, the given limit is set String type values will be appropriately escaped, they can be single values or lists of values.

executeStoredProcedure(packageName, parameters, outputIds, *, conn=None)
executeStoredProcedureWithCursor(packageName, parameters, *, conn=None)
getAllInfoForUniqueID(uniqueID)

Get all fields for a uniqueID

getCSOption(optionName, defaultValue=None)
getCounters(table, attrList, condDict, older=None, newer=None, timeStamp=None, connection=False, greater=None, smaller=None)

Count the number of records on each distinct combination of AttrList, selected with condition defined by condDict and time stamps

getDistinctAttributeValues(table, attribute, condDict=None, older=None, newer=None, timeStamp=None, connection=False, greater=None, smaller=None)

Get distinct values of a table attribute under specified conditions

getEndpointFromInstance(uniqueId)

For a given instance uniqueId it returns the asociated Endpoint in the instance table, thus the ImageName of such instance

getFields(tableName, outFields=None, condDict=None, limit=False, conn=None, older=None, newer=None, timeStamp=None, orderAttribute=None, greater=None, smaller=None, useLikeQuery=False)

Select “outFields” from “tableName” with condDict N records can match the condition return S_OK(tuple(Field, Value)) if outFields is None all fields in “tableName” are returned if limit is not False, the given limit is set inValues are properly escaped using the _escape_string method, they can be single values or lists of values. if useLikeQuery=True, then conDict can return matched rows if “%” is defined inside conDict.

getHistoryForInstanceID(instanceId)
getHistoryValues(averageBucket, selDict=None, fields2Get=False, timespan=0)
getInstanceCounters(groupField='Status', selDict=None)
getInstanceID(uniqueID)

For a given uniqueID of an instance return associated internal InstanceID

getInstanceParameter(pName, instanceID)

Get the instance parameter pName for the given instanceID

Parameters:

  • pName (str) – parameter name

  • instanceID (str) – instance unique identifier

Returns:

S_OK/S_ERROR, parameter value

getInstanceStatus(instanceID)

By dirac instanceID

getInstancesByStatus(status)

Get dictionary of Image Names with InstanceIDs in given status

getInstancesContent(selDict, sortList, start=0, limit=0)
Function to get the contents of the db

parameters are a filter to the db

getRunningInstancesBEPHistory(timespan=0, bucketSize=900)
getRunningInstancesByImageHistory(timespan=0, bucketSize=900)
getRunningInstancesByRunningPodHistory(timespan=0, bucketSize=900)
getRunningInstancesHistory(timespan=0, bucketSize=900)
getUniqueID(instanceID)

For a given dirac instanceID get the corresponding cloud endpoint uniqueID

getUniqueIDByName(instanceName)

Get the cloud provider unique ID corresponding to the DIRAC unique name

Parameters:

instanceName (str) – VM name

Returns:

S_OK/S_ERROR, cloud unique ID as value

insertFields(tableName, inFields=None, inValues=None, conn=None, inDict=None)

Insert a new row in “tableName” assigning the values “inValues” to the fields “inFields”. String type values will be appropriately escaped.

insertInstance(uniqueID, imageName, instanceName, endpoint, runningPodName)

Check Status of a given image Will insert a new Instance in the DB returns: S_OK( InstanceID ) if new Instance is properly inserted S_ERROR(ErrorMessage) otherwise

instanceIDHeartBeat(uniqueID, load, jobs, transferredFiles, transferredBytes, uptime)

Insert the heart beat info from a running instance It checks the status of the instance and the corresponding image Declares “Running” the instance and the image It returns S_ERROR if the status is not OK

recordDBHalt(instanceID, load)

Insert the heart beat info from a halting instance Declares “Halted” the instance and the image It returns S_ERROR if the status is not OK

setInstanceUniqueID(instanceID, uniqueID)

Assign a uniqueID to an instance

stallingInterval = 2400
tablesDesc = {'vm_History': {'Fields': {'InstanceID': 'INTEGER UNSIGNED NOT NULL', 'Jobs': 'INTEGER UNSIGNED NOT NULL DEFAULT 0', 'Load': 'FLOAT NOT NULL', 'Status': 'VARCHAR(32) NOT NULL', 'TransferredBytes': 'BIGINT UNSIGNED NOT NULL DEFAULT 0', 'TransferredFiles': 'INTEGER UNSIGNED NOT NULL DEFAULT 0', 'Update': 'DATETIME'}, 'Indexes': {'InstanceID': ['InstanceID']}}, 'vm_Images': {'Fields': {'ErrorMessage': 'VARCHAR(255) NOT NULL DEFAULT ""', 'LastUpdate': 'DATETIME', 'Name': 'VARCHAR(255) NOT NULL', 'Status': 'VARCHAR(16) NOT NULL', 'VMImageID': 'BIGINT UNSIGNED AUTO_INCREMENT NOT NULL'}, 'PrimaryKey': 'VMImageID'}, 'vm_Instances': {'Fields': {'Endpoint': 'VARCHAR(255) NOT NULL', 'ErrorMessage': 'VARCHAR(255) NOT NULL DEFAULT ""', 'InstanceID': 'BIGINT UNSIGNED AUTO_INCREMENT NOT NULL', 'Jobs': 'INTEGER UNSIGNED NOT NULL DEFAULT 0', 'LastUpdate': 'DATETIME', 'Load': 'FLOAT DEFAULT 0', 'MaxAllowedPrice': 'FLOAT DEFAULT NULL', 'Name': 'VARCHAR(255) NOT NULL', 'PrivateIP': 'VARCHAR(32) NOT NULL DEFAULT ""', 'PublicIP': 'VARCHAR(32) NOT NULL DEFAULT ""', 'RunningPod': 'VARCHAR(255) NOT NULL', 'Status': 'VARCHAR(32) NOT NULL', 'UniqueID': 'VARCHAR(255) NOT NULL DEFAULT ""', 'Uptime': 'INTEGER UNSIGNED DEFAULT 0', 'VMImageID': 'INTEGER UNSIGNED NOT NULL'}, 'Indexes': {'Status': ['Status']}, 'PrimaryKey': 'InstanceID'}}
transactionCommit()
transactionRollback()
transactionStart()
updateFields(tableName, updateFields=None, updateValues=None, condDict=None, limit=False, conn=None, updateDict=None, older=None, newer=None, timeStamp=None, orderAttribute=None, greater=None, smaller=None)

Update “updateFields” from “tableName” with “updateValues”. updateDict alternative way to provide the updateFields and updateValues N records can match the condition return S_OK( number of updated rows ) if limit is not False, the given limit is set String type values will be appropriately escaped.

validImageStates = ['New', 'Validated', 'Error']
validInstanceStates = ['New', 'Submitted', 'Running', 'Stopping', 'Halted', 'Stalled', 'Error']