Dirac Service Limitations

DataManagement

LFN length

Because they are stored in a database, the LFNs are limited in size. The standard size is 255 characters. It is enforced in the following database:

  • JobDB
  • TransformationTB
  • StorageManagementDB
  • DataIntegrityDB
  • FTS3DB
  • RequestDB

Shall you want to have longer LFN, then you would need to update your database manually.

A special case is the DFC. The limitations depend on the Directory and File managers you use.

In the DirectoryLevelTree and FileManager (defaults one) managers, the LFNs are split by ‘/’, yielding other limitations:

  • 128 char for the filename
  • 255 char for each directory level

In case of the Managers WithPkAndPs (LHCb):

  • 128 char for the filename
  • 255 for the base path

Performance Optimisation

Servers

When you servers are heavily loaded, you may want to tune some kernel parameters. Internet is full or resources to explain you what you should do, but a few parameters of interests certainly are the number of file descriptors allowed, as well as a few kernel tcp parameters that should be increased (https://www.kernel.org/doc/Documentation/networking/ip-sysctl.txt):

net.nf_conntrack_max
net.ipv4.tcp_max_syn_backlog
net.core.somaxconn
net.core.netdev_max_backlog

Finally, the parameter SocketBacklog for a service can be increased (man listen is your friend).

Duplications

In some cases, it is possible to run several instances of the same agent/service in order to scale.

Services

System Component Duplicate Remarque
Accounting DataStore PARTIAL One master and helpers (See DataStore Helpers)
ReportGenerator    
Configuration Configuration PARTIAL  
DataManagement DataIntegrity YES  
FileCatalog YES  
FileCatalogProxy    
FTS3Manager YES  
IRODSStorageElement    
S3Gateway YES  
StorageElement    
StorageElementProxy    
Framework BundleDelivery    
ComponentMonitoring    
Monitoring    
Notification    
Plotting    
ProxyManager YES  
RabbitMQSync    
SecurityLogging YES  
SystemAdministrator YES In principle there should be one on each and every machine
SystemLogging    
SystemLoggingReport    
UserProfileManager    
Monitoring Monitoring    
RequestManagement ReqManager YES  
ReqProxy YES  
ResourcesStatus Publisher    
ResourceManagement    
ResourceStatus    
StorageManager StorageManager    
Transformation TransformationManager    
WorkloadManagement JobManager YES  
JobMonitoring YES  
JobStateUpdate YES  
Matcher NO  
OptimizationMind    
PilotManager YES  
SandboxStore    
WMSAdministrator    

Agents

System Component Duplicate Remarque
Accounting NetworkAgent    
Configuration Bdii2CSAgent PARTIAL Select sites for different agent instances
GOCDB2CSAgent    
VOMS2CSAgent PARTIAL Split by VOs
DataManagement FTS3Agent YES  
Framework CAUpdateAgent    
MyProxyRenewalAgent    
RequestManagement CleanReqDBAgent NO  
RequestExecutingAgent YES  
ResourceStatus CacheFeederAgent    
ElementInspectorAgent    
EmailAgent NO  
SiteInspectorAgent    
SummarizeLogsAgent    
TokenAgent    
StorageManagement RequestFinalizationAgent NO  
RequestPreparationAgent NO  
StageMonitorAgent NO  
StageRequestAgent NO  
Transformation InputDataAgent PARTIAL Split by TransformationTypes
MCExtensionAgent PARTIAL Split by TransformationTypes
RequestTaskAgent PARTIAL Split by TransformationTypes
TransformationAgent PARTIAL Split by TransformationTypes
TransformationCleaningAgent PARTIAL Split by TransformationTypes
ValidateOutputDataAgent PARTIAL Split by TransformationTypes
WorkflowTaskAgent PARTIAL Split by TransformationTypes
WorkloadManagement SiteDirector PARTIAL Split by Sites, CETypes
JobAgent    
JobCleaningAgent    
PilotStatusAgent    
StalledJobAgent    
StatesAccountingAgent    
StatesMonitoringAgent