Dirac Service Limitations

DataManagement

LFN length

Because they are stored in a database, the LFNs are limited in size. The standard size is 255 characters. It is enforced in the following database:

  • JobDB

  • TransformationTB

  • StorageManagementDB

  • DataIntegrityDB

  • FTS3DB

  • RequestDB

Shall you want to have longer LFN, then you would need to update your database manually.

A special case is the DFC. The limitations depend on the Directory and File managers you use.

In the DirectoryLevelTree and FileManager (defaults one) managers, the LFNs are split by ‘/’, yielding other limitations:

  • 128 char for the filename

  • 255 char for each directory level

In case of the Managers WithPkAndPs (LHCb):

  • 128 char for the filename

  • 255 for the base path

Performance Optimisation

Servers

When you servers are heavily loaded, you may want to tune some kernel parameters. Internet is full or resources to explain you what you should do, but a few parameters of interests certainly are the number of file descriptors allowed, as well as a few kernel tcp parameters that should be increased (https://www.kernel.org/doc/Documentation/networking/ip-sysctl.txt):

net.nf_conntrack_max
net.ipv4.tcp_max_syn_backlog
net.core.somaxconn
net.core.netdev_max_backlog

Finally, the parameter SocketBacklog for a service can be increased (man listen is your friend).

Duplications

The WMS Executors can be duplicated.

In some cases, it is possible to run several instances of the same agent/service in order to scale.

NB: in the tables below, for those components where the Duplicate column is not filled, it simply means that we have not tried to duplicate them yet. For services, this probably means that they can be duplicated. For agents, instead, this probably means that they can not be duplicated.

Services

System

Component

Duplicate

Remarque

Accounting

DataStore

PARTIAL

One master and helpers (See DataStore Helpers)

ReportGenerator

Configuration

Configuration

PARTIAL

One master (rw) and slaves (ro). It’s advised to have several CS slaves

DataManagement

DataIntegrity

YES

FileCatalog

YES

FileCatalogProxy

FTS3Manager

YES

S3Gateway

YES

StorageElement

StorageElementProxy

Framework

BundleDelivery

ComponentMonitoring

YES

Monitoring

NO

Notification

Plotting

ProxyManager

YES

RabbitMQSync

SecurityLogging

YES

SystemAdministrator

MUST

There should be one on each and every machine

UserProfileManager

Monitoring

Monitoring

YES

RequestManagement

ReqManager

YES

ReqProxy

YES

ResourcesStatus

Publisher

YES

ResourceManagement

YES

ResourceStatus

YES

StorageManager

StorageManager

Transformation

TransformationManager

YES

WorkloadManagement

JobManager

YES

JobMonitoring

YES

JobStateUpdate

YES

Matcher

NO

OptimizationMind

PilotManager

PARTIAL

In case there are HTCondor CEs to deal with, the HTCondor WorkingDirectory should exist and be accessible in each and every machine (see HTCondorCE Parameters)

SandboxStore

WMSAdministrator

YES

Agents

System

Component

Duplicate

Remarque

Accounting

NetworkAgent

Configuration

Bdii2CSAgent

PARTIAL

Select sites for different agent instances

GOCDB2CSAgent

VOMS2CSAgent

PARTIAL

Split by VOs

DataManagement

FTS3Agent

YES

Framework

RequestManagement

CAUpdateAgent

CleanReqDBAgent

NO

RequestExecutingAgent

YES

ResourceStatus

CacheFeederAgent

ElementInspectorAgent

EmailAgent

NO

SiteInspectorAgent

SummarizeLogsAgent

TokenAgent

StorageManagement

RequestFinalizationAgent

NO

RequestPreparationAgent

NO

StageMonitorAgent

NO

StageRequestAgent

NO

Transformation

InputDataAgent

PARTIAL

Split by TransformationTypes

MCExtensionAgent

PARTIAL

Split by TransformationTypes

RequestTaskAgent

PARTIAL

Split by TransformationTypes

TransformationAgent

PARTIAL

Split by TransformationTypes

TransformationCleaningAgent

PARTIAL

Split by TransformationTypes

ValidateOutputDataAgent

PARTIAL

Split by TransformationTypes

WorkflowTaskAgent

PARTIAL

Split by TransformationTypes

WorkloadManagement

SiteDirector

PARTIAL

Split by Sites, CETypes

JobAgent

Installed by Pilots on Worker Nodes, not for server installations

JobCleaningAgent

PilotStatusAgent

StalledJobAgent

StatesAccountingAgent

NO