Dirac Service Limitations

DataManagement

LFN length

Because they are stored in a database, the LFNs are limited in size. The standard size is 255 characters. It is enforced in the following database:

  • JobDB

  • TransformationTB

  • StorageManagementDB

  • DataIntegrityDB

  • FTS3DB

  • RequestDB

Shall you want to have longer LFN, then you would need to update your database manually.

A special case is the DFC. The limitations depend on the Directory and File managers you use.

In the DirectoryLevelTree and FileManager (defaults one) managers, the LFNs are split by ‘/’, yielding other limitations:

  • 128 char for the filename

  • 255 char for each directory level

In case of the Managers WithPkAndPs (LHCb):

  • 128 char for the filename

  • 255 for the base path

Performance Optimisation

Servers

When you servers are heavily loaded, you may want to tune some kernel parameters. Internet is full or resources to explain you what you should do, but a few parameters of interests certainly are the number of file descriptors allowed, as well as a few kernel tcp parameters that should be increased (https://www.kernel.org/doc/Documentation/networking/ip-sysctl.txt):

net.nf_conntrack_max
net.ipv4.tcp_max_syn_backlog
net.core.somaxconn
net.core.netdev_max_backlog

Duplications

The WMS Executors can be duplicated.

In some cases, it is possible to run several instances of the same agent/service in order to scale.

NB: in the tables below, for those components where the Duplicate column is not filled, it simply means that we have not tried to duplicate them yet. For services, this probably means that they can be duplicated. For agents, instead, this probably means that they can not be duplicated.

Services

System

Component

Duplicate

Remarks

HTTPs

Accounting

DataStore

PARTIAL

One master and helpers (See DataStore Helpers)

ReportGenerator

Configuration

Configuration

PARTIAL

One master (rw) and slaves (ro). It’s advised to have several CS slaves

YES

DataManagement

DataIntegrity

YES

YES

FileCatalog

YES

YES

FileCatalogProxy

FTS3Manager

YES

YES

S3Gateway

YES

StorageElement

StorageElementProxy

Framework

BundleDelivery

YES

YES

ComponentMonitoring

YES

YES

Notification

YES

YES

ProxyManager

YES

YES

SecurityLogging

NO

SystemAdministrator

MUST

There should be one on each and every machine

TokenManager

YES

YES(only)

UserProfileManager

YES

YES

Monitoring

Monitoring

YES

YES

RequestManagement

ReqManager

YES

YES

ReqProxy

PARTIAL

Relies on local storage

ResourcesStatus

Publisher

YES

YES

ResourceManagement

YES

YES

ResourceStatus

YES

YES

StorageManager

StorageManager

Transformation

TransformationManager

YES

YES

WorkloadManagement

JobManager

YES

YES

JobMonitoring

YES

YES

JobStateUpdate

YES

YES

Matcher

NO

OptimizationMind

NO

PilotManager

PARTIAL

In case there are HTCondor CEs to deal with, the HTCondor WorkingDirectory should exist and be accessible in each and every machine

SandboxStore

WMSAdministrator

YES

YES

Agents

System

Component

Duplicate

Remarks

Accounting

NetworkAgent

Configuration

Bdii2CSAgent

PARTIAL

Select sites for different agent instances

GOCDB2CSAgent

VOMS2CSAgent

PARTIAL

Split by VOs

DataManagement

FTS3Agent

YES

Framework

CAUpdateAgent

RequestManagement

CleanReqDBAgent

NO

RequestExecutingAgent

YES

ResourceStatus

CacheFeederAgent

YES

ElementInspectorAgent

EmailAgent

NO

SiteInspectorAgent

SummarizeLogsAgent

TokenAgent

StorageManagement

RequestFinalizationAgent

NO

RequestPreparationAgent

NO

StageMonitorAgent

NO

StageRequestAgent

NO

Transformation

InputDataAgent

PARTIAL

Split by TransformationTypes

MCExtensionAgent

PARTIAL

Split by TransformationTypes

RequestTaskAgent

PARTIAL

Split by TransformationTypes

TransformationAgent

PARTIAL

Split by TransformationTypes

TransformationCleaningAgent

PARTIAL

Split by TransformationTypes

ValidateOutputDataAgent

PARTIAL

Split by TransformationTypes

WorkflowTaskAgent

PARTIAL

Split by TransformationTypes

WorkloadManagement

SiteDirector

PARTIAL

Split by Sites, CETypes

JobAgent

Installed by Pilots on Worker Nodes, not for server installations

JobCleaningAgent

YES

PilotSyncAgent

YES

PilotStatusAgent

YES

PushJobAgent

YES

Split by Sites

StalledJobAgent

YES

StatesAccountingAgent

NO

TaskQueuesAgent

YES