Resources / Sites - Subsections¶
In this section each DIRAC site available for the users is described. The convention to name the sites consist of 3 strings:
Grid site name, expressed in uppercase, for example: LCG, EELA
Institution acronym in uppercase, for example: CPPM
Country: country where the site is located, expressed in lowercase, for example fr
The three strings are concatenated with “.” to produce the name of the sites.
Name |
Description |
Example |
<DIRAC_SITE_NAME> |
Subsection named with the site name |
LCG.CPPM.fr |
<DIRAC_SITE_NAME>/Name |
Site name gave by the site administrator e.g.: the name of the site in GOCDB (optional) |
Name = in2p3 |
<DIRAC_SITE_NAME>/CE |
List of CEs using CE FQN These CEs are updated by the BDII2CSAgent in the CEs section |
CE = ce01.in2p3.fr CE += ce02.in2p3.fr |
<DIRAC_SITE_NAME>/MoUTierLevel |
Tier Level (optional) |
MoUTierLevel = 1 |
<DIRAC_SITE_NAME>/CEs/ |
Subsection used to describe each CE available |
CEs |
<DIRAC_SITE_NAME>/Coordinates |
Site geographical coordinates (optional) |
Coordinates = -8.637979:41.152461 |
<DIRAC_SITE_NAME>/Mail |
Mail address site responsable (optional) |
Mail = atsareg@in2p3.fr |
<DIRAC_SITE_NAME>/SE |
Closest SE respect to the CE (optional) |
SE = se01.in2p3.fr |
CEs sub-subsection¶
This sub-subsection specifies the attributes of each particular CE of the site. For each DIRAC site there can be more than one CE.
Name |
Description |
Example |
<CE_NAME> |
Subsection named as the CE fully qualified name |
ce01.in2p3.fr |
<CE_NAME>/architecture |
CE architecture |
architecture = x86_64 |
<CE_NAME>/CEType |
Type of CE, can take values as LCG or CREAM |
CEType = ARC |
<CE_NAME>/LocalCEType |
Type of ‘Inner’ CE, normally empty. Default = “InProcess”. Possibilities: potentially all CE types, but in practice the most valid would be: InProcess, Sudo, Singularity, Pool. Pool CE in turn uses InProcess (Default) or Sudo or Singularity. To specify, use Pool/ce_type. This option can also go at the Queue level. |
LocalCEType = Pool LocalCEType = Pool/Singularity |
<CE_NAME>/OS |
CE operating system in a DIRAC format |
OS = ScientificLinux_Boron_5.3 |
<CE_NAME>/Pilot |
Boolean attributes than indicates if the site accept pilots |
Pilot = True |
<CE_NAME>/SubmissionMode |
If the CE is a cream CE the mode of submission |
SubmissionMode = Direct |
<CE_NAME>/wnTmpDir |
Worker node temporal directory |
wnTmpDir = /tmp |
<CE_NAME>/NumberOfProcessors |
Number of available processors on worker nodes |
NumberOfProcessors = 12 |
<CE_NAME>/NumberOfNodes |
Number of available worker nodes per allocation. Values can be a number (e.g. 2 nodes) or a range of values (e.g. from 2 to 4 nodes) which lets the choice to the batch system. |
NumberOfNodes = 2 NumberOfNodes = 2 or 2-4 |
<CE_NAME>/WholeNode |
CE allows whole node jobs |
WholeNode = True |
<CE_NAME>/Tag |
List of tags specific for the CE |
Tag = GPU,96RAM |
<CE_NAME>/RequiredTag |
List of required tags that a job to be eligible must have |
RequiredTag = GPU,96RAM |
<CE_NAME>/Queues |
Subsection. Queues available for this VO in the CE |
Queues |
<CE_NAME>/Queues/<QUEUE_NAME> |
Name of the queue exactly how is published |
jobmanager-pbs-formation |
<CE_NAME>/Queues/<QUEUE_NAME>/CEQueueName |
Name of the queue in the corresponding CE if not the same as the name of the queue section |
CEQueueName = pbs-grid |
<CE_NAME>/Queues/<QUEUE_NAME>/maxCPUTime |
Maximum time allowed to jobs to run in the queue |
maxCPUTime = 1440 |
<CE_NAME>/Queues/<QUEUE_NAME>/MaxTotalJobs |
If the CE is a CREAM CE the maximum number of jobs in all the status |
MaxTotalJobs =200 |
<CE_NAME>/Queues/<QUEUE_NAME>/MaxWaitingJobs |
If the CE is a CREAM CE the maximum number of jobs in waiting status |
MaxWaitingJobs = 70 |
<CE_NAME>/Queues/<QUEUE_NAME>/OutputURL |
If the CE is a CREAM CE the URL where to find the outputs |
OutputURL = gsiftp://localhost |
<CE_NAME>/Queues/<QUEUE_NAME>/SI00 |
CE CPU Scaling Reference |
SI00 = 2130 |
<CE_NAME>/Queues/<QUEUE_NAME>/NumberOfProcessors |
overrides <CE_NAME>/NumberOfProcessors at queue level |
NumberOfProcessors = 12 |
<CE_NAME>/Queues/<QUEUE_NAME>/NumberOfNodes |
overrides <CE_NAME>/NumberOfNodes |
NumberOfNodes = 2 |
<CE_NAME>/Queues/<QUEUE_NAME>/WholeNode |
overrides <CE_NAME>/WholeNode at queue level |
WholeNode = True |
<CE_NAME>/Queues/<QUEUE_NAME>/Tag |
List of tags specific for the Queue |
Tag = GPU,96RAM |
<CE_NAME>/Queues/<QUEUE_NAME>/RequiredTag |
List of required tags that a job to be eligible must have |
RequiredTag = GPU,96RAM |
<CE_NAME>/Queues/<QUEUE_NAME>/LocalCEType |
Same as <CE_NAME>/LocalCEType (see above) but per queue. |
LocalCEType = Pool/Singularity |
An example for this session follows:
Sites
{
LCG
{
LCG.CERN.cern
{
SE = CERN-RAW
SE += CERN-RDST
SE += CERN-USER
CE = ce503.cern.ch
CE += ce504.cern.ch
Name = CERN-PROD
Coordinates = 06.0458:46.2325
Mail = grid-cern-prod-admins@cern.ch
MoUTierLevel = 0
Description = CERN European Organization for Nuclear Research
CEs
{
ce503.cern.ch
{
wnTmpDir = .
architecture = x86_64
OS = ScientificCERNSLC_Carbon_6.4
SI00 = 0
Pilot = False
CEType = HTCondorCE
SubmissionMode = Direct
Queues
{
ce503.cern.ch-condor
{
VO = lhcb
VO += LHCb
SI00 = 3100
MaxTotalJobs = 5000
MaxWaitingJobs = 200
maxCPUTime = 7776
LocalCEType = Pool/Singularity
Tag = MultiProcessor
}
}
VO = lhcb
MaxRAM = 0
UseLocalSchedd = False
DaysToKeepLogs = 1
}
ce504.cern.ch
{
wnTmpDir = .
architecture = x86_64
OS = ScientificCERNSLC_Carbon_6.4
SI00 = 0
Pilot = False
CEType = HTCondorCE
LocalCEType = Pool
SubmissionMode = Direct
Queues
{
ce504.cern.ch-condor
{
VO = lhcb
VO += LHCb
SI00 = 3100
MaxTotalJobs = 5000
MaxWaitingJobs = 200
maxCPUTime = 7776
}
}
}
}
}
}
DIRAC
{
DIRAC.HLTFarm.lhcb
{
Name = LHCb-HLTFARM
CE = OnlineCE.lhcb
CEs
{
OnlineCE.lhcb
{
CEType = CREAM
Queues
{
OnlineQueue
{
maxCPUTime = 2880
}
}
}
}
AssociatedSEs
{
Tier1-RDST = CERN-RDST
Tier1_MC-DST = CERN_MC-DST-EOS
Tier1-Buffer = CERN-BUFFER
Tier1-Failover = CERN-EOS-FAILOVER
Tier1-BUFFER = CERN-BUFFER
Tier1-USER = CERN-USER
SE-USER = CERN-USER
}
}
}
VAC
{
VAC.Manchester.uk
{
Name = UKI-NORTHGRID-MAN-HEP
CE = vac01.blackett.manchester.ac.uk
CE += vac02.blackett.manchester.ac.uk
Coordinates = -2.2302:53.4669
Mail = ops@NOSPAMtier2.hep.manchester.ac.uk
CEs
{
vac01.blackett.manchester.ac.uk
{
CEType = Vac
architecture = x86_64
OS = ScientificSL_Carbon_6.4
wnTmpDir = /scratch
SI00 = 2200
MaxCPUTime = 1000
Queues
{
default
{
maxCPUTime = 1000
}
}
}
vac02.blackett.manchester.ac.uk
{
CEType = Vac
architecture = x86_64
OS = ScientificSL_Carbon_6.4
wnTmpDir = /scratch
SI00 = 2200
MaxCPUTime = 1000
Queues
{
default
{
maxCPUTime = 1000
}
}
}
}
}
}
}