10.2.3. Pilots bootstrapping

Pilot3 needs some Python files, and a JSON file for bootstrapping. We simply call this file the pilot.json file. The pilot.json file is created starting from information found in the Configuration Service.

The pilot wrapper (the script that starts the pilot, which is effectively equivalent to what SiteDirectors send) expects to find and download such a pilot.json file from a known location, or a set of them. Such a location should be exposed via https:// by, for example, the DIRAC WebApp webserver (but not necessarily).

To make sure that the pilot.json is created and uploaded, there’s an agent that can be used for your convenience: the agent PilotsSyncAgent syncs CS and pilot files to a web server of your choice. The (list of) location(s) where the agent would upload the pilot files can be added in the CS agent option UploadLocations. If this location is not set, then the agent will try to use the Operations option Pilot/pilotFileServer.

If more than one location is used, add them as a list. We suggest to add at least the URL of the DIRAC WebApp webserver, but multiple locations are also possible, and advised.

Starting with DIRAC version v7r2, the file uploads is completely on the balancer (nginx) side. Make sure your balancer is set to load files by following the instructions.

Other options that can be set also in the Operations part of the CS include:


Pointer to git repository of DIRAC pilot

pilotRepo = https://github.com/DIRACGrid/Pilot.git The value above is the default


Pointer to git repository of VO DIRAC extension of pilot. This option is needed in case you have an extension of the pilot

pilotVORepo = https://github.com/MyDIRAC/VOPilot.git


Path to the code, inside the Git repository

pilotScriptsPath = Pilot This value is the default


Path to the code, inside the Git repository of the pilot extension

pilotVOScriptsPath = VOPilot


Branch to use, inside the Git repository, of the pilot code to be used

pilotRepoBranch = master The value above is the default


Branch to use, inside the Git repository, of the pilot code extension to be used

pilotVORepoBranch = master The value above is the default


CVMFS locations to use for finding pilot files, DIRAC installations, CAs, CRLS

CVMFS_locations = /cvmfs/lhcb.cern.ch/,/cvmfs/lhcbdev.cern.ch


Local directory of the master CS where the files will be downloaded before the upload

workDir = /tmp/pilotSyncDir There is no default (so /opt/dirac/runit/Configuration/Server) Pilots environment

There is a small number of environment variables that can be set by the pilot to control the behaviour of DIRAC jobs.


The number of processors the pilot can use


The number of processors the jobs administered by the pilot can use (a subset of DIRAC_PROCESSORS)


A boolean flag indicating the pilot is exploiting the whole node (normally, False) Install non-released code from within the Pilot

To install non-released code from within the pilot, the dirac-pilot command can be run with the –modules option. This option can be steered from the CS too.

For example:

dirac-pilot.py --modules=https://github.com/chaen/DIRAC.git:::DIRAC:::rel-v7r3_FEAT_proxyStrength

would end up installing a specific branch (rel-v7r3_FEAT_proxyStrength) pushed to github by user chaen.

An easy way to try is to add the Modules option in the CS configuration of one of your ComputingElements: