10.2.3. Pilots bootstrapping
Pilot3 needs some Python files, and a JSON file for bootstrapping. We simply call this file the
pilot.json file is created starting from information found in the Configuration Service.
The pilot wrapper (the script that starts the pilot, which is effectively equivalent to what SiteDirectors send)
expects to find and download such a
pilot.json file from a known location, or a set of them.
Such a location should be exposed via https:// by, for example, the DIRAC WebApp webserver (but not necessarily).
To make sure that the
pilot.json is created and uploaded, there’s an agent that can be used for your convenience: the agent PilotsSyncAgent syncs CS and pilot files to a web server of your choice.
The (list of) location(s) where the agent would upload the pilot files can be added in the CS agent option UploadLocations. If this location is not set, then the agent will try to use the Operations option Pilot/pilotFileServer.
If more than one location is used, add them as a list. We suggest to add at least the URL of the DIRAC WebApp webserver, but multiple locations are also possible, and advised.
Starting with DIRAC version v7r2, the file uploads is completely on the balancer (nginx) side. Make sure your balancer is set to load files by following the instructions.
Other options that can be set also in the Operations part of the CS include:
Pointer to git repository of DIRAC pilot
pilotRepo = https://github.com/DIRACGrid/Pilot.git The value above is the default
Pointer to git repository of VO DIRAC extension of pilot. This option is needed in case you have an extension of the pilot
pilotVORepo = https://github.com/MyDIRAC/VOPilot.git
Path to the code, inside the Git repository
pilotScriptsPath = Pilot This value is the default
Path to the code, inside the Git repository of the pilot extension
pilotVOScriptsPath = VOPilot
Branch to use, inside the Git repository, of the pilot code to be used
pilotRepoBranch = master The value above is the default
Branch to use, inside the Git repository, of the pilot code extension to be used
pilotVORepoBranch = master The value above is the default
Whether to try to upload the files to the list of server specified
uploadToWebApp = True The value above is the default
Local directory of the master CS where the files will be downloaded before the upload
workDir = /tmp/pilotSyncDir There is no default (so /opt/dirac/runit/Configuration/Server)
10.2.3.1. Pilots environment
There is a small number of environment variables that can be set by the pilot to control the behaviour of DIRAC jobs.
The number of processors the pilot can use
The number of processors the jobs administered by the pilot can use (a subset of DIRAC_PROCESSORS)
A boolean flag indicating the pilot is exploiting the whole node (normally, False)
10.2.3.2. Install non-released code from within the Pilot
To install non-released code from within the pilot, the dirac-pilot command can be run with the –modules option. This option can be steered from the CS too.
would end up installing a specific branch (
rel-v7r3_FEAT_proxyStrength) pushed to github.
An easy way to try is to add the
Modules option in the CS configuration of one of your ComputingElements: