10.2.9. InputDataResolution: giving job access to the data
When a job needs access to data, there are two ways data can be accessed:
either by downloading the file on the local worker node
or by reading the data remotely, aka
The resolution is done in the
JobWrapper (see DIRAC jobs: definitions). By default, the resolution logic is implemented in
InputDataResolution. It can be overwritten by the Job JDL (see
InputDataModule in Job Description Language Reference), or by the
You can look into this class for more details, but to summarize:
it will look into the
jobJDL if it can find
InputDataPolicyoption. If so, it will use that as the module.
If not, it will check whether a policy is defined for the site we are running on (in
If not, it will run the default policy specified in
InputDataPolicy parameter can either be set directly in the JDL, in which case it should be a full module, or it can be set using the
Job class (see
This module will download the files locally on the worker node for processing.
DownloadInputData for details.
This module will generate the URLs necessary to access the files remotely.
InputDataByProtocol for details.