Mainly used by Computing Elements to run several jobs/pilots in parallel within a same allocation. This is an abstract class and cannot be used directly. Use cases:
multiple nodes have been allocated and have to be used: a parallel library can execute the pilot wrapper on all the nodes in parallel.
multiple cpus have been allocated and have to be used: a parallel library could dispatch several jobs on the cores available (e.g. when sites have no external connectivity).
Parallel libraries implementations have to inherit from this class and override generateWrapper() and processOutput().
The Parallel Libraries have to be defined in a CE section via the configuration system. See the page about configuring Resources / Computing for where the options can be placed.
Generally defined in the ComputingElement source code depending on how they deal with the executables. Indicates where the parallel library wrapper script has to be defined and retrieved.
- class DIRAC.Resources.Computing.ParallelLibraries.ParallelLibrary.ParallelLibrary(name='undefined', workingDirectory='')
- __init__(name='undefined', workingDirectory='')
Initialize the parallel library
Associate the executable with a wrapper script, generated by the parallel library, to execute the same command in parallel on multiple nodes. Wrap it in a new executable file.
executableFile (str) – name of the executable file to wrap
- Return str
name of the wrapper that runs the executable via the parallel library
- processOutput(output, error, isFile=True)
Reorder the content of the output files that is possibly not easily readable after the execution of the parallel library. If isFile is enabled, then output and error have to be filenames and are modified in-place. Else the output and error contains content of the files, and the processed content is returned.