SSH (Virtual) Computing Element
For a given IP/host it will send jobs directly through ssh
Configuration for the SSHComputingElement submission can be done via the configuration system.
Area where the job errors are stored. If not defined: SharedArea + ‘/data’ is used. If not absolute: SharedArea + path is used.
Area where the job outputs are stored. If not defined: SharedArea + ‘/data’ is used. If not absolute: SharedArea + path is used.
Underlying batch system that is going to be used to orchestrate executable files. The Batch System has to be remotely accessible. By default, the SSHComputingElement submits directly on the host via the Host class. Available batch systems are defined in
Area where the executable files are stored if necessary. If not defined: SharedArea + ‘/data’ is used. If not absolute: SharedArea + path is used.
Area used to store executable/output/error files if they are not aready defined via BatchOutput, BatchError, InfoArea, ExecutableArea and/or WorkArea. The path should be absolute.
SSH host name
SSH user login
Port number if not standard, e.g. for the gsissh access
Location of the ssh private key for no-password connection
Any other SSH options to be used. Example:
SSHOptions = -o UserKnownHostsFile=/local/path/to/known_hosts
Allows to have a local copy of the
known_hostsfile, independent of the HOME directory.
String defining the use of intermediate SSH host. Example:
ssh -i /private/key/location -l final_user final_host
SSH (default) or gsissh
- class DIRAC.Resources.Computing.SSHComputingElement.SSH(host=None, parameters=None)
SSH class encapsulates passing commands and files through an SSH tunnel to a remote host. It can use either ssh or gsissh access. The final host where the commands will be executed and where the files will copied/retrieved can be reached through an intermediate host if SSHTunnel parameters is defined.
SSH constructor parameters are defined in a SSH accessible Computing Element in the Configuration System:
SSHHost: SSH host name
SSHUser: SSH user login
SSHPassword: SSH password
SSHPort: port number if not standard, e.g. for the gsissh access
SSHKey: location of the ssh private key for no-password connection
SSHOptions: any other SSH options to be used
- SSHTunnel: string defining the use of intermediate SSH host. Example:
‘ssh -i /private/key/location -l final_user final_host’
SSHType: ssh ( default ) or gsissh
The class public interface includes two methods:
sshCall( timeout, command_sequence ) scpCall( timeout, local_file, remote_file, upload = False/True )
- __init__(host=None, parameters=None)
- scpCall(timeout, localFile, remoteFile, postUploadCommand='', upload=True)
Perform file copy through an SSH magic.
timeout (int) – timeout of the command
localFile (str) – local file path, serves as source for uploading and destination for downloading. Can take ‘Memory’ as value, in this case the downloaded contents is returned as result[‘Value’]
remoteFile (str) – remote file full path
postUploadCommand (str) – command executed on the remote side after file upload
upload (bool) – upload if True, download otherwise
- class DIRAC.Resources.Computing.SSHComputingElement.SSHComputingElement(ceUniqueID)
This method returns the number of available slots in the target CE. The CE instance polls for waiting and running jobs and compares to the limits in the CE parameters.
jobIDList (list) – list of already existing job IDs to be checked against
Method to return information on running and pending jobs.
Get CE description as a dictionary.
This is called by the JobAgent for the case of “inner” CEs.
- getJobOutput(jobID, localDir=None)
Get the specified job standard output and error files. If the localDir is provided, the output is returned as file in this directory. Otherwise, the output is returned as strings.
Get the status information for the given list of jobs
Initialize the CE parameters after they are collected from various sources
Check if the stored proxy is valid
Check the sanity of the Computing Element definition
Kill a bunch of jobs
Instantiate object representing the backend batch system
batchSystemName (str) – name of the batch system
- sendOutput(stdid, line)
Callback function such that the results from the CE may be returned.
Update the CPUTime parameter of the CE classAd, necessary for running in filling mode
Add parameters from the given dictionary overriding the previous values
ceOptions (dict) – CE parameters dictionary to update already defined ones
- setProxy(proxy, valid=0)
Set and prepare proxy to use
- setToken(token, valid=0)
Optional method to shutdown the (Inner) Computing Element
- submitJob(executableFile, proxy, numberOfJobs=1)
Method to submit job, should be overridden in sub-class.
CE helper function to write a CE proxy string to a file.