5. Installing the RequestManagement System

5.1. Pre-requisite

You should:

5.2. Tutorial goal

The aim of the tutorial is to install the RequestManagement system components and to use it to perform a simple replication of file.

5.4. Installing the RMS

This section is to be executed as diracuser with a proxy with dirac_admin group.

The RMS needs the ReqManager service and the RequestExecutingAgent to work (you may want to add the CleanReqDBAgent if you scale…).

The RMS is no different than any other DIRAC system. The installation step are thus very simple:

[[email protected] ~]$ dirac-admin-sysadmin-cli --host dirac-tuto
Pinging dirac-tuto...
[dirac-tuto]$ add instance RequestManagement Production
Adding RequestManagement system as Production self.instance for MyDIRAC-Production self.setup to dirac.cfg and CS
RequestManagement system instance Production added successfully
[dirac-tuto]$ restart *
All systems are restarted, connection to SystemAdministrator is lost
[dirac-tuto]$ install db ReqDB
MySQL root password:
Adding to CS RequestManagement/ReqDB
Database ReqDB from DIRAC/RequestManagementSystem installed successfully
[dirac-tuto]$ install service RequestManagement ReqManager
Loading configuration template /home/diracuser/DiracInstallation/DIRAC/RequestManagementSystem/ConfigTemplate.cfg
Adding to CS service RequestManagement/ReqManager
service RequestManagement_ReqManager is installed, runit status: Run
[dirac-tuto]$ install agent RequestManagement RequestExecutingAgent
Loading configuration template /home/diracuser/DiracInstallation/DIRAC/RequestManagementSystem/ConfigTemplate.cfg
Adding to CS agent RequestManagement/RequestExecutingAgent
agent RequestManagement_RequestExecutingAgent is installed, runit status: Run
[dirac-tuto]$ quit

By default, the installation of the RequestExecutingAgent will configure it with a whole bunch of default Operations possible. You can see that in the Agent configuration in /Systems/RequestManagement/Production/Agents/RequestExecutingAgent/OperationHandlers

5.5. Testing the RMS

This section is to be executed with a proxy with dirac_data group.

The test we are going to do consists in transferring a file from one storage element to another, using the RequestExecutingAgent.

First, let’s add a file:

[[email protected] ~]$ echo "My Test File" > /tmp/myTestFile.txt
[[email protected] ~]$ dirac-dms-add-file /tutoVO/user/c/ciuser/myTestFile.txt /tmp/myTestFile.txt StorageElementOne

Uploading /tutoVO/user/c/ciuser/myTestFile.txt
Successfully uploaded file to StorageElementOne

We can see that our file is indeed in the StorageElementOne:

[[email protected] ~]$ dirac-dms-lfn-replicas /tutoVO/user/c/ciuser/myTestFile.txt
LFN                                  StorageElement    URL
==========================================================
/tutoVO/user/c/ciuser/myTestFile.txt StorageElementOne dips://dirac-tuto:9148/DataManagement/StorageElement/tutoVO/user/c/ciuser/myTestFile.txt

Let’s replicate it to StorageElementTwo using the RMS:

[[email protected] ~]$ dirac-dms-replicate-and-register-request myFirstRequest /tutoVO/user/c/ciuser/myTestFile.txt StorageElementTwo
Request 'myFirstRequest' has been put to ReqDB for execution.
RequestID(s): 8
You can monitor requests' status using command: 'dirac-rms-request <requestName/ID>'

The Request has a name (myFirstRequest) that we chose, but also an ID, returned by the system (here 8). The ID is guaranteed to be unique, while the name is not, so it is recommended to use the ID when you interact with the RMS. You can see the status of your Request, using its name or ID:

[[email protected] ~]$ dirac-rms-request myFirstRequest
Request name='myFirstRequest' ID=8 Status='Waiting'
Created 2019-04-23 14:37:05, Updated 2019-04-23 14:37:05, NotBefore 2019-04-23 14:37:05
Owner: '/C=ch/O=DIRAC/OU=DIRAC CI/CN=ciuser/[email protected]', Group: dirac_data
  [0] Operation Type='ReplicateAndRegister' ID=8 Order=0 Status='Waiting'
      TargetSE: StorageElementTwo - Created 2019-04-23 14:37:05, Updated 2019-04-23 14:37:05
    [01] ID=2 LFN='/tutoVO/user/c/ciuser/myTestFile.txt' Status='Waiting' Checksum='1e750431'

[[email protected] ~]$ dirac-rms-request 8
Request name='myFirstRequest' ID=8 Status='Waiting'
Created 2019-04-23 14:37:05, Updated 2019-04-23 14:37:05, NotBefore 2019-04-23 14:37:05
Owner: '/C=ch/O=DIRAC/OU=DIRAC CI/CN=ciuser/[email protected]', Group: dirac_data
  [0] Operation Type='ReplicateAndRegister' ID=8 Order=0 Status='Waiting'
      TargetSE: StorageElementTwo - Created 2019-04-23 14:37:05, Updated 2019-04-23 14:37:05
    [01] ID=2 LFN='/tutoVO/user/c/ciuser/myTestFile.txt' Status='Waiting' Checksum='1e750431'

You can here clearly see that the Request consists of one ReplicateAndRegister operation (which does what it says) targeting the LFN /tutoVO/user/c/ciuser/myTestFile.txt. The RequestExecutingAgent will pick up the request and execute it. And shortly you should be able to see it done:

[[email protected] ~]$ dirac-rms-request 8
Request name='myFirstRequest' ID=8 Status='Done'
Created 2019-04-23 14:37:05, Updated 2019-04-23 14:37:29, NotBefore 2019-04-23 14:37:05
Owner: '/C=ch/O=DIRAC/OU=DIRAC CI/CN=ciuser/[email protected]', Group: dirac_data
  [0] Operation Type='ReplicateAndRegister' ID=8 Order=0 Status='Done'
      TargetSE: StorageElementTwo - Created 2019-04-23 14:37:05, Updated 2019-04-23 14:37:29
    [01] ID=2 LFN='/tutoVO/user/c/ciuser/myTestFile.txt' Status='Done' Checksum='1e750431'

[[email protected] ~]$ dirac-dms-lfn-replicas /tutoVO/user/c/ciuser/myTestFile.txt
LFN                                  StorageElement    URL
==========================================================
/tutoVO/user/c/ciuser/myTestFile.txt StorageElementTwo dips://dirac-tuto:9147/DataManagement/StorageElementTwo/tutoVO/user/c/ciuser/myTestFile.txt
                                    StorageElementOne dips://dirac-tuto:9148/DataManagement/StorageElement/tutoVO/user/c/ciuser/myTestFile.txt

5.6. Conclusion

You now have an RMS in place, which is the base for all the asynchronous operations in DIRAC. This is used for big scale operations, failover, or even more !