Tags:
create new tag
view all tags

Usage of CMS SE

Overview

The Storage Element in T2_Beijing site uses dCache system. Now we have 10TB storage space, one poolnode and five pools. If you want to know more about the hardware of dCache system, please look here. Users need to know the name of the following functional servers:
   SRM Server= srm.ihep.ac.cn
   DCAP Server = cmspn01.ihep.ac.cn
The use directory in SE now. if not, you can create one for yourself.
   /pnfs/ihep.ac.cn/data/cms/<username>

How to Use SRM Server

Storage Resource Managers (SRMs) are middleware components whose function is to provide dynamic space allocation and file management on shared storage components on the Grid. SRMs support protocol negotiation and a reliable replication mechanism. The SRM specification standardizes the interface, thus allowing for a uniform access to heterogeneous storage elements. The common commands used by users:
   srmPing ---  is used to check the state of the SRM. It works as an "are you alive" type of call.
   srmCopy --- is used to copy files from source storage sites into the target storage sites. The source storage site or the target storage site needs to be the SRM itself that the client makes the srmCopy request. If both source and target are local to the SRM, it performed a local copy. There are two cases for remote copies: 1. Target SRM is where client makes a srmCopy request (PULL case), 2. Source SRM is where client makes a srmCopy request (PUSH case).
                1. PULL case: Upon the client’s srmCopy request, the target SRM makes a space at the target storage, and makes a request srmPrepareToGet to the source SRM. When TURL is ready at the source SRM, the target SR M transfers the file from the source TURL into the prepared target storage. After the file transfer completes, srmReleaseFiles is issued to the source SRM.
                2. PUSH case: Upon the client’s srmCopy request, the source SRM prepares a file to be transferred out to the target SRM, and makes a request srmPrepareToPut to the target SRM. When TURL is ready at the target SRM, the source SRM transfers the file from the prepared source into the prepared target TURL. After the file transfer completes, srmPutDone is issued to the target SRM.
   When specified target space token is provided, the files will be located finally in the targeted space associated with the space token.  It is an asynchronous operation, and request token must be returned. The status may be checked through srmStatusOfCopyRequest with the returned request token.
   srmLs --- returns a list of files with a basic information. This operation may be asynchronous, and in such case, requestToken must be returned.
   srmRm --- remove SURLs (the name space entries) in the storage system. 
   srmRmdir --- removes an empty directory in a local SRM space
   srmMkdir --- create a directory in a local SRM space.
If you want to know more SRM command, please refer to The Storage Resource Manager Interface Specification. Here are some examples for common comands:
   srmmkdir srm://srm.ihep.ac.cn:8443/pnfs/ihep.ac.cn/data/zhangxm/test1
   srmls srm://srm.ihep.ac.cn:8443/pnfs/ihep.ac.cn/data/cms/zhangxm/test1
   srmrmdir srm://srm.ihep.ac.cn:8443/pnfs/ihep.ac.cn/data/cms/zhangxm/test1
   srmrm srm://srm.ihep.ac.cn:8443/pnfs/ihep.ac.cn/data/cms/largefile2
   srmping srm://srm.ihep.ac.cn:8443
   srmcp -debug=true file:////home/zhangxm/test/srmv2test/test1 srm://srm.ihep.ac.cn:8443/pnfs/ihep.ac.cn/data/cms/zhangxm/test1
   srmcp -debug=true srm://srm.ihep.ac.cn:8443/pnfs/ihep.ac.cn/data/cms/zhangxm/test1 file:////home/zhangxm/test/srmv2test/test2

Additional tools-RErmdir and REcpdir

The location of it is in the directory of /opt/exp_soft/cms/bjtools/. The goal of RErmdir is to let users do the deletion of non-empty diectory. The usage of RErmdir is:
  $ /opt/exp_soft/cms/bjtools/RErmdir  
  Usage: RErmdir dir1,dir2,..... 
  dir1, dir2:  the directories after srm://srm.ihep.ac.cn:8443/pnfs/ihep.ac.cn/data/cms/<dir1,dir2>
  Delete directories in the SE server srm.ihep.ac.cn
The goal of REcpdir is to transfer a dataset from T2 to T3 and avoid SE crashed by too many parallel transferring. The usage of REcpdir is:
  /opt/exp_soft/cms/bjtools/REcpdir
Usage: REcpdir srcdir destdir
srcdir: the directory after srm://srm.ihep.ac.cn:8443/pnfs/ihep.ac.cn/data/cms/
destdir: the local directory after the file:/// protocol string

NOTE: the current version only copies the first 999 files in a single directory.

Space Management

Please feel free to give some comments.

Edit | Attach | Watch | Print version | History: r6 < r5 < r4 < r3 < r2 | Backlinks | Raw View | Raw edit | More topic actions
Topic revision: r6 - 2010-09-16 - TangZhicheng
 
This site is powered by the TWiki collaboration platform Powered by PerlCopyright © 2008-2019 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki? Send feedback