Tags:
create new tag
view all tags

Introduction to DBS

  • Data Bookkeeping Service
  • Catalog all CMS event data from MC and online
    • Store dataset and block information, including files and SE info
    • LFN info of files in datasets and blocks
  • Global DBS vs. Local DBS

Goals of CMS local DBS

  • Share datasets among local group, PAG ,etc
  • Able to deal with private datasets in crab
    • Register your private dataset into local DBS
    • Read your private dataset from local DBS
  • Manage private datasets conveniently
    • Create, delete, etc

Functions of CMS local DBS

  • Publish your private dataset with CRAB
  • Use your private dataset with CRAB
  • Delete your private dataset with CRAB
  • Query your private dataset over web (http://cmsdbs.ihep.ac.cn:8003)
  • Create and Register your private dataset with CMS DBS API

Publish your private dataset with CRAB

  • Specify these jobs published results to local DBS in crab.cfg
         publish_data=1
         #<primarydataset>/<publish_data_name>/USER, ###/Zee/xmzhang-DBS_test01-83ec*/USER
         publish_data_name = DBS_test01
         dbs_url_for_publication = http://cmsdbs.ihep.ac.cn:8080/DBS/servlet/DBSServlet
         
  • Do the submission
         crab -create -submit 1
         
  • Get output of jobs when jobs are in "Done" state
         crab --status
         crab --getoutput
         
  • Publish the dataset into the IHEP local DBS
         crab --publish http://cmsdbs.ihep.ac.cn:8080/DBS/servlet/DBSServlet
         

Use your private dataset with CRAB

  • Fill the DBS url in crab.cfg
         dbs_url = http://cmsdbs.ihep.ac.cn:8080/DBS/servlet/DBSServlet
         
  • Specify your private dataset name
         datasetpath=/Zee/xmzhang-DBS_test-83ec64b9af82f6c4b27a36630c9d7d4e/USER
         
  • Do the submission
     
         Crab –create –submit 1
         

Delete your private dataset with CRAB

  • Use the tool DBSDeleteData.py from CRAB in the directory /python
         /home/cms/zhangxm/test_crab/CRAB_2_5_1/python
         
  • Issue the command to do the deletion
           python DBSDeleteData.py 
            --DBSURL=http://cmsdbs.ihep.ac.cn:8080/DBS/servlet/DBSServlet 
            --datasetPath=/QCD_pt_0_15/JobRobot_IDEAL_V9_JobRobot/GEN-SIM-RAW-RECO
         

Query your private dataset over web

Create and Register your private dataset with CMS DBS API

  • Set up env for DBS client in cmsui01 /home/cms/zhangxm/DBS/slc4_ia32_gcc345/cms/dbs-client/DBS_2_0_7-cmp/lib/setup.(c)sh
  • Following the steps to create and register your private dataset
    1. Create a Primary Dataset (If one doesn't exist already)
    2. Create an Algorithm (Combinition of Application and ParametersetID)
    3. Insert Run(s)
    4. Create a Processed Dataset for the Primary dataset, providing data tier etc)
    5. Insert Lumi Section (s)
    6. Create a Block
    7. Insert files into Processed dataset
  • Use the following scripts in the directory /home/cms/zhangxm/DBS/slc4_ia32_gcc345/cms/dbs-client/DBS_2_0_7-cmp/lib/RegisteFile for each step
    • dbsInsertPrimary.py
    • dbsInsertAlgorithm.py
    • dbsInsertRun.py
    • dbsInsertProcessed.py
    • dbsInsertLumiSection.py
    • dbsInsertBlock.py
    • dbsInsertFiles.py
Edit | Attach | Watch | Print version | History: r5 < r4 < r3 < r2 < r1 | Backlinks | Raw View | Raw edit | More topic actions
Topic revision: r5 - 2009-12-14 - ZhangXiaomei
 
This site is powered by the TWiki collaboration platform Powered by PerlCopyright © 2008-2019 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki? Send feedback