create new tag
view all tags

Platform based on DIRAC

What is DIRAC?

  • DIRAC(Distributed Infrastructure with Remote Agent Control) is a complete solution for using distributed computing resources.
  • DIRAC forms a layer between a particular community and various computing resources to allow optimized, transparent and reliable usage, including lcg resources, clusters, clouds, etc.


  • BesDirac is built as a uniform framework or platform for the BESIII experiment and users to manage and use distributed resources for production and analysis acitivities, which also allows an efficient use of non-reliable resources.
  • It would integrate:
    • Workload management: Data-driven workload scheduling system, various compute resources
    • Data management system: File and metadata catalog, data transfer, various storage resources
    • Job and storage monitoring system: local ang global monitoring system
  • Status:
    • 2011.4 the besdirac server is set up
    • 2011.6 four LCG and PBS sites(IHEP-PBS,IHEP-Grid, GUCAS, Dubna) are joined to accept jobs
    • 2011.7 two storage elements(dcache, dpm) are joined to accept data
    • 2011.8 working with ganga to accept mass production jobs
    • 2011.9 enable inputdata functions for analysis jobs
  • Link:

Mass job management with Ganga

What is Ganga?


  • GangaBoss is the package which we developed on ganga for BOSS users with all the necessary BES-specific plug-ins
  • It includes the following plug-ins:
    • Dirac -- use DIRAC API to talk to DIRAC server
    • Boss -- support BOSS applications, including generating run script,etc
    • Dataset -- support dataset-based operations Interface with BES metadata catalog, resolving and registering BOSS dataset Interface with BES data transfer system, subscribing dataset transfers
    • Splitter and Merger -- define a set of splitters and mergers to meet requirements of BES data processing

  • Status
    • Can well support both PBS and DIRAC backend
    • Able to split and submit BESIII production jobs to PBS and DIRAC
    • Ready to support BESIII reconstruction and analysis
  • Link

How to start as a user

get your private certificate ready and join BES VO

register in dirac CS

  • your DN are automatically registered in dirac CS after you registered as a BES VO memeber
  • test your proxy with dirac command
    • source /afs/ihep.ac.cn/bes3/bvrs/ganga/diracenv (for bash)
      source /afs/ihep.ac.cn/bes3/bvrs/ganga/diracenv.csh (for tcsh)
    • proxy-init -g bes_user

set up your environment to use ganga

  • source boss env
  • source ganga env
    • source /afs/ihep.ac.cn/bes3/bvrs/ganga/gangaEnv (for bash)
      source /afs/ihep.ac.cn/bes3/bvrs/ganga/gangaEnv.csh (for tcsh)
    • cp /afs/ihep.ac.cn/users/z/zhangxm/.gangarc .gangarc
    • define your private parameters in .gangarc
      • gangadir = /home/cms/zhangxm/gangadir # log directory
      • DataOutput = /home/cms/zhangxm/gangadir # data output dir

run ganga

  • run and submit a job to pbs
    • /afs/ihep.ac.cn/bes3/bvrs/ganga/Ganga/install/5.6.2/bin/ganga /afs/ihep.ac.cn/bes3/bvrs/ganga/tutorial/sim_pbs.py

How to start as a site

  • PBS site
    • register your site in DIRAC
              (1) the name of the torque server
              (2) the queue to be used for BES
              (3) a user account which can do the submission to PBS resources.
              (4) the name of UI where the user can login to do the submission to PBS resources.
    • network requirements
             (1) UI need to be accessible from the outside
             (2) work nodes need to access abroad
    • install CVMFS client to get BOSS software deployed
              (1) The installation and configuration of CVMFS client, please refer to the CVMFS technical reports:
              (2) Add support for BES in one of configuration files(/etc/cvmfs/default.local)

-- ZhangXiaomei - 2011-10-16:

Edit | Attach | Watch | Print version | History: r16 < r15 < r14 < r13 < r12 | Backlinks | Raw View | Raw edit | More topic actions
Topic revision: r16 - 2012-02-27 - ZhangXiaomei
This site is powered by the TWiki collaboration platform Powered by PerlCopyright © 2008-2023 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki? Send feedback