create new tag
view all tags

Meeting Minutes - 2012-01-12

Attendants: WU Wenjing, LI Sha, KA Wenxiao, Fabio, Daniel

Time: 2:30 p.m.

Secretary: Fabio

* Introduction

  • Daniel is a Master student supervised by Yaodong who is interested in contributing to the project. He comes for the first time to get familiar with the project and with the people involved. Initally he will be able to devote 1 day/week on this project. Starting from September 2012, he would be able to work full time on it.
  • Fabio prepared a presentation for giving an overview of the project, the motivation, the technology used, the current status and the next steps. It can be found in the Resources section of this site.

* OpenStack Swift

  • Sha deployed Swift (1 proxy node, 3 storage nodes) on the testbed using physical machines. She experienced some problems as some dependencies were not well documented for Scientific Linux v5. In particular, the version of SQLite needs to be upgraded in order to be compatible with OpenStack's requirements.
  • The authentication service enabled is Swauth, which is an associated development to OpenStack for providing authentication service for Swift. Its internals need to be understood and in particular we have to know whether it can be a single point of failure in the installation.
  • Regarding the access control to Swift objects, Sha reports that the ACLs are assigned to containers and not to objects. One can authorize other users to read from or write to objects per container, but not to delete, which is only for the owner of the container.
  • The proxy node is reponsible for storing the configuration of the cluster. Reconfiguration of the replication parameters is possible but needs to rebuild the ring and potentially to rebalance the data. It is not yet understood what are the operational consequences of this kind of interventions. Rebalancing the ring is possible without shuting down the service, for instance, when adding or removing a machine. The rebalance must be manually triggered by a Swift administrator.
  • Since Swift already provides an S3-compatible protocol, we could use it to build the whole system or only the file-contents store. We need to improve our understanding of the system before making a decision.
  • We agreed to use this platform to perform some tests:
    • functionality tests: store, retrieve, delete, list objects both using the command line interface [Sha] and the Python API [Wenjing]
    • [Sha & Wenxiao] scalability tests: perform a significant number of operations (create, delete, retrieve, ...) and measure the time needed for each of them. Summarize the results.
    • [Wening] understand the authentication mechanism used by Swift to identify the possible ways to integrated with the authentication mechanism used by grid jobs which is based in X509 proxy certificates. Fabio has already done some work to demonstrate that using the LHCb's pyGSI module is possible to develop a client-server dialog which uses grid proxies as authentication mechanism.

* Filesystem-backed file-contents store

  • Fabio will continue working on the implementation of the file-contents store using a networked file system, such as Gluster. Such a kind of solution could be of interest for IHEP which has extensive experience running large-scale file-systems and could provide a cloud-based file storage service on top of Gluster.


  • A contribution (poster) for presenting this project was accepted to the CHEP 2102 conference to be held in New York on May 2012.
  • We agreed to target the end of February to have a complete demonstrator of the system either using Swift or using Redis + networked file system.
  • Next meeting: Thursday February 23rd 2012, 2:30 pm, Fabio's office

-- FabioHernandez - 2012-01-13

Topic revision: r1 - 2012-01-13 - FabioHernandez
This site is powered by the TWiki collaboration platform Powered by PerlCopyright © 2008-2022 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki? Send feedback