create new tag
view all tags

Meeting Minutes - 2011-08-10

Attendants: Wenjing, Ran, Fabio

Time: 2:30 p.m.

Secretary: Fabio

  • OpenStack Swift [Ran]
    • Ran has been doing some tests with Swift administration commands and observed some issues. In particular, she was not able to find the way to create unprivileged users. She did manage to create admin-level users for doing some tests.
    • She also found that in the current version of Swift the support for ACL (access control lists) was removed.
    • Swift requires that the reconfiguration of the cluster (for instance for increasing or decreasing the number of replicas) need the data to be downloaded and uploaded again. This may pose problems for the day-to-day operations of a Swift cluster.
  • Cassandra [Wenjing]
    • Wenjing was reviewing the prototype she wrote to understand Cassandra and make some initial measurements. One of the improvements she wants to implement is to keep the connection open between the client and server. She performed the initial round of tests on 3 virtual machines.
  • Redis [Fabio]
    • Fabio has been working on refining his Redis-based store for managing meta-data and made some performance measurements.
    • He has performed two kind of tests:
      • run a Python-based client interacting with a non-persistent Redis server in the same development machine (no TCP overhead)
      • run a Python-based client interacting with a non-persistent Redis server in 2 machines of the IHEP login fam (round-trip time 0.113ms).
    • A plot summarizing the results of the first test is attached. The test consisted of performing all the meta-data related operations for creating around 1 million objects (i.e. files and directories) in the file store and measuring the service time as perceived by the client. The bottom line is that such a configuration allows for a sustained rate of around 1000 meta-data operations per second on a development-level machine using a single client thread.
    • More tests need to be performed in the testbed using the recently acquired hardware (see paragraph below) and taking into account the latency between the participating machines. Also, some tests with several concurrent clients need to be performed.
    • The plot of the second tests were not available for the meeting.
  • Other businesses [All]
    • The new machines for the testbed platform were purchased and should be delivered in the coming days. Indeed, since the meeting took place, the machines were installed on August 19th and are ready for testing. The configuration of this new testbed was documented by Wenjing and is available in the Testbed page.
    • Ran will be leave for a 1-year-long stay at CERN in the coming months and is going to leave the project. In the next meeting we will decide whether it is worth continuing the exploration of Swift.

-- FabioHernandez - 2011-08-25

Topic attachments
I Attachment History Action Size Date Who Comment
PDFpdf DEP3-SUB40-FIL1000-Duration-Histogram.pdf r1 manage 33.7 K 2011-08-25 - 08:53 FabioHernandez Performance measurements of Redis-backed meta-data store
Topic revision: r1 - 2011-08-25 - FabioHernandez
This site is powered by the TWiki collaboration platform Powered by PerlCopyright © 2008-2022 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki? Send feedback