Tags:
create new tag
view all tags
Base on the keyspace I described here, I performed some stress tests on cassandra.
Hardware:

cassandra cluster: 6 cluster nodes from our testbed

test node: one head node(fsw01)

Methodology of testing

I use pycassa API to implement a namespace to perform basic namespace operations. For this test, I only use one operation(make directory), which involves key-value pair insert/list operations in cassandra.

2. As it takes longer time to make a new directory when the depth of the directory goes further, so for this test, the depth of the directory tree

is set to 4.

code for testing:

for i in range(n1,n2):
   make_dir("/dir"+str(i))
   for j in range(n3,n4):
      make_dir("/dir"+str(i)+"/dir"+str(j)) 
      for k in range(n5,n6):
          make_dir("/dir"+str(i)+"/dir"+str(j)+"/dir"+str(k)) 
         for l in range(n7,n8):
            make_dir("/dir"+str(i)+"/dir"+str(j)+"/dir"+str(k)+"/dir"+str(l)) 

each make_dir operation involves several(1-3) list operation and an insert operations in cassandra cluster.

Test Results:

I ran 2 tests, while I did a slight modification in the API call in the 2nd test which has twice improved the performance.(the average time in the 2nd test is only 50% of the value of the 1st test).

Modifications:

instead of using

column_family.get("row_key",COLUMN_COUNT=100000)['col_name1'],

I use a different API:

column_family.get("row_key",COLUMNS=['col_name1'])['col_name1']

which will significantly reduce the number of columns returned by Cassandra cluster.

in Pycassa API, the default API get('row_key') will return 100 columns.

1st result:

total time is 70797 seconds, operation coun ts is 5075500, average time for each make_dir operation is 0.0139 seconds, min imum time for each operation is 0.004 seconds, max imum time for each operation is 5.048 seconds.

(time in seconds) --> number of operations which use this much time in seconds

0.004 --> 4
0.005 --> 256
0.006 --> 2865
0.007 --> 17669
0.008 --> 68920
0.009 --> 162279
0.010 --> 295330
0.011 --> 446647
0.012 --> 572977
0.013 --> 668155
0.014 --> 707582
0.015 --> 662395
0.016 --> 554026
0.017 --> 403742
0.018 --> 261432
0.019 --> 152893
0.020 --> 66725
0.021 --> 24176
0.022 --> 3729
0.023 --> 1688
0.024 --> 763
0.025 --> 196
0.026 --> 59
0.027 --> 61
0.028 --> 40
0.029 --> 68
0.030 --> 52
0.031 --> 61
0.032 --> 56
0.033 --> 57
0.034 --> 50
0.035 --> 45
0.036 --> 72
0.037 --> 61
0.038 --> 56
0.039 --> 52
0.040 --> 38
0.041 --> 30
0.042 --> 36
0.043 --> 22
0.044 --> 16
0.045 --> 16
0.046 --> 10
0.047 --> 9
0.048 --> 5
0.049 --> 5
0.050 --> 4
0.051 --> 2
0.052 --> 4
0.053 --> 1
0.054 --> 3
0.055 --> 7
0.057 --> 1
0.058 --> 9
0.059 --> 20
0.060 --> 10
0.062 --> 1
0.065 --> 1
0.074 --> 1
0.080 --> 1
0.086 --> 1
0.089 --> 1
0.103 --> 1
0.107 --> 1
0.110 --> 1
0.120 --> 1
0.198 --> 1
5.035 --> 1
5.048 --> 1

2nd result:

total time is 25275 seconds, operation count is 4040201, average is 0.0062 seconds, minimum time for each operation is 0.003 seconds, maximum time for each operation is 0.204 seconds

(time in seconds) --> number of operations which use this much time in seconds
0.003 --> 720
0.004 --> 106371
0.005 --> 564130
0.006 --> 1602206
0.007 --> 1737708
0.008 --> 24786
0.009 --> 1993
0.010 --> 1644
0.011 --> 376
0.012 --> 28
0.013 --> 3
0.015 --> 4
0.016 --> 3
0.017 --> 2
0.018 --> 7
0.019 --> 4
0.020 --> 2
0.022 --> 1
0.024 --> 1
0.025 --> 1
0.027 --> 2
0.028 --> 2
0.029 --> 1
0.032 --> 1
0.033 --> 3
0.034 --> 2
0.035 --> 1
0.036 --> 1
0.037 --> 1
0.038 --> 1
0.039 --> 4
0.041 --> 2
0.042 --> 2
0.043 --> 6
0.044 --> 4
0.045 --> 3
0.046 --> 3
0.047 --> 2
0.048 --> 7
0.049 --> 5
0.050 --> 3
0.051 --> 10
0.052 --> 7
0.053 --> 8
0.054 --> 9
0.055 --> 5
0.056 --> 4
0.057 --> 6
0.058 --> 2
0.059 --> 6
0.060 --> 6
0.061 --> 10
0.062 --> 7
0.063 --> 8
0.064 --> 6
0.065 --> 9
0.066 --> 8
0.067 --> 5
0.068 --> 8
0.069 --> 6
0.070 --> 3
0.073 --> 1
0.074 --> 1
0.075 --> 1
0.076 --> 1
0.079 --> 1
0.083 --> 1
0.084 --> 1
0.086 --> 1
0.092 --> 1
0.093 --> 1
0.094 --> 1
0.099 --> 1
0.110 --> 1
0.114 --> 1
0.120 --> 1
0.127 --> 1
0.131 --> 1
0.132 --> 2
0.137 --> 1
0.141 --> 1
0.204 --> 1

-- WenjingWu - 2011-08-25

Edit | Attach | Watch | Print version | History: r2 < r1 | Backlinks | Raw View | Raw edit | More topic actions
Topic revision: r2 - 2011-08-26 - WenjingWu
 
This site is powered by the TWiki collaboration platform Powered by PerlCopyright © 2008-2019 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki? Send feedback