beansdb

Archived, see GoBeansDB instead.

  • 所有者: douban/beansdb
  • 平台:
  • 许可证: Other
  • 分类:
  • 主题:
  • 喜欢:
    0
      比较:

Github星跟踪图

What is Beansdb?

Beansdb is a distributed key-value storage system designed for large scale
online system, aiming for high avaliablility and easy management. It took
the ideas from Amazon's Dynamo, then made some simplify to Keep It Simple
Stupid (KISS).

The clients write to N Beansdb node, then read from R of them (solving
conflict). Data in different nodes is synced through hash tree, in cronjob.

It conforms to memcache protocol (not fully supported, see below), so any
memcached client can interactive with it without any modification.

Beansdb is heavy used in http://www.douban.com/, is used to stored images,
mp3, text fields and so on, see benchmark below.

Any suggestion or feedback is welcomed.

Features

  • High availability data storage with multi readable and writable repications

  • Soft state and final consistency, synced with hash tree

  • Easy Scaling out without interrupting online service

  • High performance read/write for a key-value based object

  • Configurable availability/consistency by N,W,R

  • Memcache protocol compatibility

Supported memcache commands

  • get
  • set(with version support)
  • append
  • incr
  • delete
  • stats
  • flush_all

Private commands

  • get @xxx, list the content of hash tree, such as @0f
  • get ?xxx, get the meta data of key.

Python Example

from dbclient import Beansdb

# three beansdb nodes on localhost
BEANSDBCFG = {
    "localhost:7901": range(16),
    "localhost:7902": range(16),
    "localhost:7903": range(16),
}

db = Beansdb(BEANSDBCFG, 16)

db.set('hello', 'world')
db.get('hello')
db.delete('hello')

Benchmark

 $ beansdb -d 
 $ memstorm -s localhost:7900 -n 1000000 -k 10 -l 100 
   
  ---- 
  Num of Records : 1000000 
  Non-Blocking IO : 0 
  TCP No-Delay : 0 
   
  Successful [SET] : 1000000 
  Failed [SET] : 0 
  Total Time [SET] : 51.77594s 
  Average Time [SET] : 0.00005s 
   
  Successful [GET] : 1000000 
  Failed [GET] : 0 
  Total Time [GET] : 40.93667s 
  Average Time [GET] : 0.00004s 

Real performance in production

  • cluster 1: 1.1B records, 55TB data, 48 nodes, 1100 get/25 set per seconds,
    med/avg/90%/99% time is 12/20/37/186 ms.
  • cluster 2: 3.3B records, 3.5TB data, 15 nodes, 1000 get/500 set per seconds,
    med/avg/90%/99% time is 1/11/15/123 ms.

主要指标

概览
名称与所有者douban/beansdb
主编程语言C
编程语言Shell (语言数: 5)
平台
许可证Other
所有者活动
创建于2012-02-23 03:24:10
推送于2015-12-12 17:14:16
最后一次提交2015-06-19 21:51:10
发布数7
最新版本名称v0.7.1.4 (发布于 )
第一版名称v0.5.6 (发布于 )
用户参与
星数865
关注者数126
派生数236
提交数314
已启用问题?
问题数23
打开的问题数9
拉请求数5
打开的拉请求数0
关闭的拉请求数3
项目设置
已启用Wiki?
已存档?
是复刻?
已锁定?
是镜像?
是私有?