- 浏览: 986977 次
- 性别:
- 来自: 杭州
文章分类
- 全部博客 (826)
- 硬件 (8)
- 软件 (24)
- 软件工程 (34)
- JAVA (229)
- C/C++/C# (77)
- JavaScript (8)
- PHP (1)
- Ruby (3)
- MySQL (14)
- 数据库 (19)
- 心情记事 (12)
- 团队管理 (19)
- Hadoop (1)
- spring (22)
- mybatis(ibatis) (7)
- tomcat (16)
- velocity (0)
- 系统架构 (6)
- JMX (8)
- proxool (1)
- 开发工具 (16)
- python (10)
- JVM (27)
- servlet (5)
- JMS (26)
- ant (2)
- 设计模式 (5)
- 智力题 (2)
- 面试题收集 (1)
- 孙子兵法 (16)
- 测试 (1)
- 数据结构 (7)
- 算法 (22)
- Android (11)
- 汽车驾驶 (1)
- lucene (1)
- memcache (12)
- 技术架构 (7)
- OTP-Erlang (7)
- memcached (17)
- redis (20)
- 浏览器插件 (3)
- sqlite (3)
- Heritrix (9)
- Java线程 (1)
- scala (0)
- Mina (6)
- 汇编 (2)
- Netty (15)
- libevent (0)
- CentOS (12)
- mongod (5)
- mac os (0)
最新评论
-
kingasdfg:
你这里面存在一个错误添加多个任务 应该是这样的 /** * ...
Quartz的任务的临时启动和暂停和恢复【转】 -
kyzeng:
纠正一个错误,long型对应的符号是J,不是L。
Jni中C++和Java的参数传递 -
zhaohaolin:
抱歉,兄弟,只是留下作记录,方便学习,如果觉得资料不好,可以到 ...
netty的个人使用心得【转】 -
cccoooccooco:
谢谢!自己一直以为虚机得使用网线才可以与主机连接呢。。
主机网卡无网线连接与虚拟机通信 -
yuqilin001:
要转别人的东西,请转清楚点嘛,少了这么多类,误人子弟
netty的个人使用心得【转】
1,250,000,000 Key/Value Pairs in Redis 2.0.0-rc3 on a 32GB Machine【转】
- 博客分类:
- redis
Following up on yesterday’s 200,000,000 Keys in Redis 2.0.0-rc3 post, which was a worst-case test scenario to see what the overhead for top-level keys in Redis is, I decided to push the boundaries in a different way. I wanted to use the new Hash data type to see if I could store over 1 billion values on a single 32GB box. To do that, I modified my previous script to create 25,000,000 top-level hashes, each of which had 50 key/value pairs in it.
The code for redisStressHash was this:
#!/usr/bin/perl -w $|++; use strict; use lib 'perl-Redis/lib'; use Redis; my $r = Redis->new(server => 'localhost:63790') or die "$!"; ## 2.5B values for my $key (1..25_000_000) { my @vals; for my $k (1..50) { my $v = int(rand($key)); push @vals, $k, $v; } $r->hmset("$key", @vals) or die "$!"; } exit; __END__
Note that I added a use lib in there to use a modified Redis Perl library that speaks the multi-bulk protocol used all over in the Redis 2.0 series.
If you do the math, that yields 1.25 billion (1,250,000,000) key/value pairs stored. This time I remembered to time the execution as well:
real 160m17.479s user 58m55.577s sys 5m53.178s
So it took about 2 hours and 40 minutes to complete. The resulting dump file (.rdb file) was 13GB in size (compared to the previous 1.8GB) and the memory usage was roughly 17GB.
Here’s the INFO output again on the master:
redis_version:1.3.16 redis_git_sha1:00000000 redis_git_dirty:0 arch_bits:64 multiplexing_api:epoll process_id:21426 uptime_in_seconds:12807 uptime_in_days:0 connected_clients:1 connected_slaves:1 blocked_clients:0 used_memory:18345759448 used_memory_human:17.09G changes_since_last_save:774247 bgsave_in_progress:1 last_save_time:1280092860 bgrewriteaof_in_progress:0 total_connections_received:22 total_commands_processed:32937310 expired_keys:0 hash_max_zipmap_entries:64 hash_max_zipmap_value:512 pubsub_channels:0 pubsub_patterns:0 vm_enabled:0 role:master db0:keys=25000000,expires=0
Not bad, really. This provides a slightly more reasonable usse case of storing many values in Redis. In most applications, I supsect people will have a number of “complex” values stored behind their top-level keys (unlike my previous simple test).
I’m kind of tempted to re-run this test using LISTS, then SETS, then SORTED SETS just to see how they all compare from a storage point of view.
In any case, a 10 machine cluster could handle 12 billion key/value pairs this way. Food for thought.
- Share this:
- Share
specializes in the manufacture of blank keys, color key. OEM and ODM
www.golden-key.com.tw
JDM Logo Shaped Key Blanks K's Logo, Silvia Logo, 5 Colors
www.Rotary13B1.com/Silvia_Key_Blank
发表评论
-
Redis入门教程
2011-07-12 22:08 2116入门教程,仅供菜鸟。 ... -
Redis数据库介绍
2011-07-12 22:07 27811. Redis是什么 这个问题的结果影响了我们怎么用R ... -
redis-sharding – 一个 Redis Proxy【转】
2011-04-28 01:43 1618redis-sharding 是一个由perl写的 Redis ... -
redis 运维实际经验纪录之一【转】
2011-04-26 11:30 1149发表于 2010/05/25 由 ... -
200,000,000 Keys in Redis 2.0.0-rc3【转】
2011-04-26 11:27 951Posted on July 24, 20 ... -
Redis配置文件各项参数说明【转】
2011-04-26 11:00 1588最近一个项目用到Redis ... -
Redis配置主从服务器【转】
2011-04-26 10:31 1249redis配置主从服务器 R ... -
More Redis internals: Tracing a GET & SET【转】
2011-04-26 10:09 1837In my previous article , I too ... -
深入Redis内部-Redis 源码讲解【转】
2011-04-26 10:08 573Redis 作为 NoSQL 数据库的杰出代表,一直广受关注, ... -
redis安装配置及使用【转】
2011-04-26 09:03 15402011-03-27 20:41 http://cod ... -
Redis安装配置【转】
2011-04-26 09:02 2011http://bbs.linuxtone.org/thread ... -
闲聊Redis【转】
2011-04-26 09:00 1128Redis 是一个有趣 的项目,与其把它说成键值存储、键 ... -
redis知识点整理【转】
2011-04-25 22:40 1018xmind做的笔记,可惜只能截图 关键字: ... -
jedis线程池的代码【转】
2011-04-25 22:36 2771备忘:(本机i5的测试数据:redis线程执行时间是:2118 ... -
在多台服务器上简单实现Redis的数据主从复制【转】
2011-04-25 22:27 1186Redis的主从复制功能非常强大,一个mas ter可以拥有 ... -
归档Redis在Windows,linux平台下的安装配置【转】
2011-04-25 22:25 2183为了方便查阅资料,特将网上搜索到的Redis相关安装配置进行归 ... -
Redis在Windows下的使用[转]
2011-04-25 22:24 1125Windows版的Redis可到此处下载,非官方版http:/ ... -
windows下redis和memcached的性能对比测试【转】
2011-04-25 22:19 1661在windows下测试了redis和memcached的性能, ... -
膜拜Redis【转】
2011-04-25 21:14 961膜拜Redis 2009年06月18 ...
相关推荐
191 |[Number of 1 Bits](https://leetcode.com/problems/number-of-1-bits/) | [C++](./C++/number-of-1-bits.cpp) [Python](./Python/number-of-1-bits.py) | _O(1)_ | _O(1)_ | Easy ||| 201 | [Bitwise AND of ...
This dataset contains 20 pairs of color multi-focus images of size 520?20 pixels and four series of multi-focus images with three sources. Please cite the following paper if you use this dataset: M...
使用Prometheus和Grafana监视Spring Data Redis项目运行Docker Compose文件该docker compose... Put 1 million random key,value pairs to Redis without any TTL. 2. Put a given key,value pair to Redis with given
./persistent-hdfs/bin/hadoop fs -rm /vol/all-pairs-shortest-path_2.10-1.0.jar ./persistent-hdfs/bin/hadoop fs -put all-pairs-shortest-path_2.10- 1.0.jar hdfs://ec2-54-146-149-83.compute-1.amazonaws....
For the past several years, I’ve been teaching a class on “Topics in Information Security.” Each time I taught this course, I’d sneak in a few more machine learning topics. For the past couple of ...
auto-pairs是vim的一个极小的插件。它对于用vim来编写程序,提供了极大的便。我ubuntu16中,用vim编写c和c++程序,安装了此插件,极大地方便了代码的输入。 二、功能介绍: 它可以自动完成大括号、小括号、中括号、...
前端开源库-ripple-keypairs波纹键对,波纹键对
https://github.com/HouJP/kaggle-quora-question-pairs 这里对应的PPT
[script] bytes1 == bytes2, comparison of two byte vectors works in the same way as string1 == string2 – content equality. [behavior:video] support of video.style.foregroundImage to access current ...
Synopse Big Table is an open source Delphi unit for very fast data storage and access, using key/values pairs, or records organized with fields. With this 1.12a version, the unit has evolved into a ...
This means that you do a binary search in the page list in log M time and get the value in O(1) time within a page. RaptorDB starts off by loading the page list and it is good to go from there and...
A multistage graph is a graph (1) G=(V,E) with V partitioned into K >= 2 disjoint subsets such that if (a,b) is in E, then a is in Vi , and b is in Vi+1 for some subsets in the partition; and (2) | ...
【1】用 LUA解释器 实现魔兽世界 数学表达式计算插件 支持通用运算符和优先级 WoW Lua Math 库函数 以及 WoW Lua 环境变量 测试用例: /calc random(1,16) / 0x10 /calc log10(10) - log(exp(1)) /calc PI - rad...
https://github.com/HouJP/kaggle-quora-question-pairs 这里对应的视频
matlab导入excel代码utl_all_possible_pairs_of_a_list 列表中所有可能的对。 关键字:sas sql join合并大数据分析宏oracle teradata mysql sas社区stackoverflow statistics人工智慧AI Python R Java Javascript ...
A search on "suffix trees Ukkonen" on a search engine should give you an idea of what I'm talking about. The interesting points are a linear indexing time, very little memory consumption (except for ...
Lua的for in和pairs1
In-shop Clothes Retrieval Benchmark evaluates the ...52,712 number of in-shop clothes images, and ~200,000 cross-pose/scale pairs; Each image is annotated by bounding box, clothing type and pose type.
(Unity源码,零积分)非常好点记忆力游戏(找彩蛋)Find The Pairs V.1.0.2 - Admob.zip
不会LeetCode_532--K-diff-Pairs-in-an-Array 给定一个整数数组和一个整数 k,您需要找到数组中唯一 k-diff 对的数量。 这里 k-diff 对定义为整数对 (i, j),其中 i 和 j 都是数组中的数字,它们的绝对差为 k。 示例...