idea's blog

Redis is great! It is super fast, easy to use, the data structures(kv, hash, zset) make it powerful to store business data.

But Redis has some problems, the most significant problem is memory size. Redis a single-instance storage, which means it is not distributed, so it could not store big data, and the data it stores must not exceed the memory size of a server for it holds all data in meory.

WHat can we do if data exceed memory size?

First, manually sharding. You will have to shard data into multi instances across machines, all by your own. Redis does not provide official solution yet.

Second, try some distributed storage system. But we still want to be able to work will data structures, not stupid pure key-value!

Third, use SSDB instead! SSDB is an alternative to Redis, it supports Redis network protocol and nearly all redis clients(redis-cli, hiredis, phpredis, etc). And the storage capacity of SSDB is usually 100 times of Redis’, for SSDB stores data in hard-disk, and holds hot data in memory.

SSDB is widely used by many internet companies, including QIHU 360, Baidu.com, TOPGAME, Autohome.com, BYD auto, and many other[see here].

If you get rails route error like this:

No route matches [GET] "/sign_out"

That’s because Devise uses HTTP DELETE to logout the session, but you will never be able to do this by clicking a link in a web page.

Then, just config your config/initializers/devise.rb to use

# The default HTTP method used to sign out a resource. Default is :delete.
config.sign_out_via = :get

Restart rails server.

The c1000k comet/push server icomet has just fixed a bug of memory leak. This bug is caused by not pairing every call to evhttp_send_reply_start() with evhttp_send_reply_end().

The icomet project use libevent’s HTTP server API to easily set up a comet server, when a client connected, it calls:

evhttp_send_reply_start(req, HTTP_OK, "OK");

To send chunked data. And after a few seconds(by configuration) the long-polling times out, it ends up the HTTP request by calling:

evhttp_send_reply_end(req);

But when the client interupts the request, the callback function registered by evhttp_connection_set_closecb() is called, and evhttp_send_reply_end() is not called, so libevent will never free that request, memory leaks!

I met a weird thing recently with my 13-inch Mac Book Pro. I have a image file, every time I double-click it to open from within Finder, I get a new copy of this file. I got a lot of copies of this file after days, I try to rename it, open it with different Apps, but it keeps generating copies.

Finally, I view the info of this file(Get Info menu), then I notice the “Stationery pad” checkbox is checked, that’s it! I unchecked the option, the copy of this file won’t generate any more!

SSDB users had been asking for Redis protocol supports for a long time, because they want to use existing Redis clients, and do not want do any modification on their codes.

SSDB now supports Redis protocol – finally.

That means you can use existing Redis clients to connect to a SSDB server. Only redis-cli and phpredis is tested by now.

Not all Redis commands are supported, see http://www.ideawu.com/ssdb/docs/redis-to-ssdb.html.

During the development, I have found Redis network protocol unneccessarily complicated, because it concerns about data types. The Redis network protocol deals with request and response, while SSDB’s network protocol treats requests and responses the same. Redis network protocol impmentation has to deal with many types of response: status, error, integer, list, … As a contrast, data types are transparent to SSDB protocol implementors.