Redis is a key-value database. It is similar to memcached but the dataset is not volatile, and keys can be strings, exactly like in memcached, but also lists and sets with atomic operations to push/pop elements. In order to be very fast but at the same time persistent the whole dataset is taken in memory and from time to time and/or when a number of changes to the dataset are performed it is written asynchronously on disk. You may lost the last few queries that is acceptable in many applications but it is as fast as an in memory DB (btw the SVN version of Redis includes support for replication in order to solve this problem by redundancy). Replication and other interesting features are a work in progress (Basic master <-> slave replication implemented in Redis SVN). Redis is written in ANSI C Redis is pretty fast!, 110000 SETs/second, 81000 GETs/second in an entry level Linux box.
James Hamilton has published a thorough summary of Facebook's Cassandra, another scalable key-value store for your perusal. It's open source and is described as a "BigTable data model running on a Dynamo-like infrastructure." Cassandra is used in Facebook as an email search system containing 25TB and over 100m mailboxes. # Google Code for Cassandra - A Structured Storage System on a P2P Network # SIGMOD 2008 Presentation. # Video Presentation at Facebook # Facebook Engineering Blog for Cassandra # Anti-RDBMS: A list of distributed key-value stores # Facebook Cassandra Architecture and Design by James Hamilton
P. Thiran, J. Hainaut, S. Bodart, A. Deflorenne, and J. Hick. Proc. of the 3rd Int. Conf. on Cooperative Information Systems, CoopIS'98, August 20--22, 1998, New York City, USA, page 54--63. IEEE Computer Society Press, (1998)
W. Hasan, and R. Motwani. VLDB '95: Proceedings of the 21th International Conference on Very Large Data Bases, page 239--250. San Francisco, CA, USA, Morgan Kaufmann Publishers Inc., (1995)
H. Mattsson, H. Nilsson, and C. Wikstrom. PADL '99: Proceedings of the First International Workshop on Practical Aspects of Declarative Languages, page 152--163. London, UK, Springer-Verlag, (1998)
J. Li, H. Gao, J. Luo, S. Shi, and W. Zhang. SIGMOD '07: Proceedings of the 2007 ACM SIGMOD international conference on Management of data, page 899--909. New York, NY, USA, ACM, (2007)
W. Han, J. Ng, V. Markl, H. Kache, and M. Kandil. SIGMOD '07: Proceedings of the 2007 ACM SIGMOD international conference on Management of data, page 809--820. New York, NY, USA, ACM, (2007)
R. Fang, B. He, M. Lu, K. Yang, N. Govindaraju, Q. Luo, and P. Sander. SIGMOD '07: Proceedings of the 2007 ACM SIGMOD international conference on Management of data, page 1061--1063. New York, NY, USA, ACM, (2007)
T. Cruanes, B. Dageville, and B. Ghosh. SIGMOD '04: Proceedings of the 2004 ACM SIGMOD international conference on Management of data, page 850--854. New York, NY, USA, ACM, (2004)
G. Cormode, and M. Garofalakis. SIGMOD '07: Proceedings of the 2007 ACM SIGMOD international conference on Management of data, page 1178--1181. New York, NY, USA, ACM, (2007)