Redis is a key-value database. It is similar to memcached but the dataset is not volatile, and keys can be strings, exactly like in memcached, but also lists and sets with atomic operations to push/pop elements. In order to be very fast but at the same time persistent the whole dataset is taken in memory and from time to time and/or when a number of changes to the dataset are performed it is written asynchronously on disk. You may lost the last few queries that is acceptable in many applications but it is as fast as an in memory DB (btw the SVN version of Redis includes support for replication in order to solve this problem by redundancy). Replication and other interesting features are a work in progress (Basic master <-> slave replication implemented in Redis SVN). Redis is written in ANSI C Redis is pretty fast!, 110000 SETs/second, 81000 GETs/second in an entry level Linux box.
James Hamilton has published a thorough summary of Facebook's Cassandra, another scalable key-value store for your perusal. It's open source and is described as a "BigTable data model running on a Dynamo-like infrastructure." Cassandra is used in Facebook as an email search system containing 25TB and over 100m mailboxes. # Google Code for Cassandra - A Structured Storage System on a P2P Network # SIGMOD 2008 Presentation. # Video Presentation at Facebook # Facebook Engineering Blog for Cassandra # Anti-RDBMS: A list of distributed key-value stores # Facebook Cassandra Architecture and Design by James Hamilton
see discussion too Today, I decided to Google the term "document oriented". Turns out it's not new, here's an article I found Towards truly document oriented Web services on the O'Reilly site. The article gives and example of a REST API that is similar to the one I will be exposing with CouchDb. Cool. "Document Oriented Development" I think this may be a poorly served yet hugely important area of application development. Particularly in storage and management. For document storage, you pretty much have two options in mainstream development, direct file system access and relational databases. Traditional file based systems are simple enough, this is how most PC applications have dealt with documents for a long time. MS Office is a prime example: all documents are files. And relational databases? There is nothing "relational" about documents. XML databases will simplify development only if your data is already XML. Lotus Notes got so much of this right over 15 years ago
Both for individual researchers as for research groups or projects, it is of major importance to organize the literature one has read. A well organized bibliography is a powerful instrument. It speeds up the search for publications one has already read and supports the user in structuring information. Aigaion provides a bibliography management software environment that supports a user in just this: Organizing and managing a complete bibliography, from small bibliographies to bibliographies for a complete research department.