Download 1.5 here.

For the end-user, the only visible change is the removal of the submit parameter from the commands (such as keyspace_client_set()), since this can be derived from the use of keyspace_client_begin() before the commands are issued in batch.

More good news: we have starting wrapping the C/C++ client library, which itself has reached a very stable state, using SWIG, the Simplified Wrapper and Interface Generator. We are planning to release fully featured Python, PHP, Java libraries next week. In parallel, we are finally going to write up proper manuals starting today.

Download 1.4.1 here.

Here are some performance numbers from a n=3 node Keyspace setup on Windows over a 100Mbit LAN:

VALUE LENGTH = 100 bytes long
batch set: 30487 tx/sec
single set: 1777 tx/sec
batch get: 45662 tx/sec
single get: 6734 tx/sec

VALUE LENGTH = 1000 bytes long
batch set: 2443 tx/sec
single set: 845 tx/sec
batch get: 8888 tx/sec
single get: 3137 tx/sec

Notable changes in the server:

• Improved stability on Windows
• PaxosLease now uses TCP instead of UDP, in an effort to make master leases last longer
• Servers now explicitly store 100.000 Paxos rounds in their local DB to help lagging nodes, in effect eliminating the need for our previous truncation hack, at the expense of disk space
• More info printed in HTTP hello message
• 7 seconds sleep() on startup uses a timer instead of a blocking sleep, making the server responsive during the time
• Max. value size decreased to 100K
• Configuration files have not changed
• Much more...

Notable changes in the C/C++ client library:

• We rewrote much of the client library
• It now handles failovers in the nodes, without the application having to call the library again
• Application can specify global and master timeouts:
• Global timeout: the client library will return no matter what within this time
• Master timeout: the client library will return if the master is lost and no new master is found within this time
• Failure conditions can be queried from the client library: transport status, connectivity status, timeout status and command status
• Much much more...

We are planning to write up proper documentation on the new client library next week, stay tuned!

A number of people have requested we go on github - thanks for the suggestions - we're discussing it with Attila.

1. now domain names as well as IP addresses can be specified in paxos.endpoints
2. the client config erronously referred to paxos.endpoints, which was in reality keyspace.endpoints (port numbers of keyspace protocol)
3. Linux build was broken

Download 1.3.1 here.

Click here to download Keyspace 1.3.

For this port we had to port our (readyness notification) event based, networking subsystem to Windows' Completion Ports API, which was more challenging than we expected as the intended use for the API is asynchronous I/O, which is slightly different than our model. After lots of coding and one all-nighter we prevailed! I'll share more details in a later post.

The next big step is getting rid of BerkeleyDB. Coding is already on its way and I'll post some details about our storage engine architecture soon (currently codenamed Blocksmith).

Back to work.

Click here to download the Keyspace Windows beta.

This is a non-versioned beta release, which will be followed by a regular 1.3 release next week after we have done some more testing on Amazon.

The release includes Visual Studio C++ 2008 Express Edition project files so Windows developers can get straight to work. Also included are the necessary BerkeleyDB include and lib files ('include' and 'lib' folders, respectively). The 'bin' folder contains statically linked binaries in case somebody wants to test it out without compiling. If it doesn't run, you may have to install the appropriate Visual Studio redistributables, which I put in the 'redist' folder for your convenience.

Technically, the Windows port uses the Completion Ports API to achieve high-performance network I/O. Thanks to our layered architecture (for platform-independence) we didn't have to modify application logic at all. Based on preliminary testing the performance numbers should be equivalent to the ones in our whitepapers.

From now on, the Keyspace server and all C/C++ code including the C/C++ libraries are licensed under the Affero GPL, as they were originally. The Python and PHP libraries are licensed under the BSD, allowing you to use them freely with your application as you wish. The 1.2.0 version, which was licensed under the BSD license, has been removed from the site to avoid confusion and protect the Keyspace source code.

1. Added Python and PHP client libraries; this means we have C, Python and PHP covered.
2. Added HTTP JSON API; For example, http://localhost:8080/json/listkeys?/test returns the following JSON object: {"/test":["/test/i:0","/test/i:1","/test/i:2"]}.
3. Fixed some bugs in server code.
4. Fixed some bugs in Python code.
5. Removed deprecated latency command.

Download 1.2 here.

1. New PaxosLease algorithm does not require clock synchronization.
2. Added COUNT and DIRTYCOUNT commands.
3. Added /html/ part to HTTP interface, which pretty prints the list output.
4. Added possibility to reverse list direction.
5. The http://server:port/ welcome page prints the version number and master information.
6. Added database.numReaders config option.

Download 1.1 here.

The most important is: Keyspace no longer requires clock synchrony! This should make Keyspace as a distributed database much more attractive to new users. The new PaxosLease algorithm is described in the PaxosLease whitepaper.

In other news, Hacker News Watch runs on Keyspace.

1. Keyspace would not start without a config file in default mode. Resolution: fixed.
2. Keyspace printed a non-informative error message when a directory 'keyspace' was present and it tried to create the database file 'keyspace'. Resolution: More informative error message.
3. Some Linux optimizations were accidentally removed in the 1.0 release. Resolution: added to 1.0.1

Download 1.0.1 here.

The relevant links are:

• Keyspace features at a glance
• Documentation
• Whitepapers
• Downloads
• License
• Help
• FAQ

In a few words, consistent replication means it's not eventually consistent, it's always consistent or always in-sync. When the server acknowledges an operation, then you can be certain that it has hit the disk on a majority of the nodes and you will never see a state not reflecting that write.

Since Keyspace is replicated, the database is highly-available and fault-tolerant. A minority of nodes can fail without the overall database service being affected. This is of course the main feature of any replicated data store: if the cleaning crew accidentally pulls the plug on one of your servers or a disk fails, then your system keeps going without service outage.

Keyspace is a high-performance, asynchronous, event-driven C/C++ UNIX server. Even with n=3 replication we can handle 100.000 writes a second on fast Linux PCs with SCSI disks. This is an order of magnitude faster than naive key-value store implementations in Java, Erlang or Python. See the whitepaper for more.

With Keyspace, you have a sensible upgrade path. You can use Keyspace in single-server mode, in which case it's reduced to a TCP wrapper around Berkeley DB. Later, you can reconfigure it to use replication. Keyspace has its own TCP protocol, which is used by the client libraries, and a built-in HTTP server and HTTP protocol so you can issue http://localhost/set?key,value type requests using your browser for testing. More about the API.

You can download source code and compile Keyspace or use the Debian package for quick installation and /etc/init.d/ sweetness. Included are the usual help files, client libraries and some sample code. Keyspace is distributed under the Affero GPL license (AGPL).

What's missing in this release? Currently, our C/C++ client library is production quality. We are finishing up PHP, Python, Java and .NET libraries, the exact order is a function of user requests.

This is a beta release, where one of the goals is to gauge user interest. If you find Keyspace interesting or useful, please drop us a line.

Scalien was formalized as an LLC in Janurary of 2009 by Marton Trencseni and Attila Gazso. Our goal is to create a distributed stack of open-source components, enabling developers to create scalable, highly-available applications able to meet their customer's expectations. Our products fit in with the 'cloud computing' paradigm, although we're not looking to become cloud providers. More precisely, sites looking to become cloud providers in the future will be looking for basic distributed software such as databases and filesystems to run their own infrastructure on.

Over the last 6 months, we've been busy with the basic building block of our distributed stack, a replicated, highly-available key-value store called Keyspace. Keyspace has similarities to Google's Chubby server, the most important being that both use Leslie Lamport's Paxos distributed consensus algorithm as the basic primitive. The first public beta of Keyspace is available for download under the Affero GPL license (AGPL). To the best of our knowledge, Keyspace is the only consistently replicated (Paxos-based) high-performance key-value store on the market.

Once the dust settles, we will begin to implement the second layer of our stack, which will add the essential property of scalability (in Keyspace, all nodes store the same data). This product will most likely be called Keyscale, and will also be licensed under the AGPL. We hope to have an early version of Keyscale finished at the end of 2009, or early 2010. Then, we'll get back to our original idea, and use these technologies to create a useful, general-purpose distributed filesystem.

Back to work.