Projects/LMDB KDB

This is an early stage project for MIT Kerberos. It is being fleshed out by its proponents. Feel free to help flesh out the details of this project. After the project is ready, it will be presented for review and approval.

Background

LMDB (Lightning Memory-mapped Database) is emerging as the preferred successor to Berkeley DB for embedded key-value databases. A KDB module using LMDB could provide better performance and crash recovery than DB2, and could in the long term remove the need to maintain an old version of the Berkeley DB source code in our tree.

Limitations

Because LMDB uses memory mapping on the database, an LMDB KDB will be limited to 4GB on 32-bit platforms. This concern will become less important as the prevalence of 32-bit platforms decreases.

Initial design notes

Because LMDB presents a similar API to that of DB2, we can start with the DB2 module as a basis. However:

• Principals and policies should be stored in the same database file using key prefixes, not in two separate files as they are in the DB2 module.

• Any code relating to locking should be carefully examined and simplified as much as possible, as LMDB allows concurrent access to the database.

• Special attention must be given to the code for creating a temporary side DB and promoting it (used by kdb5_util load).

• The marshalling code in kdb_xdr.c and pol_xdr.c should be factored out into libkdb5, not duplicated.

In the future, it may be desirable to make the LMDB KDB module the default for newly created databases, while continuing to use the DB2 module for existing databases. This cutover may require changes to the libkdb5 logic for selecting the database module in the absence of specific configuration.