Lumosql

Documentation
Login

Documentation

Node.js better-sqlite3 backed by LumoSQL

better-sqlite3 is the most popular SQLite binding for Node.js. This directory builds better-sqlite3 against LumoSQL with the LMDBv1 backend. Besides being faster for most operations, LMDBv1 behind the SQLite interface allows page-based encryption, incremental backup and checksums without further development because these are core features of LMDBv1.

The Makefile replaces only the vendored deps/sqlite3/sqlite3.c and sqlite3.h, copies the LumoSQL helper files into deps/lumo/, and extends deps/sqlite3.gyp.

The LumoSQL backend can also be built with traditional LMDB (in deep freeze maintenance as of July 2026, and occasionally still a little faster than v1.0 on some operations). To do this use something like BACKEND_SPEC=lmdb-0.9.35 LUMOSQL_TARGET=3.53.3+lmdb-0.9.35. We mostly just use LMDBv1.

Dependencies

Install the normal LumoSQL build dependencies from the top-level LumoSQL README and confirm them with:

make doctor

This example uses git as the source code management system. LumoSQL, being very close to SQLite, uses Fossil as the primary SCM, but this example uses git mirrors and does not need Fossil at all.

git clone https://codeberg.org/lumosql/lumosql     # same result as fossil clone https://lumosql.org/src/lumosql

The not-forking tool is required, and is not yet packaged by any distribution. LumoSQL uses it to assemble the SQLite and backend sources. not-forking needs perl plus the Text::Glob module (libtext-glob-perl on Debian/Ubuntu, perl-Text-Glob on Fedora/RHEL). Then:

git clone https://codeberg.org/not-forking/not-forking   # or: fossil clone https://lumosql.org/src/not-forking
cd not-forking
perl Makefile.PL && make && sudo make install

In addition, this example needs Node.js 20 or newer, npm, git, python3, make, and a C++ compiler.

Debian/Ubuntu:

sudo apt install git nodejs npm python3 make g++

Fedora/RHEL:

sudo dnf install git nodejs npm python3 make gcc-c++

Quick build

From the LumoSQL root directory:

make doctor
cd examples/node-better-sqlite3
make

The addon is:

examples/node-better-sqlite3/build/better-sqlite3/build/Release/better_sqlite3.node

Confirm LumoSQL is linked statically:

ldd build/better-sqlite3/build/Release/better_sqlite3.node | grep -i sqlite   # expect: no output

By default the Makefile builds better-sqlite3 v12.11.2, detects the SQLite version it bundles, and builds the matching +lmdbv1-1.0 LumoSQL target. Override the defaults as follows:

make BSQL_REF=master
make LUMOSQL_TARGET=3.53.3+lmdbv1-1.0
make BSQL_GIT_URL=https://github.com/WiseLibs/better-sqlite3 WORK_DIR=/tmp/lumo-bsql

If better-sqlite3 master bundles a SQLite version newer than LumoSQL has (pretty unlikely), the Makefile falls back to LumoSQL's newest SQLite.

Smoke test

make smoke

The smoke test creates a throwaway database, exercises prepared statements, transactions, .get(), .all(), .iterate(), a user-defined function, and FTS5. It also prints the result of PRAGMA journal_mode=WAL, because WAL is a SQLite pager feature and is a no-op under the LMDB backend. The test verifies that the database file has the LMDB magic bytes and a <db>-lock sibling, and no -wal or -shm files.

To try the built module from another project:

npm install --ignore-scripts /path/to/lumosql/examples/node-better-sqlite3/build/better-sqlite3

--ignore-scripts is required. Without it npm runs better-sqlite3's own install script, which fetches an upstream prebuilt addon compiled against stock SQLite and puts it in place of the LumoSQL one. npm also links a local path rather than copying it, so the consuming project uses the example build tree directly and make clean will remove it.

To force a full rebuild:

make clean && make

Testing, and Unimplemented features and deviations

Run the upstream better-sqlite3 test suite through the example Makefile:

make test

This runs each upstream better-sqlite3/test/*.js file separately with test/00.setup.js loaded first, so one expected LMDB/file-format failure does not hide later results.

npm run benchmark is not a stock-vs-LumoSQL comparison for this build. The upstream benchmark seeds one database through better-sqlite3; here that file is LMDB-format, so the comparison drivers that expect SQLite file format (node-sqlite3 and node:sqlite) fail with SQLITE_NOTADB. You should read only the better-sqlite3 result rows from that benchmark, or, use LumoSQL's own benchmark suite for native-vs-LMDB comparisons. Features tied to SQLite's pager or file format are absent or no-op under the LMDB backend including backup(), serialize(), deserialize(), checkpointing, WAL pragmas, and tests that fabricate or inspect SQLite file-format bytes. Test suite entries for these features (including ones that expect particular bytes to be present in a data file) will fail. Look at the directory better-sqlite3/test/*js to see what upstream tests do. The result should be that all relevant tests pass identically with both LMDB backends.

Note: better-sqlite3 builds SQLite with SQLITE_THREADSAFE=2, and this LumoSQL example keeps that upstream setting. This varies from LumoSQL's own tests which assume SQLITE_THREADSAFE=1.

LMDB databases always have a <db>-lock file, analogous to SQLite<db>-wal and <db>-shm files that are often generated when native SQLite runs in WAL mode.

The backend pragmas are available from JavaScript through db.pragma(). PRAGMA lumo_sync sets the durability level (full, normal, off) and defaults to normal, and PRAGMA lmdb_transaction sets the transaction type.

Data migration

An LMDB-backed LumoSQL database is not an SQLite file-format database, with conversion left to the user. As an example, you can copy rows from an existing SQLite database into a new LMDB-backed one like this:

const Stock = require('/path/to/stock/node_modules/better-sqlite3');
const Lumo = require('/path/to/lumosql/examples/node-better-sqlite3/build/better-sqlite3');
const src = new Stock('old.sqlite');
const dst = new Lumo('new.lmdb');
dst.exec('CREATE TABLE t(id INTEGER PRIMARY KEY, body TEXT)');
const insert = dst.prepare('INSERT INTO t VALUES (?, ?)');
dst.transaction(() => { for (const r of src.prepare('SELECT id, body FROM t').iterate()) insert.run(r.id, r.body); })();