Lumosql

Documentation
Login

Documentation

Detailed and more Technical details of Fossil-on-LMDB-backed-LumoSQL

This document is for things not handled by examples/fossil/Makefile as described in the LumoSQL Fossil README, such as encryption and incremental backups. It also explains step-by-step what the Makefile is doing, which is a necessary pre-requisite to understanding the encyption etc stages.

Fossil with LumoSQL under it needs to address these three things:

Correctness: Fossil and libfossil are source code management systems, so correctness is the first requirement. The Fossil and libfossil test suite validates that Fossil and Libfossil produce identical results to the equivalent non-LumoSQL unmodified binaries for a range of important functionality. We're not looking for database equivalent, just that the data results of, say, checking in a modified file are identical. Due to the way LMDB works by using the operating system's memory management system, Fossil+LumoSQL is also more crash-proof in many circumstances.

Speed: According to the Fossil speed test suite, Fossil backed by LumoSQL with LMDBv1 is usually faster than SQLite native on most measures, and about the same on all other measures. If you run the tests maybe you will get a different result, and we'd like to hear about it.

Features: With the LMDBv1.0 backend, LumoSQL can support encryption. The default Makefile build is not encrypted; the encryption section is manual. Other features of LMDBv1 probably don't mean much for Fossil users: page-based checksums, well, we have a Merkle DAG; incremental backups, well, that's what fossil update is really. The common non-encrypted setup is built by make in this directory. It fetches Fossil, builds or reuses the matching LumoSQL target, prepares the Fossil source tree, and runs Fossil's own build.

Running libfossil and Fossil on LumoSQL with LMDB backend

These instructions are for technical people familiar with building software on Linux/Unix. They explain how to build libfossil and Fossil against a LumoSQL sqlite3 amalgamation. Using LumoSQL, a Fossil repository is stored in an LMDB database which is a single regular file, accompanied by a sibling lock file named <repository>-lock. As far as Fossil is concerned nothing is different from using ordinary SQLite, because Fossil almost never needs to know about such details. The SQL language, Fossil protocol and everything else is unchanged.

Prerequisites

Everything in these instructions assumes you have a Linux/Unix development environment available to you with a C compiler and associated tools.

If the USER variable is unset, export USER=$(id -un) (needed in some really bare environment such as certain Incus container defaults.)

Install ssl dev headers according to your OS with sudo, which have various names on different operating systems:

apt update; apt install libssl-dev  # Debian, Ubuntu
dnf install openssl-devel   # Fedora
pacman -S openssl           # Arch Linux
pkg install openssl         # FreeBSD
pkgin install openssl       # NetBSD

Install fossil to bootstrap the process

Install fossil: apt install fossil or dnf install fossil depending on your operating system.

Are you behind a strict application firewall? If so, your packaged fossil version will not work, so test it:

fossil test-httpmsg https://fossil-scm.org/home/fossil

If it returns an error message such as "server says: 426 Upgrade Required" then you will need to upgrade it to be capable of HTTP 1.1 by these steps:

You need a Fossil that works in your environment. This is not the ultimate target goal of a LumoSQL-backed Fossil, just an ordinary fossil that will bootstrap you into a Fossil backed by the LumoSQL amalgamation.

Install all other system packages

# Debian/Ubuntu
apt install tcl tclx perl libtext-glob-perl zlib1g-dev libreadline-dev libncurses-dev libsodium-dev pkg-config

# Fedora/RedHat
dnf install tcl tclx perl perl-Text-Glob zlib-devel readline-devel ncurses-devel libsodium-devel pkgconf

Install Not-forking

cd $HOME
fossil clone https://lumosql.org/src/not-forking
cd not-forking
perl Makefile.PL PREFIX=$HOME/.local
make ; make install
export PATH=$HOME/.local/bin:$PATH
# not-fork's Perl modules install under $PREFIX but aren't on Perl's @INC.
# Find where they landed (the NotFork/ dir) and put its parent on PERL5LIB.
NF=$(find "$HOME/.local" -type d -name NotFork 2>/dev/null | head -1)
export PERL5LIB="$(dirname "$NF")${PERL5LIB:+:$PERL5LIB}"

Build LumoSQL

cd $HOME
fossil clone https://lumosql.org/src/lumosql
cd lumosql
not-fork --check-prereq     # should return 'Ok' for all. If not, install the things needed for LumoSQL recipes.
make doctor                 # checks all LumoSQL build dependencies. If any are missing, install them.
# build.tcl requires Tcl X. If make stops with "can't find package Tclx 8.0".

make what     # warms the cache, takes a while with no output, then prints all targets it will build by default

# Build an LMDBv1 target with sources kept for the manual recipes below.
# Use the SQLite version reported by `make what` or by the Fossil Makefile.
make build TARGETS="3.53.3+lmdbv1-1.0" KEEP_SOURCES=1 ALWAYS_REBUILD=1

The flags KEEP_SOURCES and ALWAYS_REBUILD are needed for the amalgamation. See LUMOSQL-AMALGAMATION.md if you want an explanation for them.

The last line of output from successful build will tell you what directory it is in. The recipes below reference three subdirectories from that build which we put into handy variables:

SRC=/path/to/lumosql/build/<target>/sources/sqlite3   # amalgamation + lumo helpers
LM=/path/to/lumosql/build/<target>/sources/lmdb       # LMDB .c/.o (sources/lmdbv1 for an lmdbv1 target)
LB=/path/to/lumosql/build/<target>/lumo/build         # LMDB headers + backend shim under $LB/.lumosql/backend/

The standalone build/<target>/lumo/build/sqlite3 links LMDB dynamically, so running it directly needs liblmdb.so on LD_LIBRARY_PATH; without it you get error while loading shared libraries: liblmdb.so. This is expected and does not affect the libfossil and Fossil builds below, which embed the LMDB objects directly.

Do not use ROWSUM for Fossil, because while it's an interesting LumoSQL feature, the Merkle DAG already gives most of the benefit there and so it isn't really relevant to Fossil.

Build libfossil

Get the source: cd $HOME; fossil clone https://fossil.wanderinghorse.net/r/libfossil

Prepare the source tree

cd libfossil
cp $SRC/sqlite3.c $SRC/sqlite3.h extsrc/
cp $SRC/src/lumo-vdbeInt.h  extsrc/
cp $SRC/src/lumo-vdbeAdd.c  extsrc/
cp $SRC/src/lumo-func.c     extsrc/
cp $SRC/src/lumo-sha3.c     extsrc/
cp $SRC/src/lumo-blake3*.c  extsrc/
cp $SRC/src/blake3.h        extsrc/
cp $SRC/src/blake3_impl.h   extsrc/
cp $SRC/src/lumo-siphash.c  extsrc/
cp $SRC/src/siphash.h       extsrc/
cp -r $LB/.lumosql/backend  extsrc/backend
cp $LB/lmdb.h $LB/midl.h    extsrc/
cp $LM/mdb.c $LM/midl.c     extsrc/

$LB/.lumosql/backend holds the expanded backend shim the amalgamation #includes. ($SRC/.lumosql/backend holds redirect stubs that resolve only inside the LumoSQL build tree.)

Build the LMDB objects with -fPIC (libfossil is a shared object). -I$LB lets mdb.c find lmdb.h:

cc -fPIC -O2 -I$LB -c $LM/mdb.c  -o extsrc/mdb_pic.o
cc -fPIC -O2 -I$LB -c $LM/midl.c -o extsrc/midl_pic.o

Configure and build

./configure \
  --lumosql \
  --no-fnc \
  --with-sqlite=extsrc \
  CPPFLAGS="-DLUMO_LMDB_FIXED_ROWID -DLUMO_BACKEND_PRAGMA -DSQLITE_ENABLE_RTREE" \
  LDFLAGS="$PWD/extsrc/mdb_pic.o $PWD/extsrc/midl_pic.o"

libfossil compiles sqlite3.c with a rule that uses $(CFLAGS.core) and ignores the configure-time CPPFLAGS, so the -DLUMO_LMDBV1_ENCRYPT you passed via CPPFLAGS above does not reach the amalgamation (which is where the crypto shim is compiled in). Run the same make CFLAGS.core=... line from the non-encrypted build.

make CFLAGS.core='-Wall -fPIE -fPIC -O2 -Iextsrc \
  -DLUMO_LMDB_FIXED_ROWID -DLUMO_BACKEND_PRAGMA \
  -DSQLITE_ENABLE_RTREE -DSQLITE_THREADSAFE=1 -DFSL_ENABLE_MUTEX=0 \
  -Wno-unused-function -Wno-maybe-uninitialized -Wno-unused-variable'

This produces libfossil.so and the f-apps under the checkout root.

Test libfossil

The test driver correctness-libfossil-lmdb.sh lives in LumoSQL's examples/fossil/test/ directory, but it expects to find the f-* and test-* binaries in its parent directory. Copy the script into the libfossil checkout root and run it from there:

mkdir $HOME/libfossil/test
cp $HOME/lumosql/examples/fossil/test/correctness-libfossil-lmdb.sh $HOME/libfossil/test
cd $HOME/libfossil/test
sh correctness-libfossil-lmdb.sh

Build Fossil

Get the source via Fossil, switch to the HTTP 1.1 branch and build:

cd $HOME; fossil clone https://fossil-scm.org/home/fossil
cd $HOME/fossil
fossil update http1-1-chunked

If you do not have Fossil available, use the GitHub mirror instead:

cd $HOME; git clone https://github.com/drhsqlite/fossil-mirror fossil
cd $HOME/fossil
git checkout http1-1-chunked

Prepare the source tree for the SQLite amalgamation

cd $HOME/fossil
cp $SRC/sqlite3.c $SRC/sqlite3.h extsrc/
cp $SRC/shell.c             extsrc/
cp $SRC/src/lumo-vdbeInt.h  extsrc/
cp $SRC/src/lumo-vdbeAdd.c  extsrc/
cp $SRC/src/lumo-func.c     extsrc/
cp $SRC/src/lumo-sha3.c     extsrc/
cp $SRC/src/lumo-blake3*.c  extsrc/
cp $SRC/src/blake3.h        extsrc/
cp $SRC/src/blake3_impl.h   extsrc/
cp $SRC/src/lumo-siphash.c  extsrc/
cp $SRC/src/siphash.h       extsrc/
cp -r $LB/.lumosql/backend  extsrc/backend
cp $LM/mdb.o $LM/midl.o     extsrc/
cp $LB/lmdb.h $LB/midl.h    extsrc/

shell.c replaces Fossil's extsrc/shell.c, which references shell symbols absent from the LumoSQL amalgamation. The LMDB objects are static here because Fossil links a static binary.

Configure and add LumoSQL flags

./configure , then edit src/main.mk to append to the SQLITE_OPTIONS block, ending with -DSQLITE_THREADSAFE=1:

SQLITE_OPTIONS = ... \
                 -Iextsrc \
                 -Wno-unused-function -Wno-declaration-after-statement \
                 -Wno-maybe-uninitialized \
                 -DLUMO_LMDB_FIXED_ROWID -DLUMO_BACKEND_PRAGMA \
                 -DSQLITE_ENABLE_RTREE -DSQLITE_THREADSAFE=1

With unmodified SQLite, Fossil's SQLITE_OPTIONS sets -DSQLITE_THREADSAFE=0, but the LumoSQL shim needs THREADSAFE=1 to compile. This appended -D overrides Fossil's -D.

Next edit the generated Makefile and append the LMDB objects to the LIB line:

LIB += extsrc/mdb.o extsrc/midl.o

Add the diskused stub

Fossil's src/stat.c references sqlite3_diskused_init, which the LumoSQL amalgamation omits; without it the link fails with undefined reference to sqlite3_diskused_init. Add a stub and link it in:

cat > extsrc/lumo_diskused_stub.c <<'EOF'
#include "sqlite3.h"
int sqlite3_diskused_init(sqlite3 *db, char **pzErrMsg,
                          const sqlite3_api_routines *pApi){
  (void)db; (void)pzErrMsg; (void)pApi;
  return SQLITE_OK;
}
EOF
mkdir -p bld
gcc -I. -I./src -I./extsrc -Ibld -DHAVE_AUTOCONFIG_H -Os \
    -c extsrc/lumo_diskused_stub.c -o bld/lumo_diskused_stub.o
echo 'LIB += bld/lumo_diskused_stub.o' >> Makefile

Then type make, which should produce a fossil binary in the checkout root.

Run a quick test:

rm -rf ~/.config/fossil.db ~/.config/fossil.db/   # clear config-db format mismatch
cd "$(mktemp -d)"                                  # run outside any checkout
F=$HOME/fossil/fossil
$F new /tmp/ftest.fossil
ls -la /tmp/ftest.fossil /tmp/ftest.fossil-lock   # expect a regular file + its -lock sibling
$F open /tmp/ftest.fossil --force
echo hello > a.txt
$F add a.txt
$F commit -m "first" --user "$USER"
$F timeline

Congratulations! You have just built and tested Fossil with a LumoSQL LMDB backend.

Play with your unencrypted LMDB-backed Fossil

Run the LumoSQL examples/fossil/test/stress-fossil-lmdb.sh which by default clones the substantial SQLite repo and does all manner of operations on it. The script has an example at the top of how you can pass it other repos. You should expect to be able to clone trees about ten times larger than SQLite without any difficulty.

If you are that kind of person, import a truly gigantic massive tree and watch the crashes.

Build Fossil with Encryption

Requires the lmdbv1 backend built with OPTION_LMDBV1_ENCRYPT=on:

cd $HOME/lumosql
make build TARGETS="3.53.3+lmdbv1-1.0+lmdbv1_encrypt-on" KEEP_SOURCES=1 ALWAYS_REBUILD=1

Re-point SRC/LM/LB at this target's build directory which will be LMDBv1 (use your own 3.53.x if make what showed a different default), then follow the Fossil build instructions above with these additions:

db.c reads the key, so append the flag to TCCFLAGS in the generated Makefile, and add -lsodium to the LIB line:

TCCFLAGS = ... -DHAVE_AUTOCONFIG_H -DLUMO_LMDBV1_ENCRYPT
LIB += extsrc/mdb.o extsrc/midl.o -lsodium

Fossil opens without SQLITE_OPEN_URI, so the backend never sees the lumo_key URI parameter. In db.c, before the sqlite3_open_v2 in db_open, add SQLITE_OPEN_URI to the flags and rewrite the filename from LUMO_KEY:

diff --git a/src/db.c b/src/db.c
index 43f6081..95e5c2c 100644
--- a/src/db.c
+++ b/src/db.c
@@ -2191,11 +2191,25 @@ LOCAL sqlite3 *db_open(const char *zDbName){
     g.zVfsName = "apndvfs";
   }
   blob_reset(&bNameCheck);
-  rc = sqlite3_open_v2(
-       zDbName, &db,
-       SQLITE_OPEN_READWRITE | SQLITE_OPEN_CREATE,
-       g.zVfsName
-  );
+#if defined(LUMO_LMDBV1_ENCRYPT)
+  {
+    const char *zLumoKey = fossil_getenv("LUMO_KEY");
+    const char *zBase = file_tail(zDbName);
+    if( zLumoKey && zLumoKey[0] && sqlite3_strglob("file:*", zDbName)!=0
+        && strchr(zDbName, '?')==0
+        && file_isfile(zDbName, ExtFILE)==0
+        && fossil_strcmp(zBase, "fossil.db")!=0
+        && fossil_strcmp(zBase, ".fossil")!=0
+        && fossil_strcmp(zBase, "_fossil")!=0 ){
+      char *zEnc = sqlite3_mprintf("file:%s?lumo_key=%s", zDbName, zLumoKey);
+      zDbName = fossil_strdup(zEnc);
+      sqlite3_free(zEnc);
+    }
+  }
+#endif
+  rc = sqlite3_open_v2(zDbName, &db,
+       SQLITE_OPEN_READWRITE | SQLITE_OPEN_CREATE | SQLITE_OPEN_URI,
+       g.zVfsName);
   if( rc!=SQLITE_OK ){
     db_err("[%s]: %s", zDbName, sqlite3_errmsg(db));
   }

Apply the same rewrite in db_attach so the repository and localdb attachments inherit the key. The global config can be named fossil.db, .fossil, or _fossil depending on platform and $HOME layout, so all three are excluded.

diff --git a/src/db.c b/src/db.c
index 43f6081..2039f79 100644
--- a/src/db.c
+++ b/src/db.c
@@ -2242,6 +2242,26 @@ void db_detach(const char *zLabel){
 void db_attach(const char *zDbName, const char *zLabel){
   Blob key;
   if( db_table_exists(zLabel,"sqlite_schema") ) return;
+#if defined(LUMO_LMDBV1_ENCRYPT)
+  {
+    const char *zLumoKey = fossil_getenv("LUMO_KEY");
+    const char *zBase = file_tail(zDbName);
+    if( zLumoKey && zLumoKey[0] && sqlite3_strglob("file:*", zDbName)!=0
+        && strchr(zDbName, '?')==0
+        && file_isfile(zDbName, ExtFILE)==0
+        && fossil_strcmp(zBase, "fossil.db")!=0
+        && fossil_strcmp(zBase, ".fossil")!=0
+        && fossil_strcmp(zBase, "_fossil")!=0 ){
+      char *zEnc = sqlite3_mprintf("file:%s?lumo_key=%s", zDbName, zLumoKey);
+      char *zCmd = sqlite3_mprintf("ATTACH DATABASE %Q AS %Q KEY ''",
+                                   zEnc, zLabel);
+      db_exec_sql(zCmd);
+      sqlite3_free(zCmd);
+      sqlite3_free(zEnc);
+      return;
+    }
+  }
+#endif
   blob_init(&key, 0, 0);
   db_maybe_obtain_encryption_key(zDbName, &key);
   if( fossil_getenv("FOSSIL_USE_SEE_TEXTKEY")==0 ){

Make fossil sql work on an encrypted repository

The two edits above cover every command that uses Fossil's own database connection. fossil sql closes Fossil's connection and hands off to the embedded SQLite shell, which re-opens the repository, not using db_open/db_attach. The edits below allow fossil sql to open an encrypted repository.

In src/sqlcmd.c, rewrite the name handed to the shell, and key the localdb ATTACH (the .fslckout checkout DB is itself encrypted). Leave the configdb ATTACH that follows it untouched so the global config stays plaintext. URI handling is on by default in this build, so no extra open flag is needed. First, sqlcmd_get_dbname:

diff --git a/src/sqlcmd.c b/src/sqlcmd.c
index 5175ac9..de75b25 100644
--- a/src/sqlcmd.c
+++ b/src/sqlcmd.c
@@ -276,6 +276,19 @@ static void sqlcmd_atexit(void) {
 */
 void sqlcmd_get_dbname(const char **pzRepoName){
   *pzRepoName = g.zRepositoryName;
+#if defined(LUMO_LMDBV1_ENCRYPT)
+  {
+    const char *zLumoKey = fossil_getenv("LUMO_KEY");
+    const char *zName = g.zRepositoryName;
+    if( zLumoKey && zLumoKey[0] && zName && zName[0]
+        && sqlite3_strglob("file:*", zName)!=0
+        && strchr(zName, '?')==0
+        && file_isfile(zName, ExtFILE)==0
+        && fossil_strcmp(file_tail(zName), "fossil.db")!=0 ){
+      *pzRepoName = sqlite3_mprintf("file:%s?lumo_key=%s", zName, zLumoKey);
+    }
+  }
+#endif
 }
 
 /*

Then the localdb ATTACH in sqlcmd_autoinit (the configdb block just below it is left as-is):

diff --git a/src/sqlcmd.c b/src/sqlcmd.c
index 5175ac9..b3e73f8 100644
--- a/src/sqlcmd.c
+++ b/src/sqlcmd.c
@@ -232,7 +232,22 @@ static int sqlcmd_autoinit(
     db_maybe_set_encryption_key(db, g.zRepositoryName);
   }
   if( g.zLocalDbName ){
-    char *zSql = sqlite3_mprintf("ATTACH %Q AS 'localdb' KEY ''",
+    char *zSql;
+#if defined(LUMO_LMDBV1_ENCRYPT)
+    /* The checkout localdb (.fslckout) is itself an encrypted LMDB file,
+    ** so the shell's ATTACH must carry the key too. */
+    const char *zLumoKey = fossil_getenv("LUMO_KEY");
+    if( zLumoKey && zLumoKey[0]
+        && sqlite3_strglob("file:*", g.zLocalDbName)!=0
+        && strchr(g.zLocalDbName, '?')==0
+        && file_isfile(g.zLocalDbName, ExtFILE)==0 ){
+      char *zEnc = sqlite3_mprintf("file:%s?lumo_key=%s",
+                                   g.zLocalDbName, zLumoKey);
+      zSql = sqlite3_mprintf("ATTACH %Q AS 'localdb' KEY ''", zEnc);
+      sqlite3_free(zEnc);
+    }else
+#endif
+    zSql = sqlite3_mprintf("ATTACH %Q AS 'localdb' KEY ''",
                                  g.zLocalDbName);
     sqlite3_exec(db, zSql, 0, 0, 0);
     sqlite3_free(zSql);

Fossil initialises SQLite before the embedded shell runs, so the shell prints WARNING: attempt to configure SQLite after initialization. to stdout, which corrupts the output callers are expecting from fossil sql. Send it to stderr instead:

diff --git a/extsrc/shell.c b/extsrc/shell.c
index 12a04c8..a66b1d6 100644
--- a/extsrc/shell.c
+++ b/extsrc/shell.c
@@ -36295,7 +36295,7 @@ static void usage(int showDetail){
 */
 static void verify_uninitialized(void){
   if( sqlite3_config(-1)==SQLITE_MISUSE ){
-    sputz(stdout, "WARNING: attempt to configure SQLite after"
+    sputz(stderr, "WARNING: attempt to configure SQLite after"
           " initialization.\n");
   }
 }

(change the stdout argument to stderr). Apply this after copying shell.c into extsrc/ in the source-tree step above.

Then make .

The key is a passphrase in LUMO_KEY, run through Argon2id with a salt stored in a <repository>-salt sibling file next to the repository:

export LUMO_KEY="correct-horse-battery-staple"
./fossil new  /tmp/sec.fossil
./fossil open /tmp/sec.fossil
echo topsecret > a.txt
./fossil add a.txt
./fossil commit -m "first"
./fossil cat a.txt

A wrong or missing LUMO_KEY fails the open with (SQLITE_AUTH) and the shell unhelpfully prints unable to open database ...: unknown error, which will be fixed at some point. Temp and in-memory databases are never encrypted.

Build libfossil with Encryption

Requires the same OPTION_LMDBV1_ENCRYPT=on lmdbv1 backend you built in the Fossil-encryption section above (3.53.3+lmdbv1-1.0+lmdbv1_encrypt-on). Make sure SRC/LM/LB point at that target's build directory. Note LM is sources/lmdbv1, not sources/lmdb (use your own 3.53.x if make what showed a different default), then follow the libfossil build instructions above with these changes. The same tests detect encryption support. Export LUMO_KEY before running encrypted tests.

Build the LMDB PIC object with encryption, and link libsodium:

cc -fPIC -O2 -DLUMO_LMDBV1_ENCRYPT -Iextsrc -c extsrc/mdb.c -o extsrc/mdb_pic.o
./configure --lumosql --no-fnc --with-sqlite=extsrc \
  CPPFLAGS="... -DLUMO_LMDBV1_ENCRYPT" \
  LDFLAGS="$PWD/extsrc/mdb_pic.o $PWD/extsrc/midl_pic.o -lsodium"

--lumosql only defines LIBFOSSIL_LUMOSQL for cx.c. The key handling is in db.c, so extend the switch in auto.def:

diff --git a/auto.def b/auto.def
index 37dd346..81534a7 100644
--- a/auto.def
+++ b/auto.def
@@ -596,6 +596,7 @@ apply {{} {
   array set lcflags {}
   if {[opt-bool lumosql]} {
     set lcflags(cx.c) {-DLIBFOSSIL_LUMOSQL}
+    set lcflags(db.c) {-DLIBFOSSIL_LUMOSQL}
   }
   foreach f $lfile {
     set fc \$(DIR.src)/$f

fsl_db_open opens without SQLITE_OPEN_URI. Before the sqlite3_open_v2 call, rewrite the filename from LUMO_KEY into a URI and set the flag:

diff --git a/src/db.c b/src/db.c
index 1744eda..a9ea19c 100644
--- a/src/db.c
+++ b/src/db.c
@@ -1216,7 +1216,23 @@ int fsl_db_open( fsl_db * const db, char const * dbFile,
       }
       if(!sOpenFlags) sOpenFlags = SQLITE_OPEN_READONLY;
     }
-    rc = sqlite3_open_v2( dbFile, &dbh, sOpenFlags, NULL );
+    char const * zOpen = dbFile;
+    char * zLumoUri = NULL;
+#if defined(LIBFOSSIL_LUMOSQL)
+    {
+      char const * zLumoKey = fsl_getenv("LUMO_KEY");
+      if( zLumoKey && *zLumoKey && 0!=*dbFile && ':'!=*dbFile
+          && NULL==strchr(dbFile, '?') ){
+        zLumoUri = fsl_mprintf("file:%s?lumo_key=%s", dbFile, zLumoKey);
+        if(zLumoUri){
+          zOpen = zLumoUri;
+          sOpenFlags |= SQLITE_OPEN_URI;
+        }
+      }
+    }
+#endif
+    rc = sqlite3_open_v2( zOpen, &dbh, sOpenFlags, NULL );
+    fsl_free(zLumoUri);
     if(rc){
       if(dbh){
         /* By some complete coincidence, FSL_RC_DB==SQLITE_CANTOPEN. */

Apply the same rewrite in fsl_db_attach so attached databases inherit the key:

diff --git a/src/db.c b/src/db.c
index 1744eda..b6e4ca1 100644
--- a/src/db.c
+++ b/src/db.c
@@ -222,10 +222,26 @@ void fsl_db_err_reset( fsl_db * const db ){
 
 int fsl_db_attach(fsl_db * const db, const char *zDbName,
                   const char *zLabel){
-  return (db && db->dbh && zDbName && *zDbName && zLabel && *zLabel)
-    ? fsl_db_exec(db, "ATTACH DATABASE %Q AS %!Q /*%s()*/",
-                  zDbName, zLabel, __func__)
-    : FSL_RC_MISUSE;
+  if( !(db && db->dbh && zDbName && *zDbName && zLabel && *zLabel) ){
+    return FSL_RC_MISUSE;
+  }
+#if defined(LIBFOSSIL_LUMOSQL)
+  {
+    char const * zLumoKey = fsl_getenv("LUMO_KEY");
+    if( zLumoKey && *zLumoKey && ':'!=*zDbName
+        && NULL==strchr(zDbName, '?') ){
+      char * zUri = fsl_mprintf("file:%s?lumo_key=%s", zDbName, zLumoKey);
+      int rc;
+      if(!zUri) return FSL_RC_OOM;
+      rc = fsl_db_exec(db, "ATTACH DATABASE %Q AS %!Q /*%s()*/",
+                       zUri, zLabel, __func__);
+      fsl_free(zUri);
+      return rc;
+    }
+  }
+#endif
+  return fsl_db_exec(db, "ATTACH DATABASE %Q AS %!Q /*%s()*/",
+                     zDbName, zLabel, __func__);
 }
 
 int fsl_db_detach(fsl_db * const db, const char *zLabel){

As in the non-encrypted build, libfossil compiles sqlite3.c with a rule that uses $(CFLAGS.core) and ignores the configure-time CPPFLAGS, so the -DLUMO_LMDBV1_ENCRYPT you passed via CPPFLAGS above does not reach the amalgamation (which is where the crypto shim is compiled in). Run the same make CFLAGS.core=... line as the non-encrypted build, with -DLUMO_LMDBV1_ENCRYPT added to it.

make CFLAGS.core='-Wall -fPIE -fPIC -O2 -Iextsrc \
  -DLUMO_LMDB_FIXED_ROWID -DLUMO_BACKEND_PRAGMA \
  -DSQLITE_ENABLE_RTREE -DSQLITE_THREADSAFE=1 -DFSL_ENABLE_MUTEX=0 \
  -DLUMO_LMDBV1_ENCRYPT \
  -Wno-unused-function -Wno-maybe-uninitialized -Wno-unused-variable'

Set LUMO_KEY before running any f-app:

export LUMO_KEY="correct-horse-battery-staple"
./f-new  /tmp/sec.f -m init
./f-open /tmp/sec.f --force
echo topsecret > a.txt
./f-add a.txt
./f-ci  a.txt -m "first"
./f-acat $(./f-ls | awk '/a.txt/{print $1}')

A wrong or missing LUMO_KEY fails the open with FSL_RC_DB (SQLite SQLITE_AUTH). Plaintext does not appear in the data file.

Run sh correctness-libfossil-lmdb.sh. It detects encryption support and tests crypto when available.

Development

If you are making edits to the backends under not-fork.d and then testing Fossil, you need to rm -f bld/sqlite3.o before each make. Recovering an interrupted build needs the lock directory removed with rm -rf bld/.build_lock.