RPM backed by LumoSQL

rpm is the package manager at the core of Linux ooperating systems such as Fedora, RHEL, SUSE, openSUSE, Mageia, OpenMandriva and more, and it uses SQLite as its database. LumoSQL is a modification of SQLite, explained in the main README. The recipe in this directory builds rpm against LumoSQL with the LMDBv1 backend, without any modification of rpm.

What this means:

rpm can now use a different, but identically compatible database to SQLite. Identically compatible means that all API calls are the same, and users can experience better performance, encryption or online backups. This works because LumoSQL in this case is SQLite backed by the LMDB database library, which uses the operating system for its memory management and crash recovery characteristics. LMDB v1.0 has encryption and incremental backups built-in, so LumoSQL and therefore rpm inherits these features too.

This recipe is aimed at rpm upstream contributors wanting to evaluate a different backend, and produces rpm/rpmdb binaries from rpm git master.

RPM will work equally well with lumosql.so re-labelled as libsqlite3.so, or, as an SQLite amalgamation compiled in directly. We have chosen to statically compile a LumoSQL amalgamation because its simpler.

We compile LumoSQL's SQLite plus the LMDBv1 backend into a single static libsqlite3.a, generate a sqlite3.pc that describes it, and put that .pc first on PKG_CONFIG_PATH. RPM's pkg_check_modules then discovers it and links LumoSQL statically into librpm.

Dependencies

Install the normal LumoSQL build dependencies from the top-level LumoSQL README, checking with make doctor as described there.

This recipe works using git as the source code management system. LumoSQL, being very close to SQLite, uses Fossil as the primary SCM, but this recipe uses git mirrors and does not need Fossil at all.

git clone https://codeberg.org/lumosql/lumosql     # same result as fossil clone https://lumosql.org/src/lumosql

The not-forking tool is required, and is not yet packaged by any distribution. LumoSQL uses it to assemble the SQLite and backend sources. not-forking needs perl plus the Text::Glob module (libtext-glob-perl on Debian/Ubuntu, perl-Text-Glob on Fedora/RHEL). Then:

git clone https://codeberg.org/not-forking/not-forking   # or: fossil clone https://lumosql.org/src/not-forking
cd not-forking
perl Makefile.PL && make && sudo make install

In addition, there are the RPM dependencies:

Fedora/RHEL:

sudo dnf install git cmake gcc gcc-c++ pkgconf
sudo dnf builddep rpm      # pulls RPM's own build requirements

Debian/Ubuntu:

sudo apt install git cmake build-essential pkg-config
sudo apt build-dep rpm    # wont work unless the deb-src repo is enabled

The Makefile disables RPM's optional SELinux and Sequoia integrations for the LumoSQL demo build, and uses a local scdoc fallback if scdoc is not installed.

Quick build

From the LumoSQL root directory:

make doctor        # resolve any remaining LumoSQL dependency problems
cd examples/rpm
make

The binaries are:

examples/rpm/build/rpm-build/tools/rpm
examples/rpm/build/rpm-build/tools/rpmdb

Confirm LumoSQL is linked statically:

ldd build/rpm-build/tools/rpm | grep -i sqlite   # expect: no output

By default the Makefile builds RPM master against LumoSQL target 3.53.2+lmdbv1-1.0, using build/ as its work directory. To build a specific RPM commit, pin the LumoSQL target explicitly, or override the source/work locations:

make RPM_REF=rpm-4.20.0
make LUMOSQL_TARGET=3.53.2+lmdbv1-1.0
make RPM_GIT_URL=https://github.com/rpm-software-management/rpm WORK_DIR=/tmp/lumo-rpm-build

make smoke after make reuses the binaries. If you want to force a full rebuild:

make REBUILD=1
make clean && make

Smoke test

make smoke # install, query and erase a package in a throwaway rpmdb

make smoke is a basic selftest of the build, and that the rpm binary works. See the smoke.sh helper script for more detail.

Install a package into the DB

To install something into the database created by make smoke:

"$RPM" $RPM_COMMON -ivh --define "_pkgverify_level none" some-package.rpm
"$RPM" $RPM_COMMON -qa

RPM 6 rejects unsigned packages by default, failing with package ... does not verify: no signature. _pkgverify_level none disables that check.

If you want to roll your own test package using this database, rpmbuild needs two macros:

build/rpm-build/tools/rpmbuild --rcfile build/rpm-build/rpmrc --macros build/rpm-build/macros \
    --define "_topdir /tmp/rpmtop" \
    --define "_fileattrsdir $PWD/build/rpm/fileattrs" \
    --define "_rpmconfigdir $PWD/build/rpm/scripts" \
    -bb my.spec

Migrating an existing rpmdb

To migrate an existing system rpmdb into a LumoSQL-backed one, use RPM's header export/import. This is because the LMDB file format is completely different to SQLite.

RPM=build/rpm-build/tools/rpm
RPMDB=build/rpm-build/tools/rpmdb
RPM_COMMON="--rcfile build/rpm-build/rpmrc --macros build/rpm-build/macros --root /tmp/rpmtest"
rpmdb --exportdb > /tmp/live.headers
"$RPMDB" $RPM_COMMON --importdb < /tmp/live.headers