This runs a hand-authored SQL test suite in test/sql/ that exercises the storage backend. The SQL language layer is unchanged by LumoSQL and covered by SQLite's own tests, so we do not try to repeat the thousands of tests SQLite is already doing on higher-level functionality. Builds binaries on demand, runs every *.test against each built sqlite3, and appends results to test-sql.sqlite.

Subset and direct invocation

make test-sql LUMO_TEST_FILES='spill.test churn.test'
tclsh tool/test-runner.tcl /path/to/sqlite3 test/sql/churn.test

Test blocks in .test files

do_execsql_test NAME {
  SQL ...
} {expected output}

Expected values containing error, not a database, unable to open, or malformed match any error. Whitespace is collapsed.

Roles and cross-binary testing

The command-line binary is the default role. Extra roles are declared in a header and filled from build/<target>/sqlite3, matched on capabilities parsed from the target name:

Target name	Capabilities
3.53.0	version=3.53.0, backend=btree, rowsum=off
3.53.0+lmdb-0.9.33	version=3.53.0, backend=lmdb, rowsum=off
3.53.0+rowsum-on	version=3.53.0, backend=btree, rowsum=on
3.53.0+lmdb-0.9.33+rowsum-on	version=3.53.0, backend=lmdb, rowsum=on

Requirements header:

requires role NAME { key=val ... }
requires db (shared|each) (file|:memory:)

A role no binary satisfies skips the whole file. shared file gives all roles one database; each gives each role its own. Default is each :memory:.

Cross-binary directives:

Directive	Purpose
do_execsql_test NAME [ROLE] { SQL } { expected }	Run on ROLE (default if omitted)
do_compare_test NAME { roles } [differ] { SQL }	All roles equal, or with differ each differs from first
do_golden_test NAME GOLDEN { others } { SQL }	Others match GOLDEN
do_corrupt_test NAME { marker } { read_sql }	Flip byte after marker on disk; pass if read_sql reports corruption

Querying results

tclsh tool/test-sql-filter.tcl           # summary
tclsh tool/test-sql-filter.tcl -failures # all failures
tclsh tool/test-sql-filter.tcl -diff 1 2 # changes between runs
tclsh tool/test-sql-filter.tcl -help     # full options

Modes: -runs, -summary, -failures, -skips, -categories, -diff, -compare, -test, -export. Filters: -target, -file, -category, -format.

Relationship to benchmarking

	make benchmark	make test-sql
Tests	Performance	Correctness
Input	not-fork.d/*/benchmark/*.test	test/sql/*.test
Output	benchmarks.sqlite	test-sql.sqlite
Query tool	tool/benchmark-filter.tcl	tool/test-sql-filter.tcl
Cross-binary	No	Yes
Corruption testing	No	Yes

Both use append-only SQLite databases with queryable history, comparable across runs.