Many hyperlinks are disabled.
Use anonymous login
to enable hyperlinks.
50 most recent check-ins
|
2026-05-24
| ||
| 05:30 | Fix FTS5, rtree, and WITHOUT-ROWID under OPTION_ROWSUM=on. Replace the trailing-serial-type heuristic in lumoExtensionAdd / lumoExtensionPresent . This made 6 SQL test probes suddenly pass. Leaf check-in: eea66261eb user: danshearer tags: trunk | |
| 05:27 | Long key support. After trying various different schemes, the simplest one works. We compile LMDB with MDB_MAXKEYSIZE=0, which tells LMDB to compute its key size from the page size at initialisation time. this is is approximately 1980 bytes when page sizes are 4k, which solves the problem of LMDB's default being 411 bytes. We do not have a goal of being compatible with other LMDB databases so that is fine. Anything above this returns SQLITE_TOOBIG . check-in: 89325afa90 user: danshearer tags: trunk | |
| 00:49 | One possible error case I forgot in long-keys plan check-in: fef7beea33 user: danshearer tags: trunk | |
|
2026-05-23
| ||
| 23:33 | Remove .NOTPARALLEL , but added a comment explaining what parallelism we do and don't support. `make -jN` is safe but doesn't really do anything. Read the comment to see how we achieve that effect. check-in: acc8bb60e7 user: danshearer tags: trunk | |
| 23:21 | Delete ancient container target in Makefile, and a couple of dead files. check-in: 03137915f5 user: danshearer tags: trunk | |
| 23:11 | LumoSQL was designed for safe concurrency, the build system locks anything to do with a target before working on it. So for example two build/bench processes concurrently will work, one wins and the other either skips this target or waits. What was not concurrent was global shared access that applies to all targets, things like Makefile.options or benchmarks.sqlite which any target may want access to. Testing with concurrent runs easily showed up all of the cases fixed here, and hopefully enough careful checking has been done so that we don't have any more lurking. check-in: 3b5d6f10c7 user: danshearer tags: trunk | |
| 19:09 | Comment out race condition in build.tcl, discovered when running lots of benchmarks and a build on a big shared storage machine. This isn't the best answer but it will solve the problem while I work out if its hiding other problems. check-in: 1add4f00de user: danshearer tags: trunk | |
| 16:34 | I *think* this empty commit is the way to remove an empty directory in fossil following `fossil mv --hard FROM-X TO-Y`. check-in: 0f5a130129 user: danshearer tags: trunk | |
| 16:32 | Moved more historical stuff to loft. some of this is likely to be useful, but not just yet and its important to be able to see LumoSQL at a glance. I suspect now I'm using slurm for benchmarking I should look at Ruben's kbench files again. check-in: 8c51cf0db4 user: danshearer tags: trunk | |
| 16:24 | Update for 2026 reboot check-in: f9795168bf user: danshearer tags: trunk | |
| 16:20 | Move alternative-tests to loft/ , and note in the relevant section of TODO.md. check-in: c6acf2ec66 user: danshearer tags: trunk | |
| 16:18 | Start directory for old/cruft/unknown/interesting files that don't yet have a known purpose or obvious place to live. check-in: 44265fbf3b user: danshearer tags: trunk | |
| 16:09 | Explain alternative-tests, which will be very useful when we finish off rowsum work but is not currently a particularly useful 'make test' target. check-in: 33b5cb9958 user: danshearer tags: trunk | |
| 15:53 | Fix misleading comment about empty files being dead weight. Empty files override the SQLite amalgamation mechanism as explained in the last commit. check-in: 5ef6351e85 user: danshearer tags: trunk | |
| 15:47 | This reverts "Remove dead code" (44507d22), because the SQLite amalgamation is a beast. Claudio had these there by careful design, because these are how the LMDB backend suppresses upstream SQLite's WAL/mutex code from the amalgamation. The not-fork sed pass rewrites every reference to wal.c (etc.) in src/main.mk, Makefile.in, mksqlite3c.tcl etc to lumo_wal.c, which is a 4-line shim that #includes the *empty* backend file. This makes the amalgamation tool read zero bytes where it would otherwise pull in src/wal.c, and it works fine. Oops. check-in: 77b224d81a user: danshearer tags: trunk | |
| 15:33 | Clean up the stub implementation of SQLite's backup API. Write up in TODO so that we don't forget to implement this using LMDB's backup facilities. check-in: 6f9be6e788 user: danshearer tags: trunk | |
| 15:19 | Remove dead code. These are stub files and their handling code from when we thought we were going to fill them out. Some of them were zero bytes. It's a smaller cleanup than it looks but definitely makes life easier and safer. check-in: 44507d2285 user: danshearer tags: trunk | |
| 15:02 | Finished the thinking of long keys implementation, in the end it looks like a few hundred lines in btree.c , and a lot of care. check-in: 998604be37 user: danshearer tags: trunk | |
| 14:22 | Lots more thinking about long keys and rowsums. check-in: 4314ba1a08 user: danshearer tags: trunk | |
| 09:14 | Removed obsolete distro references, fixed formatting, updated for the new rebooted LumoSQL check-in: c5a9bcdabe user: danshearer tags: trunk | |
| 09:10 | More bringing README up to date for the relaunch check-in: 0c4819cffe user: danshearer tags: trunk | |
| 08:26 | Warn when warming up an empty cache, because otherwise it can seem like nothing is happening as many Mb are downloaded the first time check-in: 5108303e28 user: danshearer tags: trunk | |
| 08:25 | Add instructions for the non-root case, typically on a shared compute cluster of the sort ideal for monster LumoSQL build runs. check-in: e4815bd084 user: danshearer tags: trunk | |
|
2026-05-15
| ||
| 11:10 | Remove BDB from everywhere including documentation except for historical factual references. BDB was a useful part of the original proof of concept, showing that multiple backends were possible. The port was done by Oracle, long-since abandoned, as is BDB. Also started to update the documentation generally. Many things have changed since 2022, and we now have a feeling for which of the many original goals are likely to be most useful. check-in: 83e970b26e user: danshearer tags: trunk | |
|
2026-05-06
| ||
| 08:33 | Updated for the many changes to 2026 check-in: a2ac713d02 user: danshearer tags: trunk | |
|
2026-05-05
| ||
| 00:24 | Update status of the checksum VFS, which is maintained and still has lessons for us check-in: 09245f16bb user: danshearer tags: trunk | |
| 00:19 | Refine the rowsum argument with respect to what exists already in 2026 check-in: 2e53111960 user: danshearer tags: trunk | |
| 00:15 | Update database support claims for 2026 check-in: 2b6a0c9245 user: danshearer tags: trunk | |
|
2026-05-04
| ||
| 23:46 | Tighten up crypto thoughts check-in: 0651665169 user: danshearer tags: trunk | |
| 23:40 | Tighten up LMDB v1.0 draft plan for comment check-in: 9894a41503 user: danshearer tags: trunk | |
|
2026-05-02
| ||
| 09:59 | Update mirror comment check-in: 24cca33fed user: danshearer tags: trunk | |
|
2026-04-30
| ||
| 17:14 | Fix strangeness with LMDB v1.0 numbering in preparation for 1.0 port Document the benchmark SQLite database schema and give examples of useful queries check-in: 5420617f59 user: danshearer tags: trunk | |
| 17:05 | Plan for and comments about LMDB 1.0 backend, which will be another not-fork recipe and involve page level crypto at the later stages. check-in: dd5e63598d user: danshearer tags: trunk | |
|
2026-04-29
| ||
| 16:34 | Notes on current status check-in: 1d0c11edf0 user: danshearer tags: trunk | |
| 12:33 | Summarise work remaining in Lumo as-is in TODO.md Add TODO-CRYPTO.md, which is only about page-level crypto via LMDB which cut a v1.0 branch yesterday Row-level crypto is only addressed in the docs, there is no code or prototype work check-in: fb7b27fd17 user: danshearer tags: trunk | |
|
2026-04-27
| ||
| 06:42 | Corrected comments in sql coverage for current supported status. There's a lot less broken now, which gives us nice regression tests for the future. check-in: 12288843ad user: danshearer tags: trunk | |
| 04:12 | Reduced the lower bound for LumoSQL to be SQLite 3.30.0 (August 2020). This needed five small #if SQLITE_VERSION_NUMBER additions not-fork.d/lmdb/files/btree.c and pager.c. Things like the pager filename, and a couple of the Btree transaction API function calls were renamed around 3.34, and a minor change to BtreeMaxPageCount's rootpage type which changed in 3.33 etc. This is probably where the lower bound will stay unless there a user who is stuck on an older version really really wants another backend. check-in: cad2ad9692 user: danshearer tags: trunk | |
|
2026-04-26
| ||
| 20:03 | Document the limitations of rowsum, which now works for all normal schemas. Later we can make it work for all cases. The fix isn't hard, just requires a care not to break vdbe. check-in: 42b1a0b6ee user: danshearer tags: trunk | |
| 19:01 | Made rowsum work: * In vdbe: fix OP_Column and OP_Insert_Idx * In vdbeAdd.c: stop returning corruption on non-rowid last column * In sql-coverage, add all the failure conditions found for rowsum (loads - 7 in total, although most were due to a few errors.) check-in: bb02b3969a user: danshearer tags: trunk | |
| 12:08 | Add rowsum tests check-in: 37835880b0 user: danshearer tags: trunk | |
|
2026-04-25
| ||
| 20:18 | Fix FTS3 cursor mis-positioning bug: sqlite3BtreeMovetoUnpacked (via either of its two helpers) lied to the caller whenever the search key compared larger than every existing row in the table. Every SQLite optimiser path issuing a "seek just past this key" trusted the cursor's current row, but was actually reading the wrong row whenever the seek key happened to exceed the table's maximum. Added one probe to the coverage suite. LMDB coverage went from 60 pass / 4 expected_fail / 0 unexpected_pass to 64 pass / 1 expected_fail / 0 unexpected_pass (with one more probe in the suite, so 65 → 65 total); no-backend went from 64 pass / 0 expected_fail to 65 pass / 0 expected_fail. check-in: c3a3bac37d user: danshearer tags: trunk | |
| 16:47 | Added FTS5 to build flags. FTS5 only works for the no-backend builds, but it stops errors on other builds that test for FTS5 existence even though FTS5 errors when exercised. Not quite as obvious as it looked: upstream enables FTS4 by default. sql-coverage now probes for FTS5. check-in: 4d7b976df6 user: danshearer tags: trunk | |
| 16:18 | Deleting cruft, dead files, an #IF 0, etc. check-in: 0965e5ccff user: danshearer tags: trunk | |
| 15:46 | LMDB doesn't have a pager so everything here is a no-op. There are 14 functions previously returning unintelligent reponses, all now return the best response I can think of. some of these changes have visible effects: - PRAGMA journal_size_limit=N now round-trips: setting a number then querying reports N back, instead of always 0. - PRAGMA journal_mode=delete|truncate|persist|memory now round-trips and behaves like SQLite. - PRAGMA journal_mode=wal is now silently suppressed (the prior journal mode is preserved). - PRAGMA locking_mode now reports normal check-in: b124ab6be3 user: danshearer tags: trunk | |
| 15:09 | Tests for SQL we're supporting. The goal is to watch the failures go down to zero. check-in: df9264f7f9 user: danshearer tags: trunk | |
| 15:08 | ANALYZE on LMDB now writes correct row counts to sqlite_stat1 (was 0 1, now 200 1 for a 200-row table) sqlite_offset() returns distinct values per row instead of always 0. Did this by replacing four LMDB-backend stubs returning bogus values of SQLITE_INTERNAL or 0 where the upstream callers expected real numbers. Specifically: sqlite3BtreeRowCountEst now returns mdb_stat().ms_entries sqlite3BtreeCursorIsValidNN now reports mdb cursor state sqlite3BtreeMaxRecordSize returns 2147483647 upper bound, which I took from the comment in Btree.c: "Everything will still work if this routine always returns 2147483647 (which is the largest record that SQLite can handle)". sqlite3BtreeOffset returns the LMDB data pointer, which I think is probably correct check-in: efc81884ad user: danshearer tags: trunk | |
| 14:22 | These are SQLite settings that have no LMDB equivalent (LMDB doesn't have a page cache, autovacuum, or a configurable page size in the SQLite sense). Stubs returning SQLITE_INTERNAL is wrong because the call sites in pragma.c, vacuum.c, etc. expect a sensible return. The fix is to return a safe and sometimes meaningful default. This converts 16 stubs to intelligent NO-OPs: SetSpillSize SetMmapLimit SetPagerFlags SetPageSize GetPageSize MaxPageCount SecureDelete GetRequestedReserve SetAutoVacuum GetAutoVacuum Checkpoint IncrVacuum SetVersion ConnectionCount GetReserveNoMutex HeaderSizeBtree check-in: 092d4d9ac5 user: danshearer tags: trunk | |
| 14:05 | Made SAVEPOINT work on LMDB backend. SQLite's native btree backend rolls back at the page level via the rollback journal. LMDB's rollback unit is a transaction, so we open MDB_txn at SAVEPOINT BEGIN, abort it at ROLLBACK TO, commit it into its parent at RELEASE. SQLite WAL is lower level, so savepoint behaviour is the same whether journalling or WAL. check-in: 1f1c4b73e9 user: danshearer tags: trunk | |
| 13:41 | Update for new benchmark commands (-average, -compare etc) check-in: 04cd997a03 user: danshearer tags: trunk | |
| 13:40 | Add -average , -compare , -group-by and the TSV (tab-separated values) format. TSV is native to Unix and generally easier and safer. check-in: 958dfddfa0 user: danshearer tags: trunk | |