Features, bugs and improvements
- improve the example recipes. lumoSQL is great, but need some diverse illustrations
- implement
not-fork lintthat checks recipes (see also grammar in next section). This will save some grief from human errors - revisit the comparator. Now we know it is good (so that people wrote papers about it not knowing it already existed in not-forking) perhaps we can think about families of versions. Different packaging ecosystems have their own schemes so we could just say "we're using python-style" even if we are actually using Go. That way we can give intercal (for example) its own scheme within which it can go nuts. There's nothing like this in the literature... it's somewhat flattering to be compared to PURL and libversion but we do it better.
- Improve fragment-patch, because in 100-200 lines we should be able to replace the unreliable, undocumented patch(1) specification and algorithm for the specific case of applying our kind of fragments (which are anchored by regex's , making them more reliable to start with).
- Make cache concurrent. The not-forking cache is a major part of its benefit for non-trivial codebases with a few variants and/or upstreams. At present concurrent access within a single controlled run is difficult-to-impossible, and concurrency between multiple runs is definitely impossible.
- Make cache robust to failures of
fossil revert, orgit checkoutfailures on a dirty tree. - Make the offline cache just as good/native as the online. At present
- Make it possible to seed a cache (no matter whether offline or online) from a given URL. Especially useful for when the URL is
file://my/directory/repo, letting us seed from a normalgit cloneorfossil clonetree. Some likelynot-forkingcandidate users have huge trees. - not-fork version_filter cannot cleanly express "exclude the 1.x line" against the LumoSQL LMDB recipe. LumoSQL uses a hack,
< 1.0-alphain upstream.conf to exploit the not-forking convention that pre-release suffixes sort below the bare version:1.0-alphais the lowest known pre-release token, so comparing against it excludes everything else including any 1.x releases. This is a not-fork bug but not urgent for LumoSQL now this hack is in place.
Strategic stuff
This should probably use some of the vocabulary from the literature, now we know that eg Conradi & Westfechtel have given names to these concepts after we implemented them
- (easy) Create a schema for the lockfile (eg TOML or similar), which records a snapshot in time of what the current answer to "latest 3.47.x release of SQLite" is, which will be 'commit abcd12345'. This will mean other tooling can integrate with not-forking straight away. This is the reproducible bit from the human intentional bit in the recipe.
- (easy) Create 'not-fork hash' command or similar which gives a content addressable identifier for the tuple (recipe + lockfile + upstream commit ID), or something like that, at which point we can compute results without building them, and we have also implemented Nix.
- Create a grammar for Claudio's not-forking specification document
- Create a very simple second implementation of the grammar that will do some basic operations to show that the grammar appears to work and it has at least two users
Not really not-forking
- When run on many VMs and behind many kinds of strict proxies, Fossil fails while git sails. Unforunately this is a problem when populating a giant cache. The following 2 line patch to Fossil solves the problem. Maybe it should be discussed upstream:
--- a/src/http.c +++ b/src/http.c @@ -153,8 +153,9 @@ }else{ zPath = g.url.path; }
- blob_appendf(pHdr, "%s %s HTTP/1.0rn",
- blob_appendf(pHdr, "%s %s HTTP/1.1rn", nPayload>0 ? "POST" : "GET", zPath);
- blob_appendf(pHdr, "Connection: closern"); if( g.url.proxyAuth ){ blob_appendf(pHdr, "Proxy-Authorization: %srn", g.url.proxyAuth); }
This is just embarassing. Especially when we can't even fetch sqlite from sqlite.org, which is seemingly the most canonical task for Fossil.