~bigbes/sr-ht-dolt

sr-ht-dolt/Makefile -rw-r--r-- 8.4 KiB
12c7ff77 — Eugene Blikh 2 days ago
ci: publish this build's own coverage and benchmarks

The board and detail projections in beads/ and the memory markdown render in
web/ are what a request costs here — there is no SQL engine behind a bare
store, so every table is read whole and projected in process. This repository
had no Benchmark at all; those two paths now have one each, driven through the
seams that already exist (a canned BrowseSession, an index built by the real
PrefixesAcross), so what is timed is the projection and not the disk.

The manifest gains a coverage and a bench task, both after publish so a
rejected upload cannot cost a shipped apk, and both guarded: the profile must
be non-empty, the benchmark names must be in the file (an empty benchfmt body
uploads and reports success), and a build handed no token says so and exits 0
with the artifact still attached.
ec98df31 — Eugene Blikh 9 days ago
ci: export the version instead of sed-ing the APKBUILD

The version task rewrote pkgver= in the tracked APKBUILD before abuild ran.
Go decides the vcs.modified half of the build stamp it records in every
binary from git status --porcelain, so that one sed made every binary in
the apk call itself <sha>-dirty for the life of the package. Measured on
go1.26.5. PKGVER now reaches abuild through ~/.buildenv and the APKBUILD
reads pkgver=${PKGVER:-0.0.0}; a local abuild builds an honest 0.0.0.

One git describe now decides it — a tag, else tag_git<n>, else the family's
0.0.<commit count> — and refs/tags/v* joins allow-refs so a tag builds.

The export alone is not enough: an untracked file at go build time sets the
same flag, and abuild works in this checkout. .gitignore takes /src/, /pkg/
and /tmp/, the last being the one abuild points GOTMPDIR at while
MAKEFLAGS=-j runs three go builds that can see each other's work dirs.

Around it, the shape the siblings converged on: cacher installed through
install.sh (checksum-verified, and the raw curl it replaces had no -f, so a
404 body was chmod +x-ed) in two tasks, since its PATH export lands in
~/.buildenv and only the next task sources it; scss through cacher --exec,
whose seed upload is best-effort, instead of an if-block that made an S3
hiccup fatal and a genuine S3 error indistinguishable from a miss;
--optional and the module-cache repair block in cache_restore; cache_save
after publish, without the cacher exists guards an upload already implies;
and publish gated on a readable ~/.apk-ci.env so a secretless submission
reports what it built instead of failing.

go mod download and NOT go mod download all: all resolves dependencies'
test dependencies and appends their hashes to go.sum, which is a tracked
file and therefore the very -dirty stamp this commit removes. On this tree
it added 426 lines, 671 -> 1097; plain download leaves go.sum untouched and
go mod verify still passes.

APKBUILD gains GOFLAGS=-trimpath -modcacherw, without which the binaries
embed builder paths and the module cache is packed read-only into the cache
tarball that the next build then cannot unpack. package() calls the new
install-files, which does not build: abuild runs it in a fresh process that
never called build(), so make install recompiled all three binaries from a
cold cache and shipped a copy nothing had checked. The bare ls of the
stylesheet becomes make check-css, which counts — web/router.go takes the
first glob match, so two stylesheets are as wrong as none. The three binary
targets lose their if [ -d ./cmd/... ] guards, which now only serve to turn
a deleted cmd/ into a green build that packages nothing.

The manifest is a varchar(16384) and over the cap a branch gets no CI at
all, so the reasoning moved to docs/ci.md and the tasks carry pointers.
1dc49e2e — Eugene Blikh 11 days ago
apk: ship dolt-git-hook as a -hook subpackage

The hook binary was the one piece of this repo not in the apk — the
deployment's Dockerfile.git cloned the repo and compiled it from source
at a separately pinned revision (SRHT_DOLT_HOOK_REV), which meant a
second version pin to keep in lockstep, a build-time dependency on the
git host, and a full Go toolchain stage in the git image rebuild.

Add dolt-git-hook to the Makefile's BINARIES (same guarded target
pattern) and split it into a dolt.sr.ht-hook subpackage: the git.sr.ht
container needs only this 9 MB binary, not the 126 MB doltsrht service
the main package carries. The deployment can now apk-add the subpackage
at the same pinned version as the service.
8ed47c67 — Eugene Blikh 30 days ago
build: pure-Go (CGO_ENABLED=0) build via a klauspost-backed gozstd shim

dolthub/dolt/go pulls in two hard cgo dependencies — go-icu-regex (SQL
REGEXP) and gozstd (NBS zstd compression) — which forced a C toolchain +
ICU headers on every build. Both are now avoided so the default build is
pure Go and statically linkable:

- ICU: build with `-tags gms_pure_go`, selecting go-mysql-server's stdlib
  regexp fallback. Safe because this service never runs the SQL engine
  (it serves bare NBS stores and browses read-only), so it never
  evaluates SQL REGEXP.
- zstd: `replace github.com/dolthub/gozstd => ./third_party/gozstd-purego`,
  a pure-Go drop-in over klauspost/compress/zstd (already in the graph).
  It reproduces the nine gozstd symbols dolt references. dolt is
  unmodified.

dolt uses gozstd only in its NBS archive subsystem; this binary hits only
the decompress side at runtime (archive dictionary TRAINING is gc/
archive-writer code we never run — the shim implements it over klauspost
but panics on the trainer errors that only that off-path use could
trigger). zstd frames and dictionaries are standard-format, so
libzstd-authored archives decode correctly; the shim's tests prove this
by decoding plain and dictionary-compressed frames produced by the zstd
CLI (libzstd).

The Makefile now defaults to CGO_ENABLED=0 + -tags gms_pure_go (override
with `make CGO_ENABLED=1 GO_TAGS=` for the cgo variant). Verified:
CGO_ENABLED=0 build of ./..., all unit tests, the real-dolt-CLI
integration + spike suites, and the shim's libzstd-interop tests, all
green with no cgo.
ca9357a0 — Eugene Blikh 30 days ago
foundation: module, core domain, schema, spike-verified dolt interop

Phase 0 foundation for dolt.sr.ht (pure-Go Dolt hosting for SourceHut).

Dependency pins:
- core-go v0.0.0-20260520082310-fdb3662452dc: the production instance's
  submodule commit (fdb3662); must never be upgraded so token/config/crypto
  behavior matches the rest of the instance.
- dolthub/dolt/go v0.40.5-0.20260626152440-45335d44ad79: pseudo-version at the
  commit tagged v2.1.10 (45335d44), matching the installed dolt CLI. The /go
  submodule's latest tag (v0.40.4, 2021) does not interop with a modern CLI;
  matching the CLI commit guarantees a common NBS format (Format_DOLT) and
  remotesapi proto.
- gopkg.in/go-jose/go-jose.v2 v2.6.3: same JOSE version dolt's creds package
  signs EdDSA keypair JWTs with, avoiding a duplicate JOSE lib.
- grpc v1.79.3, logrus v1.8.3, lib/pq v1.10.9, chi/v5 v5.3.1, brant v0.5.1.

Contents:
- core/ pure domain: models, ValidateName/ParseRepoPath, the access matrix
  (Allowed + NotFoundForPrivate), full table-driven tests.
- schema.sql + migrations/0001_initial.sql (brant format).
- config.example.ini, contrib/dolt.sr.ht.conf (nginx), Makefile, scss/main.scss,
  static/logo.svg, README, .gitignore.
- internal/smoke: throwaway import-and-build check (deleted in Phase 3).

Spike (storage/spike_test.go, tag `spike`): bare NBS store via WriteEmptyRepo,
served by remotesrv on an ephemeral single port, round-tripped through the real
dolt CLI v2.1.10 (clone -> insert -> commit -> push -> re-clone -> verify rows).
Green, no version juggling needed beyond selecting the v2.1.10 commit. FS must
be rooted at the repos dir (LocalFilesysWithWorkingDir) so sealed chunk-URL
prefixes are clean relatives.

Build note: dolt pulls go-icu-regex (CGO), which needs ICU4C headers; on this
Mac set CGO_CPPFLAGS/CGO_LDFLAGS to icu4c@78 (documented in README).