bd init: initialize beads issue tracking
ci: run the test suites against a real postgres
Not one of this repository's 28 test files had ever run on the builder: the
manifest went from cache_restore straight to abuild, and options="!check"
meant abuild did not run them either. Every apk published so far was built
from code CI compiled and never executed.
A postgres task brings up a database in the VM and exports DOLTSRHT_TEST_PG;
a test task runs make vet and make test. Measured on this tree, db/ has 10
tests and 9 of them gate on that DSN, so without it the whole persistence
layer is skipped and the build is green regardless; no other package needs
it. Hence the guard: an empty DSN fails the task loudly rather than skipping
every database suite. That guard is what makes !check an honest claim about
where the suites ran instead of a licence to ship untested code, and the
APKBUILD now says so.
make vet and make test rather than bare go commands, because the Makefile is
where -tags gms_pure_go and CGO_ENABLED=0 are named — without the tag either
command pulls go-icu-regex in and wants ICU headers the builder lacks.
remoteapi/integration_test.go prefers DOLTSRHT_TEST_PG and only falls back to
docker run when it is empty, so it never reaches for a daemon the builder
does not have. It is moot either way: the file is behind //go:build
integration and go test ./... does not compile it, as with spike.
No gofmt gate: web/beads.go and four test files are gofmt-dirty on master, so
one would be red on arrival.
ci: export the version instead of sed-ing the APKBUILD
The version task rewrote pkgver= in the tracked APKBUILD before abuild ran.
Go decides the vcs.modified half of the build stamp it records in every
binary from git status --porcelain, so that one sed made every binary in
the apk call itself <sha>-dirty for the life of the package. Measured on
go1.26.5. PKGVER now reaches abuild through ~/.buildenv and the APKBUILD
reads pkgver=${PKGVER:-0.0.0}; a local abuild builds an honest 0.0.0.
One git describe now decides it — a tag, else tag_git<n>, else the family's
0.0.<commit count> — and refs/tags/v* joins allow-refs so a tag builds.
The export alone is not enough: an untracked file at go build time sets the
same flag, and abuild works in this checkout. .gitignore takes /src/, /pkg/
and /tmp/, the last being the one abuild points GOTMPDIR at while
MAKEFLAGS=-j runs three go builds that can see each other's work dirs.
Around it, the shape the siblings converged on: cacher installed through
install.sh (checksum-verified, and the raw curl it replaces had no -f, so a
404 body was chmod +x-ed) in two tasks, since its PATH export lands in
~/.buildenv and only the next task sources it; scss through cacher --exec,
whose seed upload is best-effort, instead of an if-block that made an S3
hiccup fatal and a genuine S3 error indistinguishable from a miss;
--optional and the module-cache repair block in cache_restore; cache_save
after publish, without the cacher exists guards an upload already implies;
and publish gated on a readable ~/.apk-ci.env so a secretless submission
reports what it built instead of failing.
go mod download and NOT go mod download all: all resolves dependencies'
test dependencies and appends their hashes to go.sum, which is a tracked
file and therefore the very -dirty stamp this commit removes. On this tree
it added 426 lines, 671 -> 1097; plain download leaves go.sum untouched and
go mod verify still passes.
APKBUILD gains GOFLAGS=-trimpath -modcacherw, without which the binaries
embed builder paths and the module cache is packed read-only into the cache
tarball that the next build then cannot unpack. package() calls the new
install-files, which does not build: abuild runs it in a fresh process that
never called build(), so make install recompiled all three binaries from a
cold cache and shipped a copy nothing had checked. The bare ls of the
stylesheet becomes make check-css, which counts — web/router.go takes the
first glob match, so two stylesheets are as wrong as none. The three binary
targets lose their if [ -d ./cmd/... ] guards, which now only serve to turn
a deleted cmd/ into a green build that packages nothing.
The manifest is a varchar(16384) and over the cap a branch gets no CI at
all, so the reasoning moved to docs/ci.md and the tasks carry pointers.
deps: tidy after the ecore uplift
chrome: link the favicon through the shared head partial
The icon href was a literal in the layout, so a build with no static tree — a
test, a binary run out of a working copy — asked for /static/logo.svg once per
page and got a 404 each time. It is the chrome's FaviconHref now: our own logo
when this build ships one, checked the way the stylesheet already was, and
NewService's built-in data: URI when it does not. Both <link>s come from
srht-head-links, so the guard against <link href=""> is written once.
Also pins what the repo-list partial's optional fields do for a service that has
no timestamp in its schema: a card with no muted footer, not "0001-01-01".
pages: read forms through FormValues
Every mutation on this surface read its fields with r.PostFormValue after a bare
r.ParseForm. PostFormValue was already the right half — r.Form would merge the
query string into the body, which would let a mutation be driven from a URL
somebody was linked to, and that request is exactly the one the same-origin
guard sees nothing wrong with — but the body was bounded only by net/http's
10 MiB default, on pages anyone can reach. FormValues is both properties in one
call, and the values are passed down explicitly instead of each sub-handler
reaching back into the request.
chimw: take the chi helpers from ecore
Three things this service did not have. Read routes are registered for HEAD as
well as GET, so `curl -I` and every uptime probe stop being answered with a 405
and a kilobyte of error page; the twin shares the handler, so it cannot say 200
where the GET says 404. chi's two routing failures now render our own page
instead of net/http's plain text — an unrouted URL here was the one refusal on
the instance that did not look like the service it came from. And the request
line is a slog record rather than chi's colourised line on stdout, which was the
only line this daemon emitted that was neither structured nor on stderr.
chi's own middleware package loses the chimw alias to the package written
against; it is chimiddleware now, as ecore's package doc asks.
instconf: take the origin and required-key helpers from ecore
Two copies of one function disagreed in this repo: hostFromOrigin returned an
error for a malformed origin and web's hostOf answered "localhost", which is a
guess that looks like an answer. Both are gone; the caller now names which half
it means, and both wanted OriginAuthority — a port is part of a sealed-URL host,
a JWT audience and the synthesized commit-author domain alike.
The startup checks become one Require, so an operator filling in a fresh
config.ini reads every missing key off one boot instead of one per restart. The
hook's internal-origin read becomes InternalOrigin, which falls back to the
external origin: an instance with only a public address is not misconfigured and
used to be refused. And the git-description mirror is wired only when git.sr.ht
has an API origin — web.Config already documented a nil Git as no mirroring, but
nothing produced one, so an instance without git.sr.ht met config.GetAPI's panic
on the first push.
logging: take the log policy from ecore
setupLogging's level parser, its os.Stderr.Stat colour probe and its three-key
mask list were one of six copies. Defaults(conf, section) resolves all of it now
and the handler stays here, which is the split that package documents.
Three things follow from taking the instance's list instead of this service's
own: the mask set gains the config-file private keys and the migration DSN that
siblings had already learned to redact, NO_COLOR is honoured, and verbosity can
be set for one run with $LOG_LEVEL or -d. [dolt.sr.ht]log-level is unchanged.
login: take the unified-login cookie decode from ecore
authn's CookieName, the fernet decrypt, the auth.AuthCookie unmarshal and the
empty-name check were one of six copies of the same decode on this instance.
They are now sr-ht-ecore/login.UsernameFromRequest; what stays here is the half
that is ours, turning that name into a row in our user table.
Two things the local copy did not do. It passed the cookie's name through with
a leading '~' still on it, which meta answers for nobody, and it validated
nothing at all — a name went from an attacker-supplied cookie straight into a
GraphQL query and a log line. Both are now login's, and the middleware's own
rule is unchanged: every failure is anonymity, so public browsing and public
clones keep working.
internalauth: take both ends of the internal protocol from ecore
The guard on /internal/repos and the header cmd/dolt-git-hook minted for it
were two hand-written halves of one protocol in two packages that shared no
type, no constant and no test. Both are now sr-ht-ecore/internalauth: Guard on
the receiving end, AuthorizationAs on the calling one, over one Auth struct.
The guard also pins the caller, which the old copy did not: core-go only asks
that a token name some client and node, and on an endpoint that provisions a
database for an arbitrary user that means any holder of the network key will
do. The pinned pair lives in core so the mint and the pin cannot drift apart.
The hook test now runs internalauth.Identify — the real receiving end — over
the header the hook produced, so the two ends are checked against each other
rather than against a third copy of the decode.
deps: bump sr-ht-ecore and auxilia to v0.7.0
log: take the logrus bridge from auxilia
The bridge was written here because remotesrv.ServerArgs.Logger demands a
*logrus.Entry and nothing else, and letting it log around our handler meant
its records skipped the masks. None of that is specific to dolt or to
SourceHut, so it now lives in auxilia beside scribe, where the next library
that demands a logrus entry can reach it.
log: bridge dolt's remotesrv logger into slog
remotesrv takes a *logrus.Entry and nothing else, so passing nil left it
writing through logrus' standard logger: its own format, its own stream,
and no mask between a field named token and the journal. It is the half
of this process that serves clones and pushes — the likeliest place for
a credential to reach a log field, and the one that was logging around
everything the previous commit configured.
internal/logrusbridge hands it an entry whose only exit is a
logrus.Hook: output to io.Discard, a formatter that produces nothing,
and the logrus level left wide open so the slog handler does the
filtering from the one setting in config.ini. Fields cross as attributes
rather than a formatted blob, which is what lets a mask keyed on the
attribute path fire at all.
Verified against logrus v1.9.3 rather than assumed: Entry.log fires
hooks before it writes, before Logger.Exit and before the panic, so a
Fatal or Panic record reaches slog before the process ends. Both arms
are tested.
The package takes nothing from this service and belongs beside scribe in
auxilia; it is here because it was needed here first.
web: take the login redirect from chrome.LoginURLFor
Building a whole Page resolved the nav, the brand and the profile link
for a response that is a Location header and nothing else.
log: replace logrus with slog behind auxilia's scribe handler
Every logger field this service owned was a *logrus.Entry threaded
through a constructor, which is what logrus costs for want of a usable
default. They are slog.Default().With("component", ...) now, and the
threading is gone with them; the shared middleware's panic reports land
in the same handler, which is why the daemon sets the default before
anything that can fail.
The handler is scribe's tint handler: level from [dolt.sr.ht]log-level,
source positions, and masks keyed on the attribute path for the three
credentials this service handles — the unified-login cookie, the
Internal fernet token and the Authorization header the remotesapi reads
a PAT or a keypair JWT out of. Errors go through scribe.Err, so a culpa
error's hint reaches the operator on its own line.
logrus stays in go.mod: dolt's remotesrv.ServerArgs takes a
*logrus.Entry and nothing else. It is now confined to Config.DoltLogger,
which is the only place this service names it.
dolt-git-hook is deliberately untouched: what it writes to stderr is the
notice a pushing user reads through git, not a log.
deps: bump sr-ht-ecore for slog panic reports
middleware reports a panic through slog's default logger now, with the
method, path, panic value and stack as attributes rather than one
formatted line. It logs through the default, so this service has to set
one — which the next commit does.
test: build the fixture config and the keyset with ecoretest
The hand-built ini in web_test.go, the random fernet key in authn's
TestMain and the same seeding copied into the git-hook test are one call
to ecoretest now. The keys are fixed rather than generated on purpose:
they secure nothing inside a test process, and a constant keyset is what
lets two packages of this service initialise without the second rotating
what the first sealed with.
The synthetic instance runs in production mode, so the environment
banner is off in tests unless one asks for it.
web: guard mutations with ecore's csrf, cache and panic middleware
checkSameOrigin and originMatches are gone, and with them the three
per-handler calls that had to be remembered: csrf.Require sits over the
whole browser group, so the mutating route added next year is guarded by
being routed. The internal provisioning endpoint stays outside that
group deliberately — it is a service-to-service POST with no Origin and
its own network-key guard.
middleware.PrivateCache marks every page as one no cache may reuse for
the next viewer, which is only correct because the static handler opts
out per asset once it has found the file. RecoverPanics answers a panic
with the error page, and one that arrives after the response has started
by dropping the connection rather than appending an error to half a
document.