# lethe-web-ui-aggregates **Status:** Design **Module:** `sourcecraft.dev/bigbes/lethe` **Branch:** master **Worktree:** none **Parent RFC:** Personal AI Assistant Log Aggregator (2026-04-25) **Design source:** `docs/design_handoff_assistant_log/` (Direction 4 — Dense Data UI) **Sibling tasks:** - `lethe-server.md` (#1, ✓ Verified) - `lethe-web-ui-foundation.md` (#4, ✓ Reviewed) - `lethe-collector-claude-code.md` (#2, deferred) - `lethe-search-and-opencode.md` (#3, deferred) - `lethe-web-ui-palette-savedsearch.md` (#6, deferred) - `lethe-web-ui-search.md` (#7, blocked on #3) - `lethe-web-ui-settings-display.md` (#8, deferred) - `lethe-web-ui-health-sources.md` (#9, blocked on #2) ## Design ### Purpose Add the two backend aggregation endpoints (`/api/v1/projects`, `/api/v1/stats`) the foundation deferred, plus the three frontend screens that consume them: Projects index, Project detail, Stats. Replaces three of the foundation's stub routes (`/projects`, `/project/$`, `/stats`) with real content. **In scope**: Go aggregation queries; the new project URL pattern; chart primitives (stacked-bar, heatmap, horizontal-bar, hour-bar) inline in SVG; range/group-by sub-bar controls. **Out of scope**: Search route, Health route, Settings → Display, palette PROJECT/SESSION items, saved searches. ### Chosen approach **Backend — two endpoints, both additive**: - `GET /api/v1/projects?since=&limit=&offset=` returns rows of `{ cwd, sessions, turn_count, tokens_in_total, tokens_out_total, last_active, hosts[], tools[], top_tool }`. SQL groups by `working_dir` over the existing sessions+turns join, ordered by `MAX(ended_at) DESC`. Owner-scoped via existing auth middleware. - `GET /api/v1/stats?range=7d|30d|90d|all` returns a single fat JSON response with six cards in one round-trip: - `per_tool`: `{ tool, sessions, turns, tokens_in, tokens_out, daily_sparkline: number[N] }[]` for the requested range - `daily`: `{ date_unix, per_tool: { [tool: string]: number /*turns*/ } }[]` over the last 60 days - `heatmap`: `{ date_unix, count }[]` — 84 cells (12 weeks × 7 days), fixed window - `top_cwd`: `{ cwd, count }[]` — top 20 cwds by turn count in the requested range - `hour_of_day`: `{ hour, count }[]` — 24 buckets over the range - `host_split`: `{ host, count }[]` over the range - Both endpoints reuse the existing `apierror.Render` for error paths; both honor admin `?owner=` override exactly like sessions list. **Backend — one tiny extension to sessions list**: `GET /api/v1/sessions?cwd=` accepts a new optional filter on `working_dir`. Project detail page uses this to pull a cwd's sessions; no separate project-detail endpoint. Wire/schema unchanged. **Frontend**: - `useProjects(filters)` — TanStack Query hook for `/api/v1/projects`. - `useStats(range)` — TanStack Query hook for `/api/v1/stats`. - Routes: - `web/src/routes/projects.tsx` (replaces stub) — projects table. - `web/src/routes/project.$.tsx` (new, `$` catch-all so cwd-with-slashes works as a single URL-encoded path segment) — header card + sessions table scoped to that cwd via the new `?cwd=` filter. - `web/src/routes/stats.tsx` (replaces stub) — sub-bar with range pills (7d / 30d / 90d / all), 2-column card grid. - New chart primitives (`web/src/primitives/`): `StackedBars` (60×N stacked-bar SVG), `Heatmap` (12×7 cell grid), `HorizontalBars` (top-cwd list), `HourBars` (24-bin column chart). `Spark` from the foundation is reused for the per-tool sparkline cards. All SVGs inline; no chart library. **Project identity**: exact `working_dir` string, no canonicalization. URL is `/project/$` with a single segment carrying the URL-encoded cwd. Tradeoff: `/Users/x/proj` and `~/proj` resolved-to-the-same-place show as different projects. Acceptable for v1; canonicalization is a known limitation. ### Hands-off decisions - **udesign — stats endpoint = single fat JSON**. Six small queries server-side beat six round-trips; payloads are a few KB. - **udesign — project identity = exact `working_dir`**. Simpler, matches prototype; canonicalization needs OS-level path resolution that's out of scope. - **udesign — project detail uses `GET /api/v1/sessions?cwd=`** rather than a dedicated endpoint. Avoids duplicating the sessions DTO server-side. - **udesign — range-pill semantics are unix-seconds offsets**. `7d` = `now - 7*86400`; `all` omits the `range` param. Mirrors the existing `since` pattern on sessions list. - **udesign — `top_cwd` capped at 20**. Card has fixed height; prototype shows ~10 entries. - **udesign — heatmap window is fixed at 12 weeks (84 cells), not range-dependent**. Matches the prototype. ### Backwards-compatibility check All changes additive: - Two new endpoints; no existing consumers. - One new optional query param (`?cwd=`) on `/api/v1/sessions`; old clients ignore it and get the existing behavior. - No schema migrations. - No wire-type changes. - Frontend stub routes are replaced; no deployed user is affected. ### TDD: yes (scoped) - **Yes** for: each backend aggregate query (Go repository tests with seed data per card); the date-bucketing logic that fills missing days/hours with zeros (small Go helper, deterministic); the TS `adaptStats` adapter that pipes Go DTOs into the chart-friendly shapes. - **No** for: SVG chart primitives — visual, no logic worth testing beyond what the rendered output shows. ### Unknowns (resolve during execute) - **Catch-all `$` route compatibility with TanStack Router**: needs a quick spike — TanStack supports `$param` (segment) and `$` (full splat). Verify the splat captures slash-bearing cwds correctly and `Link` round-trips them. - **Aggregate query plan at scale**: at 10–100k turns the GROUP BY's are fine on existing PKs. If `EXPLAIN QUERY PLAN` shows a scan on `turns` for the `top_cwd` query, add a covering index `(owner, timestamp)` in the same execute. Not blocking the phase. ### Invariants - `/api/v1/projects` and `/api/v1/stats` reuse the existing auth middleware. No bypass, no special-casing. - `?owner=` admin override behaves identically across `/sessions`, `/projects`, `/stats` (all-owners when set to `*`, specific owner when set to a username, default = caller's identity). - No schema migrations; no `internal/shared/wire/` changes; no removed/renamed JSON fields. - All new HTTP error paths render `application/problem+json` via the existing `apierror.Render`. - Stats response is computed fresh per request (no server-side cache). Range parameter is the only knob; no per-tool/host filters at the endpoint level. - Project identity is exact `working_dir`; the server never normalizes paths. - All API calls in the SPA flow through TanStack Query + the existing `apiFetch` wrapper. No raw `fetch` outside `api/client.ts`. - All chart primitives are inline `` in TSX; no chart library dependency added. ### Principles - One round-trip per stats screen render. The frontend doesn't fan out to six endpoints; the server's GROUP BYs are cheap enough to bundle. - Server returns numbers; the SPA formats and colors. No HTML, no pre-rendered SVG, no localized strings on the wire. - Project detail = scoped sessions list. Resist building a parallel "project entity" model on either side. - Reuse `Spark`, `Tag`, `ToolDot`, `EmptyState`, `Sub`, `SessionsTable` from the foundation. Add chart primitives only where none of these fits. - No silent fallbacks. If a stats card's query returns zero rows for the requested range, the response includes that card as an empty array; the SPA renders an `EmptyState` per-card. No "0%" placeholders, no synthesized data. ## Plan Approach: five phases — Phase 1 ships the projects aggregate plus the `?cwd=` extension to sessions list (one commit, single backend file group); Phase 2 ships the stats endpoint with deterministic date-bucketing helpers; Phases 3-5 swap the three foundation stub routes for real screens (projects index, project detail using the new cwd filter, stats with inline-SVG chart primitives bundled in the stats commit since they are not reused). Two new Go packages mirror the `internal/domain/session/` shape: `internal/domain/project/` (Repository + Handler over `GET /projects`) and `internal/domain/stats/` (Repository + Handler over `GET /stats`). Both are registered as steward services and mounted under the existing `/api/v1` chi group alongside `Sessions` and `Ingest`. The `?cwd=` extension lives on the existing `session.Repository.List` / `session.Handler.List`. ### Phase 1 — Backend: `/api/v1/projects` + sessions `?cwd=` filter - **1.1** `internal/domain/session/repository.go:166-174` (modify) — `ListFilter` gains `Cwd *string`. `List` (lines 237-285) appends `working_dir = ?` clause when non-nil, in fixed order before `Since`/`Until` to keep deterministic clause ordering. - Invariant: query stays parameterised; column name not derived from input. Wire/schema unchanged. - **1.2** `internal/domain/session/handler.go:68-105` (modify) — `List` reads `q.Get("cwd")` and threads to `filter.Cwd`. Empty string = absent (consistent with other optional filters). - **1.3** `internal/domain/session/repository_test.go` (modify) — add `TestList_FilterByCwd`: seed three sessions with two distinct `working_dir` values + one NULL; assert `?cwd=/code/x` matches only the two with that cwd; NULL row excluded. - **1.4** `internal/domain/project/repository.go` (create) — new package `project`. Types and signatures: - `type Project struct { Cwd string; Sessions int64; TurnCount int64; TokensInTotal int64; TokensOutTotal int64; LastActive int64; Hosts []string; Tools []string; TopTool string }` with `db` and `json` tags. `Hosts`/`Tools` are populated by post-query aggregation (sqlite `GROUP_CONCAT` returns comma-joined strings; split + dedupe in Go to keep the JSON shape clean). - `type ListFilter struct { Owner session.OwnerScope; Since *int64; Limit int; Offset int }` - `func (r *Repository) List(ctx context.Context, f ListFilter) ([]Project, error)` — single SQL: `SELECT working_dir AS cwd, COUNT(*) AS sessions, SUM(turns…) ... FROM sessions LEFT JOIN turns USING (owner,tool,host,session_id) WHERE working_dir IS NOT NULL [AND owner=?] [AND started_at >= ?] GROUP BY working_dir ORDER BY MAX(ended_at) DESC LIMIT ? OFFSET ?`. `top_tool` is a correlated subquery picking the tool with the highest turn count for that cwd. - Owner clause logic mirrors `session.Repository.List` switch on `OwnerScope` (AllOwners / SpecificOwner / default). Reuse `session.OwnerScope` directly to avoid divergence. - Invariant: `?owner=` admin-override behaves identically to `/sessions`; non-admin override returns 403 at the handler layer. - **1.5** `internal/domain/project/handler.go` (create) — `Handler` steward with `Repo *Repository` (project) and `SessionScope *session.Handler` for scope-resolution reuse — actually re-implement `resolveScope` here using `auth.MustIdentity`; copy is small (≈18 lines) and avoids cross-handler coupling. - `Mount(r chi.Router)` registers `r.Get("/projects", h.List)`. Mounted by `server.go` under `/api/v1`. - `listResponse { Projects []Project; Limit int; Offset int }`. Pagination clamps via the same `clampLimit`/`clampOffset` helpers (relocate to `internal/pkg/httputil` or duplicate — duplicate, keeps each handler self-contained; <10 LoC). - **1.6** `internal/domain/project/repository_test.go` (create) — TDD targets: - empty DB → `[]` (not nil) - one cwd, two sessions, three turns total → row with `sessions=2`, `turn_count=3`, `last_active = MAX(ended_at)` - NULL `working_dir` excluded entirely - `top_tool` ties broken deterministically (smallest tool name first via `ORDER BY tool ASC` in the subquery) - `Hosts`/`Tools` deduped + sorted - owner scope: AllOwners returns rows from both owners; SpecificOwner pins to one; default = caller - **1.7** `internal/domain/project/handler_test.go` (create) — endpoint-level: seed 1 project; `GET /api/v1/projects` → 200 with row; non-admin `?owner=alice` → 403; bad `since` → 400 problem+json. - **1.8** `internal/server/server.go:54-100` (modify) — add `Projects *project.Handler` inject field; in `Init`, call `s.Projects.Mount(r)` inside the `/api/v1` Route block alongside `s.Sessions.Mount(r)`. - **1.9** `cmd/lethe/main.go` (modify) — register the new `*project.Repository` and `*project.Handler` with steward (mirrors how `session.Repository`/`session.Handler` are registered). Same for stats in Phase 2. - Commit: `project: add /api/v1/projects aggregation; sessions: add ?cwd= filter` ### Phase 2 — Backend: `/api/v1/stats` - **2.1** `internal/domain/stats/repository.go` (create) — types: - `type ToolRollup struct { Tool string; Sessions int64; Turns int64; TokensIn int64; TokensOut int64; DailySparkline []int64 }` (sparkline length = bucket count of the requested range, capped at 60) - `type DailyBucket struct { DateUnix int64; PerTool map[string]int64 }` - `type HeatmapCell struct { DateUnix int64; Count int64 }` - `type CwdRow struct { Cwd string; Count int64 }` - `type HourBucket struct { Hour int; Count int64 }` - `type HostRow struct { Host string; Count int64 }` - `type Stats struct { PerTool []ToolRollup; Daily []DailyBucket; Heatmap []HeatmapCell; TopCwd []CwdRow; HourOfDay []HourBucket; HostSplit []HostRow }` - `type Filter struct { Owner session.OwnerScope; RangeSince *int64 /* nil = all */; Now int64 /* injected for determinism in tests */ }` - `func (r *Repository) Stats(ctx context.Context, f Filter) (*Stats, error)` — runs six small queries against the `turns` table joined to `sessions` for `working_dir`/`tool` projection, fills with deterministic-window helpers (Phase 2.3) so missing days/hours appear as zero rather than absent rows. - **2.2** `internal/domain/stats/handler.go` (create) — `Handler.List` reads `?range=7d|30d|90d|all` and translates to `RangeSince` via `time.Now().Unix() - days*86400` (or nil for `all`); injects `Now = time.Now().Unix()` into Filter. Mount: `r.Get("/stats", h.List)`. `listResponse` is just `*Stats`. - **2.3** `internal/domain/stats/buckets.go` (create) — pure Go helpers, deterministic, TDD targets: - `func DailyWindow(now int64, days int) []int64` — returns `days+1` unix seconds at midnight UTC, oldest first. - `func FillDaily(window []int64, rows []DailyBucket) []DailyBucket` — left-joins `rows` onto `window`; missing days get empty `PerTool` map. - `func HeatmapWindow(now int64) []int64` — 84 cells = 12 weeks × 7 days, ending today (UTC). - `func HourWindow() []HourBucket` — 24 zero-initialised buckets. - **2.4** `internal/domain/stats/buckets_test.go` (create) — TDD targets per helper: window length, ordering, deterministic boundary at UTC midnight, fill behaviour with sparse rows. - **2.5** `internal/domain/stats/repository_test.go` (create) — TDD targets: - empty DB → `Stats` with all slices zero-length but non-nil - one tool, three turns over two days → `PerTool[0].Turns = 3`, `Daily` has the two days populated and others zero - heatmap windowed at 12 weeks regardless of `range` - `top_cwd` capped at 20 even when there are more - owner scope: `AllOwners` aggregates across owners; `SpecificOwner` excludes others - **2.6** `internal/domain/stats/handler_test.go` (create) — `?range=7d` → 200 with `Stats`; `?range=foo` → 400 problem+json; non-admin `?owner=` → 403. - **2.7** `internal/server/server.go` (modify) — register and mount `*stats.Handler` exactly like `*project.Handler` (Phase 1.8 pattern). - **2.8** `cmd/lethe/main.go` (modify) — register `*stats.Repository` and `*stats.Handler`. - Commit: `stats: add /api/v1/stats aggregate endpoint` ### Phase 3 — Frontend: Projects index route - **3.1** `web/src/api/adapters.ts` (modify, append) — add types and adapter: - `interface ProjectDTO { cwd: string; sessions: number; turn_count: number; tokens_in_total: number; tokens_out_total: number; last_active: number; hosts: string[]; tools: string[]; top_tool: string }` - `interface Project { cwd: string; sessions: number; turns: number; tokensIn: number; tokensOut: number; lastActive: string /* ISO */; hosts: string[]; tools: string[]; topTool: string }` - `adaptProject(d: ProjectDTO): Project` — last_active=0 → `lastActive=''`; otherwise ISO. - **3.2** `web/src/api/adapters.test.ts` (modify) — add `adaptProject` cases: empty arrays passthrough, last_active=0 → empty string, ISO conversion at unix=1700000000. - **3.3** `web/src/features/projects/useProjects.ts` (create) — `useProjects(filters: { since?: '7d'|'30d'|'90d'|'all' }): UseQueryResult`. Same `since`→unix translation pattern as `useSessions`. - **3.4** `web/src/features/projects/ProjectsTable.tsx` (create) — port from `proto-pages.jsx:3-32`. Columns: `cwd · sessions · tok · last · top tool · activity`. Reuses `Tag`, `ToolDot`, `Spark`. Click row → router push `/project/$`. Cursor row styling identical to `SessionsTable` (`--accent-soft` bg, 2px left border). - Signature: `ProjectsTable(props: { projects: Project[]; cursor: number; onCursor: (i: number) => void; onOpen: (p: Project) => void }): JSX.Element` - **3.5** `web/src/features/projects/useProjectsCursor.ts` (create) — clones `useHomeCursor` shape (move/activate/jumpTo). Trivially small; could be hoisted into a shared hook in a follow-up. - **3.6** `web/src/routes/projects.tsx` (replace) — drop the stub. SubBar shows ` projects · ranked by recent activity` + a single `since` FilterChip (clone the home one or inline a simple `