# @aweftjs/logs The logs battery: what a page and the server did, per visit, in the application's own store. A crash, a bad state, a regression in one account: the visit that hit it is a document you read back, by hand or with an agent. `@aweftjs/logs/client` is the browser half that records the page; the readers this package exports are for any process, an agent or a job among them. A visit is one page from load to close. Nothing records until the page calls `createLog`, and nothing is read but through the store. ## Quickstart The server side is three modules and the two store declarations they and the readers query: ```ts import { auth, paths as authPaths } from '@aweftjs/auth'; import { logs, paths as logPaths } from '@aweftjs/logs'; import { fromDirectory } from '@aweftjs/modules/node'; import { createServer } from '@aweftjs/server'; import { node } from '@aweftjs/server/node'; import { createStore, memoryDriver } from '@aweftjs/store'; const store = createStore({ driver: memoryDriver(), declare: { ...authPaths, ...logPaths } }); const server = createServer({ sources: [fromDirectory('./modules'), logs, auth], store, gate: 'auth/Gate', listener: node({ port: 8080 }), }); await server.start(); ``` The page wraps the client it already has, and uses the one the log hands back everywhere else: ```ts import { createClient } from '@aweftjs/client'; import { createLog } from '@aweftjs/logs/client'; const log = createLog(createClient({ url }), { build: BUILD_SHA }); const client = log.client; // share and ask through this from here on client.share('board'); // its commits are recorded as shape log.write({ kind: 'checkout', step: 3 }); // the page's own record ``` That is the whole wiring. From then on the page's errors, its console at error and warn, every shared document's commits, every ask, the connection status, the URL, and clicks, keys and submits are recorded, and the server's calls, routes, refusals and failures are recorded beside them in the same visit. ## What is recorded **On the page**, by `createLog`: | kind | what it holds | |---|---| | `error`, `rejection` | an uncaught error or a rejected promise: message, and stack when there is one (cut at 8000 characters on the page, and the first thing to go under the server's byte cap) | | `console` | a `console.error` or `console.warn` the page made: `level` and `message` | | `commit` | a shared document changed: the `topic`, the `paths` touched, the delta count, the bytes; never a value | | `refused`, `fault` | a share the server refused a write to, or a topic that faulted | | `ask` | a call the page made: `name`, `ms`, `ok`, and the reason when it failed; never the args or the result | | `status` | the connection went connecting, open or closed | | `url` | the router's URL changed (when a router is handed in) | | `input` | a `click`, `keydown` or `submit`: a target descriptor, and for a key, the key when it is a named one (`Enter`, `ArrowLeft`), never what a layout produces for a character | **On the server**, by `logs/Observe` through the `observe` hook (design 260): a `call` (`name`, `ms`, `ok`, and the reason when it failed), a `request` (`method`, `path`, `status`, `ms`), a `refused` commit, a `failed` hook, and the connection opening and closing. A call's args and result are recorded only for a module whose instance carries `logs: true`; otherwise their byte size. A request's body is never recorded. **Once per visit**: the user (from the gate), the build, and the browser (the UA string as it is, the `userAgentData` brands and platform where the browser offers them, the viewport, screen, pixel ratio, colour scheme, reduced motion, language and touch). ## The batches the page sends The client half posts to one route; you never call it by hand, but this is its shape, for a test or a recorder of your own. A batch is JSON, and identity comes from the request's cookie through the gate: ``` POST /api/logs { "visit": "", "build"?: "...", "browser"?: { ... }, "ended"?: true, "entries": [ { "at": 1789170542372, "kind": "error", "message": "..." }, ... ] } 200 { "kept": 3 } 400 { "reasons": [ ... ] } the body is not a batch 429 { "reasons": [ ... ] } over a cap ``` `build` and `browser` are read once, from the first batch that carries them; `ended` stamps the visit's end. An entry is any flat object with a `kind`; `at` defaults to now. There is no read route: reading a visit is the readers below, in a process you trust. The client sends one batch at a time, each at most `batch` entries (500) and under 48000 bytes, because a keepalive request and a beacon may carry 64 KiB at most in flight and the browser refuses the send over it. If you lower `logs/Visits`'s `batch`, pass the same `batch` to `createLog`, or the route answers 429 to a full batch and the page drops it. ## What is never recorded No IP address. No typed value. No character key, and no key at all from a password or hidden field. No page text. No route body. No commit value. **No value under a leading-underscore slot**: the recorder watches a shared document through core's `skip(Infinity)`, which never delivers a delta whose path passes through an object slot whose key starts with an underscore (design 259), so a `_password` or `_ssn` slot is carried as a commit's shape and never its value, with nothing to configure. On the server, a call's args and result are carried only when the called module says `logs: true`, the way `public: true` is a word the gate reads. This is why a diagnostics recorder is lawful with a privacy-policy line and stays out of the territory that gets session-replay vendors sued: it records that things happened and their shape, not their content. ## The store, and reading it back Each visit is `visit:`, each server run `process:` (a fresh one when the last fills), an ordinary document with the fields above and an `entries` list. Read one back with the readers, which take the store and run in any process: ```ts import { errors, prune, visit, visits } from '@aweftjs/logs'; const seen = await visit(store, id); // one timeline, entries in time order const theirs = await visits(store, { user, since }); // matches, newest first; each id is `visit:` const top = await errors(store, { since }); // messages grouped, most seen first await prune(store, Date.now() - 30 * 86_400_000); // drop what is older than 30 days ``` `postgres/views.sql` flattens the documents into `aweft_log_visits` and `aweft_log_entries` for a reader that speaks SQL, such as a Metabase. Apply it by hand against the store's database; the battery never runs it. ## Configuring it Each module is configured the way any battery module is (design 240): a same-named file exporting `config` in a source before this one. `logs/Visits` holds the caps and the retention, generous by default: ```ts // modules/logs/Visits.ts export const config = { keep: 30, // days a document is kept; the sweep runs on start and hourly build: process.env.BUILD_SHA ?? null, batch: 500, // entries a batch may carry; tell createLog the same when you lower it entry: 4096, // bytes an entry may take; over it the stack, args, result, paths and reasons go first, then the message is cut perVisit: 10_000, // entries a visit may hold; the last is capped and the rest are dropped batchesPerMinute: 60, visitsPerMinute: 600, sweepMs: 3_600_000, // and idleMs: at most 2147483647, what a timer can hold }; ``` The process document has the same `perVisit` cap and rotates at it: once the next entry would be its sentinel, a fresh `process:` takes over and the full one is swept in its time. An application that wants its own retention runs `prune` from a `jobs` row instead of, or beside, `keep`. `logs/Record`'s one setting is `public`, true by default so an anonymous page may post; set it false for a route that needs a signed-in user. ## What it never decides **Who may read.** No module answers a read over the wire, because "allowed" differs in every application. An application that wants a route or a call over a reader writes the module and its gate rule, the way it writes any private module. **Whether to record.** Nothing records until the page calls `createLog` and `logs` is in `sources`. **Retention beyond the defaults**, and any per-address cap: the route is handed a request and a context, never the peer, so the caps here are per visit and per process. A gate that puts the address in its context gives an application what it needs to cap by address in a module of its own. ## Known limits - **No replay.** A commit is recorded as its shape, not its values, so a visit is read, not re-run. Recording came first on purpose; replaying a visit against the real page is a later step that adds the values back through the same private-slot wildcard. - **Server output written with `console.log` inside a module is not attributed to a visit.** A module that wants a line in the record calls `logs/Visits`'s `write` (name it in `deps`). Per-module console capture would need an async-context wrapper around every hook and is not here. - **The process document is held open until it fills or the process ends.** `prune` from outside with a cutoff later than the process started removes it under the module; the module's own sweep skips it. - **A second library that replaces `console.error` after this one wins.** The page puts the console back on `stop`. The design notes are 259 (the `skip(Infinity)` wildcard), 260 (the `observe` hook on `server`) and 261 (this battery). ## API Every export of `@aweftjs/logs`, its signature as the compiler resolves it, and its block comment. ### `@aweftjs/logs` #### `Batch` ```ts interface Batch { readonly visit: string; readonly build?: string | null; readonly browser?: Readonly>; readonly ended?: boolean; readonly entries: readonly Entry[]; } ``` What the page sends: one visit's entries, with what is known once per visit when it is. #### `Entry` ```ts interface Entry { readonly at: number; readonly side: 'page' | 'server'; readonly kind: string; readonly [field: string]: Primitive | undefined; } ``` One thing that happened, flat: `at`, `side`, `kind`, and the fields of the kind. #### `ErrorGroup` ```ts interface ErrorGroup { readonly message: string; readonly kind: string; readonly build: string | null; readonly count: number; readonly visits: number; readonly firstSeen: number; readonly lastSeen: number; } ``` One message, seen `count` times across `visits` visits on one build. #### `Primitive` ```ts type Primitive = string | number | boolean | null; ``` A value a slot of an entry may hold. #### `VisitRecord` ```ts interface VisitRecord { readonly id: string; readonly kind: 'visit' | 'process'; readonly user: string | null; readonly build: string | null; readonly browser: Readonly> | null; readonly startedAt: number; readonly endedAt: number | null; readonly errors: number; readonly entries: readonly Readonly>[]; } ``` A visit or process document as plain data, entries in time order. #### `VisitSummary` ```ts interface VisitSummary { readonly id: string; readonly user: string | null; readonly build: string | null; readonly startedAt: number; readonly errors: number; } ``` A visit as the index holds it, without opening it. #### `errors` ```ts errors: (store: Store, filter?: ErrorFilter) => Promise ``` The errors seen, grouped by message, kind and build, most seen first. **Params** - `store`: the application's store - `filter`: by start time or build; `limit` is how many visits are read, 200 by default **Example** ```ts for (const group of await errors(store, { since })) console.log(group.count, group.message); ``` #### `logs` ```ts logs: Source ``` The three modules, for `sources`: `logs/Visits` keeps the documents, `logs/Record` answers `POST /api/logs`, `logs/Observe` writes what the server did (design 261). **Returns** the source. Put the application's own source first and a module of the same name there wins; a file of that name exporting only `config` configures it instead. **Example** ```ts const store = createStore({ driver, declare: { ...auth.paths, ...logs.paths } }); const server = createServer({ sources: [own, logs, auth], store, gate: 'auth/Gate', listener }); // modules/logs/Visits.ts, in `own`: export const config = { keep: 7, build: process.env.BUILD_SHA ?? null }; ``` #### `paths` ```ts paths: Readonly> ``` The paths the application declares on its store for these modules and the readers to query: `kind`, `build`, `startedAt` and `errors` on visit and process documents. `user` is the auth battery's declaration and the same path, so spreading both declares it once. **Example** ```ts const store = createStore({ driver, declare: { ...paths } }); ``` #### `prune` ```ts prune: (store: Store, olderThan: number) => Promise ``` Remove every visit and process document that started before `olderThan`. **Params** - `store`: the application's store - `olderThan`: a time in milliseconds since the epoch **Returns** how many were removed. A process document of a server that has run longer than that is removed under it, so prune from the module's own sweep, or later than an uptime. **Example** ```ts await prune(store, Date.now() - 7 * 86_400_000); ``` #### `visit` ```ts visit: (store: Store, id: string) => Promise ``` One visit as plain data, or undefined when there is none by that id. **Params** - `store`: the application's store - `id`: the visit's id, or a full document name such as `process:` **Returns** the document with its entries sorted by `at`. **Example** ```ts const seen = await visit(store, 'k9Q2...'); for (const entry of seen?.entries ?? []) console.log(entry.at, entry.kind); ``` #### `visits` ```ts visits: (store: Store, filter?: VisitFilter) => Promise ``` The visits that match, newest first, without opening any. **Params** - `store`: the application's store - `filter`: by user, build, start time, or having errors; `limit` defaults to 100 **Example** ```ts const theirs = await visits(store, { user, since: Date.now() - 86_400_000 }); ``` ### `@aweftjs/logs/client` #### `Batch` ```ts interface Batch { readonly visit: string; readonly build?: string | null; readonly browser?: Readonly>; readonly ended?: boolean; readonly entries: readonly Entry[]; } ``` What the page sends: one visit's entries, with what is known once per visit when it is. #### `Entry` ```ts interface Entry { readonly at: number; readonly side: 'page' | 'server'; readonly kind: string; readonly [field: string]: Primitive | undefined; } ``` One thing that happened, flat: `at`, `side`, `kind`, and the fields of the kind. #### `FetchInit` ```ts interface FetchInit { method: string; headers: Record; body: string; keepalive: true; credentials: 'same-origin'; } ``` What `fetch` is given, stated here so this declaration names no DOM type. #### `Fetcher` ```ts type Fetcher = (url: string, init: FetchInit) => Promise; ``` The one HTTP call this half makes. The global `fetch` is one of these already. #### `Listening` ```ts interface Listening { addEventListener(type: string, listener: (event: never) => void, options?: unknown): void; removeEventListener(type: string, listener: (event: never) => void, options?: unknown): void; } ``` A listener target, as the window and the document both are. #### `Log` ```ts interface Log { readonly client: Client; readonly visit: string; write(entry: Readonly>): void; each(fn: (entry: Entry) => void): () => void; flush(): Promise; stop(): void; } ``` A log over one page: the client that records, and the page's own entries. #### `LogOptions` ```ts interface LogOptions { readonly origin?: string | undefined; readonly build?: string | null | undefined; readonly router?: { readonly url: Derived; } | undefined; readonly fetch?: Fetcher | undefined; readonly flushMs?: number | undefined; readonly batch?: number | undefined; readonly window?: WindowLike | undefined; } ``` No block comment on this export. #### `Primitive` ```ts type Primitive = string | number | boolean | null; ``` A value a slot of an entry may hold. #### `WindowLike` ```ts interface WindowLike extends Listening { readonly document?: Listening | undefined; readonly location?: { readonly origin: string; } | undefined; readonly navigator?: { readonly userAgent?: string; readonly userAgentData?: { readonly brands?: readonly { brand: string; version: string; }[]; readonly platform?: string; readonly mobile?: boolean; }; readonly language?: string; readonly maxTouchPoints?: number; sendBeacon?(url: string, data: unknown): boolean; } | undefined; readonly screen?: { readonly width: number; readonly height: number; } | undefined; readonly innerWidth?: number | undefined; readonly innerHeight?: number | undefined; readonly devicePixelRatio?: number | undefined; matchMedia?(query: string): { readonly matches: boolean; }; readonly console?: { error(...args: unknown[]): void; warn(...args: unknown[]): void; } | undefined; readonly Blob?: (new (parts: string[], options: { type: string; }) => unknown) | undefined; } ``` The window as this half reads it, so a test can hand in one of its own. Every field is optional, and a missing one records nothing for that source. #### `createLog` ```ts createLog: (client: Client, options?: LogOptions) => Log ``` Record a page over a connection it already has. **Params** - `client`: the page's client; the answer's `client` is the one to use from here on - `options`: where the route is, the build, a router, and the seams a test hands in **Returns** the log. Nothing is recorded before this is called, and `stop` ends it. **Example** ```ts const log = createLog(createClient({ url }), { build, router }); const identity = createAuth(log.client); log.client.share('board'); ``` ## Refusals Every error this package raises carries a `reason` to switch on and a `fix` that says what to do. These are its reasons, from `errors.txt`. | reason | fix | |---|---| | `capped` | Send fewer, or raise the cap in logs/Visits's config. | | `invalid-config` | Give build a string, such as a commit hash, or null. | | `invalid-config` | Give that setting a number above zero. | | `invalid-config` | Give that setting at most 2147483647 milliseconds, about 24.8 days. | | `invalid-config` | Set public to true for a route any page may post to, or false for one that needs a signed-in user. | | `malformed` | Send { visit, entries: [{ at, kind, ... }] } as JSON. | | `no-store` | Pass store to createServer, or props: { store } to a loader you build yourself. | | `undeclared` | Spread paths from @aweftjs/logs into the store's declare. | ## Recipes The programs in the stack's gate that use this package, each a job someone would have. - [`recipes/logs`](/docs/recipes/logs): A page recorded end to end in a browser and the visit read back: an error, a rejection, a console line, a failed call on both sides, a commit's shape with a private slot absent, a typed character never stored, sign-in mid-visit - [`recipes/room`](/docs/recipes/room): An act module stored on the server, run in a frame on the page: the board it shares reaches the server, its ask carries the page's identity, its own stage navigates on the page's URL under the host act, an error inside reaches the page's logs, and the frame paints but cannot fetch