# @aweftjs/modules Modules from a directory, a bundle or a document, loaded in dependency order with their dependencies injected. Tools to load and unload. No opinion about which, when, or for whom. **Loading a module runs its code**, with the privileges of the process that loaded it. Nothing here isolates anything. Load what you trust, or put your own boundary in front of this. ## Quickstart ```ts import { createLoader, fromDocument } from '@aweftjs/modules'; import { fromDirectory } from '@aweftjs/modules/node'; const loader = createLoader({ sources: [fromDirectory('./modules'), fromDocument(plugins)], props: { store }, }); const { 'posts/Create': create } = await loader.load(['posts/Create']); await loader.unload('posts/Create'); ``` `load` instantiates what you name and everything it depends on, dependencies first, and hands back what you named. `unload` lets go of exactly one module, calling its `stop` if it has one. **`props` is for what the platform hands in**, such as the store an application made before there was a loader; anything the application itself makes is a module that others name in `deps`. `@aweftjs/server` builds its loader that way and hands in `store` alone. ## A module A module is a file, a bundle entry or a document entry that exports: ```ts export const deps = ['auth/Session', 'lib/Log']; // what it needs, by name export const defaults = { maxLength: 80 }; // its configuration when nothing configures it export default ({ imports, config, extensions, store }) => ({ make: (title) => { imports.Log.log(imports.Session.userOf()); return title.slice(0, config.maxLength); }, stop: () => { /* let go of whatever this holds */ }, }); ``` The factory receives `imports`, `config`, `extensions`, and whatever props the loader was made with. `imports` is keyed by the **last segment** of each dependency's name: `auth/Session` is `imports.Session`. Two dependencies whose names end the same way are refused at `load`. An **extension** is a same-named entry from another source that exports `config` or `extensions` and no factory. Its `config` merges over the implementation's `defaults`, plain objects one level at a time, arrays and everything else replaced whole. When several sources carry one name, the earliest source's implementation wins and every source's config contributes, earliest winning. **The load order is dependency order, then the order you listed.** A module loads after everything it depends on; between modules that do not depend on each other, the one whose source is earlier in `sources` loads first, and within one source the one listed first. A module in several sources takes the place of the source that implements it, so a file that only configures a library module does not move it. A directory lists sorted, a bundle in the order of its map, a document sorted. `loaded()` is that order, and a server walks it: the first module to answer a request no route matched is the answer, so a module that answers everything goes in the last source. Whatever the factory returns is the instance. If it has a `stop` function, `unload` calls and awaits it. Nothing else is read off an instance. ## Where modules come from A **source** lists candidates and evaluates nothing until `load` asks for a name. | source | names | evaluates by | |---|---|---| | `fromDirectory(path)`, on `@aweftjs/modules/node` | the file's path under `path`, no extension: `auth/Session` | `import()` of the file | | `fromBundle(map, { prefix })` | the key without `prefix` (or a leading `./`) and its extension | the entry, or calling it when it is a function | | `fromDocument(document, { compile })` | the document's keys | `compile(entry.source)` | `fromDirectory` is on its own subpath because it reads the filesystem; the main entry loads in a browser. `fromBundle` takes the two shapes a bundler's glob import produces, eager or lazy. `fromDirectory` hands its path to `node:fs`, which resolves a relative one against the process's working directory rather than the file that called it, so `'./modules'` finds nothing when the program is started from anywhere else. Build an absolute path from `import.meta.url`: ```ts const here = fileURLToPath(new URL('.', import.meta.url)); const loader = createLoader({ sources: [fromDirectory(join(here, 'modules'))] }); ``` **A module document** is an observable object whose keys are module names and whose values carry a `source` string. Every other field in an entry is yours: an author, a note, a list of earlier versions. The loader ignores them. ```ts const plugins = createObject({ 'plugin/Shout': createObject({ source: 'export default () => ({ ... })', author: 'ada' }), }); ``` That document is an ordinary document. Open it through a store and it is persisted. Share it over a link and a second node loads the same modules. Its commit history is its version history. This package does none of that and knows none of it: it reads `source` and nothing about how it got there. ## Compile `compile(source)` turns module text into exports. The default imports the text as an ES module through a data URL, with no dependency. If your modules need a transform, pass your own as `fromDocument(document, { compile })`. `compile` is exported on its own so you can check a source before you store it: ```ts try { await compile(draft); } catch (error) { /* refuse the draft */ } ``` Nothing here does that for you. The default keeps every distinct source in the runtime's module cache for the life of the process. [`bench/compile.ts`](https://github.com/torrinworx/aweft/blob/9a5bb24770dc7555257f8a307d79917efc58df70/bench/compile.ts) measured it on Node 25: about 5.6 KB of heap per distinct source, 53 MB for 10,000, and the same text imported twice is one module. An application that compiles many versions of many modules supplies a compile that does not keep them. ## The tools | | | |---|---| | `load(names)` | instantiate these and what they need; returns the named instances | | `unload(name)` | call `stop` if present and drop the instance; true if it was loaded | | `loaded()` | the loaded names, in the order their factories finished, always a dependency order | | `get(name)` | a loaded instance, or undefined | | `dependencies(name)` | what a loaded module declared, or undefined | | `dependents(name)` | the loaded modules that depend directly on it | `unload` unloads exactly the named module. A loaded module that depends on it keeps the reference it was handed, and will use it; ask `dependents` and unload those first if that is what you want. A `load` that fails part way leaves what it already instantiated loaded and throws naming the module that failed. A loader is an instance. Make one per tenant, one per module, or one for everything; two loaders share nothing. Every error this package raises carries a `reason` you can branch on and the `module` it is about: `missing` (in no source, or gone from it while loading), `no-implementation`, `duplicate` (one source lists a name twice, such as `thing.js` beside `thing.ts`), `invalid-name`, `cycle`, `ambiguous-import`, and `failed` (with `cause`). A `stop` that throws is the module's own error and reaches you from `unload` as it was thrown. **Declare dependencies in `deps`; do not load them from inside a factory.** A cycle written in `deps` is refused by name. A factory that calls `load` for a module whose own factory is waiting on this one cannot be told apart from an ordinary concurrent load, so it waits forever. The loader has no way to see who called it. ## Following a document Nothing here reloads a module on its own. If you want a loader's modules to track a document, say so: ```ts const stop = follow(loader, plugins, { failed: (name, error) => log.warn(name, error), applied: (name, action) => log.info(`${name} ${action}`), // 'reloaded' or 'unloaded' }); ``` While following, a loaded module whose entry's `source` changes is unloaded and loaded again, together with its loaded dependents: dependents first on the way out, dependency order on the way back. A loaded module whose entry is removed is unloaded with its dependents. Anything else does nothing: a changed field other than `source`, an entry nothing has loaded. Reloads run one at a time, in the order the changes landed. A reload happens after the commit that changed the source, not during it, so the loader has not caught up the moment your assignment returns. `applied` is told when it has, with the module's name and what happened to it; by then the new instances are in place. A reload that fails goes to `failed` instead, or is raised where nothing catches it when you gave no handler, and the process reports it as uncaught. One change failing never stops the follow: a module whose `stop` throws is reported to `failed` and the reload still lands, a handler that throws is reported the same way, and the next change is followed as usual. `stop()` stops following. ```ts const caughtUp = new Promise((resolve) => { const stop = follow(loader, plugins, { applied: () => { stop(); resolve(); } }); }); plugins['plugin/Count'].source = newSource; await caughtUp; ``` ## What this package never decides Which modules load, when, how many, for how long, in which process, or at what rate. Who may read, write, load or run one. Where a document comes from, whether anything persists it, or what its history is. Whether the code is safe to run. ## Reading on The design notes are in [`docs/design/`](https://github.com/torrinworx/aweft/tree/9a5bb24770dc7555257f8a307d79917efc58df70/docs/design) 061 to 065. The contract this package is built to is the one an application's own modules are written to. ## API Every export of `@aweftjs/modules`, its signature as the compiler resolves it, and its block comment. ### `@aweftjs/modules` #### `BundleEntry` ```ts type BundleEntry = ModuleExports | (() => Promise); ``` One entry of a bundle map: the exports, or a function that loads them. #### `Candidate` ```ts interface Candidate { readonly name: string; exports(): Promise; } ``` One module a source can hand over: its name, and the way to get its exports when asked. #### `Compile` ```ts type Compile = (source: string) => Promise; ``` Turns module text into its exports. The default is `compile`; an application may pass its own. #### `Factory` ```ts type Factory = (props: ModuleProps) => unknown; ``` Builds a module's instance. May return a promise. Whatever it returns is the instance. #### `FollowHandlers` ```ts interface FollowHandlers { readonly failed?: ((name: string, error: unknown) => void) | undefined; readonly applied?: ((name: string, action: 'reloaded' | 'unloaded') => void) | undefined; } ``` No block comment on this export. #### `Loader` ```ts interface Loader { load(names: readonly string[]): Promise>>; unload(name: string): Promise; loaded(): readonly string[]; get(name: string): unknown; dependencies(name: string): readonly string[] | undefined; dependents(name: string): readonly string[]; } ``` A loader: the loaded graph and the tools that change it. Every method is about this loader alone. Two loaders share nothing. #### `ModuleExports` ```ts interface ModuleExports { readonly deps?: readonly string[]; readonly defaults?: Readonly>; readonly config?: Readonly>; readonly extensions?: Readonly>; readonly default?: Factory; } ``` What a module exports. An implementation exports `default`, usually `deps`, and sometimes `defaults`. An extension exports `config` or `extensions` and no `default`; its contribution merges into the module of the same name from another source. One file may do both. #### `ModuleProps` ```ts interface ModuleProps { readonly imports: Readonly>; readonly config: Readonly>; readonly extensions: Readonly>; readonly [key: string]: unknown; } ``` What a module's factory receives. The three named fields win over anything of the same name in the loader's props, which are spread in first. #### `ModulesError` ```ts interface ModulesError extends Error { readonly reason: string; readonly module: string; } ``` An error this package raises, with a reason a caller can branch on. Reasons: `missing` (a name is in no source, or left it while it was being loaded), `no-implementation` (only extensions for it), `duplicate` (one source lists a name twice), `invalid-name` (a path that leaves no name), `cycle`, `ambiguous-import` (two dependencies share a last segment), `failed` (a factory threw; `cause` carries what it threw). #### `Source` ```ts interface Source { candidates(): Promise; } ``` Somewhere modules come from. Listing evaluates nothing. #### `compile` ```ts compile: Compile ``` The default compile: import the text as an ES module through a data URL. **Params** - `source`: the text of a module, exporting what a module exports **Returns** its exports. The source runs with this process's own privileges, the same as any `import()`; nothing here isolates anything (design 065). Every distinct source stays in the runtime's module cache for the life of the process: [`bench/compile.ts`](https://github.com/torrinworx/aweft/blob/9a5bb24770dc7555257f8a307d79917efc58df70/bench/compile.ts) measured about 5.6 KB of heap per distinct source on Node 25 (10,000 sources, 53 MB), and the same text imported twice is the same module. An application that compiles many versions of many modules passes its own compile to `fromDocument` instead. **Example** ```ts const { deps, default: factory } = await compile(entry.source); ``` #### `createLoader` ```ts createLoader: ({ sources, props }: { sources: readonly Source[]; props?: Readonly> | undefined; }) => Loader ``` Make a loader. **Params** - `sources`: where modules come from, in order of precedence; the first source with an implementation of a name wins, and every source's configuration for it contributes - `props`: spread into every factory's props, under `imports`, `config` and `extensions` **Returns** a loader with nothing loaded. **Example** ```ts const loader = createLoader({ sources: [fromDirectory('./modules'), fromDocument(plugins)] }); await loader.load(['posts/Create']); ``` #### `follow` ```ts follow: (loader: Loader, document: object, handlers?: FollowHandlers) => () => void ``` Keep a loader's loaded modules matching a document. **Params** - `loader`: the loader whose loaded modules follow the document - `document`: the same object handed to `fromDocument` - `handlers.failed`: where a reload failure goes - `handlers.applied`: told when a reload or an unload has finished, by module name **Returns** the function that stops following. While following: a loaded module whose entry's `source` changes is unloaded and loaded again, together with the loaded modules that depend on it, dependents first on the way out and dependency order on the way back. A loaded module whose entry is removed is unloaded together with its loaded dependents. A change to any other field of an entry, or to an entry nothing has loaded, does nothing. Reloads run one at a time, in the order the changes landed, and `applied` hears each one as it finishes. **Example** ```ts const stop = follow(loader, plugins, { failed: (name, error) => log.warn(name, error), applied: (name, action) => log.info(`${name} ${action}`), }); ``` #### `fromBundle` ```ts fromBundle: (map: Readonly>, options?: { readonly prefix?: string | undefined; }) => Source ``` A source over a bundle map. **Params** - `map`: path to entry, in the shape a bundler's glob import produces, eager or lazy - `options.prefix`: the leading part of every key to drop; without it, a leading `./` is dropped **Returns** a source whose names are the keys without that prefix and without the file extension, so `./modules/auth/Session.js` is `auth/Session` under `prefix: './modules/'`. **Throws** a `ModulesError` with reason `invalid-name`, at once, for a key that leaves no name once its extension is removed. **Example** ```ts const source = fromBundle(import.meta.glob('./modules/**\/*.js'), { prefix: './modules/' }); ``` #### `fromDocument` ```ts fromDocument: (document: object, options?: { readonly compile?: Compile | undefined; }) => Source ``` A source over a document. **Params** - `document`: an observable object whose keys are module names and whose values carry a `source` string; other fields are yours and are ignored - `options.compile`: what turns a source into exports; the default imports it as an ES module **Returns** a source. It reads the document each time it is listed, so a module added to the document is a candidate on the next `load`, and `exports()` compiles the source as it is at that moment. **Throws** a `ModulesError` with reason `missing`, out of the `load` that is using it, when an entry leaves the document between being listed and being compiled. **Example** ```ts const plugins = createObject({ 'plugin/Shout': { source: 'export default () => ({ ... })' } }); const loader = createLoader({ sources: [fromDocument(plugins)] }); ``` ### `@aweftjs/modules/node` #### `fromDirectory` ```ts fromDirectory: (path: string) => Source ``` A source over a directory tree. **Params** - `path`: the directory. A relative path is resolved by `node:fs` against the process's working directory, not against the file that called this, so a program run from anywhere else finds nothing; pass an absolute path built from `import.meta.url`, as the example does **Returns** a source over every `.js`, `.mjs` and `.ts` file under the directory (never a `.d.ts`), named by the file's path relative to the directory with `/` separators and no extension, so `/auth/Session.ts` is `auth/Session`. Listing walks the tree; evaluating a candidate is `import()` of its file. Two files that would share one name, `thing.js` beside `thing.ts`, are refused by the loader as `duplicate`. **Throws** a `ModulesError` with reason `invalid-name`, out of the `load` that lists it, for a file whose name is nothing but an extension. **Example** ```ts const here = fileURLToPath(new URL('.', import.meta.url)); const loader = createLoader({ sources: [fromDirectory(join(here, 'modules'))] }); ``` ## Refusals Every error this package raises carries a `reason` to switch on and a `fix` that says what to do. These are its reasons, from `errors.txt`. | reason | fix | |---|---| | `ambiguous-import` | Rename one of the two modules so their last segments differ. | | `cycle` | Drop one of the deps, or move what the two share into a third module. | | `duplicate` | Keep one file per module name in a source. | | `failed` | Fix what the factory threw; the error carries it as its cause. | | `invalid-name` | Give the file a name before its extension. | | `missing` | Add the module to one of the loader sources, or fix the name. | | `missing` | Do not unload a module from a factory while a load is running. | | `missing` | Leave the entry in the document until the load is over. | | `missing` | Load the dependency first, or list it in the module deps. | | `no-implementation` | Give one of the files for this name a default export, the factory. | ## Recipes The programs in the stack's gate that use this package, each a job someone would have. - [`recipes/full-stack`](/docs/recipes/full-stack): A page and a server in one directory: the page reaching the server in development through the dev server's proxy, and a sign-in that sets the cookie on that one origin - [`recipes/translated-site`](/docs/recipes/translated-site): A site written in one language and launched in three: the build finds every string and writes the catalog an agent fills, each language is a tree of pages with the plural rules of its own, a stored act is compiled where it runs, and the Ukrainian page hydrates in place - [`recipes/static`](/docs/recipes/static): A generated site served by the stack's own server: one process is the whole deployment, and the page still comes alive where it stands - [`recipes/health`](/docs/recipes/health): A deploy's verification: the health endpoint polled until the shipped build is the one answering, and the two states a poll must not mistake for health - [`recipes/logs`](/docs/recipes/logs): A page recorded end to end in a browser and the visit read back: an error, a rejection, a console line, a failed call on both sides, a commit's shape with a private slot absent, a typed character never stored, sign-in mid-visit - [`recipes/uploads`](/docs/recipes/uploads): A page uploads pictures under the gate and they paint from `/files/`; each refusal reaches the page with its reason, a module makes a file of its own, and the static battery behind it never sees a file - [`recipes/notify`](/docs/recipes/notify): Two pages of one user hear a send live and mark it read for each other, a device registered from the page, email and push against two fakes, a failed mail kept, a forged write refused, a restart, and a server with no store sending a contact form's mail - [`recipes/posts-to-pages`](/docs/recipes/posts-to-pages): Pages written while the application runs: a post published over a socket becomes a page, and a scheduled full write refreshes the sitemap - [`recipes/backend`](/docs/recipes/backend): The boot pattern to copy: a twelve-line boot file and a folder of modules, one holding a document, one the rules, one a scheduler, one the gate, and one configuring a battery module - [`recipes/modules`](/docs/recipes/modules): the package's own recipe