Sharding, Gateway & Execution (s200/shard, s200/gateway, s200/executor, s200/unit-metrics, s200/shard-dev)
Route policies, shard declarations, generated gateway configs, supervised shard execution, and per-unit observability.
Shard (s200/shard)
Route policies and shard declarations as pure data — gateway generators, dev dispatchers, and executor supervisors all consume the same ShardSpec shape, which is what keeps their outputs consistent with each other.
import { policy, shardSpecs, shardApp, matchShard } from 's200/shard';
policy(app, 'GET', '/admin/:path*', { timeoutMs: 30_000, streaming: true });
const specs = shardSpecs(app, [
{ id: 'api', prefix: '/api', policy: { timeoutMs: 5_000 } },
{ id: 'admin', prefix: '/admin', entry: './shards/admin.ts' },
{ id: 'main', prefix: '/' },
]);
const adminApp = shardApp(app, { id: 'admin', prefix: '/admin' });
const serving = matchShard(specs, '/api/users'); // longest-prefix dispatch| Function | Meaning |
|---|---|
policy(app, method, pattern, p) | Annotates every route registered for method + pattern with a RoutePolicy; no-op when nothing matches, a second call replaces the first |
getPolicy(route) | The policy attached to a route, when policy set one |
shardSpecs(app, groups) | Resolves ShardGroup declarations against the actual route table → readonly ShardSpec[]; specs keep declaration order, routes keep registration order (deterministic generation). Duplicate group ids throw |
shardApp(app, group) | Cuts one group's routes out of the app as a new App (pure data transform — the source app is never mutated) |
matchShard(specs, pathname) | Longest-prefix dispatch over specs — the seam gateway config, the dev dispatcher, and any custom router share; returns the winning spec or undefined |
RoutePolicy: { timeoutMs? (response deadline hint), memoryMb? (executor resource limit), maxConcurrency? (scheduler cap — a group's merged policy takes the max), streaming? (generated proxy config disables buffering), shard? (explicit shard id override — claims the route regardless of prefix) }.
ShardGroup: { id (DNS-safe label — becomes upstream/worker names), prefix ('/'-rooted; '/' declares the catch-all), policy? (group-level base policy), entry? (executor module path/URL for thread/process supervision) }.
ShardSpec: { id, prefix, routes: readonly RouteTableEntry[] (the s200/route-table entry shape), policy (group + route policies merged, always stamped shard: id) }.
Membership rule: an explicit policy.shard: id claim decides (claimed-elsewhere routes stay behind); otherwise the prefix subtree. A parent shard may carry routes a nested shard also serves — longest-prefix dispatch keeps their traffic separate.
Gateway (s200/gateway)
Generated gateway configuration from ShardSpec[] — three text views, all literal-prefix semantics (^~, PathPrefix, path-pattern), all prefixed # generated by s200/gateway. Policy mapping: timeoutMs → proxy_read_timeout, streaming → proxy_buffering off; a shard without the policy field simply omits the directive. Specs whose prefix is not an absolute path throw, naming every offender.
| Function | Options | Output |
|---|---|---|
nginxConf(specs, opts?) | NginxConfOptions: socketDir? (default /run/s200, unix sockets ${id}.sock) or portBase? (shard i → 127.0.0.1:(portBase + i)) | nginx upstream blocks + location ^~ <prefix> server blocks |
gatewayRoutes(specs, opts?) | GatewayRoutesOptions: namespace?, gatewayName? (default s200), hostname? | Kubernetes HTTPRoute manifest (Gateway API) as YAML |
albRules(specs, opts?) | AlbRulesOptions: listenerArn?, targetGroupArns: Record<shardId, arn> (every shard needs one — a rule that forwards nowhere fails here, at generation) | AWS ALB listener rules as JSON |
Executor (s200/executor)
Supervised shard execution: inline keeps everything in-process, thread/process run the entry in a child realm with increasing isolation, external marks a unit this supervisor neither starts nor owns.
import { runShards } from 's200/executor';
const supervisor = runShards([
{ spec: apiSpec, executor: { kind: 'thread', entry: './shards/api.ts' } },
{ spec: mainSpec, executor: { kind: 'inline', app } },
]);
await supervisor.start();
console.log(supervisor.units());
await supervisor.stop();runShards(plans) → Supervisor:
| Member | Meaning |
|---|---|
start() | Boots every owned unit: process units fork immediately; thread units only pre-bind their port (a paused listener) — the worker thread spawns lazily on the first connection, so idle shards cost zero threads. Inline units are the caller's |
stop() | Drains every owned unit within STOP_GRACE_MS and force-kills whatever ignores it |
units() | Snapshots UnitStatus[]: ` |
ShardPlan: { spec, executor: Executor, port? } — explicit port override; omitted, thread/process units draw from the 31000+ counter. Executor: { kind: 'inline'; app } \| { kind: 'thread'; entry } \| { kind: 'process'; entry } \| { kind: 'external'; address }. UnitState: 'spawning' | 'ready' | 'draining' | 'stopped' | 'failed' | 'external'. Unexpected child exits restart with exponential backoff (100ms doubling, capped at 5s); a stable period resets it.
The thread hand-off is dual-mode: on runtimes that can transfer server handles (node ≥ 26) the pre-bound listener and its parked sockets move into the worker in one postMessage (zero proxying); on older runtimes the worker serves a private port instead and the supervisor pipes accepted sockets to it byte-for-byte — same lazy spawn, same first-request guarantee, the supervisor never parses HTTP. A worker that dies before adopting its sockets destroys the parked ones so their clients fail fast instead of hanging.
Also exported: installShardResolution(), appFromEntry(entry), createShardServer(app), startShardWorker(...), runThreadShard(...) — the child-realm machinery runShards drives (a thread/process unit re-imports this module as its runtime) — plus WorkerLaunch ({ mode?: 'lazy' | 'eager'; port? }), the supervisor→worker launch message.
Unit metrics (s200/unit-metrics)
One point-in-time reading of a unit. All counters are cumulative since the last createUnitMetrics call or reset(); eventLoopUtilization is the 0..1 busy ratio between the previous snapshot and this one, so a supervisor polling at its own cadence gets utilization over exactly that interval.
import { createUnitMetrics, unitMetricsEndpoint } from 's200/unit-metrics';
const m = createUnitMetrics({ maxConcurrency: 64 });
use(app, m.middleware); // register early — before routes
get(app, '/metrics', unitMetricsEndpoint(m)); // JSON (default)
get(app, '/metrics', unitMetricsEndpoint(m, { format: 'prometheus' })); // Prometheus text 0.0.4createUnitMetrics(options?) → { middleware, snapshot(): UnitMetrics, reset(): void }. Register middleware early so every request — handled, 404, or errored — is counted on both entry and exit; a request that errored still counts as completed. UnitMetricsOptions: { maxConcurrency? (the concurrency the *dispatcher* announced — shapes queueDepth), warmAfter? (default 1) }.
UnitMetrics: { inFlight, maxInFlight (the peak the unit actually absorbed), requests (completions, not arrivals), queueDepth (max(0, inFlight - maxConcurrency), always 0 when no cap was announced), eventLoopUtilization, rss, heapUsed, warm (sticky: true once requestsreachedwarmAfter, until reset()), startedAt }.
unitMetricsEndpoint(m) (or unitMetricsEndpoint(m, { format })): a handler serving snapshot() — as JSON by default, or as Prometheus text exposition with format: 'prometheus'. Mount it before or after m.middleware, it stays correct either way (unwrapped it reports totals; wrapped it honestly shows itself as the one request currently in flight). The prometheus flavor is served with Content-Type: text/plain; version=0.0.4; charset=utf-8 — point a Prometheus scraper at the same /metrics route.
prometheusText(snapshot): the pure stringifier behind that flavor — renders a snapshot as Prometheus text exposition format 0.0.4: # HELP/# TYPE lines plus one gauge per field, no labels, deterministic order (the UnitMetrics declaration order). Metric names are s200_unit_* with camelCase fields snake_cased (inFlight → s200_unit_in_flight); byte measures carry the _bytes base-unit suffix (rss → s200_unit_rss_bytes, heapUsed → s200_unit_heap_used_bytes), startedAt is rescaled to unix seconds (s200_unit_started_at_seconds), and warm maps to 0/1. The format has no NaN/Infinity literals — a non-finite reading degrades to 0 rather than poison the scrape. PROMETHEUS_CONTENT_TYPE is the exported content-type constant.
Dev dispatcher (s200/shard-dev)
Local development over shards: createDispatcher pairs each ShardSpec with the app that backs it (typically cut out of the monolith with shardApp) and returns a plain (request) => Promise<Response> doing longest-prefix dispatch — the same seam matchShard and the generated nginx config share. fallback is the app for pathnames no shard prefix claims (in production the gateway's catch-all upstream plays that role).
import { createDispatcher, dispatchPlan } from 's200/shard-dev';
const dispatch = createDispatcher([
{ spec: apiSpec, app: apiApp },
{ spec: adminSpec, app: adminApp },
], fallbackApp);
const res = await dispatch(new Request('http://localhost/api/users'));The specs snapshot is taken once at creation, mirroring how a generated nginx config is a fixed artifact: re-registering routes on a shard app still works (apps are live), but adding shards means building a new dispatcher. dispatchPlan(specs) is the pure data view of the decision table — shards ordered longest-prefix-first, for tests and docs (render it next to the generated nginx map and the two must tell the same story).