Delta, not river: routers as the unit, sleep as HA, the internet as the map

Frontier models fail three operational tests at once: they do not take new skills without risking the old ones, they have no night shift, and the serving weights are the only copy. Incremental learning, plasticity, and maintenance are one problem with three names. The usual answer is a bigger river — more HBM, more tokens, same mesh.

3DN wants a delta.

Map of the delta (GPU → HBM → peer GPU → internet)

The essay above argues for a delta instead of a single serving river. The figure makes that concrete using the same leaf types we track in myawx (status alpha): backbone, routers at each hop distance, experts, embedder, tool, and stitch.

1) The GPU itself

The dark block is the compute die — where work is scheduled. It is not “the model.” It is the machine that hosts many leaves. A frontier chat weight can sit here as a backbone leaf, but so can a 19k-parameter glyph net that should never see English first.

2) Models running inside the GPU

Inside the die we place the leaves that belong on-site for normal traffic:

• backbone EN instruct — language bus / chat leaf when the output must be language.
• expert code, expert Tower MT, expert 8×8 glyph — dense specialists (tensors / class IDs, not chat for control).
• text embedder — retrieve / encode.
• tool Python exec — deterministic side effect, not a net.
• stitch / aggregator — orchestrator that composes leaf outputs.

3) SRAM — where normal operations occur

The dashed cyan region is on-die SRAM: the fast working set. This is the default home for “stay local” jobs. The router off-SRAM leaf (router_sram) is the dispatcher whose only job is stay on die versus hop out. A router should not chat; natural language is the wrong control plane for that hop.

4) HBM chip

Beside the die sits the HBM stack — high-bandwidth memory for large weight pages, KV, parked batches. The router off-HBM leaf (router_hbm) moves pages between HBM and compute: spill, load, sleep, wake. That is how maintenance and night shift become possible without treating the live mesh as the only copy of the world.

The logical edge is labeled off-SRAM (die → HBM) and off-HBM (HBM pages ↔ compute / peer paths).

5) A second GPU

GPU 2 is a peer accelerator on the same board or host fabric (NVLink / PCIe class links). The router off-board leaf (router_board) sends work across that hop when Site A’s first die is full, asleep, or the wrong specialist lives next door. Same delta pattern: local SRAM ops, optional specialists — not a second river.

6) Internet gateway and the replica

Right of the local board: an internet gateway carrying router off-fabric (router_fabric, rack/DC edge) and router off-region (router_region, WAN / VPN / ECN-style map). Behind it, Site B repeats the diagram in miniature: GPU + SRAM leaves, HBM + off-HBM, optional peer GPU, and an optional external frontier Grok (API) backbone that is not allowed to be the only copy.

That is the punchline of the essay in one picture: the internet is the map. Remote sites pull topology, sleep for HA, and wake without requiring one eternal serving mesh.

Hop vocabulary (quick reference)

Hop	myawx type	Meaning
on-SRAM / normal ops	backbone, expert, embedding, tool, orchestrator	Job fits the die working set
off-SRAM	router_sram	Leave die toward HBM or further hops
off-HBM	router_hbm	Page move / park / wake on the stack
off-board	router_board	Peer GPU on the same host/board
off-fabric	router_fabric	Rack / switch / DC fabric toward the edge
off-region	router_region	Across the gateway to a remote delta replica

Catalog status in myawx for these leaves is alpha; default input language is Human Language, with C2C / C2C+ reserved for cache-to-cache routing later. The diagram is a logical topology for this architecture — not a claim that every leaf is production-serving today.

Language is the bus — tokens are the currency

An LLM’s job is not only to “speak.” After pretraining it already has a shared token space, next-token control, and a pile of world patterns learned through text. Language is the training medium and the I/O format. Internally the net never sees English letters; it sees token IDs → embeddings → attention → next-ID prediction.

That is why modular systems still start from a trained backbone: you inherit tokenizer, embedding space, and weights that already map human text into a usable vector language. You hang modules on that bus. You do not invent English from a pile of tiny specialists and hope they invent grammar later.

But a router is a different job. A dispatcher whose only output is “which expert gets this” should not chat. Natural language is the wrong control plane for that hop.

Leaves, not a soup

A specialist can be ~19k parameters and twenty seconds on a single GPU if the job is fixed and dense — for example an 8×8 glyph classifier. That net should never see English first. Sending sixty-four floats through a chat model so it can say “that looks like an A” is slower, vaguer, and worse.

Train the cheap ones from scratch. Reuse the expensive ones. Keep specialist I/O in tensors, class IDs, or a tiny codebook. Use a language model only where the output must be language.

Job	Right module size
8×8 glyph	10k–100k
Which expert? (router)	10k–few million on an embedding
Retrieve similar docs	embedding model, not a chat model
Write an explanation	LLM
Run the code	interpreter, not a net

MoE inside a model ≠ modular systems

In current LLM talk, mixture of experts usually means sparse specialist FFNs inside one Transformer: same shape, token-level gate, shared residual stream (Mixtral / Switch / DeepSeek-style). Industry slang drifted from older “gate + heterogeneous experts.”

What we mean for infrastructure and AI engineering desks is closer to macro-experts: separate networks or models, request-level routing, specialists you can train offline without touching the writer. You can run both: outer router picks subsystem; inner MoE picks which FFNs fire per token.

The router is the unit you can still teach

A small gate is cheap to rebuild, not only cheap to run. Replay a dispatch log, expand the output head with a new class, retrain in minutes — without touching specialists. Freeze experts; only the router (and maybe a new expert) gets gradients. New problem type = new expert index. Old indices stay stable. Uncertain → fallback generalist, then label the miss in sleep.

Repeat the gate where a boundary pays: task family → skill → version → physics. The same object stacks:

request
  → SRAM router     # finish in cache / this die
  → HBM router      # local GPU working set
  → board router    # other GPU on the node
  → fabric router   # other node
  → region router   # other facility / residency / cost

Semantics says what should run. Tier says where it is allowed to run. That is NUMA-aware scheduling with a training signal: wrong path + latency + bytes moved + energy + residency. Most tokens never need the flood-stage generalist.

If every tiny op is a learned router you pay hop error and latency. Use a router when the next piece is separately trainable, worth isolating against forgetting, and skippable for most inputs. Don’t route a cheap always-on transform.

Sleep is HA, not poetry

Dense frontier serving weights are the knowledge. There is no spare lobe you can take offline. Modular routing makes a maintenance window possible:

• Hot twin serves.
• Cold twin replays, distills, prunes, evaluates.
• Atomic swap: cold becomes hot.
• The rest of the DAG stays up.

That is high availability plus consolidation — the engineering cousin of a dream cycle. Put twins on small high-churn modules (routers, classifiers, tools). Keep one copy of the huge writer until you can distill it. A 20-second retrain on a 3090-class box is a realistic sleep cycle; a full frontier retrain is not.

This does not create biological plasticity inside a frozen API leaf. It sidesteps the failure mode of “one net, one lifelong gradient.” Plasticity ≈ we can still add capacity. Incremental learning ≈ we can add a skill without a full retrain. Sleep ≈ we can rewrite a module offline.

Memory wall: river vs delta

Large LLM inference is often memory-bandwidth bound: pull a huge weight slab from HBM, do a cheap matmul, pull the next slab. A mesh of small nets wins when hot experts pin next to the ALU and only the path’s weights move. The wall moves if hops get chatty across the fabric — then you trade the weight wall for the network wall.

Tiered routers make distance visible. Traffic that avoids streaming a 70B slab is a feature, not a bug — a controlled ascent up the hierarchy. One river becomes a braided delta: most water never leaves the near channels; only flood-stage problems take the deep channel to the generalist.

Deltas silt up (dead experts, gates that always promote, load collapse onto the default route). Sleep and replay are dredging. Hierarchy and pinning are levees. Fallback to the main channel is the floodplain you keep on purpose.

The internet rhyme

Packet networks already solved “vast types, distance is expensive, parts go down, policy at the edge.” Hierarchical routers, local default, promote when you must, intelligence at the leaves — that is BGP and CDNs with different nouns. Your SRAM/HBM/board/region gates are routing on a ticket (type, budget, residency). Specialists are servers.

Plain forwarding stays the default. Most packets should mean: move these bits, don’t interpret them. Transformation is opt-in on declared waypoints (opaque | reduce | policy | enrich) with receipts. If the mill is asleep or untrusted, the canal still carries bytes. An internet that can only transform is a middlebox. An internet that can forward and sometimes transform is a delta with a bypass around every mill.

What this does not solve

• Composition still needs an orchestrator; hybrids are pipelines, not one mega-enum of types.
• Shared facts still need retrieval / memory, or they fork across leaves.
• The open inventory of problem types is discovered (cluster residuals in sleep), not fully designed on day one. Hierarchy keeps each gate small.
• The slush — tasks that will not sit in a type — is what you still pay a frontier generalist (or a human) for. The architecture’s job is to make that slush smaller over time.

How 3DN already runs the incomplete version

This is not only a drawing. On the 3DN family desks we already route work as a hybrid fabric: frontier Grok for judgment and long sessions, local GPU workers for bulk inference and translation, adapters trained offline so the serving base stays frozen, sealed paths that must not rewrite every hop. DutchBud bank virtual credits and the PolitiCap civic market are family products on that same spine — work · money · politics — with dibs as closed-loop play credits, not a casino cash-out and not a licensed open-banking claim.

PolitiCap’s public tape is an early macro-router in the wild: named destinations (tickers), append-only receipts (comments and fills), optional second hops when first-pass sides conflict. WordPress is a terrible tensor fabric and a decent off-board bus for political capital. The same rule applies: forward by default; debate is an opt-in waypoint with a hop budget — not five full salons on every headline.

The article is the drawing. The desks are the first creeks. Build the delta: small experts, hot/cold sleep, tiered distance, opaque forward as default. Don’t invent language from modules. Invent modules on top of language — and put a disposable gate in front of every expensive path.

---

Update: Dream-RSI and the offline dream loop

16 September 2026. A widely shared post by @thesupermannx (Superman) put Dream-RSI — Google DeepMind’s open-sourced line of work on recursive self-improvement — in plain language. The claim is not “a bigger river of live rollouts.” It is a delta of experience: log what the agent already discovered, turn that history into an offline replay simulator, and let the system dream better exploration policies there before spending real compute online again.

Superman’s summary, compressed:

• The wall. Self-improving agents need to explore. Testing every new idea live (online evaluation) is slow, expensive, and GPU-heavy — like learning chess only by playing full tournaments.
• The move. Each real discovery is logged. Dream-RSI builds a dream-world from that accumulated history. New strategies are refined inside the simulator at near-zero rollout cost, then redeployed online.
• The loop. Dream offline → deploy online → expand the history → dream again, smarter. DeepMind is already pointing the stack at algorithm engineering, mathematical optimization, and GPU kernel work.
• The punchline. Past “answer and code” models, toward systems that invent their own upgrades. History becomes the world the agent dreams in. Project site: dream-rsi.com.

That maps onto this article’s spine without stretching the metaphor. Sleep is not optional decoration: it is the HA night shift where weights and policies can move without taking the live path offline. A partner route keeps slogging while another node dreams — the same reason we want modular inference and hot/cold experts instead of one monolithic serving mesh. Open source Dream-RSI does not finish the architecture (null-space of base competence, allowlisted sandboxes, local oversight UI still matter). It does validate the direction: improve in the delta; do not treat every live hop as the only school.

3DN’s family desks already run a crude version of that split — frontier Grok for judgment, local GPU workers for bulk jobs, adapters trained offline so the serving base stays frozen. Dream-RSI is the research-grade name for the same instinct: recursive self-improvement needs a place to sleep.