Sealed batches under load: Bangkok and Berlin PolitiCap ECN trials

We ran two live production trials of PolitiCap regional ECN nodes — one on a fast LAN edge (Bangkok), one on a slow WAN edge (Berlin). The goal was not marketing volume. It was to measure how MarketMaker fills, DutchBud ledger seals, batch upload, forged-seal rejection, and operator quarantine behave when the fabric is under real load.

This is a 3DN engineering write-up: numbers from our metrics store, behavior we observed, and design choices that held. It continues our earlier notes on pull-not-push regional updates and ledger-bound trade batches.

Why ECN lag still matters (a short historical detour)

Electronic communication networks are not a new idea. Equity markets spent decades learning that who sees the book first is an economic weapon. The Berlin Stock Exchange — and European venues more broadly — have long been part of debates about delayed quotes, fragmented liquidity, and abusive short-selling patterns that thrived whenever one path was slower than another. Naked shorting scandals and “fail-to-deliver” stories were never only about greed; they were about asymmetric information over time.

PolitiCap is a civic market simulation with virtual credits (DutchBud dibs), not a cash securities exchange. Still, the engineering lesson transfers: if a regional node can invent fills that the hub must trust, or if lag can be gamed into a privileged view of the tape, you have rebuilt the old ECN problem in miniature. Our trials deliberately stressed the opposite design — sealed batches only, pull-based software, and quarantine when the seal fails.

What we tested

Trial	Edge character	Load shape	Adversary
Bangkok	LAN-class path to hub	MarketMaker volume ramp mild → medium over ~1 hour	Periodic forged ledger rows in the local outbox
Berlin	WAN edge (slow software pull; kB/s-class transfers)	Short sealed MM burst (~3 minutes, 28 fills)	Same forged-seal inject; auto-quarantine path

Both nodes used the same rules: local matching stays local; syndication to the hub requires a DutchBud receipt (ledger_ref + HMAC ledger_code). The hub never accepts “because the node said so.”

Figure 1. Happy path vs forged seal. Local matching can still run; the global tape does not accept unsigned fiction.

Bangkok trial — ramp under a clean path

Setup. Two MarketMaker bots crossed a demo symbol on the Bangkok regional API. Volume ramped from a mild rate (~8 shares/min average) toward a medium rate (~40 shares/min) over about one hour, with small random spread on price and lot size so the path was not a metronome.

Seals. Every successful cross attempted a DutchBud ledger seal before the fill was eligible for the syndication outbox. On this edge, seal traffic reached the ledger service through the hub’s authenticated proxy path (the regional VLAN does not need a direct bank port). Seal success tracked the MM crosses during the ramp.

Syndication result. Hub metrics and the syndicated-trade table agreed on the headline: on the order of ~580 ledger-trusted fills from the Bangkok node landed on the hub tape during the trial window (trust label ledger). Batch upload ran on a short interval (one minute) so Grafana and the metrics store could see accept/reject counts without waiting on a long poll.

Adversary. A separate process injected outbox rows with guessed ledger references and codes (origin keys prefixed for metrics as rogue). Those rows rode the same batch channel. Hub verify rejected them. They never became hub tape. That is the entire point of binding batches to DutchBud receipts rather than to “the node’s word.”

Figure 2. Bangkok: sealed volume dominates; forged seals are rejected (retry noise inflates reject counters, not the tape).

CPU. Hub and ledger hosts did not show a noteworthy compute spike at this mild–medium rate. HMAC-SHA256 for receipts is cheap next to database commits and HTTP. The expensive historical “hash” story (proof-of-work mining) is a different workload: millions of failed tries per success. A receipt is one MAC per fill.

Berlin trial — WAN edge, short burst, quarantine

Edge character. The Berlin node sits on a constrained WAN path. Software self-update over that link is measured in kilobytes per second — fine for catalog deltas and trade JSON, painful for multi‑megabyte binaries. For the trial we still used the pull channel’s version advertisement; when the pull was too slow for the session, we delivered the same hashed artifact out-of-band so the node and hub agreed on build identity. Production expectation remains: nodes pull; operators accept that WAN physics apply.

Load. A short MarketMaker burst produced 28 locally sealed crosses in a few minutes. Seals again went through the hub proxy to DutchBud. Each cross returned a ledger receipt before outbox eligibility.

First batch. On the first successful upload after the burst, the hub accepted sealed rows and rejected the forged companion. Metrics and logs showed the split: legit path OK, rogue path not_found / invalid seal.

Quarantine. Because the batch contained a clear forged-seal signal, the hub automatically quarantined the Berlin node. Effects:

Further POST …/trades from that node API key received 403 with a quarantine flag — no new syndicate writes.
The registered operator received an urgent email (reason + evidence). In this trial the operator mailbox was the desk address on the node record.
Local matching was not intentionally slowed. Quarantine punishes uplink privilege, not customer latency on the regional book. (Slowing local trading as a sanction would be attacker-aligned: a hostile operator could self-trip quarantine to harm brokers on purpose.)

Lift and the “trap door.” When quarantine was lifted while a leftover rogue outbox row still sat beside already-consumed legit receipts, the next flush mixed replay rejects with one forged row and auto-quarantine fired again — plus a second operator email. That is correct security and slightly comic ops. We tightened auto-quarantine so it only trips when all rejects in a batch are rogue-style, not when a single leftover forged row rides next to harmless replays. Leftover rogue rows should still be purged after an incident.

Figure 3. Berlin timeline: burst → mixed batch → quarantine → operator mail → controlled lift.

What the metrics store actually showed

We export Prometheus counters from the hub (and LAN edges we scrape). Sparse batch events are easier to read as counts over a window than as per-second rates — after a process restart, rate() panels go empty even when the system is healthy. That is a dashboard lesson, not a fabric failure.

Node identity gauges (running build vs hub latest, last hello) stayed continuous for Bangkok and Berlin once both reported the trial build.
Hub ingest counters separated kind=legit vs kind=rogue and result=accepted|rejected.
Node upload counters exist on the node process; WAN edges that are not scraped will not paint a “node upload” panel even when logs show minute-by-minute POSTs. Hub ingest (or access logs) remain the source of truth for those edges.
Quarantine retries inflate hub reject totals (same pending outbox every minute). The syndicated-trade table does not grow with those retries.

Syndicated rows for the day carried trust ledger — receipt verified and consumed. Bangkok contributed the bulk of sealed volume; Berlin contributed the sealed burst that made it through before quarantine.

Design conclusions

Pull still wins on hostile or slow networks. The hub does not need deploy credentials on the edge. WAN binary pull can be slow; the channel remains correct. Catalog and trade JSON stay cheap.
Seals are the trust root; API keys are only mouths. Anyone with a node key can POST a batch (curl included). Only DutchBud HMAC receipts make words count on the hub tape.
Rogue operators (or compromised keys) get uplink isolation + mail, not customer DoS. Quarantine blocks syndicate writes and notifies the operator — useful when the operator might be the victim of a hack, not the villain. We do not “punish” by slowing local brokers; that would be a gift to a malicious operator who wants lag.
Historical ECN lag abuse is a reminder, not a product feature. Markets that tolerate unsigned or delayed truth recreate the conditions that made delayed venues and short-sale games profitable. PolitiCap’s simulation stays boring on purpose: no global tape without a seal.
Ops hygiene matters as much as crypto. Leftover forged outbox rows after an incident will re-fire alarms. Purge, then lift. Auto-quarantine should require a clean forged-batch signal, not a single leftover needle in a replay haystack.

What we will measure next

Longer multi-node ramps, explicit scrape of every trusted edge where network policy allows, bond/reputation for high-trust operators whose names brokers can see, and tighter dashboards that treat batch events as counts rather than continuous rates. The fabric is ready for more volume; the next bottleneck will be storage and human ops, not hashing.

PolitiCap is part of the 3DN family — work · money · politics — with DutchBud as the closed-loop ledger for virtual credits. If you operate infrastructure that must stay honest across networks you do not fully own, pull-plus-seal is a pattern worth stealing.