diff --git a/ai/prompt-io/claude/20260813T001102Z_27c34aeb_prompt_io.md b/ai/prompt-io/claude/20260813T001102Z_27c34aeb_prompt_io.md new file mode 100644 index 00000000..ee919337 --- /dev/null +++ b/ai/prompt-io/claude/20260813T001102Z_27c34aeb_prompt_io.md @@ -0,0 +1,136 @@ +--- +model: claude-opus-5 +service: claude +session: 7b9c97c4-fff7-4ac4-97fb-35720453308e +timestamp: 2026-08-13T00:11:02Z +git_ref: 27c34aeb +scope: docs+code +substantive: true +raw_file: 20260813T001102Z_27c34aeb_prompt_io.raw.md +--- + +## Prompt + +> draft hyper detailed implementation plans for [three] +> prospective new transport (tpt) backends for tractor's `.ipc` +> layer, from four GitHub issues: TIPC (gh #378) using built-in +> linux socket API w/ `trio` interfacing, leveraging TIPC's +> built-in discovery machinery; QUIC (gh #353) using the `iroh` +> lib, ideally with the py asyncio support (via ffi) rewritten +> for trio; wg (gh #482 and/or #443) with other shuttle-able +> tpts, using `pyroute2`, as much trio wrapping as possible +> where any other async support can be replaced. + +With constraints: "be only slightly speculative"; realistic given +the linked info; the plans must be workable "by another +model/provider without much discrepancy on the design or lib +selections needed"; do it all in a new `/open-wkt` off main; and +be "as modular and modern-async-python-as-nearly-functional-style +as possible." + +Key follow-up prompts that changed the design mid-session: + +> wg requires binding a socket on both ends of the tunnel, the +> new support we added to `py-multiaddr` in their #108 enables +> declaring this in the maddr? […] so the +> `/ip4/10.0.0.1/udp/666/ip4/192.168.0.13/tcp/80` bullet example +> from #443 isn't correct for a wg tunnel maddr and would +> actually end up looking like, +> `/ip4/10.0.0.1/udp/666/wg//ip4/192.168.0.13/tcp/80` no? + +> right, we're likely going to need less `tuple`-formed +> `UnwrappedAddress` usage to make them all work and/or to extend +> the unwrapped form with a proto-str, likely the key from +> `multiaddrs` no? + +> better encapsulating `UnwrappedAddress` for use externally is +> really way overdue; we should likely be encouraging (if not +> preventing unwrapped) wrapped addrs always much like other +> (std) libs, `ipaddress` comes to mind. + +> so iirc netns mgmt becomes a lower-level runtime config-api +> rather then an actor-app-code API which can be dynamically +> changed once the actor is up ya? + +## Response summary + +A research-and-planning branch, `ng_tpts_planning` — **no runtime +code touched**. Five plan docs under `ai/tpt-backends/` plus a +corrected `wg` example set under `examples/multihost/wg_lan/`, +across 9 commits. + +Three design conclusions came out of the planning that outlive +the branch: + +1. **`wg` is a bindspace, not a `MsgTransport`.** The `/wg/` + maddr segment is *infix*: bearer (underlay ip+udp + `ListenPort`, bound by the kernel) → `/wg/u` (an + identity, bound by nothing) → overlay (the only part tractor + binds). Verified empirically by installing + `baudco/py-multiaddr@wg_support` (py-multiaddr#108) in a + throwaway venv and round-tripping each candidate form; gh + #482's original suffix form parses but is semantically + inverted. +2. **Proto-key the `UnwrappedAddress` form**, spelled with the + `multiaddr` protocol names, dispatched via + `_address_types[addr[0]]`. Kills a whole collision class + (TIPC's `(str, int)` ≡ `TCPAddress`; iroh's `(str, str)` + swallowed by the UDS case) and is the recommended migration + *before* any new backend lands. +3. **netns is a runtime/boot-time config API, not an app-code + one** — `setns(2)` is per-thread and won't move + already-created sockets, so there is deliberately no + `await actor.enter_netns(...)`. + +Also verified that `trio.SocketStream`/`SocketListener` are +address-family agnostic (no `AF_*` check anywhere), which is what +makes TIPC the cheapest of the three backends to add. + +Four related issues were annotated with the results (#378, #353, +#482, #443); #443's body was rewritten to reflect the corrected +grammar, with no existing checkbox state changed. + +## Files changed + +- `ai/tpt-backends/00_shared_backend_contract.md` — normative + backend duck-type contract, registration checklist, §1.1 + proto-key conclusion +- `ai/tpt-backends/01_tipc_backend.md` — TIPC plan; service + addressing, `TIPC_TOP_SRV` push registry, instance-collision + hazard, step-0 probe +- `ai/tpt-backends/02_quic_iroh_backend.md` — `iroh` plan; + `uniffi`→`trio` bridge, listener/stream adapters, API-truth + table +- `ai/tpt-backends/03_wg_tunnel_bindspace.md` — `wg`-as-bindspace + plan; verified maddr grammar, 3-owner split, netns reality +- `ai/tpt-backends/README.md` — index +- `examples/multihost/wg_lan/wg_maddr.py` — frozen `msgspec` + tunnelled addr + pure parse/render helpers; impure + `verify_wg_peer()` kept separate +- `examples/multihost/wg_lan/host_a_srv.py` — host-A actor tree +- `examples/multihost/wg_lan/host_b_client.py` — host-B dialer +- `examples/multihost/wg_lan/README.md` — grammar, owner table, + setup, "what changed vs #482" + +## Human edits + +Substantial human steering rather than post-hoc editing; the +corrections were applied by the model in-session after being +challenged: + +- rejected an initial claim that `wg` has "nothing to bind at the + tunnel layer" and supplied the correct composed maddr form, + which forced a rewrite of plan 03 §3.2 and a retraction in the + already-posted #443 comment +- rejected a supporting claim that `/ip4/../udp/443/quic-v1` was + "also composed" +- directed the proto-key/`ipaddress`-discipline conclusion and + the netns-as-runtime-config framing, both of which were then + folded back into the docs +- chose the commit boundaries and authored all commits; ran every + `git` mutation (commit, rebase, push) themselves + +One model-initiated correction pre-publication: a self-review +downgraded two overconfident claims (the `uniffi`/asyncio thesis +and TIPC duplicate-binder behaviour) to explicitly-flagged +assumptions before the #353/#378 comments were posted. diff --git a/ai/prompt-io/claude/20260813T001102Z_27c34aeb_prompt_io.raw.md b/ai/prompt-io/claude/20260813T001102Z_27c34aeb_prompt_io.raw.md new file mode 100644 index 00000000..0e082229 --- /dev/null +++ b/ai/prompt-io/claude/20260813T001102Z_27c34aeb_prompt_io.raw.md @@ -0,0 +1,165 @@ +--- +model: claude-opus-5 +service: claude +timestamp: 2026-08-13T00:11:02Z +git_ref: 27c34aeb +diff_cmd: git diff main..ng_tpts_planning +--- + +# Raw output — next-gen tpt-backend implementation plans + +## Generated planning docs + +> `git diff main..ng_tpts_planning -- ai/tpt-backends/` + +Five markdown docs. `00_shared_backend_contract.md` is normative +and the other three are written against it so they can be worked +independently: + +- **`00_shared_backend_contract.md`** — the backend duck-type + (`Address(msgspec.Struct, frozen=True)` + module-level + `start_listener()`/`close_listener()` + a + `MsgpackStream(MsgpackTransport)`), the + `inspect.getmodule(self.addr)` reflection in + `Endpoint.start_listener()` that forces the Address class and + its listener fns to share a module, a 10-item registration + checklist, the dep policy, the test-harness shape, and §1.1's + proto-key conclusion (below). +- **`01_tipc_backend.md`** — service addressing via + `TIPC_ADDR_NAMESEQ` (bind/publish) and `TIPC_ADDR_NAME` + (connect/lookup), `TIPC_TOP_SRV` topology subscriptions as a + push-based registry, the `get_random()` instance-collision + hazard, and a step-0 capability-probe spike. +- **`02_quic_iroh_backend.md`** — `iroh` over + `aioquic`/`quiche`/`trio-asyncio`, a `_uniffi_trio.py` bridge + built on `TrioToken.run_sync_soon()`, `trio.abc.Listener`/ + `HalfCloseableStream` adapters, and an API-truth table to fill + in during step 0. +- **`03_wg_tunnel_bindspace.md`** — `wg` as a *bindspace* rather + than a `MsgTransport`, a `TunnelledAddress` wrapper delegating + `.proto_key`/`.unwrap()` to `.inner`, `pyroute2` for layer B, + and `@acm`-managed netns/iface for layer C. +- **`README.md`** — index. + +## Generated example code + +> `git diff main..ng_tpts_planning -- examples/multihost/wg_lan/` + +- `wg_maddr.py` — `WGTunnelledAddr(msgspec.Struct, frozen=True)` + carrying `bearer: tuple[str, int]`, `peer_pubkey: str`, + `inner: tuple[str, int]`, `inner_proto: Literal['tcp']`, plus a + `.maddr` property that re-renders the canonical form. Pure + helpers `mb_pubkey()`, `wg8_pubkey()`, `parse_wg_maddr()`, and + `_segments()` (with a marked stopgap for when the `wg` codec + isn't installed). `verify_wg_peer()` is impure **by design** and + kept out of the parse path. +- `host_a_srv.py` / `host_b_client.py` — the two-host runs; both + pass only `addr.inner` to `open_nursery()`/`open_root_actor()`. +- `README.md` — grammar, owner table, `#108`-branch install line, + tunnel setup, "what changed vs #482". + +## Verified findings (non-code, verbatim) + +### `trio` is address-family agnostic + +Read against the installed `trio`. `SocketStream`/`SocketListener` +ctor checks are only "is a trio sock object" + `type == +SOCK_STREAM`, plus an `OSError`-**suppressed** `SO_ACCEPTCONN` +probe. No `AF_*` check anywhere; `TCP_NODELAY`/`TCP_NOTSENT_LOWAT` +are set under `suppress(OSError)`. A TIPC `SOCK_STREAM` sock should +therefore drop straight into `trio.serve_listeners()` with the +existing `MsgpackTransport` framing, making TIPC mostly +table-registration boilerplate w/ zero new deps. + +### the `wg` maddr grammar — `/wg/` is infix, not suffix + +Installed `baudco/py-multiaddr@wg_support` (PR +multiformats/py-multiaddr#108) into a throwaway venv and +round-tripped every candidate form: + +| maddr | `[p.name for p in m.protocols()]` | +| --- | --- | +| `/ip4/1.2.3.4/udp/51820/wg/u` | `['ip4','udp','wg']` | +| `/ip4/../udp/../wg/u/ip4/../tcp/..` | `['ip4','udp','wg','ip4','tcp']` | +| `/ip4/10.0.11.1/tcp/1616/wg/u` | `['ip4','tcp','wg']` | + +``` +/ip4/192.168.1.50/udp/51820/wg/u/ip4/10.0.11.1/tcp/1616 +\_______ bearer __________/\__ key __/\______ overlay ______/ +``` + +Segments *before* `/wg/` are the bearer — the underlay +`(ip, udp-port)` that `wg(8)` itself listens on (`ListenPort`). +Segments *after* are the overlay endpoint, the only part tractor +binds. The third row above is #482's original suffix form: it +parses, but is semantically inverted. + +Three parts, three owners — and only one is an `Endpoint`: + +| part | bound by | in the runtime? | +| --- | --- | --- | +| bearer | kernel, via `wg-quick`/`pyroute2` | no | +| `/wg/u` | nothing — an identity | no, verified out-of-band | +| overlay | `tractor`'s `IPCServer` | yes, as `.inner` | + +### proto-key-tagged `UnwrappedAddress` + +Shape-matching in `wrap_address()` does not survive four backends. +TIPC's natural unwrapped form is a `(str, int)`, indistinguishable +from `TCPAddress`; iroh's is a `(str, str)`, already swallowed by +the existing UDS case (`case (_, filename) if type(filename) is +str`). Ordering hacks and prefix-tagging only paper over it. + +Recommended prerequisite for all three backends: carry an explicit +proto-key spelled with the `multiaddr` protocol name — +`('tcp', host, port)`, `('unix', path)`, +`('tipc', stype, inst, scope)` — so `wrap_address()` collapses to +`_address_types[addr[0]]` and the collision class stops existing. +This also makes the on-wire form agree with +`mk_maddr()`/`parse_maddr()` instead of being an independent +invention. It is a wire-format change (`SpawnSpec`, +`_root_mailbox`, `_registry_addrs`) plus every fixture and +downstream config, so it wants its own migration commit landed +before any new backend — and it is the moment to stop handing raw +tuples to users at all, making `Address` the public currency and +`UnwrappedAddress` an internal serialization detail (the +discipline `ipaddress` uses). + +### netns is a runtime-level config API + +`setns(2)` affects the calling thread only and does not move +already-created sockets. So a netns is a spawn/boot-time input +alongside `enable_transports`/`tpt_bind_addrs`, and there is +deliberately no `await actor.enter_netns(...)` — a mid-life API +would silently leave the IPC server bound in the old namespace. +Corollary for layer B: pass `netns=` down to `pyroute2` rather +than assuming a `trio.to_thread` worker inherits it. + +### `examples/` collection would have failed CI + +`tests/test_docs_examples.py` walks `examples/` recursively and +subproc-runs every collected file asserting `rc == 0`. Its filter +never checks the extension, so all four `wg_lan` files were +collected — including `README.md`, which would have been run as +`python README.md`. `'multihost' not in p[0]` was already in the +exclusion list with no directory using it. Moving the set under +`examples/multihost/wg_lan/` drops collection 24 → 20 with zero +test changes; confirmed via `pytest --collect-only`. + +## Corrections applied during the session + +The human corrected two claims that had been asserted without +verification, both since retracted in-place in the docs and in the +posted issue comments: + +1. that `wg` has "nothing to bind at the tunnel layer, exactly one + bind" — wrong; a wg stack is genuinely composed, and the real + axis is *who owns* each layer's endpoint. +2. that `/ip4/../udp/443/quic-v1` was "also composed" — wrong; + that is one endpoint with a protocol qualifier, not a tunnel. + +A self-review before publication also downgraded two +overconfident claims to explicitly-flagged assumptions: the +`uniffi`-uses-asyncio-only-as-executor thesis (contradicted that +plan's own "do not guess from memory" step 0) and TIPC's +duplicate-binder round-robin behaviour (unverified).