From e25e018370e526b902c18dff163f00484877d9f4 Mon Sep 17 00:00:00 2001 From: goodboy Date: Tue, 11 Aug 2026 23:11:10 -0400 Subject: [PATCH 01/18] Add the `.ipc` tpt-backend contract spec MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit First doc of a new `ai/tpt-backends/` set: the normative description of what a `tractor` tpt backend *is* as of `main`, written so the 3 sibling plans (TIPC, QUIC, `wg`) can be worked independently (by another model/provider) w/o design drift. Deats, - the backend duck-type as empirically derived from `_tcp.py`/`_uds.py`: the `Address` protocol surface, the mod-level `start_listener()`/`close_listener()` pair and `MsgpackStream(MsgpackTransport)`. - the ONE reflection you can't break: `Endpoint.start_listener()` resolves the tpt mod via `inspect.getmodule(self.addr)`, so an `Address` type and its listener fns MUST live in the same mod. - a 10-item registration checklist (`_address_types`, `_key_to_transport`, `_addr_to_transport`, `wrap_address()` match-cases, `TransportProtocolKey`, maddr tables, ..) incl. the import-time `_default_lo_addrs` trap. - where the `trio.SocketListener` assumption is *actually* load-bearing (just the `getsockname()` reconcile) vs. merely annotated. - the handshake/discovery invariants a new backend inherits, dep policy (extras + import-laziness per the #470 boot-latency budget), `--tpt-proto` harness plumbing and code style. Also, records a verified finding the plans lean on hard: `trio.SocketStream`/`SocketListener` are addr-*family* agnostic — the only ctor checks are "is a trio sock" + `SOCK_STREAM` (+ an `OSError`-suppressed `SO_ACCEPTCONN`) — so any `SOCK_STREAM` family CPython can make drops into the existing `trio.serve_listeners()` path unmodified. (this patch was generated in some part by `claude-code` using `claude-opus-5` (`anthropic`)) --- ai/tpt-backends/00_shared_backend_contract.md | 368 ++++++++++++++++++ 1 file changed, 368 insertions(+) create mode 100644 ai/tpt-backends/00_shared_backend_contract.md diff --git a/ai/tpt-backends/00_shared_backend_contract.md b/ai/tpt-backends/00_shared_backend_contract.md new file mode 100644 index 00000000..690ffa8b --- /dev/null +++ b/ai/tpt-backends/00_shared_backend_contract.md @@ -0,0 +1,368 @@ +# `tractor.ipc` next-gen transport backends: the shared contract + +Status: design doc / implementation spec. +Audience: any model or human implementing one of the three +sibling plans in this directory. + +- [`01_tipc_backend.md`](./01_tipc_backend.md) — `AF_TIPC` + (gh #378) +- [`02_quic_iroh_backend.md`](./02_quic_iroh_backend.md) — QUIC + via `iroh` FFI, uniffi-async rewritten onto `trio` (gh #353) +- [`03_wg_tunnel_bindspace.md`](./03_wg_tunnel_bindspace.md) — + WireGuard (and other shuttle-able) tunnels as a *nested + bindspace* layer via `pyroute2` (gh #482, #443) + +This doc is the **normative** description of what a `tractor` +transport backend *is* as of `main@83b34884`. Each sibling plan +assumes it and only documents its own deltas. Read this first; +do not re-derive it from the code. + +--- + +## 0. Why a shared contract doc + +The three plans are meant to be implementable *independently and +concurrently* by different models/providers without design +drift. Everything they share — the backend duck-type, the +registration tables, the test harness plumbing, the naming and +code-style rules — lives here exactly once. If an implementer +finds this doc disagrees with `main`, **the code wins**; fix this +doc in the same PR. + +--- + +## 1. The backend duck-type (empirical, from `_tcp.py`/`_uds.py`) + +A transport backend is **one module** under `tractor/ipc/` +exposing exactly four things. There is no ABC to subclass and no +plugin entrypoint; wiring is by explicit table registration +(§2) plus one piece of reflection (§1.3). + +### 1.1 `class Address(msgspec.Struct, frozen=True)` + +Structurally conforms to the `Address` `Protocol` in +`tractor/discovery/_addr.py:82`. Required surface: + +| member | kind | notes | +| --- | --- | --- | +| `proto_key` | `ClassVar[str]` | the wire/registry key, e.g. `'tcp'`, `'uds'` | +| `unwrapped_type` | `ClassVar[type]` | the primitive tuple shape | +| `def_bindspace` | `ClassVar` | default bindspace value | +| `is_valid` | `@property -> bool` | "is this a *dialable/bindable* addr" | +| `bindspace` | `@property` | the "set of hosts"-ish scope (see below) | +| `from_addr(cls, addr)` | `@classmethod` | primitive -> wrapped, `match`-based | +| `unwrap(self)` | method | wrapped -> primitive (must be msgpack-native!) | +| `get_random(cls, bindspace=...)` | `@classmethod` | per-subactor ephemeral addr | +| `get_root(cls)` | `@classmethod` | host-singleton default registrar addr | +| `__repr__` | method | `f'{type(self).__name__}[{...}]'` house style | + +Hard constraints learned from the existing two: + +- **`frozen=True`.** Addresses are dict keys + (`Server.epsdict()`, `Endpoint.peer_tpts`) and are compared by + value all over the runtime. +- **`.unwrap()` output must round-trip through `msgspec` and + through `wrap_address()`.** It is what actually crosses the + wire in `SpawnSpec`/`_root_mailbox`/`_registry_addrs`, and it + is what `Actor.reg_addrs` and every test compares against. If + your unwrapped form is not *uniquely* pattern-matchable + against the other backends' forms in + `wrap_address()` (`_addr.py:230`), you have a bug that + manifests as the wrong transport being loaded — the file's own + `XXX NOTE` warns about precisely this. +- **`.get_random()` must be collision-free without a live + runtime.** See the `UDSAddress.get_random()` uuid-token + comment (`_uds.py:207-220`): with no `current_actor()` the + sockname degenerates to a pure fn of `(prefix, pid)` and two + calls in one proc alias. Mix in a `uuid4().hex[:8]` token. +- **`.bindspace` semantics**: "the address' bindable space" — + ip/host for `tcp`, the socket-file *directory* for `uds`. For + the new backends: the TIPC *scope* (§1 of plan 01), the iroh + *ALPN + relay/discovery realm* (plan 02), the netns (plan 03). + `Address.namespace` is already spec'd in the Protocol as + "the if-available OS-specific network namespace key" and is + currently unimplemented by both backends — plan 03 is the + first real consumer. + +### 1.2 module-level listener lifecycle + +```python +async def start_listener( + addr: Address, + **kwargs, +) -> trio.SocketListener # or a trio.abc.Listener, see §3 + ... + +def close_listener( # OPTIONAL + addr: Address, + lstnr: trio.abc.Listener, +) -> None: + ... +``` + +`close_listener()` is optional; `Endpoint.close_listener()` +(`_server.py:674`) `getattr`s it and treats absence as "closing +is implicit". `uds` needs it (unlinks the sock-file), `tcp` +does not. + +### 1.3 the ONE piece of reflection you must not break + +`Endpoint.start_listener()` (`_server.py:656`): + +```python +tpt_mod: ModuleType = inspect.getmodule(self.addr) +lstnr = await tpt_mod.start_listener(addr=self.addr) +``` + +The transport module is found by `inspect.getmodule()` **on the +`Address` instance**. Therefore: *the `Address` class and its +`start_listener()`/`close_listener()` MUST live in the same +module.* Do not define the address type in `_types.py` or a +`_addrs.py` and the listener elsewhere. + +Immediately after, the same method does: + +```python +if (unwrapped := lstnr.socket.getsockname()) != self.addr.unwrap(): + self.addr = self.addr.from_addr(unwrapped) +``` + +i.e. it assumes `lstnr.socket.getsockname()` exists and that its +return value is a valid `from_addr()` input. This is fine for +TIPC (§3 of plan 01) and **is the main integration hazard for +iroh** (§3 of plan 02) — plans that break it must say so +explicitly and propose the upstream `_server.py` patch. + +### 1.4 `class MsgpackStream(MsgpackTransport)` + +Subclass `tractor.ipc._transport.MsgpackTransport`. You inherit +all framing (`Address` class | +| `layer_key: int` | OSI-ish layer, `4` for both current backends | +| `maddr` `@property` | `-> Multiaddr\|str`, via `mk_maddr(self.raddr)` | +| `connected(self) -> bool` | `tcp`/`uds` both use `self.stream.socket.fileno() != -1` | +| `connect_to(cls, addr, prefix_size=4, codec=None, **kw)` | `@classmethod`, returns an instance | +| `get_stream_addrs(cls, stream) -> (laddr, raddr)` | `@classmethod`, called from `MsgpackTransport.__init__` | + +`MsgpackTransport.__init__` requires the object passed as +`stream` to satisfy: + +- `await stream.send_all(bytes)` +- usable as `tricycle.BufferedReceiveStream(transport_stream=stream)`, + i.e. `await stream.receive_some(n)` +- `trio.BrokenResourceError` / `trio.ClosedResourceError` / + `ValueError('...unclean EOF...')` on the failure paths that + `_iter_packets()` and `send()` already `match` on + (`_transport.py:221-304`, `:436-499`). + +That is **`trio.abc.Stream`, not `trio.SocketStream`**. The +`MsgTransport` Protocol's `stream: trio.SocketStream` +annotation (`_transport.py:83`) is a lie of convenience — the +actual `MsgpackTransport.__init__` param is typed +`trio.abc.Stream` and nothing in the msg path touches +`.socket`. Only `connected()` (which each backend defines) and +`Endpoint.start_listener()`'s `getsockname()` do. + +### 1.5 verified-good news for socket-family backends + +Both `trio.SocketStream` and `trio.SocketListener` are +**address-family agnostic**. Verified against the installed +`trio` (`trio/_highlevel_socket.py`): the only constructor +checks are + +- `isinstance(socket, trio.socket.SocketType)` +- `socket.type == SOCK_STREAM` +- (listener) `getsockopt(SOL_SOCKET, SO_ACCEPTCONN)` is truthy, + with `OSError` **suppressed** (the macOS carve-out, which + also covers exotic families that reject the opt) + +There is no `AF_*` check and no `IPPROTO_TCP` hard dependency +(`TCP_NODELAY`/`TCP_NOTSENT_LOWAT` are set under +`suppress(OSError)`). Consequence: **any `SOCK_STREAM` family +CPython can create — including `AF_TIPC` — drops straight into +the existing `trio.SocketStream` + `trio.serve_listeners()` +path.** This is why plan 01 is small and plan 02 is not. + +--- + +## 2. Registration tables (the full wiring checklist) + +Adding a backend touches these and only these: + +1. `tractor/runtime/_state.py:46` + `TransportProtocolKey = Literal['tcp', 'uds', ...]` — add the + key. This `Literal` is the canonical set; `_testing/pytest.py` + drives `--tpt-proto` validation off `_addr._address_types`, + and the spawn-backend fixture already models the + "drive-the-set-from-the-Literal" pattern + (`pytest.py:870-880`) — do the same rather than hardcoding. +2. `tractor/discovery/_addr.py:173` `_address_types: bidict` — + `{'': Address}`. Note it is a **`bidict`**, so + the mapping must stay 1:1. +3. `tractor/discovery/_addr.py:181` `_default_lo_addrs` — + `'': Address.get_root().unwrap()`. + ⚠️ this dict is built at **import time**, so + `get_root()` must not require a live runtime, a loaded kernel + module, or network I/O. (`UDSAddress.def_bindspace = + get_rt_dir()` is the precedent for "cheap, pure, filesystem- + ish".) A backend whose root addr needs I/O must make this + entry lazy — propose that refactor explicitly. +4. `tractor/discovery/_addr.py:230` `wrap_address()` `match` — + add a case iff your `unwrapped_type` isn't already uniquely + matched. Prefer unwrapped forms that are *self-tagging* + (see plan 01 §2.2 and plan 02 §2.2) so this stays cheap. +5. `tractor/ipc/_types.py` — `Address` union alias, + `_msg_transports` list, `_key_to_transport[('msgpack', key)]`, + `_addr_to_transport[Address]`. +6. `tractor/ipc/_types.py:92` `transport_from_stream()` — the + `sock.family` `match`. For a non-socket stream type (iroh) + this needs a different discriminator; see plan 02 §3.3. +7. `tractor/discovery/_multiaddr.py` — + `_tpt_proto_to_maddr`, and a `case` in both `mk_maddr()` and + `parse_maddr()`. +8. `tractor/ipc/__init__.py` — re-export if the backend has a + public surface. +9. `tractor/_testing/addr.py::get_rando_addr()` — per-proto + branch so the whole suite can run under `--tpt-proto `. +10. `pyproject.toml` — new deps go in an **optional extra**, never + in `[project].dependencies`. See §5. + +## 3. Where the `trio.SocketListener` assumption is load-bearing + +`_serve_ipc_eps()` (`_server.py:1041`) annotates +`listener: trio.abc.Listener` and hands the list to +`trio.serve_listeners(handler=handle_stream_from_peer, +listeners=..., handler_nursery=stream_handler_tn)`. +`trio.serve_listeners` itself is generic over +`trio.abc.Listener`. So the *only* `SocketListener`-specific +code in the server path is the `getsockname()` reconciliation in +`Endpoint.start_listener()` (§1.3) and the type annotations. + +`handle_stream_from_peer()` (`_server.py:298`) then does +`Channel.from_stream(stream)` → +`transport_from_stream(stream)` → `sock.family` match (§2.6). + +**Therefore**: a non-socket backend needs (a) a +`trio.abc.Listener` subclass, (b) a change to +`Endpoint.start_listener()` to not blindly `getsockname()`, and +(c) a change to `transport_from_stream()`'s discrimination. +All three are small, upstream-able, and *should be landed as +their own prep PR* before the backend itself — see plan 02 §3. + +## 4. Handshake / discovery invariants you inherit + +- Every accepted stream immediately does + `chan._do_handshake(aid=actor.aid)`; a peer that fails it is + logged at `runtime` and dropped, **not** raised + (`_server.py:334-365`). Discovery-sys "pings" rely on this, + so your `connect_to()` must raise something that normalizes + to `TransportClosed`/`ConnectionError` on a dead peer, never + a novel exception type. +- `_root.py:381-406` fail-fasts when a `registry_addrs` entry's + `proto_key` is not in `enable_transports`. Your key must be + spellable in both. +- `_root.py:256` currently enforces `len(enable_transports) == 1`. + Multi-tpt actors are a separate work item; none of these three + plans may depend on lifting it. +- Sub-actor bind addrs come from + `_runtime.py:1600-1610`: for each key in the parent-supplied + `enable_transports`, `get_address_cls(key).get_random()`. + So `get_random()` runs *in the child, post-fork, pre-listen*. + Anything it needs (kernel module, netns membership, an iroh + secret key) must already be true at that moment. + +## 5. Dependency policy + +`[project].dependencies` stays lean (see the boot-latency work, +gh #470: `import tractor` is budgeted at ~0.145s). Every new +backend dep is an extra: + +```toml +[project.optional-dependencies] +tipc = [] # stdlib-only! +quic = ["iroh>=0.35"] # pin per plan 02 §1 +wg = ["pyroute2>=0.9"] # pin per plan 03 §1 +``` + +and every backend module must be **import-lazy**: a +`tractor/ipc/_.py` that imports its 3rd-party dep at +module scope must not be imported by `tractor/__init__.py`, +`tractor/ipc/__init__.py`, or `tractor/discovery/_addr.py`'s +import-time table construction. The `_addr._default_lo_addrs` +eager-dict (§2.3) is the trap: keep the backend's `get_root()` +dep-free, or make that table lazy. + +## 6. Test-harness plumbing (identical for all three) + +- `--tpt-proto ` (`_testing/pytest.py:409`) selects the + session-wide proto; the `tpt_proto` fixture mutates + `_state._def_tpt_proto` + `_runtime_vars['_enable_tpts']` + (`pytest.py:807-835`). Adding the key to `_address_types` is + what makes `--tpt-proto ` legal (`pytest.py:795-800` + asserts the lookup). +- The **acceptance bar** for every backend is: the *entire* + existing suite passes under `--tpt-proto `, unmodified. + That is the whole point of the abstraction. Backend-specific + unit tests go in `tests/ipc/test_each_tpt.py` (the existing + `test_uds_bindspace_created_implicitly` / + `test_uds_double_listen_raises_connerr` are the model). +- Capability gating: each backend needs a **cheap, pure + predicate** + a `pytest.mark.skipif`, because these are all + environment-dependent. Verified example: on this dev box + `socket.socket(AF_TIPC, SOCK_STREAM)` raises + `OSError(97, 'Address family not supported by protocol')` + because the `tipc` module isn't loaded. Put the predicate in + the backend module (so apps can use it too), not in the test. +- New pytest marks must be registered in `pyproject.toml`, per + the project's fix-warnings-at-source rule (gh #469). + +## 7. Code style (non-negotiable, matches the repo) + +- module header tagline: `# tractor: distributed structured + concurrency.` for **new** files (not the legacy + `structured concurrent "actors".` form the existing `_tcp.py` + carries). +- AGPL header block copied verbatim from `_tcp.py`. +- `from __future__ import annotations` first. +- annotate *everything*, including locals: + `sockpath: Path = addr.sockpath`. +- `match`/`case` over `isinstance` chains for address and + error dispatch. +- multi-line call/`import` style with trailing commas. +- never emit a whitespace-only line. +- error messages are multi-line f-strings ending in `\n`, with + the `f'...\n' f'...\n'` implicit-concat layout and the + `>[`/`[>`/`<=(` nested-op sigils where a `nest_from_op()` is + in play. +- prefer pure functions + module-level helpers over methods; + keep `Address` types data-only. Where a helper needs + scoped setup/teardown, it's an `@acm` — not a class with + `.start()`/`.stop()`. +- pure getters: no `get_*(..., mutate=True)` flags; split into + a read-only getter and an explicit sibling setter. + +--- + +## 8. Cross-plan sequencing + +The three are independent *except*: + +- plan 02 (iroh) needs the `_server.py` / + `transport_from_stream()` generalization (§3) — plan 01 does + **not**, and should therefore land first as the cheap proof + that the table-registration story works for a genuinely new + proto. +- plan 03 (wg) composes *under* whatever L4 tpt is in use and + its netns work is what finally implements + `Address.namespace`. It can land before or after 02, but its + `TunnelledAddress` design must be reviewed against plan 02's + address shape so the "tunnelled maddr" grammar (gh #443) + covers `/…/quic-v1/p2p/…` inner addrs too. +- All three want first-class `wg`/`quic`/`tipc` protos in + `py-multiaddr`; that upstream track is gh #483 and + multiformats/py-multiaddr#107/#108. From f81dd82b89a4e9b4183ea8329b6e49dea096af77 Mon Sep 17 00:00:00 2001 From: goodboy Date: Tue, 11 Aug 2026 23:11:52 -0400 Subject: [PATCH 02/18] Add `TIPC` tpt-backend impl plan MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Plan doc for gh #378, the cheapest new backend we can add: it's stdlib-only (CPython ships `AF_TIPC` + 23 `TIPC_*` consts) and per the contract doc `trio`'s stream/listener wrappers don't care about the addr family, so `MsgpackTransport` framing and `trio.serve_listeners()` are reused verbatim. Deats, - `TIPCAddress` as a *service name* `(type, instance)` w/ scope as the `.bindspace`; `bind()` publishes the singleton name-range, peers `connect()` by name and the kernel resolves + load-balances. I.e. registration/lookup for free, no registrar in the loop. - the self-tagging `('tipc::', instance)` unwrapped form + why it must be match-ordered before `TCPAddress`'s. - `get_random()` via a blake2b digest of the actor id (there's no `port=0` analogue) and the silent-crosstalk risk that follows: TIPC *allows* dup binders and round-robins, so a collision doesn't `EADDRINUSE`, it cross-talks. - an `Address.rebind_from_sockname` ClassVar to opt out of `Endpoint.start_listener()`'s `getsockname()` reconcile, which for TIPC always returns a port-id, never the bound name. - the `TIPC_TOP_SRV` topology-service subscription as an `@acm` yielding a chan of typed name-table events — push-based register/dereg, the real "end game cluster proto" bit. - commit sequencing, hard capability gating (`modprobe tipc`; bare `AF_TIPC` is `EAFNOSUPPORT` on a stock box), CI matrix notes, risks + follow-up seeds. (this patch was generated in some part by `claude-code` using `claude-opus-5` (`anthropic`)) --- ai/tpt-backends/01_tipc_backend.md | 701 +++++++++++++++++++++++++++++ 1 file changed, 701 insertions(+) create mode 100644 ai/tpt-backends/01_tipc_backend.md diff --git a/ai/tpt-backends/01_tipc_backend.md b/ai/tpt-backends/01_tipc_backend.md new file mode 100644 index 00000000..8a0fa413 --- /dev/null +++ b/ai/tpt-backends/01_tipc_backend.md @@ -0,0 +1,701 @@ +# Plan 01 — `TIPC` transport backend (`tractor/ipc/_tipc.py`) + +Tracks gh [#378]. Prereq reading: +[`00_shared_backend_contract.md`](./00_shared_backend_contract.md). + +**Thesis**: TIPC is the *cheapest* new backend we can add and +simultaneously the only one that gives us cluster-wide service +discovery **for free, in the kernel**, replacing (for +TIPC-capable deployments) the whole `tractor.discovery` +registrar round-trip with a `bind()`/`connect()` on a +*service name*. It is stdlib-only: zero new dependencies. + +[#378]: https://github.com/goodboy/tractor/issues/378 + +--- + +## 1. Why this is small: three verified facts + +1. **CPython already speaks TIPC.** `socket.AF_TIPC` plus 23 + `TIPC_*` constants are present in the stdlib on Linux + (verified on the dev box, py3.13): + `AF_TIPC, SOL_TIPC, TIPC_ADDR_ID, TIPC_ADDR_NAME, + TIPC_ADDR_NAMESEQ, TIPC_CFG_SRV, TIPC_CLUSTER_SCOPE, + TIPC_CONN_TIMEOUT, TIPC_{CRITICAL,HIGH,MEDIUM,LOW}_IMPORTANCE, + TIPC_DEST_DROPPABLE, TIPC_IMPORTANCE, TIPC_NODE_SCOPE, + TIPC_PUBLISHED, TIPC_SRC_DROPPABLE, TIPC_SUBSCR_TIMEOUT, + TIPC_SUB_CANCEL, TIPC_SUB_PORTS, TIPC_SUB_SERVICE, + TIPC_TOP_SRV, TIPC_WAIT_FOREVER, TIPC_WITHDRAWN, + TIPC_ZONE_SCOPE`. + `sock.bind()/connect()/getsockname()` take/return the + 5-tuple `(addr_type, v1, v2, v3, scope)` — the last element + is optional on input and defaults to `0`. +2. **`trio` doesn't care about the address family.** Per + contract §1.5, `trio.SocketStream` and `trio.SocketListener` + only require a trio socket object of type `SOCK_STREAM`. + TIPC's `SOCK_STREAM` is a real connection-oriented reliable + byte stream. So we reuse `trio.SocketStream`, + `trio.SocketListener`, `trio.serve_listeners()`, + `MsgpackTransport`'s framing — *all of it*. +3. **It is not available by default.** On this box + `socket.socket(AF_TIPC, SOCK_STREAM)` → + `OSError(97, 'Address family not supported by protocol')` + with no `tipc` in `/proc/modules`. `modprobe tipc` is + required; cross-node needs a bearer + (`tipc bearer enable media eth device ` or + `media udp name localip `). Everything about this + plan's testability hinges on gating (§7). + +Non-goals: `SOCK_RDM`/`SOCK_DGRAM`/`SOCK_SEQPACKET` message +modes, multicast fan-out, and TIPC group messaging. They are +genuinely interesting for a future `tractor` broadcast/pubsub +transport but they do **not** fit `MsgTransport`'s +stream-of-length-prefixed-msgs shape. Note them in the +follow-up issue, do not build them here. + +--- + +## 2. `TIPCAddress` + +### 2.1 the three TIPC address flavours, and which we use + +| flavour | tuple | meaning | +| --- | --- | --- | +| `TIPC_ADDR_NAMESEQ` | `(type, lower, upper, scope)` | a *published range* — what a server `bind()`s | +| `TIPC_ADDR_NAME` | `(type, instance, domain, scope)` | a *lookup* — what a client `connect()`s | +| `TIPC_ADDR_ID` | `(node, ref, 0, scope)` | a concrete port id — the "physical" address | + +The design decision that makes this backend coherent: + +> **A `tractor` actor's TIPC address is a *service name* +> `(type, instance)`; `bind()` publishes the singleton range +> `(type, instance, instance)`; peers `connect()` by name and +> the kernel resolves + load-balances. `TIPC_ADDR_ID` is only +> ever an *observed* address (`getpeername()`), never a +> user-facing one.** + +This is exactly the "leverage the built-in discovery machinery" +ask in #378: publishing a bind *is* registration, and +`connect()` on a name *is* a lookup, with no registrar actor in +the loop. + +### 2.2 the struct + +```python +class TIPCAddress( + msgspec.Struct, + frozen=True, +): + _stype: int # TIPC "type" == service class + _instance: int # service instance within the type + _scope: int = TIPC_CLUSTER_SCOPE + # observed-only, never part of identity/equality-by-intent + maybe_node: int|None = None # from TIPC_ADDR_ID getpeername() + maybe_ref: int|None = None + + proto_key: ClassVar[str] = 'tipc' + unwrapped_type: ClassVar[type] = tuple[str, int] + def_bindspace: ClassVar[int] = TIPC_CLUSTER_SCOPE +``` + +**Unwrapped form** (the wire/`SpawnSpec` shape) — must be +uniquely matchable in `wrap_address()`: + +```python +def unwrap(self) -> tuple[str, int]: + # ('tipc::', instance) + return (f'tipc:{self._stype}:{self._scope}', self._instance) +``` + +i.e. `(str, int)` — *the same coarse shape as `TCPAddress`*, so +`wrap_address()`'s `case (str(), int())` would steal it. Two +options; **pick (a)**: + +- **(a) self-tagging prefix + a `case` ordered before the TCP + one**, guarded on `addr[0].startswith('tipc:')`: + ```python + case (str() as h, int()) if h.startswith('tipc:'): + cls = TIPCAddress + ``` + Cheap, string-y, but honest about the fact that + `UnwrappedAddress` is a deliberately-degenerate + `tuple[str, int|str]` (see the `_addr.py:43-65` TODO block). + It also keeps `TCPAddress.__post_init__`'s + `ipaddress.ip_address()` validation from being reached with a + non-IP host. +- (b) widen `UnwrappedAddress` to a 3-tuple for tipc. Rejected: + ripples into every `reg_addr` fixture, `_root_mailbox`, and + downstream (`piker`) config. + +Note `TIPCAddress` is the first backend where `.unwrap()` is +**not** a lossless view of the live socket — `maybe_node`/ +`maybe_ref` are observed metadata, exactly like +`UDSAddress.maybe_pid` (which is likewise excluded from +`.unwrap()`). Follow that precedent, including its `__repr__` +treatment (`_uds.py:242`). + +### 2.3 how to pick `_stype` and `_instance` + +- `_stype` = a `tractor`-reserved service class. TIPC reserves + 0..63 for internal use (`TIPC_TOP_SRV == 1`, + `TIPC_CFG_SRV == 0`). Use a module constant + `TRACTOR_STYPE: int = 0x74_72_00_00` ("tr\0\0") as the default + and make it overridable via `TIPCAddress._stype` so an app + can partition service classes. Document that two `tractor` + trees sharing a cluster **and** a `_stype` share a namespace. +- `_instance` for `get_root()`: `1616` — mirrors the + `TCPAddress.get_root()` port and the `registry@1616.sock` + UDS filename, so the "1616 is tractor's registrar" idiom + holds across all backends. +- `_instance` for `get_random()`: TIPC gives us no + kernel-assigned-instance analogue of `port=0`, so we must + choose. Use a *pure* fn of the actor identity so it is + reproducible and collision-free: + ```python + # 32-bit instance derived from the actor's uuid4 (+ pid when + # there's no live runtime, per the UDS precedent). + inst: int = int.from_bytes( + blake2b(seed.encode(), digest_size=4).digest(), + 'big', + ) + ``` + where `seed = f'{actor.aid.name}@{pid}'` if + `current_actor(err_on_no_runtime=False)` else + `f'{prefix}.{uuid4().hex[:8]}@{pid}'`. Must avoid the reserved + low range: `inst = 64 + (inst % (2**32 - 64))`. + ⚠️ *unlike* `port=0`, a collision here surfaces as a + successful-but-shared publication (TIPC allows multiple + binders on the same name and round-robins!) rather than + `EADDRINUSE`. That is a silent-crosstalk failure mode; §7 has + the test that proves the 4-byte digest is enough and §9 has + the mitigation if it isn't. +- `_scope`: `TIPC_NODE_SCOPE` for a same-host-only actor (the + UDS-equivalent), `TIPC_CLUSTER_SCOPE` (default) for + cluster-visible. **This is `.bindspace`**: + ```python + @property + def bindspace(self) -> int: + return self._scope + ``` + It is the honest analogue of "the set of hosts this bind is + reachable from", which is precisely the docstring in + `Address.bindspace`. (`TIPC_ZONE_SCOPE` is deprecated/aliased + to cluster in modern kernels — accept it on input, normalize + to cluster, log at `transport` level.) + +### 2.4 `is_valid` + +```python +@property +def is_valid(self) -> bool: + return ( + self._instance != 0 + and + self._stype not in _tipc_reserved_stypes # {0, 1, ...} + and + self._scope in (TIPC_NODE_SCOPE, TIPC_CLUSTER_SCOPE) + ) +``` + +--- + +## 3. Listener + stream + +### 3.1 `start_listener()` + +```python +async def start_listener( + addr: TIPCAddress, + backlog: int = 128, + **kwargs, +) -> SocketListener: + sock = trio.socket.socket( + socket.AF_TIPC, + socket.SOCK_STREAM, + ) + # publish the singleton name-range == "register the service" + await sock.bind(( + socket.TIPC_ADDR_NAMESEQ, + addr._stype, + addr._instance, + addr._instance, + addr._scope, + )) + sock.listen(backlog) + return SocketListener(sock) +``` + +Notes / hazards: + +- `bind()` on `AF_TIPC` is **not** a filesystem or port-table + operation and can't block on DNS, but keep it `await`ed + through `trio.socket` anyway for uniformity. +- `backlog=128` matching `_uds.start_listener()`'s hard-won + value (see its comment at `_uds.py:317-331` re: concurrent + deregistration storms). Do not use `1`. +- **no `close_listener()` needed** — nothing to unlink. Omit the + function entirely (contract §1.2: absence means implicit). + Withdrawal of the published name happens on socket close. +- ⚠️ `SocketListener.__init__` will try + `getsockopt(SOL_SOCKET, SO_ACCEPTCONN)`. If TIPC rejects it, + trio's `except OSError: pass` covers us. Assert this in a + unit test rather than assuming. +- Wrap the bind in a `_reraise_as_connerr()`-style `@cm` (copy + the `_uds.py:256` pattern) so `EADDRINUSE`-ish and + `EAFNOSUPPORT` become `ConnectionError` with the addr in the + message. `EAFNOSUPPORT` here means "kernel module not + loaded" and deserves a *specifically actionable* message: + `'TIPC unavailable — try `sudo modprobe tipc`\n'`. + +### 3.2 the `getsockname()` reconciliation + +`Endpoint.start_listener()` does +`if lstnr.socket.getsockname() != self.addr.unwrap(): self.addr = +self.addr.from_addr(unwrapped)`. + +For TIPC, `getsockname()` on a bound-but-listening socket +returns a `TIPC_ADDR_ID`-flavoured 5-tuple (the port id), *not* +the name-seq we bound. So the `!=` is **always true** and +`from_addr()` will be handed a 5-tuple. + +Handle it inside `TIPCAddress.from_addr()` — do **not** patch +`_server.py`: + +```python +@classmethod +def from_addr(cls, addr) -> TIPCAddress: + match addr: + # our own unwrapped form + case (str() as tag, int() as inst) if tag.startswith('tipc:'): + _, stype, scope = tag.split(':') + return TIPCAddress(int(stype), inst, int(scope)) + + # a kernel-observed TIPC_ADDR_ID 5-tuple: keep the + # *service* identity we already know and only annotate + # the observed port-id. + case (int() as atype, *rest) if atype == socket.TIPC_ADDR_ID: + ... +``` + +The `TIPC_ADDR_ID` case cannot reconstruct `(stype, instance)` +— that info isn't in a port id. So `from_addr()` alone is +insufficient for the reconciliation path. **Resolution**: make +`from_addr()` raise a clear `ValueError` for the bare +`TIPC_ADDR_ID` case, and instead prevent the reconciliation +from firing by having `start_listener()` return a listener +whose `getsockname()` we never need — i.e. land this two-line +upstream fix in `_server.py:664`: + +```python +if ( + (unwrapped := lstnr.socket.getsockname()) != self.addr.unwrap() + and + self.addr.rebind_from_sockname # ClassVar[bool] = True on tcp/uds +): +``` + +with `TIPCAddress.rebind_from_sockname: ClassVar[bool] = False` +(and `True` on `TCPAddress`/`UDSAddress`, preserving today's +behaviour exactly). Rationale: the reconciliation exists *only* +to learn the kernel-assigned port for `port=0` TCP binds (its +own comment says so, `_server.py:662`); TIPC has no such +late-binding, so opting out is semantically right rather than a +hack. **Land this as its own commit, ahead of the backend**, +with a test that `tcp`'s `port=0` behaviour is unchanged. + +Keep the observed port-id available anyway: annotate +`ep.addr = ep.addr.with_port_id(*getsockname()[1:3])` (a pure +`msgspec.structs.replace()` helper) purely for logging/repr. + +### 3.3 `MsgpackTIPCStream` + +```python +class MsgpackTIPCStream(MsgpackTransport): + address_type = TIPCAddress + layer_key: int = 4 + + @property + def maddr(self) -> Multiaddr|str: + return mk_maddr(self.raddr) + + def connected(self) -> bool: + return self.stream.socket.fileno() != -1 + + @classmethod + async def connect_to( + cls, + destaddr: TIPCAddress, + prefix_size: int = 4, + codec: MsgCodec|None = None, + **kwargs, + ) -> MsgpackTIPCStream: + sock = trio.socket.socket(AF_TIPC, SOCK_STREAM) + with close_on_error(sock): + # NOTE: connect by *name* -> kernel does the lookup, + # so this is our "discovery" call. + await sock.connect(( + socket.TIPC_ADDR_NAME, + destaddr._stype, + destaddr._instance, + 0, # domain: 0 == "anywhere in scope" + destaddr._scope, + )) + return cls( + trio.SocketStream(sock), + prefix_size=prefix_size, + codec=codec, + ) +``` + +- reuse `trio._highlevel_open_unix_stream.close_on_error` (the + UDS backend already imports it) or inline the equivalent + `try/except: sock.close(); raise`. +- `SO_/TIPC_` opts worth setting and documenting: + - `setsockopt(SOL_TIPC, TIPC_IMPORTANCE, TIPC_HIGH_IMPORTANCE)` + for the *parent<->child* lifetime channel — this is a real + win TIPC gives us that TCP can't: the runtime's + supervision channel can outrank bulk app traffic under + congestion. Wire it as a `connect_to(..., importance=...)` + kwarg defaulted from a module constant, and have + `_runtime.py`'s parent-chan path pass the high value **in a + follow-up** (don't couple it to this PR). + - `TIPC_CONN_TIMEOUT` — the kernel-side connect timeout; + leave at default, we have `trio` cancel scopes. + - `TIPC_DEST_DROPPABLE = 0` on the connection so undeliverable + msgs come back as errors rather than being silently dropped. +- **`connect_to()` on a name with no publisher**: TIPC returns + `ECONNREFUSED`/`EHOSTUNREACH` promptly (no SYN-timeout wait), + which is *better* discovery-ping behaviour than TCP. Confirm + the errno and make sure it surfaces as `ConnectionError` + (contract §4 — the registrar ping path depends on it). + +### 3.4 `get_stream_addrs()` + +```python +@classmethod +def get_stream_addrs(cls, stream) -> tuple[TIPCAddress, TIPCAddress]: + sock = stream.socket + # both return TIPC_ADDR_ID 5-tuples for a connected sock + l_id = sock.getsockname() + r_id = sock.getpeername() + ... +``` + +Problem: neither end's port-id tells us the *service name*. The +`laddr`/`raddr` are used for logging, `Channel.raddr`, +`Server._peers` keying-adjacent repr, and `maddr`. Design: + +- the **connecting** side knows the destaddr it dialled → + `connect_to()` overrides `_raddr` after construction with the + known-good `TIPCAddress`, exactly as + `MsgpackUDSStream.connect_to()` does for the peer-pid case + (`_uds.py:539-543`). +- the **accepting** side does not know the peer's service name + from the socket. Two honest options: + - **(a) accept it: `raddr` carries only `(node, ref)`** via + `maybe_node`/`maybe_ref`, `_stype/_instance` set to a + sentinel `-1`, and `__repr__` renders + `TIPCAddress[:]`. The `Aid` from the + handshake already gives us the peer's logical identity, so + nothing in the runtime actually *needs* the peer's service + name. **Recommended.** + - (b) piggyback the peer's own bound name in the handshake. + Rejected for this PR: touches `Aid`/msg-spec. +- `laddr` on the accepting side: the `Endpoint` knows its own + `addr`; but `get_stream_addrs()` is a `@classmethod` with only + the stream. Use `TIPC_ADDR_ID` for `laddr` too and let + `Endpoint.peer_tpts` keying (which is by *peer* addr) still + work. Verify nothing asserts `laddr == ep.addr` — grep for + `.laddr` uses before committing (`_server.py`'s + `con_status` logging, `Channel.pformat()`). + +--- + +## 4. Multiaddr representation + +There is no `/tipc` in the multiaddr protocol table. Interim +grammar, mirroring how `uds` maps to the spec-legal `/unix`: + +``` +/tipc// # scope implied = cluster +/tipc/// # explicit +``` + +- `_tpt_proto_to_maddr['tipc'] = 'tipc'` and a `mk_maddr()` + `case 'tipc':` building the above. +- `parse_maddr()` gets `case ['tipc']:` — but note + `py-multiaddr` will reject an unregistered protocol name + outright, so this **requires an upstream registration** (same + track as the `wg` work, gh #483 / + multiformats/py-multiaddr#107). Until that lands: + - `MsgpackTIPCStream.maddr` returns the **`str`** form (the + `MsgTransport.maddr` return type is already + `Multiaddr|str`, and `MsgpackUDSStream.maddr` already + exercises the `str` branch), and + - `parse_maddr()` special-cases the `/tipc/` prefix *before* + handing the string to `Multiaddr()`. + Document this as the reason gh #443's "standardize on + returning `Multiaddr` everywhere" item stays blocked. + +Propose `/tipc/` upstream as: name `tipc`, code TBD, size +variable, value `::` — or as three +composed protos. Prefer *one* proto with a structured value so +the maddr stays 2-segment like `/unix/...`. + +--- + +## 5. Discovery: the actually-interesting part + +Two independently-shippable layers. **Layer A is in scope for +the first PR; layer B is a fast-follow.** + +### 5.1 Layer A — "discovery by bind" (free) + +Because `bind(TIPC_ADDR_NAMESEQ)` publishes and +`connect(TIPC_ADDR_NAME)` resolves, a `tractor` tree whose +`registry_addrs` are TIPC service names needs **no registrar +liveness at all** for the connect path: `find_actor()`'s +"connect to the registrar and ask" becomes "connect to the +service name directly". Concretely: + +- `tractor.discovery._api.find_actor()` etc. keep working + unchanged (they go through the registrar), *and* +- a new, TIPC-only fast path becomes possible: derive an actor's + service name from its `(name, uuid)` and dial it without any + registrar hop. + +Do **not** build the fast path in PR 1. Instead, prove the +property with a test (§7.4) and file the follow-up: it changes +`discovery` semantics (name→instance derivation must be a +documented, stable, cross-language-able hash) and deserves its +own design. + +### 5.2 Layer B — the topology service (`TIPC_TOP_SRV`) + +This is what makes #378's "end game cluster proto" claim real: +a *subscription* to name-table events, i.e. push-based +`register`/`deregister` for free, replacing the registrar's +polled `find_actor()`. + +Mechanics (verify each field against +`linux/include/uapi/linux/tipc.h` + `net/tipc/topsrv.c` at +implementation time — the struct layout below is from the uapi +header and the byte-order caveat is real): + +```python +# SOCK_SEQPACKET connected to the topology server +sock = trio.socket.socket(AF_TIPC, SOCK_SEQPACKET) +await sock.connect(( + socket.TIPC_ADDR_NAME, + socket.TIPC_TOP_SRV, # == 1 + socket.TIPC_TOP_SRV, + 0, +)) + +# struct tipc_subscr { +# struct tipc_name_seq seq; /* 3 * __u32: type, lower, upper */ +# __u32 timeout; /* TIPC_WAIT_FOREVER == ~0 */ +# __u32 filter; /* TIPC_SUB_{PORTS,SERVICE,CANCEL} */ +# char usr_handle[8]; +# } /* == 28 bytes */ +_SUBSCR_FMT: str = '=IIIII8s' # ⚠ 5*I is 20 -> use '=5I8s' +``` + +- **byte order**: the topology server historically accepts both + host and swapped order and auto-detects; modern kernels are + strict-ish. Pack native (`'='`) first, and if the server + closes the connection immediately, retry with `'>'`. Encode + that as a one-time probe helper + `_detect_topsrv_endianness()` cached at module level — and + put a `# ?TODO` pointing at `net/tipc/topsrv.c` for someone + to make it deterministic. +- **events**: `struct tipc_event` is `event: u32`, + `found_lower: u32`, `found_upper: u32`, + `port: {ref: u32, node: u32}`, then the 28-byte subscription + echo → 40 bytes. `event ∈ {TIPC_PUBLISHED, TIPC_WITHDRAWN, + TIPC_SUBSCR_TIMEOUT}`. +- **trio shape** — this is where the "nearly-functional, + modern-async" style pays off; expose it as an `@acm` yielding + a `trio` receive-channel of typed events, *not* a class: + +```python +@acm +async def open_topology_events( + stype: int = TRACTOR_STYPE, + lower: int = 0, + upper: int = 0xFFFFFFFF, + filter: int = TIPC_SUB_SERVICE, + timeout: int = TIPC_WAIT_FOREVER, + buf_size: int = 64, +) -> AsyncGenerator[ + trio.MemoryReceiveChannel[TIPCNameEvent], + None, +]: + ... +``` + + with `TIPCNameEvent(msgspec.Struct, frozen=True)` fields + `kind: Literal['published','withdrawn','timeout']`, + `addr: TIPCAddress`, `node: int`, `ref: int`. One + `trio.lowlevel`-free implementation: a nursery-spawned reader + task doing `await sock.recv(40)` in a loop and + `send_nowait()`ing decoded events, with the `@acm` closing the + socket on exit → reader gets `ClosedResourceError` → cancel + scope collapses. Standard `tractor` `@acm` discipline. +- **consumer**: `tractor/discovery/_registry.py` gains an + optional "watch" mode so a registrar (or any actor) can keep + a live view of the actor set without polling. Sketch the + integration in the follow-up issue; do not wire it in PR 1. +- **`SOCK_SEQPACKET` is fine here** because this socket never + goes through `MsgpackTransport` — it's a plain trio socket + used with `recv()`. The contract's "`SOCK_STREAM` only" + constraint applies to `MsgTransport` streams, not to this. + +--- + +## 6. Commit sequencing (each independently reviewable + green) + +1. `_server.py`: add `Address.rebind_from_sockname: + ClassVar[bool]`, gate the `getsockname()` reconciliation on + it, `True` for tcp/uds. Test: tcp `port=0` unchanged. +2. `tractor/ipc/_tipc.py`: `TIPCAddress` + `is_tipc_available()` + predicate + `start_listener()`. No transport yet. + Tests: address round-trip (`unwrap`/`from_addr`/`wrap_address`), + `get_random()` uniqueness, bind/listen + `SO_ACCEPTCONN` + tolerance, `EAFNOSUPPORT` → actionable `ConnectionError`. +3. `MsgpackTIPCStream` + `connect_to()` + `get_stream_addrs()`. + Test: two `trio` tasks in one proc exchange a msg over + `Msgpack` framing (no `tractor` runtime). +4. registration tables (contract §2 items 1-6, 9) + + `pyproject.toml` mark/extra. Test: full suite under + `--tpt-proto tipc` (§7.3). +5. maddr support (`str` form + prefix special-case) + docs. +6. `open_topology_events()` @acm + its tests (layer B). +7. docs page + `docs/` example. + +Per project convention, a reproducing/guard test lands in its +own commit **before** the fix it guards. + +--- + +## 7. Testing + +### 7.1 the capability predicate (in `_tipc.py`, public) + +```python +def is_tipc_available() -> bool: + ''' + True iff this kernel can create an `AF_TIPC` socket, i.e. + the `tipc` module is loaded. + + ''' + try: + socket.socket(socket.AF_TIPC, socket.SOCK_STREAM).close() + return True + except OSError: + return False +``` + +Cache it in a module global (it can't change without a +`modprobe`, and a cold call costs a syscall). Pure predicate, no +side effects, no logging. + +### 7.2 gating + +- `pytest.mark.tipc` registered in `pyproject.toml`. +- module-level + `pytestmark = pytest.mark.skipif(not is_tipc_available(), + reason='`tipc` kernel module not loaded (`modprobe tipc`)')` + in `tests/ipc/test_tipc.py`. +- `--tpt-proto tipc` with no module must fail **loudly and + early** with the actionable message, not with 400 confusing + timeouts. Add the check to the `tpt_protos` fixture's existing + per-proto validation loop (`_testing/pytest.py:795`): if the + chosen `Address` type exposes an `is_available()`-style + classmethod, call it and `pytest.fail()` with its reason. + Generalize (don't special-case tipc) — plans 02/03 need the + same hook. + +### 7.3 CI + +- add a job matrix entry `--tpt-proto tipc` that runs + `sudo modprobe tipc` in a `before` step. GH's + `ubuntu-latest` runners do allow `modprobe tipc` (the module + ships with the standard Ubuntu kernel package); verify in a + throwaway workflow before wiring the matrix. If it turns out + to be unavailable, fall back to a container job with + `--privileged`/`--cap-add NET_ADMIN`, and mark the job + `continue-on-error` until it's proven stable. +- cross-node TIPC (bearer) cannot be CI'd; cover it with a + documented manual smoke test in the docs page, in the style + of gh #482's LAN examples. + +### 7.4 backend-specific tests worth writing + +- **name-publication is discovery**: bind a listener on + `(stype, inst)`, then from a second task `connect()` by name + and assert it lands — *without* any `tractor` registrar. +- **`get_random()` collision resistance**: 10k `get_random()` + calls with no live runtime → 10k distinct `_instance`s. + (This is the silent-crosstalk risk from §2.3; if the 4-byte + digest ever collides in this test, escalate to §9.) +- **round-robin surprise**: two listeners bound to the *same* + `(stype, inst)` both succeed (TIPC allows it) and connects + distribute. Assert the observed behaviour and reference it + from the `get_random()` docstring so the next reader knows + why the hash matters. +- **scope isolation**: a `TIPC_NODE_SCOPE` bind is not visible + to a cluster-scope lookup from another node (manual/marked). +- **importance opt** round-trips via `getsockopt`. +- **graceful + abrupt close** produce `TransportClosed` with the + same `loglevel` classification as tcp/uds — i.e. re-run the + relevant `tests/ipc/test_each_tpt.py` cases parametrized over + the new proto rather than writing new ones. + +--- + +## 8. Deployment / docs deliverable + +A `docs/` page (and/or an `examples/` script) covering: + +```bash +# single host, node-scope only +sudo modprobe tipc +tipc node get addr + +# multi-host over ethernet (pairs beautifully with plan 03's wg) +sudo tipc bearer enable media eth device eth0 +# ...or over UDP when L2 isn't available: +sudo tipc bearer enable media udp name uc localip 10.0.11.1 +tipc link list +tipc nametable show # <- see tractor's published services! +``` + +`tipc nametable show` displaying live `tractor` actors is the +single best demo this backend has; lead with it. + +--- + +## 9. Known risks + escalations + +| risk | mitigation | +| --- | --- | +| `_instance` hash collision → silent crosstalk (two actors share a service name, TIPC round-robins connects between them) | §7.4 test; if it bites, add a post-bind verification handshake, or bump to a 6-byte digest folded into `(stype_low, instance)` | +| kernel/module unavailability everywhere (dev boxes, macOS, CI) | hard gating (§7.2); TIPC is explicitly an *opt-in cluster* transport, never a default | +| `getsockname()` returns port-id not name | the `rebind_from_sockname` opt-out (§3.2), landed first | +| unregistered `/tipc` multiaddr proto | `str` maddr fallback (§4) + upstream track gh #483 | +| stale docs (#378 notes tipc.io docs may be out of date) | treat `include/uapi/linux/tipc.h` + `net/tipc/` as the only normative source; cite file+symbol in code comments | +| `SOCK_SEQPACKET` topology framing byte-order | probe helper + `?TODO` (§5.2) | + +## 10. Follow-up issue seeds + +- registrar-less discovery fast path via name derivation (§5.1) +- `TIPC_TOP_SRV`-driven push registry in + `discovery/_registry.py` (§5.2) +- `TIPC_IMPORTANCE` for the parent<->child lifetime channel + (§3.3) — genuinely novel supervision QoS, no other backend + can do it +- TIPC multicast / group messaging as a *broadcast* transport + for `tractor.trionics` fan-out (explicitly not `MsgTransport`) +- dual-link resiliency / multi-homing (#378's "hybrid dual link") + once bearers are scripted in the docs From 07ba2be07d91a3f6c68a233a420172ccd176e7c0 Mon Sep 17 00:00:00 2001 From: goodboy Date: Tue, 11 Aug 2026 23:12:03 -0400 Subject: [PATCH 03/18] Add `QUIC`-via-`iroh` tpt-backend plan MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Plan doc for gh #353. Picks `iroh` (the `uniffi` FFI pkg) over `aioquic`/`quiche` bc node-id addressing + hole-punching + relay fallback is the whole point; `aioquic` stays documented as the fallback since ~90% of the adapters here are reusable against a sans-io core. Deats, - the layering: iroh `Endpoint` per actor, `Connection` per peer (pooled via `trionics.maybe_open_context()`, not a hand-rolled cache), one bi-stream per `Channel`. 4-byte prefix framing stays so `MsgpackTransport` is untouched. - `_uniffi_trio.py`: uniffi only uses `asyncio` as the executor for its rust-future poll loop, so a ~40-line `TrioToken.run_sync_soon()` bridge replaces it. Spells out the real hazards — strong ref on the `ctypes` trampoline, poll-code propagation, and a *bounded* shielded cancel-drain so a wedged rust future can't make an actor un-cancellable. - `IrohAddress` w/ ALPN as the `.bindspace`, the `(str, str)` unwrapped form's collision w/ the UDS match-case, and why `get_root()` needs a persisted secret key -> a lazy `default_lo_addrs()` + a pure-getter/explicit-setter split. - `QuicMsgStream(trio.abc.HalfCloseableStream)` + `QuicListener(trio.abc.Listener)`, incl. the exact EOF/reset/use-after-close semantics `_transport.py` already match-cases on, and hanging the acceptor tasks off the existing `Endpoint.listen_tn`. - a prep-PR boundary: annotation widening, the shared `rebind_from_sockname` gate and a `tpt_key`-based `transport_from_stream()` dispatch, all landable w/ tcp/uds as the only backends. Further, notes this is our first tpt w/ real transport security + peer auth, so an inbound node-id allowlist hook belongs here — and that it says nothing about the other backends. (this patch was generated in some part by `claude-code` using `claude-opus-5` (`anthropic`)) --- ai/tpt-backends/02_quic_iroh_backend.md | 566 ++++++++++++++++++++++++ 1 file changed, 566 insertions(+) create mode 100644 ai/tpt-backends/02_quic_iroh_backend.md diff --git a/ai/tpt-backends/02_quic_iroh_backend.md b/ai/tpt-backends/02_quic_iroh_backend.md new file mode 100644 index 00000000..53b93999 --- /dev/null +++ b/ai/tpt-backends/02_quic_iroh_backend.md @@ -0,0 +1,566 @@ +# Plan 02 — QUIC backend via `iroh` FFI, uniffi-async rewritten onto `trio` + +Tracks gh [#353]. Prereq reading: +[`00_shared_backend_contract.md`](./00_shared_backend_contract.md). + +**Thesis**: the value of `iroh` over "just QUIC" is +`NodeId`-addressed, NAT-traversing, relay-fallback endpoints — +i.e. a `tractor` actor tree that spans hosts *without* a +reachable listening socket. The cost is that `iroh`'s python +surface is `uniffi`-generated **asyncio** and its listener is not +a socket. This plan spends its complexity budget in exactly two +places: a `trio`-native uniffi future bridge, and a +`trio.abc.Listener`/`Stream` adapter pair. Everything else is +contract boilerplate. + +[#353]: https://github.com/goodboy/tractor/issues/353 + +--- + +## 1. Library selection (decided, with the rejected alternatives) + +**Chosen: `iroh` (PyPI, from `n0-computer/iroh-ffi`), pinned to +a single minor.** The `iroh` python package is a `uniffi` +binding over the rust `iroh` crate (QUIC via `quinn`/`noq`). + +Rejected, and why — record these so the next implementer doesn't +relitigate: + +- **`aioquic`** (sans-io + asyncio): genuinely trio-portable + (`hypercorn` already pairs its sans-io core with a trio UDP + server, see the links in #353) and dependency-light. But it + gives us *only* QUIC — no NodeId identity, no hole punching, + no relay. We'd be reimplementing iroh's whole reason for + existing. **Keep as the documented fallback** if the FFI + bridge (§2) proves unmaintainable; the `MsgTransport` and + `Listener` adapters from §3 are ~90% reusable against an + `aioquic` core, which is a deliberate design property of this + plan. +- **`quiche` / `quinn` via a hand-rolled PyO3 ext**: strictly + more work than reusing `iroh-ffi`, and puts us in the + build-wheels business. +- **`trio-asyncio`**: viable *shortcut* to run the asyncio-shaped + bindings under trio, and `tractor` already ships + infected-asyncio machinery (`tractor.to_asyncio`, + `tests/test_infected_asyncio.py`). Rejected as the *primary* + design because it makes every IPC send/recv cross a + loop-boundary shim in the hot path, and because #353 asks + explicitly for the asyncio support to be "rewritten for trio". + **But**: build it first as the throwaway spike (§6 step 0) to + de-risk the iroh API surface before writing the bridge. + +Version pinning: `iroh` moves fast and has had breaking +API renames across minors. Pin `iroh>=X.Y, str` | | +| node addr (relay + direct) | `iroh.NodeAddr` | | +| dial | `await endpoint.connect(node_addr, alpn)` | | +| accept conn | `await endpoint.accept()` | | +| open bi-stream | `await conn.open_bi()` | | +| accept bi-stream | `await conn.accept_bi()` | | +| send | `await send_stream.write_all(b)` | | +| recv | `await recv_stream.read(n) -> bytes\|None` | | +| half-close | `await send_stream.finish()` | | + +--- + +## 2. The `trio`-native uniffi future bridge (`tractor/ipc/_uniffi_trio.py`) + +### 2.1 what uniffi actually generates + +`uniffi`'s async support does not use asyncio *semantically* — +it uses asyncio only as the *executor* for a poll loop. The +generated python for an `async fn` is, in shape: + +1. call `_uniffi_..._(...)` → returns an opaque + `RustFuture` handle (a `void*`/`u64`). +2. loop: call + `ffi_..._rust_future_poll_(handle, callback, callback_data)`. + The callback is a C-ABI fn pointer invoked **from an + arbitrary rust thread** with a poll-result code + (`READY`/`MAYBE_READY`). +3. the generated glue's callback resolves an + `asyncio.Future` via `loop.call_soon_threadsafe(...)`; the + coroutine awaits it, then re-polls. +4. on ready: `ffi_..._rust_future_complete_(handle, + &call_status)` → the value; then + `ffi_..._rust_future_free_(handle)`. + +**The asyncio dependency is confined to step 3.** That is the +whole insight: the bridge is ~40 lines. + +### 2.2 the trio version + +```python +async def await_rust_future( + poll: Callable, # ffi_..._rust_future_poll_ + complete: Callable, # ffi_..._rust_future_complete_ + free: Callable, # ffi_..._rust_future_free_ + handle: int, + lift: Callable[[Any], Any], +) -> Any: + ''' + Drive a `uniffi` rust-future to completion on the current + `trio` task, bridging rust-thread wakeups via + `TrioToken.run_sync_soon()`. + + ''' + token = trio.lowlevel.current_trio_token() + while True: + wake = trio.Event() + # NOTE, invoked from a *rust* thread! + def _cb(_data, poll_code): + token.run_sync_soon(wake.set) + + cb = _UNIFFI_FUTURE_CALLBACK(_cb) # keep a strong ref! + poll(handle, cb, 0) + await wake.wait() + if : + break + try: + status = _UniffiRustCallStatus.default() + res = complete(handle, status) + _uniffi_check_call_status(status) # reuse generated helper + return lift(res) + finally: + free(handle) +``` + +Critical details, each a real bug if missed: + +- **`token.run_sync_soon()` is the only trio API callable from a + foreign thread**, and it is documented as such. Use it; do + *not* use `trio.from_thread.run_sync` (requires a trio thread + context) and do not touch the `Event` directly from the + callback. +- **the poll code must reach the trio side.** Capture it in a + `nonlocal`/1-slot list written by the callback *before* + `run_sync_soon`, since the callback owns the value. Handle + `MAYBE_READY` by re-polling (the loop above does). +- **keep the `ctypes` callback object alive** across the await — + a GC'd `CFUNCTYPE` trampoline is a segfault. Bind it to a + local *and* make sure the local outlives the `poll()` call + window. +- **cancellation.** `await wake.wait()` is a trio checkpoint, so + a `Cancelled` can fire while rust still owns the future. On + cancel we must still `free(handle)` — and per uniffi, the + correct sequence is to call the generated + `ffi_..._rust_future_cancel_(handle)` then continue + polling to completion before `free`. Wrap the whole thing so + the cancel path does: + `with trio.CancelScope(shield=True): cancel(handle); ; free(handle)`. **Bounded** shield (add a + `trio.move_on_after()` with a module-level constant) so a + wedged rust future can't make an actor un-cancellable — + `tractor` is SC-first and an unbounded shield here would + violate that. +- **`trio.lowlevel.current_trio_token()`** must be captured on + the trio side (not in the callback). + +### 2.3 how to apply it to the generated bindings + +Do **not** fork/vendor the generated `iroh` python. Instead ship +a *narrow* re-dispatch shim: + +- write `tractor/ipc/_uniffi_trio.py` with `await_rust_future()` + plus a `@cm patch_uniffi_for_trio()` that monkey-patches the + generated module's single async-driver entrypoint (in current + uniffi that's `_uniffi_rust_call_async` / `_rust_call_async`, + one function) to the trio implementation. +- verify at import time that the expected symbol exists and + raise a clear, actionable error naming the pinned `iroh` + version if not. A silent fallback to asyncio would be a + nightmare to debug. +- **plan for this to break on `iroh`/`uniffi` upgrades.** Mitigate + with (a) a unit test that drives one trivial `iroh` async call + under bare `trio.run()` and asserts no event loop was ever + created (`asyncio.get_event_loop_policy()` untouched / + `asyncio._get_running_loop() is None`), and (b) a docstring + pointing at the uniffi codegen template this mirrors. + +If step 0 reveals the generated code is *structurally* hostile +to this (e.g. `asyncio` imported and used at module scope for +more than the driver), fall back to option (b): run iroh under +`tractor.to_asyncio` infected mode and open the follow-up to +revisit. Say so in the PR rather than fighting it. + +--- + +## 3. Mapping QUIC onto `MsgTransport` + +### 3.1 the layering decision + +QUIC natively multiplexes streams inside one connection. The +mapping that preserves *all* existing `tractor` semantics with +the least new code: + +``` +iroh Endpoint == one per actor (process) -> the "listener" +iroh Connection == one per peer actor -> pooled +iroh bi-stream == one `Channel`/`MsgTransport` -> 1:1 +``` + +- keep the 4-byte ` (node_id_str, alpn_str)`** — a `(str, str)` + tuple, which is *unambiguously distinct* from + `TCPAddress`'s `(str, int)`. But careful: + `wrap_address()`'s UDS case is + `case (_, filename) if type(filename) is str` — which + **already catches `(str, str)`**. So the iroh `case` MUST be + ordered *before* the UDS case and guarded, e.g. + `case (str() as nid, str() as alpn) if _is_node_id(nid):` + with `_is_node_id()` a cheap length+alphabet check. Add a + regression test asserting a UDS `(dir, filename)` pair still + wraps to `UDSAddress` — this is the exact "wrong transport + loaded" hazard `_addr.py:214` warns about. +- `.bindspace` → `self._alpn`. This is the honest analogue: + the ALPN is the set of endpoints willing to talk to you, and + two `tractor` deployments sharing an iroh network are + separated by ALPN exactly as two UDS deployments are + separated by directory. Include a `tractor` version/proto + epoch in the default ALPN so incompatible runtimes can't + handshake. +- `.is_valid` → node-id parses, alpn non-empty. +- **`get_root()` is the hard one.** There is no + well-known-port analogue: an iroh node id is a *keypair*, so + "the host's default registrar addr" requires a *persisted + secret key*. Design: + - the root/registrar's secret key lives at + `get_rt_dir() / 'iroh_registrar.key'` (0600), created on + first use. + - `get_root()` must stay **pure and import-time-safe** + (contract §2.3: `_default_lo_addrs` is built at import!). + So `get_root()` *reads* the key file if present and + otherwise returns an `IrohAddress` with + `_node_id=''`/sentinel, and the **generation** happens in + an explicit sibling — `ensure_registrar_key() -> + IrohAddress` — called from the listen path. Pure getter, + explicit setter; do not smuggle key generation into + `get_root()`. + - this almost certainly means `_default_lo_addrs` must become + lazy for this backend. **Land that refactor as its own prep + commit** (a `default_lo_addrs()` that computes per-call + instead of the import-time dict) — it also unblocks plan + 03's netns-scoped defaults. +- `get_random()`: generate a fresh `SecretKey` per subactor and + return its node-id. Note this runs post-fork pre-listen + (contract §4) and costs an ed25519 keygen (~µs, fine). The + *secret* can't live in a frozen `Address`, so it must be + stashed where the listen path can find it: a module-level + `dict[node_id, SecretKey]` populated by `get_random()` and + consumed+popped by `start_listener()`. Ugly but honest; + document it and note the alternative (thread the key through + `Endpoint`) as a follow-up. + +#### 3.2.1 `proto_key`: `'iroh'` vs `'quic'` + +Use **`'quic'`** for the `proto_key`/`--tpt-proto` name and +name the module `_quic.py`, with `iroh` as the *implementation*. +Rationale: it keeps the door open for the `aioquic` fallback +(§1) without a user-visible rename, and it matches how `uds` is +a proto name rather than a lib name. Put `iroh`-specific bits +behind an internal `_iroh` submodule if the file gets big. + +### 3.3 the `trio.abc` adapters — where the real work is + +Contract §3 says a non-socket backend needs three upstream +generalizations. Land them **as a prep PR, before any iroh +code**, so they can be reviewed on their own merits with +tcp/uds still the only backends: + +1. **`Endpoint.start_listener()` must not assume + `.socket.getsockname()`.** Use the same + `Address.rebind_from_sockname: ClassVar[bool]` gate that + plan 01 §3.2 introduces — coordinate so it lands once. (If + plan 01 lands first, this is free.) +2. **`transport_from_stream()` (`_types.py:92`) must not assume + `trio.SocketStream`.** Replace the `sock.family` match with: + check `isinstance(stream, trio.SocketStream)` → existing + family match; else look for a + `stream.tpt_key: ClassVar[MsgTransportKey]` attribute on the + adapter and use it. Keeps the existing path byte-identical + and makes new stream types self-describing (a much better + shape than growing an `isinstance` ladder). +3. **type annotations**: `handle_stream_from_peer(stream: + trio.SocketStream)` → `trio.abc.Stream`; `Endpoint._listener: + SocketListener|None` → `trio.abc.Listener|None`; + `MsgTransport.stream: trio.SocketStream` → + `trio.abc.Stream`. Annotation-only, zero behaviour change. + +Then the adapters: + +```python +class QuicMsgStream(trio.abc.HalfCloseableStream): + ''' + A single `iroh` bi-directional QUIC stream presented as + a `trio` byte-stream so `MsgpackTransport` can frame over + it unmodified. + + ''' + tpt_key: ClassVar[MsgTransportKey] = ('msgpack', 'quic') + + def __init__(self, conn, send, recv) -> None: ... + async def send_all(self, data: bytes) -> None: ... + async def wait_send_all_might_not_block(self) -> None: ... + async def receive_some(self, max_bytes: int|None = None) -> bytes: ... + async def send_eof(self) -> None: ... + async def aclose(self) -> None: ... +``` + +Non-negotiable behaviours (each maps to a `match` case that +already exists in `_transport.py` and must keep working): + +- `receive_some()` returns `b''` at clean EOF → + `MsgpackTransport._iter_packets()` sees `header == b''` and + raises `TransportClosed(loglevel='transport')`. **This is the + graceful-disconnect path the whole runtime relies on**; get it + right first. +- a reset/aborted stream → raise `trio.BrokenResourceError`. +- use after local close → raise `trio.ClosedResourceError` + (ideally with `'another task closed this fd'`-equivalent text + absent, so the `raise_on_report` branch at + `_transport.py:290` stays quiet). +- `send_all()` on a closed peer → `trio.BrokenResourceError`. +- honour `trio`'s one-task-per-direction rule: guard with + `trio._util.ConflictDetector` equivalents (or just document + + assert), because `MsgpackTransport` already serializes sends + with a `StrictFIFOLock` but recvs are single-task by + construction. +- **buffering**: if iroh's `read()` doesn't support + "read up to n", `receive_some()` must maintain an internal + leftover buffer. Note `MsgpackTransport` wraps us in + `tricycle.BufferedReceiveStream` anyway, so `receive_some()` + just needs *some* nonzero-progress contract. + +```python +class QuicListener(trio.abc.Listener): + ''' + Accepts iroh `Connection`s and yields one `QuicMsgStream` + per accepted bi-stream, so `trio.serve_listeners()` spawns + one `handle_stream_from_peer()` per `Channel`. + + ''' + async def accept(self) -> QuicMsgStream: ... + async def aclose(self) -> None: ... +``` + +The accept-side subtlety: `trio.abc.Listener.accept()` yields +one stream per call, but iroh gives us *connections* which then +yield *streams*. So `QuicListener` needs an internal +`trio.MemoryReceiveChannel[QuicMsgStream]` fed by a background +task-pair (one task accepting connections, one per connection +accepting bi-streams). `trio.abc.Listener` has no nursery, so: +make the listener **constructed by an `@acm`** that owns the +nursery, and have `start_listener()` be that `@acm`'s driver. + +⚠️ this collides with `Endpoint.start_listener()` being a plain +`async def` returning a listener. Two options: +- **(a)** hang the nursery off the `Endpoint`'s existing + `listen_tn` — `_serve_ipc_eps()` already creates `listen_tn` + and passes it into every `Endpoint` (`_server.py:1063-1074`), + and `Endpoint.listen_tn` is right there. So + `start_listener()` can `self.listen_tn.start_soon(...)` the + acceptor tasks. **Recommended**: no upstream signature change, + correct lifetime (dies with the ep group), and it's why + `listen_tn` is on the struct in the first place. +- (b) change `start_listener()` to a `@acm`. Bigger blast + radius; only if (a) proves insufficient. + +Since `start_listener()` is called via +`inspect.getmodule(addr)` with only `addr=` (contract §1.3), +option (a) needs the `Endpoint` itself. Either add `ep=` to the +module-level `start_listener()` call signature (all backends +ignore it except quic → small upstream change, do it as part of +the prep PR and make it keyword-only with a default) or have +`QuicListener.accept()` lazily spawn via +`trio.lowlevel.current_task().parent_nursery` (**rejected** — +fragile, implicit). Do the explicit `ep=` kwarg. + +### 3.4 `maddr` + +Multiaddr already standardizes the pieces: + +``` +/ip4//udp/

/quic-v1 # direct +/ip4//udp/

/quic-v1/p2p/ # direct + identity +/dns//tcp/443/tls/ws/p2p/ # relay-ish +``` + +- primary form: `/p2p/` alone is a legal maddr and is + the *only* required component for iroh dialling — relay + + direct addrs are discovery hints. So `mk_maddr()` emits + `/p2p/` and, when known, prefixes the direct + `/ip4/../udp/../quic-v1/`. +- `/p2p/` values are multihash-encoded peer ids; an iroh node-id + is a raw ed25519 key. Converting requires the identity + multihash + libp2p key protobuf wrapper. **Decide**: emit the + raw node-id under a *tractor-local* `/iroh/` segment + (needs upstream registration, same track as `wg`/`tipc`, + gh #483) rather than pretending to be a libp2p peer-id we + can't round-trip. Return the `str` form until upstream lands + (`MsgTransport.maddr` is `Multiaddr|str`). +- this backend is the strongest argument for gh #443's + **tunnelled/composed maddr** item: `/ip4/../udp/../quic-v1/..` + *is* a composed stack. Cross-reference plan 03 §5 so the two + grammars land compatibly. + +--- + +## 4. Discovery integration + +- iroh's node-id addressing means the `tractor` registrar can + hold `IrohAddress`es that are **reachable from anywhere** with + no port-forwarding — that is the headline feature. The + registrar itself works unchanged. +- iroh has its own discovery (DNS/pkarr/mdns). **Out of scope**; + note in the follow-up that `tractor.discovery` could + eventually delegate to it, which would be the direct analogue + of plan 01's TIPC-topology idea. +- relay servers: default to n0's public relays for the demo, + document self-hosting (docs.iroh.computer's dedicated-infra + page is linked from #353), and make the relay set a + `start_listener()` kwarg. + +## 5. Security note + +QUIC is TLS-1.3-always and iroh authenticates by node-id, so +this backend is the first `tractor` transport with real +transport security and peer authentication. Two things follow: +1. an **allowlist hook** — an actor should be able to reject + inbound connections from unknown node-ids *before* the + `Aid` handshake. Natural home: a predicate kwarg on + `start_listener()`, evaluated in `QuicListener`'s connection + acceptor task. Sketch it; ship it in PR 1 if cheap (it is). +2. do **not** claim any security property for the other + backends by association. `tcp`/`uds`/`tipc` remain + unauthenticated; that's what plan 03 (wg) is for. + +## 6. Commit sequencing + +0. **spike (throwaway, not committed)**: drive iroh under + `trio-asyncio`/`tractor.to_asyncio`, echo bytes over a + bi-stream between two procs. Fills in §1.1. Timebox it. +1. prep PR: annotation widening + `rebind_from_sockname` gate + + `transport_from_stream()` `tpt_key` dispatch + `ep=` kwarg on + `start_listener()` + lazy `default_lo_addrs()`. **No new + backend.** Full suite green on tcp *and* uds. +2. `_uniffi_trio.py` + its tests (drive one iroh async call + under bare `trio.run()`; assert no asyncio loop; assert + cancellation frees the future). +3. `QuicMsgStream` + tests against a *loopback* iroh endpoint + pair in one process (no `tractor` runtime): send/recv, clean + EOF → `b''`, reset → `BrokenResourceError`, use-after-close + → `ClosedResourceError`. +4. `QuicListener` + `start_listener()` + `IrohAddress` + + key-file mgmt. +5. `MsgpackQuicStream(MsgpackTransport)` + `connect_to()` + + `maybe_open_context()` connection pooling. +6. registration tables + `--tpt-proto quic` + full suite. +7. maddr + docs + a two-host example (pairs with #482's format). + +## 7. Testing + +- capability predicate `is_quic_available()` → `iroh` importable + *and* the uniffi driver symbol present at the pinned version. + Same `pytest.fail`-early hook as plan 01 §7.2. +- **the acceptance bar is the same**: whole suite green under + `--tpt-proto quic`. Expect this to shake out real bugs in the + adapters (esp. teardown ordering and `TransportClosed` + classification) — that's the point. +- expect to need **timeout headroom**: iroh endpoint bind + + first connect (relay discovery) is orders of magnitude slower + than a UDS bind. Before touching any test deadline, rule out + the CPU-throttle false-positive (see the project's + `env_cpu_throttle_masquerades_as_regression` note); then, if + real, add a per-proto timeout multiplier to the test harness + rather than editing individual tests. +- a no-network test mode: iroh with relays disabled + + loopback direct addrs only, so CI doesn't depend on n0's + infra. **Make this the default in CI**; mark the relay tests + `pytest.mark.net` and keep them out of the default run. +- leak checks: assert every `SecretKey`/`Endpoint` is closed on + actor teardown (an `Endpoint` left open holds UDP sockets and + relay connections; a leak here shows up as hung tests, not + errors). + +## 8. Risks + +| risk | mitigation | +| --- | --- | +| uniffi codegen internals shift on upgrade | pinned minor, symbol assertion at import, the "no asyncio loop" test, documented fallback to `to_asyncio` | +| rust-thread callback → trio wakeup mishandled (segfault / lost wakeup / un-cancellable task) | strong ref on the ctypes trampoline; `run_sync_soon` only; **bounded** shielded cancel-drain; run the `conc-anal` skill over the bridge | +| `iroh` wheel availability for 3.13/3.14 on linux+macos | verify in step 0; if missing, that alone may force the `aioquic` fallback | +| QUIC latency/jitter destabilizes the existing suite's timing assumptions | per-proto timeout multiplier, relay-less CI mode | +| `(str, str)` unwrapped form collides with UDS in `wrap_address()` | guarded case ordered first + explicit regression test (§3.2) | +| scope creep into iroh's docs/blobs/gossip crates | this backend is `Endpoint`+`Connection`+bi-streams only; anything else is a separate issue | + +## 9. Follow-up issue seeds + +- `tractor.discovery` delegating to iroh discovery (DNS/pkarr/mdns) +- per-`Context` QUIC sub-streams: today one `Channel` == one + stream; QUIC would let each `tractor.Context` own its own + stream with independent flow-control and cancellation — this + is the genuinely novel win #353 gestures at, and it's a + runtime-layer change, not a transport one +- unreliable QUIC datagrams for a lossy-ok broadcast transport + (pairs with plan 01's TIPC-multicast seed) +- node-id allowlist → a real `tractor` authz story +- `aioquic` sans-io backend reusing §3's adapters From 325369af2252f7a63e487017e1b05a918255cbe4 Mon Sep 17 00:00:00 2001 From: goodboy Date: Tue, 11 Aug 2026 23:12:17 -0400 Subject: [PATCH 04/18] Add `wg`-as-nested-bindspace plan doc MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Plan doc for gh #482 + the tunnelled-maddr item of #443. Pushes back on the framing that `wg` is a tpt: it's transparent to `socket(2)`, so it belongs as a *bindspace* — a scoped `@acm`-managed net ctx that an existing L4 tpt binds *inside* — and it's what finally implements the long-spec'd (never implemented) `Address.namespace`. Deats, 3 independently-shippable layers, - A) declarative: commit #482's examples, teach `parse_maddr()` the `/…/wg/u` suffix -> a `TunnelledAddress` wrapper whose `.proto_key`/`.unwrap()` delegate to `.inner` so nothing new crosses the wire and every existing table lookup keeps working. - B) swap the `subprocess.run(['sudo', 'wg', 'show'])` shelling for `pyroute2`. Default to `trio.to_thread` around the sync API (these are one-shot ops at bind/teardown, never hot-path), w/ sans-io codecs + a trio `AF_NETLINK` sock as the follow-up for the read paths. Explicitly forbids dragging `trio-asyncio` in. - C) `open_bindspace()`/`open_netns()`/`open_wg_iface()` `@acm`s folded w/ an `AsyncExitStack`, + filling in the `# !TODO, always be ns aware!` placeholder already sitting in `Endpoint.pformat()`. Also flags the subtlest bug in the whole thing: `setns(2)` is *per-thread*, so a `pyroute2` query issued via `trio.to_thread` lands in the *original* netns. Test-first, per usual. Further, designs for the generalization (`TunnelSpec` union + `match` dispatch) while only implementing `wg`+netns, and calls out `veth`-in-netns as the better *first* one bc it makes a fully self-contained two-"host" integration test possible w/o `wg` at all. (this patch was generated in some part by `claude-code` using `claude-opus-5` (`anthropic`)) --- ai/tpt-backends/03_wg_tunnel_bindspace.md | 401 ++++++++++++++++++++++ 1 file changed, 401 insertions(+) create mode 100644 ai/tpt-backends/03_wg_tunnel_bindspace.md diff --git a/ai/tpt-backends/03_wg_tunnel_bindspace.md b/ai/tpt-backends/03_wg_tunnel_bindspace.md new file mode 100644 index 00000000..5f7427e1 --- /dev/null +++ b/ai/tpt-backends/03_wg_tunnel_bindspace.md @@ -0,0 +1,401 @@ +# Plan 03 — WireGuard (and other tunnels) as a *nested bindspace* via `pyroute2` + +Tracks gh [#482] + the tunnelled-maddr item of [#443]. +Prereq reading: +[`00_shared_backend_contract.md`](./00_shared_backend_contract.md). + +**Thesis**: WireGuard is **not** a `MsgTransport`. It is an +interface-layer tunnel that is transparent to `socket(2)`, so +the correct abstraction is a *bindspace* — a scoped, +`@acm`-managed network context that an existing L4 transport +(`tcp`, and later `quic`/`tipc`-over-UDP-bearer) binds *inside*. +This plan implements `Address.namespace` (spec'd but unused +since day one) and the composed/tunnelled maddr grammar, with +`pyroute2` as the netlink codec and as much of the I/O moved +onto `trio` as the library's sans-io layer allows. + +[#482]: https://github.com/goodboy/tractor/issues/482 +[#443]: https://github.com/goodboy/tractor/issues/443 + +--- + +## 1. What exists today (verified, per #482) + +- `wrap_address()` accepts maddr `str`s (leading-`/` dispatch, + `_addr.py:262`) but `parse_maddr()` only knows + `/ip4|ip6//tcp/

` and `/unix/

`; a `.../wg/u` + maddr raises `ValueError('Unsupported multiaddr protocol + combo')`. +- there is no `wg` proto in the multiaddr spec; the first-draft + upstream PR is multiformats/py-multiaddr#108 with key form + `u` (commit `8be3a8b`), tracked by + multiformats/py-multiaddr#107 and gh #483. +- so **today's deployable story is declarative**: run `wg-quick` + out-of-band, parse the maddr, strip to the inner + `(host, port)`, verify the pubkey against the live tunnel, + hand the inner addr to `registry_addrs=`/`tpt_bind_addrs=`. + #482 already contains working example code for exactly this. +- `Address.namespace` exists in the Protocol + (`_addr.py:94-101`, "the if-available OS-specific network + namespace key") and **no backend implements it**. This plan is + its first consumer. + +## 2. Three layers, three PRs + +| layer | what | dep | ships | +| --- | --- | --- | --- | +| **A. declarative** | commit #482's examples; `parse_maddr()` learns `/wg/u` → inner `Address` + verified pubkey | `multiaddr` (already), `wg(8)` CLI | first | +| **B. `pyroute2` read/verify** | replace the `subprocess.run(['sudo','wg','show'])` shelling with netlink queries | `pyroute2` extra | second | +| **C. `@acm` lifecycle** | create/configure/tear down wg ifaces + netns *from the runtime*, as nested bindspaces; implement `Address.namespace` | `pyroute2` + `CAP_NET_ADMIN` | third | + +Each is independently valuable and independently reviewable. +**Do not attempt C first** — the interesting design (nested +bindspace `@acm`s) is only well-posed once A has pinned the +address grammar and B has proven the netlink path under trio. + +--- + +## 3. Layer A — declarative `wg` maddrs + +### 3.1 the address shape + +The decision: **a wg segment annotates an existing address, it +does not create a new address type.** Two candidate encodings; +**pick (a)**: + +- **(a) `TunnelledAddress` wrapper** (recommended): + ```python + class TunnelledAddress( + msgspec.Struct, + frozen=True, + ): + inner: Address # e.g. TCPAddress + tunnel: WGTunnelSpec # proto-specific, frozen + ``` + with `.proto_key` **delegating to `inner.proto_key`** so every + existing table lookup (`_addr_to_transport`, + `enable_transports` guard at `_root.py:391`, + `transport_from_addr()`) keeps working untouched, and + `.unwrap()` delegating to `inner.unwrap()` so **nothing new + crosses the wire**. `.namespace` and `.bindspace` come from + the tunnel spec. The wrapper is stripped (`→ .inner`) at the + moment of bind/connect. + - ⚠️ `is_wrapped_addr()` (`_addr.py:194`) tests + `type(addr) in _address_types.values()` — a `bidict` of + proto_key→type. `TunnelledAddress` isn't in it and must not + be (it's not 1:1 with a proto). So either add an explicit + `isinstance(addr, TunnelledAddress)` clause there, or give + the wrapper a marker and test structurally. Do the former; + it's two lines and honest. + - the reflection in `Endpoint.start_listener()` + (`inspect.getmodule(self.addr)`) would resolve to the + *wrapper's* module, not the transport's. **So the wrapper + must be unwrapped before it reaches `Endpoint`** — i.e. by + the bindspace `@acm` (layer C) or by `parse_maddr()` + (layer A). State this loudly in the docstring; it's the #1 + way to get this wrong. +- (b) add fields to each existing `Address` type. Rejected: + duplicates tunnel logic per-backend and pollutes `.unwrap()`. + +```python +class WGTunnelSpec( + msgspec.Struct, + frozen=True, +): + peer_pubkey: str # std-base64 `wg(8)` form + iface: str = 'wg0' + netns: str|None = None + # layer-C-only fields, unset in layer A + maybe_endpoint: tuple[str, int]|None = None + maybe_allowed_ips: tuple[str, ...] = () +``` + +### 3.2 `parse_maddr()`/`mk_maddr()` + +Grammar (matches py-multiaddr#108): the `wg` segment is a +*suffix* whose value is the multibase `u` pubkey. + +``` +/ip4/10.0.11.1/tcp/1616/wg/u +``` + +- `parse_maddr()` gains + `case [('ip4'|'ip6'), 'tcp', 'wg']:` → build the inner + `TCPAddress`, decode the multibase key to std-base64, return + `TunnelledAddress(inner=..., tunnel=WGTunnelSpec(...))`. +- keep the existing 2-proto cases byte-identical; add the new + case *after* them. +- generalize the match so the *inner* stack is parsed by the + existing logic and `wg` is peeled off first — this is what + makes `/…/udp/…/quic-v1/wg/u` work later without another + case (plan 02 §3.4). Write it as a small pure function: + `_peel_tunnel_segs(proto_names) -> (inner_names, + tunnel_specs)`. +- `mk_maddr()` inverse for `TunnelledAddress`. +- **blocked on upstream**: `Multiaddr('/…/wg/u…')` only parses + once py-multiaddr#108 lands. Until then: pin the branch in the + `wg` extra / dev-group and gate the tests on + `_have_wg_maddr_proto()` (a cheap try/except around + `Multiaddr('/wg/uAAAA')`). Do **not** hand-roll a `wg` parser + in `tractor` — the whole point of #429 was dropping the NIH + parser. + +### 3.3 verification helper (pure, composable) + +Port #482 §2's helpers into `tractor/discovery/_tunnel.py` as +*pure functions* + one impure probe, cleanly separated: + +```python +def parse_wg_maddr(maddr: str) -> TunnelledAddress: ... # pure +def wg8_pubkey(multibase_key: str) -> str: ... # pure +def verify_wg_peer(spec: WGTunnelSpec) -> bool: ... # impure probe +``` + +In layer A `verify_wg_peer()` may shell out (`wg show +peers`), but it must be a *single* function so layer B swaps +only its body. Never call it implicitly from +`wrap_address()`/`parse_maddr()` — parsing must stay pure and +side-effect-free; verification is the *caller's* explicit step +(and later, the bindspace `@acm`'s). + +### 3.4 deliverables + +- `examples/` scripts distilled from #482 §§3-5 (this is the + unchecked "commit examples from ^" bullet in #443). +- a `docs/` page: tunnel setup, the maddr form, the two-host + run. Keep prose in the docs; keep the examples runnable and + minimal. +- tests: maddr round-trip, `TunnelledAddress` delegation + (`proto_key`/`unwrap` identical to inner), `wrap_address()` + regression (a tunnelled maddr `str` → `TunnelledAddress`; a + plain one → unchanged), and **a real end-to-end over a + locally-created wg pair** gated on `CAP_NET_ADMIN` (see §5.3). + +--- + +## 4. Layer B — `pyroute2` under `trio` + +### 4.1 the library situation (verify at implementation time) + +`pyroute2` ≥0.9 rewrote its core onto **asyncio** +(`AsyncIPRoute`; the sync `IPRoute` wraps it with its own loop). +It also ships a `WireGuard` netlink (generic-netlink) module +supporting `.set(iface, private_key=..., peer={...})` and +`.info(iface)`, plus `pyroute2.netns` / `NetNS` for namespaces, +and `IPRoute.link('add', kind='wireguard', ifname=...)`. + +Three integration options, in increasing trio-nativeness: + +- **(1) `trio.to_thread.run_sync()` around the sync API.** + Netlink ops here are one-shot, sub-millisecond, and happen at + bind/teardown time only — *not* in the msg hot path. This is + the **correct default**: it's ~10 lines, uses a battle-tested + API, and costs nothing where it's used. +- **(2) sans-io: `trio.socket` + pyroute2's message codecs.** + `pyroute2`'s message classes + (`pyroute2.netlink.rtnl.*`, `pyroute2.netlink.generic.wireguard.wgmsg`) + encode/decode independently of its I/O core. So a + `tractor/ipc/_netlink.py` with a small trio `NetlinkSocket` + (`trio.socket.socket(AF_NETLINK, SOCK_RAW|SOCK_DGRAM, proto)`, + `sendto`/`recv`, seq/pid matching, `NLMSG_DONE`/`NLMSG_ERROR` + handling) + pyroute2 codecs is very achievable and is the + honest reading of "as much trio wrapping as possible where any + other async support can be replaced". + **Do this for the paths we actually need** (link add/del, + addr add, wg get/set, netns bind) and *only* those — a + general netlink client is out of scope. +- (3) reimplement the codecs. Never. + +**Recommended split**: ship (1) first so layer B is a small, +reviewable, behaviour-preserving swap of `verify_wg_peer()`'s +body; then land (2) as a follow-up commit for the read path +(`wg get`, `link get`) where the sans-io surface is smallest, +and keep (1) for the privileged mutating ops. Measure before +converting anything else — there is no perf argument here, only +a "no foreign event loop in a trio actor" argument, which (1) +already satisfies (a thread is not an event loop). + +Explicitly **do not** pull in `trio-asyncio` for pyroute2: it +would be the one place in the runtime where an asyncio loop +exists for no reason. + +### 4.2 API shape + +Pure-ish, functional, `@acm` for anything with teardown: + +```python +async def read_wg_peers( + iface: str = 'wg0', + netns: str|None = None, +) -> tuple[str, ...]: ... # base64 pubkeys + +async def read_wg_pubkey(iface: str = 'wg0', ...) -> str: ... +``` + +and `verify_wg_peer()` becomes a thin composition over the two. +Note the pure-getter rule: no `read_wg_peers(..., create=True)`. + +--- + +## 5. Layer C — nested bindspace `@acm`s + `Address.namespace` + +This is the part #443 and `multiaddr_declare_eps.md` actually +ask for: *"for any tunneled maddr-`str`-entry we deliver a +data-structure which can easily be passed to nested `@acm`s +which consecutively setup nested net bindspaces for binding the +endpoint addrs"*. + +### 5.1 the composition + +```python +@acm +async def open_bindspace( + addr: TunnelledAddress, +) -> AsyncGenerator[Address, None]: + ''' + Enter the net-bindspace implied by `addr`'s tunnel stack, + yielding the *inner* `Address` ready to bind/connect. + + Nests: one `@acm` per tunnel segment, outermost-first, so + a 2-deep stack is just two nested `async with`s and the + teardown order is guaranteed by `trio`. + + ''' +``` + +with per-tunnel-kind implementations: + +```python +@acm +async def open_netns(name: str) -> AsyncGenerator[None, None]: ... +@acm +async def open_wg_iface(spec: WGTunnelSpec) -> AsyncGenerator[WGTunnelSpec, None]: ... +``` + +and a driver that folds a list of specs into nested contexts +(`contextlib.AsyncExitStack` for the N-deep case). The +`parse_endpoints()` API (`_multiaddr.py:153`) is the front door: +it already returns `dict[name, list[Address]]` and the +`multiaddr_declare_eps.md` sketch anticipates the recursive +`dict[str, list[Address]]|dict[...]` return for tunnelled +entries. Extend it to carry the tunnel stack, not to *enter* it. + +### 5.2 `Address.namespace`, at last + +- `TunnelledAddress.namespace` → `(kind, id)` e.g. + `('netns', 'tractor-wg0')`. +- **and** the existing backends should implement it as `None` + explicitly (they currently just don't define it), so the + Protocol stops lying. +- consumers to audit: nothing reads `.namespace` today — so + adding it is safe, but the *point* is that + `Endpoint`/`Server.pformat()` should start showing it (there's + already a `# !TODO, always be ns aware!` + + `f'|_netns: {netns}\n'` placeholder sitting in + `Endpoint.pformat()`, `_server.py:645`). Fill that in; it's + the cheapest possible proof the layer is wired. + +### 5.3 the netns/process reality — read this before designing + +- `setns(2)` with `CLONE_NEWNET` affects **the calling thread + only**, and sockets already created keep their original netns. + A trio actor is effectively single-threaded for our purposes, + so "enter the netns, *then* bind" works — but any + `to_thread` worker (§4.1 option 1!) is in the **original** + netns unless it also `setns`. Concretely: a wg query issued + via `trio.to_thread` will hit the wrong namespace. Either + pass `netns=` down to `pyroute2` (which does the + fork/setns dance itself) or pin a dedicated worker. **This is + the single subtlest bug in this plan — write the test first.** +- entering a netns is *process-global-ish and irreversible-ish* + in practice. Therefore: **netns membership belongs to the + actor process, decided before the runtime binds**, not to a + mid-life `@acm`. Design: + - the root/parent decides the netns for a subactor and passes + it in the spawn spec (there's already + `enable_transports`/`accept_addrs` plumbing at + `_runtime.py:1595-1615` — the netns rides alongside). + - the child, in `_runtime.async_main()` **before** + `IPCServer.listen_on()`, enters it. + - the mid-life `@acm` form is then only for the *root* / + single-actor case, and for iface creation (which is + genuinely scoped). + - document the constraint rather than hiding it; a + `RuntimeError` if `open_netns()` is entered after any + listener exists. +- privileges: iface/netns creation needs `CAP_NET_ADMIN`. + Never `sudo` from inside the runtime. Two supported modes: + (i) pre-provisioned out-of-band (layers A/B — the default, + and what #482 documents), (ii) runtime-managed when the + process already holds the cap. Detect with a cheap + `os.geteuid()==0 or CAP_NET_ADMIN in /proc/self/status` + probe and *fail loudly with an actionable message* otherwise. +- teardown must be idempotent and tolerant: an iface/netns + already gone must not strand the rest of the teardown — the + exact lesson `_uds.close_listener()`'s `FileNotFoundError` + tolerance and `_serve_ipc_eps()`'s per-ep `try/except` + encode. Mirror both. + +### 5.4 tests for layer C + +- unit: fold-N-tunnel-specs-into-nested-`@acm`s, with fakes; assert + enter/exit ordering (outermost-last-out) via a trace list. +- integration, gated on `CAP_NET_ADMIN` (skip otherwise, and in + CI run it in a `--cap-add NET_ADMIN` container job): create two + netns + a wg pair entirely in-process, boot a `tractor` root in + one and a subactor in the other, `find_actor()` across the + tunnel. This is a *fantastic* test to have and is fully + self-contained — no second host, no `sudo` in the test body. +- the `to_thread`-netns-mismatch regression from §5.3, written + **first** (red), then the fix (green), per project convention. + +--- + +## 6. "Other shuttle-able tpts" + +The generalization the #482 follow-up gestures at: once +`TunnelledAddress` + `open_bindspace()` exist, the same +machinery covers any iface-layer tunnel `pyroute2` can drive — +`ipip`/`gre`/`sit`/`vxlan`/`geneve`/`bridge`/`veth`. Keep +`WGTunnelSpec` as *one* frozen struct among a +`TunnelSpec = WGTunnelSpec|VxlanTunnelSpec|...` union with a +`kind: ClassVar[str]`, and dispatch `open_*` by `match` on it. +Design for it now (union + `match`), implement only `wg` + +`netns`. `veth`-pairs-in-netns is the natural second one because +it makes the §5.4 integration test possible without wg at all — +consider doing it *first* for exactly that reason. + +## 7. Non-goals + +- no wg userspace implementation, no key exchange, no + `wg-quick` reimplementation (config-file parsing is + out of scope; take structured input). +- no persistence of private keys beyond what layer C's iface + creation needs (and that stays in `get_rt_dir()`, 0600). +- macOS/Windows: layers B/C are Linux-only. Layer A (declarative) + works anywhere `wg` does. Gate accordingly and say so in the + docs — do not silently no-op. + +## 8. Risks + +| risk | mitigation | +| --- | --- | +| `to_thread` worker runs in the wrong netns | §5.3; pass `netns=` to pyroute2 or pin a worker; test-first | +| py-multiaddr#108 not merged | branch pin + `_have_wg_maddr_proto()` gate; layer A's inner-addr path works regardless | +| `TunnelledAddress` leaks into `Endpoint` and breaks `inspect.getmodule()` | unwrap at parse/bindspace boundary; assert `not isinstance(ep.addr, TunnelledAddress)` in `Endpoint.__post_init__` | +| privileged ops in a library | never `sudo`; explicit cap probe + actionable error; pre-provisioned is the default | +| pyroute2 0.9 asyncio core drags a loop into the actor | option (1) is a *thread*, not a loop; forbid `trio-asyncio` here (§4.1) | +| netns teardown strands actor teardown | idempotent/tolerant teardown mirroring `_uds.close_listener()` | + +## 9. Follow-up issue seeds + +- `veth`-in-netns bindspace (unblocks capless-ish integration + testing, and is a great local multi-"host" test rig) +- composed/tunnelled maddr grammar shared with plan 02's + `/…/quic-v1/…` stacks (gh #443) +- `wg` proto into the multiaddr **spec** (gh #483), then flip + `MsgTransport.maddr` to always return `Multiaddr` (the third + #443 bullet) +- runtime-managed wg key rotation / peer add-remove as a + `tractor` service actor — the natural "actor that owns the + network" demo From 9cdbe36bca3f62d37ee0ab5b13cf39bdc8f9d8c2 Mon Sep 17 00:00:00 2001 From: goodboy Date: Tue, 11 Aug 2026 23:12:30 -0400 Subject: [PATCH 05/18] Index the tpt-backend plans w/ a README Landing page for `ai/tpt-backends/`: points at the contract spec as required first reading, tables the 3 plans against their issues/deps/size, and states the landing order + why. Deats, - TIPC first as the cheap proof the table-registration story generalizes to a genuinely new proto (stdlib-only, and `trio`'s sock wrappers are family-agnostic). - `wg` layer-A next since it's deployable-today doc/example work. - QUIC last, gated on its own prep PR. - notes that plans 01 and 02 both want the same `Address.rebind_from_sockname` gate, so whichever lands first ships it. (this patch was generated in some part by `claude-code` using `claude-opus-5` (`anthropic`)) --- ai/tpt-backends/README.md | 54 +++++++++++++++++++++++++++++++++++++++ 1 file changed, 54 insertions(+) create mode 100644 ai/tpt-backends/README.md diff --git a/ai/tpt-backends/README.md b/ai/tpt-backends/README.md new file mode 100644 index 00000000..2f64b12f --- /dev/null +++ b/ai/tpt-backends/README.md @@ -0,0 +1,54 @@ +# next-gen `tractor.ipc` transport backend plans + +Implementation specs for three prospective `.ipc` transport +backends, written so each can be worked independently (by a +different model/provider) without design or lib-selection drift. + +**Read [`00_shared_backend_contract.md`](./00_shared_backend_contract.md) +first** — it is the normative description of what a `tractor` +transport backend *is* as of `main@83b34884` (the backend +duck-type, the 10-item registration checklist, the test-harness +plumbing, the code-style rules). The three plans assume it and +document only their own deltas. + +| plan | issue | dep | size | lands | +| --- | --- | --- | --- | --- | +| [01 — TIPC](./01_tipc_backend.md) | [#378] | **none** (stdlib) | small | first | +| [02 — QUIC/`iroh`](./02_quic_iroh_backend.md) | [#353] | `iroh` (uniffi FFI) | large | needs a prep PR | +| [03 — `wg` bindspace](./03_wg_tunnel_bindspace.md) | [#482], [#443] | `pyroute2` | medium, 3 layers | layer A now | + +Headline conclusions: + +- **TIPC is the cheap win.** Verified: `trio.SocketStream` and + `trio.SocketListener` are address-family agnostic (only + `SOCK_STREAM` + a trio socket), and CPython ships `AF_TIPC` + + 23 `TIPC_*` constants. So the backend is ~one module of + contract boilerplate, zero new deps, and it buys + *kernel-native* service discovery: `bind()` publishes, + `connect()`-by-name resolves — no registrar in the loop. + (`modprobe tipc` is required; hard-gate everything.) +- **QUIC's cost is entirely in two adapters**, not in QUIC. The + `iroh` python bindings are `uniffi`-generated asyncio, but the + asyncio dependency is confined to *one* future-poll callback — + a ~40-line `trio` bridge (`TrioToken.run_sync_soon`) replaces + it. The second cost is that an iroh listener isn't a socket, + which needs a small, independently-reviewable prep PR to + `_server.py`/`_types.py`. +- **WireGuard is not a transport.** It's an iface-layer tunnel, + so it belongs as a *nested bindspace* (`TunnelledAddress` + + `open_bindspace()` `@acm`s) wrapping whatever L4 tpt is in + use — which is also what finally implements the long-spec'd + `Address.namespace`, and what generalizes to + `veth`/`vxlan`/`gre`. + +Ordering rationale: plan 01 first as the cheap proof the +table-registration story generalizes to a genuinely new proto; +plan 03 layer A is already deployable-today doc/example work; +plan 02 last (and gated on its prep PR). Plans 01 and 02 both +want the same `Address.rebind_from_sockname` gate — whichever +lands first ships it. + +[#378]: https://github.com/goodboy/tractor/issues/378 +[#353]: https://github.com/goodboy/tractor/issues/353 +[#482]: https://github.com/goodboy/tractor/issues/482 +[#443]: https://github.com/goodboy/tractor/issues/443 From a03b52619a51d32909bc319ec3de7151626e1811 Mon Sep 17 00:00:00 2001 From: goodboy Date: Wed, 12 Aug 2026 19:37:08 -0400 Subject: [PATCH 06/18] Proto-key the unwrapped-addr form in the plans MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Shape-matching in `wrap_address()` doesn't survive 4 backends and the plans were papering over it: TIPC's natural unwrapped form is a `(str, int)`, indistinguishable from `TCPAddress`, and iroh's is a `(str, str)`, which the *existing* UDS case (`case (_, filename) if type(filename) is str`) already swallows. So the contract doc (§1.1) now carries the conclusion as a **recommended prerequisite for all three backends**: make the unwrapped form carry an explicit proto-key spelled with the `multiaddr` protocol name — `('tcp', host, port)`, `('unix', path)`, `('tipc', stype, inst, scope)`. `wrap_address()` then collapses from an order-sensitive `match` to `_address_types[addr[0]]` and the whole collision class stops existing, while the on-wire form finally agrees w/ `mk_maddr()`/`parse_maddr()` instead of being an independent invention. Two consequences spelled out: it's a wire-format change (`SpawnSpec`, `_root_mailbox`, `_registry_addrs`) + every fixture + downstream config, so it wants its own migration commit landed *before* any new backend; and it's the moment to stop handing raw tuples to users at all — `Address` becomes the public currency and `UnwrappedAddress` an internal serialization detail, the same discipline `ipaddress` uses (you pass `IPv4Address`, never a 4-tuple). Plan 01 §2.2 is rewritten to match and to explicitly **retract** its own earlier `('tipc::', instance)` self-tagging prefix hack — it keeps `wrap_address()` order-sensitive and does nothing for the iroh/UDS collision, so the doc says don't resurrect it. Registration checklist item 4 likewise becomes "do the migration first, then this is a one-line `_address_types` entry". Also seeds a `/tipc` multiaddr-spec submission as a follow-up, mirroring the `wg` track (multiformats/py-multiaddr#107/#108 + gh (this patch was generated in some part by `claude-code` using `claude-opus-5` (`anthropic`)) --- ai/tpt-backends/00_shared_backend_contract.md | 39 +++++++++++++- ai/tpt-backends/01_tipc_backend.md | 52 +++++++++++-------- 2 files changed, 66 insertions(+), 25 deletions(-) diff --git a/ai/tpt-backends/00_shared_backend_contract.md b/ai/tpt-backends/00_shared_backend_contract.md index 690ffa8b..f46873a2 100644 --- a/ai/tpt-backends/00_shared_backend_contract.md +++ b/ai/tpt-backends/00_shared_backend_contract.md @@ -70,6 +70,40 @@ Hard constraints learned from the existing two: `wrap_address()` (`_addr.py:230`), you have a bug that manifests as the wrong transport being loaded — the file's own `XXX NOTE` warns about precisely this. + + ⚠️ **and shape-matching does not survive 4 backends.** Adding + TIPC and iroh breaks it outright: TIPC's natural form is a + `(str, int)` — indistinguishable from `TCPAddress` — and + iroh's is a `(str, str)`, which the *existing* UDS case + (`case (_, filename) if type(filename) is str`) already + swallows. Ordering hacks and prefix-tagging (an earlier + revision of plan 01 proposed `('tipc::', inst)`) + paper over it at best. + + **The fix, and the recommended prerequisite for all three + backends: make the unwrapped form carry an explicit + proto-key, using the `multiaddr` protocol name as the + canonical spelling** — `('tcp', host, port)`, + `('unix', path)`, `('udp', ...)`, `('tipc', stype, inst, + scope)`. Then `wrap_address()` collapses from an + order-sensitive `match` to `_address_types[addr[0]]`, and the + whole collision class stops existing. Note this *also* aligns + the on-wire form with `mk_maddr()`/`parse_maddr()`, so the two + representations stop being independent inventions. + + Two consequences to plan for: + - it's a **wire-format change** (`SpawnSpec`, + `_root_mailbox`, `_registry_addrs`) plus every test fixture + and downstream config (`piker`'s `[network]` table). It + wants its **own migration commit, landed before any new + backend**, not smuggled into one. + - it's the moment to **stop handing raw unwrapped tuples to + users at all.** The long-term shape is: `Address` subtypes + are the public currency and `UnwrappedAddress` becomes an + internal serialization detail — the same discipline + `ipaddress` uses (you pass `IPv4Address`, not a 4-tuple). + Public API should accept `Address|maddr-str` and treat bare + tuples as legacy-tolerated input, ideally deprecated. - **`.get_random()` must be collision-free without a live runtime.** See the `UDSAddress.get_random()` uuid-token comment (`_uds.py:207-220`): with no `current_actor()` the @@ -214,8 +248,9 @@ Adding a backend touches these and only these: entry lazy — propose that refactor explicitly. 4. `tractor/discovery/_addr.py:230` `wrap_address()` `match` — add a case iff your `unwrapped_type` isn't already uniquely - matched. Prefer unwrapped forms that are *self-tagging* - (see plan 01 §2.2 and plan 02 §2.2) so this stays cheap. + matched. **Preferably do the proto-key migration in §1.1 + first**, after which this step becomes a one-line + `_address_types` entry instead of an order-sensitive `case`. 5. `tractor/ipc/_types.py` — `Address` union alias, `_msg_transports` list, `_key_to_transport[('msgpack', key)]`, `_addr_to_transport[Address]`. diff --git a/ai/tpt-backends/01_tipc_backend.md b/ai/tpt-backends/01_tipc_backend.md index 8a0fa413..ccbdf3be 100644 --- a/ai/tpt-backends/01_tipc_backend.md +++ b/ai/tpt-backends/01_tipc_backend.md @@ -98,34 +98,34 @@ class TIPCAddress( def_bindspace: ClassVar[int] = TIPC_CLUSTER_SCOPE ``` -**Unwrapped form** (the wire/`SpawnSpec` shape) — must be -uniquely matchable in `wrap_address()`: +**Unwrapped form** (the wire/`SpawnSpec` shape). + +TIPC's natural form is `(stype, instance, scope)` — but a +2-tuple squeeze of it is a `(str, int)`, i.e. *the same coarse +shape as `TCPAddress`*, so `wrap_address()`'s +`case (str(), int())` steals it. This backend is therefore the +forcing function for the contract-doc's conclusion (§1.1): + +> **make the unwrapped form carry an explicit proto-key, spelled +> with the `multiaddr` protocol name.** ```python -def unwrap(self) -> tuple[str, int]: - # ('tipc::', instance) - return (f'tipc:{self._stype}:{self._scope}', self._instance) +def unwrap(self) -> tuple[str, int, int, int]: + return ('tipc', self._stype, self._instance, self._scope) ``` -i.e. `(str, int)` — *the same coarse shape as `TCPAddress`*, so -`wrap_address()`'s `case (str(), int())` would steal it. Two -options; **pick (a)**: +`wrap_address()` then dispatches `_address_types[addr[0]]` and +the collision class disappears. **This is a prerequisite +migration commit, not part of this backend** — see contract §1.1 +for its blast radius (wire format + every fixture + `piker` +config) and for the follow-on "stop handing raw tuples to users +at all, à la `ipaddress`" direction. -- **(a) self-tagging prefix + a `case` ordered before the TCP - one**, guarded on `addr[0].startswith('tipc:')`: - ```python - case (str() as h, int()) if h.startswith('tipc:'): - cls = TIPCAddress - ``` - Cheap, string-y, but honest about the fact that - `UnwrappedAddress` is a deliberately-degenerate - `tuple[str, int|str]` (see the `_addr.py:43-65` TODO block). - It also keeps `TCPAddress.__post_init__`'s - `ipaddress.ip_address()` validation from being reached with a - non-IP host. -- (b) widen `UnwrappedAddress` to a 3-tuple for tipc. Rejected: - ripples into every `reg_addr` fixture, `_root_mailbox`, and - downstream (`piker`) config. +⚠️ an earlier revision of this plan proposed a self-tagging +`('tipc::', instance)` string-prefix hack with an +ordered `case` guard. **Dropped** — it papers over the problem, +keeps `wrap_address()` order-sensitive, and doesn't help iroh's +`(str, str)`-vs-UDS collision at all. Do not resurrect it. Note `TIPCAddress` is the first backend where `.unwrap()` is **not** a lossless view of the live socket — `maybe_node`/ @@ -689,6 +689,12 @@ single best demo this backend has; lead with it. ## 10. Follow-up issue seeds +- **register `/tipc` in the multiaddr spec**, mirroring the `wg` + track (multiformats/py-multiaddr#107/#108 + gh #483). Same + shape of work: propose the proto + code, land a codec in + `py-multiaddr`, then drop our `str`-maddr fallback (§4). Worth + filing *alongside* the `wg` spec-submission issue so both + proposals go up together rather than as one-offs. - registrar-less discovery fast path via name derivation (§5.1) - `TIPC_TOP_SRV`-driven push registry in `discovery/_registry.py` (§5.2) From 86417fa19ca23c7affc071437cfe672357539315 Mon Sep 17 00:00:00 2001 From: goodboy Date: Wed, 12 Aug 2026 19:37:19 -0400 Subject: [PATCH 07/18] Fix the `wg` maddr grammar, `/wg/` is *infix* MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The prior revision (and gh #482's examples) had it as a suffix, `/ip4/10.0.11.1/tcp/1616/wg/u`. Wrong: verified against `baudco/py-multiaddr@wg_support` (py-multiaddr#108) installed in a throwaway venv, the canonical form is /ip4/192.168.1.50/udp/51820/wg/u/ip4/10.0.11.1/tcp/1616 where segs *before* `/wg/` are the **bearer** — the underlay `(ip, udp-port)` `wg(8)` itself listens on (`ListenPort`), per the codec docstring's own example — and segs *after* are the **overlay** ep, the only part we ever bind. The suffix form does parse, which is why it slipped through, but it's semantically inverted: overlay addr where the bearer belongs, `tcp` where wg's `udp` goes, and no overlay ep declared at all. Records the observed `[p.name for p in m.protocols()]` lists so the `match` can be written against fact, and replaces the "composed vs not" framing w/ what's actually the design axis: three parts, three **owners** — bearer bound by the kernel via `wg-quick`/`pyroute2`, `/wg/u` bound by nothing (it's an identity, verified out-of-band), overlay bound by our `IPCServer` as `.inner`. `_peel_tunnel_segs()` correspondingly grows a 3rd return, splitting *at* the tunnel seg so nested tunnels fall out for free. Also hoists the netns conclusion to the top of §5.3 where it can't be missed: netns is a **runtime-level config API, not an actor-app-code one**. It's a spawn/boot-time input alongside `enable_transports`/`tpt_bind_addrs`, deliberately w/ no `await actor.enter_netns(...)`, because `setns(2)` neither moves already-created sockets nor applies beyond the calling thread — so a mid-life API would silently leave the IPC server bound in the old ns. (this patch was generated in some part by `claude-code` using `claude-opus-5` (`anthropic`)) --- ai/tpt-backends/03_wg_tunnel_bindspace.md | 74 +++++++++++++++++++---- 1 file changed, 61 insertions(+), 13 deletions(-) diff --git a/ai/tpt-backends/03_wg_tunnel_bindspace.md b/ai/tpt-backends/03_wg_tunnel_bindspace.md index 5f7427e1..6f44c7a9 100644 --- a/ai/tpt-backends/03_wg_tunnel_bindspace.md +++ b/ai/tpt-backends/03_wg_tunnel_bindspace.md @@ -112,25 +112,61 @@ class WGTunnelSpec( ### 3.2 `parse_maddr()`/`mk_maddr()` -Grammar (matches py-multiaddr#108): the `wg` segment is a -*suffix* whose value is the multibase `u` pubkey. +Grammar — **verified** against py-multiaddr#108 +(`baudco/py-multiaddr@wg_support`, installed in a throwaway venv; +all three forms below parse *and* round-trip): ``` -/ip4/10.0.11.1/tcp/1616/wg/u +/ip4/192.168.1.50/udp/51820/wg/u/ip4/10.0.11.1/tcp/1616 +\_______ bearer __________/\__ key __/\______ overlay ______/ + underlay, wg `ListenPort` the ONLY part we bind ``` -- `parse_maddr()` gains - `case [('ip4'|'ip6'), 'tcp', 'wg']:` → build the inner - `TCPAddress`, decode the multibase key to std-base64, return - `TunnelledAddress(inner=..., tunnel=WGTunnelSpec(...))`. +The `/wg/` segment is **infix, not suffix** — the segments +*before* it are the wg **bearer** (the underlay `(ip, udp-port)` +that `wg(8)` itself listens on, per the codec docstring's own +`/ip4/1.2.3.4/udp/51820/wg/{key}` example), and the segments +*after* are the **overlay** endpoint that `tractor` binds. + +⚠️ **CORRECTION** — an earlier revision of this plan (and the +examples in gh #482) used a *suffix* form +`/ip4/10.0.11.1/tcp/1616/wg/u`. That parses, but it is +semantically inverted: it puts the overlay addr where the bearer +belongs, `tcp` where wg's `udp` `ListenPort` goes, and declares +no overlay endpoint at all. `parse_wg_maddr()` in +`examples/wg_lan/` now rejects it with an actionable error. +Observed protocol-name lists, for writing the `match`: + +| maddr | `[p.name for p in m.protocols()]` | +| --- | --- | +| `/ip4/1.2.3.4/udp/51820/wg/u` | `['ip4','udp','wg']` | +| `/ip4/../udp/../wg/u/ip4/../tcp/..` | `['ip4','udp','wg','ip4','tcp']` | + +- so the three parts have **three different owners**, and only the + third is an `Endpoint`: + + | part | bound by | in the runtime? | + | --- | --- | --- | + | bearer | kernel, via `wg-quick`/`pyroute2` | no | + | `/wg/u` | nothing — it's an identity | no, verified out-of-band | + | overlay | `tractor`'s `IPCServer` | **yes**, as `.inner` | + + This owner-split is the real axis of the design, *not* whether + the maddr stack is "composed" (it is). +- `parse_maddr()` gains a case on + `[('ip4'|'ip6'), 'udp', 'wg', ('ip4'|'ip6'), ]` → + build the inner `Address` from the trailing segments, decode + the multibase key to std-base64, and return + `TunnelledAddress(inner=..., tunnel=WGTunnelSpec(...))` with + the bearer recorded in the spec. - keep the existing 2-proto cases byte-identical; add the new case *after* them. -- generalize the match so the *inner* stack is parsed by the - existing logic and `wg` is peeled off first — this is what - makes `/…/udp/…/quic-v1/wg/u` work later without another - case (plan 02 §3.4). Write it as a small pure function: - `_peel_tunnel_segs(proto_names) -> (inner_names, - tunnel_specs)`. +- generalize by **peeling at the tunnel segment**: split + `proto_names` at `'wg'`, hand the trailing list to the existing + inner-stack logic, and recurse for nested tunnels. Write it as + a small pure fn `_peel_tunnel_segs(proto_names) -> + (bearer_names, tunnel_specs, inner_names)`. This is also what + makes a wg-inside-wg stack fall out for free. - `mk_maddr()` inverse for `TunnelledAddress`. - **blocked on upstream**: `Multiaddr('/…/wg/u…')` only parses once py-multiaddr#108 lands. Until then: pin the branch in the @@ -297,6 +333,18 @@ entries. Extend it to carry the tunnel stack, not to *enter* it. ### 5.3 the netns/process reality — read this before designing +**The headline consequence, stated up front**: netns is a +**runtime-level config API, not an actor-app-code API.** It is +declared as part of how an actor process is *brought up* — a +spawn-time/boot-time input alongside `enable_transports` and +`tpt_bind_addrs` — and it is **not** dynamically re-enterable by +app code once the actor is live. There is deliberately no +`await actor.enter_netns(...)`. Two hard reasons, both below: +`setns(2)` doesn't retroactively move existing sockets, and it's +per-thread rather than per-process. Anything that *looks* like a +mid-life API here would be a footgun that silently leaves the IPC +server bound in the old namespace. + - `setns(2)` with `CLONE_NEWNET` affects **the calling thread only**, and sockets already created keep their original netns. A trio actor is effectively single-threaded for our purposes, From 0457e8f3b13b479be7cf49bffab64467f944611d Mon Sep 17 00:00:00 2001 From: goodboy Date: Wed, 12 Aug 2026 19:37:29 -0400 Subject: [PATCH 08/18] Add a `wg`-tunnelled 2-host example set MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Re-renders gh #482's examples w/ the corrected (infix) maddr grammar, as the "layer A" slice of the wg plan: declarative maddrs only, tunnel pre-provisioned out-of-band, zero runtime changes. - `wg_maddr.py`: a `frozen=True` `msgspec.Struct` addr carrying `bearer`/`peer_pubkey`/`inner` (+ `inner_proto`), a `.maddr` property that re-renders the canonical form, and pure `mb_pubkey()`/`wg8_pubkey()`/`parse_wg_maddr()`. The parser rejects #482's inverted suffix form w/ an actionable error and stays **side-effect free** — `verify_wg_peer()` is a separate, explicitly impure step the caller composes, never something a parse path shells out to. - `host_a_srv.py`/`host_b_client.py`: the two-host runs, passing only `addr.inner` into `open_nursery()`/`open_root_actor()`, which is the whole point — the bearer + key layers are already established before any bind happens. - `README.md`: the grammar + the 3-owners table, the `#108` branch install line, tunnel setup, and a "what changed vs #482" section enumerating the corrections. Runnable-shaped but **not yet run against a live tunnel**; that's next, and the reason these sit on the planning branch rather than in `examples/` proper. `_segments()` marks its stopgap for when the `wg` codec isn't installed. (this patch was generated in some part by `claude-code` using `claude-opus-5` (`anthropic`)) --- examples/wg_lan/README.md | 153 ++++++++++++++++++++++ examples/wg_lan/host_a_srv.py | 61 +++++++++ examples/wg_lan/host_b_client.py | 52 ++++++++ examples/wg_lan/wg_maddr.py | 215 +++++++++++++++++++++++++++++++ 4 files changed, 481 insertions(+) create mode 100644 examples/wg_lan/README.md create mode 100644 examples/wg_lan/host_a_srv.py create mode 100644 examples/wg_lan/host_b_client.py create mode 100644 examples/wg_lan/wg_maddr.py diff --git a/examples/wg_lan/README.md b/examples/wg_lan/README.md new file mode 100644 index 00000000..21396d5b --- /dev/null +++ b/examples/wg_lan/README.md @@ -0,0 +1,153 @@ +# `tractor` over a WireGuard tunnel, declared as one maddr + +A two-host LAN setup: a `tractor` actor tree on host A, dialed +from host B, with the endpoint declared as a single `wg` +multiaddr. + +Supersedes the example set in gh +[#482](https://github.com/goodboy/tractor/issues/482) — see +[what changed](#what-changed-vs-482). + +## the maddr form + +``` +/ip4/192.168.1.50/udp/51820/wg/u/ip4/10.0.11.1/tcp/1616 +\____ wg bearer ___________/\__ key __/\____ tractor ep _____/ + underlay, wg `ListenPort` overlay, on the wg iface + (kernel/`wg(8)` owns it) (the ONLY part tractor binds) +``` + +Three parts, three different owners: + +| part | who binds it | in the runtime? | +| --- | --- | --- | +| `/ip4/../udp/51820` bearer | kernel via `wg-quick`/`pyroute2` | no | +| `/wg/u` | nothing — it's an identity | no, verified out-of-band | +| `/ip4/../tcp/1616` overlay | `tractor`'s `IPCServer` | **yes**, as `.inner` | + +Verified against py-multiaddr +[#108](https://github.com/multiformats/py-multiaddr/pull/108): +this composed form parses and round-trips +(`['ip4','udp','wg','ip4','tcp']`). + +## requirements + +The `wg` proto isn't in released `py-multiaddr` yet (`0.2.0` has +no `wg` codec), so until #108 lands: + +```bash +uv pip install 'git+https://github.com/baudco/py-multiaddr.git@wg_support' multibase +``` + +`wg_maddr.py` degrades to a plain segment split when the codec is +absent, so the examples still run — but you lose per-segment +validation. It deliberately does **not** hand-roll a `wg` codec +(gh #429 was about *dropping* our NIH parser). + +## 0. tunnel setup (out-of-band, both hosts) + +Host A is the service host (underlay e.g. `192.168.1.50`), host B +your workstation. Overlay net `10.0.11.0/24`. + +```bash +umask 077 +wg genkey | tee wg_priv.key | wg pubkey > wg_pub.key +``` + +`/etc/wireguard/wg0.conf` on **host A**: + +```ini +[Interface] +PrivateKey = +Address = 10.0.11.1/24 +ListenPort = 51820 +``` +```ini +[Peer] +PublicKey = +AllowedIPs = 10.0.11.2/32 +``` + +on **host B**: + +```ini +[Interface] +PrivateKey = +Address = 10.0.11.2/24 +``` +```ini +[Peer] +PublicKey = +Endpoint = 192.168.1.50:51820 +AllowedIPs = 10.0.11.1/32 +PersistentKeepalive = 25 +``` + +Note how `ListenPort` and `Endpoint` are exactly the maddr's +bearer segment, and `[Interface] Address` is its overlay host. + +```bash +sudo wg-quick up wg0 # both hosts +ping -c1 10.0.11.1 # from B +``` + +## 1. get your pubkey into the maddr + +```bash +python -c " +import base64, multibase +key = open('wg_pub.key').read().strip() +print(multibase.encode('base64url', base64.b64decode(key)).decode()) +" +``` + +Paste the `u...` output into `WG_MADDR` in both scripts (they use +the same string — A's bearer, A's key, A's overlay ep). + +## 2. run + +```bash +# host A +python host_a_srv.py + +# host B +python host_b_client.py +``` + +`host_a_srv.py` must be importable on host B too, since +`portal.run()` refs the fn by module path — standard `tractor` +RPC semantics. + +## what changed vs #482 + +Four corrections, all from +`ai/tpt-backends/03_wg_tunnel_bindspace.md`: + +1. **the maddr semantics were inverted.** #482 used + `/ip4/10.0.11.1/tcp/1616/wg/u` — that parses, but it puts + the *overlay* addr where the bearer belongs and `tcp` where + wg's `udp` `ListenPort` goes, and it declares no overlay ep at + all. `parse_wg_maddr()` now rejects it with an actionable + error. +2. **parsing is pure.** #482's helper had the key-check adjacent + to the parse; `verify_wg_peer()` is now a separate, explicitly + composed step that the caller invokes. A parser that shells + out is a nasty surprise. +3. **no `sudo`.** #482 ran `sudo wg show`; a library/example must + never escalate. `wg show` works unprivileged for read on most + setups; if yours needs root, run the script as root rather + than embedding `sudo`. +4. **no new `Address` proto-type.** The tunnel rides *beside* the + inner addr in a frozen `WGTunnelledAddr`, and only `.inner` + crosses into `open_nursery()`. #482 §6 floated a `WGAddress` + registered in `_address_types` — that table is a `bidict` + (1:1 proto-key↔type) and `_addr_to_transport` wants a + `MsgTransport` per addr-type, which `wg` doesn't have. + +## next + +`WGTunnelledAddr` is deliberately example-local. Promoting it to +`tractor.discovery` as a `TunnelledAddress` whose +`.proto_key`/`.unwrap()` delegate to `.inner`, plus +`open_bindspace()` `@acm`s that create/tear down the iface + +netns via `pyroute2`, is layers A→C of the plan doc. diff --git a/examples/wg_lan/host_a_srv.py b/examples/wg_lan/host_a_srv.py new file mode 100644 index 00000000..16432cfb --- /dev/null +++ b/examples/wg_lan/host_a_srv.py @@ -0,0 +1,61 @@ +# tractor: distributed structured concurrency. +''' +Host A: the service host, reachable over a `wg` tunnel. + +Binds `tractor`'s registrar + an `echo_srv` sub-actor on the +tunnel's *overlay* addr, declared as a single `wg` maddr. + +''' +from __future__ import annotations + +import tractor +import trio + +from wg_maddr import ( + parse_wg_maddr, + verify_wg_peer, + WGTunnelledAddr, +) + +# bearer = host A's underlay `(ip, wg ListenPort)` +# key = host A's OWN tunnel pubkey +# overlay = the ep `tractor` binds, on the wg iface's addr +WG_MADDR: str = ( + '/ip4/192.168.1.50/udp/51820' + '/wg/u' + '/ip4/10.0.11.1/tcp/1616' +) + + +async def echo(msg: str) -> str: + actor = tractor.current_actor() + return f'{actor.aid.name!r} echoes: {msg}' + + +async def main(): + addr: WGTunnelledAddr = parse_wg_maddr(WG_MADDR) + assert verify_wg_peer(addr), ( + f'wg pubkey from maddr not active on wg0 !\n' + f'maddr: {WG_MADDR}\n' + f'key: {addr.peer_pubkey}\n' + ) + print( + f'wg bearer (kernel-owned): {addr.bearer}\n' + f'tractor overlay ep: {addr.inner}\n' + ) + async with tractor.open_nursery( + # XXX only `.inner` crosses into the runtime; the bearer + # + key are iface-layer concerns `tractor` never binds. + registry_addrs=[addr.inner], + enable_transports=[addr.inner_proto], + ) as an: + await an.start_actor( + 'echo_srv', + enable_modules=[__name__], + ) + print(f'echo_srv up on\n {addr.maddr}\n') + await trio.sleep_forever() + + +if __name__ == '__main__': + trio.run(main) diff --git a/examples/wg_lan/host_b_client.py b/examples/wg_lan/host_b_client.py new file mode 100644 index 00000000..4806f5de --- /dev/null +++ b/examples/wg_lan/host_b_client.py @@ -0,0 +1,52 @@ +# tractor: distributed structured concurrency. +''' +Host B: workstation dialing host A's actor tree through the +`wg` tunnel. + +''' +from __future__ import annotations + +import tractor +import trio + +from host_a_srv import echo # noqa: F401 (RPC refs it by mod path) +from wg_maddr import ( + parse_wg_maddr, + verify_wg_peer, + WGTunnelledAddr, +) + +# same maddr as host A: A's bearer, A's key, A's overlay ep +WG_MADDR: str = ( + '/ip4/192.168.1.50/udp/51820' + '/wg/u' + '/ip4/10.0.11.1/tcp/1616' +) + + +async def main(): + addr: WGTunnelledAddr = parse_wg_maddr(WG_MADDR) + assert verify_wg_peer(addr), ( + f'wg pubkey from maddr not a peer on wg0 !\n' + f'maddr: {WG_MADDR}\n' + ) + async with ( + tractor.open_root_actor( + name='wg_client', + registry_addrs=[addr.inner], + enable_transports=[addr.inner_proto], + ), + tractor.find_actor( + 'echo_srv', + registry_addrs=[addr.inner], + ) as portal, + ): + res: str = await portal.run( + echo, + msg='hello over wg!', + ) + print(res) + + +if __name__ == '__main__': + trio.run(main) diff --git a/examples/wg_lan/wg_maddr.py b/examples/wg_lan/wg_maddr.py new file mode 100644 index 00000000..44d22e0a --- /dev/null +++ b/examples/wg_lan/wg_maddr.py @@ -0,0 +1,215 @@ +# tractor: distributed structured concurrency. +r''' +Parse `wg`-tunnelled multiaddrs into `tractor`-ready addrs. + +The canonical form (per py-multiaddr PR #108, verified to parse + +round-trip on that branch) nests the *overlay* endpoint **after** +the `/wg/` segment: + + /ip4/10.0.0.1/udp/51820/wg/u/ip4/10.0.11.1/tcp/1616 + \_______ wg bearer ______/\_ key _/\____ tractor ep _____/ + (underlay, wg + `ListenPort`) + +- the segments *before* `/wg/` are the **bearer**: the underlay + `(ip, udp-port)` that `wg(8)` itself listens on. Nothing in + `tractor` ever binds this — the kernel/`wg` iface owns it. +- `/wg/u` carries the tunnel peer's Curve25519 pubkey as + multibase base64url (std base64 from `wg(8)` contains `/` and + can't go in a `/`-delimited maddr). +- the segments *after* are the **overlay** endpoint, i.e. the + addr `tractor` actually binds/dials. This is the only part the + runtime sees. + +XXX NOTE, `tractor`'s own `parse_maddr()` can't parse this yet +(`ValueError('Unsupported multiaddr protocol combo')`), which is +why this module exists: parse here, hand `.inner` to the runtime. + +Design rules this module follows (see +`ai/tpt-backends/03_wg_tunnel_bindspace.md`): + +- **parsing is pure**. `parse_wg_maddr()` does no I/O, no + `subprocess`, no netlink. A parser that shells out is a nasty + surprise. +- **verification is an explicit, separate step**. The caller + composes `verify_wg_peer()` when it wants it; nothing implicit. +- **no new `Address` proto-type**. `wg` gets no entry in + `tractor.discovery._addr._address_types` (a `bidict`, so 1:1 + proto-key<->type) bc it has no `MsgTransport` of its own. The + tunnel is a *bindspace*, so we carry it beside the inner addr + and strip to `.inner` at bind/dial time. + +''' +from __future__ import annotations +import base64 +import subprocess +from typing import Literal + +import msgspec + + +class WGTunnelledAddr( + msgspec.Struct, + frozen=True, +): + ''' + A `wg`-tunnelled endpoint: the underlay bearer, the tunnel + peer key, and the overlay addr `tractor` binds/dials. + + ''' + # underlay, owned by `wg(8)`/the kernel — NEVER bound by us + bearer: tuple[str, int] + + # tunnel peer pubkey in the std-base64 `wg(8)` form, i.e. + # directly comparable to `wg show peers` output + peer_pubkey: str + + # overlay ep: an `UnwrappedAddress` as accepted by + # `tractor.discovery.wrap_address()` + inner: tuple[str, int] + inner_proto: Literal['tcp'] = 'tcp' + + @property + def maddr(self) -> str: + ''' + Re-render the canonical maddr `str` form. + + ''' + b_host, b_port = self.bearer + i_host, i_port = self.inner + return ( + f'/ip4/{b_host}/udp/{b_port}' + f'/wg/{mb_pubkey(self.peer_pubkey)}' + f'/ip4/{i_host}/{self.inner_proto}/{i_port}' + ) + + +def mb_pubkey(wg8_key: str) -> str: + ''' + `wg(8)` std-base64 pubkey -> multibase base64url (`u`-prefixed). + + ''' + import multibase + raw: bytes = base64.b64decode(wg8_key) + return multibase.encode('base64url', raw).decode('ascii') + + +def wg8_pubkey(mb_key: str) -> str: + ''' + Inverse of `mb_pubkey()`: multibase -> `wg(8)` std-base64. + + ''' + import multibase + raw: bytes = multibase.decode(mb_key) + return base64.b64encode(raw).decode('ascii') + + +def parse_wg_maddr( + maddr: str, +) -> WGTunnelledAddr: + ''' + Split a `wg`-tunnelled maddr into its bearer/key/overlay + parts. Pure — no I/O. + + Uses `py-multiaddr` when it knows the `wg` proto (PR #108), + else falls back to a minimal segment split. + + ''' + segs: list[str] = _segments(maddr) + try: + wg_at: int = segs.index('wg') + except ValueError: + raise ValueError( + f'Not a `wg`-tunnelled maddr, no `/wg/` segment ??\n' + f'maddr: {maddr!r}\n' + ) + + bearer_segs: list[str] = segs[:wg_at] + mb_key: str = segs[wg_at + 1] + inner_segs: list[str] = segs[wg_at + 2:] + + match bearer_segs: + case ['ip4'|'ip6', str() as b_host, 'udp', str() as b_port]: + bearer = (b_host, int(b_port)) + case _: + raise ValueError( + f'Bad `wg` bearer, expected `/ip4|ip6//udp/

`\n' + f'got: {"/".join(bearer_segs)!r}\n' + f'from maddr: {maddr!r}\n' + ) + + match inner_segs: + case ['ip4'|'ip6', str() as i_host, 'tcp', str() as i_port]: + inner = (i_host, int(i_port)) + inner_proto = 'tcp' + case []: + raise ValueError( + f'`wg` maddr declares no overlay endpoint!\n' + f'A bare `/…/wg/` names only the tunnel; ' + f'append the ep `tractor` should bind, e.g.\n' + f' {maddr}/ip4/10.0.11.1/tcp/1616\n' + ) + case _: + raise ValueError( + f'Unsupported `wg` overlay proto combo\n' + f'got: {"/".join(inner_segs)!r}\n' + f'from maddr: {maddr!r}\n' + ) + + return WGTunnelledAddr( + bearer=bearer, + peer_pubkey=wg8_pubkey(mb_key), + inner=inner, + inner_proto=inner_proto, + ) + + +def _segments(maddr: str) -> list[str]: + ''' + Deliver a maddr's `/`-split segments, preferring the real + parser when it supports `wg`. + + ''' + from multiaddr import Multiaddr + try: + # the real thing: validates every proto + value + Multiaddr(maddr) + except Exception: + # XXX STOPGAP, only until py-multiaddr#108 lands; then + # this branch is dead and `Multiaddr` is authoritative. + # We deliberately DON'T hand-roll a `wg` codec (the whole + # point of gh #429 was dropping the NIH parser). + pass + return [s for s in maddr.split('/') if s] + + +def verify_wg_peer( + addr: WGTunnelledAddr, + iface: str = 'wg0', +) -> bool: + ''' + True iff `addr.peer_pubkey` is a configured peer (or our own + pubkey) on `iface`. + + IMPURE + explicit by design: never called from + `parse_wg_maddr()`. + + ?TODO, per plan-03 layer B, swap this body for `pyroute2` + (keeping the signature) — and note `setns(2)` is *per-thread*, + so a query issued via `trio.to_thread` lands in the ORIGINAL + netns unless `netns=` is passed down. + + ''' + def _wg(*args: str) -> str: + return subprocess.run( + ['wg', 'show', iface, *args], + capture_output=True, + text=True, + check=True, + ).stdout + + return ( + addr.peer_pubkey in _wg('peers').split() + or + addr.peer_pubkey == _wg('public-key').strip() + ) From 3c6bd928b7e30573afae87f6f8649d17f9cdf479 Mon Sep 17 00:00:00 2001 From: goodboy Date: Wed, 12 Aug 2026 20:08:08 -0400 Subject: [PATCH 09/18] Move the `wg_lan` examples under `examples/multihost/` MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit `tests/test_docs_examples.py` walks `examples/` **recursively** and subproc-runs every collected file asserting `rc == 0`. Ran its exact filter against the tree: all 4 of our files were being collected — including `README.md`, since the filter never checks the extension, so CI would have literally tried `python README.md`. These need a real second host + a live `wg` tunnel, so they can't ever satisfy that gate. `'multihost' not in p[0]` is already in the test's exclusion list w/ no dir yet using it, so this is a pure `git mv` — zero test changes — and it's what the exclusion was plainly there for. Collection drops 24 -> 20 files, 0 of them ours. Also records *why* in the two places someone would look before adding the next one: a callout at the top of the example README and a note on plan 03's §3.4 deliverables. Anything needing a second host or live tunnel goes under `examples/multihost/`. (this patch was generated in some part by `claude-code` using `claude-opus-5` (`anthropic`)) --- ai/tpt-backends/03_wg_tunnel_bindspace.md | 12 ++++++++++-- examples/{ => multihost}/wg_lan/README.md | 7 +++++++ examples/{ => multihost}/wg_lan/host_a_srv.py | 0 examples/{ => multihost}/wg_lan/host_b_client.py | 0 examples/{ => multihost}/wg_lan/wg_maddr.py | 0 5 files changed, 17 insertions(+), 2 deletions(-) rename examples/{ => multihost}/wg_lan/README.md (92%) rename examples/{ => multihost}/wg_lan/host_a_srv.py (100%) rename examples/{ => multihost}/wg_lan/host_b_client.py (100%) rename examples/{ => multihost}/wg_lan/wg_maddr.py (100%) diff --git a/ai/tpt-backends/03_wg_tunnel_bindspace.md b/ai/tpt-backends/03_wg_tunnel_bindspace.md index 6f44c7a9..78d5f1ed 100644 --- a/ai/tpt-backends/03_wg_tunnel_bindspace.md +++ b/ai/tpt-backends/03_wg_tunnel_bindspace.md @@ -134,7 +134,8 @@ examples in gh #482) used a *suffix* form semantically inverted: it puts the overlay addr where the bearer belongs, `tcp` where wg's `udp` `ListenPort` goes, and declares no overlay endpoint at all. `parse_wg_maddr()` in -`examples/wg_lan/` now rejects it with an actionable error. +`examples/multihost/wg_lan/` now rejects it with an actionable +error. Observed protocol-name lists, for writing the `match`: | maddr | `[p.name for p in m.protocols()]` | @@ -197,7 +198,14 @@ side-effect-free; verification is the *caller's* explicit step ### 3.4 deliverables - `examples/` scripts distilled from #482 §§3-5 (this is the - unchecked "commit examples from ^" bullet in #443). + unchecked "commit examples from ^" bullet in #443). They live + under `examples/multihost/` — `test_docs_examples.py` walks + `examples/` recursively and runs every collected file as a + subproc asserting `rc == 0` (it doesn't even filter by + extension, so a stray `README.md` would be `python`-run too), + and `'multihost' not in p[0]` is already in its exclusion + list. Anything needing a real second host or a live tunnel + belongs there. - a `docs/` page: tunnel setup, the maddr form, the two-host run. Keep prose in the docs; keep the examples runnable and minimal. diff --git a/examples/wg_lan/README.md b/examples/multihost/wg_lan/README.md similarity index 92% rename from examples/wg_lan/README.md rename to examples/multihost/wg_lan/README.md index 21396d5b..64c1b36b 100644 --- a/examples/wg_lan/README.md +++ b/examples/multihost/wg_lan/README.md @@ -8,6 +8,13 @@ Supersedes the example set in gh [#482](https://github.com/goodboy/tractor/issues/482) — see [what changed](#what-changed-vs-482). +> **Why `examples/multihost/`?** `tests/test_docs_examples.py` +> walks `examples/` recursively and runs everything it collects +> as a subproc, asserting `rc == 0`. These need a real second +> host and a live `wg` tunnel, so they can't satisfy that; +> `'multihost' not in p[0]` is already in the test's exclusion +> list, which is what keeps them out of CI. + ## the maddr form ``` diff --git a/examples/wg_lan/host_a_srv.py b/examples/multihost/wg_lan/host_a_srv.py similarity index 100% rename from examples/wg_lan/host_a_srv.py rename to examples/multihost/wg_lan/host_a_srv.py diff --git a/examples/wg_lan/host_b_client.py b/examples/multihost/wg_lan/host_b_client.py similarity index 100% rename from examples/wg_lan/host_b_client.py rename to examples/multihost/wg_lan/host_b_client.py diff --git a/examples/wg_lan/wg_maddr.py b/examples/multihost/wg_lan/wg_maddr.py similarity index 100% rename from examples/wg_lan/wg_maddr.py rename to examples/multihost/wg_lan/wg_maddr.py From cd1e8160fe087966542e8f8d0640845951858a84 Mon Sep 17 00:00:00 2001 From: goodboy Date: Wed, 12 Aug 2026 20:36:03 -0400 Subject: [PATCH 10/18] Log prompt-io for the tpt-backend planning arc MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit One record covering all 9 commits on this branch, per the NLNet generative-AI policy and the existing `ai/prompt-io/claude/` convention. Uses diff-ref mode for both the plan docs and the example code (`git diff main..ng_tpts_planning -- `) rather than duplicating content already in `git log -p`. Kept verbatim in the `.raw.md`: the four verified findings (trio's family-agnostic `SocketStream`/`SocketListener`, the round-trip table proving `/wg/` is infix, the proto-key `UnwrappedAddress` rationale, and `setns(2)`'s per-thread reality), since those are reasoning rather than diffable output. `## Human edits` records that the steering here was substantial and mid-session rather than post-hoc: two model claims about wg maddr semantics were challenged and retracted (incl. in an already-posted issue comment), and the proto-key + netns-as-runtime-config framings were human-directed. Also notes the one model-initiated correction — a pre-publication self-review that downgraded the `uniffi`/asyncio thesis and the TIPC duplicate-binder claim to explicitly-flagged assumptions. Prompt-IO: ai/prompt-io/claude/20260813T001102Z_27c34aeb_prompt_io.md (this patch was generated in some part by `claude-code` using `claude-opus-5` (`anthropic`)) --- .../20260813T001102Z_27c34aeb_prompt_io.md | 136 +++++++++++++++ ...20260813T001102Z_27c34aeb_prompt_io.raw.md | 165 ++++++++++++++++++ 2 files changed, 301 insertions(+) create mode 100644 ai/prompt-io/claude/20260813T001102Z_27c34aeb_prompt_io.md create mode 100644 ai/prompt-io/claude/20260813T001102Z_27c34aeb_prompt_io.raw.md diff --git a/ai/prompt-io/claude/20260813T001102Z_27c34aeb_prompt_io.md b/ai/prompt-io/claude/20260813T001102Z_27c34aeb_prompt_io.md new file mode 100644 index 00000000..ee919337 --- /dev/null +++ b/ai/prompt-io/claude/20260813T001102Z_27c34aeb_prompt_io.md @@ -0,0 +1,136 @@ +--- +model: claude-opus-5 +service: claude +session: 7b9c97c4-fff7-4ac4-97fb-35720453308e +timestamp: 2026-08-13T00:11:02Z +git_ref: 27c34aeb +scope: docs+code +substantive: true +raw_file: 20260813T001102Z_27c34aeb_prompt_io.raw.md +--- + +## Prompt + +> draft hyper detailed implementation plans for [three] +> prospective new transport (tpt) backends for tractor's `.ipc` +> layer, from four GitHub issues: TIPC (gh #378) using built-in +> linux socket API w/ `trio` interfacing, leveraging TIPC's +> built-in discovery machinery; QUIC (gh #353) using the `iroh` +> lib, ideally with the py asyncio support (via ffi) rewritten +> for trio; wg (gh #482 and/or #443) with other shuttle-able +> tpts, using `pyroute2`, as much trio wrapping as possible +> where any other async support can be replaced. + +With constraints: "be only slightly speculative"; realistic given +the linked info; the plans must be workable "by another +model/provider without much discrepancy on the design or lib +selections needed"; do it all in a new `/open-wkt` off main; and +be "as modular and modern-async-python-as-nearly-functional-style +as possible." + +Key follow-up prompts that changed the design mid-session: + +> wg requires binding a socket on both ends of the tunnel, the +> new support we added to `py-multiaddr` in their #108 enables +> declaring this in the maddr? […] so the +> `/ip4/10.0.0.1/udp/666/ip4/192.168.0.13/tcp/80` bullet example +> from #443 isn't correct for a wg tunnel maddr and would +> actually end up looking like, +> `/ip4/10.0.0.1/udp/666/wg//ip4/192.168.0.13/tcp/80` no? + +> right, we're likely going to need less `tuple`-formed +> `UnwrappedAddress` usage to make them all work and/or to extend +> the unwrapped form with a proto-str, likely the key from +> `multiaddrs` no? + +> better encapsulating `UnwrappedAddress` for use externally is +> really way overdue; we should likely be encouraging (if not +> preventing unwrapped) wrapped addrs always much like other +> (std) libs, `ipaddress` comes to mind. + +> so iirc netns mgmt becomes a lower-level runtime config-api +> rather then an actor-app-code API which can be dynamically +> changed once the actor is up ya? + +## Response summary + +A research-and-planning branch, `ng_tpts_planning` — **no runtime +code touched**. Five plan docs under `ai/tpt-backends/` plus a +corrected `wg` example set under `examples/multihost/wg_lan/`, +across 9 commits. + +Three design conclusions came out of the planning that outlive +the branch: + +1. **`wg` is a bindspace, not a `MsgTransport`.** The `/wg/` + maddr segment is *infix*: bearer (underlay ip+udp + `ListenPort`, bound by the kernel) → `/wg/u` (an + identity, bound by nothing) → overlay (the only part tractor + binds). Verified empirically by installing + `baudco/py-multiaddr@wg_support` (py-multiaddr#108) in a + throwaway venv and round-tripping each candidate form; gh + #482's original suffix form parses but is semantically + inverted. +2. **Proto-key the `UnwrappedAddress` form**, spelled with the + `multiaddr` protocol names, dispatched via + `_address_types[addr[0]]`. Kills a whole collision class + (TIPC's `(str, int)` ≡ `TCPAddress`; iroh's `(str, str)` + swallowed by the UDS case) and is the recommended migration + *before* any new backend lands. +3. **netns is a runtime/boot-time config API, not an app-code + one** — `setns(2)` is per-thread and won't move + already-created sockets, so there is deliberately no + `await actor.enter_netns(...)`. + +Also verified that `trio.SocketStream`/`SocketListener` are +address-family agnostic (no `AF_*` check anywhere), which is what +makes TIPC the cheapest of the three backends to add. + +Four related issues were annotated with the results (#378, #353, +#482, #443); #443's body was rewritten to reflect the corrected +grammar, with no existing checkbox state changed. + +## Files changed + +- `ai/tpt-backends/00_shared_backend_contract.md` — normative + backend duck-type contract, registration checklist, §1.1 + proto-key conclusion +- `ai/tpt-backends/01_tipc_backend.md` — TIPC plan; service + addressing, `TIPC_TOP_SRV` push registry, instance-collision + hazard, step-0 probe +- `ai/tpt-backends/02_quic_iroh_backend.md` — `iroh` plan; + `uniffi`→`trio` bridge, listener/stream adapters, API-truth + table +- `ai/tpt-backends/03_wg_tunnel_bindspace.md` — `wg`-as-bindspace + plan; verified maddr grammar, 3-owner split, netns reality +- `ai/tpt-backends/README.md` — index +- `examples/multihost/wg_lan/wg_maddr.py` — frozen `msgspec` + tunnelled addr + pure parse/render helpers; impure + `verify_wg_peer()` kept separate +- `examples/multihost/wg_lan/host_a_srv.py` — host-A actor tree +- `examples/multihost/wg_lan/host_b_client.py` — host-B dialer +- `examples/multihost/wg_lan/README.md` — grammar, owner table, + setup, "what changed vs #482" + +## Human edits + +Substantial human steering rather than post-hoc editing; the +corrections were applied by the model in-session after being +challenged: + +- rejected an initial claim that `wg` has "nothing to bind at the + tunnel layer" and supplied the correct composed maddr form, + which forced a rewrite of plan 03 §3.2 and a retraction in the + already-posted #443 comment +- rejected a supporting claim that `/ip4/../udp/443/quic-v1` was + "also composed" +- directed the proto-key/`ipaddress`-discipline conclusion and + the netns-as-runtime-config framing, both of which were then + folded back into the docs +- chose the commit boundaries and authored all commits; ran every + `git` mutation (commit, rebase, push) themselves + +One model-initiated correction pre-publication: a self-review +downgraded two overconfident claims (the `uniffi`/asyncio thesis +and TIPC duplicate-binder behaviour) to explicitly-flagged +assumptions before the #353/#378 comments were posted. diff --git a/ai/prompt-io/claude/20260813T001102Z_27c34aeb_prompt_io.raw.md b/ai/prompt-io/claude/20260813T001102Z_27c34aeb_prompt_io.raw.md new file mode 100644 index 00000000..0e082229 --- /dev/null +++ b/ai/prompt-io/claude/20260813T001102Z_27c34aeb_prompt_io.raw.md @@ -0,0 +1,165 @@ +--- +model: claude-opus-5 +service: claude +timestamp: 2026-08-13T00:11:02Z +git_ref: 27c34aeb +diff_cmd: git diff main..ng_tpts_planning +--- + +# Raw output — next-gen tpt-backend implementation plans + +## Generated planning docs + +> `git diff main..ng_tpts_planning -- ai/tpt-backends/` + +Five markdown docs. `00_shared_backend_contract.md` is normative +and the other three are written against it so they can be worked +independently: + +- **`00_shared_backend_contract.md`** — the backend duck-type + (`Address(msgspec.Struct, frozen=True)` + module-level + `start_listener()`/`close_listener()` + a + `MsgpackStream(MsgpackTransport)`), the + `inspect.getmodule(self.addr)` reflection in + `Endpoint.start_listener()` that forces the Address class and + its listener fns to share a module, a 10-item registration + checklist, the dep policy, the test-harness shape, and §1.1's + proto-key conclusion (below). +- **`01_tipc_backend.md`** — service addressing via + `TIPC_ADDR_NAMESEQ` (bind/publish) and `TIPC_ADDR_NAME` + (connect/lookup), `TIPC_TOP_SRV` topology subscriptions as a + push-based registry, the `get_random()` instance-collision + hazard, and a step-0 capability-probe spike. +- **`02_quic_iroh_backend.md`** — `iroh` over + `aioquic`/`quiche`/`trio-asyncio`, a `_uniffi_trio.py` bridge + built on `TrioToken.run_sync_soon()`, `trio.abc.Listener`/ + `HalfCloseableStream` adapters, and an API-truth table to fill + in during step 0. +- **`03_wg_tunnel_bindspace.md`** — `wg` as a *bindspace* rather + than a `MsgTransport`, a `TunnelledAddress` wrapper delegating + `.proto_key`/`.unwrap()` to `.inner`, `pyroute2` for layer B, + and `@acm`-managed netns/iface for layer C. +- **`README.md`** — index. + +## Generated example code + +> `git diff main..ng_tpts_planning -- examples/multihost/wg_lan/` + +- `wg_maddr.py` — `WGTunnelledAddr(msgspec.Struct, frozen=True)` + carrying `bearer: tuple[str, int]`, `peer_pubkey: str`, + `inner: tuple[str, int]`, `inner_proto: Literal['tcp']`, plus a + `.maddr` property that re-renders the canonical form. Pure + helpers `mb_pubkey()`, `wg8_pubkey()`, `parse_wg_maddr()`, and + `_segments()` (with a marked stopgap for when the `wg` codec + isn't installed). `verify_wg_peer()` is impure **by design** and + kept out of the parse path. +- `host_a_srv.py` / `host_b_client.py` — the two-host runs; both + pass only `addr.inner` to `open_nursery()`/`open_root_actor()`. +- `README.md` — grammar, owner table, `#108`-branch install line, + tunnel setup, "what changed vs #482". + +## Verified findings (non-code, verbatim) + +### `trio` is address-family agnostic + +Read against the installed `trio`. `SocketStream`/`SocketListener` +ctor checks are only "is a trio sock object" + `type == +SOCK_STREAM`, plus an `OSError`-**suppressed** `SO_ACCEPTCONN` +probe. No `AF_*` check anywhere; `TCP_NODELAY`/`TCP_NOTSENT_LOWAT` +are set under `suppress(OSError)`. A TIPC `SOCK_STREAM` sock should +therefore drop straight into `trio.serve_listeners()` with the +existing `MsgpackTransport` framing, making TIPC mostly +table-registration boilerplate w/ zero new deps. + +### the `wg` maddr grammar — `/wg/` is infix, not suffix + +Installed `baudco/py-multiaddr@wg_support` (PR +multiformats/py-multiaddr#108) into a throwaway venv and +round-tripped every candidate form: + +| maddr | `[p.name for p in m.protocols()]` | +| --- | --- | +| `/ip4/1.2.3.4/udp/51820/wg/u` | `['ip4','udp','wg']` | +| `/ip4/../udp/../wg/u/ip4/../tcp/..` | `['ip4','udp','wg','ip4','tcp']` | +| `/ip4/10.0.11.1/tcp/1616/wg/u` | `['ip4','tcp','wg']` | + +``` +/ip4/192.168.1.50/udp/51820/wg/u/ip4/10.0.11.1/tcp/1616 +\_______ bearer __________/\__ key __/\______ overlay ______/ +``` + +Segments *before* `/wg/` are the bearer — the underlay +`(ip, udp-port)` that `wg(8)` itself listens on (`ListenPort`). +Segments *after* are the overlay endpoint, the only part tractor +binds. The third row above is #482's original suffix form: it +parses, but is semantically inverted. + +Three parts, three owners — and only one is an `Endpoint`: + +| part | bound by | in the runtime? | +| --- | --- | --- | +| bearer | kernel, via `wg-quick`/`pyroute2` | no | +| `/wg/u` | nothing — an identity | no, verified out-of-band | +| overlay | `tractor`'s `IPCServer` | yes, as `.inner` | + +### proto-key-tagged `UnwrappedAddress` + +Shape-matching in `wrap_address()` does not survive four backends. +TIPC's natural unwrapped form is a `(str, int)`, indistinguishable +from `TCPAddress`; iroh's is a `(str, str)`, already swallowed by +the existing UDS case (`case (_, filename) if type(filename) is +str`). Ordering hacks and prefix-tagging only paper over it. + +Recommended prerequisite for all three backends: carry an explicit +proto-key spelled with the `multiaddr` protocol name — +`('tcp', host, port)`, `('unix', path)`, +`('tipc', stype, inst, scope)` — so `wrap_address()` collapses to +`_address_types[addr[0]]` and the collision class stops existing. +This also makes the on-wire form agree with +`mk_maddr()`/`parse_maddr()` instead of being an independent +invention. It is a wire-format change (`SpawnSpec`, +`_root_mailbox`, `_registry_addrs`) plus every fixture and +downstream config, so it wants its own migration commit landed +before any new backend — and it is the moment to stop handing raw +tuples to users at all, making `Address` the public currency and +`UnwrappedAddress` an internal serialization detail (the +discipline `ipaddress` uses). + +### netns is a runtime-level config API + +`setns(2)` affects the calling thread only and does not move +already-created sockets. So a netns is a spawn/boot-time input +alongside `enable_transports`/`tpt_bind_addrs`, and there is +deliberately no `await actor.enter_netns(...)` — a mid-life API +would silently leave the IPC server bound in the old namespace. +Corollary for layer B: pass `netns=` down to `pyroute2` rather +than assuming a `trio.to_thread` worker inherits it. + +### `examples/` collection would have failed CI + +`tests/test_docs_examples.py` walks `examples/` recursively and +subproc-runs every collected file asserting `rc == 0`. Its filter +never checks the extension, so all four `wg_lan` files were +collected — including `README.md`, which would have been run as +`python README.md`. `'multihost' not in p[0]` was already in the +exclusion list with no directory using it. Moving the set under +`examples/multihost/wg_lan/` drops collection 24 → 20 with zero +test changes; confirmed via `pytest --collect-only`. + +## Corrections applied during the session + +The human corrected two claims that had been asserted without +verification, both since retracted in-place in the docs and in the +posted issue comments: + +1. that `wg` has "nothing to bind at the tunnel layer, exactly one + bind" — wrong; a wg stack is genuinely composed, and the real + axis is *who owns* each layer's endpoint. +2. that `/ip4/../udp/443/quic-v1` was "also composed" — wrong; + that is one endpoint with a protocol qualifier, not a tunnel. + +A self-review before publication also downgraded two +overconfident claims to explicitly-flagged assumptions: the +`uniffi`-uses-asyncio-only-as-executor thesis (contradicted that +plan's own "do not guess from memory" step 0) and TIPC's +duplicate-binder round-robin behaviour (unverified). From ba6b181cf5d14791a0336e8fb79fc07fa3dc30cc Mon Sep 17 00:00:00 2001 From: goodboy Date: Fri, 14 Aug 2026 09:38:58 -0400 Subject: [PATCH 11/18] Pin `multiaddr` to the merged `wg` codec rev MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit py-multiaddr#108 (the `/wg/u` maddr proto) merged upstream on 2026-07-28 as `f86519da`, but ships in no release yet — the latest `0.2.0` predates it by ~4 months and carries no `wg` codec at all. So `examples/multihost/wg_lan/` can't parse its own maddrs off PyPI. Pinned by `rev` and not `branch` so CI stays reproducible. Note the lock now records the git source *instead of* the `>=0.2.0` specifier, i.e. the dep floor above is fully overridden for as long as this pin lives. TODO, drop the pin (and bump that floor) the moment a release carries the codec; the only consumer is the `wg_lan` example set. (this patch was generated in some part by `claude-code` using `claude-opus-5` (`anthropic`)) --- pyproject.toml | 11 +++++++++++ 1 file changed, 11 insertions(+) diff --git a/pyproject.toml b/pyproject.toml index 123c4108..e51f3035 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -169,6 +169,17 @@ sync_pause = {requires-python = ">=3.13, <3.14"} # linux kernel networking # 'pyroute2 +# XXX TEMP, the `/wg/u` maddr proto is MERGED upstream (in +# py-multiaddr#108, 2026-07-28) but is in NO release yet; the +# latest `0.2.0` (2026-03-17) predates the merge by ~4 months. +# Pinned by `rev` (not `branch`) so CI stays reproducible. +# +# Drop this pin (and bump the `multiaddr` dep floor above) the +# moment a release carries the `wg` codec; the only consumer is +# `examples/multihost/wg_lan/`. +# |_https://github.com/multiformats/py-multiaddr/pull/108 +multiaddr = { git = 'https://github.com/multiformats/py-multiaddr.git', rev = 'f86519daaa21699023d0037c58cdff600313dd09' } + # ------ tool.uv.sources ------ [tool.uv] From 13cdc43f0188d568b2cfcfd1e20471a678138844 Mon Sep 17 00:00:00 2001 From: goodboy Date: Fri, 14 Aug 2026 09:50:48 -0400 Subject: [PATCH 12/18] Fix silently-corrupt keys in `parse_wg_maddr()` MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit `_segments()` called `Multiaddr(maddr)` purely to validate, then swallowed every failure under `except Exception: pass`. That was harmless pre-#108 — w/o a `wg` codec there was nothing to validate — but now that the codec is pinned in, the swallow is load-bearing and disabled: a malformed key sails past validation into `wg8_pubkey()`, which happily emits a corrupt b64 str, and the returned struct then fails its own `.maddr` round-trip. No raise, just quietly wrong output. Deats, - add `_have_wg_maddr_proto()`, the gate plan-03 already referenced but which never actually existed. Impl'd as `protocols.protocol_with_name('wg')` under `except ProtocolNotFoundError` and cached in a mod global, same shape as the TIPC plan's `is_tipc_available()`. - only validate when that gate is `True`, and let `StringParseError` propagate — a maddr which doesn't parse must NOT reach `wg8_pubkey()`. - keep the degraded split for a pre-#108 install, now w/ an explicit `XXX` naming the validation you give up. So parsing stays pure but becomes total-or-raises. Our own `ValueError`s (missing `/wg/` seg, bare tunnel w/o an overlay ep) are unaffected, as is the `wg(8)` b64 round-trip. (this patch was generated in some part by `claude-code` using `claude-opus-5` (`anthropic`)) --- examples/multihost/wg_lan/wg_maddr.py | 67 ++++++++++++++++++++------- 1 file changed, 50 insertions(+), 17 deletions(-) diff --git a/examples/multihost/wg_lan/wg_maddr.py b/examples/multihost/wg_lan/wg_maddr.py index 44d22e0a..0a4c3385 100644 --- a/examples/multihost/wg_lan/wg_maddr.py +++ b/examples/multihost/wg_lan/wg_maddr.py @@ -2,9 +2,9 @@ r''' Parse `wg`-tunnelled multiaddrs into `tractor`-ready addrs. -The canonical form (per py-multiaddr PR #108, verified to parse + -round-trip on that branch) nests the *overlay* endpoint **after** -the `/wg/` segment: +The canonical form (per py-multiaddr #108, verified to parse + +round-trip against its upstream merge) nests the *overlay* +endpoint **after** the `/wg/` segment: /ip4/10.0.0.1/udp/51820/wg/u/ip4/10.0.11.1/tcp/1616 \_______ wg bearer ______/\_ key _/\____ tractor ep _____/ @@ -111,8 +111,10 @@ def parse_wg_maddr( Split a `wg`-tunnelled maddr into its bearer/key/overlay parts. Pure — no I/O. - Uses `py-multiaddr` when it knows the `wg` proto (PR #108), - else falls back to a minimal segment split. + Total-or-raises: with a `wg`-aware `py-multiaddr` (#108) an + unparseable maddr raises instead of yielding a struct built + from garbage segments. See `_segments()` for the degraded + pre-#108 path. ''' segs: list[str] = _segments(maddr) @@ -164,22 +166,53 @@ def parse_wg_maddr( ) -def _segments(maddr: str) -> list[str]: +_wg_proto_known: bool|None = None + + +def _have_wg_maddr_proto() -> bool: ''' - Deliver a maddr's `/`-split segments, preferring the real - parser when it supports `wg`. + True iff the installed `py-multiaddr` knows the `/wg/` proto, + i.e. carries py-multiaddr#108. + + Merged upstream 2026-07-28 but in no release as of `0.2.0`, + hence the `[tool.uv.sources]` `rev` pin. + + Pure predicate; result cached since it can't change without a + reinstall. ''' - from multiaddr import Multiaddr - try: - # the real thing: validates every proto + value + global _wg_proto_known + if _wg_proto_known is None: + from multiaddr.protocols import protocol_with_name + from multiaddr.exceptions import ProtocolNotFoundError + try: + protocol_with_name('wg') + _wg_proto_known = True + except ProtocolNotFoundError: + _wg_proto_known = False + + return _wg_proto_known + + +def _segments(maddr: str) -> list[str]: + ''' + Deliver a maddr's `/`-split segments, validating via the real + parser whenever it knows `wg`. + + ''' + if _have_wg_maddr_proto(): + from multiaddr import Multiaddr + # the real thing: validates every proto + value, incl. + # that the `wg` key decodes to exactly 32 bytes. Let it + # raise — a maddr that doesn't parse must NOT reach + # `wg8_pubkey()`, which would happily emit a corrupt key. Multiaddr(maddr) - except Exception: - # XXX STOPGAP, only until py-multiaddr#108 lands; then - # this branch is dead and `Multiaddr` is authoritative. - # We deliberately DON'T hand-roll a `wg` codec (the whole - # point of gh #429 was dropping the NIH parser). - pass + + # XXX, degraded path for a pre-#108 `py-multiaddr` ONLY: no + # per-segment validation, so a malformed key survives to the + # returned struct. We deliberately DON'T hand-roll a `wg` + # codec (the whole point of gh #429 was dropping the NIH + # parser) — install the pinned rev to get validation back. return [s for s in maddr.split('/') if s] From 8e39fd8b077e10c8fa528748d395b73eaeef3178 Mon Sep 17 00:00:00 2001 From: goodboy Date: Fri, 14 Aug 2026 09:51:36 -0400 Subject: [PATCH 13/18] Update `wg` docs for the merged py-multiaddr#108 MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit it lands" framing in plan-03 and the example README was stale in both directions: the branch pin is obsolete, yet you still can't just `pip install multiaddr`. Deats, - §3.2's grammar table is now re-verified against the upstream merge (`f86519da`) rather than only `baudco@wg_support` in a throwaway venv. Also notes the codec enforces a 32-byte key, so a truncated one is a `StringParseError` and not a silently mangled parse. - §1 says merged-but-unreleased; the still-open work is spec registration (py-multiaddr#107 + gh #483). - §3.4 swaps "pin the branch" for the `[tool.uv.sources]` `rev` pin, and fixes the `_have_wg_maddr_proto()` recipe it suggested — probing w/ `Multiaddr('/wg/uAAAA')` now ALWAYS raises bc the codec wants 32B, i.e. that feature-detect would report `False` even w/ the proto perfectly well known. - risk table row goes "#108 not merged" -> "merged but unreleased". - example README: `uv sync` alone now suffices bc of the pin; documents the 32B check and points at `_have_wg_maddr_proto()` as the gate. The one surviving `baudco` mention is deliberate, it records where the grammar was *first* verified. (this patch was generated in some part by `claude-code` using `claude-opus-5` (`anthropic`)) --- ai/tpt-backends/03_wg_tunnel_bindspace.md | 37 ++++++++++++++--------- examples/multihost/wg_lan/README.md | 23 ++++++++++---- 2 files changed, 39 insertions(+), 21 deletions(-) diff --git a/ai/tpt-backends/03_wg_tunnel_bindspace.md b/ai/tpt-backends/03_wg_tunnel_bindspace.md index 78d5f1ed..ffb0dfd3 100644 --- a/ai/tpt-backends/03_wg_tunnel_bindspace.md +++ b/ai/tpt-backends/03_wg_tunnel_bindspace.md @@ -26,10 +26,11 @@ onto `trio` as the library's sans-io layer allows. `/ip4|ip6//tcp/

` and `/unix/

`; a `.../wg/u` maddr raises `ValueError('Unsupported multiaddr protocol combo')`. -- there is no `wg` proto in the multiaddr spec; the first-draft - upstream PR is multiformats/py-multiaddr#108 with key form - `u` (commit `8be3a8b`), tracked by - multiformats/py-multiaddr#107 and gh #483. +- there is no `wg` proto in the multiaddr *spec* yet, but + multiformats/py-multiaddr#108 (key form `u`) is + **merged** as of 2026-07-28 (`f86519da`) — and unreleased, the + latest `0.2.0` predating it. Spec registration is still tracked + by multiformats/py-multiaddr#107 and gh #483. - so **today's deployable story is declarative**: run `wg-quick` out-of-band, parse the maddr, strip to the inner `(host, port)`, verify the pubkey against the live tunnel, @@ -112,9 +113,12 @@ class WGTunnelSpec( ### 3.2 `parse_maddr()`/`mk_maddr()` -Grammar — **verified** against py-multiaddr#108 -(`baudco/py-multiaddr@wg_support`, installed in a throwaway venv; -all three forms below parse *and* round-trip): +Grammar — **verified** against py-multiaddr#108, first on the +`baudco/py-multiaddr@wg_support` branch and re-verified after it +merged upstream (`multiformats/py-multiaddr@f86519da`); all three +forms below parse *and* round-trip. Note the codec also validates +that the key decodes to exactly 32 bytes, so a truncated key is a +`StringParseError`, not a silently-mangled parse: ``` /ip4/192.168.1.50/udp/51820/wg/u/ip4/10.0.11.1/tcp/1616 @@ -169,13 +173,16 @@ Observed protocol-name lists, for writing the `match`: (bearer_names, tunnel_specs, inner_names)`. This is also what makes a wg-inside-wg stack fall out for free. - `mk_maddr()` inverse for `TunnelledAddress`. -- **blocked on upstream**: `Multiaddr('/…/wg/u…')` only parses - once py-multiaddr#108 lands. Until then: pin the branch in the - `wg` extra / dev-group and gate the tests on - `_have_wg_maddr_proto()` (a cheap try/except around - `Multiaddr('/wg/uAAAA')`). Do **not** hand-roll a `wg` parser - in `tractor` — the whole point of #429 was dropping the NIH - parser. +- **pending an upstream release**: py-multiaddr#108 is merged, so + `Multiaddr('/…/wg/u…')` parses — but off a `[tool.uv.sources]` + `rev` pin, since no release carries the codec. Gate the tests + on `_have_wg_maddr_proto()`, implemented as + `protocols.protocol_with_name('wg')` under + `except ProtocolNotFoundError`. Do **not** probe by parsing a + dummy like `Multiaddr('/wg/uAAAA')` — the codec enforces a + 32-byte key, so that raises even when the proto *is* known. Do + **not** hand-roll a `wg` parser in `tractor` — the whole point + of #429 was dropping the NIH parser. ### 3.3 verification helper (pure, composable) @@ -437,7 +444,7 @@ consider doing it *first* for exactly that reason. | risk | mitigation | | --- | --- | | `to_thread` worker runs in the wrong netns | §5.3; pass `netns=` to pyroute2 or pin a worker; test-first | -| py-multiaddr#108 not merged | branch pin + `_have_wg_maddr_proto()` gate; layer A's inner-addr path works regardless | +| py-multiaddr#108 merged but unreleased | `[tool.uv.sources]` `rev` pin + `_have_wg_maddr_proto()` gate; layer A's inner-addr path works regardless | | `TunnelledAddress` leaks into `Endpoint` and breaks `inspect.getmodule()` | unwrap at parse/bindspace boundary; assert `not isinstance(ep.addr, TunnelledAddress)` in `Endpoint.__post_init__` | | privileged ops in a library | never `sudo`; explicit cap probe + actionable error; pre-provisioned is the default | | pyroute2 0.9 asyncio core drags a loop into the actor | option (1) is a *thread*, not a loop; forbid `trio-asyncio` here (§4.1) | diff --git a/examples/multihost/wg_lan/README.md b/examples/multihost/wg_lan/README.md index 64c1b36b..8bea3344 100644 --- a/examples/multihost/wg_lan/README.md +++ b/examples/multihost/wg_lan/README.md @@ -39,16 +39,27 @@ this composed form parses and round-trips ## requirements -The `wg` proto isn't in released `py-multiaddr` yet (`0.2.0` has -no `wg` codec), so until #108 lands: +py-multiaddr #108 is **merged** (2026-07-28) but ships in no +release yet — the latest `0.2.0` (2026-03-17) predates it and has +no `wg` codec. So `pyproject.toml` carries a temporary +`[tool.uv.sources]` `rev` pin at the merge commit, and a plain ```bash -uv pip install 'git+https://github.com/baudco/py-multiaddr.git@wg_support' multibase +uv sync ``` -`wg_maddr.py` degrades to a plain segment split when the codec is -absent, so the examples still run — but you lose per-segment -validation. It deliberately does **not** hand-roll a `wg` codec +gets you a `wg`-aware `multiaddr`. That pin goes away once a +release carries the codec. You also need `multibase`: + +```bash +uv pip install multibase +``` + +Without the codec `wg_maddr.py` degrades to a plain segment split +— the examples still run, but you lose per-segment validation +(incl. the 32-byte key-length check), so a malformed key reaches +the returned struct instead of raising. `_have_wg_maddr_proto()` +is the gate. It deliberately does **not** hand-roll a `wg` codec (gh #429 was about *dropping* our NIH parser). ## 0. tunnel setup (out-of-band, both hosts) From 4c8188652b4380155bcfefba98511ffe410b33d8 Mon Sep 17 00:00:00 2001 From: goodboy Date: Mon, 17 Aug 2026 17:25:07 -0400 Subject: [PATCH 14/18] Peel `wg` maddrs w/ `py-multiaddr`'s own tunnel API MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit `py-multiaddr` already ships the entire tunnel compose/peel surface and this module was reimplementing it — a raw `maddr.split('/')` plus index arithmetic, sitting directly under a comment congratulating itself for not hand-rolling a parser. Same NIH trap gh #429 existed to close, just one layer up. The API was linked from gh #443's own 2nd bullet the whole time. So every cut now goes through the real thing, | need | API | | --- | --- | | isolate the bearer | `.decapsulate_code(P_WG)` | | per-seg maddrs | `.split()` | | rejoin a seg tail | `Multiaddr.join()` | | read the key | `.value_for_protocol('wg')` | | recompose | `.encapsulate()` | `.decapsulate_code()` turns out to handle the infix `/wg/` seg cleanly *because* it cuts on proto-code and never tries to match an addr value — the key seg has no addr of its own, which was the exact thing I'd assumed would need bespoke handling. Deats, - rename the role fields `inner`/`inner_proto` -> `overlay`/`overlay_proto`, matching `py-multiaddr`'s encapsulation model (earlier segs wrap later ones) and #443's owner table. `inner` collided head-on w/ call-stack `inner`, where it reads as higher-up + later-called, while here the encapsulated addr is bound *first* and sits deeper. - drop `_segments()` and its degraded hand-split path entirely. W/o the codec there's now one actionable `RuntimeError` instead of a silent downgrade, superseding the swallow fix in 7d6e7955. - add `.as_multiaddr()` so callers can stay in `Multiaddr` land; `.maddr` is now just `str()` of it. - accept `str|Multiaddr` on the way in. - carry `bearer_ip`/`overlay_ip` so a v6 stack re-renders as v6 — the old `.maddr` hardcoded `/ip4/` and would silently mangle it. - both host scripts follow the rename to `.overlay`. ⚠️ `value_for_protocol('ip4')` on a *full* tunnelled maddr silently returns the **first** match, i.e. the bearer's host, so it's only ever called here on an already-peeled sub-maddr. (this patch was generated in some part by `claude-code` using `claude-opus-5` (`anthropic`)) --- examples/multihost/wg_lan/host_a_srv.py | 8 +- examples/multihost/wg_lan/host_b_client.py | 6 +- examples/multihost/wg_lan/wg_maddr.py | 257 ++++++++++++--------- 3 files changed, 161 insertions(+), 110 deletions(-) diff --git a/examples/multihost/wg_lan/host_a_srv.py b/examples/multihost/wg_lan/host_a_srv.py index 16432cfb..21ccc50a 100644 --- a/examples/multihost/wg_lan/host_a_srv.py +++ b/examples/multihost/wg_lan/host_a_srv.py @@ -41,13 +41,13 @@ async def main(): ) print( f'wg bearer (kernel-owned): {addr.bearer}\n' - f'tractor overlay ep: {addr.inner}\n' + f'tractor overlay ep: {addr.overlay}\n' ) async with tractor.open_nursery( - # XXX only `.inner` crosses into the runtime; the bearer + # XXX only `.overlay` crosses into the runtime; the bearer # + key are iface-layer concerns `tractor` never binds. - registry_addrs=[addr.inner], - enable_transports=[addr.inner_proto], + registry_addrs=[addr.overlay], + enable_transports=[addr.overlay_proto], ) as an: await an.start_actor( 'echo_srv', diff --git a/examples/multihost/wg_lan/host_b_client.py b/examples/multihost/wg_lan/host_b_client.py index 4806f5de..cd9970be 100644 --- a/examples/multihost/wg_lan/host_b_client.py +++ b/examples/multihost/wg_lan/host_b_client.py @@ -33,12 +33,12 @@ async def main(): async with ( tractor.open_root_actor( name='wg_client', - registry_addrs=[addr.inner], - enable_transports=[addr.inner_proto], + registry_addrs=[addr.overlay], + enable_transports=[addr.overlay_proto], ), tractor.find_actor( 'echo_srv', - registry_addrs=[addr.inner], + registry_addrs=[addr.overlay], ) as portal, ): res: str = await portal.run( diff --git a/examples/multihost/wg_lan/wg_maddr.py b/examples/multihost/wg_lan/wg_maddr.py index 0a4c3385..67549568 100644 --- a/examples/multihost/wg_lan/wg_maddr.py +++ b/examples/multihost/wg_lan/wg_maddr.py @@ -2,32 +2,49 @@ r''' Parse `wg`-tunnelled multiaddrs into `tractor`-ready addrs. -The canonical form (per py-multiaddr #108, verified to parse + -round-trip against its upstream merge) nests the *overlay* -endpoint **after** the `/wg/` segment: +The canonical form (per py-multiaddr #108, verified against its +upstream merge) nests the *overlay* endpoint **after** the `/wg/` +segment: /ip4/10.0.0.1/udp/51820/wg/u/ip4/10.0.11.1/tcp/1616 \_______ wg bearer ______/\_ key _/\____ tractor ep _____/ (underlay, wg `ListenPort`) -- the segments *before* `/wg/` are the **bearer**: the underlay +Naming follows `py-multiaddr`'s own encapsulation model, where +earlier segments *wrap* later ones (`.encapsulate()` appends), so +the two roles are: + +- **bearer**: the segs *before* `/wg/`, i.e. the underlay `(ip, udp-port)` that `wg(8)` itself listens on. Nothing in `tractor` ever binds this — the kernel/`wg` iface owns it. -- `/wg/u` carries the tunnel peer's Curve25519 pubkey as - multibase base64url (std base64 from `wg(8)` contains `/` and - can't go in a `/`-delimited maddr). -- the segments *after* are the **overlay** endpoint, i.e. the - addr `tractor` actually binds/dials. This is the only part the - runtime sees. +- **overlay**: the segs *after*, i.e. the addr `tractor` actually + binds/dials. The only part the runtime ever sees. + +We deliberately avoid `inner`/`outer` for these two: in a *call* +stack "inner" reads as higher-up and later-called, whereas here +the encapsulated addr is bound *first* and sits deeper in the +maddr — two opposite intuitions on one word. + +`/wg/u` itself carries the tunnel peer's Curve25519 pubkey +as multibase base64url (std base64 from `wg(8)` contains `/` and +so can't go in a `/`-delimited maddr). It binds nothing at all; +it's an identity, verified out-of-band. XXX NOTE, `tractor`'s own `parse_maddr()` can't parse this yet (`ValueError('Unsupported multiaddr protocol combo')`), which is -why this module exists: parse here, hand `.inner` to the runtime. +why this module exists: peel here, hand `.overlay` to the +runtime. Design rules this module follows (see `ai/tpt-backends/03_wg_tunnel_bindspace.md`): +- **let `py-multiaddr` do the parsing**. Every peel/compose goes + through `.decapsulate_code()`, `.split()`, `.join()`, + `.encapsulate()` and `.value_for_protocol()`. We hand-roll no + segment splitting whatsoever — the whole point of gh #429 was + dropping the NIH parser, and that applies to *peeling a tunnel + stack* every bit as much as to decoding a single proto. - **parsing is pure**. `parse_wg_maddr()` does no I/O, no `subprocess`, no netlink. A parser that shells out is a nasty surprise. @@ -36,8 +53,8 @@ Design rules this module follows (see - **no new `Address` proto-type**. `wg` gets no entry in `tractor.discovery._addr._address_types` (a `bidict`, so 1:1 proto-key<->type) bc it has no `MsgTransport` of its own. The - tunnel is a *bindspace*, so we carry it beside the inner addr - and strip to `.inner` at bind/dial time. + tunnel is a *bindspace*, so we carry it beside the overlay + addr and strip to `.overlay` at bind/dial time. ''' from __future__ import annotations @@ -46,6 +63,11 @@ import subprocess from typing import Literal import msgspec +from multiaddr import Multiaddr +from multiaddr.protocols import P_WG + + +IPProto = Literal['ip4', 'ip6'] class WGTunnelledAddr( @@ -66,22 +88,43 @@ class WGTunnelledAddr( # overlay ep: an `UnwrappedAddress` as accepted by # `tractor.discovery.wrap_address()` - inner: tuple[str, int] - inner_proto: Literal['tcp'] = 'tcp' + overlay: tuple[str, int] + overlay_proto: Literal['tcp'] = 'tcp' + + # kept so `.as_multiaddr()` re-renders the same ip family it + # was parsed from, rather than assuming v4 + bearer_ip: IPProto = 'ip4' + overlay_ip: IPProto = 'ip4' + + def as_multiaddr(self) -> Multiaddr: + ''' + Re-compose the canonical `Multiaddr`, bearer outward-in, + using `.encapsulate()` exactly as py-multiaddr's own + tunneling example does. + + ''' + b_host, b_port = self.bearer + o_host, o_port = self.overlay + return ( + Multiaddr(f'/{self.bearer_ip}/{b_host}/udp/{b_port}') + .encapsulate( + Multiaddr(f'/wg/{mb_pubkey(self.peer_pubkey)}') + ) + .encapsulate( + Multiaddr( + f'/{self.overlay_ip}/{o_host}' + f'/{self.overlay_proto}/{o_port}' + ) + ) + ) @property def maddr(self) -> str: ''' - Re-render the canonical maddr `str` form. + The canonical maddr `str` form. ''' - b_host, b_port = self.bearer - i_host, i_port = self.inner - return ( - f'/ip4/{b_host}/udp/{b_port}' - f'/wg/{mb_pubkey(self.peer_pubkey)}' - f'/ip4/{i_host}/{self.inner_proto}/{i_port}' - ) + return str(self.as_multiaddr()) def mb_pubkey(wg8_key: str) -> str: @@ -104,68 +147,6 @@ def wg8_pubkey(mb_key: str) -> str: return base64.b64encode(raw).decode('ascii') -def parse_wg_maddr( - maddr: str, -) -> WGTunnelledAddr: - ''' - Split a `wg`-tunnelled maddr into its bearer/key/overlay - parts. Pure — no I/O. - - Total-or-raises: with a `wg`-aware `py-multiaddr` (#108) an - unparseable maddr raises instead of yielding a struct built - from garbage segments. See `_segments()` for the degraded - pre-#108 path. - - ''' - segs: list[str] = _segments(maddr) - try: - wg_at: int = segs.index('wg') - except ValueError: - raise ValueError( - f'Not a `wg`-tunnelled maddr, no `/wg/` segment ??\n' - f'maddr: {maddr!r}\n' - ) - - bearer_segs: list[str] = segs[:wg_at] - mb_key: str = segs[wg_at + 1] - inner_segs: list[str] = segs[wg_at + 2:] - - match bearer_segs: - case ['ip4'|'ip6', str() as b_host, 'udp', str() as b_port]: - bearer = (b_host, int(b_port)) - case _: - raise ValueError( - f'Bad `wg` bearer, expected `/ip4|ip6//udp/

`\n' - f'got: {"/".join(bearer_segs)!r}\n' - f'from maddr: {maddr!r}\n' - ) - - match inner_segs: - case ['ip4'|'ip6', str() as i_host, 'tcp', str() as i_port]: - inner = (i_host, int(i_port)) - inner_proto = 'tcp' - case []: - raise ValueError( - f'`wg` maddr declares no overlay endpoint!\n' - f'A bare `/…/wg/` names only the tunnel; ' - f'append the ep `tractor` should bind, e.g.\n' - f' {maddr}/ip4/10.0.11.1/tcp/1616\n' - ) - case _: - raise ValueError( - f'Unsupported `wg` overlay proto combo\n' - f'got: {"/".join(inner_segs)!r}\n' - f'from maddr: {maddr!r}\n' - ) - - return WGTunnelledAddr( - bearer=bearer, - peer_pubkey=wg8_pubkey(mb_key), - inner=inner, - inner_proto=inner_proto, - ) - - _wg_proto_known: bool|None = None @@ -174,8 +155,9 @@ def _have_wg_maddr_proto() -> bool: True iff the installed `py-multiaddr` knows the `/wg/` proto, i.e. carries py-multiaddr#108. - Merged upstream 2026-07-28 but in no release as of `0.2.0`, - hence the `[tool.uv.sources]` `rev` pin. + Merged upstream 2026-07-28 (`f86519da`) but in no release as + of `0.2.0`, hence the `[tool.uv.sources]` `rev` pin in + `pyproject.toml`. Pure predicate; result cached since it can't change without a reinstall. @@ -194,26 +176,95 @@ def _have_wg_maddr_proto() -> bool: return _wg_proto_known -def _segments(maddr: str) -> list[str]: +def parse_wg_maddr( + maddr: str|Multiaddr, +) -> WGTunnelledAddr: ''' - Deliver a maddr's `/`-split segments, validating via the real - parser whenever it knows `wg`. + Peel a `wg`-tunnelled maddr into its bearer/key/overlay + parts. Pure — no I/O. + + Every cut is made by `py-multiaddr`, so a malformed maddr + (incl. a `wg` key that isn't exactly 32B) raises out of + `Multiaddr()` rather than yielding a struct quietly built + from garbage segs. ''' - if _have_wg_maddr_proto(): - from multiaddr import Multiaddr - # the real thing: validates every proto + value, incl. - # that the `wg` key decodes to exactly 32 bytes. Let it - # raise — a maddr that doesn't parse must NOT reach - # `wg8_pubkey()`, which would happily emit a corrupt key. - Multiaddr(maddr) + if not _have_wg_maddr_proto(): + raise RuntimeError( + f'Installed `py-multiaddr` has no `/wg/` proto!\n' + f'Needs py-multiaddr#108, merged upstream but not\n' + f'yet released; a `uv sync` picks up the pinned rev.\n' + f'maddr: {maddr!r}\n' + ) - # XXX, degraded path for a pre-#108 `py-multiaddr` ONLY: no - # per-segment validation, so a malformed key survives to the - # returned struct. We deliberately DON'T hand-roll a `wg` - # codec (the whole point of gh #429 was dropping the NIH - # parser) — install the pinned rev to get validation back. - return [s for s in maddr.split('/') if s] + ma: Multiaddr = ( + maddr + if isinstance(maddr, Multiaddr) + else Multiaddr(maddr) + ) + segs: list[Multiaddr] = ma.split() + names: list[str] = [ + proto.name + for seg in segs + for proto in seg.protocols() + ] + if 'wg' not in names: + raise ValueError( + f'Not a `wg`-tunnelled maddr, no `/wg/` segment ??\n' + f'maddr: {ma}\n' + ) + + # NOTE, `.decapsulate_code()` cuts at the LAST occurrence of + # the proto and keeps the *prefix*, which is exactly the + # bearer. It handles `/wg/` cleanly precisely bc it cuts on + # proto-code and never tries to match an addr value — the + # key seg has no addr of its own. + bearer_ma: Multiaddr = ma.decapsulate_code(P_WG) + overlay_ma: Multiaddr = Multiaddr.join( + *segs[names.index('wg') + 1:] + ) + + match [proto.name for proto in bearer_ma.protocols()]: + case [('ip4' | 'ip6') as b_ip, 'udp']: + bearer = ( + bearer_ma.value_for_protocol(b_ip), + int(bearer_ma.value_for_protocol('udp')), + ) + case _: + raise ValueError( + f'Bad `wg` bearer, expected `/ip4|ip6//udp/

`\n' + f'got: {bearer_ma}\n' + f'from maddr: {ma}\n' + ) + + match [proto.name for proto in overlay_ma.protocols()]: + case [('ip4' | 'ip6') as o_ip, ('tcp') as l4]: + overlay = ( + overlay_ma.value_for_protocol(o_ip), + int(overlay_ma.value_for_protocol(l4)), + ) + case []: + raise ValueError( + f'`wg` maddr declares no overlay endpoint!\n' + f'A bare `/…/wg/` names only the tunnel; ' + f'append the ep `tractor` should bind, e.g.\n' + f' {ma}/ip4/10.0.11.1/tcp/1616\n' + ) + case _: + raise ValueError( + f'Unsupported `wg` overlay proto combo\n' + f'got: {overlay_ma}\n' + f'from maddr: {ma}\n' + ) + + return WGTunnelledAddr( + bearer=bearer, + peer_pubkey=wg8_pubkey(ma.value_for_protocol('wg')), + overlay=overlay, + overlay_proto=l4, + bearer_ip=b_ip, + overlay_ip=o_ip, + ) def verify_wg_peer( From 768b531662a88037e3988885102d2ff24a7ce34e Mon Sep 17 00:00:00 2001 From: goodboy Date: Mon, 17 Aug 2026 17:25:20 -0400 Subject: [PATCH 15/18] Retract the hand-rolled tunnel peeler from plan-03 MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit §3.2 specced a pure fn `_peel_tunnel_segs(proto_names) -> (bearer_names, tunnel_specs, overlay_names)` to split a maddr at its tunnel seg. It should never be written: `py-multiaddr` ships that whole surface already and the plan simply missed it, even though gh #443's 2nd bullet links the README sections in question. Replaced w/ a ⚠️ CORRECTION carrying the verified API table (`.decapsulate_code(P_WG)` for the bearer, `.split()`/`.join()` for a seg tail, `.value_for_protocol()` to read a value, `.encapsulate()` to recompose) plus *why* it works on an infix `/wg/` seg: the cut is by proto-code, never by matching an addr value, and the key seg has no addr of its own. Also, - adopt `bearer`/`overlay` as the role names throughout, and say plainly why not `inner`/`outer` — the call-stack reading of "inner" is the exact opposite of the encapsulation one. - warn that `value_for_protocol('ip4')` on a full tunnelled maddr silently yields the *bearer's* host; only call it on a peeled sub-maddr. - note nesting (wg-in-wg) falls out of `.decapsulate_code()` cutting at the *last* occurrence, so peel repeatedly rather than recursing through a bespoke splitter. - `mk_maddr()` for `TunnelledAddress` is `.encapsulate()` composition, not `str` building. - README: drop the "degrades to a plain segment split" para, since that path is gone — no codec now means one actionable raise. (this patch was generated in some part by `claude-code` using `claude-opus-5` (`anthropic`)) --- ai/tpt-backends/03_wg_tunnel_bindspace.md | 69 +++++++++++++++-------- examples/multihost/wg_lan/README.md | 23 +++++--- 2 files changed, 60 insertions(+), 32 deletions(-) diff --git a/ai/tpt-backends/03_wg_tunnel_bindspace.md b/ai/tpt-backends/03_wg_tunnel_bindspace.md index ffb0dfd3..36c975b5 100644 --- a/ai/tpt-backends/03_wg_tunnel_bindspace.md +++ b/ai/tpt-backends/03_wg_tunnel_bindspace.md @@ -32,9 +32,9 @@ onto `trio` as the library's sans-io layer allows. latest `0.2.0` predating it. Spec registration is still tracked by multiformats/py-multiaddr#107 and gh #483. - so **today's deployable story is declarative**: run `wg-quick` - out-of-band, parse the maddr, strip to the inner + out-of-band, parse the maddr, strip to the overlay `(host, port)`, verify the pubkey against the live tunnel, - hand the inner addr to `registry_addrs=`/`tpt_bind_addrs=`. + hand the overlay addr to `registry_addrs=`/`tpt_bind_addrs=`. #482 already contains working example code for exactly this. - `Address.namespace` exists in the Protocol (`_addr.py:94-101`, "the if-available OS-specific network @@ -45,7 +45,7 @@ onto `trio` as the library's sans-io layer allows. | layer | what | dep | ships | | --- | --- | --- | --- | -| **A. declarative** | commit #482's examples; `parse_maddr()` learns `/wg/u` → inner `Address` + verified pubkey | `multiaddr` (already), `wg(8)` CLI | first | +| **A. declarative** | commit #482's examples; `parse_maddr()` learns `/wg/u` → overlay `Address` + verified pubkey | `multiaddr` (already), `wg(8)` CLI | first | | **B. `pyroute2` read/verify** | replace the `subprocess.run(['sudo','wg','show'])` shelling with netlink queries | `pyroute2` extra | second | | **C. `@acm` lifecycle** | create/configure/tear down wg ifaces + netns *from the runtime*, as nested bindspaces; implement `Address.namespace` | `pyroute2` + `CAP_NET_ADMIN` | third | @@ -70,16 +70,16 @@ does not create a new address type.** Two candidate encodings; msgspec.Struct, frozen=True, ): - inner: Address # e.g. TCPAddress + overlay: Address # e.g. TCPAddress tunnel: WGTunnelSpec # proto-specific, frozen ``` - with `.proto_key` **delegating to `inner.proto_key`** so every + with `.proto_key` **delegating to `overlay.proto_key`** so every existing table lookup (`_addr_to_transport`, `enable_transports` guard at `_root.py:391`, `transport_from_addr()`) keeps working untouched, and - `.unwrap()` delegating to `inner.unwrap()` so **nothing new + `.unwrap()` delegating to `overlay.unwrap()` so **nothing new crosses the wire**. `.namespace` and `.bindspace` come from - the tunnel spec. The wrapper is stripped (`→ .inner`) at the + the tunnel spec. The wrapper is stripped (`→ .overlay`) at the moment of bind/connect. - ⚠️ `is_wrapped_addr()` (`_addr.py:194`) tests `type(addr) in _address_types.values()` — a `bidict` of @@ -154,25 +154,48 @@ Observed protocol-name lists, for writing the `match`: | --- | --- | --- | | bearer | kernel, via `wg-quick`/`pyroute2` | no | | `/wg/u` | nothing — it's an identity | no, verified out-of-band | - | overlay | `tractor`'s `IPCServer` | **yes**, as `.inner` | + | overlay | `tractor`'s `IPCServer` | **yes**, as `.overlay` | This owner-split is the real axis of the design, *not* whether the maddr stack is "composed" (it is). +- ⚠️ **CORRECTION**, an earlier draft of this section specced a + hand-rolled `_peel_tunnel_segs(proto_names) -> (bearer_names, + tunnel_specs, overlay_names)`. **Do not write it.** + `py-multiaddr` already ships the whole tunnel compose/peel API + and it was simply missed here — see its README "En/decapsulate" + and "Tunneling" sections, and gh #443's 2nd bullet which links + them. Verified against the pinned rev: + + | need | API | + | --- | --- | + | isolate the bearer | `ma.decapsulate_code(P_WG)` | + | drop the overlay, keep bearer+key | `ma.decapsulate(overlay_ma)` | + | per-seg maddrs | `ma.split()` | + | rejoin a seg tail | `Multiaddr.join(*segs)` | + | read the key | `ma.value_for_protocol('wg')` | + | recompose | `bearer.encapsulate(key).encapsulate(overlay)` | + + `.decapsulate_code()` handles the infix `/wg/` seg cleanly + *because* it cuts on proto-code and never tries to match an + addr value — the key seg has no addr of its own. This is the + same NIH trap gh #429 existed to close, one layer up. + +- ⚠️ `value_for_protocol('ip4')` on a *full* tunnelled maddr + silently returns the **first** match, i.e. the bearer's host. + Always call it on a peeled sub-maddr, never the whole stack. + - `parse_maddr()` gains a case on - `[('ip4'|'ip6'), 'udp', 'wg', ('ip4'|'ip6'), ]` → - build the inner `Address` from the trailing segments, decode - the multibase key to std-base64, and return - `TunnelledAddress(inner=..., tunnel=WGTunnelSpec(...))` with - the bearer recorded in the spec. + `[('ip4'|'ip6'), 'udp', 'wg', ('ip4'|'ip6'), ]` → + peel w/ the API above, decode the multibase key to std-base64, + and return `TunnelledAddress(overlay=..., tunnel=WGTunnelSpec( + ...))` w/ the bearer recorded in the spec. - keep the existing 2-proto cases byte-identical; add the new case *after* them. -- generalize by **peeling at the tunnel segment**: split - `proto_names` at `'wg'`, hand the trailing list to the existing - inner-stack logic, and recurse for nested tunnels. Write it as - a small pure fn `_peel_tunnel_segs(proto_names) -> - (bearer_names, tunnel_specs, inner_names)`. This is also what - makes a wg-inside-wg stack fall out for free. -- `mk_maddr()` inverse for `TunnelledAddress`. +- nesting (wg-in-wg) falls out of `.decapsulate_code()` cutting + at the *last* occurrence — peel repeatedly rather than + recursing through a bespoke splitter. +- `mk_maddr()` inverse for `TunnelledAddress` is just + `.encapsulate()` composition; don't rebuild `str`s by hand. - **pending an upstream release**: py-multiaddr#108 is merged, so `Multiaddr('/…/wg/u…')` parses — but off a `[tool.uv.sources]` `rev` pin, since no release carries the codec. Gate the tests @@ -217,7 +240,7 @@ side-effect-free; verification is the *caller's* explicit step run. Keep prose in the docs; keep the examples runnable and minimal. - tests: maddr round-trip, `TunnelledAddress` delegation - (`proto_key`/`unwrap` identical to inner), `wrap_address()` + (`proto_key`/`unwrap` identical to overlay), `wrap_address()` regression (a tunnelled maddr `str` → `TunnelledAddress`; a plain one → unchanged), and **a real end-to-end over a locally-created wg pair** gated on `CAP_NET_ADMIN` (see §5.3). @@ -305,7 +328,7 @@ async def open_bindspace( ) -> AsyncGenerator[Address, None]: ''' Enter the net-bindspace implied by `addr`'s tunnel stack, - yielding the *inner* `Address` ready to bind/connect. + yielding the *overlay* `Address` ready to bind/connect. Nests: one `@acm` per tunnel segment, outermost-first, so a 2-deep stack is just two nested `async with`s and the @@ -444,7 +467,7 @@ consider doing it *first* for exactly that reason. | risk | mitigation | | --- | --- | | `to_thread` worker runs in the wrong netns | §5.3; pass `netns=` to pyroute2 or pin a worker; test-first | -| py-multiaddr#108 merged but unreleased | `[tool.uv.sources]` `rev` pin + `_have_wg_maddr_proto()` gate; layer A's inner-addr path works regardless | +| py-multiaddr#108 merged but unreleased | `[tool.uv.sources]` `rev` pin + `_have_wg_maddr_proto()` gate; layer A's overlay-addr path works regardless | | `TunnelledAddress` leaks into `Endpoint` and breaks `inspect.getmodule()` | unwrap at parse/bindspace boundary; assert `not isinstance(ep.addr, TunnelledAddress)` in `Endpoint.__post_init__` | | privileged ops in a library | never `sudo`; explicit cap probe + actionable error; pre-provisioned is the default | | pyroute2 0.9 asyncio core drags a loop into the actor | option (1) is a *thread*, not a loop; forbid `trio-asyncio` here (§4.1) | diff --git a/examples/multihost/wg_lan/README.md b/examples/multihost/wg_lan/README.md index 8bea3344..c8e5ba54 100644 --- a/examples/multihost/wg_lan/README.md +++ b/examples/multihost/wg_lan/README.md @@ -30,7 +30,7 @@ Three parts, three different owners: | --- | --- | --- | | `/ip4/../udp/51820` bearer | kernel via `wg-quick`/`pyroute2` | no | | `/wg/u` | nothing — it's an identity | no, verified out-of-band | -| `/ip4/../tcp/1616` overlay | `tractor`'s `IPCServer` | **yes**, as `.inner` | +| `/ip4/../tcp/1616` overlay | `tractor`'s `IPCServer` | **yes**, as `.overlay` | Verified against py-multiaddr [#108](https://github.com/multiformats/py-multiaddr/pull/108): @@ -55,12 +55,17 @@ release carries the codec. You also need `multibase`: uv pip install multibase ``` -Without the codec `wg_maddr.py` degrades to a plain segment split -— the examples still run, but you lose per-segment validation -(incl. the 32-byte key-length check), so a malformed key reaches -the returned struct instead of raising. `_have_wg_maddr_proto()` -is the gate. It deliberately does **not** hand-roll a `wg` codec -(gh #429 was about *dropping* our NIH parser). +Without the codec `parse_wg_maddr()` raises immediately with an +actionable message — there is deliberately **no** degraded +hand-split fallback. `_have_wg_maddr_proto()` is the predicate. + +Every peel and re-compose here goes through `py-multiaddr`'s own +tunnel API (`.decapsulate_code()`, `.split()`, `.join()`, +`.encapsulate()`, `.value_for_protocol()`) rather than any +bespoke segment slicing — see its README "En/decapsulate" and +"Tunneling" sections. gh #429 was about *dropping* our NIH +parser, and that applies to peeling a tunnel stack just as much +as to decoding one proto. ## 0. tunnel setup (out-of-band, both hosts) @@ -156,7 +161,7 @@ Four corrections, all from setups; if yours needs root, run the script as root rather than embedding `sudo`. 4. **no new `Address` proto-type.** The tunnel rides *beside* the - inner addr in a frozen `WGTunnelledAddr`, and only `.inner` + overlay addr in a frozen `WGTunnelledAddr`, and only `.overlay` crosses into `open_nursery()`. #482 §6 floated a `WGAddress` registered in `_address_types` — that table is a `bidict` (1:1 proto-key↔type) and `_addr_to_transport` wants a @@ -166,6 +171,6 @@ Four corrections, all from `WGTunnelledAddr` is deliberately example-local. Promoting it to `tractor.discovery` as a `TunnelledAddress` whose -`.proto_key`/`.unwrap()` delegate to `.inner`, plus +`.proto_key`/`.unwrap()` delegate to `.overlay`, plus `open_bindspace()` `@acm`s that create/tear down the iface + netns via `pyroute2`, is layers A→C of the plan doc. From 83e4af169caf8d4bf14316946d7493981686ea07 Mon Sep 17 00:00:00 2001 From: goodboy Date: Mon, 31 Aug 2026 12:27:23 -0400 Subject: [PATCH 16/18] Pin tpt prompt provenance to immutable refs Replace mutable branch pointers with the exact source commit and historical base-to-commit range. Normalize the substantive scope while leaving the recorded raw response unchanged. Prompt-IO: ai/prompt-io/claude/20260813T001102Z_27c34aeb_prompt_io.md (this patch was generated in some part by `opencode` using `gpt-5.6-sol` (`openai`)) --- .../claude/20260813T001102Z_27c34aeb_prompt_io.md | 4 ++-- .../claude/20260813T001102Z_27c34aeb_prompt_io.raw.md | 8 ++++---- 2 files changed, 6 insertions(+), 6 deletions(-) diff --git a/ai/prompt-io/claude/20260813T001102Z_27c34aeb_prompt_io.md b/ai/prompt-io/claude/20260813T001102Z_27c34aeb_prompt_io.md index ee919337..01c3a492 100644 --- a/ai/prompt-io/claude/20260813T001102Z_27c34aeb_prompt_io.md +++ b/ai/prompt-io/claude/20260813T001102Z_27c34aeb_prompt_io.md @@ -3,8 +3,8 @@ model: claude-opus-5 service: claude session: 7b9c97c4-fff7-4ac4-97fb-35720453308e timestamp: 2026-08-13T00:11:02Z -git_ref: 27c34aeb -scope: docs+code +git_ref: 27c34aebb615c30d4039fa399f4ce2766ed7ba2c +scope: code substantive: true raw_file: 20260813T001102Z_27c34aeb_prompt_io.raw.md --- diff --git a/ai/prompt-io/claude/20260813T001102Z_27c34aeb_prompt_io.raw.md b/ai/prompt-io/claude/20260813T001102Z_27c34aeb_prompt_io.raw.md index 0e082229..3cd7918c 100644 --- a/ai/prompt-io/claude/20260813T001102Z_27c34aeb_prompt_io.raw.md +++ b/ai/prompt-io/claude/20260813T001102Z_27c34aeb_prompt_io.raw.md @@ -2,15 +2,15 @@ model: claude-opus-5 service: claude timestamp: 2026-08-13T00:11:02Z -git_ref: 27c34aeb -diff_cmd: git diff main..ng_tpts_planning +git_ref: 27c34aebb615c30d4039fa399f4ce2766ed7ba2c +diff_cmd: git diff 3ad7e7e5dc28aa336499725a03744fc10c6c0026..27c34aebb615c30d4039fa399f4ce2766ed7ba2c --- # Raw output — next-gen tpt-backend implementation plans ## Generated planning docs -> `git diff main..ng_tpts_planning -- ai/tpt-backends/` +> `git diff 3ad7e7e5dc28aa336499725a03744fc10c6c0026..27c34aebb615c30d4039fa399f4ce2766ed7ba2c -- ai/tpt-backends/` Five markdown docs. `00_shared_backend_contract.md` is normative and the other three are written against it so they can be worked @@ -43,7 +43,7 @@ independently: ## Generated example code -> `git diff main..ng_tpts_planning -- examples/multihost/wg_lan/` +> `git diff 3ad7e7e5dc28aa336499725a03744fc10c6c0026..27c34aebb615c30d4039fa399f4ce2766ed7ba2c -- examples/multihost/wg_lan/` - `wg_maddr.py` — `WGTunnelledAddr(msgspec.Struct, frozen=True)` carrying `bearer: tuple[str, int]`, `peer_pubkey: str`, From 7c2de6c359d724ee651f91fa79156ea5e6946428 Mon Sep 17 00:00:00 2001 From: goodboy Date: Mon, 31 Aug 2026 12:27:43 -0400 Subject: [PATCH 17/18] Reconcile tpt plans with runtime contracts Bring the shared backend contract and TIPC plan in line with current runtime behavior and downstream implementation evidence. Rework the QUIC plan around actor-owned endpoint, bootstrap, stream, listener and cleanup lifecycles, with explicit validation gates for the still-unverified UniFFI details. (this patch was generated in some part by `opencode` using `gpt-5.6-sol` (`openai`)) --- ai/tpt-backends/00_shared_backend_contract.md | 135 ++-- ai/tpt-backends/01_tipc_backend.md | 434 ++++++----- ai/tpt-backends/02_quic_iroh_backend.md | 677 +++++++++++------- ai/tpt-backends/README.md | 16 +- 4 files changed, 780 insertions(+), 482 deletions(-) diff --git a/ai/tpt-backends/00_shared_backend_contract.md b/ai/tpt-backends/00_shared_backend_contract.md index f46873a2..e6bb6b30 100644 --- a/ai/tpt-backends/00_shared_backend_contract.md +++ b/ai/tpt-backends/00_shared_backend_contract.md @@ -33,19 +33,35 @@ doc in the same PR. ## 1. The backend duck-type (empirical, from `_tcp.py`/`_uds.py`) -A transport backend is **one module** under `tractor/ipc/` -exposing exactly four things. There is no ABC to subclass and no -plugin entrypoint; wiring is by explicit table registration -(§2) plus one piece of reflection (§1.3). +A transport backend is **one module** under `tractor/ipc/`. +There is no ABC to subclass and no plugin entrypoint; wiring is by +explicit table registration (§2) plus one piece of reflection +(§1.3). + +Keep two contracts distinct: + +- `tractor.discovery._addr.Address` is a static `Protocol`. It + declares address-wrapper members including `namespace`, + `open_listener()` and `close_listener()`. +- the runtime's empirical contract is what `_tcp.py`, `_uds.py` + and `_server.py` actually call. The current address classes do + not implement every declared `Address` member: listener + lifecycle is module-level, `def_bindspace` is used despite not + being declared by the `Protocol`, and `namespace` remains + aspirational. + +Until those surfaces are deliberately reconciled, implement the +empirical module contract below and update the static `Protocol` +only when the runtime really consumes the new member. Do not claim +that structural conformance alone defines a backend. ### 1.1 `class Address(msgspec.Struct, frozen=True)` -Structurally conforms to the `Address` `Protocol` in -`tractor/discovery/_addr.py:82`. Required surface: +The runtime-consumed address-wrapper surface is: | member | kind | notes | | --- | --- | --- | -| `proto_key` | `ClassVar[str]` | the wire/registry key, e.g. `'tcp'`, `'uds'` | +| `proto_key` | `ClassVar[str]` | internal transport key, e.g. `'tcp'`, `'uds'` | | `unwrapped_type` | `ClassVar[type]` | the primitive tuple shape | | `def_bindspace` | `ClassVar` | default bindspace value | | `is_valid` | `@property -> bool` | "is this a *dialable/bindable* addr" | @@ -81,22 +97,32 @@ Hard constraints learned from the existing two: paper over it at best. **The fix, and the recommended prerequisite for all three - backends: make the unwrapped form carry an explicit - proto-key, using the `multiaddr` protocol name as the - canonical spelling** — `('tcp', host, port)`, - `('unix', path)`, `('udp', ...)`, `('tipc', stype, inst, - scope)`. Then `wrap_address()` collapses from an - order-sensitive `match` to `_address_types[addr[0]]`, and the - whole collision class stops existing. Note this *also* aligns - the on-wire form with `mk_maddr()`/`parse_maddr()`, so the two - representations stop being independent inventions. + backends: make the unwrapped form carry an explicit internal + proto-key** — `('tcp', host, port)`, + `('uds', filedir, filename)`, `('tipc', stype, inst, scope)`. + The tag must be a `TransportProtocolKey`/registry key. In + particular it is **`'uds'`, not the external multiaddr spelling + `'unix'`**. If a wire or display format uses a different name, + name that translation explicitly; today `_multiaddr.py` maps + internal `uds` to external `/unix/`. Then `wrap_address()` can + dispatch through `_address_types[addr[0]]` without an + order-sensitive shape match. Two consequences to plan for: - - it's a **wire-format change** (`SpawnSpec`, - `_root_mailbox`, `_registry_addrs`) plus every test fixture - and downstream config (`piker`'s `[network]` table). It - wants its **own migration commit, landed before any new - backend**, not smuggled into one. + - it's a **wire-format change**. Widen and keep synchronized + `discovery._addr.UnwrappedAddress` and the duplicate wire + alias in `msg.types`; change `SpawnSpec.reg_addrs` and + `.bind_addrs`, not only `_root_mailbox` and + `_registry_addrs`. Audit the related `RuntimeVars` + `_root_mailbox`/`_root_addrs` annotations, `Actor.reg_addrs` + and accept-address annotations, channel/spawn signatures, + fixtures, and downstream config (`piker`'s `[network]` + table). `msgspec` rejects a union containing multiple + array-like tuple shapes, so #493 used + `tuple[str|int, ...]` as the truthful transitional wire type; + the complete proto-key migration can restore per-proto + validation. This wants its **own migration commit, landed + before any new backend**, not smuggled into one. - it's the moment to **stop handing raw unwrapped tuples to users at all.** The long-term shape is: `Address` subtypes are the public currency and `UnwrappedAddress` becomes an @@ -104,8 +130,8 @@ Hard constraints learned from the existing two: `ipaddress` uses (you pass `IPv4Address`, not a 4-tuple). Public API should accept `Address|maddr-str` and treat bare tuples as legacy-tolerated input, ideally deprecated. -- **`.get_random()` must be collision-free without a live - runtime.** See the `UDSAddress.get_random()` uuid-token +- **`.get_random()` must not deterministically alias without a + live runtime.** See the `UDSAddress.get_random()` uuid-token comment (`_uds.py:207-220`): with no `current_actor()` the sockname degenerates to a pure fn of `(prefix, pid)` and two calls in one proc alias. Mix in a `uuid4().hex[:8]` token. @@ -162,10 +188,11 @@ if (unwrapped := lstnr.socket.getsockname()) != self.addr.unwrap(): ``` i.e. it assumes `lstnr.socket.getsockname()` exists and that its -return value is a valid `from_addr()` input. This is fine for -TIPC (§3 of plan 01) and **is the main integration hazard for -iroh** (§3 of plan 02) — plans that break it must say so -explicitly and propose the upstream `_server.py` patch. +return value is a valid `from_addr()` input. That is false for +TIPC, whose listener sockname is an undialable port ID, and for +non-socket iroh. Both plans must use the explicit backend rebind +policy added at this integration point rather than pretending a +sockname is always an address replacement. ### 1.4 `class MsgpackStream(MsgpackTransport)` @@ -224,20 +251,26 @@ path.** This is why plan 01 is small and plan 02 is not. --- -## 2. Registration tables (the full wiring checklist) +## 2. Registration and policy wiring -Adding a backend touches these and only these: +Adding a backend requires this complete audit. Not every item +changes for every backend, but none may be assumed from the others: 1. `tractor/runtime/_state.py:46` `TransportProtocolKey = Literal['tcp', 'uds', ...]` — add the - key. This `Literal` is the canonical set; `_testing/pytest.py` - drives `--tpt-proto` validation off `_addr._address_types`, - and the spawn-backend fixture already models the - "drive-the-set-from-the-Literal" pattern - (`pytest.py:870-880`) — do the same rather than hardcoding. -2. `tractor/discovery/_addr.py:173` `_address_types: bidict` — - `{'': Address}`. Note it is a **`bidict`**, so - the mapping must stay 1:1. + internal key. This `Literal` is the **declared protocol-key + set**, not proof that a backend is usable on this host. +2. `tractor/discovery/_addr.py` `_address_protos` and + `_address_types: dict[str, Type[Address]]` — register + `'': Address`. `_address_types` is a plain + **`dict`, not a `bidict`**, and represents the backends this + build registers for import and dispatch. UDS is conditional on + `HAS_UDS`, while TIPC can remain registered on a host where its + kernel support is unavailable. An importable backend with a + runtime capability requirement therefore needs a separate + availability check. Never conflate this dispatch registry with + either host usability or the declared `TransportProtocolKey` + universe. 3. `tractor/discovery/_addr.py:181` `_default_lo_addrs` — `'': Address.get_root().unwrap()`. ⚠️ this dict is built at **import time**, so @@ -250,7 +283,7 @@ Adding a backend touches these and only these: add a case iff your `unwrapped_type` isn't already uniquely matched. **Preferably do the proto-key migration in §1.1 first**, after which this step becomes a one-line - `_address_types` entry instead of an order-sensitive `case`. + `_address_types` lookup instead of an order-sensitive `case`. 5. `tractor/ipc/_types.py` — `Address` union alias, `_msg_transports` list, `_key_to_transport[('msgpack', key)]`, `_addr_to_transport[Address]`. @@ -262,9 +295,17 @@ Adding a backend touches these and only these: `parse_maddr()`. 8. `tractor/ipc/__init__.py` — re-export if the backend has a public surface. -9. `tractor/_testing/addr.py::get_rando_addr()` — per-proto - branch so the whole suite can run under `--tpt-proto `. -10. `pyproject.toml` — new deps go in an **optional extra**, never +9. `tractor/discovery/_api.py::_is_local_addr()` and + `prefer_addr()` — define and test the backend's locality and + selection tier. The current order is UDS, local TCP, then + remote. A new backend must not silently fall into `remote` by + accident: for example TIPC node scope is local, cluster scope + is not known-local, and an observed address with unknown scope + must not be promoted. Preserve the last-registered tie-break + unless intentionally changing policy. +10. `tractor/_testing/addr.py::get_rando_addr()` — per-proto + branch so the whole suite can run under `--tpt-proto `. +11. `pyproject.toml` — new deps go in an **optional extra**, never in `[project].dependencies`. See §5. ## 3. Where the `trio.SocketListener` assumption is load-bearing @@ -339,7 +380,10 @@ dep-free, or make that table lazy. `_state._def_tpt_proto` + `_runtime_vars['_enable_tpts']` (`pytest.py:807-835`). Adding the key to `_address_types` is what makes `--tpt-proto ` legal (`pytest.py:795-800` - asserts the lookup). + asserts the lookup). Thus CLI acceptance follows the + build-registered `_address_types`, while type-level declarations + follow `TransportProtocolKey` and host usability follows each + backend's capability probe; test all three layers separately. - The **acceptance bar** for every backend is: the *entire* existing suite passes under `--tpt-proto `, unmodified. That is the whole point of the abstraction. Backend-specific @@ -353,8 +397,11 @@ dep-free, or make that table lazy. `OSError(97, 'Address family not supported by protocol')` because the `tipc` module isn't loaded. Put the predicate in the backend module (so apps can use it too), not in the test. -- New pytest marks must be registered in `pyproject.toml`, per - the project's fix-warnings-at-source rule (gh #469). +- New pytest marks must be registered in + `_testing/pytest.py::pytest_configure()` with + `config.addinivalue_line()`, alongside the existing custom + marks. The repo has no `pyproject.toml` marker table. This is + still part of the fix-warnings-at-source rule (gh #469). ## 7. Code style (non-negotiable, matches the repo) diff --git a/ai/tpt-backends/01_tipc_backend.md b/ai/tpt-backends/01_tipc_backend.md index ccbdf3be..0e43aef9 100644 --- a/ai/tpt-backends/01_tipc_backend.md +++ b/ai/tpt-backends/01_tipc_backend.md @@ -4,13 +4,18 @@ Tracks gh [#378]. Prereq reading: [`00_shared_backend_contract.md`](./00_shared_backend_contract.md). **Thesis**: TIPC is the *cheapest* new backend we can add and -simultaneously the only one that gives us cluster-wide service -discovery **for free, in the kernel**, replacing (for -TIPC-capable deployments) the whole `tractor.discovery` -registrar round-trip with a `bind()`/`connect()` on a -*service name*. It is stdlib-only: zero new dependencies. +gives us kernel-native service-name publication, known-address +dialling, and topology events. Those are primitives for reducing +registrar traffic; they do **not** by themselves replace +`tractor.discovery`, derive an actor's address from its name, or +elect one registrar. It is stdlib-only: zero new dependencies. + +This plan is reconciled against downstream PR [#493]'s code and +tests. Treat that implementation as prior art without mistaking +implemented transport primitives for completed discovery policy. [#378]: https://github.com/goodboy/tractor/issues/378 +[#493]: https://github.com/goodboy/tractor/pull/493 --- @@ -74,10 +79,12 @@ The design decision that makes this backend coherent: > ever an *observed* address (`getpeername()`), never a > user-facing one.** -This is exactly the "leverage the built-in discovery machinery" -ask in #378: publishing a bind *is* registration, and -`connect()` on a name *is* a lookup, with no registrar actor in -the loop. +This is the "leverage the built-in discovery machinery" part of +#378: publishing a bind is kernel name-table registration and +`connect()` on an already-known name is a kernel lookup, with no +registrar actor on that **dial** path. Mapping an application name +to that address and maintaining Tractor's actor registry remain +separate work (§5). ### 2.2 the struct @@ -89,12 +96,12 @@ class TIPCAddress( _stype: int # TIPC "type" == service class _instance: int # service instance within the type _scope: int = TIPC_CLUSTER_SCOPE - # observed-only, never part of identity/equality-by-intent + # observed-only, excluded from the unwrapped service identity maybe_node: int|None = None # from TIPC_ADDR_ID getpeername() maybe_ref: int|None = None proto_key: ClassVar[str] = 'tipc' - unwrapped_type: ClassVar[type] = tuple[str, int] + unwrapped_type: ClassVar[type] = tuple[str, int, int, int] def_bindspace: ClassVar[int] = TIPC_CLUSTER_SCOPE ``` @@ -106,8 +113,8 @@ shape as `TCPAddress`*, so `wrap_address()`'s `case (str(), int())` steals it. This backend is therefore the forcing function for the contract-doc's conclusion (§1.1): -> **make the unwrapped form carry an explicit proto-key, spelled -> with the `multiaddr` protocol name.** +> **make the unwrapped form carry the explicit internal +> `TransportProtocolKey`.** ```python def unwrap(self) -> tuple[str, int, int, int]: @@ -115,11 +122,31 @@ def unwrap(self) -> tuple[str, int, int, int]: ``` `wrap_address()` then dispatches `_address_types[addr[0]]` and -the collision class disappears. **This is a prerequisite -migration commit, not part of this backend** — see contract §1.1 -for its blast radius (wire format + every fixture + `piker` -config) and for the follow-on "stop handing raw tuples to users -at all, à la `ipaddress`" direction. +the collision class disappears. The complete all-backend change +is a prerequisite migration; #493 necessarily carried the +transitional `UnwrappedAddress`/`SpawnSpec.reg_addrs`/ +`.bind_addrs` widening needed for TIPC. See contract §1.1 for the +remaining runtime annotations, fixtures and `piker` config. Here +`tipc` is both the internal and external spelling; UDS remains +internally `uds` and translates explicitly to external `/unix/`. + +`msgpack` decodes tuples as lists, so both forms are part of the +round-trip contract. Match only the exact three- or four-element +tagged shapes and test all four routes: + +```python +case ( + ('tipc', int() as stype, int() as inst, int() as scope) + | + ['tipc', int() as stype, int() as inst, int() as scope] +): + ... +``` + +Also test the scope-defaulted three-element form through +`TIPCAddress.from_addr()`, and tuple/list forms through the global +`wrap_address()`. A normal two-element TCP/UDS address whose first +element happens to be `'tipc'` must retain its classic dispatch. ⚠️ an earlier revision of this plan proposed a self-tagging `('tipc::', instance)` string-prefix hack with an @@ -150,25 +177,28 @@ treatment (`_uds.py:242`). - `_instance` for `get_random()`: TIPC gives us no kernel-assigned-instance analogue of `port=0`, so we must choose. Use a *pure* fn of the actor identity so it is - reproducible and collision-free: + reproducible and well-distributed, **not collision-free**: ```python - # 32-bit instance derived from the actor's uuid4 (+ pid when - # there's no live runtime, per the UDS precedent). + # 32-bit instance derived from the actor's Aid.uid, or from a + # per-call token + pid when there is no live runtime. inst: int = int.from_bytes( blake2b(seed.encode(), digest_size=4).digest(), 'big', ) ``` - where `seed = f'{actor.aid.name}@{pid}'` if + where `seed = '.'.join(actor.aid.uid)` if `current_actor(err_on_no_runtime=False)` else `f'{prefix}.{uuid4().hex[:8]}@{pid}'`. Must avoid the reserved low range: `inst = 64 + (inst % (2**32 - 64))`. + The UUID is load-bearing because TIPC names are cluster-wide + while PIDs are only host-local: `(actor name, pid)` can alias on + different hosts. ⚠️ *unlike* `port=0`, a collision here surfaces as a successful-but-shared publication (TIPC allows multiple binders on the same name and round-robins!) rather than `EADDRINUSE`. That is a silent-crosstalk failure mode; §7 has - the test that proves the 4-byte digest is enough and §9 has - the mitigation if it isn't. + a statistical test and §9 records the unresolved recovery work + in [#501]. - `_scope`: `TIPC_NODE_SCOPE` for a same-host-only actor (the UDS-equivalent), `TIPC_CLUSTER_SCOPE` (default) for cluster-visible. **This is `.bindspace`**: @@ -189,7 +219,9 @@ treatment (`_uds.py:242`). @property def is_valid(self) -> bool: return ( - self._instance != 0 + self._instance > 0 + and + self._stype > 0 and self._stype not in _tipc_reserved_stypes # {0, 1, ...} and @@ -236,10 +268,11 @@ Notes / hazards: - **no `close_listener()` needed** — nothing to unlink. Omit the function entirely (contract §1.2: absence means implicit). Withdrawal of the published name happens on socket close. -- ⚠️ `SocketListener.__init__` will try - `getsockopt(SOL_SOCKET, SO_ACCEPTCONN)`. If TIPC rejects it, - trio's `except OSError: pass` covers us. Assert this in a - unit test rather than assuming. +- `SocketListener.__init__` calls + `getsockopt(SOL_SOCKET, SO_ACCEPTCONN)`. The live-kernel probe + used by #493 answers `1`; retain the unit test so a kernel-side + change is visible rather than relying on trio's suppressed- + `OSError` carve-out. - Wrap the bind in a `_reraise_as_connerr()`-style `@cm` (copy the `_uds.py:256` pattern) so `EADDRINUSE`-ish and `EAFNOSUPPORT` become `ConnectionError` with the addr in the @@ -258,33 +291,31 @@ returns a `TIPC_ADDR_ID`-flavoured 5-tuple (the port id), *not* the name-seq we bound. So the `!=` is **always true** and `from_addr()` will be handed a 5-tuple. -Handle it inside `TIPCAddress.from_addr()` — do **not** patch -`_server.py`: +`TIPCAddress.from_addr()` must accept only proto-keyed service +names. It must reject a bare port ID because no conversion can +recover `(stype, instance)`: ```python @classmethod def from_addr(cls, addr) -> TIPCAddress: match addr: - # our own unwrapped form - case (str() as tag, int() as inst) if tag.startswith('tipc:'): - _, stype, scope = tag.split(':') - return TIPCAddress(int(stype), inst, int(scope)) + # our proto-keyed tuple or decoded-list wire form + case ( + ('tipc', int() as stype, int() as inst, int() as scope) + | + ['tipc', int() as stype, int() as inst, int() as scope] + ): + return TIPCAddress(stype, inst, _norm_scope(scope)) - # a kernel-observed TIPC_ADDR_ID 5-tuple: keep the - # *service* identity we already know and only annotate - # the observed port-id. + # a bare kernel-observed TIPC_ADDR_ID 5-tuple has no + # service identity to annotate. case (int() as atype, *rest) if atype == socket.TIPC_ADDR_ID: - ... + raise ValueError(...) ``` -The `TIPC_ADDR_ID` case cannot reconstruct `(stype, instance)` -— that info isn't in a port id. So `from_addr()` alone is -insufficient for the reconciliation path. **Resolution**: make -`from_addr()` raise a clear `ValueError` for the bare -`TIPC_ADDR_ID` case, and instead prevent the reconciliation -from firing by having `start_listener()` return a listener -whose `getsockname()` we never need — i.e. land this two-line -upstream fix in `_server.py:664`: +The `TIPC_ADDR_ID` case cannot reconstruct `(stype, instance)`. +The resolution is the explicit listener-rebind policy added ahead +of the backend in #493: ```python if ( @@ -300,12 +331,13 @@ behaviour exactly). Rationale: the reconciliation exists *only* to learn the kernel-assigned port for `port=0` TCP binds (its own comment says so, `_server.py:662`); TIPC has no such late-binding, so opting out is semantically right rather than a -hack. **Land this as its own commit, ahead of the backend**, -with a test that `tcp`'s `port=0` behaviour is unchanged. +hack. Keep the guard test that TCP's `port=0` behaviour is +unchanged. -Keep the observed port-id available anyway: annotate -`ep.addr = ep.addr.with_port_id(*getsockname()[1:3])` (a pure -`msgspec.structs.replace()` helper) purely for logging/repr. +Do **not** annotate `Endpoint.addr` from `getsockname()`: the +listener endpoint must remain the dialable service name. Port IDs +are observed only on connected streams and may annotate a copy via +`with_port_id()` purely for logging/repr. ### 3.3 `MsgpackTIPCStream` @@ -340,11 +372,12 @@ class MsgpackTIPCStream(MsgpackTransport): 0, # domain: 0 == "anywhere in scope" destaddr._scope, )) - return cls( - trio.SocketStream(sock), - prefix_size=prefix_size, - codec=codec, - ) + stream = trio.SocketStream(sock) + return cls( + stream, + prefix_size=prefix_size, + codec=codec, + ) ``` - reuse `trio._highlevel_open_unix_stream.close_on_error` (the @@ -363,11 +396,11 @@ class MsgpackTIPCStream(MsgpackTransport): leave at default, we have `trio` cancel scopes. - `TIPC_DEST_DROPPABLE = 0` on the connection so undeliverable msgs come back as errors rather than being silently dropped. -- **`connect_to()` on a name with no publisher**: TIPC returns - `ECONNREFUSED`/`EHOSTUNREACH` promptly (no SYN-timeout wait), - which is *better* discovery-ping behaviour than TCP. Confirm - the errno and make sure it surfaces as `ConnectionError` - (contract §4 — the registrar ping path depends on it). +- **`connect_to()` on a name with no publisher**: the live-kernel + result is immediate `EHOSTUNREACH`. Python exposes that as a + bare `OSError`, not a `ConnectionError` subtype, so + `_reraise_as_connerr()` is load-bearing for contract §4. Keep + the exact errno and normalization under test. ### 3.4 `get_stream_addrs()` @@ -385,29 +418,27 @@ Problem: neither end's port-id tells us the *service name*. The `laddr`/`raddr` are used for logging, `Channel.raddr`, `Server._peers` keying-adjacent repr, and `maddr`. Design: -- the **connecting** side knows the destaddr it dialled → - `connect_to()` overrides `_raddr` after construction with the - known-good `TIPCAddress`, exactly as - `MsgpackUDSStream.connect_to()` does for the peer-pid case - (`_uds.py:539-543`). -- the **accepting** side does not know the peer's service name - from the socket. Two honest options: - - **(a) accept it: `raddr` carries only `(node, ref)`** via - `maybe_node`/`maybe_ref`, `_stype/_instance` set to a - sentinel `-1`, and `__repr__` renders - `TIPCAddress[:]`. The `Aid` from the - handshake already gives us the peer's logical identity, so - nothing in the runtime actually *needs* the peer's service - name. **Recommended.** - - (b) piggyback the peer's own bound name in the handshake. - Rejected for this PR: touches `Aid`/msg-spec. -- `laddr` on the accepting side: the `Endpoint` knows its own - `addr`; but `get_stream_addrs()` is a `@classmethod` with only - the stream. Use `TIPC_ADDR_ID` for `laddr` too and let - `Endpoint.peer_tpts` keying (which is by *peer* addr) still - work. Verify nothing asserts `laddr == ep.addr` — grep for - `.laddr` uses before committing (`_server.py`'s - `con_status` logging, `Channel.pformat()`). +- `get_stream_addrs()` converts both socket results into + **observed-only** addresses: `_stype`/`_instance` use the + `TIPC_NAME_UNKNOWN = -1` sentinel and `maybe_node`/`maybe_ref` + carry the port ID. Such addresses are invalid for dialling. +- the **connecting** side knows the service name it dialled, so + `connect_to()` replaces `_raddr` after construction with that + known `TIPCAddress` while retaining the constructor's one + tolerant port-ID observation. Do not call `getpeername()` a + second time: the peer can withdraw between the two calls. +- the **accepting** side genuinely cannot recover the peer's + service name from a port ID. Keep the observed-only `raddr`; + the handshake's `Aid` supplies logical identity. Piggybacking a + bound name in the handshake is outside this backend. +- `laddr` is observed-only as well. It is used for repr/logging, + not to replace the endpoint's known service name. +- unlike TCP/UDS, TIPC can answer `ENOTCONN` from + `getpeername()` after a connect-then-drop. This lookup happens + during `MsgpackTransport` construction, before handshake error + tolerance. Wrap `getsockname()` and `getpeername()` in a + tolerant helper and degrade to a port-ID-less observed address; + a dropped peer must cost an observation, not kill the actor. --- @@ -444,43 +475,47 @@ the maddr stays 2-segment like `/unix/...`. --- -## 5. Discovery: the actually-interesting part +## 5. Discovery primitives and explicit limits -Two independently-shippable layers. **Layer A is in scope for -the first PR; layer B is a fast-follow.** +The backend provides independently-shippable kernel primitives. +Neither primitive alone implements Tractor's actor-name discovery, +registry ownership, or registrar election. ### 5.1 Layer A — "discovery by bind" (free) Because `bind(TIPC_ADDR_NAMESEQ)` publishes and -`connect(TIPC_ADDR_NAME)` resolves, a `tractor` tree whose -`registry_addrs` are TIPC service names needs **no registrar -liveness at all** for the connect path: `find_actor()`'s -"connect to the registrar and ask" becomes "connect to the -service name directly". Concretely: +`connect(TIPC_ADDR_NAME)` resolves, a caller that **already knows** +a TIPC service address can dial it without a registrar lookup. +This is narrower than registrar-less `find_actor(name)`: -- `tractor.discovery._api.find_actor()` etc. keep working - unchanged (they go through the registrar), *and* -- a new, TIPC-only fast path becomes possible: derive an actor's - service name from its `(name, uuid)` and dial it without any - registrar hop. +- `tractor.discovery._api.find_actor()` and peers still query a + registrar; #493 does not change them. +- deriving a stable service address from `(name, uuid)` and + dialling it directly is follow-up [#499]. The mapping must be + documented and cross-language stable. +- `registry_addrs` still identify registrars. Connecting to a + known registrar by TIPC name removes no registrar bookkeeping + or ownership semantics. -Do **not** build the fast path in PR 1. Instead, prove the -property with a test (§7.4) and file the follow-up: it changes -`discovery` semantics (name→instance derivation must be a -documented, stable, cross-language-able hash) and deserves its -own design. +There is also an unresolved **split-brain election** problem. +Duplicate TIPC name publication succeeds and round-robins, so two +roots can both probe an unoccupied registrar name, both bind it, +and both believe they won. The backend provides no atomic +compare-and-publish, lease, quorum, or deterministic winner. A +topology subscription can reveal multiple publisher port IDs but +does not elect or fence one. Do not describe registrar election as +solved until a separate protocol closes this race. ### 5.2 Layer B — the topology service (`TIPC_TOP_SRV`) -This is what makes #378's "end game cluster proto" claim real: -a *subscription* to name-table events, i.e. push-based -`register`/`deregister` for free, replacing the registrar's -polled `find_actor()`. +This is the push primitive behind #378's "end game cluster proto" +direction: a subscription to kernel name-table publish/withdraw +events. #493 implements `open_topology_events()`; consuming that +feed in `discovery._registry` is follow-up [#496]. Until then it +does not replace registrar state or `find_actor()`. -Mechanics (verify each field against -`linux/include/uapi/linux/tipc.h` + `net/tipc/topsrv.c` at -implementation time — the struct layout below is from the uapi -header and the byte-order caveat is real): +Mechanics, verified against `linux/include/uapi/linux/tipc.h`, +`net/tipc/topsrv.c` and #493's live-kernel probe: ```python # SOCK_SEQPACKET connected to the topology server @@ -498,22 +533,25 @@ await sock.connect(( # __u32 filter; /* TIPC_SUB_{PORTS,SERVICE,CANCEL} */ # char usr_handle[8]; # } /* == 28 bytes */ -_SUBSCR_FMT: str = '=IIIII8s' # ⚠ 5*I is 20 -> use '=5I8s' +_SUBSCR_FMT: str = '=5I8s' ``` -- **byte order**: the topology server historically accepts both - host and swapped order and auto-detects; modern kernels are - strict-ish. Pack native (`'='`) first, and if the server - closes the connection immediately, retry with `'>'`. Encode - that as a one-time probe helper - `_detect_topsrv_endianness()` cached at module level — and - put a `# ?TODO` pointing at `net/tipc/topsrv.c` for someone - to make it deterministic. +- **byte order**: #493's live-kernel probe verified native + standard-size (`'='`) packing for publish and withdraw events. + Use `'=5I8s'` for the 28-byte subscription. Do not retain the + speculative `'>'` retry/probe as if it were required. Preserve + the earlier `# ?TODO` to verify the deterministic rule directly + against `net/tipc/topsrv.c`; it is source-audit work, not a + runtime retry requirement. - **events**: `struct tipc_event` is `event: u32`, `found_lower: u32`, `found_upper: u32`, `port: {ref: u32, node: u32}`, then the 28-byte subscription - echo → 40 bytes. `event ∈ {TIPC_PUBLISHED, TIPC_WITHDRAWN, - TIPC_SUBSCR_TIMEOUT}`. + echo: **48 bytes** (`4 + 4 + 4 + 8 + 28`), not 40. Use + `'=10I8s'` and assert `struct.calcsize(...) == 48`. + `event ∈ {TIPC_PUBLISHED, TIPC_WITHDRAWN, + TIPC_SUBSCR_TIMEOUT}`. Python exposes `TIPC_WAIT_FOREVER` as + `-1`, so mask it with `& 0xFFFF_FFFF` before packing an + unsigned `I`. - **trio shape** — this is where the "nearly-functional, modern-async" style pays off; expose it as an `@acm` yielding a `trio` receive-channel of typed events, *not* a class: @@ -538,14 +576,22 @@ async def open_topology_events( `kind: Literal['published','withdrawn','timeout']`, `addr: TIPCAddress`, `node: int`, `ref: int`. One `trio.lowlevel`-free implementation: a nursery-spawned reader - task doing `await sock.recv(40)` in a loop and - `send_nowait()`ing decoded events, with the `@acm` closing the - socket on exit → reader gets `ClosedResourceError` → cancel - scope collapses. Standard `tractor` `@acm` discipline. -- **consumer**: `tractor/discovery/_registry.py` gains an - optional "watch" mode so a registrar (or any actor) can keep - a live view of the actor set without polling. Sketch the - integration in the follow-up issue; do not wire it in PR 1. + task doing `await sock.recv(48)` in a loop. The feed is + authoritative and may neither block the socket reader nor drop + transitions silently. Use `send_nowait()` and, on + `trio.WouldBlock`, raise a dedicated + `TIPCNameEventOverflow` that aborts the subscription and tells + the consumer to resubscribe and rebuild its view. A timeout + event is delivered once and then closes the channel. The + `@acm` cancels its reader before closing the fd so teardown + cannot race a retried `recv()` into `EBADF`. +- **scope**: topology events carry no publication scope. Use an + explicit unknown-scope sentinel and keep the resulting address + non-dialable; never copy caller/subscription context into + supposedly observed data. +- **consumer**: [#496] owns the optional watch mode and the + decision whether the feed subsumes or merely accelerates + existing registrar bookkeeping. - **`SOCK_SEQPACKET` is fine here** because this socket never goes through `MsgpackTransport` — it's a plain trio socket used with `recv()`. The contract's "`SOCK_STREAM` only" @@ -561,14 +607,16 @@ async def open_topology_events( 2. `tractor/ipc/_tipc.py`: `TIPCAddress` + `is_tipc_available()` predicate + `start_listener()`. No transport yet. Tests: address round-trip (`unwrap`/`from_addr`/`wrap_address`), - `get_random()` uniqueness, bind/listen + `SO_ACCEPTCONN` + `get_random()` distribution, bind/listen + `SO_ACCEPTCONN` tolerance, `EAFNOSUPPORT` → actionable `ConnectionError`. 3. `MsgpackTIPCStream` + `connect_to()` + `get_stream_addrs()`. Test: two `trio` tasks in one proc exchange a msg over `Msgpack` framing (no `tractor` runtime). -4. registration tables (contract §2 items 1-6, 9) + +4. registration tables (contract §2 items 1-8 and 10) + `pyproject.toml` mark/extra. Test: full suite under - `--tpt-proto tipc` (§7.3). + `--tpt-proto tipc` (§7.3). Keep TIPC in the conservative remote + preference tier until a follow-up implements and tests contract + item 9's node-scope locality policy. 5. maddr support (`str` form + prefix special-case) + docs. 6. `open_topology_events()` @acm + its tests (layer B). 7. docs page + `docs/` example. @@ -589,6 +637,8 @@ def is_tipc_available() -> bool: the `tipc` module is loaded. ''' + if sys.platform != 'linux': + return False try: socket.socket(socket.AF_TIPC, socket.SOCK_STREAM).close() return True @@ -596,17 +646,22 @@ def is_tipc_available() -> bool: return False ``` -Cache it in a module global (it can't change without a -`modprobe`, and a cold call costs a syscall). Pure predicate, no -side effects, no logging. +Do not permanently memoize the result: `modprobe tipc` and module +removal can change it during a long-lived process. Probe once per +runtime startup, or use an explicitly refreshable cache whose +owner invalidates it after module-management operations. The +predicate itself remains side-effect-free and silent. ### 7.2 gating -- `pytest.mark.tipc` registered in `pyproject.toml`. -- module-level - `pytestmark = pytest.mark.skipif(not is_tipc_available(), - reason='`tipc` kernel module not loaded (`modprobe tipc`)')` - in `tests/ipc/test_tipc.py`. +- `pytest.mark.tipc` registered in + `_testing/pytest.py::pytest_configure()` via + `config.addinivalue_line()`, where this repo declares its other + custom marks. Do not invent a `pyproject.toml` marker table. +- keep pure address, serialization, and topology-codec tests + runnable on every host. Apply a shared `requires_tipc` marker + only to tests that create sockets or otherwise touch the kernel; + do not module-skip `tests/ipc/test_tipc.py`. - `--tpt-proto tipc` with no module must fail **loudly and early** with the actionable message, not with 400 confusing timeouts. Add the check to the `tpt_protos` fixture's existing @@ -622,23 +677,29 @@ side effects, no logging. `sudo modprobe tipc` in a `before` step. GH's `ubuntu-latest` runners do allow `modprobe tipc` (the module ships with the standard Ubuntu kernel package); verify in a - throwaway workflow before wiring the matrix. If it turns out - to be unavailable, fall back to a container job with - `--privileged`/`--cap-add NET_ADMIN`, and mark the job - `continue-on-error` until it's proven stable. + throwaway workflow before wiring the matrix. #493's TIPC leg + is now blocking. If runners cease permitting the module load, + fix the environment or use a suitable container rather than + silently restoring `continue-on-error`. - cross-node TIPC (bearer) cannot be CI'd; cover it with a documented manual smoke test in the docs page, in the style of gh #482's LAN examples. ### 7.4 backend-specific tests worth writing -- **name-publication is discovery**: bind a listener on +- **known-name publication/resolution**: bind a listener on `(stype, inst)`, then from a second task `connect()` by name and assert it lands — *without* any `tractor` registrar. -- **`get_random()` collision resistance**: 10k `get_random()` - calls with no live runtime → 10k distinct `_instance`s. - (This is the silent-crosstalk risk from §2.3; if the 4-byte - digest ever collides in this test, escalate to §9.) +- **`get_random()` distribution**: 10k `get_random()` calls with + no live runtime. Do **not** assert 10k distinct values: the + no-runtime seeds and outputs are both only 32 bits. Including + duplicate seeds plus distinct-seed hash collisions puts the + modeled chance of at least one duplicate near 2.3% for 10k + calls. #493 uses `>= n - 2` (modeled probability of more than + two collisions around `2e-6`) and separately proves + `instance_from_seed()` is a pure function. Also hold actor + name/PID fixed while varying only `Aid.uuid` to prove live + actors seed from `Aid.uid`. - **round-robin surprise**: two listeners bound to the *same* `(stype, inst)` both succeed (TIPC allows it) and connects distribute. Assert the observed behaviour and reference it @@ -678,14 +739,61 @@ single best demo this backend has; lead with it. ## 9. Known risks + escalations -| risk | mitigation | -| --- | --- | -| `_instance` hash collision → silent crosstalk (two actors share a service name, TIPC round-robins connects between them) | §7.4 test; if it bites, add a post-bind verification handshake, or bump to a 6-byte digest folded into `(stype_low, instance)` | -| kernel/module unavailability everywhere (dev boxes, macOS, CI) | hard gating (§7.2); TIPC is explicitly an *opt-in cluster* transport, never a default | -| `getsockname()` returns port-id not name | the `rebind_from_sockname` opt-out (§3.2), landed first | -| unregistered `/tipc` multiaddr proto | `str` maddr fallback (§4) + upstream track gh #483 | -| stale docs (#378 notes tipc.io docs may be out of date) | treat `include/uapi/linux/tipc.h` + `net/tipc/` as the only normative source; cite file+symbol in code comments | -| `SOCK_SEQPACKET` topology framing byte-order | probe helper + `?TODO` (§5.2) | +- **Instance collision / silent crosstalk remains unresolved.** + `Aid.uid` seeding and §7.4 tests reduce and measure risk, but + the instance field is still a hard 32 bits. [#501] owns + post-bind verification and recovery. Do not fold bits into + `_stype`: topology can watch only one service type. +- **Concurrent registrar startup can split brain.** Topology can + observe duplicate publisher port IDs but cannot elect or fence + a winner; a separate election protocol is required (§5.1). +- **Kernel/module availability is opt-in.** Keep the hard gate in + §7.2; TIPC is never the default transport. +- **A listener sockname is a port ID, not its service name.** Keep + the `rebind_from_sockname` opt-out (§3.2). +- **`/tipc` is not yet a registered multiaddr protocol.** Keep + the interim `str` maddr fallback (§4) and upstream gh #483. +- **The public TIPC docs can be stale.** Treat + `include/uapi/linux/tipc.h` and `net/tipc/` as normative and + cite file/symbol names in code comments. +- **A slow topology consumer loses continuity.** Fail fast with + `TIPCNameEventOverflow`; resubscribe and rebuild rather than + block the reader or retain stale state (§5.2). +- **TIPC locality preference is not implemented.** Current + `_is_local_addr()` handles only UDS and TCP, so node- and + cluster-scope TIPC both remain in the conservative remote tier. + Add explicit scope-aware policy and multihomed selection tests + before claiming node-scope preference (contract §2.9). + +### 9.1 remaining constructor/error cleanup + +#493 closes the peer-withdrawal race in transport construction, +but it is not a blanket error-path cleanup. Keep these gaps +explicit rather than reporting the backend as fully hardened: + +- direct `TIPCAddress(...)` construction bypasses + `from_addr()` scope normalization; `is_valid` is queried later + rather than enforcing validity at construction. Decide whether + constructors should reject bad service types/instances/scopes + or document direct construction as trusted-internal. +- `maybe_node`/`maybe_ref` are excluded from `.unwrap()` but, as + `msgspec.Struct` fields, still participate in structural + equality/hash. If service-name identity must ignore observation + metadata, represent or compare it explicitly instead of relying + on the current "observed-only" description. +- `start_listener()` must keep ownership of the raw socket through + `bind()`, `listen()` and `SocketListener(...)`. The downstream + implementation normalizes bind errors but does not yet wrap the + complete listener-construction sequence in close-on-error, so a + later setup failure can leak the fd. +- `_maybe_sockaddr()` currently degrades every `OSError` to an + unknown observed address. Narrow that tolerance to expected + peer-withdrawal errors (notably `ENOTCONN`) so unrelated bad-fd + or programming failures remain visible. +- error normalization is intentionally required for an + unpublished-name `EHOSTUNREACH`, but setup `setsockopt`, + listener-constructor, and topology setup failures still need a + consistent policy and focused regression tests. ## 10. Follow-up issue seeds @@ -695,9 +803,11 @@ single best demo this backend has; lead with it. `py-multiaddr`, then drop our `str`-maddr fallback (§4). Worth filing *alongside* the `wg` spec-submission issue so both proposals go up together rather than as one-offs. -- registrar-less discovery fast path via name derivation (§5.1) +- registrar-less discovery fast path via name derivation ([#499], + §5.1) - `TIPC_TOP_SRV`-driven push registry in - `discovery/_registry.py` (§5.2) + `discovery/_registry.py` ([#496], §5.2) +- post-bind collision verification and recovery ([#501], §9) - `TIPC_IMPORTANCE` for the parent<->child lifetime channel (§3.3) — genuinely novel supervision QoS, no other backend can do it @@ -705,3 +815,7 @@ single best demo this backend has; lead with it. for `tractor.trionics` fan-out (explicitly not `MsgTransport`) - dual-link resiliency / multi-homing (#378's "hybrid dual link") once bearers are scripted in the docs + +[#496]: https://github.com/goodboy/tractor/issues/496 +[#499]: https://github.com/goodboy/tractor/issues/499 +[#501]: https://github.com/goodboy/tractor/issues/501 diff --git a/ai/tpt-backends/02_quic_iroh_backend.md b/ai/tpt-backends/02_quic_iroh_backend.md index 53b93999..a349e5a7 100644 --- a/ai/tpt-backends/02_quic_iroh_backend.md +++ b/ai/tpt-backends/02_quic_iroh_backend.md @@ -3,6 +3,12 @@ Tracks gh [#353]. Prereq reading: [`00_shared_backend_contract.md`](./00_shared_backend_contract.md). +**External-fact rule**: every claim here about `iroh`, UniFFI, +generated bindings, QUIC wire/security behavior, or multiaddr +support is provisional until the step-0 API-truth pass records a +source or probe. Tractor/Trio behavior read from this checkout is +the only locally proven basis for the plan. + **Thesis**: the value of `iroh` over "just QUIC" is `NodeId`-addressed, NAT-traversing, relay-fallback endpoints — i.e. a `tractor` actor tree that spans hosts *without* a @@ -49,20 +55,22 @@ relitigate: **But**: build it first as the throwaway spike (§6 step 0) to de-risk the iroh API surface before writing the bridge. -Version pinning: `iroh` moves fast and has had breaking -API renames across minors. Pin `iroh>=X.Y,=X.Y, bytes\|None` | | | half-close | `await send_stream.finish()` | | +| endpoint close + completion | `close()` / `await closed()` | | +| resolved node address | relay URL + direct socket addrs | | +| future start/poll callback ABI | generated symbols + args | | +| future cancel/complete/free | generated symbols + ordering | | +| callback quiescence guarantee | after poll/complete/free? | | +| cancellation terminal poll code | generated enum/value | | +| iroh exception/status taxonomy | per operation | | --- ## 2. The `trio`-native uniffi future bridge (`tractor/ipc/_uniffi_trio.py`) -### 2.1 what uniffi actually generates +### 2.1 Step-0 generated-ABI gate -`uniffi`'s async support does not use asyncio *semantically* — -it uses asyncio only as the *executor* for a poll loop. The -generated python for an `async fn` is, in shape: +The expected generated shape is: start an opaque Rust future, +poll it with a C callback, cancel through a generated cancel +symbol, consume its terminal value/status through `complete`, +then call `free`. The expected callback may arrive on a foreign +Rust thread. **All of that is external and provisional.** Step 0 +must identify the exact generated driver and prove, from its +template/source plus probes: -1. call `_uniffi_..._(...)` → returns an opaque - `RustFuture` handle (a `void*`/`u64`). -2. loop: call - `ffi_..._rust_future_poll_(handle, callback, callback_data)`. - The callback is a C-ABI fn pointer invoked **from an - arbitrary rust thread** with a poll-result code - (`READY`/`MAYBE_READY`). -3. the generated glue's callback resolves an - `asyncio.Future` via `loop.call_soon_threadsafe(...)`; the - coroutine awaits it, then re-polls. -4. on ready: `ffi_..._rust_future_complete_(handle, - &call_status)` → the value; then - `ffi_..._rust_future_free_(handle)`. +1. the start, poll, cancel, complete, and free signatures for + every return-type family used by `iroh`; +2. poll result values and whether callbacks can be synchronous, + concurrent, repeated, or late; +3. which terminal state permits `complete`, when `free` is + legal, and when no callback can still reference Python; +4. whether generated callback-data and call-status objects must + remain alive, and how generated lifting/errors are applied; +5. whether one narrow generated async-driver entrypoint can be + replaced without importing or requiring an asyncio loop. -**The asyncio dependency is confined to step 3.** That is the -whole insight: the bridge is ~40 lines. +Do not implement from a remembered UniFFI version. If cancel +does not have a documented path to a terminal, safely freeable +state, the native Trio bridge fails the spike gate and the first +backend uses the infected-asyncio fallback. -### 2.2 the trio version +### 2.2 Cancellation-safe ownership -```python -async def await_rust_future( - poll: Callable, # ffi_..._rust_future_poll_ - complete: Callable, # ffi_..._rust_future_complete_ - free: Callable, # ffi_..._rust_future_free_ - handle: int, - lift: Callable[[Any], Any], -) -> Any: - ''' - Drive a `uniffi` rust-future to completion on the current - `trio` task, bridging rust-thread wakeups via - `TrioToken.run_sync_soon()`. +Do not let the caller task own a raw handle across an `await`. +Introduce an actor-scoped `UniffiFutureSupervisor` running in the +dedicated transport nursery specified in §3.2.1. That nursery +must span parent bootstrap, the service nurseries, and final +deregistration. For each call, its operation task owns the +**entire** generated lifecycle: - ''' - token = trio.lowlevel.current_trio_token() - while True: - wake = trio.Event() - # NOTE, invoked from a *rust* thread! - def _cb(_data, poll_code): - token.run_sync_soon(wake.set) - - cb = _UNIFFI_FUTURE_CALLBACK(_cb) # keep a strong ref! - poll(handle, cb, 0) - await wake.wait() - if : - break - try: - status = _UniffiRustCallStatus.default() - res = complete(handle, status) - _uniffi_check_call_status(status) # reuse generated helper - return lift(res) - finally: - free(handle) +```text +create handle -> poll/callback loop -> complete -> lift/status + -> free -> publish result + ^ + cancel request uses generated cancel, then follows + the verified terminal poll/complete/free protocol ``` -Critical details, each a real bug if missed: +The operation task, not the awaiting caller, creates the handle. +Creation and insertion in the supervisor's live-operation set +must have no cancellation checkpoint between them. The operation +retains strong references to the C callback trampoline, callback +data, wake state, call status, and handle until step 0 proves all +callbacks are quiescent and `free` has returned. Use one stable +callback per operation unless the verified ABI requires a fresh +one per poll; in either case, retain every potentially callable +trampoline. Capture `current_trio_token()` in the Trio owner and +schedule the wake into Trio with `token.run_sync_soon(...)`; the +foreign callback only stores its poll result and schedules that +wake. -- **`token.run_sync_soon()` is the only trio API callable from a - foreign thread**, and it is documented as such. Use it; do - *not* use `trio.from_thread.run_sync` (requires a trio thread - context) and do not touch the `Event` directly from the - callback. -- **the poll code must reach the trio side.** Capture it in a - `nonlocal`/1-slot list written by the callback *before* - `run_sync_soon`, since the callback owns the value. Handle - `MAYBE_READY` by re-polling (the loop above does). -- **keep the `ctypes` callback object alive** across the await — - a GC'd `CFUNCTYPE` trampoline is a segfault. Bind it to a - local *and* make sure the local outlives the `poll()` call - window. -- **cancellation.** `await wake.wait()` is a trio checkpoint, so - a `Cancelled` can fire while rust still owns the future. On - cancel we must still `free(handle)` — and per uniffi, the - correct sequence is to call the generated - `ffi_..._rust_future_cancel_(handle)` then continue - polling to completion before `free`. Wrap the whole thing so - the cancel path does: - `with trio.CancelScope(shield=True): cancel(handle); ; free(handle)`. **Bounded** shield (add a - `trio.move_on_after()` with a module-level constant) so a - wedged rust future can't make an actor un-cancellable — - `tractor` is SC-first and an unbounded shield here would - violate that. -- **`trio.lowlevel.current_trio_token()`** must be captured on - the trio side (not in the callback). +Caller cancellation is a request, not handle ownership transfer: + +1. the caller sends an idempotent cancel request and waits under + a short shield for the operation to acknowledge it; +2. the owner invokes the generated cancel function exactly once + and continues the **verified** poll/complete/free sequence; +3. once caller cancellation is observed, cleanup completion never + wins the race by returning a value. After acknowledgement the + caller continues propagating its original Trio cancellation; + if cleanup outlives the grace period it first abandons its + result channel while the actor supervisor keeps ownership; +4. actor endpoint teardown stops accepting new calls, requests + cancellation of all live operations, and joins the supervisor + before destroying endpoint/key state. + +There is deliberately no `move_on_after(...): free(handle)` +path. A timeout proves only that cleanup is slow; it does not +prove that callbacks are quiescent or that `free` is legal. A +wedged operation therefore remains visible in the supervisor and +can delay graceful actor shutdown; process-level termination is +the final escalation, not an unsafe FFI free. + +Structured-concurrency race to test: caller cancellation may land +after handle creation, after each poll, during callback delivery, +after terminal readiness, during `complete`, and before result +publication. At every checkpoint exactly one operation task owns +the handle, exactly one `free` is possible, and the supervisor +cannot exit while that task or a callable trampoline remains. ### 2.3 how to apply it to the generated bindings -Do **not** fork/vendor the generated `iroh` python. Instead ship -a *narrow* re-dispatch shim: +Do **not** fork/vendor the generated `iroh` Python. Subject to the +step-0 gate, ship a *narrow* re-dispatch shim: -- write `tractor/ipc/_uniffi_trio.py` with `await_rust_future()` - plus a `@cm patch_uniffi_for_trio()` that monkey-patches the - generated module's single async-driver entrypoint (in current - uniffi that's `_uniffi_rust_call_async` / `_rust_call_async`, - one function) to the trio implementation. +- write `tractor/ipc/_uniffi_trio.py` with the supervisor and a + `@cm patch_uniffi_for_trio()` that patches only the generated + async-driver entrypoint recorded in §1.1; - verify at import time that the expected symbol exists and raise a clear, actionable error naming the pinned `iroh` version if not. A silent fallback to asyncio would be a nightmare to debug. -- **plan for this to break on `iroh`/`uniffi` upgrades.** Mitigate - with (a) a unit test that drives one trivial `iroh` async call - under bare `trio.run()` and asserts no event loop was ever - created (`asyncio.get_event_loop_policy()` untouched / - `asyncio._get_running_loop() is None`), and (b) a docstring - pointing at the uniffi codegen template this mirrors. +- treat every `iroh`/UniFFI upgrade as requiring the step-0 ABI + gate again. Keep a test that drives one trivial call under bare + `trio.run()`, asserts no asyncio loop, and injects cancellation + at every lifecycle checkpoint. Point the module docstring at + the exact generated template/revision mirrored by the shim. If step 0 reveals the generated code is *structurally* hostile to this (e.g. `asyncio` imported and used at module scope for @@ -228,18 +234,52 @@ iroh bi-stream == one `Channel`/`MsgTransport` -> 1:1 - `layer_key: int = 4` still (QUIC is L4-ish); note in a comment that this backend is really 4+security+multiplex. -**Connection pooling** is the one place we add state the other -backends don't have: dialing the same peer twice should reuse -the `Connection` and open a second bi-stream. Implement as a -module-level `dict[NodeId, Connection]` guarded by a -`trio.Lock`... **no** — that's a per-process cache with -lifetime/teardown hazards. Instead reuse the codebase's existing -idiom: `tractor.trionics.maybe_open_context()` keyed on the -node-id, which already solves exactly this (one-cached-resource- -per-key, refcounted, teardown-on-last-exit) and whose teardown -semantics were just hardened (gh #488). Use it; do not hand-roll -a cache. Anything concurrency-subtle here should get the -`conc-anal` skill run over it. +**Connection pooling** is actor-endpoint state, never module +state. Its key is exactly +`(local_endpoint_identity, remote_node_id, alpn)`, where local +endpoint identity is the local NodeId derived from the actor key. +Remote NodeId alone would incorrectly share connections across +local keys or protocol epochs. Build it over the codebase's +`maybe_open_context()` idiom only after a concurrency review of +its actual last-user teardown behavior in the implementation +revision. Do not assume an issue reference proves the required +ordering. + +`acquire_connection()` returns a `ConnectionLease`, not a bare +connection. An outgoing `QuicMsgStream` owns that entered lease +for its whole lifetime; `connect_to()` must not exit the cached +context immediately after `open_bi()`. Exact transfer paths: + +- dial/acquire or `open_bi()` failure releases the lease in a + shielded `finally` before raising; +- successful stream construction atomically transfers the lease + to `QuicMsgStream` before the first cancellation checkpoint; +- `send_eof()` closes only the send half and does not release; +- clean receive EOF closes only the receive half and does not + release while the send half remains usable; +- one guarded terminal-state transition releases exactly once + when both halves have become terminal, in either order; +- `aclose()`, reset, or terminal connection failure closes both + halves as applicable and idempotently releases exactly once; +- a stream queued by `QuicListener` already owns its lease; if + never accepted, listener draining closes it and releases it. + +After `accept()` returns, the server dispatch path owns the stream +until a handler task starts and must close it if task start fails. +The handler then takes ownership, with an outer `finally` that +calls `stream.aclose()` on normal return, handshake failure, and +cancellation. Lease release itself is an idempotent pool state +transition; if last-user connection teardown awaits FFI, the actor +endpoint's pool supervisor owns that await so cancellation of the +handler cannot strand the lease. + +For inbound connections, the connection-feeder owns a base lease +while accepting streams and each queued/returned stream gets a +child lease. The base lease is released only after the accept +loop ends; the pool closes the connection after the base and all +stream leases are gone. Reject or deterministically reconcile a +simultaneous inbound/outbound duplicate for the same full key; +record the chosen iroh-compatible rule during step 0. ### 3.2 `IrohAddress` @@ -248,29 +288,29 @@ class IrohAddress( msgspec.Struct, frozen=True, ): - _node_id: str # 32B ed25519 pubkey, hex or z32 - _alpn: str = 'tractor/0' # the bindspace! - # optional dial hints; NOT part of identity - maybe_relay_url: str|None = None - maybe_direct_addrs: tuple[str, ...] = () + _node_id: str + _alpn: str + _relay_url: str|None + _direct_addrs: tuple[str, ...] - proto_key: ClassVar[str] = 'iroh' # ?or 'quic'; see §3.2.1 - unwrapped_type: ClassVar[type] = tuple[str, str] + proto_key: ClassVar[str] = 'quic' + unwrapped_type: ClassVar[type] = tuple def_bindspace: ClassVar[str] = 'tractor/0' ``` -- **`.unwrap() -> (node_id_str, alpn_str)`** — a `(str, str)` - tuple, which is *unambiguously distinct* from - `TCPAddress`'s `(str, int)`. But careful: - `wrap_address()`'s UDS case is - `case (_, filename) if type(filename) is str` — which - **already catches `(str, str)`**. So the iroh `case` MUST be - ordered *before* the UDS case and guarded, e.g. - `case (str() as nid, str() as alpn) if _is_node_id(nid):` - with `_is_node_id()` a cheap length+alphabet check. Add a - regression test asserting a UDS `(dir, filename)` pair still - wraps to `UDSAddress` — this is the exact "wrong transport - loaded" hazard `_addr.py:214` warns about. +- **`.unwrap()` is the complete, tagged wire descriptor**: + `('quic', node_id, alpn, relay_url, direct_addrs)`. All values + are msgpack-native and `direct_addrs` is canonicalized to a + tuple. `from_addr()` requires that exact tag and shape; never + infer QUIC from a `(str, str)` pair. This depends on the shared + contract's tagged-address migration and removes the UDS + collision rather than ordering around it. +- The descriptor always carries NodeId, ALPN, and both route-hint + fields. For this discovery-free first backend, `.is_valid` + requires a parseable NodeId, non-empty ALPN, and at least one + relay URL or direct address. Whether NodeId-only dialing works + through optional iroh discovery is a step-0 API check and is + not part of the first implementation. - `.bindspace` → `self._alpn`. This is the honest analogue: the ALPN is the set of endpoints willing to talk to you, and two `tractor` deployments sharing an iroh network are @@ -278,39 +318,80 @@ class IrohAddress( separated by directory. Include a `tractor` version/proto epoch in the default ALPN so incompatible runtimes can't handshake. -- `.is_valid` → node-id parses, alpn non-empty. -- **`get_root()` is the hard one.** There is no - well-known-port analogue: an iroh node id is a *keypair*, so - "the host's default registrar addr" requires a *persisted - secret key*. Design: - - the root/registrar's secret key lives at - `get_rt_dir() / 'iroh_registrar.key'` (0600), created on - first use. - - `get_root()` must stay **pure and import-time-safe** - (contract §2.3: `_default_lo_addrs` is built at import!). - So `get_root()` *reads* the key file if present and - otherwise returns an `IrohAddress` with - `_node_id=''`/sentinel, and the **generation** happens in - an explicit sibling — `ensure_registrar_key() -> - IrohAddress` — called from the listen path. Pure getter, - explicit setter; do not smuggle key generation into - `get_root()`. - - this almost certainly means `_default_lo_addrs` must become - lazy for this backend. **Land that refactor as its own prep - commit** (a `default_lo_addrs()` that computes per-call - instead of the import-time dict) — it also unblocks plan - 03's netns-scoped defaults. -- `get_random()`: generate a fresh `SecretKey` per subactor and - return its node-id. Note this runs post-fork pre-listen - (contract §4) and costs an ed25519 keygen (~µs, fine). The - *secret* can't live in a frozen `Address`, so it must be - stashed where the listen path can find it: a module-level - `dict[node_id, SecretKey]` populated by `get_random()` and - consumed+popped by `start_listener()`. Ugly but honest; - document it and note the alternative (thread the key through - `Endpoint`) as a follow-up. -#### 3.2.1 `proto_key`: `'iroh'` vs `'quic'` +### 3.2.1 One actor endpoint and key + +Add an actor-scoped `QuicActorEndpoint` resource containing the +secret key, one bound iroh endpoint, the UniFFI supervisor, the +connection pool, and its latest resolved `IrohAddress`. It cannot +live in `_service_tn`: a child dials its parent before that nursery +opens, while final deregistration may dial after it closes. + +Add a dedicated `transport_tn` around the complete actor runtime: +the task that opens this nursery must start the complete +`async_main` sequence as a **child** of it and wait for that child. +That makes `transport_tn` an ancestor of every parent-dial, +service, and deregistration caller, satisfying +`maybe_open_context(tn=transport_tn)` rather than asking the +nursery-opening task to use its own child nursery. The child keeps +the nursery around `_root_tn` and `_service_tn`, performs final +deregistration while it remains open, then returns so the owner can +close the transport resource and nursery. Root startup needs the +equivalent outer owner around actor construction, service, and +teardown. If this shape cannot be preserved, the connection pool +must stop depending on `maybe_open_context()`'s ancestor-nursery +contract. No path creates a second endpoint for the actor. + +The child currently receives transport configuration only in the +`SpawnSpec` sent over its already-open parent channel. QUIC cannot +derive its local key, ALPN, or requested bind policy from that late +message. Add a small msgpack/pickle-native +`ChildTransportBootstrap` to every process-launch path. It carries +the selected protocol and the QUIC-local key reference/generation +policy, ALPN, relay policy, and requested bind constraints. It is +available before `_from_parent()`; the later `SpawnSpec` repeats +the public configuration and startup rejects any mismatch. Root +actors derive the same bootstrap record directly from +`open_root_actor()` inputs before address selection. + +With that prep in place, the order is: + +1. consume the launch-time bootstrap record, select one key + (persisted and explicitly provisioned for a registrar, fresh + for an ordinary actor), and construct/bind the endpoint in the + transport owner task; +2. await the step-0-verified address-ready API and build a valid + descriptor from the endpoint's NodeId, ALPN, relay URL, and + direct addresses; +3. only then dial `_from_parent()` through this endpoint; +4. start `QuicListener` over this endpoint's accept API; +5. publish the resolved descriptor as `Endpoint.addr` and + `Actor.accept_addrs` before parent/registrar registration; +6. after service nurseries close, keep the endpoint available for + deregistration; then close listeners and streams, drain + connection leases and FFI operations, close/join the endpoint, + and release key state. + +`IrohAddress.get_random()` is therefore a descriptor lookup on +the active actor transport resource, not key generation. Broaden +the shared `get_random()` contract for resource-backed transports +and make root/subactor address selection consume the bootstrap +resource instead of calling it before that resource exists. Do not +hide a secret in a module-level side table. Calls without an active +resource fail clearly rather than allocating an unowned key. + +`get_root()` never returns an empty/sentinel NodeId. Make default +addresses lazy, and have QUIC load a provisioned public registrar +descriptor. Registrar provisioning writes its secret separately +with mode 0600 and writes the matching complete public descriptor +atomically; endpoint startup verifies the derived NodeId. If no +descriptor exists, default QUIC registrar discovery fails with an +actionable configuration error. Automatic first-process election +is deferred until a safe key-file locking and endpoint-binding +protocol is proven; key generation never occurs in the listen +path. + +#### 3.2.2 `proto_key`: `'iroh'` vs `'quic'` Use **`'quic'`** for the `proto_key`/`--tpt-proto` name and name the module `_quic.py`, with `iroh` as the *implementation*. @@ -357,7 +438,7 @@ class QuicMsgStream(trio.abc.HalfCloseableStream): ''' tpt_key: ClassVar[MsgTransportKey] = ('msgpack', 'quic') - def __init__(self, conn, send, recv) -> None: ... + def __init__(self, conn, send, recv, lease) -> None: ... async def send_all(self, data: bytes) -> None: ... async def wait_send_all_might_not_block(self) -> None: ... async def receive_some(self, max_bytes: int|None = None) -> bytes: ... @@ -379,17 +460,38 @@ already exists in `_transport.py` and must keep working): absent, so the `raise_on_report` branch at `_transport.py:290` stays quiet). - `send_all()` on a closed peer → `trio.BrokenResourceError`. -- honour `trio`'s one-task-per-direction rule: guard with - `trio._util.ConflictDetector` equivalents (or just document + - assert), because `MsgpackTransport` already serializes sends - with a `StrictFIFOLock` but recvs are single-task by - construction. +- honour Trio's one-task-per-direction rule with public, + implementation-local guards that raise + `trio.BusyResourceError`; do not depend on `trio._util`. + `MsgpackTransport` already serializes sends, while receives are + single-task by construction. - **buffering**: if iroh's `read()` doesn't support "read up to n", `receive_some()` must maintain an internal leftover buffer. Note `MsgpackTransport` wraps us in `tricycle.BufferedReceiveStream` anyway, so `receive_some()` just needs *some* nonzero-progress contract. +Centralize exception translation at every iroh/UniFFI boundary; +no generated exception may escape into `Channel` or server code. +Step 0 must record actual exception classes/status payloads and +build an exhaustive operation-specific mapping: + +| observed condition | adapter result | +| --- | --- | +| receive clean EOF | `b''` | +| local stream/listener/endpoint already closed | `trio.ClosedResourceError` | +| concurrent same-direction operation | `trio.BusyResourceError` | +| peer reset, stopped stream, lost connection | `trio.BrokenResourceError` | +| dial rejected or no usable route | `ConnectionRefusedError` or `ConnectionError` | +| caller's Trio deadline/cancellation | preserve Trio cancellation semantics | +| unexpected FFI status/panic | chained `RuntimeError` identifying operation and pinned version | + +Preserve the original exception as `__cause__`, but sanitize +messages so `_transport.py` sees stable Trio/Tractor categories, +not version-specific iroh text. Endpoint accept failure becomes a +listener `BrokenResourceError`; normal endpoint shutdown becomes +`ClosedResourceError`. Add one test per observed step-0 status. + ```python class QuicListener(trio.abc.Listener): ''' @@ -402,41 +504,63 @@ class QuicListener(trio.abc.Listener): async def aclose(self) -> None: ... ``` -The accept-side subtlety: `trio.abc.Listener.accept()` yields -one stream per call, but iroh gives us *connections* which then -yield *streams*. So `QuicListener` needs an internal -`trio.MemoryReceiveChannel[QuicMsgStream]` fed by a background -task-pair (one task accepting connections, one per connection -accepting bi-streams). `trio.abc.Listener` has no nursery, so: -make the listener **constructed by an `@acm`** that owns the -nursery, and have `start_listener()` be that `@acm`'s driver. +The accept-side subtlety is fan-out: one actor transport accepts +connections and each connection accepts streams, while +`Listener.accept()` returns one stream. Give **each** listener a +supervisor task started with +`await server_ep.listen_tn.start(...)`. +That task creates and owns a cancel scope, a child nursery for the +endpoint feeder plus per-connection feeders, a guarded stream +queue, and a completion event. +`start_listener(addr=, server_ep=, actor_tpt=)` does not return +until the supervisor has reported all of those ready. +Do not borrow an implicit parent nursery or spawn feeders lazily +from `accept()`. -⚠️ this collides with `Endpoint.start_listener()` being a plain -`async def` returning a listener. Two options: -- **(a)** hang the nursery off the `Endpoint`'s existing - `listen_tn` — `_serve_ipc_eps()` already creates `listen_tn` - and passes it into every `Endpoint` (`_server.py:1063-1074`), - and `Endpoint.listen_tn` is right there. So - `start_listener()` can `self.listen_tn.start_soon(...)` the - acceptor tasks. **Recommended**: no upstream signature change, - correct lifetime (dies with the ep group), and it's why - `listen_tn` is on the struct in the first place. -- (b) change `start_listener()` to a `@acm`. Bigger blast - radius; only if (a) proves insufficient. +The queue is a guarded `deque`, not an unowned memory-channel +buffer. A feeder transfers a fully constructed, lease-owning +stream into it only while the listener is open; if close wins the +race, the feeder closes the stream itself. `accept()` atomically +pops one item or waits on the queue condition. Once close is +marked and the queue is empty, it raises +`trio.ClosedResourceError`. -Since `start_listener()` is called via -`inspect.getmodule(addr)` with only `addr=` (contract §1.3), -option (a) needs the `Endpoint` itself. Either add `ep=` to the -module-level `start_listener()` call signature (all backends -ignore it except quic → small upstream change, do it as part of -the prep PR and make it keyword-only with a default) or have -`QuicListener.accept()` lazily spawn via -`trio.lowlevel.current_task().parent_nursery` (**rejected** — -fragile, implicit). Do the explicit `ep=` kwarg. +`QuicListener.aclose()` is idempotent and has this exact order: + +1. under the queue guard, mark closed and wake all `accept()` + waiters without a checkpoint between the state change and + notification; +2. cancel the listener-owned supervisor scope; +3. the supervisor's shielded `finally` joins the endpoint and all + connection feeders, atomically detaches the queue, closes every + queued stream, releases their leases, and closes the queue; +4. only after that finalizer finishes, the supervisor sets its + completion event; +5. `aclose()` waits under a shield for that event and returns; + concurrent closers wait for the same event. + +The same supervisor finalizer runs if its parent nursery is +cancelled before someone calls `aclose()`. This makes the +supervisor, not an arbitrarily cancelled caller, the sole final +cleanup owner. Test cancellation at feeder accept, stream +construction, queue transfer, `accept()` wakeup, and each close +checkpoint; no feeder may outlive the listener and no queued +lease may survive completion. + +This needs two explicit, typed references in the module-level +listener call: `server_ep=` is the IPC server `Endpoint` that owns +`listen_tn`, while `actor_tpt=` is the already-open +`QuicActorEndpoint` whose iroh accept API supplies connections. +Store `actor_tpt` on the server endpoint during actor transport +bootstrap and pass both keyword-only arguments; socket backends +ignore `actor_tpt`. `Endpoint.start_listener()` then stores the +listener's already-resolved address instead of calling +`getsockname()`. ### 3.4 `maddr` -Multiaddr already standardizes the pieces: +Expected multiaddr spellings for direct QUIC and relay routes are +**step-0 verification items**, not assumptions: ``` /ip4//udp/

/quic-v1 # direct @@ -444,19 +568,16 @@ Multiaddr already standardizes the pieces: /dns//tcp/443/tls/ws/p2p/ # relay-ish ``` -- primary form: `/p2p/` alone is a legal maddr and is - the *only* required component for iroh dialling — relay + - direct addrs are discovery hints. So `mk_maddr()` emits - `/p2p/` and, when known, prefixes the direct - `/ip4/../udp/../quic-v1/`. -- `/p2p/` values are multihash-encoded peer ids; an iroh node-id - is a raw ed25519 key. Converting requires the identity - multihash + libp2p key protobuf wrapper. **Decide**: emit the - raw node-id under a *tractor-local* `/iroh/` segment - (needs upstream registration, same track as `wg`/`tipc`, - gh #483) rather than pretending to be a libp2p peer-id we - can't round-trip. Return the `str` form until upstream lands - (`MsgTransport.maddr` is `Multiaddr|str`). +- Do not emit NodeId alone in the first backend: without enabled + discovery it would discard the route required by the complete + `IrohAddress`. `mk_maddr()` must preserve NodeId, ALPN, relay + URL, and all direct addresses, or return a canonical Tractor + string form that does until a multiaddr grammar can round-trip + every field. +- Verify whether an iroh NodeId can losslessly map to `/p2p/`. + If not, use a tractor-local `/iroh/` segment rather + than pretending to be a libp2p peer-id. This needs upstream + registration, on the same track as `wg`/`tipc` (gh #483). - this backend is the strongest argument for gh #443's **tunnelled/composed maddr** item: `/ip4/../udp/../quic-v1/..` *is* a composed stack. Cross-reference plan 03 §5 so the two @@ -466,24 +587,25 @@ Multiaddr already standardizes the pieces: ## 4. Discovery integration -- iroh's node-id addressing means the `tractor` registrar can - hold `IrohAddress`es that are **reachable from anywhere** with - no port-forwarding — that is the headline feature. The - registrar itself works unchanged. -- iroh has its own discovery (DNS/pkarr/mdns). **Out of scope**; - note in the follow-up that `tractor.discovery` could - eventually delegate to it, which would be the direct analogue - of plan 01's TIPC-topology idea. -- relay servers: default to n0's public relays for the demo, - document self-hosting (docs.iroh.computer's dedicated-infra - page is linked from #353), and make the relay set a - `start_listener()` kwarg. +- The registrar stores the complete `IrohAddress`, not only a + NodeId. Registration is forbidden until endpoint address + resolution has produced that descriptor. If route hints change + later, dynamic re-registration is a follow-up; the spike uses + the pre-registration snapshot. +- Optional iroh discovery mechanisms and their names/capabilities + are step-0 verification items and out of scope for the first + backend. No NodeId-only reachability claim is made. +- Relay configuration belongs to `QuicActorEndpoint` creation, + not `start_listener()`, because dialing and listening reuse the + same endpoint. The demo's relay choice and self-hosted option + are selected only after step 0 verifies the pinned API. ## 5. Security note -QUIC is TLS-1.3-always and iroh authenticates by node-id, so -this backend is the first `tractor` transport with real -transport security and peer authentication. Two things follow: +The transport-security and NodeId-authentication properties of the +pinned iroh stack are **step-0 documentation-verification items**. +Claim only the properties supported by that version's source and +docs. Two design consequences remain: 1. an **allowlist hook** — an actor should be able to reject inbound connections from unknown node-ids *before* the `Aid` handshake. Natural home: a predicate kwarg on @@ -497,59 +619,72 @@ transport security and peer authentication. Two things follow: 0. **spike (throwaway, not committed)**: drive iroh under `trio-asyncio`/`tractor.to_asyncio`, echo bytes over a - bi-stream between two procs. Fills in §1.1. Timebox it. -1. prep PR: annotation widening + `rebind_from_sockname` gate + - `transport_from_stream()` `tpt_key` dispatch + `ep=` kwarg on - `start_listener()` + lazy `default_lo_addrs()`. **No new - backend.** Full suite green on tcp *and* uds. -2. `_uniffi_trio.py` + its tests (drive one iroh async call - under bare `trio.run()`; assert no asyncio loop; assert - cancellation frees the future). -3. `QuicMsgStream` + tests against a *loopback* iroh endpoint - pair in one process (no `tractor` runtime): send/recv, clean - EOF → `b''`, reset → `BrokenResourceError`, use-after-close - → `ClosedResourceError`. -4. `QuicListener` + `start_listener()` + `IrohAddress` + - key-file mgmt. -5. `MsgpackQuicStream(MsgpackTransport)` + `connect_to()` + - `maybe_open_context()` connection pooling. -6. registration tables + `--tpt-proto quic` + full suite. -7. maddr + docs + a two-host example (pairs with #482's format). + bi-stream between two processes. Fill §1.1 with generated ABI, + endpoint resolution, close/join, and error observations. Probe + cancel at every generated lifecycle phase. Timebox it and use + the fallback if any mandatory ownership fact stays unknown. +1. prep PR: tagged address migration, annotation widening, + non-socket listener reconciliation, `tpt_key` dispatch, + typed `server_ep=`/`actor_tpt=` listener inputs, and lazy + default addresses. **No new backend.** Keep tcp and uds + behavior unchanged. +2. bootstrap prep: pass `ChildTransportBootstrap` through every + process-launch path and add the transport nursery around child + parent-dial, service, deregistration, and teardown. Resolve the + endpoint address before registration. Add no iroh-specific + global state. +3. `_uniffi_trio.py` supervisor + lifecycle fault-injection tests: + no asyncio loop, one owner/complete/free, callback retention, + bounded caller handoff, and joined durable cleanup. +4. `QuicActorEndpoint` + provisioned registrar descriptor + + loopback direct-address tests; prove one endpoint handles dial, + listen, address lookup, and ordered teardown. +5. `QuicMsgStream` + exhaustive error-normalization and lease + release tests against the loopback endpoint pair. +6. `QuicListener` supervisor + cancellation-at-every-checkpoint + tests, including queued-stream draining and feeder joins. +7. `MsgpackQuicStream`, full-key connection pooling, registration + tables, and `--tpt-proto quic`; then run the full suite. +8. routable maddr/string form + docs + a two-host example (pairs + with #482's format). ## 7. Testing - capability predicate `is_quic_available()` → `iroh` importable - *and* the uniffi driver symbol present at the pinned version. - Same `pytest.fail`-early hook as plan 01 §7.2. + *and* every step-0-recorded driver symbol present at the pinned + version. Same `pytest.fail`-early hook as plan 01 §7.2. - **the acceptance bar is the same**: whole suite green under `--tpt-proto quic`. Expect this to shake out real bugs in the adapters (esp. teardown ordering and `TransportClosed` classification) — that's the point. -- expect to need **timeout headroom**: iroh endpoint bind + - first connect (relay discovery) is orders of magnitude slower - than a UDS bind. Before touching any test deadline, rule out - the CPU-throttle false-positive (see the project's - `env_cpu_throttle_masquerades_as_regression` note); then, if - real, add a per-proto timeout multiplier to the test harness - rather than editing individual tests. -- a no-network test mode: iroh with relays disabled + - loopback direct addrs only, so CI doesn't depend on n0's - infra. **Make this the default in CI**; mark the relay tests - `pytest.mark.net` and keep them out of the default run. -- leak checks: assert every `SecretKey`/`Endpoint` is closed on - actor teardown (an `Endpoint` left open holds UDP sockets and - relay connections; a leak here shows up as hung tests, not - errors). +- Measure endpoint bind and first-connect latency in step 0; do + not assume a multiplier. Before changing a deadline, rule out + the project's CPU-throttle false-positive, then prefer one + per-proto harness multiplier over individual-test edits. +- Use the step-0-verified relay-disable configuration with direct + loopback addresses for default CI. Mark separately verified + relay tests `pytest.mark.net` and keep them out of default CI. +- leak checks: assert the actor has one key/endpoint, every FFI + operation completed/freed once, all listener feeders joined, + all queued streams closed, every connection lease released, + and endpoint close completion observed before actor teardown. +- address-ordering check: block registration until a descriptor + with NodeId, ALPN, and at least one route is published; reject + sentinel, NodeId-only, and post-registration mutation cases. ## 8. Risks | risk | mitigation | | --- | --- | | uniffi codegen internals shift on upgrade | pinned minor, symbol assertion at import, the "no asyncio loop" test, documented fallback to `to_asyncio` | -| rust-thread callback → trio wakeup mishandled (segfault / lost wakeup / un-cancellable task) | strong ref on the ctypes trampoline; `run_sync_soon` only; **bounded** shielded cancel-drain; run the `conc-anal` skill over the bridge | +| callback wakeup/lifetime semantics differ from the hypothesis | step-0 source + probe gate; retain callback/data through verified quiescence; durable owner; never timeout-free | +| cancelled foreign future never reaches a freeable state | bounded caller handoff to visible actor supervisor; joined graceful shutdown or process-level escalation; never speculative free | | `iroh` wheel availability for 3.13/3.14 on linux+macos | verify in step 0; if missing, that alone may force the `aioquic` fallback | -| QUIC latency/jitter destabilizes the existing suite's timing assumptions | per-proto timeout multiplier, relay-less CI mode | -| `(str, str)` unwrapped form collides with UDS in `wrap_address()` | guarded case ordered first + explicit regression test (§3.2) | +| endpoint or route resolution is not ready before parent dial/registration | actor endpoint bootstrap barrier; publish only a complete resolved descriptor | +| connection closes while a stream still uses it | full-key pool + stream-held leases + exact-once release tests | +| listener close strands feeder tasks or queued streams | listener-owned scope/completion event; cancel, join, drain, then return | +| QUIC latency/jitter destabilizes suite timing assumptions | measure first; per-proto multiplier only if demonstrated; relay-less CI mode | +| address tuple collides with another backend | required `'quic'` tag and exact-shape dispatch | | scope creep into iroh's docs/blobs/gossip crates | this backend is `Endpoint`+`Connection`+bi-streams only; anything else is a separate issue | ## 9. Follow-up issue seeds diff --git a/ai/tpt-backends/README.md b/ai/tpt-backends/README.md index 2f64b12f..0ff9ce0f 100644 --- a/ai/tpt-backends/README.md +++ b/ai/tpt-backends/README.md @@ -7,9 +7,9 @@ different model/provider) without design or lib-selection drift. **Read [`00_shared_backend_contract.md`](./00_shared_backend_contract.md) first** — it is the normative description of what a `tractor` transport backend *is* as of `main@83b34884` (the backend -duck-type, the 10-item registration checklist, the test-harness -plumbing, the code-style rules). The three plans assume it and -document only their own deltas. +duck-type, registration and address-selection wiring, the +test-harness plumbing, the code-style rules). The three plans +assume it and document only their own deltas. | plan | issue | dep | size | lands | | --- | --- | --- | --- | --- | @@ -23,10 +23,12 @@ Headline conclusions: `trio.SocketListener` are address-family agnostic (only `SOCK_STREAM` + a trio socket), and CPython ships `AF_TIPC` + 23 `TIPC_*` constants. So the backend is ~one module of - contract boilerplate, zero new deps, and it buys - *kernel-native* service discovery: `bind()` publishes, - `connect()`-by-name resolves — no registrar in the loop. - (`modprobe tipc` is required; hard-gate everything.) + contract boilerplate, zero new deps, and it buys kernel-native + service primitives: `bind()` publishes, known-address + `connect()` resolves, and topology events report publication + changes. Actor-name lookup, registrar state and split-brain-safe + election remain separate work. (`modprobe tipc` is required; + hard-gate everything.) - **QUIC's cost is entirely in two adapters**, not in QUIC. The `iroh` python bindings are `uniffi`-generated asyncio, but the asyncio dependency is confined to *one* future-poll callback — From e4e27ec71defdec300bee17ba55d61d2bfdd0cc4 Mon Sep 17 00:00:00 2001 From: goodboy Date: Mon, 31 Aug 2026 12:28:19 -0400 Subject: [PATCH 18/18] Harden the wg LAN example for real hosts Validate WireGuard keys and tunnel descriptors strictly. Inspect iface keys asynchronously by local/peer role and reject unsupported nested tunnels. Correct both host binds and service publication, document an unprivileged two-host setup and pin the merged `py-multiaddr` codec revision in the lock. Prompt-IO: ai/prompt-io/opencode/20260831T022317Z_768b5316_prompt_io.md (this patch was generated in some part by `opencode` using `gpt-5.6-sol` (`openai`)) --- .../20260831T022317Z_768b5316_prompt_io.md | 72 +++++++++++ ...20260831T022317Z_768b5316_prompt_io.raw.md | 60 +++++++++ ai/tpt-backends/03_wg_tunnel_bindspace.md | 39 +++--- examples/multihost/wg_lan/README.md | 79 ++++++++---- examples/multihost/wg_lan/host_a_srv.py | 25 ++-- examples/multihost/wg_lan/host_b_client.py | 23 +++- examples/multihost/wg_lan/wg_maddr.py | 117 +++++++++++++----- uv.lock | 14 +-- 8 files changed, 339 insertions(+), 90 deletions(-) create mode 100644 ai/prompt-io/opencode/20260831T022317Z_768b5316_prompt_io.md create mode 100644 ai/prompt-io/opencode/20260831T022317Z_768b5316_prompt_io.raw.md diff --git a/ai/prompt-io/opencode/20260831T022317Z_768b5316_prompt_io.md b/ai/prompt-io/opencode/20260831T022317Z_768b5316_prompt_io.md new file mode 100644 index 00000000..4d8fa7d5 --- /dev/null +++ b/ai/prompt-io/opencode/20260831T022317Z_768b5316_prompt_io.md @@ -0,0 +1,72 @@ +--- +model: gpt-5.6-sol +service: opencode +session: f3a5eef0-9695-48ab-a336-861187473f8a +timestamp: 2026-08-31T02:23:17Z +git_ref: ng_tpts_planning +diff_cmd: git diff 768b531662a88037e3988885102d2ff24a7ce34e..HEAD +scope: code +substantive: true +raw_file: 20260831T022317Z_768b5316_prompt_io.raw.md +--- + +## Prompt + +The human asked to continue the retained PR landing sequence and +selected "Repair all blockers" after a read-only review of PR #492. +They then corrected the TIPC assessment: + +> oh one thing on the tipc findings, they may have already been +> addressed in the downstream patch branch no? + +The repair therefore had to use downstream #493 as implementation +evidence, avoid duplicating solved TIPC work, preserve the dirty +downstream worktree, and repair the remaining shared, QUIC, +WireGuard, lock, and provenance defects without committing, +pushing, or publishing. + +## Response summary + +Reworked PR #492's plans and executable WireGuard example into a +consistent landing candidate. The shared and TIPC documents now +match current runtime/downstream behavior while naming remaining +risks. The QUIC plan now defines transport ownership across the +whole actor lifecycle. The WireGuard example validates keys and +binds both hosts and the service actor correctly without elevating +Tractor. The dependency lock and historical provenance pointers +are synchronized. + +## Files changed + +- `ai/tpt-backends/00_shared_backend_contract.md` - current backend + contract, dispatch, capability, and locality rules. +- `ai/tpt-backends/01_tipc_backend.md` - #493 reconciliation and + bounded unresolved work. +- `ai/tpt-backends/02_quic_iroh_backend.md` - lifecycle-safe QUIC + and UniFFI implementation plan. +- `ai/tpt-backends/03_wg_tunnel_bindspace.md` - corrected address + registry and privilege semantics. +- `ai/tpt-backends/README.md` - refreshed plan summaries. +- `examples/multihost/wg_lan/README.md` - unprivileged two-host + setup and accurate verification limits. +- `examples/multihost/wg_lan/host_a_srv.py` - local key check, + overlay child bind, and stable RPC exposure. +- `examples/multihost/wg_lan/host_b_client.py` - peer key check, + host-B bind, and explicit missing-service failure. +- `examples/multihost/wg_lan/wg_maddr.py` - strict parsing and + asynchronous role-specific key inspection. +- `uv.lock` - exact `py-multiaddr` Git source resolution. +- `ai/prompt-io/claude/20260813T001102Z_27c34aeb_prompt_io.md` - + valid scope and immutable Git reference. +- `ai/prompt-io/claude/20260813T001102Z_27c34aeb_prompt_io.raw.md` + - immutable historical diff pointers. + +## Human edits + +The human chose the full repair path rather than reducing the PR +to planning documents or publishing the initial review. They also +identified that the initial TIPC review had not accounted for the +downstream implementation branch. That correction materially +changed the work: solved TIPC items were backported into the plan, +remaining defects were separated from implemented behavior, and +the dirty downstream worktree was kept read-only. diff --git a/ai/prompt-io/opencode/20260831T022317Z_768b5316_prompt_io.raw.md b/ai/prompt-io/opencode/20260831T022317Z_768b5316_prompt_io.raw.md new file mode 100644 index 00000000..a41cc032 --- /dev/null +++ b/ai/prompt-io/opencode/20260831T022317Z_768b5316_prompt_io.raw.md @@ -0,0 +1,60 @@ +--- +model: gpt-5.6-sol +service: opencode +timestamp: 2026-08-31T02:23:17Z +git_ref: ng_tpts_planning +diff_cmd: git diff 768b531662a88037e3988885102d2ff24a7ce34e..HEAD +--- + +# Raw output - repair PR #492 for landing + +## Generated changes + +> `git diff 768b531662a88037e3988885102d2ff24a7ce34e..HEAD -- ai/tpt-backends/` + +- Reconciled the shared backend contract with current address, + dispatch, capability, locality, and listener-rebind APIs. +- Updated the TIPC plan from downstream #493 implementation and + tests, preserving unresolved registrar election, collision, + locality, and socket-cleanup work as explicit follow-ups. +- Reworked the QUIC plan around a launch-time transport bootstrap, + one actor-owned endpoint, a transport nursery spanning parent + dial through deregistration, supervised UniFFI cleanup, complete + routable addresses, connection leases, and listener-owned tasks. +- Corrected the WireGuard bindspace plan to distinguish the + build-registered address registry from runtime capability. + +> `git diff 768b531662a88037e3988885102d2ff24a7ce34e..HEAD -- examples/multihost/wg_lan/` + +- Hardened WireGuard key conversion and parsing with strict + 32-byte validation, lazy protocol lookup, and explicit rejection + of unsupported nested tunnel descriptors. +- Made interface inspection asynchronous, bounded at its + cancellation request, role-specific, and separable from + privileged preflight commands. +- Corrected host-local binds, subactor overlay publication, stable + RPC module exposure, and missing-actor handling in the two-host + example. +- Updated the README so Tractor remains unprivileged and the local + versus remote overlay roles are explicit. + +> `git diff 768b531662a88037e3988885102d2ff24a7ce34e..HEAD -- pyproject.toml uv.lock ai/prompt-io/` + +- Regenerated `uv.lock` for the exact merged `py-multiaddr` WireGuard + codec revision. +- Replaced mutable historical prompt diff pointers with immutable + refs and normalized the substantive scope without rewriting the + historical raw response. + +## Verification output + +- `git diff --check`: passed. +- `uv lock --check`: passed. +- Ruff on the WireGuard example directory: passed. +- Python compilation of the WireGuard example directory: passed. +- Tractor imported from the PR worktree's existing environment. +- WireGuard address round-trip, local/peer role checks, malformed + base64 rejection, and nested-tunnel rejection: passed. +- Three independent final re-reviews reported no actionable + findings in the shared/TIPC, QUIC, or WireGuard slices. +- No live TIPC, iroh/UniFFI, or WireGuard network test was run. diff --git a/ai/tpt-backends/03_wg_tunnel_bindspace.md b/ai/tpt-backends/03_wg_tunnel_bindspace.md index 36c975b5..bdb0d579 100644 --- a/ai/tpt-backends/03_wg_tunnel_bindspace.md +++ b/ai/tpt-backends/03_wg_tunnel_bindspace.md @@ -33,7 +33,7 @@ onto `trio` as the library's sans-io layer allows. by multiformats/py-multiaddr#107 and gh #483. - so **today's deployable story is declarative**: run `wg-quick` out-of-band, parse the maddr, strip to the overlay - `(host, port)`, verify the pubkey against the live tunnel, + `(host, port)`, verify the pubkey in its host-specific role, hand the overlay addr to `registry_addrs=`/`tpt_bind_addrs=`. #482 already contains working example code for exactly this. - `Address.namespace` exists in the Protocol @@ -82,9 +82,10 @@ does not create a new address type.** Two candidate encodings; the tunnel spec. The wrapper is stripped (`→ .overlay`) at the moment of bind/connect. - ⚠️ `is_wrapped_addr()` (`_addr.py:194`) tests - `type(addr) in _address_types.values()` — a `bidict` of - proto_key→type. `TunnelledAddress` isn't in it and must not - be (it's not 1:1 with a proto). So either add an explicit + `type(addr) in _address_types.values()` — the build-registered + protocol-key-to-address-type registry. `TunnelledAddress` + isn't in it and must not be (it has no transport of its own). + So either add an explicit `isinstance(addr, TunnelledAddress)` clause there, or give the wrapper a marker and test structurally. Do the former; it's two lines and honest. @@ -103,7 +104,7 @@ class WGTunnelSpec( msgspec.Struct, frozen=True, ): - peer_pubkey: str # std-base64 `wg(8)` form + pubkey: str # std-base64 `wg(8)` form iface: str = 'wg0' netns: str|None = None # layer-C-only fields, unset in layer A @@ -191,9 +192,10 @@ Observed protocol-name lists, for writing the `match`: ...))` w/ the bearer recorded in the spec. - keep the existing 2-proto cases byte-identical; add the new case *after* them. -- nesting (wg-in-wg) falls out of `.decapsulate_code()` cutting - at the *last* occurrence — peel repeatedly rather than - recursing through a bespoke splitter. +- layer A rejects more than one `/wg/` segment. Its wrapper stores + one bearer, key, and overlay, so accepting wg-in-wg would + silently misrepresent the maddr. Nested tunnel support needs a + different data shape and belongs in a later layer. - `mk_maddr()` inverse for `TunnelledAddress` is just `.encapsulate()` composition; don't rebuild `str`s by hand. - **pending an upstream release**: py-multiaddr#108 is merged, so @@ -215,12 +217,21 @@ Port #482 §2's helpers into `tractor/discovery/_tunnel.py` as ```python def parse_wg_maddr(maddr: str) -> TunnelledAddress: ... # pure def wg8_pubkey(multibase_key: str) -> str: ... # pure -def verify_wg_peer(spec: WGTunnelSpec) -> bool: ... # impure probe +async def verify_wg_key( + spec: WGTunnelSpec, + role: Literal['local', 'peer'], + inspection: str|None = None, +) -> bool: ... # impure probe ``` -In layer A `verify_wg_peer()` may shell out (`wg show -peers`), but it must be a *single* function so layer B swaps -only its body. Never call it implicitly from +In layer A `verify_wg_key()` may shell out to role-specific +`wg show public-key|peers` queries, but it must be a *single* +async, time-bounded function so it never blocks trio's run thread +and layer B swaps only its body. It verifies key presence only, +not `Endpoint`, `AllowedIPs`, handshake state, or routing. Never +run `tractor` as root: privileged inspection stays a separate +step whose public-key output can be passed as `inspection`. Never +call it implicitly from `wrap_address()`/`parse_maddr()` — parsing must stay pure and side-effect-free; verification is the *caller's* explicit step (and later, the bindspace `@acm`'s). @@ -281,7 +292,7 @@ Three integration options, in increasing trio-nativeness: - (3) reimplement the codecs. Never. **Recommended split**: ship (1) first so layer B is a small, -reviewable, behaviour-preserving swap of `verify_wg_peer()`'s +reviewable, behaviour-preserving swap of `verify_wg_key()`'s body; then land (2) as a follow-up commit for the read path (`wg get`, `link get`) where the sans-io surface is smallest, and keep (1) for the privileged mutating ops. Measure before @@ -306,7 +317,7 @@ async def read_wg_peers( async def read_wg_pubkey(iface: str = 'wg0', ...) -> str: ... ``` -and `verify_wg_peer()` becomes a thin composition over the two. +and `verify_wg_key()` becomes a thin composition over the two. Note the pure-getter rule: no `read_wg_peers(..., create=True)`. --- diff --git a/examples/multihost/wg_lan/README.md b/examples/multihost/wg_lan/README.md index c8e5ba54..4a1c75f8 100644 --- a/examples/multihost/wg_lan/README.md +++ b/examples/multihost/wg_lan/README.md @@ -49,11 +49,9 @@ uv sync ``` gets you a `wg`-aware `multiaddr`. That pin goes away once a -release carries the codec. You also need `multibase`: - -```bash -uv pip install multibase -``` +release carries the codec. Its `py-multibase` dependency provides +the imported `multibase` module; no separate install command is +needed. Without the codec `parse_wg_maddr()` raises immediately with an actionable message — there is deliberately **no** degraded @@ -64,8 +62,9 @@ tunnel API (`.decapsulate_code()`, `.split()`, `.join()`, `.encapsulate()`, `.value_for_protocol()`) rather than any bespoke segment slicing — see its README "En/decapsulate" and "Tunneling" sections. gh #429 was about *dropping* our NIH -parser, and that applies to peeling a tunnel stack just as much -as to decoding one proto. +parser, and that applies to peeling this composed maddr just as +much as to decoding one proto. This example rejects multiple +`/wg/` segments because `WGTunnelledAddr` stores one tunnel. ## 0. tunnel setup (out-of-band, both hosts) @@ -106,8 +105,11 @@ AllowedIPs = 10.0.11.1/32 PersistentKeepalive = 25 ``` -Note how `ListenPort` and `Endpoint` are exactly the maddr's -bearer segment, and `[Interface] Address` is its overlay host. +This example configures host A's `ListenPort` and host B's +`Endpoint` from the maddr bearer, and configures host A's +`[Interface] Address` from its overlay host. The verification +step below checks keys only; it does not inspect those fields or +either peer's `AllowedIPs`. ```bash sudo wg-quick up wg0 # both hosts @@ -118,16 +120,40 @@ ping -c1 10.0.11.1 # from B ```bash python -c " -import base64, multibase +from wg_maddr import mb_pubkey key = open('wg_pub.key').read().strip() -print(multibase.encode('base64url', base64.b64decode(key)).decode()) +print(mb_pubkey(key)) " ``` Paste the `u...` output into `WG_MADDR` in both scripts (they use the same string — A's bearer, A's key, A's overlay ep). -## 2. run +## 2. verify the keys + +Interface inspection commonly needs `CAP_NET_ADMIN`. Keep that +privileged operation separate from the `tractor` processes: + +```bash +# host A: output must equal the maddr's A_pub key +export WG_KEY_INSPECTION="$(sudo wg show wg0 public-key)" + +# host B: output must contain the maddr's A_pub key +export WG_KEY_INSPECTION="$(sudo wg show wg0 peers)" +``` + +These checks establish only that host A uses the declared local +key and host B has that key as a configured peer. They do not +verify `Endpoint`, `AllowedIPs`, a recent handshake, or routing. +The exported text contains public keys only. Each script passes it +to `verify_wg_key()` with its host-specific role before starting +`tractor`. Callers that already have permission to inspect the +interface may omit that argument; the helper's direct query is +async and requests cancellation after five seconds. Trio's +subprocess termination escalation can make final process cleanup +take longer than that cancellation deadline. + +## 3. run ```bash # host A @@ -137,6 +163,17 @@ python host_a_srv.py python host_b_client.py ``` +Run both `tractor` programs as the normal application account, +not as root. Privilege is needed only for tunnel setup and the +separate inspection above. If using that preflight, keep the +host-specific `WG_KEY_INSPECTION` value exported in each +program's shell. + +The client binds its own actor listener to `10.0.11.2:0`, while +the service actor binds to host A's `10.0.11.1` overlay host with +a random port. Keep `LOCAL_OVERLAY_BIND` aligned with host B's +WireGuard interface address if adapting this example. + `host_a_srv.py` must be importable on host B too, since `portal.run()` refs the fn by module path — standard `tractor` RPC semantics. @@ -153,19 +190,19 @@ Four corrections, all from all. `parse_wg_maddr()` now rejects it with an actionable error. 2. **parsing is pure.** #482's helper had the key-check adjacent - to the parse; `verify_wg_peer()` is now a separate, explicitly - composed step that the caller invokes. A parser that shells - out is a nasty surprise. + to the parse; `verify_wg_key()` is now a separate, explicitly + composed step for inspection-capable callers. A parser that + shells out is a nasty surprise. 3. **no `sudo`.** #482 ran `sudo wg show`; a library/example must - never escalate. `wg show` works unprivileged for read on most - setups; if yours needs root, run the script as root rather - than embedding `sudo`. + never escalate or run `tractor` as root. Privileged tunnel + setup and key inspection are separate shell steps. 4. **no new `Address` proto-type.** The tunnel rides *beside* the overlay addr in a frozen `WGTunnelledAddr`, and only `.overlay` crosses into `open_nursery()`. #482 §6 floated a `WGAddress` - registered in `_address_types` — that table is a `bidict` - (1:1 proto-key↔type) and `_addr_to_transport` wants a - `MsgTransport` per addr-type, which `wg` doesn't have. + registered in `_address_types` — that registry maps available + transport keys to concrete address types, and + `_addr_to_transport` wants a `MsgTransport` per addr-type, + which `wg` doesn't have. ## next diff --git a/examples/multihost/wg_lan/host_a_srv.py b/examples/multihost/wg_lan/host_a_srv.py index 21ccc50a..6d545dbe 100644 --- a/examples/multihost/wg_lan/host_a_srv.py +++ b/examples/multihost/wg_lan/host_a_srv.py @@ -6,14 +6,17 @@ Binds `tractor`'s registrar + an `echo_srv` sub-actor on the tunnel's *overlay* addr, declared as a single `wg` maddr. ''' + from __future__ import annotations +import os + import tractor import trio from wg_maddr import ( parse_wg_maddr, - verify_wg_peer, + verify_wg_key, WGTunnelledAddr, ) @@ -34,11 +37,17 @@ async def echo(msg: str) -> str: async def main(): addr: WGTunnelledAddr = parse_wg_maddr(WG_MADDR) - assert verify_wg_peer(addr), ( - f'wg pubkey from maddr not active on wg0 !\n' - f'maddr: {WG_MADDR}\n' - f'key: {addr.peer_pubkey}\n' - ) + inspection: str | None = os.environ.get('WG_KEY_INSPECTION') + if not await verify_wg_key( + addr, + role='local', + inspection=inspection, + ): + raise RuntimeError( + f'Maddr key is not wg0 local public key!\n' + f'maddr: {WG_MADDR}\n' + f'key: {addr.wg_pubkey}\n' + ) print( f'wg bearer (kernel-owned): {addr.bearer}\n' f'tractor overlay ep: {addr.overlay}\n' @@ -51,7 +60,9 @@ async def main(): ) as an: await an.start_actor( 'echo_srv', - enable_modules=[__name__], + bind_addrs=[(addr.overlay[0], 0)], + enable_transports=[addr.overlay_proto], + enable_modules=['host_a_srv'], ) print(f'echo_srv up on\n {addr.maddr}\n') await trio.sleep_forever() diff --git a/examples/multihost/wg_lan/host_b_client.py b/examples/multihost/wg_lan/host_b_client.py index cd9970be..86e3e59c 100644 --- a/examples/multihost/wg_lan/host_b_client.py +++ b/examples/multihost/wg_lan/host_b_client.py @@ -4,15 +4,18 @@ Host B: workstation dialing host A's actor tree through the `wg` tunnel. ''' + from __future__ import annotations +import os + import tractor import trio from host_a_srv import echo # noqa: F401 (RPC refs it by mod path) from wg_maddr import ( parse_wg_maddr, - verify_wg_peer, + verify_wg_key, WGTunnelledAddr, ) @@ -22,23 +25,33 @@ WG_MADDR: str = ( '/wg/u' '/ip4/10.0.11.1/tcp/1616' ) +LOCAL_OVERLAY_BIND: tuple[str, int] = ('10.0.11.2', 0) async def main(): addr: WGTunnelledAddr = parse_wg_maddr(WG_MADDR) - assert verify_wg_peer(addr), ( - f'wg pubkey from maddr not a peer on wg0 !\n' - f'maddr: {WG_MADDR}\n' - ) + inspection: str | None = os.environ.get('WG_KEY_INSPECTION') + if not await verify_wg_key( + addr, + role='peer', + inspection=inspection, + ): + raise RuntimeError( + f'Maddr key is not a configured wg0 peer!\n' + f'maddr: {WG_MADDR}\n' + f'key: {addr.wg_pubkey}\n' + ) async with ( tractor.open_root_actor( name='wg_client', + tpt_bind_addrs=[LOCAL_OVERLAY_BIND], registry_addrs=[addr.overlay], enable_transports=[addr.overlay_proto], ), tractor.find_actor( 'echo_srv', registry_addrs=[addr.overlay], + raise_on_none=True, ) as portal, ): res: str = await portal.run( diff --git a/examples/multihost/wg_lan/wg_maddr.py b/examples/multihost/wg_lan/wg_maddr.py index 67549568..f6e664ee 100644 --- a/examples/multihost/wg_lan/wg_maddr.py +++ b/examples/multihost/wg_lan/wg_maddr.py @@ -26,7 +26,7 @@ stack "inner" reads as higher-up and later-called, whereas here the encapsulated addr is bound *first* and sits deeper in the maddr — two opposite intuitions on one word. -`/wg/u` itself carries the tunnel peer's Curve25519 pubkey +`/wg/u` itself carries a declared Curve25519 pubkey as multibase base64url (std base64 from `wg(8)` contains `/` and so can't go in a `/`-delimited maddr). It binds nothing at all; it's an identity, verified out-of-band. @@ -43,28 +43,33 @@ Design rules this module follows (see through `.decapsulate_code()`, `.split()`, `.join()`, `.encapsulate()` and `.value_for_protocol()`. We hand-roll no segment splitting whatsoever — the whole point of gh #429 was - dropping the NIH parser, and that applies to *peeling a tunnel - stack* every bit as much as to decoding a single proto. + dropping the NIH parser, and that applies to peeling this + composed maddr every bit as much as to decoding one proto. - **parsing is pure**. `parse_wg_maddr()` does no I/O, no `subprocess`, no netlink. A parser that shells out is a nasty surprise. - **verification is an explicit, separate step**. The caller - composes `verify_wg_peer()` when it wants it; nothing implicit. + composes `verify_wg_key()` when it has permission to inspect + the iface; nothing implicit. +- **exactly one `wg` segment is supported**. `WGTunnelledAddr` + stores one bearer and one key, so accepting another segment + would silently misrepresent the maddr. - **no new `Address` proto-type**. `wg` gets no entry in - `tractor.discovery._addr._address_types` (a `bidict`, so 1:1 - proto-key<->type) bc it has no `MsgTransport` of its own. The + `tractor.discovery._addr._address_types`, which maps available + transport keys to concrete address types, bc it has no + `MsgTransport` of its own. The tunnel is a *bindspace*, so we carry it beside the overlay addr and strip to `.overlay` at bind/dial time. ''' + from __future__ import annotations import base64 -import subprocess from typing import Literal import msgspec from multiaddr import Multiaddr -from multiaddr.protocols import P_WG +import trio IPProto = Literal['ip4', 'ip6'] @@ -76,15 +81,16 @@ class WGTunnelledAddr( ): ''' A `wg`-tunnelled endpoint: the underlay bearer, the tunnel - peer key, and the overlay addr `tractor` binds/dials. + key, and the overlay addr `tractor` binds/dials. ''' + # underlay, owned by `wg(8)`/the kernel — NEVER bound by us bearer: tuple[str, int] - # tunnel peer pubkey in the std-base64 `wg(8)` form, i.e. - # directly comparable to `wg show peers` output - peer_pubkey: str + # declared wg pubkey in std-base64 `wg(8)` form; it is the + # local key on the bearer host and a configured peer on a dialer + wg_pubkey: str # overlay ep: an `UnwrappedAddress` as accepted by # `tractor.discovery.wrap_address()` @@ -108,7 +114,7 @@ class WGTunnelledAddr( return ( Multiaddr(f'/{self.bearer_ip}/{b_host}/udp/{b_port}') .encapsulate( - Multiaddr(f'/wg/{mb_pubkey(self.peer_pubkey)}') + Multiaddr(f'/wg/{mb_pubkey(self.wg_pubkey)}') ) .encapsulate( Multiaddr( @@ -133,7 +139,13 @@ def mb_pubkey(wg8_key: str) -> str: ''' import multibase - raw: bytes = base64.b64decode(wg8_key) + + raw: bytes = base64.b64decode(wg8_key, validate=True) + if len(raw) != 32: + raise ValueError( + f'WireGuard public keys must decode to 32 bytes, ' + f'not {len(raw)}' + ) return multibase.encode('base64url', raw).decode('ascii') @@ -143,11 +155,17 @@ def wg8_pubkey(mb_key: str) -> str: ''' import multibase + raw: bytes = multibase.decode(mb_key) + if len(raw) != 32: + raise ValueError( + f'WireGuard public keys must decode to 32 bytes, ' + f'not {len(raw)}' + ) return base64.b64encode(raw).decode('ascii') -_wg_proto_known: bool|None = None +_wg_proto_known: bool | None = None def _have_wg_maddr_proto() -> bool: @@ -167,6 +185,7 @@ def _have_wg_maddr_proto() -> bool: if _wg_proto_known is None: from multiaddr.protocols import protocol_with_name from multiaddr.exceptions import ProtocolNotFoundError + try: protocol_with_name('wg') _wg_proto_known = True @@ -177,7 +196,7 @@ def _have_wg_maddr_proto() -> bool: def parse_wg_maddr( - maddr: str|Multiaddr, + maddr: str | Multiaddr, ) -> WGTunnelledAddr: ''' Peel a `wg`-tunnelled maddr into its bearer/key/overlay @@ -208,18 +227,31 @@ def parse_wg_maddr( for seg in segs for proto in seg.protocols() ] - if 'wg' not in names: + wg_count: int = names.count('wg') + if not wg_count: raise ValueError( f'Not a `wg`-tunnelled maddr, no `/wg/` segment ??\n' f'maddr: {ma}\n' ) + if wg_count > 1: + raise ValueError( + f'Nested `wg` segments are not supported; ' + f'`WGTunnelledAddr` stores one tunnel only.\n' + f'maddr: {ma}\n' + ) + + # Resolve the unreleased protocol only after the capability + # check, so importing this module works with released multiaddr. + from multiaddr.protocols import protocol_with_name + + wg_code: int = protocol_with_name('wg').code # NOTE, `.decapsulate_code()` cuts at the LAST occurrence of # the proto and keeps the *prefix*, which is exactly the # bearer. It handles `/wg/` cleanly precisely bc it cuts on # proto-code and never tries to match an addr value — the # key seg has no addr of its own. - bearer_ma: Multiaddr = ma.decapsulate_code(P_WG) + bearer_ma: Multiaddr = ma.decapsulate_code(wg_code) overlay_ma: Multiaddr = Multiaddr.join( *segs[names.index('wg') + 1:] ) @@ -259,7 +291,7 @@ def parse_wg_maddr( return WGTunnelledAddr( bearer=bearer, - peer_pubkey=wg8_pubkey(ma.value_for_protocol('wg')), + wg_pubkey=wg8_pubkey(ma.value_for_protocol('wg')), overlay=overlay, overlay_proto=l4, bearer_ip=b_ip, @@ -267,13 +299,23 @@ def parse_wg_maddr( ) -def verify_wg_peer( +async def verify_wg_key( addr: WGTunnelledAddr, + role: Literal['local', 'peer'], iface: str = 'wg0', + timeout: float = 5, + inspection: str | None = None, ) -> bool: ''' - True iff `addr.peer_pubkey` is a configured peer (or our own - pubkey) on `iface`. + Verify the declared key in the role required on this host. + + A bearer host uses `role='local'`; a dialer uses `role='peer'`. + This verifies only key presence. It does not inspect the peer's + endpoint, AllowedIPs, handshake state, or iface addresses. + + `inspection` accepts output captured by a separate privileged + `wg show` step. Without it, query asynchronously for callers + which already have interface-inspection permission. IMPURE + explicit by design: never called from `parse_wg_maddr()`. @@ -284,16 +326,23 @@ def verify_wg_peer( netns unless `netns=` is passed down. ''' - def _wg(*args: str) -> str: - return subprocess.run( - ['wg', 'show', iface, *args], - capture_output=True, - text=True, - check=True, - ).stdout + match role: + case 'local': + field = 'public-key' + case 'peer': + field = 'peers' + case _: + raise ValueError(f'Unknown WireGuard key role: {role!r}') - return ( - addr.peer_pubkey in _wg('peers').split() - or - addr.peer_pubkey == _wg('public-key').strip() - ) + if inspection is None: + with trio.fail_after(timeout): + proc = await trio.run_process( + ['wg', 'show', iface, field], + capture_stdout=True, + check=True, + ) + inspection = proc.stdout.decode() + + if role == 'local': + return addr.wg_pubkey == inspection.strip() + return addr.wg_pubkey in inspection.split() diff --git a/uv.lock b/uv.lock index fb63d5ce..8a033627 100644 --- a/uv.lock +++ b/uv.lock @@ -273,9 +273,9 @@ name = "greenback" version = "1.2.1" source = { registry = "https://pypi.org/simple" } dependencies = [ - { name = "greenlet", marker = "python_full_version < '3.14'" }, - { name = "outcome", marker = "python_full_version < '3.14'" }, - { name = "sniffio", marker = "python_full_version < '3.14'" }, + { name = "greenlet" }, + { name = "outcome" }, + { name = "sniffio" }, ] sdist = { url = "https://files.pythonhosted.org/packages/dc/c1/ab3a42c0f3ed56df9cd33de1539b3198d98c6ccbaf88a73d6be0b72d85e0/greenback-1.2.1.tar.gz", hash = "sha256:de3ca656885c03b96dab36079f3de74bb5ba061da9bfe3bb69dccc866ef95ea3", size = 42597, upload-time = "2024-02-20T21:23:13.239Z" } wheels = [ @@ -518,7 +518,7 @@ wheels = [ [[package]] name = "multiaddr" version = "0.2.0" -source = { registry = "https://pypi.org/simple" } +source = { git = "https://github.com/multiformats/py-multiaddr.git?rev=f86519daaa21699023d0037c58cdff600313dd09#f86519daaa21699023d0037c58cdff600313dd09" } dependencies = [ { name = "base58" }, { name = "dnspython" }, @@ -533,10 +533,6 @@ dependencies = [ { name = "trio-typing" }, { name = "varint" }, ] -sdist = { url = "https://files.pythonhosted.org/packages/c7/10/4e26a8577cfce1c0febc8d83087e1373e93c695c6e73ad010546fb67e229/multiaddr-0.2.0.tar.gz", hash = "sha256:acb6b25c332ec1b2f1f8fef8d03a8c63385d34a87d690df0f4bba43cdf6efe8d", size = 58356, upload-time = "2026-03-17T21:51:00.274Z" } -wheels = [ - { url = "https://files.pythonhosted.org/packages/b5/13/56e503d01218d1ca27ea9fda862045a4b400cae5e756f47315f5aaba0eee/multiaddr-0.2.0-py3-none-any.whl", hash = "sha256:bcff7bf3d7de3d6da0b865b25423bcb411de1d20d70cc6abfacf75170d17866c", size = 40424, upload-time = "2026-03-17T21:50:58.833Z" }, -] [[package]] name = "mypy-extensions" @@ -1193,7 +1189,7 @@ requires-dist = [ { name = "bidict", specifier = ">=0.23.1" }, { name = "colorlog", specifier = ">=6.8.2,<7" }, { name = "msgspec", specifier = ">=0.20.0" }, - { name = "multiaddr", specifier = ">=0.2.0" }, + { name = "multiaddr", git = "https://github.com/multiformats/py-multiaddr.git?rev=f86519daaa21699023d0037c58cdff600313dd09" }, { name = "pdbp", specifier = ">=1.8.2,<2" }, { name = "platformdirs", specifier = ">=4.4.0" }, { name = "setproctitle", specifier = ">=1.3,<2" },