From d9a6e2e9b4213bb0900b99deda851cb2eaaa2b1b Mon Sep 17 00:00:00 2001 From: goodboy Date: Mon, 17 Aug 2026 17:25:20 -0400 Subject: [PATCH] Retract the hand-rolled tunnel peeler from plan-03 MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit §3.2 specced a pure fn `_peel_tunnel_segs(proto_names) -> (bearer_names, tunnel_specs, overlay_names)` to split a maddr at its tunnel seg. It should never be written: `py-multiaddr` ships that whole surface already and the plan simply missed it, even though gh #443's 2nd bullet links the README sections in question. Replaced w/ a ⚠️ CORRECTION carrying the verified API table (`.decapsulate_code(P_WG)` for the bearer, `.split()`/`.join()` for a seg tail, `.value_for_protocol()` to read a value, `.encapsulate()` to recompose) plus *why* it works on an infix `/wg/` seg: the cut is by proto-code, never by matching an addr value, and the key seg has no addr of its own. Also, - adopt `bearer`/`overlay` as the role names throughout, and say plainly why not `inner`/`outer` — the call-stack reading of "inner" is the exact opposite of the encapsulation one. - warn that `value_for_protocol('ip4')` on a full tunnelled maddr silently yields the *bearer's* host; only call it on a peeled sub-maddr. - note nesting (wg-in-wg) falls out of `.decapsulate_code()` cutting at the *last* occurrence, so peel repeatedly rather than recursing through a bespoke splitter. - `mk_maddr()` for `TunnelledAddress` is `.encapsulate()` composition, not `str` building. - README: drop the "degrades to a plain segment split" para, since that path is gone — no codec now means one actionable raise. (this patch was generated in some part by `claude-code` using `claude-opus-5` (`anthropic`)) --- ai/tpt-backends/03_wg_tunnel_bindspace.md | 69 +++++++++++++++-------- examples/multihost/wg_lan/README.md | 23 +++++--- 2 files changed, 60 insertions(+), 32 deletions(-) diff --git a/ai/tpt-backends/03_wg_tunnel_bindspace.md b/ai/tpt-backends/03_wg_tunnel_bindspace.md index ffb0dfd3..36c975b5 100644 --- a/ai/tpt-backends/03_wg_tunnel_bindspace.md +++ b/ai/tpt-backends/03_wg_tunnel_bindspace.md @@ -32,9 +32,9 @@ onto `trio` as the library's sans-io layer allows. latest `0.2.0` predating it. Spec registration is still tracked by multiformats/py-multiaddr#107 and gh #483. - so **today's deployable story is declarative**: run `wg-quick` - out-of-band, parse the maddr, strip to the inner + out-of-band, parse the maddr, strip to the overlay `(host, port)`, verify the pubkey against the live tunnel, - hand the inner addr to `registry_addrs=`/`tpt_bind_addrs=`. + hand the overlay addr to `registry_addrs=`/`tpt_bind_addrs=`. #482 already contains working example code for exactly this. - `Address.namespace` exists in the Protocol (`_addr.py:94-101`, "the if-available OS-specific network @@ -45,7 +45,7 @@ onto `trio` as the library's sans-io layer allows. | layer | what | dep | ships | | --- | --- | --- | --- | -| **A. declarative** | commit #482's examples; `parse_maddr()` learns `/wg/u` → inner `Address` + verified pubkey | `multiaddr` (already), `wg(8)` CLI | first | +| **A. declarative** | commit #482's examples; `parse_maddr()` learns `/wg/u` → overlay `Address` + verified pubkey | `multiaddr` (already), `wg(8)` CLI | first | | **B. `pyroute2` read/verify** | replace the `subprocess.run(['sudo','wg','show'])` shelling with netlink queries | `pyroute2` extra | second | | **C. `@acm` lifecycle** | create/configure/tear down wg ifaces + netns *from the runtime*, as nested bindspaces; implement `Address.namespace` | `pyroute2` + `CAP_NET_ADMIN` | third | @@ -70,16 +70,16 @@ does not create a new address type.** Two candidate encodings; msgspec.Struct, frozen=True, ): - inner: Address # e.g. TCPAddress + overlay: Address # e.g. TCPAddress tunnel: WGTunnelSpec # proto-specific, frozen ``` - with `.proto_key` **delegating to `inner.proto_key`** so every + with `.proto_key` **delegating to `overlay.proto_key`** so every existing table lookup (`_addr_to_transport`, `enable_transports` guard at `_root.py:391`, `transport_from_addr()`) keeps working untouched, and - `.unwrap()` delegating to `inner.unwrap()` so **nothing new + `.unwrap()` delegating to `overlay.unwrap()` so **nothing new crosses the wire**. `.namespace` and `.bindspace` come from - the tunnel spec. The wrapper is stripped (`→ .inner`) at the + the tunnel spec. The wrapper is stripped (`→ .overlay`) at the moment of bind/connect. - ⚠️ `is_wrapped_addr()` (`_addr.py:194`) tests `type(addr) in _address_types.values()` — a `bidict` of @@ -154,25 +154,48 @@ Observed protocol-name lists, for writing the `match`: | --- | --- | --- | | bearer | kernel, via `wg-quick`/`pyroute2` | no | | `/wg/u` | nothing — it's an identity | no, verified out-of-band | - | overlay | `tractor`'s `IPCServer` | **yes**, as `.inner` | + | overlay | `tractor`'s `IPCServer` | **yes**, as `.overlay` | This owner-split is the real axis of the design, *not* whether the maddr stack is "composed" (it is). +- ⚠️ **CORRECTION**, an earlier draft of this section specced a + hand-rolled `_peel_tunnel_segs(proto_names) -> (bearer_names, + tunnel_specs, overlay_names)`. **Do not write it.** + `py-multiaddr` already ships the whole tunnel compose/peel API + and it was simply missed here — see its README "En/decapsulate" + and "Tunneling" sections, and gh #443's 2nd bullet which links + them. Verified against the pinned rev: + + | need | API | + | --- | --- | + | isolate the bearer | `ma.decapsulate_code(P_WG)` | + | drop the overlay, keep bearer+key | `ma.decapsulate(overlay_ma)` | + | per-seg maddrs | `ma.split()` | + | rejoin a seg tail | `Multiaddr.join(*segs)` | + | read the key | `ma.value_for_protocol('wg')` | + | recompose | `bearer.encapsulate(key).encapsulate(overlay)` | + + `.decapsulate_code()` handles the infix `/wg/` seg cleanly + *because* it cuts on proto-code and never tries to match an + addr value — the key seg has no addr of its own. This is the + same NIH trap gh #429 existed to close, one layer up. + +- ⚠️ `value_for_protocol('ip4')` on a *full* tunnelled maddr + silently returns the **first** match, i.e. the bearer's host. + Always call it on a peeled sub-maddr, never the whole stack. + - `parse_maddr()` gains a case on - `[('ip4'|'ip6'), 'udp', 'wg', ('ip4'|'ip6'), ]` → - build the inner `Address` from the trailing segments, decode - the multibase key to std-base64, and return - `TunnelledAddress(inner=..., tunnel=WGTunnelSpec(...))` with - the bearer recorded in the spec. + `[('ip4'|'ip6'), 'udp', 'wg', ('ip4'|'ip6'), ]` → + peel w/ the API above, decode the multibase key to std-base64, + and return `TunnelledAddress(overlay=..., tunnel=WGTunnelSpec( + ...))` w/ the bearer recorded in the spec. - keep the existing 2-proto cases byte-identical; add the new case *after* them. -- generalize by **peeling at the tunnel segment**: split - `proto_names` at `'wg'`, hand the trailing list to the existing - inner-stack logic, and recurse for nested tunnels. Write it as - a small pure fn `_peel_tunnel_segs(proto_names) -> - (bearer_names, tunnel_specs, inner_names)`. This is also what - makes a wg-inside-wg stack fall out for free. -- `mk_maddr()` inverse for `TunnelledAddress`. +- nesting (wg-in-wg) falls out of `.decapsulate_code()` cutting + at the *last* occurrence — peel repeatedly rather than + recursing through a bespoke splitter. +- `mk_maddr()` inverse for `TunnelledAddress` is just + `.encapsulate()` composition; don't rebuild `str`s by hand. - **pending an upstream release**: py-multiaddr#108 is merged, so `Multiaddr('/…/wg/u…')` parses — but off a `[tool.uv.sources]` `rev` pin, since no release carries the codec. Gate the tests @@ -217,7 +240,7 @@ side-effect-free; verification is the *caller's* explicit step run. Keep prose in the docs; keep the examples runnable and minimal. - tests: maddr round-trip, `TunnelledAddress` delegation - (`proto_key`/`unwrap` identical to inner), `wrap_address()` + (`proto_key`/`unwrap` identical to overlay), `wrap_address()` regression (a tunnelled maddr `str` → `TunnelledAddress`; a plain one → unchanged), and **a real end-to-end over a locally-created wg pair** gated on `CAP_NET_ADMIN` (see §5.3). @@ -305,7 +328,7 @@ async def open_bindspace( ) -> AsyncGenerator[Address, None]: ''' Enter the net-bindspace implied by `addr`'s tunnel stack, - yielding the *inner* `Address` ready to bind/connect. + yielding the *overlay* `Address` ready to bind/connect. Nests: one `@acm` per tunnel segment, outermost-first, so a 2-deep stack is just two nested `async with`s and the @@ -444,7 +467,7 @@ consider doing it *first* for exactly that reason. | risk | mitigation | | --- | --- | | `to_thread` worker runs in the wrong netns | §5.3; pass `netns=` to pyroute2 or pin a worker; test-first | -| py-multiaddr#108 merged but unreleased | `[tool.uv.sources]` `rev` pin + `_have_wg_maddr_proto()` gate; layer A's inner-addr path works regardless | +| py-multiaddr#108 merged but unreleased | `[tool.uv.sources]` `rev` pin + `_have_wg_maddr_proto()` gate; layer A's overlay-addr path works regardless | | `TunnelledAddress` leaks into `Endpoint` and breaks `inspect.getmodule()` | unwrap at parse/bindspace boundary; assert `not isinstance(ep.addr, TunnelledAddress)` in `Endpoint.__post_init__` | | privileged ops in a library | never `sudo`; explicit cap probe + actionable error; pre-provisioned is the default | | pyroute2 0.9 asyncio core drags a loop into the actor | option (1) is a *thread*, not a loop; forbid `trio-asyncio` here (§4.1) | diff --git a/examples/multihost/wg_lan/README.md b/examples/multihost/wg_lan/README.md index 8bea3344..c8e5ba54 100644 --- a/examples/multihost/wg_lan/README.md +++ b/examples/multihost/wg_lan/README.md @@ -30,7 +30,7 @@ Three parts, three different owners: | --- | --- | --- | | `/ip4/../udp/51820` bearer | kernel via `wg-quick`/`pyroute2` | no | | `/wg/u` | nothing — it's an identity | no, verified out-of-band | -| `/ip4/../tcp/1616` overlay | `tractor`'s `IPCServer` | **yes**, as `.inner` | +| `/ip4/../tcp/1616` overlay | `tractor`'s `IPCServer` | **yes**, as `.overlay` | Verified against py-multiaddr [#108](https://github.com/multiformats/py-multiaddr/pull/108): @@ -55,12 +55,17 @@ release carries the codec. You also need `multibase`: uv pip install multibase ``` -Without the codec `wg_maddr.py` degrades to a plain segment split -— the examples still run, but you lose per-segment validation -(incl. the 32-byte key-length check), so a malformed key reaches -the returned struct instead of raising. `_have_wg_maddr_proto()` -is the gate. It deliberately does **not** hand-roll a `wg` codec -(gh #429 was about *dropping* our NIH parser). +Without the codec `parse_wg_maddr()` raises immediately with an +actionable message — there is deliberately **no** degraded +hand-split fallback. `_have_wg_maddr_proto()` is the predicate. + +Every peel and re-compose here goes through `py-multiaddr`'s own +tunnel API (`.decapsulate_code()`, `.split()`, `.join()`, +`.encapsulate()`, `.value_for_protocol()`) rather than any +bespoke segment slicing — see its README "En/decapsulate" and +"Tunneling" sections. gh #429 was about *dropping* our NIH +parser, and that applies to peeling a tunnel stack just as much +as to decoding one proto. ## 0. tunnel setup (out-of-band, both hosts) @@ -156,7 +161,7 @@ Four corrections, all from setups; if yours needs root, run the script as root rather than embedding `sudo`. 4. **no new `Address` proto-type.** The tunnel rides *beside* the - inner addr in a frozen `WGTunnelledAddr`, and only `.inner` + overlay addr in a frozen `WGTunnelledAddr`, and only `.overlay` crosses into `open_nursery()`. #482 §6 floated a `WGAddress` registered in `_address_types` — that table is a `bidict` (1:1 proto-key↔type) and `_addr_to_transport` wants a @@ -166,6 +171,6 @@ Four corrections, all from `WGTunnelledAddr` is deliberately example-local. Promoting it to `tractor.discovery` as a `TunnelledAddress` whose -`.proto_key`/`.unwrap()` delegate to `.inner`, plus +`.proto_key`/`.unwrap()` delegate to `.overlay`, plus `open_bindspace()` `@acm`s that create/tear down the iface + netns via `pyroute2`, is layers A→C of the plan doc.