diff --git a/ai/tpt-backends/03_wg_tunnel_bindspace.md b/ai/tpt-backends/03_wg_tunnel_bindspace.md index bc754a4c..9d71acba 100644 --- a/ai/tpt-backends/03_wg_tunnel_bindspace.md +++ b/ai/tpt-backends/03_wg_tunnel_bindspace.md @@ -22,24 +22,23 @@ onto `trio` as the library's sans-io layer allows. ## 1. What exists today (verified, per #482) - `wrap_address()` accepts maddr `str`s (leading-`/` dispatch, - `_addr.py:262`) but `parse_maddr()` only knows - `/ip4|ip6//tcp/

` and `/unix/

`; a `.../wg/u` - maddr raises `ValueError('Unsupported multiaddr protocol - combo')`. + `_addr.py:262`). `parse_maddr()` and `mk_maddr()` support plain + TCP/UDS addresses plus nested, canonical bearer-first `/wg/` + stacks represented locally as `TunnelledAddress` wrappers. - there is no `wg` proto in the multiaddr *spec* yet, but multiformats/py-multiaddr#108 (key form `u`) is **merged** as of 2026-07-28 (`f86519da`) — and unreleased, the latest `0.2.0` predating it. Spec registration is still tracked by multiformats/py-multiaddr#107 and gh #483. -- so **today's deployable story is declarative**: run `wg-quick` - out-of-band, parse the maddr, strip to the overlay +- **today's deployable story remains declarative**: run `wg-quick` + out-of-band, parse the maddr, strip its wrapper to the overlay `(host, port)`, verify the pubkey against the live tunnel, hand the overlay addr to `registry_addrs=`/`tpt_bind_addrs=`. #482 already contains working example code for exactly this. - `Address.namespace` exists in the Protocol (`_addr.py:94-101`, "the if-available OS-specific network - namespace key") and **no backend implements it**. This plan is - its first consumer. + namespace key"). `TunnelledAddress` implements it from its spec; + no concrete transport backend implements it yet. ## 2. Three layers, three PRs @@ -197,9 +196,9 @@ Observed protocol-name lists, for writing the `match`: - `mk_maddr()` inverse for `TunnelledAddress` is just `.encapsulate()` composition; don't rebuild `str`s by hand. - **pending an upstream release**: py-multiaddr#108 is merged, so - `Multiaddr('/…/wg/u…')` parses — but off a `[tool.uv.sources]` - `rev` pin, since no release carries the codec. Gate the tests - on `_have_wg_maddr_proto()`, implemented as + `Multiaddr('/…/wg/u…')` parses off a PEP 621 direct-revision pin, + since no release carries the codec. Gate parser entry on + `_wg_proto_code()`, implemented as `protocols.protocol_with_name('wg')` under `except ProtocolNotFoundError`. Do **not** probe by parsing a dummy like `Multiaddr('/wg/uAAAA')` — the codec enforces a @@ -209,13 +208,14 @@ Observed protocol-name lists, for writing the `match`: ### 3.3 verification helper (pure, composable) -Port #482 §2's helpers into `tractor/discovery/_tunnel.py` as -*pure functions* + one impure probe, cleanly separated: +Port #482 §2's pure helpers into +`tractor/discovery/_tunnel.py`, keeping the impure probe cleanly +separated until layer B: ```python def parse_wg_maddr(maddr: str) -> TunnelledAddress: ... # pure def wg8_pubkey(multibase_key: str) -> str: ... # pure -def verify_wg_peer(spec: WGTunnelSpec) -> bool: ... # impure probe +def verify_wg_peer(spec: WGTunnelSpec) -> bool: ... # layer B ``` In layer A `verify_wg_peer()` may shell out (`wg show @@ -475,7 +475,7 @@ consider doing it *first* for exactly that reason. | risk | mitigation | | --- | --- | | `to_thread` worker runs in the wrong netns | §5.3; pass `netns=` to pyroute2 or pin a worker; test-first | -| py-multiaddr#108 merged but unreleased | `[tool.uv.sources]` `rev` pin + `_have_wg_maddr_proto()` gate; layer A's overlay-addr path works regardless | +| py-multiaddr#108 merged but unreleased | PEP 621 direct-revision pin + `_wg_proto_code()` gate; replace with a release floor once published | | `TunnelledAddress` leaks into `Endpoint` and breaks `inspect.getmodule()` | unwrap at parse/bindspace boundary; assert `not isinstance(ep.addr, TunnelledAddress)` in `Endpoint.__post_init__` | | privileged ops in a library | never `sudo`; explicit cap probe + actionable error; pre-provisioned is the default | | pyroute2 0.9 asyncio core drags a loop into the actor | option (1) is a *thread*, not a loop; forbid `trio-asyncio` here (§4.1) | diff --git a/examples/multihost/wg_lan/README.md b/examples/multihost/wg_lan/README.md index c8e5ba54..36fe080a 100644 --- a/examples/multihost/wg_lan/README.md +++ b/examples/multihost/wg_lan/README.md @@ -21,16 +21,16 @@ Supersedes the example set in gh /ip4/192.168.1.50/udp/51820/wg/u/ip4/10.0.11.1/tcp/1616 \____ wg bearer ___________/\__ key __/\____ tractor ep _____/ underlay, wg `ListenPort` overlay, on the wg iface - (kernel/`wg(8)` owns it) (the ONLY part tractor binds) + (kernel owns the socket) (`MsgTransport` binds this) ``` Three parts, three different owners: -| part | who binds it | in the runtime? | +| part | socket owner / provisioner | runtime role | | --- | --- | --- | -| `/ip4/../udp/51820` bearer | kernel via `wg-quick`/`pyroute2` | no | -| `/wg/u` | nothing — it's an identity | no, verified out-of-band | -| `/ip4/../tcp/1616` overlay | `tractor`'s `IPCServer` | **yes**, as `.overlay` | +| `/ip4/../udp/51820` bearer | kernel-owned; `wg-quick` now, tractor bindspace later | control-plane metadata | +| `/wg/u` | nothing — it's an identity | parsed, verified explicitly | +| `/ip4/../tcp/1616` overlay | `tractor`'s `IPCServer` | application `MsgTransport` | Verified against py-multiaddr [#108](https://github.com/multiformats/py-multiaddr/pull/108): @@ -41,23 +41,20 @@ this composed form parses and round-trips py-multiaddr #108 is **merged** (2026-07-28) but ships in no release yet — the latest `0.2.0` (2026-03-17) predates it and has -no `wg` codec. So `pyproject.toml` carries a temporary -`[tool.uv.sources]` `rev` pin at the merge commit, and a plain +no `wg` codec. So `pyproject.toml` temporarily pins the merge commit +in its PEP 621 dependency metadata, and a plain ```bash uv sync ``` gets you a `wg`-aware `multiaddr`. That pin goes away once a -release carries the codec. You also need `multibase`: - -```bash -uv pip install multibase -``` +release carries the codec. `py-multibase` is a direct dependency. Without the codec `parse_wg_maddr()` raises immediately with an actionable message — there is deliberately **no** degraded -hand-split fallback. `_have_wg_maddr_proto()` is the predicate. +hand-split fallback. `_wg_proto_code()` performs the capability +check before parsing. Every peel and re-compose here goes through `py-multiaddr`'s own tunnel API (`.decapsulate_code()`, `.split()`, `.join()`, @@ -118,9 +115,9 @@ ping -c1 10.0.11.1 # from B ```bash python -c " -import base64, multibase -key = open('wg_pub.key').read().strip() -print(multibase.encode('base64url', base64.b64decode(key)).decode()) + from tractor.discovery import mb_pubkey + key = open('wg_pub.key').read().strip() + print(mb_pubkey(key)) " ``` @@ -161,7 +158,7 @@ Four corrections, all from setups; if yours needs root, run the script as root rather than embedding `sudo`. 4. **no new `Address` proto-type.** The tunnel rides *beside* the - overlay addr in a frozen `WGTunnelledAddr`, and only `.overlay` + overlay addr in a frozen `TunnelledAddress`, and only `.overlay` crosses into `open_nursery()`. #482 §6 floated a `WGAddress` registered in `_address_types` — that table is a `bidict` (1:1 proto-key↔type) and `_addr_to_transport` wants a @@ -169,8 +166,7 @@ Four corrections, all from ## next -`WGTunnelledAddr` is deliberately example-local. Promoting it to -`tractor.discovery` as a `TunnelledAddress` whose -`.proto_key`/`.unwrap()` delegate to `.overlay`, plus -`open_bindspace()` `@acm`s that create/tear down the iface + -netns via `pyroute2`, is layers A→C of the plan doc. +Layer A's `TunnelledAddress` and native maddr parser now live in +`tractor.discovery`. Next, replace this example's `wg(8)` verification +probe with `pyroute2`, then add `open_bindspace()` `@acm`s which +create/tear down the iface and netns. diff --git a/examples/multihost/wg_lan/host_a_srv.py b/examples/multihost/wg_lan/host_a_srv.py index 21ccc50a..297b0389 100644 --- a/examples/multihost/wg_lan/host_a_srv.py +++ b/examples/multihost/wg_lan/host_a_srv.py @@ -10,13 +10,14 @@ from __future__ import annotations import tractor import trio - -from wg_maddr import ( +from tractor.discovery import ( + TunnelledAddress, + mk_maddr, parse_wg_maddr, - verify_wg_peer, - WGTunnelledAddr, ) +from wg_maddr import verify_wg_peer + # bearer = host A's underlay `(ip, wg ListenPort)` # key = host A's OWN tunnel pubkey # overlay = the ep `tractor` binds, on the wg iface's addr @@ -33,27 +34,27 @@ async def echo(msg: str) -> str: async def main(): - addr: WGTunnelledAddr = parse_wg_maddr(WG_MADDR) + addr: TunnelledAddress = parse_wg_maddr(WG_MADDR) assert verify_wg_peer(addr), ( f'wg pubkey from maddr not active on wg0 !\n' f'maddr: {WG_MADDR}\n' - f'key: {addr.peer_pubkey}\n' + f'key: {addr.tunnel.peer_pubkey}\n' ) print( - f'wg bearer (kernel-owned): {addr.bearer}\n' + f'wg bearer (kernel-owned): {addr.tunnel.bearer}\n' f'tractor overlay ep: {addr.overlay}\n' ) async with tractor.open_nursery( # XXX only `.overlay` crosses into the runtime; the bearer - # + key are iface-layer concerns `tractor` never binds. + # + key are bindspace metadata, never `Endpoint` addrs. registry_addrs=[addr.overlay], - enable_transports=[addr.overlay_proto], + enable_transports=[addr.overlay.proto_key], ) as an: await an.start_actor( 'echo_srv', enable_modules=[__name__], ) - print(f'echo_srv up on\n {addr.maddr}\n') + print(f'echo_srv up on\n {mk_maddr(addr)}\n') await trio.sleep_forever() diff --git a/examples/multihost/wg_lan/host_b_client.py b/examples/multihost/wg_lan/host_b_client.py index cd9970be..2c5d61f3 100644 --- a/examples/multihost/wg_lan/host_b_client.py +++ b/examples/multihost/wg_lan/host_b_client.py @@ -8,13 +8,13 @@ from __future__ import annotations import tractor import trio +from tractor.discovery import ( + TunnelledAddress, + parse_wg_maddr, +) from host_a_srv import echo # noqa: F401 (RPC refs it by mod path) -from wg_maddr import ( - parse_wg_maddr, - verify_wg_peer, - WGTunnelledAddr, -) +from wg_maddr import verify_wg_peer # same maddr as host A: A's bearer, A's key, A's overlay ep WG_MADDR: str = ( @@ -25,7 +25,7 @@ WG_MADDR: str = ( async def main(): - addr: WGTunnelledAddr = parse_wg_maddr(WG_MADDR) + addr: TunnelledAddress = parse_wg_maddr(WG_MADDR) assert verify_wg_peer(addr), ( f'wg pubkey from maddr not a peer on wg0 !\n' f'maddr: {WG_MADDR}\n' @@ -34,7 +34,7 @@ async def main(): tractor.open_root_actor( name='wg_client', registry_addrs=[addr.overlay], - enable_transports=[addr.overlay_proto], + enable_transports=[addr.overlay.proto_key], ), tractor.find_actor( 'echo_srv', diff --git a/examples/multihost/wg_lan/wg_maddr.py b/examples/multihost/wg_lan/wg_maddr.py index 67549568..28b36dac 100644 --- a/examples/multihost/wg_lan/wg_maddr.py +++ b/examples/multihost/wg_lan/wg_maddr.py @@ -1,289 +1,53 @@ # tractor: distributed structured concurrency. r''' -Parse `wg`-tunnelled multiaddrs into `tractor`-ready addrs. +Verify `wg` peers declared by tractor's multiaddr parser. -The canonical form (per py-multiaddr #108, verified against its -upstream merge) nests the *overlay* endpoint **after** the `/wg/` -segment: +`tractor.discovery.parse_wg_maddr()` owns pure parsing and delegates +all tunnel peeling to `py-multiaddr`. This example keeps only the +explicit impure probe used by the two-host demo; parsing never shells +out or verifies local interface state implicitly. + +The canonical maddr form is: /ip4/10.0.0.1/udp/51820/wg/u/ip4/10.0.11.1/tcp/1616 \_______ wg bearer ______/\_ key _/\____ tractor ep _____/ - (underlay, wg - `ListenPort`) -Naming follows `py-multiaddr`'s own encapsulation model, where -earlier segments *wrap* later ones (`.encapsulate()` appends), so -the two roles are: - -- **bearer**: the segs *before* `/wg/`, i.e. the underlay - `(ip, udp-port)` that `wg(8)` itself listens on. Nothing in - `tractor` ever binds this — the kernel/`wg` iface owns it. -- **overlay**: the segs *after*, i.e. the addr `tractor` actually - binds/dials. The only part the runtime ever sees. - -We deliberately avoid `inner`/`outer` for these two: in a *call* -stack "inner" reads as higher-up and later-called, whereas here -the encapsulated addr is bound *first* and sits deeper in the -maddr — two opposite intuitions on one word. - -`/wg/u` itself carries the tunnel peer's Curve25519 pubkey -as multibase base64url (std base64 from `wg(8)` contains `/` and -so can't go in a `/`-delimited maddr). It binds nothing at all; -it's an identity, verified out-of-band. - -XXX NOTE, `tractor`'s own `parse_maddr()` can't parse this yet -(`ValueError('Unsupported multiaddr protocol combo')`), which is -why this module exists: peel here, hand `.overlay` to the -runtime. - -Design rules this module follows (see -`ai/tpt-backends/03_wg_tunnel_bindspace.md`): - -- **let `py-multiaddr` do the parsing**. Every peel/compose goes - through `.decapsulate_code()`, `.split()`, `.join()`, - `.encapsulate()` and `.value_for_protocol()`. We hand-roll no - segment splitting whatsoever — the whole point of gh #429 was - dropping the NIH parser, and that applies to *peeling a tunnel - stack* every bit as much as to decoding a single proto. -- **parsing is pure**. `parse_wg_maddr()` does no I/O, no - `subprocess`, no netlink. A parser that shells out is a nasty - surprise. -- **verification is an explicit, separate step**. The caller - composes `verify_wg_peer()` when it wants it; nothing implicit. -- **no new `Address` proto-type**. `wg` gets no entry in - `tractor.discovery._addr._address_types` (a `bidict`, so 1:1 - proto-key<->type) bc it has no `MsgTransport` of its own. The - tunnel is a *bindspace*, so we carry it beside the overlay - addr and strip to `.overlay` at bind/dial time. +The kernel owns the bearer socket. A future tractor bindspace may +provision it through netlink, but only the overlay is an application +`MsgTransport` endpoint. ''' from __future__ import annotations -import base64 import subprocess -from typing import Literal -import msgspec -from multiaddr import Multiaddr -from multiaddr.protocols import P_WG - - -IPProto = Literal['ip4', 'ip6'] - - -class WGTunnelledAddr( - msgspec.Struct, - frozen=True, -): - ''' - A `wg`-tunnelled endpoint: the underlay bearer, the tunnel - peer key, and the overlay addr `tractor` binds/dials. - - ''' - # underlay, owned by `wg(8)`/the kernel — NEVER bound by us - bearer: tuple[str, int] - - # tunnel peer pubkey in the std-base64 `wg(8)` form, i.e. - # directly comparable to `wg show peers` output - peer_pubkey: str - - # overlay ep: an `UnwrappedAddress` as accepted by - # `tractor.discovery.wrap_address()` - overlay: tuple[str, int] - overlay_proto: Literal['tcp'] = 'tcp' - - # kept so `.as_multiaddr()` re-renders the same ip family it - # was parsed from, rather than assuming v4 - bearer_ip: IPProto = 'ip4' - overlay_ip: IPProto = 'ip4' - - def as_multiaddr(self) -> Multiaddr: - ''' - Re-compose the canonical `Multiaddr`, bearer outward-in, - using `.encapsulate()` exactly as py-multiaddr's own - tunneling example does. - - ''' - b_host, b_port = self.bearer - o_host, o_port = self.overlay - return ( - Multiaddr(f'/{self.bearer_ip}/{b_host}/udp/{b_port}') - .encapsulate( - Multiaddr(f'/wg/{mb_pubkey(self.peer_pubkey)}') - ) - .encapsulate( - Multiaddr( - f'/{self.overlay_ip}/{o_host}' - f'/{self.overlay_proto}/{o_port}' - ) - ) - ) - - @property - def maddr(self) -> str: - ''' - The canonical maddr `str` form. - - ''' - return str(self.as_multiaddr()) - - -def mb_pubkey(wg8_key: str) -> str: - ''' - `wg(8)` std-base64 pubkey -> multibase base64url (`u`-prefixed). - - ''' - import multibase - raw: bytes = base64.b64decode(wg8_key) - return multibase.encode('base64url', raw).decode('ascii') - - -def wg8_pubkey(mb_key: str) -> str: - ''' - Inverse of `mb_pubkey()`: multibase -> `wg(8)` std-base64. - - ''' - import multibase - raw: bytes = multibase.decode(mb_key) - return base64.b64encode(raw).decode('ascii') - - -_wg_proto_known: bool|None = None - - -def _have_wg_maddr_proto() -> bool: - ''' - True iff the installed `py-multiaddr` knows the `/wg/` proto, - i.e. carries py-multiaddr#108. - - Merged upstream 2026-07-28 (`f86519da`) but in no release as - of `0.2.0`, hence the `[tool.uv.sources]` `rev` pin in - `pyproject.toml`. - - Pure predicate; result cached since it can't change without a - reinstall. - - ''' - global _wg_proto_known - if _wg_proto_known is None: - from multiaddr.protocols import protocol_with_name - from multiaddr.exceptions import ProtocolNotFoundError - try: - protocol_with_name('wg') - _wg_proto_known = True - except ProtocolNotFoundError: - _wg_proto_known = False - - return _wg_proto_known - - -def parse_wg_maddr( - maddr: str|Multiaddr, -) -> WGTunnelledAddr: - ''' - Peel a `wg`-tunnelled maddr into its bearer/key/overlay - parts. Pure — no I/O. - - Every cut is made by `py-multiaddr`, so a malformed maddr - (incl. a `wg` key that isn't exactly 32B) raises out of - `Multiaddr()` rather than yielding a struct quietly built - from garbage segs. - - ''' - if not _have_wg_maddr_proto(): - raise RuntimeError( - f'Installed `py-multiaddr` has no `/wg/` proto!\n' - f'Needs py-multiaddr#108, merged upstream but not\n' - f'yet released; a `uv sync` picks up the pinned rev.\n' - f'maddr: {maddr!r}\n' - ) - - ma: Multiaddr = ( - maddr - if isinstance(maddr, Multiaddr) - else Multiaddr(maddr) - ) - segs: list[Multiaddr] = ma.split() - names: list[str] = [ - proto.name - for seg in segs - for proto in seg.protocols() - ] - if 'wg' not in names: - raise ValueError( - f'Not a `wg`-tunnelled maddr, no `/wg/` segment ??\n' - f'maddr: {ma}\n' - ) - - # NOTE, `.decapsulate_code()` cuts at the LAST occurrence of - # the proto and keeps the *prefix*, which is exactly the - # bearer. It handles `/wg/` cleanly precisely bc it cuts on - # proto-code and never tries to match an addr value — the - # key seg has no addr of its own. - bearer_ma: Multiaddr = ma.decapsulate_code(P_WG) - overlay_ma: Multiaddr = Multiaddr.join( - *segs[names.index('wg') + 1:] - ) - - match [proto.name for proto in bearer_ma.protocols()]: - case [('ip4' | 'ip6') as b_ip, 'udp']: - bearer = ( - bearer_ma.value_for_protocol(b_ip), - int(bearer_ma.value_for_protocol('udp')), - ) - case _: - raise ValueError( - f'Bad `wg` bearer, expected `/ip4|ip6//udp/

`\n' - f'got: {bearer_ma}\n' - f'from maddr: {ma}\n' - ) - - match [proto.name for proto in overlay_ma.protocols()]: - case [('ip4' | 'ip6') as o_ip, ('tcp') as l4]: - overlay = ( - overlay_ma.value_for_protocol(o_ip), - int(overlay_ma.value_for_protocol(l4)), - ) - case []: - raise ValueError( - f'`wg` maddr declares no overlay endpoint!\n' - f'A bare `/…/wg/` names only the tunnel; ' - f'append the ep `tractor` should bind, e.g.\n' - f' {ma}/ip4/10.0.11.1/tcp/1616\n' - ) - case _: - raise ValueError( - f'Unsupported `wg` overlay proto combo\n' - f'got: {overlay_ma}\n' - f'from maddr: {ma}\n' - ) - - return WGTunnelledAddr( - bearer=bearer, - peer_pubkey=wg8_pubkey(ma.value_for_protocol('wg')), - overlay=overlay, - overlay_proto=l4, - bearer_ip=b_ip, - overlay_ip=o_ip, - ) +from tractor.discovery import ( + TunnelledAddress, + WGTunnelSpec, +) def verify_wg_peer( - addr: WGTunnelledAddr, - iface: str = 'wg0', + addr: TunnelledAddress, + iface: str|None = None, ) -> bool: ''' - True iff `addr.peer_pubkey` is a configured peer (or our own - pubkey) on `iface`. + Check the outer tunnel's key against one local `wg` iface. - IMPURE + explicit by design: never called from - `parse_wg_maddr()`. + IMPURE + explicit by design: neither `parse_wg_maddr()` nor + `tractor.discovery.parse_maddr()` calls this probe. ?TODO, per plan-03 layer B, swap this body for `pyroute2` - (keeping the signature) — and note `setns(2)` is *per-thread*, - so a query issued via `trio.to_thread` lands in the ORIGINAL - netns unless `netns=` is passed down. + while retaining the explicit verification boundary. ''' + spec = addr.tunnel + if not isinstance(spec, WGTunnelSpec): + raise TypeError( + f'Unsupported tunnel spec: {type(spec)!r}' + ) + + iface = iface or spec.iface + def _wg(*args: str) -> str: return subprocess.run( ['wg', 'show', iface, *args], @@ -293,7 +57,7 @@ def verify_wg_peer( ).stdout return ( - addr.peer_pubkey in _wg('peers').split() + spec.peer_pubkey in _wg('peers').split() or - addr.peer_pubkey == _wg('public-key').strip() + spec.peer_pubkey == _wg('public-key').strip() )