goodboy
/
tractor
1
0
Fork 2
Code Issues 5 Pull Requests 6 Packages Projects Releases Wiki Activity

Merge pull request #492 from goodboy/ng_tpts_planning 

Add impl plans for `TIPC`/`QUIC`/`wg` tpt backends
piker_pin
Bd 2026-08-31 22:26:01 -04:00 committed by GitHub
parent f220b4641d e4e27ec71d
commit 0c52f2770a
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
15 changed files with 3673 additions and 9 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

136
ai/prompt-io/claude/20260813T001102Z_27c34aeb_prompt_io.md 100644
View File

@ -0,0 +1,136 @@
---
model: claude-opus-5
service: claude
session: 7b9c97c4-fff7-4ac4-97fb-35720453308e
timestamp: 2026-08-13T00:11:02Z
git_ref: 27c34aebb615c30d4039fa399f4ce2766ed7ba2c
scope: code
substantive: true
raw_file: 20260813T001102Z_27c34aeb_prompt_io.raw.md
---
## Prompt
> draft hyper detailed implementation plans for [three]
> prospective new transport (tpt) backends for tractor's `.ipc`
> layer, from four GitHub issues: TIPC (gh #378) using built-in
> linux socket API w/ `trio` interfacing, leveraging TIPC's
> built-in discovery machinery; QUIC (gh #353) using the `iroh`
> lib, ideally with the py asyncio support (via ffi) rewritten
> for trio; wg (gh #482 and/or #443) with other shuttle-able
> tpts, using `pyroute2`, as much trio wrapping as possible
> where any other async support can be replaced.
With constraints: "be only slightly speculative"; realistic given
the linked info; the plans must be workable "by another
model/provider without much discrepancy on the design or lib
selections needed"; do it all in a new `/open-wkt` off main; and
be "as modular and modern-async-python-as-nearly-functional-style
as possible."
Key follow-up prompts that changed the design mid-session:
> wg requires binding a socket on both ends of the tunnel, the
> new support we added to `py-multiaddr` in their #108 enables
> declaring this in the maddr? […] so the
> `/ip4/10.0.0.1/udp/666/ip4/192.168.0.13/tcp/80` bullet example
> from #443 isn't correct for a wg tunnel maddr and would
> actually end up looking like,
> `/ip4/10.0.0.1/udp/666/wg/<pub-key>/ip4/192.168.0.13/tcp/80` no?
> right, we're likely going to need less `tuple`-formed
> `UnwrappedAddress` usage to make them all work and/or to extend
> the unwrapped form with a proto-str, likely the key from
> `multiaddrs` no?
> better encapsulating `UnwrappedAddress` for use externally is
> really way overdue; we should likely be encouraging (if not
> preventing unwrapped) wrapped addrs always much like other
> (std) libs, `ipaddress` comes to mind.
> so iirc netns mgmt becomes a lower-level runtime config-api
> rather then an actor-app-code API which can be dynamically
> changed once the actor is up ya?
## Response summary
A research-and-planning branch, `ng_tpts_planning` — **no runtime
code touched**. Five plan docs under `ai/tpt-backends/` plus a
corrected `wg` example set under `examples/multihost/wg_lan/`,
across 9 commits.
Three design conclusions came out of the planning that outlive
the branch:
1. **`wg` is a bindspace, not a `MsgTransport`.** The `/wg/`
maddr segment is *infix*: bearer (underlay ip+udp
`ListenPort`, bound by the kernel) → `/wg/u<key>` (an
identity, bound by nothing) → overlay (the only part tractor
binds). Verified empirically by installing
`baudco/py-multiaddr@wg_support` (py-multiaddr#108) in a
throwaway venv and round-tripping each candidate form; gh
#482's original suffix form parses but is semantically
inverted.
2. **Proto-key the `UnwrappedAddress` form**, spelled with the
`multiaddr` protocol names, dispatched via
`_address_types[addr[0]]`. Kills a whole collision class
(TIPC's `(str, int)` ≡ `TCPAddress`; iroh's `(str, str)`
swallowed by the UDS case) and is the recommended migration
*before* any new backend lands.
3. **netns is a runtime/boot-time config API, not an app-code
one** — `setns(2)` is per-thread and won't move
already-created sockets, so there is deliberately no
`await actor.enter_netns(...)`.
Also verified that `trio.SocketStream`/`SocketListener` are
address-family agnostic (no `AF_*` check anywhere), which is what
makes TIPC the cheapest of the three backends to add.
Four related issues were annotated with the results (#378, #353,
#482, #443); #443's body was rewritten to reflect the corrected
grammar, with no existing checkbox state changed.
## Files changed
- `ai/tpt-backends/00_shared_backend_contract.md` — normative
backend duck-type contract, registration checklist, §1.1
proto-key conclusion
- `ai/tpt-backends/01_tipc_backend.md` — TIPC plan; service
addressing, `TIPC_TOP_SRV` push registry, instance-collision
hazard, step-0 probe
- `ai/tpt-backends/02_quic_iroh_backend.md` — `iroh` plan;
`uniffi`→`trio` bridge, listener/stream adapters, API-truth
table
- `ai/tpt-backends/03_wg_tunnel_bindspace.md` — `wg`-as-bindspace
plan; verified maddr grammar, 3-owner split, netns reality
- `ai/tpt-backends/README.md` — index
- `examples/multihost/wg_lan/wg_maddr.py` — frozen `msgspec`
tunnelled addr + pure parse/render helpers; impure
`verify_wg_peer()` kept separate
- `examples/multihost/wg_lan/host_a_srv.py` — host-A actor tree
- `examples/multihost/wg_lan/host_b_client.py` — host-B dialer
- `examples/multihost/wg_lan/README.md` — grammar, owner table,
setup, "what changed vs #482"
## Human edits
Substantial human steering rather than post-hoc editing; the
corrections were applied by the model in-session after being
challenged:
- rejected an initial claim that `wg` has "nothing to bind at the
tunnel layer" and supplied the correct composed maddr form,
which forced a rewrite of plan 03 §3.2 and a retraction in the
already-posted #443 comment
- rejected a supporting claim that `/ip4/../udp/443/quic-v1` was
"also composed"
- directed the proto-key/`ipaddress`-discipline conclusion and
the netns-as-runtime-config framing, both of which were then
folded back into the docs
- chose the commit boundaries and authored all commits; ran every
`git` mutation (commit, rebase, push) themselves
One model-initiated correction pre-publication: a self-review
downgraded two overconfident claims (the `uniffi`/asyncio thesis
and TIPC duplicate-binder behaviour) to explicitly-flagged
assumptions before the #353/#378 comments were posted.

165
ai/prompt-io/claude/20260813T001102Z_27c34aeb_prompt_io.raw.md 100644
View File

@ -0,0 +1,165 @@
---
model: claude-opus-5
service: claude
timestamp: 2026-08-13T00:11:02Z
git_ref: 27c34aebb615c30d4039fa399f4ce2766ed7ba2c
diff_cmd: git diff 3ad7e7e5dc28aa336499725a03744fc10c6c0026..27c34aebb615c30d4039fa399f4ce2766ed7ba2c
---
# Raw output — next-gen tpt-backend implementation plans
## Generated planning docs
> `git diff 3ad7e7e5dc28aa336499725a03744fc10c6c0026..27c34aebb615c30d4039fa399f4ce2766ed7ba2c -- ai/tpt-backends/`
Five markdown docs. `00_shared_backend_contract.md` is normative
and the other three are written against it so they can be worked
independently:
- **`00_shared_backend_contract.md`** — the backend duck-type
(`<Proto>Address(msgspec.Struct, frozen=True)` + module-level
`start_listener()`/`close_listener()` + a
`Msgpack<Proto>Stream(MsgpackTransport)`), the
`inspect.getmodule(self.addr)` reflection in
`Endpoint.start_listener()` that forces the Address class and
its listener fns to share a module, a 10-item registration
checklist, the dep policy, the test-harness shape, and §1.1's
proto-key conclusion (below).
- **`01_tipc_backend.md`** — service addressing via
`TIPC_ADDR_NAMESEQ` (bind/publish) and `TIPC_ADDR_NAME`
(connect/lookup), `TIPC_TOP_SRV` topology subscriptions as a
push-based registry, the `get_random()` instance-collision
hazard, and a step-0 capability-probe spike.
- **`02_quic_iroh_backend.md`** — `iroh` over
`aioquic`/`quiche`/`trio-asyncio`, a `_uniffi_trio.py` bridge
built on `TrioToken.run_sync_soon()`, `trio.abc.Listener`/
`HalfCloseableStream` adapters, and an API-truth table to fill
in during step 0.
- **`03_wg_tunnel_bindspace.md`** — `wg` as a *bindspace* rather
than a `MsgTransport`, a `TunnelledAddress` wrapper delegating
`.proto_key`/`.unwrap()` to `.inner`, `pyroute2` for layer B,
and `@acm`-managed netns/iface for layer C.
- **`README.md`** — index.
## Generated example code
> `git diff 3ad7e7e5dc28aa336499725a03744fc10c6c0026..27c34aebb615c30d4039fa399f4ce2766ed7ba2c -- examples/multihost/wg_lan/`
- `wg_maddr.py` — `WGTunnelledAddr(msgspec.Struct, frozen=True)`
carrying `bearer: tuple[str, int]`, `peer_pubkey: str`,
`inner: tuple[str, int]`, `inner_proto: Literal['tcp']`, plus a
`.maddr` property that re-renders the canonical form. Pure
helpers `mb_pubkey()`, `wg8_pubkey()`, `parse_wg_maddr()`, and
`_segments()` (with a marked stopgap for when the `wg` codec
isn't installed). `verify_wg_peer()` is impure **by design** and
kept out of the parse path.
- `host_a_srv.py` / `host_b_client.py` — the two-host runs; both
pass only `addr.inner` to `open_nursery()`/`open_root_actor()`.
- `README.md` — grammar, owner table, `#108`-branch install line,
tunnel setup, "what changed vs #482".
## Verified findings (non-code, verbatim)
### `trio` is address-family agnostic
Read against the installed `trio`. `SocketStream`/`SocketListener`
ctor checks are only "is a trio sock object" + `type ==
SOCK_STREAM`, plus an `OSError`-**suppressed** `SO_ACCEPTCONN`
probe. No `AF_*` check anywhere; `TCP_NODELAY`/`TCP_NOTSENT_LOWAT`
are set under `suppress(OSError)`. A TIPC `SOCK_STREAM` sock should
therefore drop straight into `trio.serve_listeners()` with the
existing `MsgpackTransport` framing, making TIPC mostly
table-registration boilerplate w/ zero new deps.
### the `wg` maddr grammar — `/wg/` is infix, not suffix
Installed `baudco/py-multiaddr@wg_support` (PR
multiformats/py-multiaddr#108) into a throwaway venv and
round-tripped every candidate form:
| maddr | `[p.name for p in m.protocols()]` |
| --- | --- |
| `/ip4/1.2.3.4/udp/51820/wg/u<k>` | `['ip4','udp','wg']` |
| `/ip4/../udp/../wg/u<k>/ip4/../tcp/..` | `['ip4','udp','wg','ip4','tcp']` |
| `/ip4/10.0.11.1/tcp/1616/wg/u<k>` | `['ip4','tcp','wg']` |
```
/ip4/192.168.1.50/udp/51820/wg/u<A_pub>/ip4/10.0.11.1/tcp/1616
\_______ bearer __________/\__ key __/\______ overlay ______/
```
Segments *before* `/wg/` are the bearer — the underlay
`(ip, udp-port)` that `wg(8)` itself listens on (`ListenPort`).
Segments *after* are the overlay endpoint, the only part tractor
binds. The third row above is #482's original suffix form: it
parses, but is semantically inverted.
Three parts, three owners — and only one is an `Endpoint`:
| part | bound by | in the runtime? |
| --- | --- | --- |
| bearer | kernel, via `wg-quick`/`pyroute2` | no |
| `/wg/u<key>` | nothing — an identity | no, verified out-of-band |
| overlay | `tractor`'s `IPCServer` | yes, as `.inner` |
### proto-key-tagged `UnwrappedAddress`
Shape-matching in `wrap_address()` does not survive four backends.
TIPC's natural unwrapped form is a `(str, int)`, indistinguishable
from `TCPAddress`; iroh's is a `(str, str)`, already swallowed by
the existing UDS case (`case (_, filename) if type(filename) is
str`). Ordering hacks and prefix-tagging only paper over it.
Recommended prerequisite for all three backends: carry an explicit
proto-key spelled with the `multiaddr` protocol name —
`('tcp', host, port)`, `('unix', path)`,
`('tipc', stype, inst, scope)` — so `wrap_address()` collapses to
`_address_types[addr[0]]` and the collision class stops existing.
This also makes the on-wire form agree with
`mk_maddr()`/`parse_maddr()` instead of being an independent
invention. It is a wire-format change (`SpawnSpec`,
`_root_mailbox`, `_registry_addrs`) plus every fixture and
downstream config, so it wants its own migration commit landed
before any new backend — and it is the moment to stop handing raw
tuples to users at all, making `Address` the public currency and
`UnwrappedAddress` an internal serialization detail (the
discipline `ipaddress` uses).
### netns is a runtime-level config API
`setns(2)` affects the calling thread only and does not move
already-created sockets. So a netns is a spawn/boot-time input
alongside `enable_transports`/`tpt_bind_addrs`, and there is
deliberately no `await actor.enter_netns(...)` — a mid-life API
would silently leave the IPC server bound in the old namespace.
Corollary for layer B: pass `netns=` down to `pyroute2` rather
than assuming a `trio.to_thread` worker inherits it.
### `examples/` collection would have failed CI
`tests/test_docs_examples.py` walks `examples/` recursively and
subproc-runs every collected file asserting `rc == 0`. Its filter
never checks the extension, so all four `wg_lan` files were
collected — including `README.md`, which would have been run as
`python README.md`. `'multihost' not in p[0]` was already in the
exclusion list with no directory using it. Moving the set under
`examples/multihost/wg_lan/` drops collection 24 → 20 with zero
test changes; confirmed via `pytest --collect-only`.
## Corrections applied during the session
The human corrected two claims that had been asserted without
verification, both since retracted in-place in the docs and in the
posted issue comments:
1. that `wg` has "nothing to bind at the tunnel layer, exactly one
bind" — wrong; a wg stack is genuinely composed, and the real
axis is *who owns* each layer's endpoint.
2. that `/ip4/../udp/443/quic-v1` was "also composed" — wrong;
that is one endpoint with a protocol qualifier, not a tunnel.
A self-review before publication also downgraded two
overconfident claims to explicitly-flagged assumptions: the
`uniffi`-uses-asyncio-only-as-executor thesis (contradicted that
plan's own "do not guess from memory" step 0) and TIPC's
duplicate-binder round-robin behaviour (unverified).

72
ai/prompt-io/opencode/20260831T022317Z_768b5316_prompt_io.md 100644
View File

@ -0,0 +1,72 @@
---
model: gpt-5.6-sol
service: opencode
session: f3a5eef0-9695-48ab-a336-861187473f8a
timestamp: 2026-08-31T02:23:17Z
git_ref: ng_tpts_planning
diff_cmd: git diff 768b531662a88037e3988885102d2ff24a7ce34e..HEAD
scope: code
substantive: true
raw_file: 20260831T022317Z_768b5316_prompt_io.raw.md
---
## Prompt
The human asked to continue the retained PR landing sequence and
selected "Repair all blockers" after a read-only review of PR #492.
They then corrected the TIPC assessment:
> oh one thing on the tipc findings, they may have already been
> addressed in the downstream patch branch no?
The repair therefore had to use downstream #493 as implementation
evidence, avoid duplicating solved TIPC work, preserve the dirty
downstream worktree, and repair the remaining shared, QUIC,
WireGuard, lock, and provenance defects without committing,
pushing, or publishing.
## Response summary
Reworked PR #492's plans and executable WireGuard example into a
consistent landing candidate. The shared and TIPC documents now
match current runtime/downstream behavior while naming remaining
risks. The QUIC plan now defines transport ownership across the
whole actor lifecycle. The WireGuard example validates keys and
binds both hosts and the service actor correctly without elevating
Tractor. The dependency lock and historical provenance pointers
are synchronized.
## Files changed
- `ai/tpt-backends/00_shared_backend_contract.md` - current backend
contract, dispatch, capability, and locality rules.
- `ai/tpt-backends/01_tipc_backend.md` - #493 reconciliation and
bounded unresolved work.
- `ai/tpt-backends/02_quic_iroh_backend.md` - lifecycle-safe QUIC
and UniFFI implementation plan.
- `ai/tpt-backends/03_wg_tunnel_bindspace.md` - corrected address
registry and privilege semantics.
- `ai/tpt-backends/README.md` - refreshed plan summaries.
- `examples/multihost/wg_lan/README.md` - unprivileged two-host
setup and accurate verification limits.
- `examples/multihost/wg_lan/host_a_srv.py` - local key check,
overlay child bind, and stable RPC exposure.
- `examples/multihost/wg_lan/host_b_client.py` - peer key check,
host-B bind, and explicit missing-service failure.
- `examples/multihost/wg_lan/wg_maddr.py` - strict parsing and
asynchronous role-specific key inspection.
- `uv.lock` - exact `py-multiaddr` Git source resolution.
- `ai/prompt-io/claude/20260813T001102Z_27c34aeb_prompt_io.md` -
valid scope and immutable Git reference.
- `ai/prompt-io/claude/20260813T001102Z_27c34aeb_prompt_io.raw.md`
- immutable historical diff pointers.
## Human edits
The human chose the full repair path rather than reducing the PR
to planning documents or publishing the initial review. They also
identified that the initial TIPC review had not accounted for the
downstream implementation branch. That correction materially
changed the work: solved TIPC items were backported into the plan,
remaining defects were separated from implemented behavior, and
the dirty downstream worktree was kept read-only.

60
ai/prompt-io/opencode/20260831T022317Z_768b5316_prompt_io.raw.md 100644
View File

@ -0,0 +1,60 @@
---
model: gpt-5.6-sol
service: opencode
timestamp: 2026-08-31T02:23:17Z
git_ref: ng_tpts_planning
diff_cmd: git diff 768b531662a88037e3988885102d2ff24a7ce34e..HEAD
---
# Raw output - repair PR #492 for landing
## Generated changes
> `git diff 768b531662a88037e3988885102d2ff24a7ce34e..HEAD -- ai/tpt-backends/`
- Reconciled the shared backend contract with current address,
dispatch, capability, locality, and listener-rebind APIs.
- Updated the TIPC plan from downstream #493 implementation and
tests, preserving unresolved registrar election, collision,
locality, and socket-cleanup work as explicit follow-ups.
- Reworked the QUIC plan around a launch-time transport bootstrap,
one actor-owned endpoint, a transport nursery spanning parent
dial through deregistration, supervised UniFFI cleanup, complete
routable addresses, connection leases, and listener-owned tasks.
- Corrected the WireGuard bindspace plan to distinguish the
build-registered address registry from runtime capability.
> `git diff 768b531662a88037e3988885102d2ff24a7ce34e..HEAD -- examples/multihost/wg_lan/`
- Hardened WireGuard key conversion and parsing with strict
32-byte validation, lazy protocol lookup, and explicit rejection
of unsupported nested tunnel descriptors.
- Made interface inspection asynchronous, bounded at its
cancellation request, role-specific, and separable from
privileged preflight commands.
- Corrected host-local binds, subactor overlay publication, stable
RPC module exposure, and missing-actor handling in the two-host
example.
- Updated the README so Tractor remains unprivileged and the local
versus remote overlay roles are explicit.
> `git diff 768b531662a88037e3988885102d2ff24a7ce34e..HEAD -- pyproject.toml uv.lock ai/prompt-io/`
- Regenerated `uv.lock` for the exact merged `py-multiaddr` WireGuard
codec revision.
- Replaced mutable historical prompt diff pointers with immutable
refs and normalized the substantive scope without rewriting the
historical raw response.
## Verification output
- `git diff --check`: passed.
- `uv lock --check`: passed.
- Ruff on the WireGuard example directory: passed.
- Python compilation of the WireGuard example directory: passed.
- Tractor imported from the PR worktree's existing environment.
- WireGuard address round-trip, local/peer role checks, malformed
base64 rejection, and nested-tunnel rejection: passed.
- Three independent final re-reviews reported no actionable
findings in the shared/TIPC, QUIC, or WireGuard slices.
- No live TIPC, iroh/UniFFI, or WireGuard network test was run.

450
ai/tpt-backends/00_shared_backend_contract.md 100644
View File

@ -0,0 +1,450 @@
# `tractor.ipc` next-gen transport backends: the shared contract
Status: design doc / implementation spec.
Audience: any model or human implementing one of the three
sibling plans in this directory.
- [`01_tipc_backend.md`](./01_tipc_backend.md) — `AF_TIPC`
(gh #378)
- [`02_quic_iroh_backend.md`](./02_quic_iroh_backend.md) — QUIC
via `iroh` FFI, uniffi-async rewritten onto `trio` (gh #353)
- [`03_wg_tunnel_bindspace.md`](./03_wg_tunnel_bindspace.md) —
WireGuard (and other shuttle-able) tunnels as a *nested
bindspace* layer via `pyroute2` (gh #482, #443)
This doc is the **normative** description of what a `tractor`
transport backend *is* as of `main@83b34884`. Each sibling plan
assumes it and only documents its own deltas. Read this first;
do not re-derive it from the code.
---
## 0. Why a shared contract doc
The three plans are meant to be implementable *independently and
concurrently* by different models/providers without design
drift. Everything they share — the backend duck-type, the
registration tables, the test harness plumbing, the naming and
code-style rules — lives here exactly once. If an implementer
finds this doc disagrees with `main`, **the code wins**; fix this
doc in the same PR.
---
## 1. The backend duck-type (empirical, from `_tcp.py`/`_uds.py`)
A transport backend is **one module** under `tractor/ipc/`.
There is no ABC to subclass and no plugin entrypoint; wiring is by
explicit table registration (§2) plus one piece of reflection
(§1.3).
Keep two contracts distinct:
- `tractor.discovery._addr.Address` is a static `Protocol`. It
declares address-wrapper members including `namespace`,
`open_listener()` and `close_listener()`.
- the runtime's empirical contract is what `_tcp.py`, `_uds.py`
and `_server.py` actually call. The current address classes do
not implement every declared `Address` member: listener
lifecycle is module-level, `def_bindspace` is used despite not
being declared by the `Protocol`, and `namespace` remains
aspirational.
Until those surfaces are deliberately reconciled, implement the
empirical module contract below and update the static `Protocol`
only when the runtime really consumes the new member. Do not claim
that structural conformance alone defines a backend.
### 1.1 `class <Proto>Address(msgspec.Struct, frozen=True)`
The runtime-consumed address-wrapper surface is:
| member | kind | notes |
| --- | --- | --- |
| `proto_key` | `ClassVar[str]` | internal transport key, e.g. `'tcp'`, `'uds'` |
| `unwrapped_type` | `ClassVar[type]` | the primitive tuple shape |
| `def_bindspace` | `ClassVar` | default bindspace value |
| `is_valid` | `@property -> bool` | "is this a *dialable/bindable* addr" |
| `bindspace` | `@property` | the "set of hosts"-ish scope (see below) |
| `from_addr(cls, addr)` | `@classmethod` | primitive -> wrapped, `match`-based |
| `unwrap(self)` | method | wrapped -> primitive (must be msgpack-native!) |
| `get_random(cls, bindspace=...)` | `@classmethod` | per-subactor ephemeral addr |
| `get_root(cls)` | `@classmethod` | host-singleton default registrar addr |
| `__repr__` | method | `f'{type(self).__name__}[{...}]'` house style |
Hard constraints learned from the existing two:
- **`frozen=True`.** Addresses are dict keys
(`Server.epsdict()`, `Endpoint.peer_tpts`) and are compared by
value all over the runtime.
- **`.unwrap()` output must round-trip through `msgspec` and
through `wrap_address()`.** It is what actually crosses the
wire in `SpawnSpec`/`_root_mailbox`/`_registry_addrs`, and it
is what `Actor.reg_addrs` and every test compares against. If
your unwrapped form is not *uniquely* pattern-matchable
against the other backends' forms in
`wrap_address()` (`_addr.py:230`), you have a bug that
manifests as the wrong transport being loaded — the file's own
`XXX NOTE` warns about precisely this.
⚠️ **and shape-matching does not survive 4 backends.** Adding
TIPC and iroh breaks it outright: TIPC's natural form is a
`(str, int)` — indistinguishable from `TCPAddress` — and
iroh's is a `(str, str)`, which the *existing* UDS case
(`case (_, filename) if type(filename) is str`) already
swallows. Ordering hacks and prefix-tagging (an earlier
revision of plan 01 proposed `('tipc:<stype>:<scope>', inst)`)
paper over it at best.
**The fix, and the recommended prerequisite for all three
backends: make the unwrapped form carry an explicit internal
proto-key** — `('tcp', host, port)`,
`('uds', filedir, filename)`, `('tipc', stype, inst, scope)`.
The tag must be a `TransportProtocolKey`/registry key. In
particular it is **`'uds'`, not the external multiaddr spelling
`'unix'`**. If a wire or display format uses a different name,
name that translation explicitly; today `_multiaddr.py` maps
internal `uds` to external `/unix/`. Then `wrap_address()` can
dispatch through `_address_types[addr[0]]` without an
order-sensitive shape match.
Two consequences to plan for:
- it's a **wire-format change**. Widen and keep synchronized
`discovery._addr.UnwrappedAddress` and the duplicate wire
alias in `msg.types`; change `SpawnSpec.reg_addrs` and
`.bind_addrs`, not only `_root_mailbox` and
`_registry_addrs`. Audit the related `RuntimeVars`
`_root_mailbox`/`_root_addrs` annotations, `Actor.reg_addrs`
and accept-address annotations, channel/spawn signatures,
fixtures, and downstream config (`piker`'s `[network]`
table). `msgspec` rejects a union containing multiple
array-like tuple shapes, so #493 used
`tuple[str|int, ...]` as the truthful transitional wire type;
the complete proto-key migration can restore per-proto
validation. This wants its **own migration commit, landed
before any new backend**, not smuggled into one.
- it's the moment to **stop handing raw unwrapped tuples to
users at all.** The long-term shape is: `Address` subtypes
are the public currency and `UnwrappedAddress` becomes an
internal serialization detail — the same discipline
`ipaddress` uses (you pass `IPv4Address`, not a 4-tuple).
Public API should accept `Address|maddr-str` and treat bare
tuples as legacy-tolerated input, ideally deprecated.
- **`.get_random()` must not deterministically alias without a
live runtime.** See the `UDSAddress.get_random()` uuid-token
comment (`_uds.py:207-220`): with no `current_actor()` the
sockname degenerates to a pure fn of `(prefix, pid)` and two
calls in one proc alias. Mix in a `uuid4().hex[:8]` token.
- **`.bindspace` semantics**: "the address' bindable space" —
ip/host for `tcp`, the socket-file *directory* for `uds`. For
the new backends: the TIPC *scope* (§1 of plan 01), the iroh
*ALPN + relay/discovery realm* (plan 02), the netns (plan 03).
`Address.namespace` is already spec'd in the Protocol as
"the if-available OS-specific network namespace key" and is
currently unimplemented by both backends — plan 03 is the
first real consumer.
### 1.2 module-level listener lifecycle
```python
async def start_listener(
addr: <Proto>Address,
**kwargs,
) -> trio.SocketListener # or a trio.abc.Listener, see §3
...
def close_listener( # OPTIONAL
addr: <Proto>Address,
lstnr: trio.abc.Listener,
) -> None:
...
```
`close_listener()` is optional; `Endpoint.close_listener()`
(`_server.py:674`) `getattr`s it and treats absence as "closing
is implicit". `uds` needs it (unlinks the sock-file), `tcp`
does not.
### 1.3 the ONE piece of reflection you must not break
`Endpoint.start_listener()` (`_server.py:656`):
```python
tpt_mod: ModuleType = inspect.getmodule(self.addr)
lstnr = await tpt_mod.start_listener(addr=self.addr)
```
The transport module is found by `inspect.getmodule()` **on the
`Address` instance**. Therefore: *the `Address` class and its
`start_listener()`/`close_listener()` MUST live in the same
module.* Do not define the address type in `_types.py` or a
`_addrs.py` and the listener elsewhere.
Immediately after, the same method does:
```python
if (unwrapped := lstnr.socket.getsockname()) != self.addr.unwrap():
self.addr = self.addr.from_addr(unwrapped)
```
i.e. it assumes `lstnr.socket.getsockname()` exists and that its
return value is a valid `from_addr()` input. That is false for
TIPC, whose listener sockname is an undialable port ID, and for
non-socket iroh. Both plans must use the explicit backend rebind
policy added at this integration point rather than pretending a
sockname is always an address replacement.
### 1.4 `class Msgpack<Proto>Stream(MsgpackTransport)`
Subclass `tractor.ipc._transport.MsgpackTransport`. You inherit
all framing (`<I` 4-byte little-endian length prefix),
`msgspec` codec ctx-var lookup, `TransportClosed` normalization,
`.drain()`, `__aiter__`. You implement only:
| member | notes |
| --- | --- |
| `address_type` | the `<Proto>Address` class |
| `layer_key: int` | OSI-ish layer, `4` for both current backends |
| `maddr` `@property` | `-> Multiaddr\|str`, via `mk_maddr(self.raddr)` |
| `connected(self) -> bool` | `tcp`/`uds` both use `self.stream.socket.fileno() != -1` |
| `connect_to(cls, addr, prefix_size=4, codec=None, **kw)` | `@classmethod`, returns an instance |
| `get_stream_addrs(cls, stream) -> (laddr, raddr)` | `@classmethod`, called from `MsgpackTransport.__init__` |
`MsgpackTransport.__init__` requires the object passed as
`stream` to satisfy:
- `await stream.send_all(bytes)`
- usable as `tricycle.BufferedReceiveStream(transport_stream=stream)`,
i.e. `await stream.receive_some(n)`
- `trio.BrokenResourceError` / `trio.ClosedResourceError` /
`ValueError('...unclean EOF...')` on the failure paths that
`_iter_packets()` and `send()` already `match` on
(`_transport.py:221-304`, `:436-499`).
That is **`trio.abc.Stream`, not `trio.SocketStream`**. The
`MsgTransport` Protocol's `stream: trio.SocketStream`
annotation (`_transport.py:83`) is a lie of convenience — the
actual `MsgpackTransport.__init__` param is typed
`trio.abc.Stream` and nothing in the msg path touches
`.socket`. Only `connected()` (which each backend defines) and
`Endpoint.start_listener()`'s `getsockname()` do.
### 1.5 verified-good news for socket-family backends
Both `trio.SocketStream` and `trio.SocketListener` are
**address-family agnostic**. Verified against the installed
`trio` (`trio/_highlevel_socket.py`): the only constructor
checks are
- `isinstance(socket, trio.socket.SocketType)`
- `socket.type == SOCK_STREAM`
- (listener) `getsockopt(SOL_SOCKET, SO_ACCEPTCONN)` is truthy,
with `OSError` **suppressed** (the macOS carve-out, which
also covers exotic families that reject the opt)
There is no `AF_*` check and no `IPPROTO_TCP` hard dependency
(`TCP_NODELAY`/`TCP_NOTSENT_LOWAT` are set under
`suppress(OSError)`). Consequence: **any `SOCK_STREAM` family
CPython can create — including `AF_TIPC` — drops straight into
the existing `trio.SocketStream` + `trio.serve_listeners()`
path.** This is why plan 01 is small and plan 02 is not.
---
## 2. Registration and policy wiring
Adding a backend requires this complete audit. Not every item
changes for every backend, but none may be assumed from the others:
1. `tractor/runtime/_state.py:46`
`TransportProtocolKey = Literal['tcp', 'uds', ...]` — add the
internal key. This `Literal` is the **declared protocol-key
set**, not proof that a backend is usable on this host.
2. `tractor/discovery/_addr.py` `_address_protos` and
`_address_types: dict[str, Type[Address]]` — register
`'<key>': <Proto>Address`. `_address_types` is a plain
**`dict`, not a `bidict`**, and represents the backends this
build registers for import and dispatch. UDS is conditional on
`HAS_UDS`, while TIPC can remain registered on a host where its
kernel support is unavailable. An importable backend with a
runtime capability requirement therefore needs a separate
availability check. Never conflate this dispatch registry with
either host usability or the declared `TransportProtocolKey`
universe.
3. `tractor/discovery/_addr.py:181` `_default_lo_addrs` —
`'<key>': <Proto>Address.get_root().unwrap()`.
⚠️ this dict is built at **import time**, so
`get_root()` must not require a live runtime, a loaded kernel
module, or network I/O. (`UDSAddress.def_bindspace =
get_rt_dir()` is the precedent for "cheap, pure, filesystem-
ish".) A backend whose root addr needs I/O must make this
entry lazy — propose that refactor explicitly.
4. `tractor/discovery/_addr.py:230` `wrap_address()` `match` —
add a case iff your `unwrapped_type` isn't already uniquely
matched. **Preferably do the proto-key migration in §1.1
first**, after which this step becomes a one-line
`_address_types` lookup instead of an order-sensitive `case`.
5. `tractor/ipc/_types.py` — `Address` union alias,
`_msg_transports` list, `_key_to_transport[('msgpack', key)]`,
`_addr_to_transport[<Proto>Address]`.
6. `tractor/ipc/_types.py:92` `transport_from_stream()` — the
`sock.family` `match`. For a non-socket stream type (iroh)
this needs a different discriminator; see plan 02 §3.3.
7. `tractor/discovery/_multiaddr.py` —
`_tpt_proto_to_maddr`, and a `case` in both `mk_maddr()` and
`parse_maddr()`.
8. `tractor/ipc/__init__.py` — re-export if the backend has a
public surface.
9. `tractor/discovery/_api.py::_is_local_addr()` and
`prefer_addr()` — define and test the backend's locality and
selection tier. The current order is UDS, local TCP, then
remote. A new backend must not silently fall into `remote` by
accident: for example TIPC node scope is local, cluster scope
is not known-local, and an observed address with unknown scope
must not be promoted. Preserve the last-registered tie-break
unless intentionally changing policy.
10. `tractor/_testing/addr.py::get_rando_addr()` — per-proto
branch so the whole suite can run under `--tpt-proto <key>`.
11. `pyproject.toml` — new deps go in an **optional extra**, never
in `[project].dependencies`. See §5.
## 3. Where the `trio.SocketListener` assumption is load-bearing
`_serve_ipc_eps()` (`_server.py:1041`) annotates
`listener: trio.abc.Listener` and hands the list to
`trio.serve_listeners(handler=handle_stream_from_peer,
listeners=..., handler_nursery=stream_handler_tn)`.
`trio.serve_listeners` itself is generic over
`trio.abc.Listener`. So the *only* `SocketListener`-specific
code in the server path is the `getsockname()` reconciliation in
`Endpoint.start_listener()` (§1.3) and the type annotations.
`handle_stream_from_peer()` (`_server.py:298`) then does
`Channel.from_stream(stream)` →
`transport_from_stream(stream)` → `sock.family` match (§2.6).
**Therefore**: a non-socket backend needs (a) a
`trio.abc.Listener` subclass, (b) a change to
`Endpoint.start_listener()` to not blindly `getsockname()`, and
(c) a change to `transport_from_stream()`'s discrimination.
All three are small, upstream-able, and *should be landed as
their own prep PR* before the backend itself — see plan 02 §3.
## 4. Handshake / discovery invariants you inherit
- Every accepted stream immediately does
`chan._do_handshake(aid=actor.aid)`; a peer that fails it is
logged at `runtime` and dropped, **not** raised
(`_server.py:334-365`). Discovery-sys "pings" rely on this,
so your `connect_to()` must raise something that normalizes
to `TransportClosed`/`ConnectionError` on a dead peer, never
a novel exception type.
- `_root.py:381-406` fail-fasts when a `registry_addrs` entry's
`proto_key` is not in `enable_transports`. Your key must be
spellable in both.
- `_root.py:256` currently enforces `len(enable_transports) == 1`.
Multi-tpt actors are a separate work item; none of these three
plans may depend on lifting it.
- Sub-actor bind addrs come from
`_runtime.py:1600-1610`: for each key in the parent-supplied
`enable_transports`, `get_address_cls(key).get_random()`.
So `get_random()` runs *in the child, post-fork, pre-listen*.
Anything it needs (kernel module, netns membership, an iroh
secret key) must already be true at that moment.
## 5. Dependency policy
`[project].dependencies` stays lean (see the boot-latency work,
gh #470: `import tractor` is budgeted at ~0.145s). Every new
backend dep is an extra:
```toml
[project.optional-dependencies]
tipc = [] # stdlib-only!
quic = ["iroh>=0.35"] # pin per plan 02 §1
wg = ["pyroute2>=0.9"] # pin per plan 03 §1
```
and every backend module must be **import-lazy**: a
`tractor/ipc/_<proto>.py` that imports its 3rd-party dep at
module scope must not be imported by `tractor/__init__.py`,
`tractor/ipc/__init__.py`, or `tractor/discovery/_addr.py`'s
import-time table construction. The `_addr._default_lo_addrs`
eager-dict (§2.3) is the trap: keep the backend's `get_root()`
dep-free, or make that table lazy.
## 6. Test-harness plumbing (identical for all three)
- `--tpt-proto <key>` (`_testing/pytest.py:409`) selects the
session-wide proto; the `tpt_proto` fixture mutates
`_state._def_tpt_proto` + `_runtime_vars['_enable_tpts']`
(`pytest.py:807-835`). Adding the key to `_address_types` is
what makes `--tpt-proto <key>` legal (`pytest.py:795-800`
asserts the lookup). Thus CLI acceptance follows the
build-registered `_address_types`, while type-level declarations
follow `TransportProtocolKey` and host usability follows each
backend's capability probe; test all three layers separately.
- The **acceptance bar** for every backend is: the *entire*
existing suite passes under `--tpt-proto <key>`, unmodified.
That is the whole point of the abstraction. Backend-specific
unit tests go in `tests/ipc/test_each_tpt.py` (the existing
`test_uds_bindspace_created_implicitly` /
`test_uds_double_listen_raises_connerr` are the model).
- Capability gating: each backend needs a **cheap, pure
predicate** + a `pytest.mark.skipif`, because these are all
environment-dependent. Verified example: on this dev box
`socket.socket(AF_TIPC, SOCK_STREAM)` raises
`OSError(97, 'Address family not supported by protocol')`
because the `tipc` module isn't loaded. Put the predicate in
the backend module (so apps can use it too), not in the test.
- New pytest marks must be registered in
`_testing/pytest.py::pytest_configure()` with
`config.addinivalue_line()`, alongside the existing custom
marks. The repo has no `pyproject.toml` marker table. This is
still part of the fix-warnings-at-source rule (gh #469).
## 7. Code style (non-negotiable, matches the repo)
- module header tagline: `# tractor: distributed structured
concurrency.` for **new** files (not the legacy
`structured concurrent "actors".` form the existing `_tcp.py`
carries).
- AGPL header block copied verbatim from `_tcp.py`.
- `from __future__ import annotations` first.
- annotate *everything*, including locals:
`sockpath: Path = addr.sockpath`.
- `match`/`case` over `isinstance` chains for address and
error dispatch.
- multi-line call/`import` style with trailing commas.
- never emit a whitespace-only line.
- error messages are multi-line f-strings ending in `\n`, with
the `f'...\n' f'...\n'` implicit-concat layout and the
`>[`/`[>`/`<=(` nested-op sigils where a `nest_from_op()` is
in play.
- prefer pure functions + module-level helpers over methods;
keep `Address` types data-only. Where a helper needs
scoped setup/teardown, it's an `@acm` — not a class with
`.start()`/`.stop()`.
- pure getters: no `get_*(..., mutate=True)` flags; split into
a read-only getter and an explicit sibling setter.
---
## 8. Cross-plan sequencing
The three are independent *except*:
- plan 02 (iroh) needs the `_server.py` /
`transport_from_stream()` generalization (§3) — plan 01 does
**not**, and should therefore land first as the cheap proof
that the table-registration story works for a genuinely new
proto.
- plan 03 (wg) composes *under* whatever L4 tpt is in use and
its netns work is what finally implements
`Address.namespace`. It can land before or after 02, but its
`TunnelledAddress` design must be reviewed against plan 02's
address shape so the "tunnelled maddr" grammar (gh #443)
covers `/…/quic-v1/p2p/…` inner addrs too.
- All three want first-class `wg`/`quic`/`tipc` protos in
`py-multiaddr`; that upstream track is gh #483 and
multiformats/py-multiaddr#107/#108.

821
ai/tpt-backends/01_tipc_backend.md 100644
View File

@ -0,0 +1,821 @@
# Plan 01 — `TIPC` transport backend (`tractor/ipc/_tipc.py`)
Tracks gh [#378]. Prereq reading:
[`00_shared_backend_contract.md`](./00_shared_backend_contract.md).
**Thesis**: TIPC is the *cheapest* new backend we can add and
gives us kernel-native service-name publication, known-address
dialling, and topology events. Those are primitives for reducing
registrar traffic; they do **not** by themselves replace
`tractor.discovery`, derive an actor's address from its name, or
elect one registrar. It is stdlib-only: zero new dependencies.
This plan is reconciled against downstream PR [#493]'s code and
tests. Treat that implementation as prior art without mistaking
implemented transport primitives for completed discovery policy.
[#378]: https://github.com/goodboy/tractor/issues/378
[#493]: https://github.com/goodboy/tractor/pull/493
---
## 1. Why this is small: three verified facts
1. **CPython already speaks TIPC.** `socket.AF_TIPC` plus 23
`TIPC_*` constants are present in the stdlib on Linux
(verified on the dev box, py3.13):
`AF_TIPC, SOL_TIPC, TIPC_ADDR_ID, TIPC_ADDR_NAME,
TIPC_ADDR_NAMESEQ, TIPC_CFG_SRV, TIPC_CLUSTER_SCOPE,
TIPC_CONN_TIMEOUT, TIPC_{CRITICAL,HIGH,MEDIUM,LOW}_IMPORTANCE,
TIPC_DEST_DROPPABLE, TIPC_IMPORTANCE, TIPC_NODE_SCOPE,
TIPC_PUBLISHED, TIPC_SRC_DROPPABLE, TIPC_SUBSCR_TIMEOUT,
TIPC_SUB_CANCEL, TIPC_SUB_PORTS, TIPC_SUB_SERVICE,
TIPC_TOP_SRV, TIPC_WAIT_FOREVER, TIPC_WITHDRAWN,
TIPC_ZONE_SCOPE`.
`sock.bind()/connect()/getsockname()` take/return the
5-tuple `(addr_type, v1, v2, v3, scope)` — the last element
is optional on input and defaults to `0`.
2. **`trio` doesn't care about the address family.** Per
contract §1.5, `trio.SocketStream` and `trio.SocketListener`
only require a trio socket object of type `SOCK_STREAM`.
TIPC's `SOCK_STREAM` is a real connection-oriented reliable
byte stream. So we reuse `trio.SocketStream`,
`trio.SocketListener`, `trio.serve_listeners()`,
`MsgpackTransport`'s framing — *all of it*.
3. **It is not available by default.** On this box
`socket.socket(AF_TIPC, SOCK_STREAM)` →
`OSError(97, 'Address family not supported by protocol')`
with no `tipc` in `/proc/modules`. `modprobe tipc` is
required; cross-node needs a bearer
(`tipc bearer enable media eth device <if>` or
`media udp name <n> localip <ip>`). Everything about this
plan's testability hinges on gating (§7).
Non-goals: `SOCK_RDM`/`SOCK_DGRAM`/`SOCK_SEQPACKET` message
modes, multicast fan-out, and TIPC group messaging. They are
genuinely interesting for a future `tractor` broadcast/pubsub
transport but they do **not** fit `MsgTransport`'s
stream-of-length-prefixed-msgs shape. Note them in the
follow-up issue, do not build them here.
---
## 2. `TIPCAddress`
### 2.1 the three TIPC address flavours, and which we use
| flavour | tuple | meaning |
| --- | --- | --- |
| `TIPC_ADDR_NAMESEQ` | `(type, lower, upper, scope)` | a *published range* — what a server `bind()`s |
| `TIPC_ADDR_NAME` | `(type, instance, domain, scope)` | a *lookup* — what a client `connect()`s |
| `TIPC_ADDR_ID` | `(node, ref, 0, scope)` | a concrete port id — the "physical" address |
The design decision that makes this backend coherent:
> **A `tractor` actor's TIPC address is a *service name*
> `(type, instance)`; `bind()` publishes the singleton range
> `(type, instance, instance)`; peers `connect()` by name and
> the kernel resolves + load-balances. `TIPC_ADDR_ID` is only
> ever an *observed* address (`getpeername()`), never a
> user-facing one.**
This is the "leverage the built-in discovery machinery" part of
#378: publishing a bind is kernel name-table registration and
`connect()` on an already-known name is a kernel lookup, with no
registrar actor on that **dial** path. Mapping an application name
to that address and maintaining Tractor's actor registry remain
separate work (§5).
### 2.2 the struct
```python
class TIPCAddress(
msgspec.Struct,
frozen=True,
):
_stype: int # TIPC "type" == service class
_instance: int # service instance within the type
_scope: int = TIPC_CLUSTER_SCOPE
# observed-only, excluded from the unwrapped service identity
maybe_node: int|None = None # from TIPC_ADDR_ID getpeername()
maybe_ref: int|None = None
proto_key: ClassVar[str] = 'tipc'
unwrapped_type: ClassVar[type] = tuple[str, int, int, int]
def_bindspace: ClassVar[int] = TIPC_CLUSTER_SCOPE
```
**Unwrapped form** (the wire/`SpawnSpec` shape).
TIPC's natural form is `(stype, instance, scope)` — but a
2-tuple squeeze of it is a `(str, int)`, i.e. *the same coarse
shape as `TCPAddress`*, so `wrap_address()`'s
`case (str(), int())` steals it. This backend is therefore the
forcing function for the contract-doc's conclusion (§1.1):
> **make the unwrapped form carry the explicit internal
> `TransportProtocolKey`.**
```python
def unwrap(self) -> tuple[str, int, int, int]:
return ('tipc', self._stype, self._instance, self._scope)
```
`wrap_address()` then dispatches `_address_types[addr[0]]` and
the collision class disappears. The complete all-backend change
is a prerequisite migration; #493 necessarily carried the
transitional `UnwrappedAddress`/`SpawnSpec.reg_addrs`/
`.bind_addrs` widening needed for TIPC. See contract §1.1 for the
remaining runtime annotations, fixtures and `piker` config. Here
`tipc` is both the internal and external spelling; UDS remains
internally `uds` and translates explicitly to external `/unix/`.
`msgpack` decodes tuples as lists, so both forms are part of the
round-trip contract. Match only the exact three- or four-element
tagged shapes and test all four routes:
```python
case (
('tipc', int() as stype, int() as inst, int() as scope)
|
['tipc', int() as stype, int() as inst, int() as scope]
):
...
```
Also test the scope-defaulted three-element form through
`TIPCAddress.from_addr()`, and tuple/list forms through the global
`wrap_address()`. A normal two-element TCP/UDS address whose first
element happens to be `'tipc'` must retain its classic dispatch.
⚠️ an earlier revision of this plan proposed a self-tagging
`('tipc:<stype>:<scope>', instance)` string-prefix hack with an
ordered `case` guard. **Dropped** — it papers over the problem,
keeps `wrap_address()` order-sensitive, and doesn't help iroh's
`(str, str)`-vs-UDS collision at all. Do not resurrect it.
Note `TIPCAddress` is the first backend where `.unwrap()` is
**not** a lossless view of the live socket — `maybe_node`/
`maybe_ref` are observed metadata, exactly like
`UDSAddress.maybe_pid` (which is likewise excluded from
`.unwrap()`). Follow that precedent, including its `__repr__`
treatment (`_uds.py:242`).
### 2.3 how to pick `_stype` and `_instance`
- `_stype` = a `tractor`-reserved service class. TIPC reserves
0..63 for internal use (`TIPC_TOP_SRV == 1`,
`TIPC_CFG_SRV == 0`). Use a module constant
`TRACTOR_STYPE: int = 0x74_72_00_00` ("tr\0\0") as the default
and make it overridable via `TIPCAddress._stype` so an app
can partition service classes. Document that two `tractor`
trees sharing a cluster **and** a `_stype` share a namespace.
- `_instance` for `get_root()`: `1616` — mirrors the
`TCPAddress.get_root()` port and the `registry@1616.sock`
UDS filename, so the "1616 is tractor's registrar" idiom
holds across all backends.
- `_instance` for `get_random()`: TIPC gives us no
kernel-assigned-instance analogue of `port=0`, so we must
choose. Use a *pure* fn of the actor identity so it is
reproducible and well-distributed, **not collision-free**:
```python
# 32-bit instance derived from the actor's Aid.uid, or from a
# per-call token + pid when there is no live runtime.
inst: int = int.from_bytes(
blake2b(seed.encode(), digest_size=4).digest(),
'big',
)
```
where `seed = '.'.join(actor.aid.uid)` if
`current_actor(err_on_no_runtime=False)` else
`f'{prefix}.{uuid4().hex[:8]}@{pid}'`. Must avoid the reserved
low range: `inst = 64 + (inst % (2**32 - 64))`.
The UUID is load-bearing because TIPC names are cluster-wide
while PIDs are only host-local: `(actor name, pid)` can alias on
different hosts.
⚠️ *unlike* `port=0`, a collision here surfaces as a
successful-but-shared publication (TIPC allows multiple
binders on the same name and round-robins!) rather than
`EADDRINUSE`. That is a silent-crosstalk failure mode; §7 has
a statistical test and §9 records the unresolved recovery work
in [#501].
- `_scope`: `TIPC_NODE_SCOPE` for a same-host-only actor (the
UDS-equivalent), `TIPC_CLUSTER_SCOPE` (default) for
cluster-visible. **This is `.bindspace`**:
```python
@property
def bindspace(self) -> int:
return self._scope
```
It is the honest analogue of "the set of hosts this bind is
reachable from", which is precisely the docstring in
`Address.bindspace`. (`TIPC_ZONE_SCOPE` is deprecated/aliased
to cluster in modern kernels — accept it on input, normalize
to cluster, log at `transport` level.)
### 2.4 `is_valid`
```python
@property
def is_valid(self) -> bool:
return (
self._instance > 0
and
self._stype > 0
and
self._stype not in _tipc_reserved_stypes # {0, 1, ...}
and
self._scope in (TIPC_NODE_SCOPE, TIPC_CLUSTER_SCOPE)
)
```
---
## 3. Listener + stream
### 3.1 `start_listener()`
```python
async def start_listener(
addr: TIPCAddress,
backlog: int = 128,
**kwargs,
) -> SocketListener:
sock = trio.socket.socket(
socket.AF_TIPC,
socket.SOCK_STREAM,
)
# publish the singleton name-range == "register the service"
await sock.bind((
socket.TIPC_ADDR_NAMESEQ,
addr._stype,
addr._instance,
addr._instance,
addr._scope,
))
sock.listen(backlog)
return SocketListener(sock)
```
Notes / hazards:
- `bind()` on `AF_TIPC` is **not** a filesystem or port-table
operation and can't block on DNS, but keep it `await`ed
through `trio.socket` anyway for uniformity.
- `backlog=128` matching `_uds.start_listener()`'s hard-won
value (see its comment at `_uds.py:317-331` re: concurrent
deregistration storms). Do not use `1`.
- **no `close_listener()` needed** — nothing to unlink. Omit the
function entirely (contract §1.2: absence means implicit).
Withdrawal of the published name happens on socket close.
- `SocketListener.__init__` calls
`getsockopt(SOL_SOCKET, SO_ACCEPTCONN)`. The live-kernel probe
used by #493 answers `1`; retain the unit test so a kernel-side
change is visible rather than relying on trio's suppressed-
`OSError` carve-out.
- Wrap the bind in a `_reraise_as_connerr()`-style `@cm` (copy
the `_uds.py:256` pattern) so `EADDRINUSE`-ish and
`EAFNOSUPPORT` become `ConnectionError` with the addr in the
message. `EAFNOSUPPORT` here means "kernel module not
loaded" and deserves a *specifically actionable* message:
`'TIPC unavailable — try `sudo modprobe tipc`\n'`.
### 3.2 the `getsockname()` reconciliation
`Endpoint.start_listener()` does
`if lstnr.socket.getsockname() != self.addr.unwrap(): self.addr =
self.addr.from_addr(unwrapped)`.
For TIPC, `getsockname()` on a bound-but-listening socket
returns a `TIPC_ADDR_ID`-flavoured 5-tuple (the port id), *not*
the name-seq we bound. So the `!=` is **always true** and
`from_addr()` will be handed a 5-tuple.
`TIPCAddress.from_addr()` must accept only proto-keyed service
names. It must reject a bare port ID because no conversion can
recover `(stype, instance)`:
```python
@classmethod
def from_addr(cls, addr) -> TIPCAddress:
match addr:
# our proto-keyed tuple or decoded-list wire form
case (
('tipc', int() as stype, int() as inst, int() as scope)
|
['tipc', int() as stype, int() as inst, int() as scope]
):
return TIPCAddress(stype, inst, _norm_scope(scope))
# a bare kernel-observed TIPC_ADDR_ID 5-tuple has no
# service identity to annotate.
case (int() as atype, *rest) if atype == socket.TIPC_ADDR_ID:
raise ValueError(...)
```
The `TIPC_ADDR_ID` case cannot reconstruct `(stype, instance)`.
The resolution is the explicit listener-rebind policy added ahead
of the backend in #493:
```python
if (
(unwrapped := lstnr.socket.getsockname()) != self.addr.unwrap()
and
self.addr.rebind_from_sockname # ClassVar[bool] = True on tcp/uds
):
```
with `TIPCAddress.rebind_from_sockname: ClassVar[bool] = False`
(and `True` on `TCPAddress`/`UDSAddress`, preserving today's
behaviour exactly). Rationale: the reconciliation exists *only*
to learn the kernel-assigned port for `port=0` TCP binds (its
own comment says so, `_server.py:662`); TIPC has no such
late-binding, so opting out is semantically right rather than a
hack. Keep the guard test that TCP's `port=0` behaviour is
unchanged.
Do **not** annotate `Endpoint.addr` from `getsockname()`: the
listener endpoint must remain the dialable service name. Port IDs
are observed only on connected streams and may annotate a copy via
`with_port_id()` purely for logging/repr.
### 3.3 `MsgpackTIPCStream`
```python
class MsgpackTIPCStream(MsgpackTransport):
address_type = TIPCAddress
layer_key: int = 4
@property
def maddr(self) -> Multiaddr|str:
return mk_maddr(self.raddr)
def connected(self) -> bool:
return self.stream.socket.fileno() != -1
@classmethod
async def connect_to(
cls,
destaddr: TIPCAddress,
prefix_size: int = 4,
codec: MsgCodec|None = None,
**kwargs,
) -> MsgpackTIPCStream:
sock = trio.socket.socket(AF_TIPC, SOCK_STREAM)
with close_on_error(sock):
# NOTE: connect by *name* -> kernel does the lookup,
# so this is our "discovery" call.
await sock.connect((
socket.TIPC_ADDR_NAME,
destaddr._stype,
destaddr._instance,
0, # domain: 0 == "anywhere in scope"
destaddr._scope,
))
stream = trio.SocketStream(sock)
return cls(
stream,
prefix_size=prefix_size,
codec=codec,
)
```
- reuse `trio._highlevel_open_unix_stream.close_on_error` (the
UDS backend already imports it) or inline the equivalent
`try/except: sock.close(); raise`.
- `SO_/TIPC_` opts worth setting and documenting:
- `setsockopt(SOL_TIPC, TIPC_IMPORTANCE, TIPC_HIGH_IMPORTANCE)`
for the *parent<->child* lifetime channel — this is a real
win TIPC gives us that TCP can't: the runtime's
supervision channel can outrank bulk app traffic under
congestion. Wire it as a `connect_to(..., importance=...)`
kwarg defaulted from a module constant, and have
`_runtime.py`'s parent-chan path pass the high value **in a
follow-up** (don't couple it to this PR).
- `TIPC_CONN_TIMEOUT` — the kernel-side connect timeout;
leave at default, we have `trio` cancel scopes.
- `TIPC_DEST_DROPPABLE = 0` on the connection so undeliverable
msgs come back as errors rather than being silently dropped.
- **`connect_to()` on a name with no publisher**: the live-kernel
result is immediate `EHOSTUNREACH`. Python exposes that as a
bare `OSError`, not a `ConnectionError` subtype, so
`_reraise_as_connerr()` is load-bearing for contract §4. Keep
the exact errno and normalization under test.
### 3.4 `get_stream_addrs()`
```python
@classmethod
def get_stream_addrs(cls, stream) -> tuple[TIPCAddress, TIPCAddress]:
sock = stream.socket
# both return TIPC_ADDR_ID 5-tuples for a connected sock
l_id = sock.getsockname()
r_id = sock.getpeername()
...
```
Problem: neither end's port-id tells us the *service name*. The
`laddr`/`raddr` are used for logging, `Channel.raddr`,
`Server._peers` keying-adjacent repr, and `maddr`. Design:
- `get_stream_addrs()` converts both socket results into
**observed-only** addresses: `_stype`/`_instance` use the
`TIPC_NAME_UNKNOWN = -1` sentinel and `maybe_node`/`maybe_ref`
carry the port ID. Such addresses are invalid for dialling.
- the **connecting** side knows the service name it dialled, so
`connect_to()` replaces `_raddr` after construction with that
known `TIPCAddress` while retaining the constructor's one
tolerant port-ID observation. Do not call `getpeername()` a
second time: the peer can withdraw between the two calls.
- the **accepting** side genuinely cannot recover the peer's
service name from a port ID. Keep the observed-only `raddr`;
the handshake's `Aid` supplies logical identity. Piggybacking a
bound name in the handshake is outside this backend.
- `laddr` is observed-only as well. It is used for repr/logging,
not to replace the endpoint's known service name.
- unlike TCP/UDS, TIPC can answer `ENOTCONN` from
`getpeername()` after a connect-then-drop. This lookup happens
during `MsgpackTransport` construction, before handshake error
tolerance. Wrap `getsockname()` and `getpeername()` in a
tolerant helper and degrade to a port-ID-less observed address;
a dropped peer must cost an observation, not kill the actor.
---
## 4. Multiaddr representation
There is no `/tipc` in the multiaddr protocol table. Interim
grammar, mirroring how `uds` maps to the spec-legal `/unix`:
```
/tipc/<stype>/<instance> # scope implied = cluster
/tipc/<stype>/<instance>/<scope> # explicit
```
- `_tpt_proto_to_maddr['tipc'] = 'tipc'` and a `mk_maddr()`
`case 'tipc':` building the above.
- `parse_maddr()` gets `case ['tipc']:` — but note
`py-multiaddr` will reject an unregistered protocol name
outright, so this **requires an upstream registration** (same
track as the `wg` work, gh #483 /
multiformats/py-multiaddr#107). Until that lands:
- `MsgpackTIPCStream.maddr` returns the **`str`** form (the
`MsgTransport.maddr` return type is already
`Multiaddr|str`, and `MsgpackUDSStream.maddr` already
exercises the `str` branch), and
- `parse_maddr()` special-cases the `/tipc/` prefix *before*
handing the string to `Multiaddr()`.
Document this as the reason gh #443's "standardize on
returning `Multiaddr` everywhere" item stays blocked.
Propose `/tipc/` upstream as: name `tipc`, code TBD, size
variable, value `<stype>:<instance>:<scope>` — or as three
composed protos. Prefer *one* proto with a structured value so
the maddr stays 2-segment like `/unix/...`.
---
## 5. Discovery primitives and explicit limits
The backend provides independently-shippable kernel primitives.
Neither primitive alone implements Tractor's actor-name discovery,
registry ownership, or registrar election.
### 5.1 Layer A — "discovery by bind" (free)
Because `bind(TIPC_ADDR_NAMESEQ)` publishes and
`connect(TIPC_ADDR_NAME)` resolves, a caller that **already knows**
a TIPC service address can dial it without a registrar lookup.
This is narrower than registrar-less `find_actor(name)`:
- `tractor.discovery._api.find_actor()` and peers still query a
registrar; #493 does not change them.
- deriving a stable service address from `(name, uuid)` and
dialling it directly is follow-up [#499]. The mapping must be
documented and cross-language stable.
- `registry_addrs` still identify registrars. Connecting to a
known registrar by TIPC name removes no registrar bookkeeping
or ownership semantics.
There is also an unresolved **split-brain election** problem.
Duplicate TIPC name publication succeeds and round-robins, so two
roots can both probe an unoccupied registrar name, both bind it,
and both believe they won. The backend provides no atomic
compare-and-publish, lease, quorum, or deterministic winner. A
topology subscription can reveal multiple publisher port IDs but
does not elect or fence one. Do not describe registrar election as
solved until a separate protocol closes this race.
### 5.2 Layer B — the topology service (`TIPC_TOP_SRV`)
This is the push primitive behind #378's "end game cluster proto"
direction: a subscription to kernel name-table publish/withdraw
events. #493 implements `open_topology_events()`; consuming that
feed in `discovery._registry` is follow-up [#496]. Until then it
does not replace registrar state or `find_actor()`.
Mechanics, verified against `linux/include/uapi/linux/tipc.h`,
`net/tipc/topsrv.c` and #493's live-kernel probe:
```python
# SOCK_SEQPACKET connected to the topology server
sock = trio.socket.socket(AF_TIPC, SOCK_SEQPACKET)
await sock.connect((
socket.TIPC_ADDR_NAME,
socket.TIPC_TOP_SRV, # == 1
socket.TIPC_TOP_SRV,
0,
))
# struct tipc_subscr {
# struct tipc_name_seq seq; /* 3 * __u32: type, lower, upper */
# __u32 timeout; /* TIPC_WAIT_FOREVER == ~0 */
# __u32 filter; /* TIPC_SUB_{PORTS,SERVICE,CANCEL} */
# char usr_handle[8];
# } /* == 28 bytes */
_SUBSCR_FMT: str = '=5I8s'
```
- **byte order**: #493's live-kernel probe verified native
standard-size (`'='`) packing for publish and withdraw events.
Use `'=5I8s'` for the 28-byte subscription. Do not retain the
speculative `'>'` retry/probe as if it were required. Preserve
the earlier `# ?TODO` to verify the deterministic rule directly
against `net/tipc/topsrv.c`; it is source-audit work, not a
runtime retry requirement.
- **events**: `struct tipc_event` is `event: u32`,
`found_lower: u32`, `found_upper: u32`,
`port: {ref: u32, node: u32}`, then the 28-byte subscription
echo: **48 bytes** (`4 + 4 + 4 + 8 + 28`), not 40. Use
`'=10I8s'` and assert `struct.calcsize(...) == 48`.
`event ∈ {TIPC_PUBLISHED, TIPC_WITHDRAWN,
TIPC_SUBSCR_TIMEOUT}`. Python exposes `TIPC_WAIT_FOREVER` as
`-1`, so mask it with `& 0xFFFF_FFFF` before packing an
unsigned `I`.
- **trio shape** — this is where the "nearly-functional,
modern-async" style pays off; expose it as an `@acm` yielding
a `trio` receive-channel of typed events, *not* a class:
```python
@acm
async def open_topology_events(
stype: int = TRACTOR_STYPE,
lower: int = 0,
upper: int = 0xFFFFFFFF,
filter: int = TIPC_SUB_SERVICE,
timeout: int = TIPC_WAIT_FOREVER,
buf_size: int = 64,
) -> AsyncGenerator[
trio.MemoryReceiveChannel[TIPCNameEvent],
None,
]:
...
```
with `TIPCNameEvent(msgspec.Struct, frozen=True)` fields
`kind: Literal['published','withdrawn','timeout']`,
`addr: TIPCAddress`, `node: int`, `ref: int`. One
`trio.lowlevel`-free implementation: a nursery-spawned reader
task doing `await sock.recv(48)` in a loop. The feed is
authoritative and may neither block the socket reader nor drop
transitions silently. Use `send_nowait()` and, on
`trio.WouldBlock`, raise a dedicated
`TIPCNameEventOverflow` that aborts the subscription and tells
the consumer to resubscribe and rebuild its view. A timeout
event is delivered once and then closes the channel. The
`@acm` cancels its reader before closing the fd so teardown
cannot race a retried `recv()` into `EBADF`.
- **scope**: topology events carry no publication scope. Use an
explicit unknown-scope sentinel and keep the resulting address
non-dialable; never copy caller/subscription context into
supposedly observed data.
- **consumer**: [#496] owns the optional watch mode and the
decision whether the feed subsumes or merely accelerates
existing registrar bookkeeping.
- **`SOCK_SEQPACKET` is fine here** because this socket never
goes through `MsgpackTransport` — it's a plain trio socket
used with `recv()`. The contract's "`SOCK_STREAM` only"
constraint applies to `MsgTransport` streams, not to this.
---
## 6. Commit sequencing (each independently reviewable + green)
1. `_server.py`: add `Address.rebind_from_sockname:
ClassVar[bool]`, gate the `getsockname()` reconciliation on
it, `True` for tcp/uds. Test: tcp `port=0` unchanged.
2. `tractor/ipc/_tipc.py`: `TIPCAddress` + `is_tipc_available()`
predicate + `start_listener()`. No transport yet.
Tests: address round-trip (`unwrap`/`from_addr`/`wrap_address`),
`get_random()` distribution, bind/listen + `SO_ACCEPTCONN`
tolerance, `EAFNOSUPPORT` → actionable `ConnectionError`.
3. `MsgpackTIPCStream` + `connect_to()` + `get_stream_addrs()`.
Test: two `trio` tasks in one proc exchange a msg over
`Msgpack` framing (no `tractor` runtime).
4. registration tables (contract §2 items 1-8 and 10) +
`pyproject.toml` mark/extra. Test: full suite under
`--tpt-proto tipc` (§7.3). Keep TIPC in the conservative remote
preference tier until a follow-up implements and tests contract
item 9's node-scope locality policy.
5. maddr support (`str` form + prefix special-case) + docs.
6. `open_topology_events()` @acm + its tests (layer B).
7. docs page + `docs/` example.
Per project convention, a reproducing/guard test lands in its
own commit **before** the fix it guards.
---
## 7. Testing
### 7.1 the capability predicate (in `_tipc.py`, public)
```python
def is_tipc_available() -> bool:
'''
True iff this kernel can create an `AF_TIPC` socket, i.e.
the `tipc` module is loaded.
'''
if sys.platform != 'linux':
return False
try:
socket.socket(socket.AF_TIPC, socket.SOCK_STREAM).close()
return True
except OSError:
return False
```
Do not permanently memoize the result: `modprobe tipc` and module
removal can change it during a long-lived process. Probe once per
runtime startup, or use an explicitly refreshable cache whose
owner invalidates it after module-management operations. The
predicate itself remains side-effect-free and silent.
### 7.2 gating
- `pytest.mark.tipc` registered in
`_testing/pytest.py::pytest_configure()` via
`config.addinivalue_line()`, where this repo declares its other
custom marks. Do not invent a `pyproject.toml` marker table.
- keep pure address, serialization, and topology-codec tests
runnable on every host. Apply a shared `requires_tipc` marker
only to tests that create sockets or otherwise touch the kernel;
do not module-skip `tests/ipc/test_tipc.py`.
- `--tpt-proto tipc` with no module must fail **loudly and
early** with the actionable message, not with 400 confusing
timeouts. Add the check to the `tpt_protos` fixture's existing
per-proto validation loop (`_testing/pytest.py:795`): if the
chosen `Address` type exposes an `is_available()`-style
classmethod, call it and `pytest.fail()` with its reason.
Generalize (don't special-case tipc) — plans 02/03 need the
same hook.
### 7.3 CI
- add a job matrix entry `--tpt-proto tipc` that runs
`sudo modprobe tipc` in a `before` step. GH's
`ubuntu-latest` runners do allow `modprobe tipc` (the module
ships with the standard Ubuntu kernel package); verify in a
throwaway workflow before wiring the matrix. #493's TIPC leg
is now blocking. If runners cease permitting the module load,
fix the environment or use a suitable container rather than
silently restoring `continue-on-error`.
- cross-node TIPC (bearer) cannot be CI'd; cover it with a
documented manual smoke test in the docs page, in the style
of gh #482's LAN examples.
### 7.4 backend-specific tests worth writing
- **known-name publication/resolution**: bind a listener on
`(stype, inst)`, then from a second task `connect()` by name
and assert it lands — *without* any `tractor` registrar.
- **`get_random()` distribution**: 10k `get_random()` calls with
no live runtime. Do **not** assert 10k distinct values: the
no-runtime seeds and outputs are both only 32 bits. Including
duplicate seeds plus distinct-seed hash collisions puts the
modeled chance of at least one duplicate near 2.3% for 10k
calls. #493 uses `>= n - 2` (modeled probability of more than
two collisions around `2e-6`) and separately proves
`instance_from_seed()` is a pure function. Also hold actor
name/PID fixed while varying only `Aid.uuid` to prove live
actors seed from `Aid.uid`.
- **round-robin surprise**: two listeners bound to the *same*
`(stype, inst)` both succeed (TIPC allows it) and connects
distribute. Assert the observed behaviour and reference it
from the `get_random()` docstring so the next reader knows
why the hash matters.
- **scope isolation**: a `TIPC_NODE_SCOPE` bind is not visible
to a cluster-scope lookup from another node (manual/marked).
- **importance opt** round-trips via `getsockopt`.
- **graceful + abrupt close** produce `TransportClosed` with the
same `loglevel` classification as tcp/uds — i.e. re-run the
relevant `tests/ipc/test_each_tpt.py` cases parametrized over
the new proto rather than writing new ones.
---
## 8. Deployment / docs deliverable
A `docs/` page (and/or an `examples/` script) covering:
```bash
# single host, node-scope only
sudo modprobe tipc
tipc node get addr
# multi-host over ethernet (pairs beautifully with plan 03's wg)
sudo tipc bearer enable media eth device eth0
# ...or over UDP when L2 isn't available:
sudo tipc bearer enable media udp name uc localip 10.0.11.1
tipc link list
tipc nametable show # <- see tractor's published services!
```
`tipc nametable show` displaying live `tractor` actors is the
single best demo this backend has; lead with it.
---
## 9. Known risks + escalations
- **Instance collision / silent crosstalk remains unresolved.**
`Aid.uid` seeding and §7.4 tests reduce and measure risk, but
the instance field is still a hard 32 bits. [#501] owns
post-bind verification and recovery. Do not fold bits into
`_stype`: topology can watch only one service type.
- **Concurrent registrar startup can split brain.** Topology can
observe duplicate publisher port IDs but cannot elect or fence
a winner; a separate election protocol is required (§5.1).
- **Kernel/module availability is opt-in.** Keep the hard gate in
§7.2; TIPC is never the default transport.
- **A listener sockname is a port ID, not its service name.** Keep
the `rebind_from_sockname` opt-out (§3.2).
- **`/tipc` is not yet a registered multiaddr protocol.** Keep
the interim `str` maddr fallback (§4) and upstream gh #483.
- **The public TIPC docs can be stale.** Treat
`include/uapi/linux/tipc.h` and `net/tipc/` as normative and
cite file/symbol names in code comments.
- **A slow topology consumer loses continuity.** Fail fast with
`TIPCNameEventOverflow`; resubscribe and rebuild rather than
block the reader or retain stale state (§5.2).
- **TIPC locality preference is not implemented.** Current
`_is_local_addr()` handles only UDS and TCP, so node- and
cluster-scope TIPC both remain in the conservative remote tier.
Add explicit scope-aware policy and multihomed selection tests
before claiming node-scope preference (contract §2.9).
### 9.1 remaining constructor/error cleanup
#493 closes the peer-withdrawal race in transport construction,
but it is not a blanket error-path cleanup. Keep these gaps
explicit rather than reporting the backend as fully hardened:
- direct `TIPCAddress(...)` construction bypasses
`from_addr()` scope normalization; `is_valid` is queried later
rather than enforcing validity at construction. Decide whether
constructors should reject bad service types/instances/scopes
or document direct construction as trusted-internal.
- `maybe_node`/`maybe_ref` are excluded from `.unwrap()` but, as
`msgspec.Struct` fields, still participate in structural
equality/hash. If service-name identity must ignore observation
metadata, represent or compare it explicitly instead of relying
on the current "observed-only" description.
- `start_listener()` must keep ownership of the raw socket through
`bind()`, `listen()` and `SocketListener(...)`. The downstream
implementation normalizes bind errors but does not yet wrap the
complete listener-construction sequence in close-on-error, so a
later setup failure can leak the fd.
- `_maybe_sockaddr()` currently degrades every `OSError` to an
unknown observed address. Narrow that tolerance to expected
peer-withdrawal errors (notably `ENOTCONN`) so unrelated bad-fd
or programming failures remain visible.
- error normalization is intentionally required for an
unpublished-name `EHOSTUNREACH`, but setup `setsockopt`,
listener-constructor, and topology setup failures still need a
consistent policy and focused regression tests.
## 10. Follow-up issue seeds
- **register `/tipc` in the multiaddr spec**, mirroring the `wg`
track (multiformats/py-multiaddr#107/#108 + gh #483). Same
shape of work: propose the proto + code, land a codec in
`py-multiaddr`, then drop our `str`-maddr fallback (§4). Worth
filing *alongside* the `wg` spec-submission issue so both
proposals go up together rather than as one-offs.
- registrar-less discovery fast path via name derivation ([#499],
§5.1)
- `TIPC_TOP_SRV`-driven push registry in
`discovery/_registry.py` ([#496], §5.2)
- post-bind collision verification and recovery ([#501], §9)
- `TIPC_IMPORTANCE` for the parent<->child lifetime channel
(§3.3) — genuinely novel supervision QoS, no other backend
can do it
- TIPC multicast / group messaging as a *broadcast* transport
for `tractor.trionics` fan-out (explicitly not `MsgTransport`)
- dual-link resiliency / multi-homing (#378's "hybrid dual link")
once bearers are scripted in the docs
[#496]: https://github.com/goodboy/tractor/issues/496
[#499]: https://github.com/goodboy/tractor/issues/499
[#501]: https://github.com/goodboy/tractor/issues/501

701
ai/tpt-backends/02_quic_iroh_backend.md 100644
View File

@ -0,0 +1,701 @@
# Plan 02 — QUIC backend via `iroh` FFI, uniffi-async rewritten onto `trio`
Tracks gh [#353]. Prereq reading:
[`00_shared_backend_contract.md`](./00_shared_backend_contract.md).
**External-fact rule**: every claim here about `iroh`, UniFFI,
generated bindings, QUIC wire/security behavior, or multiaddr
support is provisional until the step-0 API-truth pass records a
source or probe. Tractor/Trio behavior read from this checkout is
the only locally proven basis for the plan.
**Thesis**: the value of `iroh` over "just QUIC" is
`NodeId`-addressed, NAT-traversing, relay-fallback endpoints —
i.e. a `tractor` actor tree that spans hosts *without* a
reachable listening socket. The cost is that `iroh`'s python
surface is `uniffi`-generated **asyncio** and its listener is not
a socket. This plan spends its complexity budget in exactly two
places: a `trio`-native uniffi future bridge, and a
`trio.abc.Listener`/`Stream` adapter pair. Everything else is
contract boilerplate.
[#353]: https://github.com/goodboy/tractor/issues/353
---
## 1. Library selection (decided, with the rejected alternatives)
**Chosen: `iroh` (PyPI, from `n0-computer/iroh-ffi`), pinned to
a single minor.** The `iroh` python package is a `uniffi`
binding over the rust `iroh` crate (QUIC via `quinn`/`noq`).
Rejected, and why — record these so the next implementer doesn't
relitigate:
- **`aioquic`** (sans-io + asyncio): genuinely trio-portable
(`hypercorn` already pairs its sans-io core with a trio UDP
server, see the links in #353) and dependency-light. But it
gives us *only* QUIC — no NodeId identity, no hole punching,
no relay. We'd be reimplementing iroh's whole reason for
existing. **Keep as the documented fallback** if the FFI
bridge (§2) proves unmaintainable; the `MsgTransport` and
`Listener` adapters from §3 are ~90% reusable against an
`aioquic` core, which is a deliberate design property of this
plan.
- **`quiche` / `quinn` via a hand-rolled PyO3 ext**: strictly
more work than reusing `iroh-ffi`, and puts us in the
build-wheels business.
- **`trio-asyncio`**: viable *shortcut* to run the asyncio-shaped
bindings under trio, and `tractor` already ships
infected-asyncio machinery (`tractor.to_asyncio`,
`tests/test_infected_asyncio.py`). Rejected as the *primary*
design because it makes every IPC send/recv cross a
loop-boundary shim in the hot path, and because #353 asks
explicitly for the asyncio support to be "rewritten for trio".
**But**: build it first as the throwaway spike (§6 step 0) to
de-risk the iroh API surface before writing the bridge.
Version pinning: treat API stability across `iroh` minors as an
**unverified external constraint** until step 0. Pin the version
exercised by the spike to `iroh>=X.Y,<X.Y+1` in a `quic` extra,
and **write down the exact resolved version + generated
`iroh/_uniffi*` module layout** in the module docstring, because
§2 depends on generated-code internals.
**Step 0 of implementation is an API-truth pass**: install the
pinned `iroh`, inspect both its generated Python and loaded FFI
symbols, and run the throwaway two-process spike. Record in
§1.1 the real names and observed contracts. Every statement
below about `iroh`, UniFFI, Rust callbacks, or generated symbols
is a **step-0 hypothesis**, not a locally proven fact, unless it
is copied into the completed API-truth table with a source or
probe. Tractor and Trio behavior cited from this checkout is not
subject to that qualifier.
### 1.1 API-truth table (fill in during step 0)
| concept | provisional name | actual (fill in) |
| --- | --- | --- |
| secret key | `iroh.SecretKey.generate()` | |
| endpoint builder | `iroh.Endpoint.builder(...).bind()` | |
| node id | `endpoint.node_id() -> str` | |
| node addr (relay + direct) | `iroh.NodeAddr` | |
| dial | `await endpoint.connect(node_addr, alpn)` | |
| accept conn | `await endpoint.accept()` | |
| open bi-stream | `await conn.open_bi()` | |
| accept bi-stream | `await conn.accept_bi()` | |
| send | `await send_stream.write_all(b)` | |
| recv | `await recv_stream.read(n) -> bytes\|None` | |
| half-close | `await send_stream.finish()` | |
| endpoint close + completion | `close()` / `await closed()` | |
| resolved node address | relay URL + direct socket addrs | |
| future start/poll callback ABI | generated symbols + args | |
| future cancel/complete/free | generated symbols + ordering | |
| callback quiescence guarantee | after poll/complete/free? | |
| cancellation terminal poll code | generated enum/value | |
| iroh exception/status taxonomy | per operation | |
---
## 2. The `trio`-native uniffi future bridge (`tractor/ipc/_uniffi_trio.py`)
### 2.1 Step-0 generated-ABI gate
The expected generated shape is: start an opaque Rust future,
poll it with a C callback, cancel through a generated cancel
symbol, consume its terminal value/status through `complete`,
then call `free`. The expected callback may arrive on a foreign
Rust thread. **All of that is external and provisional.** Step 0
must identify the exact generated driver and prove, from its
template/source plus probes:
1. the start, poll, cancel, complete, and free signatures for
every return-type family used by `iroh`;
2. poll result values and whether callbacks can be synchronous,
concurrent, repeated, or late;
3. which terminal state permits `complete`, when `free` is
legal, and when no callback can still reference Python;
4. whether generated callback-data and call-status objects must
remain alive, and how generated lifting/errors are applied;
5. whether one narrow generated async-driver entrypoint can be
replaced without importing or requiring an asyncio loop.
Do not implement from a remembered UniFFI version. If cancel
does not have a documented path to a terminal, safely freeable
state, the native Trio bridge fails the spike gate and the first
backend uses the infected-asyncio fallback.
### 2.2 Cancellation-safe ownership
Do not let the caller task own a raw handle across an `await`.
Introduce an actor-scoped `UniffiFutureSupervisor` running in the
dedicated transport nursery specified in §3.2.1. That nursery
must span parent bootstrap, the service nurseries, and final
deregistration. For each call, its operation task owns the
**entire** generated lifecycle:
```text
create handle -> poll/callback loop -> complete -> lift/status
-> free -> publish result
^
cancel request uses generated cancel, then follows
the verified terminal poll/complete/free protocol
```
The operation task, not the awaiting caller, creates the handle.
Creation and insertion in the supervisor's live-operation set
must have no cancellation checkpoint between them. The operation
retains strong references to the C callback trampoline, callback
data, wake state, call status, and handle until step 0 proves all
callbacks are quiescent and `free` has returned. Use one stable
callback per operation unless the verified ABI requires a fresh
one per poll; in either case, retain every potentially callable
trampoline. Capture `current_trio_token()` in the Trio owner and
schedule the wake into Trio with `token.run_sync_soon(...)`; the
foreign callback only stores its poll result and schedules that
wake.
Caller cancellation is a request, not handle ownership transfer:
1. the caller sends an idempotent cancel request and waits under
a short shield for the operation to acknowledge it;
2. the owner invokes the generated cancel function exactly once
and continues the **verified** poll/complete/free sequence;
3. once caller cancellation is observed, cleanup completion never
wins the race by returning a value. After acknowledgement the
caller continues propagating its original Trio cancellation;
if cleanup outlives the grace period it first abandons its
result channel while the actor supervisor keeps ownership;
4. actor endpoint teardown stops accepting new calls, requests
cancellation of all live operations, and joins the supervisor
before destroying endpoint/key state.
There is deliberately no `move_on_after(...): free(handle)`
path. A timeout proves only that cleanup is slow; it does not
prove that callbacks are quiescent or that `free` is legal. A
wedged operation therefore remains visible in the supervisor and
can delay graceful actor shutdown; process-level termination is
the final escalation, not an unsafe FFI free.
Structured-concurrency race to test: caller cancellation may land
after handle creation, after each poll, during callback delivery,
after terminal readiness, during `complete`, and before result
publication. At every checkpoint exactly one operation task owns
the handle, exactly one `free` is possible, and the supervisor
cannot exit while that task or a callable trampoline remains.
### 2.3 how to apply it to the generated bindings
Do **not** fork/vendor the generated `iroh` Python. Subject to the
step-0 gate, ship a *narrow* re-dispatch shim:
- write `tractor/ipc/_uniffi_trio.py` with the supervisor and a
`@cm patch_uniffi_for_trio()` that patches only the generated
async-driver entrypoint recorded in §1.1;
- verify at import time that the expected symbol exists and
raise a clear, actionable error naming the pinned `iroh`
version if not. A silent fallback to asyncio would be a
nightmare to debug.
- treat every `iroh`/UniFFI upgrade as requiring the step-0 ABI
gate again. Keep a test that drives one trivial call under bare
`trio.run()`, asserts no asyncio loop, and injects cancellation
at every lifecycle checkpoint. Point the module docstring at
the exact generated template/revision mirrored by the shim.
If step 0 reveals the generated code is *structurally* hostile
to this (e.g. `asyncio` imported and used at module scope for
more than the driver), fall back to option (b): run iroh under
`tractor.to_asyncio` infected mode and open the follow-up to
revisit. Say so in the PR rather than fighting it.
---
## 3. Mapping QUIC onto `MsgTransport`
### 3.1 the layering decision
QUIC natively multiplexes streams inside one connection. The
mapping that preserves *all* existing `tractor` semantics with
the least new code:
```
iroh Endpoint == one per actor (process) -> the "listener"
iroh Connection == one per peer actor -> pooled
iroh bi-stream == one `Channel`/`MsgTransport` -> 1:1
```
- keep the 4-byte `<I` length-prefix framing **unchanged**. It's
redundant-ish over a QUIC stream but it means
`MsgpackTransport` is reused verbatim, and framing is cheap.
Revisit only after it works.
- **one-task-per-stream** falls out naturally, which is exactly
the #353 note about QUIC sub-stream QoS/cancellation fitting
`trio`.
- `layer_key: int = 4` still (QUIC is L4-ish); note in a comment
that this backend is really 4+security+multiplex.
**Connection pooling** is actor-endpoint state, never module
state. Its key is exactly
`(local_endpoint_identity, remote_node_id, alpn)`, where local
endpoint identity is the local NodeId derived from the actor key.
Remote NodeId alone would incorrectly share connections across
local keys or protocol epochs. Build it over the codebase's
`maybe_open_context()` idiom only after a concurrency review of
its actual last-user teardown behavior in the implementation
revision. Do not assume an issue reference proves the required
ordering.
`acquire_connection()` returns a `ConnectionLease`, not a bare
connection. An outgoing `QuicMsgStream` owns that entered lease
for its whole lifetime; `connect_to()` must not exit the cached
context immediately after `open_bi()`. Exact transfer paths:
- dial/acquire or `open_bi()` failure releases the lease in a
shielded `finally` before raising;
- successful stream construction atomically transfers the lease
to `QuicMsgStream` before the first cancellation checkpoint;
- `send_eof()` closes only the send half and does not release;
- clean receive EOF closes only the receive half and does not
release while the send half remains usable;
- one guarded terminal-state transition releases exactly once
when both halves have become terminal, in either order;
- `aclose()`, reset, or terminal connection failure closes both
halves as applicable and idempotently releases exactly once;
- a stream queued by `QuicListener` already owns its lease; if
never accepted, listener draining closes it and releases it.
After `accept()` returns, the server dispatch path owns the stream
until a handler task starts and must close it if task start fails.
The handler then takes ownership, with an outer `finally` that
calls `stream.aclose()` on normal return, handshake failure, and
cancellation. Lease release itself is an idempotent pool state
transition; if last-user connection teardown awaits FFI, the actor
endpoint's pool supervisor owns that await so cancellation of the
handler cannot strand the lease.
For inbound connections, the connection-feeder owns a base lease
while accepting streams and each queued/returned stream gets a
child lease. The base lease is released only after the accept
loop ends; the pool closes the connection after the base and all
stream leases are gone. Reject or deterministically reconcile a
simultaneous inbound/outbound duplicate for the same full key;
record the chosen iroh-compatible rule during step 0.
### 3.2 `IrohAddress`
```python
class IrohAddress(
msgspec.Struct,
frozen=True,
):
_node_id: str
_alpn: str
_relay_url: str|None
_direct_addrs: tuple[str, ...]
proto_key: ClassVar[str] = 'quic'
unwrapped_type: ClassVar[type] = tuple
def_bindspace: ClassVar[str] = 'tractor/0'
```
- **`.unwrap()` is the complete, tagged wire descriptor**:
`('quic', node_id, alpn, relay_url, direct_addrs)`. All values
are msgpack-native and `direct_addrs` is canonicalized to a
tuple. `from_addr()` requires that exact tag and shape; never
infer QUIC from a `(str, str)` pair. This depends on the shared
contract's tagged-address migration and removes the UDS
collision rather than ordering around it.
- The descriptor always carries NodeId, ALPN, and both route-hint
fields. For this discovery-free first backend, `.is_valid`
requires a parseable NodeId, non-empty ALPN, and at least one
relay URL or direct address. Whether NodeId-only dialing works
through optional iroh discovery is a step-0 API check and is
not part of the first implementation.
- `.bindspace` → `self._alpn`. This is the honest analogue:
the ALPN is the set of endpoints willing to talk to you, and
two `tractor` deployments sharing an iroh network are
separated by ALPN exactly as two UDS deployments are
separated by directory. Include a `tractor` version/proto
epoch in the default ALPN so incompatible runtimes can't
handshake.
### 3.2.1 One actor endpoint and key
Add an actor-scoped `QuicActorEndpoint` resource containing the
secret key, one bound iroh endpoint, the UniFFI supervisor, the
connection pool, and its latest resolved `IrohAddress`. It cannot
live in `_service_tn`: a child dials its parent before that nursery
opens, while final deregistration may dial after it closes.
Add a dedicated `transport_tn` around the complete actor runtime:
the task that opens this nursery must start the complete
`async_main` sequence as a **child** of it and wait for that child.
That makes `transport_tn` an ancestor of every parent-dial,
service, and deregistration caller, satisfying
`maybe_open_context(tn=transport_tn)` rather than asking the
nursery-opening task to use its own child nursery. The child keeps
the nursery around `_root_tn` and `_service_tn`, performs final
deregistration while it remains open, then returns so the owner can
close the transport resource and nursery. Root startup needs the
equivalent outer owner around actor construction, service, and
teardown. If this shape cannot be preserved, the connection pool
must stop depending on `maybe_open_context()`'s ancestor-nursery
contract. No path creates a second endpoint for the actor.
The child currently receives transport configuration only in the
`SpawnSpec` sent over its already-open parent channel. QUIC cannot
derive its local key, ALPN, or requested bind policy from that late
message. Add a small msgpack/pickle-native
`ChildTransportBootstrap` to every process-launch path. It carries
the selected protocol and the QUIC-local key reference/generation
policy, ALPN, relay policy, and requested bind constraints. It is
available before `_from_parent()`; the later `SpawnSpec` repeats
the public configuration and startup rejects any mismatch. Root
actors derive the same bootstrap record directly from
`open_root_actor()` inputs before address selection.
With that prep in place, the order is:
1. consume the launch-time bootstrap record, select one key
(persisted and explicitly provisioned for a registrar, fresh
for an ordinary actor), and construct/bind the endpoint in the
transport owner task;
2. await the step-0-verified address-ready API and build a valid
descriptor from the endpoint's NodeId, ALPN, relay URL, and
direct addresses;
3. only then dial `_from_parent()` through this endpoint;
4. start `QuicListener` over this endpoint's accept API;
5. publish the resolved descriptor as `Endpoint.addr` and
`Actor.accept_addrs` before parent/registrar registration;
6. after service nurseries close, keep the endpoint available for
deregistration; then close listeners and streams, drain
connection leases and FFI operations, close/join the endpoint,
and release key state.
`IrohAddress.get_random()` is therefore a descriptor lookup on
the active actor transport resource, not key generation. Broaden
the shared `get_random()` contract for resource-backed transports
and make root/subactor address selection consume the bootstrap
resource instead of calling it before that resource exists. Do not
hide a secret in a module-level side table. Calls without an active
resource fail clearly rather than allocating an unowned key.
`get_root()` never returns an empty/sentinel NodeId. Make default
addresses lazy, and have QUIC load a provisioned public registrar
descriptor. Registrar provisioning writes its secret separately
with mode 0600 and writes the matching complete public descriptor
atomically; endpoint startup verifies the derived NodeId. If no
descriptor exists, default QUIC registrar discovery fails with an
actionable configuration error. Automatic first-process election
is deferred until a safe key-file locking and endpoint-binding
protocol is proven; key generation never occurs in the listen
path.
#### 3.2.2 `proto_key`: `'iroh'` vs `'quic'`
Use **`'quic'`** for the `proto_key`/`--tpt-proto` name and
name the module `_quic.py`, with `iroh` as the *implementation*.
Rationale: it keeps the door open for the `aioquic` fallback
(§1) without a user-visible rename, and it matches how `uds` is
a proto name rather than a lib name. Put `iroh`-specific bits
behind an internal `_iroh` submodule if the file gets big.
### 3.3 the `trio.abc` adapters — where the real work is
Contract §3 says a non-socket backend needs three upstream
generalizations. Land them **as a prep PR, before any iroh
code**, so they can be reviewed on their own merits with
tcp/uds still the only backends:
1. **`Endpoint.start_listener()` must not assume
`.socket.getsockname()`.** Use the same
`Address.rebind_from_sockname: ClassVar[bool]` gate that
plan 01 §3.2 introduces — coordinate so it lands once. (If
plan 01 lands first, this is free.)
2. **`transport_from_stream()` (`_types.py:92`) must not assume
`trio.SocketStream`.** Replace the `sock.family` match with:
check `isinstance(stream, trio.SocketStream)` → existing
family match; else look for a
`stream.tpt_key: ClassVar[MsgTransportKey]` attribute on the
adapter and use it. Keeps the existing path byte-identical
and makes new stream types self-describing (a much better
shape than growing an `isinstance` ladder).
3. **type annotations**: `handle_stream_from_peer(stream:
trio.SocketStream)` → `trio.abc.Stream`; `Endpoint._listener:
SocketListener|None` → `trio.abc.Listener|None`;
`MsgTransport.stream: trio.SocketStream` →
`trio.abc.Stream`. Annotation-only, zero behaviour change.
Then the adapters:
```python
class QuicMsgStream(trio.abc.HalfCloseableStream):
'''
A single `iroh` bi-directional QUIC stream presented as
a `trio` byte-stream so `MsgpackTransport` can frame over
it unmodified.
'''
tpt_key: ClassVar[MsgTransportKey] = ('msgpack', 'quic')
def __init__(self, conn, send, recv, lease) -> None: ...
async def send_all(self, data: bytes) -> None: ...
async def wait_send_all_might_not_block(self) -> None: ...
async def receive_some(self, max_bytes: int|None = None) -> bytes: ...
async def send_eof(self) -> None: ...
async def aclose(self) -> None: ...
```
Non-negotiable behaviours (each maps to a `match` case that
already exists in `_transport.py` and must keep working):
- `receive_some()` returns `b''` at clean EOF →
`MsgpackTransport._iter_packets()` sees `header == b''` and
raises `TransportClosed(loglevel='transport')`. **This is the
graceful-disconnect path the whole runtime relies on**; get it
right first.
- a reset/aborted stream → raise `trio.BrokenResourceError`.
- use after local close → raise `trio.ClosedResourceError`
(ideally with `'another task closed this fd'`-equivalent text
absent, so the `raise_on_report` branch at
`_transport.py:290` stays quiet).
- `send_all()` on a closed peer → `trio.BrokenResourceError`.
- honour Trio's one-task-per-direction rule with public,
implementation-local guards that raise
`trio.BusyResourceError`; do not depend on `trio._util`.
`MsgpackTransport` already serializes sends, while receives are
single-task by construction.
- **buffering**: if iroh's `read()` doesn't support
"read up to n", `receive_some()` must maintain an internal
leftover buffer. Note `MsgpackTransport` wraps us in
`tricycle.BufferedReceiveStream` anyway, so `receive_some()`
just needs *some* nonzero-progress contract.
Centralize exception translation at every iroh/UniFFI boundary;
no generated exception may escape into `Channel` or server code.
Step 0 must record actual exception classes/status payloads and
build an exhaustive operation-specific mapping:
| observed condition | adapter result |
| --- | --- |
| receive clean EOF | `b''` |
| local stream/listener/endpoint already closed | `trio.ClosedResourceError` |
| concurrent same-direction operation | `trio.BusyResourceError` |
| peer reset, stopped stream, lost connection | `trio.BrokenResourceError` |
| dial rejected or no usable route | `ConnectionRefusedError` or `ConnectionError` |
| caller's Trio deadline/cancellation | preserve Trio cancellation semantics |
| unexpected FFI status/panic | chained `RuntimeError` identifying operation and pinned version |
Preserve the original exception as `__cause__`, but sanitize
messages so `_transport.py` sees stable Trio/Tractor categories,
not version-specific iroh text. Endpoint accept failure becomes a
listener `BrokenResourceError`; normal endpoint shutdown becomes
`ClosedResourceError`. Add one test per observed step-0 status.
```python
class QuicListener(trio.abc.Listener):
'''
Accepts iroh `Connection`s and yields one `QuicMsgStream`
per accepted bi-stream, so `trio.serve_listeners()` spawns
one `handle_stream_from_peer()` per `Channel`.
'''
async def accept(self) -> QuicMsgStream: ...
async def aclose(self) -> None: ...
```
The accept-side subtlety is fan-out: one actor transport accepts
connections and each connection accepts streams, while
`Listener.accept()` returns one stream. Give **each** listener a
supervisor task started with
`await server_ep.listen_tn.start(...)`.
That task creates and owns a cancel scope, a child nursery for the
endpoint feeder plus per-connection feeders, a guarded stream
queue, and a completion event.
`start_listener(addr=, server_ep=, actor_tpt=)` does not return
until the supervisor has reported all of those ready.
Do not borrow an implicit parent nursery or spawn feeders lazily
from `accept()`.
The queue is a guarded `deque`, not an unowned memory-channel
buffer. A feeder transfers a fully constructed, lease-owning
stream into it only while the listener is open; if close wins the
race, the feeder closes the stream itself. `accept()` atomically
pops one item or waits on the queue condition. Once close is
marked and the queue is empty, it raises
`trio.ClosedResourceError`.
`QuicListener.aclose()` is idempotent and has this exact order:
1. under the queue guard, mark closed and wake all `accept()`
waiters without a checkpoint between the state change and
notification;
2. cancel the listener-owned supervisor scope;
3. the supervisor's shielded `finally` joins the endpoint and all
connection feeders, atomically detaches the queue, closes every
queued stream, releases their leases, and closes the queue;
4. only after that finalizer finishes, the supervisor sets its
completion event;
5. `aclose()` waits under a shield for that event and returns;
concurrent closers wait for the same event.
The same supervisor finalizer runs if its parent nursery is
cancelled before someone calls `aclose()`. This makes the
supervisor, not an arbitrarily cancelled caller, the sole final
cleanup owner. Test cancellation at feeder accept, stream
construction, queue transfer, `accept()` wakeup, and each close
checkpoint; no feeder may outlive the listener and no queued
lease may survive completion.
This needs two explicit, typed references in the module-level
listener call: `server_ep=` is the IPC server `Endpoint` that owns
`listen_tn`, while `actor_tpt=` is the already-open
`QuicActorEndpoint` whose iroh accept API supplies connections.
Store `actor_tpt` on the server endpoint during actor transport
bootstrap and pass both keyword-only arguments; socket backends
ignore `actor_tpt`. `Endpoint.start_listener()` then stores the
listener's already-resolved address instead of calling
`getsockname()`.
### 3.4 `maddr`
Expected multiaddr spellings for direct QUIC and relay routes are
**step-0 verification items**, not assumptions:
```
/ip4/<h>/udp/<p>/quic-v1 # direct
/ip4/<h>/udp/<p>/quic-v1/p2p/<node-id> # direct + identity
/dns/<relay-host>/tcp/443/tls/ws/p2p/<node> # relay-ish
```
- Do not emit NodeId alone in the first backend: without enabled
discovery it would discard the route required by the complete
`IrohAddress`. `mk_maddr()` must preserve NodeId, ALPN, relay
URL, and all direct addresses, or return a canonical Tractor
string form that does until a multiaddr grammar can round-trip
every field.
- Verify whether an iroh NodeId can losslessly map to `/p2p/`.
If not, use a tractor-local `/iroh/<node-id>` segment rather
than pretending to be a libp2p peer-id. This needs upstream
registration, on the same track as `wg`/`tipc` (gh #483).
- this backend is the strongest argument for gh #443's
**tunnelled/composed maddr** item: `/ip4/../udp/../quic-v1/..`
*is* a composed stack. Cross-reference plan 03 §5 so the two
grammars land compatibly.
---
## 4. Discovery integration
- The registrar stores the complete `IrohAddress`, not only a
NodeId. Registration is forbidden until endpoint address
resolution has produced that descriptor. If route hints change
later, dynamic re-registration is a follow-up; the spike uses
the pre-registration snapshot.
- Optional iroh discovery mechanisms and their names/capabilities
are step-0 verification items and out of scope for the first
backend. No NodeId-only reachability claim is made.
- Relay configuration belongs to `QuicActorEndpoint` creation,
not `start_listener()`, because dialing and listening reuse the
same endpoint. The demo's relay choice and self-hosted option
are selected only after step 0 verifies the pinned API.
## 5. Security note
The transport-security and NodeId-authentication properties of the
pinned iroh stack are **step-0 documentation-verification items**.
Claim only the properties supported by that version's source and
docs. Two design consequences remain:
1. an **allowlist hook** — an actor should be able to reject
inbound connections from unknown node-ids *before* the
`Aid` handshake. Natural home: a predicate kwarg on
`start_listener()`, evaluated in `QuicListener`'s connection
acceptor task. Sketch it; ship it in PR 1 if cheap (it is).
2. do **not** claim any security property for the other
backends by association. `tcp`/`uds`/`tipc` remain
unauthenticated; that's what plan 03 (wg) is for.
## 6. Commit sequencing
0. **spike (throwaway, not committed)**: drive iroh under
`trio-asyncio`/`tractor.to_asyncio`, echo bytes over a
bi-stream between two processes. Fill §1.1 with generated ABI,
endpoint resolution, close/join, and error observations. Probe
cancel at every generated lifecycle phase. Timebox it and use
the fallback if any mandatory ownership fact stays unknown.
1. prep PR: tagged address migration, annotation widening,
non-socket listener reconciliation, `tpt_key` dispatch,
typed `server_ep=`/`actor_tpt=` listener inputs, and lazy
default addresses. **No new backend.** Keep tcp and uds
behavior unchanged.
2. bootstrap prep: pass `ChildTransportBootstrap` through every
process-launch path and add the transport nursery around child
parent-dial, service, deregistration, and teardown. Resolve the
endpoint address before registration. Add no iroh-specific
global state.
3. `_uniffi_trio.py` supervisor + lifecycle fault-injection tests:
no asyncio loop, one owner/complete/free, callback retention,
bounded caller handoff, and joined durable cleanup.
4. `QuicActorEndpoint` + provisioned registrar descriptor +
loopback direct-address tests; prove one endpoint handles dial,
listen, address lookup, and ordered teardown.
5. `QuicMsgStream` + exhaustive error-normalization and lease
release tests against the loopback endpoint pair.
6. `QuicListener` supervisor + cancellation-at-every-checkpoint
tests, including queued-stream draining and feeder joins.
7. `MsgpackQuicStream`, full-key connection pooling, registration
tables, and `--tpt-proto quic`; then run the full suite.
8. routable maddr/string form + docs + a two-host example (pairs
with #482's format).
## 7. Testing
- capability predicate `is_quic_available()` → `iroh` importable
*and* every step-0-recorded driver symbol present at the pinned
version. Same `pytest.fail`-early hook as plan 01 §7.2.
- **the acceptance bar is the same**: whole suite green under
`--tpt-proto quic`. Expect this to shake out real bugs in the
adapters (esp. teardown ordering and `TransportClosed`
classification) — that's the point.
- Measure endpoint bind and first-connect latency in step 0; do
not assume a multiplier. Before changing a deadline, rule out
the project's CPU-throttle false-positive, then prefer one
per-proto harness multiplier over individual-test edits.
- Use the step-0-verified relay-disable configuration with direct
loopback addresses for default CI. Mark separately verified
relay tests `pytest.mark.net` and keep them out of default CI.
- leak checks: assert the actor has one key/endpoint, every FFI
operation completed/freed once, all listener feeders joined,
all queued streams closed, every connection lease released,
and endpoint close completion observed before actor teardown.
- address-ordering check: block registration until a descriptor
with NodeId, ALPN, and at least one route is published; reject
sentinel, NodeId-only, and post-registration mutation cases.
## 8. Risks
| risk | mitigation |
| --- | --- |
| uniffi codegen internals shift on upgrade | pinned minor, symbol assertion at import, the "no asyncio loop" test, documented fallback to `to_asyncio` |
| callback wakeup/lifetime semantics differ from the hypothesis | step-0 source + probe gate; retain callback/data through verified quiescence; durable owner; never timeout-free |
| cancelled foreign future never reaches a freeable state | bounded caller handoff to visible actor supervisor; joined graceful shutdown or process-level escalation; never speculative free |
| `iroh` wheel availability for 3.13/3.14 on linux+macos | verify in step 0; if missing, that alone may force the `aioquic` fallback |
| endpoint or route resolution is not ready before parent dial/registration | actor endpoint bootstrap barrier; publish only a complete resolved descriptor |
| connection closes while a stream still uses it | full-key pool + stream-held leases + exact-once release tests |
| listener close strands feeder tasks or queued streams | listener-owned scope/completion event; cancel, join, drain, then return |
| QUIC latency/jitter destabilizes suite timing assumptions | measure first; per-proto multiplier only if demonstrated; relay-less CI mode |
| address tuple collides with another backend | required `'quic'` tag and exact-shape dispatch |
| scope creep into iroh's docs/blobs/gossip crates | this backend is `Endpoint`+`Connection`+bi-streams only; anything else is a separate issue |
## 9. Follow-up issue seeds
- `tractor.discovery` delegating to iroh discovery (DNS/pkarr/mdns)
- per-`Context` QUIC sub-streams: today one `Channel` == one
stream; QUIC would let each `tractor.Context` own its own
stream with independent flow-control and cancellation — this
is the genuinely novel win #353 gestures at, and it's a
runtime-layer change, not a transport one
- unreliable QUIC datagrams for a lossy-ok broadcast transport
(pairs with plan 01's TIPC-multicast seed)
- node-id allowlist → a real `tractor` authz story
- `aioquic` sans-io backend reusing §3's adapters

498
ai/tpt-backends/03_wg_tunnel_bindspace.md 100644
View File

@ -0,0 +1,498 @@
# Plan 03 — WireGuard (and other tunnels) as a *nested bindspace* via `pyroute2`
Tracks gh [#482] + the tunnelled-maddr item of [#443].
Prereq reading:
[`00_shared_backend_contract.md`](./00_shared_backend_contract.md).
**Thesis**: WireGuard is **not** a `MsgTransport`. It is an
interface-layer tunnel that is transparent to `socket(2)`, so
the correct abstraction is a *bindspace* — a scoped,
`@acm`-managed network context that an existing L4 transport
(`tcp`, and later `quic`/`tipc`-over-UDP-bearer) binds *inside*.
This plan implements `Address.namespace` (spec'd but unused
since day one) and the composed/tunnelled maddr grammar, with
`pyroute2` as the netlink codec and as much of the I/O moved
onto `trio` as the library's sans-io layer allows.
[#482]: https://github.com/goodboy/tractor/issues/482
[#443]: https://github.com/goodboy/tractor/issues/443
---
## 1. What exists today (verified, per #482)
- `wrap_address()` accepts maddr `str`s (leading-`/` dispatch,
`_addr.py:262`) but `parse_maddr()` only knows
`/ip4|ip6/<h>/tcp/<p>` and `/unix/<p>`; a `.../wg/u<key>`
maddr raises `ValueError('Unsupported multiaddr protocol
combo')`.
- there is no `wg` proto in the multiaddr *spec* yet, but
multiformats/py-multiaddr#108 (key form `u<base64url>`) is
**merged** as of 2026-07-28 (`f86519da`) — and unreleased, the
latest `0.2.0` predating it. Spec registration is still tracked
by multiformats/py-multiaddr#107 and gh #483.
- so **today's deployable story is declarative**: run `wg-quick`
out-of-band, parse the maddr, strip to the overlay
`(host, port)`, verify the pubkey in its host-specific role,
hand the overlay addr to `registry_addrs=`/`tpt_bind_addrs=`.
#482 already contains working example code for exactly this.
- `Address.namespace` exists in the Protocol
(`_addr.py:94-101`, "the if-available OS-specific network
namespace key") and **no backend implements it**. This plan is
its first consumer.
## 2. Three layers, three PRs
| layer | what | dep | ships |
| --- | --- | --- | --- |
| **A. declarative** | commit #482's examples; `parse_maddr()` learns `/wg/u<key>` → overlay `Address` + verified pubkey | `multiaddr` (already), `wg(8)` CLI | first |
| **B. `pyroute2` read/verify** | replace the `subprocess.run(['sudo','wg','show'])` shelling with netlink queries | `pyroute2` extra | second |
| **C. `@acm` lifecycle** | create/configure/tear down wg ifaces + netns *from the runtime*, as nested bindspaces; implement `Address.namespace` | `pyroute2` + `CAP_NET_ADMIN` | third |
Each is independently valuable and independently reviewable.
**Do not attempt C first** — the interesting design (nested
bindspace `@acm`s) is only well-posed once A has pinned the
address grammar and B has proven the netlink path under trio.
---
## 3. Layer A — declarative `wg` maddrs
### 3.1 the address shape
The decision: **a wg segment annotates an existing address, it
does not create a new address type.** Two candidate encodings;
**pick (a)**:
- **(a) `TunnelledAddress` wrapper** (recommended):
```python
class TunnelledAddress(
msgspec.Struct,
frozen=True,
):
overlay: Address # e.g. TCPAddress
tunnel: WGTunnelSpec # proto-specific, frozen
```
with `.proto_key` **delegating to `overlay.proto_key`** so every
existing table lookup (`_addr_to_transport`,
`enable_transports` guard at `_root.py:391`,
`transport_from_addr()`) keeps working untouched, and
`.unwrap()` delegating to `overlay.unwrap()` so **nothing new
crosses the wire**. `.namespace` and `.bindspace` come from
the tunnel spec. The wrapper is stripped (`→ .overlay`) at the
moment of bind/connect.
- ⚠️ `is_wrapped_addr()` (`_addr.py:194`) tests
`type(addr) in _address_types.values()` — the build-registered
protocol-key-to-address-type registry. `TunnelledAddress`
isn't in it and must not be (it has no transport of its own).
So either add an explicit
`isinstance(addr, TunnelledAddress)` clause there, or give
the wrapper a marker and test structurally. Do the former;
it's two lines and honest.
- the reflection in `Endpoint.start_listener()`
(`inspect.getmodule(self.addr)`) would resolve to the
*wrapper's* module, not the transport's. **So the wrapper
must be unwrapped before it reaches `Endpoint`** — i.e. by
the bindspace `@acm` (layer C) or by `parse_maddr()`
(layer A). State this loudly in the docstring; it's the #1
way to get this wrong.
- (b) add fields to each existing `Address` type. Rejected:
duplicates tunnel logic per-backend and pollutes `.unwrap()`.
```python
class WGTunnelSpec(
msgspec.Struct,
frozen=True,
):
pubkey: str # std-base64 `wg(8)` form
iface: str = 'wg0'
netns: str|None = None
# layer-C-only fields, unset in layer A
maybe_endpoint: tuple[str, int]|None = None
maybe_allowed_ips: tuple[str, ...] = ()
```
### 3.2 `parse_maddr()`/`mk_maddr()`
Grammar — **verified** against py-multiaddr#108, first on the
`baudco/py-multiaddr@wg_support` branch and re-verified after it
merged upstream (`multiformats/py-multiaddr@f86519da`); all three
forms below parse *and* round-trip. Note the codec also validates
that the key decodes to exactly 32 bytes, so a truncated key is a
`StringParseError`, not a silently-mangled parse:
```
/ip4/192.168.1.50/udp/51820/wg/u<A_pub>/ip4/10.0.11.1/tcp/1616
\_______ bearer __________/\__ key __/\______ overlay ______/
underlay, wg `ListenPort` the ONLY part we bind
```
The `/wg/` segment is **infix, not suffix** — the segments
*before* it are the wg **bearer** (the underlay `(ip, udp-port)`
that `wg(8)` itself listens on, per the codec docstring's own
`/ip4/1.2.3.4/udp/51820/wg/{key}` example), and the segments
*after* are the **overlay** endpoint that `tractor` binds.
⚠️ **CORRECTION** — an earlier revision of this plan (and the
examples in gh #482) used a *suffix* form
`/ip4/10.0.11.1/tcp/1616/wg/u<key>`. That parses, but it is
semantically inverted: it puts the overlay addr where the bearer
belongs, `tcp` where wg's `udp` `ListenPort` goes, and declares
no overlay endpoint at all. `parse_wg_maddr()` in
`examples/multihost/wg_lan/` now rejects it with an actionable
error.
Observed protocol-name lists, for writing the `match`:
| maddr | `[p.name for p in m.protocols()]` |
| --- | --- |
| `/ip4/1.2.3.4/udp/51820/wg/u<k>` | `['ip4','udp','wg']` |
| `/ip4/../udp/../wg/u<k>/ip4/../tcp/..` | `['ip4','udp','wg','ip4','tcp']` |
- so the three parts have **three different owners**, and only the
third is an `Endpoint`:
| part | bound by | in the runtime? |
| --- | --- | --- |
| bearer | kernel, via `wg-quick`/`pyroute2` | no |
| `/wg/u<key>` | nothing — it's an identity | no, verified out-of-band |
| overlay | `tractor`'s `IPCServer` | **yes**, as `.overlay` |
This owner-split is the real axis of the design, *not* whether
the maddr stack is "composed" (it is).
- ⚠️ **CORRECTION**, an earlier draft of this section specced a
hand-rolled `_peel_tunnel_segs(proto_names) -> (bearer_names,
tunnel_specs, overlay_names)`. **Do not write it.**
`py-multiaddr` already ships the whole tunnel compose/peel API
and it was simply missed here — see its README "En/decapsulate"
and "Tunneling" sections, and gh #443's 2nd bullet which links
them. Verified against the pinned rev:
| need | API |
| --- | --- |
| isolate the bearer | `ma.decapsulate_code(P_WG)` |
| drop the overlay, keep bearer+key | `ma.decapsulate(overlay_ma)` |
| per-seg maddrs | `ma.split()` |
| rejoin a seg tail | `Multiaddr.join(*segs)` |
| read the key | `ma.value_for_protocol('wg')` |
| recompose | `bearer.encapsulate(key).encapsulate(overlay)` |
`.decapsulate_code()` handles the infix `/wg/` seg cleanly
*because* it cuts on proto-code and never tries to match an
addr value — the key seg has no addr of its own. This is the
same NIH trap gh #429 existed to close, one layer up.
- ⚠️ `value_for_protocol('ip4')` on a *full* tunnelled maddr
silently returns the **first** match, i.e. the bearer's host.
Always call it on a peeled sub-maddr, never the whole stack.
- `parse_maddr()` gains a case on
`[('ip4'|'ip6'), 'udp', 'wg', ('ip4'|'ip6'), <overlay-l4>]` →
peel w/ the API above, decode the multibase key to std-base64,
and return `TunnelledAddress(overlay=..., tunnel=WGTunnelSpec(
...))` w/ the bearer recorded in the spec.
- keep the existing 2-proto cases byte-identical; add the new
case *after* them.
- layer A rejects more than one `/wg/` segment. Its wrapper stores
one bearer, key, and overlay, so accepting wg-in-wg would
silently misrepresent the maddr. Nested tunnel support needs a
different data shape and belongs in a later layer.
- `mk_maddr()` inverse for `TunnelledAddress` is just
`.encapsulate()` composition; don't rebuild `str`s by hand.
- **pending an upstream release**: py-multiaddr#108 is merged, so
`Multiaddr('/…/wg/u…')` parses — but off a `[tool.uv.sources]`
`rev` pin, since no release carries the codec. Gate the tests
on `_have_wg_maddr_proto()`, implemented as
`protocols.protocol_with_name('wg')` under
`except ProtocolNotFoundError`. Do **not** probe by parsing a
dummy like `Multiaddr('/wg/uAAAA')` — the codec enforces a
32-byte key, so that raises even when the proto *is* known. Do
**not** hand-roll a `wg` parser in `tractor` — the whole point
of #429 was dropping the NIH parser.
### 3.3 verification helper (pure, composable)
Port #482 §2's helpers into `tractor/discovery/_tunnel.py` as
*pure functions* + one impure probe, cleanly separated:
```python
def parse_wg_maddr(maddr: str) -> TunnelledAddress: ... # pure
def wg8_pubkey(multibase_key: str) -> str: ... # pure
async def verify_wg_key(
spec: WGTunnelSpec,
role: Literal['local', 'peer'],
inspection: str|None = None,
) -> bool: ... # impure probe
```
In layer A `verify_wg_key()` may shell out to role-specific
`wg show <if> public-key|peers` queries, but it must be a *single*
async, time-bounded function so it never blocks trio's run thread
and layer B swaps only its body. It verifies key presence only,
not `Endpoint`, `AllowedIPs`, handshake state, or routing. Never
run `tractor` as root: privileged inspection stays a separate
step whose public-key output can be passed as `inspection`. Never
call it implicitly from
`wrap_address()`/`parse_maddr()` — parsing must stay pure and
side-effect-free; verification is the *caller's* explicit step
(and later, the bindspace `@acm`'s).
### 3.4 deliverables
- `examples/` scripts distilled from #482 §§3-5 (this is the
unchecked "commit examples from ^" bullet in #443). They live
under `examples/multihost/` — `test_docs_examples.py` walks
`examples/` recursively and runs every collected file as a
subproc asserting `rc == 0` (it doesn't even filter by
extension, so a stray `README.md` would be `python`-run too),
and `'multihost' not in p[0]` is already in its exclusion
list. Anything needing a real second host or a live tunnel
belongs there.
- a `docs/` page: tunnel setup, the maddr form, the two-host
run. Keep prose in the docs; keep the examples runnable and
minimal.
- tests: maddr round-trip, `TunnelledAddress` delegation
(`proto_key`/`unwrap` identical to overlay), `wrap_address()`
regression (a tunnelled maddr `str` → `TunnelledAddress`; a
plain one → unchanged), and **a real end-to-end over a
locally-created wg pair** gated on `CAP_NET_ADMIN` (see §5.3).
---
## 4. Layer B — `pyroute2` under `trio`
### 4.1 the library situation (verify at implementation time)
`pyroute2` ≥0.9 rewrote its core onto **asyncio**
(`AsyncIPRoute`; the sync `IPRoute` wraps it with its own loop).
It also ships a `WireGuard` netlink (generic-netlink) module
supporting `.set(iface, private_key=..., peer={...})` and
`.info(iface)`, plus `pyroute2.netns` / `NetNS` for namespaces,
and `IPRoute.link('add', kind='wireguard', ifname=...)`.
Three integration options, in increasing trio-nativeness:
- **(1) `trio.to_thread.run_sync()` around the sync API.**
Netlink ops here are one-shot, sub-millisecond, and happen at
bind/teardown time only — *not* in the msg hot path. This is
the **correct default**: it's ~10 lines, uses a battle-tested
API, and costs nothing where it's used.
- **(2) sans-io: `trio.socket` + pyroute2's message codecs.**
`pyroute2`'s message classes
(`pyroute2.netlink.rtnl.*`, `pyroute2.netlink.generic.wireguard.wgmsg`)
encode/decode independently of its I/O core. So a
`tractor/ipc/_netlink.py` with a small trio `NetlinkSocket`
(`trio.socket.socket(AF_NETLINK, SOCK_RAW|SOCK_DGRAM, proto)`,
`sendto`/`recv`, seq/pid matching, `NLMSG_DONE`/`NLMSG_ERROR`
handling) + pyroute2 codecs is very achievable and is the
honest reading of "as much trio wrapping as possible where any
other async support can be replaced".
**Do this for the paths we actually need** (link add/del,
addr add, wg get/set, netns bind) and *only* those — a
general netlink client is out of scope.
- (3) reimplement the codecs. Never.
**Recommended split**: ship (1) first so layer B is a small,
reviewable, behaviour-preserving swap of `verify_wg_key()`'s
body; then land (2) as a follow-up commit for the read path
(`wg get`, `link get`) where the sans-io surface is smallest,
and keep (1) for the privileged mutating ops. Measure before
converting anything else — there is no perf argument here, only
a "no foreign event loop in a trio actor" argument, which (1)
already satisfies (a thread is not an event loop).
Explicitly **do not** pull in `trio-asyncio` for pyroute2: it
would be the one place in the runtime where an asyncio loop
exists for no reason.
### 4.2 API shape
Pure-ish, functional, `@acm` for anything with teardown:
```python
async def read_wg_peers(
iface: str = 'wg0',
netns: str|None = None,
) -> tuple[str, ...]: ... # base64 pubkeys
async def read_wg_pubkey(iface: str = 'wg0', ...) -> str: ...
```
and `verify_wg_key()` becomes a thin composition over the two.
Note the pure-getter rule: no `read_wg_peers(..., create=True)`.
---
## 5. Layer C — nested bindspace `@acm`s + `Address.namespace`
This is the part #443 and `multiaddr_declare_eps.md` actually
ask for: *"for any tunneled maddr-`str`-entry we deliver a
data-structure which can easily be passed to nested `@acm`s
which consecutively setup nested net bindspaces for binding the
endpoint addrs"*.
### 5.1 the composition
```python
@acm
async def open_bindspace(
addr: TunnelledAddress,
) -> AsyncGenerator[Address, None]:
'''
Enter the net-bindspace implied by `addr`'s tunnel stack,
yielding the *overlay* `Address` ready to bind/connect.
Nests: one `@acm` per tunnel segment, outermost-first, so
a 2-deep stack is just two nested `async with`s and the
teardown order is guaranteed by `trio`.
'''
```
with per-tunnel-kind implementations:
```python
@acm
async def open_netns(name: str) -> AsyncGenerator[None, None]: ...
@acm
async def open_wg_iface(spec: WGTunnelSpec) -> AsyncGenerator[WGTunnelSpec, None]: ...
```
and a driver that folds a list of specs into nested contexts
(`contextlib.AsyncExitStack` for the N-deep case). The
`parse_endpoints()` API (`_multiaddr.py:153`) is the front door:
it already returns `dict[name, list[Address]]` and the
`multiaddr_declare_eps.md` sketch anticipates the recursive
`dict[str, list[Address]]|dict[...]` return for tunnelled
entries. Extend it to carry the tunnel stack, not to *enter* it.
### 5.2 `Address.namespace`, at last
- `TunnelledAddress.namespace` → `(kind, id)` e.g.
`('netns', 'tractor-wg0')`.
- **and** the existing backends should implement it as `None`
explicitly (they currently just don't define it), so the
Protocol stops lying.
- consumers to audit: nothing reads `.namespace` today — so
adding it is safe, but the *point* is that
`Endpoint`/`Server.pformat()` should start showing it (there's
already a `# !TODO, always be ns aware!` +
`f'|_netns: {netns}\n'` placeholder sitting in
`Endpoint.pformat()`, `_server.py:645`). Fill that in; it's
the cheapest possible proof the layer is wired.
### 5.3 the netns/process reality — read this before designing
**The headline consequence, stated up front**: netns is a
**runtime-level config API, not an actor-app-code API.** It is
declared as part of how an actor process is *brought up* — a
spawn-time/boot-time input alongside `enable_transports` and
`tpt_bind_addrs` — and it is **not** dynamically re-enterable by
app code once the actor is live. There is deliberately no
`await actor.enter_netns(...)`. Two hard reasons, both below:
`setns(2)` doesn't retroactively move existing sockets, and it's
per-thread rather than per-process. Anything that *looks* like a
mid-life API here would be a footgun that silently leaves the IPC
server bound in the old namespace.
- `setns(2)` with `CLONE_NEWNET` affects **the calling thread
only**, and sockets already created keep their original netns.
A trio actor is effectively single-threaded for our purposes,
so "enter the netns, *then* bind" works — but any
`to_thread` worker (§4.1 option 1!) is in the **original**
netns unless it also `setns`. Concretely: a wg query issued
via `trio.to_thread` will hit the wrong namespace. Either
pass `netns=` down to `pyroute2` (which does the
fork/setns dance itself) or pin a dedicated worker. **This is
the single subtlest bug in this plan — write the test first.**
- entering a netns is *process-global-ish and irreversible-ish*
in practice. Therefore: **netns membership belongs to the
actor process, decided before the runtime binds**, not to a
mid-life `@acm`. Design:
- the root/parent decides the netns for a subactor and passes
it in the spawn spec (there's already
`enable_transports`/`accept_addrs` plumbing at
`_runtime.py:1595-1615` — the netns rides alongside).
- the child, in `_runtime.async_main()` **before**
`IPCServer.listen_on()`, enters it.
- the mid-life `@acm` form is then only for the *root* /
single-actor case, and for iface creation (which is
genuinely scoped).
- document the constraint rather than hiding it; a
`RuntimeError` if `open_netns()` is entered after any
listener exists.
- privileges: iface/netns creation needs `CAP_NET_ADMIN`.
Never `sudo` from inside the runtime. Two supported modes:
(i) pre-provisioned out-of-band (layers A/B — the default,
and what #482 documents), (ii) runtime-managed when the
process already holds the cap. Detect with a cheap
`os.geteuid()==0 or CAP_NET_ADMIN in /proc/self/status`
probe and *fail loudly with an actionable message* otherwise.
- teardown must be idempotent and tolerant: an iface/netns
already gone must not strand the rest of the teardown — the
exact lesson `_uds.close_listener()`'s `FileNotFoundError`
tolerance and `_serve_ipc_eps()`'s per-ep `try/except`
encode. Mirror both.
### 5.4 tests for layer C
- unit: fold-N-tunnel-specs-into-nested-`@acm`s, with fakes; assert
enter/exit ordering (outermost-last-out) via a trace list.
- integration, gated on `CAP_NET_ADMIN` (skip otherwise, and in
CI run it in a `--cap-add NET_ADMIN` container job): create two
netns + a wg pair entirely in-process, boot a `tractor` root in
one and a subactor in the other, `find_actor()` across the
tunnel. This is a *fantastic* test to have and is fully
self-contained — no second host, no `sudo` in the test body.
- the `to_thread`-netns-mismatch regression from §5.3, written
**first** (red), then the fix (green), per project convention.
---
## 6. "Other shuttle-able tpts"
The generalization the #482 follow-up gestures at: once
`TunnelledAddress` + `open_bindspace()` exist, the same
machinery covers any iface-layer tunnel `pyroute2` can drive —
`ipip`/`gre`/`sit`/`vxlan`/`geneve`/`bridge`/`veth`. Keep
`WGTunnelSpec` as *one* frozen struct among a
`TunnelSpec = WGTunnelSpec|VxlanTunnelSpec|...` union with a
`kind: ClassVar[str]`, and dispatch `open_*` by `match` on it.
Design for it now (union + `match`), implement only `wg` +
`netns`. `veth`-pairs-in-netns is the natural second one because
it makes the §5.4 integration test possible without wg at all —
consider doing it *first* for exactly that reason.
## 7. Non-goals
- no wg userspace implementation, no key exchange, no
`wg-quick` reimplementation (config-file parsing is
out of scope; take structured input).
- no persistence of private keys beyond what layer C's iface
creation needs (and that stays in `get_rt_dir()`, 0600).
- macOS/Windows: layers B/C are Linux-only. Layer A (declarative)
works anywhere `wg` does. Gate accordingly and say so in the
docs — do not silently no-op.
## 8. Risks
| risk | mitigation |
| --- | --- |
| `to_thread` worker runs in the wrong netns | §5.3; pass `netns=` to pyroute2 or pin a worker; test-first |
| py-multiaddr#108 merged but unreleased | `[tool.uv.sources]` `rev` pin + `_have_wg_maddr_proto()` gate; layer A's overlay-addr path works regardless |
| `TunnelledAddress` leaks into `Endpoint` and breaks `inspect.getmodule()` | unwrap at parse/bindspace boundary; assert `not isinstance(ep.addr, TunnelledAddress)` in `Endpoint.__post_init__` |
| privileged ops in a library | never `sudo`; explicit cap probe + actionable error; pre-provisioned is the default |
| pyroute2 0.9 asyncio core drags a loop into the actor | option (1) is a *thread*, not a loop; forbid `trio-asyncio` here (§4.1) |
| netns teardown strands actor teardown | idempotent/tolerant teardown mirroring `_uds.close_listener()` |
## 9. Follow-up issue seeds
- `veth`-in-netns bindspace (unblocks capless-ish integration
testing, and is a great local multi-"host" test rig)
- composed/tunnelled maddr grammar shared with plan 02's
`/…/quic-v1/…` stacks (gh #443)
- `wg` proto into the multiaddr **spec** (gh #483), then flip
`MsgTransport.maddr` to always return `Multiaddr` (the third
#443 bullet)
- runtime-managed wg key rotation / peer add-remove as a
`tractor` service actor — the natural "actor that owns the
network" demo

56
ai/tpt-backends/README.md 100644
View File

@ -0,0 +1,56 @@
# next-gen `tractor.ipc` transport backend plans
Implementation specs for three prospective `.ipc` transport
backends, written so each can be worked independently (by a
different model/provider) without design or lib-selection drift.
**Read [`00_shared_backend_contract.md`](./00_shared_backend_contract.md)
first** — it is the normative description of what a `tractor`
transport backend *is* as of `main@83b34884` (the backend
duck-type, registration and address-selection wiring, the
test-harness plumbing, the code-style rules). The three plans
assume it and document only their own deltas.
| plan | issue | dep | size | lands |
| --- | --- | --- | --- | --- |
| [01 — TIPC](./01_tipc_backend.md) | [#378] | **none** (stdlib) | small | first |
| [02 — QUIC/`iroh`](./02_quic_iroh_backend.md) | [#353] | `iroh` (uniffi FFI) | large | needs a prep PR |
| [03 — `wg` bindspace](./03_wg_tunnel_bindspace.md) | [#482], [#443] | `pyroute2` | medium, 3 layers | layer A now |
Headline conclusions:
- **TIPC is the cheap win.** Verified: `trio.SocketStream` and
`trio.SocketListener` are address-family agnostic (only
`SOCK_STREAM` + a trio socket), and CPython ships `AF_TIPC` +
23 `TIPC_*` constants. So the backend is ~one module of
contract boilerplate, zero new deps, and it buys kernel-native
service primitives: `bind()` publishes, known-address
`connect()` resolves, and topology events report publication
changes. Actor-name lookup, registrar state and split-brain-safe
election remain separate work. (`modprobe tipc` is required;
hard-gate everything.)
- **QUIC's cost is entirely in two adapters**, not in QUIC. The
`iroh` python bindings are `uniffi`-generated asyncio, but the
asyncio dependency is confined to *one* future-poll callback —
a ~40-line `trio` bridge (`TrioToken.run_sync_soon`) replaces
it. The second cost is that an iroh listener isn't a socket,
which needs a small, independently-reviewable prep PR to
`_server.py`/`_types.py`.
- **WireGuard is not a transport.** It's an iface-layer tunnel,
so it belongs as a *nested bindspace* (`TunnelledAddress` +
`open_bindspace()` `@acm`s) wrapping whatever L4 tpt is in
use — which is also what finally implements the long-spec'd
`Address.namespace`, and what generalizes to
`veth`/`vxlan`/`gre`.
Ordering rationale: plan 01 first as the cheap proof the
table-registration story generalizes to a genuinely new proto;
plan 03 layer A is already deployable-today doc/example work;
plan 02 last (and gated on its prep PR). Plans 01 and 02 both
want the same `Address.rebind_from_sockname` gate — whichever
lands first ships it.
[#378]: https://github.com/goodboy/tractor/issues/378
[#353]: https://github.com/goodboy/tractor/issues/353
[#482]: https://github.com/goodboy/tractor/issues/482
[#443]: https://github.com/goodboy/tractor/issues/443

213
examples/multihost/wg_lan/README.md 100644
View File

@ -0,0 +1,213 @@
# `tractor` over a WireGuard tunnel, declared as one maddr
A two-host LAN setup: a `tractor` actor tree on host A, dialed
from host B, with the endpoint declared as a single `wg`
multiaddr.
Supersedes the example set in gh
[#482](https://github.com/goodboy/tractor/issues/482) — see
[what changed](#what-changed-vs-482).
> **Why `examples/multihost/`?** `tests/test_docs_examples.py`
> walks `examples/` recursively and runs everything it collects
> as a subproc, asserting `rc == 0`. These need a real second
> host and a live `wg` tunnel, so they can't satisfy that;
> `'multihost' not in p[0]` is already in the test's exclusion
> list, which is what keeps them out of CI.
## the maddr form
```
/ip4/192.168.1.50/udp/51820/wg/u<A_pub>/ip4/10.0.11.1/tcp/1616
\____ wg bearer ___________/\__ key __/\____ tractor ep _____/
underlay, wg `ListenPort` overlay, on the wg iface
(kernel/`wg(8)` owns it) (the ONLY part tractor binds)
```
Three parts, three different owners:
| part | who binds it | in the runtime? |
| --- | --- | --- |
| `/ip4/../udp/51820` bearer | kernel via `wg-quick`/`pyroute2` | no |
| `/wg/u<key>` | nothing — it's an identity | no, verified out-of-band |
| `/ip4/../tcp/1616` overlay | `tractor`'s `IPCServer` | **yes**, as `.overlay` |
Verified against py-multiaddr
[#108](https://github.com/multiformats/py-multiaddr/pull/108):
this composed form parses and round-trips
(`['ip4','udp','wg','ip4','tcp']`).
## requirements
py-multiaddr #108 is **merged** (2026-07-28) but ships in no
release yet — the latest `0.2.0` (2026-03-17) predates it and has
no `wg` codec. So `pyproject.toml` carries a temporary
`[tool.uv.sources]` `rev` pin at the merge commit, and a plain
```bash
uv sync
```
gets you a `wg`-aware `multiaddr`. That pin goes away once a
release carries the codec. Its `py-multibase` dependency provides
the imported `multibase` module; no separate install command is
needed.
Without the codec `parse_wg_maddr()` raises immediately with an
actionable message — there is deliberately **no** degraded
hand-split fallback. `_have_wg_maddr_proto()` is the predicate.
Every peel and re-compose here goes through `py-multiaddr`'s own
tunnel API (`.decapsulate_code()`, `.split()`, `.join()`,
`.encapsulate()`, `.value_for_protocol()`) rather than any
bespoke segment slicing — see its README "En/decapsulate" and
"Tunneling" sections. gh #429 was about *dropping* our NIH
parser, and that applies to peeling this composed maddr just as
much as to decoding one proto. This example rejects multiple
`/wg/` segments because `WGTunnelledAddr` stores one tunnel.
## 0. tunnel setup (out-of-band, both hosts)
Host A is the service host (underlay e.g. `192.168.1.50`), host B
your workstation. Overlay net `10.0.11.0/24`.
```bash
umask 077
wg genkey | tee wg_priv.key | wg pubkey > wg_pub.key
```
`/etc/wireguard/wg0.conf` on **host A**:
```ini
[Interface]
PrivateKey = <A_priv>
Address = 10.0.11.1/24
ListenPort = 51820
```
```ini
[Peer]
PublicKey = <B_pub>
AllowedIPs = 10.0.11.2/32
```
on **host B**:
```ini
[Interface]
PrivateKey = <B_priv>
Address = 10.0.11.2/24
```
```ini
[Peer]
PublicKey = <A_pub>
Endpoint = 192.168.1.50:51820
AllowedIPs = 10.0.11.1/32
PersistentKeepalive = 25
```
This example configures host A's `ListenPort` and host B's
`Endpoint` from the maddr bearer, and configures host A's
`[Interface] Address` from its overlay host. The verification
step below checks keys only; it does not inspect those fields or
either peer's `AllowedIPs`.
```bash
sudo wg-quick up wg0 # both hosts
ping -c1 10.0.11.1 # from B
```
## 1. get your pubkey into the maddr
```bash
python -c "
from wg_maddr import mb_pubkey
key = open('wg_pub.key').read().strip()
print(mb_pubkey(key))
"
```
Paste the `u...` output into `WG_MADDR` in both scripts (they use
the same string — A's bearer, A's key, A's overlay ep).
## 2. verify the keys
Interface inspection commonly needs `CAP_NET_ADMIN`. Keep that
privileged operation separate from the `tractor` processes:
```bash
# host A: output must equal the maddr's A_pub key
export WG_KEY_INSPECTION="$(sudo wg show wg0 public-key)"
# host B: output must contain the maddr's A_pub key
export WG_KEY_INSPECTION="$(sudo wg show wg0 peers)"
```
These checks establish only that host A uses the declared local
key and host B has that key as a configured peer. They do not
verify `Endpoint`, `AllowedIPs`, a recent handshake, or routing.
The exported text contains public keys only. Each script passes it
to `verify_wg_key()` with its host-specific role before starting
`tractor`. Callers that already have permission to inspect the
interface may omit that argument; the helper's direct query is
async and requests cancellation after five seconds. Trio's
subprocess termination escalation can make final process cleanup
take longer than that cancellation deadline.
## 3. run
```bash
# host A
python host_a_srv.py
# host B
python host_b_client.py
```
Run both `tractor` programs as the normal application account,
not as root. Privilege is needed only for tunnel setup and the
separate inspection above. If using that preflight, keep the
host-specific `WG_KEY_INSPECTION` value exported in each
program's shell.
The client binds its own actor listener to `10.0.11.2:0`, while
the service actor binds to host A's `10.0.11.1` overlay host with
a random port. Keep `LOCAL_OVERLAY_BIND` aligned with host B's
WireGuard interface address if adapting this example.
`host_a_srv.py` must be importable on host B too, since
`portal.run()` refs the fn by module path — standard `tractor`
RPC semantics.
## what changed vs #482
Four corrections, all from
`ai/tpt-backends/03_wg_tunnel_bindspace.md`:
1. **the maddr semantics were inverted.** #482 used
`/ip4/10.0.11.1/tcp/1616/wg/u<key>` — that parses, but it puts
the *overlay* addr where the bearer belongs and `tcp` where
wg's `udp` `ListenPort` goes, and it declares no overlay ep at
all. `parse_wg_maddr()` now rejects it with an actionable
error.
2. **parsing is pure.** #482's helper had the key-check adjacent
to the parse; `verify_wg_key()` is now a separate, explicitly
composed step for inspection-capable callers. A parser that
shells out is a nasty surprise.
3. **no `sudo`.** #482 ran `sudo wg show`; a library/example must
never escalate or run `tractor` as root. Privileged tunnel
setup and key inspection are separate shell steps.
4. **no new `Address` proto-type.** The tunnel rides *beside* the
overlay addr in a frozen `WGTunnelledAddr`, and only `.overlay`
crosses into `open_nursery()`. #482 §6 floated a `WGAddress`
registered in `_address_types` — that registry maps available
transport keys to concrete address types, and
`_addr_to_transport` wants a `MsgTransport` per addr-type,
which `wg` doesn't have.
## next
`WGTunnelledAddr` is deliberately example-local. Promoting it to
`tractor.discovery` as a `TunnelledAddress` whose
`.proto_key`/`.unwrap()` delegate to `.overlay`, plus
`open_bindspace()` `@acm`s that create/tear down the iface +
netns via `pyroute2`, is layers A→C of the plan doc.

72
examples/multihost/wg_lan/host_a_srv.py 100644
View File

@ -0,0 +1,72 @@
# tractor: distributed structured concurrency.
'''
Host A: the service host, reachable over a `wg` tunnel.
Binds `tractor`'s registrar + an `echo_srv` sub-actor on the
tunnel's *overlay* addr, declared as a single `wg` maddr.
'''
from __future__ import annotations
import os
import tractor
import trio
from wg_maddr import (
parse_wg_maddr,
verify_wg_key,
WGTunnelledAddr,
)
# bearer = host A's underlay `(ip, wg ListenPort)`
# key = host A's OWN tunnel pubkey
# overlay = the ep `tractor` binds, on the wg iface's addr
WG_MADDR: str = (
'/ip4/192.168.1.50/udp/51820'
'/wg/u<A_pub_b64url>'
'/ip4/10.0.11.1/tcp/1616'
)
async def echo(msg: str) -> str:
actor = tractor.current_actor()
return f'{actor.aid.name!r} echoes: {msg}'
async def main():
addr: WGTunnelledAddr = parse_wg_maddr(WG_MADDR)
inspection: str | None = os.environ.get('WG_KEY_INSPECTION')
if not await verify_wg_key(
addr,
role='local',
inspection=inspection,
):
raise RuntimeError(
f'Maddr key is not wg0 local public key!\n'
f'maddr: {WG_MADDR}\n'
f'key: {addr.wg_pubkey}\n'
)
print(
f'wg bearer (kernel-owned): {addr.bearer}\n'
f'tractor overlay ep: {addr.overlay}\n'
)
async with tractor.open_nursery(
# XXX only `.overlay` crosses into the runtime; the bearer
# + key are iface-layer concerns `tractor` never binds.
registry_addrs=[addr.overlay],
enable_transports=[addr.overlay_proto],
) as an:
await an.start_actor(
'echo_srv',
bind_addrs=[(addr.overlay[0], 0)],
enable_transports=[addr.overlay_proto],
enable_modules=['host_a_srv'],
)
print(f'echo_srv up on\n {addr.maddr}\n')
await trio.sleep_forever()
if __name__ == '__main__':
trio.run(main)

65
examples/multihost/wg_lan/host_b_client.py 100644
View File

@ -0,0 +1,65 @@
# tractor: distributed structured concurrency.
'''
Host B: workstation dialing host A's actor tree through the
`wg` tunnel.
'''
from __future__ import annotations
import os
import tractor
import trio
from host_a_srv import echo # noqa: F401 (RPC refs it by mod path)
from wg_maddr import (
parse_wg_maddr,
verify_wg_key,
WGTunnelledAddr,
)
# same maddr as host A: A's bearer, A's key, A's overlay ep
WG_MADDR: str = (
'/ip4/192.168.1.50/udp/51820'
'/wg/u<A_pub_b64url>'
'/ip4/10.0.11.1/tcp/1616'
)
LOCAL_OVERLAY_BIND: tuple[str, int] = ('10.0.11.2', 0)
async def main():
addr: WGTunnelledAddr = parse_wg_maddr(WG_MADDR)
inspection: str | None = os.environ.get('WG_KEY_INSPECTION')
if not await verify_wg_key(
addr,
role='peer',
inspection=inspection,
):
raise RuntimeError(
f'Maddr key is not a configured wg0 peer!\n'
f'maddr: {WG_MADDR}\n'
f'key: {addr.wg_pubkey}\n'
)
async with (
tractor.open_root_actor(
name='wg_client',
tpt_bind_addrs=[LOCAL_OVERLAY_BIND],
registry_addrs=[addr.overlay],
enable_transports=[addr.overlay_proto],
),
tractor.find_actor(
'echo_srv',
registry_addrs=[addr.overlay],
raise_on_none=True,
) as portal,
):
res: str = await portal.run(
echo,
msg='hello over wg!',
)
print(res)
if __name__ == '__main__':
trio.run(main)

348
examples/multihost/wg_lan/wg_maddr.py 100644
View File

@ -0,0 +1,348 @@
# tractor: distributed structured concurrency.
r'''
Parse `wg`-tunnelled multiaddrs into `tractor`-ready addrs.
The canonical form (per py-multiaddr #108, verified against its
upstream merge) nests the *overlay* endpoint **after** the `/wg/`
segment:
/ip4/10.0.0.1/udp/51820/wg/u<key>/ip4/10.0.11.1/tcp/1616
\_______ wg bearer ______/\_ key _/\____ tractor ep _____/
(underlay, wg
`ListenPort`)
Naming follows `py-multiaddr`'s own encapsulation model, where
earlier segments *wrap* later ones (`.encapsulate()` appends), so
the two roles are:
- **bearer**: the segs *before* `/wg/`, i.e. the underlay
`(ip, udp-port)` that `wg(8)` itself listens on. Nothing in
`tractor` ever binds this — the kernel/`wg` iface owns it.
- **overlay**: the segs *after*, i.e. the addr `tractor` actually
binds/dials. The only part the runtime ever sees.
We deliberately avoid `inner`/`outer` for these two: in a *call*
stack "inner" reads as higher-up and later-called, whereas here
the encapsulated addr is bound *first* and sits deeper in the
maddr — two opposite intuitions on one word.
`/wg/u<key>` itself carries a declared Curve25519 pubkey
as multibase base64url (std base64 from `wg(8)` contains `/` and
so can't go in a `/`-delimited maddr). It binds nothing at all;
it's an identity, verified out-of-band.
XXX NOTE, `tractor`'s own `parse_maddr()` can't parse this yet
(`ValueError('Unsupported multiaddr protocol combo')`), which is
why this module exists: peel here, hand `.overlay` to the
runtime.
Design rules this module follows (see
`ai/tpt-backends/03_wg_tunnel_bindspace.md`):
- **let `py-multiaddr` do the parsing**. Every peel/compose goes
through `.decapsulate_code()`, `.split()`, `.join()`,
`.encapsulate()` and `.value_for_protocol()`. We hand-roll no
segment splitting whatsoever — the whole point of gh #429 was
dropping the NIH parser, and that applies to peeling this
composed maddr every bit as much as to decoding one proto.
- **parsing is pure**. `parse_wg_maddr()` does no I/O, no
`subprocess`, no netlink. A parser that shells out is a nasty
surprise.
- **verification is an explicit, separate step**. The caller
composes `verify_wg_key()` when it has permission to inspect
the iface; nothing implicit.
- **exactly one `wg` segment is supported**. `WGTunnelledAddr`
stores one bearer and one key, so accepting another segment
would silently misrepresent the maddr.
- **no new `Address` proto-type**. `wg` gets no entry in
`tractor.discovery._addr._address_types`, which maps available
transport keys to concrete address types, bc it has no
`MsgTransport` of its own. The
tunnel is a *bindspace*, so we carry it beside the overlay
addr and strip to `.overlay` at bind/dial time.
'''
from __future__ import annotations
import base64
from typing import Literal
import msgspec
from multiaddr import Multiaddr
import trio
IPProto = Literal['ip4', 'ip6']
class WGTunnelledAddr(
msgspec.Struct,
frozen=True,
):
'''
A `wg`-tunnelled endpoint: the underlay bearer, the tunnel
key, and the overlay addr `tractor` binds/dials.
'''
# underlay, owned by `wg(8)`/the kernel — NEVER bound by us
bearer: tuple[str, int]
# declared wg pubkey in std-base64 `wg(8)` form; it is the
# local key on the bearer host and a configured peer on a dialer
wg_pubkey: str
# overlay ep: an `UnwrappedAddress` as accepted by
# `tractor.discovery.wrap_address()`
overlay: tuple[str, int]
overlay_proto: Literal['tcp'] = 'tcp'
# kept so `.as_multiaddr()` re-renders the same ip family it
# was parsed from, rather than assuming v4
bearer_ip: IPProto = 'ip4'
overlay_ip: IPProto = 'ip4'
def as_multiaddr(self) -> Multiaddr:
'''
Re-compose the canonical `Multiaddr`, bearer outward-in,
using `.encapsulate()` exactly as py-multiaddr's own
tunneling example does.
'''
b_host, b_port = self.bearer
o_host, o_port = self.overlay
return (
Multiaddr(f'/{self.bearer_ip}/{b_host}/udp/{b_port}')
.encapsulate(
Multiaddr(f'/wg/{mb_pubkey(self.wg_pubkey)}')
)
.encapsulate(
Multiaddr(
f'/{self.overlay_ip}/{o_host}'
f'/{self.overlay_proto}/{o_port}'
)
)
)
@property
def maddr(self) -> str:
'''
The canonical maddr `str` form.
'''
return str(self.as_multiaddr())
def mb_pubkey(wg8_key: str) -> str:
'''
`wg(8)` std-base64 pubkey -> multibase base64url (`u`-prefixed).
'''
import multibase
raw: bytes = base64.b64decode(wg8_key, validate=True)
if len(raw) != 32:
raise ValueError(
f'WireGuard public keys must decode to 32 bytes, '
f'not {len(raw)}'
)
return multibase.encode('base64url', raw).decode('ascii')
def wg8_pubkey(mb_key: str) -> str:
'''
Inverse of `mb_pubkey()`: multibase -> `wg(8)` std-base64.
'''
import multibase
raw: bytes = multibase.decode(mb_key)
if len(raw) != 32:
raise ValueError(
f'WireGuard public keys must decode to 32 bytes, '
f'not {len(raw)}'
)
return base64.b64encode(raw).decode('ascii')
_wg_proto_known: bool | None = None
def _have_wg_maddr_proto() -> bool:
'''
True iff the installed `py-multiaddr` knows the `/wg/` proto,
i.e. carries py-multiaddr#108.
Merged upstream 2026-07-28 (`f86519da`) but in no release as
of `0.2.0`, hence the `[tool.uv.sources]` `rev` pin in
`pyproject.toml`.
Pure predicate; result cached since it can't change without a
reinstall.
'''
global _wg_proto_known
if _wg_proto_known is None:
from multiaddr.protocols import protocol_with_name
from multiaddr.exceptions import ProtocolNotFoundError
try:
protocol_with_name('wg')
_wg_proto_known = True
except ProtocolNotFoundError:
_wg_proto_known = False
return _wg_proto_known
def parse_wg_maddr(
maddr: str | Multiaddr,
) -> WGTunnelledAddr:
'''
Peel a `wg`-tunnelled maddr into its bearer/key/overlay
parts. Pure — no I/O.
Every cut is made by `py-multiaddr`, so a malformed maddr
(incl. a `wg` key that isn't exactly 32B) raises out of
`Multiaddr()` rather than yielding a struct quietly built
from garbage segs.
'''
if not _have_wg_maddr_proto():
raise RuntimeError(
f'Installed `py-multiaddr` has no `/wg/` proto!\n'
f'Needs py-multiaddr#108, merged upstream but not\n'
f'yet released; a `uv sync` picks up the pinned rev.\n'
f'maddr: {maddr!r}\n'
)
ma: Multiaddr = (
maddr
if isinstance(maddr, Multiaddr)
else Multiaddr(maddr)
)
segs: list[Multiaddr] = ma.split()
names: list[str] = [
proto.name
for seg in segs
for proto in seg.protocols()
]
wg_count: int = names.count('wg')
if not wg_count:
raise ValueError(
f'Not a `wg`-tunnelled maddr, no `/wg/` segment ??\n'
f'maddr: {ma}\n'
)
if wg_count > 1:
raise ValueError(
f'Nested `wg` segments are not supported; '
f'`WGTunnelledAddr` stores one tunnel only.\n'
f'maddr: {ma}\n'
)
# Resolve the unreleased protocol only after the capability
# check, so importing this module works with released multiaddr.
from multiaddr.protocols import protocol_with_name
wg_code: int = protocol_with_name('wg').code
# NOTE, `.decapsulate_code()` cuts at the LAST occurrence of
# the proto and keeps the *prefix*, which is exactly the
# bearer. It handles `/wg/` cleanly precisely bc it cuts on
# proto-code and never tries to match an addr value — the
# key seg has no addr of its own.
bearer_ma: Multiaddr = ma.decapsulate_code(wg_code)
overlay_ma: Multiaddr = Multiaddr.join(
*segs[names.index('wg') + 1:]
)
match [proto.name for proto in bearer_ma.protocols()]:
case [('ip4' | 'ip6') as b_ip, 'udp']:
bearer = (
bearer_ma.value_for_protocol(b_ip),
int(bearer_ma.value_for_protocol('udp')),
)
case _:
raise ValueError(
f'Bad `wg` bearer, expected `/ip4|ip6/<h>/udp/<p>`\n'
f'got: {bearer_ma}\n'
f'from maddr: {ma}\n'
)
match [proto.name for proto in overlay_ma.protocols()]:
case [('ip4' | 'ip6') as o_ip, ('tcp') as l4]:
overlay = (
overlay_ma.value_for_protocol(o_ip),
int(overlay_ma.value_for_protocol(l4)),
)
case []:
raise ValueError(
f'`wg` maddr declares no overlay endpoint!\n'
f'A bare `/…/wg/<key>` names only the tunnel; '
f'append the ep `tractor` should bind, e.g.\n'
f' {ma}/ip4/10.0.11.1/tcp/1616\n'
)
case _:
raise ValueError(
f'Unsupported `wg` overlay proto combo\n'
f'got: {overlay_ma}\n'
f'from maddr: {ma}\n'
)
return WGTunnelledAddr(
bearer=bearer,
wg_pubkey=wg8_pubkey(ma.value_for_protocol('wg')),
overlay=overlay,
overlay_proto=l4,
bearer_ip=b_ip,
overlay_ip=o_ip,
)
async def verify_wg_key(
addr: WGTunnelledAddr,
role: Literal['local', 'peer'],
iface: str = 'wg0',
timeout: float = 5,
inspection: str | None = None,
) -> bool:
'''
Verify the declared key in the role required on this host.
A bearer host uses `role='local'`; a dialer uses `role='peer'`.
This verifies only key presence. It does not inspect the peer's
endpoint, AllowedIPs, handshake state, or iface addresses.
`inspection` accepts output captured by a separate privileged
`wg show` step. Without it, query asynchronously for callers
which already have interface-inspection permission.
IMPURE + explicit by design: never called from
`parse_wg_maddr()`.
?TODO, per plan-03 layer B, swap this body for `pyroute2`
(keeping the signature) — and note `setns(2)` is *per-thread*,
so a query issued via `trio.to_thread` lands in the ORIGINAL
netns unless `netns=` is passed down.
'''
match role:
case 'local':
field = 'public-key'
case 'peer':
field = 'peers'
case _:
raise ValueError(f'Unknown WireGuard key role: {role!r}')
if inspection is None:
with trio.fail_after(timeout):
proc = await trio.run_process(
['wg', 'show', iface, field],
capture_stdout=True,
check=True,
)
inspection = proc.stdout.decode()
if role == 'local':
return addr.wg_pubkey == inspection.strip()
return addr.wg_pubkey in inspection.split()

11
pyproject.toml
View File

@ -169,6 +169,17 @@ sync_pause = {requires-python = ">=3.13, <3.14"}
# linux kernel networking
# 'pyroute2
# XXX TEMP, the `/wg/u<key>` maddr proto is MERGED upstream (in
# py-multiaddr#108, 2026-07-28) but is in NO release yet; the
# latest `0.2.0` (2026-03-17) predates the merge by ~4 months.
# Pinned by `rev` (not `branch`) so CI stays reproducible.
#
# Drop this pin (and bump the `multiaddr` dep floor above) the
# moment a release carries the `wg` codec; the only consumer is
# `examples/multihost/wg_lan/`.
# |_https://github.com/multiformats/py-multiaddr/pull/108
multiaddr = { git = 'https://github.com/multiformats/py-multiaddr.git', rev = 'f86519daaa21699023d0037c58cdff600313dd09' }
# ------ tool.uv.sources ------
[tool.uv]

14
uv.lock
View File

@ -273,9 +273,9 @@ name = "greenback"
version = "1.2.1"
source = { registry = "https://pypi.org/simple" }
dependencies = [
{ name = "greenlet", marker = "python_full_version < '3.14'" },
{ name = "outcome", marker = "python_full_version < '3.14'" },
{ name = "sniffio", marker = "python_full_version < '3.14'" },
{ name = "greenlet" },
{ name = "outcome" },
{ name = "sniffio" },
]
sdist = { url = "https://files.pythonhosted.org/packages/dc/c1/ab3a42c0f3ed56df9cd33de1539b3198d98c6ccbaf88a73d6be0b72d85e0/greenback-1.2.1.tar.gz", hash = "sha256:de3ca656885c03b96dab36079f3de74bb5ba061da9bfe3bb69dccc866ef95ea3", size = 42597, upload-time = "2024-02-20T21:23:13.239Z" }
wheels = [
@ -518,7 +518,7 @@ wheels = [
[[package]]
name = "multiaddr"
version = "0.2.0"
source = { registry = "https://pypi.org/simple" }
source = { git = "https://github.com/multiformats/py-multiaddr.git?rev=f86519daaa21699023d0037c58cdff600313dd09#f86519daaa21699023d0037c58cdff600313dd09" }
dependencies = [
{ name = "base58" },
{ name = "dnspython" },
@ -533,10 +533,6 @@ dependencies = [
{ name = "trio-typing" },
{ name = "varint" },
]
sdist = { url = "https://files.pythonhosted.org/packages/c7/10/4e26a8577cfce1c0febc8d83087e1373e93c695c6e73ad010546fb67e229/multiaddr-0.2.0.tar.gz", hash = "sha256:acb6b25c332ec1b2f1f8fef8d03a8c63385d34a87d690df0f4bba43cdf6efe8d", size = 58356, upload-time = "2026-03-17T21:51:00.274Z" }
wheels = [
{ url = "https://files.pythonhosted.org/packages/b5/13/56e503d01218d1ca27ea9fda862045a4b400cae5e756f47315f5aaba0eee/multiaddr-0.2.0-py3-none-any.whl", hash = "sha256:bcff7bf3d7de3d6da0b865b25423bcb411de1d20d70cc6abfacf75170d17866c", size = 40424, upload-time = "2026-03-17T21:50:58.833Z" },
]
[[package]]
name = "mypy-extensions"
@ -1193,7 +1189,7 @@ requires-dist = [
{ name = "bidict", specifier = ">=0.23.1" },
{ name = "colorlog", specifier = ">=6.8.2,<7" },
{ name = "msgspec", specifier = ">=0.20.0" },
{ name = "multiaddr", specifier = ">=0.2.0" },
{ name = "multiaddr", git = "https://github.com/multiformats/py-multiaddr.git?rev=f86519daaa21699023d0037c58cdff600313dd09" },
{ name = "pdbp", specifier = ">=1.8.2,<2" },
{ name = "platformdirs", specifier = ">=4.4.0" },
{ name = "setproctitle", specifier = ">=1.3,<2" },