Document TIPC-over-`wg`, add a handoff doc

Anticipating gh #502 — TIPC over a WireGuard mesh as our go-to multihost tpt deployment — plus a cold-start handoff for whoever (or whatever) picks this up next. The wg deats, both verified locally, - a wg iface is L3/`tun` (`POINTOPOINT,NOARP`, `link/none`, no L2 addr) so TIPC's `eth` media **cannot** bind it; the udp bearer is *mandatory* over wg, not merely an alternative. Also its ~1420 MTU sits under ethernet's 1500. - the composed deployment maddr is `/ip4/<pub>/udp/51820/wg/u<key>/tipc/<stype>/<inst>/<scope>`. XXX note the tipc segment has NO locative part unlike tcp's inner `/ip4/../tcp/..` — a service name is location-independent, so wg carries routing and tipc carries identity. That's the argument for one `/tipc` proto w/ a structured value in the #498 spec proposal. XXX ALSO correcting a premise: TIPC is **not** unencrypted. It ships AES-GCM crypto (`tipc node set key`, linux 5.9+) w/ cluster/master/per-node keys + rekeying. Those keys are symmetric+pre-shared tho, so wg is still preferred for public-key identity, NAT traversal, and one overlay every tpt can share. `01_tipc_HANDOFF.md` is deliberately provider-neutral: env setup, the hard-won kernel facts table, the two closed design decisions (+why), what landed, the pre-land TODOs and the repo's working conventions. (this patch was generated in some part by `claude-code` using `claude-opus-5` (`anthropic`))
2026-08-17 17:38:05 -04:00 · 2026-08-17 17:38:05 -04:00 · 4aa7a890cb
parent 1802641e41
commit 4aa7a890cb
4 changed files with 326 additions and 6 deletions
--- a/ai/tpt-backends/01_tipc_HANDOFF.md
+++ b/ai/tpt-backends/01_tipc_HANDOFF.md
@ -0,0 +1,249 @@
+# TIPC backend — handoff
+
+Status: **PR [#493] is feature-complete and green**; what remains
+is landing logistics plus a named follow-up track.
+
+Audience: any agent or human picking this up cold, from any
+provider. Nothing here assumes a particular harness or tooling.
+
+Read in this order,
+
+1. this file (orientation + what's already settled)
+2. [`00_shared_backend_contract.md`](./00_shared_backend_contract.md)
+   — **normative** description of what a `tractor` transport
+   backend *is*
+3. [`01_tipc_backend.md`](./01_tipc_backend.md) — the plan,
+   already reconciled against the live-kernel findings
+
+Do **not** re-derive the design or re-select libraries. Where
+this doc and the code disagree, **the code wins** — fix the doc
+in the same change (contract §0).
+
+[#493]: https://github.com/goodboy/tractor/pull/493
+
+---
+
+## 1. Environment
+
+The backend needs a linux kernel module that is **not loaded by
+default**:
+
+```bash
+sudo modprobe tipc
+tipc node get address        # confirms the module is live
+```
+
+Everything else is stdlib — TIPC adds **zero** dependencies.
+
+This repo is a git worktree with a `uv`-managed venv at
+`./py313`. Run things through it:
+
+```bash
+./py313/bin/python -m pytest tests/ipc -q
+./py313/bin/python -m pytest tests/ -q --tpt-proto tipc
+```
+
+To rebuild docs you need the docs dep-group, which *mutates*
+that venv — `uv sync` afterwards to restore it:
+
+```bash
+UV_PROJECT_ENVIRONMENT=py313 uv run --group docs \
+    python -m sphinx -b html docs /tmp/docbuild
+UV_PROJECT_ENVIRONMENT=py313 uv sync
+```
+
+Without the module, `--tpt-proto tipc` fails loudly and
+immediately (by design); `tests/ipc/test_tipc.py`'s
+kernel-touching cases self-skip.
+
+## 2. What the backend is, in three sentences
+
+An actor's TIPC address is a **service name** `(stype, instance)`
+— no host, no port. Binding the singleton `TIPC_ADDR_NAMESEQ`
+range *publishes* it into a kernel-maintained cluster-wide name
+table (visible via `tipc nametable show`), and a peer's
+`.connect()`-by-name *is* the discovery lookup, resolved
+in-kernel. `TIPC_ADDR_ID` port-ids are only ever **observed**,
+never user-facing.
+
+Everything lives in `tractor/ipc/_tipc.py`.
+
+## 3. Hard-won facts — do not re-litigate these
+
+All verified against a live kernel. Several contradict what the
+plan originally assumed.
+
+| fact | why it matters |
+| --- | --- |
+| A duplicate name bind **succeeds**, and connects **round-robin** between publishers | An instance collision is *silent crosstalk*, never `EADDRINUSE`. Hence the `blake2b` instance digest. |
+| Dialing an unpublished name gives `EHOSTUNREACH` **instantly**, as a **bare `OSError`** — not a `ConnectionError` subtype | `_reraise_as_connerr()` is REQUIRED by contract §4, not polish |
+| `getsockname()` always answers a `TIPC_ADDR_ID` port-id, even pre-bind | why `TIPCAddress.rebind_from_sockname = False` |
+| A connect-then-drop peer makes `getpeername()` raise `ENOTCONN` | Unguarded, this **kills the whole actor** — `.get_stream_addrs()` runs *before* the handshake, so it escapes handshake tolerance. See `_maybe_sockaddr()`. |
+| `SO_ACCEPTCONN` works fine (answers `1`) | trio's `except OSError` carve-out is not load-bearing here |
+| Graceful peer close arrives as `BrokenResourceError`/`ECONNRESET`, not a clean 0-byte EOF | benign; `_iter_packets()` already classifies it as a normal disconnect |
+| Topology `struct tipc_event` is **48 bytes** (`4+4+4+8+28`) | the plan said 40 |
+| The topology server **accepts native `'='` byte-order** | the plan's proposed `'>'`-retry endianness probe was deleted as unnecessary |
+| `TIPC_WAIT_FOREVER` is `-1` in python | must be masked (`& 0xFFFFFFFF`) before packing as unsigned |
+| TIPC **has** AES-GCM encryption (`tipc node set key`, linux 5.9+) | cluster/master/per-node keys + rekeying. Symmetric pre-shared, which is why a wg mesh is still preferred — see §6. |
+| A wg interface is L3/`tun` (`POINTOPOINT,NOARP`, `link/none`) | TIPC's `eth` media **cannot** bind it; the udp bearer is mandatory over wg |
+
+Two design decisions that are **closed**, with reasons:
+
+- **The accepting side does not learn the peer's service name.**
+  A port-id can't be reversed into one. It gets a
+  `TIPC_NAME_UNKNOWN` sentinel plus the observed `(node, ref)`,
+  and that's fine — the `Aid` from the handshake already carries
+  the peer's logical identity. (`uds` has the same wart.)
+- **Do not fold digest bits into the service `_stype` to widen
+  the collision space.** A topology subscription can only watch
+  **one** `stype`, so varying it per-actor would need 65536
+  subscriptions and kills the push-registry work outright. If
+  crosstalk ever bites, the escalation is a post-bind
+  verification handshake ([#501]).
+
+[#501]: https://github.com/goodboy/tractor/issues/501
+
+## 4. What landed
+
+15 commits, `22ef362d..2d082373`, on `wkt/tipc_backend_378`,
+based on `ng_tpts_planning` (PR [#492], docs-only).
+
+- `TIPCAddress` + `is_tipc_available()` + `start_listener()`
+- `MsgpackTIPCStream` (`connect_to()`, `get_stream_addrs()`)
+- `open_topology_events()` — the `TIPC_TOP_SRV` push feed
+- registration across every table in contract §2
+- interim `str`-only `/tipc/…` maddr grammar
+- a `--tpt-proto=tipc` CI leg, **non-blocking** for now
+- `docs/guide/tipc.rst` + `examples/multihost/tipc_cluster/`
+
+Two fixes fell out that are **not** TIPC-specific:
+
+- `devx/pformat.py` — `pformat_caller_frame()` passed an
+  `indent=''` kwarg `pformat_boxed_tb()` never accepted, so every
+  send-side `MsgTypeError` died with a `TypeError` while
+  formatting itself. On `main` and every branch since
+  `888af602`. **Wants cherry-picking out of this stack.**
+- `SpawnSpec.reg_addrs`/`.bind_addrs` pinned the wire shape to a
+  2-tuple. Widened to `UnwrappedAddress`, which had to become
+  **variadic** (`tuple[str|int, ...]`) because `msgspec` refuses
+  a union holding more than one array-like type.
+
+**Acceptance bar met**: 122 passed / 1 xfailed / 2 xpassed under
+`--tpt-proto tipc` across `ipc`, `discovery`, `runtime`,
+`spawning`, `local`, `rpc`, `cancellation`. `tcp`/`uds`
+unchanged.
+
+[#492]: https://github.com/goodboy/tractor/pull/492
+
+## 5. Immediate next steps (pre-land)
+
+These live on [#493]'s body as `### TODOs before landing`. They
+are **not** mirrored into an issue — if the PR is ever superseded
+they need re-homing.
+
+1. **Cherry-pick the `pformat` fix onto `main`** as its own PR —
+   `22ef362d` (red guard test) then `f9f98eeb` (the 1-line fix).
+   Unrelated to TIPC; every branch has the bug.
+2. **Watch the `tipc` CI leg.** It's gated
+   `continue-on-error: ${{ matrix.tpt_proto == 'tipc' }}` because
+   GH's runners have never been asked to `modprobe` for us. Once
+   it has a few green runs, drop the gate. If the runners refuse
+   the `modprobe`, fall back to a container job with
+   `--cap-add NET_ADMIN`.
+3. **Rebase onto `main` once #492 merges.**
+
+## 6. The follow-up track
+
+All filed with the `follow-up` label.
+
+| issue | what |
+| --- | --- |
+| [#495] | `TIPC_IMPORTANCE` supervision QoS on the parent↔child chan |
+| [#496] | `TIPC_TOP_SRV` push registry in `discovery._registry` |
+| [#497] | dual-link resiliency / multi-homing |
+| [#498] | `/tipc` multiaddr spec submission |
+| [#499] | registrar-less discovery via name derivation |
+| [#500] | multicast/group msging as a *broadcast* transport |
+| [#501] | post-bind verification for instance collisions |
+| [#502] | **TIPC over a `wg` mesh — the reference multihost deployment** |
+
+### the wg direction
+
+[#502] is the strategic one. The intent is that TIPC-over-`wg`
+becomes our go-to multihost transport deployment, with composed
+addresses of the form:
+
+```
+/ip4/<pub>/udp/51820/wg/u<key>/tipc/<stype>/<inst>/<scope>
+\____ wg bearer ________/\_key_/\______ tractor ep ________/
+```
+
+Note the structural point, which matters for the spec proposal
+in [#498]: the tcp equivalent repeats an `/ip4/…/tcp/…` inner
+segment because a tcp endpoint is *located*. The tipc inner
+segment has **no locative component at all** — a service name is
+location-independent by design. So in the composed form the wg
+segments carry all the routing and the tipc segment carries pure
+*identity*.
+
+Prerequisites already established:
+
+- py-multiaddr [#108] (merged) proved the composed `wg` + tcp
+  form parses and round-trips (`['ip4','udp','wg','ip4','tcp']`)
+- `examples/multihost/wg_lan/` is the existing wg example set to
+  generalize from rather than duplicate
+- the udp-bearer-only constraint and MTU caveat are documented in
+  both `docs/guide/tipc.rst` and the `tipc_cluster` README
+
+[#495]: https://github.com/goodboy/tractor/issues/495
+[#496]: https://github.com/goodboy/tractor/issues/496
+[#497]: https://github.com/goodboy/tractor/issues/497
+[#498]: https://github.com/goodboy/tractor/issues/498
+[#499]: https://github.com/goodboy/tractor/issues/499
+[#500]: https://github.com/goodboy/tractor/issues/500
+[#502]: https://github.com/goodboy/tractor/issues/502
+[#108]: https://github.com/multiformats/py-multiaddr/pull/108
+
+## 7. Working conventions in this repo
+
+Provider-neutral, but they *are* enforced by review:
+
+- **Never commit, push, rebase or amend on your own.** Prepare
+  changes, report them, and let the maintainer stage. Asking
+  "should we commit?" is a question *for you to answer*, not
+  permission to act.
+- **A failing/guard test lands in its own commit before the fix
+  it guards.** Red first, then green.
+- **One commit per logical step**, so history shows *why*. Never
+  squash unrelated changes.
+- Commit subjects: present-tense verb, ~50 chars (hard max 67),
+  backticks around every code element. Bodies wrap at 67 cols.
+- **Never write a line containing only whitespace.**
+- Annotate everything including locals; prefer `match`/`case`
+  over `isinstance` chains; multi-line calls with trailing
+  commas; `@acm` over classes with `.start()`/`.stop()`.
+- New modules get the `# tractor: distributed structured
+  concurrency.` header tagline plus the AGPL block.
+- **Do not change task/checkbox state** in issues, plans or
+  trackers unless explicitly asked for that exact transition.
+- Fix warnings at source; only genuinely-unfixable ones get
+  filtered, with a documented reason.
+
+## 8. Where things are
+
+```
+tractor/ipc/_tipc.py                     the whole backend
+tests/ipc/test_tipc.py                   28 backend unit tests
+tests/ipc/test_server.py                 the reconciliation guard
+tests/devx/test_pformat.py               the cherry-pick candidate
+docs/guide/tipc.rst                      the docs page
+examples/multihost/tipc_cluster/         runnable demos + manual
+                                         smoke test
+ai/tpt-backends/00_shared_backend_contract.md
+ai/tpt-backends/01_tipc_backend.md       the (reconciled) plan
+.github/workflows/ci.yml                 the gated tipc leg
+```
+
+Both single-host examples have been **run against a live
+kernel** — the output pasted in their README is real, not
+illustrative.
--- a/ai/tpt-backends/README.md
+++ b/ai/tpt-backends/README.md
@ -13,7 +13,7 @@ document only their own deltas.

 | plan | issue | dep | size | lands |
 | --- | --- | --- | --- | --- |
-| [01 — TIPC](./01_tipc_backend.md) | [#378] | **none** (stdlib) | small | first |
+| [01 — TIPC](./01_tipc_backend.md) | [#378] | **none** (stdlib) | small | **landed**, PR [#493] — see the [handoff](./01_tipc_HANDOFF.md) |
 | [02 — QUIC/`iroh`](./02_quic_iroh_backend.md) | [#353] | `iroh` (uniffi FFI) | large | needs a prep PR |
 | [03 — `wg` bindspace](./03_wg_tunnel_bindspace.md) | [#482], [#443] | `pyroute2` | medium, 3 layers | layer A now |

@ -52,3 +52,5 @@ lands first ships it.
 [#353]: https://github.com/goodboy/tractor/issues/353
 [#482]: https://github.com/goodboy/tractor/issues/482
 [#443]: https://github.com/goodboy/tractor/issues/443
+
+[#493]: https://github.com/goodboy/tractor/pull/493
--- a/docs/guide/tipc.rst
+++ b/docs/guide/tipc.rst
@ -157,11 +157,40 @@ node and the client's dial keeps working, unchanged. See
 (that directory is excluded from CI precisely because it needs
 real hardware).

+Over a WireGuard mesh
+~~~~~~~~~~~~~~~~~~~~~
+
+TIPC over a `wg` mesh is the intended reference deployment for
+multihost ``tractor`` (see gh #502), composing with the tunnel
+examples in ``examples/multihost/wg_lan/``.
+
+.. warning::
+
+   A wg interface is L3/``tun`` — ``POINTOPOINT,NOARP`` with
+   ``link/none`` and no L2 address — so TIPC's ``eth`` media
+   **cannot** bind it. Over wg the udp bearer is *mandatory*,
+   not merely an alternative:
+
+   .. code:: bash
+
+       # NOT possible over wg
+       sudo tipc bearer enable media eth device wg0
+
+       # required instead, bound to the wg overlay IP
+       sudo tipc bearer enable media udp name wgmesh \
+           localip 10.0.11.1
+
+   Mind the MTU too: wg links typically sit at 1420, under
+   ethernet's 1500.
+
 .. note::

-   TIPC over a UDP bearer composes with the WireGuard tunnel
-   examples in ``examples/multihost/wg_lan/`` — cluster-wide
-   kernel service discovery across an encrypted overlay.
+   TIPC is **not** unencrypted — it ships AES-GCM crypto of its
+   own (``tipc node set key``, linux 5.9+) with cluster, master
+   and per-node keys plus rekeying intervals. Those keys are
+   symmetric and pre-shared though, so a wg mesh is still
+   preferred for public-key identity, NAT traversal, and an
+   overlay every transport can share.

 Gotchas
 -------
@ -198,6 +227,19 @@ protocol in the multiaddr table yet, so the grammar is

    /tipc/<stype>/<instance>/<scope>

+Composed with a wg bearer — the form that actually matters for
+multihost — that becomes:
+
+.. code:: text
+
+    /ip4/<pub>/udp/51820/wg/u<key>/tipc/<stype>/<inst>/<scope>
+
+Note the tipc segment carries **no** locative component, unlike
+the ``/ip4/../tcp/..`` inner segment of the equivalent tcp maddr
+— a TIPC service name is location-independent by design, so the
+wg segments carry all the routing and the tipc segment is pure
+identity.
+
 Running the suite over TIPC
 ---------------------------

--- a/examples/multihost/tipc_cluster/README.md
+++ b/examples/multihost/tipc_cluster/README.md
@ -92,8 +92,8 @@ sudo modprobe tipc
 # over ethernet (L2) — simplest when the hosts share a segment
 sudo tipc bearer enable media eth device eth0

-# ..or over UDP when L2 isn't available (pairs nicely with the
-# `wg` tunnel examples in ../wg_lan/)
+# ..or over UDP when L2 isn't available — and MANDATORY over a
+# `wg` mesh, see below
 sudo tipc bearer enable media udp name uc localip 10.0.11.1

 # verify BEFORE running anything: this must list the peer
@ -116,6 +116,33 @@ Both sides name the *same service*, and the kernel routes it.
 Move `host_a_srv.py` to a third node and host B's dial keeps
 working, unchanged.

+### over a `wg` mesh
+
+TIPC over WireGuard is the intended reference multihost
+deployment (gh #502). One hard constraint: a wg interface is
+L3/`tun` — `POINTOPOINT,NOARP`, `link/none`, no L2 address — so
+TIPC's `eth` media **cannot** bind it. The udp bearer is
+mandatory there, bound to the wg overlay IP, and wg's typical
+1420 MTU sits under ethernet's 1500 so link MTU wants checking.
+
+Composed, the deployment address is:
+
+```
+/ip4/<pub>/udp/51820/wg/u<key>/tipc/<stype>/<inst>/<scope>
+\____ wg bearer ________/\_key_/\______ tractor ep ________/
+```
+
+Note the tipc segment has no locative part, unlike tcp's inner
+`/ip4/../tcp/..` — a service name is location-independent, so wg
+carries the routing and tipc carries identity.
+
+Worth knowing: TIPC is **not** unencrypted. It ships AES-GCM
+crypto of its own (`tipc node set key`, linux 5.9+) with
+cluster/master/per-node keys and rekeying. Those keys are
+symmetric and pre-shared, which is why a wg mesh is still
+preferred — public-key identity, NAT traversal, and one overlay
+shared by every transport.
+
 ### scope

 `TIPCAddress._scope` is the backend's `.bindspace` — literally