# `tractor` over `AF_TIPC`, where the address *is* the service name TIPC is a linux-kernel cluster IPC protocol whose service names live in a **cluster-wide name table maintained by the kernel**. It is also described as **Cluster Domain Sockets**: the Unix-domain socket model extended from one kernel to a cluster. That name is a useful explanation for new users, while the code keeps `tipc` as its protocol key to match Linux's `AF_TIPC`, kernel module and tooling. For `tractor` that means: - an actor's IPC address is a service name `(stype, instance)`, not a host/port, - `.bind()`ing it **is** service registration, - a peer's `.connect()`-by-name **is** the lookup. So the discovery machinery `tractor.discovery` normally implements with a registrar actor comes for free, in-kernel — which is the ask in gh [#378](https://github.com/goodboy/tractor/issues/378). > **Why `examples/multihost/`?** `tests/test_docs_examples.py` > walks `examples/` recursively and runs everything it collects > as a subproc, asserting `rc == 0`. These need the `tipc` > kernel module (and, for the two-host pair, a live bearer), so > they can't satisfy that; `'multihost' not in p[0]` is already > in the test's exclusion list, which is what keeps them out of > CI. See "CI" below for the separate matrix-entry plan. ## the single best demo ```bash sudo modprobe tipc python single_host.py ``` Four actors boot, four service names appear in the kernel's table, and all four are withdrawn on teardown — observed with `tipc(8)`, entirely outside `tractor`: ``` --- `tipc nametable show` :: root + 3 subactors --- Type Lower Upper Scope Port 1953628160 1616 1616 cluster 3161982128 1953628160 1219427151 1219427151 cluster 1587358717 1953628160 2641339936 2641339936 cluster 1864021571 1953628160 3344505866 3344505866 cluster 3816483388 --- `tipc nametable show` :: after teardown (all withdrawn) --- Type Lower Upper Scope Port ``` `1953628160` is `0x74720000` — `tractor`'s reserved service type, ascii `tr` in the high half. `1616` is the host-singleton registrar, the same idiom as the TCP port and the `registry@1616.sock` UDS filename. The other three instances are per-actor digests (see "silent crosstalk" below). ## push-based discovery ```bash python watch_nametable.py ``` Subscribes to the kernel's *topology service* and prints name table transitions as they happen — no polling, no registrar round-trip: ``` watching the TIPC name table.. [+] published instance=1616 port=0x00000000:2375440573 spawning subactors.. [+] published instance=186947472 port=0x00000000:3960753074 [+] published instance=2191362136 port=0x00000000:2263898853 [+] published instance=3484369663 port=0x00000000:2126817956 tearing down.. [-] withdrawn instance=186947472 port=0x00000000:3960753074 ... ``` This is the groundwork for a push registry in `tractor.discovery._registry` (gh [#184](https://github.com/goodboy/tractor/issues/184), [#216](https://github.com/goodboy/tractor/issues/216)) — a registrar that *never polls* `find_actor()`. ## two hosts Everything above is single-node (`modprobe` is enough). To span hosts you need a **bearer** on both, which is the one thing that can't be CI'd. For the first physical test, use two wired Linux hosts on the same L2 segment. Prefer a direct cable or uncomplicated switch; avoid Wi-Fi, guest VLANs and port isolation until the basic link works. Use the same checkout and Python environment on both hosts: ```bash # on BOTH hosts git rev-parse HEAD # must match on A and B uv sync --all-extras --dev sudo modprobe tipc # choose the real wired iface; do not assume `eth0` ip -br link IFACE=enp3s0 # inspect existing cluster identity before changing anything tipc node get address # must differ between hosts tipc node get netid # must match between hosts # use one private test netid on BOTH hosts, before enabling bearers sudo tipc node set netid 37801 # ethernet is simplest when the hosts share an L2 segment sudo tipc bearer enable media eth device "$IFACE" # ..or over UDP when L2 isn't available — and MANDATORY over a # `wg` mesh, see below sudo tipc bearer enable media udp name uc localip 10.0.11.1 # verify BEFORE running anything: this must list the peer tipc bearer list tipc link list tipc node list ``` If the link does not appear, first verify carrier, a common TIPC network ID, distinct node addresses, a common VLAN and compatible MTUs. Ethernet TIPC uses EtherType traffic rather than IP routing, so a successful `ping` alone does not prove the bearer can work. Run each command from `examples/multihost/tipc_cluster/`: ```bash # host A uv run python host_a_srv.py # host B watch -n 0.5 tipc nametable show # optional second terminal uv run python host_b_client.py ``` Note what's absent from both scripts: any IP, hostname or port. Both sides name the *same service*, and the kernel routes it. Move `host_a_srv.py` to a third node and host B's dial keeps working, unchanged. For a first resilience pass, use a local console or separate management link so the test does not cut off your own SSH session: ```bash # host B: record the healthy baseline tipc link list tipc link statistics show # either host: withdraw and recreate the bearer sudo tipc bearer disable media eth device "$IFACE" tipc link list sudo tipc bearer enable media eth device "$IFACE" tipc link list # prove name withdrawal/republication and RPC recovery tipc nametable show uv run python host_b_client.py ``` Capture `uname -a`, both node addresses, `tipc bearer list`, `tipc link list`, `tipc link statistics show`, the name table and both Python transcripts. Those artifacts distinguish an actor bug from bearer discovery, cluster identity or switch configuration. Clean up a disposable Ethernet test on both hosts with: ```bash sudo tipc bearer disable media eth device "$IFACE" tipc link list ``` Restore any pre-existing network ID only after all bearers are disabled. The first useful automation target is a two-node network namespace fixture that asserts link-up, remote publication, RPC, withdrawal and republication in that order; physical hardware then remains the validation layer for real NIC and switch behaviour. The commands above use iproute2's `tipc` frontend, which speaks the kernel's `TIPCv2` generic-netlink family. The planned `pyroute2` dependency already manages WireGuard, interfaces and namespaces and provides generic-netlink primitives, but it does not currently ship a TIPC message codec. Adding one upstream would let `tractor` replace these manual commands with one Python netlink stack instead of shelling out; until then, `tipc(8)` remains the canonical frontend. ### over a `wg` mesh TIPC over WireGuard is the intended reference multihost deployment (gh #502). One hard constraint: a wg interface is L3/`tun` — `POINTOPOINT,NOARP`, `link/none`, no L2 address — so TIPC's `eth` media **cannot** bind it. The udp bearer is mandatory there, bound to the wg overlay IP, and wg's typical 1420 MTU sits under ethernet's 1500 so link MTU wants checking. Composed, the deployment address is: ``` /ip4//udp/51820/wg/u/tipc/// \____ wg bearer ________/\_key_/\______ tractor ep ________/ ``` Note the tipc segment has no locative part, unlike tcp's inner `/ip4/../tcp/..` — a service name is location-independent, so wg carries the routing and tipc carries identity. **wg here is not about confidentiality.** TIPC ships AES-GCM crypto of its own (`tipc node set key`, linux 5.9+) with cluster/master/per-node keys and rekeying, so "wg adds the encryption TIPC lacks" is wrong. The motivation is different but real: TIPC's keys are *symmetric and pre-shared* — distribution, rotation and revocation are all on you — whereas wg gives public-key identity and a handshake, NAT traversal, and one overlay shared by every transport instead of a TIPC-only mechanism. Worth benchmarking either way; native crypto skips the tunnel hop. Note too that the ethernet-bearer pairing #378 imagined does **not** apply over wg: on a given link the L2 path and the wg path are mutually exclusive. ### scope `TIPCAddress._scope` is the backend's `.bindspace` — literally "the set of hosts this published name is reachable from": | scope | meaning | | --- | --- | | `TIPC_NODE_SCOPE` | same host only — the UDS analogue | | `TIPC_CLUSTER_SCOPE` | cluster-visible (the default) | `TIPC_ZONE_SCOPE` is deprecated and aliased to cluster by modern kernels; `tractor` accepts it on input and folds it, logging at `transport` level. ## gotchas worth knowing before you deploy **Silent crosstalk.** Unlike every other backend, a duplicate bind does **not** raise `EADDRINUSE` — TIPC happily accepts multiple publishers of one name and *round-robins* connects between them (verified: 6 dials alternated `b,a,b,a,b,a`). So an instance collision is silent traffic-splitting, not an error. That's why `TIPCAddress.get_random()` derives the instance from a `blake2b` digest of the actor identity rather than a counter. Two `tractor` trees sharing both a cluster **and** an `_stype` share a name space; partition them by passing a distinct `_stype`. **Graceful close looks like a reset.** A peer closing cleanly surfaces as `BrokenResourceError`/`ECONNRESET` rather than the clean 0-byte EOF you get from TCP/UDS. It's benign — the transport layer already classifies it as a normal disconnect — but it does look alarming in `transport`-level logs. **Dialing an unpublished name** answers `EHOSTUNREACH` *instantly* (no SYN-timeout wait), which is much better discovery-ping behaviour than TCP. `tractor` normalizes it to `ConnectionError`. **It's opt-in, never a default.** The module isn't loaded on most boxes and doesn't exist off-linux, so `enable_transports=['tipc']` is always explicit. Check `tractor.ipc._tipc.is_tipc_available()` before assuming. ## maddr form There is no registered `/tipc` protocol in the multiaddr table yet (upstream track: gh [#483](https://github.com/goodboy/tractor/issues/483) + multiformats/py-multiaddr#107), so the grammar is interim and `str`-only: ``` /tipc/// ``` `parse_maddr()` special-cases this prefix *before* handing anything to `Multiaddr()`, which would otherwise reject the unregistered name outright. Registering it upstream is what would unblock gh [#443](https://github.com/goodboy/tractor/issues/443)'s "return `Multiaddr` everywhere" item. ## running the suite over TIPC The whole test suite runs under the backend: ```bash sudo modprobe tipc pytest --tpt-proto tipc ``` Without the module that fails loudly and immediately with an actionable message rather than a few hundred connect timeouts. Backend-specific unit tests live in `tests/ipc/test_tipc.py` and self-skip when the module is absent. ## CI Single-host TIPC *is* CI-able — the module ships with the standard Ubuntu kernel package. CI loads it with `sudo modprobe tipc` and runs the suite with `--tpt-proto tipc` as a blocking matrix leg. Cross-node bearer testing stays manual — this README is that smoke test. ## normative refs The tipc.io docs are stale in places (gh #378 says as much). Treat the kernel sources as the only normative reference: - `include/uapi/linux/tipc.h` — address flavours, sockopts, the topology `struct`s - `net/tipc/socket.c`, `net/tipc/topsrv.c` - `man 8 tipc`