9.4 KiB

Raw Blame History

TIPC: when the kernel does discovery

Every other tractor transport gives you a pipe and leaves discovery to us: the registrar actor, the find_actor() round-trip, the whole discovery story. TIPC (Transparent Inter-Process Communication) is different — it's a linux-kernel cluster protocol whose service names live in a cluster-wide name table the kernel itself maintains.

Which flips the model:

an actor's IPC address is a service name (stype, instance) — no host, no port,
.bind()-ing that name is service registration,
a peer's .connect()-by-name is the lookup, resolved and load-balanced in-kernel.

So for TIPC-capable deployments the registrar round-trip stops being the only way peers find each other. Enable it per actor like any other backend,

async with tractor.open_nursery(
    enable_transports=['tipc'],
) as an:
    ...

Warning

TIPC is opt-in and linux-only. The tipc kernel module is not loaded on most boxes (sudo modprobe tipc), and the address family doesn't exist off-linux at all. Check tractor.ipc._tipc.is_tipc_available before assuming; tractor never selects this backend for you.

Your actor tree, in the kernel's name table

The single best demo this backend has needs no tractor API at all — boot a tree and ask tipc(8) what it sees:

sudo modprobe tipc
python examples/multihost/tipc_cluster/single_host.py

--- `tipc nametable show` :: root + 3 subactors ---
  Type       Lower      Upper      Scope    Port
  1953628160 1616       1616       cluster  3161982128
  1953628160 1219427151 1219427151 cluster  1587358717
  1953628160 2641339936 2641339936 cluster  1864021571
  1953628160 3344505866 3344505866 cluster  3816483388

--- `tipc nametable show` :: after teardown (all withdrawn) ---
  Type       Lower      Upper      Scope    Port

Reading the rows,

1953628160 is 0x74720000, tractor's reserved service type — ascii tr in the high half, with the low 16 bits free so an app can partition its own service classes via TIPCAddress._stype,
1616 is the host-singleton registrar instance, the same "1616 is tractor's registrar" idiom as the TCP port and the registry@1616.sock UDS filename,
the other three are per-actor instances derived from a blake2b digest of the actor's identity (see Silent crosstalk),
Scope is the address' bindspace — see Scope is the bindspace.

Push-based discovery

TIPC also exposes a topology service: subscribe and the kernel pushes you name-table transitions as they happen. tractor.ipc._tipc.open_topology_events wraps it as an @acm yielding a trio receive-channel,

from tractor.ipc._tipc import open_topology_events

async with open_topology_events() as events:
    async for ev in events:
        print(f'{ev.kind}: {ev.addr}')

watching the TIPC name table..
  [+] published  instance=1616         port=0x00000000:2375440573
spawning subactors..
  [+] published  instance=186947472    port=0x00000000:3960753074
  [+] published  instance=2191362136   port=0x00000000:2263898853
tearing down..
  [-] withdrawn  instance=186947472    port=0x00000000:3960753074

No polling, no registrar round-trip — this is the groundwork for a registrar that keeps a live view of the actor set without ever calling find_actor().

filt picks the granularity: TIPC_SUB_SERVICE gives one event per name becoming (un)available, TIPC_SUB_PORTS one per publisher — which is what makes the duplicate-name case below externally observable.

Scope is the bindspace

Every tractor address type has a .bindspace — "the set of hosts this bind is reachable from". For TCP that's the IP, for UDS the socket-file directory. For TIPC it's the scope, which is about as literal a reading of that docstring as exists:

scope	meaning
`TIPC_NODE_SCOPE`	same host only — the UDS analogue
`TIPC_CLUSTER_SCOPE`	cluster-visible (the default)

TIPC_ZONE_SCOPE is deprecated and aliased to cluster-scope by modern kernels; tractor accepts it on input, folds it to cluster and logs at transport level.

Spanning hosts

Single-host TIPC needs only modprobe. Crossing hosts needs a bearer enabled on both — an ethernet (L2) or UDP underlay the kernel routes service names over:

# on BOTH hosts
sudo tipc bearer enable media eth device eth0
# ..or, when L2 isn't available:
sudo tipc bearer enable media udp name uc localip 10.0.11.1

tipc link list      # must list the peer before you proceed

The two-host example pair then talks with no IP, hostname or port anywhere in either script — both sides name the same service and the kernel routes it. Move the server to a third node and the client's dial keeps working, unchanged. See examples/multihost/tipc_cluster/ for the full walkthrough (that directory is excluded from CI precisely because it needs real hardware).

Over a WireGuard mesh

TIPC over a wg mesh is the intended reference deployment for multihost tractor (see gh #502), composing with the tunnel examples in examples/multihost/wg_lan/.

Warning

A wg interface is L3/tun — POINTOPOINT,NOARP with link/none and no L2 address — so TIPC's eth media cannot bind it. Over wg the udp bearer is mandatory, not merely an alternative:

# NOT possible over wg
sudo tipc bearer enable media eth device wg0

# required instead, bound to the wg overlay IP
sudo tipc bearer enable media udp name wgmesh \
    localip 10.0.11.1

Mind the MTU too: wg links typically sit at 1420, under ethernet's 1500.

Note

wg is not about confidentiality here. TIPC ships AES-GCM crypto of its own (tipc node set key, linux 5.9+) with cluster, master and per-node keys plus rekeying intervals, so "wg adds the encryption TIPC lacks" is simply wrong.

The motivation is different but real: TIPC's keys are symmetric and pre-shared, leaving distribution, rotation and revocation to the operator, whereas wg brings public-key identity and a handshake — plus NAT traversal, and one overlay that every transport can share rather than a TIPC-only mechanism. Which to prefer is worth benchmarking; native crypto avoids a tunnel hop entirely.

Gotchas

Silent crosstalk. Unlike every other backend, a duplicate bind does not raise EADDRINUSE. TIPC accepts multiple publishers of one name and round-robins connects between them — verified: six dials alternated strictly between two listeners. So an instance collision splits traffic silently instead of erroring. That's why TIPCAddress.get_random() derives its instance from a blake2b digest of the actor identity rather than a counter, and why two tractor trees sharing both a cluster and an _stype share a namespace — partition them with a distinct _stype.

Graceful close looks like a reset. A peer closing cleanly surfaces as BrokenResourceError/ECONNRESET rather than the clean 0-byte EOF TCP and UDS give you. Benign — the transport layer already classifies it as a normal disconnect — but it does look alarming in transport-level logs.

Dialing an unpublished name answers EHOSTUNREACH instantly, with no SYN-timeout wait. That's markedly better discovery-ping behaviour than TCP; tractor normalizes it to ConnectionError so the usual lookup paths work unchanged.

Multiaddrs are interim. There's no registered /tipc protocol in the multiaddr table yet, so the grammar is str-only:

/tipc/<stype>/<instance>/<scope>

Composed with a wg bearer — the form that actually matters for multihost — that becomes:

/ip4/<pub>/udp/51820/wg/u<key>/tipc/<stype>/<inst>/<scope>

Note the tipc segment carries no locative component, unlike the /ip4/../tcp/.. inner segment of the equivalent tcp maddr — a TIPC service name is location-independent by design, so the wg segments carry all the routing and the tipc segment is pure identity.

Running the suite over TIPC

The backend is a first-class suite mode — the entire existing test suite runs over it unmodified, which is the acceptance bar for any tractor transport:

sudo modprobe tipc
pytest --tpt-proto tipc

Without the module that fails loudly and immediately with an actionable message rather than a few hundred confusing connect timeouts. Backend-specific unit tests live in tests/ipc/test_tipc.py and self-skip when the module is absent.

Normative references

The tipc.io documentation is stale in places. Treat the kernel sources as the only authority:

include/uapi/linux/tipc.h — address flavours, sockopts, the topology structs
net/tipc/socket.c, net/tipc/topsrv.c
man 8 tipc

9.4 KiB Raw Blame History