Compare commits

...

14 Commits

Author SHA1 Message Date
Gud Boi 451e0acf8a Add prompt-io log for GH #473 UDS-on-macOS work
Provenance entry (+ unedited raw output) for the root-cause
session behind the prior three patches, per the NLNet
generative-AI policy tracked under `ai/prompt-io/`.

(this patch was generated in some part by [`claude-code`][claude-code-gh])
[claude-code-gh]: https://github.com/anthropics/claude-code
2026-08-13 15:46:56 -04:00
Gud Boi 3d02a8569e Run UDS-on-macOS CI leg, un-skip the example
UDS-on-macOS is otherwise un-exercised: the matrix
explicitly excludes the `macos-latest` + `uds` combo, so the
`uds_transport_actor_tree.py` example (skipped on macOS CI
since PR #460) is the only thing that ever touches that
path.

- drop the matrix `exclude` so the full suite runs with
  `--tpt-proto=uds` on `macos-latest`.
- un-skip the example on macOS+CI; with example-stderr
  surfacing in place a still-red run now yields the full
  traceback GH #473 asks for instead of a bare returncode
  assert.

Task-bullets 3 + 4 of GH #473.

Prompt-IO: ai/prompt-io/claude/20260702T155006Z_65bf9df5_prompt_io.md

(this patch was generated in some part by [`claude-code`][claude-code-gh])
[claude-code-gh]: https://github.com/anthropics/claude-code
2026-08-13 15:46:56 -04:00
Gud Boi bceca74eb7 Fix UDS addr corruption sans-autobind (macOS)
`MsgpackUDSStream.get_stream_addrs()` matches the
`(peername, sockname)` pair by type to find the listener's
fs-path, but the `(str, str)` arm unconditionally takes
`peername`: on platforms without linux's
`SO_PASSCRED`-triggered autobind (macOS!) the accept side's
`getpeername()` is `''`, so every accepted conn gets garbage
`Path('')` laddr/raddr structs.

Proven on linux by disabling `SO_PASSCRED` (no autobind ->
same `''` shape as darwin): the `uds_transport_actor_tree.py`
example reports `listener sock file: .` pre-fix and the real
registry sockpath post-fix.

- pick the non-empty name in the `(str, str)` arm: `peername`
  on the connect side, `sockname` on the accept side; raise
  `ValueError` on an (unexpected) empty pair.
- document the linux-autobind origin of the `bytes` arms
  which the original impl noted as "unclear".
- `start_listener()`: create the bindspace dir with
  `parents=True, exist_ok=True` (nested custom `filedir`s +
  racing actors).
- example docstring: peer-pid comes via `SO_PEERCRED` on
  linux but `LOCAL_PEERPID` on macOS.

May not be the (only) macOS crasher for GH #473 — it is
non-fatal on the linux sim — but with stderr surfacing now
in place the next macOS CI run pins any remaining layer.

Prompt-IO: ai/prompt-io/claude/20260702T155006Z_65bf9df5_prompt_io.md

(this patch was generated in some part by [`claude-code`][claude-code-gh])
[claude-code-gh]: https://github.com/anthropics/claude-code
2026-08-13 15:46:56 -04:00
Gud Boi 18faffcff7 Surface example stderr on any non-zero exit
The docs-example harness only re-raises captured subproc
stderr when the LAST line contains 'Error', but a `tractor`
root-actor crash always ends stderr with the strict-EG
collapse note `( ^^^ this exc was collapsed from a group ^^^ )`
— so every possible crash is swallowed down to a bare
`assert 1 == 0`, exactly what the macOS CI leg shows for the
UDS example in GH #473.

- raise with the FULL stderr (+stdout) whenever the example
  subproc exits non-zero, regardless of stderr shape.
- keep the legacy last-line 'Error' check for zero-rc runs
  which still emit error-ish output.

First task-bullet of GH #473.

Prompt-IO: ai/prompt-io/claude/20260702T155006Z_65bf9df5_prompt_io.md

(this patch was generated in some part by [`claude-code`][claude-code-gh])
[claude-code-gh]: https://github.com/anthropics/claude-code
2026-08-13 15:46:56 -04:00
Bd 7554f90e59
Merge pull request #478 from goodboy/wkt/boot_latency_470
Trim `import tractor` 0.42s -> 0.15s (gh #470)
2026-08-13 15:02:13 -04:00
Gud Boi 3705bbe594 Guard the cold import latency budget
Run seven fresh interpreters and gate the median `import tractor`
time below a conservative 0.35s ceiling. Provide an explicit
environment override for platforms with a different baseline.

Also assert the optional modules deferred by this patch remain
unloaded so a timing pass cannot hide an eager-import regression.

Review: PR #478 (goodboy)
https://github.com/goodboy/tractor/pull/478#pullrequestreview-4922213201

(this patch was generated in some part by `opencode` using
`gpt-5.6-sol` (`openai`))
2026-08-13 02:35:07 -04:00
Gud Boi 66ac7863b5 Advertise the lazy `to_asyncio` API
Preserve the existing public wildcard surface while adding the lazy
`to_asyncio` submodule to `__all__` and `dir(tractor)`.

Exercise both APIs in cold interpreters and verify normal package
import still leaves `asyncio` unloaded.

Review: PR #478 (goodboy)
https://github.com/goodboy/tractor/pull/478#pullrequestreview-4922213201

(this patch was generated in some part by `opencode` using
`gpt-5.6-sol` (`openai`))
2026-08-12 20:55:41 -04:00
Gud Boi 089e158da9 Retain dynamic logger caller discovery
Keep the `sys.modules` lookup as the normal fast path, then use
`inspect.getmodule()` for unregistered `runpy`, plugin, and `exec()`
namespaces.

Cover an unregistered module name backed by a real package file and
verify implicit logger naming still resolves to that package.

Review: PR #478 (goodboy)
https://github.com/goodboy/tractor/pull/478#pullrequestreview-4922213201

(this patch was generated in some part by `opencode` using
`gpt-5.6-sol` (`openai`))
2026-08-12 20:52:47 -04:00
Gud Boi 7987f6b8d7 Keep lazy annotations runtime-resolvable
Provide import-free runtime aliases for annotation-only actor and
multiaddr types so `typing.get_type_hints()` remains usable without
eagerly loading optional dependencies.

Correct `_address_types` to its actual `dict` shape and cover the
affected discovery and transport APIs.

Caught-during: review remediation
Found-via: `/run-tests` test_lazy_annotation_names_resolve

Review: PR #478 (goodboy)
https://github.com/goodboy/tractor/pull/478#pullrequestreview-4922213201

(this patch was generated in some part by `opencode` using
`gpt-5.6-sol` (`openai`))
2026-08-12 20:52:15 -04:00
Gud Boi 70c7e334a7 Make sub-spawn strategy toggleable in `we_are_processes`
The #470 boot-latency example hard-coded spawning each `worker_<i>`
subactor concurrently from a bg `trio.Task` (so each child's cold
`import tractor` overlaps). Add a `main()` `spawn_subs_in_bg_tasks`
flag so the serial-spawn path can be demo'd/compared too: flip it
`False` to `start_actor()` each sub inline in the loop before
handing the ready `Portal` to the bg task.

Deats,
- factor an `open_ep(ptl, i)` helper out of `spawn_and_open_ep()` -
  just the `Portal.open_context()` + `wait_for_result()` half, now
  that the spawn step is caller-optional.
- `spawn_and_open_ep()` grows a `maybe_ptl: Portal|None = None`
  param: spawn the subactor itself when unset (bg-task path), OW
  reuse the pre-spawned one (serial path).
- move the "overlap cold imports" rationale comment onto the new
  `main()` param where the toggle now lives.

(this commit msg was generated in some part by [`claude-code`][claude-code-gh])
[claude-code-gh]: https://github.com/anthropics/claude-code
2026-08-12 19:59:49 -04:00
Gud Boi 62b729a106 Add prompt-io entries for gh #470 latency work
Log the AI-assisted session per the NLNet generative-AI
policy: prompt, profiling findings, per-file diff pointers,
measured results and the unimplemented `pdbp`/`platformdirs`
deferral follow-ups.

(this patch was generated in some part by [`claude-code`][claude-code-gh])
[claude-code-gh]: https://github.com/anthropics/claude-code
2026-08-12 19:57:28 -04:00
Gud Boi 213a298aad Defer `asyncio` import via PEP-562 `.to_asyncio`
`asyncio` (~5ms) only matters for infected-aio actors yet gets
imported by every cold `import tractor` via module-lvl
`.to_asyncio` imports in the debug-REPL + spawn-entry mods.

Deats,
- `devx.debug._trace`/`._tty_lock`: mv `import asyncio` under
  `TYPE_CHECKING` + fn-local it at the two
  `asyncio.current_task()` call-sites; fn-local the
  `run_trio_task_in_future` imports in the infected-aio-only
  branches.
- `spawn._entry`: fn-local `run_as_asyncio_guest` inside the
  `infect_asyncio=True` branches of `_mp_main()`/
  `_trio_main()`.
- `tractor/__init__.py`: add a PEP-562 module `__getattr__`
  lazy-loading `.to_asyncio` on first attr-access so the
  public `tractor.to_asyncio.<attr>` API (e.g.
  `LinkedTaskChannel` annots in
  `test_child_manages_service_nursery.py` + downstream users)
  keeps working unchanged.

Prompt-IO: ai/prompt-io/claude/20260702T155626Z_65bf9df5_prompt_io.md
(this patch was generated in some part by [`claude-code`][claude-code-gh])
[claude-code-gh]: https://github.com/anthropics/claude-code
2026-08-12 19:57:28 -04:00
Gud Boi d1d3fc58bb Lazy-import optional 3rd-party deps (gh #470)
Move every import-time-only-by-accident dep off the eager
`import tractor` path so cold child-actor boots only pay for
what they actually use:

- `bidict` -> `TYPE_CHECKING` in `discovery._addr`
  (annotation-only; `_address_types` is a plain `dict`
  literal).
- `multiaddr` -> `TYPE_CHECKING` + fn-local imports in
  `discovery._multiaddr.mk_maddr()`/`parse_maddr()`; also
  `TYPE_CHECKING` the `Multiaddr` annots in `ipc._tcp`/`._uds`
  (adds future-annots to `._multiaddr`).
- `colorlog` -> fn-local in `log.get_console_log()`.
- `pdbp` + `wrapt` -> fn-local in
  `devx._frame_stack.hide_runtime_frames()`/`api_frame()`.
- `platformdirs` -> fn-local in `runtime._state.get_rt_dir()`.

Still eager (documented follow-ups),
- `pdbp` via `devx.debug._repl` class-bases
  (`PdbREPL(pdbp.Pdb)`) + the module-lvl `@pdbp.hideframe` in
  `._tty_lock`; needs a `._repl` restructure.
- `platformdirs` via the `UDSAddress.def_bindspace: ClassVar`
  class-body eval of `get_rt_dir()`; needs an `Address`-proto
  rework.
- `stackscope` is already fn-local; `setproctitle` is not
  imported anywhere.

Prompt-IO: ai/prompt-io/claude/20260702T155626Z_65bf9df5_prompt_io.md
(this patch was generated in some part by [`claude-code`][claude-code-gh])
[claude-code-gh]: https://github.com/anthropics/claude-code
2026-08-12 19:57:28 -04:00
Gud Boi a2c8af558e Use `sys._getframe()` in `get_logger()` mod lookup
`get_caller_mod()` (nested in `get_logger()`) walks the WHOLE
call-stack via `inspect.stack()`, which also resolves src-file
info for every frame and scans all of `sys.modules` per frame
via `inspect.getmodule()`. During nested imports (deep
importlib stacks) each module-level `get_logger()` call costs
~5-10ms, making the ~39 such calls dominate `import tractor`
wall-time: ~244ms of the ~420ms total (see gh #470).

Deats,
- resolve the caller frame with `sys._getframe(frames_up)` and
  map its `f_globals['__name__']` through `sys.modules`: O(1)
  vs. O(stack x sys.modules).
- guard `ValueError` (stack too shallow) -> `None`, matching
  the existing null-caller handling at all use-sites.
- drop the now-unused `inspect` imports; pull `FrameType` from
  `types` instead.

Results: `import tractor` drops 0.42s -> ~0.155s; sequential
`.start_actor()` spawn latency ~0.42 -> ~0.18s/actor.

Prompt-IO: ai/prompt-io/claude/20260702T155626Z_65bf9df5_prompt_io.md
(this patch was generated in some part by [`claude-code`][claude-code-gh])
[claude-code-gh]: https://github.com/anthropics/claude-code
2026-08-12 19:57:28 -04:00
21 changed files with 850 additions and 77 deletions

View File

@ -118,11 +118,6 @@ jobs:
'tcp', 'tcp',
'uds', 'uds',
] ]
# https://github.com/orgs/community/discussions/26253#discussioncomment-3250989
exclude:
# don't do UDS run on macOS (for now)
- os: macos-latest
tpt_proto: 'uds'
steps: steps:
- uses: actions/checkout@v4 - uses: actions/checkout@v4

View File

@ -0,0 +1,77 @@
---
model: claude-fable-5
service: claude
session: b8ec2ffe-1e47-4db5-8bc3-5bebc5feaaf6
timestamp: 2026-07-02T15:50:06Z
git_ref: 65bf9df5
scope: code
substantive: true
raw_file: 20260702T155006Z_65bf9df5_prompt_io.raw.md
---
## Prompt
From `ai/prompt-io/prompts/issue_473.md`:
> NOTE: you MUST pause this work at 12:50PM EST (BEFORE
> your weekly limit reset) for review by a human!
>
> resolve GH #473: https://github.com/goodboy/tractor/issues/474
> do it with /open-wkt.
(The prompt's issue-number/URL mismatch was resolved in
favor of #473 — file name, session name and prompt text
all say #473; the `/474` URL is a typo.)
GH #473: "Root-cause UDS-on-macOS —
`uds_transport_actor_tree` example fails on CI".
## Response summary
Worked all four #473 next-step checkboxes in wkt
`uds_macos_473`:
1. **stderr surfacing**: `test_docs_examples.py` now
raises with the full subproc stderr (+stdout) on ANY
non-zero exit. The old last-line-'Error' check could
never fire for `tractor` crashes since root-actor
stderr always ends with the strict-EG collapse note
`( ^^^ this exc was collapsed from a group ^^^ )`
proven against the real PR #460 macOS CI log (bare
`assert 1 == 0`, no traceback).
2. **root-cause (linux-provable layer)**: macOS-only
addr corruption in
`MsgpackUDSStream.get_stream_addrs()` — no
`SO_PASSCRED`/autobind on darwin means the accept
side's `getpeername()` is `''`, and the
`(str(), str())` arm took `peername` unconditionally →
`Path('')` garbage addrs on every accepted conn.
Proven + fixed via linux no-autobind simulation.
Possibly not the final macOS crasher (non-fatal on
linux-sim); the diagnostic patch guarantees the next
macOS CI run shows any remaining layer.
3. **CI matrix**: removed the `macos-latest`+`uds`
exclude.
4. **un-skip**: dropped the macOS+CI skip of the example.
Also: `start_listener()` bindspace mkdir hardened
(`parents=True, exist_ok=True`), example docstring
peer-pid mechanism corrected for macOS.
## Files changed
- `tests/test_docs_examples.py` — surface full stderr on
non-zero exit; remove macOS skip of the UDS example
- `tractor/ipc/_uds.py` — fix no-autobind
`get_stream_addrs()` arm; harden bindspace mkdir;
document autobind semantics
- `.github/workflows/ci.yml` — add macOS UDS leg (remove
matrix exclude)
- `examples/uds_transport_actor_tree.py` — platform-
correct peer-pid docs
## Human edits
None yet — pending human review (hard-stop review
checkpoint at 12:50PM EST per prompt); commit staging and
any edits are the human's.

View File

@ -0,0 +1,101 @@
---
model: claude-fable-5
service: claude
timestamp: 2026-07-02T15:50:06Z
git_ref: 65bf9df5
diff_cmd: git diff main..wkt/uds_macos_473
---
# Raw output — GH #473 UDS-on-macOS root-cause session
NOTE: code output is diff-referenced (not copied) per
prompt-io diff-ref mode; all generated code is on branch
`wkt/uds_macos_473` relative to `main` (65bf9df5).
## Diagnostic narrative (verbatim)
Root-cause work proceeded by linux-side simulation since no
macOS host is available:
1. Pulled the actual macOS CI failure log from PR #460 run
28343878933 (job 84115107992): the failure is a bare
`assert 1 == 0` on `proc.returncode` with NO stderr
surfaced — the harness only re-raised when the LAST
stderr line contained 'Error'.
2. Reproduced the stderr-swallow mechanism on linux: any
`tractor` root-actor crash ends stderr with the
strict-EG collapse note
`( ^^^ this exc was collapsed from a group ^^^ )` which
never matches 'Error' — so EVERY possible crash was
swallowed. (Verified by sabotaging the runtime dir via
an over-long `XDG_RUNTIME_DIR` → `OSError: AF_UNIX path
too long` → rc=1 + swallowed.)
3. Found + proved a macOS-only addr-corruption bug in
`MsgpackUDSStream.get_stream_addrs()`: the
`(str(), str())` match-arm unconditionally took
`peername`, but on no-autobind platforms (macOS lacks
linux's `SO_PASSCRED`-triggered autobind) the accept
side's `getpeername()` is `''``Path('')` garbage
laddr/raddr on EVERY accepted UDS conn. Simulated on
linux by nulling `SO_PASSCRED` (no autobind → same `''`
shape): pre-fix the example printed
`listener sock file: .`; post-fix it prints the real
registry sockpath. Non-fatal on linux-sim (rc=0), so
possibly not the final macOS crasher — the diagnostic
patch guarantees the next macOS CI run reveals any
remaining layer.
4. Falsified the missing-parent-dir theory:
`get_rt_dir()` already `mkdir(parents=True,
exist_ok=True)`s at import (and macOS TCP CI passes),
so `~/Library/Caches/TemporaryItems` absence cannot be
the crasher. Hardened `start_listener()`'s bindspace
mkdir anyway (custom `filedir` case + racing actors).
## Generated changes (diff pointers)
> `git diff main..wkt/uds_macos_473 -- tests/test_docs_examples.py`
- always raise with FULL subproc stderr (+stdout) on any
non-zero example exit; keep legacy last-line 'Error'
check for zero-rc cases; drop the macOS+CI skip of
`uds_transport_actor_tree.py` (GH #473 next-step).
> `git diff main..wkt/uds_macos_473 -- tractor/ipc/_uds.py`
- `get_stream_addrs()`: document the autobind semantics
(bytes = linux abstract-ns autobind artifact), add
no-autobind `(str, str)` arm picking the non-empty name
(`peername` connect-side, `sockname` accept-side) with
an empty-pair `ValueError` guard.
- `start_listener()`: `bs.mkdir(parents=True,
exist_ok=True)`.
> `git diff main..wkt/uds_macos_473 -- .github/workflows/ci.yml`
- remove the `macos-latest`+`uds` matrix exclude so
UDS-on-macOS is exercised by CI (GH #473 next-step).
> `git diff main..wkt/uds_macos_473 -- examples/uds_transport_actor_tree.py`
- docs nit: peer-pid mechanism is `SO_PEERCRED` on linux,
`LOCAL_PEERPID` on macOS.
## Verification (verbatim summary)
- macOS-shape sim (no autobind) + fix: example rc=0 with
correct listener sockpath.
- native linux post-fix: example rc=0, autobind arms
unchanged.
- sabotage smoke-test: diagnostic patch surfaces the full
EG traceback incl. collapse-note line.
- `pytest tests/ipc/ tests/test_2way.py tests/discovery/
--tpt-proto uds`: 61 passed, 1 `TooSlowError` flake
(test_simple_rpc) that passes solo in 2.2s — attributed
to a concurrent full-suite run (other session) loading
the host.
- full `tests/test_docs_examples.py` run pending at
raw-file write time (queued behind the concurrent
session's suite).

View File

@ -0,0 +1,83 @@
---
model: claude-fable-5
service: claude
session: b6b42e23-0454-4d9b-be1c-0a89d65aed1b
timestamp: 2026-07-02T15:56:26Z
git_ref: 65bf9df5
scope: code
substantive: true
raw_file: 20260702T155626Z_65bf9df5_prompt_io.raw.md
---
## Prompt
From `ai/prompt-io/prompts/issue_470.md`:
> attempt to resolve
> https://github.com/goodboy/tractor/issues/470
> do it with /open-wkt, ensure you /run-tests on all
> changes
(plus a hard pause-for-human-review deadline of
12:50PM EST.)
Issue #470 asks to trim the ~0.42s `import tractor`
cost — which dominates per-actor spawn latency on the
`trio` backend — by lazy-importing heavy/optional deps
(`pdbp`, `stackscope`, `multiaddr`, + audit of
`colorlog`/`bidict`/`wrapt`/`setproctitle`).
## Response summary
Profiling showed the issue's dep-list only accounted
for ~20ms; the dominant cost (~244ms) was
`log.get_logger()`'s `get_caller_mod()` calling
`inspect.stack()` at module level in ~39 modules —
each call walks every stack frame (deep during nested
imports) and scans `sys.modules` per frame via
`inspect.getmodule()`.
Changes, in impact order:
1. `get_caller_mod()` -> `sys._getframe()` +
`f_globals['__name__']` `sys.modules` lookup
(~240ms saved).
2. Issue's lazy-import checklist: `bidict`,
`multiaddr`, `colorlog`, `wrapt` moved to
`TYPE_CHECKING`/function-local imports;
`platformdirs` function-local; `asyncio` +
`.to_asyncio` deferred out of the `devx.debug` +
`spawn._entry` eager paths (~15ms saved).
3. PEP 562 `__getattr__` on `tractor/__init__.py`
preserving public `tractor.to_asyncio` attr access.
Results: `import tractor` 0.42s -> ~0.145s (~65%);
sequential `start_actor` latency 0.40-0.44s ->
~0.179s/actor. `pdbp` (needs `_repl.py` class-base
restructure) + `platformdirs` (needs
`UDSAddress.def_bindspace` protocol rework) documented
as follow-ups.
## Files changed
- `tractor/log.py``get_caller_mod()` perf fix +
lazy `colorlog`
- `tractor/__init__.py` — PEP 562 lazy `to_asyncio`
- `tractor/discovery/_addr.py``bidict` ->
`TYPE_CHECKING`
- `tractor/discovery/_multiaddr.py` — lazy `multiaddr`
- `tractor/ipc/_tcp.py`, `tractor/ipc/_uds.py`
`Multiaddr` -> `TYPE_CHECKING`
- `tractor/runtime/_state.py` — lazy `platformdirs`
- `tractor/devx/_frame_stack.py` — lazy `pdbp` +
`wrapt`
- `tractor/devx/debug/_trace.py`,
`tractor/devx/debug/_tty_lock.py` — lazy `asyncio` +
`.to_asyncio`
- `tractor/spawn/_entry.py` — lazy
`run_as_asyncio_guest`
## Human edits
None yet — pending user review at the 12:50PM EST
pause gate (test-suite results reported in-session).

View File

@ -0,0 +1,122 @@
---
model: claude-fable-5
service: claude
timestamp: 2026-07-02T15:56:26Z
git_ref: 65bf9df5
diff_cmd: git diff main..wkt/boot_latency_470
---
# Raw AI output — gh #470 `import tractor` latency trim
All generated code is committed on the
`wkt/boot_latency_470` branch; per diff-ref mode each
file's content is referenced via its diff instead of
copied verbatim.
## Profiling findings (verbatim analysis output)
Baseline: `import tractor` ~0.39-0.42s wall.
`python -X importtime` + `cProfile` traced the cost NOT
primarily to third-party deps (the issue's hypothesis)
but to `tractor/log.py:get_logger()` calling
`get_caller_mod()` -> `inspect.stack()` at module level
in ~39 tractor modules:
- `inspect.stack()` builds `FrameInfo` (incl. src-file
and line-context resolution) for EVERY frame on the
stack; during nested imports the stack is dozens of
importlib frames deep.
- each `FrameInfo` resolution calls
`inspect.getmodule()` which scans all of
`sys.modules` per frame (1.4M `ismodule()` calls in
one profiled import).
- aggregate: ~244ms of tractor-own module "self" time
vs ~20ms for ALL the issue-listed third-party deps
(`pdbp` ~10ms, `bidict` ~4.5ms, `multiaddr` ~3.5ms,
`wrapt`/`colorlog` ~1ms each); `trio` itself is
~70-100ms and unavoidable.
## Generated changes
> `git diff main..wkt/boot_latency_470 -- tractor/log.py`
`get_caller_mod()` rewritten from `inspect.stack()` +
`inspect.getmodule()` to `sys._getframe(frames_up)` +
`frame.f_globals['__name__']` -> `sys.modules` lookup
(O(1) vs O(stack x sys.modules)). Unused `inspect`
imports dropped; `FrameType` imported from `types`.
Also `colorlog` lazy-imported inside
`get_console_log()`.
> `git diff main..wkt/boot_latency_470 -- tractor/discovery/_addr.py`
`bidict` import moved under `TYPE_CHECKING`
(annotation-only use; `_address_types` is a plain dict
literal).
> `git diff main..wkt/boot_latency_470 -- tractor/discovery/_multiaddr.py`
`from __future__ import annotations` added; `multiaddr`
import moved under `TYPE_CHECKING` + function-local
imports in `mk_maddr()`/`parse_maddr()`.
> `git diff main..wkt/boot_latency_470 -- tractor/ipc/_tcp.py tractor/ipc/_uds.py`
`Multiaddr` imports moved under `TYPE_CHECKING`
(annotation-only in both transports).
> `git diff main..wkt/boot_latency_470 -- tractor/runtime/_state.py`
`platformdirs` lazy-imported inside `get_rt_dir()`
(NOTE: still imported eagerly via
`UDSAddress.def_bindspace` class-var eval; see
follow-ups).
> `git diff main..wkt/boot_latency_470 -- tractor/devx/_frame_stack.py`
`pdbp` + `wrapt` lazy-imported inside
`hide_runtime_frames()` / `api_frame()` respectively.
> `git diff main..wkt/boot_latency_470 -- tractor/devx/debug/_trace.py tractor/devx/debug/_tty_lock.py`
`asyncio` moved to `TYPE_CHECKING` + call-site local
imports (`asyncio.current_task()` sites);
`tractor.to_asyncio.run_trio_task_in_future` imports
moved into the infected-aio runtime branches.
> `git diff main..wkt/boot_latency_470 -- tractor/spawn/_entry.py`
`run_as_asyncio_guest` import moved into the
`infect_asyncio=True` branches of `_mp_main()` /
`_trio_main()`.
> `git diff main..wkt/boot_latency_470 -- tractor/__init__.py`
PEP 562 module `__getattr__` added so
`tractor.to_asyncio` attr-access still works (required
by `tests/test_child_manages_service_nursery.py` and
any downstream user) while keeping `asyncio` off the
eager import path.
## Measured results (verbatim)
- `import tractor`: 0.39-0.42s -> ~0.145s (~65% cut)
- `start_actor` spawn+boot+reg+cancel: ~0.40-0.44s ->
~0.179s/actor (n=5 sequential, warm parent)
- post-change eager-module check: only `pdbp` +
`platformdirs` of the issue's list remain eager.
## Known follow-ups (not implemented, deadline-bound)
- `pdbp` (~10ms): still eager via
`devx/debug/_repl.py` class bases
(`class PdbREPL(pdbp.Pdb)`) + `_tty_lock.py`
module-level `@pdbp.hideframe`; needs `_repl`
restructure + PEP 562 in `devx.debug.__init__`.
- `platformdirs` (~1.5ms): eager via
`UDSAddress.def_bindspace: ClassVar = get_rt_dir()`
class-body call; needs `Address`-protocol rework of
`def_bindspace` to a lazy accessor.
- `stackscope` + `setproctitle`: already lazy/absent —
no change needed.

View File

@ -23,18 +23,10 @@ async def endpoint(
await trio.sleep_forever() await trio.sleep_forever()
async def spawn_and_open_ep( async def open_ep(
an: tractor.ActorNursery, ptl: tractor.Portal,
i: int, i: int,
) -> None: ) -> None:
'''
Spawn a subactor, start a remote `endpoint()`-task in it.
'''
ptl: tractor.Portal = await an.start_actor(
name=f'worker_{i}',
enable_modules=[__name__],
)
ctx: tractor.Context ctx: tractor.Context
async with ptl.open_context(endpoint) as ( async with ptl.open_context(endpoint) as (
ctx, ctx,
@ -47,7 +39,33 @@ async def spawn_and_open_ep(
await ctx.wait_for_result() await ctx.wait_for_result()
async def main(): async def spawn_and_open_ep(
an: tractor.ActorNursery,
i: int,
maybe_ptl: tractor.Portal|None = None,
) -> None:
'''
Spawn a subactor, start a remote `endpoint()`-task in it.
'''
if maybe_ptl is None:
maybe_ptl: tractor.Portal = await an.start_actor(
name=f'worker_{i}',
enable_modules=[__name__],
)
await open_ep(
ptl=maybe_ptl,
i=i,
)
async def main(
# spawn subs concurrently (in bg `trio.Task`s) so each
# actor's cold `import tractor` (~0.4s, see #470) overlaps
# instead of stacking; once forkserver (#463) lands, spawn
# is cheap enough to just loop sequentially.
spawn_subs_in_bg_tasks: bool = True,
):
''' '''
Spawn a subactor-per-CPU then self-destruct the cluster. Spawn a subactor-per-CPU then self-destruct the cluster.
@ -60,17 +78,21 @@ async def main():
# https://github.com/goodboy/tractor/pull/463 # https://github.com/goodboy/tractor/pull/463
# start_method='main_thread_forkserver', # start_method='main_thread_forkserver',
) as an, ) as an,
# spawn subs concurrently (in bg `trio.Task`s) so each
# actor's cold `import tractor` (~0.4s, see #470) overlaps
# instead of stacking; once forkserver (#463) lands, spawn
# is cheap enough to just loop sequentially.
trio.open_nursery() as tn, trio.open_nursery() as tn,
): ):
for i in range(cpu_count()): for i in range(cpu_count()):
maybe_ptl: tractor.Portal|None = None
if not spawn_subs_in_bg_tasks:
maybe_ptl: tractor.Portal = await an.start_actor(
name=f'worker_{i}',
enable_modules=[__name__],
)
tn.start_soon( tn.start_soon(
spawn_and_open_ep, spawn_and_open_ep,
an, an,
i, i,
maybe_ptl,
) )
destruct_in: int = 2 destruct_in: int = 2
print( print(

View File

@ -6,7 +6,8 @@ subactor inherits the preference.
Every channel address is a filesystem socket path (no TCP port Every channel address is a filesystem socket path (no TCP port
in sight!) and, as a kernel-provided bonus, the peer's pid is in sight!) and, as a kernel-provided bonus, the peer's pid is
exchanged for free via `SO_PEERCRED`. exchanged for free via `SO_PEERCRED` on linux,
`LOCAL_PEERPID` on macOS.
''' '''
import os import os
@ -42,7 +43,7 @@ async def main() -> None:
# (named for the root registrar) this channel rode in # (named for the root registrar) this channel rode in
# on, NOT a per-child path; the child-specific identity # on, NOT a per-child path; the child-specific identity
# we get for free is the kernel-reported peer pid (via # we get for free is the kernel-reported peer pid (via
# `SO_PEERCRED`). # `SO_PEERCRED` on linux, `LOCAL_PEERPID` on macOS).
print( print(
f'portal chan tpt proto: {raddr.proto_key!r}\n' f'portal chan tpt proto: {raddr.proto_key!r}\n'
f'listener sock file: {raddr.sockpath}\n' f'listener sock file: {raddr.sockpath}\n'

View File

@ -145,21 +145,6 @@ def test_example(
'This test does run just fine "in person" however..' 'This test does run just fine "in person" however..'
) )
if (
'uds_transport_actor_tree' in ex_file
and
_friggin_macos
and
ci_env
):
pytest.skip(
'UDS-transport example reliably fails on macOS CI.\n'
'UDS-on-macOS is otherwise un-exercised by the matrix\n'
'(no `tpt_proto=uds` macOS job), so this new example is\n'
'the first to surface it; the macOS UDS path needs\n'
'root-causing. Passes on Linux.'
)
from .conftest import cpu_perf_headroom from .conftest import cpu_perf_headroom
timeout: float = ( timeout: float = (
@ -178,10 +163,11 @@ def test_example(
code = ex.read() code = ex.read()
with run_example_in_subproc(code) as proc: with run_example_in_subproc(code) as proc:
out = None
err = None err = None
try: try:
if not proc.poll(): if not proc.poll():
_, err = proc.communicate(timeout=timeout) out, err = proc.communicate(timeout=timeout)
except subprocess.TimeoutExpired as e: except subprocess.TimeoutExpired as e:
test_log.exception( test_log.exception(
@ -190,9 +176,34 @@ def test_example(
proc.kill() proc.kill()
err = e.stderr err = e.stderr
errmsg: str = err.decode() if err else ''
# XXX, ALWAYS surface the subproc's full stderr
# whenever it exits non-zero!
#
# The prior impl only raised when the LAST stderr
# line contained 'Error', swallowing any crash whose
# traceback ends in a non-`XxxError:` line; in
# particular EVERY `tractor` root-actor crash ends
# with the strict-EG collapse note,
# '( ^^^ this exc was collapsed from a group ^^^ )',
# so ALL such failures were reduced to a bare
# `assert 1 == 0` in CI logs.. see GH #473.
rc: int|None = proc.returncode
if rc:
outmsg: str = out.decode() if out else ''
raise Exception(
f'Example script exited with rc={rc} !?\n'
f'\n'
f'stdout:\n'
f'{outmsg}\n'
f'\n'
f'stderr:\n'
f'{errmsg}\n'
)
# if we get some gnarly output let's aggregate and raise # if we get some gnarly output let's aggregate and raise
if err: if errmsg:
errmsg = err.decode()
errlines = errmsg.splitlines() errlines = errmsg.splitlines()
last_error = errlines[-1] last_error = errlines[-1]
if ( if (

View File

@ -0,0 +1,160 @@
'''
Regression tests for the cold package import surface.
'''
import json
import os
from statistics import median
import subprocess
import sys
from typing import (
Any,
get_type_hints,
)
from tractor.discovery import (
_addr,
_multiaddr,
)
from tractor.ipc import (
_tcp,
_uds,
)
def run_cold_import(code: str) -> dict[str, object]:
result = subprocess.run(
[
sys.executable,
'-c',
code,
],
check=True,
capture_output=True,
text=True,
)
return json.loads(result.stdout)
def test_lazy_to_asyncio_package_api():
'''
Keep the public lazy submodule discoverable without eagerly
importing it.
Before the lazy conversion, package import side effects exposed
`to_asyncio` to `dir()` and wildcard imports. Exercise those APIs
in cold interpreters so this test proves normal `import tractor`
leaves `asyncio` unloaded, while discovery and wildcard access
still advertise and resolve the public submodule.
'''
cold = run_cold_import(
'import json, sys, tractor; '
'print(json.dumps({'
'"advertised": "to_asyncio" in dir(tractor), '
'"asyncio_loaded": "asyncio" in sys.modules}))'
)
assert cold == {
'advertised': True,
'asyncio_loaded': False,
}
wildcard = run_cold_import(
'import json; '
'from tractor import *; '
'print(json.dumps({'
'"module": to_asyncio.__name__}))'
)
assert wildcard == {
'module': 'tractor.to_asyncio',
}
def test_cold_import_budget():
'''
Keep cold package import below the pre-optimization regression.
The original `inspect.stack()` caller lookup made a fresh
`import tractor` take about 0.42s and dominate actor startup.
Run seven independent interpreters and gate their median at a
deliberately broad 0.35s: over twice the measured ~0.145s
baseline, but low enough to catch restoration of that hot path.
Taking the median absorbs process-start and shared-runner noise.
The child measures only its import, rather than parent-side
process creation. `TRACTOR_IMPORT_BUDGET_S` provides an explicit,
reviewable override for platforms that establish a different
baseline instead of silently weakening the project default.
Each child also reports the modules whose eager loading this PR
intentionally removes, proving a timing pass cannot hide a
dependency-import regression.
'''
budget_s = float(
os.environ.get(
'TRACTOR_IMPORT_BUDGET_S',
'0.35',
)
)
optional_mods = (
'asyncio',
'bidict',
'colorlog',
'multiaddr',
'wrapt',
)
code = (
'import json, sys, time; '
'started = time.perf_counter(); '
'import tractor; '
'elapsed = time.perf_counter() - started; '
f'optional = {optional_mods!r}; '
'print(json.dumps({'
'"elapsed": elapsed, '
'"loaded": [name for name in optional '
'if name in sys.modules]}))'
)
samples = [
run_cold_import(code)
for _ in range(7)
]
elapsed = [
float(sample['elapsed'])
for sample in samples
]
loaded = {
name
for sample in samples
for name in sample['loaded']
}
assert not loaded
assert median(elapsed) < budget_s, (
f'cold import median exceeded {budget_s:.3f}s budget: '
f'{elapsed!r}'
)
def test_lazy_annotation_names_resolve():
'''
Resolve annotations without importing optional dependencies.
Moving annotation-only third-party names under `TYPE_CHECKING`
left their runtime globals undefined, causing
`typing.get_type_hints()` to raise `NameError`. Resolve every
affected API and prove the lazy aliases retain import-free runtime
introspection.
'''
assert get_type_hints(_multiaddr.mk_maddr)['return'] is Any
assert get_type_hints(_tcp.MsgpackTCPStream.maddr.fget)[
'return'
] is Any
assert get_type_hints(_uds.MsgpackUDSStream.maddr.fget)[
'return'
] == Any|str
assert get_type_hints(_addr.Address.get_random)[
'current_actor'
] is Any
assert _addr.__annotations__['_address_types'].startswith('dict')

View File

@ -2,9 +2,11 @@
`tractor.log`-wrapping unit tests. `tractor.log`-wrapping unit tests.
''' '''
import importlib
import logging import logging
from pathlib import Path from pathlib import Path
import shutil import shutil
import sys
from types import ModuleType from types import ModuleType
import pytest import pytest
@ -165,6 +167,53 @@ def test_implicit_mod_name_applied_for_child(
assert submod.log.logger in sub_logs assert submod.log.logger in sub_logs
def test_implicit_mod_name_from_unregistered_namespace(
tmp_path: Path,
):
'''
Preserve implicit logger naming for dynamic module namespaces.
The fast `sys.modules` caller lookup cannot resolve `runpy`,
plugin-loader, or `exec()` namespaces that are not registered.
Compile a real package file under an unregistered module name so
the rare filename fallback must recover its imported package and
retain the same package-level logger name.
'''
pkg_name = 'dynamic_logger_pkg'
pkg_dir = tmp_path / pkg_name
pkg_dir.mkdir()
init_path = pkg_dir / '__init__.py'
init_path.write_text('')
mod_path = pkg_dir / 'plugin.py'
mod_path.write_text('')
sys.path.insert(0, str(tmp_path))
try:
importlib.import_module(pkg_name)
namespace = {
'__name__': f'{pkg_name}.unregistered',
'__package__': pkg_name,
'tractor': tractor,
}
exec(
compile(
'log = tractor.log.get_logger('
f'pkg_name={pkg_name!r})',
str(mod_path),
'exec',
),
namespace,
)
dynamic_log = namespace.get('log')
finally:
sys.path.remove(str(tmp_path))
sys.modules.pop(pkg_name, None)
assert dynamic_log is not None
assert dynamic_log.name == pkg_name
def test_io_custom_level_registered(): def test_io_custom_level_registered():
''' '''
The `IO`(21) level (registered via `add_log_level()` at The `IO`(21) level (registered via `add_log_level()` at

View File

@ -74,3 +74,39 @@ from .discovery._registry import (
Arbiter as Arbiter, Arbiter as Arbiter,
) )
# from . import hilevel as hilevel # from . import hilevel as hilevel
__all__: tuple[str, ...] = tuple(
name
for name in globals()
if not name.startswith('_')
) + (
'to_asyncio',
)
def __dir__() -> list[str]:
return sorted(set(globals()) | set(__all__))
def __getattr__(name: str):
'''
PEP 562 lazy sub-module loading, presently only for
`.to_asyncio` which (transitively) imports `asyncio`
itself: a non-trivial multi-ms chunk of the eager
`import tractor` cost (gh #470) unneeded by
`trio`-only apps.
Any `tractor.to_asyncio.<attr>` access (or a
`from tractor import to_asyncio`) still works, the
sub-mod is simply imported on first-access instead
of at pkg-import time.
'''
if name == 'to_asyncio':
from importlib import import_module
return import_module('.to_asyncio', __name__)
raise AttributeError(
f'module {__name__!r} has no attribute {name!r}'
)

View File

@ -39,14 +39,15 @@ from typing import (
Type, Type,
) )
import pdbp # NOTE, `pdbp` + `wrapt` are lazy-imported at their
# single use-sites below to keep them off the eager
# `import tractor` path (gh #470).
from tractor.log import get_logger from tractor.log import get_logger
import trio import trio
from tractor.msg import ( from tractor.msg import (
pretty_struct, pretty_struct,
NamespacePath, NamespacePath,
) )
import wrapt
log = get_logger() log = get_logger()
@ -257,6 +258,7 @@ def api_frame(
caller_frames_up: int = 1, caller_frames_up: int = 1,
) -> Callable: ) -> Callable:
import wrapt
# handle the decorator called WITHOUT () case, # handle the decorator called WITHOUT () case,
# i.e. just @api_frame, NOT @api_frame(extra=<blah>) # i.e. just @api_frame, NOT @api_frame(extra=<blah>)
@ -320,6 +322,8 @@ def hide_runtime_frames() -> dict[FunctionType, CodeType]:
as possible, particularly from inside a `PdbREPL`. as possible, particularly from inside a `PdbREPL`.
''' '''
import pdbp
# XXX HACKZONE XXX # XXX HACKZONE XXX
# hide exit stack frames on nurseries and cancel-scopes! # hide exit stack frames on nurseries and cancel-scopes!
# |_ so avoid seeing it when the `pdbp` REPL is first engaged from # |_ so avoid seeing it when the `pdbp` REPL is first engaged from

View File

@ -24,7 +24,6 @@ mult-process support within a single actor tree.
''' '''
from __future__ import annotations from __future__ import annotations
import asyncio
import bdb import bdb
from contextlib import ( from contextlib import (
AbstractContextManager, AbstractContextManager,
@ -53,7 +52,6 @@ from trio import (
) )
import tractor import tractor
from tractor.log import get_logger from tractor.log import get_logger
from tractor.to_asyncio import run_trio_task_in_future
from tractor._context import Context from tractor._context import Context
from tractor.runtime import _state from tractor.runtime import _state
from tractor._exceptions import ( from tractor._exceptions import (
@ -85,6 +83,10 @@ from ..pformat import (
) )
if TYPE_CHECKING: if TYPE_CHECKING:
# NOTE, `asyncio` (and `.to_asyncio`) are
# lazy-imported at their use-sites to keep them off
# the eager `import tractor` path (gh #470).
import asyncio
from trio.lowlevel import Task from trio.lowlevel import Task
from threading import Thread from threading import Thread
from tractor.runtime._runtime import ( from tractor.runtime._runtime import (
@ -164,6 +166,7 @@ async def _pause(
'An `asyncio` task should not be calling this!?' 'An `asyncio` task should not be calling this!?'
) from rte ) from rte
else: else:
import asyncio
task = asyncio.current_task() task = asyncio.current_task()
if debug_func is not None: if debug_func is not None:
@ -946,6 +949,7 @@ def pause_from_sync(
asyncio_task: asyncio.Task|None = None asyncio_task: asyncio.Task|None = None
if is_infected_aio: if is_infected_aio:
import asyncio
asyncio_task = asyncio.current_task() asyncio_task = asyncio.current_task()
# TODO: we could also check for a non-`.to_thread` context # TODO: we could also check for a non-`.to_thread` context
@ -1059,6 +1063,9 @@ def pause_from_sync(
greenback: ModuleType = maybe_import_greenback() greenback: ModuleType = maybe_import_greenback()
if greenback.has_portal(): if greenback.has_portal():
from tractor.to_asyncio import (
run_trio_task_in_future,
)
DebugStatus.shield_sigint() DebugStatus.shield_sigint()
fute: asyncio.Future = run_trio_task_in_future( fute: asyncio.Future = run_trio_task_in_future(
partial( partial(

View File

@ -20,7 +20,6 @@ Root-actor TTY mutex-locking machinery.
''' '''
from __future__ import annotations from __future__ import annotations
import asyncio
from contextlib import ( from contextlib import (
AbstractContextManager, AbstractContextManager,
asynccontextmanager as acm, asynccontextmanager as acm,
@ -52,7 +51,6 @@ from trio import (
TaskStatus, TaskStatus,
) )
import tractor import tractor
from tractor.to_asyncio import run_trio_task_in_future
from tractor.log import get_logger from tractor.log import get_logger
from tractor._context import Context from tractor._context import Context
from tractor.runtime import _state from tractor.runtime import _state
@ -66,6 +64,10 @@ from tractor.runtime._state import (
) )
if TYPE_CHECKING: if TYPE_CHECKING:
# NOTE, `asyncio` (and `.to_asyncio`) are
# lazy-imported at their use-sites to keep them off
# the eager `import tractor` path (gh #470).
import asyncio
from trio.lowlevel import Task from trio.lowlevel import Task
from threading import Thread from threading import Thread
from tractor.ipc import ( from tractor.ipc import (
@ -910,6 +912,9 @@ class DebugStatus:
async def _set_repl_release(): async def _set_repl_release():
repl_release.set() repl_release.set()
from tractor.to_asyncio import (
run_trio_task_in_future,
)
fute: asyncio.Future = run_trio_task_in_future( fute: asyncio.Future = run_trio_task_in_future(
_set_repl_release _set_repl_release
) )

View File

@ -16,13 +16,13 @@
from __future__ import annotations from __future__ import annotations
from uuid import uuid4 from uuid import uuid4
from typing import ( from typing import (
Any,
Protocol, Protocol,
ClassVar, ClassVar,
Type, Type,
TYPE_CHECKING, TYPE_CHECKING,
) )
from bidict import bidict
from trio import ( from trio import (
SocketListener, SocketListener,
) )
@ -35,7 +35,11 @@ from ..ipc._tcp import TCPAddress
from ..ipc._uds import UDSAddress from ..ipc._uds import UDSAddress
if TYPE_CHECKING: if TYPE_CHECKING:
# ONLY type-annots, the eager import costs ~4.5ms
# of `import tractor` wall-time (gh #470).
from ..runtime._runtime import Actor from ..runtime._runtime import Actor
else:
Actor = Any
log = get_logger() log = get_logger()
@ -170,7 +174,7 @@ class Address(Protocol):
... ...
_address_types: bidict[str, Type[Address]] = { _address_types: dict[str, Type[Address]] = {
'tcp': TCPAddress, 'tcp': TCPAddress,
'uds': UDSAddress 'uds': UDSAddress
} }

View File

@ -24,14 +24,23 @@ Multiaddress support using the upstream `py-multiaddr` lib
- https://github.com/multiformats/multiaddr/blob/master/protocols/unix.md - https://github.com/multiformats/multiaddr/blob/master/protocols/unix.md
''' '''
from __future__ import annotations
import ipaddress import ipaddress
from pathlib import Path from pathlib import Path
from typing import TYPE_CHECKING from typing import (
Any,
from multiaddr import Multiaddr TYPE_CHECKING,
)
if TYPE_CHECKING: if TYPE_CHECKING:
# NOTE, `multiaddr` is lazy-imported at first use
# (in the fns below) to keep it off the eager
# `import tractor` path (gh #470).
from multiaddr import Multiaddr
from tractor.discovery._addr import Address from tractor.discovery._addr import Address
else:
Multiaddr = Any
Address = Any
# map from tractor-internal `proto_key` identifiers # map from tractor-internal `proto_key` identifiers
# to the standard multiaddr protocol name strings. # to the standard multiaddr protocol name strings.
@ -56,6 +65,8 @@ def mk_maddr(
multiaddr-spec-compliant protocol path. multiaddr-spec-compliant protocol path.
''' '''
from multiaddr import Multiaddr
proto_key: str = addr.proto_key proto_key: str = addr.proto_key
maddr_proto: str|None = _tpt_proto_to_maddr.get(proto_key) maddr_proto: str|None = _tpt_proto_to_maddr.get(proto_key)
if maddr_proto is None: if maddr_proto is None:
@ -98,6 +109,7 @@ def parse_maddr(
''' '''
# lazy imports to avoid circular deps # lazy imports to avoid circular deps
from multiaddr import Multiaddr
from tractor.ipc._tcp import TCPAddress from tractor.ipc._tcp import TCPAddress
from tractor.ipc._uds import UDSAddress from tractor.ipc._uds import UDSAddress

View File

@ -20,7 +20,9 @@ TCP implementation of tractor.ipc._transport.MsgTransport protocol
from __future__ import annotations from __future__ import annotations
import ipaddress import ipaddress
from typing import ( from typing import (
Any,
ClassVar, ClassVar,
TYPE_CHECKING,
) )
# from contextlib import ( # from contextlib import (
# asynccontextmanager as acm, # asynccontextmanager as acm,
@ -33,7 +35,6 @@ from trio import (
open_tcp_listeners, open_tcp_listeners,
) )
from multiaddr import Multiaddr
from tractor.msg import MsgCodec from tractor.msg import MsgCodec
from tractor.log import get_logger from tractor.log import get_logger
from tractor.discovery._multiaddr import mk_maddr from tractor.discovery._multiaddr import mk_maddr
@ -42,6 +43,13 @@ from tractor.ipc._transport import (
MsgpackTransport, MsgpackTransport,
) )
if TYPE_CHECKING:
# ONLY type-annots, the eager import costs
# `import tractor` wall-time (gh #470).
from multiaddr import Multiaddr
else:
Multiaddr = Any
log = get_logger() log = get_logger()

View File

@ -32,6 +32,7 @@ from socket import (
) )
import struct import struct
from typing import ( from typing import (
Any,
Type, Type,
TYPE_CHECKING, TYPE_CHECKING,
ClassVar, ClassVar,
@ -49,7 +50,6 @@ from trio._highlevel_open_unix_stream import (
has_unix, has_unix,
) )
from multiaddr import Multiaddr
from tractor.msg import MsgCodec from tractor.msg import MsgCodec
from tractor.log import get_logger from tractor.log import get_logger
from tractor.discovery._multiaddr import mk_maddr from tractor.discovery._multiaddr import mk_maddr
@ -63,7 +63,13 @@ from tractor.runtime._state import (
) )
if TYPE_CHECKING: if TYPE_CHECKING:
# ONLY type-annots, the eager import costs
# `import tractor` wall-time (gh #470).
from multiaddr import Multiaddr
from tractor.runtime._runtime import Actor from tractor.runtime._runtime import Actor
else:
Multiaddr = Any
Actor = Any
# Platform-specific credential passing constants # Platform-specific credential passing constants
@ -301,7 +307,16 @@ async def start_listener(
f'>{{\n' f'>{{\n'
f'|_{bs!r}\n' f'|_{bs!r}\n'
) )
bs.mkdir() bs.mkdir(
# ensure the full ancestor tree for any nested
# (custom `filedir`) bindspace; the default
# `get_rt_dir()` space is pre-created but a custom
# one may have missing parents.
parents=True,
# avoid `FileExistsError` from racing actors, same
# guard as in `get_rt_dir()`.
exist_ok=True,
)
with _reraise_as_connerr( with _reraise_as_connerr(
src_excs=( src_excs=(
@ -554,11 +569,18 @@ class MsgpackUDSStream(MsgpackTransport):
]: ]:
sock: trio.socket.socket = stream.socket sock: trio.socket.socket = stream.socket
# NOTE XXX, it's unclear why one or the other ends up being # NOTE, the `bytes` case is a linux-only artifact: setting
# `bytes` versus the socket-file-path, i presume it's # `SO_PASSCRED` (see `open_unix_socket_w_passcred()`)
# something to do with who is the server (called `.listen()`)? # causes the kernel to *autobind* the un-named client
# maybe could be better implemented using another info-query # sock to an abstract-namespace addr which python
# on the socket like, # delivers as `bytes`; the listener-bound end is always
# the fs-path `str`. On platforms WITHOUT autobind
# (macOS et al) the un-bound end instead reports as an
# empty `str` so BOTH names arrive as `str`s and the
# real fs-path is whichever is non-empty: `peername` on
# the connect side, `sockname` on the accept side.
#
# for socket-api deats see,
# https://beej.us/guide/bgnet/html/split-wide/system-calls-or-bust.html#gethostnamewho-am-i # https://beej.us/guide/bgnet/html/split-wide/system-calls-or-bust.html#gethostnamewho-am-i
sockname: str|bytes = sock.getsockname() sockname: str|bytes = sock.getsockname()
# https://beej.us/guide/bgnet/html/split-wide/system-calls-or-bust.html#getpeernamewho-are-you # https://beej.us/guide/bgnet/html/split-wide/system-calls-or-bust.html#getpeernamewho-are-you
@ -570,8 +592,24 @@ class MsgpackUDSStream(MsgpackTransport):
case (bytes(), str()): case (bytes(), str()):
sock_path: Path = Path(sockname) sock_path: Path = Path(sockname)
case (str(), str()): # XXX, likely macOS # XXX, no-autobind case (macOS): the un-bound end
sock_path: Path = Path(peername) # is `''`, NOT a `bytes` abstract-ns addr; taking
# `peername` unconditionally (as prior impl did)
# delivers garbage `Path('')` addrs on the accept
# side!
case (str(), str()):
bound_name: str = (
peername
or
sockname
)
if not bound_name:
raise ValueError(
f'Empty UDS (peername, sockname) pair ??\n'
f'peername: {peername!r}\n'
f'sockname: {sockname!r}\n'
)
sock_path: Path = Path(bound_name)
case _: case _:
raise TypeError( raise TypeError(

View File

@ -26,11 +26,6 @@ built on `tractor`.
''' '''
from collections.abc import Mapping from collections.abc import Mapping
from functools import partial from functools import partial
from inspect import (
FrameInfo,
getmodule,
stack,
)
import sys import sys
import logging import logging
from logging import ( from logging import (
@ -38,10 +33,16 @@ from logging import (
Logger, Logger,
StreamHandler, StreamHandler,
) )
from types import ModuleType from types import (
FrameType,
ModuleType,
)
import warnings import warnings
import colorlog # type: ignore # NOTE, `colorlog` is lazy-imported in
# `get_console_log()` to keep it off the eager
# `import tractor` path (gh #470).
#
# ?TODO, some other (modern) alt libs? # ?TODO, some other (modern) alt libs?
# import coloredlogs # import coloredlogs
# import colored_traceback.auto # ?TODO, need better config? # import colored_traceback.auto # ?TODO, need better config?
@ -436,17 +437,41 @@ def get_logger(
pkg_name: str = _root_name pkg_name: str = _root_name
def get_caller_mod( def get_caller_mod(
frames_up:int = 2 frames_up: int = 2,
): ) -> ModuleType|None:
''' '''
Attempt to get the module which called `tractor.get_logger()`. Attempt to get the module which called
`tractor.get_logger()`.
Resolve the caller's frame with `sys._getframe()` and
map its `__name__` through `sys.modules`; `inspect.stack()`
(the previous impl) builds src-file info for EVERY frame
on the stack, scanning all of `sys.modules` per frame via
`inspect.getmodule()`, which made module-level
`get_logger()` calls dominate `import tractor` time
(see gh #470).
''' '''
callstack: list[FrameInfo] = stack() try:
caller_fi: FrameInfo = callstack[frames_up] caller_frame: FrameType = sys._getframe(frames_up)
caller_mod: ModuleType = getmodule(caller_fi.frame) except ValueError:
return None
mod_name: str|None = caller_frame.f_globals.get(
'__name__',
)
if mod_name is None:
return None
if caller_mod := sys.modules.get(mod_name):
return caller_mod return caller_mod
# Preserve caller discovery for `runpy`, plugin loaders,
# and `exec()` namespaces not registered in `sys.modules`.
# Import `inspect` only on this rare fallback path.
from inspect import getmodule
return getmodule(caller_frame)
# --- Auto--naming-CASE --- # --- Auto--naming-CASE ---
# ------------------------- # -------------------------
# Implicitly introspect the caller's module-name whenever `name` # Implicitly introspect the caller's module-name whenever `name`
@ -780,6 +805,10 @@ def get_console_log(
None, None,
) )
): ):
# lazy-imported to keep it off the eager
# `import tractor` path (gh #470).
import colorlog # type: ignore
fmt: str = LOG_FORMAT # always apply our format? fmt: str = LOG_FORMAT # always apply our format?
handler = StreamHandler() handler = StreamHandler()
formatter = colorlog.ColoredFormatter( formatter = colorlog.ColoredFormatter(

View File

@ -30,7 +30,6 @@ from typing import (
TYPE_CHECKING, TYPE_CHECKING,
) )
import platformdirs
from trio.lowlevel import current_task from trio.lowlevel import current_task
from msgspec import ( from msgspec import (
@ -332,6 +331,10 @@ def get_rt_dir(
the lovely `platformdirs` lib. the lovely `platformdirs` lib.
''' '''
# lazy-imported to keep it off the eager
# `import tractor` path (gh #470).
import platformdirs
rt_dir: Path = Path( rt_dir: Path = Path(
platformdirs.user_runtime_dir( platformdirs.user_runtime_dir(
appname=appname, appname=appname,

View File

@ -38,7 +38,11 @@ from ..devx import (
pformat, pformat,
) )
# from ..msg import pretty_struct # from ..msg import pretty_struct
from ..to_asyncio import run_as_asyncio_guest #
# NOTE, `.to_asyncio` (and thus `asyncio` itself) is
# lazy-imported at the `infect_asyncio=True` use-sites
# below to keep it off the eager `import tractor` path
# (gh #470).
from ..discovery._addr import UnwrappedAddress from ..discovery._addr import UnwrappedAddress
from ..runtime._runtime import ( from ..runtime._runtime import (
async_main, async_main,
@ -96,6 +100,7 @@ def _mp_main(
) )
try: try:
if infect_asyncio: if infect_asyncio:
from ..to_asyncio import run_as_asyncio_guest
actor._infected_aio = True actor._infected_aio = True
run_as_asyncio_guest(trio_main) run_as_asyncio_guest(trio_main)
else: else:
@ -156,6 +161,7 @@ def _trio_main(
) )
try: try:
if infect_asyncio: if infect_asyncio:
from ..to_asyncio import run_as_asyncio_guest
actor._infected_aio = True actor._infected_aio = True
run_as_asyncio_guest(trio_main) run_as_asyncio_guest(trio_main)
else: else: