Compare commits
14 Commits
02022d6487
...
451e0acf8a
| Author | SHA1 | Date |
|---|---|---|
|
|
451e0acf8a | |
|
|
3d02a8569e | |
|
|
bceca74eb7 | |
|
|
18faffcff7 | |
|
|
7554f90e59 | |
|
|
3705bbe594 | |
|
|
66ac7863b5 | |
|
|
089e158da9 | |
|
|
7987f6b8d7 | |
|
|
70c7e334a7 | |
|
|
62b729a106 | |
|
|
213a298aad | |
|
|
d1d3fc58bb | |
|
|
a2c8af558e |
|
|
@ -118,11 +118,6 @@ jobs:
|
|||
'tcp',
|
||||
'uds',
|
||||
]
|
||||
# https://github.com/orgs/community/discussions/26253#discussioncomment-3250989
|
||||
exclude:
|
||||
# don't do UDS run on macOS (for now)
|
||||
- os: macos-latest
|
||||
tpt_proto: 'uds'
|
||||
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
|
|
|
|||
|
|
@ -0,0 +1,77 @@
|
|||
---
|
||||
model: claude-fable-5
|
||||
service: claude
|
||||
session: b8ec2ffe-1e47-4db5-8bc3-5bebc5feaaf6
|
||||
timestamp: 2026-07-02T15:50:06Z
|
||||
git_ref: 65bf9df5
|
||||
scope: code
|
||||
substantive: true
|
||||
raw_file: 20260702T155006Z_65bf9df5_prompt_io.raw.md
|
||||
---
|
||||
|
||||
## Prompt
|
||||
|
||||
From `ai/prompt-io/prompts/issue_473.md`:
|
||||
|
||||
> NOTE: you MUST pause this work at 12:50PM EST (BEFORE
|
||||
> your weekly limit reset) for review by a human!
|
||||
>
|
||||
> resolve GH #473: https://github.com/goodboy/tractor/issues/474
|
||||
> do it with /open-wkt.
|
||||
|
||||
(The prompt's issue-number/URL mismatch was resolved in
|
||||
favor of #473 — file name, session name and prompt text
|
||||
all say #473; the `/474` URL is a typo.)
|
||||
|
||||
GH #473: "Root-cause UDS-on-macOS —
|
||||
`uds_transport_actor_tree` example fails on CI".
|
||||
|
||||
## Response summary
|
||||
|
||||
Worked all four #473 next-step checkboxes in wkt
|
||||
`uds_macos_473`:
|
||||
|
||||
1. **stderr surfacing**: `test_docs_examples.py` now
|
||||
raises with the full subproc stderr (+stdout) on ANY
|
||||
non-zero exit. The old last-line-'Error' check could
|
||||
never fire for `tractor` crashes since root-actor
|
||||
stderr always ends with the strict-EG collapse note
|
||||
`( ^^^ this exc was collapsed from a group ^^^ )` —
|
||||
proven against the real PR #460 macOS CI log (bare
|
||||
`assert 1 == 0`, no traceback).
|
||||
2. **root-cause (linux-provable layer)**: macOS-only
|
||||
addr corruption in
|
||||
`MsgpackUDSStream.get_stream_addrs()` — no
|
||||
`SO_PASSCRED`/autobind on darwin means the accept
|
||||
side's `getpeername()` is `''`, and the
|
||||
`(str(), str())` arm took `peername` unconditionally →
|
||||
`Path('')` garbage addrs on every accepted conn.
|
||||
Proven + fixed via linux no-autobind simulation.
|
||||
Possibly not the final macOS crasher (non-fatal on
|
||||
linux-sim); the diagnostic patch guarantees the next
|
||||
macOS CI run shows any remaining layer.
|
||||
3. **CI matrix**: removed the `macos-latest`+`uds`
|
||||
exclude.
|
||||
4. **un-skip**: dropped the macOS+CI skip of the example.
|
||||
|
||||
Also: `start_listener()` bindspace mkdir hardened
|
||||
(`parents=True, exist_ok=True`), example docstring
|
||||
peer-pid mechanism corrected for macOS.
|
||||
|
||||
## Files changed
|
||||
|
||||
- `tests/test_docs_examples.py` — surface full stderr on
|
||||
non-zero exit; remove macOS skip of the UDS example
|
||||
- `tractor/ipc/_uds.py` — fix no-autobind
|
||||
`get_stream_addrs()` arm; harden bindspace mkdir;
|
||||
document autobind semantics
|
||||
- `.github/workflows/ci.yml` — add macOS UDS leg (remove
|
||||
matrix exclude)
|
||||
- `examples/uds_transport_actor_tree.py` — platform-
|
||||
correct peer-pid docs
|
||||
|
||||
## Human edits
|
||||
|
||||
None yet — pending human review (hard-stop review
|
||||
checkpoint at 12:50PM EST per prompt); commit staging and
|
||||
any edits are the human's.
|
||||
|
|
@ -0,0 +1,101 @@
|
|||
---
|
||||
model: claude-fable-5
|
||||
service: claude
|
||||
timestamp: 2026-07-02T15:50:06Z
|
||||
git_ref: 65bf9df5
|
||||
diff_cmd: git diff main..wkt/uds_macos_473
|
||||
---
|
||||
|
||||
# Raw output — GH #473 UDS-on-macOS root-cause session
|
||||
|
||||
NOTE: code output is diff-referenced (not copied) per
|
||||
prompt-io diff-ref mode; all generated code is on branch
|
||||
`wkt/uds_macos_473` relative to `main` (65bf9df5).
|
||||
|
||||
## Diagnostic narrative (verbatim)
|
||||
|
||||
Root-cause work proceeded by linux-side simulation since no
|
||||
macOS host is available:
|
||||
|
||||
1. Pulled the actual macOS CI failure log from PR #460 run
|
||||
28343878933 (job 84115107992): the failure is a bare
|
||||
`assert 1 == 0` on `proc.returncode` with NO stderr
|
||||
surfaced — the harness only re-raised when the LAST
|
||||
stderr line contained 'Error'.
|
||||
|
||||
2. Reproduced the stderr-swallow mechanism on linux: any
|
||||
`tractor` root-actor crash ends stderr with the
|
||||
strict-EG collapse note
|
||||
`( ^^^ this exc was collapsed from a group ^^^ )` which
|
||||
never matches 'Error' — so EVERY possible crash was
|
||||
swallowed. (Verified by sabotaging the runtime dir via
|
||||
an over-long `XDG_RUNTIME_DIR` → `OSError: AF_UNIX path
|
||||
too long` → rc=1 + swallowed.)
|
||||
|
||||
3. Found + proved a macOS-only addr-corruption bug in
|
||||
`MsgpackUDSStream.get_stream_addrs()`: the
|
||||
`(str(), str())` match-arm unconditionally took
|
||||
`peername`, but on no-autobind platforms (macOS lacks
|
||||
linux's `SO_PASSCRED`-triggered autobind) the accept
|
||||
side's `getpeername()` is `''` → `Path('')` garbage
|
||||
laddr/raddr on EVERY accepted UDS conn. Simulated on
|
||||
linux by nulling `SO_PASSCRED` (no autobind → same `''`
|
||||
shape): pre-fix the example printed
|
||||
`listener sock file: .`; post-fix it prints the real
|
||||
registry sockpath. Non-fatal on linux-sim (rc=0), so
|
||||
possibly not the final macOS crasher — the diagnostic
|
||||
patch guarantees the next macOS CI run reveals any
|
||||
remaining layer.
|
||||
|
||||
4. Falsified the missing-parent-dir theory:
|
||||
`get_rt_dir()` already `mkdir(parents=True,
|
||||
exist_ok=True)`s at import (and macOS TCP CI passes),
|
||||
so `~/Library/Caches/TemporaryItems` absence cannot be
|
||||
the crasher. Hardened `start_listener()`'s bindspace
|
||||
mkdir anyway (custom `filedir` case + racing actors).
|
||||
|
||||
## Generated changes (diff pointers)
|
||||
|
||||
> `git diff main..wkt/uds_macos_473 -- tests/test_docs_examples.py`
|
||||
|
||||
- always raise with FULL subproc stderr (+stdout) on any
|
||||
non-zero example exit; keep legacy last-line 'Error'
|
||||
check for zero-rc cases; drop the macOS+CI skip of
|
||||
`uds_transport_actor_tree.py` (GH #473 next-step).
|
||||
|
||||
> `git diff main..wkt/uds_macos_473 -- tractor/ipc/_uds.py`
|
||||
|
||||
- `get_stream_addrs()`: document the autobind semantics
|
||||
(bytes = linux abstract-ns autobind artifact), add
|
||||
no-autobind `(str, str)` arm picking the non-empty name
|
||||
(`peername` connect-side, `sockname` accept-side) with
|
||||
an empty-pair `ValueError` guard.
|
||||
- `start_listener()`: `bs.mkdir(parents=True,
|
||||
exist_ok=True)`.
|
||||
|
||||
> `git diff main..wkt/uds_macos_473 -- .github/workflows/ci.yml`
|
||||
|
||||
- remove the `macos-latest`+`uds` matrix exclude so
|
||||
UDS-on-macOS is exercised by CI (GH #473 next-step).
|
||||
|
||||
> `git diff main..wkt/uds_macos_473 -- examples/uds_transport_actor_tree.py`
|
||||
|
||||
- docs nit: peer-pid mechanism is `SO_PEERCRED` on linux,
|
||||
`LOCAL_PEERPID` on macOS.
|
||||
|
||||
## Verification (verbatim summary)
|
||||
|
||||
- macOS-shape sim (no autobind) + fix: example rc=0 with
|
||||
correct listener sockpath.
|
||||
- native linux post-fix: example rc=0, autobind arms
|
||||
unchanged.
|
||||
- sabotage smoke-test: diagnostic patch surfaces the full
|
||||
EG traceback incl. collapse-note line.
|
||||
- `pytest tests/ipc/ tests/test_2way.py tests/discovery/
|
||||
--tpt-proto uds`: 61 passed, 1 `TooSlowError` flake
|
||||
(test_simple_rpc) that passes solo in 2.2s — attributed
|
||||
to a concurrent full-suite run (other session) loading
|
||||
the host.
|
||||
- full `tests/test_docs_examples.py` run pending at
|
||||
raw-file write time (queued behind the concurrent
|
||||
session's suite).
|
||||
|
|
@ -0,0 +1,83 @@
|
|||
---
|
||||
model: claude-fable-5
|
||||
service: claude
|
||||
session: b6b42e23-0454-4d9b-be1c-0a89d65aed1b
|
||||
timestamp: 2026-07-02T15:56:26Z
|
||||
git_ref: 65bf9df5
|
||||
scope: code
|
||||
substantive: true
|
||||
raw_file: 20260702T155626Z_65bf9df5_prompt_io.raw.md
|
||||
---
|
||||
|
||||
## Prompt
|
||||
|
||||
From `ai/prompt-io/prompts/issue_470.md`:
|
||||
|
||||
> attempt to resolve
|
||||
> https://github.com/goodboy/tractor/issues/470
|
||||
> do it with /open-wkt, ensure you /run-tests on all
|
||||
> changes
|
||||
|
||||
(plus a hard pause-for-human-review deadline of
|
||||
12:50PM EST.)
|
||||
|
||||
Issue #470 asks to trim the ~0.42s `import tractor`
|
||||
cost — which dominates per-actor spawn latency on the
|
||||
`trio` backend — by lazy-importing heavy/optional deps
|
||||
(`pdbp`, `stackscope`, `multiaddr`, + audit of
|
||||
`colorlog`/`bidict`/`wrapt`/`setproctitle`).
|
||||
|
||||
## Response summary
|
||||
|
||||
Profiling showed the issue's dep-list only accounted
|
||||
for ~20ms; the dominant cost (~244ms) was
|
||||
`log.get_logger()`'s `get_caller_mod()` calling
|
||||
`inspect.stack()` at module level in ~39 modules —
|
||||
each call walks every stack frame (deep during nested
|
||||
imports) and scans `sys.modules` per frame via
|
||||
`inspect.getmodule()`.
|
||||
|
||||
Changes, in impact order:
|
||||
|
||||
1. `get_caller_mod()` -> `sys._getframe()` +
|
||||
`f_globals['__name__']` `sys.modules` lookup
|
||||
(~240ms saved).
|
||||
2. Issue's lazy-import checklist: `bidict`,
|
||||
`multiaddr`, `colorlog`, `wrapt` moved to
|
||||
`TYPE_CHECKING`/function-local imports;
|
||||
`platformdirs` function-local; `asyncio` +
|
||||
`.to_asyncio` deferred out of the `devx.debug` +
|
||||
`spawn._entry` eager paths (~15ms saved).
|
||||
3. PEP 562 `__getattr__` on `tractor/__init__.py`
|
||||
preserving public `tractor.to_asyncio` attr access.
|
||||
|
||||
Results: `import tractor` 0.42s -> ~0.145s (~65%);
|
||||
sequential `start_actor` latency 0.40-0.44s ->
|
||||
~0.179s/actor. `pdbp` (needs `_repl.py` class-base
|
||||
restructure) + `platformdirs` (needs
|
||||
`UDSAddress.def_bindspace` protocol rework) documented
|
||||
as follow-ups.
|
||||
|
||||
## Files changed
|
||||
|
||||
- `tractor/log.py` — `get_caller_mod()` perf fix +
|
||||
lazy `colorlog`
|
||||
- `tractor/__init__.py` — PEP 562 lazy `to_asyncio`
|
||||
- `tractor/discovery/_addr.py` — `bidict` ->
|
||||
`TYPE_CHECKING`
|
||||
- `tractor/discovery/_multiaddr.py` — lazy `multiaddr`
|
||||
- `tractor/ipc/_tcp.py`, `tractor/ipc/_uds.py` —
|
||||
`Multiaddr` -> `TYPE_CHECKING`
|
||||
- `tractor/runtime/_state.py` — lazy `platformdirs`
|
||||
- `tractor/devx/_frame_stack.py` — lazy `pdbp` +
|
||||
`wrapt`
|
||||
- `tractor/devx/debug/_trace.py`,
|
||||
`tractor/devx/debug/_tty_lock.py` — lazy `asyncio` +
|
||||
`.to_asyncio`
|
||||
- `tractor/spawn/_entry.py` — lazy
|
||||
`run_as_asyncio_guest`
|
||||
|
||||
## Human edits
|
||||
|
||||
None yet — pending user review at the 12:50PM EST
|
||||
pause gate (test-suite results reported in-session).
|
||||
|
|
@ -0,0 +1,122 @@
|
|||
---
|
||||
model: claude-fable-5
|
||||
service: claude
|
||||
timestamp: 2026-07-02T15:56:26Z
|
||||
git_ref: 65bf9df5
|
||||
diff_cmd: git diff main..wkt/boot_latency_470
|
||||
---
|
||||
|
||||
# Raw AI output — gh #470 `import tractor` latency trim
|
||||
|
||||
All generated code is committed on the
|
||||
`wkt/boot_latency_470` branch; per diff-ref mode each
|
||||
file's content is referenced via its diff instead of
|
||||
copied verbatim.
|
||||
|
||||
## Profiling findings (verbatim analysis output)
|
||||
|
||||
Baseline: `import tractor` ~0.39-0.42s wall.
|
||||
|
||||
`python -X importtime` + `cProfile` traced the cost NOT
|
||||
primarily to third-party deps (the issue's hypothesis)
|
||||
but to `tractor/log.py:get_logger()` calling
|
||||
`get_caller_mod()` -> `inspect.stack()` at module level
|
||||
in ~39 tractor modules:
|
||||
|
||||
- `inspect.stack()` builds `FrameInfo` (incl. src-file
|
||||
and line-context resolution) for EVERY frame on the
|
||||
stack; during nested imports the stack is dozens of
|
||||
importlib frames deep.
|
||||
- each `FrameInfo` resolution calls
|
||||
`inspect.getmodule()` which scans all of
|
||||
`sys.modules` per frame (1.4M `ismodule()` calls in
|
||||
one profiled import).
|
||||
- aggregate: ~244ms of tractor-own module "self" time
|
||||
vs ~20ms for ALL the issue-listed third-party deps
|
||||
(`pdbp` ~10ms, `bidict` ~4.5ms, `multiaddr` ~3.5ms,
|
||||
`wrapt`/`colorlog` ~1ms each); `trio` itself is
|
||||
~70-100ms and unavoidable.
|
||||
|
||||
## Generated changes
|
||||
|
||||
> `git diff main..wkt/boot_latency_470 -- tractor/log.py`
|
||||
|
||||
`get_caller_mod()` rewritten from `inspect.stack()` +
|
||||
`inspect.getmodule()` to `sys._getframe(frames_up)` +
|
||||
`frame.f_globals['__name__']` -> `sys.modules` lookup
|
||||
(O(1) vs O(stack x sys.modules)). Unused `inspect`
|
||||
imports dropped; `FrameType` imported from `types`.
|
||||
Also `colorlog` lazy-imported inside
|
||||
`get_console_log()`.
|
||||
|
||||
> `git diff main..wkt/boot_latency_470 -- tractor/discovery/_addr.py`
|
||||
|
||||
`bidict` import moved under `TYPE_CHECKING`
|
||||
(annotation-only use; `_address_types` is a plain dict
|
||||
literal).
|
||||
|
||||
> `git diff main..wkt/boot_latency_470 -- tractor/discovery/_multiaddr.py`
|
||||
|
||||
`from __future__ import annotations` added; `multiaddr`
|
||||
import moved under `TYPE_CHECKING` + function-local
|
||||
imports in `mk_maddr()`/`parse_maddr()`.
|
||||
|
||||
> `git diff main..wkt/boot_latency_470 -- tractor/ipc/_tcp.py tractor/ipc/_uds.py`
|
||||
|
||||
`Multiaddr` imports moved under `TYPE_CHECKING`
|
||||
(annotation-only in both transports).
|
||||
|
||||
> `git diff main..wkt/boot_latency_470 -- tractor/runtime/_state.py`
|
||||
|
||||
`platformdirs` lazy-imported inside `get_rt_dir()`
|
||||
(NOTE: still imported eagerly via
|
||||
`UDSAddress.def_bindspace` class-var eval; see
|
||||
follow-ups).
|
||||
|
||||
> `git diff main..wkt/boot_latency_470 -- tractor/devx/_frame_stack.py`
|
||||
|
||||
`pdbp` + `wrapt` lazy-imported inside
|
||||
`hide_runtime_frames()` / `api_frame()` respectively.
|
||||
|
||||
> `git diff main..wkt/boot_latency_470 -- tractor/devx/debug/_trace.py tractor/devx/debug/_tty_lock.py`
|
||||
|
||||
`asyncio` moved to `TYPE_CHECKING` + call-site local
|
||||
imports (`asyncio.current_task()` sites);
|
||||
`tractor.to_asyncio.run_trio_task_in_future` imports
|
||||
moved into the infected-aio runtime branches.
|
||||
|
||||
> `git diff main..wkt/boot_latency_470 -- tractor/spawn/_entry.py`
|
||||
|
||||
`run_as_asyncio_guest` import moved into the
|
||||
`infect_asyncio=True` branches of `_mp_main()` /
|
||||
`_trio_main()`.
|
||||
|
||||
> `git diff main..wkt/boot_latency_470 -- tractor/__init__.py`
|
||||
|
||||
PEP 562 module `__getattr__` added so
|
||||
`tractor.to_asyncio` attr-access still works (required
|
||||
by `tests/test_child_manages_service_nursery.py` and
|
||||
any downstream user) while keeping `asyncio` off the
|
||||
eager import path.
|
||||
|
||||
## Measured results (verbatim)
|
||||
|
||||
- `import tractor`: 0.39-0.42s -> ~0.145s (~65% cut)
|
||||
- `start_actor` spawn+boot+reg+cancel: ~0.40-0.44s ->
|
||||
~0.179s/actor (n=5 sequential, warm parent)
|
||||
- post-change eager-module check: only `pdbp` +
|
||||
`platformdirs` of the issue's list remain eager.
|
||||
|
||||
## Known follow-ups (not implemented, deadline-bound)
|
||||
|
||||
- `pdbp` (~10ms): still eager via
|
||||
`devx/debug/_repl.py` class bases
|
||||
(`class PdbREPL(pdbp.Pdb)`) + `_tty_lock.py`
|
||||
module-level `@pdbp.hideframe`; needs `_repl`
|
||||
restructure + PEP 562 in `devx.debug.__init__`.
|
||||
- `platformdirs` (~1.5ms): eager via
|
||||
`UDSAddress.def_bindspace: ClassVar = get_rt_dir()`
|
||||
class-body call; needs `Address`-protocol rework of
|
||||
`def_bindspace` to a lazy accessor.
|
||||
- `stackscope` + `setproctitle`: already lazy/absent —
|
||||
no change needed.
|
||||
|
|
@ -23,18 +23,10 @@ async def endpoint(
|
|||
await trio.sleep_forever()
|
||||
|
||||
|
||||
async def spawn_and_open_ep(
|
||||
an: tractor.ActorNursery,
|
||||
async def open_ep(
|
||||
ptl: tractor.Portal,
|
||||
i: int,
|
||||
) -> None:
|
||||
'''
|
||||
Spawn a subactor, start a remote `endpoint()`-task in it.
|
||||
|
||||
'''
|
||||
ptl: tractor.Portal = await an.start_actor(
|
||||
name=f'worker_{i}',
|
||||
enable_modules=[__name__],
|
||||
)
|
||||
ctx: tractor.Context
|
||||
async with ptl.open_context(endpoint) as (
|
||||
ctx,
|
||||
|
|
@ -47,7 +39,33 @@ async def spawn_and_open_ep(
|
|||
await ctx.wait_for_result()
|
||||
|
||||
|
||||
async def main():
|
||||
async def spawn_and_open_ep(
|
||||
an: tractor.ActorNursery,
|
||||
i: int,
|
||||
maybe_ptl: tractor.Portal|None = None,
|
||||
) -> None:
|
||||
'''
|
||||
Spawn a subactor, start a remote `endpoint()`-task in it.
|
||||
|
||||
'''
|
||||
if maybe_ptl is None:
|
||||
maybe_ptl: tractor.Portal = await an.start_actor(
|
||||
name=f'worker_{i}',
|
||||
enable_modules=[__name__],
|
||||
)
|
||||
await open_ep(
|
||||
ptl=maybe_ptl,
|
||||
i=i,
|
||||
)
|
||||
|
||||
|
||||
async def main(
|
||||
# spawn subs concurrently (in bg `trio.Task`s) so each
|
||||
# actor's cold `import tractor` (~0.4s, see #470) overlaps
|
||||
# instead of stacking; once forkserver (#463) lands, spawn
|
||||
# is cheap enough to just loop sequentially.
|
||||
spawn_subs_in_bg_tasks: bool = True,
|
||||
):
|
||||
'''
|
||||
Spawn a subactor-per-CPU then self-destruct the cluster.
|
||||
|
||||
|
|
@ -60,17 +78,21 @@ async def main():
|
|||
# https://github.com/goodboy/tractor/pull/463
|
||||
# start_method='main_thread_forkserver',
|
||||
) as an,
|
||||
# spawn subs concurrently (in bg `trio.Task`s) so each
|
||||
# actor's cold `import tractor` (~0.4s, see #470) overlaps
|
||||
# instead of stacking; once forkserver (#463) lands, spawn
|
||||
# is cheap enough to just loop sequentially.
|
||||
trio.open_nursery() as tn,
|
||||
):
|
||||
for i in range(cpu_count()):
|
||||
|
||||
maybe_ptl: tractor.Portal|None = None
|
||||
if not spawn_subs_in_bg_tasks:
|
||||
maybe_ptl: tractor.Portal = await an.start_actor(
|
||||
name=f'worker_{i}',
|
||||
enable_modules=[__name__],
|
||||
)
|
||||
tn.start_soon(
|
||||
spawn_and_open_ep,
|
||||
an,
|
||||
i,
|
||||
maybe_ptl,
|
||||
)
|
||||
destruct_in: int = 2
|
||||
print(
|
||||
|
|
|
|||
|
|
@ -6,7 +6,8 @@ subactor inherits the preference.
|
|||
|
||||
Every channel address is a filesystem socket path (no TCP port
|
||||
in sight!) and, as a kernel-provided bonus, the peer's pid is
|
||||
exchanged for free via `SO_PEERCRED`.
|
||||
exchanged for free via `SO_PEERCRED` on linux,
|
||||
`LOCAL_PEERPID` on macOS.
|
||||
|
||||
'''
|
||||
import os
|
||||
|
|
@ -42,7 +43,7 @@ async def main() -> None:
|
|||
# (named for the root registrar) this channel rode in
|
||||
# on, NOT a per-child path; the child-specific identity
|
||||
# we get for free is the kernel-reported peer pid (via
|
||||
# `SO_PEERCRED`).
|
||||
# `SO_PEERCRED` on linux, `LOCAL_PEERPID` on macOS).
|
||||
print(
|
||||
f'portal chan tpt proto: {raddr.proto_key!r}\n'
|
||||
f'listener sock file: {raddr.sockpath}\n'
|
||||
|
|
|
|||
|
|
@ -145,21 +145,6 @@ def test_example(
|
|||
'This test does run just fine "in person" however..'
|
||||
)
|
||||
|
||||
if (
|
||||
'uds_transport_actor_tree' in ex_file
|
||||
and
|
||||
_friggin_macos
|
||||
and
|
||||
ci_env
|
||||
):
|
||||
pytest.skip(
|
||||
'UDS-transport example reliably fails on macOS CI.\n'
|
||||
'UDS-on-macOS is otherwise un-exercised by the matrix\n'
|
||||
'(no `tpt_proto=uds` macOS job), so this new example is\n'
|
||||
'the first to surface it; the macOS UDS path needs\n'
|
||||
'root-causing. Passes on Linux.'
|
||||
)
|
||||
|
||||
from .conftest import cpu_perf_headroom
|
||||
|
||||
timeout: float = (
|
||||
|
|
@ -178,10 +163,11 @@ def test_example(
|
|||
code = ex.read()
|
||||
|
||||
with run_example_in_subproc(code) as proc:
|
||||
out = None
|
||||
err = None
|
||||
try:
|
||||
if not proc.poll():
|
||||
_, err = proc.communicate(timeout=timeout)
|
||||
out, err = proc.communicate(timeout=timeout)
|
||||
|
||||
except subprocess.TimeoutExpired as e:
|
||||
test_log.exception(
|
||||
|
|
@ -190,9 +176,34 @@ def test_example(
|
|||
proc.kill()
|
||||
err = e.stderr
|
||||
|
||||
errmsg: str = err.decode() if err else ''
|
||||
|
||||
# XXX, ALWAYS surface the subproc's full stderr
|
||||
# whenever it exits non-zero!
|
||||
#
|
||||
# The prior impl only raised when the LAST stderr
|
||||
# line contained 'Error', swallowing any crash whose
|
||||
# traceback ends in a non-`XxxError:` line; in
|
||||
# particular EVERY `tractor` root-actor crash ends
|
||||
# with the strict-EG collapse note,
|
||||
# '( ^^^ this exc was collapsed from a group ^^^ )',
|
||||
# so ALL such failures were reduced to a bare
|
||||
# `assert 1 == 0` in CI logs.. see GH #473.
|
||||
rc: int|None = proc.returncode
|
||||
if rc:
|
||||
outmsg: str = out.decode() if out else ''
|
||||
raise Exception(
|
||||
f'Example script exited with rc={rc} !?\n'
|
||||
f'\n'
|
||||
f'stdout:\n'
|
||||
f'{outmsg}\n'
|
||||
f'\n'
|
||||
f'stderr:\n'
|
||||
f'{errmsg}\n'
|
||||
)
|
||||
|
||||
# if we get some gnarly output let's aggregate and raise
|
||||
if err:
|
||||
errmsg = err.decode()
|
||||
if errmsg:
|
||||
errlines = errmsg.splitlines()
|
||||
last_error = errlines[-1]
|
||||
if (
|
||||
|
|
|
|||
|
|
@ -0,0 +1,160 @@
|
|||
'''
|
||||
Regression tests for the cold package import surface.
|
||||
|
||||
'''
|
||||
import json
|
||||
import os
|
||||
from statistics import median
|
||||
import subprocess
|
||||
import sys
|
||||
from typing import (
|
||||
Any,
|
||||
get_type_hints,
|
||||
)
|
||||
|
||||
from tractor.discovery import (
|
||||
_addr,
|
||||
_multiaddr,
|
||||
)
|
||||
from tractor.ipc import (
|
||||
_tcp,
|
||||
_uds,
|
||||
)
|
||||
|
||||
|
||||
def run_cold_import(code: str) -> dict[str, object]:
|
||||
result = subprocess.run(
|
||||
[
|
||||
sys.executable,
|
||||
'-c',
|
||||
code,
|
||||
],
|
||||
check=True,
|
||||
capture_output=True,
|
||||
text=True,
|
||||
)
|
||||
return json.loads(result.stdout)
|
||||
|
||||
|
||||
def test_lazy_to_asyncio_package_api():
|
||||
'''
|
||||
Keep the public lazy submodule discoverable without eagerly
|
||||
importing it.
|
||||
|
||||
Before the lazy conversion, package import side effects exposed
|
||||
`to_asyncio` to `dir()` and wildcard imports. Exercise those APIs
|
||||
in cold interpreters so this test proves normal `import tractor`
|
||||
leaves `asyncio` unloaded, while discovery and wildcard access
|
||||
still advertise and resolve the public submodule.
|
||||
|
||||
'''
|
||||
cold = run_cold_import(
|
||||
'import json, sys, tractor; '
|
||||
'print(json.dumps({'
|
||||
'"advertised": "to_asyncio" in dir(tractor), '
|
||||
'"asyncio_loaded": "asyncio" in sys.modules}))'
|
||||
)
|
||||
assert cold == {
|
||||
'advertised': True,
|
||||
'asyncio_loaded': False,
|
||||
}
|
||||
|
||||
wildcard = run_cold_import(
|
||||
'import json; '
|
||||
'from tractor import *; '
|
||||
'print(json.dumps({'
|
||||
'"module": to_asyncio.__name__}))'
|
||||
)
|
||||
assert wildcard == {
|
||||
'module': 'tractor.to_asyncio',
|
||||
}
|
||||
|
||||
|
||||
def test_cold_import_budget():
|
||||
'''
|
||||
Keep cold package import below the pre-optimization regression.
|
||||
|
||||
The original `inspect.stack()` caller lookup made a fresh
|
||||
`import tractor` take about 0.42s and dominate actor startup.
|
||||
Run seven independent interpreters and gate their median at a
|
||||
deliberately broad 0.35s: over twice the measured ~0.145s
|
||||
baseline, but low enough to catch restoration of that hot path.
|
||||
|
||||
Taking the median absorbs process-start and shared-runner noise.
|
||||
The child measures only its import, rather than parent-side
|
||||
process creation. `TRACTOR_IMPORT_BUDGET_S` provides an explicit,
|
||||
reviewable override for platforms that establish a different
|
||||
baseline instead of silently weakening the project default.
|
||||
|
||||
Each child also reports the modules whose eager loading this PR
|
||||
intentionally removes, proving a timing pass cannot hide a
|
||||
dependency-import regression.
|
||||
|
||||
'''
|
||||
budget_s = float(
|
||||
os.environ.get(
|
||||
'TRACTOR_IMPORT_BUDGET_S',
|
||||
'0.35',
|
||||
)
|
||||
)
|
||||
optional_mods = (
|
||||
'asyncio',
|
||||
'bidict',
|
||||
'colorlog',
|
||||
'multiaddr',
|
||||
'wrapt',
|
||||
)
|
||||
code = (
|
||||
'import json, sys, time; '
|
||||
'started = time.perf_counter(); '
|
||||
'import tractor; '
|
||||
'elapsed = time.perf_counter() - started; '
|
||||
f'optional = {optional_mods!r}; '
|
||||
'print(json.dumps({'
|
||||
'"elapsed": elapsed, '
|
||||
'"loaded": [name for name in optional '
|
||||
'if name in sys.modules]}))'
|
||||
)
|
||||
samples = [
|
||||
run_cold_import(code)
|
||||
for _ in range(7)
|
||||
]
|
||||
elapsed = [
|
||||
float(sample['elapsed'])
|
||||
for sample in samples
|
||||
]
|
||||
loaded = {
|
||||
name
|
||||
for sample in samples
|
||||
for name in sample['loaded']
|
||||
}
|
||||
|
||||
assert not loaded
|
||||
assert median(elapsed) < budget_s, (
|
||||
f'cold import median exceeded {budget_s:.3f}s budget: '
|
||||
f'{elapsed!r}'
|
||||
)
|
||||
|
||||
|
||||
def test_lazy_annotation_names_resolve():
|
||||
'''
|
||||
Resolve annotations without importing optional dependencies.
|
||||
|
||||
Moving annotation-only third-party names under `TYPE_CHECKING`
|
||||
left their runtime globals undefined, causing
|
||||
`typing.get_type_hints()` to raise `NameError`. Resolve every
|
||||
affected API and prove the lazy aliases retain import-free runtime
|
||||
introspection.
|
||||
|
||||
'''
|
||||
assert get_type_hints(_multiaddr.mk_maddr)['return'] is Any
|
||||
assert get_type_hints(_tcp.MsgpackTCPStream.maddr.fget)[
|
||||
'return'
|
||||
] is Any
|
||||
assert get_type_hints(_uds.MsgpackUDSStream.maddr.fget)[
|
||||
'return'
|
||||
] == Any|str
|
||||
assert get_type_hints(_addr.Address.get_random)[
|
||||
'current_actor'
|
||||
] is Any
|
||||
assert _addr.__annotations__['_address_types'].startswith('dict')
|
||||
|
|
@ -2,9 +2,11 @@
|
|||
`tractor.log`-wrapping unit tests.
|
||||
|
||||
'''
|
||||
import importlib
|
||||
import logging
|
||||
from pathlib import Path
|
||||
import shutil
|
||||
import sys
|
||||
from types import ModuleType
|
||||
|
||||
import pytest
|
||||
|
|
@ -165,6 +167,53 @@ def test_implicit_mod_name_applied_for_child(
|
|||
assert submod.log.logger in sub_logs
|
||||
|
||||
|
||||
def test_implicit_mod_name_from_unregistered_namespace(
|
||||
tmp_path: Path,
|
||||
):
|
||||
'''
|
||||
Preserve implicit logger naming for dynamic module namespaces.
|
||||
|
||||
The fast `sys.modules` caller lookup cannot resolve `runpy`,
|
||||
plugin-loader, or `exec()` namespaces that are not registered.
|
||||
Compile a real package file under an unregistered module name so
|
||||
the rare filename fallback must recover its imported package and
|
||||
retain the same package-level logger name.
|
||||
|
||||
'''
|
||||
pkg_name = 'dynamic_logger_pkg'
|
||||
pkg_dir = tmp_path / pkg_name
|
||||
pkg_dir.mkdir()
|
||||
init_path = pkg_dir / '__init__.py'
|
||||
init_path.write_text('')
|
||||
mod_path = pkg_dir / 'plugin.py'
|
||||
mod_path.write_text('')
|
||||
|
||||
sys.path.insert(0, str(tmp_path))
|
||||
try:
|
||||
importlib.import_module(pkg_name)
|
||||
namespace = {
|
||||
'__name__': f'{pkg_name}.unregistered',
|
||||
'__package__': pkg_name,
|
||||
'tractor': tractor,
|
||||
}
|
||||
exec(
|
||||
compile(
|
||||
'log = tractor.log.get_logger('
|
||||
f'pkg_name={pkg_name!r})',
|
||||
str(mod_path),
|
||||
'exec',
|
||||
),
|
||||
namespace,
|
||||
)
|
||||
dynamic_log = namespace.get('log')
|
||||
finally:
|
||||
sys.path.remove(str(tmp_path))
|
||||
sys.modules.pop(pkg_name, None)
|
||||
|
||||
assert dynamic_log is not None
|
||||
assert dynamic_log.name == pkg_name
|
||||
|
||||
|
||||
def test_io_custom_level_registered():
|
||||
'''
|
||||
The `IO`(21) level (registered via `add_log_level()` at
|
||||
|
|
|
|||
|
|
@ -74,3 +74,39 @@ from .discovery._registry import (
|
|||
Arbiter as Arbiter,
|
||||
)
|
||||
# from . import hilevel as hilevel
|
||||
|
||||
|
||||
__all__: tuple[str, ...] = tuple(
|
||||
name
|
||||
for name in globals()
|
||||
if not name.startswith('_')
|
||||
) + (
|
||||
'to_asyncio',
|
||||
)
|
||||
|
||||
|
||||
def __dir__() -> list[str]:
|
||||
return sorted(set(globals()) | set(__all__))
|
||||
|
||||
|
||||
def __getattr__(name: str):
|
||||
'''
|
||||
PEP 562 lazy sub-module loading, presently only for
|
||||
`.to_asyncio` which (transitively) imports `asyncio`
|
||||
itself: a non-trivial multi-ms chunk of the eager
|
||||
`import tractor` cost (gh #470) unneeded by
|
||||
`trio`-only apps.
|
||||
|
||||
Any `tractor.to_asyncio.<attr>` access (or a
|
||||
`from tractor import to_asyncio`) still works, the
|
||||
sub-mod is simply imported on first-access instead
|
||||
of at pkg-import time.
|
||||
|
||||
'''
|
||||
if name == 'to_asyncio':
|
||||
from importlib import import_module
|
||||
return import_module('.to_asyncio', __name__)
|
||||
|
||||
raise AttributeError(
|
||||
f'module {__name__!r} has no attribute {name!r}'
|
||||
)
|
||||
|
|
|
|||
|
|
@ -39,14 +39,15 @@ from typing import (
|
|||
Type,
|
||||
)
|
||||
|
||||
import pdbp
|
||||
# NOTE, `pdbp` + `wrapt` are lazy-imported at their
|
||||
# single use-sites below to keep them off the eager
|
||||
# `import tractor` path (gh #470).
|
||||
from tractor.log import get_logger
|
||||
import trio
|
||||
from tractor.msg import (
|
||||
pretty_struct,
|
||||
NamespacePath,
|
||||
)
|
||||
import wrapt
|
||||
|
||||
|
||||
log = get_logger()
|
||||
|
|
@ -257,6 +258,7 @@ def api_frame(
|
|||
caller_frames_up: int = 1,
|
||||
|
||||
) -> Callable:
|
||||
import wrapt
|
||||
|
||||
# handle the decorator called WITHOUT () case,
|
||||
# i.e. just @api_frame, NOT @api_frame(extra=<blah>)
|
||||
|
|
@ -320,6 +322,8 @@ def hide_runtime_frames() -> dict[FunctionType, CodeType]:
|
|||
as possible, particularly from inside a `PdbREPL`.
|
||||
|
||||
'''
|
||||
import pdbp
|
||||
|
||||
# XXX HACKZONE XXX
|
||||
# hide exit stack frames on nurseries and cancel-scopes!
|
||||
# |_ so avoid seeing it when the `pdbp` REPL is first engaged from
|
||||
|
|
|
|||
|
|
@ -24,7 +24,6 @@ mult-process support within a single actor tree.
|
|||
|
||||
'''
|
||||
from __future__ import annotations
|
||||
import asyncio
|
||||
import bdb
|
||||
from contextlib import (
|
||||
AbstractContextManager,
|
||||
|
|
@ -53,7 +52,6 @@ from trio import (
|
|||
)
|
||||
import tractor
|
||||
from tractor.log import get_logger
|
||||
from tractor.to_asyncio import run_trio_task_in_future
|
||||
from tractor._context import Context
|
||||
from tractor.runtime import _state
|
||||
from tractor._exceptions import (
|
||||
|
|
@ -85,6 +83,10 @@ from ..pformat import (
|
|||
)
|
||||
|
||||
if TYPE_CHECKING:
|
||||
# NOTE, `asyncio` (and `.to_asyncio`) are
|
||||
# lazy-imported at their use-sites to keep them off
|
||||
# the eager `import tractor` path (gh #470).
|
||||
import asyncio
|
||||
from trio.lowlevel import Task
|
||||
from threading import Thread
|
||||
from tractor.runtime._runtime import (
|
||||
|
|
@ -164,6 +166,7 @@ async def _pause(
|
|||
'An `asyncio` task should not be calling this!?'
|
||||
) from rte
|
||||
else:
|
||||
import asyncio
|
||||
task = asyncio.current_task()
|
||||
|
||||
if debug_func is not None:
|
||||
|
|
@ -946,6 +949,7 @@ def pause_from_sync(
|
|||
|
||||
asyncio_task: asyncio.Task|None = None
|
||||
if is_infected_aio:
|
||||
import asyncio
|
||||
asyncio_task = asyncio.current_task()
|
||||
|
||||
# TODO: we could also check for a non-`.to_thread` context
|
||||
|
|
@ -1059,6 +1063,9 @@ def pause_from_sync(
|
|||
greenback: ModuleType = maybe_import_greenback()
|
||||
|
||||
if greenback.has_portal():
|
||||
from tractor.to_asyncio import (
|
||||
run_trio_task_in_future,
|
||||
)
|
||||
DebugStatus.shield_sigint()
|
||||
fute: asyncio.Future = run_trio_task_in_future(
|
||||
partial(
|
||||
|
|
|
|||
|
|
@ -20,7 +20,6 @@ Root-actor TTY mutex-locking machinery.
|
|||
|
||||
'''
|
||||
from __future__ import annotations
|
||||
import asyncio
|
||||
from contextlib import (
|
||||
AbstractContextManager,
|
||||
asynccontextmanager as acm,
|
||||
|
|
@ -52,7 +51,6 @@ from trio import (
|
|||
TaskStatus,
|
||||
)
|
||||
import tractor
|
||||
from tractor.to_asyncio import run_trio_task_in_future
|
||||
from tractor.log import get_logger
|
||||
from tractor._context import Context
|
||||
from tractor.runtime import _state
|
||||
|
|
@ -66,6 +64,10 @@ from tractor.runtime._state import (
|
|||
)
|
||||
|
||||
if TYPE_CHECKING:
|
||||
# NOTE, `asyncio` (and `.to_asyncio`) are
|
||||
# lazy-imported at their use-sites to keep them off
|
||||
# the eager `import tractor` path (gh #470).
|
||||
import asyncio
|
||||
from trio.lowlevel import Task
|
||||
from threading import Thread
|
||||
from tractor.ipc import (
|
||||
|
|
@ -910,6 +912,9 @@ class DebugStatus:
|
|||
async def _set_repl_release():
|
||||
repl_release.set()
|
||||
|
||||
from tractor.to_asyncio import (
|
||||
run_trio_task_in_future,
|
||||
)
|
||||
fute: asyncio.Future = run_trio_task_in_future(
|
||||
_set_repl_release
|
||||
)
|
||||
|
|
|
|||
|
|
@ -16,13 +16,13 @@
|
|||
from __future__ import annotations
|
||||
from uuid import uuid4
|
||||
from typing import (
|
||||
Any,
|
||||
Protocol,
|
||||
ClassVar,
|
||||
Type,
|
||||
TYPE_CHECKING,
|
||||
)
|
||||
|
||||
from bidict import bidict
|
||||
from trio import (
|
||||
SocketListener,
|
||||
)
|
||||
|
|
@ -35,7 +35,11 @@ from ..ipc._tcp import TCPAddress
|
|||
from ..ipc._uds import UDSAddress
|
||||
|
||||
if TYPE_CHECKING:
|
||||
# ONLY type-annots, the eager import costs ~4.5ms
|
||||
# of `import tractor` wall-time (gh #470).
|
||||
from ..runtime._runtime import Actor
|
||||
else:
|
||||
Actor = Any
|
||||
|
||||
log = get_logger()
|
||||
|
||||
|
|
@ -170,7 +174,7 @@ class Address(Protocol):
|
|||
...
|
||||
|
||||
|
||||
_address_types: bidict[str, Type[Address]] = {
|
||||
_address_types: dict[str, Type[Address]] = {
|
||||
'tcp': TCPAddress,
|
||||
'uds': UDSAddress
|
||||
}
|
||||
|
|
|
|||
|
|
@ -24,14 +24,23 @@ Multiaddress support using the upstream `py-multiaddr` lib
|
|||
- https://github.com/multiformats/multiaddr/blob/master/protocols/unix.md
|
||||
|
||||
'''
|
||||
from __future__ import annotations
|
||||
import ipaddress
|
||||
from pathlib import Path
|
||||
from typing import TYPE_CHECKING
|
||||
|
||||
from multiaddr import Multiaddr
|
||||
from typing import (
|
||||
Any,
|
||||
TYPE_CHECKING,
|
||||
)
|
||||
|
||||
if TYPE_CHECKING:
|
||||
# NOTE, `multiaddr` is lazy-imported at first use
|
||||
# (in the fns below) to keep it off the eager
|
||||
# `import tractor` path (gh #470).
|
||||
from multiaddr import Multiaddr
|
||||
from tractor.discovery._addr import Address
|
||||
else:
|
||||
Multiaddr = Any
|
||||
Address = Any
|
||||
|
||||
# map from tractor-internal `proto_key` identifiers
|
||||
# to the standard multiaddr protocol name strings.
|
||||
|
|
@ -56,6 +65,8 @@ def mk_maddr(
|
|||
multiaddr-spec-compliant protocol path.
|
||||
|
||||
'''
|
||||
from multiaddr import Multiaddr
|
||||
|
||||
proto_key: str = addr.proto_key
|
||||
maddr_proto: str|None = _tpt_proto_to_maddr.get(proto_key)
|
||||
if maddr_proto is None:
|
||||
|
|
@ -98,6 +109,7 @@ def parse_maddr(
|
|||
|
||||
'''
|
||||
# lazy imports to avoid circular deps
|
||||
from multiaddr import Multiaddr
|
||||
from tractor.ipc._tcp import TCPAddress
|
||||
from tractor.ipc._uds import UDSAddress
|
||||
|
||||
|
|
|
|||
|
|
@ -20,7 +20,9 @@ TCP implementation of tractor.ipc._transport.MsgTransport protocol
|
|||
from __future__ import annotations
|
||||
import ipaddress
|
||||
from typing import (
|
||||
Any,
|
||||
ClassVar,
|
||||
TYPE_CHECKING,
|
||||
)
|
||||
# from contextlib import (
|
||||
# asynccontextmanager as acm,
|
||||
|
|
@ -33,7 +35,6 @@ from trio import (
|
|||
open_tcp_listeners,
|
||||
)
|
||||
|
||||
from multiaddr import Multiaddr
|
||||
from tractor.msg import MsgCodec
|
||||
from tractor.log import get_logger
|
||||
from tractor.discovery._multiaddr import mk_maddr
|
||||
|
|
@ -42,6 +43,13 @@ from tractor.ipc._transport import (
|
|||
MsgpackTransport,
|
||||
)
|
||||
|
||||
if TYPE_CHECKING:
|
||||
# ONLY type-annots, the eager import costs
|
||||
# `import tractor` wall-time (gh #470).
|
||||
from multiaddr import Multiaddr
|
||||
else:
|
||||
Multiaddr = Any
|
||||
|
||||
|
||||
log = get_logger()
|
||||
|
||||
|
|
|
|||
|
|
@ -32,6 +32,7 @@ from socket import (
|
|||
)
|
||||
import struct
|
||||
from typing import (
|
||||
Any,
|
||||
Type,
|
||||
TYPE_CHECKING,
|
||||
ClassVar,
|
||||
|
|
@ -49,7 +50,6 @@ from trio._highlevel_open_unix_stream import (
|
|||
has_unix,
|
||||
)
|
||||
|
||||
from multiaddr import Multiaddr
|
||||
from tractor.msg import MsgCodec
|
||||
from tractor.log import get_logger
|
||||
from tractor.discovery._multiaddr import mk_maddr
|
||||
|
|
@ -63,7 +63,13 @@ from tractor.runtime._state import (
|
|||
)
|
||||
|
||||
if TYPE_CHECKING:
|
||||
# ONLY type-annots, the eager import costs
|
||||
# `import tractor` wall-time (gh #470).
|
||||
from multiaddr import Multiaddr
|
||||
from tractor.runtime._runtime import Actor
|
||||
else:
|
||||
Multiaddr = Any
|
||||
Actor = Any
|
||||
|
||||
|
||||
# Platform-specific credential passing constants
|
||||
|
|
@ -301,7 +307,16 @@ async def start_listener(
|
|||
f'>{{\n'
|
||||
f'|_{bs!r}\n'
|
||||
)
|
||||
bs.mkdir()
|
||||
bs.mkdir(
|
||||
# ensure the full ancestor tree for any nested
|
||||
# (custom `filedir`) bindspace; the default
|
||||
# `get_rt_dir()` space is pre-created but a custom
|
||||
# one may have missing parents.
|
||||
parents=True,
|
||||
# avoid `FileExistsError` from racing actors, same
|
||||
# guard as in `get_rt_dir()`.
|
||||
exist_ok=True,
|
||||
)
|
||||
|
||||
with _reraise_as_connerr(
|
||||
src_excs=(
|
||||
|
|
@ -554,11 +569,18 @@ class MsgpackUDSStream(MsgpackTransport):
|
|||
]:
|
||||
sock: trio.socket.socket = stream.socket
|
||||
|
||||
# NOTE XXX, it's unclear why one or the other ends up being
|
||||
# `bytes` versus the socket-file-path, i presume it's
|
||||
# something to do with who is the server (called `.listen()`)?
|
||||
# maybe could be better implemented using another info-query
|
||||
# on the socket like,
|
||||
# NOTE, the `bytes` case is a linux-only artifact: setting
|
||||
# `SO_PASSCRED` (see `open_unix_socket_w_passcred()`)
|
||||
# causes the kernel to *autobind* the un-named client
|
||||
# sock to an abstract-namespace addr which python
|
||||
# delivers as `bytes`; the listener-bound end is always
|
||||
# the fs-path `str`. On platforms WITHOUT autobind
|
||||
# (macOS et al) the un-bound end instead reports as an
|
||||
# empty `str` so BOTH names arrive as `str`s and the
|
||||
# real fs-path is whichever is non-empty: `peername` on
|
||||
# the connect side, `sockname` on the accept side.
|
||||
#
|
||||
# for socket-api deats see,
|
||||
# https://beej.us/guide/bgnet/html/split-wide/system-calls-or-bust.html#gethostnamewho-am-i
|
||||
sockname: str|bytes = sock.getsockname()
|
||||
# https://beej.us/guide/bgnet/html/split-wide/system-calls-or-bust.html#getpeernamewho-are-you
|
||||
|
|
@ -570,8 +592,24 @@ class MsgpackUDSStream(MsgpackTransport):
|
|||
case (bytes(), str()):
|
||||
sock_path: Path = Path(sockname)
|
||||
|
||||
case (str(), str()): # XXX, likely macOS
|
||||
sock_path: Path = Path(peername)
|
||||
# XXX, no-autobind case (macOS): the un-bound end
|
||||
# is `''`, NOT a `bytes` abstract-ns addr; taking
|
||||
# `peername` unconditionally (as prior impl did)
|
||||
# delivers garbage `Path('')` addrs on the accept
|
||||
# side!
|
||||
case (str(), str()):
|
||||
bound_name: str = (
|
||||
peername
|
||||
or
|
||||
sockname
|
||||
)
|
||||
if not bound_name:
|
||||
raise ValueError(
|
||||
f'Empty UDS (peername, sockname) pair ??\n'
|
||||
f'peername: {peername!r}\n'
|
||||
f'sockname: {sockname!r}\n'
|
||||
)
|
||||
sock_path: Path = Path(bound_name)
|
||||
|
||||
case _:
|
||||
raise TypeError(
|
||||
|
|
|
|||
|
|
@ -26,11 +26,6 @@ built on `tractor`.
|
|||
'''
|
||||
from collections.abc import Mapping
|
||||
from functools import partial
|
||||
from inspect import (
|
||||
FrameInfo,
|
||||
getmodule,
|
||||
stack,
|
||||
)
|
||||
import sys
|
||||
import logging
|
||||
from logging import (
|
||||
|
|
@ -38,10 +33,16 @@ from logging import (
|
|||
Logger,
|
||||
StreamHandler,
|
||||
)
|
||||
from types import ModuleType
|
||||
from types import (
|
||||
FrameType,
|
||||
ModuleType,
|
||||
)
|
||||
import warnings
|
||||
|
||||
import colorlog # type: ignore
|
||||
# NOTE, `colorlog` is lazy-imported in
|
||||
# `get_console_log()` to keep it off the eager
|
||||
# `import tractor` path (gh #470).
|
||||
#
|
||||
# ?TODO, some other (modern) alt libs?
|
||||
# import coloredlogs
|
||||
# import colored_traceback.auto # ?TODO, need better config?
|
||||
|
|
@ -436,16 +437,40 @@ def get_logger(
|
|||
pkg_name: str = _root_name
|
||||
|
||||
def get_caller_mod(
|
||||
frames_up:int = 2
|
||||
):
|
||||
frames_up: int = 2,
|
||||
) -> ModuleType|None:
|
||||
'''
|
||||
Attempt to get the module which called `tractor.get_logger()`.
|
||||
Attempt to get the module which called
|
||||
`tractor.get_logger()`.
|
||||
|
||||
Resolve the caller's frame with `sys._getframe()` and
|
||||
map its `__name__` through `sys.modules`; `inspect.stack()`
|
||||
(the previous impl) builds src-file info for EVERY frame
|
||||
on the stack, scanning all of `sys.modules` per frame via
|
||||
`inspect.getmodule()`, which made module-level
|
||||
`get_logger()` calls dominate `import tractor` time
|
||||
(see gh #470).
|
||||
|
||||
'''
|
||||
callstack: list[FrameInfo] = stack()
|
||||
caller_fi: FrameInfo = callstack[frames_up]
|
||||
caller_mod: ModuleType = getmodule(caller_fi.frame)
|
||||
return caller_mod
|
||||
try:
|
||||
caller_frame: FrameType = sys._getframe(frames_up)
|
||||
except ValueError:
|
||||
return None
|
||||
|
||||
mod_name: str|None = caller_frame.f_globals.get(
|
||||
'__name__',
|
||||
)
|
||||
if mod_name is None:
|
||||
return None
|
||||
|
||||
if caller_mod := sys.modules.get(mod_name):
|
||||
return caller_mod
|
||||
|
||||
# Preserve caller discovery for `runpy`, plugin loaders,
|
||||
# and `exec()` namespaces not registered in `sys.modules`.
|
||||
# Import `inspect` only on this rare fallback path.
|
||||
from inspect import getmodule
|
||||
return getmodule(caller_frame)
|
||||
|
||||
# --- Auto--naming-CASE ---
|
||||
# -------------------------
|
||||
|
|
@ -780,6 +805,10 @@ def get_console_log(
|
|||
None,
|
||||
)
|
||||
):
|
||||
# lazy-imported to keep it off the eager
|
||||
# `import tractor` path (gh #470).
|
||||
import colorlog # type: ignore
|
||||
|
||||
fmt: str = LOG_FORMAT # always apply our format?
|
||||
handler = StreamHandler()
|
||||
formatter = colorlog.ColoredFormatter(
|
||||
|
|
|
|||
|
|
@ -30,7 +30,6 @@ from typing import (
|
|||
TYPE_CHECKING,
|
||||
)
|
||||
|
||||
import platformdirs
|
||||
from trio.lowlevel import current_task
|
||||
|
||||
from msgspec import (
|
||||
|
|
@ -332,6 +331,10 @@ def get_rt_dir(
|
|||
the lovely `platformdirs` lib.
|
||||
|
||||
'''
|
||||
# lazy-imported to keep it off the eager
|
||||
# `import tractor` path (gh #470).
|
||||
import platformdirs
|
||||
|
||||
rt_dir: Path = Path(
|
||||
platformdirs.user_runtime_dir(
|
||||
appname=appname,
|
||||
|
|
|
|||
|
|
@ -38,7 +38,11 @@ from ..devx import (
|
|||
pformat,
|
||||
)
|
||||
# from ..msg import pretty_struct
|
||||
from ..to_asyncio import run_as_asyncio_guest
|
||||
#
|
||||
# NOTE, `.to_asyncio` (and thus `asyncio` itself) is
|
||||
# lazy-imported at the `infect_asyncio=True` use-sites
|
||||
# below to keep it off the eager `import tractor` path
|
||||
# (gh #470).
|
||||
from ..discovery._addr import UnwrappedAddress
|
||||
from ..runtime._runtime import (
|
||||
async_main,
|
||||
|
|
@ -96,6 +100,7 @@ def _mp_main(
|
|||
)
|
||||
try:
|
||||
if infect_asyncio:
|
||||
from ..to_asyncio import run_as_asyncio_guest
|
||||
actor._infected_aio = True
|
||||
run_as_asyncio_guest(trio_main)
|
||||
else:
|
||||
|
|
@ -156,6 +161,7 @@ def _trio_main(
|
|||
)
|
||||
try:
|
||||
if infect_asyncio:
|
||||
from ..to_asyncio import run_as_asyncio_guest
|
||||
actor._infected_aio = True
|
||||
run_as_asyncio_guest(trio_main)
|
||||
else:
|
||||
|
|
|
|||
Loading…
Reference in New Issue