Compare commits

..

20 Commits

Author SHA1 Message Date
Gud Boi 3bc6680982 Use `cpu_perf_headroom()` for timing-test deadlines
`cpu_perf_headroom()` is a strict SUPERSET of `cpu_scaling_factor()`
— it folds in static cpu-freq scaling + the slow-CI bump AND the
sustained-load power-cap throttle probe. Point the timing-deadline
call sites at it so they're robust to sustained throttling too (the
gremlin behind mass `trio` deadline-miss flakes),

- `test_legacy_one_way_streaming`: `time_quad_ex` cancel-deadline +
  `test_a_quadruple_example`'s `this_fast` smoke bound.
- `test_cancellation`: `test_cancel_via_SIGINT_other_task` +
  `test_fast_graceful_cancel_*` deadlines.

`cpu_perf_headroom >= cpu_scaling_factor` always, so never less
headroom (CI stays green); `cpu_scaling_factor()` remains the
internal static component.

(this patch was generated in some part by [`claude-code`][claude-code-gh])
[claude-code-gh]: https://github.com/anthropics/claude-code
2026-06-24 20:08:03 -04:00
Gud Boi 23580e9cb8 Add `cpu-perf-check` sustained-throttle gate script
Standalone CLI companion to `cpu_perf_headroom()` (20cb99ec): idle
freq snapshots LIE — every static knob (`governor`, EPP,
`platform_profile`, `scaling_max_freq`) can read "performance"
while a firmware/EC power cap (AMD PPT/STAPM + friends) clamps the
package to ~30% the moment a sustained multi-core load lands,
masquerading as a `trio`-backend deadline-miss "regression" on
byte-identical code.

Deats,
- burns every core for `CPU_PERF_SECS` (default 4s) and samples the
  ACHIEVED `scaling_cur_freq` steady-state (post boost-ramp) vs the
  package max ceiling,
- exits 0 when the sustained fraction clears
  `CPU_PERF_HEALTHY_FRAC` (default 0.45), 1 when throttled — so it
  gates a suite run: `scripts/cpu-perf-check && pytest tests/ ...`,
- prints the static knobs first (to show they all read fine) then
  the remediation list on failure (`platform_profile` bounce, USB-C
  PD replug, `ryzenadj`, reboot) w/ the key reminder: do NOT bump
  test budgets — the box is slow, not the code.

(this patch was generated in some part by [`claude-code`][claude-code-gh])
[claude-code-gh]: https://github.com/anthropics/claude-code
2026-06-24 20:08:03 -04:00
Gud Boi 2a7c46add2 Add `cpu_perf_headroom()` for throttle-aware deadlines
Mass `trio` deadline-miss failures on byte-identical code turned
out to be a firmware/EC power-cap (AMD PPT/STAPM) clamping the
all-core sustained clock while every static knob (`governor`,
`scaling_max_freq`, EPP, platform-profile) still read "performance"
— invisible to the existing `cpu_scaling_factor()` check. See
`scripts/cpu-perf-check` + the
`ai/conc-anal/trio_033_cancel_cascade_slowdown_depth3_issue.md`
notes.

Deats,
- add `_measure_sustained_headroom()` to `tests/conftest.py`: a
  one-shot ~0.9s all-core burn (explicit `fork`-ctx `mp` procs)
  sampling achieved-vs-max freq AFTER the boost window; under a 0.6
  gate it returns the full inverse fraction (capped 4x), else 1.0;
  best-effort 1.0 on non-linux or any error,
- add `cpu_perf_headroom()`: `max()` of the static scaling factor
  and the (session-cached) sustained probe,
- inflate deadline budgets by it in `test_dynamic_pub_sub`, both
  `test_clustering` cases, the
  `test_multi_nested_subactors_error_through_nurseries` pexpect
  waits + `test_nested_multierrors`,
- `xfail(strict=False)` `test_nested_multierrors` depth=3 under
  throttle: the deep tree trips tractor's INTERNAL reap deadlines
  (`soft_kill`/`hard_kill` `terminate_after=1.6`) minting a
  `Cancelled` inside the runtime — not fixable by test-budget
  inflation; auto-clears once the box un-throttles.

(this patch was generated in some part by [`claude-code`][claude-code-gh])
[claude-code-gh]: https://github.com/anthropics/claude-code
2026-06-24 20:08:03 -04:00
Gud Boi 6764251e6b Clarify eg-wrap contract + tighten `stdout` typing
Doc/type accuracy in `supervise_run_process()`:

- the docstring claimed the rc!=0 `CalledProcessError` had
  "no nursery-eg-wrapped CPE to catch/`collapse_eg`", but it
  IS raised as a task into the parent `tn`, so under
  `trio>=0.33` it surfaces `ExceptionGroup`-wrapped like any
  `tn.start()`-task raise. Say so + point at `except*` /
  `collapse_eg()` (matching the inline rc-check comment).
- widen `stdout: int` to `int|None` — the param accepts an
  explicit `None` (inherit) per its own docstring.
- note that stdout is NOT captured, so the CPE note carries
  only stderr (a cmd logging errors to stdout should
  `relay_stdout=True`).

Review: PR #465 (copilot-pull-request-reviewer)
https://github.com/goodboy/tractor/pull/465#pullrequestreview-4548191641

(this patch was generated in some part by [`claude-code`][claude-code-gh])
[claude-code-gh]: https://github.com/anthropics/claude-code
2026-06-24 19:09:45 -04:00
Gud Boi 02cb789901 Bound retained `stderr` to a tail + de-dup decode
`_relay_stream_lines()` kept ALL captured `stderr` for the
rc!=0 `CalledProcessError` note, so a long-lived `check=True`
child could buffer its entire stderr history in parent mem.
Add an `accum_cap` (the new `_STDERR_NOTE_TAIL_BYTES`=16KiB)
that trims the buffer to its TAIL — the bytes nearest the
failure, which are what the note wants anyway.

While here, factor the per-line decode/`rstrip` into a
`_decode_line()` helper so the in-loop + trailing-flush
emits no longer duplicate it.

Review: PR #465 (copilot-pull-request-reviewer)
https://github.com/goodboy/tractor/pull/465#pullrequestreview-4548191641

(this patch was generated in some part by [`claude-code`][claude-code-gh])
[claude-code-gh]: https://github.com/anthropics/claude-code
2026-06-24 19:08:19 -04:00
Gud Boi a3f8765264 Resolve relay `emit` only when relaying
`supervise_run_process()` bound the relay `emit` method
(`getattr(log, relay_level)`) unconditionally at entry, so a
bad/typo'd `relay_level` raised `AttributeError` even for a
call that requested NO relay (the common silent path). Bind
it lazily behind `relay_stdout or relay_stderr` so only an
actual relay request validates the level.

Review: PR #465 (copilot-pull-request-reviewer)
https://github.com/goodboy/tractor/pull/465#pullrequestreview-4548191641

(this patch was generated in some part by [`claude-code`][claude-code-gh])
[claude-code-gh]: https://github.com/anthropics/claude-code
2026-06-24 19:05:55 -04:00
Gud Boi e076296cda Guard `stdout=PIPE`/`capture_*` footguns
Two `supervise_run_process()` caller-kwarg footguns that
previously failed obscurely now raise a clear, early
`ValueError`:

- a bare `stdout=subprocess.PIPE` override spawned NO drain
  reader (only `relay_stdout` does), so a child emitting
  >64KiB would deadlock on a full ~64KiB pipe buffer.
- `trio`'s `capture_stdout`/`capture_stderr` alias our
  MANAGED `stdout`/`stderr` keys; forwarding them made
  `trio.run_process` raise an opaque "can't specify both"
  error, so reject them up-front pointing at `relay_*` /
  the `stdout=` override.

Review: PR #465 (copilot-pull-request-reviewer)
https://github.com/goodboy/tractor/pull/465#pullrequestreview-4548191641

(this patch was generated in some part by [`claude-code`][claude-code-gh])
[claude-code-gh]: https://github.com/anthropics/claude-code
2026-06-24 19:04:35 -04:00
Gud Boi 7f6782b9be Fix `test_simple_context` for `trio>=0.33` groups
Under `trio>=0.33`'s strict exception-groups a lone
`error_parent` arrives wrapped in a 1-exc `ExceptionGroup`
instead of collapsing to the bare exc, so it skips the
`except error_parent` arm and lands in the group arm — where
`is_multi_cancelled()` is (correctly) `False`, tripping the
assert. Also accept `beg.subgroup(error_parent) is not None`.

Review: PR #465 (copilot + self CI-triage)
https://github.com/goodboy/tractor/pull/465#pullrequestreview-4548191641

(this patch was generated in some part by [`claude-code`][claude-code-gh])
[claude-code-gh]: https://github.com/anthropics/claude-code
2026-06-24 18:19:16 -04:00
Gud Boi 35414a42a9 Scale `test_most_beautiful_word` deadline for CI
Flat `trio.fail_after(1)` wraps a whole actor spawn + IPC
round-trip in `test_most_beautiful_word` — sub-second on a
warm box, but slow/noisy CI runners (esp. macOS) blow the
1s deadline with a `trio.TooSlowError`. Scale the budget by
`cpu_scaling_factor()`: a strict `1s` locally (factor
`1.0`), CI/CPU-throttle headroom (~3x) on macOS.

Review: PR #465 (copilot + self CI-triage)
https://github.com/goodboy/tractor/pull/465#pullrequestreview-4548191641

(this patch was generated in some part by [`claude-code`][claude-code-gh])
[claude-code-gh]: https://github.com/anthropics/claude-code
2026-06-24 18:19:16 -04:00
Gud Boi 8fcb9eb7d6 Skip `test_parent_tty_isolated` off-Linux (`/proc`-only) 2026-06-24 18:19:16 -04:00
Gud Boi 14329ec1d6 Strip ANSI + accept `_create(...)` in devx tests
Two version-compat fixes for the `devx` debugger
test-suite, both about matching upstream output that
got more verbose w/ recent lib releases.

ANSI stripping (`tests/devx/conftest.py`),
- Add `ansi_strip(text)` helper + `_ansi_re` pattern
  (regex per https://stackoverflow.com/a/14693789).
- Apply inside `in_prompt_msg()` + `assert_before()` so
  substring matches against REPL/traceback output stay
  robust to color leakage.
- Motivated by py3.13's colored tracebacks +
  `pdbp`/pygments highlighting leaking ANSI even when
  `PYTHON_COLORS=0` is set in the `spawn` fixture (not
  every renderer in the spawned subproc honors it).
- Replaces the longstanding inline TODO that linked
  the SO answer w/o impl'ing.

trio 0.30+ `Cancelled._create(` match (`test_debugger`),
- In `test_shield_pause` swap the two
  `"raise Cancelled._create()"` assertion patterns →
  `"raise Cancelled._create("` (open-paren form, no
  closing).
- trio >=0.30 raises a multi-line
  `raise Cancelled._create(source=.., reason=..,
  source_task=..)` w/ cancel-reason metadata, so the
  legacy bare-`()` form no longer matches. Inline
  comment documents the trio-version pivot.

(this commit msg was generated in some part by [`claude-code`][claude-code-gh])
[claude-code-gh]: https://github.com/anthropics/claude-code

(cherry picked from commit 3854cf5ecb)
(cherry picked from commit c07cf2546b)
2026-06-24 18:19:16 -04:00
Gud Boi 008824d650 Add `supervise_run_process` to `trionics._subproc`
A `trio.Nursery.start()`-style wrapper around
`trio.run_process()` that surfaces rc!=0 errors
deterministically, ALWAYS isolates the parent
controlling-tty, and optionally live-relays the child's
std-streams to `log.<level>` per-line. Suits both
short-lived test-runners + long-lived daemons.

`supervise_run_process()`,
- Deterministic rc!=0: pass `check=False` to `trio`
  and do our OWN post-drain rc-check from the
  supervisor coro body AFTER `own_tn.__aexit__` — NOT
  inside the internal nursery, since that would
  race-cancel the still-draining relay reader and lose
  stderr lines. (Re)build + raise a BARE
  `subprocess.CalledProcessError`: `.stderr=` for
  programmatic callers + an `add_note()`'d
  `|_.stderr:` block for human teardown logs. No
  nursery-eg-wrapped CPE to `collapse_eg` around.
- Parent controlling-tty isolation: `stdin=DEVNULL`
  always, `stdout=DEVNULL` unless relayed/overridden
  (via `stdout=` kwarg w/ `_UNSET` sentinel so explicit
  `None` = inherit still works). Prevents a spawned
  program from clobbering the launching tty's scrollback
  w/ control-seqs.
- Live per-line relay: `relay_stdout=True`/
  `relay_stderr=True` → relayed to `log.<relay_level>`
  (default `'io'`, our custom level 21). Picked to sort
  just above stdlib `INFO`=20 so it shows at usual
  `info`/`devx` levels yet stays separately filterable;
  `runtime`=15 was REJECTED as a default since it'd be
  silently filtered at usual verbosity — footgun for
  daemon supervisors whose whole point is visibility.
  STREAMED, not buffered-until-exit.
- Non-blocking `tn.start()` semantics: live
  `trio.Process` handed up via
  `task_status.started()` immediately (else
  `tn.start()` would block till child exit, losing
  the long-lived-daemon use case). Supervise/relay bg
  tasks run to completion in this coro.
- `**run_process_kwargs` forwarded verbatim (env, shell,
  cwd, start_new_session, executable, ...); MANAGED keys
  (`stdin`/`stdout`/`stderr`/`check`) win on conflict.
- Crash-handling layer intentionally NOT baked in —
  compose `maybe_open_crash_handler()` ON TOP at the
  call-site.

`_relay_stream_lines()` helper,
- Concurrent pipe-drain reader. MANDATORY whenever piping
  w/o `capture_*` since nothing else drains the OS pipe —
  child blocks on `write()` once kernel buf (~64KiB) fills
  → deadlock.
- Modes (combine freely): `emit`-only live relay,
  `accum`-only silent drain+capture (for the CPE note),
  or both. Per-line splitting handles cross-chunk
  residuals + flushes any trailing un-newline-term'd line
  at EOF.

`_add_stderr_note()` helper,
- Attaches an indented `|_.stderr:` note to a CPE via
  `add_note()` for legible rc!=0 reporting at teardown.

Tests (`tests/trionics/test_subproc.py`),
- Hermetic `trio`-only (no actor-runtime).
- `test_stdout_relayed_per_line`: per-line stdout relay.
- `test_parent_tty_isolated`: child fd1 is OUR pipe (no
  `/dev/pts/*`), fd0 pinned to `/dev/null`.
- `test_no_deadlock_on_big_unnewlined_output`: 200KiB
  no-newline output completes under `fail_after(2)` —
  exercises the concurrent drain (without it, the child
  blocks at ~64KiB).
- `test_stderr_relay_and_cpe_rebuild`: rc!=0 w/
  `relay_stderr=True` → bare `CalledProcessError` w/ the
  `.stderr` note + per-line live relay.
- `test_nonrelay_cpe_note`: rc!=0 w/o relay → same
  deterministic post-drain CPE w/ `.stderr` note (silent
  drain+capture path).

Re-export `supervise_run_process` from `tractor.trionics`.

Prompt-IO: ai/prompt-io/claude/20260601T231429Z_0e3e008b_prompt_io.md

(this patch was generated in some part by [`claude-code`][claude-code-gh])
[claude-code-gh]: https://github.com/anthropics/claude-code

(cherry picked from commit f595acc76c)
(cherry picked from commit 6df9ee11bc)
2026-06-24 18:19:16 -04:00
Gud Boi 6255c17db9 Use `is not None` check for peer-connect `event`
Matches the explicit `dict.pop(uid, None)` contract one
line above; same semantics as the prior truthy check.

(this commit msg was generated in some part by [`claude-code`][claude-code-gh])
[claude-code-gh]: https://github.com/anthropics/claude-code

(cherry picked from commit 0e3e008b0c)
(cherry picked from commit 13ed668512)
2026-06-24 18:19:16 -04:00
Bd 6a1149d401
Merge pull request #467 from goodboy/custom_log_levels_api
Add `add_log_level()` custom log-level API
2026-06-24 18:18:37 -04:00
Gud Boi 88cc68edd3 Fix `add_log_level()` re-registration value drift
The bound `StackLevelAdapter.<name>` emit method captured its
level via a `_level: int = value` default-arg at bind time, so
a later `add_log_level(name, <new-value>)` re-registration left
the method emitting at the *stale* original level. Read
`CUSTOM_LEVELS[name_up]` at call time instead — the method binds
once but always tracks the current registered value.

Also,
- correct the `add_log_level()` docstring's "Idempotent" note
  to describe the call-time value lookup (was the misleading
  "no-op-ish refresh (won't clobber an already-bound method)").
- sync stale `_reap.py` doc/comment markers `tractor[` ->
  `_subactor[` to match the actual `_proctitle._def_prefix`
  proc-title prefix (doc-only drift; the code markers already
  referenced `_proctitle._def_prefix`).

Review: PR #467 (copilot-pull-request-reviewer)
https://github.com/goodboy/tractor/pull/467#pullrequestreview-4562945515

(this patch was generated in some part by [`claude-code`][claude-code-gh])
[claude-code-gh]: https://github.com/anthropics/claude-code
2026-06-24 18:03:28 -04:00
Gud Boi 3727ce9e6f Bump lockfile for `xonsh>=0.23.8` release 2026-06-24 16:20:50 -04:00
Gud Boi 90c316f90c Code-style, couple newline/ws tweaks
(cherry picked from commit 8526985c97)
(cherry picked from commit 0952b33a9e)
2026-06-24 16:20:50 -04:00
Gud Boi 6db796ed83 Pin to latest `xonsh` release
(cherry picked from commit c4cad921b9)
(cherry picked from commit ade15b4204)
2026-06-24 16:20:50 -04:00
Gud Boi eb7ef3f0bf Hoist proc-title prefix to `_def_prefix` const
Make the sub-actor proc-title prefix a single
authoritative constant (`_proctitle._def_prefix`) so
the reap-recognition markers and `xontrib` banner pick
it up automatically — one place to flip the prefix
shape going fwd.

Deats,
- `_proctitle._def_prefix: str = '_subactor'`. New
  module-level const consumed by everything that needs
  to know the prefix.
- `set_actor_proctitle(actor, prefix=_def_prefix)`:
  takes an explicit `prefix` arg (default = the const)
  so callers can override per-spawn if they want.
- Default proc-title format:
  `'tractor[<reprol>]'` → `f'{prefix}[<reprol>]'`
  i.e. `_subactor[<reprol>]` by default.
- `_testing/_reap.py`: cmdline + comm markers source
  the prefix from `_proctitle._def_prefix` instead of
  the hardcoded `'tractor['`. So
  `_is_tractor_subactor()` tracks the const
  automatically.
- `xontrib/tractor_diag.xsh`: `acli.reap` orphan-mode
  banner now interpolates the
  `_TRACTOR_PROC_CMDLINE_MARKERS` tuple directly so
  the human-readable mode line stays in sync if the
  prefix shape changes again.

(this commit msg was generated in some part by [`claude-code`][claude-code-gh])
[claude-code-gh]: https://github.com/anthropics/claude-code

(cherry picked from commit 3a45dbd503)
(cherry picked from commit fd8d39c0ce)
2026-06-24 16:20:50 -04:00
Gud Boi bb0b2ffa3c Add `add_log_level()` factory + register `IO`=21
Follow-up to f595acc7 (`supervise_run_process`) which
called `log.io(...)` for std-stream relay assuming an
`IO=21` level existed. Add the registration via a new
factory + tests covering both the factory and the new
level.

`add_log_level()` factory,
- One call wires the four (otherwise hand-synced) pieces:
  - `CUSTOM_LEVELS[NAME]` — drives the `stacklevel` bump
    in `StackLevelAdapter.log()` + `get_logger()`'s
    per-level audit.
  - `logging.addLevelName()` — stdlib name registration.
  - `STD_PALETTE[NAME]` + `BOLD_PALETTE['bold'][NAME]` —
    color entries consumed by `get_console_log()`'s
    `ColoredFormatter` build.
  - Same-named (lowercase) emit method bound on
    `StackLevelAdapter` so `log.<name>('msg')` works +
    `get_logger()`'s per-level method audit passes.
- Idempotent: re-registering an existing name is a
  no-op-ish refresh that won't clobber an already-bound
  method.
- Method binding uses a default-arg `_level=value` so
  the level int is captured (not late-bound across
  multiple registrations).

`IO=21` level (first user),
- Purple. Used by `tractor.trionics._subproc`'s
  std-stream relay (see f595acc7).
- Value 21 picked to sit just ABOVE stdlib `INFO`=20 so
  it's SHOWN BY DEFAULT at usual `info`/`devx` console
  levels — a `runtime`=15 relay would be silently
  filtered (footgun for daemon supervisors whose whole
  point is visibility). Still distinctly labeled +
  filterable.

Tests (`tests/test_log_sys.py`),
- `test_io_custom_level_registered`: validates the IO
  level is fully wired (`CUSTOM_LEVELS`, `addLevelName`,
  both palettes, `StackLevelAdapter.io()` callable);
  emits a record + sanity-asserts `21 >= INFO(20)`.
- `test_add_log_level_pluggable`: registers a fresh
  `XLVL=19` (cyan) via `add_log_level()`, asserts all
  four wires + the bound `xlog.xlvl()` emit, then
  try/finally cleans up the module-global mutations so
  later `get_logger()` audits don't trip on a
  half-removed level.

(this patch was generated in some part by [`claude-code`][claude-code-gh])
[claude-code-gh]: https://github.com/anthropics/claude-code

(cherry picked from commit 7bd7dd50c7)
(cherry picked from commit 93558fe3c9)
2026-06-24 16:20:50 -04:00
6 changed files with 116 additions and 33 deletions

View File

@ -263,9 +263,18 @@ def test_simple_context(
except error_parent: except error_parent:
pass pass
except BaseExceptionGroup as beg: except BaseExceptionGroup as beg:
# XXX: on windows it seems we may have to expect the group error # XXX: on windows — and under `trio>=0.33`'s strict
# exception-groups, where a lone `error_parent` now
# arrives wrapped in a 1-exc `ExceptionGroup` rather than
# collapsed to the bare exc — accept either the
# `error_parent` nested in the group OR a cancel-only
# (multi-cancelled) group.
from tractor.trionics import is_multi_cancelled from tractor.trionics import is_multi_cancelled
assert is_multi_cancelled(beg) assert (
is_multi_cancelled(beg)
or
beg.subgroup(error_parent) is not None
)
else: else:
trio.run(main) trio.run(main)

View File

@ -150,7 +150,12 @@ async def test_most_beautiful_word(
The main ``tractor`` routine. The main ``tractor`` routine.
''' '''
with trio.fail_after(1): # actor spawn + IPC round-trip is comfortably sub-second on a
# warm box, but slow/noisy CI runners (esp. macOS) blow a flat
# 1s deadline. Scale for CI/CPU-throttle headroom — `== 1s`
# locally where `cpu_scaling_factor()` is `1.0`.
from .conftest import cpu_scaling_factor
with trio.fail_after(1 * cpu_scaling_factor()):
async with tractor.open_nursery( async with tractor.open_nursery(
debug_mode=debug_mode, debug_mode=debug_mode,
) as an: ) as an:

View File

@ -14,6 +14,7 @@ Hermetic `trio`-only coverage (no actor-runtime needed):
''' '''
from functools import partial from functools import partial
import platform
import subprocess import subprocess
import pytest import pytest
@ -69,6 +70,10 @@ def test_stdout_relayed_per_line(monkeypatch):
assert any('line=3' in r for r in out_lines) assert any('line=3' in r for r in out_lines)
@pytest.mark.skipif(
platform.system() != 'Linux',
reason='reads `/proc/self/fd` — Linux-only',
)
def test_parent_tty_isolated(monkeypatch): def test_parent_tty_isolated(monkeypatch):
records = _capture_relay(monkeypatch) records = _capture_relay(monkeypatch)

View File

@ -216,18 +216,18 @@ def _read_comm(pid: int) -> str:
# regardless of cwd / venv path / launch context. Used by # regardless of cwd / venv path / launch context. Used by
# `_is_tractor_subactor()` below. # `_is_tractor_subactor()` below.
# #
# - cmdline `tractor[`: matches the `setproctitle`-set form # - cmdline `_subactor[` (`_proctitle._def_prefix`): matches
# (`tractor[<aid.reprol()>]`) — set in # the `setproctitle`-set form (`_subactor[<aid.reprol()>]`)
# `_actor_child_main` for ALL backends, mutates argv via # — set in `_actor_child_main` for ALL backends, mutates
# libc so visible in `/proc/<pid>/cmdline`. # argv via libc so visible in `/proc/<pid>/cmdline`.
# - cmdline `tractor._child`: matches the legacy # - cmdline `tractor._child`: matches the legacy
# `python -m tractor._child --uid (...)` form. Catches # `python -m tractor._child --uid (...)` form. Catches
# procs that died before `_actor_child_main` got to call # procs that died before `_actor_child_main` got to call
# `setproctitle()` — argv from exec is still kernel- # `setproctitle()` — argv from exec is still kernel-
# visible at that point. # visible at that point.
# - comm `tractor[`: same proctitle-set form, but visible # - comm `_subactor[`: same proctitle-set form, but visible
# via `/proc/<pid>/comm` (kernel-truncated to ~15 bytes, # via `/proc/<pid>/comm` (kernel-truncated to ~15 bytes,
# `tractor[doggy:`). Critical for ZOMBIES — kernel # `_subactor[doggy:`). Critical for ZOMBIES — kernel
# preserves `comm` past task-exit until parent reaps, # preserves `comm` past task-exit until parent reaps,
# while `cmdline` for zombies often reads as empty. # while `cmdline` for zombies often reads as empty.
_TRACTOR_PROC_CMDLINE_MARKERS: tuple[str, ...] = ( _TRACTOR_PROC_CMDLINE_MARKERS: tuple[str, ...] = (
@ -253,7 +253,7 @@ def _is_tractor_subactor(pid: int) -> bool:
''' '''
# 1. cmdline match — catches both `setproctitle`-set # 1. cmdline match — catches both `setproctitle`-set
# `tractor[<aid>]` (live) AND legacy `python -m # `_subactor[<aid>]` (live) AND legacy `python -m
# tractor._child` (any) form. # tractor._child` (any) form.
cmdline: str = _read_cmdline(pid) cmdline: str = _read_cmdline(pid)
if any(m in cmdline for m in _TRACTOR_PROC_CMDLINE_MARKERS): if any(m in cmdline for m in _TRACTOR_PROC_CMDLINE_MARKERS):

View File

@ -281,8 +281,11 @@ def add_log_level(
`StackLevelAdapter` so `log.<name>('msg')` works (and so `StackLevelAdapter` so `log.<name>('msg')` works (and so
`get_logger()`'s per-level method audit passes). `get_logger()`'s per-level method audit passes).
Idempotent: re-registering an existing name is a no-op-ish Idempotent: re-registering an existing name refreshes its
refresh (won't clobber an already-bound method). value + color in place. The bound `StackLevelAdapter.<name>`
method reads the current `CUSTOM_LEVELS` value at call time, so
it tracks the new level across re-registration (the method
object is bound once, never re-bound/clobbered).
''' '''
name_up: str = name.upper() name_up: str = name.upper()
@ -294,16 +297,18 @@ def add_log_level(
BOLD_PALETTE['bold'][name_up] = f'bold_{color}' BOLD_PALETTE['bold'][name_up] = f'bold_{color}'
if not hasattr(StackLevelAdapter, name_lo): if not hasattr(StackLevelAdapter, name_lo):
# bind via default-arg so `value` is captured (not # read the level from `CUSTOM_LEVELS[name_up]` at CALL time
# late-bound); delegates to `.log()` exactly like the # (NOT captured at bind time) so a later
# hand-written level methods above. # `add_log_level(name, <new-value>)` re-registration stays
# consistent with this once-bound method. The closure binds
# *this* call's `name_up` local (stable, never reassigned),
# so there's no late-binding hazard. Delegates to `.log()`
# exactly like the hand-written level methods above.
def _emit( def _emit(
self, self,
msg: str, msg: str,
*,
_level: int = value,
) -> None: ) -> None:
return self.log(_level, msg) return self.log(CUSTOM_LEVELS[name_up], msg)
_emit.__name__ = name_lo _emit.__name__ = name_lo
_emit.__qualname__ = f'StackLevelAdapter.{name_lo}' _emit.__qualname__ = f'StackLevelAdapter.{name_lo}'

View File

@ -42,6 +42,22 @@ log = get_logger()
_UNSET = object() _UNSET = object()
# cap the stderr bytes retained for a rc!=0 `CalledProcessError`
# note: a long-lived (`check=True`) child could otherwise buffer
# its ENTIRE stderr history in parent mem — keep only the TAIL
# (the bytes nearest the failure, which are what you want to see).
_STDERR_NOTE_TAIL_BYTES: int = 16 * 1024
def _decode_line(raw: bytes) -> str:
'''
Decode one relayed line: replace undecodable bytes and trim a
trailing carriage-return (so CRLF-term'd lines read clean).
'''
return raw.decode(errors='replace').rstrip('\r')
def _add_stderr_note( def _add_stderr_note(
cpe: subprocess.CalledProcessError, cpe: subprocess.CalledProcessError,
stderr_bytes: bytes, stderr_bytes: bytes,
@ -65,6 +81,7 @@ async def _relay_stream_lines(
emit: Callable[[str], None]|None = None, emit: Callable[[str], None]|None = None,
tag: str = '', tag: str = '',
accum: bytearray|None = None, accum: bytearray|None = None,
accum_cap: int|None = None,
) -> None: ) -> None:
''' '''
Concurrently drain a child subproc's `stdout`/`stderr` Concurrently drain a child subproc's `stdout`/`stderr`
@ -95,15 +112,21 @@ async def _relay_stream_lines(
async for chunk in stream: # ends at child-exit EOF async for chunk in stream: # ends at child-exit EOF
if accum is not None: if accum is not None:
accum += chunk accum += chunk
# bound retained bytes to the TAIL (see caller +
# `_STDERR_NOTE_TAIL_BYTES`) so a chatty long-lived
# child can't grow parent mem without limit.
if (
accum_cap is not None
and
len(accum) > accum_cap
):
del accum[:len(accum) - accum_cap]
if emit is None: if emit is None:
continue # drain(+accum)-only continue # drain(+accum)-only
buf: bytes = residual + chunk buf: bytes = residual + chunk
*lines, residual = buf.split(b'\n') *lines, residual = buf.split(b'\n')
for raw in lines: for raw in lines:
line: str = raw.decode( emit(f'[{tag}] {_decode_line(raw)}')
errors='replace',
).rstrip('\r')
emit(f'[{tag}] {line}')
# flush any trailing partial (un-newline-term'd) line @ EOF # flush any trailing partial (un-newline-term'd) line @ EOF
if ( if (
@ -111,10 +134,7 @@ async def _relay_stream_lines(
and and
residual residual
): ):
line: str = residual.decode( emit(f'[{tag}] {_decode_line(residual)}')
errors='replace',
).rstrip('\r')
emit(f'[{tag}] {line}')
async def supervise_run_process( async def supervise_run_process(
@ -140,7 +160,7 @@ async def supervise_run_process(
# non-relay `stdout` override; defaults (via `_UNSET`) to # non-relay `stdout` override; defaults (via `_UNSET`) to
# `DEVNULL` so we NEVER inherit (+ thus can't clobber) the # `DEVNULL` so we NEVER inherit (+ thus can't clobber) the
# parent controlling-tty. # parent controlling-tty.
stdout: int = _UNSET, stdout: int|None = _UNSET,
task_status: trio.TaskStatus[ task_status: trio.TaskStatus[
trio.Process trio.Process
@ -158,11 +178,14 @@ async def supervise_run_process(
- surfaces a rc!=0 `subprocess.CalledProcessError` - surfaces a rc!=0 `subprocess.CalledProcessError`
DETERMINISTICALLY: we pass `check=False` to `trio` and DETERMINISTICALLY: we pass `check=False` to `trio` and
do our OWN post-drain rc-check, (re)building + raising a do our OWN post-drain rc-check, (re)building + raising the
BARE CPE (with a `.stderr` note) from this coro's body CPE (with a `.stderr` note) from this coro's body AFTER the
AFTER the child exits so there's no nursery-eg-wrapped child exits so it's never wrapped by `trio.run_process`'s
CPE to catch/`collapse_eg`, and the relay reader is never INTERNAL nursery nor race-cancelled mid-drain. It IS still
race-cancelled mid-drain. raised as a task into the *parent* `tn`, so like any
`tn.start()`-task raise it surfaces `ExceptionGroup`-
wrapped under `trio>=0.33`; unwrap via `except*` or
`trionics.collapse_eg()`.
- ALWAYS isolates the parent controlling-tty - ALWAYS isolates the parent controlling-tty
(`stdin=DEVNULL`, and `stdout=DEVNULL` unless (`stdin=DEVNULL`, and `stdout=DEVNULL` unless
@ -190,7 +213,14 @@ async def supervise_run_process(
) )
''' '''
emit: Callable[[str], None] = getattr(log, relay_level) # resolve the relay emit-method ONLY when actually relaying so
# a bad/typo'd `relay_level` can't crash a NON-relay call (and
# we don't bind an unused method on the common silent path).
emit: Callable[[str], None]|None = (
getattr(log, relay_level)
if (relay_stdout or relay_stderr)
else None
)
tag: str = ( tag: str = (
label label
or or
@ -201,6 +231,20 @@ async def supervise_run_process(
# MANAGED keys below override on conflict. # MANAGED keys below override on conflict.
rp_kwargs: dict = dict(run_process_kwargs) rp_kwargs: dict = dict(run_process_kwargs)
# XXX reject `trio`'s `capture_stdout`/`capture_stderr`: they
# ALIAS our MANAGED `stdout`/`stderr` keys, so forwarding them
# makes `trio.run_process` raise an opaque
# `ValueError("can't specify both ...")`. Fail LOUD + early
# with a pointer to the supported knobs instead.
for _cap_kw in ('capture_stdout', 'capture_stderr'):
if _cap_kw in rp_kwargs:
raise ValueError(
f'{_cap_kw!r} is unsupported here; use '
f'`relay_stdout=`/`relay_stderr=` for a live relay, '
f'or the `stdout=` override / default `check=True` '
f'stderr-capture instead.'
)
# XXX ALWAYS isolate the controlling-tty's stdin. # XXX ALWAYS isolate the controlling-tty's stdin.
rp_kwargs['stdin'] = subprocess.DEVNULL rp_kwargs['stdin'] = subprocess.DEVNULL
@ -210,6 +254,16 @@ async def supervise_run_process(
if relay_stdout: if relay_stdout:
rp_kwargs['stdout'] = subprocess.PIPE rp_kwargs['stdout'] = subprocess.PIPE
elif stdout is not _UNSET: elif stdout is not _UNSET:
# XXX a bare `stdout=PIPE` override has NO drain reader
# (only `relay_stdout` spins one up), so it would deadlock
# once the ~64KiB OS pipe buffer fills — reject it.
if stdout is subprocess.PIPE:
raise ValueError(
'Use `relay_stdout=True` to PIPE *and* drain '
'stdout; a bare `stdout=subprocess.PIPE` override '
'has no reader and will deadlock once the ~64KiB '
'pipe buffer fills.'
)
rp_kwargs['stdout'] = stdout rp_kwargs['stdout'] = stdout
else: else:
rp_kwargs['stdout'] = subprocess.DEVNULL rp_kwargs['stdout'] = subprocess.DEVNULL
@ -262,6 +316,7 @@ async def supervise_run_process(
emit=emit if relay_stderr else None, emit=emit if relay_stderr else None,
tag=f'{tag}:err', tag=f'{tag}:err',
accum=stderr_accum, accum=stderr_accum,
accum_cap=_STDERR_NOTE_TAIL_BYTES,
) )
) )
@ -287,6 +342,10 @@ async def supervise_run_process(
if stderr_accum is not None if stderr_accum is not None
else b'' else b''
) )
# NOTE: stdout is NOT captured (DEVNULL unless overridden)
# so the CPE carries only (tail-bounded) stderr; a cmd that
# logs failures to *stdout* should `relay_stdout=True` to
# surface them in the note.
cpe = subprocess.CalledProcessError( cpe = subprocess.CalledProcessError(
returncode=trio_proc.returncode, returncode=trio_proc.returncode,
cmd=trio_proc.args, cmd=trio_proc.args,