Compare commits

...

175 Commits

Author SHA1 Message Date
Guillermo Rodriguez 2bee54e137 Refinements, fix dec_hook builtins and same type bug 2026-04-05 13:40:48 -04:00
Guillermo Rodriguez 180c701352 Fix typing on mk_boxed_ext_structs 2026-04-05 13:40:48 -04:00
Guillermo Rodriguez c6fe840c77 Change test structs name to not get conflicts with pytest 2026-04-05 13:40:48 -04:00
Guillermo Rodriguez 29f9966682 Re-add boxed struct type system on _codec & create enc/dec hook auto factory 2026-04-05 13:40:48 -04:00
Bd bbf01d5161
Merge pull request #430 from goodboy/dependabot/uv/pygments-2.20.0
Bump pygments from 2.19.2 to 2.20.0
2026-04-05 13:33:33 -04:00
Bd ec8e8a2786
Merge pull request #426 from goodboy/remote_exc_type_registry
Fix remote exc relay + add `reg_err_types()` tests
2026-04-02 22:44:36 -04:00
Gud Boi c3d1ec22eb Fix `Type[BaseException]` annots, guard `.src_type` resolve
- Use `Type[BaseException]` (not bare `BaseException`)
  for all err-type references: `get_err_type()` return,
  `._src_type`, `boxed_type` in `unpack_error()`.
- Add `|None` where types can be unresolvable
  (`get_err_type()`, `.boxed_type` property).
- Add `._src_type_resolved` flag to prevent repeated
  lookups and guard against `._ipc_msg is None`.
- Fix `recevier` and `exeptions` typos.

Review: PR #426 (Copilot)
https://github.com/goodboy/tractor/pull/426

(this patch was generated in some part by [`claude-code`][claude-code-gh])
[claude-code-gh]: https://github.com/anthropics/claude-code
2026-04-02 18:21:19 -04:00
Gud Boi 8f44efa327 Drop stale `.cancel()`, fix docstring typo in tests
- Remove leftover `await an.cancel()` in
  `test_registered_custom_err_relayed`; the
  nursery already cancels on scope exit.
- Fix `This document` -> `This documents` typo in
  `test_unregistered_err_still_relayed` docstring.

Review: PR #426 (Copilot)
https://github.com/goodboy/tractor/pull/426

(this patch was generated in some part by [`claude-code`][claude-code-gh])
[claude-code-gh]: https://github.com/anthropics/claude-code
2026-04-02 18:21:19 -04:00
Gud Boi 5968a3c773 Use `'<unknown>'` for unresolvable `.boxed_type_str`
Add a teensie unit test to match.

(this patch was generated in some part by [`claude-code`][claude-code-gh])
[claude-code-gh]: https://github.com/anthropics/claude-code
2026-04-02 18:21:19 -04:00
Gud Boi 80597b80bf Add passing test for unregistered err relay
Drop the `xfail` test and instead add a new one that ensures the
`tractor._exceptions` fixes enable graceful relay of
remote-but-unregistered error types via the unboxing of just the
`rae.src_type_str/boxed_type_str` content. The test also ensures
a warning is included with remote error content indicating the user
should register their error type for effective cross-actor re-raising.

Deats,
- add `test_unregistered_err_still_relayed`: verify the
  `RemoteActorError` IS raised with `.boxed_type`
  as `None` but `.src_type_str`, `.boxed_type_str`,
  and `.tb_str` all preserved from the IPC msg.
- drop `test_unregistered_boxed_type_resolution_xfail`
  since the new above case covers it and we don't need to have
  an effectively entirely repeated test just with an inverse assert
  as it's last line..

(this patch was generated in some part by [`claude-code`][claude-code-gh])
[claude-code-gh]: https://github.com/anthropics/claude-code
2026-04-02 18:21:19 -04:00
Gud Boi a41c6d5c70 Fix unregistered-remote-error-type relay crash
Make `RemoteActorError` resilient to unresolved
custom error types so that errors from remote actors
always relay back to the caller - even when the user
hasn't called `reg_err_types()` to register the exc type.

Deats,
- `.src_type`: log warning + return `None` instead
  of raising `TypeError` which was crashing the
  entire `_deliver_msg()` -> `pformat()` chain
  before the error could be relayed.
- `.boxed_type_str`: fallback to `_ipc_msg.boxed_type_str`
  when the type obj can't be resolved so the type *name* is always
  available.
- `unwrap_src_err()`: fallback to `RuntimeError` preserving
  original type name + traceback.
- `unpack_error()`: log warning when `get_err_type()` returns
  `None` telling the user to call `reg_err_types()`.

(this patch was generated in some part by [`claude-code`][claude-code-gh])
[claude-code-gh]: https://github.com/anthropics/claude-code
2026-04-02 18:21:19 -04:00
Gud Boi 9c37b3f956 Add `reg_err_types()` test suite for remote exc relay
Verify registered custom error types round-trip correctly over IPC via
`reg_err_types()` + `get_err_type()`.

Deats,
- `TestRegErrTypesPlumbing`: 5 unit tests for the type-registry plumbing
  (register, lookup, builtins, tractor-native types, unregistered
  returns `None`)
- `test_registered_custom_err_relayed`: IPC end-to-end for a registered
  `CustomAppError` checking `.boxed_type`, `.src_type`, and `.tb_str`
- `test_registered_another_err_relayed`: same for `AnotherAppError`
  (multi-type coverage)
- `test_unregistered_custom_err_fails_lookup`: `xfail` documenting that
  `.boxed_type` can't resolve without `reg_err_types()` registration

(this patch was generated in some part by [`claude-code`][claude-code-gh])
[claude-code-gh]: https://github.com/anthropics/claude-code
2026-04-02 18:21:19 -04:00
Bd 8f6bc56174
Merge pull request #427 from goodboy/subsys_reorg
Mv core mods to `runtime/`, `spawn/`, `discovery/` subpkgs
2026-04-02 18:21:00 -04:00
Gud Boi b14dbde77b Skip `test_empty_mngrs_input_raises` on UDS tpt
The `open_actor_cluster()` teardown hangs
intermittently on UDS when `gather_contexts(mngrs=())`
raises `ValueError` mid-setup; likely a race in the
actor-nursery cleanup vs UDS socket shutdown. TCP
passes reliably (5/5 runs).

- Add `tpt_proto` fixture param to the test
- `pytest.skip()` on UDS with a TODO for deeper
  investigation of `._clustering`/`._supervise`
  teardown paths

(this patch was generated in some part by [`claude-code`][claude-code-gh])
[claude-code-gh]: https://github.com/anthropics/claude-code
2026-04-02 17:59:13 -04:00
Gud Boi cd6509b724 Fix `tractor_test` kwarg check and Windows `start_method` default
- Use `kw in kwargs` membership test instead of
  `kwargs[kw]` to avoid `KeyError` on missing params.
- Restructure Windows `start_method` logic to properly
  default to `'trio'` when unset; only raise on an
  explicit non-trio value.

Review: PR #427 (Copilot)
https://github.com/goodboy/tractor/pull/427#pullrequestreview-4009934142

(this patch was generated in some part by [`claude-code`][claude-code-gh])
[claude-code-gh]: https://github.com/anthropics/claude-code
2026-04-02 17:59:13 -04:00
Gud Boi 93d99ed2eb Move `get_cpu_state()` to `conftest` as shared latency headroom
Factor the CPU-freq-scaling helper out of
`test_legacy_one_way_streaming` into `conftest.py`
alongside a new `cpu_scaling_factor()` convenience fn
that returns a latency-headroom multiplier (>= 1.0).

Apply it to the two other flaky-timeout tests,
- `test_cancel_via_SIGINT_other_task`: 2s -> scaled
- `test_example[we_are_processes.py]`: 16s -> scaled

Deats,
- add `get_cpu_state()` + `cpu_scaling_factor()` to
  `conftest.py` so all test mods can share the logic.
- catch `IndexError` (empty glob) in addition to
  `FileNotFoundError`.
- rename `factor` var -> `headroom` at call sites for
  clarity on intent.

(this patch was generated in some part by [`claude-code`][claude-code-gh])
[claude-code-gh]: https://github.com/anthropics/claude-code
2026-04-02 17:59:13 -04:00
Gud Boi 6215e3b2dd Adjust `test_a_quadruple_example` time-limit for CPU scaling
Add `get_cpu_state()` helper to read CPU freq settings
from `/sys/devices/system/cpu/` and use it to compensate
the perf time-limit when `auto-cpufreq` (or similar)
scales down the max frequency.

Deats,
- read `*_pstate_max_freq` and `scaling_max_freq`
  to compute a `cpu_scaled` ratio.
- when `cpu_scaled != 1.`, increase `this_fast` limit
  proportionally (factoring dual-threaded cores).
- log a warning via `test_log` when compensating.

(this commit msg was generated in some part by [`claude-code`][claude-code-gh])
[claude-code-gh]: https://github.com/anthropics/claude-code
2026-04-02 17:59:13 -04:00
Gud Boi be5d8da8c0 Just alias `Arbiter` via assignment 2026-04-02 17:59:13 -04:00
Gud Boi 21ed181835 Filter `__pycache__` from example discovery in tests
(this commit msg was generated in some part by [`claude-code`][claude-code-gh])
[claude-code-gh]: https://github.com/anthropics/claude-code
2026-04-02 17:59:13 -04:00
Gud Boi 9ec2749ab7 Rename `Arbiter` -> `Registrar`, mv to `discovery._registry`
Move the `Arbiter` class out of `runtime._runtime` into its
logical home at `discovery._registry` as `Registrar(Actor)`.
This completes the long-standing terminology migration from
"arbiter" to "registrar/registry" throughout the codebase.

Deats,
- add new `discovery/_registry.py` mod with `Registrar`
  class + backward-compat `Arbiter = Registrar` alias.
- rename `Actor.is_arbiter` attr -> `.is_registrar`;
  old attr now a `@property` with `DeprecationWarning`.
- `_root.py` imports `Registrar` directly for
  root-actor instantiation.
- export `Registrar` + `Arbiter` from `tractor.__init__`.
- `_runtime.py` re-imports from `discovery._registry`
  for backward compat.

Also,
- update all test files to use `.is_registrar`
  (`test_local`, `test_rpc`, `test_spawning`,
  `test_discovery`, `test_multi_program`).
- update "arbiter" -> "registrar" in comments/docstrings
  across `_discovery.py`, `_server.py`, `_transport.py`,
  `_testing/pytest.py`, and examples.
- drop resolved TODOs from `_runtime.py` and `_root.py`.

(this patch was generated in some part by [`claude-code`][claude-code-gh])
[claude-code-gh]: https://github.com/anthropics/claude-code
2026-04-02 17:59:13 -04:00
Gud Boi f3441a6790 Update tests+examples imports for new subpkgs
Adjust all `tractor._state`, `tractor._addr`,
`tractor._supervise`, etc. refs in tests and examples
to use the new `runtime/`, `discovery/`, `spawn/` paths.

Also,
- use `tractor.debug_mode()` pub API instead of
  `tractor._state.debug_mode()` in a few test mods
- add explicit `timeout=20` to `test_respawn_consumer_task`
  `@tractor_test` deco call

(this patch was generated in some part by [`claude-code`][claude-code-gh])
[claude-code-gh]: https://github.com/anthropics/claude-code
2026-04-02 17:59:13 -04:00
Gud Boi cc42d38284 Mv core mods to `runtime/`, `spawn/`, `discovery/` subpkgs
Restructure the flat `tractor/` top-level private mods
into (more nested) subpackages:

- `runtime/`: `_runtime`, `_portal`, `_rpc`, `_state`,
  `_supervise`
- `spawn/`: `_spawn`, `_entry`, `_forkserver_override`,
  `_mp_fixup_main`
- `discovery/`: `_addr`, `_discovery`, `_multiaddr`

Each subpkg `__init__.py` is kept lazy (no eager
imports) to avoid circular import issues.

Also,
- update all intra-pkg imports across ~35 mods to use
  the new subpkg paths (e.g. `from .runtime._state`
  instead of `from ._state`)

(this patch was generated in some part by [`claude-code`][claude-code-gh])
[claude-code-gh]: https://github.com/anthropics/claude-code
2026-04-02 17:59:13 -04:00
Gud Boi 6827ceba12 Use `wrapt` for `tractor_test()` decorator
Refactor the test-fn deco to use `wrapt.decorator`
instead of `functools.wraps` for better fn-sig
preservation and optional-args support via
`PartialCallableObjectProxy`.

Deats,
- add `timeout` and `hide_tb` deco params
- wrap test-fn body with `trio.fail_after(timeout)`
- consolidate per-fixture `if` checks into a loop
- add `iscoroutinefunction()` type-check on wrapped fn
- set `__tracebackhide__` at each wrapper level

Also,
- update imports for new subpkg paths:
  `tractor.spawn._spawn`, `tractor.discovery._addr`,
  `tractor.runtime._state`
  (see upcoming, likely large patch commit ;)

(this commit msg was generated in some part by [`claude-code`][claude-code-gh])
[claude-code-gh]: https://github.com/anthropics/claude-code
2026-04-02 17:59:13 -04:00
Gud Boi 94458807ce Expose `RuntimeVars` + `get_runtime_vars()` from pkg
Export the new `RuntimeVars` struct and `get_runtime_vars()`
from `tractor.__init__` and improve the accessor to
optionally return the struct form.

Deats,
- add `RuntimeVars` and `get_runtime_vars` to
  `__init__.py` exports; alphabetize `_state` imports.
- move `get_runtime_vars()` up in `_state.py` to sit
  right below `_runtime_vars` dict definition.
- add `as_dict: bool = True` param so callers can get
  either the legacy `dict` or the new `RuntimeVars`
  struct.
- drop the old stub fn at bottom of `_state.py`.
- rm stale `from .msg.pretty_struct import Struct` comment.

(this commit msg was generated in some part by [`claude-code`][claude-code-gh])
[claude-code-gh]: https://github.com/anthropics/claude-code
2026-04-02 17:59:13 -04:00
Gud Boi be5e7e446b Proto a `._state.RuntimeVars` struct
So we can start transition from runtime-vars `dict` to a typed struct
for better clarity and wire-ready monitoring potential, as well as
better traceability when .

Deats,
- add a new `RuntimeVars(Struct)` with all fields from `_runtime_vars`
  dict typed out
- include `__setattr__()` with `breakpoint()` for debugging
  any unexpected mutations.
- add `.update()` method for batch-updating compat with `dict`.
- keep old `_runtime_vars: dict` in place (we need to port a ton of
  stuff to adjust..).

(this commit msg was generated in some part by [`claude-code`][claude-code-gh])
[claude-code-gh]: https://github.com/anthropics/claude-code
2026-04-02 17:59:13 -04:00
Gud Boi 571b2b320e Add `reg_err_types()` for custom remote exc lookup
Allow external app code to register custom exception types
on `._exceptions` so they can be re-raised on the receiver
side of an IPC dialog via `get_err_type()`.

(this commit msg was generated in some part by [`claude-code`][claude-code-gh])
[claude-code-gh]: https://github.com/anthropics/claude-code
2026-04-02 17:59:13 -04:00
Gud Boi c7b5d00f19 Add `get_runtime_vars()` accessor to `._state`
Expose a copy of the current actor's `_runtime_vars` dict
via a public fn; TODO to convert to `RuntimeVars` struct.

(this commit msg was generated in some part by [`claude-code`][claude-code-gh])
[claude-code-gh]: https://github.com/anthropics/claude-code
2026-04-02 17:59:13 -04:00
dependabot[bot] 1049f7bf38
Bump pygments from 2.19.2 to 2.20.0
Bumps [pygments](https://github.com/pygments/pygments) from 2.19.2 to 2.20.0.
- [Release notes](https://github.com/pygments/pygments/releases)
- [Changelog](https://github.com/pygments/pygments/blob/master/CHANGES)
- [Commits](https://github.com/pygments/pygments/compare/2.19.2...2.20.0)

---
updated-dependencies:
- dependency-name: pygments
  dependency-version: 2.20.0
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
2026-03-30 20:28:44 +00:00
Bd cc3bfac741
Merge pull request #366 from goodboy/dereg_on_oserror
Make `find_actor()` delete stale sockaddr entries from registrar on `OSError`
2026-03-25 03:27:27 -04:00
Gud Boi e71eec07de Refine type annots in `_discovery` and `_runtime`
- Add `LocalPortal` union to `query_actor()` return
  type and `reg_portal` var annotation since the
  registrar yields a `LocalPortal` instance.
- Update docstring to note the `LocalPortal` case.
- Widen `.delete_addr()` `addr` param to accept
  `list[str|int]` bc msgpack deserializes tuples as
  lists over IPC.
- Tighten `uid` annotation to `tuple[str, str]|None`.

(this patch was generated in some part by [`claude-code`][claude-code-gh])
[claude-code-gh]: https://github.com/anthropics/claude-code
2026-03-25 02:16:48 -04:00
Gud Boi b557ec20a7 Coerce IPC `addr` to `tuple` in `.delete_addr()`
`msgpack` deserializes tuples as lists over IPC so
the `bidict.inverse.pop()` needs a `tuple`-cast to
match registry keys.

Regressed-by: 85457cb (`registry_addrs` change)
Found-via: `/run-tests` test_stale_entry_is_deleted

(this patch was generated in some part by [`claude-code`][claude-code-gh])
[claude-code-gh]: https://github.com/anthropics/claude-code
2026-03-25 01:36:58 -04:00
Gud Boi 85457cb839 Address Copilot review suggestions on PR #366
- Use `bidict.forceput()` in `register_actor()` to handle
  duplicate addr values from stale entries or actor restarts.
- Fix `uid` annotation to `tuple[str, str]|None` in
  `maybe_open_portal()` and handle the `None` return from
  `delete_addr()` in log output.
- Pass explicit `registry_addrs=[reg_addr]` to `open_nursery()`
  and `find_actor()` in `test_stale_entry_is_deleted` to ensure
  the test uses the remote registrar.
- Update `query_actor()` docstring to document the new
  `(addr, reg_portal)` yield shape.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-25 00:21:09 -04:00
Gud Boi 850219f60c Guard `reg_portal` for `None` in `maybe_open_portal()`
Fix potential `AttributeError` when `query_actor()` yields
a `None` portal (peer-found-locally path) and an `OSError`
is raised during transport connect.

Also,
- fix `Arbiter.delete_addr()` return type to
  `tuple[str, str]|None` bc it can return `None`.
- fix "registar" typo -> "registrar" in comment.

(this patch was generated in some part by [`claude-code`][claude-code-gh])
[claude-code-gh]: https://github.com/anthropics/claude-code
2026-03-17 17:26:57 -04:00
Tyler Goodlet d929fb75b5 Rename `.delete_sockaddr()` -> `.delete_addr()` 2026-03-13 21:51:15 -04:00
Tyler Goodlet 403c2174a1 Always no-raise try-to-pop registry addrs 2026-03-13 21:51:15 -04:00
Tyler Goodlet 528012f35f Add stale entry deleted from registrar test
By spawning an actor task that immediately shuts down the transport
server and then sleeps, verify that attempting to connect via the
`._discovery.find_actor()` helper delivers `None` for the `Portal`
value.

Relates to #184 and #216
2026-03-13 21:51:15 -04:00
Tyler Goodlet 0dfa6f4a8a Don't unwrap and unwrapped addr, just warn on delete XD 2026-03-13 21:51:15 -04:00
Tyler Goodlet a0d3741fac Ensure `._registry` values are hashable, since `bidict`! 2026-03-13 21:51:15 -04:00
Tyler Goodlet 149b800c9f Handle stale registrar entries; detect and delete
In cases where an actor's transport server task (by default handling new
TCP connections) terminates early but does not de-register from the
pertaining registry (aka the registrar) actor's address table, the
trying-to-connect client actor will get a connection error on that
address. In the case where client handles a (local) `OSError` (meaning
the target actor address is likely being contacted over `localhost`)
exception, make a further call to the registrar to delete the stale
entry and `yield None` gracefully indicating to calling code that no
`Portal` can be delivered to the target address.

This issue was originally discovered in `piker` where the `emsd`
(clearing engine) actor would sometimes crash on rapid client
re-connects and then leave a `pikerd` stale entry. With this fix new
clients will attempt connect via an endpoint which will re-spawn the
`emsd` when a `None` portal is delivered (via `maybe_spawn_em()`).
2026-03-13 21:51:15 -04:00
Tyler Goodlet 03f458a45c Add `Arbiter.delete_sockaddr()` to remove addrs
Since stale addrs can be leaked where the actor transport server task
crashes but doesn't (successfully) unregister from the registrar, we
need a remote way to remove such entries; hence this new (registrar)
method.

To implement this make use of the `bidict` lib for the `._registry`
table thus making it super simple to do reverse uuid lookups from an
input socket-address.
2026-03-13 21:51:15 -04:00
Bd e77198bb64
Merge pull request #422 from goodboy/global_uds_in_test_harness
Run (some) test suites in CI with `--tpt-proto uds`
2026-03-13 21:50:45 -04:00
Gud Boi 5b8f6cf4c7 Use `.aid.uid` to avoid deprecation warns in tests
- `test_inter_peer_cancellation`: swap all `.uid` refs
  on `Actor`, `Channel`, and `Portal` to `.aid.uid`
- `test_legacy_one_way_streaming`: same + fix `print()`
  to multiline style

(this commit msg was generated in some part by [`claude-code`][claude-code-gh])
[claude-code-gh]: https://github.com/anthropics/claude-code
2026-03-13 21:10:52 -04:00
Gud Boi 8868ff19f3 Flip to `ActorNursery.cancel_called` API
Avoid deprecation warnings, prepping for property removal.
2026-03-13 21:10:52 -04:00
Gud Boi 066011b83d Bump `fail_after` delay on non-linux for sync-sleep test
Use 6s timeout on non-linux (vs 4s) in
`test_cancel_while_childs_child_in_sync_sleep()` to avoid
flaky `TooSlowError` on slower CI runners.

(this commit msg was generated in some part by [`claude-code`][claude-code-gh])
[claude-code-gh]: https://github.com/anthropics/claude-code
2026-03-13 21:10:52 -04:00
Gud Boi b1d003d850 Add `--tpt-proto` CI matrix and wire to `pytest`
- add `tpt_proto: ['tcp', 'uds']` matrix dimension
  to the `testing` job.
- exclude `uds` on `macos-latest` for now.
- pass `--tpt-proto=${{ matrix.tpt_proto }}` to the
  `pytest` invocation.

(this commit msg was generated in some part by [`claude-code`][claude-code-gh])
[claude-code-gh]: https://github.com/anthropics/claude-code
2026-03-13 21:10:52 -04:00
Gud Boi 8991ec2bf5 Fix warns and de-reg race in `test_discovery`
Removes the `pytest` deprecation warns and attempts to avoid
some de-registration raciness, though i'm starting to think the
real issue is due to not having the fixes from #366 (which handle
the new dereg on `OSError` case from UDS)?

- use `.channel.aid.uid` over deprecated `.channel.uid`
  throughout `test_discovery.py`.
- add polling loop (up to 5s) for subactor de-reg check
  in `spawn_and_check_registry()` to handle slower
  transports like UDS where teardown takes longer.

(this commit msg was generated in some part by [`claude-code`][claude-code-gh])
[claude-code-gh]: https://github.com/anthropics/claude-code
2026-03-13 21:10:52 -04:00
Gud Boi dfc153c228 'Bump `daemon` pre-wait for 'uds' parametrization' 2026-03-13 21:10:52 -04:00
Gud Boi 52e8fb43ee Tighten UDS addr validation and sockname prefixes
- add `is_valid` and `sockpath.resolve()` asserts in
  `get_rando_addr()` for the `'uds'` case plus an
  explicit `UDSAddress` type annotation.
- rename no-runtime sockname prefixes from
  `'<unknown-actor>'`/`'root'` to
  `'no_runtime_root'`/`'no_runtime_actor'` with a proper
  if/else branch in `UDSAddress.get_random()`.

(this commit msg was generated in some part by [`claude-code`][claude-code-gh])
[claude-code-gh]: https://github.com/anthropics/claude-code
2026-03-13 21:10:52 -04:00
Gud Boi 99577b719a Skip cluster test on UDS, wire `tpt_proto` fixture
Add UDS skip-guard to `test_streaming_to_actor_cluster()`
and plumb `tpt_proto` through the `@tractor_test` wrapper
so transport-parametrized tests can receive it.

Deats,
- skip cluster test when `tpt_proto == 'uds'` with
  descriptive msg, add TODO about `@pytest.mark.no_tpt`.
- add `tpt_proto: str|None` param to inner wrapper in
  `tractor_test()`, forward to decorated fn when its sig
  accepts it.
- register custom `no_tpt` marker via `pytest_configure()`
  to avoid unknown-marker warnings.
- add masked todo for `no_tpt` marker-check code in `tpt_proto` fixture
  (needs fn-scope to work, left as TODO).
- add `request` param to `tpt_proto` fixture for future
  marker inspection.

Also,
- add doc-string to `test_streaming_to_actor_cluster()`.

(this commit msg was generated in some part by [`claude-code`][claude-code-gh])
[claude-code-gh]: https://github.com/anthropics/claude-code
2026-03-13 21:10:52 -04:00
Gud Boi 4092db60b2 Revert advanced-fault UDS edge case handling
Namely the workaround expected exc branches added in ef7ed7a for the UDS
parametrization. With the new boxing of the underlying CREs as
tpt-closed, we can expect the same exc outcomes as in the TCP cases.

Also this tweaks some error report logging content used while debugging
this,
- properly `repr()` the `TransportClosed.src_exc`-type from
  the maybe emit in `.report_n_maybe_raise()`.
- remove the redudant `chan.raddr` from the "closed abruptly"
  header in the tpt-closed handler of `._rpc.process_messages()`,
  the `Channel.__repr__()` now contains it by default.
2026-03-13 21:10:52 -04:00
Gud Boi 4f333dee05 Pass `enable_transports` in `daemon` fixture
Forward the `tpt_proto` fixture val into spawned daemon
subprocesses via `run_daemon(enable_transports=..)` and
sync `_runtime_vars['_enable_tpts']` in the `tpt_proto`
fixture so sub-actors inherit the transport setting.

Deats,
- add `enable_transports={enable_tpts}` to the daemon
  spawn-cmd template in `tests/conftest.py`.
- set `_state._runtime_vars['_enable_tpts']` in the
  `tpt_proto` fixture in `_testing/pytest.py`.

(this patch was generated in some part by [`claude-code`][claude-code-gh])
[claude-code-gh]: https://github.com/anthropics/claude-code
2026-03-13 21:10:52 -04:00
Gud Boi 8a2f74da2c Bump `_PROC_SPAWN_WAIT` and use `test_log` in `daemon`
For more reliability with the oob registrar using tests
via the `daemon` fixture,
- increase spawn-wait to `2` in CI, `1` OW; drop
  the old py<3.7 branch.
- move `_ci_env` to module-level (above `_non_linux`)
  so `_PROC_SPAWN_WAIT` can reference it at parse time.
- add `test_log` fixture param to `daemon()`, use it
  for the error-on-exit log line instead of bare `log`.

(this commit msg was generated in some part by [`claude-code`][claude-code-gh])
[claude-code-gh]: https://github.com/anthropics/claude-code
2026-03-13 21:10:52 -04:00
Gud Boi 2bf155131d Make `spawn()` `expect_timeout` configurable
Add `expect_timeout: float` param to `_spawn()`
so individual tests can tune `pexpect` timeouts
instead of relying on the hard-coded 3/10 split.

Deats,
- default to 4s, bump by +6 on non-linux CI.
- use walrus `:=` to capture resolved timeout and  assert
  `spawned.timeout == timeout` for sanity.

(this commit msg was generated in some part by [`claude-code`][claude-code-gh])
[claude-code-gh]: https://github.com/anthropics/claude-code
2026-03-13 21:10:52 -04:00
Gud Boi 0f29f5717a `ci.yml` add todos for mp-backend runs and for supporting subints soon! 2026-03-13 21:10:52 -04:00
Gud Boi 5ea721683b Use `.aid.uid` to avoid deprecation warns
I started getting annoyed by all the warnings from `pytest` during work
on macos suport in CI, so this replaces all `Actor.uid`/`Channel.uid`
accesses with `.aid.uid` (or `.aid.reprol()` for log msgs) across the
core runtime and IPC subsystems to avoid the noise.

This also provides incentive to start the adjustment to all
`.uid`-holding/tracking internal `dict`-tables/data-structures to
instead use `.msg.types.Aid`. Hopefully that will come a (vibed?) follow
up shortly B)

Deats,
- `._context`: swap all `self._actor.uid`, `self.chan.uid`,
  and `portal.actor.uid` refs to `.aid.uid`; use
  `.aid.reprol()` for log/error formatting.
- `._rpc`: same treatment for `actor.uid`, `chan.uid` in
  log msgs and cancel-scope handling; fix `str(err)` typo
  in `ContextCancelled` log.
- `._runtime`: update `chan.uid` -> `chan.aid.uid` in ctx
  cache lookups, RPC `Start` msg, registration and
  cancel-request handling; improve ctxc log formatting.
- `._spawn`: replace all `subactor.uid` with
  `.aid.uid` for child-proc tracking, IPC peer waiting,
  debug-lock acquisition, and nursery child dict ops.
- `._supervise`: same for `subactor.uid` in cancel and
  portal-wait paths; use `actor.aid.uid` for error dict.
- `._state`: fix `last.uid` -> `last.aid.uid` in
  `current_actor()` error msg.

Also,
- `._chan`: make `Channel.aid` a proper `@property` backed
  by `._aid` so we can add validation/typing later.
- `.log`: use `current_actor().aid.uuid` instead of
  `.uid[1]` for actor-uid log field.
- `.msg.types`: add TODO comment for `Start.aid` field
  conversion.

(this commit msg was generated in some part by [`claude-code`][claude-code-gh])
[claude-code-gh]: https://github.com/anthropics/claude-code
2026-03-13 21:10:52 -04:00
Gud Boi f84ef44992 Repair lifetime-stack suite's flakiness
Event on linux i was noticing lotsa false negatives based on sub
teardown race conditions, so this tries to both make way for
(eventually?) expanding the set of suite cases and ensure the current
ones are more reliable on every run.

The main change is to hange the `error_in_child=False` case to use
parent-side-cancellation via a new `trio.move_on_after(timeout)` instead
of `actor.cancel_soon()` (which is now toggled by a new `self_cancel:
bool` but unused rn), and add better teardown assertions.

Low level deats,
- add `rent_cancel`/`self_cancel` params to
  `crash_and_clean_tmpdir()` for different cancel paths;
  default to `rent_cancel=True` which just sleeps forever
  letting the parent's timeout do the work.
- use `trio.move_on_after()` with longer timeouts per
  case: 1.6s for error, 1s for cancel.
- use the `.move_on_after()` cancel-scope to assert `.cancel_called`
  pnly when `error_in_child=False`, indicating we
  parent-graceful-cancelled the sub.
- add `loglevel` fixture, pass to `open_nursery()`.
- log caught `RemoteActorError` via console logger.
- add `ids=` to parametrize for readable test names.

(this commit msg was generated in some part by [`claude-code`][claude-code-gh])
[claude-code-gh]: https://github.com/anthropics/claude-code
2026-03-13 21:10:52 -04:00
Gud Boi 1e0c57c6c5 Wrap cluster test in `trio.fail_after()`
Add a 6s timeout guard around `test_streaming_to_actor_cluster()`
to catch hangs, and nest the `async with` block inside it.
Found this when running `pytest tests/ --tpt-proto uds`.

(this commit msg was generated in some part by [`claude-code`][claude-code-gh])
[claude-code-gh]: https://github.com/anthropics/claude-code
2026-03-13 21:10:52 -04:00
Gud Boi 65660c77c7 Add note about `--tpt-proto` controlling `reg_addr`-type 2026-03-13 21:10:52 -04:00
Bd c9b415475f
Merge pull request #413 from goodboy/to_asyncio_channel_iface
Extend the `to_asyncio` inter-loop-task channel iface
2026-03-13 21:09:13 -04:00
Gud Boi 359bcf691f Update `docs/README.rst` to use `chan` API style
Sync the inline "infected asyncio" echo-server example
with the new `LinkedTaskChannel` iface from prior commits.

- `to_trio`/`from_trio` params -> `chan: LinkedTaskChannel`
- use `chan.started_nowait()`, `.send_nowait()`, `.get()`
- swap yield order to `(chan, first)`
- update blurb to describe the new unified channel API

(this patch was generated in some part by [`claude-code`][claude-code-gh])
[claude-code-gh]: https://github.com/anthropics/claude-code
2026-03-13 20:54:49 -04:00
Gud Boi b3ce5ab4f6 Swap `open_channel_from()` to yield `(chan, first)`
Deliver `(LinkedTaskChannel, Any)` instead of the prior `(first, chan)`
order from `open_channel_from()` to match the type annotation and be
consistent with `trio.open_*_channel()` style where the channel obj
comes first.

- flip `yield first, chan` -> `yield chan, first`
- update type annotation + docstring to match
- swap all unpack sites in tests and examples

(this patch was generated in some part by [`claude-code`][claude-code-gh])
[claude-code-gh]: https://github.com/anthropics/claude-code
2026-03-13 19:28:57 -04:00
Gud Boi e89fe03da7 Fix `LinkedTaskChannel` docstrings from GH bot review
Address valid findings from copilot's PR #413 review
(https://github.com/goodboy/tractor/pull/413
 #pullrequestreview-3925876037):

- `.get()` docstring referenced non-existent
  `._from_trio` attr, correct to `._to_aio`.
- `.send()` docstring falsely claimed error-raising
  on missing `from_trio` arg; reword to describe the
  actual `.put_nowait()` enqueue behaviour.
- `.open_channel_from()` return type annotation had
  `tuple[LinkedTaskChannel, Any]` but `yield` order
  is `(first, chan)`; fix annotation + docstring to
  match actual `tuple[Any, LinkedTaskChannel]`.

(this patch was generated in some part by [`claude-code`][claude-code-gh])
[claude-code-gh]: https://github.com/anthropics/claude-code
2026-03-13 19:28:57 -04:00
Gud Boi 417b796169 Use `chan: LinkedTaskChannel` API in all aio-task fns
Convert every remaining `to_trio`/`from_trio` fn-sig style
to the new unified `chan: LinkedTaskChannel` iface added in
prior commit (c46e9ee8).

Deats,
- `to_trio.send_nowait(val)` (1st call) -> `chan.started_nowait(val)`
- `to_trio.send_nowait(val)` (subsequent) -> `chan.send_nowait(val)`
- `await from_trio.get()` -> `await chan.get()`

Converted fns,
- `sleep_and_err()`, `push_from_aio_task()` in
  `tests/test_infected_asyncio.py`
- `sync_and_err()` in `tests/test_root_infect_asyncio.py`
- `aio_streamer()` in
  `tests/test_child_manages_service_nursery.py`
- `aio_echo_server()` in
  `examples/infected_asyncio_echo_server.py`
- `bp_then_error()` in `examples/debugging/asyncio_bp.py`

Also,
- drop stale comments referencing old param names.

(this patch was generated in some part by [`claude-code`][claude-code-gh])
[claude-code-gh]: https://github.com/anthropics/claude-code
2026-03-13 19:28:57 -04:00
Gud Boi 36cbc07602 Tried out an alt approach for `.to_asyncio` crashes
This change is masked out now BUT i'm leaving it in for reference.

I was debugging a multi-actor fault where the primary source actor was
an infected-aio-subactor (`brokerd.ib`) and it seemed like the REPL was only
entering on the `trio` side (at a `.open_channel_from()`) and not
eventually breaking in the `asyncio.Task`. But, since (changing
something?) it seems to be working now, it's just that the `trio` side
seems to sometimes handle before the (source/causing and more
child-ish) `asyncio`-task, which is a bit odd and not expected..
We could likely refine (maybe with an inter-loop-task REPL lock?) this
at some point and ensure a child-`asyncio` task which errors always
grabs the REPL **first**?

Lowlevel deats/further-todos,
- add (masked) `maybe_open_crash_handler()` block around
  `asyncio.Task` execution with notes about weird parent-addr
  delivery bug in `test_sync_pause_from_aio_task`
  * yeah dunno what that's about but made a bug; seems to be IPC
    serialization of the `TCPAddress` struct somewhere??
- add inter-loop lock TODO for avoiding aio-task clobbering
  trio-tasks when both crash in debug-mode

Also,
- change import from `tractor.devx.debug` to `tractor.devx`
- adjust `get_logger()` call to use new implicit mod-name detection
  added to `.log.get_logger()`, i.e. sin `name=__name__`.
- some teensie refinements to `open_channel_from()`:
  * swap return type annotation for  to `tuple[LinkedTaskChannel, Any]`
    (was `Any`).
  * update doc-string to clarify started-value delivery
  * add err-log before `.pause()` in what should be an unreachable path.
  * add todo to swap the `(first, chan)` pair to match that of ctx..

(this commit msg was generated in some part by [`claude-code`][claude-code-gh])

[claude-code-gh]: https://github.com/anthropics/claude-code
2026-03-13 19:28:57 -04:00
Tyler Goodlet 1f2fad22ee Extend `.to_asyncio.LinkedTaskChannel` for aio side
With methods to comms similar to those that exist for the `trio` side,
- `.get()` which proxies verbatim to the `._to_aio: asyncio.Queue`,
- `.send_nowait()` which thin-wraps to `._to_trio: trio.MemorySendChannel`.

Obviously the more correct design is to break up the channel type into
a pair of handle types, one for each "side's" task in each event-loop,
that's hopefully coming shortly in a follow up patch B)

Also,
- fill in some missing doc strings, tweak some explanation comments and
  update todos.
- adjust the `test_aio_errors_and_channel_propagates_and_closes()` suite
  to use the new `chan` fn-sig-API with `.open_channel_from()` including
  the new methods for msg comms; ensures everything added here works e2e.
2026-03-13 19:28:57 -04:00
Tyler Goodlet ca5f6f50a8 Explain the `infect_asyncio: bool` param to pass in RTE msg 2026-03-13 19:28:57 -04:00
Bd a7ff1387c7
Merge pull request #414 from goodboy/struct_field_filtering
Hide private fields in `Struct.pformat()` output
2026-03-13 19:22:22 -04:00
Gud Boi abbb4a79c8 Drop unused import noticed by `copilot` 2026-03-13 11:52:18 -04:00
Gud Boi 1529095c32 Add `tests/msg/` sub-pkg, audit `pformat()` filtering
Reorganize existing msg-related test suites under
a new `tests/msg/` subdir (matching `tests/devx/`
and `tests/ipc/` convention) and add unit tests for
the `_`-prefixed field filtering in `pformat()`.

Deats,
- `git mv` `test_ext_types_msgspec` and `test_pldrx_limiting` into
  `tests/msg/`.
- add `__init__.py` + `conftest.py` for the new test sub-pkg.
- add new `test_pretty_struct.py` suite with 8 unit tests:
  - parametrized field visibility (public shown, `_`-private hidden,
    mixed)
  - direct `iter_struct_ppfmt_lines()` assertion
  - nested struct recursion filtering
  - empty struct edge case
  - real `MsgDec` via `mk_dec()` hiding `_dec`
  - `repr()` integration via `Struct.__repr__()`

(this patch was generated in some part by [`claude-code`][claude-code-gh])
[claude-code-gh]: https://github.com/anthropics/claude-code
2026-03-12 18:32:31 -04:00
Gud Boi 8215a7ba34 Hide private fields in `Struct.pformat()` output
Skip fields starting with `_` in pretty-printed struct output
to avoid cluttering displays with internal/private state (and/or accessing
private properties which have errors Bp).

Deats,
- add `if k[0] == '_': continue` check to skip private fields
- change nested `if isinstance(v, Struct)` to `elif` since we
  now have early-continue for private fields
- mv `else:` comment to clarify it handles top-level fields
- fix indentation of `yield` statement to only output
  non-private, non-nested fields

(this commit msg was generated in some part by [`claude-code`][claude-code-gh])
[claude-code-gh]: https://github.com/anthropics/claude-code
2026-03-10 00:33:32 -04:00
Bd c1c4d85958
Merge pull request #406 from goodboy/macos_support
macOS support (for the fancy peeps with nice furniture)
2026-03-10 00:28:54 -04:00
Gud Boi 88b084802f Merge `testing-macos` into unified `testing` matrix
Drop the separate `testing-macos` job and add
`macos-latest` to the existing OS matrix; bump
timeout to 16 min to accommodate macOS runs.

(this patch was generated in some part by [`claude-code`][claude-code-gh])
[claude-code-gh]: https://github.com/anthropics/claude-code
2026-03-09 23:28:58 -04:00
Gud Boi bf1dcea9d1 Announce macOS support in `pyproject` and README
- add `"Operating System :: MacOS"` classifier.
- add macOS bullet to README's TODO/status section.

(this patch was generated in some part by [`claude-code`][claude-code-gh])
[claude-code-gh]: https://github.com/anthropics/claude-code
2026-03-09 23:23:58 -04:00
Bd 5c270b89d5
Merge pull request #342 from goodboy/macos_in_ci
Macos in ci
2026-03-09 20:33:38 -04:00
Gud Boi 6ee0149e8d Another cancellation test timeout bump for non-linux 2026-03-09 19:46:42 -04:00
Gud Boi 9c4cd869fb OK-FINE, skip streaming docs example on macos!
It seems something is up with their VM-img or wtv bc i keep increasing
the subproc timeout and nothing is changing. Since i can't try
a `-xlarge` one without paying i'm just muting this test for now.
2026-03-09 19:46:42 -04:00
Gud Boi afd66ce3b7 Final try, drop logging level in streaming example to see if macos can cope.. 2026-03-09 19:46:42 -04:00
Gud Boi f9bdb1b35d Try one more timeout bumps for flaky docs streaming ex.. 2026-03-09 19:46:42 -04:00
Gud Boi d135ce94af Restyle `test_legacy_one_way_streaming` mod
- convert all doc-strings to `'''` multiline style.
- rename `nursery` -> `an`, `n` -> `tn` to match
  project-wide conventions.
- add type annotations to fn params (fixtures, test
  helpers).
- break long lines into multiline style for fn calls,
  assertions, and `parametrize` decorator lists.
- add `ids=` to `@pytest.mark.parametrize`.
- use `'` over `"` for string literals.
- add `from typing import Callable` import.
- drop spurious blank lines inside generators.

(this commit msg was generated in some part by [`claude-code`][claude-code-gh])
[claude-code-gh]: https://github.com/anthropics/claude-code
2026-03-09 19:46:25 -04:00
Gud Boi fb94aa0095 Tidy a typing-typo, add explicit `ids=` for paramed suites 2026-03-09 19:35:47 -04:00
Gud Boi b71e8575e5 Skip a couple more `ctlc` flaking suites 2026-03-09 19:31:16 -04:00
Gud Boi bbc028e84c Increase macos job timeout to 16s 2026-03-09 19:31:16 -04:00
Gud Boi 016306adf5 Allow `ctlcs_bish(<condition-args>)` skipping
Via ensuring `all(mark.args)` on wtv expressions are arg-passed to the
mark decorator; use it to skip the `test_subactor_breakpoint` suite when
`ctlc=True` since it seems too unreliable in CI.
2026-03-09 19:31:15 -04:00
Gud Boi 712c009790 Hike `testdir.spawn()` timeout on non-linux in CI 2026-03-09 19:30:41 -04:00
Gud Boi 79396b4a26 2x the ctl-c loop prompt-timeout for non-linux in CI 2026-03-09 19:30:41 -04:00
Gud Boi 5b2905b702 Xplatform tweaks for `daemon` fixture
There's a very sloppy registrar-actor-bootup syncing approach used in
this fixture (basically just guessing how long to sleep to wait for it
to init and bind the registry socket) using a `global _PROC_SPAWN_WAIT`
that needs to be made more reliable. But, for now i'm just playing along
with what's there to try and make less CI runs flaky by,

- sleeping *another* 1s when run from non-linux CI.
- reporting stdout (if any) alongside stderr on teardown.
- not strictly requiring a `proc.returncode == -2` indicating successful
  graceful cancellation via SIGINT; instead we now error-log and only
  raise the RTE on `< 0` exit code.
  * though i can't think of why this would happen other then an
    underlying crash which should propagate.. but i don't think any test
    suite does this intentionally rn?
  * though i don't think it should ever happen, having a CI run
    "error"-fail bc of this isn't all that illuminating, if there is
    some weird `.returncode == 0` termination case it's likely not
    a failure?

For later, see the new todo list; we should sync to some kind of "ping"
polling of the tpt address if possible which is already easy enough for
TCP reusing an internal closure from `._root.open_root_actor()`.
2026-03-09 19:30:41 -04:00
Gud Boi 776af3fce6 Register our `ctlcs_bish` marker to avoid warnings 2026-03-09 19:30:41 -04:00
Gud Boi 4639685770 Fill out types in `test_discovery` mod 2026-03-09 19:30:41 -04:00
Gud Boi 98a7d69341 Always pre-sleep in `daemon` fixture when in non-linux CI.. 2026-03-09 19:30:41 -04:00
Gud Boi ab6c955949 Lol fine! bump it a bit more XD 2026-03-09 19:30:41 -04:00
Gud Boi a72bb9321e Bleh, bump timeout again for docs-exs suite when in CI 2026-03-09 19:30:41 -04:00
Gud Boi 0e2949ea59 Bump docs-exs subproc timeout, exception log any timeouts 2026-03-09 19:30:41 -04:00
Gud Boi fb73935dbc Add a `test_log` fixture for emitting from *within* test bodies or fixtures 2026-03-09 19:30:41 -04:00
Gud Boi 94dfeb1441 Add delay before root-actor open, macos in CI.. 2026-03-09 19:30:41 -04:00
Gud Boi 9c1bcb23af Skip legacy-one-way suites on non-linux in CI 2026-03-09 19:30:41 -04:00
Gud Boi a1ea373f34 Ok.. try a longer prompt timeout? 2026-03-09 19:30:41 -04:00
Gud Boi e8f3d64e71 Increase prompt timeout for macos in CI 2026-03-09 19:30:41 -04:00
Gud Boi b30faaca82 Adjust debugger test suites for macos
Namely, after trying to get `test_multi_daemon_subactors` to work for
the `ctlc=True` case (for way too long), give up on that (see
todo/comments) and skip it; the normal case works just fine. Also tweak
the `test_ctxep_pauses_n_maybe_ipc_breaks` pattern matching for
non-`'UDS'` per the previous script commit; we can't use UDS alongside
`pytest`'s tmp dir generation, mega lulz.
2026-03-09 19:30:40 -04:00
Gud Boi 51701fc8dc Ok just skip `test_shield_pause` for macos..
Something something the SIGINT handler isn't being swapped correctly?
2026-03-09 19:29:47 -04:00
Gud Boi 7b89204afd Tweak `do_ctlc()`'s `delay` default
To be a null default and set to `0.1` when not passed by the caller so
as to avoid having to pass `0.1` if you wanted the
param-defined-default.

Also,
- in the `spawn()` fixtures's `unset_colors()` closure, add in a masked
  `os.environ['NO_COLOR'] = '1'` since i found it while trying to debug
  debugger tests.
- always return the `child.before` content from `assert_before()`
  helper; again it comes in handy when debugging console matching tests.
2026-03-09 19:29:18 -04:00
Gud Boi 82d02ef404 Lul, never use `'uds'` tpt for macos test-scripts
It's explained in the comment and i really think it's getting more
hilarious the more i learn about the arbitrary limitations of user space
with this tina platform.
2026-03-09 19:29:18 -04:00
Gud Boi b7546fd221 Longer timeout for `test_one_end_stream_not_opened`
On non-linux that is.
2026-03-09 19:29:18 -04:00
Gud Boi 86c95539ca Loosen shml test assert for key shortening on macos 2026-03-09 19:29:18 -04:00
Gud Boi 706a4b761b Add 6sec timeout around `test_simple_rpc` suite for macos 2026-03-09 19:29:18 -04:00
Gud Boi c5af2fa778 Add a `@no_macos` skipif deco 2026-03-09 19:29:18 -04:00
Gud Boi 86489cc453 Use py version in job `name`, consider macos in linux matrix? 2026-03-09 19:29:18 -04:00
Gud Boi 2631fb4ff3 Only run CI on <py3.14 2026-03-09 19:29:18 -04:00
Gud Boi aee86f2544 Run macos job on `uv` and newer `actions@v4` 2026-03-09 19:29:18 -04:00
Tyler Goodlet 83c8a8ad78 Add macos run using only the `trio` spawner 2026-03-09 19:29:18 -04:00
Gud Boi daae196048 Warn if `.ipc._uds.get_peer_pid()` returns null 2026-03-08 19:17:16 -04:00
Gud Boi 70efcb09a0 Slight refinements to `._state.get_rt_dir()`
Per the `copilot` review,
https://github.com/goodboy/tractor/pull/406#pullrequestreview-3893270953

now we also,
- pass `exists_ok=True` to `.mkdir()` to avoid conc races.
- expose `appname: str` param for caller override.
- normalize `subdir` to avoid escaping the base rt-dir location.
2026-03-08 19:17:16 -04:00
Gud Boi a7e74acdff Doc `getsockopt()` args (for macOS)
Per the questionable `copilot` review which is detailed for follow up in
https://github.com/goodboy/tractor/issues/418. These constants are
directly linked from the kernel sources fwiw.
2026-03-08 19:17:16 -04:00
Gud Boi 9c3d3bcec1 Add prompt flush hack for `bash` on macos as well.. 2026-03-08 19:17:16 -04:00
Gud Boi 521fb97fe9 Support UDS on macos (for realz)
Though it was a good (vibed) try by @dnks, the previous "fix" was not
actually adding unix socket support but merely sidestepping a crash due
to `get_peer_info()`'s impl never going to work on MacOS (and it was
never intended to).

This patch instead solves the underlying issue by implementing a new
`get_peer_pid()` helper which does in fact retrieve the peer's PID in
a more generic/cross-platform way (:fingers_crossed:); much thanks to
the linked SO answer for this solution!

Impl deats,
- add `get_peer_pid()` and call it from
  `MsgpackUDSStream.get_stream_addrs()` when we detect a non-'linux'
  platform, OW use the original soln: `get_stream_addrs()`.
- add a new case for the `match (peername, sockname)` with a
  `case (str(), str()):` which seems to at least work on macos.
- drop all the `LOCAL_PEERCRED` dynamic import branching since it was
  never needed and was never going to work.
2026-03-08 19:17:16 -04:00
Gud Boi d8a3969048 Also shorten shm-key for `ShmList` on macos
Same problem as for the `ShmArray` tokens, so tweak and reuse
the `_shorten_key_for_macos()` helper and call it from
`open_shm_list()` similarly.

Some tweaks/updates to the various helpers,
- support `prefix/suffix` inputs and if provided take their lengths and
  subtract them from the known *macOS shm_open() has a 31 char limit
  (PSHMNAMLEN)* when generating and using the `hashlib.sha256()` value
  which overrides (for now..) wtv `key` is passed by the caller.
- pass the appropriate `suffix='_first/_last'` values for the `ShmArray`
  token generators case.
- add a `prefix: str = 'shml_'` param to `open_shm_list()`.
- better log formatting with `!r` to report any key shortening.
2026-03-08 19:17:16 -04:00
Gud Boi 01c0db651a Port macOS shm 31-char name limit hack from `piker`
Adapt the `PSHMNAMLEN` fix from `piker.data._sharedmem` (orig commit
96fb79ec thx @dnks!) to `tractor.ipc._shm` accounting for the
module-local differences:

- Add `hashlib` import for sha256 key hashing
- Add `key: str|None` field to `NDToken` for storing
  the original descriptive key separate from the
  (possibly shortened) OS-level `shm_name`
- Add `__eq__()`/`__hash__()` to `NDToken` excluding
  the `key` field from identity comparison
- Add `_shorten_key_for_macos()` using `t_` prefix
  (vs piker's `p_`) with 16 hex chars of sha256
- Use `platform.system() == 'Darwin'` in `_make_token()`
  (tractor already imports the `platform` module vs
  piker's `sys.platform`)
- Wrap `shm_unlink()` in `ShmArray.destroy()` with
  `try/except FileNotFoundError` for teardown races
  (was already done in `SharedInt.destroy()`)
- Move token creation before `SharedMemory()` alloc in
  `open_shm_ndarray()` so `token.shm_name` is used
  as the OS-level name
- Use `lookup_key` pattern in `attach_shm_ndarray()`
  to decouple `_known_tokens` dict key from OS name

(this patch was generated in some part by [`claude-code`][claude-code-gh])
[claude-code-gh]: https://github.com/anthropics/claude-code
2026-03-08 19:17:16 -04:00
Gud Boi 7bcd7aca2b Reorg `socket` conditional imports a bit
Move the multi-platorm-supporting conditional/dynamic `socket` constant
imports to *after* the main cross-platform ones.
Also add constant typing and reformat comments a bit for the macOS case.
2026-03-08 19:17:16 -04:00
Gud Boi 920d0043b4 Force parent subdirs for macos 2026-03-08 19:17:16 -04:00
wygud 93b9a6cd97 Add macOS compatibility for Unix socket credential passing
Make socket credential imports platform-conditional in `.ipc._uds`.
- Linux: use `SO_PASSCRED`/`SO_PEERCRED` from socket module
- macOS: use `LOCAL_PEERCRED` (0x0001) instead, no need for `SO_PASSCRED`
- Conditionally call `setsockopt(SO_PASSCRED)` only on Linux

Fixes AttributeError on macOS where SO_PASSCRED doesn't exist.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2026-03-08 19:17:16 -04:00
Tyler Goodlet e7cefba67f Use `platformdirs` for `.config.get_rt_dir()`
Thanks to the `tox`-dev community for such a lovely pkg which seems to
solves all the current cross-platform user-dir problems B)

Also this,
- now passes `platformdirs.user_runtime_dir(appname='tractor')`
  and allows caller to pass an optional `subdir` under `tractor/`
  if desired.
- drops the `.config._rtdir: Path` mod var.
- bumps the lock file with the new dep.
2026-03-08 19:16:49 -04:00
Bd 683476cc96
Merge pull request #421 from goodboy/py_pkging_update
Py pkging and version support update, go 3.12 and 3.13
2026-03-08 19:14:39 -04:00
Gud Boi ad24df0ed7 Drop `pytest.ini`, now covered in `pyproject.toml` 2026-03-08 18:58:30 -04:00
Gud Boi a1622c0b94 Bump `ruff.toml` to target py313 2026-03-08 18:39:08 -04:00
Gud Boi a385d20810 Disable the `xonsh` autoloaded `pytest` plugin 2026-03-08 18:38:31 -04:00
Gud Boi 7f9044c1ef Bump pkg classifiers to match py versions range 2026-03-08 18:16:30 -04:00
Gud Boi d0618e3cb4 Pin to py<3.14 (particularly for macos) 2026-03-08 18:16:30 -04:00
Gud Boi a5bebf76d5 Pin to py-3.12+ and pin-up some deps
Namely to get a fix (i patched in) to `pdbp` and the latest
prompt-injection feats from `xonsh` B)

Also leave in a (masked) `.uv.sources.pdbp` section for easy
patch-test-submit in the future from my own fork and bump the lock file
to match!
2026-03-08 18:16:21 -04:00
Bd 814b2e7e62
Merge pull request #416 from goodboy/claudy_skillz
Claudy skillz: kicking off some `claude` code skill-files
2026-03-04 21:36:49 -05:00
Gud Boi 1704f73504 Add local `claude` settings for commit-msg perms
Auto-allow the tool calls used by the `/commit-msg` skill
so the workflow requires minimal user prompting.

(this patch was generated in some part by [`claude-code`][claude-code-gh])
[claude-code-gh]: https://github.com/anthropics/claude-code
2026-03-04 16:09:37 -05:00
Gud Boi c735fc8544 Update `.gitignore` for `gish` local files
Rename `gitea/` comment to `gish`, add `gh/` ignore, and
expand TODOs about syncing with git hosting services.
Also mute any `.claude/*_commit*.txt` files.

(this commit msg was generated in some part by [`claude-code`][claude-code-gh])
[claude-code-gh]: https://github.com/anthropics/claude-code
2026-03-04 16:06:27 -05:00
Gud Boi c5ea6040bf Improve `/commit-msg` skill fmting + consistency
Align `SKILL.md` and `style-guide-reference.md` with the
claude-code skills docs and actual usage conventions.

Deats,
- add missing frontmatter fields: `argument-hint`,
  `disable-model-invocation`, and scope `Bash` tool to
  `Bash(git *)` prefix pattern per docs.
- add `Grep`/`Glob` to `allowed-tools`.
- restructure `!`backtick`` usage for proper dynamic
  context injection (not mixed with instructional text).
- use markdown link for `style-guide-reference.md` ref
  per docs' supporting-files convention.
- switch timestamp format to cross-OS-safe
  `date -u +%Y%m%dT%H%M%SZ`.

Also,
- rm errant blank lines between footer attribution and
  reference-def lines in both files.
- fix double space in style guide subject length desc.
- reflow long lines for 67 char limit.

(this patch was generated in some part by [`claude-code`][claude-code-gh])
[claude-code-gh]: https://github.com/anthropics/claude-code
2026-03-04 15:05:17 -05:00
Gud Boi d4f2fa547a Extend `.gitignore` for dev/gen artifacts
Add ignore rules for commit-msg gen tmp files, nix dev
profiles, vim sessions, macOS metadata, gitea local docs,
and private LLM conversation logs.

(this commit msg was generated in some part by [`claude-code`][claude-code-gh])
[claude-code-gh]: https://github.com/anthropics/claude-code
2026-03-01 17:46:03 -05:00
Gud Boi 20896bfbab Add `commit-msg` skill + style guide reference
Add a `claude-code` skill for generating commit msgs
matching `tractor`'s style, plus a companion style guide
derived from analysis of 500 repo commits.

Deats,
- `SKILL.md` defines the `commit-msg` skill with YAML
  frontmatter, outlining the generation process, formatting
  rules, and footer conventions.
- `style-guide-reference.md` documents verb frequencies,
  backtick usage patterns, section markers, abbreviations,
  and tone from historical commit analysis.

(this patch was generated in some part by [`claude-code`][claude-code-gh])
[claude-code-gh]: https://github.com/anthropics/claude-code
2026-03-01 15:46:27 -05:00
Bd 70bb77280e
Merge pull request #411 from goodboy/tpt_tolerance
Tpt-tolerance: more lowlevel `trio` CRE/BRE -> `TransportClosed` translations
2026-02-19 16:40:17 -05:00
Gud Boi 916f88a070 Less newlines in `._rpc` log msg 2026-02-19 16:31:54 -05:00
Gud Boi 91f2f3ec10 Use test-harness `loglevel` in inter-peer suite 2026-02-19 16:29:20 -05:00
Tyler Goodlet 3e5124e184 Hide `._rpc._invoke()` frame, again.. 2026-02-19 16:28:22 -05:00
Gud Boi fa86269e30 Stuff from auto-review in https://github.com/goodboy/tractor/pull/412 .. 2026-02-19 16:20:21 -05:00
Gud Boi d0b92bbeba Clean up `._transport` error-case comment
Expand and clarify the comment for the default `case _`
block in the `.send()` error matcher, noting that we
console-error and raise-thru for unexpected disconnect
conditions.

(this patch was suggested by copilot in,
 https://github.com/goodboy/tractor/pull/411)

(this commit msg was generated in some part by [`claude-code`][claude-code-gh])
[claude-code-gh]: https://github.com/anthropics/claude-code
2026-02-19 16:18:39 -05:00
Gud Boi 9470815f5a Fix `spawn` fixture cleanup + test assertions
Improve the `spawn` fixture teardown logic in
`tests/devx/conftest.py` fixing the while-else bug, and fix
`test_advanced_faults` genexp for `TransportClosed` exc type
checking.

Deats,
- replace broken `while-else` pattern with direct
  `if ptyproc.isalive()` check after the SIGINT loop.
- fix undefined `spawned` ref -> `ptyproc.isalive()` in
  while condition.
- improve walrus expr formatting in timeout check (multiline
  style).

Also fix `test_ipc_channel_break_during_stream()` assertion,
- wrap genexp in `all()` call so it actually checks all excs
  are `TransportClosed` instead of just creating an unused
  generator.

(this patch was suggested by copilot in,
 https://github.com/goodboy/tractor/pull/411)

(this commit msg was generated in some part by [`claude-code`][claude-code-gh])
[claude-code-gh]: https://github.com/anthropics/claude-code
2026-02-19 16:14:11 -05:00
Gud Boi 592d918394 Tweak `test_inter_peer_cancellation` for races
Adjust `basic_echo_server()` default sequence len to avoid the race
where the 'tell_little_bro()` finished streaming **before** the
echo-server sub is cancelled by its peer subactor (which is the whole
thing we're testing!).

Deats,
- bump `rng_seed` default from 50 -> 100 to ensure peer
  cancel req arrives before echo dialog completes on fast hw.
- add `trio.sleep(0.001)` between send/receive in msg loop on the
  "client" streamer side to give cancel request transit more time to
  arrive.

Also,
- add more native `tractor`-type hints.
- reflow `basic_echo_server()` doc-string for 67 char limit
- add masked `pause()` call with comment about unreachable
  code path
- alphabetize imports: mv `current_actor` and `open_nursery`
  below typed imports

(this commit msg was generated in some part by [`claude-code`][claude-code-gh])
[claude-code-gh]: https://github.com/anthropics/claude-code
2026-02-19 15:24:42 -05:00
Gud Boi 0cddc67bdb Add doc-strs to `get_root()` + `maybe_open_portal()`
Brief descriptions for both fns in `._discovery` clarifying
what each delivers and under what conditions.

(this commit msg was generated in some part by [`claude-code`][claude-code-gh])
[claude-code-gh]: https://github.com/anthropics/claude-code
2026-02-19 13:55:02 -05:00
Gud Boi 052fe2435f Improve `Channel` doc-strs + minor cleanups
Flesh out missing method doc-strings, improve log msg formatting and
assert -> `RuntimeError` for un-inited tpt layer.

Deats,
- add doc-string to `.send()` noting `TransportClosed` raise
  on comms failures.
- add doc-string to `.recv()`.
- expand `._aiter_msgs()` doc-string, line-len reflow.
- add doc-string to `.connected()`.
- convert `assert self._transport` -> `RuntimeError` raise
  in `._aiter_msgs()` for more explicit crashing.
- expand `_connect_chan()` doc-string, note it's lowlevel
  and suggest `.open_portal()` to user instead.
- factor out `src_exc_str` in `TransportClosed` log handler
  to avoid double-call
- use multiline style for `.connected()` return expr.

(this commit msg was generated in some part by [`claude-code`][claude-code-gh])
[claude-code-gh]: https://github.com/anthropics/claude-code
2026-02-19 13:55:02 -05:00
Gud Boi 28819bf5d3 Add `Actor.is_root()` convenience predicate meth 2026-02-19 13:55:02 -05:00
Gud Boi 07c2ba5c0d Drop `trio`-exc-catching if tpt-closed covers them
Remove the `trio.ClosedResourceError` and `trio.BrokenResourceError`
handling that should now be subsumed by `TransportClosed` re-raising out
of the `.ipc` stack.

Deats,
- drop CRE and BRE from `._streaming.MsgStream.aclose()/.send()` blocks.
- similarly rm from `._context.open_context_from_portal()`.
- also from `._portal.Portal.cancel_actor()` and drop the
  (now-completed-todo) comment about this exact thing.

Also add comment in `._rpc.try_ship_error_to_remote()` noting the
remaining `trio` catches there are bc the `.ipc` layers *should* be
wrapping them; thus `log.critical()` use is warranted.

(this commit msg was generated in some part by [`claude-code`][claude-code-gh])
[claude-code-gh]: https://github.com/anthropics/claude-code
2026-02-19 13:55:02 -05:00
Gud Boi 50f40f427b Include `TransportClosed` in tpt-layer err handling
Add `TransportClosed` to except clauses where `trio`'s own
resource-closed errors are already caught, ensuring our
higher-level tpt exc is also tolerated in those same spots.
Likely i will follow up with a removal of the `trio` variants since most
*should be* caught and re-raised as tpt-closed out of the `.ipc` stack
now?

Add `TransportClosed` to various handler blocks,
- `._streaming.MsgStream.aclose()/.send()` except blocks.
- the broken-channel except in `._context.open_context_from_portal()`.
- obvi import it where necessary in those ^ mods.

Adjust `test_advanced_faults` suite + exs-script to match,
- update `ipc_failure_during_stream.py` example to catch
  `TransportClosed` alongside `trio.ClosedResourceError`
  in both the break and send-check paths.
- shield the `trio.sleep(0.01)` after tpt close in example to avoid
  taskc-raise/masking on that checkpoint since we want to simulate
  waiting for a user to send a KBI.
- loosen `ExceptionGroup` assertion to `len(excs) <= 2` and ensure all
  excs are `TransportClosed`.
- improve multi-line formatting, minor style/formatting fixes in
  condition expressions.

(this commit msg was generated in some part by [`claude-code`][claude-code-gh])
[claude-code-gh]: https://github.com/anthropics/claude-code
2026-02-19 13:55:02 -05:00
Gud Boi bf6de55865 Improve tpt-closed msg-fmt/content and CRE case matching
Refine tpt-error reporting to include closure attribution (`'locally'`
vs `'by peer'`), tighten match conditions and reduce needless newlines
in exc reprs.

Deats,
- factor out `trans_err_msg: str` and `by_whom: str` into a `dict`
  lookup before the `match:` block to pair specific err msgs to closure
  attribution strings.
- use `by_whom` directly as `CRE` case guard condition
  (truthy when msg matches known underlying CRE msg content).
- conveniently include `by_whom!r` in `TransportClosed` message.
- fix `'locally ?'` -> `'locally?'` in send-side `CRE`
  handler (drop errant space).
- add masked `maybe_pause_bp()` calls at both `CRE` sites (from when
  i was tracing a test harness issue where the UDS socket path wasn't
  being cleaned up on teardown).
- drop trailing `\n` from `body=` args to `TransportClosed`.
- reuse `trans_err_msg` for the `BRE`/broken-pipe guard.

Also adjust testing, namely `test_ctxep_pauses_n_maybe_ipc_breaks`'s
expected patts-set for new msg formats to be raised out of
`.ipc._transport`.

(this commit msg was generated in some part by [`claude-code`][claude-code-gh])
[claude-code-gh]: https://github.com/anthropics/claude-code
2026-02-19 13:55:02 -05:00
Gud Boi 5ded99a886 Add a `._trace.maybe_pause_bp()` for tpt-broken cases
Internal helper which falls back to sync `pdb` when the
child actor can't reach root to acquire the TTY lock.

Useful when debugging tpt layer failures (intentional or
otherwise) where a sub-actor can no longer IPC-contact the
root to coordinate REPL access; root uses `.pause()` as
normal while non-root falls back to `mk_pdb().set_trace()`.

(this commit msg was generated in some part by [`claude-code`][claude-code-gh])
[claude-code-gh]: https://github.com/anthropics/claude-code
2026-02-19 13:55:02 -05:00
Gud Boi 7145fa364f Add `SIGINT` cleanup to `spawn` fixture in `devx/conftest`
Convert `spawn` fixture to a generator and add post-test graceful
subproc cleanup via `SIGINT`/`SIGKILL` to avoid leaving stale `pexpect`
child procs around between test runs as well as any UDS-tpt socket files
under the system runtime-dir.

Deats,
- convert `return _spawn` -> `yield _spawn` to enable
  post-yield teardown logic.
- add a new `nonlocal spawned` ref so teardown logic can access the last
  spawned child from outside the delivered spawner fn-closure.
- add `SIGINT`-loop after yield with 5s timeout, then
  `SIGKILL` if proc still alive.
- add masked `breakpoint()` and TODO about UDS path cleanup

(this commit msg was generated in some part by [`claude-code`][claude-code-gh])
[claude-code-gh]: https://github.com/anthropics/claude-code
2026-02-19 13:55:02 -05:00
Gud Boi f8e25688c7 Unmask `ClosedResourceError` handling in `._transport`
Unmask the CRE case block for peer-closed socket errors which already
had a TODO about reproducing the condition. It appears this case can
happen during inter-actor comms teardowns in `piker`, but i haven't been
able to figure out exactly what reproduces it yet..

So activate the block again for that 'socket already closed'-msg case,
and add a TODO questioning how to reproduce it.

(this commit msg was generated in some part by [`claude-code`][claude-code-gh])
[claude-code-gh]: https://github.com/anthropics/claude-code
2026-02-12 00:51:50 -05:00
Tyler Goodlet c3f455a8ec Mask tpt-closed handling of `chan.send(return_msg)`
A partial revert of commit c05d08e426 since it seem we already
suppress tpt-closed errors lower down in `.ipc.Channel.send()`; given
that i'm pretty sure this new handler code should basically never run?

Left in a todo to remove the masked content once i'm done more
thoroughly testing under `piker`.
2026-02-12 00:51:50 -05:00
Tyler Goodlet f78e842fba More `TransportClosed`-handling around IPC-IO
For IPC-disconnects-during-teardown edge cases, augment some `._rpc`
machinery,
- in `._invoke()` around the `await chan.send(return_msg)` where we
  suppress if the underlying `Channel` already disconnected.
- add a disjoint handler in `_errors_relayed_via_ipc()` which just
  reports-n-reraises the exc (same as prior behaviour).
  * originally i thought it needed to be handled specially (to avoid
    being crash handled) but turns out that isn't necessary?
  * hence the also-added-bu-masked-out `debug_filter` / guard expression
    around the `await debug._maybe_enter_pm()` line.
- show the `._invoke()` frame for the moment.
2026-02-12 00:51:50 -05:00
Bd 3638b80c9d
Merge pull request #412 from goodboy/root_actor_raddrs_fix
Non-registrar, root actor `_root_addrs` runtime-vars fix
2026-02-12 00:49:40 -05:00
Gud Boi 2ed9e65530 Clear rtvs state on root shutdown..
Fixes the bug discovered in last test update, not sure how this wasn't
caught already XD
2026-02-11 22:17:26 -05:00
Gud Boi 6cab363c51 Catch-n-fail on stale `_root_addrs` state..
Turns out we aren't clearing the `._state._runtime_vars` entries in
between `open_root_actor` calls.. This test refinement catches that by
adding runtime-vars asserts on the expected root-addrs value; ensure
`_runtime_vars['_root_addrs'] ONLY match the values provided by the
test's CURRENT root actor.

This causes a failure when the (just added)
`test_non_registrar_spawns_child` is run as part of the module suite,
it's fine when run standalone.
2026-02-11 22:17:26 -05:00
Gud Boi 8aee24e83f Fix when root-actor addrs is set as rtvs
Move `_root_addrs` assignment to after `async_main()` unblocks (via
`.started()`) which now delivers the bind addrs , ensuring correct
`UnwrappedAddress` propagation into `._state._runtime_vars` for
non-registar root actors..

Previously for non-registrar root actors the `._state._runtime_vars`
entries were being set as `Address` values which ofc IPC serialize
incorrectly rn vs. the unwrapped versions, (well until we add a msgspec
for their structs anyway) and thus are passed in incorrect form to
children/subactors during spawning..

This fixes the issue by waiting for the `.ipc.*` stack to
bind-and-resolve any randomly allocated addrs (by the OS) until after
the initial `Actor` startup is complete.

Deats,
- primarily, mv `_root_addrs` assignment from before `root_tn.start()`
  to after, using started(-ed) `accept_addrs` now delivered from
  `._runtime.async_main()`..
- update `task_status` type hints to match.
- unpack and set the `(accept_addrs, reg_addrs)` tuple from
  `root_tn.start()` call into `._state._runtime_vars` entries.
- improve and embolden comments distinguishing registrar vs non-registrar
  init paths, ensure typing reflects wrapped vs. unwrapped addrs.

Also,
- add a masked `mk_pdb().set_trace()` for debugging `raddrs` values
  being "off".
- add TODO about using UDS on linux for root mailbox
- rename `trans_bind_addrs` -> `tpt_bind_addrs` for clarity.
- expand comment about random port allocation for
  non-registrar case

(this commit msg was generated in some part by [`claude-code`][claude-code-gh])
[claude-code-gh]: https://github.com/anthropics/claude-code
2026-02-11 22:17:26 -05:00
Gud Boi cdcc1b42fc Add test for non-registrar root sub-spawning
Ensure non-registrar root actors can spawn children and that
those children receive correct parent contact info. This test
catches the bug reported in,

https://github.com/goodboy/tractor/issues/410

Add new `test_non_registrar_spawns_child()` which spawns a sub-actor
from a non-registrar root and verifies the child can manually connect
back to its parent using `get_root()` API, auditing
`._state._runtime_vars` addr propagation from rent to child.

Also,
- improve type hints throughout test suites
  (`subprocess.Popen`, `UnwrappedAddress`, `Aid` etc.)
- rename `n` -> `an` for actor nursery vars
- use multiline style for function signatures

(this commit msg was generated in some part by [`claude-code`][claude-code-gh])
[claude-code-gh]: https://github.com/anthropics/claude-code
2026-02-11 22:17:26 -05:00
Bd 51ac0c623e
Merge pull request #402 from goodboy/log_sys_testing
Log sys testing, starting to get "serious" about it..
2026-02-11 22:13:17 -05:00
Gud Boi 3f0bde1bf8 Use bare `get_logger()` in `.to_asyncio` 2026-02-11 22:02:41 -05:00
Gud Boi fa1a15dce8 Cleaups per copilot PR review 2026-02-11 21:51:40 -05:00
Gud Boi 5850844297 Mk `test_implicit_mod_name_applied_for_child()` check init-mods
Test pkg-level init module and sub-pkg module logger naming
to better validate auto-naming logic.

Deats,
- create `pkg_init_mod` and write `mod_code` to it for
  testing pkg-level `__init__.py` logger instance creation.
  * assert `snakelib.__init__` logger name is `proj_name`.
- write `mod_code` to `subpkg/__init__.py`` as well and check the same.

Also,
- rename some vars,
  * `pkg_mod` -> `pkg_submod`,
  * `pkgmod` -> `subpkgmod`
- add `ModuleType` import for type hints
- improve comments explaining pkg init vs first-level
  sub-module naming expectations.
- drop trailing whitespace and unused TODO comment
- remove masked `breakpoint()` call

(this commit msg was generated in some part by [`claude-code`][claude-code-gh])
[claude-code-gh]: https://github.com/anthropics/claude-code
2026-02-11 21:43:37 -05:00
Gud Boi ff02939213 Toss in some `colorlog` alts to try 2026-02-11 21:05:16 -05:00
Gud Boi d61e8caab2 Improve `test_log_sys` for new auto-naming logic
Add assertions and comments to better test the reworked
implicit module-name detection in `get_logger()`.

Deats,
- add `assert not tractor.current_actor()` check to verify
  no runtime is active during test.
- import `.log` submod directly for use.
- add masked `breakpoint()` for debugging mod loading.
- add comment about using `ranger` to inspect `testdir` layout
  of auto-generated py pkg + module-files.
- improve comments explaining pkg-root-log creation.
- add TODO for testing `get_logger()` call from pkg
  `__init__.py`
- add comment about first-pkg-level module naming.

(this commit msg was generated in some part by [`claude-code`][claude-code-gh])
[claude-code-gh]: https://github.com/anthropics/claude-code
2026-02-11 21:05:07 -05:00
Gud Boi 0b0c83e9da Drop `name=__name__` from all `get_logger()` calls
Use new implicit module-name detection throughout codebase to simplify
logger creation and leverage auto-naming from caller mod .

Main changes,
- drop `name=__name__` arg from all `get_logger()` calls
  (across 29 modules).
- update `get_console_log()` calls to include `name='tractor'` for
  enabling root logger in test harness and entry points; this ensures
  logic in `get_logger()` triggers so that **all** `tractor`-internal
  logging emits to console.
- add info log msg in test `conftest.py` showing test-harness
  log level

Also,
- fix `.actor.uid` ref to `.actor.aid.uid` in `._trace`.
- adjust a `._context` log msg formatting for clarity.
- add TODO comments in `._addr`, `._uds` for when we mv to
  using `multiaddr`.
- add todo for `RuntimeVars` type hint TODO in `.msg.types` (once we
  eventually get that all going obvi!)

(this commit msg was generated in some part by [`claude-code`][claude-code-gh])
[claude-code-gh]: https://github.com/anthropics/claude-code
2026-02-11 21:04:49 -05:00
Gud Boi 5e7c0f264d Rework `.get_logger()`, better autonaming, deduping
Overhaul of the automatic-calling-module-name detection and sub-log
creation logic to avoid (at least warn) on duplication(s) and still
handle the common usage of a call with `name=__name__` from a mod's top
level scope.

Main `get_logger()` changes,
- refactor auto-naming logic for implicit `name=None` case such that we
  handle at least `tractor` internal "bare" calls from internal submods.
- factor out the `get_caller_mod()` closure (still inside
  `get_logger()`)for introspecting caller's module with configurable
  frame depth.
- use `.removeprefix()` instead of `.lstrip()` for stripping pkg-name
  from mod paths
- mv root-logger creation before sub-logger name processing
- improve duplicate detection for `pkg_name` in `name`
- add `_strict_debug=True`-only-emitted warnings for duplicate
  pkg/leaf-mod names.
- use `print()` fallback for warnings when no actor runtime is up at
  call time.

Surrounding tweaks,
- add `.level` property to `StackLevelAdapter` for getting
  current emit level as lowercase `str`.
- mv `_proj_name` def to just above `get_logger()`
- use `_curr_actor_no_exc` partial in `_conc_name_getters`
  to avoid runtime errors
- improve comments/doc-strings throughout
- keep some masked `breakpoint()` calls for future debugging

(this commit msg was generated in some part by [`claude-code`][claude-code-gh])
[claude-code-gh]: https://github.com/anthropics/claude-code
2026-02-11 21:04:29 -05:00
Gud Boi edf1189fe0 Multi-line styling in `test.devx.conftest` 2026-02-11 21:04:22 -05:00
Tyler Goodlet de24bfe052 Mv `load_module_from_path()` to a new `._code_load` submod 2026-02-11 21:03:29 -05:00
Tyler Goodlet e235b96894 Use new `pkg_name` in log-sys test suites 2026-02-11 21:03:07 -05:00
Tyler Goodlet dea4b9fd93 Implicitly name sub-logs by caller's mod
That is when no `name` is passed to `get_logger()`, try to introspect
the caller's `module.__name__` and use it to infer/get the "namespace
path" to that module the same as if using `name=__name__` as in the most
common usage.

Further, change the `_root_name` to be `pkg_name: str`, a public and
more obvious param name, and deprecate the former. This obviously adds
the necessary impl to make the new
`test_sys_log::test_implicit_mod_name_applied_for_child` test pass.

Impl detalles for `get_logger()`,
- add `pkg_name` and deprecate `_root_name`, include failover logic
  and a warning.
- implement calling module introspection using
  `inspect.stack()/getmodule()` to get both the `.__name__` and
  `.__package__` info alongside adjusted logic to set the `name`
  when not provided but only when a new `mk_sublog: bool` is set.
- tweak the `name` processing for implicitly set case,
  - rename `sub_name` -> `pkg_path: str` which is the path
    to the calling module minus that module's name component.
  - only partition `name` if `pkg_name` is `in` it.
  - use the `_root_log` for `pkg_name` duplication warnings.

Other/related,
- add types to various public mod vars missing them.
- rename `.log.log` -> `.log._root_log`.
2026-02-11 21:03:07 -05:00
Tyler Goodlet 557e2cec6a Add an implicit-pkg-path-as-logger-name test
A bit of test driven dev to anticipate support  of `.log.get_logger()`
usage such that it can be called from arbitrary sub-modules, themselves
embedded in arbitrary sub-pkgs, of some project; the when not provided,
the `sub_name` passed to the `Logger.getChild(<sub_name>)` will be set
as the sub-pkg path "down to" the calling module.

IOW if you call something like,

`log = tractor.log.get_logger(pkg_name='mypylib')`

from some `submod.py` in a project-dir that looks like,

mypylib/
  mod.py
  subpkg/
    submod.py  <- calling module

the `log: StackLevelAdapter` child-`Logger` instance will have a
`.name: str = 'mypylib.subpkg'`, discluding the `submod` part since this
already rendered as the `{filename}` header in `log.LOG_FORMAT`.

Previously similar behaviour would be obtained by passing
`get_logger(name=__name__)` in the calling module and so much so it
motivated me to make this the default, presuming we can introspect for
the info.

Impl deats,
- duplicated a `load_module_from_path()` from `modden` to load the
  `testdir` rendered py project dir from its path.
 |_should prolly factor it down to this lib anyway bc we're going to
   need it for hot code reload? (well that and `watchfiles` Bp)
- in each of `mod.py` and `submod.py` render the `get_logger()` code
  sin `name`, expecting the (coming shortly) implicit introspection
  feat to do this.
- do `.name` and `.parent` checks against expected sub-logger values
  from `StackLevelAdapter.logger.getChildren()`.
2026-02-11 21:03:07 -05:00
Tyler Goodlet 0e3229f16d Start a logging-sys unit-test module
To start ensuring that when `name=__name__` is passed we try to
de-duplicate the `_root_name` and any `leaf_mod: str` since it's already
included in the headers as `{filename}`.

Deats,
- heavily document the de-duplication `str.partition()`s in
  `.log.get_logger()` and provide the end fix by changing the predicate,
  `if rname == 'tractor':` -> `if rname == _root_name`.
  * also toss in some warnings for when we still detect duplicates.
- add todo comments around logging "filters" (vs. our "adapter").
- create the new `test_log_sys.test_root_pkg_not_duplicated()` which
  runs green with the fixes from ^.
- add a ton of test-suite todos both for existing and anticipated
  logging sys feats in the new mod.
2026-02-11 21:03:07 -05:00
Bd 448d25aef4
Merge pull request #409 from goodboy/nixos_flake
Nixos flake, for the *too-hip-for-arch-ers*
2026-02-11 21:02:37 -05:00
Gud Boi 343c9e0034 Tweaks per the `copilot` PR review 2026-02-11 20:55:08 -05:00
Gud Boi 1dc27c5161 Add a dev-overlay nix flake
Based on the impure template from `pyproject.nix` and providing
a dev-shell for easy bypass-n-hack on nix(os) using `uv`.

Deats,
- include bash completion pkgs for devx/happiness.
- pull `ruff` from <nixpkgs> to avoid wheel (build) issues.
- pin to py313 `cpython` for now.
2026-01-23 16:27:19 -05:00
Gud Boi 14aefa4b11 Reorg dev deps into nested groups
Namely,
- `devx` for console debugging extras used in `tractor.devx`.
- `repl` for @goodboy's `xonsh` hackin utils.
- `testing` for harness stuffs.
- `lint` for whenever we start doing that; it requires special
  separation on nixos in order to pull `ruff` from pkgs.

Oh and bump the lock file.
2026-01-23 16:24:24 -05:00
105 changed files with 5193 additions and 1377 deletions

View File

@ -0,0 +1,15 @@
{
"permissions": {
"allow": [
"Write(.claude/*commit_msg*)",
"Write(.claude/git_commit_msg_LATEST.md)",
"Bash(date *)",
"Bash(cp .claude/*)",
"Bash(git diff *)",
"Bash(git log *)",
"Bash(git status)"
],
"deny": [],
"ask": []
}
}

View File

@ -0,0 +1,86 @@
---
name: commit-msg
description: >
Generate git commit messages following project style. Use when user
wants to create a commit or asks for a commit message.
argument-hint: "[optional-scope-or-description]"
disable-model-invocation: true
allowed-tools:
- Bash(git *)
- Bash(date *)
- Bash(cp *)
- Read
- Grep
- Glob
- Write
---
When generating commit messages, always follow this process:
0. **Check for staged changes**: if `git diff --staged` is
empty, STOP and tell the user "nothing is staged!" with
a reminder to `git add` before invoking this skill.
1. **Gather context** from the staged diff and recent
history:
- Staged changes: !`git diff --staged --stat`
- Recent commits: !`git log --oneline -5`
2. **Analyze the diff**: understand what files changed and
the nature of the changes (new feature, bug fix, refactor,
etc.)
3. **Write the commit message** following these rules:
**Use the accompanying style guide:**
- See [style-guide-reference.md](style-guide-reference.md)
for detailed analysis of past commits in this repo.
**Subject Line Format:**
- Present tense verbs: Add, Drop, Fix, Use, Move, Adjust, etc.
- Target 50 chars (hard max: 67)
- Backticks around ALL code elements (classes, functions, modules, vars)
- Specific about what changed
**Body Format (optional - keep simple if warranted):**
- Max 67 char line length
- Use `-` bullets for lists
- Section markers: `Also,` `Deats,` `Other,` `Further,`
- Abbreviations: msg, bg, ctx, impl, mod, obvi, tn, fn, bc, var, prolly
- Casual yet technically precise tone
- Never write lines with only whitespace
**Common Opening Patterns:**
- New features: "Add `feature` to `module`"
- Removals: "Drop `attr` from `class`"
- Bug fixes: "Fix `issue` in `function`"
- Code moves: "Move `thing` to `new_location`"
- Adoption: "Use `new_tool` for `task`"
- Minor tweaks: "Adjust `behavior` in `component`"
4. **Write to TWO files**:
- `.claude/<timestamp>_<hash>_commit_msg.md`
* with `<timestamp>` from `date -u +%Y%m%dT%H%M%SZ` or similar
filesystem-safe format.
* and `<hash>` from `git log -1 --format=%h` first 7 chars.
- `.claude/git_commit_msg_LATEST.md` (overwrite)
5. **Always include credit footer**:
When no part of the patch was written by `claude`,
```
(this commit msg was generated in some part by [`claude-code`][claude-code-gh])
[claude-code-gh]: https://github.com/anthropics/claude-code
```
when some or all of the patch was written by `claude`
instead use,
```
(this patch was generated in some part by [`claude-code`][claude-code-gh])
[claude-code-gh]: https://github.com/anthropics/claude-code
```
Keep it concise. Match the tone of recent commits. For simple
changes, use subject line only.

View File

@ -0,0 +1,225 @@
# Commit Message Style Guide for `tractor`
Analysis based on 500 recent commits from the `tractor` repository.
## Core Principles
Write commit messages that are technically precise yet casual in
tone. Use abbreviations and informal language while maintaining
clarity about what changed and why.
## Subject Line Format
### Length and Structure
- Target: ~50 chars with a hard-max of 67.
- Use backticks around code elements (72.2% of commits)
- Rarely use colons (5.2%), except for file prefixes
- End with '?' for uncertain changes (rare: 0.8%)
- End with '!' for important changes (rare: 2.0%)
### Opening Verbs (Present Tense)
Most common verbs from analysis:
- `Add` (14.4%) - wholly new features/functionality
- `Use` (4.4%) - adopt new approach/tool
- `Drop` (3.6%) - remove code/feature
- `Fix` (2.4%) - bug fixes
- `Move`/`Mv` (3.6%) - relocate code
- `Adjust` (2.0%) - minor tweaks
- `Update` (1.6%) - enhance existing feature
- `Bump` (1.2%) - dependency updates
- `Rename` (1.2%) - identifier changes
- `Set` (1.2%) - configuration changes
- `Handle` (1.0%) - add handling logic
- `Raise` (1.0%) - add error raising
- `Pass` (0.8%) - pass parameters/values
- `Support` (0.8%) - add support for something
- `Hide` (1.4%) - make private/internal
- `Always` (1.4%) - enforce consistent behavior
- `Mk` (1.4%) - make/create (abbreviated)
- `Start` (1.0%) - begin implementation
Other frequent verbs: `More`, `Change`, `Extend`, `Disable`, `Log`,
`Enable`, `Ensure`, `Expose`, `Allow`
### Backtick Usage
Always use backticks for:
- Module names: `trio`, `asyncio`, `msgspec`, `greenback`, `stackscope`
- Class names: `Context`, `Actor`, `Address`, `PldRx`, `SpawnSpec`
- Method names: `.pause_from_sync()`, `._pause()`, `.cancel()`
- Function names: `breakpoint()`, `collapse_eg()`, `open_root_actor()`
- Decorators: `@acm`, `@context`
- Exceptions: `Cancelled`, `TransportClosed`, `MsgTypeError`
- Keywords: `finally`, `None`, `False`
- Variable names: `tn`, `debug_mode`
- Complex expressions: `trio.Cancelled`, `asyncio.Task`
Most backticked terms in tractor:
`trio`, `asyncio`, `Context`, `.pause_from_sync()`, `tn`,
`._pause()`, `breakpoint()`, `collapse_eg()`, `Actor`, `@acm`,
`.cancel()`, `Cancelled`, `open_root_actor()`, `greenback`
### Examples
Good subject lines:
```
Add `uds` to `._multiaddr`, tweak typing
Drop `DebugStatus.shield` attr, add `.req_finished`
Use `stackscope` for all actor-tree rendered "views"
Fix `.to_asyncio` inter-task-cancellation!
Bump `ruff.toml` to target py313
Mv `load_module_from_path()` to new `._code_load` submod
Always use `tuple`-cast for singleton parent addrs
```
## Body Format
### General Structure
- 43.2% of commits have no body (simple changes)
- Use blank line after subject
- Max line length: 67 chars
- Use `-` bullets for lists (28.0% of commits)
- Rarely use `*` bullets (2.4%)
### Section Markers
Use these markers to organize longer commit bodies:
- `Also,` (most common: 26 occurrences)
- `Other,` (13 occurrences)
- `Deats,` (11 occurrences) - for implementation details
- `Further,` (7 occurrences)
- `TODO,` (3 occurrences)
- `Impl details,` (2 occurrences)
- `Notes,` (1 occurrence)
### Common Abbreviations
Use these freely (sorted by frequency):
- `msg` (63) - message
- `bg` (37) - background
- `ctx` (30) - context
- `impl` (27) - implementation
- `mod` (26) - module
- `obvi` (17) - obviously
- `tn` (16) - task name
- `fn` (15) - function
- `vs` (15) - versus
- `bc` (14) - because
- `var` (14) - variable
- `prolly` (9) - probably
- `ep` (6) - entry point
- `OW` (5) - otherwise
- `rn` (4) - right now
- `sig` (4) - signal/signature
- `deps` (3) - dependencies
- `iface` (2) - interface
- `subproc` (2) - subprocess
- `tho` (2) - though
- `ofc` (2) - of course
### Tone and Style
- Casual but technical (use `XD` for humor: 23 times)
- Use `..` for trailing thoughts (108 occurrences)
- Use `Woops,` to acknowledge mistakes (4 subject lines)
- Don't be afraid to show personality while being precise
### Example Bodies
Simple with bullets:
```
Add `multiaddr` and bump up some deps
Since we're planning to use it for (discovery)
addressing, allowing replacement of the hacky (pretend)
attempt in `tractor._multiaddr` Bp
Also pin some deps,
- make us py312+
- use `pdbp` with my frame indexing fix.
- mv to latest `xonsh` for fancy cmd/suggestion injections.
Bump lock file to match obvi!
```
With section markers:
```
Use `stackscope` for all actor-tree rendered "views"
Instead of the (much more) limited and hacky `.devx._code`
impls, move to using the new `.devx._stackscope` API which
wraps the `stackscope` project.
Deats,
- make new `stackscope.extract_stack()` wrapper
- port over frame-descing to `_stackscope.pformat_stack()`
- move `PdbREPL` to use `stackscope` render approach
- update tests for new stack output format
Also,
- tweak log formatting for consistency
- add typing hints throughout
```
## Special Patterns
### WIP Commits
Rare (0.2%) - avoid committing WIP if possible
### Merge Commits
Auto-generated (4.4%), don't worry about style
### File References
- Use `module.py` or `.submodule` style
- Rarely use `file.py:line` references (0 in analysis)
### Links
- GitHub links used sparingly (3 total)
- Prefer code references over external links
## Footer
The default footer should credit `claude` (you) for helping generate
the commit msg content:
```
(this commit msg was generated in some part by [`claude-code`][claude-code-gh])
[claude-code-gh]: https://github.com/anthropics/claude-code
```
Further, if the patch was solely or in part written
by `claude`, instead add:
```
(this patch was generated in some part by [`claude-code`][claude-code-gh])
[claude-code-gh]: https://github.com/anthropics/claude-code
```
## Summary Checklist
Before committing, verify:
- [ ] Subject line uses present tense verb
- [ ] Subject line ~50 chars (hard max 67)
- [ ] Code elements wrapped in backticks
- [ ] Body lines ≤67 chars
- [ ] Abbreviations used where natural
- [ ] Casual yet precise tone
- [ ] Section markers if body >3 paragraphs
- [ ] Technical accuracy maintained
## Analysis Metadata
```
Source: tractor repository
Commits analyzed: 500
Date range: 2019-2025
Analysis date: 2026-02-08
```
---
(this style guide was generated by [`claude-code`][claude-code-gh]
analyzing commit history)
[claude-code-gh]: https://github.com/anthropics/claude-code

View File

@ -74,24 +74,44 @@ jobs:
# run: mypy tractor/ --ignore-missing-imports --show-traceback # run: mypy tractor/ --ignore-missing-imports --show-traceback
testing-linux: testing:
name: '${{ matrix.os }} Python ${{ matrix.python }} - ${{ matrix.spawn_backend }}' name: '${{ matrix.os }} Python${{ matrix.python-version }} spawn_backend=${{ matrix.spawn_backend }} tpt_proto=${{ matrix.tpt_proto }}'
timeout-minutes: 10 timeout-minutes: 16
runs-on: ${{ matrix.os }} runs-on: ${{ matrix.os }}
strategy: strategy:
fail-fast: false fail-fast: false
matrix: matrix:
os: [ubuntu-latest] os: [
python-version: ['3.13'] ubuntu-latest,
macos-latest,
]
python-version: [
'3.13',
# '3.14',
]
spawn_backend: [ spawn_backend: [
'trio', 'trio',
# 'mp_spawn', # 'mp_spawn',
# 'mp_forkserver', # 'mp_forkserver',
# ?TODO^ is it worth it to get these running again?
#
# - [ ] next-gen backends, on 3.13+
# https://github.com/goodboy/tractor/issues/379
# 'subinterpreter',
# 'subint',
] ]
tpt_proto: [
'tcp',
'uds',
]
# https://github.com/orgs/community/discussions/26253#discussioncomment-3250989
exclude:
# don't do UDS run on macOS (for now)
- os: macos-latest
tpt_proto: 'uds'
steps: steps:
- uses: actions/checkout@v4 - uses: actions/checkout@v4
- name: 'Install uv + py-${{ matrix.python-version }}' - name: 'Install uv + py-${{ matrix.python-version }}'
@ -118,7 +138,11 @@ jobs:
run: uv tree run: uv tree
- name: Run tests - name: Run tests
run: uv run pytest tests/ --spawn-backend=${{ matrix.spawn_backend }} -rsx run: >
uv run
pytest tests/ -rsx
--spawn-backend=${{ matrix.spawn_backend }}
--tpt-proto=${{ matrix.tpt_proto }}
# XXX legacy NOTE XXX # XXX legacy NOTE XXX
# #

27
.gitignore vendored
View File

@ -102,3 +102,30 @@ venv.bak/
# mypy # mypy
.mypy_cache/ .mypy_cache/
# all files under
.git/
# any commit-msg gen tmp files
.claude/*_commit_*.md
.claude/*_commit*.toml
.claude/*_commit*.txt
# nix develop --profile .nixdev
.nixdev*
# :Obsession .
Session.vim
# `gish` local `.md`-files
# TODO? better all around automation!
# -[ ] it'd be handy to also commit and sync with wtv git service?
# -[ ] everything should be put under a `.gish/` no?
gitea/
gh/
# ------ macOS ------
# Finder metadata
**/.DS_Store
# LLM conversations that should remain private
docs/conversations/

View File

@ -11,9 +11,4 @@ pkgs.mkShell {
LD_LIBRARY_PATH = pkgs.lib.makeLibraryPath nativeBuildInputs; LD_LIBRARY_PATH = pkgs.lib.makeLibraryPath nativeBuildInputs;
TMPDIR = "/tmp"; TMPDIR = "/tmp";
shellHook = ''
set -e
uv venv .venv --python=3.12
'';
} }

View File

@ -420,20 +420,17 @@ Check out our experimental system for `guest`_-mode controlled
async def aio_echo_server( async def aio_echo_server(
to_trio: trio.MemorySendChannel, chan: tractor.to_asyncio.LinkedTaskChannel,
from_trio: asyncio.Queue,
) -> None: ) -> None:
# a first message must be sent **from** this ``asyncio`` # a first message must be sent **from** this ``asyncio``
# task or the ``trio`` side will never unblock from # task or the ``trio`` side will never unblock from
# ``tractor.to_asyncio.open_channel_from():`` # ``tractor.to_asyncio.open_channel_from():``
to_trio.send_nowait('start') chan.started_nowait('start')
# XXX: this uses an ``from_trio: asyncio.Queue`` currently but we
# should probably offer something better.
while True: while True:
# echo the msg back # echo the msg back
to_trio.send_nowait(await from_trio.get()) chan.send_nowait(await chan.get())
await asyncio.sleep(0) await asyncio.sleep(0)
@ -445,7 +442,7 @@ Check out our experimental system for `guest`_-mode controlled
# message. # message.
async with tractor.to_asyncio.open_channel_from( async with tractor.to_asyncio.open_channel_from(
aio_echo_server, aio_echo_server,
) as (first, chan): ) as (chan, first):
assert first == 'start' assert first == 'start'
await ctx.started(first) await ctx.started(first)
@ -504,8 +501,10 @@ Yes, we spawn a python process, run ``asyncio``, start ``trio`` on the
``asyncio`` loop, then send commands to the ``trio`` scheduled tasks to ``asyncio`` loop, then send commands to the ``trio`` scheduled tasks to
tell ``asyncio`` tasks what to do XD tell ``asyncio`` tasks what to do XD
We need help refining the `asyncio`-side channel API to be more The ``asyncio``-side task receives a single
`trio`-like. Feel free to sling your opinion in `#273`_! ``chan: LinkedTaskChannel`` handle providing a ``trio``-like
API: ``.started_nowait()``, ``.send_nowait()``, ``.get()``
and more. Feel free to sling your opinion in `#273`_!
.. _#273: https://github.com/goodboy/tractor/issues/273 .. _#273: https://github.com/goodboy/tractor/issues/273
@ -641,13 +640,15 @@ Help us push toward the future of distributed `Python`.
- Typed capability-based (dialog) protocols ( see `#196 - Typed capability-based (dialog) protocols ( see `#196
<https://github.com/goodboy/tractor/issues/196>`_ with draft work <https://github.com/goodboy/tractor/issues/196>`_ with draft work
started in `#311 <https://github.com/goodboy/tractor/pull/311>`_) started in `#311 <https://github.com/goodboy/tractor/pull/311>`_)
- We **recently disabled CI-testing on windows** and need help getting - **macOS is now officially supported** and tested in CI
it running again! (see `#327 alongside Linux!
<https://github.com/goodboy/tractor/pull/327>`_). **We do have windows - We **recently disabled CI-testing on windows** and need
support** (and have for quite a while) but since no active hacker help getting it running again! (see `#327
exists in the user-base to help test on that OS, for now we're not <https://github.com/goodboy/tractor/pull/327>`_). **We do
actively maintaining testing due to the added hassle and general have windows support** (and have for quite a while) but
latency.. since no active hacker exists in the user-base to help
test on that OS, for now we're not actively maintaining
testing due to the added hassle and general latency..
Feel like saying hi? Feel like saying hi?

View File

@ -17,6 +17,7 @@ from tractor import (
MsgStream, MsgStream,
_testing, _testing,
trionics, trionics,
TransportClosed,
) )
import trio import trio
import pytest import pytest
@ -208,11 +209,15 @@ async def main(
# TODO: is this needed or no? # TODO: is this needed or no?
raise raise
except trio.ClosedResourceError: except (
trio.ClosedResourceError,
TransportClosed,
) as _tpt_err:
# NOTE: don't send if we already broke the # NOTE: don't send if we already broke the
# connection to avoid raising a closed-error # connection to avoid raising a closed-error
# such that we drop through to the ctl-c # such that we drop through to the ctl-c
# mashing by user. # mashing by user.
with trio.CancelScope(shield=True):
await trio.sleep(0.01) await trio.sleep(0.01)
# timeout: int = 1 # timeout: int = 1
@ -247,6 +252,7 @@ async def main(
await stream.send(i) await stream.send(i)
pytest.fail('stream not closed?') pytest.fail('stream not closed?')
except ( except (
TransportClosed,
trio.ClosedResourceError, trio.ClosedResourceError,
trio.EndOfChannel, trio.EndOfChannel,
) as send_err: ) as send_err:

View File

@ -18,15 +18,14 @@ async def aio_sleep_forever():
async def bp_then_error( async def bp_then_error(
to_trio: trio.MemorySendChannel, chan: to_asyncio.LinkedTaskChannel,
from_trio: asyncio.Queue,
raise_after_bp: bool = True, raise_after_bp: bool = True,
) -> None: ) -> None:
# sync with `trio`-side (caller) task # sync with `trio`-side (caller) task
to_trio.send_nowait('start') chan.started_nowait('start')
# NOTE: what happens here inside the hook needs some refinement.. # NOTE: what happens here inside the hook needs some refinement..
# => seems like it's still `.debug._set_trace()` but # => seems like it's still `.debug._set_trace()` but
@ -60,7 +59,7 @@ async def trio_ctx(
to_asyncio.open_channel_from( to_asyncio.open_channel_from(
bp_then_error, bp_then_error,
# raise_after_bp=not bp_before_started, # raise_after_bp=not bp_before_started,
) as (first, chan), ) as (chan, first),
trio.open_nursery() as tn, trio.open_nursery() as tn,
): ):

View File

@ -20,7 +20,7 @@ async def sleep(
async def open_ctx( async def open_ctx(
n: tractor._supervise.ActorNursery n: tractor.runtime._supervise.ActorNursery
): ):
# spawn both actors # spawn both actors

View File

@ -3,6 +3,7 @@ Verify we can dump a `stackscope` tree on a hang.
''' '''
import os import os
import platform
import signal import signal
import trio import trio
@ -31,13 +32,26 @@ async def main(
from_test: bool = False, from_test: bool = False,
) -> None: ) -> None:
if platform.system() != 'Darwin':
tpt = 'uds'
else:
# XXX, precisely we can't use pytest's tmp-path generation
# for tests.. apparently because:
#
# > The OSError: AF_UNIX path too long in macOS Python occurs
# > because the path to the Unix domain socket exceeds the
# > operating system's maximum path length limit (around 104
#
# WHICH IS just, wtf hillarious XD
tpt = 'tcp'
async with ( async with (
tractor.open_nursery( tractor.open_nursery(
debug_mode=True, debug_mode=True,
enable_stack_on_sig=True, enable_stack_on_sig=True,
# maybe_enable_greenback=False, # maybe_enable_greenback=False,
loglevel='devx', loglevel='devx',
enable_transports=['uds'], enable_transports=[tpt],
) as an, ) as an,
): ):
ptl: tractor.Portal = await an.start_actor( ptl: tractor.Portal = await an.start_actor(

View File

@ -1,3 +1,5 @@
import platform
import tractor import tractor
import trio import trio
@ -34,9 +36,22 @@ async def just_bp(
async def main(): async def main():
if platform.system() != 'Darwin':
tpt = 'uds'
else:
# XXX, precisely we can't use pytest's tmp-path generation
# for tests.. apparently because:
#
# > The OSError: AF_UNIX path too long in macOS Python occurs
# > because the path to the Unix domain socket exceeds the
# > operating system's maximum path length limit (around 104
#
# WHICH IS just, wtf hillarious XD
tpt = 'tcp'
async with tractor.open_nursery( async with tractor.open_nursery(
debug_mode=True, debug_mode=True,
enable_transports=['uds'], enable_transports=[tpt],
loglevel='devx', loglevel='devx',
) as n: ) as n:
p = await n.start_actor( p = await n.start_actor(

View File

@ -90,7 +90,7 @@ async def main() -> list[int]:
# yes, a nursery which spawns `trio`-"actors" B) # yes, a nursery which spawns `trio`-"actors" B)
an: ActorNursery an: ActorNursery
async with tractor.open_nursery( async with tractor.open_nursery(
loglevel='cancel', loglevel='error',
# debug_mode=True, # debug_mode=True,
) as an: ) as an:
@ -118,8 +118,10 @@ async def main() -> list[int]:
cancelled: bool = await portal.cancel_actor() cancelled: bool = await portal.cancel_actor()
assert cancelled assert cancelled
print(f"STREAM TIME = {time.time() - start}") print(
print(f"STREAM + SPAWN TIME = {time.time() - pre_start}") f"STREAM TIME = {time.time() - start}\n"
f"STREAM + SPAWN TIME = {time.time() - pre_start}\n"
)
assert result_stream == list(range(seed)) assert result_stream == list(range(seed))
return result_stream return result_stream

View File

@ -11,21 +11,17 @@ import tractor
async def aio_echo_server( async def aio_echo_server(
to_trio: trio.MemorySendChannel, chan: tractor.to_asyncio.LinkedTaskChannel,
from_trio: asyncio.Queue,
) -> None: ) -> None:
# a first message must be sent **from** this ``asyncio`` # a first message must be sent **from** this ``asyncio``
# task or the ``trio`` side will never unblock from # task or the ``trio`` side will never unblock from
# ``tractor.to_asyncio.open_channel_from():`` # ``tractor.to_asyncio.open_channel_from():``
to_trio.send_nowait('start') chan.started_nowait('start')
# XXX: this uses an ``from_trio: asyncio.Queue`` currently but we
# should probably offer something better.
while True: while True:
# echo the msg back # echo the msg back
to_trio.send_nowait(await from_trio.get()) chan.send_nowait(await chan.get())
await asyncio.sleep(0) await asyncio.sleep(0)
@ -37,7 +33,7 @@ async def trio_to_aio_echo_server(
# message. # message.
async with tractor.to_asyncio.open_channel_from( async with tractor.to_asyncio.open_channel_from(
aio_echo_server, aio_echo_server,
) as (first, chan): ) as (chan, first):
assert first == 'start' assert first == 'start'
await ctx.started(first) await ctx.started(first)

View File

@ -10,7 +10,7 @@ async def main(service_name):
await an.start_actor(service_name) await an.start_actor(service_name)
async with tractor.get_registry() as portal: async with tractor.get_registry() as portal:
print(f"Arbiter is listening on {portal.channel}") print(f"Registrar is listening on {portal.channel}")
async with tractor.wait_for_actor(service_name) as sockaddr: async with tractor.wait_for_actor(service_name) as sockaddr:
print(f"my_service is found at {sockaddr}") print(f"my_service is found at {sockaddr}")

27
flake.lock 100644
View File

@ -0,0 +1,27 @@
{
"nodes": {
"nixpkgs": {
"locked": {
"lastModified": 1769018530,
"narHash": "sha256-MJ27Cy2NtBEV5tsK+YraYr2g851f3Fl1LpNHDzDX15c=",
"owner": "nixos",
"repo": "nixpkgs",
"rev": "88d3861acdd3d2f0e361767018218e51810df8a1",
"type": "github"
},
"original": {
"owner": "nixos",
"ref": "nixos-unstable",
"repo": "nixpkgs",
"type": "github"
}
},
"root": {
"inputs": {
"nixpkgs": "nixpkgs"
}
}
},
"root": "root",
"version": 7
}

70
flake.nix 100644
View File

@ -0,0 +1,70 @@
# An "impure" template thx to `pyproject.nix`,
# https://pyproject-nix.github.io/pyproject.nix/templates.html#impure
# https://github.com/pyproject-nix/pyproject.nix/blob/master/templates/impure/flake.nix
{
description = "An impure overlay (w dev-shell) using `uv`";
inputs = {
nixpkgs.url = "github:nixos/nixpkgs/nixos-unstable";
};
outputs =
{ nixpkgs, ... }:
let
inherit (nixpkgs) lib;
forAllSystems = lib.genAttrs lib.systems.flakeExposed;
in
{
devShells = forAllSystems (
system:
let
pkgs = nixpkgs.legacyPackages.${system};
# XXX NOTE XXX, for now we overlay specific pkgs via
# a major-version-pinned-`cpython`
cpython = "python313";
venv_dir = "py313";
pypkgs = pkgs."${cpython}Packages";
in
{
default = pkgs.mkShell {
packages = [
# XXX, ensure sh completions activate!
pkgs.bashInteractive
pkgs.bash-completion
# XXX, on nix(os), use pkgs version to avoid
# build/sys-sh-integration issues
pkgs.ruff
pkgs.uv
pkgs.${cpython}# ?TODO^ how to set from `cpython` above?
];
shellHook = ''
# unmask to debug **this** dev-shell-hook
# set -e
# link-in c++ stdlib for various AOT-ext-pkgs (numpy, etc.)
LD_LIBRARY_PATH="${pkgs.stdenv.cc.cc.lib}/lib:$LD_LIBRARY_PATH"
export LD_LIBRARY_PATH
# RUNTIME-SETTINGS
# ------ uv ------
# - always use the ./py313/ venv-subdir
# - sync env with all extras
export UV_PROJECT_ENVIRONMENT=${venv_dir}
uv sync --dev --all-extras
# ------ TIPS ------
# NOTE, to launch the py-venv installed `xonsh` (like @goodboy)
# run the `nix develop` cmd with,
# >> nix develop -c uv run xonsh
'';
};
}
);
};
}

View File

@ -9,7 +9,7 @@ name = "tractor"
version = "0.1.0a6dev0" version = "0.1.0a6dev0"
description = 'structured concurrent `trio`-"actors"' description = 'structured concurrent `trio`-"actors"'
authors = [{ name = "Tyler Goodlet", email = "goodboy_foss@protonmail.com" }] authors = [{ name = "Tyler Goodlet", email = "goodboy_foss@protonmail.com" }]
requires-python = ">= 3.11" requires-python = ">=3.12, <3.14"
readme = "docs/README.rst" readme = "docs/README.rst"
license = "AGPL-3.0-or-later" license = "AGPL-3.0-or-later"
keywords = [ keywords = [
@ -24,11 +24,13 @@ keywords = [
classifiers = [ classifiers = [
"Development Status :: 3 - Alpha", "Development Status :: 3 - Alpha",
"Operating System :: POSIX :: Linux", "Operating System :: POSIX :: Linux",
"Operating System :: MacOS",
"Framework :: Trio", "Framework :: Trio",
"License :: OSI Approved :: GNU Affero General Public License v3 or later (AGPLv3+)", "License :: OSI Approved :: GNU Affero General Public License v3 or later (AGPLv3+)",
"Programming Language :: Python :: Implementation :: CPython", "Programming Language :: Python :: Implementation :: CPython",
"Programming Language :: Python :: 3 :: Only", "Programming Language :: Python :: 3 :: Only",
"Programming Language :: Python :: 3.11", "Programming Language :: Python :: 3.12",
"Programming Language :: Python :: 3.13",
"Topic :: System :: Distributed Computing", "Topic :: System :: Distributed Computing",
] ]
dependencies = [ dependencies = [
@ -42,48 +44,64 @@ dependencies = [
"wrapt>=1.16.0,<2", "wrapt>=1.16.0,<2",
"colorlog>=6.8.2,<7", "colorlog>=6.8.2,<7",
# built-in multi-actor `pdb` REPL # built-in multi-actor `pdb` REPL
"pdbp>=1.6,<2", # windows only (from `pdbp`) "pdbp>=1.8.2,<2", # windows only (from `pdbp`)
# typed IPC msging # typed IPC msging
"msgspec>=0.19.0", "msgspec>=0.19.0",
"cffi>=1.17.1", "cffi>=1.17.1",
"bidict>=0.23.1", "bidict>=0.23.1",
"platformdirs>=4.4.0",
] ]
# ------ project ------ # ------ project ------
[dependency-groups] [dependency-groups]
dev = [ dev = [
# test suite {include-group = 'devx'},
# TODO: maybe some of these layout choices? {include-group = 'testing'},
# https://docs.pytest.org/en/8.0.x/explanation/goodpractices.html#choosing-a-test-layout-import-rules {include-group = 'repl'},
"pytest>=8.3.5", ]
"pexpect>=4.9.0,<5", devx = [
# `tractor.devx` tooling # `tractor.devx` tooling
"greenback>=1.2.1,<2", "greenback>=1.2.1,<2",
"stackscope>=0.2.2,<0.3", "stackscope>=0.2.2,<0.3",
# ^ requires this? # ^ requires this?
"typing-extensions>=4.14.1", "typing-extensions>=4.14.1",
]
testing = [
# test suite
# TODO: maybe some of these layout choices?
# https://docs.pytest.org/en/8.0.x/explanation/goodpractices.html#choosing-a-test-layout-import-rules
"pytest>=8.3.5",
"pexpect>=4.9.0,<5",
]
repl = [
"pyperclip>=1.9.0", "pyperclip>=1.9.0",
"prompt-toolkit>=3.0.50", "prompt-toolkit>=3.0.50",
"xonsh>=0.19.2", "xonsh>=0.22.2",
"psutil>=7.0.0", "psutil>=7.0.0",
] ]
lint = [
"ruff>=0.9.6"
]
# TODO, add these with sane versions; were originally in # TODO, add these with sane versions; were originally in
# `requirements-docs.txt`.. # `requirements-docs.txt`..
# docs = [ # docs = [
# "sphinx>=" # "sphinx>="
# "sphinx_book_theme>=" # "sphinx_book_theme>="
# ] # ]
# ------ dependency-groups ------ # ------ dependency-groups ------
# ------ dependency-groups ------
[tool.uv.sources] [tool.uv.sources]
# XXX NOTE, only for @goodboy's hacking on `pprint(sort_dicts=False)` # XXX NOTE, only for @goodboy's hacking on `pprint(sort_dicts=False)`
# for the `pp` alias.. # for the `pp` alias..
# pdbp = { path = "../pdbp", editable = true }
# [tool.uv.sources.pdbp]
# XXX, in case we need to tmp patch again.
# git = "https://github.com/goodboy/pdbp.git"
# branch ="repair_stack_trace_frame_indexing"
# path = "../pdbp"
# editable = true
# ------ tool.uv.sources ------ # ------ tool.uv.sources ------
# TODO, distributed (multi-host) extensions # TODO, distributed (multi-host) extensions
@ -145,6 +163,7 @@ all_bullets = true
[tool.pytest.ini_options] [tool.pytest.ini_options]
minversion = '6.0' minversion = '6.0'
# https://docs.pytest.org/en/stable/reference/reference.html#configuration-options
testpaths = [ testpaths = [
'tests' 'tests'
] ]
@ -155,10 +174,17 @@ addopts = [
'--import-mode=importlib', '--import-mode=importlib',
# don't show frickin captured logs AGAIN in the report.. # don't show frickin captured logs AGAIN in the report..
'--show-capture=no', '--show-capture=no',
# disable `xonsh` plugin
# https://docs.pytest.org/en/stable/how-to/plugins.html#disabling-plugins-from-autoloading
# https://docs.pytest.org/en/stable/how-to/plugins.html#deactivating-unregistering-a-plugin-by-name
'-p no:xonsh'
] ]
log_cli = false log_cli = false
# TODO: maybe some of these layout choices? # TODO: maybe some of these layout choices?
# https://docs.pytest.org/en/8.0.x/explanation/goodpractices.html#choosing-a-test-layout-import-rules # https://docs.pytest.org/en/8.0.x/explanation/goodpractices.html#choosing-a-test-layout-import-rules
# pythonpath = "src" # pythonpath = "src"
# https://docs.pytest.org/en/stable/reference/reference.html#confval-console_output_style
console_output_style = 'progress'
# ------ tool.pytest ------ # ------ tool.pytest ------

View File

@ -1,8 +0,0 @@
# vim: ft=ini
# pytest.ini for tractor
[pytest]
# don't show frickin captured logs AGAIN in the report..
addopts = --show-capture='no'
log_cli = false
; minversion = 6.0

View File

@ -35,8 +35,8 @@ exclude = [
line-length = 88 line-length = 88
indent-width = 4 indent-width = 4
# Assume Python 3.9 # assume latest minor cpython
target-version = "py311" target-version = "py313"
[lint] [lint]
# Enable Pyflakes (`F`) and a subset of the pycodestyle (`E`) codes by default. # Enable Pyflakes (`F`) and a subset of the pycodestyle (`E`) codes by default.

View File

@ -9,8 +9,11 @@ import os
import signal import signal
import platform import platform
import time import time
from pathlib import Path
from typing import Literal
import pytest import pytest
import tractor
from tractor._testing import ( from tractor._testing import (
examples_dir as examples_dir, examples_dir as examples_dir,
tractor_test as tractor_test, tractor_test as tractor_test,
@ -22,6 +25,8 @@ pytest_plugins: list[str] = [
'tractor._testing.pytest', 'tractor._testing.pytest',
] ]
_ci_env: bool = os.environ.get('CI', False)
_non_linux: bool = platform.system() != 'Linux'
# Sending signal.SIGINT on subprocess fails on windows. Use CTRL_* alternatives # Sending signal.SIGINT on subprocess fails on windows. Use CTRL_* alternatives
if platform.system() == 'Windows': if platform.system() == 'Windows':
@ -34,9 +39,8 @@ else:
_INT_SIGNAL = signal.SIGINT _INT_SIGNAL = signal.SIGINT
_INT_RETURN_CODE = 1 if sys.version_info < (3, 8) else -signal.SIGINT.value _INT_RETURN_CODE = 1 if sys.version_info < (3, 8) else -signal.SIGINT.value
_PROC_SPAWN_WAIT = ( _PROC_SPAWN_WAIT = (
0.6 2 if _ci_env
if sys.version_info < (3, 7) else 1
else 0.4
) )
@ -44,6 +48,80 @@ no_windows = pytest.mark.skipif(
platform.system() == "Windows", platform.system() == "Windows",
reason="Test is unsupported on windows", reason="Test is unsupported on windows",
) )
no_macos = pytest.mark.skipif(
platform.system() == "Darwin",
reason="Test is unsupported on MacOS",
)
def get_cpu_state(
icpu: int = 0,
setting: Literal[
'scaling_governor',
'*_pstate_max_freq',
'scaling_max_freq',
# 'scaling_cur_freq',
] = '*_pstate_max_freq',
) -> tuple[
Path,
str|int,
]|None:
'''
Attempt to read the (first) CPU's setting according
to the set `setting` from under the file-sys,
/sys/devices/system/cpu/cpu0/cpufreq/{setting}
Useful to determine latency headroom for various perf affected
test suites.
'''
try:
# Read governor for core 0 (usually same for all)
setting_path: Path = list(
Path(f'/sys/devices/system/cpu/cpu{icpu}/cpufreq/')
.glob(f'{setting}')
)[0] # <- XXX must be single match!
with open(
setting_path,
'r',
) as f:
return (
setting_path,
f.read().strip(),
)
except (FileNotFoundError, IndexError):
return None
def cpu_scaling_factor() -> float:
'''
Return a latency-headroom multiplier (>= 1.0) reflecting how
much to inflate time-limits when CPU-freq scaling is active on
linux.
When no scaling info is available (non-linux, missing sysfs),
returns 1.0 (i.e. no headroom adjustment needed).
'''
if _non_linux:
return 1.
mx = get_cpu_state()
cur = get_cpu_state(setting='scaling_max_freq')
if mx is None or cur is None:
return 1.
_mx_pth, max_freq = mx
_cur_pth, cur_freq = cur
cpu_scaled: float = int(cur_freq) / int(max_freq)
if cpu_scaled != 1.:
return 1. / (
cpu_scaled * 2 # <- bc likely "dual threaded"
)
return 1.
def pytest_addoption( def pytest_addoption(
@ -61,16 +139,52 @@ def pytest_addoption(
@pytest.fixture(scope='session', autouse=True) @pytest.fixture(scope='session', autouse=True)
def loglevel(request): def loglevel(request) -> str:
import tractor import tractor
orig = tractor.log._default_loglevel orig = tractor.log._default_loglevel
level = tractor.log._default_loglevel = request.config.option.loglevel level = tractor.log._default_loglevel = request.config.option.loglevel
tractor.log.get_console_log(level) log = tractor.log.get_console_log(
level=level,
name='tractor', # <- enable root logger
)
log.info(
f'Test-harness set runtime loglevel: {level!r}\n'
)
yield level yield level
tractor.log._default_loglevel = orig tractor.log._default_loglevel = orig
_ci_env: bool = os.environ.get('CI', False) @pytest.fixture(scope='function')
def test_log(
request,
loglevel: str,
) -> tractor.log.StackLevelAdapter:
'''
Deliver a per test-module-fn logger instance for reporting from
within actual test bodies/fixtures.
For example this can be handy to report certain error cases from
exception handlers using `test_log.exception()`.
'''
modname: str = request.function.__module__
log = tractor.log.get_logger(
name=modname, # <- enable root logger
# pkg_name='tests',
)
_log = tractor.log.get_console_log(
level=loglevel,
logger=log,
name=modname,
# pkg_name='tests',
)
_log.debug(
f'In-test-logging requested\n'
f'test_log.name: {log.name!r}\n'
f'level: {loglevel!r}\n'
)
yield _log
@pytest.fixture(scope='session') @pytest.fixture(scope='session')
@ -106,6 +220,8 @@ def daemon(
testdir: pytest.Pytester, testdir: pytest.Pytester,
reg_addr: tuple[str, int], reg_addr: tuple[str, int],
tpt_proto: str, tpt_proto: str,
ci_env: bool,
test_log: tractor.log.StackLevelAdapter,
) -> subprocess.Popen: ) -> subprocess.Popen:
''' '''
@ -121,10 +237,12 @@ def daemon(
"import tractor; " "import tractor; "
"tractor.run_daemon([], " "tractor.run_daemon([], "
"registry_addrs={reg_addrs}, " "registry_addrs={reg_addrs}, "
"enable_transports={enable_tpts}, "
"debug_mode={debug_mode}, " "debug_mode={debug_mode}, "
"loglevel={ll})" "loglevel={ll})"
).format( ).format(
reg_addrs=str([reg_addr]), reg_addrs=str([reg_addr]),
enable_tpts=str([tpt_proto]),
ll="'{}'".format(loglevel) if loglevel else None, ll="'{}'".format(loglevel) if loglevel else None,
debug_mode=debug_mode, debug_mode=debug_mode,
) )
@ -143,13 +261,25 @@ def daemon(
**kwargs, **kwargs,
) )
# TODO! we should poll for the registry socket-bind to take place
# and only once that's done yield to the requester!
# -[ ] TCP: use the `._root.open_root_actor()`::`ping_tpt_socket()`
# closure!
# -[ ] UDS: can we do something similar for 'pinging" the
# file-socket?
#
global _PROC_SPAWN_WAIT
# UDS sockets are **really** fast to bind()/listen()/connect() # UDS sockets are **really** fast to bind()/listen()/connect()
# so it's often required that we delay a bit more starting # so it's often required that we delay a bit more starting
# the first actor-tree.. # the first actor-tree..
if tpt_proto == 'uds': if tpt_proto == 'uds':
global _PROC_SPAWN_WAIT _PROC_SPAWN_WAIT += 1.6
_PROC_SPAWN_WAIT = 0.6
if _non_linux and ci_env:
_PROC_SPAWN_WAIT += 1
# XXX, allow time for the sub-py-proc to boot up.
# !TODO, see ping-polling ideas above!
time.sleep(_PROC_SPAWN_WAIT) time.sleep(_PROC_SPAWN_WAIT)
assert not proc.returncode assert not proc.returncode
@ -159,18 +289,30 @@ def daemon(
# XXX! yeah.. just be reaaal careful with this bc sometimes it # XXX! yeah.. just be reaaal careful with this bc sometimes it
# can lock up on the `_io.BufferedReader` and hang.. # can lock up on the `_io.BufferedReader` and hang..
stderr: str = proc.stderr.read().decode() stderr: str = proc.stderr.read().decode()
if stderr: stdout: str = proc.stdout.read().decode()
if (
stderr
or
stdout
):
print( print(
f'Daemon actor tree produced STDERR:\n' f'Daemon actor tree produced output:\n'
f'{proc.args}\n' f'{proc.args}\n'
f'\n' f'\n'
f'{stderr}\n' f'stderr: {stderr!r}\n'
f'stdout: {stdout!r}\n'
) )
if proc.returncode != -2:
raise RuntimeError( if (rc := proc.returncode) != -2:
'Daemon actor tree failed !?\n' msg: str = (
f'{proc.args}\n' f'Daemon actor tree was not cancelled !?\n'
f'proc.args: {proc.args!r}\n'
f'proc.returncode: {rc!r}\n'
) )
if rc < 0:
raise RuntimeError(msg)
test_log.error(msg)
# @pytest.fixture(autouse=True) # @pytest.fixture(autouse=True)

View File

@ -3,6 +3,8 @@
''' '''
from __future__ import annotations from __future__ import annotations
import platform
import signal
import time import time
from typing import ( from typing import (
Callable, Callable,
@ -32,9 +34,23 @@ if TYPE_CHECKING:
from pexpect import pty_spawn from pexpect import pty_spawn
_non_linux: bool = platform.system() != 'Linux'
def pytest_configure(config):
# register custom marks to avoid warnings see,
# https://docs.pytest.org/en/stable/how-to/writing_plugins.html#registering-custom-markers
config.addinivalue_line(
'markers',
'ctlcs_bish: test will (likely) not behave under SIGINT..'
)
# a fn that sub-instantiates a `pexpect.spawn()` # a fn that sub-instantiates a `pexpect.spawn()`
# and returns it. # and returns it.
type PexpectSpawner = Callable[[str], pty_spawn.spawn] type PexpectSpawner = Callable[
[str],
pty_spawn.spawn,
]
@pytest.fixture @pytest.fixture
@ -64,26 +80,65 @@ def spawn(
''' '''
import os import os
# disable colored tbs
os.environ['PYTHON_COLORS'] = '0' os.environ['PYTHON_COLORS'] = '0'
# disable all ANSI color output
# os.environ['NO_COLOR'] = '1'
spawned: PexpectSpawner|None = None
def _spawn( def _spawn(
cmd: str, cmd: str,
expect_timeout: float = 4,
**mkcmd_kwargs, **mkcmd_kwargs,
) -> pty_spawn.spawn: ) -> pty_spawn.spawn:
nonlocal spawned
unset_colors() unset_colors()
return testdir.spawn( spawned = testdir.spawn(
cmd=mk_cmd( cmd=mk_cmd(
cmd, cmd,
**mkcmd_kwargs, **mkcmd_kwargs,
), ),
expect_timeout=3, expect_timeout=(timeout:=(
expect_timeout + 6
if _non_linux and _ci_env
else expect_timeout
)),
# preexec_fn=unset_colors, # preexec_fn=unset_colors,
# ^TODO? get `pytest` core to expose underlying # ^TODO? get `pytest` core to expose underlying
# `pexpect.spawn()` stuff? # `pexpect.spawn()` stuff?
) )
# sanity
assert spawned.timeout == timeout
return spawned
# such that test-dep can pass input script name. # such that test-dep can pass input script name.
return _spawn # the `PexpectSpawner`, type alias. yield _spawn # the `PexpectSpawner`, type alias.
if (
spawned
and
(ptyproc := spawned.ptyproc)
):
start: float = time.time()
timeout: float = 5
while (
ptyproc.isalive()
and
(
(_time_took := (time.time() - start))
<
timeout
)
):
ptyproc.kill(signal.SIGINT)
time.sleep(0.01)
if ptyproc.isalive():
ptyproc.kill(signal.SIGKILL)
# TODO? ensure we've cleaned up any UDS-paths?
# breakpoint()
@pytest.fixture( @pytest.fixture(
@ -109,7 +164,13 @@ def ctlc(
'https://github.com/goodboy/tractor/issues/320' 'https://github.com/goodboy/tractor/issues/320'
) )
if mark.name == 'ctlcs_bish': if (
mark.name == 'ctlcs_bish'
and
use_ctlc
and
all(mark.args)
):
pytest.skip( pytest.skip(
f'Test {node} prolly uses something from the stdlib (namely `asyncio`..)\n' f'Test {node} prolly uses something from the stdlib (namely `asyncio`..)\n'
f'The test and/or underlying example script can *sometimes* run fine ' f'The test and/or underlying example script can *sometimes* run fine '
@ -214,12 +275,13 @@ def assert_before(
err_on_false=True, err_on_false=True,
**kwargs **kwargs
) )
return str(child.before.decode())
def do_ctlc( def do_ctlc(
child, child,
count: int = 3, count: int = 3,
delay: float = 0.1, delay: float|None = None,
patt: str|None = None, patt: str|None = None,
# expect repl UX to reprint the prompt after every # expect repl UX to reprint the prompt after every
@ -231,6 +293,7 @@ def do_ctlc(
) -> str|None: ) -> str|None:
before: str|None = None before: str|None = None
delay = delay or 0.1
# make sure ctl-c sends don't do anything but repeat output # make sure ctl-c sends don't do anything but repeat output
for _ in range(count): for _ in range(count):
@ -241,7 +304,10 @@ def do_ctlc(
# if you run this test manually it works just fine.. # if you run this test manually it works just fine..
if expect_prompt: if expect_prompt:
time.sleep(delay) time.sleep(delay)
child.expect(PROMPT) child.expect(
PROMPT,
timeout=(child.timeout * 2) if _ci_env else child.timeout,
)
before = str(child.before.decode()) before = str(child.before.decode())
time.sleep(delay) time.sleep(delay)

View File

@ -37,6 +37,9 @@ from .conftest import (
in_prompt_msg, in_prompt_msg,
assert_before, assert_before,
) )
from ..conftest import (
_ci_env,
)
if TYPE_CHECKING: if TYPE_CHECKING:
from ..conftest import PexpectSpawner from ..conftest import PexpectSpawner
@ -51,13 +54,14 @@ if TYPE_CHECKING:
# - recurrent root errors # - recurrent root errors
_non_linux: bool = platform.system() != 'Linux'
if platform.system() == 'Windows': if platform.system() == 'Windows':
pytest.skip( pytest.skip(
'Debugger tests have no windows support (yet)', 'Debugger tests have no windows support (yet)',
allow_module_level=True, allow_module_level=True,
) )
# TODO: was trying to this xfail style but some weird bug i see in CI # TODO: was trying to this xfail style but some weird bug i see in CI
# that's happening at collect time.. pretty soon gonna dump actions i'm # that's happening at collect time.. pretty soon gonna dump actions i'm
# thinkin... # thinkin...
@ -193,6 +197,11 @@ def test_root_actor_bp_forever(
child.expect(EOF) child.expect(EOF)
# skip on non-Linux CI
@pytest.mark.ctlcs_bish(
_non_linux,
_ci_env,
)
@pytest.mark.parametrize( @pytest.mark.parametrize(
'do_next', 'do_next',
(True, False), (True, False),
@ -258,6 +267,11 @@ def test_subactor_error(
child.expect(EOF) child.expect(EOF)
# skip on non-Linux CI
@pytest.mark.ctlcs_bish(
_non_linux,
_ci_env,
)
def test_subactor_breakpoint( def test_subactor_breakpoint(
spawn, spawn,
ctlc: bool, ctlc: bool,
@ -480,8 +494,24 @@ def test_multi_daemon_subactors(
stream. stream.
''' '''
child = spawn('multi_daemon_subactors') non_linux = _non_linux
if non_linux and ctlc:
pytest.skip(
'Ctl-c + MacOS is too unreliable/racy for this test..\n'
)
# !TODO, if someone with more patience then i wants to muck
# with the timings on this please feel free to see all the
# `non_linux` branching logic i added on my first attempt
# below!
#
# my conclusion was that if i were to run the script
# manually, and thus as slowly as a human would, the test
# would and should pass as described in this test fn, however
# after fighting with it for >= 1hr. i decided more then
# likely the more extensive `linux` testing should cover most
# regressions.
child = spawn('multi_daemon_subactors')
child.expect(PROMPT) child.expect(PROMPT)
# there can be a race for which subactor will acquire # there can be a race for which subactor will acquire
@ -511,8 +541,19 @@ def test_multi_daemon_subactors(
else: else:
raise ValueError('Neither log msg was found !?') raise ValueError('Neither log msg was found !?')
non_linux_delay: float = 0.3
if ctlc: if ctlc:
do_ctlc(child) do_ctlc(
child,
delay=(
non_linux_delay
if non_linux
else None
),
)
if non_linux:
time.sleep(1)
# NOTE: previously since we did not have clobber prevention # NOTE: previously since we did not have clobber prevention
# in the root actor this final resume could result in the debugger # in the root actor this final resume could result in the debugger
@ -543,33 +584,66 @@ def test_multi_daemon_subactors(
# assert "in use by child ('bp_forever'," in before # assert "in use by child ('bp_forever'," in before
if ctlc: if ctlc:
do_ctlc(child) do_ctlc(
child,
delay=(
non_linux_delay
if non_linux
else None
),
)
if non_linux:
time.sleep(1)
# expect another breakpoint actor entry # expect another breakpoint actor entry
child.sendline('c') child.sendline('c')
child.expect(PROMPT) child.expect(PROMPT)
try: try:
assert_before( before: str = assert_before(
child, child,
bp_forev_parts, bp_forev_parts,
) )
except AssertionError: except AssertionError:
assert_before( before: str = assert_before(
child, child,
name_error_parts, name_error_parts,
) )
else: else:
if ctlc: if ctlc:
do_ctlc(child) before: str = do_ctlc(
child,
delay=(
non_linux_delay
if non_linux
else None
),
)
if non_linux:
time.sleep(1)
# should crash with the 2nd name error (simulates # should crash with the 2nd name error (simulates
# a retry) and then the root eventually (boxed) errors # a retry) and then the root eventually (boxed) errors
# after 1 or more further bp actor entries. # after 1 or more further bp actor entries.
child.sendline('c') child.sendline('c')
child.expect(PROMPT) try:
child.expect(
PROMPT,
timeout=3,
)
except EOF:
before: str = child.before.decode()
print(
f'\n'
f'??? NEVER RXED `pdb` PROMPT ???\n'
f'\n'
f'{before}\n'
)
raise
assert_before( assert_before(
child, child,
name_error_parts, name_error_parts,
@ -689,7 +763,8 @@ def test_multi_subactors_root_errors(
@has_nested_actors @has_nested_actors
def test_multi_nested_subactors_error_through_nurseries( def test_multi_nested_subactors_error_through_nurseries(
spawn, ci_env: bool,
spawn: PexpectSpawner,
# TODO: address debugger issue for nested tree: # TODO: address debugger issue for nested tree:
# https://github.com/goodboy/tractor/issues/320 # https://github.com/goodboy/tractor/issues/320
@ -712,7 +787,16 @@ def test_multi_nested_subactors_error_through_nurseries(
for send_char in itertools.cycle(['c', 'q']): for send_char in itertools.cycle(['c', 'q']):
try: try:
child.expect(PROMPT) child.expect(
PROMPT,
timeout=(
6 if (
_non_linux
and
ci_env
) else -1
),
)
child.sendline(send_char) child.sendline(send_char)
time.sleep(0.01) time.sleep(0.01)
@ -889,6 +973,11 @@ def test_different_debug_mode_per_actor(
) )
# skip on non-Linux CI
@pytest.mark.ctlcs_bish(
_non_linux,
_ci_env,
)
def test_post_mortem_api( def test_post_mortem_api(
spawn, spawn,
ctlc: bool, ctlc: bool,
@ -1133,12 +1222,21 @@ def test_ctxep_pauses_n_maybe_ipc_breaks(
# closed so verify we see error reporting as well as # closed so verify we see error reporting as well as
# a failed crash-REPL request msg and can CTL-c our way # a failed crash-REPL request msg and can CTL-c our way
# out. # out.
# ?TODO, match depending on `tpt_proto(s)`?
# - [ ] how can we pass it into the script tho?
tpt: str = 'UDS'
if _non_linux:
tpt: str = 'TCP'
assert_before( assert_before(
child, child,
['peer IPC channel closed abruptly?', ['peer IPC channel closed abruptly?',
'another task closed this fd', 'another task closed this fd',
'Debug lock request was CANCELLED?', 'Debug lock request was CANCELLED?',
"TransportClosed: 'MsgpackUDSStream' was already closed locally ?",] f"'Msgpack{tpt}Stream' was already closed locally?",
f"TransportClosed: 'Msgpack{tpt}Stream' was already closed 'by peer'?",
]
# XXX races on whether these show/hit? # XXX races on whether these show/hit?
# 'Failed to REPl via `_pause()` You called `tractor.pause()` from an already cancelled scope!', # 'Failed to REPl via `_pause()` You called `tractor.pause()` from an already cancelled scope!',

View File

@ -31,6 +31,9 @@ from .conftest import (
PROMPT, PROMPT,
_pause_msg, _pause_msg,
) )
from ..conftest import (
no_macos,
)
import pytest import pytest
from pexpect.exceptions import ( from pexpect.exceptions import (
@ -42,6 +45,7 @@ if TYPE_CHECKING:
from ..conftest import PexpectSpawner from ..conftest import PexpectSpawner
@no_macos
def test_shield_pause( def test_shield_pause(
spawn: PexpectSpawner, spawn: PexpectSpawner,
): ):
@ -57,6 +61,7 @@ def test_shield_pause(
expect( expect(
child, child,
'Yo my child hanging..?', 'Yo my child hanging..?',
timeout=3,
) )
assert_before( assert_before(
child, child,
@ -121,7 +126,7 @@ def test_shield_pause(
child.pid, child.pid,
signal.SIGINT, signal.SIGINT,
) )
from tractor._supervise import _shutdown_msg from tractor.runtime._supervise import _shutdown_msg
expect( expect(
child, child,
# 'Shutting down actor runtime', # 'Shutting down actor runtime',

View File

@ -8,17 +8,16 @@ from pathlib import Path
import pytest import pytest
import trio import trio
import tractor import tractor
from tractor import ( from tractor import Actor
Actor, from tractor.runtime import _state
_state, from tractor.discovery import _addr
_addr,
)
@pytest.fixture @pytest.fixture
def bindspace_dir_str() -> str: def bindspace_dir_str() -> str:
rt_dir: Path = tractor._state.get_rt_dir() from tractor.runtime._state import get_rt_dir
rt_dir: Path = get_rt_dir()
bs_dir: Path = rt_dir / 'doggy' bs_dir: Path = rt_dir / 'doggy'
bs_dir_str: str = str(bs_dir) bs_dir_str: str = str(bs_dir)
assert not bs_dir.is_dir() assert not bs_dir.is_dir()

View File

@ -13,9 +13,9 @@ from tractor import (
Portal, Portal,
ipc, ipc,
msg, msg,
_state,
_addr,
) )
from tractor.runtime import _state
from tractor.discovery import _addr
@tractor.context @tractor.context
async def chk_tpts( async def chk_tpts(
@ -62,6 +62,13 @@ def test_root_passes_tpt_to_sub(
reg_addr: tuple, reg_addr: tuple,
debug_mode: bool, debug_mode: bool,
): ):
# XXX NOTE, the `reg_addr` addr won't be the same type as the
# `tpt_proto_key` would deliver here unless you pass `--tpt-proto
# <tpt_proto_key>` on the CLI.
#
# if tpt_proto_key == 'uds':
# breakpoint()
async def main(): async def main():
async with tractor.open_nursery( async with tractor.open_nursery(
enable_transports=[tpt_proto_key], enable_transports=[tpt_proto_key],

View File

@ -0,0 +1,4 @@
'''
`tractor.msg.*` sub-sys test suite.
'''

View File

@ -0,0 +1,4 @@
'''
`tractor.msg.*` test sub-pkg conf.
'''

View File

@ -5,6 +5,7 @@ Low-level functional audits for our
B~) B~)
''' '''
from __future__ import annotations
from contextlib import ( from contextlib import (
contextmanager as cm, contextmanager as cm,
# nullcontext, # nullcontext,
@ -20,7 +21,7 @@ from msgspec import (
# structs, # structs,
# msgpack, # msgpack,
Raw, Raw,
# Struct, Struct,
ValidationError, ValidationError,
) )
import pytest import pytest
@ -46,6 +47,11 @@ from tractor.msg import (
apply_codec, apply_codec,
current_codec, current_codec,
) )
from tractor.msg._codec import (
default_builtins,
mk_dec_hook,
mk_codec_from_spec,
)
from tractor.msg.types import ( from tractor.msg.types import (
log, log,
Started, Started,
@ -743,6 +749,143 @@ def test_ext_types_over_ipc(
assert exc.boxed_type is TypeError assert exc.boxed_type is TypeError
'''
Test the auto enc & dec hooks
Create a codec which will work for:
- builtins
- custom types
- lists of custom types
'''
class BytesTestClass(Struct, tag=True):
raw: bytes
def encode(self) -> bytes:
return self.raw
@classmethod
def from_bytes(self, raw: bytes) -> BytesTestClass:
return BytesTestClass(raw=raw)
class StrTestClass(Struct, tag=True):
s: str
def encode(self) -> str:
return self.s
@classmethod
def from_str(self, s: str) -> StrTestClass:
return StrTestClass(s=s)
class IntTestClass(Struct, tag=True):
num: int
def encode(self) -> int:
return self.num
@classmethod
def from_int(self, num: int) -> IntTestClass:
return IntTestClass(num=num)
builtins = tuple((
builtin
for builtin in default_builtins
if builtin is not list
))
TestClasses = (BytesTestClass, StrTestClass, IntTestClass)
TestSpec = (
*TestClasses, list[Union[*TestClasses]]
)
test_codec = mk_codec_from_spec(
spec=TestSpec
)
@tractor.context
async def child_custom_codec(
ctx: tractor.Context,
msgs: list[Union[*TestSpec]],
):
'''
Apply codec and send all msgs passed through stream
'''
with (
apply_codec(test_codec),
limit_plds(
test_codec.pld_spec,
dec_hook=mk_dec_hook(TestSpec),
ext_types=TestSpec + builtins
),
):
await ctx.started(None)
async with ctx.open_stream() as stream:
for msg in msgs:
await stream.send(msg)
def test_multi_custom_codec():
'''
Open subactor setup codec and pld_rx and wait to receive & assert from
stream
'''
msgs = [
None,
True, False,
0xdeadbeef,
.42069,
b'deadbeef',
BytesTestClass(raw=b'deadbeef'),
StrTestClass(s='deadbeef'),
IntTestClass(num=0xdeadbeef),
[
BytesTestClass(raw=b'deadbeef'),
StrTestClass(s='deadbeef'),
IntTestClass(num=0xdeadbeef),
]
]
async def main():
async with tractor.open_nursery() as an:
p: tractor.Portal = await an.start_actor(
'child',
enable_modules=[__name__],
)
async with (
p.open_context(
child_custom_codec,
msgs=msgs,
) as (ctx, _),
ctx.open_stream() as ipc
):
with (
apply_codec(test_codec),
limit_plds(
test_codec.pld_spec,
dec_hook=mk_dec_hook(TestSpec),
ext_types=TestSpec + builtins
)
):
msg_iter = iter(msgs)
async for recv_msg in ipc:
assert recv_msg == next(msg_iter)
await p.cancel_actor()
trio.run(main)
# def chk_pld_type( # def chk_pld_type(
# payload_spec: Type[Struct]|Any, # payload_spec: Type[Struct]|Any,
# pld: Any, # pld: Any,

View File

@ -61,7 +61,7 @@ async def maybe_expect_raises(
Async wrapper for ensuring errors propagate from the inner scope. Async wrapper for ensuring errors propagate from the inner scope.
''' '''
if tractor._state.debug_mode(): if tractor.debug_mode():
timeout += 999 timeout += 999
with trio.fail_after(timeout): with trio.fail_after(timeout):

View File

@ -0,0 +1,240 @@
'''
Unit tests for `tractor.msg.pretty_struct`
private-field filtering in `pformat()`.
'''
import pytest
from tractor.msg.pretty_struct import (
Struct,
pformat,
iter_struct_ppfmt_lines,
)
from tractor.msg._codec import (
MsgDec,
mk_dec,
)
# ------ test struct definitions ------ #
class PublicOnly(Struct):
'''
All-public fields for baseline testing.
'''
name: str = 'alice'
age: int = 30
class PrivateOnly(Struct):
'''
Only underscore-prefixed (private) fields.
'''
_secret: str = 'hidden'
_internal: int = 99
class MixedFields(Struct):
'''
Mix of public and private fields.
'''
name: str = 'bob'
_hidden: int = 42
value: float = 3.14
_meta: str = 'internal'
class Inner(
Struct,
frozen=True,
):
'''
Frozen inner struct with a private field,
for nesting tests.
'''
x: int = 1
_secret: str = 'nope'
class Outer(Struct):
'''
Outer struct nesting an `Inner`.
'''
label: str = 'outer'
inner: Inner = Inner()
class EmptyStruct(Struct):
'''
Struct with zero fields.
'''
pass
# ------ tests ------ #
@pytest.mark.parametrize(
'struct_and_expected',
[
(
PublicOnly(),
{
'shown': ['name', 'age'],
'hidden': [],
},
),
(
MixedFields(),
{
'shown': ['name', 'value'],
'hidden': ['_hidden', '_meta'],
},
),
(
PrivateOnly(),
{
'shown': [],
'hidden': ['_secret', '_internal'],
},
),
],
ids=[
'all-public',
'mixed-pub-priv',
'all-private',
],
)
def test_field_visibility_in_pformat(
struct_and_expected: tuple[
Struct,
dict[str, list[str]],
],
):
'''
Verify `pformat()` shows public fields
and hides `_`-prefixed private fields.
'''
(
struct,
expected,
) = struct_and_expected
output: str = pformat(struct)
for field_name in expected['shown']:
assert field_name in output, (
f'{field_name!r} should appear in:\n'
f'{output}'
)
for field_name in expected['hidden']:
assert field_name not in output, (
f'{field_name!r} should NOT appear in:\n'
f'{output}'
)
def test_iter_ppfmt_lines_skips_private():
'''
Directly verify `iter_struct_ppfmt_lines()`
never yields tuples with `_`-prefixed field
names.
'''
struct = MixedFields()
lines: list[tuple[str, str]] = list(
iter_struct_ppfmt_lines(
struct,
field_indent=2,
)
)
# should have lines for public fields only
assert len(lines) == 2
for _prefix, line_content in lines:
field_name: str = (
line_content.split(':')[0].strip()
)
assert not field_name.startswith('_'), (
f'private field leaked: {field_name!r}'
)
def test_nested_struct_filters_inner_private():
'''
Verify that nested struct's private fields
are also filtered out during recursion.
'''
outer = Outer()
output: str = pformat(outer)
# outer's public field
assert 'label' in output
# inner's public field (recursed into)
assert 'x' in output
# inner's private field must be hidden
assert '_secret' not in output
def test_empty_struct_pformat():
'''
An empty struct should produce a valid
`pformat()` result with no field lines.
'''
output: str = pformat(EmptyStruct())
assert 'EmptyStruct(' in output
assert output.rstrip().endswith(')')
# no field lines => only struct header+footer
lines: list[tuple[str, str]] = list(
iter_struct_ppfmt_lines(
EmptyStruct(),
field_indent=2,
)
)
assert lines == []
def test_real_msgdec_pformat_hides_private():
'''
Verify `pformat()` on a real `MsgDec`
hides the `_dec` internal field.
NOTE: `MsgDec.__repr__` is custom and does
NOT call `pformat()`, so we call it directly.
'''
dec: MsgDec = mk_dec(spec=int)
output: str = pformat(dec)
# the private `_dec` field should be filtered
assert '_dec' not in output
# but the struct type name should be present
assert 'MsgDec(' in output
def test_pformat_repr_integration():
'''
Verify that `Struct.__repr__()` (which calls
`pformat()`) also hides private fields for
custom structs that do NOT override `__repr__`.
'''
mixed = MixedFields()
output: str = repr(mixed)
assert 'name' in output
assert 'value' in output
assert '_hidden' not in output
assert '_meta' not in output

View File

@ -1,7 +1,12 @@
""" '''
Bidirectional streaming. Audit the simplest inter-actor bidirectional (streaming)
msg patterns.
""" '''
from __future__ import annotations
from typing import (
Callable,
)
import pytest import pytest
import trio import trio
import tractor import tractor
@ -9,10 +14,8 @@ import tractor
@tractor.context @tractor.context
async def simple_rpc( async def simple_rpc(
ctx: tractor.Context, ctx: tractor.Context,
data: int, data: int,
) -> None: ) -> None:
''' '''
Test a small ping-pong server. Test a small ping-pong server.
@ -39,15 +42,13 @@ async def simple_rpc(
@tractor.context @tractor.context
async def simple_rpc_with_forloop( async def simple_rpc_with_forloop(
ctx: tractor.Context, ctx: tractor.Context,
data: int, data: int,
) -> None: ) -> None:
"""Same as previous test but using ``async for`` syntax/api. '''
Same as previous test but using `async for` syntax/api.
"""
'''
# signal to parent that we're up # signal to parent that we're up
await ctx.started(data + 1) await ctx.started(data + 1)
@ -68,21 +69,37 @@ async def simple_rpc_with_forloop(
@pytest.mark.parametrize( @pytest.mark.parametrize(
'use_async_for', 'use_async_for',
[True, False], [
True,
False,
],
ids='use_async_for={}'.format,
) )
@pytest.mark.parametrize( @pytest.mark.parametrize(
'server_func', 'server_func',
[simple_rpc, simple_rpc_with_forloop], [
simple_rpc,
simple_rpc_with_forloop,
],
ids='server_func={}'.format,
) )
def test_simple_rpc(server_func, use_async_for): def test_simple_rpc(
server_func: Callable,
use_async_for: bool,
loglevel: str,
debug_mode: bool,
):
''' '''
The simplest request response pattern. The simplest request response pattern.
''' '''
async def main(): async def main():
async with tractor.open_nursery() as n: with trio.fail_after(6):
async with tractor.open_nursery(
portal = await n.start_actor( loglevel=loglevel,
debug_mode=debug_mode,
) as an:
portal: tractor.Portal = await an.start_actor(
'rpc_server', 'rpc_server',
enable_modules=[__name__], enable_modules=[__name__],
) )

View File

@ -98,7 +98,8 @@ def test_ipc_channel_break_during_stream(
expect_final_exc = TransportClosed expect_final_exc = TransportClosed
mod: ModuleType = import_path( mod: ModuleType = import_path(
examples_dir() / 'advanced_faults' examples_dir()
/ 'advanced_faults'
/ 'ipc_failure_during_stream.py', / 'ipc_failure_during_stream.py',
root=examples_dir(), root=examples_dir(),
consider_namespace_packages=False, consider_namespace_packages=False,
@ -113,8 +114,9 @@ def test_ipc_channel_break_during_stream(
if ( if (
# only expect EoC if trans is broken on the child side, # only expect EoC if trans is broken on the child side,
ipc_break['break_child_ipc_after'] is not False ipc_break['break_child_ipc_after'] is not False
and
# AND we tell the child to call `MsgStream.aclose()`. # AND we tell the child to call `MsgStream.aclose()`.
and pre_aclose_msgstream pre_aclose_msgstream
): ):
# expect_final_exc = trio.EndOfChannel # expect_final_exc = trio.EndOfChannel
# ^XXX NOPE! XXX^ since now `.open_stream()` absorbs this # ^XXX NOPE! XXX^ since now `.open_stream()` absorbs this
@ -144,9 +146,6 @@ def test_ipc_channel_break_during_stream(
# a user sending ctl-c by raising a KBI. # a user sending ctl-c by raising a KBI.
if pre_aclose_msgstream: if pre_aclose_msgstream:
expect_final_exc = KeyboardInterrupt expect_final_exc = KeyboardInterrupt
if tpt_proto == 'uds':
expect_final_exc = TransportClosed
expect_final_cause = trio.BrokenResourceError
# XXX OLD XXX # XXX OLD XXX
# if child calls `MsgStream.aclose()` then expect EoC. # if child calls `MsgStream.aclose()` then expect EoC.
@ -160,16 +159,13 @@ def test_ipc_channel_break_during_stream(
ipc_break['break_child_ipc_after'] is not False ipc_break['break_child_ipc_after'] is not False
and ( and (
ipc_break['break_parent_ipc_after'] ipc_break['break_parent_ipc_after']
> ipc_break['break_child_ipc_after'] >
ipc_break['break_child_ipc_after']
) )
): ):
if pre_aclose_msgstream: if pre_aclose_msgstream:
expect_final_exc = KeyboardInterrupt expect_final_exc = KeyboardInterrupt
if tpt_proto == 'uds':
expect_final_exc = TransportClosed
expect_final_cause = trio.BrokenResourceError
# NOTE when the parent IPC side dies (even if the child does as well # NOTE when the parent IPC side dies (even if the child does as well
# but the child fails BEFORE the parent) we always expect the # but the child fails BEFORE the parent) we always expect the
# IPC layer to raise a closed-resource, NEVER do we expect # IPC layer to raise a closed-resource, NEVER do we expect
@ -248,8 +244,15 @@ def test_ipc_channel_break_during_stream(
# get raw instance from pytest wrapper # get raw instance from pytest wrapper
value = excinfo.value value = excinfo.value
if isinstance(value, ExceptionGroup): if isinstance(value, ExceptionGroup):
excs = value.exceptions excs: tuple[Exception] = value.exceptions
assert len(excs) == 1 assert (
len(excs) <= 2
and
all(
isinstance(exc, TransportClosed)
for exc in excs
)
)
final_exc = excs[0] final_exc = excs[0]
assert isinstance(final_exc, expect_final_exc) assert isinstance(final_exc, expect_final_exc)

View File

@ -17,8 +17,8 @@ from tractor._testing import (
from .conftest import no_windows from .conftest import no_windows
def is_win(): _non_linux: bool = platform.system() != 'Linux'
return platform.system() == 'Windows' _friggin_windows: bool = platform.system() == 'Windows'
async def assert_err(delay=0): async def assert_err(delay=0):
@ -431,7 +431,7 @@ async def test_nested_multierrors(loglevel, start_method):
for subexc in err.exceptions: for subexc in err.exceptions:
# verify first level actor errors are wrapped as remote # verify first level actor errors are wrapped as remote
if is_win(): if _friggin_windows:
# windows is often too slow and cancellation seems # windows is often too slow and cancellation seems
# to happen before an actor is spawned # to happen before an actor is spawned
@ -464,7 +464,7 @@ async def test_nested_multierrors(loglevel, start_method):
# XXX not sure what's up with this.. # XXX not sure what's up with this..
# on windows sometimes spawning is just too slow and # on windows sometimes spawning is just too slow and
# we get back the (sent) cancel signal instead # we get back the (sent) cancel signal instead
if is_win(): if _friggin_windows:
if isinstance(subexc, tractor.RemoteActorError): if isinstance(subexc, tractor.RemoteActorError):
assert subexc.boxed_type in ( assert subexc.boxed_type in (
BaseExceptionGroup, BaseExceptionGroup,
@ -490,7 +490,7 @@ def test_cancel_via_SIGINT(
"""Ensure that a control-C (SIGINT) signal cancels both the parent and """Ensure that a control-C (SIGINT) signal cancels both the parent and
child processes in trionic fashion child processes in trionic fashion
""" """
pid = os.getpid() pid: int = os.getpid()
async def main(): async def main():
with trio.fail_after(2): with trio.fail_after(2):
@ -507,19 +507,31 @@ def test_cancel_via_SIGINT(
@no_windows @no_windows
def test_cancel_via_SIGINT_other_task( def test_cancel_via_SIGINT_other_task(
loglevel, loglevel: str,
start_method, start_method: str,
spawn_backend, spawn_backend: str,
): ):
"""Ensure that a control-C (SIGINT) signal cancels both the parent '''
and child processes in trionic fashion even a subprocess is started Ensure that a control-C (SIGINT) signal cancels both the parent
from a seperate ``trio`` child task. and child processes in trionic fashion even a subprocess is
""" started from a seperate ``trio`` child task.
pid = os.getpid()
timeout: float = 2 '''
if is_win(): # smh from .conftest import cpu_scaling_factor
pid: int = os.getpid()
timeout: float = (
4 if _non_linux
else 2
)
if _friggin_windows: # smh
timeout += 1 timeout += 1
# add latency headroom for CPU freq scaling (auto-cpufreq et al.)
headroom: float = cpu_scaling_factor()
if headroom != 1.:
timeout *= headroom
async def spawn_and_sleep_forever( async def spawn_and_sleep_forever(
task_status=trio.TASK_STATUS_IGNORED task_status=trio.TASK_STATUS_IGNORED
): ):
@ -644,7 +656,11 @@ def test_cancel_while_childs_child_in_sync_sleep(
# #
# delay = 1 # no AssertionError in eg, TooSlowError raised. # delay = 1 # no AssertionError in eg, TooSlowError raised.
# delay = 2 # is AssertionError in eg AND no TooSlowError !? # delay = 2 # is AssertionError in eg AND no TooSlowError !?
delay = 4 # is AssertionError in eg AND no _cs cancellation. # is AssertionError in eg AND no _cs cancellation.
delay = (
6 if _non_linux
else 4
)
with trio.fail_after(delay) as _cs: with trio.fail_after(delay) as _cs:
# with trio.CancelScope() as cs: # with trio.CancelScope() as cs:
@ -696,7 +712,7 @@ def test_fast_graceful_cancel_when_spawn_task_in_soft_proc_wait_for_daemon(
kbi_delay = 0.5 kbi_delay = 0.5
timeout: float = 2.9 timeout: float = 2.9
if is_win(): # smh if _friggin_windows: # smh
timeout += 1 timeout += 1
async def main(): async def main():

View File

@ -18,16 +18,15 @@ from tractor import RemoteActorError
async def aio_streamer( async def aio_streamer(
from_trio: asyncio.Queue, chan: tractor.to_asyncio.LinkedTaskChannel,
to_trio: trio.abc.SendChannel,
) -> trio.abc.ReceiveChannel: ) -> trio.abc.ReceiveChannel:
# required first msg to sync caller # required first msg to sync caller
to_trio.send_nowait(None) chan.started_nowait(None)
from itertools import cycle from itertools import cycle
for i in cycle(range(10)): for i in cycle(range(10)):
to_trio.send_nowait(i) chan.send_nowait(i)
await asyncio.sleep(0.01) await asyncio.sleep(0.01)
@ -69,7 +68,7 @@ async def wrapper_mngr(
else: else:
async with tractor.to_asyncio.open_channel_from( async with tractor.to_asyncio.open_channel_from(
aio_streamer, aio_streamer,
) as (first, from_aio): ) as (from_aio, first):
assert not first assert not first
# cache it so next task uses broadcast receiver # cache it so next task uses broadcast receiver

View File

@ -10,7 +10,19 @@ from tractor._testing import tractor_test
MESSAGE = 'tractoring at full speed' MESSAGE = 'tractoring at full speed'
def test_empty_mngrs_input_raises() -> None: def test_empty_mngrs_input_raises(
tpt_proto: str,
) -> None:
# TODO, the `open_actor_cluster()` teardown hangs
# intermittently on UDS when `gather_contexts(mngrs=())`
# raises `ValueError` mid-setup; likely a race in the
# actor-nursery cleanup vs UDS socket shutdown. Needs
# a deeper look at `._clustering`/`._supervise` teardown
# paths with the UDS transport.
if tpt_proto == 'uds':
pytest.skip(
'actor-cluster teardown hangs intermittently on UDS'
)
async def main(): async def main():
with trio.fail_after(3): with trio.fail_after(3):
@ -56,13 +68,27 @@ async def worker(
print(msg) print(msg)
assert msg == MESSAGE assert msg == MESSAGE
# TODO: does this ever cause a hang # ?TODO, does this ever cause a hang?
# assert 0 # assert 0
# ?TODO, but needs a fn-scoped tpt_proto fixture..
# @pytest.mark.no_tpt('uds')
@tractor_test @tractor_test
async def test_streaming_to_actor_cluster() -> None: async def test_streaming_to_actor_cluster(
tpt_proto: str,
):
'''
Open an actor "cluster" using the (experimental) `._clustering`
API and conduct standard inter-task-ctx streaming.
'''
if tpt_proto == 'uds':
pytest.skip(
f'Test currently fails with tpt-proto={tpt_proto!r}\n'
)
with trio.fail_after(6):
async with ( async with (
open_actor_cluster(modules=[__name__]) as portals, open_actor_cluster(modules=[__name__]) as portals,

View File

@ -9,6 +9,7 @@ from itertools import count
import math import math
import platform import platform
from pprint import pformat from pprint import pformat
import sys
from typing import ( from typing import (
Callable, Callable,
) )
@ -25,7 +26,7 @@ from tractor._exceptions import (
StreamOverrun, StreamOverrun,
ContextCancelled, ContextCancelled,
) )
from tractor._state import current_ipc_ctx from tractor.runtime._state import current_ipc_ctx
from tractor._testing import ( from tractor._testing import (
tractor_test, tractor_test,
@ -938,9 +939,14 @@ def test_one_end_stream_not_opened(
''' '''
overrunner, buf_size_increase, entrypoint = overrun_by overrunner, buf_size_increase, entrypoint = overrun_by
from tractor._runtime import Actor from tractor.runtime._runtime import Actor
buf_size = buf_size_increase + Actor.msg_buffer_size buf_size = buf_size_increase + Actor.msg_buffer_size
timeout: float = (
1 if sys.platform == 'linux'
else 3
)
async def main(): async def main():
async with tractor.open_nursery( async with tractor.open_nursery(
debug_mode=debug_mode, debug_mode=debug_mode,
@ -950,7 +956,7 @@ def test_one_end_stream_not_opened(
enable_modules=[__name__], enable_modules=[__name__],
) )
with trio.fail_after(1): with trio.fail_after(timeout):
async with portal.open_context( async with portal.open_context(
entrypoint, entrypoint,
) as (ctx, sent): ) as (ctx, sent):

View File

@ -1,11 +1,14 @@
""" '''
Actor "discovery" testing Discovery subsystem via a "registrar" actor scenarios.
"""
'''
import os import os
import signal import signal
import platform import platform
from functools import partial from functools import partial
import itertools import itertools
import time
from typing import Callable
import psutil import psutil
import pytest import pytest
@ -17,9 +20,11 @@ import trio
@tractor_test @tractor_test
async def test_reg_then_unreg(reg_addr): async def test_reg_then_unreg(
reg_addr: tuple,
):
actor = tractor.current_actor() actor = tractor.current_actor()
assert actor.is_arbiter assert actor.is_registrar
assert len(actor._registry) == 1 # only self is registered assert len(actor._registry) == 1 # only self is registered
async with tractor.open_nursery( async with tractor.open_nursery(
@ -27,10 +32,10 @@ async def test_reg_then_unreg(reg_addr):
) as n: ) as n:
portal = await n.start_actor('actor', enable_modules=[__name__]) portal = await n.start_actor('actor', enable_modules=[__name__])
uid = portal.channel.uid uid = portal.channel.aid.uid
async with tractor.get_registry(reg_addr) as aportal: async with tractor.get_registry(reg_addr) as aportal:
# this local actor should be the arbiter # this local actor should be the registrar
assert actor is aportal.actor assert actor is aportal.actor
async with tractor.wait_for_actor('actor'): async with tractor.wait_for_actor('actor'):
@ -82,11 +87,15 @@ async def say_hello_use_wait(
@tractor_test @tractor_test
@pytest.mark.parametrize('func', [say_hello, say_hello_use_wait]) @pytest.mark.parametrize(
'func',
[say_hello,
say_hello_use_wait]
)
async def test_trynamic_trio( async def test_trynamic_trio(
func, func: Callable,
start_method, start_method: str,
reg_addr, reg_addr: tuple,
): ):
''' '''
Root actor acting as the "director" and running one-shot-task-actors Root actor acting as the "director" and running one-shot-task-actors
@ -119,7 +128,10 @@ async def stream_forever():
await trio.sleep(0.01) await trio.sleep(0.01)
async def cancel(use_signal, delay=0): async def cancel(
use_signal: bool,
delay: float = 0,
):
# hold on there sally # hold on there sally
await trio.sleep(delay) await trio.sleep(delay)
@ -132,15 +144,17 @@ async def cancel(use_signal, delay=0):
raise KeyboardInterrupt raise KeyboardInterrupt
async def stream_from(portal): async def stream_from(portal: tractor.Portal):
async with portal.open_stream_from(stream_forever) as stream: async with portal.open_stream_from(stream_forever) as stream:
async for value in stream: async for value in stream:
print(value) print(value)
async def unpack_reg(actor_or_portal): async def unpack_reg(
actor_or_portal: tractor.Portal|tractor.Actor,
):
''' '''
Get and unpack a "registry" RPC request from the "arbiter" registry Get and unpack a "registry" RPC request from the registrar
system. system.
''' '''
@ -149,7 +163,10 @@ async def unpack_reg(actor_or_portal):
else: else:
msg = await actor_or_portal.run_from_ns('self', 'get_registry') msg = await actor_or_portal.run_from_ns('self', 'get_registry')
return {tuple(key.split('.')): val for key, val in msg.items()} return {
tuple(key.split('.')): val
for key, val in msg.items()
}
async def spawn_and_check_registry( async def spawn_and_check_registry(
@ -173,24 +190,26 @@ async def spawn_and_check_registry(
registry_addrs=[reg_addr], registry_addrs=[reg_addr],
debug_mode=debug_mode, debug_mode=debug_mode,
): ):
async with tractor.get_registry(reg_addr) as portal: async with tractor.get_registry(
addr=reg_addr,
) as portal:
# runtime needs to be up to call this # runtime needs to be up to call this
actor = tractor.current_actor() actor = tractor.current_actor()
if remote_arbiter: if remote_arbiter:
assert not actor.is_arbiter assert not actor.is_registrar
if actor.is_arbiter: if actor.is_registrar:
extra = 1 # arbiter is local root actor extra = 1 # registrar is local root actor
get_reg = partial(unpack_reg, actor) get_reg = partial(unpack_reg, actor)
else: else:
get_reg = partial(unpack_reg, portal) get_reg = partial(unpack_reg, portal)
extra = 2 # local root actor + remote arbiter extra = 2 # local root actor + remote registrar
# ensure current actor is registered # ensure current actor is registered
registry: dict = await get_reg() registry: dict = await get_reg()
assert actor.uid in registry assert actor.aid.uid in registry
try: try:
async with tractor.open_nursery() as an: async with tractor.open_nursery() as an:
@ -238,22 +257,35 @@ async def spawn_and_check_registry(
# all subactors should have de-registered # all subactors should have de-registered
registry = await get_reg() registry = await get_reg()
start: float = time.time()
while (
not (len(registry) == extra)
and
(time.time() - start) < 5
):
print(
f'Waiting for remaining subs to dereg..\n'
f'{registry!r}\n'
)
await trio.sleep(0.3)
else:
assert len(registry) == extra assert len(registry) == extra
assert actor.uid in registry
assert actor.aid.uid in registry
@pytest.mark.parametrize('use_signal', [False, True]) @pytest.mark.parametrize('use_signal', [False, True])
@pytest.mark.parametrize('with_streaming', [False, True]) @pytest.mark.parametrize('with_streaming', [False, True])
def test_subactors_unregister_on_cancel( def test_subactors_unregister_on_cancel(
debug_mode: bool, debug_mode: bool,
start_method, start_method: str,
use_signal, use_signal: bool,
reg_addr, reg_addr: tuple,
with_streaming, with_streaming: bool,
): ):
''' '''
Verify that cancelling a nursery results in all subactors Verify that cancelling a nursery results in all subactors
deregistering themselves with the arbiter. deregistering themselves with the registrar.
''' '''
with pytest.raises(KeyboardInterrupt): with pytest.raises(KeyboardInterrupt):
@ -274,15 +306,17 @@ def test_subactors_unregister_on_cancel(
def test_subactors_unregister_on_cancel_remote_daemon( def test_subactors_unregister_on_cancel_remote_daemon(
daemon: subprocess.Popen, daemon: subprocess.Popen,
debug_mode: bool, debug_mode: bool,
start_method, start_method: str,
use_signal, use_signal: bool,
reg_addr, reg_addr: tuple,
with_streaming, with_streaming: bool,
): ):
"""Verify that cancelling a nursery results in all subactors '''
deregistering themselves with a **remote** (not in the local process Verify that cancelling a nursery results in all subactors
tree) arbiter. deregistering themselves with a **remote** (not in the local
""" process tree) registrar.
'''
with pytest.raises(KeyboardInterrupt): with pytest.raises(KeyboardInterrupt):
trio.run( trio.run(
partial( partial(
@ -325,20 +359,24 @@ async def close_chans_before_nursery(
try: try:
get_reg = partial(unpack_reg, aportal) get_reg = partial(unpack_reg, aportal)
async with tractor.open_nursery() as tn: async with tractor.open_nursery() as an:
portal1 = await tn.start_actor( portal1 = await an.start_actor(
name='consumer1', enable_modules=[__name__]) name='consumer1',
portal2 = await tn.start_actor( enable_modules=[__name__],
'consumer2', enable_modules=[__name__]) )
portal2 = await an.start_actor(
'consumer2',
enable_modules=[__name__],
)
# TODO: compact this back as was in last commit once async with (
# 3.9+, see https://github.com/goodboy/tractor/issues/207 portal1.open_stream_from(
async with portal1.open_stream_from(
stream_forever stream_forever
) as agen1: ) as agen1,
async with portal2.open_stream_from( portal2.open_stream_from(
stream_forever stream_forever
) as agen2: ) as agen2,
):
async with ( async with (
collapse_eg(), collapse_eg(),
trio.open_nursery() as tn, trio.open_nursery() as tn,
@ -349,7 +387,7 @@ async def close_chans_before_nursery(
await streamer(agen2) await streamer(agen2)
finally: finally:
# Kill the root nursery thus resulting in # Kill the root nursery thus resulting in
# normal arbiter channel ops to fail during # normal registrar channel ops to fail during
# teardown. It doesn't seem like this is # teardown. It doesn't seem like this is
# reliably triggered by an external SIGINT. # reliably triggered by an external SIGINT.
# tractor.current_actor()._root_nursery.cancel_scope.cancel() # tractor.current_actor()._root_nursery.cancel_scope.cancel()
@ -361,27 +399,30 @@ async def close_chans_before_nursery(
# also kill off channels cuz why not # also kill off channels cuz why not
await agen1.aclose() await agen1.aclose()
await agen2.aclose() await agen2.aclose()
finally: finally:
with trio.CancelScope(shield=True): with trio.CancelScope(shield=True):
await trio.sleep(1) await trio.sleep(1)
# all subactors should have de-registered # all subactors should have de-registered
registry = await get_reg() registry = await get_reg()
assert portal1.channel.uid not in registry assert portal1.channel.aid.uid not in registry
assert portal2.channel.uid not in registry assert portal2.channel.aid.uid not in registry
assert len(registry) == entries_at_end assert len(registry) == entries_at_end
@pytest.mark.parametrize('use_signal', [False, True]) @pytest.mark.parametrize('use_signal', [False, True])
def test_close_channel_explicit( def test_close_channel_explicit(
start_method, start_method: str,
use_signal, use_signal: bool,
reg_addr, reg_addr: tuple,
): ):
"""Verify that closing a stream explicitly and killing the actor's '''
Verify that closing a stream explicitly and killing the actor's
"root nursery" **before** the containing nursery tears down also "root nursery" **before** the containing nursery tears down also
results in subactor(s) deregistering from the arbiter. results in subactor(s) deregistering from the registrar.
"""
'''
with pytest.raises(KeyboardInterrupt): with pytest.raises(KeyboardInterrupt):
trio.run( trio.run(
partial( partial(
@ -394,16 +435,18 @@ def test_close_channel_explicit(
@pytest.mark.parametrize('use_signal', [False, True]) @pytest.mark.parametrize('use_signal', [False, True])
def test_close_channel_explicit_remote_arbiter( def test_close_channel_explicit_remote_registrar(
daemon: subprocess.Popen, daemon: subprocess.Popen,
start_method, start_method: str,
use_signal, use_signal: bool,
reg_addr, reg_addr: tuple,
): ):
"""Verify that closing a stream explicitly and killing the actor's '''
Verify that closing a stream explicitly and killing the actor's
"root nursery" **before** the containing nursery tears down also "root nursery" **before** the containing nursery tears down also
results in subactor(s) deregistering from the arbiter. results in subactor(s) deregistering from the registrar.
"""
'''
with pytest.raises(KeyboardInterrupt): with pytest.raises(KeyboardInterrupt):
trio.run( trio.run(
partial( partial(
@ -413,3 +456,65 @@ def test_close_channel_explicit_remote_arbiter(
remote_arbiter=True, remote_arbiter=True,
), ),
) )
@tractor.context
async def kill_transport(
ctx: tractor.Context,
) -> None:
await ctx.started()
actor: tractor.Actor = tractor.current_actor()
actor.ipc_server.cancel()
await trio.sleep_forever()
# @pytest.mark.parametrize('use_signal', [False, True])
def test_stale_entry_is_deleted(
debug_mode: bool,
daemon: subprocess.Popen,
start_method: str,
reg_addr: tuple,
):
'''
Ensure that when a stale entry is detected in the registrar's
table that the `find_actor()` API takes care of deleting the
stale entry and not delivering a bad portal.
'''
async def main():
name: str = 'transport_fails_actor'
_reg_ptl: tractor.Portal
an: tractor.ActorNursery
async with (
tractor.open_nursery(
debug_mode=debug_mode,
registry_addrs=[reg_addr],
) as an,
tractor.get_registry(reg_addr) as _reg_ptl,
):
ptl: tractor.Portal = await an.start_actor(
name,
enable_modules=[__name__],
)
async with ptl.open_context(
kill_transport,
) as (first, ctx):
async with tractor.find_actor(
name,
registry_addrs=[reg_addr],
) as maybe_portal:
# because the transitive
# `._discovery.maybe_open_portal()` call should
# fail and implicitly call `.delete_addr()`
assert maybe_portal is None
registry: dict = await unpack_reg(_reg_ptl)
assert ptl.chan.aid.uid not in registry
# should fail since we knocked out the IPC tpt XD
await ptl.cancel_actor()
await an.cancel()
trio.run(main)

View File

@ -9,12 +9,17 @@ import sys
import subprocess import subprocess
import platform import platform
import shutil import shutil
from typing import Callable
import pytest import pytest
import tractor
from tractor._testing import ( from tractor._testing import (
examples_dir, examples_dir,
) )
_non_linux: bool = platform.system() != 'Linux'
_friggin_macos: bool = platform.system() == 'Darwin'
@pytest.fixture @pytest.fixture
def run_example_in_subproc( def run_example_in_subproc(
@ -89,8 +94,10 @@ def run_example_in_subproc(
for f in p[2] for f in p[2]
if ( if (
'__' not in f '__' not in f # ignore any pkg-mods
and f[0] != '_' # ignore any `__pycache__` subdir
and '__pycache__' not in str(p[0])
and f[0] != '_' # ignore any WIP "examplel mods"
and 'debugging' not in p[0] and 'debugging' not in p[0]
and 'integration' not in p[0] and 'integration' not in p[0]
and 'advanced_faults' not in p[0] and 'advanced_faults' not in p[0]
@ -101,8 +108,10 @@ def run_example_in_subproc(
ids=lambda t: t[1], ids=lambda t: t[1],
) )
def test_example( def test_example(
run_example_in_subproc, run_example_in_subproc: Callable,
example_script, example_script: str,
test_log: tractor.log.StackLevelAdapter,
ci_env: bool,
): ):
''' '''
Load and run scripts from this repo's ``examples/`` dir as a user Load and run scripts from this repo's ``examples/`` dir as a user
@ -116,9 +125,39 @@ def test_example(
''' '''
ex_file: str = os.path.join(*example_script) ex_file: str = os.path.join(*example_script)
if 'rpc_bidir_streaming' in ex_file and sys.version_info < (3, 9): if (
'rpc_bidir_streaming' in ex_file
and
sys.version_info < (3, 9)
):
pytest.skip("2-way streaming example requires py3.9 async with syntax") pytest.skip("2-way streaming example requires py3.9 async with syntax")
if (
'full_fledged_streaming_service' in ex_file
and
_friggin_macos
and
ci_env
):
pytest.skip(
'Streaming example is too flaky in CI\n'
'AND their competitor runs this CI service..\n'
'This test does run just fine "in person" however..'
)
from .conftest import cpu_scaling_factor
timeout: float = (
60
if ci_env and _non_linux
else 16
)
# add latency headroom for CPU freq scaling (auto-cpufreq et al.)
headroom: float = cpu_scaling_factor()
if headroom != 1.:
timeout *= headroom
with open(ex_file, 'r') as ex: with open(ex_file, 'r') as ex:
code = ex.read() code = ex.read()
@ -126,9 +165,12 @@ def test_example(
err = None err = None
try: try:
if not proc.poll(): if not proc.poll():
_, err = proc.communicate(timeout=15) _, err = proc.communicate(timeout=timeout)
except subprocess.TimeoutExpired as e: except subprocess.TimeoutExpired as e:
test_log.exception(
f'Example failed to finish within {timeout}s ??\n'
)
proc.kill() proc.kill()
err = e.stderr err = e.stderr

View File

@ -26,8 +26,8 @@ from tractor import (
to_asyncio, to_asyncio,
RemoteActorError, RemoteActorError,
ContextCancelled, ContextCancelled,
_state,
) )
from tractor.runtime import _state
from tractor.trionics import BroadcastReceiver from tractor.trionics import BroadcastReceiver
from tractor._testing import expect_ctxc from tractor._testing import expect_ctxc
@ -47,12 +47,11 @@ async def sleep_and_err(
# just signature placeholders for compat with # just signature placeholders for compat with
# ``to_asyncio.open_channel_from()`` # ``to_asyncio.open_channel_from()``
to_trio: trio.MemorySendChannel|None = None, chan: to_asyncio.LinkedTaskChannel|None = None,
from_trio: asyncio.Queue|None = None,
): ):
if to_trio: if chan:
to_trio.send_nowait('start') chan.started_nowait('start')
await asyncio.sleep(sleep_for) await asyncio.sleep(sleep_for)
assert 0 assert 0
@ -238,7 +237,7 @@ async def trio_ctx(
trio.open_nursery() as tn, trio.open_nursery() as tn,
tractor.to_asyncio.open_channel_from( tractor.to_asyncio.open_channel_from(
sleep_and_err, sleep_and_err,
) as (first, chan), ) as (chan, first),
): ):
assert first == 'start' assert first == 'start'
@ -399,7 +398,7 @@ async def no_to_trio_in_args():
async def push_from_aio_task( async def push_from_aio_task(
sequence: Iterable, sequence: Iterable,
to_trio: trio.abc.SendChannel, chan: to_asyncio.LinkedTaskChannel,
expect_cancel: False, expect_cancel: False,
fail_early: bool, fail_early: bool,
exit_early: bool, exit_early: bool,
@ -407,15 +406,12 @@ async def push_from_aio_task(
) -> None: ) -> None:
try: try:
# print('trying breakpoint')
# breakpoint()
# sync caller ctx manager # sync caller ctx manager
to_trio.send_nowait(True) chan.started_nowait(True)
for i in sequence: for i in sequence:
print(f'asyncio sending {i}') print(f'asyncio sending {i}')
to_trio.send_nowait(i) chan.send_nowait(i)
await asyncio.sleep(0.001) await asyncio.sleep(0.001)
if ( if (
@ -478,7 +474,7 @@ async def stream_from_aio(
trio_exit_early trio_exit_early
)) ))
) as (first, chan): ) as (chan, first):
assert first is True assert first is True
@ -732,15 +728,21 @@ def test_aio_errors_and_channel_propagates_and_closes(
async def aio_echo_server( async def aio_echo_server(
to_trio: trio.MemorySendChannel, chan: to_asyncio.LinkedTaskChannel,
from_trio: asyncio.Queue,
) -> None: ) -> None:
'''
An IPC-msg "echo server" with msgs received and relayed by
a parent `trio.Task` into a child `asyncio.Task`
and then repeated back to that local parent (`trio.Task`)
and sent again back to the original calling remote actor.
to_trio.send_nowait('start') '''
# same semantics as `trio.TaskStatus.started()`
chan.started_nowait('start')
while True: while True:
try: try:
msg = await from_trio.get() msg = await chan.get()
except to_asyncio.TrioTaskExited: except to_asyncio.TrioTaskExited:
print( print(
'breaking aio echo loop due to `trio` exit!' 'breaking aio echo loop due to `trio` exit!'
@ -748,7 +750,7 @@ async def aio_echo_server(
break break
# echo the msg back # echo the msg back
to_trio.send_nowait(msg) chan.send_nowait(msg)
# if we get the terminate sentinel # if we get the terminate sentinel
# break the echo loop # break the echo loop
@ -765,7 +767,10 @@ async def trio_to_aio_echo_server(
): ):
async with to_asyncio.open_channel_from( async with to_asyncio.open_channel_from(
aio_echo_server, aio_echo_server,
) as (first, chan): ) as (
chan,
first, # value from `chan.started_nowait()` above
):
assert first == 'start' assert first == 'start'
await ctx.started(first) await ctx.started(first)
@ -776,7 +781,8 @@ async def trio_to_aio_echo_server(
await chan.send(msg) await chan.send(msg)
out = await chan.receive() out = await chan.receive()
# echo back to parent actor-task
# echo back to parent-actor's remote parent-ctx-task!
await stream.send(out) await stream.send(out)
if out is None: if out is None:
@ -1090,24 +1096,21 @@ def test_sigint_closes_lifetime_stack(
# ?TODO asyncio.Task fn-deco? # ?TODO asyncio.Task fn-deco?
# -[ ] do sig checkingat import time like @context?
# -[ ] maybe name it @aio_task ??
# -[ ] chan: to_asyncio.InterloopChannel ?? # -[ ] chan: to_asyncio.InterloopChannel ??
# -[ ] do fn-sig checking at import time like @context?
# |_[ ] maybe name it @a(sync)io_task ??
# @asyncio_task <- not bad ??
async def raise_before_started( async def raise_before_started(
# from_trio: asyncio.Queue,
# to_trio: trio.abc.SendChannel,
chan: to_asyncio.LinkedTaskChannel, chan: to_asyncio.LinkedTaskChannel,
) -> None: ) -> None:
''' '''
`asyncio.Task` entry point which RTEs before calling `asyncio.Task` entry point which RTEs before calling
`to_trio.send_nowait()`. `chan.started_nowait()`.
''' '''
await asyncio.sleep(0.2) await asyncio.sleep(0.2)
raise RuntimeError('Some shite went wrong before `.send_nowait()`!!') raise RuntimeError('Some shite went wrong before `.send_nowait()`!!')
# to_trio.send_nowait('Uhh we shouldve RTE-d ^^ ??')
chan.started_nowait('Uhh we shouldve RTE-d ^^ ??') chan.started_nowait('Uhh we shouldve RTE-d ^^ ??')
await asyncio.sleep(float('inf')) await asyncio.sleep(float('inf'))

View File

@ -11,12 +11,13 @@ import trio
import tractor import tractor
from tractor import ( # typing from tractor import ( # typing
Actor, Actor,
current_actor,
open_nursery,
Portal,
Context, Context,
ContextCancelled, ContextCancelled,
MsgStream,
Portal,
RemoteActorError, RemoteActorError,
current_actor,
open_nursery,
) )
from tractor._testing import ( from tractor._testing import (
# tractor_test, # tractor_test,
@ -200,7 +201,7 @@ async def stream_from_peer(
) -> None: ) -> None:
# sanity # sanity
assert tractor._state.debug_mode() == debug_mode assert tractor.debug_mode() == debug_mode
peer: Portal peer: Portal
try: try:
@ -580,7 +581,7 @@ def test_peer_canceller(
assert ( assert (
re.canceller re.canceller
== ==
root.uid root.aid.uid
) )
else: # the other 2 ctxs else: # the other 2 ctxs
@ -589,7 +590,7 @@ def test_peer_canceller(
and ( and (
re.canceller re.canceller
== ==
canceller.channel.uid canceller.channel.aid.uid
) )
) )
@ -744,7 +745,7 @@ def test_peer_canceller(
# -> each context should have received # -> each context should have received
# a silently absorbed context cancellation # a silently absorbed context cancellation
# in its remote nursery scope. # in its remote nursery scope.
# assert ctx.chan.uid == ctx.canceller # assert ctx.chan.aid.uid == ctx.canceller
# NOTE: when an inter-peer cancellation # NOTE: when an inter-peer cancellation
# occurred, we DO NOT expect this # occurred, we DO NOT expect this
@ -796,12 +797,12 @@ async def basic_echo_server(
) -> None: ) -> None:
''' '''
Just the simplest `MsgStream` echo server which resays what Just the simplest `MsgStream` echo server which resays what you
you told it but with its uid in front ;) told it but with its uid in front ;)
''' '''
actor: Actor = tractor.current_actor() actor: Actor = tractor.current_actor()
uid: tuple = actor.uid uid: tuple = actor.aid.uid
await ctx.started(uid) await ctx.started(uid)
async with ctx.open_stream() as ipc: async with ctx.open_stream() as ipc:
async for msg in ipc: async for msg in ipc:
@ -840,7 +841,7 @@ async def serve_subactors(
async with open_nursery() as an: async with open_nursery() as an:
# sanity # sanity
assert tractor._state.debug_mode() == debug_mode assert tractor.debug_mode() == debug_mode
await ctx.started(peer_name) await ctx.started(peer_name)
async with ctx.open_stream() as ipc: async with ctx.open_stream() as ipc:
@ -856,7 +857,7 @@ async def serve_subactors(
f'|_{peer}\n' f'|_{peer}\n'
) )
await ipc.send(( await ipc.send((
peer.chan.uid, peer.chan.aid.uid,
peer.chan.raddr.unwrap(), peer.chan.raddr.unwrap(),
)) ))
@ -879,7 +880,7 @@ async def client_req_subactor(
) -> None: ) -> None:
# sanity # sanity
if debug_mode: if debug_mode:
assert tractor._state.debug_mode() assert tractor.debug_mode()
# TODO: other cases to do with sub lifetimes: # TODO: other cases to do with sub lifetimes:
# -[ ] test that we can have the server spawn a sub # -[ ] test that we can have the server spawn a sub
@ -966,9 +967,14 @@ async def tell_little_bro(
caller: str = '', caller: str = '',
err_after: float|None = None, err_after: float|None = None,
rng_seed: int = 50, rng_seed: int = 100,
# NOTE, ensure ^ is large enough (on fast hw anyway)
# to ensure the peer cancel req arrives before the
# echoing dialog does itself Bp
): ):
# contact target actor, do a stream dialog. # contact target actor, do a stream dialog.
lb: Portal
echo_ipc: MsgStream
async with ( async with (
tractor.wait_for_actor( tractor.wait_for_actor(
name=actor_name name=actor_name
@ -983,17 +989,17 @@ async def tell_little_bro(
else None else None
), ),
) as (sub_ctx, first), ) as (sub_ctx, first),
sub_ctx.open_stream() as echo_ipc, sub_ctx.open_stream() as echo_ipc,
): ):
actor: Actor = current_actor() actor: Actor = current_actor()
uid: tuple = actor.uid uid: tuple = actor.aid.uid
for i in range(rng_seed): for i in range(rng_seed):
msg: tuple = ( msg: tuple = (
uid, uid,
i, i,
) )
await echo_ipc.send(msg) await echo_ipc.send(msg)
await trio.sleep(0.001)
resp = await echo_ipc.receive() resp = await echo_ipc.receive()
print( print(
f'{caller} => {actor_name}: {msg}\n' f'{caller} => {actor_name}: {msg}\n'
@ -1006,6 +1012,9 @@ async def tell_little_bro(
assert sub_uid != uid assert sub_uid != uid
assert _i == i assert _i == i
# XXX, usually should never get here!
# await tractor.pause()
@pytest.mark.parametrize( @pytest.mark.parametrize(
'raise_client_error', 'raise_client_error',
@ -1020,6 +1029,9 @@ def test_peer_spawns_and_cancels_service_subactor(
raise_client_error: str, raise_client_error: str,
reg_addr: tuple[str, int], reg_addr: tuple[str, int],
raise_sub_spawn_error_after: float|None, raise_sub_spawn_error_after: float|None,
loglevel: str,
# ^XXX, set to 'warning' to see masked-exc warnings
# that may transpire during actor-nursery teardown.
): ):
# NOTE: this tests for the modden `mod wks open piker` bug # NOTE: this tests for the modden `mod wks open piker` bug
# discovered as part of implementing workspace ctx # discovered as part of implementing workspace ctx
@ -1049,6 +1061,7 @@ def test_peer_spawns_and_cancels_service_subactor(
# NOTE: to halt the peer tasks on ctxc, uncomment this. # NOTE: to halt the peer tasks on ctxc, uncomment this.
debug_mode=debug_mode, debug_mode=debug_mode,
registry_addrs=[reg_addr], registry_addrs=[reg_addr],
loglevel=loglevel,
) as an: ) as an:
server: Portal = await an.start_actor( server: Portal = await an.start_actor(
(server_name := 'spawn_server'), (server_name := 'spawn_server'),
@ -1084,7 +1097,7 @@ def test_peer_spawns_and_cancels_service_subactor(
) as (client_ctx, client_says), ) as (client_ctx, client_says),
): ):
root: Actor = current_actor() root: Actor = current_actor()
spawner_uid: tuple = spawn_ctx.chan.uid spawner_uid: tuple = spawn_ctx.chan.aid.uid
print( print(
f'Server says: {first}\n' f'Server says: {first}\n'
f'Client says: {client_says}\n' f'Client says: {client_says}\n'
@ -1103,7 +1116,7 @@ def test_peer_spawns_and_cancels_service_subactor(
print( print(
'Sub-spawn came online\n' 'Sub-spawn came online\n'
f'portal: {sub}\n' f'portal: {sub}\n'
f'.uid: {sub.actor.uid}\n' f'.uid: {sub.actor.aid.uid}\n'
f'chan.raddr: {sub.chan.raddr}\n' f'chan.raddr: {sub.chan.raddr}\n'
) )
@ -1137,7 +1150,7 @@ def test_peer_spawns_and_cancels_service_subactor(
assert isinstance(res, ContextCancelled) assert isinstance(res, ContextCancelled)
assert client_ctx.cancel_acked assert client_ctx.cancel_acked
assert res.canceller == root.uid assert res.canceller == root.aid.uid
assert not raise_sub_spawn_error_after assert not raise_sub_spawn_error_after
# cancelling the spawner sub should # cancelling the spawner sub should
@ -1171,8 +1184,8 @@ def test_peer_spawns_and_cancels_service_subactor(
# little_bro: a `RuntimeError`. # little_bro: a `RuntimeError`.
# #
check_inner_rte(rae) check_inner_rte(rae)
assert rae.relay_uid == client.chan.uid assert rae.relay_uid == client.chan.aid.uid
assert rae.src_uid == sub.chan.uid assert rae.src_uid == sub.chan.aid.uid
assert not client_ctx.cancel_acked assert not client_ctx.cancel_acked
assert ( assert (
@ -1201,12 +1214,12 @@ def test_peer_spawns_and_cancels_service_subactor(
except ContextCancelled as ctxc: except ContextCancelled as ctxc:
_ctxc = ctxc _ctxc = ctxc
print( print(
f'{root.uid} caught ctxc from ctx with {client_ctx.chan.uid}\n' f'{root.aid.uid} caught ctxc from ctx with {client_ctx.chan.aid.uid}\n'
f'{repr(ctxc)}\n' f'{repr(ctxc)}\n'
) )
if not raise_sub_spawn_error_after: if not raise_sub_spawn_error_after:
assert ctxc.canceller == root.uid assert ctxc.canceller == root.aid.uid
else: else:
assert ctxc.canceller == spawner_uid assert ctxc.canceller == spawner_uid

View File

@ -1,9 +1,11 @@
""" """
Streaming via async gen api Streaming via the, now legacy, "async-gen API".
""" """
import time import time
from functools import partial from functools import partial
import platform import platform
from typing import Callable
import trio import trio
import tractor import tractor
@ -19,7 +21,11 @@ def test_must_define_ctx():
async def no_ctx(): async def no_ctx():
pass pass
assert "no_ctx must be `ctx: tractor.Context" in str(err.value) assert (
"no_ctx must be `ctx: tractor.Context"
in
str(err.value)
)
@tractor.stream @tractor.stream
async def has_ctx(ctx): async def has_ctx(ctx):
@ -69,14 +75,14 @@ async def stream_from_single_subactor(
async with tractor.open_nursery( async with tractor.open_nursery(
registry_addrs=[reg_addr], registry_addrs=[reg_addr],
start_method=start_method, start_method=start_method,
) as nursery: ) as an:
async with tractor.find_actor('streamerd') as portals: async with tractor.find_actor('streamerd') as portals:
if not portals: if not portals:
# no brokerd actor found # no brokerd actor found
portal = await nursery.start_actor( portal = await an.start_actor(
'streamerd', 'streamerd',
enable_modules=[__name__], enable_modules=[__name__],
) )
@ -116,11 +122,22 @@ async def stream_from_single_subactor(
@pytest.mark.parametrize( @pytest.mark.parametrize(
'stream_func', [async_gen_stream, context_stream] 'stream_func',
[
async_gen_stream,
context_stream,
],
ids='stream_func={}'.format
) )
def test_stream_from_single_subactor(reg_addr, start_method, stream_func): def test_stream_from_single_subactor(
"""Verify streaming from a spawned async generator. reg_addr: tuple,
""" start_method: str,
stream_func: Callable,
):
'''
Verify streaming from a spawned async generator.
'''
trio.run( trio.run(
partial( partial(
stream_from_single_subactor, stream_from_single_subactor,
@ -132,10 +149,9 @@ def test_stream_from_single_subactor(reg_addr, start_method, stream_func):
# this is the first 2 actors, streamer_1 and streamer_2 # this is the first 2 actors, streamer_1 and streamer_2
async def stream_data(seed): async def stream_data(seed: int):
for i in range(seed): for i in range(seed):
yield i yield i
# trigger scheduler to simulate practical usage # trigger scheduler to simulate practical usage
@ -143,15 +159,17 @@ async def stream_data(seed):
# this is the third actor; the aggregator # this is the third actor; the aggregator
async def aggregate(seed): async def aggregate(seed: int):
"""Ensure that the two streams we receive match but only stream '''
Ensure that the two streams we receive match but only stream
a single set of values to the parent. a single set of values to the parent.
"""
async with tractor.open_nursery() as nursery: '''
async with tractor.open_nursery() as an:
portals = [] portals = []
for i in range(1, 3): for i in range(1, 3):
# fork point # fork point
portal = await nursery.start_actor( portal = await an.start_actor(
name=f'streamer_{i}', name=f'streamer_{i}',
enable_modules=[__name__], enable_modules=[__name__],
) )
@ -164,20 +182,28 @@ async def aggregate(seed):
async with send_chan: async with send_chan:
async with portal.open_stream_from( async with portal.open_stream_from(
stream_data, seed=seed, stream_data,
seed=seed,
) as stream: ) as stream:
async for value in stream: async for value in stream:
# leverage trio's built-in backpressure # leverage trio's built-in backpressure
await send_chan.send(value) await send_chan.send(value)
print(f"FINISHED ITERATING {portal.channel.uid}") print(
f'FINISHED ITERATING!\n'
f'peer: {portal.channel.aid.uid}'
)
# spawn 2 trio tasks to collect streams and push to a local queue # spawn 2 trio tasks to collect streams and push to a local queue
async with trio.open_nursery() as n: async with trio.open_nursery() as tn:
for portal in portals: for portal in portals:
n.start_soon(push_to_chan, portal, send_chan.clone()) tn.start_soon(
push_to_chan,
portal,
send_chan.clone(),
)
# close this local task's reference to send side # close this local task's reference to send side
await send_chan.aclose() await send_chan.aclose()
@ -194,20 +220,21 @@ async def aggregate(seed):
print("FINISHED ITERATING in aggregator") print("FINISHED ITERATING in aggregator")
await nursery.cancel() await an.cancel()
print("WAITING on `ActorNursery` to finish") print("WAITING on `ActorNursery` to finish")
print("AGGREGATOR COMPLETE!") print("AGGREGATOR COMPLETE!")
# this is the main actor and *arbiter* async def a_quadruple_example() -> list[int]:
async def a_quadruple_example(): '''
# a nursery which spawns "actors" Open the root-actor which is also a "registrar".
async with tractor.open_nursery() as nursery:
'''
async with tractor.open_nursery() as an:
seed = int(1e3) seed = int(1e3)
pre_start = time.time() pre_start = time.time()
portal = await nursery.start_actor( portal = await an.start_actor(
name='aggregator', name='aggregator',
enable_modules=[__name__], enable_modules=[__name__],
) )
@ -228,8 +255,14 @@ async def a_quadruple_example():
return result_stream return result_stream
async def cancel_after(wait, reg_addr): async def cancel_after(
async with tractor.open_root_actor(registry_addrs=[reg_addr]): wait: float,
reg_addr: tuple,
) -> list[int]:
async with tractor.open_root_actor(
registry_addrs=[reg_addr],
):
with trio.move_on_after(wait): with trio.move_on_after(wait):
return await a_quadruple_example() return await a_quadruple_example()
@ -240,6 +273,10 @@ def time_quad_ex(
ci_env: bool, ci_env: bool,
spawn_backend: str, spawn_backend: str,
): ):
non_linux: bool = (_sys := platform.system()) != 'Linux'
if ci_env and non_linux:
pytest.skip(f'Test is too flaky on {_sys!r} in CI')
if spawn_backend == 'mp': if spawn_backend == 'mp':
''' '''
no idea but the mp *nix runs are flaking out here often... no idea but the mp *nix runs are flaking out here often...
@ -247,32 +284,59 @@ def time_quad_ex(
''' '''
pytest.skip("Test is too flaky on mp in CI") pytest.skip("Test is too flaky on mp in CI")
timeout = 7 if platform.system() in ('Windows', 'Darwin') else 4 timeout = 7 if non_linux else 4
start = time.time() start = time.time()
results = trio.run(cancel_after, timeout, reg_addr) results: list[int] = trio.run(
diff = time.time() - start cancel_after,
timeout,
reg_addr,
)
diff: float = time.time() - start
assert results assert results
return results, diff return results, diff
def test_a_quadruple_example( def test_a_quadruple_example(
time_quad_ex: tuple, time_quad_ex: tuple[list[int], float],
ci_env: bool, ci_env: bool,
spawn_backend: str, spawn_backend: str,
test_log: tractor.log.StackLevelAdapter,
): ):
''' '''
This also serves as a kind of "we'd like to be this fast test". This also serves as a "we'd like to be this fast" smoke test
given past empirical eval of this suite.
''' '''
non_linux: bool = (_sys := platform.system()) != 'Linux'
this_fast_on_linux: float = 3
this_fast = (
6 if non_linux
else this_fast_on_linux
)
# ^ XXX NOTE,
# i've noticed that tweaking the CPU governor setting
# to not "always" enable "turbo" mode can result in latency
# which causes this limit to be too little. Not sure if it'd
# be worth it to adjust the linux value based on reading the
# CPU conf from the sys?
#
# For ex, see the `auto-cpufreq` docs on such settings,
# https://github.com/AdnanHodzic/auto-cpufreq?tab=readme-ov-file#example-config-file-contents
#
# HENCE this below latency-headroom compensation logic..
from .conftest import cpu_scaling_factor
headroom: float = cpu_scaling_factor()
if headroom != 1.:
this_fast = this_fast_on_linux * headroom
test_log.warning(
f'Adding latency headroom on linux bc CPU scaling,\n'
f'headroom: {headroom}\n'
f'this_fast_on_linux: {this_fast_on_linux} -> {this_fast}\n'
)
results, diff = time_quad_ex results, diff = time_quad_ex
assert results assert results
this_fast = (
6 if platform.system() in (
'Windows',
'Darwin',
)
else 3
)
assert diff < this_fast assert diff < this_fast
@ -281,43 +345,58 @@ def test_a_quadruple_example(
list(map(lambda i: i/10, range(3, 9))) list(map(lambda i: i/10, range(3, 9)))
) )
def test_not_fast_enough_quad( def test_not_fast_enough_quad(
reg_addr, time_quad_ex, cancel_delay, ci_env, spawn_backend reg_addr: tuple,
time_quad_ex: tuple[list[int], float],
cancel_delay: float,
ci_env: bool,
spawn_backend: str,
): ):
"""Verify we can cancel midway through the quad example and all actors '''
cancel gracefully. Verify we can cancel midway through the quad example and all
""" actors cancel gracefully.
'''
results, diff = time_quad_ex results, diff = time_quad_ex
delay = max(diff - cancel_delay, 0) delay = max(diff - cancel_delay, 0)
results = trio.run(cancel_after, delay, reg_addr) results = trio.run(
system = platform.system() cancel_after,
if system in ('Windows', 'Darwin') and results is not None: delay,
reg_addr,
)
system: str = platform.system()
if (
system in ('Windows', 'Darwin')
and
results is not None
):
# In CI envoirments it seems later runs are quicker then the first # In CI envoirments it seems later runs are quicker then the first
# so just ignore these # so just ignore these
print(f"Woa there {system} caught your breath eh?") print(f'Woa there {system} caught your breath eh?')
else: else:
# should be cancelled mid-streaming # should be cancelled mid-streaming
assert results is None assert results is None
@tractor_test @tractor_test(timeout=20)
async def test_respawn_consumer_task( async def test_respawn_consumer_task(
reg_addr, reg_addr: tuple,
spawn_backend, spawn_backend: str,
loglevel, loglevel: str,
): ):
"""Verify that ``._portal.ReceiveStream.shield()`` '''
Verify that ``._portal.ReceiveStream.shield()``
sucessfully protects the underlying IPC channel from being closed sucessfully protects the underlying IPC channel from being closed
when cancelling and respawning a consumer task. when cancelling and respawning a consumer task.
This also serves to verify that all values from the stream can be This also serves to verify that all values from the stream can be
received despite the respawns. received despite the respawns.
""" '''
stream = None stream = None
async with tractor.open_nursery() as n: async with tractor.open_nursery() as an:
portal = await n.start_actor( portal = await an.start_actor(
name='streamer', name='streamer',
enable_modules=[__name__] enable_modules=[__name__]
) )

View File

@ -1,5 +1,5 @@
""" """
Arbiter and "local" actor api Registrar and "local" actor api
""" """
import time import time
@ -12,11 +12,11 @@ from tractor._testing import tractor_test
@pytest.mark.trio @pytest.mark.trio
async def test_no_runtime(): async def test_no_runtime():
"""An arbitter must be established before any nurseries """A registrar must be established before any nurseries
can be created. can be created.
(In other words ``tractor.open_root_actor()`` must be engaged at (In other words ``tractor.open_root_actor()`` must be
some point?) engaged at some point?)
""" """
with pytest.raises(RuntimeError) : with pytest.raises(RuntimeError) :
async with tractor.find_actor('doggy'): async with tractor.find_actor('doggy'):
@ -25,9 +25,9 @@ async def test_no_runtime():
@tractor_test @tractor_test
async def test_self_is_registered(reg_addr): async def test_self_is_registered(reg_addr):
"Verify waiting on the arbiter to register itself using the standard api." "Verify waiting on the registrar to register itself using the standard api."
actor = tractor.current_actor() actor = tractor.current_actor()
assert actor.is_arbiter assert actor.is_registrar
with trio.fail_after(0.2): with trio.fail_after(0.2):
async with tractor.wait_for_actor('root') as portal: async with tractor.wait_for_actor('root') as portal:
assert portal.channel.uid[0] == 'root' assert portal.channel.uid[0] == 'root'
@ -35,11 +35,11 @@ async def test_self_is_registered(reg_addr):
@tractor_test @tractor_test
async def test_self_is_registered_localportal(reg_addr): async def test_self_is_registered_localportal(reg_addr):
"Verify waiting on the arbiter to register itself using a local portal." "Verify waiting on the registrar to register itself using a local portal."
actor = tractor.current_actor() actor = tractor.current_actor()
assert actor.is_arbiter assert actor.is_registrar
async with tractor.get_registry(reg_addr) as portal: async with tractor.get_registry(reg_addr) as portal:
assert isinstance(portal, tractor._portal.LocalPortal) assert isinstance(portal, tractor.runtime._portal.LocalPortal)
with trio.fail_after(0.2): with trio.fail_after(0.2):
sockaddr = await portal.run_from_ns( sockaddr = await portal.run_from_ns(
@ -57,8 +57,8 @@ def test_local_actor_async_func(reg_addr):
async with tractor.open_root_actor( async with tractor.open_root_actor(
registry_addrs=[reg_addr], registry_addrs=[reg_addr],
): ):
# arbiter is started in-proc if dne # registrar is started in-proc if dne
assert tractor.current_actor().is_arbiter assert tractor.current_actor().is_registrar
for i in range(10): for i in range(10):
nums.append(i) nums.append(i)

View File

@ -0,0 +1,185 @@
'''
`tractor.log`-wrapping unit tests.
'''
from pathlib import Path
import shutil
from types import ModuleType
import pytest
import tractor
from tractor import (
_code_load,
log,
)
def test_root_pkg_not_duplicated_in_logger_name():
'''
When both `pkg_name` and `name` are passed and they have
a common `<root_name>.< >` prefix, ensure that it is not
duplicated in the child's `StackLevelAdapter.name: str`.
'''
project_name: str = 'pylib'
pkg_path: str = 'pylib.subpkg.mod'
assert not tractor.current_actor(
err_on_no_runtime=False,
)
proj_log = log.get_logger(
pkg_name=project_name,
mk_sublog=False,
)
sublog = log.get_logger(
pkg_name=project_name,
name=pkg_path,
)
assert proj_log is not sublog
assert sublog.name.count(proj_log.name) == 1
assert 'mod' not in sublog.name
def test_implicit_mod_name_applied_for_child(
testdir: pytest.Pytester,
loglevel: str,
):
'''
Verify that when `.log.get_logger(pkg_name='pylib')` is called
from a given sub-mod from within the `pylib` pkg-path, we
implicitly set the equiv of `name=__name__` from the caller's
module.
'''
# tractor.log.get_console_log(level=loglevel)
proj_name: str = 'snakelib'
mod_code: str = (
f'import tractor\n'
f'\n'
# if you need to trace `testdir` stuff @ import-time..
# f'breakpoint()\n'
f'log = tractor.log.get_logger(pkg_name="{proj_name}")\n'
)
# create a sub-module for each pkg layer
_lib = testdir.mkpydir(proj_name)
pkg: Path = Path(_lib)
pkg_init_mod: Path = pkg / "__init__.py"
pkg_init_mod.write_text(mod_code)
subpkg: Path = pkg / 'subpkg'
subpkg.mkdir()
subpkgmod: Path = subpkg / "__init__.py"
subpkgmod.touch()
subpkgmod.write_text(mod_code)
_submod: Path = testdir.makepyfile(
_mod=mod_code,
)
pkg_submod = pkg / 'mod.py'
pkg_subpkg_submod = subpkg / 'submod.py'
shutil.copyfile(
_submod,
pkg_submod,
)
shutil.copyfile(
_submod,
pkg_subpkg_submod,
)
testdir.chdir()
# NOTE, to introspect the py-file-module-layout use (in .xsh
# syntax): `ranger @str(testdir)`
# XXX NOTE, once the "top level" pkg mod has been
# imported, we can then use `import` syntax to
# import it's sub-pkgs and modules.
subpkgmod: ModuleType = _code_load.load_module_from_path(
Path(pkg / '__init__.py'),
module_name=proj_name,
)
pkg_root_log = log.get_logger(
pkg_name=proj_name,
mk_sublog=False,
)
# the top level pkg-mod, created just now,
# by above API call.
assert pkg_root_log.name == proj_name
assert not pkg_root_log.logger.getChildren()
#
# ^TODO! test this same output but created via a `get_logger()`
# call in the `snakelib.__init__py`!!
# NOTE, the pkg-level "init mod" should of course
# have the same name as the package ns-path.
import snakelib as init_mod
assert init_mod.log.name == proj_name
# NOTE, a first-pkg-level sub-module should only
# use the package-name since the leaf-node-module
# will be included in log headers by default.
from snakelib import mod
assert mod.log.name == proj_name
from snakelib import subpkg
assert (
subpkg.log.name
==
subpkg.__package__
==
f'{proj_name}.subpkg'
)
from snakelib.subpkg import submod
assert (
submod.log.name
==
submod.__package__
==
f'{proj_name}.subpkg'
)
sub_logs = pkg_root_log.logger.getChildren()
assert len(sub_logs) == 1 # only one nested sub-pkg module
assert submod.log.logger in sub_logs
# TODO, moar tests against existing feats:
# ------ - ------
# - [ ] color settings?
# - [ ] header contents like,
# - actor + thread + task names from various conc-primitives,
# - [ ] `StackLevelAdapter` extensions,
# - our custom levels/methods: `transport|runtime|cance|pdb|devx`
# - [ ] custom-headers support?
#
# TODO, test driven dev of new-ideas/long-wanted feats,
# ------ - ------
# - [ ] https://github.com/goodboy/tractor/issues/244
# - [ ] @catern mentioned using a sync / deterministic sys
# and in particular `svlogd`?
# |_ https://smarden.org/runit/svlogd.8
# - [ ] using adapter vs. filters?
# - https://stackoverflow.com/questions/60691759/add-information-to-every-log-message-in-python-logging/61830838#61830838
# - [ ] `.at_least_level()` optimization which short circuits wtv
# `logging` is doing behind the scenes when the level filters
# the emission..?
# - [ ] use of `.log.get_console_log()` in subactors and the
# subtleties of ensuring it actually emits from a subproc.
# - [ ] this idea of activating per-subsys emissions with some
# kind of `.name` filter passed to the runtime or maybe configured
# via the root `StackLevelAdapter`?
# - [ ] use of `logging.dict.dictConfig()` to simplify the impl
# of any of ^^ ??
# - https://stackoverflow.com/questions/7507825/where-is-a-complete-example-of-logging-config-dictconfig
# - https://docs.python.org/3/library/logging.config.html#configuration-dictionary-schema
# - https://docs.python.org/3/library/logging.config.html#logging.config.dictConfig

View File

@ -1,8 +1,13 @@
""" """
Multiple python programs invoking the runtime. Multiple python programs invoking the runtime.
""" """
from __future__ import annotations
import platform import platform
import subprocess
import time import time
from typing import (
TYPE_CHECKING,
)
import pytest import pytest
import trio import trio
@ -10,14 +15,32 @@ import tractor
from tractor._testing import ( from tractor._testing import (
tractor_test, tractor_test,
) )
from tractor import (
current_actor,
Actor,
Context,
Portal,
)
from tractor.runtime import _state
from .conftest import ( from .conftest import (
sig_prog, sig_prog,
_INT_SIGNAL, _INT_SIGNAL,
_INT_RETURN_CODE, _INT_RETURN_CODE,
) )
if TYPE_CHECKING:
from tractor.msg import Aid
from tractor.discovery._addr import (
UnwrappedAddress,
)
def test_abort_on_sigint(daemon):
_non_linux: bool = platform.system() != 'Linux'
def test_abort_on_sigint(
daemon: subprocess.Popen,
):
assert daemon.returncode is None assert daemon.returncode is None
time.sleep(0.1) time.sleep(0.1)
sig_prog(daemon, _INT_SIGNAL) sig_prog(daemon, _INT_SIGNAL)
@ -30,39 +53,131 @@ def test_abort_on_sigint(daemon):
@tractor_test @tractor_test
async def test_cancel_remote_arbiter(daemon, reg_addr): async def test_cancel_remote_registrar(
assert not tractor.current_actor().is_arbiter daemon: subprocess.Popen,
reg_addr: UnwrappedAddress,
):
assert not current_actor().is_registrar
async with tractor.get_registry(reg_addr) as portal: async with tractor.get_registry(reg_addr) as portal:
await portal.cancel_actor() await portal.cancel_actor()
time.sleep(0.1) time.sleep(0.1)
# the arbiter channel server is cancelled but not its main task # the registrar channel server is cancelled but not its main task
assert daemon.returncode is None assert daemon.returncode is None
# no arbiter socket should exist # no registrar socket should exist
with pytest.raises(OSError): with pytest.raises(OSError):
async with tractor.get_registry(reg_addr) as portal: async with tractor.get_registry(reg_addr) as portal:
pass pass
def test_register_duplicate_name(daemon, reg_addr): def test_register_duplicate_name(
daemon: subprocess.Popen,
reg_addr: UnwrappedAddress,
):
async def main(): async def main():
async with tractor.open_nursery( async with tractor.open_nursery(
registry_addrs=[reg_addr], registry_addrs=[reg_addr],
) as n: ) as an:
assert not tractor.current_actor().is_arbiter assert not current_actor().is_registrar
p1 = await n.start_actor('doggy') p1 = await an.start_actor('doggy')
p2 = await n.start_actor('doggy') p2 = await an.start_actor('doggy')
async with tractor.wait_for_actor('doggy') as portal: async with tractor.wait_for_actor('doggy') as portal:
assert portal.channel.uid in (p2.channel.uid, p1.channel.uid) assert portal.channel.uid in (p2.channel.uid, p1.channel.uid)
await n.cancel() await an.cancel()
# run it manually since we want to start **after** # XXX, run manually since we want to start this root **after**
# the other "daemon" program # the other "daemon" program with it's own root.
trio.run(main)
@tractor.context
async def get_root_portal(
ctx: Context,
):
'''
Connect back to the root actor manually (using `._discovery` API)
and ensure it's contact info is the same as our immediate parent.
'''
sub: Actor = current_actor()
rtvs: dict = _state._runtime_vars
raddrs: list[UnwrappedAddress] = rtvs['_root_addrs']
# await tractor.pause()
# XXX, in case the sub->root discovery breaks you might need
# this (i know i did Xp)!!
# from tractor.devx import mk_pdb
# mk_pdb().set_trace()
assert (
len(raddrs) == 1
and
list(sub._parent_chan.raddr.unwrap()) in raddrs
)
# connect back to our immediate parent which should also
# be the actor-tree's root.
from tractor.discovery._discovery import get_root
ptl: Portal
async with get_root() as ptl:
root_aid: Aid = ptl.chan.aid
parent_ptl: Portal = current_actor().get_parent()
assert (
root_aid.name == 'root'
and
parent_ptl.chan.aid == root_aid
)
await ctx.started()
def test_non_registrar_spawns_child(
daemon: subprocess.Popen,
reg_addr: UnwrappedAddress,
loglevel: str,
debug_mode: bool,
ci_env: bool,
):
'''
Ensure a non-regristar (serving) root actor can spawn a sub and
that sub can connect back (manually) to it's rent that is the
root without issue.
More or less this audits the global contact info in
`._state._runtime_vars`.
'''
async def main():
# XXX, since apparently on macos in GH's CI it can be a race
# with the `daemon` registrar on grabbing the socket-addr..
if ci_env and _non_linux:
await trio.sleep(.5)
async with tractor.open_nursery(
registry_addrs=[reg_addr],
loglevel=loglevel,
debug_mode=debug_mode,
) as an:
actor: Actor = tractor.current_actor()
assert not actor.is_registrar
sub_ptl: Portal = await an.start_actor(
name='sub',
enable_modules=[__name__],
)
async with sub_ptl.open_context(
get_root_portal,
) as (ctx, _):
print('Waiting for `sub` to connect back to us..')
await an.cancel()
# XXX, run manually since we want to start this root **after**
# the other "daemon" program with it's own root.
trio.run(main) trio.run(main)

View File

@ -0,0 +1,333 @@
'''
Verify that externally registered remote actor error
types are correctly relayed, boxed, and re-raised across
IPC actor hops via `reg_err_types()`.
Also ensure that when custom error types are NOT registered
the framework indicates the lookup failure to the user.
'''
import pytest
import trio
import tractor
from tractor import (
Context,
Portal,
RemoteActorError,
)
from tractor._exceptions import (
get_err_type,
reg_err_types,
)
# -- custom app-level errors for testing --
class CustomAppError(Exception):
'''
A hypothetical user-app error that should be
boxed+relayed by `tractor` IPC when registered.
'''
class AnotherAppError(Exception):
'''
A second custom error for multi-type registration.
'''
class UnregisteredAppError(Exception):
'''
A custom error that is intentionally NEVER
registered via `reg_err_types()` so we can
verify the framework's failure indication.
'''
# -- remote-task endpoints --
@tractor.context
async def raise_custom_err(
ctx: Context,
) -> None:
'''
Remote ep that raises a `CustomAppError`
after sync-ing with the caller.
'''
await ctx.started()
raise CustomAppError(
'the app exploded remotely'
)
@tractor.context
async def raise_another_err(
ctx: Context,
) -> None:
'''
Remote ep that raises `AnotherAppError`.
'''
await ctx.started()
raise AnotherAppError(
'another app-level kaboom'
)
@tractor.context
async def raise_unreg_err(
ctx: Context,
) -> None:
'''
Remote ep that raises an `UnregisteredAppError`
which has NOT been `reg_err_types()`-registered.
'''
await ctx.started()
raise UnregisteredAppError(
'this error type is unknown to tractor'
)
# -- unit tests for the type-registry plumbing --
class TestRegErrTypesPlumbing:
'''
Low-level checks on `reg_err_types()` and
`get_err_type()` without requiring IPC.
'''
def test_unregistered_type_returns_none(self):
'''
An unregistered custom error name should yield
`None` from `get_err_type()`.
'''
result = get_err_type('CustomAppError')
assert result is None
def test_register_and_lookup(self):
'''
After `reg_err_types()`, the custom type should
be discoverable via `get_err_type()`.
'''
reg_err_types([CustomAppError])
result = get_err_type('CustomAppError')
assert result is CustomAppError
def test_register_multiple_types(self):
'''
Registering a list of types should make each
one individually resolvable.
'''
reg_err_types([
CustomAppError,
AnotherAppError,
])
assert (
get_err_type('CustomAppError')
is CustomAppError
)
assert (
get_err_type('AnotherAppError')
is AnotherAppError
)
def test_builtin_types_always_resolve(self):
'''
Builtin error types like `RuntimeError` and
`ValueError` should always be found without
any prior registration.
'''
assert (
get_err_type('RuntimeError')
is RuntimeError
)
assert (
get_err_type('ValueError')
is ValueError
)
def test_tractor_native_types_resolve(self):
'''
`tractor`-internal exc types (e.g.
`ContextCancelled`) should always resolve.
'''
assert (
get_err_type('ContextCancelled')
is tractor.ContextCancelled
)
def test_boxed_type_str_without_ipc_msg(self):
'''
When a `RemoteActorError` is constructed
without an IPC msg (and no resolvable type),
`.boxed_type_str` should return `'<unknown>'`.
'''
rae = RemoteActorError('test')
assert rae.boxed_type_str == '<unknown>'
# -- IPC-level integration tests --
def test_registered_custom_err_relayed(
debug_mode: bool,
tpt_proto: str,
):
'''
When a custom error type is registered via
`reg_err_types()` on BOTH sides of an IPC dialog,
the parent should receive a `RemoteActorError`
whose `.boxed_type` matches the original custom
error type.
'''
reg_err_types([CustomAppError])
async def main():
async with tractor.open_nursery(
debug_mode=debug_mode,
enable_transports=[tpt_proto],
) as an:
ptl: Portal = await an.start_actor(
'custom-err-raiser',
enable_modules=[__name__],
)
async with ptl.open_context(
raise_custom_err,
) as (ctx, sent):
assert not sent
try:
await ctx.wait_for_result()
except RemoteActorError as rae:
assert rae.boxed_type is CustomAppError
assert rae.src_type is CustomAppError
assert 'the app exploded remotely' in str(
rae.tb_str
)
raise
with pytest.raises(RemoteActorError) as excinfo:
trio.run(main)
rae = excinfo.value
assert rae.boxed_type is CustomAppError
def test_registered_another_err_relayed(
debug_mode: bool,
tpt_proto: str,
):
'''
Same as above but for a different custom error
type to verify multi-type registration works
end-to-end over IPC.
'''
reg_err_types([AnotherAppError])
async def main():
async with tractor.open_nursery(
debug_mode=debug_mode,
enable_transports=[tpt_proto],
) as an:
ptl: Portal = await an.start_actor(
'another-err-raiser',
enable_modules=[__name__],
)
async with ptl.open_context(
raise_another_err,
) as (ctx, sent):
assert not sent
try:
await ctx.wait_for_result()
except RemoteActorError as rae:
assert (
rae.boxed_type
is AnotherAppError
)
raise
await an.cancel()
with pytest.raises(RemoteActorError) as excinfo:
trio.run(main)
rae = excinfo.value
assert rae.boxed_type is AnotherAppError
def test_unregistered_err_still_relayed(
debug_mode: bool,
tpt_proto: str,
):
'''
Verify that even when a custom error type is NOT registered via
`reg_err_types()`, the remote error is still relayed as
a `RemoteActorError` with all string-level info preserved
(traceback, type name, source actor uid).
The `.boxed_type` will be `None` (type obj can't be resolved) but
`.boxed_type_str` and `.src_type_str` still report the original
type name from the IPC msg.
This documents the expected limitation: without `reg_err_types()`
the `.boxed_type` property can NOT resolve to the original Python
type.
'''
# NOTE: intentionally do NOT call
# `reg_err_types([UnregisteredAppError])`
async def main():
async with tractor.open_nursery(
debug_mode=debug_mode,
enable_transports=[tpt_proto],
) as an:
ptl: Portal = await an.start_actor(
'unreg-err-raiser',
enable_modules=[__name__],
)
async with ptl.open_context(
raise_unreg_err,
) as (ctx, sent):
assert not sent
await ctx.wait_for_result()
await an.cancel()
with pytest.raises(RemoteActorError) as excinfo:
trio.run(main)
rae = excinfo.value
# the error IS relayed even without
# registration; type obj is unresolvable but
# all string-level info is preserved.
assert rae.boxed_type is None # NOT `UnregisteredAppError`
assert rae.src_type is None
# string names survive the IPC round-trip
# via the `Error` msg fields.
assert (
rae.src_type_str
==
'UnregisteredAppError'
)
assert (
rae.boxed_type_str
==
'UnregisteredAppError'
)
# original traceback content is preserved
assert 'this error type is unknown' in rae.tb_str
assert 'UnregisteredAppError' in rae.tb_str

View File

@ -17,9 +17,8 @@ from tractor.log import (
get_console_log, get_console_log,
get_logger, get_logger,
) )
log = get_logger(__name__)
log = get_logger()
_resource: int = 0 _resource: int = 0

View File

@ -49,7 +49,7 @@ def test_infected_root_actor(
), ),
to_asyncio.open_channel_from( to_asyncio.open_channel_from(
aio_echo_server, aio_echo_server,
) as (first, chan), ) as (chan, first),
): ):
assert first == 'start' assert first == 'start'
@ -91,13 +91,12 @@ def test_infected_root_actor(
async def sync_and_err( async def sync_and_err(
# just signature placeholders for compat with # just signature placeholders for compat with
# ``to_asyncio.open_channel_from()`` # ``to_asyncio.open_channel_from()``
to_trio: trio.MemorySendChannel, chan: tractor.to_asyncio.LinkedTaskChannel,
from_trio: asyncio.Queue,
ev: asyncio.Event, ev: asyncio.Event,
): ):
if to_trio: if chan:
to_trio.send_nowait('start') chan.started_nowait('start')
await ev.wait() await ev.wait()
raise RuntimeError('asyncio-side') raise RuntimeError('asyncio-side')
@ -174,7 +173,7 @@ def test_trio_prestarted_task_bubbles(
sync_and_err, sync_and_err,
ev=aio_ev, ev=aio_ev,
) )
) as (first, chan), ) as (chan, first),
): ):
for i in range(5): for i in range(5):

View File

@ -94,15 +94,15 @@ def test_runtime_vars_unset(
after the root actor-runtime exits! after the root actor-runtime exits!
''' '''
assert not tractor._state._runtime_vars['_debug_mode'] assert not tractor.runtime._state._runtime_vars['_debug_mode']
async def main(): async def main():
assert not tractor._state._runtime_vars['_debug_mode'] assert not tractor.runtime._state._runtime_vars['_debug_mode']
async with tractor.open_nursery( async with tractor.open_nursery(
debug_mode=True, debug_mode=True,
): ):
assert tractor._state._runtime_vars['_debug_mode'] assert tractor.runtime._state._runtime_vars['_debug_mode']
# after runtime closure, should be reverted! # after runtime closure, should be reverted!
assert not tractor._state._runtime_vars['_debug_mode'] assert not tractor.runtime._state._runtime_vars['_debug_mode']
trio.run(main) trio.run(main)

View File

@ -110,7 +110,7 @@ def test_rpc_errors(
) as n: ) as n:
actor = tractor.current_actor() actor = tractor.current_actor()
assert actor.is_arbiter assert actor.is_registrar
await n.run_in_actor( await n.run_in_actor(
sleep_back_actor, sleep_back_actor,
actor_name=subactor_requests_to, actor_name=subactor_requests_to,

View File

@ -22,6 +22,10 @@ def unlink_file():
async def crash_and_clean_tmpdir( async def crash_and_clean_tmpdir(
tmp_file_path: str, tmp_file_path: str,
error: bool = True, error: bool = True,
rent_cancel: bool = True,
# XXX unused, but do we really need to test these cases?
self_cancel: bool = False,
): ):
global _file_path global _file_path
_file_path = tmp_file_path _file_path = tmp_file_path
@ -32,43 +36,75 @@ async def crash_and_clean_tmpdir(
assert os.path.isfile(tmp_file_path) assert os.path.isfile(tmp_file_path)
await trio.sleep(0.1) await trio.sleep(0.1)
if error: if error:
print('erroring in subactor!')
assert 0 assert 0
else:
elif self_cancel:
print('SELF-cancelling subactor!')
actor.cancel_soon() actor.cancel_soon()
elif rent_cancel:
await trio.sleep_forever()
print('subactor exiting task!')
@pytest.mark.parametrize( @pytest.mark.parametrize(
'error_in_child', 'error_in_child',
[True, False], [True, False],
ids='error_in_child={}'.format,
) )
@tractor_test @tractor_test
async def test_lifetime_stack_wipes_tmpfile( async def test_lifetime_stack_wipes_tmpfile(
tmp_path, tmp_path,
error_in_child: bool, error_in_child: bool,
loglevel: str,
# log: tractor.log.StackLevelAdapter,
# ^TODO, once landed via macos support!
): ):
child_tmp_file = tmp_path / "child.txt" child_tmp_file = tmp_path / "child.txt"
child_tmp_file.touch() child_tmp_file.touch()
assert child_tmp_file.exists() assert child_tmp_file.exists()
path = str(child_tmp_file) path = str(child_tmp_file)
# NOTE, this is expected to cancel the sub
# in the `error_in_child=False` case!
timeout: float = (
1.6 if error_in_child
else 1
)
try: try:
with trio.move_on_after(0.5): with trio.move_on_after(timeout) as cs:
async with tractor.open_nursery() as n: async with tractor.open_nursery(
await ( # inlined portal loglevel=loglevel,
await n.run_in_actor( ) as an:
await ( # inlined `tractor.Portal`
await an.run_in_actor(
crash_and_clean_tmpdir, crash_and_clean_tmpdir,
tmp_file_path=path, tmp_file_path=path,
error=error_in_child, error=error_in_child,
) )
).result() ).result()
except ( except (
tractor.RemoteActorError, tractor.RemoteActorError,
# tractor.BaseExceptionGroup,
BaseExceptionGroup, BaseExceptionGroup,
): ) as _exc:
pass exc = _exc
from tractor.log import get_console_log
log = get_console_log(
level=loglevel,
name=__name__,
)
log.exception(
f'Subactor failed as expected with {type(exc)!r}\n'
)
# tmp file should have been wiped by # tmp file should have been wiped by
# teardown stack. # teardown stack.
assert not child_tmp_file.exists() assert not child_tmp_file.exists()
if error_in_child:
assert not cs.cancel_called
else:
# expect timeout in some cases?
assert cs.cancel_called

View File

@ -2,6 +2,7 @@
Shared mem primitives and APIs. Shared mem primitives and APIs.
""" """
import platform
import uuid import uuid
# import numpy # import numpy
@ -53,7 +54,18 @@ def test_child_attaches_alot():
shm_key=shml.key, shm_key=shml.key,
) as (ctx, start_val), ) as (ctx, start_val),
): ):
assert start_val == key assert (_key := shml.key) == start_val
if platform.system() != 'Darwin':
# XXX, macOS has a char limit..
# see `ipc._shm._shorten_key_for_macos`
assert (
start_val
==
key
==
_key
)
await ctx.result() await ctx.result()
await portal.cancel_actor() await portal.cancel_actor()

View File

@ -39,7 +39,7 @@ async def spawn(
): ):
# now runtime exists # now runtime exists
actor: tractor.Actor = tractor.current_actor() actor: tractor.Actor = tractor.current_actor()
assert actor.is_arbiter == should_be_root assert actor.is_registrar == should_be_root
# spawns subproc here # spawns subproc here
portal: tractor.Portal = await an.run_in_actor( portal: tractor.Portal = await an.run_in_actor(
@ -68,7 +68,7 @@ async def spawn(
assert result == 10 assert result == 10
return result return result
else: else:
assert actor.is_arbiter == should_be_root assert actor.is_registrar == should_be_root
return 10 return 10
@ -181,7 +181,7 @@ def test_loglevel_propagated_to_subactor(
async def main(): async def main():
async with tractor.open_nursery( async with tractor.open_nursery(
name='arbiter', name='registrar',
start_method=start_method, start_method=start_method,
arbiter_addr=reg_addr, arbiter_addr=reg_addr,

View File

@ -30,21 +30,23 @@ from ._streaming import (
MsgStream as MsgStream, MsgStream as MsgStream,
stream as stream, stream as stream,
) )
from ._discovery import ( from .discovery._discovery import (
get_registry as get_registry, get_registry as get_registry,
find_actor as find_actor, find_actor as find_actor,
wait_for_actor as wait_for_actor, wait_for_actor as wait_for_actor,
query_actor as query_actor, query_actor as query_actor,
) )
from ._supervise import ( from .runtime._supervise import (
open_nursery as open_nursery, open_nursery as open_nursery,
ActorNursery as ActorNursery, ActorNursery as ActorNursery,
) )
from ._state import ( from .runtime._state import (
RuntimeVars as RuntimeVars,
current_actor as current_actor, current_actor as current_actor,
is_root_process as is_root_process,
current_ipc_ctx as current_ipc_ctx, current_ipc_ctx as current_ipc_ctx,
debug_mode as debug_mode debug_mode as debug_mode,
get_runtime_vars as get_runtime_vars,
is_root_process as is_root_process,
) )
from ._exceptions import ( from ._exceptions import (
ContextCancelled as ContextCancelled, ContextCancelled as ContextCancelled,
@ -65,6 +67,10 @@ from ._root import (
open_root_actor as open_root_actor, open_root_actor as open_root_actor,
) )
from .ipc import Channel as Channel from .ipc import Channel as Channel
from ._portal import Portal as Portal from .runtime._portal import Portal as Portal
from ._runtime import Actor as Actor from .runtime._runtime import Actor as Actor
from .discovery._registry import (
Registrar as Registrar,
Arbiter as Arbiter,
)
# from . import hilevel as hilevel # from . import hilevel as hilevel

View File

@ -22,8 +22,8 @@ import argparse
from ast import literal_eval from ast import literal_eval
from ._runtime import Actor from .runtime._runtime import Actor
from ._entry import _trio_main from .spawn._entry import _trio_main
def parse_uid(arg): def parse_uid(arg):

View File

@ -66,7 +66,7 @@ async def open_actor_cluster(
trio.open_nursery() as tn, trio.open_nursery() as tn,
tractor.trionics.maybe_raise_from_masking_exc() tractor.trionics.maybe_raise_from_masking_exc()
): ):
uid = tractor.current_actor().uid uid = tractor.current_actor().aid.uid
async def _start(name: str) -> None: async def _start(name: str) -> None:
name = f'{uid[0]}.{name}' name = f'{uid[0]}.{name}'

View File

@ -0,0 +1,48 @@
# tractor: structured concurrent "actors".
# Copyright 2018-eternity Tyler Goodlet.
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU Affero General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU Affero General Public License for more details.
# You should have received a copy of the GNU Affero General Public License
# along with this program. If not, see <https://www.gnu.org/licenses/>.
'''
(Hot) coad (re-)load utils for python.
'''
import importlib
from pathlib import Path
import sys
from types import ModuleType
# ?TODO, move this into internal libs?
# -[ ] we already use it in `modden.config._pymod` as well
def load_module_from_path(
path: Path,
module_name: str|None = None,
) -> ModuleType:
'''
Taken from SO,
https://stackoverflow.com/a/67208147
which is based on stdlib docs,
https://docs.python.org/3/library/importlib.html#importing-a-source-file-directly
'''
module_name = module_name or path.stem
spec = importlib.util.spec_from_file_location(
module_name,
str(path),
)
module = importlib.util.module_from_spec(spec)
sys.modules[module_name] = module
spec.loader.exec_module(module)
return module

View File

@ -70,6 +70,7 @@ from ._exceptions import (
MsgTypeError, MsgTypeError,
RemoteActorError, RemoteActorError,
StreamOverrun, StreamOverrun,
TransportClosed,
pack_from_raise, pack_from_raise,
unpack_error, unpack_error,
) )
@ -96,7 +97,7 @@ from ._streaming import (
MsgStream, MsgStream,
open_stream_from_ctx, open_stream_from_ctx,
) )
from ._state import ( from .runtime._state import (
current_actor, current_actor,
debug_mode, debug_mode,
_ctxvar_Context, _ctxvar_Context,
@ -106,14 +107,14 @@ from .trionics import (
) )
# ------ - ------ # ------ - ------
if TYPE_CHECKING: if TYPE_CHECKING:
from ._portal import Portal from .runtime._portal import Portal
from ._runtime import Actor from .runtime._runtime import Actor
from .ipc._transport import MsgTransport from .ipc._transport import MsgTransport
from .devx._frame_stack import ( from .devx._frame_stack import (
CallerInfo, CallerInfo,
) )
log = get_logger(__name__) log = get_logger()
class Unresolved: class Unresolved:
@ -462,10 +463,11 @@ class Context:
# self._cancel_called = val # self._cancel_called = val
# TODO, use the `Actor.aid: Aid` instead!
@property @property
def canceller(self) -> tuple[str, str]|None: def canceller(self) -> tuple[str, str]|None:
''' '''
`Actor.uid: tuple[str, str]` of the (remote) `Actor.aid.uid: tuple[str, str]` of the (remote)
actor-process who's task was cancelled thus causing this actor-process who's task was cancelled thus causing this
(side of the) context to also be cancelled. (side of the) context to also be cancelled.
@ -498,12 +500,12 @@ class Context:
if from_uid := re.src_uid: if from_uid := re.src_uid:
from_uid: tuple = tuple(from_uid) from_uid: tuple = tuple(from_uid)
our_uid: tuple = self._actor.uid our_uid: tuple = self._actor.aid.uid
our_canceller = self.canceller our_canceller = self.canceller
return bool( return bool(
isinstance((ctxc := re), ContextCancelled) isinstance((ctxc := re), ContextCancelled)
and from_uid == self.chan.uid and from_uid == self.chan.aid.uid
and ctxc.canceller == our_uid and ctxc.canceller == our_uid
and our_canceller == our_uid and our_canceller == our_uid
) )
@ -514,7 +516,7 @@ class Context:
Records whether the task on the remote side of this IPC Records whether the task on the remote side of this IPC
context acknowledged a cancel request via a relayed context acknowledged a cancel request via a relayed
`ContextCancelled` with the `.canceller` attr set to the `ContextCancelled` with the `.canceller` attr set to the
`Actor.uid` of the local actor who's task entered `Actor.aid.uid` of the local actor who's task entered
`Portal.open_context()`. `Portal.open_context()`.
This will only be `True` when `.cancel()` is called and This will only be `True` when `.cancel()` is called and
@ -788,8 +790,8 @@ class Context:
# appropriately. # appropriately.
log.runtime( log.runtime(
'Setting remote error for ctx\n\n' 'Setting remote error for ctx\n\n'
f'<= {self.peer_side!r}: {self.chan.uid}\n' f'<= {self.peer_side!r}: {self.chan.aid.reprol()}\n'
f'=> {self.side!r}: {self._actor.uid}\n\n' f'=> {self.side!r}: {self._actor.aid.reprol()}\n\n'
f'{error!r}' f'{error!r}'
) )
self._remote_error: BaseException = error self._remote_error: BaseException = error
@ -810,7 +812,7 @@ class Context:
# cancelled. # cancelled.
# #
# !TODO, switching to `Actor.aid` here! # !TODO, switching to `Actor.aid` here!
if (canc := error.canceller) == self._actor.uid: if (canc := error.canceller) == self._actor.aid.uid:
whom: str = 'us' whom: str = 'us'
self._canceller = canc self._canceller = canc
else: else:
@ -1035,7 +1037,7 @@ class Context:
--------- ---------
- after the far end cancels, the `.cancel()` calling side - after the far end cancels, the `.cancel()` calling side
should receive a `ContextCancelled` with the should receive a `ContextCancelled` with the
`.canceller: tuple` uid set to the current `Actor.uid`. `.canceller: tuple` uid set to the current `Actor.aid.uid`.
- timeout (quickly) on failure to rx this ACK error-msg in - timeout (quickly) on failure to rx this ACK error-msg in
an attempt to sidestep 2-generals when the transport an attempt to sidestep 2-generals when the transport
@ -1064,9 +1066,9 @@ class Context:
) )
reminfo: str = ( reminfo: str = (
# ' =>\n' # ' =>\n'
# f'Context.cancel() => {self.chan.uid}\n' # f'Context.cancel() => {self.chan.aid.uid}\n'
f'\n' f'\n'
f'c)=> {self.chan.uid}\n' f'c)=> {self.chan.aid.reprol()}\n'
f' |_[{self.dst_maddr}\n' f' |_[{self.dst_maddr}\n'
f' >> {self.repr_rpc}\n' f' >> {self.repr_rpc}\n'
# f' >> {self._nsf}() -> {codec}[dict]:\n\n' # f' >> {self._nsf}() -> {codec}[dict]:\n\n'
@ -1210,7 +1212,7 @@ class Context:
''' '''
__tracebackhide__: bool = hide_tb __tracebackhide__: bool = hide_tb
peer_uid: tuple = self.chan.uid peer_uid: tuple = self.chan.aid.uid
# XXX NOTE XXX: `ContextCancelled`/`StreamOverrun` absorption # XXX NOTE XXX: `ContextCancelled`/`StreamOverrun` absorption
# for "graceful cancellation" case(s): # for "graceful cancellation" case(s):
@ -1227,7 +1229,7 @@ class Context:
# (`ContextCancelled`) as an expected # (`ContextCancelled`) as an expected
# error-msg-is-cancellation-ack IFF said # error-msg-is-cancellation-ack IFF said
# `remote_error: ContextCancelled` has `.canceller` # `remote_error: ContextCancelled` has `.canceller`
# set to the `Actor.uid` of THIS task (i.e. the # set to the `Actor.aid.uid` of THIS task (i.e. the
# cancellation requesting task's actor is the actor # cancellation requesting task's actor is the actor
# checking whether it should absorb the ctxc). # checking whether it should absorb the ctxc).
self_ctxc: bool = self._is_self_cancelled(remote_error) self_ctxc: bool = self._is_self_cancelled(remote_error)
@ -1678,7 +1680,7 @@ class Context:
elif self._started_called: elif self._started_called:
raise RuntimeError( raise RuntimeError(
f'called `.started()` twice on context with {self.chan.uid}' f'called `.started()` twice on context with {self.chan.aid.uid}'
) )
started_msg = Started( started_msg = Started(
@ -1811,7 +1813,7 @@ class Context:
''' '''
cid: str = self.cid cid: str = self.cid
chan: Channel = self.chan chan: Channel = self.chan
from_uid: tuple[str, str] = chan.uid from_uid: tuple[str, str] = chan.aid.uid
send_chan: trio.MemorySendChannel = self._send_chan send_chan: trio.MemorySendChannel = self._send_chan
nsf: NamespacePath = self._nsf nsf: NamespacePath = self._nsf
@ -1952,20 +1954,22 @@ class Context:
# overrun state and that msg isn't stuck in an # overrun state and that msg isn't stuck in an
# overflow queue what happens?!? # overflow queue what happens?!?
local_uid = self._actor.uid local_aid = self._actor.aid
txt: str = ( txt: str = (
'on IPC context:\n' 'on IPC context:\n'
f'<= sender: {from_uid}\n' f'<= sender: {from_uid}\n'
f' |_ {self._nsf}()\n\n' f' |_ {self._nsf}()\n\n'
f'=> overrun: {local_uid}\n' f'=> overrun: {local_aid.reprol()!r}\n'
f' |_cid: {cid}\n' f' |_cid: {cid}\n'
f' |_task: {self._task}\n' f' |_task: {self._task}\n'
) )
if not self._stream_opened: if not self._stream_opened:
txt += ( txt += (
f'\n*** No stream open on `{local_uid[0]}` side! ***\n\n' f'\n'
f'*** No stream open on `{local_aid.name}` side! ***\n'
f'\n'
f'{msg}\n' f'{msg}\n'
) )
@ -2114,7 +2118,11 @@ async def open_context_from_portal(
# XXX NOTE XXX: currenly we do NOT allow opening a contex # XXX NOTE XXX: currenly we do NOT allow opening a contex
# with "self" since the local feeder mem-chan processing # with "self" since the local feeder mem-chan processing
# is not built for it. # is not built for it.
if (uid := portal.channel.uid) == portal.actor.uid: if (
(uid := portal.channel.aid.uid)
==
portal.actor.aid.uid
):
raise RuntimeError( raise RuntimeError(
'** !! Invalid Operation !! **\n' '** !! Invalid Operation !! **\n'
'Can not open an IPC ctx with the local actor!\n' 'Can not open an IPC ctx with the local actor!\n'
@ -2328,7 +2336,7 @@ async def open_context_from_portal(
and and
ctxc is ctx._remote_error ctxc is ctx._remote_error
and and
ctxc.canceller == portal.actor.uid ctxc.canceller == portal.actor.aid.uid
): ):
log.cancel( log.cancel(
f'Context (cid=[{ctx.cid[-6:]}..] cancelled gracefully with:\n' f'Context (cid=[{ctx.cid[-6:]}..] cancelled gracefully with:\n'
@ -2391,19 +2399,21 @@ async def open_context_from_portal(
case trio.Cancelled(): case trio.Cancelled():
logmeth = log.cancel logmeth = log.cancel
cause: str = 'cancelled' cause: str = 'cancelled'
msg: str = (
f'ctx {ctx.side!r}-side {cause!r} with,\n'
f'{ctx.repr_outcome()!r}\n'
)
# XXX explicitly report on any non-graceful-taskc cases # XXX explicitly report on any non-graceful-taskc cases
case _: case _:
cause: str = 'errored' cause: str = 'errored'
logmeth = log.exception logmeth = log.exception
msg: str = f'ctx {ctx.side!r}-side {cause!r} with,\n'
logmeth( logmeth(msg)
f'ctx {ctx.side!r}-side {cause!r} with,\n'
f'{ctx.repr_outcome()!r}\n'
)
if debug_mode(): if debug_mode():
# async with debug.acquire_debug_lock(portal.actor.uid): # async with debug.acquire_debug_lock(portal.actor.aid.uid):
# pass # pass
# TODO: factor ^ into below for non-root cases? # TODO: factor ^ into below for non-root cases?
# #
@ -2426,10 +2436,7 @@ async def open_context_from_portal(
try: try:
# await pause(shield=True) # await pause(shield=True)
await ctx.cancel() await ctx.cancel()
except ( except TransportClosed:
trio.BrokenResourceError,
trio.ClosedResourceError,
):
log.warning( log.warning(
'IPC connection for context is broken?\n' 'IPC connection for context is broken?\n'
f'task: {ctx.cid}\n' f'task: {ctx.cid}\n'

View File

@ -43,7 +43,7 @@ from msgspec import (
ValidationError, ValidationError,
) )
from tractor._state import current_actor from tractor.runtime._state import current_actor
from tractor.log import get_logger from tractor.log import get_logger
from tractor.msg import ( from tractor.msg import (
Error, Error,
@ -187,7 +187,31 @@ _body_fields: list[str] = list(
) )
def get_err_type(type_name: str) -> BaseException|None: def reg_err_types(
exc_types: list[Type[Exception]],
) -> None:
'''
Register custom exception types for local lookup.
Such that error types can be registered by an external
`tractor`-use-app code base which are expected to be raised
remotely; enables them being re-raised on the receiver side of
some inter-actor IPC dialog.
'''
for exc_type in exc_types:
log.debug(
f'Register custom exception,\n'
f'{exc_type!r}\n'
)
setattr(
_this_mod,
exc_type.__name__,
exc_type,
)
def get_err_type(type_name: str) -> Type[BaseException]|None:
''' '''
Look up an exception type by name from the set of locally known Look up an exception type by name from the set of locally known
namespaces: namespaces:
@ -301,7 +325,8 @@ class RemoteActorError(Exception):
# also pertains to our long long oustanding issue XD # also pertains to our long long oustanding issue XD
# https://github.com/goodboy/tractor/issues/5 # https://github.com/goodboy/tractor/issues/5
self._boxed_type: BaseException = boxed_type self._boxed_type: BaseException = boxed_type
self._src_type: BaseException|None = None self._src_type: Type[BaseException]|None = None
self._src_type_resolved: bool = False
self._ipc_msg: Error|None = ipc_msg self._ipc_msg: Error|None = ipc_msg
self._extra_msgdata = extra_msgdata self._extra_msgdata = extra_msgdata
@ -410,24 +435,41 @@ class RemoteActorError(Exception):
return self._ipc_msg.src_type_str return self._ipc_msg.src_type_str
@property @property
def src_type(self) -> str: def src_type(self) -> Type[BaseException]|None:
''' '''
Error type raised by original remote faulting actor. Error type raised by original remote faulting
actor.
When the error has only been relayed a single actor-hop When the error has only been relayed a single
this will be the same as the `.boxed_type`. actor-hop this will be the same as
`.boxed_type`.
If the type can not be resolved locally (i.e.
it was not registered via `reg_err_types()`)
a warning is logged and `None` is returned;
all string-level error info (`.src_type_str`,
`.tb_str`, etc.) remains available.
''' '''
if self._src_type is None: if not self._src_type_resolved:
self._src_type_resolved = True
if self._ipc_msg is None:
return None
self._src_type = get_err_type( self._src_type = get_err_type(
self._ipc_msg.src_type_str self._ipc_msg.src_type_str
) )
if not self._src_type: if not self._src_type:
raise TypeError( log.warning(
f'Failed to lookup src error type with ' f'Failed to lookup src error type via\n'
f'`tractor._exceptions.get_err_type()` :\n' f'`tractor._exceptions.get_err_type()`:\n'
f'{self.src_type_str}' f'\n'
f'`{self._ipc_msg.src_type_str}`'
f' is not registered!\n'
f'\n'
f'Call `reg_err_types()` to enable'
f' full type reconstruction.\n'
) )
return self._src_type return self._src_type
@ -435,20 +477,30 @@ class RemoteActorError(Exception):
@property @property
def boxed_type_str(self) -> str: def boxed_type_str(self) -> str:
''' '''
String-name of the (last hop's) boxed error type. String-name of the (last hop's) boxed error
type.
Falls back to the IPC-msg-encoded type-name
str when the type can not be resolved locally
(e.g. unregistered custom errors).
''' '''
# TODO, maybe support also serializing the # TODO, maybe support also serializing the
# `ExceptionGroup.exeptions: list[BaseException]` set under # `ExceptionGroup.exceptions: list[BaseException]`
# certain conditions? # set under certain conditions?
bt: Type[BaseException] = self.boxed_type bt: Type[BaseException] = self.boxed_type
if bt: if bt:
return str(bt.__name__) return str(bt.__name__)
return '' # fallback to the str name from the IPC msg
# when the type obj can't be resolved.
if self._ipc_msg:
return self._ipc_msg.boxed_type_str
return '<unknown>'
@property @property
def boxed_type(self) -> Type[BaseException]: def boxed_type(self) -> Type[BaseException]|None:
''' '''
Error type boxed by last actor IPC hop. Error type boxed by last actor IPC hop.
@ -677,10 +729,22 @@ class RemoteActorError(Exception):
failing actor's remote env. failing actor's remote env.
''' '''
# TODO: better tb insertion and all the fancier dunder # TODO: better tb insertion and all the fancier
# metadata stuff as per `.__context__` etc. and friends: # dunder metadata stuff as per `.__context__`
# etc. and friends:
# https://github.com/python-trio/trio/issues/611 # https://github.com/python-trio/trio/issues/611
src_type_ref: Type[BaseException] = self.src_type src_type_ref: Type[BaseException]|None = (
self.src_type
)
if src_type_ref is None:
# unresolvable type: fall back to
# a `RuntimeError` preserving original
# traceback + type name.
return RuntimeError(
f'{self.src_type_str}: '
f'{self.tb_str}'
)
return src_type_ref(self.tb_str) return src_type_ref(self.tb_str)
# TODO: local recontruction of nested inception for a given # TODO: local recontruction of nested inception for a given
@ -982,6 +1046,7 @@ class TransportClosed(Exception):
''' '''
__tracebackhide__: bool = hide_tb __tracebackhide__: bool = hide_tb
message: str = message or self.message message: str = message or self.message
# when a cause is set, slap it onto the log emission. # when a cause is set, slap it onto the log emission.
if cause := self.src_exc: if cause := self.src_exc:
cause_tb_str: str = ''.join( cause_tb_str: str = ''.join(
@ -989,7 +1054,7 @@ class TransportClosed(Exception):
) )
message += ( message += (
f'{cause_tb_str}\n' # tb f'{cause_tb_str}\n' # tb
f' {cause}\n' # exc repr f'{cause!r}\n' # exc repr
) )
getattr( getattr(
@ -1208,14 +1273,31 @@ def unpack_error(
if not isinstance(msg, Error): if not isinstance(msg, Error):
return None return None
# try to lookup a suitable error type from the local runtime # try to lookup a suitable error type from the
# env then use it to construct a local instance. # local runtime env then use it to construct a
# boxed_type_str: str = error_dict['boxed_type_str'] # local instance.
boxed_type_str: str = msg.boxed_type_str boxed_type_str: str = msg.boxed_type_str
boxed_type: Type[BaseException] = get_err_type(boxed_type_str) boxed_type: Type[BaseException]|None = get_err_type(
boxed_type_str
)
# retrieve the error's msg-encoded remotoe-env info if boxed_type is None:
message: str = f'remote task raised a {msg.boxed_type_str!r}\n' log.warning(
f'Failed to resolve remote error type\n'
f'`{boxed_type_str}` - boxing as\n'
f'`RemoteActorError` with original\n'
f'traceback preserved.\n'
f'\n'
f'Call `reg_err_types()` to enable\n'
f'full type reconstruction.\n'
)
# retrieve the error's msg-encoded remote-env
# info
message: str = (
f'remote task raised a '
f'{msg.boxed_type_str!r}\n'
)
# TODO: do we even really need these checks for RAEs? # TODO: do we even really need these checks for RAEs?
if boxed_type_str in [ if boxed_type_str in [

View File

@ -37,19 +37,20 @@ import warnings
import trio import trio
from . import _runtime from .runtime import _runtime
from .discovery._registry import Registrar
from .devx import ( from .devx import (
debug, debug,
_frame_stack, _frame_stack,
pformat as _pformat, pformat as _pformat,
) )
from . import _spawn from .spawn import _spawn
from . import _state from .runtime import _state
from . import log from . import log
from .ipc import ( from .ipc import (
_connect_chan, _connect_chan,
) )
from ._addr import ( from .discovery._addr import (
Address, Address,
UnwrappedAddress, UnwrappedAddress,
default_lo_addrs, default_lo_addrs,
@ -88,7 +89,8 @@ async def maybe_block_bp(
bp_blocked: bool bp_blocked: bool
if ( if (
debug_mode debug_mode
and maybe_enable_greenback and
maybe_enable_greenback
and ( and (
maybe_mod := await debug.maybe_init_greenback( maybe_mod := await debug.maybe_init_greenback(
raise_not_found=False, raise_not_found=False,
@ -266,7 +268,6 @@ async def open_root_actor(
if start_method is not None: if start_method is not None:
_spawn.try_set_start_method(start_method) _spawn.try_set_start_method(start_method)
# TODO! remove this ASAP!
if arbiter_addr is not None: if arbiter_addr is not None:
warnings.warn( warnings.warn(
'`arbiter_addr` is now deprecated\n' '`arbiter_addr` is now deprecated\n'
@ -289,10 +290,12 @@ async def open_root_actor(
for uw_addr in uw_reg_addrs for uw_addr in uw_reg_addrs
] ]
loglevel = ( loglevel: str = (
loglevel loglevel
or log._default_loglevel or
).upper() log._default_loglevel
)
loglevel: str = loglevel.upper()
if ( if (
debug_mode debug_mode
@ -323,7 +326,10 @@ async def open_root_actor(
) )
assert loglevel assert loglevel
_log = log.get_console_log(loglevel) _log = log.get_console_log(
level=loglevel,
name='tractor',
)
assert _log assert _log
# TODO: factor this into `.devx._stackscope`!! # TODO: factor this into `.devx._stackscope`!!
@ -380,10 +386,13 @@ async def open_root_actor(
addr, addr,
) )
trans_bind_addrs: list[UnwrappedAddress] = [] tpt_bind_addrs: list[
Address # `Address.get_random()` case
|UnwrappedAddress # registrar case `= uw_reg_addrs`
] = []
# Create a new local root-actor instance which IS NOT THE # ------ NON-REGISTRAR ------
# REGISTRAR # create a new root-actor instance.
if ponged_addrs: if ponged_addrs:
if ensure_registry: if ensure_registry:
raise RuntimeError( raise RuntimeError(
@ -391,7 +400,7 @@ async def open_root_actor(
'registry socket(s) already bound' 'registry socket(s) already bound'
) )
# we were able to connect to an arbiter # we were able to connect to a registrar
logger.info( logger.info(
f'Registry(s) seem(s) to exist @ {ponged_addrs}' f'Registry(s) seem(s) to exist @ {ponged_addrs}'
) )
@ -410,12 +419,21 @@ async def open_root_actor(
# XXX INSTEAD, bind random addrs using the same tpt # XXX INSTEAD, bind random addrs using the same tpt
# proto. # proto.
for addr in ponged_addrs: for addr in ponged_addrs:
trans_bind_addrs.append( tpt_bind_addrs.append(
# XXX, these are `Address` NOT `UnwrappedAddress`.
#
# NOTE, in the case of posix/berkley socket
# protos we allocate port=0 such that the system
# allocates a random value at bind time; this
# happens in the `.ipc.*` stack's backend.
addr.get_random( addr.get_random(
bindspace=addr.bindspace, bindspace=addr.bindspace,
) )
) )
# ------ REGISTRAR ------
# create a new "registry providing" root-actor instance.
#
# Start this local actor as the "registrar", aka a regular # Start this local actor as the "registrar", aka a regular
# actor who manages the local registry of "mailboxes" of # actor who manages the local registry of "mailboxes" of
# other process-tree-local sub-actors. # other process-tree-local sub-actors.
@ -424,7 +442,7 @@ async def open_root_actor(
# following init steps are taken: # following init steps are taken:
# - the tranport layer server is bound to each addr # - the tranport layer server is bound to each addr
# pair defined in provided registry_addrs, or the default. # pair defined in provided registry_addrs, or the default.
trans_bind_addrs = uw_reg_addrs tpt_bind_addrs = uw_reg_addrs
# - it is normally desirable for any registrar to stay up # - it is normally desirable for any registrar to stay up
# indefinitely until either all registered (child/sub) # indefinitely until either all registered (child/sub)
@ -435,8 +453,7 @@ async def open_root_actor(
# https://github.com/goodboy/tractor/pull/348 # https://github.com/goodboy/tractor/pull/348
# https://github.com/goodboy/tractor/issues/296 # https://github.com/goodboy/tractor/issues/296
# TODO: rename as `RootActor` or is that even necessary? actor = Registrar(
actor = _runtime.Arbiter(
name=name or 'registrar', name=name or 'registrar',
uuid=mk_uuid(), uuid=mk_uuid(),
registry_addrs=registry_addrs, registry_addrs=registry_addrs,
@ -444,20 +461,10 @@ async def open_root_actor(
enable_modules=enable_modules, enable_modules=enable_modules,
) )
# XXX, in case the root actor runtime was actually run from # XXX, in case the root actor runtime was actually run from
# `tractor.to_asyncio.run_as_asyncio_guest()` and NOt # `tractor.to_asyncio.run_as_asyncio_guest()` and NOT
# `.trio.run()`. # `.trio.run()`.
actor._infected_aio = _state._runtime_vars['_is_infected_aio'] actor._infected_aio = _state._runtime_vars['_is_infected_aio']
# NOTE, only set the loopback addr for the
# process-tree-global "root" mailbox since all sub-actors
# should be able to speak to their root actor over that
# channel.
raddrs: list[Address] = _state._runtime_vars['_root_addrs']
raddrs.extend(trans_bind_addrs)
# TODO, remove once we have also removed all usage;
# eventually all (root-)registry apis should expect > 1 addr.
_state._runtime_vars['_root_mailbox'] = raddrs[0]
# Start up main task set via core actor-runtime nurseries. # Start up main task set via core actor-runtime nurseries.
try: try:
# assign process-local actor # assign process-local actor
@ -494,14 +501,39 @@ async def open_root_actor(
# "actor runtime" primitives are SC-compat and thus all # "actor runtime" primitives are SC-compat and thus all
# transitively spawned actors/processes must be as # transitively spawned actors/processes must be as
# well. # well.
await root_tn.start( accept_addrs: list[UnwrappedAddress]
reg_addrs: list[UnwrappedAddress]
(
accept_addrs,
reg_addrs,
) = await root_tn.start(
partial( partial(
_runtime.async_main, _runtime.async_main,
actor, actor,
accept_addrs=trans_bind_addrs, accept_addrs=tpt_bind_addrs,
parent_addr=None parent_addr=None
) )
) )
# NOTE, only set a local-host addr (i.e. like
# `lo`-loopback for TCP) for the process-tree-global
# "root"-process (its tree-wide "mailbox") since all
# sub-actors should be able to speak to their root
# actor over that channel.
#
# ?TODO, per-OS non-network-proto alt options?
# -[ ] on linux we should be able to always use UDS?
#
raddrs: list[UnwrappedAddress] = _state._runtime_vars['_root_addrs']
raddrs.extend(
accept_addrs,
)
# TODO, remove once we have also removed all usage;
# eventually all (root-)registry apis should expect > 1 addr.
_state._runtime_vars['_root_mailbox'] = raddrs[0]
# if 'chart' in actor.aid.name:
# from tractor.devx import mk_pdb
# mk_pdb().set_trace()
try: try:
yield actor yield actor
except ( except (
@ -583,6 +615,13 @@ async def open_root_actor(
): ):
_state._runtime_vars['_debug_mode'] = False _state._runtime_vars['_debug_mode'] = False
# !XXX, clear ALL prior contact info state, this is MEGA
# important if you are opening the runtime multiple times
# from the same parent process (like in our test
# harness)!
_state._runtime_vars['_root_addrs'].clear()
_state._runtime_vars['_root_mailbox'] = None
_state._current_actor = None _state._current_actor = None
_state._last_actor_terminated = actor _state._last_actor_terminated = actor

View File

@ -38,6 +38,7 @@ import trio
from ._exceptions import ( from ._exceptions import (
ContextCancelled, ContextCancelled,
RemoteActorError, RemoteActorError,
TransportClosed,
) )
from .log import get_logger from .log import get_logger
from .trionics import ( from .trionics import (
@ -54,12 +55,12 @@ from tractor.msg import (
) )
if TYPE_CHECKING: if TYPE_CHECKING:
from ._runtime import Actor from .runtime._runtime import Actor
from ._context import Context from ._context import Context
from .ipc import Channel from .ipc import Channel
log = get_logger(__name__) log = get_logger()
# TODO: the list # TODO: the list
@ -409,10 +410,8 @@ class MsgStream(trio.abc.Channel):
# it). # it).
with trio.CancelScope(shield=True): with trio.CancelScope(shield=True):
await self._ctx.send_stop() await self._ctx.send_stop()
except ( except (
trio.BrokenResourceError, TransportClosed,
trio.ClosedResourceError
) as re: ) as re:
# the underlying channel may already have been pulled # the underlying channel may already have been pulled
# in which case our stop message is meaningless since # in which case our stop message is meaningless since
@ -593,9 +592,8 @@ class MsgStream(trio.abc.Channel):
), ),
) )
except ( except (
trio.ClosedResourceError,
trio.BrokenResourceError,
BrokenPipeError, BrokenPipeError,
TransportClosed,
) as _trans_err: ) as _trans_err:
trans_err = _trans_err trans_err = _trans_err
if ( if (

View File

@ -26,9 +26,7 @@ import random
from typing import ( from typing import (
Type, Type,
) )
from tractor import ( from tractor.discovery import _addr
_addr,
)
def get_rando_addr( def get_rando_addr(
@ -61,7 +59,11 @@ def get_rando_addr(
# NOTE, file-name uniqueness (no-collisions) will be based on # NOTE, file-name uniqueness (no-collisions) will be based on
# the runtime-directory and root (pytest-proc's) pid. # the runtime-directory and root (pytest-proc's) pid.
case 'uds': case 'uds':
testrun_reg_addr = addr_type.get_random().unwrap() from tractor.ipc._uds import UDSAddress
addr: UDSAddress = addr_type.get_random()
assert addr.is_valid
assert addr.sockpath.resolve()
testrun_reg_addr = addr.unwrap()
# XXX, as sanity it should never the same as the default for the # XXX, as sanity it should never the same as the default for the
# host-singleton registry actor. # host-singleton registry actor.

View File

@ -21,17 +21,27 @@ and applications.
''' '''
from functools import ( from functools import (
partial, partial,
wraps,
) )
import inspect import inspect
import platform import platform
from typing import (
Callable,
Type,
)
import pytest import pytest
import tractor import tractor
import trio import trio
import wrapt
def tractor_test(fn): def tractor_test(
wrapped: Callable|None = None,
*,
# @tractor_test(<deco-params>)
timeout:float = 30,
hide_tb: bool = True,
):
''' '''
Decorator for async test fns to decorator-wrap them as "native" Decorator for async test fns to decorator-wrap them as "native"
looking sync funcs runnable by `pytest` and auto invoked with looking sync funcs runnable by `pytest` and auto invoked with
@ -45,8 +55,18 @@ def tractor_test(fn):
Basic deco use: Basic deco use:
--------------- ---------------
@tractor_test @tractor_test(
async def test_whatever(): timeout=10,
)
async def test_whatever(
# fixture param declarations
loglevel: str,
start_method: str,
reg_addr: tuple,
tpt_proto: str,
debug_mode: bool,
):
# already inside a root-actor runtime `trio.Task`
await ... await ...
@ -55,7 +75,7 @@ def tractor_test(fn):
If any of the following fixture are requested by the wrapped test If any of the following fixture are requested by the wrapped test
fn (via normal func-args declaration), fn (via normal func-args declaration),
- `reg_addr` (a socket addr tuple where arbiter is listening) - `reg_addr` (a socket addr tuple where registrar is listening)
- `loglevel` (logging level passed to tractor internals) - `loglevel` (logging level passed to tractor internals)
- `start_method` (subprocess spawning backend) - `start_method` (subprocess spawning backend)
@ -67,48 +87,69 @@ def tractor_test(fn):
`tractor.open_root_actor()` funcargs. `tractor.open_root_actor()` funcargs.
''' '''
@wraps(fn) __tracebackhide__: bool = hide_tb
# handle the decorator not called with () case.
# i.e. in `wrapt` support a deco-with-optional-args,
# https://wrapt.readthedocs.io/en/master/decorators.html#decorators-with-optional-arguments
if wrapped is None:
return wrapt.PartialCallableObjectProxy(
tractor_test,
timeout=timeout,
hide_tb=hide_tb
)
@wrapt.decorator
def wrapper( def wrapper(
wrapped: Callable,
instance: object|Type|None,
args: tuple,
kwargs: dict,
):
__tracebackhide__: bool = hide_tb
# NOTE, ensure we inject any test-fn declared fixture names.
for kw in [
'reg_addr',
'loglevel',
'start_method',
'debug_mode',
'tpt_proto',
'timeout',
]:
if kw in inspect.signature(wrapped).parameters:
assert kw in kwargs
start_method = kwargs.get('start_method')
if platform.system() == "Windows":
if start_method is None:
kwargs['start_method'] = 'trio'
elif start_method != 'trio':
raise ValueError(
'ONLY the `start_method="trio"` is supported on Windows.'
)
# open a root-actor, passing certain
# `tractor`-runtime-settings, then invoke the test-fn body as
# the root-most task.
#
# https://wrapt.readthedocs.io/en/master/decorators.html#processing-function-arguments
async def _main(
*args, *args,
loglevel=None,
reg_addr=None, # runtime-settings
loglevel:str|None = None,
reg_addr:tuple|None = None,
start_method: str|None = None, start_method: str|None = None,
debug_mode: bool = False, debug_mode: bool = False,
**kwargs tpt_proto: str|None = None,
**kwargs,
): ):
# __tracebackhide__ = True __tracebackhide__: bool = hide_tb
# NOTE: inject ant test func declared fixture with trio.fail_after(timeout):
# names by manually checking!
if 'reg_addr' in inspect.signature(fn).parameters:
# injects test suite fixture value to test as well
# as `run()`
kwargs['reg_addr'] = reg_addr
if 'loglevel' in inspect.signature(fn).parameters:
# allows test suites to define a 'loglevel' fixture
# that activates the internal logging
kwargs['loglevel'] = loglevel
if start_method is None:
if platform.system() == "Windows":
start_method = 'trio'
if 'start_method' in inspect.signature(fn).parameters:
# set of subprocess spawning backends
kwargs['start_method'] = start_method
if 'debug_mode' in inspect.signature(fn).parameters:
# set of subprocess spawning backends
kwargs['debug_mode'] = debug_mode
if kwargs:
# use explicit root actor start
async def _main():
async with tractor.open_root_actor( async with tractor.open_root_actor(
# **kwargs,
registry_addrs=[reg_addr] if reg_addr else None, registry_addrs=[reg_addr] if reg_addr else None,
loglevel=loglevel, loglevel=loglevel,
start_method=start_method, start_method=start_method,
@ -117,17 +158,31 @@ def tractor_test(fn):
debug_mode=debug_mode, debug_mode=debug_mode,
): ):
await fn(*args, **kwargs) # invoke test-fn body IN THIS task
await wrapped(
*args,
**kwargs,
)
main = _main funcname = wrapped.__name__
if not inspect.iscoroutinefunction(wrapped):
raise TypeError(
f"Test-fn {funcname!r} must be an async-function !!"
)
else: # invoke runtime via a root task.
# use implicit root actor start return trio.run(
main = partial(fn, *args, **kwargs) partial(
_main,
*args,
**kwargs,
)
)
return trio.run(main)
return wrapper return wrapper(
wrapped,
)
def pytest_addoption( def pytest_addoption(
@ -175,7 +230,15 @@ def pytest_addoption(
def pytest_configure(config): def pytest_configure(config):
backend = config.option.spawn_backend backend = config.option.spawn_backend
tractor._spawn.try_set_start_method(backend) from tractor.spawn._spawn import try_set_start_method
try_set_start_method(backend)
# register custom marks to avoid warnings see,
# https://docs.pytest.org/en/stable/how-to/writing_plugins.html#registering-custom-markers
config.addinivalue_line(
'markers',
'no_tpt(proto_key): test will (likely) not behave with tpt backend'
)
@pytest.fixture(scope='session') @pytest.fixture(scope='session')
@ -214,7 +277,8 @@ def tpt_protos(request) -> list[str]:
# XXX ensure we support the protocol by name via lookup! # XXX ensure we support the protocol by name via lookup!
for proto_key in proto_keys: for proto_key in proto_keys:
addr_type = tractor._addr._address_types[proto_key] from tractor.discovery import _addr
addr_type = _addr._address_types[proto_key]
assert addr_type.proto_key == proto_key assert addr_type.proto_key == proto_key
yield proto_keys yield proto_keys
@ -225,13 +289,32 @@ def tpt_protos(request) -> list[str]:
autouse=True, autouse=True,
) )
def tpt_proto( def tpt_proto(
request,
tpt_protos: list[str], tpt_protos: list[str],
) -> str: ) -> str:
proto_key: str = tpt_protos[0] proto_key: str = tpt_protos[0]
from tractor import _state # ?TODO, but needs a fn-scoped tpt_proto fixture..
# @pytest.mark.no_tpt('uds')
# node = request.node
# markers = node.own_markers
# for mark in markers:
# if (
# mark.name == 'no_tpt'
# and
# proto_key in mark.args
# ):
# pytest.skip(
# f'Test {node} normally fails with '
# f'tpt-proto={proto_key!r}\n'
# )
from tractor.runtime import _state
if _state._def_tpt_proto != proto_key: if _state._def_tpt_proto != proto_key:
_state._def_tpt_proto = proto_key _state._def_tpt_proto = proto_key
_state._runtime_vars['_enable_tpts'] = [
proto_key,
]
yield proto_key yield proto_key

View File

@ -49,7 +49,7 @@ from tractor.msg import (
import wrapt import wrapt
log = get_logger(__name__) log = get_logger()
# TODO: yeah, i don't love this and we should prolly just # TODO: yeah, i don't love this and we should prolly just
# write a decorator that actually keeps a stupid ref to the func # write a decorator that actually keeps a stupid ref to the func

View File

@ -45,17 +45,15 @@ from typing import (
) )
import trio import trio
from tractor import ( from tractor.runtime import _state
_state, from tractor import log as logmod
log as logmod,
)
from tractor.devx import debug from tractor.devx import debug
log = logmod.get_logger(__name__) log = logmod.get_logger()
if TYPE_CHECKING: if TYPE_CHECKING:
from tractor._spawn import ProcessType from tractor.spawn._spawn import ProcessType
from tractor import ( from tractor import (
Actor, Actor,
ActorNursery, ActorNursery,

View File

@ -59,7 +59,7 @@ from ._sigint import (
_ctlc_ignore_header as _ctlc_ignore_header _ctlc_ignore_header as _ctlc_ignore_header
) )
log = get_logger(__name__) log = get_logger()
# ---------------- # ----------------
# XXX PKG TODO XXX # XXX PKG TODO XXX

View File

@ -53,8 +53,8 @@ import trio
from tractor._exceptions import ( from tractor._exceptions import (
NoRuntime, NoRuntime,
) )
from tractor import _state from tractor.runtime import _state
from tractor._state import ( from tractor.runtime._state import (
current_actor, current_actor,
debug_mode, debug_mode,
) )
@ -76,7 +76,7 @@ from ._repl import (
if TYPE_CHECKING: if TYPE_CHECKING:
from trio.lowlevel import Task from trio.lowlevel import Task
from tractor._runtime import ( from tractor.runtime._runtime import (
Actor, Actor,
) )
@ -84,7 +84,7 @@ _crash_msg: str = (
'Opening a pdb REPL in crashed actor' 'Opening a pdb REPL in crashed actor'
) )
log = get_logger(__package__) log = get_logger()
class BoxedMaybeException(Struct): class BoxedMaybeException(Struct):

View File

@ -25,7 +25,7 @@ from functools import (
import os import os
import pdbp import pdbp
from tractor._state import ( from tractor.runtime._state import (
is_root_process, is_root_process,
) )

View File

@ -21,12 +21,13 @@ cancellation during REPL interaction.
''' '''
from __future__ import annotations from __future__ import annotations
import platform
from typing import ( from typing import (
TYPE_CHECKING, TYPE_CHECKING,
) )
import trio import trio
from tractor.log import get_logger from tractor.log import get_logger
from tractor._state import ( from tractor.runtime._state import (
current_actor, current_actor,
is_root_process, is_root_process,
) )
@ -43,12 +44,13 @@ if TYPE_CHECKING:
from tractor.ipc import ( from tractor.ipc import (
Channel, Channel,
) )
from tractor._runtime import ( from tractor.runtime._runtime import (
Actor, Actor,
) )
log = get_logger(__name__) log = get_logger()
_is_macos: bool = platform.system() == 'Darwin'
_ctlc_ignore_header: str = ( _ctlc_ignore_header: str = (
'Ignoring SIGINT while debug REPL in use' 'Ignoring SIGINT while debug REPL in use'
) )
@ -300,6 +302,11 @@ def sigint_shield(
# XXX: yah, mega hack, but how else do we catch this madness XD # XXX: yah, mega hack, but how else do we catch this madness XD
if ( if (
repl.shname == 'xonsh' repl.shname == 'xonsh'
or (
repl.shname == 'bash'
and
_is_macos
)
): ):
flush_status += ( flush_status += (
'-> ALSO re-flushing due to `xonsh`..\n' '-> ALSO re-flushing due to `xonsh`..\n'

View File

@ -40,7 +40,7 @@ from trio.lowlevel import (
Task, Task,
) )
from tractor._context import Context from tractor._context import Context
from tractor._state import ( from tractor.runtime._state import (
current_actor, current_actor,
debug_mode, debug_mode,
is_root_process, is_root_process,
@ -58,7 +58,7 @@ from ._sigint import (
_ctlc_ignore_header as _ctlc_ignore_header _ctlc_ignore_header as _ctlc_ignore_header
) )
log = get_logger(__package__) log = get_logger()
async def maybe_wait_for_debugger( async def maybe_wait_for_debugger(

View File

@ -55,12 +55,12 @@ import tractor
from tractor.log import get_logger from tractor.log import get_logger
from tractor.to_asyncio import run_trio_task_in_future from tractor.to_asyncio import run_trio_task_in_future
from tractor._context import Context from tractor._context import Context
from tractor import _state from tractor.runtime import _state
from tractor._exceptions import ( from tractor._exceptions import (
NoRuntime, NoRuntime,
InternalError, InternalError,
) )
from tractor._state import ( from tractor.runtime._state import (
current_actor, current_actor,
current_ipc_ctx, current_ipc_ctx,
is_root_process, is_root_process,
@ -87,13 +87,13 @@ from ..pformat import (
if TYPE_CHECKING: if TYPE_CHECKING:
from trio.lowlevel import Task from trio.lowlevel import Task
from threading import Thread from threading import Thread
from tractor._runtime import ( from tractor.runtime._runtime import (
Actor, Actor,
) )
# from ._post_mortem import BoxedMaybeException # from ._post_mortem import BoxedMaybeException
from ._repl import PdbREPL from ._repl import PdbREPL
log = get_logger(__package__) log = get_logger()
_pause_msg: str = 'Opening a pdb REPL in paused actor' _pause_msg: str = 'Opening a pdb REPL in paused actor'
_repl_fail_msg: str|None = ( _repl_fail_msg: str|None = (
@ -628,7 +628,7 @@ def _set_trace(
log.pdb( log.pdb(
f'{_pause_msg}\n' f'{_pause_msg}\n'
f'>(\n' f'>(\n'
f'|_{actor.uid}\n' f'|_{actor.aid.uid}\n'
f' |_{task}\n' # @ {actor.uid}\n' f' |_{task}\n' # @ {actor.uid}\n'
# f'|_{task}\n' # f'|_{task}\n'
# ^-TODO-^ more compact pformating? # ^-TODO-^ more compact pformating?
@ -1257,3 +1257,26 @@ async def breakpoint(
api_frame=inspect.currentframe(), api_frame=inspect.currentframe(),
**kwargs, **kwargs,
) )
async def maybe_pause_bp():
'''
Internal (ONLY for now) `breakpoint()`-er fn which only tries to
use the multi-actor `.pause()` API when the current actor is the
root.
?! BUT WHY !?
-------
This is useful when debugging cases where the tpt layer breaks
(or is intentionally broken, say during resiliency testing) in
the case where a child can no longer contact the root process to
acquire the process-tree-singleton TTY lock.
'''
import tractor
actor = tractor.current_actor()
if actor.aid.name == 'root':
await tractor.pause(shield=True)
else:
tractor.devx.mk_pdb().set_trace()

View File

@ -55,12 +55,12 @@ import tractor
from tractor.to_asyncio import run_trio_task_in_future from tractor.to_asyncio import run_trio_task_in_future
from tractor.log import get_logger from tractor.log import get_logger
from tractor._context import Context from tractor._context import Context
from tractor import _state from tractor.runtime import _state
from tractor._exceptions import ( from tractor._exceptions import (
DebugRequestError, DebugRequestError,
InternalError, InternalError,
) )
from tractor._state import ( from tractor.runtime._state import (
current_actor, current_actor,
is_root_process, is_root_process,
) )
@ -71,7 +71,7 @@ if TYPE_CHECKING:
from tractor.ipc import ( from tractor.ipc import (
IPCServer, IPCServer,
) )
from tractor._runtime import ( from tractor.runtime._runtime import (
Actor, Actor,
) )
from ._repl import ( from ._repl import (
@ -81,7 +81,7 @@ if TYPE_CHECKING:
BoxedMaybeException, BoxedMaybeException,
) )
log = get_logger(__name__) log = get_logger()
class LockStatus( class LockStatus(
@ -1013,7 +1013,7 @@ async def request_root_stdio_lock(
DebugStatus.req_task = current_task() DebugStatus.req_task = current_task()
req_err: BaseException|None = None req_err: BaseException|None = None
try: try:
from tractor._discovery import get_root from tractor.discovery._discovery import get_root
# NOTE: we need this to ensure that this task exits # NOTE: we need this to ensure that this task exits
# BEFORE the REPl instance raises an error like # BEFORE the REPl instance raises an error like
# `bdb.BdbQuit` directly, OW you get a trio cs stack # `bdb.BdbQuit` directly, OW you get a trio cs stack

View File

@ -0,0 +1,26 @@
# tractor: structured concurrent "actors".
# Copyright 2018-eternity Tyler Goodlet.
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU Affero General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU Affero General Public License for more details.
# You should have received a copy of the GNU Affero General Public License
# along with this program. If not, see <https://www.gnu.org/licenses/>.
'''
Discovery (protocols) API for automatic addressing
and location management of (service) actors.
NOTE: to avoid circular imports, this ``__init__``
does NOT eagerly import submodules. Use direct
module paths like ``tractor.discovery._addr`` or
``tractor.discovery._discovery`` instead.
'''

View File

@ -27,17 +27,17 @@ from trio import (
SocketListener, SocketListener,
) )
from .log import get_logger from ..log import get_logger
from ._state import ( from ..runtime._state import (
_def_tpt_proto, _def_tpt_proto,
) )
from .ipc._tcp import TCPAddress from ..ipc._tcp import TCPAddress
from .ipc._uds import UDSAddress from ..ipc._uds import UDSAddress
if TYPE_CHECKING: if TYPE_CHECKING:
from ._runtime import Actor from ..runtime._runtime import Actor
log = get_logger(__name__) log = get_logger()
# TODO, maybe breakout the netns key to a struct? # TODO, maybe breakout the netns key to a struct?
@ -259,6 +259,8 @@ def wrap_address(
case _: case _:
# import pdbp; pdbp.set_trace() # import pdbp; pdbp.set_trace()
# from tractor.devx import mk_pdb
# mk_pdb().set_trace()
raise TypeError( raise TypeError(
f'Can not wrap unwrapped-address ??\n' f'Can not wrap unwrapped-address ??\n'
f'type(addr): {type(addr)!r}\n' f'type(addr): {type(addr)!r}\n'

View File

@ -28,39 +28,39 @@ from typing import (
from contextlib import asynccontextmanager as acm from contextlib import asynccontextmanager as acm
from tractor.log import get_logger from tractor.log import get_logger
from .trionics import ( from ..trionics import (
gather_contexts, gather_contexts,
collapse_eg, collapse_eg,
) )
from .ipc import _connect_chan, Channel from ..ipc import _connect_chan, Channel
from ._addr import ( from ._addr import (
UnwrappedAddress, UnwrappedAddress,
Address, Address,
wrap_address wrap_address
) )
from ._portal import ( from ..runtime._portal import (
Portal, Portal,
open_portal, open_portal,
LocalPortal, LocalPortal,
) )
from ._state import ( from ..runtime._state import (
current_actor, current_actor,
_runtime_vars, _runtime_vars,
_def_tpt_proto, _def_tpt_proto,
) )
if TYPE_CHECKING: if TYPE_CHECKING:
from ._runtime import Actor from ..runtime._runtime import Actor
log = get_logger(__name__) log = get_logger()
@acm @acm
async def get_registry( async def get_registry(
addr: UnwrappedAddress|None = None, addr: UnwrappedAddress|None = None,
) -> AsyncGenerator[ ) -> AsyncGenerator[
Portal | LocalPortal | None, Portal|LocalPortal|None,
None, None,
]: ]:
''' '''
@ -72,8 +72,8 @@ async def get_registry(
''' '''
actor: Actor = current_actor() actor: Actor = current_actor()
if actor.is_registrar: if actor.is_registrar:
# we're already the arbiter # we're already the registrar
# (likely a re-entrant call from the arbiter actor) # (likely a re-entrant call from the registrar actor)
yield LocalPortal( yield LocalPortal(
actor, actor,
Channel(transport=None) Channel(transport=None)
@ -91,10 +91,13 @@ async def get_registry(
@acm @acm
async def get_root( async def get_root(**kwargs) -> AsyncGenerator[Portal, None]:
**kwargs, '''
) -> AsyncGenerator[Portal, None]: Deliver the current actor's "root process" actor (yes in actor
and proc tree terms) by delivering a `Portal` from the spawn-time
provided contact address.
'''
# TODO: rename mailbox to `_root_maddr` when we finally # TODO: rename mailbox to `_root_maddr` when we finally
# add and impl libp2p multi-addrs? # add and impl libp2p multi-addrs?
addr = _runtime_vars['_root_mailbox'] addr = _runtime_vars['_root_mailbox']
@ -150,21 +153,27 @@ async def query_actor(
regaddr: UnwrappedAddress|None = None, regaddr: UnwrappedAddress|None = None,
) -> AsyncGenerator[ ) -> AsyncGenerator[
UnwrappedAddress|None, tuple[UnwrappedAddress|None, Portal|LocalPortal|None],
None, None,
]: ]:
''' '''
Lookup a transport address (by actor name) via querying a registrar Lookup a transport address (by actor name) via querying a registrar
listening @ `regaddr`. listening @ `regaddr`.
Returns the transport protocol (socket) address or `None` if no Yields a `tuple` of `(addr, reg_portal)` where,
entry under that name exists. - `addr` is the transport protocol (socket) address or `None` if
no entry under that name exists,
- `reg_portal` is the `Portal` (or `LocalPortal` when the
current actor is the registrar) used for the lookup (or
`None` when the peer was found locally via
`get_peer_by_name()`).
''' '''
actor: Actor = current_actor() actor: Actor = current_actor()
if ( if (
name == 'registrar' name == 'registrar'
and actor.is_registrar and
actor.is_registrar
): ):
raise RuntimeError( raise RuntimeError(
'The current actor IS the registry!?' 'The current actor IS the registry!?'
@ -172,10 +181,10 @@ async def query_actor(
maybe_peers: list[Channel]|None = get_peer_by_name(name) maybe_peers: list[Channel]|None = get_peer_by_name(name)
if maybe_peers: if maybe_peers:
yield maybe_peers[0].raddr yield maybe_peers[0].raddr, None
return return
reg_portal: Portal reg_portal: Portal|LocalPortal
regaddr: Address = wrap_address(regaddr) or actor.reg_addrs[0] regaddr: Address = wrap_address(regaddr) or actor.reg_addrs[0]
async with get_registry(regaddr) as reg_portal: async with get_registry(regaddr) as reg_portal:
# TODO: return portals to all available actors - for now # TODO: return portals to all available actors - for now
@ -185,25 +194,63 @@ async def query_actor(
'find_actor', 'find_actor',
name=name, name=name,
) )
yield addr yield addr, reg_portal
@acm @acm
async def maybe_open_portal( async def maybe_open_portal(
addr: UnwrappedAddress, addr: UnwrappedAddress,
name: str, name: str,
): ):
'''
Open a `Portal` to the actor serving @ `addr` or `None` if no
peer can be contacted or found.
'''
async with query_actor( async with query_actor(
name=name, name=name,
regaddr=addr, regaddr=addr,
) as addr: ) as (addr, reg_portal):
pass if not addr:
yield None
return
if addr: try:
async with _connect_chan(addr) as chan: async with _connect_chan(addr) as chan:
async with open_portal(chan) as portal: async with open_portal(chan) as portal:
yield portal yield portal
# most likely we were unable to connect the
# transport and there is likely a stale entry in
# the registry actor's table, thus we need to
# instruct it to clear that stale entry and then
# more silently (pretend there was no reason but
# to) indicate that the target actor can't be
# contacted at that addr.
except OSError:
# NOTE: ensure we delete the stale entry
# from the registrar actor when available.
if reg_portal is not None:
uid: tuple[str, str]|None = await reg_portal.run_from_ns(
'self',
'delete_addr',
addr=addr,
)
if uid:
log.warning(
f'Deleted stale registry entry !\n'
f'addr: {addr!r}\n'
f'uid: {uid!r}\n'
)
else: else:
log.warning(
f'No registry entry found for addr: {addr!r}'
)
else:
log.warning(
f'Connection to {addr!r} failed'
f' and no registry portal available'
f' to delete stale entry.'
)
yield None yield None
@ -221,10 +268,10 @@ async def find_actor(
None, None,
]: ]:
''' '''
Ask the arbiter to find actor(s) by name. Ask the registrar to find actor(s) by name.
Returns a connected portal to the last registered matching actor Returns a connected portal to the last registered
known to the arbiter. matching actor known to the registrar.
''' '''
# optimization path, use any pre-existing peer channel # optimization path, use any pre-existing peer channel
@ -272,7 +319,7 @@ async def find_actor(
if not any(portals): if not any(portals):
if raise_on_none: if raise_on_none:
raise RuntimeError( raise RuntimeError(
f'No actor "{name}" found registered @ {registry_addrs}' f'No actor {name!r} found registered @ {registry_addrs!r}'
) )
yield None yield None
return return

View File

@ -0,0 +1,253 @@
# tractor: structured concurrent "actors".
# Copyright 2018-eternity Tyler Goodlet.
# This program is free software: you can redistribute it and/or
# modify it under the terms of the GNU Affero General Public
# License as published by the Free Software Foundation, either
# version 3 of the License, or (at your option) any later
# version.
# This program is distributed in the hope that it will be
# useful, but WITHOUT ANY WARRANTY; without even the implied
# warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR
# PURPOSE. See the GNU Affero General Public License for more
# details.
# You should have received a copy of the GNU Affero General
# Public License along with this program. If not, see
# <https://www.gnu.org/licenses/>.
'''
Actor-registry for process-tree service discovery.
The `Registrar` is a special `Actor` subtype that serves as
the process-tree's name-registry, tracking actor
name-to-address mappings so peers can discover each other.
'''
from __future__ import annotations
from bidict import bidict
import trio
from ..runtime._runtime import Actor
from ._addr import (
UnwrappedAddress,
Address,
wrap_address,
)
from ..devx import debug
from ..log import get_logger
log = get_logger('tractor')
class Registrar(Actor):
'''
A special registrar `Actor` who can contact all other
actors within its immediate process tree and keeps
a registry of others meant to be discoverable in
a distributed application.
Normally the registrar is also the "root actor" and
thus always has access to the top-most-level actor
(process) nursery.
By default, the registrar is always initialized when
and if no other registrar socket addrs have been
specified to runtime init entry-points (such as
`open_root_actor()` or `open_nursery()`). Any time
a new main process is launched (and thus a new root
actor created) and, no existing registrar can be
contacted at the provided `registry_addr`, then
a new one is always created; however, if one can be
reached it is used.
Normally a distributed app requires at least one
registrar per logical host where for that given
"host space" (aka localhost IPC domain of addresses)
it is responsible for making all other host (local
address) bound actors *discoverable* to external
actor trees running on remote hosts.
'''
is_registrar = True
def is_registry(self) -> bool:
return self.is_registrar
def __init__(
self,
*args,
**kwargs,
) -> None:
self._registry: bidict[
tuple[str, str],
UnwrappedAddress,
] = bidict({})
self._waiters: dict[
str,
# either an event to sync to receiving an
# actor uid (which is filled in once the actor
# has sucessfully registered), or that uid
# after registry is complete.
list[trio.Event|tuple[str, str]]
] = {}
super().__init__(*args, **kwargs)
async def find_actor(
self,
name: str,
) -> UnwrappedAddress|None:
for uid, addr in self._registry.items():
if name in uid:
return addr
return None
async def get_registry(
self
) -> dict[str, UnwrappedAddress]:
'''
Return current name registry.
This method is async to allow for cross-actor
invocation.
'''
# NOTE: requires ``strict_map_key=False`` to the
# msgpack unpacker since we have tuples as keys
# (note this makes the registrar suscetible to
# hashdos):
# https://github.com/msgpack/msgpack-python#major-breaking-changes-in-msgpack-10
return {
'.'.join(key): val
for key, val in self._registry.items()
}
async def wait_for_actor(
self,
name: str,
) -> list[UnwrappedAddress]:
'''
Wait for a particular actor to register.
This is a blocking call if no actor by the
provided name is currently registered.
'''
addrs: list[UnwrappedAddress] = []
addr: UnwrappedAddress
mailbox_info: str = (
'Actor registry contact infos:\n'
)
for uid, addr in self._registry.items():
mailbox_info += (
f'|_uid: {uid}\n'
f'|_addr: {addr}\n\n'
)
if name == uid[0]:
addrs.append(addr)
if not addrs:
waiter = trio.Event()
self._waiters.setdefault(
name, []
).append(waiter)
await waiter.wait()
for uid in self._waiters[name]:
if not isinstance(uid, trio.Event):
addrs.append(
self._registry[uid]
)
log.runtime(mailbox_info)
return addrs
async def register_actor(
self,
uid: tuple[str, str],
addr: UnwrappedAddress
) -> None:
uid = name, hash = (
str(uid[0]),
str(uid[1]),
)
waddr: Address = wrap_address(addr)
if not waddr.is_valid:
# should never be 0-dynamic-os-alloc
await debug.pause()
# XXX NOTE, value must also be hashable AND since
# `._registry` is a `bidict` values must be unique;
# use `.forceput()` to replace any prior (stale)
# entries that might map a different uid to the same
# addr (e.g. after an unclean shutdown or
# actor-restart reusing the same address).
self._registry.forceput(uid, tuple(addr))
# pop and signal all waiter events
events = self._waiters.pop(name, [])
self._waiters.setdefault(
name, []
).append(uid)
for event in events:
if isinstance(event, trio.Event):
event.set()
async def unregister_actor(
self,
uid: tuple[str, str]
) -> None:
uid = (str(uid[0]), str(uid[1]))
entry: tuple = self._registry.pop(
uid, None
)
if entry is None:
log.warning(
f'Request to de-register'
f' {uid!r} failed?'
)
async def delete_addr(
self,
addr: tuple[str, int|str]|list[str|int],
) -> tuple[str, str]|None:
# NOTE: `addr` arrives as a `list` over IPC
# (msgpack deserializes tuples -> lists) so
# coerce to `tuple` for the bidict hash lookup.
uid: tuple[str, str]|None = (
self._registry.inverse.pop(
tuple(addr),
None,
)
)
if uid:
report: str = (
'Deleting registry-entry for,\n'
)
else:
report: str = (
'No registry entry for,\n'
)
log.warning(
report
+
f'{addr!r}@{uid!r}'
)
return uid
# Backward compat alias
Arbiter = Registrar

View File

@ -146,7 +146,7 @@ _pubtask2lock: dict[str, trio.StrictFIFOLock] = {}
def pub( def pub(
wrapped: typing.Callable | None = None, wrapped: typing.Callable|None = None,
*, *,
tasks: set[str] = set(), tasks: set[str] = set(),
): ):
@ -244,8 +244,12 @@ def pub(
task2lock[name] = trio.StrictFIFOLock() task2lock[name] = trio.StrictFIFOLock()
@wrapt.decorator @wrapt.decorator
async def wrapper(agen, instance, args, kwargs): async def wrapper(
agen,
instance,
args,
kwargs,
):
# XXX: this is used to extract arguments properly as per the # XXX: this is used to extract arguments properly as per the
# `wrapt` docs # `wrapt` docs
async def _execute( async def _execute(

View File

@ -39,7 +39,7 @@ from ._types import (
transport_from_addr, transport_from_addr,
transport_from_stream, transport_from_stream,
) )
from tractor._addr import ( from tractor.discovery._addr import (
is_wrapped_addr, is_wrapped_addr,
wrap_address, wrap_address,
Address, Address,
@ -60,7 +60,7 @@ if TYPE_CHECKING:
from ._transport import MsgTransport from ._transport import MsgTransport
log = get_logger(__name__) log = get_logger()
_is_windows = platform.system() == 'Windows' _is_windows = platform.system() == 'Windows'
@ -94,7 +94,7 @@ class Channel:
self._transport: MsgTransport|None = transport self._transport: MsgTransport|None = transport
# set after handshake - always info from peer end # set after handshake - always info from peer end
self.aid: Aid|None = None self._aid: Aid|None = None
self._aiter_msgs = self._iter_msgs() self._aiter_msgs = self._iter_msgs()
self._exc: Exception|None = None self._exc: Exception|None = None
@ -122,6 +122,14 @@ class Channel:
''' '''
return self._cancel_called return self._cancel_called
@property
def aid(self) -> Aid:
'''
Peer actor's ID.
'''
return self._aid
@property @property
def uid(self) -> tuple[str, str]: def uid(self) -> tuple[str, str]:
''' '''
@ -307,7 +315,12 @@ class Channel:
) -> None: ) -> None:
''' '''
Send a coded msg-blob over the transport. Send a coded msg-blob over the underlying IPC transport.
This fn raises `TransportClosed` on comms failures and is
normally handled by higher level runtime machinery for the
expected-graceful cases, normally ephemercal
(re/dis)connects.
''' '''
__tracebackhide__: bool = hide_tb __tracebackhide__: bool = hide_tb
@ -334,9 +347,10 @@ class Channel:
except KeyError: except KeyError:
raise err raise err
case TransportClosed(): case TransportClosed():
src_exc_str: str = err.repr_src_exc()
log.transport( log.transport(
f'Transport stream closed due to\n' f'Transport stream closed due to,\n'
f'{err.repr_src_exc()}\n' f'{src_exc_str}'
) )
case _: case _:
@ -345,6 +359,11 @@ class Channel:
raise raise
async def recv(self) -> Any: async def recv(self) -> Any:
'''
Receive the latest (queued) msg-blob from the underlying IPC
transport.
'''
assert self._transport assert self._transport
return await self._transport.recv() return await self._transport.recv()
@ -418,16 +437,18 @@ class Channel:
self self
) -> AsyncGenerator[Any, None]: ) -> AsyncGenerator[Any, None]:
''' '''
Yield `MsgType` IPC msgs decoded and deliverd from Yield `MsgType` IPC msgs decoded and deliverd from an
an underlying `MsgTransport` protocol. underlying `MsgTransport` protocol.
This is a streaming routine alo implemented as an async-gen This is a streaming routine alo implemented as an
func (same a `MsgTransport._iter_pkts()`) gets allocated by async-generator func (same a `MsgTransport._iter_pkts()`)
a `.__call__()` inside `.__init__()` where it is assigned to gets allocated by a `.__call__()` inside `.__init__()` where
the `._aiter_msgs` attr. it is assigned to the `._aiter_msgs` attr.
''' '''
assert self._transport if not self._transport:
raise RuntimeError('No IPC transport initialized!?')
while True: while True:
try: try:
async for msg in self._transport: async for msg in self._transport:
@ -462,7 +483,15 @@ class Channel:
# continue # continue
def connected(self) -> bool: def connected(self) -> bool:
return self._transport.connected() if self._transport else False '''
Predicate whether underlying IPC tpt is connected.
'''
return (
self._transport.connected()
if self._transport
else False
)
async def _do_handshake( async def _do_handshake(
self, self,
@ -484,7 +513,7 @@ class Channel:
f'<= {peer_aid.reprol(sin_uuid=False)}\n' f'<= {peer_aid.reprol(sin_uuid=False)}\n'
) )
# NOTE, we always are referencing the remote peer! # NOTE, we always are referencing the remote peer!
self.aid = peer_aid self._aid = peer_aid
return peer_aid return peer_aid
@ -493,8 +522,11 @@ async def _connect_chan(
addr: UnwrappedAddress addr: UnwrappedAddress
) -> typing.AsyncGenerator[Channel, None]: ) -> typing.AsyncGenerator[Channel, None]:
''' '''
Create and connect a channel with disconnect on context manager Create and connect a `Channel` to the provided `addr`, disconnect
teardown. it on cm exit.
NOTE, this is a lowlevel, normally internal-only iface. You
should likely use `.open_portal()` instead.
''' '''
chan = await Channel.from_addr(addr) chan = await Channel.from_addr(addr)

View File

@ -50,29 +50,27 @@ from ..devx.pformat import (
from .._exceptions import ( from .._exceptions import (
TransportClosed, TransportClosed,
) )
from .. import _rpc from ..runtime import _rpc
from ..msg import ( from ..msg import (
MsgType, MsgType,
Struct, Struct,
types as msgtypes, types as msgtypes,
) )
from ..trionics import maybe_open_nursery from ..trionics import maybe_open_nursery
from .. import ( from ..runtime import _state
_state, from .. import log
log, from ..discovery._addr import Address
)
from .._addr import Address
from ._chan import Channel from ._chan import Channel
from ._transport import MsgTransport from ._transport import MsgTransport
from ._uds import UDSAddress from ._uds import UDSAddress
from ._tcp import TCPAddress from ._tcp import TCPAddress
if TYPE_CHECKING: if TYPE_CHECKING:
from .._runtime import Actor from ..runtime._runtime import Actor
from .._supervise import ActorNursery from ..runtime._supervise import ActorNursery
log = log.get_logger(__name__) log = log.get_logger()
async def maybe_wait_on_canced_subs( async def maybe_wait_on_canced_subs(
@ -357,7 +355,7 @@ async def handle_stream_from_peer(
# and `MsgpackStream._inter_packets()` on a read from the # and `MsgpackStream._inter_packets()` on a read from the
# stream particularly when the runtime is first starting up # stream particularly when the runtime is first starting up
# inside `open_root_actor()` where there is a check for # inside `open_root_actor()` where there is a check for
# a bound listener on the "arbiter" addr. the reset will be # a bound listener on the registrar addr. the reset will be
# because the handshake was never meant took place. # because the handshake was never meant took place.
log.runtime( log.runtime(
con_status con_status
@ -970,7 +968,7 @@ class Server(Struct):
in `accept_addrs`. in `accept_addrs`.
''' '''
from .._addr import ( from ..discovery._addr import (
default_lo_addrs, default_lo_addrs,
wrap_address, wrap_address,
) )

View File

@ -23,6 +23,7 @@ considered optional within the context of this runtime-library.
""" """
from __future__ import annotations from __future__ import annotations
import hashlib
from multiprocessing import shared_memory as shm from multiprocessing import shared_memory as shm
from multiprocessing.shared_memory import ( from multiprocessing.shared_memory import (
# SharedMemory, # SharedMemory,
@ -59,7 +60,7 @@ except ImportError:
pass pass
log = get_logger(__name__) log = get_logger()
SharedMemory = disable_mantracker() SharedMemory = disable_mantracker()
@ -106,11 +107,12 @@ class NDToken(Struct, frozen=True):
This type is msg safe. This type is msg safe.
''' '''
shm_name: str # this servers as a "key" value shm_name: str # actual OS-level name (may be shortened on macOS)
shm_first_index_name: str shm_first_index_name: str
shm_last_index_name: str shm_last_index_name: str
dtype_descr: tuple dtype_descr: tuple
size: int # in struct-array index / row terms size: int # in struct-array index / row terms
key: str|None = None # original descriptive key (for lookup)
# TODO: use nptyping here on dtypes # TODO: use nptyping here on dtypes
@property @property
@ -124,6 +126,41 @@ class NDToken(Struct, frozen=True):
def as_msg(self): def as_msg(self):
return to_builtins(self) return to_builtins(self)
def __eq__(self, other) -> bool:
'''
Compare tokens based on shm names and dtype,
ignoring the `key` field.
The `key` field is only used for lookups,
not for token identity.
'''
if not isinstance(other, NDToken):
return False
return (
self.shm_name == other.shm_name
and self.shm_first_index_name
== other.shm_first_index_name
and self.shm_last_index_name
== other.shm_last_index_name
and self.dtype_descr == other.dtype_descr
and self.size == other.size
)
def __hash__(self) -> int:
'''
Hash based on the same fields used
in `.__eq__()`.
'''
return hash((
self.shm_name,
self.shm_first_index_name,
self.shm_last_index_name,
self.dtype_descr,
self.size,
))
@classmethod @classmethod
def from_msg(cls, msg: dict) -> NDToken: def from_msg(cls, msg: dict) -> NDToken:
if isinstance(msg, NDToken): if isinstance(msg, NDToken):
@ -160,6 +197,50 @@ def get_shm_token(key: str) -> NDToken | None:
return _known_tokens.get(key) return _known_tokens.get(key)
def _shorten_key_for_macos(
key: str,
prefix: str = '',
suffix: str = '',
) -> str:
'''
MacOS has a (hillarious) 31 character limit for POSIX shared
memory names. Hash long keys to fit within this limit while
maintaining uniqueness.
'''
# macOS shm_open() has a 31 char limit (PSHMNAMLEN)
# format: /t_<hash16> = 19 chars, well under limit
max_len: int = 31
if len(key) <= max_len:
return key
_hash: str = hashlib.sha256(
key.encode()
).hexdigest()
hash_len: int = (
(max_len - 1)
- len(prefix)
- len(suffix)
)
key_hash: str = _hash[:hash_len]
short_key = (
prefix
+
f'{key_hash}'
+
suffix
)
log.debug(
f'Shortened shm key for macOS:\n'
f' original: {key!r} ({len(key)!r} chars)\n'
f' shortened: {short_key!r}'
f' ({len(short_key)!r} chars)'
)
return short_key
def _make_token( def _make_token(
key: str, key: str,
size: int, size: int,
@ -171,12 +252,32 @@ def _make_token(
to access a shared array. to access a shared array.
''' '''
# On macOS, shorten keys that exceed the
# 31 character limit
if platform.system() == 'Darwin':
shm_name = _shorten_key_for_macos(
key=key,
)
shm_first = _shorten_key_for_macos(
key=key,
suffix='_first',
)
shm_last = _shorten_key_for_macos(
key=key,
suffix='_last',
)
else:
shm_name = key
shm_first = key + '_first'
shm_last = key + '_last'
return NDToken( return NDToken(
shm_name=key, shm_name=shm_name,
shm_first_index_name=key + "_first", shm_first_index_name=shm_first,
shm_last_index_name=key + "_last", shm_last_index_name=shm_last,
dtype_descr=tuple(np.dtype(dtype).descr), dtype_descr=tuple(np.dtype(dtype).descr),
size=size, size=size,
key=key, # store original key for lookup
) )
@ -431,9 +532,17 @@ class ShmArray:
def destroy(self) -> None: def destroy(self) -> None:
if _USE_POSIX: if _USE_POSIX:
# We manually unlink to bypass all the "resource tracker" # We manually unlink to bypass all the
# nonsense meant for non-SC systems. # "resource tracker" nonsense meant for
shm_unlink(self._shm.name) # non-SC systems.
name = self._shm.name
try:
shm_unlink(name)
except FileNotFoundError:
# might be a teardown race here?
log.warning(
f'Shm for {name} already unlinked?'
)
self._first.destroy() self._first.destroy()
self._last.destroy() self._last.destroy()
@ -463,8 +572,16 @@ def open_shm_ndarray(
a = np.zeros(size, dtype=dtype) a = np.zeros(size, dtype=dtype)
a['index'] = np.arange(len(a)) a['index'] = np.arange(len(a))
# Create token first to get the (possibly
# shortened) shm name
token = _make_token(
key=key,
size=size,
dtype=dtype,
)
shm = SharedMemory( shm = SharedMemory(
name=key, name=token.shm_name,
create=True, create=True,
size=a.nbytes size=a.nbytes
) )
@ -476,12 +593,6 @@ def open_shm_ndarray(
array[:] = a[:] array[:] = a[:]
array.setflags(write=int(not readonly)) array.setflags(write=int(not readonly))
token = _make_token(
key=key,
size=size,
dtype=dtype,
)
# create single entry arrays for storing an first and last indices # create single entry arrays for storing an first and last indices
first = SharedInt( first = SharedInt(
shm=SharedMemory( shm=SharedMemory(
@ -554,13 +665,23 @@ def attach_shm_ndarray(
''' '''
token = NDToken.from_msg(token) token = NDToken.from_msg(token)
key = token.shm_name # Use original key for _known_tokens lookup,
# shm_name for OS calls
lookup_key = (
token.key if token.key
else token.shm_name
)
if key in _known_tokens: if lookup_key in _known_tokens:
assert NDToken.from_msg(_known_tokens[key]) == token, "WTF" assert (
NDToken.from_msg(
_known_tokens[lookup_key]
) == token
), 'WTF'
# XXX: ugh, looks like due to the ``shm_open()`` C api we can't # XXX: ugh, looks like due to the ``shm_open()``
# actually place files in a subdir, see discussion here: # C api we can't actually place files in a subdir,
# see discussion here:
# https://stackoverflow.com/a/11103289 # https://stackoverflow.com/a/11103289
# attach to array buffer and view as per dtype # attach to array buffer and view as per dtype
@ -568,7 +689,7 @@ def attach_shm_ndarray(
for _ in range(3): for _ in range(3):
try: try:
shm = SharedMemory( shm = SharedMemory(
name=key, name=token.shm_name,
create=False, create=False,
) )
break break
@ -614,10 +735,10 @@ def attach_shm_ndarray(
sha.array sha.array
# Stash key -> token knowledge for future queries # Stash key -> token knowledge for future queries
# via `maybe_opepn_shm_array()` but only after we know # via `maybe_open_shm_ndarray()` but only after
# we can attach. # we know we can attach.
if key not in _known_tokens: if lookup_key not in _known_tokens:
_known_tokens[key] = token _known_tokens[lookup_key] = token
# "close" attached shm on actor teardown # "close" attached shm on actor teardown
tractor.current_actor().lifetime_stack.callback(sha.close) tractor.current_actor().lifetime_stack.callback(sha.close)
@ -661,7 +782,10 @@ def maybe_open_shm_ndarray(
False, # not newly opened False, # not newly opened
) )
except KeyError: except KeyError:
log.warning(f"Could not find {key} in shms cache") log.warning(
f'Could not find key in shms cache,\n'
f'key: {key!r}\n'
)
if dtype: if dtype:
token = _make_token( token = _make_token(
key, key,
@ -771,6 +895,7 @@ def open_shm_list(
size: int = int(2 ** 10), size: int = int(2 ** 10),
dtype: float | int | bool | str | bytes | None = float, dtype: float | int | bool | str | bytes | None = float,
readonly: bool = True, readonly: bool = True,
prefix: str = 'shml_',
) -> ShmList: ) -> ShmList:
@ -784,6 +909,12 @@ def open_shm_list(
}[dtype] }[dtype]
sequence = [default] * size sequence = [default] * size
if platform.system() == 'Darwin':
key: str = _shorten_key_for_macos(
key=key,
prefix=prefix,
)
shml = ShmList( shml = ShmList(
sequence=sequence, sequence=sequence,
name=key, name=key,

View File

@ -41,7 +41,7 @@ from tractor.ipc._transport import (
) )
log = get_logger(__name__) log = get_logger()
class TCPAddress( class TCPAddress(

View File

@ -54,9 +54,9 @@ from tractor.msg import (
) )
if TYPE_CHECKING: if TYPE_CHECKING:
from tractor._addr import Address from tractor.discovery._addr import Address
log = get_logger(__name__) log = get_logger()
# (codec, transport) # (codec, transport)
@ -154,7 +154,6 @@ class MsgTransport(Protocol):
# ... # ...
class MsgpackTransport(MsgTransport): class MsgpackTransport(MsgTransport):
# TODO: better naming for this? # TODO: better naming for this?
@ -226,7 +225,7 @@ class MsgpackTransport(MsgTransport):
# not sure entirely why we need this but without it we # not sure entirely why we need this but without it we
# seem to be getting racy failures here on # seem to be getting racy failures here on
# arbiter/registry name subs.. # registrar name subs..
trio.BrokenResourceError, trio.BrokenResourceError,
) as trans_err: ) as trans_err:
@ -278,14 +277,18 @@ class MsgpackTransport(MsgTransport):
except trio.ClosedResourceError as cre: except trio.ClosedResourceError as cre:
closure_err = cre closure_err = cre
# await tractor.devx._trace.maybe_pause_bp()
raise TransportClosed( raise TransportClosed(
message=( message=(
f'{tpt_name} was already closed locally ?\n' f'{tpt_name} was already closed locally?'
), ),
src_exc=closure_err, src_exc=closure_err,
loglevel='error', loglevel='error',
raise_on_report=( raise_on_report=(
'another task closed this fd' in closure_err.args 'another task closed this fd'
in
closure_err.args
), ),
) from closure_err ) from closure_err
@ -435,6 +438,11 @@ class MsgpackTransport(MsgTransport):
trans_err = _re trans_err = _re
tpt_name: str = f'{type(self).__name__!r}' tpt_name: str = f'{type(self).__name__!r}'
trans_err_msg: str = trans_err.args[0]
by_whom: str = {
'another task closed this fd': 'locally',
'this socket was already closed': 'by peer',
}.get(trans_err_msg)
match trans_err: match trans_err:
# XXX, specifc to UDS transport and its, # XXX, specifc to UDS transport and its,
@ -446,38 +454,42 @@ class MsgpackTransport(MsgTransport):
case trio.BrokenResourceError() if ( case trio.BrokenResourceError() if (
'[Errno 32] Broken pipe' '[Errno 32] Broken pipe'
in in
trans_err.args[0] trans_err_msg
): ):
tpt_closed = TransportClosed.from_src_exc( tpt_closed = TransportClosed.from_src_exc(
message=( message=(
f'{tpt_name} already closed by peer\n' f'{tpt_name} already closed by peer\n'
), ),
body=f'{self}\n', body=f'{self}',
src_exc=trans_err, src_exc=trans_err,
raise_on_report=True, raise_on_report=True,
loglevel='transport', loglevel='transport',
) )
raise tpt_closed from trans_err raise tpt_closed from trans_err
# case trio.ClosedResourceError() if ( # ??TODO??, what case in piker does this and HOW
# 'this socket was already closed' # CAN WE RE-PRODUCE IT?!?!?
# in case trio.ClosedResourceError() if (
# trans_err.args[0] by_whom
# ): ):
# tpt_closed = TransportClosed.from_src_exc( tpt_closed = TransportClosed.from_src_exc(
# message=( message=(
# f'{tpt_name} already closed by peer\n' f'{tpt_name} was already closed {by_whom!r}?\n'
# ), ),
# body=f'{self}\n', body=f'{self}',
# src_exc=trans_err, src_exc=trans_err,
# raise_on_report=True, raise_on_report=True,
# loglevel='transport', loglevel='transport',
# ) )
# raise tpt_closed from trans_err
# unless the disconnect condition falls under "a # await tractor.devx._trace.maybe_pause_bp()
# normal operation breakage" we usualy console warn raise tpt_closed from trans_err
# about it.
# XXX, unless the disconnect condition falls
# under "a normal/expected operating breakage"
# (per the `trans_err_msg` guards in the cases
# above) we usualy console-error about it and
# raise-thru. about it.
case _: case _:
log.exception( log.exception(
f'{tpt_name} layer failed pre-send ??\n' f'{tpt_name} layer failed pre-send ??\n'

View File

@ -23,12 +23,12 @@ from contextlib import (
) )
from pathlib import Path from pathlib import Path
import os import os
import sys
from socket import ( from socket import (
AF_UNIX, AF_UNIX,
SOCK_STREAM, SOCK_STREAM,
SO_PASSCRED,
SO_PEERCRED,
SOL_SOCKET, SOL_SOCKET,
error as socket_error,
) )
import struct import struct
from typing import ( from typing import (
@ -53,17 +53,39 @@ from tractor.log import get_logger
from tractor.ipc._transport import ( from tractor.ipc._transport import (
MsgpackTransport, MsgpackTransport,
) )
from .._state import ( from tractor.runtime._state import (
get_rt_dir, get_rt_dir,
current_actor, current_actor,
is_root_process, is_root_process,
) )
if TYPE_CHECKING: if TYPE_CHECKING:
from ._runtime import Actor from tractor.runtime._runtime import Actor
log = get_logger(__name__) # Platform-specific credential passing constants
# See: https://stackoverflow.com/a/7982749
if sys.platform == 'linux':
from socket import (
SO_PASSCRED,
SO_PEERCRED,
)
else:
# Other (Unix) platforms - though further testing is required and
# others may need additional special handling?
SO_PASSCRED = None
SO_PEERCRED = None
# NOTE, macOS uses `LOCAL_PEERCRED` instead of `SO_PEERCRED` and
# doesn't need `SO_PASSCRED` (credential passing is always enabled).
# See code in <sys/un.h>: `#define LOCAL_PEERCRED 0x001`
#
# XXX INSTEAD we use the (hopefully) more generic
# `get_peer_pid()` below for other OSes.
log = get_logger()
def unwrap_sockpath( def unwrap_sockpath(
@ -165,11 +187,21 @@ class UDSAddress(
err_on_no_runtime=False, err_on_no_runtime=False,
) )
if actor: if actor:
sockname: str = '::'.join(actor.uid) + f'@{pid}' sockname: str = f'{actor.aid.name}@{pid}'
# XXX, orig version which broke both macOS (file-name
# length) and `multiaddrs` ('::' invalid separator).
# sockname: str = '::'.join(actor.uid) + f'@{pid}'
#
# ?^TODO, for `multiaddr`'s parser we can't use the `::`
# above^, SO maybe a `.` or something else here?
# sockname: str = '.'.join(actor.uid) + f'@{pid}'
# -[ ] CURRENTLY using `.` BREAKS TEST SUITE tho..
else: else:
prefix: str = '<unknown-actor>'
if is_root_process(): if is_root_process():
prefix: str = 'root' prefix: str = 'no_runtime_root'
else:
prefix: str = 'no_runtime_actor'
sockname: str = f'{prefix}@{pid}' sockname: str = f'{prefix}@{pid}'
sockpath: Path = Path(f'{sockname}.sock') sockpath: Path = Path(f'{sockname}.sock')
@ -288,7 +320,12 @@ def close_listener(
async def open_unix_socket_w_passcred( async def open_unix_socket_w_passcred(
filename: str|bytes|os.PathLike[str]|os.PathLike[bytes], filename: (
str
|bytes
|os.PathLike[str]
|os.PathLike[bytes]
),
) -> trio.SocketStream: ) -> trio.SocketStream:
''' '''
Literally the exact same as `trio.open_unix_socket()` except we set the additiona Literally the exact same as `trio.open_unix_socket()` except we set the additiona
@ -306,21 +343,66 @@ async def open_unix_socket_w_passcred(
# much more simplified logic vs tcp sockets - one socket type and only one # much more simplified logic vs tcp sockets - one socket type and only one
# possible location to connect to # possible location to connect to
sock = trio.socket.socket(AF_UNIX, SOCK_STREAM) sock = trio.socket.socket(AF_UNIX, SOCK_STREAM)
# Only set SO_PASSCRED on Linux (not needed/available on macOS)
if SO_PASSCRED is not None:
sock.setsockopt(SOL_SOCKET, SO_PASSCRED, 1) sock.setsockopt(SOL_SOCKET, SO_PASSCRED, 1)
with close_on_error(sock): with close_on_error(sock):
await sock.connect(os.fspath(filename)) await sock.connect(os.fspath(filename))
return trio.SocketStream(sock) return trio.SocketStream(sock)
def get_peer_info(sock: trio.socket.socket) -> tuple[ def get_peer_pid(sock) -> int|None:
'''
Gets the PID of the process connected to the other end of a Unix
domain socket on macOS, or `None` if that fails.
NOTE, should work on MacOS (and others?).
'''
# try to get the peer PID using a naive soln found from,
# https://stackoverflow.com/a/67971484
#
# NOTE, a more correct soln is likely needed here according to
# the complaints of `copilot` which led to digging into the
# underlying `go`lang issue linked from the above SO answer,
# XXX, darwin-xnu kernel srces defining these constants,
# - SOL_LOCAL
# |_https://github.com/apple/darwin-xnu/blob/main/bsd/sys/un.h#L85
# - LOCAL_PEERPID
# |_https://github.com/apple/darwin-xnu/blob/main/bsd/sys/un.h#L89
#
SOL_LOCAL: int = 0
LOCAL_PEERPID: int = 0x002
try:
pid: int = sock.getsockopt(
SOL_LOCAL,
LOCAL_PEERPID,
)
return pid
except socket_error as e:
log.exception(
f"Failed to get peer PID: {e}"
)
return None
def get_peer_info(
sock: trio.socket.socket,
) -> tuple[
int, # pid int, # pid
int, # uid int, # uid
int, # guid int, # guid
]: ]:
''' '''
Deliver the connecting peer's "credentials"-info as defined in Deliver the connecting peer's "credentials"-info as defined in
a very Linux specific way.. a platform-specific way.
Linux-ONLY, uses SO_PEERCRED.
For more deats see, For more deats see,
- `man accept`, - `man accept`,
@ -333,6 +415,11 @@ def get_peer_info(sock: trio.socket.socket) -> tuple[
- https://stackoverflow.com/a/7982749 - https://stackoverflow.com/a/7982749
''' '''
if SO_PEERCRED is None:
raise RuntimeError(
f'Peer credential retrieval not supported on {sys.platform}!'
)
creds: bytes = sock.getsockopt( creds: bytes = sock.getsockopt(
SOL_SOCKET, SOL_SOCKET,
SO_PEERCRED, SO_PEERCRED,
@ -436,14 +523,38 @@ class MsgpackUDSStream(MsgpackTransport):
match (peername, sockname): match (peername, sockname):
case (str(), bytes()): case (str(), bytes()):
sock_path: Path = Path(peername) sock_path: Path = Path(peername)
case (bytes(), str()): case (bytes(), str()):
sock_path: Path = Path(sockname) sock_path: Path = Path(sockname)
case (str(), str()): # XXX, likely macOS
sock_path: Path = Path(peername)
case _:
raise TypeError(
f'Failed to match (peername, sockname) types?\n'
f'peername: {peername!r}\n'
f'sockname: {sockname!r}\n'
)
if sys.platform == 'linux':
( (
peer_pid, peer_pid,
_, _,
_, _,
) = get_peer_info(sock) ) = get_peer_info(sock)
# NOTE known to at least works on,
# - macos
else:
peer_pid: int|None = get_peer_pid(sock)
if peer_pid is None:
log.warning(
f'Unable to get peer PID?\n'
f'sock: {sock!r}\n'
f'peer_pid: {peer_pid!r}\n'
)
filedir, filename = unwrap_sockpath(sock_path) filedir, filename = unwrap_sockpath(sock_path)
laddr = UDSAddress( laddr = UDSAddress(
filedir=filedir, filedir=filedir,

View File

@ -14,11 +14,23 @@
# You should have received a copy of the GNU Affero General Public License # You should have received a copy of the GNU Affero General Public License
# along with this program. If not, see <https://www.gnu.org/licenses/>. # along with this program. If not, see <https://www.gnu.org/licenses/>.
""" '''
Log like a forester! An enhanced logging subsys.
""" An extended logging layer using (for now) the stdlib's `logging`
+ `colorlog` which embeds concurrency-primitive/runtime info into
records (headers) to help you better grok your distributed systems
built on `tractor`.
'''
from collections.abc import Mapping from collections.abc import Mapping
from functools import partial
from inspect import (
FrameInfo,
getmodule,
stack,
)
import sys import sys
import logging import logging
from logging import ( from logging import (
@ -26,20 +38,24 @@ from logging import (
Logger, Logger,
StreamHandler, StreamHandler,
) )
import colorlog # type: ignore from types import ModuleType
import warnings
import colorlog # type: ignore
# ?TODO, some other (modern) alt libs?
# import coloredlogs
# import colored_traceback.auto # ?TODO, need better config?
import trio import trio
from ._state import current_actor from .runtime._state import current_actor
_proj_name: str = 'tractor'
_default_loglevel: str = 'ERROR' _default_loglevel: str = 'ERROR'
# Super sexy formatting thanks to ``colorlog``. # Super sexy formatting thanks to ``colorlog``.
# (NOTE: we use the '{' format style) # (NOTE: we use the '{' format style)
# Here, `thin_white` is just the layperson's gray. # Here, `thin_white` is just the layperson's gray.
LOG_FORMAT = ( LOG_FORMAT: str = (
# "{bold_white}{log_color}{asctime}{reset}" # "{bold_white}{log_color}{asctime}{reset}"
"{log_color}{asctime}{reset}" "{log_color}{asctime}{reset}"
" {bold_white}{thin_white}({reset}" " {bold_white}{thin_white}({reset}"
@ -51,7 +67,7 @@ LOG_FORMAT = (
" {reset}{bold_white}{thin_white}{message}" " {reset}{bold_white}{thin_white}{message}"
) )
DATE_FORMAT = '%b %d %H:%M:%S' DATE_FORMAT: str = '%b %d %H:%M:%S'
# FYI, ERROR is 40 # FYI, ERROR is 40
# TODO: use a `bidict` to avoid the :155 check? # TODO: use a `bidict` to avoid the :155 check?
@ -75,7 +91,10 @@ STD_PALETTE = {
'TRANSPORT': 'cyan', 'TRANSPORT': 'cyan',
} }
BOLD_PALETTE = { BOLD_PALETTE: dict[
str,
dict[int, str],
] = {
'bold': { 'bold': {
level: f"bold_{color}" for level, color in STD_PALETTE.items()} level: f"bold_{color}" for level, color in STD_PALETTE.items()}
} }
@ -97,9 +116,26 @@ def at_least_level(
return False return False
# TODO: this isn't showing the correct '{filename}' # TODO, compare with using a "filter" instead?
# as it did before.. # - https://stackoverflow.com/questions/60691759/add-information-to-every-log-message-in-python-logging/61830838#61830838
# |_corresponding dict-config,
# https://stackoverflow.com/questions/7507825/where-is-a-complete-example-of-logging-config-dictconfig/7507842#7507842
# - [ ] what's the benefit/tradeoffs?
#
class StackLevelAdapter(LoggerAdapter): class StackLevelAdapter(LoggerAdapter):
'''
A (software) stack oriented logger "adapter".
'''
@property
def level(self) -> str:
'''
The currently set `str` emit level (in lowercase).
'''
return logging.getLevelName(
self.getEffectiveLevel()
).lower()
def at_least_level( def at_least_level(
self, self,
@ -248,11 +284,16 @@ def pformat_task_uid(
return f'{task.name}[{tid_part}]' return f'{task.name}[{tid_part}]'
_curr_actor_no_exc = partial(
current_actor,
err_on_no_runtime=False,
)
_conc_name_getters = { _conc_name_getters = {
'task': pformat_task_uid, 'task': pformat_task_uid,
'actor': lambda: current_actor(), 'actor': lambda: _curr_actor_no_exc(),
'actor_name': lambda: current_actor().name, 'actor_name': lambda: current_actor().name,
'actor_uid': lambda: current_actor().uid[1][:6], 'actor_uid': lambda: current_actor().aid.uuid[:6],
} }
@ -282,9 +323,16 @@ class ActorContextInfo(Mapping):
return f'no {key} context' return f'no {key} context'
_proj_name: str = 'tractor'
def get_logger( def get_logger(
name: str|None = None, name: str|None = None,
_root_name: str = _proj_name, # ^NOTE, setting `name=_proj_name=='tractor'` enables the "root
# logger" for `tractor` itself.
pkg_name: str = _proj_name,
# XXX, deprecated, use ^
_root_name: str|None = None,
logger: Logger|None = None, logger: Logger|None = None,
@ -293,23 +341,129 @@ def get_logger(
# |_https://stackoverflow.com/questions/7507825/where-is-a-complete-example-of-logging-config-dictconfig # |_https://stackoverflow.com/questions/7507825/where-is-a-complete-example-of-logging-config-dictconfig
# |_https://docs.python.org/3/library/logging.config.html#configuration-dictionary-schema # |_https://docs.python.org/3/library/logging.config.html#configuration-dictionary-schema
subsys_spec: str|None = None, subsys_spec: str|None = None,
mk_sublog: bool = True,
_strict_debug: bool = False,
) -> StackLevelAdapter: ) -> StackLevelAdapter:
''' '''
Return the `tractor`-library root logger or a sub-logger for Return the `tractor`-library root logger or a sub-logger for
`name` if provided. `name` if provided.
When `name` is left null we try to auto-detect the caller's
`mod.__name__` and use that as a the sub-logger key.
This allows for example creating a module level instance like,
.. code:: python
log = tractor.log.get_logger(_root_name='mylib')
and by default all console record headers will show the caller's
(of any `log.<level>()`-method) correct sub-pkg's
+ py-module-file.
''' '''
log: Logger if _root_name:
log = rlog = logger or logging.getLogger(_root_name) msg: str = (
'The `_root_name: str` param of `get_logger()` is now deprecated.\n'
'Use the new `pkg_name: str` instead, it is the same usage.\n'
)
warnings.warn(
msg,
DeprecationWarning,
stacklevel=2,
)
if ( pkg_name: str = _root_name
name
and def get_caller_mod(
name != _proj_name frames_up:int = 2
): ):
'''
Attempt to get the module which called `tractor.get_logger()`.
# NOTE: for handling for modules that use `get_logger(__name__)` '''
callstack: list[FrameInfo] = stack()
caller_fi: FrameInfo = callstack[frames_up]
caller_mod: ModuleType = getmodule(caller_fi.frame)
return caller_mod
# --- Auto--naming-CASE ---
# -------------------------
# Implicitly introspect the caller's module-name whenever `name`
# if left as the null default.
#
# When the `pkg_name` is `in` in the `mod.__name__` we presume
# this instance can be created as a sub-`StackLevelAdapter` and
# that the intention is to get free module-path tracing and
# filtering (well once we implement that) oriented around the
# py-module code hierarchy of the consuming project.
#
if (
mk_sublog
and
name is None
and
pkg_name
):
if (caller_mod := get_caller_mod()):
# ?XXX how is this `caller_mod.__name__` defined?
# => well by how the mod is imported.. XD
# |_https://stackoverflow.com/a/15883682
#
# if pkg_name in caller_mod.__package__:
# from tractor.devx.debug import mk_pdb
# mk_pdb().set_trace()
mod_ns_path: str = caller_mod.__name__
mod_pkg_ns_path: str = caller_mod.__package__
if (
mod_pkg_ns_path in mod_ns_path
or
pkg_name in mod_ns_path
):
# proper_mod_name = mod_ns_path.lstrip(
proper_mod_name = mod_pkg_ns_path.removeprefix(
f'{pkg_name}.'
)
name = proper_mod_name
elif (
pkg_name
# and
# pkg_name in mod_ns_path
):
name = mod_ns_path
if _strict_debug:
msg: str = (
f'@ {get_caller_mod()}\n'
f'Generating sub-logger name,\n'
f'{pkg_name}.{name}\n'
)
if _curr_actor_no_exc():
_root_log.debug(msg)
elif pkg_name != _proj_name:
print(
f'=> tractor.log.get_logger():\n'
f'{msg}\n'
)
# build a root logger instance
log: Logger
rlog = log = (
logger
or
logging.getLogger(pkg_name)
)
# XXX, lowlevel debuggin..
# if pkg_name != _proj_name:
# from tractor.devx.debug import mk_pdb
# mk_pdb().set_trace()
# NOTE: for handling for modules that use the unecessary,
# `get_logger(__name__)`
#
# we make the following stylistic choice: # we make the following stylistic choice:
# - always avoid duplicate project-package token # - always avoid duplicate project-package token
# in msg output: i.e. tractor.tractor.ipc._chan.py in header # in msg output: i.e. tractor.tractor.ipc._chan.py in header
@ -317,25 +471,157 @@ def get_logger(
# - never show the leaf module name in the {name} part # - never show the leaf module name in the {name} part
# since in python the {filename} is always this same # since in python the {filename} is always this same
# module-file. # module-file.
if (
sub_name: None|str = None name
rname, _, sub_name = name.partition('.') and
pkgpath, _, modfilename = sub_name.rpartition('.') # ?TODO? more correct?
# _proj_name not in name
# NOTE: for tractor itself never include the last level name != pkg_name
# module key in the name such that something like: eg. ):
# 'tractor.trionics._broadcast` only includes the first # ex. modden.runtime.progman
# 2 tokens in the (coloured) name part. # -> rname='modden', _, pkg_path='runtime.progman'
if rname == 'tractor': if (
sub_name = pkgpath pkg_name
and
if _root_name in sub_name: pkg_name in name
duplicate, _, sub_name = sub_name.partition('.') ):
proper_name: str = name.removeprefix(
if not sub_name: f'{pkg_name}.'
log = rlog )
msg: str = (
f'@ {get_caller_mod()}\n'
f'Duplicate pkg-name in sub-logger `name`-key?\n'
f'pkg_name = {pkg_name!r}\n'
f'name = {name!r}\n'
f'\n'
f'=> You should change your input params to,\n'
f'get_logger(\n'
f' pkg_name={pkg_name!r}\n'
f' name={proper_name!r}\n'
f')'
)
# assert _duplicate == rname
if _curr_actor_no_exc():
_root_log.warning(msg)
else: else:
log = rlog.getChild(sub_name) print(
f'=> tractor.log.get_logger() ERROR:\n'
f'{msg}\n'
)
name = proper_name
rname: str = pkg_name
pkg_path: str = name
# (
# rname,
# _,
# pkg_path,
# ) = name.partition('.')
# For ex. 'modden.runtime.progman'
# -> pkgpath='runtime', _, leaf_mod='progman'
(
subpkg_path,
_,
leaf_mod,
) = pkg_path.rpartition('.')
# NOTE: special usage for passing `name=__name__`,
#
# - remove duplication of any root-pkg-name in the
# (sub/child-)logger name; i.e. never include the
# `pkg_name` *twice* in the top-most-pkg-name/level
#
# -> this happens normally since it is added to `.getChild()`
# and as the name of its root-logger.
#
# => So for ex. (module key in the name) something like
# `name='tractor.trionics._broadcast` is passed,
# only includes the first 2 sub-pkg name-tokens in the
# child-logger's name; the colored "pkg-namespace" header
# will then correctly show the same value as `name`.
if (
# XXX, TRY to remove duplication cases
# which get warn-logged on below!
(
# when, subpkg_path == pkg_path
subpkg_path
and
rname == pkg_name
)
# ) or (
# # when, pkg_path == leaf_mod
# pkg_path
# and
# leaf_mod == pkg_path
# )
):
pkg_path = subpkg_path
# XXX, do some double-checks for duplication of,
# - root-pkg-name, already in root logger
# - leaf-module-name already in `{filename}` header-field
if (
_strict_debug
and
pkg_name
and
pkg_name in pkg_path
):
_duplicate, _, pkg_path = pkg_path.partition('.')
if _duplicate:
msg: str = (
f'@ {get_caller_mod()}\n'
f'Duplicate pkg-name in sub-logger key?\n'
f'pkg_name = {pkg_name!r}\n'
f'pkg_path = {pkg_path!r}\n'
)
# assert _duplicate == rname
if _curr_actor_no_exc():
_root_log.warning(msg)
else:
print(
f'=> tractor.log.get_logger() ERROR:\n'
f'{msg}\n'
)
# XXX, should never get here?
breakpoint()
if (
_strict_debug
and
leaf_mod
and
leaf_mod in pkg_path
):
msg: str = (
f'@ {get_caller_mod()}\n'
f'Duplicate leaf-module-name in sub-logger key?\n'
f'leaf_mod = {leaf_mod!r}\n'
f'pkg_path = {pkg_path!r}\n'
)
if _curr_actor_no_exc():
_root_log.warning(msg)
else:
print(
f'=> tractor.log.get_logger() ERROR:\n'
f'{msg}\n'
)
# mk/get underlying (sub-)`Logger`
if (
not pkg_path
and
leaf_mod == pkg_name
):
# breakpoint()
log = rlog
elif mk_sublog:
# breakpoint()
log = rlog.getChild(pkg_path)
log.level = rlog.level log.level = rlog.level
@ -350,8 +636,13 @@ def get_logger(
for name, val in CUSTOM_LEVELS.items(): for name, val in CUSTOM_LEVELS.items():
logging.addLevelName(val, name) logging.addLevelName(val, name)
# ensure customs levels exist as methods # ensure our custom adapter levels exist as methods
assert getattr(logger, name.lower()), f'Logger does not define {name}' assert getattr(
logger,
name.lower()
), (
f'Logger does not define {name}'
)
return logger return logger
@ -425,4 +716,4 @@ def get_loglevel() -> str:
# global module logger for tractor itself # global module logger for tractor itself
log: StackLevelAdapter = get_logger('tractor') _root_log: StackLevelAdapter = get_logger('tractor')

View File

@ -39,13 +39,11 @@ from contextvars import (
) )
import textwrap import textwrap
from typing import ( from typing import (
Any,
Callable,
Protocol,
Type,
TYPE_CHECKING, TYPE_CHECKING,
TypeVar, Any,
Type,
Union, Union,
Callable,
) )
from types import ModuleType from types import ModuleType
@ -54,6 +52,13 @@ from msgspec import (
msgpack, msgpack,
Raw, Raw,
) )
from msgspec.inspect import (
CustomType,
UnionType,
SetType,
ListType,
TupleType
)
# TODO: see notes below from @mikenerone.. # TODO: see notes below from @mikenerone..
# from tricycle import TreeVar # from tricycle import TreeVar
@ -68,7 +73,7 @@ from tractor.log import get_logger
if TYPE_CHECKING: if TYPE_CHECKING:
from tractor._context import Context from tractor._context import Context
log = get_logger(__name__) log = get_logger()
# TODO: unify with `MsgCodec` by making `._dec` part this? # TODO: unify with `MsgCodec` by making `._dec` part this?
@ -81,7 +86,7 @@ class MsgDec(Struct):
''' '''
_dec: msgpack.Decoder _dec: msgpack.Decoder
# _ext_types_box: Struct|None = None _ext_types_boxes: dict[Type, Struct] = {}
@property @property
def dec(self) -> msgpack.Decoder: def dec(self) -> msgpack.Decoder:
@ -226,6 +231,8 @@ def mk_dec(
f'ext_types = {ext_types!r}\n' f'ext_types = {ext_types!r}\n'
) )
_boxed_structs: dict[Type, Struct] = {}
if dec_hook: if dec_hook:
if ext_types is None: if ext_types is None:
raise TypeError( raise TypeError(
@ -237,17 +244,15 @@ def mk_dec(
f'ext_types = {ext_types!r}\n' f'ext_types = {ext_types!r}\n'
) )
# XXX, i *thought* we would require a boxing struct as per docs, if len(ext_types) > 1:
# https://jcristharif.com/msgspec/extending.html#mapping-to-from-native-types _boxed_structs = mk_boxed_ext_structs(ext_types)
# |_ see comment, ext_types = [
# > Note that typed deserialization is required for etype
# > successful roundtripping here, so we pass `MyMessage` to for etype in ext_types
# > `Decoder`. if etype not in _boxed_structs
# ]
# BUT, turns out as long as you spec a union with `Raw` it ext_types += list(_boxed_structs.values())
# will work? kk B)
#
# maybe_box_struct = mk_boxed_ext_struct(ext_types)
spec = Raw | Union[*ext_types] spec = Raw | Union[*ext_types]
return MsgDec( return MsgDec(
@ -255,29 +260,26 @@ def mk_dec(
type=spec, # like `MsgType[Any]` type=spec, # like `MsgType[Any]`
dec_hook=dec_hook, dec_hook=dec_hook,
), ),
_ext_types_boxes=_boxed_structs
) )
# TODO? remove since didn't end up needing this? def mk_boxed_ext_structs(
def mk_boxed_ext_struct(
ext_types: list[Type], ext_types: list[Type],
) -> Struct: ) -> dict[Type, Struct]:
# NOTE, originally was to wrap non-msgpack-supported "extension box_types: dict[Type, Struct] = {}
# types" in a field-typed boxing struct, see notes around the for ext_type in ext_types:
# `dec_hook()` branch in `mk_dec()`. info = msgspec.inspect.type_info(ext_type)
ext_types_union = Union[*ext_types] if isinstance(info, CustomType):
repr_ext_types_union: str = ( box_types[ext_type] = msgspec.defstruct(
str(ext_types_union) f'Box{ext_type.__name__}',
or tag=True,
"|".join(ext_types)
)
BoxedExtType = msgspec.defstruct(
f'BoxedExts[{repr_ext_types_union}]',
fields=[ fields=[
('boxed', ext_types_union), ('inner', ext_type),
], ],
) )
return BoxedExtType
return box_types
def unpack_spec_types( def unpack_spec_types(
@ -378,7 +380,7 @@ class MsgCodec(Struct):
_dec: msgpack.Decoder _dec: msgpack.Decoder
_pld_spec: Type[Struct]|Raw|Any _pld_spec: Type[Struct]|Raw|Any
# _ext_types_box: Struct|None = None _ext_types_boxes: dict[Type, Struct] = {}
def __repr__(self) -> str: def __repr__(self) -> str:
speclines: str = textwrap.indent( speclines: str = textwrap.indent(
@ -465,45 +467,29 @@ class MsgCodec(Struct):
''' '''
__tracebackhide__: bool = hide_tb __tracebackhide__: bool = hide_tb
try:
box: Struct|None = self._ext_types_boxes.get(type(py_obj), None)
if (
as_ext_type
or
box
):
py_obj = box(inner=py_obj)
if use_buf: if use_buf:
self._enc.encode_into(py_obj, self._buf) self._enc.encode_into(py_obj, self._buf)
return self._buf return self._buf
return self._enc.encode(py_obj) return self._enc.encode(py_obj)
# try:
# return self._enc.encode(py_obj)
# except TypeError as typerr:
# typerr.add_note(
# '|_src error from `msgspec`'
# # f'|_{self._enc.encode!r}'
# )
# raise typerr
# TODO! REMOVE once i'm confident we won't ever need it! except TypeError as typerr:
# typerr.add_note(
# box: Struct = self._ext_types_box '|_src error from `msgspec`'
# if ( # f'|_{self._enc.encode!r}'
# as_ext_type )
# or raise typerr
# (
# # XXX NOTE, auto-detect if the input type
# box
# and
# (ext_types := unpack_spec_types(
# spec=box.__annotations__['boxed'])
# )
# )
# ):
# match py_obj:
# # case PayloadMsg(pld=pld) if (
# # type(pld) in ext_types
# # ):
# # py_obj.pld = box(boxed=py_obj)
# # breakpoint()
# case _ if (
# type(py_obj) in ext_types
# ):
# py_obj = box(boxed=py_obj)
@property @property
def dec(self) -> msgpack.Decoder: def dec(self) -> msgpack.Decoder:
@ -565,11 +551,6 @@ def mk_codec(
enc_hook: Callable|None = None, enc_hook: Callable|None = None,
ext_types: list[Type]|None = None, ext_types: list[Type]|None = None,
# optionally provided msg-decoder from which we pull its,
# |_.dec_hook()
# |_.type
ext_dec: MsgDec|None = None
#
# ?TODO? other params we might want to support # ?TODO? other params we might want to support
# Encoder: # Encoder:
# write_buffer_size=write_buffer_size, # write_buffer_size=write_buffer_size,
@ -597,12 +578,6 @@ def mk_codec(
) )
dec_hook: Callable|None = None dec_hook: Callable|None = None
if ext_dec:
dec: msgspec.Decoder = ext_dec.dec
dec_hook = dec.dec_hook
pld_spec |= dec.type
if ext_types:
pld_spec |= Union[*ext_types]
# (manually) generate a msg-spec (how appropes) for all relevant # (manually) generate a msg-spec (how appropes) for all relevant
# payload-boxing-struct-msg-types, parameterizing the # payload-boxing-struct-msg-types, parameterizing the
@ -630,10 +605,16 @@ def mk_codec(
enc = msgpack.Encoder( enc = msgpack.Encoder(
enc_hook=enc_hook, enc_hook=enc_hook,
) )
boxes = {}
if ext_types and len(ext_types) > 1:
boxes = mk_boxed_ext_structs(ext_types)
codec = MsgCodec( codec = MsgCodec(
_enc=enc, _enc=enc,
_dec=dec, _dec=dec,
_pld_spec=pld_spec, _pld_spec=pld_spec,
_ext_types_boxes=boxes
) )
# sanity on expected backend support # sanity on expected backend support
assert codec.lib.__name__ == libname assert codec.lib.__name__ == libname
@ -809,78 +790,298 @@ def limit_msg_spec(
assert curr_codec is current_codec() assert curr_codec is current_codec()
# XXX: msgspec won't allow this with non-struct custom types '''
# like `NamespacePath`!@! Encoder / Decoder generic hook factory
# @cm
# def extend_msg_spec(
# payload_spec: Union[Type[Struct]],
# ) -> MsgCodec: '''
# '''
# Extend the current `MsgCodec.pld_spec` (type set) by extending
# the payload spec to **include** the types specified by
# `payload_spec`.
# '''
# codec: MsgCodec = current_codec()
# pld_spec: Union[Type] = codec.pld_spec
# extended_spec: Union[Type] = pld_spec|payload_spec
# with limit_msg_spec(payload_types=extended_spec) as ext_codec:
# # import pdbp; pdbp.set_trace()
# assert ext_codec.pld_spec == extended_spec
# yield ext_codec
#
# ^-TODO-^ is it impossible to make something like this orr!?
# TODO: make an auto-custom hook generator from a set of input custom
# types?
# -[ ] below is a proto design using a `TypeCodec` idea?
#
# type var for the expected interchange-lib's
# IPC-transport type when not available as a built-in
# serialization output.
WireT = TypeVar('WireT')
# TODO: some kinda (decorator) API for built-in subtypes # builtins we can have in same pld_spec as custom types
# that builds this implicitly by inspecting the `mro()`? default_builtins = (
class TypeCodec(Protocol): None,
bool,
int,
float,
bytes,
list
)
# spec definition type
TypeSpec = (
Type |
Union[Type] |
list[Type] |
tuple[Type] |
set[Type]
)
class TypeCodec:
''' '''
A per-custom-type wire-transport serialization translator This class describes a way of encoding to or decoding from a "wire type",
description type. objects that have `encode_fn` and `decode_fn` can be used with
`.encode/.decode`.
''' '''
src_type: Type
wire_type: WireT
def encode(obj: Type) -> WireT: def __init__(
... self,
wire_type: Type,
decode_fn: str,
encode_fn: str = 'encode',
):
self._encode_fn: str = encode_fn
self._decode_fn: str = decode_fn
self._wire_type: Type = wire_type
def decode( def __repr__(self) -> str:
obj_type: Type[WireT], return (
obj: WireT, f'{type(self).__name__}('
) -> Type: f'{self._encode_fn}, '
... f'{self._decode_fn}) '
f'-> {self._wire_type}'
)
@property
def encode_fn(self) -> str:
return self._encode_fn
@property
def decode_fn(self) -> str:
return self._decode_fn
@property
def wire_type(self) -> str:
return self._wire_type
def is_type_compat(self, obj: any) -> bool:
return (
hasattr(obj, self._encode_fn)
and
hasattr(obj, self._decode_fn)
)
def encode(self, obj: any) -> any:
return getattr(obj, self._encode_fn)()
def decode(self, cls: Type, raw: any) -> any:
return getattr(cls, self._decode_fn)(raw)
class MsgpackTypeCodec(TypeCodec): '''
... Default codec descriptions for wire types:
- bytes
- str
- int
'''
def mk_codec_hooks( BytesCodec = TypeCodec(
type_codecs: list[TypeCodec], decode_fn='from_bytes',
wire_type=bytes
)
) -> tuple[Callable, Callable]:
StrCodec = TypeCodec(
decode_fn='from_str',
wire_type=str
)
IntCodec = TypeCodec(
decode_fn='from_int',
wire_type=int
)
default_codecs: dict[Type, TypeCodec] = {
bytes: BytesCodec,
str: StrCodec,
int: IntCodec
}
def mk_spec_set(
spec: TypeSpec
) -> set[Type]:
''' '''
Deliver a `enc_hook()`/`dec_hook()` pair which handle Given any of the different spec definitions, always return a `set[Type]`
manual convertion from an input `Type` set such that whenever with each spec type as an item.
the `TypeCodec.filter()` predicate matches the
`TypeCodec.decode()` is called on the input native object by - When passed list|tuple|set do nothing
the `dec_hook()` and whenever the - When passed a single type we wrap it in tuple
`isiinstance(obj, TypeCodec.type)` matches against an - When passed a Union we wrap its inner types in tuple
`enc_hook(obj=obj)` the return value is taken from a
`TypeCodec.encode(obj)` callback.
''' '''
... if not (
isinstance(spec, set)
or
isinstance(spec, list)
or
isinstance(spec, tuple)
):
spec_info = msgspec.inspect.type_info(spec)
match spec_info:
case UnionType():
return set((
t.cls
for t in spec_info.types
))
case _:
return set((spec, ))
return set(spec)
def mk_codec_map_from_spec(
spec: TypeSpec,
codecs: dict[Type, TypeCodec] = default_codecs
) -> dict[Type, TypeCodec]:
'''
Generate a map of spec type -> supported codec
'''
spec: set[Type] = mk_spec_set(spec)
spec_codecs: dict[Type, TypeCodec] = {}
for t in spec:
if t in spec_codecs:
continue
for codec_type in (int, bytes, str):
codec = codecs[codec_type]
if codec.is_type_compat(t):
spec_codecs[t] = codec
break
return spec_codecs
def mk_enc_hook(
spec: TypeSpec,
with_builtins: bool = True,
builtins: set[Type] = default_builtins,
codecs: dict[Type, TypeCodec] = default_codecs
) -> Callable:
'''
Given a type specification return a msgspec enc_hook fn
'''
spec_codecs = mk_codec_map_from_spec(spec)
def enc_hook(obj: any) -> any:
try:
t = type(obj)
maybe_codec = spec_codecs.get(t, None)
if maybe_codec:
return maybe_codec.encode(obj)
# passthrough builtins
if builtins and t in builtins:
return obj
raise NotImplementedError(
f"Objects of type {type(obj)} are not supported:\n{obj}"
)
except* Exception as e:
e.add_note(f'enc_hook: {t}, {type(obj)} {obj}')
raise
return enc_hook
def mk_dec_hook(
spec: TypeSpec,
with_builtins: bool = True,
builtins: set[Type] = default_builtins,
codecs: dict[Type, TypeCodec] = default_codecs
) -> Callable:
'''
Given a type specification return a msgspec dec_hook fn
'''
spec_codecs = mk_codec_map_from_spec(spec)
def dec_hook(t: Type, obj: any) -> any:
try:
if t is type(obj):
return obj
maybe_codec = spec_codecs.get(t, None)
if maybe_codec:
return maybe_codec.decode(t, obj)
# passthrough builtins
if builtins and type(obj) in builtins:
return obj
raise NotImplementedError(
f"Objects of type {type} are not supported from {obj}"
)
except* Exception as e:
e.add_note(f'dec_hook: {t}, {type(obj)} {obj}')
raise
return dec_hook
def mk_codec_hooks(*args, **kwargs) -> tuple[Callable, Callable]:
'''
Given a type specification return a msgspec enc & dec hook fn pair
'''
return (
mk_enc_hook(*args, **kwargs),
mk_dec_hook(*args, **kwargs)
)
def mk_codec_from_spec(
spec: TypeSpec,
with_builtins: bool = True,
builtins: set[Type] = default_builtins,
codecs: dict[Type, TypeCodec] = default_codecs
) -> MsgCodec:
'''
Given a type specification return a MsgCodec
'''
spec: set[Type] = mk_spec_set(spec)
return mk_codec(
enc_hook=mk_enc_hook(
spec,
with_builtins=with_builtins,
builtins=builtins,
codecs=codecs
),
ext_types=spec
)
def mk_msgpack_codec(
spec: TypeSpec,
with_builtins: bool = True,
builtins: set[Type] = default_builtins,
codecs: dict[Type, TypeCodec] = default_codecs
) -> tuple[msgpack.Encoder, msgpack.Decoder]:
'''
Get a msgpack Encoder, Decoder pair for a given type spec
'''
enc_hook, dec_hook = mk_codec_hooks(
spec,
with_builtins=with_builtins,
builtins=builtins,
codecs=codecs
)
encoder = msgpack.Encoder(enc_hook=enc_hook)
decoder = msgpack.Decoder(spec, dec_hook=dec_hook)
return encoder, decoder

View File

@ -50,7 +50,7 @@ from tractor._exceptions import (
_mk_recv_mte, _mk_recv_mte,
pack_error, pack_error,
) )
from tractor._state import ( from tractor.runtime._state import (
current_ipc_ctx, current_ipc_ctx,
) )
from ._codec import ( from ._codec import (
@ -77,7 +77,7 @@ if TYPE_CHECKING:
from tractor._streaming import MsgStream from tractor._streaming import MsgStream
log = get_logger(__name__) log = get_logger()
_def_any_pldec: MsgDec[Any] = mk_dec(spec=Any) _def_any_pldec: MsgDec[Any] = mk_dec(spec=Any)

View File

@ -126,13 +126,17 @@ def iter_struct_ppfmt_lines(
str(ft) str(ft)
).replace(' ', '') ).replace(' ', '')
if k[0] == '_':
continue
# recurse to get sub-struct's `.pformat()` output Bo # recurse to get sub-struct's `.pformat()` output Bo
if isinstance(v, Struct): elif isinstance(v, Struct):
yield from iter_struct_ppfmt_lines( yield from iter_struct_ppfmt_lines(
struct=v, struct=v,
field_indent=field_indent+field_indent, field_indent=field_indent+field_indent,
) )
else:
else: # top-level field
val_str: str = repr(v) val_str: str = repr(v)
# XXX LOL, below just seems to be f#$%in causing # XXX LOL, below just seems to be f#$%in causing

View File

@ -51,7 +51,7 @@ from tractor.log import get_logger
# from tractor._addr import UnwrappedAddress # from tractor._addr import UnwrappedAddress
log = get_logger('tractor.msgspec') log = get_logger()
# type variable for the boxed payload field `.pld` # type variable for the boxed payload field `.pld`
PayloadT = TypeVar('PayloadT') PayloadT = TypeVar('PayloadT')
@ -202,7 +202,10 @@ class SpawnSpec(
# TODO: similar to the `Start` kwargs spec needed below, we need # TODO: similar to the `Start` kwargs spec needed below, we need
# a hard `Struct` def for all of these fields! # a hard `Struct` def for all of these fields!
_parent_main_data: dict _parent_main_data: dict
_runtime_vars: dict[str, Any] _runtime_vars: (
dict[str, Any]
#|RuntimeVars # !TODO
)
# ^NOTE see `._state._runtime_vars: dict` # ^NOTE see `._state._runtime_vars: dict`
# module import capability # module import capability
@ -321,6 +324,8 @@ class Start(
# => SEE ABOVE <= # => SEE ABOVE <=
kwargs: dict[str, Any] kwargs: dict[str, Any]
uid: tuple[str, str] # (calling) actor-id uid: tuple[str, str] # (calling) actor-id
# aid: Aid
# ^TODO, convert stat!
# TODO: enforcing a msg-spec in terms `Msg.pld` # TODO: enforcing a msg-spec in terms `Msg.pld`
# parameterizable msgs to be used in the appls IPC dialog. # parameterizable msgs to be used in the appls IPC dialog.

View File

@ -0,0 +1,26 @@
# tractor: structured concurrent "actors".
# Copyright 2018-eternity Tyler Goodlet.
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU Affero General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU Affero General Public License for more details.
# You should have received a copy of the GNU Affero General Public License
# along with this program. If not, see <https://www.gnu.org/licenses/>.
'''
The actor runtime: core machinery for the
actor-model implemented on a `trio` task runtime.
NOTE: to avoid circular imports, this ``__init__``
does NOT eagerly import submodules. Use direct
module paths like ``tractor.runtime._state`` or
``tractor.runtime._runtime`` instead.
'''

View File

@ -39,37 +39,37 @@ import warnings
import trio import trio
from .trionics import ( from ..trionics import (
maybe_open_nursery, maybe_open_nursery,
collapse_eg, collapse_eg,
) )
from ._state import ( from ._state import (
current_actor, current_actor,
) )
from .ipc import Channel from ..ipc import Channel
from .log import get_logger from ..log import get_logger
from .msg import ( from ..msg import (
# Error, # Error,
PayloadMsg, PayloadMsg,
NamespacePath, NamespacePath,
Return, Return,
) )
from ._exceptions import ( from .._exceptions import (
NoResult, NoResult,
TransportClosed, TransportClosed,
) )
from ._context import ( from .._context import (
Context, Context,
open_context_from_portal, open_context_from_portal,
) )
from ._streaming import ( from .._streaming import (
MsgStream, MsgStream,
) )
if TYPE_CHECKING: if TYPE_CHECKING:
from ._runtime import Actor from ._runtime import Actor
log = get_logger(__name__) log = get_logger()
class Portal: class Portal:
@ -329,18 +329,7 @@ class Portal:
# if we get here some weird cancellation case happened # if we get here some weird cancellation case happened
return False return False
except ( except TransportClosed as tpt_err:
# XXX, should never really get raised unless we aren't
# wrapping them in the below type by mistake?
#
# Leaving the catch here for now until we're very sure
# all the cases (for various tpt protos) have indeed been
# re-wrapped ;p
trio.ClosedResourceError,
trio.BrokenResourceError,
TransportClosed,
) as tpt_err:
ipc_borked_report: str = ( ipc_borked_report: str = (
f'IPC for actor already closed/broken?\n\n' f'IPC for actor already closed/broken?\n\n'
f'\n' f'\n'

View File

@ -43,11 +43,11 @@ from trio import (
TaskStatus, TaskStatus,
) )
from .ipc import Channel from ..ipc import Channel
from ._context import ( from .._context import (
Context, Context,
) )
from ._exceptions import ( from .._exceptions import (
ContextCancelled, ContextCancelled,
RemoteActorError, RemoteActorError,
ModuleNotExposed, ModuleNotExposed,
@ -56,19 +56,19 @@ from ._exceptions import (
pack_error, pack_error,
unpack_error, unpack_error,
) )
from .trionics import ( from ..trionics import (
collapse_eg, collapse_eg,
is_multi_cancelled, is_multi_cancelled,
maybe_raise_from_masking_exc, maybe_raise_from_masking_exc,
) )
from .devx import ( from ..devx import (
debug, debug,
add_div, add_div,
pformat as _pformat, pformat as _pformat,
) )
from . import _state from . import _state
from .log import get_logger from ..log import get_logger
from .msg import ( from ..msg import (
current_codec, current_codec,
MsgCodec, MsgCodec,
PayloadT, PayloadT,
@ -252,8 +252,8 @@ async def _invoke_non_context(
): ):
log.warning( log.warning(
'Failed to send RPC result?\n' 'Failed to send RPC result?\n'
f'|_{func}@{actor.uid}() -> {ret_msg}\n\n' f'|_{func}@{actor.aid.reprol()}() -> {ret_msg}\n\n'
f'x=> peer: {chan.uid}\n' f'x=> peer: {chan.aid.reprol()}\n'
) )
@acm @acm
@ -284,6 +284,15 @@ async def _errors_relayed_via_ipc(
try: try:
yield # run RPC invoke body yield # run RPC invoke body
# NOTE, never REPL any pseudo-expected tpt-disconnect.
except TransportClosed as err:
rpc_err = err
log.warning(
f'Tpt disconnect during remote-exc relay due to,\n'
f'{err!r}\n'
)
raise err
# box and ship RPC errors for wire-transit via # box and ship RPC errors for wire-transit via
# the task's requesting parent IPC-channel. # the task's requesting parent IPC-channel.
except ( except (
@ -327,10 +336,15 @@ async def _errors_relayed_via_ipc(
# recovery logic - the only case is some kind of # recovery logic - the only case is some kind of
# strange bug in our transport layer itself? Going # strange bug in our transport layer itself? Going
# to keep this open ended for now. # to keep this open ended for now.
log.debug(
'RPC task crashed, attempting to enter debugger\n' if _state.debug_mode():
f'|_{ctx}' log.exception(
f'RPC task crashed!\n'
f'Attempting to enter debugger\n'
f'\n'
f'{ctx}'
) )
entered_debug = await debug._maybe_enter_pm( entered_debug = await debug._maybe_enter_pm(
err, err,
api_frame=inspect.currentframe(), api_frame=inspect.currentframe(),
@ -419,7 +433,7 @@ async def _errors_relayed_via_ipc(
# cancel scope will not have been inserted yet # cancel scope will not have been inserted yet
if is_rpc: if is_rpc:
log.warning( log.warning(
'RPC task likely errored or cancelled before start?\n' 'RPC task likely crashed or cancelled before start?\n'
f'|_{ctx._task}\n' f'|_{ctx._task}\n'
f' >> {ctx.repr_rpc}\n' f' >> {ctx.repr_rpc}\n'
) )
@ -684,7 +698,7 @@ async def _invoke(
# which cancels the scope presuming the input error # which cancels the scope presuming the input error
# is not a `.cancel_acked` pleaser. # is not a `.cancel_acked` pleaser.
if rpc_ctx_cs.cancelled_caught: if rpc_ctx_cs.cancelled_caught:
our_uid: tuple = actor.uid our_uid: tuple = actor.aid.uid
# first check for and raise any remote error # first check for and raise any remote error
# before raising any context cancelled case # before raising any context cancelled case
@ -716,7 +730,7 @@ async def _invoke(
# TODO: determine if the ctx peer task was the # TODO: determine if the ctx peer task was the
# exact task which cancelled, vs. some other # exact task which cancelled, vs. some other
# task in the same actor. # task in the same actor.
elif canceller == ctx.chan.uid: elif canceller == ctx.chan.aid.uid:
explain += f'its {ctx.peer_side!r}-side peer' explain += f'its {ctx.peer_side!r}-side peer'
elif canceller == our_uid: elif canceller == our_uid:
@ -811,7 +825,7 @@ async def _invoke(
# associated child isn't in debug any more # associated child isn't in debug any more
await debug.maybe_wait_for_debugger() await debug.maybe_wait_for_debugger()
ctx: Context = actor._contexts.pop(( ctx: Context = actor._contexts.pop((
chan.uid, chan.aid.uid,
cid, cid,
)) ))
@ -862,9 +876,9 @@ async def _invoke(
) )
logmeth( logmeth(
f'{message}\n' f'{message}'
f'\n' f'\n'
f'{descr_str}\n' f'{descr_str}'
) )
@ -900,6 +914,11 @@ async def try_ship_error_to_remote(
# XXX NOTE XXX in SC terms this is one of the worst things # XXX NOTE XXX in SC terms this is one of the worst things
# that can happen and provides for a 2-general's dilemma.. # that can happen and provides for a 2-general's dilemma..
#
# FURHTER, we should never really have to handle these
# lowlevel excs from `trio` since the `Channel.send()` layers
# downward should be mostly wrapping such cases in a
# tpt-closed; the `.critical()` usage is warranted.
except ( except (
trio.ClosedResourceError, trio.ClosedResourceError,
trio.BrokenResourceError, trio.BrokenResourceError,
@ -908,7 +927,7 @@ async def try_ship_error_to_remote(
log.critical( log.critical(
'IPC transport failure -> ' 'IPC transport failure -> '
f'failed to ship error to {remote_descr}!\n\n' f'failed to ship error to {remote_descr}!\n\n'
f'{type(msg)!r}[{msg.boxed_type_str}] X=> {channel.uid}\n' f'{type(msg)!r}[{msg.boxed_type_str}] X=> {channel.aid.uid}\n'
f'\n' f'\n'
# TODO: use `.msg.preetty_struct` for this! # TODO: use `.msg.preetty_struct` for this!
f'{msg}\n' f'{msg}\n'
@ -986,7 +1005,7 @@ async def process_messages(
async for msg in chan: async for msg in chan:
log.transport( # type: ignore log.transport( # type: ignore
f'IPC msg from peer\n' f'IPC msg from peer\n'
f'<= {chan.uid}\n\n' f'<= {chan.aid.reprol()}\n\n'
# TODO: use of the pprinting of structs is # TODO: use of the pprinting of structs is
# FRAGILE and should prolly not be # FRAGILE and should prolly not be
@ -1090,7 +1109,7 @@ async def process_messages(
except BaseException: except BaseException:
log.exception( log.exception(
'Failed to cancel task?\n' 'Failed to cancel task?\n'
f'<= canceller: {chan.uid}\n' f'<= canceller: {chan.aid.reprol()}\n'
f' |_{chan}\n\n' f' |_{chan}\n\n'
f'=> {actor}\n' f'=> {actor}\n'
f' |_cid: {target_cid}\n' f' |_cid: {target_cid}\n'
@ -1245,7 +1264,7 @@ async def process_messages(
log.transport( log.transport(
'Waiting on next IPC msg from\n' 'Waiting on next IPC msg from\n'
f'peer: {chan.uid}\n' f'peer: {chan.aid.reprol()}\n'
f'|_{chan}\n' f'|_{chan}\n'
) )
@ -1294,12 +1313,10 @@ async def process_messages(
f'peer IPC channel closed abruptly?\n' f'peer IPC channel closed abruptly?\n'
f'\n' f'\n'
f'<=x[\n' f'<=x[\n'
f' {chan}\n' f'{chan}\n'
f' |_{chan.raddr}\n\n'
) )
+ +
tc.message tc.message
) )
# transport **WAS** disconnected # transport **WAS** disconnected
@ -1322,8 +1339,8 @@ async def process_messages(
match err: match err:
case ContextCancelled(): case ContextCancelled():
log.cancel( log.cancel(
f'Actor: {actor.uid} was context-cancelled with,\n' f'Actor: {actor.aid.reprol()!r} is ctxc with,\n'
f'str(err)' f'{str(err)}'
) )
case _: case _:
log.exception("Actor errored:") log.exception("Actor errored:")

View File

@ -83,46 +83,46 @@ from tractor.msg import (
pretty_struct, pretty_struct,
types as msgtypes, types as msgtypes,
) )
from .trionics import ( from ..trionics import (
collapse_eg, collapse_eg,
maybe_open_nursery, maybe_open_nursery,
) )
from .ipc import ( from ..ipc import (
Channel, Channel,
# IPCServer, # causes cycles atm.. # IPCServer, # causes cycles atm..
_server, _server,
) )
from ._addr import ( from ..discovery._addr import (
UnwrappedAddress, UnwrappedAddress,
Address, Address,
# default_lo_addrs, # default_lo_addrs,
get_address_cls, get_address_cls,
wrap_address, wrap_address,
) )
from ._context import ( from .._context import (
mk_context, mk_context,
Context, Context,
) )
from .log import get_logger from ..log import get_logger
from ._exceptions import ( from .._exceptions import (
ContextCancelled, ContextCancelled,
InternalError, InternalError,
ModuleNotExposed, ModuleNotExposed,
MsgTypeError, MsgTypeError,
unpack_error, unpack_error,
) )
from .devx import ( from ..devx import (
debug, debug,
pformat as _pformat pformat as _pformat
) )
from ._discovery import get_registry from ..discovery._discovery import get_registry
from ._portal import Portal from ._portal import Portal
from . import _state from . import _state
from . import _mp_fixup_main from ..spawn import _mp_fixup_main
from . import _rpc from . import _rpc
if TYPE_CHECKING: if TYPE_CHECKING:
from ._supervise import ActorNursery from ._supervise import ActorNursery # noqa
from trio._channel import MemoryChannelState from trio._channel import MemoryChannelState
@ -147,6 +147,8 @@ def get_mod_nsps2fps(mod_ns_paths: list[str]) -> dict[str, str]:
return nsp2fp return nsp2fp
_bp = False
class Actor: class Actor:
''' '''
The fundamental "runtime" concurrency primitive. The fundamental "runtime" concurrency primitive.
@ -173,13 +175,29 @@ class Actor:
dialog. dialog.
''' '''
# ugh, we need to get rid of this and replace with a "registry" sys is_registrar: bool = False
# https://github.com/goodboy/tractor/issues/216
is_arbiter: bool = False
@property @property
def is_registrar(self) -> bool: def is_arbiter(self) -> bool:
return self.is_arbiter '''
Deprecated, use `.is_registrar`.
'''
warnings.warn(
'`Actor.is_arbiter` is deprecated.\n'
'Use `.is_registrar` instead.',
DeprecationWarning,
stacklevel=2,
)
return self.is_registrar
@property
def is_root(self) -> bool:
'''
This actor is the parent most in the tree?
'''
return _state.is_root_process()
msg_buffer_size: int = 2**6 msg_buffer_size: int = 2**6
@ -227,7 +245,6 @@ class Actor:
registry_addrs: list[Address]|None = None, registry_addrs: list[Address]|None = None,
spawn_method: str|None = None, spawn_method: str|None = None,
# TODO: remove!
arbiter_addr: UnwrappedAddress|None = None, arbiter_addr: UnwrappedAddress|None = None,
) -> None: ) -> None:
@ -272,11 +289,13 @@ class Actor:
stacklevel=2, stacklevel=2,
) )
registry_addrs: list[Address] = [wrap_address(arbiter_addr)] registry_addrs: list[Address] = [
wrap_address(arbiter_addr)
]
# marked by the process spawning backend at startup # marked by the process spawning backend at startup
# will be None for the parent most process started manually # will be None for the parent most process started
# by the user (currently called the "arbiter") # manually by the user (the "registrar")
self._spawn_method: str = spawn_method self._spawn_method: str = spawn_method
# RPC state # RPC state
@ -679,7 +698,7 @@ class Actor:
''' '''
# ?TODO, use Aid here as well? # ?TODO, use Aid here as well?
actor_uid = chan.uid actor_uid = chan.aid.uid
assert actor_uid assert actor_uid
try: try:
ctx = self._contexts[( ctx = self._contexts[(
@ -689,7 +708,7 @@ class Actor:
)] )]
log.debug( log.debug(
f'Retreived cached IPC ctx for\n' f'Retreived cached IPC ctx for\n'
f'peer: {chan.uid}\n' f'peer: {chan.aid.uid}\n'
f'cid:{cid}\n' f'cid:{cid}\n'
) )
ctx._allow_overruns: bool = allow_overruns ctx._allow_overruns: bool = allow_overruns
@ -706,7 +725,7 @@ class Actor:
except KeyError: except KeyError:
log.debug( log.debug(
f'Allocate new IPC ctx for\n' f'Allocate new IPC ctx for\n'
f'peer: {chan.uid}\n' f'peer: {chan.aid.uid}\n'
f'cid: {cid}\n' f'cid: {cid}\n'
) )
ctx = mk_context( ctx = mk_context(
@ -752,7 +771,7 @@ class Actor:
''' '''
cid: str = str(uuid.uuid4()) cid: str = str(uuid.uuid4())
assert chan.uid assert chan.aid.uid
ctx = self.get_context( ctx = self.get_context(
chan=chan, chan=chan,
cid=cid, cid=cid,
@ -779,12 +798,12 @@ class Actor:
ns=ns, ns=ns,
func=func, func=func,
kwargs=kwargs, kwargs=kwargs,
uid=self.uid, uid=self.aid.uid, # <- !TODO use .aid!
cid=cid, cid=cid,
) )
log.runtime( log.runtime(
'Sending RPC `Start`\n\n' 'Sending RPC `Start`\n\n'
f'=> peer: {chan.uid}\n' f'=> peer: {chan.aid.uid}\n'
f' |_ {ns}.{func}({kwargs})\n\n' f' |_ {ns}.{func}({kwargs})\n\n'
f'{pretty_struct.pformat(msg)}' f'{pretty_struct.pformat(msg)}'
@ -895,7 +914,7 @@ class Actor:
# TODO! -[ ] another `Struct` for rtvs.. # TODO! -[ ] another `Struct` for rtvs..
rvs: dict[str, Any] = spawnspec._runtime_vars rvs: dict[str, Any] = spawnspec._runtime_vars
if rvs['_debug_mode']: if rvs['_debug_mode']:
from .devx import ( from ..devx import (
enable_stack_on_sig, enable_stack_on_sig,
maybe_init_greenback, maybe_init_greenback,
) )
@ -959,6 +978,21 @@ class Actor:
rvs['_is_root'] = False # obvi XD rvs['_is_root'] = False # obvi XD
# TODO, remove! left in just while protoing init fix!
# global _bp
# if (
# 'chart' in self.aid.name
# and
# isinstance(
# rvs['_root_addrs'][0],
# dict,
# )
# and
# not _bp
# ):
# _bp = True
# breakpoint()
_state._runtime_vars.update(rvs) _state._runtime_vars.update(rvs)
# `SpawnSpec.reg_addrs` # `SpawnSpec.reg_addrs`
@ -1217,7 +1251,7 @@ class Actor:
'Cancel request for invalid RPC task.\n' 'Cancel request for invalid RPC task.\n'
'The task likely already completed or was never started!\n\n' 'The task likely already completed or was never started!\n\n'
f'<= canceller: {requesting_aid}\n' f'<= canceller: {requesting_aid}\n'
f'=> {cid}@{parent_chan.uid}\n' f'=> {cid}@{parent_chan.aid.uid}\n'
f' |_{parent_chan}\n' f' |_{parent_chan}\n'
) )
return True return True
@ -1354,7 +1388,7 @@ class Actor:
f'Cancelling {descr} RPC tasks\n\n' f'Cancelling {descr} RPC tasks\n\n'
f'<=c) {req_aid} [canceller]\n' f'<=c) {req_aid} [canceller]\n'
f'{rent_chan_repr}' f'{rent_chan_repr}'
f'c)=> {self.uid} [cancellee]\n' f'c)=> {self.aid.uid} [cancellee]\n'
f' |_{self} [with {len(tasks)} tasks]\n' f' |_{self} [with {len(tasks)} tasks]\n'
# f' |_tasks: {len(tasks)}\n' # f' |_tasks: {len(tasks)}\n'
# f'{tasks_str}' # f'{tasks_str}'
@ -1455,7 +1489,12 @@ async def async_main(
# be False when running as root actor and True when as # be False when running as root actor and True when as
# a subactor. # a subactor.
parent_addr: UnwrappedAddress|None = None, parent_addr: UnwrappedAddress|None = None,
task_status: TaskStatus[None] = trio.TASK_STATUS_IGNORED, task_status: TaskStatus[
tuple[
list[UnwrappedAddress], # accept_addrs
list[UnwrappedAddress], # reg_addrs
]
] = trio.TASK_STATUS_IGNORED,
) -> None: ) -> None:
''' '''
@ -1624,7 +1663,7 @@ async def async_main(
# TODO, just read direct from ipc_server? # TODO, just read direct from ipc_server?
accept_addrs: list[UnwrappedAddress] = actor.accept_addrs accept_addrs: list[UnwrappedAddress] = actor.accept_addrs
# Register with the arbiter if we're told its addr # Register with the registrar if we're told its addr
log.runtime( log.runtime(
f'Registering `{actor.name}` => {pformat(accept_addrs)}\n' f'Registering `{actor.name}` => {pformat(accept_addrs)}\n'
# ^-TODO-^ we should instead show the maddr here^^ # ^-TODO-^ we should instead show the maddr here^^
@ -1634,6 +1673,7 @@ async def async_main(
# if addresses point to the same actor.. # if addresses point to the same actor..
# So we need a way to detect that? maybe iterate # So we need a way to detect that? maybe iterate
# only on unique actor uids? # only on unique actor uids?
addr: UnwrappedAddress
for addr in actor.reg_addrs: for addr in actor.reg_addrs:
try: try:
waddr = wrap_address(addr) waddr = wrap_address(addr)
@ -1642,7 +1682,9 @@ async def async_main(
await debug.pause() await debug.pause()
# !TODO, get rid of the local-portal crap XD # !TODO, get rid of the local-portal crap XD
reg_portal: Portal
async with get_registry(addr) as reg_portal: async with get_registry(addr) as reg_portal:
accept_addr: UnwrappedAddress
for accept_addr in accept_addrs: for accept_addr in accept_addrs:
accept_addr = wrap_address(accept_addr) accept_addr = wrap_address(accept_addr)
@ -1652,14 +1694,18 @@ async def async_main(
await reg_portal.run_from_ns( await reg_portal.run_from_ns(
'self', 'self',
'register_actor', 'register_actor',
uid=actor.uid, uid=actor.aid.uid,
addr=accept_addr.unwrap(), addr=accept_addr.unwrap(),
) )
is_registered: bool = True is_registered: bool = True
# init steps complete # init steps complete, deliver IPC-server and
task_status.started() # registrar addrs back to caller.
task_status.started((
accept_addrs,
actor.reg_addrs,
))
# Begin handling our new connection back to our # Begin handling our new connection back to our
# parent. This is done last since we don't want to # parent. This is done last since we don't want to
@ -1719,9 +1765,11 @@ async def async_main(
# always! # always!
match internal_err: match internal_err:
case ContextCancelled(): case ContextCancelled():
reprol: str = actor.aid.reprol()
log.cancel( log.cancel(
f'Actor: {actor.uid} was task-context-cancelled with,\n' f'Actor {reprol!r} was task-ctx-cancelled with,\n'
f'str(internal_err)' f'\n'
f'{internal_err!r}'
) )
case _: case _:
log.exception( log.exception(
@ -1793,7 +1841,7 @@ async def async_main(
await reg_portal.run_from_ns( await reg_portal.run_from_ns(
'self', 'self',
'unregister_actor', 'unregister_actor',
uid=actor.uid uid=actor.aid.uid,
) )
except OSError: except OSError:
failed = True failed = True
@ -1839,153 +1887,8 @@ async def async_main(
log.runtime(teardown_report) log.runtime(teardown_report)
# TODO: rename to `Registry` and move to `.discovery._registry`! # Backward compat: class moved to discovery._registry
class Arbiter(Actor): from ..discovery._registry import (
''' Registrar as Registrar,
A special registrar (and for now..) `Actor` who can contact all )
other actors within its immediate process tree and possibly keeps Arbiter = Registrar
a registry of others meant to be discoverable in a distributed
application. Normally the registrar is also the "root actor" and
thus always has access to the top-most-level actor (process)
nursery.
By default, the registrar is always initialized when and if no
other registrar socket addrs have been specified to runtime
init entry-points (such as `open_root_actor()` or
`open_nursery()`). Any time a new main process is launched (and
thus thus a new root actor created) and, no existing registrar
can be contacted at the provided `registry_addr`, then a new
one is always created; however, if one can be reached it is
used.
Normally a distributed app requires at least registrar per
logical host where for that given "host space" (aka localhost
IPC domain of addresses) it is responsible for making all other
host (local address) bound actors *discoverable* to external
actor trees running on remote hosts.
'''
is_arbiter = True
# TODO, implement this as a read on there existing a `._state` of
# some sort setup by whenever we impl this all as
# a `.discovery._registry.open_registry()` API
def is_registry(self) -> bool:
return self.is_arbiter
def __init__(
self,
*args,
**kwargs,
) -> None:
self._registry: dict[
tuple[str, str],
UnwrappedAddress,
] = {}
self._waiters: dict[
str,
# either an event to sync to receiving an actor uid (which
# is filled in once the actor has sucessfully registered),
# or that uid after registry is complete.
list[trio.Event | tuple[str, str]]
] = {}
super().__init__(*args, **kwargs)
async def find_actor(
self,
name: str,
) -> UnwrappedAddress|None:
for uid, addr in self._registry.items():
if name in uid:
return addr
return None
async def get_registry(
self
) -> dict[str, UnwrappedAddress]:
'''
Return current name registry.
This method is async to allow for cross-actor invocation.
'''
# NOTE: requires ``strict_map_key=False`` to the msgpack
# unpacker since we have tuples as keys (not this makes the
# arbiter suscetible to hashdos):
# https://github.com/msgpack/msgpack-python#major-breaking-changes-in-msgpack-10
return {
'.'.join(key): val
for key, val in self._registry.items()
}
async def wait_for_actor(
self,
name: str,
) -> list[UnwrappedAddress]:
'''
Wait for a particular actor to register.
This is a blocking call if no actor by the provided name is currently
registered.
'''
addrs: list[UnwrappedAddress] = []
addr: UnwrappedAddress
mailbox_info: str = 'Actor registry contact infos:\n'
for uid, addr in self._registry.items():
mailbox_info += (
f'|_uid: {uid}\n'
f'|_addr: {addr}\n\n'
)
if name == uid[0]:
addrs.append(addr)
if not addrs:
waiter = trio.Event()
self._waiters.setdefault(name, []).append(waiter)
await waiter.wait()
for uid in self._waiters[name]:
if not isinstance(uid, trio.Event):
addrs.append(self._registry[uid])
log.runtime(mailbox_info)
return addrs
async def register_actor(
self,
uid: tuple[str, str],
addr: UnwrappedAddress
) -> None:
uid = name, hash = (str(uid[0]), str(uid[1]))
waddr: Address = wrap_address(addr)
if not waddr.is_valid:
# should never be 0-dynamic-os-alloc
await debug.pause()
self._registry[uid] = addr
# pop and signal all waiter events
events = self._waiters.pop(name, [])
self._waiters.setdefault(name, []).append(uid)
for event in events:
if isinstance(event, trio.Event):
event.set()
async def unregister_actor(
self,
uid: tuple[str, str]
) -> None:
uid = (str(uid[0]), str(uid[1]))
entry: tuple = self._registry.pop(uid, None)
if entry is None:
log.warning(f'Request to de-register {uid} failed?')

View File

@ -22,19 +22,25 @@ from __future__ import annotations
from contextvars import ( from contextvars import (
ContextVar, ContextVar,
) )
import os
from pathlib import Path from pathlib import Path
from typing import ( from typing import (
Any, Any,
Callable,
Literal, Literal,
TYPE_CHECKING, TYPE_CHECKING,
) )
import platformdirs
from trio.lowlevel import current_task from trio.lowlevel import current_task
from msgspec import (
field,
Struct,
)
if TYPE_CHECKING: if TYPE_CHECKING:
from ._runtime import Actor from ._runtime import Actor
from ._context import Context from .._context import Context
# default IPC transport protocol settings # default IPC transport protocol settings
@ -47,9 +53,70 @@ _def_tpt_proto: TransportProtocolKey = 'tcp'
_current_actor: Actor|None = None # type: ignore # noqa _current_actor: Actor|None = None # type: ignore # noqa
_last_actor_terminated: Actor|None = None _last_actor_terminated: Actor|None = None
# TODO: mk this a `msgspec.Struct`! # TODO: mk this a `msgspec.Struct`!
# -[ ] type out all fields obvi! # -[x] type out all fields obvi!
# -[ ] (eventually) mk wire-ready for monitoring? # -[ ] (eventually) mk wire-ready for monitoring?
class RuntimeVars(Struct):
'''
Actor-(and thus process)-global runtime state.
This struct is relayed from parent to child during sub-actor
spawning and is a singleton instance per process.
Generally contains,
- root-actor indicator.
- comms-info: addrs for both (public) process/service-discovery
and in-tree contact with other actors.
- transport-layer IPC protocol server(s) settings.
- debug-mode settings for enabling sync breakpointing and any
surrounding REPL-fixture hooking.
- infected-`asyncio` via guest-mode toggle(s)/cohfig.
'''
_is_root: bool = False # bool
_root_mailbox: tuple[str, str|int] = (None, None) # tuple[str|None, str|None]
_root_addrs: list[
tuple[str, str|int],
] = [] # tuple[str|None, str|None]
# parent->chld ipc protocol caps
_enable_tpts: list[TransportProtocolKey] = field(
default_factory=lambda: [_def_tpt_proto],
)
# registrar info
_registry_addrs: list[tuple] = []
# `debug_mode: bool` settings
_debug_mode: bool = False # bool
repl_fixture: bool|Callable = False # |AbstractContextManager[bool]
# for `tractor.pause_from_sync()` & `breakpoint()` support
use_greenback: bool = False
# infected-`asyncio`-mode: `trio` running as guest.
_is_infected_aio: bool = False
def __setattr__(
self,
key,
val,
) -> None:
breakpoint()
super().__setattr__(key, val)
def update(
self,
from_dict: dict|Struct,
) -> None:
for attr, val in from_dict.items():
setattr(
self,
attr,
val,
)
_runtime_vars: dict[str, Any] = { _runtime_vars: dict[str, Any] = {
# root of actor-process tree info # root of actor-process tree info
'_is_root': False, # bool '_is_root': False, # bool
@ -73,6 +140,23 @@ _runtime_vars: dict[str, Any] = {
} }
def get_runtime_vars(
as_dict: bool = True,
) -> dict:
'''
Deliver a **copy** of the current `Actor`'s "runtime variables".
By default, for historical impl reasons, this delivers the `dict`
form, but the `RuntimeVars` struct should be utilized as possible
for future calls.
'''
if as_dict:
return dict(_runtime_vars)
return RuntimeVars(**_runtime_vars)
def last_actor() -> Actor|None: def last_actor() -> Actor|None:
''' '''
Try to return last active `Actor` singleton Try to return last active `Actor` singleton
@ -98,13 +182,13 @@ def current_actor(
_current_actor is None _current_actor is None
): ):
msg: str = 'No local actor has been initialized yet?\n' msg: str = 'No local actor has been initialized yet?\n'
from ._exceptions import NoRuntime from .._exceptions import NoRuntime
if last := last_actor(): if last := last_actor():
msg += ( msg += (
f'Apparently the lact active actor was\n' f'Apparently the lact active actor was\n'
f'|_{last}\n' f'|_{last}\n'
f'|_{last.uid}\n' f'|_{last.aid.uid}\n'
) )
# no actor runtime has (as of yet) ever been started for # no actor runtime has (as of yet) ever been started for
# this process. # this process.
@ -164,7 +248,7 @@ def current_ipc_ctx(
not ctx not ctx
and error_on_not_set and error_on_not_set
): ):
from ._exceptions import InternalError from .._exceptions import InternalError
raise InternalError( raise InternalError(
'No IPC context has been allocated for this task yet?\n' 'No IPC context has been allocated for this task yet?\n'
f'|_{current_task()}\n' f'|_{current_task()}\n'
@ -172,23 +256,56 @@ def current_ipc_ctx(
return ctx return ctx
# std ODE (mutable) app state location
_rtdir: Path = Path(os.environ['XDG_RUNTIME_DIR'])
def get_rt_dir( def get_rt_dir(
subdir: str = 'tractor' subdir: str|Path|None = None,
appname: str = 'tractor',
) -> Path: ) -> Path:
''' '''
Return the user "runtime dir" where most userspace apps stick Return the user "runtime dir", the file-sys location where most
their IPC and cache related system util-files; we take hold userspace apps stick their IPC and cache related system
of a `'XDG_RUNTIME_DIR'/tractor/` subdir by default. util-files.
On linux we use a `${XDG_RUNTIME_DIR}/tractor/` subdir by
default, but equivalents are mapped for each platform using
the lovely `platformdirs` lib.
''' '''
rtdir: Path = _rtdir / subdir rt_dir: Path = Path(
if not rtdir.is_dir(): platformdirs.user_runtime_dir(
rtdir.mkdir() appname=appname,
return rtdir ),
)
# Normalize and validate that `subdir` is a relative path
# without any parent-directory ("..") components, to prevent
# escaping the runtime directory.
if subdir:
subdir_path = (
subdir
if isinstance(subdir, Path)
else Path(subdir)
)
if subdir_path.is_absolute():
raise ValueError(
f'`subdir` must be a relative path!\n'
f'{subdir!r}\n'
)
if any(part == '..' for part in subdir_path.parts):
raise ValueError(
"`subdir` must not contain '..' components!\n"
f'{subdir!r}\n'
)
rt_dir: Path = rt_dir / subdir_path
if not rt_dir.is_dir():
rt_dir.mkdir(
parents=True,
exist_ok=True, # avoid `FileExistsError` from conc calls
)
return rt_dir
def current_ipc_protos() -> list[str]: def current_ipc_protos() -> list[str]:

View File

@ -30,39 +30,39 @@ import warnings
import trio import trio
from .devx import ( from ..devx import (
debug, debug,
pformat as _pformat, pformat as _pformat,
) )
from ._addr import ( from ..discovery._addr import (
UnwrappedAddress, UnwrappedAddress,
mk_uuid, mk_uuid,
) )
from ._state import current_actor, is_main_process from ._state import current_actor, is_main_process
from .log import get_logger, get_loglevel from ..log import get_logger, get_loglevel
from ._runtime import Actor from ._runtime import Actor
from ._portal import Portal from ._portal import Portal
from .trionics import ( from ..trionics import (
is_multi_cancelled, is_multi_cancelled,
collapse_eg, collapse_eg,
) )
from ._exceptions import ( from .._exceptions import (
ContextCancelled, ContextCancelled,
) )
from ._root import ( from .._root import (
open_root_actor, open_root_actor,
) )
from . import _state from . import _state
from . import _spawn from ..spawn import _spawn
if TYPE_CHECKING: if TYPE_CHECKING:
import multiprocessing as mp import multiprocessing as mp
# from .ipc._server import IPCServer # from ..ipc._server import IPCServer
from .ipc import IPCServer from ..ipc import IPCServer
log = get_logger(__name__) log = get_logger()
class ActorNursery: class ActorNursery:
@ -391,15 +391,17 @@ class ActorNursery:
else: else:
if portal is None: # actor hasn't fully spawned yet if portal is None: # actor hasn't fully spawned yet
event: trio.Event = server._peer_connected[subactor.uid] event: trio.Event = server._peer_connected[
subactor.aid.uid
]
log.warning( log.warning(
f"{subactor.uid} never 't finished spawning?" f"{subactor.aid.uid} never 't finished spawning?"
) )
await event.wait() await event.wait()
# channel/portal should now be up # channel/portal should now be up
_, _, portal = children[subactor.uid] _, _, portal = children[subactor.aid.uid]
# XXX should be impossible to get here # XXX should be impossible to get here
# unless method was called from within # unless method was called from within
@ -407,7 +409,7 @@ class ActorNursery:
if portal is None: if portal is None:
# cancelled while waiting on the event # cancelled while waiting on the event
# to arrive # to arrive
chan = server._peers[subactor.uid][-1] chan = server._peers[subactor.aid.uid][-1]
if chan: if chan:
portal = Portal(chan) portal = Portal(chan)
else: # there's no other choice left else: # there's no other choice left
@ -506,7 +508,7 @@ async def _open_and_supervise_one_cancels_all_nursery(
except BaseException as _inner_err: except BaseException as _inner_err:
inner_err = _inner_err inner_err = _inner_err
errors[actor.uid] = inner_err errors[actor.aid.uid] = inner_err
# If we error in the root but the debugger is # If we error in the root but the debugger is
# engaged we don't want to prematurely kill (and # engaged we don't want to prematurely kill (and
@ -539,7 +541,7 @@ async def _open_and_supervise_one_cancels_all_nursery(
log.cancel( log.cancel(
f'Actor-nursery cancelled by {etype}\n\n' f'Actor-nursery cancelled by {etype}\n\n'
f'{current_actor().uid}\n' f'{current_actor().aid.uid}\n'
f' |_{an}\n\n' f' |_{an}\n\n'
# TODO: show tb str? # TODO: show tb str?
@ -630,7 +632,7 @@ async def _open_and_supervise_one_cancels_all_nursery(
# show frame on any (likely) internal error # show frame on any (likely) internal error
if ( if (
not an.cancelled not an.cancel_called
and an._scope_error and an._scope_error
): ):
__tracebackhide__: bool = False __tracebackhide__: bool = False
@ -726,7 +728,7 @@ async def open_nursery(
if ( if (
an an
and and
not an.cancelled not an.cancel_called
and and
an._scope_error an._scope_error
): ):

View File

@ -0,0 +1,26 @@
# tractor: structured concurrent "actors".
# Copyright 2018-eternity Tyler Goodlet.
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU Affero General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU Affero General Public License for more details.
# You should have received a copy of the GNU Affero General Public License
# along with this program. If not, see <https://www.gnu.org/licenses/>.
'''
Actor process spawning machinery using
multiple backends (trio, multiprocessing).
NOTE: to avoid circular imports, this ``__init__``
does NOT eagerly import submodules. Use direct
module paths like ``tractor.spawn._spawn`` or
``tractor.spawn._entry`` instead.
'''

View File

@ -29,19 +29,19 @@ from typing import (
import trio # type: ignore import trio # type: ignore
from .log import ( from ..log import (
get_console_log, get_console_log,
get_logger, get_logger,
) )
from . import _state from ..runtime import _state
from .devx import ( from ..devx import (
_frame_stack, _frame_stack,
pformat, pformat,
) )
# from .msg import pretty_struct # from ..msg import pretty_struct
from .to_asyncio import run_as_asyncio_guest from ..to_asyncio import run_as_asyncio_guest
from ._addr import UnwrappedAddress from ..discovery._addr import UnwrappedAddress
from ._runtime import ( from ..runtime._runtime import (
async_main, async_main,
Actor, Actor,
) )
@ -50,7 +50,7 @@ if TYPE_CHECKING:
from ._spawn import SpawnMethodKey from ._spawn import SpawnMethodKey
log = get_logger(__name__) log = get_logger()
def _mp_main( def _mp_main(
@ -72,11 +72,15 @@ def _mp_main(
spawn_ctx: mp.context.BaseContext = try_set_start_method(start_method) spawn_ctx: mp.context.BaseContext = try_set_start_method(start_method)
assert spawn_ctx assert spawn_ctx
# XXX, enable root log at level
if actor.loglevel is not None: if actor.loglevel is not None:
log.info( log.info(
f'Setting loglevel for {actor.uid} to {actor.loglevel}' f'Setting loglevel for {actor.uid} to {actor.loglevel!r}'
)
get_console_log(
level=actor.loglevel,
name='tractor',
) )
get_console_log(actor.loglevel)
# TODO: use scops headers like for `trio` below! # TODO: use scops headers like for `trio` below!
# (well after we libify it maybe..) # (well after we libify it maybe..)
@ -126,8 +130,12 @@ def _trio_main(
parent_addr=parent_addr parent_addr=parent_addr
) )
# XXX, enable root log at level
if actor.loglevel is not None: if actor.loglevel is not None:
get_console_log(actor.loglevel) get_console_log(
level=actor.loglevel,
name='tractor',
)
log.info( log.info(
f'Starting `trio` subactor from parent @ ' f'Starting `trio` subactor from parent @ '
f'{parent_addr}\n' f'{parent_addr}\n'

View File

@ -125,7 +125,7 @@ class PatchedForkServer(ForkServer):
self._forkserver_pid = None self._forkserver_pid = None
# XXX only thing that changed! # XXX only thing that changed!
cmd = ('from tractor._forkserver_override import main; ' + cmd = ('from tractor.spawn._forkserver_override import main; ' +
'main(%d, %d, %r, **%r)') 'main(%d, %d, %r, **%r)')
if self._preload_modules: if self._preload_modules:

View File

@ -34,11 +34,11 @@ from typing import (
import trio import trio
from trio import TaskStatus from trio import TaskStatus
from .devx import ( from ..devx import (
debug, debug,
pformat as _pformat pformat as _pformat
) )
from tractor._state import ( from tractor.runtime._state import (
current_actor, current_actor,
is_main_process, is_main_process,
is_root_process, is_root_process,
@ -46,10 +46,10 @@ from tractor._state import (
_runtime_vars, _runtime_vars,
) )
from tractor.log import get_logger from tractor.log import get_logger
from tractor._addr import UnwrappedAddress from tractor.discovery._addr import UnwrappedAddress
from tractor._portal import Portal from tractor.runtime._portal import Portal
from tractor._runtime import Actor from tractor.runtime._runtime import Actor
from tractor._entry import _mp_main from ._entry import _mp_main
from tractor._exceptions import ActorFailure from tractor._exceptions import ActorFailure
from tractor.msg import ( from tractor.msg import (
types as msgtypes, types as msgtypes,
@ -58,11 +58,11 @@ from tractor.msg import (
if TYPE_CHECKING: if TYPE_CHECKING:
from ipc import ( from tractor.ipc import (
_server, _server,
Channel, Channel,
) )
from ._supervise import ActorNursery from tractor.runtime._supervise import ActorNursery
ProcessType = TypeVar('ProcessType', mp.Process, trio.Process) ProcessType = TypeVar('ProcessType', mp.Process, trio.Process)
@ -151,7 +151,7 @@ async def exhaust_portal(
__tracebackhide__ = True __tracebackhide__ = True
try: try:
log.debug( log.debug(
f'Waiting on final result from {actor.uid}' f'Waiting on final result from {actor.aid.uid}'
) )
# XXX: streams should never be reaped here since they should # XXX: streams should never be reaped here since they should
@ -210,17 +210,17 @@ async def cancel_on_completion(
actor, actor,
) )
if isinstance(result, Exception): if isinstance(result, Exception):
errors[actor.uid]: Exception = result errors[actor.aid.uid]: Exception = result
log.cancel( log.cancel(
'Cancelling subactor runtime due to error:\n\n' 'Cancelling subactor runtime due to error:\n\n'
f'Portal.cancel_actor() => {portal.channel.uid}\n\n' f'Portal.cancel_actor() => {portal.channel.aid}\n\n'
f'error: {result}\n' f'error: {result}\n'
) )
else: else:
log.runtime( log.runtime(
'Cancelling subactor gracefully:\n\n' 'Cancelling subactor gracefully:\n\n'
f'Portal.cancel_actor() => {portal.channel.uid}\n\n' f'Portal.cancel_actor() => {portal.channel.aid}\n\n'
f'result: {result}\n' f'result: {result}\n'
) )
@ -308,7 +308,7 @@ async def hard_kill(
# ) # )
# with trio.CancelScope(shield=True): # with trio.CancelScope(shield=True):
# async with debug.acquire_debug_lock( # async with debug.acquire_debug_lock(
# subactor_uid=current_actor().uid, # subactor_uid=current_actor().aid.uid,
# ) as _ctx: # ) as _ctx:
# log.warning( # log.warning(
# 'Acquired debug lock, child ready to be killed ??\n' # 'Acquired debug lock, child ready to be killed ??\n'
@ -483,7 +483,7 @@ async def trio_proc(
# TODO, how to pass this over "wire" encodings like # TODO, how to pass this over "wire" encodings like
# cmdline args? # cmdline args?
# -[ ] maybe we can add an `msgtypes.Aid.min_tuple()` ? # -[ ] maybe we can add an `msgtypes.Aid.min_tuple()` ?
str(subactor.uid), str(subactor.aid.uid),
# Address the child must connect to on startup # Address the child must connect to on startup
"--parent_addr", "--parent_addr",
str(parent_addr) str(parent_addr)
@ -514,7 +514,7 @@ async def trio_proc(
# channel should have handshake completed by the # channel should have handshake completed by the
# local actor by the time we get a ref to it # local actor by the time we get a ref to it
event, chan = await ipc_server.wait_for_peer( event, chan = await ipc_server.wait_for_peer(
subactor.uid subactor.aid.uid
) )
except trio.Cancelled: except trio.Cancelled:
@ -528,7 +528,9 @@ async def trio_proc(
await debug.maybe_wait_for_debugger() await debug.maybe_wait_for_debugger()
elif proc is not None: elif proc is not None:
async with debug.acquire_debug_lock(subactor.uid): async with debug.acquire_debug_lock(
subactor_uid=subactor.aid.uid
):
# soft wait on the proc to terminate # soft wait on the proc to terminate
with trio.move_on_after(0.5): with trio.move_on_after(0.5):
await proc.wait() await proc.wait()
@ -538,7 +540,7 @@ async def trio_proc(
assert proc assert proc
portal = Portal(chan) portal = Portal(chan)
actor_nursery._children[subactor.uid] = ( actor_nursery._children[subactor.aid.uid] = (
subactor, subactor,
proc, proc,
portal, portal,
@ -563,7 +565,7 @@ async def trio_proc(
# track subactor in current nursery # track subactor in current nursery
curr_actor: Actor = current_actor() curr_actor: Actor = current_actor()
curr_actor._actoruid2nursery[subactor.uid] = actor_nursery curr_actor._actoruid2nursery[subactor.aid.uid] = actor_nursery
# resume caller at next checkpoint now that child is up # resume caller at next checkpoint now that child is up
task_status.started(portal) task_status.started(portal)
@ -616,7 +618,9 @@ async def trio_proc(
# don't clobber an ongoing pdb # don't clobber an ongoing pdb
if cancelled_during_spawn: if cancelled_during_spawn:
# Try again to avoid TTY clobbering. # Try again to avoid TTY clobbering.
async with debug.acquire_debug_lock(subactor.uid): async with debug.acquire_debug_lock(
subactor_uid=subactor.aid.uid
):
with trio.move_on_after(0.5): with trio.move_on_after(0.5):
await proc.wait() await proc.wait()
@ -662,7 +666,7 @@ async def trio_proc(
if not cancelled_during_spawn: if not cancelled_during_spawn:
# pop child entry to indicate we no longer managing this # pop child entry to indicate we no longer managing this
# subactor # subactor
actor_nursery._children.pop(subactor.uid) actor_nursery._children.pop(subactor.aid.uid)
async def mp_proc( async def mp_proc(
@ -744,7 +748,7 @@ async def mp_proc(
# register the process before start in case we get a cancel # register the process before start in case we get a cancel
# request before the actor has fully spawned - then we can wait # request before the actor has fully spawned - then we can wait
# for it to fully come up before sending a cancel request # for it to fully come up before sending a cancel request
actor_nursery._children[subactor.uid] = (subactor, proc, None) actor_nursery._children[subactor.aid.uid] = (subactor, proc, None)
proc.start() proc.start()
if not proc.is_alive(): if not proc.is_alive():
@ -758,7 +762,7 @@ async def mp_proc(
# channel should have handshake completed by the # channel should have handshake completed by the
# local actor by the time we get a ref to it # local actor by the time we get a ref to it
event, chan = await ipc_server.wait_for_peer( event, chan = await ipc_server.wait_for_peer(
subactor.uid, subactor.aid.uid,
) )
# XXX: monkey patch poll API to match the ``subprocess`` API.. # XXX: monkey patch poll API to match the ``subprocess`` API..
@ -771,7 +775,7 @@ async def mp_proc(
# any process we may have started. # any process we may have started.
portal = Portal(chan) portal = Portal(chan)
actor_nursery._children[subactor.uid] = (subactor, proc, portal) actor_nursery._children[subactor.aid.uid] = (subactor, proc, portal)
# unblock parent task # unblock parent task
task_status.started(portal) task_status.started(portal)
@ -810,7 +814,7 @@ async def mp_proc(
# tandem if not done already # tandem if not done already
log.warning( log.warning(
"Cancelling existing result waiter task for " "Cancelling existing result waiter task for "
f"{subactor.uid}") f"{subactor.aid.uid}")
nursery.cancel_scope.cancel() nursery.cancel_scope.cancel()
finally: finally:
@ -828,7 +832,7 @@ async def mp_proc(
log.debug(f"Joined {proc}") log.debug(f"Joined {proc}")
# pop child entry to indicate we are no longer managing subactor # pop child entry to indicate we are no longer managing subactor
actor_nursery._children.pop(subactor.uid) actor_nursery._children.pop(subactor.aid.uid)
# TODO: prolly report to ``mypy`` how this causes all sorts of # TODO: prolly report to ``mypy`` how this causes all sorts of
# false errors.. # false errors..

Some files were not shown because too many files have changed in this diff Show More