Only `ctx.canccel()` when non-callee ctxc received

Since presumably if the ctxc *is* received from the callee side, there's
no point in also sending it a cancel request ;)

Also, try out shielding the `ctx.cancel()` call inside
`Portal.open_context()` since that seems to resolve testing assertions
where you'd expect `.cancel_called` to have been set in the case where
an inter-peer cancellation is raised by some local ctx-handle and that
local ctxc causes other surrounding `.open_context()` blocks to raise
and then implicitly call `ctx.cancel()`. However, there's a possible
race where the callee side might deliver a ctxc while all that is
happening, `._scope.cancel()` gets called and then by the time
`ctx.cancel()` is called it raises `trio.Cancelled` instead continuing
to set `._cancel_called`? This needs more testing and docs to get the
minute details correct .. yet again.

.github/workflows/ci.yml

@@ -20,7 +20,7 @@ jobs:
       - name: Setup python
         uses: actions/setup-python@v2
         with:
-          python-version: '3.11'
+          python-version: '3.10'
 
       - name: Install dependencies
         run: pip install -U . --upgrade-strategy eager -r requirements-test.txt
@@ -41,7 +41,7 @@ jobs:
       - name: Setup python
         uses: actions/setup-python@v2
         with:
-          python-version: '3.11'
+          python-version: '3.10'
 
       - name: Build sdist
         run: python setup.py sdist --formats=zip
@@ -59,7 +59,7 @@ jobs:
       fail-fast: false
       matrix:
         os: [ubuntu-latest]
-        python: ['3.11']
+        python: ['3.10']
         spawn_backend: [
           'trio',
           'mp_spawn',

docs/README.rst

@@ -1,23 +1,20 @@
-|logo| ``tractor``: distributed structurred concurrency
+|logo| ``tractor``: next-gen Python parallelism
 
 |gh_actions|
 |docs|
 
-``tractor`` is a `structured concurrency`_ (SC), multi-processing_ runtime built on trio_.
+``tractor`` is a `structured concurrent`_, (optionally
+distributed_) multi-processing_ runtime built on trio_.
 
-Fundamentally, ``tractor`` provides parallelism via
-``trio``-"*actors*": independent Python **processes** (i.e.
-*non-shared-memory threads*) which can schedule ``trio`` tasks whilst
-maintaining *end-to-end SC* inside a *distributed supervision tree*.
+Fundamentally, ``tractor`` gives you parallelism via
+``trio``-"*actors*": independent Python processes (aka
+non-shared-memory threads) which maintain structured
+concurrency (SC) *end-to-end* inside a *supervision tree*.
 
 Cross-process (and thus cross-host) SC is accomplished through the
-combined use of our,
-
-- "actor nurseries_" which provide for spawning multiple, and
-  possibly nested, Python processes each running a ``trio`` scheduled
-  runtime - a call to ``trio.run()``,
-- an "SC-transitive supervision protocol" enforced as an
-  IPC-message-spec encapsulating all RPC-dialogs.
+combined use of our "actor nurseries_" and an "SC-transitive IPC
+protocol" constructed on top of multiple Pythons each running a ``trio``
+scheduled runtime - a call to ``trio.run()``.
 
 We believe the system adheres to the `3 axioms`_ of an "`actor model`_"
 but likely **does not** look like what **you** probably *think* an "actor
@@ -30,7 +27,6 @@ The first step to grok ``tractor`` is to get an intermediate
 knowledge of ``trio`` and **structured concurrency** B)
 
 Some great places to start are,
-
 - the seminal `blog post`_
 - obviously the `trio docs`_
 - wikipedia's nascent SC_ page
@@ -39,84 +35,22 @@ Some great places to start are,
 
 Features
 --------
-- **It's just** a ``trio`` API!
-- *Infinitely nesteable* process trees running embedded ``trio`` tasks.
-- Swappable, OS-specific, process spawning via multiple backends.
-- Modular IPC stack, allowing for custom interchange formats (eg.
-  as offered from `msgspec`_), varied transport protocols (TCP, RUDP,
-  QUIC, wireguard), and OS-env specific higher-perf primitives (UDS,
-  shm-ring-buffers).
-- Optionally distributed_: all IPC and RPC APIs work over multi-host
-  transports the same as local.
-- Builtin high-level streaming API that enables your app to easily
-  leverage the benefits of a "`cheap or nasty`_" `(un)protocol`_.
-- A "native UX" around a multi-process safe debugger REPL using
-  `pdbp`_ (a fork & fix of `pdb++`_)
-- "Infected ``asyncio``" mode: support for starting an actor's
-  runtime as a `guest`_ on the ``asyncio`` loop allowing us to
-  provide stringent SC-style ``trio.Task``-supervision around any
-  ``asyncio.Task`` spawned via our ``tractor.to_asyncio`` APIs.
-- A **very naive** and still very much work-in-progress inter-actor
-  `discovery`_ sys with plans to support multiple `modern protocol`_
-  approaches.
-- Various ``trio`` extension APIs via ``tractor.trionics`` such as,
-  - task fan-out `broadcasting`_,
-  - multi-task-single-resource-caching and fan-out-to-multi
-    ``__aenter__()`` APIs for ``@acm`` functions,
-  - (WIP) a ``TaskMngr``: one-cancels-one style nursery supervisor.
-
-
-Install
--------
-``tractor`` is still in a *alpha-near-beta-stage* for many
-of its subsystems, however we are very close to having a stable
-lowlevel runtime and API.
-
-As such, it's currently recommended that you clone and install the
-repo from source::
-
-    pip install git+git://github.com/goodboy/tractor.git
-
-
-We use the very hip `uv`_ for project mgmt::
-
-    git clone https://github.com/goodboy/tractor.git
-    cd tractor
-    uv sync --dev
-    uv run python examples/rpc_bidir_streaming.py
-
-Consider activating a virtual/project-env before starting to hack on
-the code base::
-
-    # you could use plain ol' venvs
-    # https://docs.astral.sh/uv/pip/environments/
-    uv venv tractor_py313 --python 3.13
-
-    # but @goodboy prefers the more explicit (and shell agnostic)
-    # https://docs.astral.sh/uv/configuration/environment/#uv_project_environment
-    UV_PROJECT_ENVIRONMENT="tractor_py313
-
-    # hint hint, enter @goodboy's fave shell B)
-    uv run --dev xonsh
-
-Alongside all this we ofc offer "releases" on PyPi::
-
-    pip install tractor
-
-Just note that YMMV since the main git branch is often much further
-ahead then any latest release.
-
-
-Example codez
--------------
-In ``tractor``'s (very lacking) documention we prefer to point to
-example scripts in the repo over duplicating them in docs, but with
-that in mind here are some definitive snippets to try and hook you
-into digging deeper.
+- **It's just** a ``trio`` API
+- *Infinitely nesteable* process trees
+- Builtin IPC streaming APIs with task fan-out broadcasting
+- A "native" multi-core debugger REPL using `pdbp`_ (a fork & fix of
+  `pdb++`_ thanks to @mdmintz!)
+- Support for a swappable, OS specific, process spawning layer
+- A modular transport stack, allowing for custom serialization (eg. with
+  `msgspec`_), communications protocols, and environment specific IPC
+  primitives
+- Support for spawning process-level-SC, inter-loop one-to-one-task oriented
+  ``asyncio`` actors via "infected ``asyncio``" mode
+- `structured chadcurrency`_ from the ground up
 
 
 Run a func in a process
-***********************
+-----------------------
 Use ``trio``'s style of focussing on *tasks as functions*:
 
 .. code:: python
@@ -174,7 +108,7 @@ might want to check out `trio-parallel`_.
 
 
 Zombie safe: self-destruct a process tree
-*****************************************
+-----------------------------------------
 ``tractor`` tries to protect you from zombies, no matter what.
 
 .. code:: python
@@ -230,7 +164,7 @@ it **is a bug**.
 
 
 "Native" multi-process debugging
-********************************
+--------------------------------
 Using the magic of `pdbp`_ and our internal IPC, we've
 been able to create a native feeling debugging experience for
 any (sub-)process in your ``tractor`` tree.
@@ -285,7 +219,7 @@ We're hoping to add a respawn-from-repl system soon!
 
 
 SC compatible bi-directional streaming
-**************************************
+--------------------------------------
 Yes, you saw it here first; we provide 2-way streams
 with reliable, transitive setup/teardown semantics.
 
@@ -377,7 +311,7 @@ hear your thoughts on!
 
 
 Worker poolz are easy peasy
-***************************
+---------------------------
 The initial ask from most new users is *"how do I make a worker
 pool thing?"*.
 
@@ -399,10 +333,10 @@ This uses no extra threads, fancy semaphores or futures; all we need
 is ``tractor``'s IPC!
 
 "Infected ``asyncio``" mode
-***************************
+---------------------------
 Have a bunch of ``asyncio`` code you want to force to be SC at the process level?
 
-Check out our experimental system for `guest`_-mode controlled
+Check out our experimental system for `guest-mode`_ controlled
 ``asyncio`` actors:
 
 .. code:: python
@@ -508,7 +442,7 @@ We need help refining the `asyncio`-side channel API to be more
 
 
 Higher level "cluster" APIs
-***************************
+---------------------------
 To be extra terse the ``tractor`` devs have started hacking some "higher
 level" APIs for managing actor trees/clusters. These interfaces should
 generally be condsidered provisional for now but we encourage you to try
@@ -565,6 +499,18 @@ spawn a flat cluster:
 .. _full worker pool re-implementation: https://github.com/goodboy/tractor/blob/master/examples/parallelism/concurrent_actors_primes.py
 
 
+Install
+-------
+From PyPi::
+
+    pip install tractor
+
+
+From git::
+
+    pip install git+git://github.com/goodboy/tractor.git
+
+
 Under the hood
 --------------
 ``tractor`` is an attempt to pair trionic_ `structured concurrency`_ with
@@ -668,26 +614,21 @@ channel`_!
 .. _adherance to: https://www.youtube.com/watch?v=7erJ1DV_Tlo&t=1821s
 .. _trio gitter channel: https://gitter.im/python-trio/general
 .. _matrix channel: https://matrix.to/#/!tractor:matrix.org
-.. _broadcasting: https://github.com/goodboy/tractor/pull/229
-.. _modern procotol: https://en.wikipedia.org/wiki/Rendezvous_protocol
 .. _pdbp: https://github.com/mdmintz/pdbp
 .. _pdb++: https://github.com/pdbpp/pdbpp
-.. _cheap or nasty: https://zguide.zeromq.org/docs/chapter7/#The-Cheap-or-Nasty-Pattern
-.. _(un)protocol: https://zguide.zeromq.org/docs/chapter7/#Unprotocols
-.. _discovery: https://zguide.zeromq.org/docs/chapter8/#Discovery
-.. _modern protocol: https://en.wikipedia.org/wiki/Rendezvous_protocol
+.. _guest mode: https://trio.readthedocs.io/en/stable/reference-lowlevel.html?highlight=guest%20mode#using-guest-mode-to-run-trio-on-top-of-other-event-loops
 .. _messages: https://en.wikipedia.org/wiki/Message_passing
 .. _trio docs: https://trio.readthedocs.io/en/latest/
 .. _blog post: https://vorpus.org/blog/notes-on-structured-concurrency-or-go-statement-considered-harmful/
 .. _structured concurrency: https://en.wikipedia.org/wiki/Structured_concurrency
 .. _SC: https://en.wikipedia.org/wiki/Structured_concurrency
 .. _libdill-docs: https://sustrik.github.io/libdill/structured-concurrency.html
+.. _structured chadcurrency: https://en.wikipedia.org/wiki/Structured_concurrency
 .. _unrequirements: https://en.wikipedia.org/wiki/Actor_model#Direct_communication_and_asynchrony
 .. _async generators: https://www.python.org/dev/peps/pep-0525/
 .. _trio-parallel: https://github.com/richardsheridan/trio-parallel
-.. _uv: https://docs.astral.sh/uv/
 .. _msgspec: https://jcristharif.com/msgspec/
-.. _guest: https://trio.readthedocs.io/en/stable/reference-lowlevel.html?highlight=guest%20mode#using-guest-mode-to-run-trio-on-top-of-other-event-loops
+.. _guest-mode: https://trio.readthedocs.io/en/stable/reference-lowlevel.html?highlight=guest%20mode#using-guest-mode-to-run-trio-on-top-of-other-event-loops
 
 
 .. |gh_actions| image:: https://img.shields.io/endpoint.svg?url=https%3A%2F%2Factions-badge.atrox.dev%2Fgoodboy%2Ftractor%2Fbadge&style=popout-square

examples/advanced_faults/ipc_failure_during_stream.py

@@ -6,120 +6,81 @@ been an outage) and we want to ensure that despite being in debug mode
 actor tree will eventually be cancelled without leaving any zombies.
 
 '''
-from contextlib import asynccontextmanager as acm
-from functools import partial
-
+import trio
 from tractor import (
     open_nursery,
     context,
     Context,
-    ContextCancelled,
     MsgStream,
-    _testing,
 )
-import trio
-import pytest
 
 
-async def break_ipc_then_error(
+async def break_channel_silently_then_error(
     stream: MsgStream,
-    break_ipc_with: str|None = None,
-    pre_close: bool = False,
-):
-    await _testing.break_ipc(
-        stream=stream,
-        method=break_ipc_with,
-        pre_close=pre_close,
-    )
-    async for msg in stream:
-        await stream.send(msg)
-
-    assert 0
-
-
-async def iter_ipc_stream(
-    stream: MsgStream,
-    break_ipc_with: str|None = None,
-    pre_close: bool = False,
 ):
     async for msg in stream:
         await stream.send(msg)
 
+        # XXX: close the channel right after an error is raised
+        # purposely breaking the IPC transport to make sure the parent
+        # doesn't get stuck in debug or hang on the connection join.
+        # this more or less simulates an infinite msg-receive hang on
+        # the other end.
+        await stream._ctx.chan.send(None)
+        assert 0
+
+
+async def close_stream_and_error(
+    stream: MsgStream,
+):
+    async for msg in stream:
+        await stream.send(msg)
+
+        # wipe out channel right before raising
+        await stream._ctx.chan.send(None)
+        await stream.aclose()
+        assert 0
+
 
 @context
 async def recv_and_spawn_net_killers(
 
     ctx: Context,
-    break_ipc_after: bool|int = False,
-    pre_close: bool = False,
+    break_ipc_after: bool | int = False,
 
 ) -> None:
     '''
     Receive stream msgs and spawn some IPC killers mid-stream.
 
     '''
-    broke_ipc: bool = False
     await ctx.started()
     async with (
         ctx.open_stream() as stream,
-        trio.open_nursery(
-            strict_exception_groups=False,
-        ) as tn,
+        trio.open_nursery() as n,
     ):
         async for i in stream:
             print(f'child echoing {i}')
-            if not broke_ipc:
-                await stream.send(i)
-            else:
-                await trio.sleep(0.01)
-
+            await stream.send(i)
             if (
                 break_ipc_after
-                and
-                i >= break_ipc_after
+                and i > break_ipc_after
             ):
-                broke_ipc = True
-                tn.start_soon(
-                    iter_ipc_stream,
-                    stream,
-                )
-                tn.start_soon(
-                    partial(
-                        break_ipc_then_error,
-                        stream=stream,
-                        pre_close=pre_close,
-                    )
-                )
-
-
-@acm
-async def stuff_hangin_ctlc(timeout: float = 1) -> None:
-
-    with trio.move_on_after(timeout) as cs:
-        yield timeout
-
-    if cs.cancelled_caught:
-        # pretend to be a user seeing no streaming action
-        # thinking it's a hang, and then hitting ctl-c..
-        print(
-            f"i'm a user on the PARENT side and thingz hangin "
-            f'after timeout={timeout} ???\n\n'
-            'MASHING CTlR-C..!?\n'
-        )
-        raise KeyboardInterrupt
+                '#################################\n'
+                'Simulating child-side IPC BREAK!\n'
+                '#################################'
+                n.start_soon(break_channel_silently_then_error, stream)
+                n.start_soon(close_stream_and_error, stream)
 
 
 async def main(
     debug_mode: bool = False,
     start_method: str = 'trio',
-    loglevel: str = 'cancel',
 
     # by default we break the parent IPC first (if configured to break
     # at all), but this can be changed so the child does first (even if
     # both are set to break).
-    break_parent_ipc_after: int|bool = False,
-    break_child_ipc_after: int|bool = False,
-    pre_close: bool = False,
+    break_parent_ipc_after: int | bool = False,
+    break_child_ipc_after: int | bool = False,
 
 ) -> None:
 
@@ -130,129 +91,60 @@ async def main(
             # NOTE: even debugger is used we shouldn't get
             # a hang since it never engages due to broken IPC
             debug_mode=debug_mode,
-            loglevel=loglevel,
+            loglevel='warning',
 
         ) as an,
     ):
-        sub_name: str = 'chitty_hijo'
         portal = await an.start_actor(
-            sub_name,
+            'chitty_hijo',
             enable_modules=[__name__],
         )
 
-        async with (
-            stuff_hangin_ctlc(timeout=2) as timeout,
-            _testing.expect_ctxc(
-                yay=(
-                    break_parent_ipc_after
-                    or break_child_ipc_after
-                ),
-                # TODO: we CAN'T remove this right?
-                # since we need the ctxc to bubble up from either
-                # the stream API after the `None` msg is sent
-                # (which actually implicitly cancels all remote
-                # tasks in the hijo) or from simluated
-                # KBI-mash-from-user
-                # or should we expect that a KBI triggers the ctxc
-                # and KBI in an eg?
-                reraise=True,
-            ),
+        async with portal.open_context(
+            recv_and_spawn_net_killers,
+            break_ipc_after=break_child_ipc_after,
 
-            portal.open_context(
-                recv_and_spawn_net_killers,
-                break_ipc_after=break_child_ipc_after,
-                pre_close=pre_close,
-            ) as (ctx, sent),
-        ):
-            rx_eoc: bool = False
-            ipc_break_sent: bool = False
+        ) as (ctx, sent):
             async with ctx.open_stream() as stream:
                 for i in range(1000):
 
                     if (
                         break_parent_ipc_after
-                        and
-                        i > break_parent_ipc_after
-                        and
-                        not ipc_break_sent
+                        and i > break_parent_ipc_after
                     ):
                         print(
                             '#################################\n'
-                            'Simulating PARENT-side IPC BREAK!\n'
-                            '#################################\n'
+                            'Simulating parent-side IPC BREAK!\n'
+                            '#################################'
                         )
-
-                        # TODO: other methods? see break func above.
-                        # await stream._ctx.chan.send(None)
-                        # await stream._ctx.chan.transport.stream.send_eof()
-                        await stream._ctx.chan.transport.stream.aclose()
-                        ipc_break_sent = True
+                        await stream._ctx.chan.send(None)
 
                     # it actually breaks right here in the
-                    # mp_spawn/forkserver backends and thus the
-                    # zombie reaper never even kicks in?
-                    try:
-                        print(f'parent sending {i}')
-                        await stream.send(i)
-                    except ContextCancelled as ctxc:
-                        print(
-                            'parent received ctxc on `stream.send()`\n'
-                            f'{ctxc}\n'
-                        )
-                        assert 'root' in ctxc.canceller
-                        assert sub_name in ctx.canceller
+                    # mp_spawn/forkserver backends and thus the zombie
+                    # reaper never even kicks in?
+                    print(f'parent sending {i}')
+                    await stream.send(i)
 
-                        # TODO: is this needed or no?
-                        raise
+                    with trio.move_on_after(2) as cs:
 
-                    except trio.ClosedResourceError:
-                        # NOTE: don't send if we already broke the
-                        # connection to avoid raising a closed-error
-                        # such that we drop through to the ctl-c
-                        # mashing by user.
-                        await trio.sleep(0.01)
-
-                    # timeout: int = 1
-                    # with trio.move_on_after(timeout) as cs:
-                    async with stuff_hangin_ctlc() as timeout:
-                        print(
-                            f'PARENT `stream.receive()` with timeout={timeout}\n'
-                        )
                         # NOTE: in the parent side IPC failure case this
                         # will raise an ``EndOfChannel`` after the child
                         # is killed and sends a stop msg back to it's
                         # caller/this-parent.
-                        try:
-                            rx = await stream.receive()
-                            print(
-                                "I'm a happy PARENT user and echoed to me is\n"
-                                f'{rx}\n'
-                            )
-                        except trio.EndOfChannel:
-                            rx_eoc: bool = True
-                            print('MsgStream got EoC for PARENT')
-                            raise
+                        rx = await stream.receive()
 
-            print(
-                'Streaming finished and we got Eoc.\n'
-                'Canceling `.open_context()` in root with\n'
-                'CTlR-C..'
-            )
-            if rx_eoc:
-                assert stream.closed
-                try:
-                    await stream.send(i)
-                    pytest.fail('stream not closed?')
-                except (
-                    trio.ClosedResourceError,
-                    trio.EndOfChannel,
-                ) as send_err:
-                    if rx_eoc:
-                        assert send_err is stream._eoc
-                    else:
-                        assert send_err is stream._closed
+                        print(f"I'm a happy user and echoed to me is {rx}")
 
-            raise KeyboardInterrupt
+                    if cs.cancelled_caught:
+                        # pretend to be a user seeing no streaming action
+                        # thinking it's a hang, and then hitting ctl-c..
+                        print("YOO i'm a user anddd thingz hangin..")
+
+                print(
+                    "YOO i'm mad send side dun but thingz hangin..\n"
+                    'MASHING CTlR-C Ctl-c..'
+                )
+                raise KeyboardInterrupt
 
 
 if __name__ == '__main__':

examples/debugging/asyncio_bp.py

@@ -1,16 +1,8 @@
-'''
-Examples of using the builtin `breakpoint()` from an `asyncio.Task`
-running in a subactor spawned with `infect_asyncio=True`.
-
-'''
 import asyncio
 
 import trio
 import tractor
-from tractor import (
-    to_asyncio,
-    Portal,
-)
+from tractor import to_asyncio
 
 
 async def aio_sleep_forever():
@@ -25,21 +17,21 @@ async def bp_then_error(
 
 ) -> None:
 
-    # sync with `trio`-side (caller) task
+    # sync with ``trio``-side (caller) task
     to_trio.send_nowait('start')
 
     # NOTE: what happens here inside the hook needs some refinement..
     # => seems like it's still `._debug._set_trace()` but
     #    we set `Lock.local_task_in_debug = 'sync'`, we probably want
-    #    some further, at least, meta-data about the task/actor in debug
-    #    in terms of making it clear it's `asyncio` mucking about.
-    breakpoint()  # asyncio-side
+    #    some further, at least, meta-data about the task/actoq in debug
+    #    in terms of making it clear it's asyncio mucking about.
+    breakpoint()
 
     # short checkpoint / delay
-    await asyncio.sleep(0.5)  # asyncio-side
+    await asyncio.sleep(0.5)
 
     if raise_after_bp:
-        raise ValueError('asyncio side error!')
+        raise ValueError('blah')
 
     # TODO: test case with this so that it gets cancelled?
     else:
@@ -57,21 +49,23 @@ async def trio_ctx(
     # this will block until the ``asyncio`` task sends a "first"
     # message, see first line in above func.
     async with (
+
         to_asyncio.open_channel_from(
             bp_then_error,
-            # raise_after_bp=not bp_before_started,
+            raise_after_bp=not bp_before_started,
         ) as (first, chan),
 
-        trio.open_nursery() as tn,
+        trio.open_nursery() as n,
     ):
+
         assert first == 'start'
 
         if bp_before_started:
-            await tractor.pause()  # trio-side
+            await tractor.breakpoint()
 
-        await ctx.started(first)  # trio-side
+        await ctx.started(first)
 
-        tn.start_soon(
+        n.start_soon(
             to_asyncio.run_task,
             aio_sleep_forever,
         )
@@ -79,50 +73,37 @@ async def trio_ctx(
 
 
 async def main(
-    bps_all_over: bool = True,
-
-    # TODO, WHICH OF THESE HAZ BUGZ?
-    cancel_from_root: bool = False,
-    err_from_root: bool = False,
+    bps_all_over: bool = False,
 
 ) -> None:
 
-    async with tractor.open_nursery(
-        debug_mode=True,
-        maybe_enable_greenback=True,
-        # loglevel='devx',
-    ) as an:
-        ptl: Portal = await an.start_actor(
+    async with tractor.open_nursery() as n:
+
+        p = await n.start_actor(
             'aio_daemon',
             enable_modules=[__name__],
             infect_asyncio=True,
             debug_mode=True,
-            # loglevel='cancel',
+            loglevel='cancel',
         )
 
-        async with ptl.open_context(
+        async with p.open_context(
             trio_ctx,
             bp_before_started=bps_all_over,
         ) as (ctx, first):
 
             assert first == 'start'
 
-            # pause in parent to ensure no cross-actor
-            # locking problems exist!
-            await tractor.pause()  # trio-root
-
-            if cancel_from_root:
-                await ctx.cancel()
-
-            if err_from_root:
-                assert 0
-            else:
-                await trio.sleep_forever()
+            if bps_all_over:
+                await tractor.breakpoint()
 
+            # await trio.sleep_forever()
+            await ctx.cancel()
+            assert 0
 
         # TODO: case where we cancel from trio-side while asyncio task
         # has debugger lock?
-        # await ptl.cancel_actor()
+        # await p.cancel_actor()
 
 
 if __name__ == '__main__':

examples/debugging/debug_mode_hang.py

@@ -1,9 +0,0 @@
-'''
-Reproduce a bug where enabling debug mode for a sub-actor actually causes
-a hang on teardown...
-
-'''
-import asyncio
-
-import trio
-import tractor

examples/debugging/fast_error_in_root_after_spawn.py

@@ -1,5 +1,5 @@
 '''
-Fast fail test with a `Context`.
+Fast fail test with a context.
 
 Ensure the partially initialized sub-actor process
 doesn't cause a hang on error/cancel of the parent

examples/debugging/multi_daemon_subactors.py

@@ -4,15 +4,9 @@ import trio
 
 async def breakpoint_forever():
     "Indefinitely re-enter debugger in child actor."
-    try:
-        while True:
-            yield 'yo'
-            await tractor.pause()
-    except BaseException:
-        tractor.log.get_console_log().exception(
-            'Cancelled while trying to enter pause point!'
-        )
-        raise
+    while True:
+        yield 'yo'
+        await tractor.breakpoint()
 
 
 async def name_error():
@@ -21,14 +15,11 @@ async def name_error():
 
 
 async def main():
-    '''
-    Test breakpoint in a streaming actor.
-
-    '''
+    """Test breakpoint in a streaming actor.
+    """
     async with tractor.open_nursery(
         debug_mode=True,
-        loglevel='cancel',
-        # loglevel='devx',
+        loglevel='error',
     ) as n:
 
         p0 = await n.start_actor('bp_forever', enable_modules=[__name__])
@@ -41,7 +32,7 @@ async def main():
             try:
                 await p1.run(name_error)
             except tractor.RemoteActorError as rae:
-                assert rae.boxed_type is NameError
+                assert rae.type is NameError
 
             async for i in stream:
 

examples/debugging/multi_nested_subactors_error_up_through_nurseries.py

@@ -10,7 +10,7 @@ async def name_error():
 async def breakpoint_forever():
     "Indefinitely re-enter debugger in child actor."
     while True:
-        await tractor.pause()
+        await tractor.breakpoint()
 
         # NOTE: if the test never sent 'q'/'quit' commands
         # on the pdb repl, without this checkpoint line the
@@ -45,7 +45,6 @@ async def spawn_until(depth=0):
             )
 
 
-# TODO: notes on the new boxed-relayed errors through proxy actors
 async def main():
     """The main ``tractor`` routine.
 

examples/debugging/multi_subactor_root_errors.py

@@ -40,7 +40,7 @@ async def main():
     """
     async with tractor.open_nursery(
         debug_mode=True,
-        loglevel='devx',
+        # loglevel='cancel',
     ) as n:
 
         # spawn both actors

examples/debugging/multi_subactors.py

@@ -6,7 +6,7 @@ async def breakpoint_forever():
     "Indefinitely re-enter debugger in child actor."
     while True:
         await trio.sleep(0.1)
-        await tractor.pause()
+        await tractor.breakpoint()
 
 
 async def name_error():
@@ -38,7 +38,6 @@ async def main():
     """
     async with tractor.open_nursery(
         debug_mode=True,
-        # loglevel='runtime',
     ) as n:
 
         # Spawn both actors, don't bother with collecting results

examples/debugging/per_actor_debug.py

@@ -23,6 +23,5 @@ async def main():
             n.start_soon(debug_actor.run, die)
             n.start_soon(crash_boi.run, die)
 
-
 if __name__ == '__main__':
     trio.run(main)

examples/debugging/pm_in_subactor.py

@@ -1,56 +0,0 @@
-import trio
-import tractor
-
-
-@tractor.context
-async def name_error(
-    ctx: tractor.Context,
-):
-    '''
-    Raise a `NameError`, catch it and enter `.post_mortem()`, then
-    expect the `._rpc._invoke()` crash handler to also engage.
-
-    '''
-    try:
-        getattr(doggypants)  # noqa (on purpose)
-    except NameError:
-        await tractor.post_mortem()
-        raise
-
-
-async def main():
-    '''
-    Test 3 `PdbREPL` entries:
-      - one in the child due to manual `.post_mortem()`,
-      - another in the child due to runtime RPC crash handling.
-      - final one here in parent from the RAE.
-
-    '''
-    # XXX NOTE: ideally the REPL arrives at this frame in the parent
-    # ONE UP FROM the inner ctx block below!
-    async with tractor.open_nursery(
-        debug_mode=True,
-        # loglevel='cancel',
-    ) as an:
-        p: tractor.Portal = await an.start_actor(
-            'child',
-            enable_modules=[__name__],
-        )
-
-        # XXX should raise `RemoteActorError[NameError]`
-        # AND be the active frame when REPL enters!
-        try:
-            async with p.open_context(name_error) as (ctx, first):
-                assert first
-        except tractor.RemoteActorError as rae:
-            assert rae.boxed_type is NameError
-
-            # manually handle in root's parent task
-            await tractor.post_mortem()
-            raise
-        else:
-            raise RuntimeError('IPC ctx should have remote errored!?')
-
-
-if __name__ == '__main__':
-    trio.run(main)

examples/debugging/restore_builtin_breakpoint.py

@@ -6,44 +6,19 @@ import tractor
 
 
 async def main() -> None:
+    async with tractor.open_nursery(debug_mode=True) as an:
 
-    # intially unset, no entry.
-    orig_pybp_var: int = os.environ.get('PYTHONBREAKPOINT')
-    assert orig_pybp_var in {None, "0"}
-
-    async with tractor.open_nursery(
-        debug_mode=True,
-    ) as an:
-        assert an
-        assert (
-            (pybp_var := os.environ['PYTHONBREAKPOINT'])
-            ==
-            'tractor.devx._debug._sync_pause_from_builtin'
-        )
+        assert os.environ['PYTHONBREAKPOINT'] == 'tractor._debug._set_trace'
 
         # TODO: an assert that verifies the hook has indeed been, hooked
         # XD
-        assert (
-            (pybp_hook := sys.breakpointhook)
-            is not tractor.devx._debug._set_trace
-        )
+        assert sys.breakpointhook is not tractor._debug._set_trace
 
-        print(
-            f'$PYTHONOBREAKPOINT: {pybp_var!r}\n'
-            f'`sys.breakpointhook`: {pybp_hook!r}\n'
-        )
-        breakpoint()  # first bp, tractor hook set.
+        breakpoint()
 
-    # XXX AFTER EXIT (of actor-runtime) verify the hook is unset..
-    #
-    # YES, this is weird but it's how stdlib docs say to do it..
-    # https://docs.python.org/3/library/sys.html#sys.breakpointhook
-    assert os.environ.get('PYTHONBREAKPOINT') is orig_pybp_var
+    # TODO: an assert that verifies the hook is unhooked..
     assert sys.breakpointhook
-
-    # now ensure a regular builtin pause still works
-    breakpoint()  # last bp, stdlib hook restored
-
+    breakpoint()
 
 if __name__ == '__main__':
     trio.run(main)

examples/debugging/root_actor_breakpoint.py

@@ -10,7 +10,7 @@ async def main():
 
         await trio.sleep(0.1)
 
-        await tractor.pause()
+        await tractor.breakpoint()
 
         await trio.sleep(0.1)
 

examples/debugging/root_actor_breakpoint_forever.py

@@ -2,16 +2,13 @@ import trio
 import tractor
 
 
-async def main(
-    registry_addrs: tuple[str, int]|None = None
-):
+async def main():
 
     async with tractor.open_root_actor(
         debug_mode=True,
-        # loglevel='runtime',
     ):
         while True:
-            await tractor.pause()
+            await tractor.breakpoint()
 
 
 if __name__ == '__main__':

examples/debugging/shield_hang_in_sub.py

@@ -1,83 +0,0 @@
-'''
-Verify we can dump a `stackscope` tree on a hang.
-
-'''
-import os
-import signal
-
-import trio
-import tractor
-
-@tractor.context
-async def start_n_shield_hang(
-    ctx: tractor.Context,
-):
-    # actor: tractor.Actor = tractor.current_actor()
-
-    # sync to parent-side task
-    await ctx.started(os.getpid())
-
-    print('Entering shield sleep..')
-    with trio.CancelScope(shield=True):
-        await trio.sleep_forever()  # in subactor
-
-    # XXX NOTE ^^^ since this shields, we expect
-    # the zombie reaper (aka T800) to engage on
-    # SIGINT from the user and eventually hard-kill
-    # this subprocess!
-
-
-async def main(
-    from_test: bool = False,
-) -> None:
-
-    async with (
-        tractor.open_nursery(
-            debug_mode=True,
-            enable_stack_on_sig=True,
-            # maybe_enable_greenback=False,
-            loglevel='devx',
-        ) as an,
-    ):
-        ptl: tractor.Portal  = await an.start_actor(
-            'hanger',
-            enable_modules=[__name__],
-            debug_mode=True,
-        )
-        async with ptl.open_context(
-            start_n_shield_hang,
-        ) as (ctx, cpid):
-
-            _, proc, _ = an._children[ptl.chan.uid]
-            assert cpid == proc.pid
-
-            print(
-                'Yo my child hanging..?\n'
-                # "i'm a user who wants to see a `stackscope` tree!\n"
-            )
-
-            # XXX simulate the wrapping test's "user actions"
-            # (i.e. if a human didn't run this manually but wants to
-            # know what they should do to reproduce test behaviour)
-            if from_test:
-                print(
-                    f'Sending SIGUSR1 to {cpid!r}!\n'
-                )
-                os.kill(
-                    cpid,
-                    signal.SIGUSR1,
-                )
-
-                # simulate user cancelling program
-                await trio.sleep(0.5)
-                os.kill(
-                    os.getpid(),
-                    signal.SIGINT,
-                )
-            else:
-                # actually let user send the ctl-c
-                await trio.sleep_forever()  # in root
-
-
-if __name__ == '__main__':
-    trio.run(main)

examples/debugging/shielded_pause.py

@@ -1,88 +0,0 @@
-import trio
-import tractor
-
-
-async def cancellable_pause_loop(
-    task_status: trio.TaskStatus[trio.CancelScope] = trio.TASK_STATUS_IGNORED
-):
-    with trio.CancelScope() as cs:
-        task_status.started(cs)
-        for _ in range(3):
-            try:
-                # ON first entry, there is no level triggered
-                # cancellation yet, so this cp does a parent task
-                # ctx-switch so that this scope raises for the NEXT
-                # checkpoint we hit.
-                await trio.lowlevel.checkpoint()
-                await tractor.pause()
-
-                cs.cancel()
-
-                # parent should have called `cs.cancel()` by now
-                await trio.lowlevel.checkpoint()
-
-            except trio.Cancelled:
-                print('INSIDE SHIELDED PAUSE')
-                await tractor.pause(shield=True)
-        else:
-            # should raise it again, bubbling up to parent
-            print('BUBBLING trio.Cancelled to parent task-nursery')
-            await trio.lowlevel.checkpoint()
-
-
-async def pm_on_cancelled():
-    async with trio.open_nursery() as tn:
-        tn.cancel_scope.cancel()
-        try:
-            await trio.sleep_forever()
-        except trio.Cancelled:
-            # should also raise `Cancelled` since
-            # we didn't pass `shield=True`.
-            try:
-                await tractor.post_mortem(hide_tb=False)
-            except trio.Cancelled as taskc:
-
-                # should enter just fine, in fact it should
-                # be debugging the internals of the previous
-                # sin-shield call above Bo
-                await tractor.post_mortem(
-                    hide_tb=False,
-                    shield=True,
-                )
-                raise taskc
-
-        else:
-            raise RuntimeError('Dint cancel as expected!?')
-
-
-async def cancelled_before_pause(
-):
-    '''
-    Verify that using a shielded pause works despite surrounding
-    cancellation called state in the calling task.
-
-    '''
-    async with trio.open_nursery() as tn:
-        cs: trio.CancelScope = await tn.start(cancellable_pause_loop)
-        await trio.sleep(0.1)
-
-    assert cs.cancelled_caught
-
-    await pm_on_cancelled()
-
-
-async def main():
-    async with tractor.open_nursery(
-        debug_mode=True,
-    ) as n:
-        portal: tractor.Portal = await n.run_in_actor(
-            cancelled_before_pause,
-        )
-        await portal.result()
-
-        # ensure the same works in the root actor!
-        await pm_on_cancelled()
-
-
-if __name__ == '__main__':
-    trio.run(main)

examples/debugging/subactor_bp_in_ctx.py

@@ -4,9 +4,9 @@ import trio
 
 async def gen():
     yield 'yo'
-    await tractor.pause()
+    await tractor.breakpoint()
     yield 'yo'
-    await tractor.pause()
+    await tractor.breakpoint()
 
 
 @tractor.context
@@ -15,7 +15,7 @@ async def just_bp(
 ) -> None:
 
     await ctx.started()
-    await tractor.pause()
+    await tractor.breakpoint()
 
     # TODO: bps and errors in this call..
     async for val in gen():

examples/debugging/subactor_breakpoint.py

@@ -3,20 +3,17 @@ import tractor
 
 
 async def breakpoint_forever():
-    '''
-    Indefinitely re-enter debugger in child actor.
-
-    '''
+    """Indefinitely re-enter debugger in child actor.
+    """
     while True:
         await trio.sleep(0.1)
-        await tractor.pause()
+        await tractor.breakpoint()
 
 
 async def main():
 
     async with tractor.open_nursery(
         debug_mode=True,
-        loglevel='cancel',
     ) as n:
 
         portal = await n.run_in_actor(

examples/debugging/subactor_error.py

@@ -3,26 +3,16 @@ import tractor
 
 
 async def name_error():
-    getattr(doggypants)  # noqa (on purpose)
+    getattr(doggypants)
 
 
 async def main():
     async with tractor.open_nursery(
         debug_mode=True,
-        # loglevel='transport',
-    ) as an:
+    ) as n:
 
-        # TODO: ideally the REPL arrives at this frame in the parent,
-        # ABOVE the @api_frame of `Portal.run_in_actor()` (which
-        # should eventually not even be a portal method ... XD)
-        # await tractor.pause()
-        p: tractor.Portal = await an.run_in_actor(name_error)
-
-        # with this style, should raise on this line
-        await p.result()
-
-        # with this alt style should raise at `open_nusery()`
-        # return await p.result()
+        portal = await n.run_in_actor(name_error)
+        await portal.result()
 
 
 if __name__ == '__main__':

examples/debugging/sync_bp.py

@@ -1,169 +0,0 @@
-from functools import partial
-import time
-
-import trio
-import tractor
-
-# TODO: only import these when not running from test harness?
-# can we detect `pexpect` usage maybe?
-# from tractor.devx._debug import (
-#     get_lock,
-#     get_debug_req,
-# )
-
-
-def sync_pause(
-    use_builtin: bool = False,
-    error: bool = False,
-    hide_tb: bool = True,
-    pre_sleep: float|None = None,
-):
-    if pre_sleep:
-        time.sleep(pre_sleep)
-
-    if use_builtin:
-        breakpoint(hide_tb=hide_tb)
-
-    else:
-        # TODO: maybe for testing some kind of cm style interface
-        # where the `._set_trace()` call doesn't happen until block
-        # exit?
-        # assert get_lock().ctx_in_debug is None
-        # assert get_debug_req().repl is None
-        tractor.pause_from_sync()
-        # assert get_debug_req().repl is None
-
-    if error:
-        raise RuntimeError('yoyo sync code error')
-
-
-@tractor.context
-async def start_n_sync_pause(
-    ctx: tractor.Context,
-):
-    actor: tractor.Actor = tractor.current_actor()
-
-    # sync to parent-side task
-    await ctx.started()
-
-    print(f'Entering `sync_pause()` in subactor: {actor.uid}\n')
-    sync_pause()
-    print(f'Exited `sync_pause()` in subactor: {actor.uid}\n')
-
-
-async def main() -> None:
-    async with (
-        tractor.open_nursery(
-            debug_mode=True,
-            maybe_enable_greenback=True,
-            enable_stack_on_sig=True,
-            # loglevel='warning',
-            # loglevel='devx',
-        ) as an,
-        trio.open_nursery() as tn,
-    ):
-        # just from root task
-        sync_pause()
-
-        p: tractor.Portal  = await an.start_actor(
-            'subactor',
-            enable_modules=[__name__],
-            # infect_asyncio=True,
-            debug_mode=True,
-        )
-
-        # TODO: 3 sub-actor usage cases:
-        # -[x] via a `.open_context()`
-        # -[ ] via a `.run_in_actor()` call
-        # -[ ] via a `.run()`
-        # -[ ] via a `.to_thread.run_sync()` in subactor
-        async with p.open_context(
-            start_n_sync_pause,
-        ) as (ctx, first):
-            assert first is None
-
-            # TODO: handle bg-thread-in-root-actor special cases!
-            #
-            # there are a couple very subtle situations possible here
-            # and they are likely to become more important as cpython
-            # moves to support no-GIL.
-            #
-            # Cases:
-            # 1. root-actor bg-threads that call `.pause_from_sync()`
-            #   whilst an in-tree subactor also is using ` .pause()`.
-            # |_ since the root-actor bg thread can not
-            #   `Lock._debug_lock.acquire_nowait()` without running
-            #   a `trio.Task`, AND because the
-            #   `PdbREPL.set_continue()` is called from that
-            #   bg-thread, we can not `._debug_lock.release()`
-            #   either!
-            #  |_ this results in no actor-tree `Lock` being used
-            #    on behalf of the bg-thread and thus the subactor's
-            #    task and the thread trying to to use stdio
-            #    simultaneously which results in the classic TTY
-            #    clobbering!
-            #
-            # 2. mutiple sync-bg-threads that call
-            #   `.pause_from_sync()` where one is scheduled via
-            #   `Nursery.start_soon(to_thread.run_sync)` in a bg
-            #   task.
-            #
-            #   Due to the GIL, the threads never truly try to step
-            #   through the REPL simultaneously, BUT their `logging`
-            #   and traceback outputs are interleaved since the GIL
-            #   (seemingly) on every REPL-input from the user
-            #   switches threads..
-            #
-            #   Soo, the context switching semantics of the GIL
-            #   result in a very confusing and messy interaction UX
-            #   since eval and (tb) print output is NOT synced to
-            #   each REPL-cycle (like we normally make it via
-            #   a `.set_continue()` callback triggering the
-            #   `Lock.release()`). Ideally we can solve this
-            #   usability issue NOW because this will of course be
-            #   that much more important when eventually there is no
-            #   GIL!
-
-            # XXX should cause double REPL entry and thus TTY
-            # clobbering due to case 1. above!
-            tn.start_soon(
-                partial(
-                    trio.to_thread.run_sync,
-                    partial(
-                        sync_pause,
-                        use_builtin=False,
-                        # pre_sleep=0.5,
-                    ),
-                    abandon_on_cancel=True,
-                    thread_name='start_soon_root_bg_thread',
-                )
-            )
-
-            await tractor.pause()
-
-            # XXX should cause double REPL entry and thus TTY
-            # clobbering due to case 2. above!
-            await trio.to_thread.run_sync(
-                partial(
-                    sync_pause,
-                    # NOTE this already works fine since in the new
-                    # thread the `breakpoint()` built-in is never
-                    # overloaded, thus NO locking is used, HOWEVER
-                    # the case 2. from above still exists!
-                    use_builtin=True,
-                ),
-                # TODO: with this `False` we can hang!??!
-                # abandon_on_cancel=False,
-                abandon_on_cancel=True,
-                thread_name='inline_root_bg_thread',
-            )
-
-        await ctx.cancel()
-
-        # TODO: case where we cancel from trio-side while asyncio task
-        # has debugger lock?
-        await p.cancel_actor()
-
-
-if __name__ == '__main__':
-    trio.run(main)

examples/full_fledged_streaming_service.py

@@ -1,11 +1,6 @@
 import time
 import trio
 import tractor
-from tractor import (
-    ActorNursery,
-    MsgStream,
-    Portal,
-)
 
 
 # this is the first 2 actors, streamer_1 and streamer_2
@@ -17,18 +12,14 @@ async def stream_data(seed):
 
 # this is the third actor; the aggregator
 async def aggregate(seed):
-    '''
-    Ensure that the two streams we receive match but only stream
+    """Ensure that the two streams we receive match but only stream
     a single set of values to the parent.
-
-    '''
-    an: ActorNursery
-    async with tractor.open_nursery() as an:
-        portals: list[Portal] = []
+    """
+    async with tractor.open_nursery() as nursery:
+        portals = []
         for i in range(1, 3):
-
-            # fork/spawn call
-            portal = await an.start_actor(
+            # fork point
+            portal = await nursery.start_actor(
                 name=f'streamer_{i}',
                 enable_modules=[__name__],
             )
@@ -52,11 +43,7 @@ async def aggregate(seed):
         async with trio.open_nursery() as n:
 
             for portal in portals:
-                n.start_soon(
-                    push_to_chan,
-                    portal,
-                    send_chan.clone(),
-                )
+                n.start_soon(push_to_chan, portal, send_chan.clone())
 
             # close this local task's reference to send side
             await send_chan.aclose()
@@ -73,7 +60,7 @@ async def aggregate(seed):
 
             print("FINISHED ITERATING in aggregator")
 
-        await an.cancel()
+        await nursery.cancel()
         print("WAITING on `ActorNursery` to finish")
     print("AGGREGATOR COMPLETE!")
 
@@ -88,21 +75,18 @@ async def main() -> list[int]:
 
     '''
     # yes, a nursery which spawns `trio`-"actors" B)
-    an: ActorNursery
-    async with tractor.open_nursery(
-        loglevel='cancel',
-        # debug_mode=True,
-    ) as an:
+    nursery: tractor.ActorNursery
+    async with tractor.open_nursery() as nursery:
 
         seed = int(1e3)
         pre_start = time.time()
 
-        portal: Portal = await an.start_actor(
+        portal: tractor.Portal = await nursery.start_actor(
             name='aggregator',
             enable_modules=[__name__],
         )
 
-        stream: MsgStream
+        stream: tractor.MsgStream
         async with portal.open_stream_from(
             aggregate,
             seed=seed,
@@ -111,12 +95,11 @@ async def main() -> list[int]:
             start = time.time()
             # the portal call returns exactly what you'd expect
             # as if the remote "aggregate" function was called locally
-            result_stream: list[int] = []
+            result_stream = []
             async for value in stream:
                 result_stream.append(value)
 
-        cancelled: bool = await portal.cancel_actor()
-        assert cancelled
+        await portal.cancel_actor()
 
         print(f"STREAM TIME = {time.time() - start}")
         print(f"STREAM + SPAWN TIME = {time.time() - pre_start}")

examples/parallelism/concurrent_actors_primes.py

@@ -8,10 +8,7 @@ This uses no extra threads, fancy semaphores or futures; all we need
 is ``tractor``'s channels.
 
 """
-from contextlib import (
-    asynccontextmanager as acm,
-    aclosing,
-)
+from contextlib import asynccontextmanager
 from typing import Callable
 import itertools
 import math
@@ -19,6 +16,7 @@ import time
 
 import tractor
 import trio
+from async_generator import aclosing
 
 
 PRIMES = [
@@ -46,7 +44,7 @@ async def is_prime(n):
     return True
 
 
-@acm
+@asynccontextmanager
 async def worker_pool(workers=4):
     """Though it's a trivial special case for ``tractor``, the well
     known "worker pool" seems to be the defacto "but, I want this

examples/quick_cluster.py

@@ -3,18 +3,20 @@ import trio
 import tractor
 
 
-async def sleepy_jane() -> None:
-    uid: tuple = tractor.current_actor().uid
+async def sleepy_jane():
+    uid = tractor.current_actor().uid
     print(f'Yo i am actor {uid}')
     await trio.sleep_forever()
 
 
 async def main():
     '''
-    Spawn a flat actor cluster, with one process per detected core.
+    Spawn a flat actor cluster, with one process per
+    detected core.
 
     '''
     portal_map: dict[str, tractor.Portal]
+    results: dict[str, str]
 
     # look at this hip new syntax!
     async with (
@@ -23,16 +25,11 @@ async def main():
             modules=[__name__]
         ) as portal_map,
 
-        trio.open_nursery(
-            strict_exception_groups=False,
-        ) as tn,
+        trio.open_nursery() as n,
     ):
 
         for (name, portal) in portal_map.items():
-            tn.start_soon(
-                portal.run,
-                sleepy_jane,
-            )
+            n.start_soon(portal.run, sleepy_jane)
 
         await trio.sleep(0.5)
 
@@ -44,4 +41,4 @@ if __name__ == '__main__':
     try:
         trio.run(main)
     except KeyboardInterrupt:
-        print('trio cancelled by KBI')
+        pass

examples/rpc_bidir_streaming.py

@@ -13,7 +13,7 @@ async def simple_rpc(
 
     '''
     # signal to parent that we're up much like
-    # ``trio.TaskStatus.started()``
+    # ``trio_typing.TaskStatus.started()``
     await ctx.started(data + 1)
 
     async with ctx.open_stream() as stream:

examples/service_discovery.py

@@ -9,7 +9,7 @@ async def main(service_name):
     async with tractor.open_nursery() as an:
         await an.start_actor(service_name)
 
-        async with tractor.get_registry('127.0.0.1', 1616) as portal:
+        async with tractor.get_arbiter('127.0.0.1', 1616) as portal:
             print(f"Arbiter is listening on {portal.channel}")
 
         async with tractor.wait_for_actor(service_name) as sockaddr:

notes_to_self/howtorelease.md

@@ -1,18 +0,0 @@
-First generate a built disti:
-
-```
-python -m pip install --upgrade build
-python -m build --sdist --outdir dist/alpha5/
-```
-
-Then try a test ``pypi`` upload:
-
-```
-python -m twine upload --repository testpypi dist/alpha5/*
-```
-
-The push to `pypi` for realz.
-
-```
-python -m twine upload --repository testpypi dist/alpha5/*
-```

pyproject.toml

@@ -1,111 +1,3 @@
-[build-system]
-requires = ["hatchling"]
-build-backend = "hatchling.build"
-
-# ------ build-system ------
-
-[project]
-name = "tractor"
-version = "0.1.0a6dev0"
-description = 'structured concurrent `trio`-"actors"'
-authors = [{ name = "Tyler Goodlet", email = "goodboy_foss@protonmail.com" }]
-requires-python = ">= 3.11"
-readme = "docs/README.rst"
-license = "AGPL-3.0-or-later"
-keywords = [
-  "trio",
-  "async",
-  "concurrency",
-  "structured concurrency",
-  "actor model",
-  "distributed",
-  "multiprocessing",
-]
-classifiers = [
-  "Development Status :: 3 - Alpha",
-  "Operating System :: POSIX :: Linux",
-  "Framework :: Trio",
-  "License :: OSI Approved :: GNU Affero General Public License v3 or later (AGPLv3+)",
-  "Programming Language :: Python :: Implementation :: CPython",
-  "Programming Language :: Python :: 3 :: Only",
-  "Programming Language :: Python :: 3.11",
-  "Topic :: System :: Distributed Computing",
-]
-dependencies = [
-  # trio runtime and friends
-  # (poetry) proper range specs,
-  # https://packaging.python.org/en/latest/discussions/install-requires-vs-requirements/#id5
-  # TODO, for 3.13 we must go go `0.27` which means we have to
-  # disable strict egs or port to handling them internally!
-  "trio>0.27",
-  "tricycle>=0.4.1,<0.5",
-  "wrapt>=1.16.0,<2",
-  "colorlog>=6.8.2,<7",
-  # built-in multi-actor `pdb` REPL
-  "pdbp>=1.6,<2", # windows only (from `pdbp`)
-  # typed IPC msging
-  "msgspec>=0.19.0",
-]
-
-# ------ project ------
-
-[dependency-groups]
-dev = [
-  # test suite
-  # TODO: maybe some of these layout choices?
-  # https://docs.pytest.org/en/8.0.x/explanation/goodpractices.html#choosing-a-test-layout-import-rules
-  "pytest>=8.3.5",
-  "pexpect>=4.9.0,<5",
-  # `tractor.devx` tooling
-  "greenback>=1.2.1,<2",
-  "stackscope>=0.2.2,<0.3",
-  "pyperclip>=1.9.0",
-  "prompt-toolkit>=3.0.50",
-  "xonsh>=0.19.2",
-]
-# TODO, add these with sane versions; were originally in
-# `requirements-docs.txt`..
-# docs = [
-#   "sphinx>="
-#   "sphinx_book_theme>="
-# ]
-
-# ------ dependency-groups ------
-
-# ------ dependency-groups ------
-
-[tool.uv.sources]
-# XXX NOTE, only for @goodboy's hacking on `pprint(sort_dicts=False)`
-# for the `pp` alias..
-# pdbp = { path = "../pdbp", editable = true }
-
-# ------ tool.uv.sources ------
-# TODO, distributed (multi-host) extensions
-# linux kernel networking
-# 'pyroute2
-
-# ------ tool.uv.sources ------
-
-[tool.uv]
-# XXX NOTE, prefer the sys python bc apparently the distis from
-# `astral` are built in a way that breaks `pdbp`+`tabcompleter`'s
-# likely due to linking against `libedit` over `readline`..
-# |_https://docs.astral.sh/uv/concepts/python-versions/#managed-python-distributions
-# |_https://gregoryszorc.com/docs/python-build-standalone/main/quirks.html#use-of-libedit-on-linux
-#
-# https://docs.astral.sh/uv/reference/settings/#python-preference
-python-preference = 'system'
-
-# ------ tool.uv ------
-
-[tool.hatch.build.targets.sdist]
-include = ["tractor"]
-
-[tool.hatch.build.targets.wheel]
-include = ["tractor"]
-
-# ------ tool.hatch ------
-
 [tool.towncrier]
 package = "tractor"
 filename = "NEWS.rst"
@@ -115,44 +7,22 @@ title_format = "tractor {version} ({project_date})"
 template = "nooz/_template.rst"
 all_bullets = true
 
-[[tool.towncrier.type]]
+  [[tool.towncrier.type]]
   directory = "feature"
   name = "Features"
   showcontent = true
 
-[[tool.towncrier.type]]
+  [[tool.towncrier.type]]
   directory = "bugfix"
   name = "Bug Fixes"
   showcontent = true
 
-[[tool.towncrier.type]]
+  [[tool.towncrier.type]]
   directory = "doc"
   name = "Improved Documentation"
   showcontent = true
 
-[[tool.towncrier.type]]
+  [[tool.towncrier.type]]
   directory = "trivial"
   name = "Trivial/Internal Changes"
   showcontent = true
-
-# ------ tool.towncrier ------
-
-[tool.pytest.ini_options]
-minversion = '6.0'
-testpaths = [
-  'tests'
-]
-addopts = [
-  # TODO: figure out why this isn't working..
-  '--rootdir=./tests',
-
-  '--import-mode=importlib',
-  # don't show frickin captured logs AGAIN in the report..
-  '--show-capture=no',
-]
-log_cli = false
-# TODO: maybe some of these layout choices?
-# https://docs.pytest.org/en/8.0.x/explanation/goodpractices.html#choosing-a-test-layout-import-rules
-# pythonpath = "src"
-
-# ------ tool.pytest ------

requirements-docs.txt

@@ -0,0 +1,2 @@
+sphinx
+sphinx_book_theme

requirements-test.txt

@@ -0,0 +1,9 @@
+pytest
+pytest-trio
+pytest-timeout
+pdbp
+mypy
+trio_typing
+pexpect
+towncrier
+numpy

ruff.toml

@@ -1,82 +0,0 @@
-# from default `ruff.toml` @
-# https://docs.astral.sh/ruff/configuration/
-
-# Exclude a variety of commonly ignored directories.
-exclude = [
-    ".bzr",
-    ".direnv",
-    ".eggs",
-    ".git",
-    ".git-rewrite",
-    ".hg",
-    ".ipynb_checkpoints",
-    ".mypy_cache",
-    ".nox",
-    ".pants.d",
-    ".pyenv",
-    ".pytest_cache",
-    ".pytype",
-    ".ruff_cache",
-    ".svn",
-    ".tox",
-    ".venv",
-    ".vscode",
-    "__pypackages__",
-    "_build",
-    "buck-out",
-    "build",
-    "dist",
-    "node_modules",
-    "site-packages",
-    "venv",
-]
-
-# Same as Black.
-line-length = 88
-indent-width = 4
-
-# Assume Python 3.9
-target-version = "py311"
-
-[lint]
-# Enable Pyflakes (`F`) and a subset of the pycodestyle (`E`)  codes by default.
-# Unlike Flake8, Ruff doesn't enable pycodestyle warnings (`W`) or
-# McCabe complexity (`C901`) by default.
-select = ["E4", "E7", "E9", "F"]
-ignore = [
-  'E402',  # https://docs.astral.sh/ruff/rules/module-import-not-at-top-of-file/
-]
-
-# Allow fix for all enabled rules (when `--fix`) is provided.
-fixable = ["ALL"]
-unfixable = []
-
-# Allow unused variables when underscore-prefixed.
-# dummy-variable-rgx = "^(_+|(_+[a-zA-Z0-9_]*[a-zA-Z0-9]+?))$"
-
-[format]
-# Use single quotes in `ruff format`.
-quote-style = "single"
-
-# Like Black, indent with spaces, rather than tabs.
-indent-style = "space"
-
-# Like Black, respect magic trailing commas.
-skip-magic-trailing-comma = false
-
-# Like Black, automatically detect the appropriate line ending.
-line-ending = "auto"
-
-# Enable auto-formatting of code examples in docstrings. Markdown,
-# reStructuredText code/literal blocks and doctests are all supported.
-#
-# This is currently disabled by default, but it is planned for this
-# to be opt-out in the future.
-docstring-code-format = false
-
-# Set the line length limit used when formatting code snippets in
-# docstrings.
-#
-# This only has an effect when the `docstring-code-format` setting is
-# enabled.
-docstring-code-line-length = "dynamic"

setup.py

@@ -0,0 +1,101 @@
+#!/usr/bin/env python
+#
+# tractor: structured concurrent "actors".
+#
+# Copyright 2018-eternity Tyler Goodlet.
+
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU Affero General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU Affero General Public License for more details.
+
+# You should have received a copy of the GNU Affero General Public License
+# along with this program.  If not, see <https://www.gnu.org/licenses/>.
+
+from setuptools import setup
+
+with open('docs/README.rst', encoding='utf-8') as f:
+    readme = f.read()
+
+
+setup(
+    name="tractor",
+    version='0.1.0a6dev0',  # alpha zone
+    description='structured concurrent `trio`-"actors"',
+    long_description=readme,
+    license='AGPLv3',
+    author='Tyler Goodlet',
+    maintainer='Tyler Goodlet',
+    maintainer_email='goodboy_foss@protonmail.com',
+    url='https://github.com/goodboy/tractor',
+    platforms=['linux', 'windows'],
+    packages=[
+        'tractor',
+        'tractor.experimental',  # wacky ideas
+        'tractor.trionics',  # trio extensions
+        'tractor.msg',  # lowlevel data types
+        'tractor.devx',  # "dev-experience"
+    ],
+    install_requires=[
+
+        # trio related
+        # proper range spec:
+        # https://packaging.python.org/en/latest/discussions/install-requires-vs-requirements/#id5
+        'trio >= 0.22',
+        'async_generator',
+        'trio_typing',
+        'exceptiongroup',
+
+        # tooling
+        'stackscope',
+        'tricycle',
+        'trio_typing',
+        'colorlog',
+        'wrapt',
+
+        # IPC serialization
+        'msgspec',
+
+        # debug mode REPL
+        'pdbp',
+
+        # TODO: distributed transport using
+        # linux kernel networking
+        # 'pyroute2',
+
+        # pip ref docs on these specs:
+        # https://pip.pypa.io/en/stable/reference/requirement-specifiers/#examples
+        # and pep:
+        # https://peps.python.org/pep-0440/#version-specifiers
+
+    ],
+    tests_require=['pytest'],
+    python_requires=">=3.10",
+    keywords=[
+        'trio',
+        'async',
+        'concurrency',
+        'structured concurrency',
+        'actor model',
+        'distributed',
+        'multiprocessing'
+    ],
+    classifiers=[
+        "Development Status :: 3 - Alpha",
+        "Operating System :: POSIX :: Linux",
+        "Operating System :: Microsoft :: Windows",
+        "Framework :: Trio",
+        "License :: OSI Approved :: GNU Affero General Public License v3 or later (AGPLv3+)",
+        "Programming Language :: Python :: Implementation :: CPython",
+        "Programming Language :: Python :: 3 :: Only",
+        "Programming Language :: Python :: 3.10",
+        "Intended Audience :: Science/Research",
+        "Intended Audience :: Developers",
+        "Topic :: System :: Distributed Computing",
+    ],
+)

tests/conftest.py

@@ -7,19 +7,98 @@ import os
 import random
 import signal
 import platform
+import pathlib
 import time
+import inspect
+from functools import partial, wraps
 
 import pytest
+import trio
 import tractor
-from tractor._testing import (
-    examples_dir as examples_dir,
-    tractor_test as tractor_test,
-    expect_ctxc as expect_ctxc,
-)
 
-# TODO: include wtv plugin(s) we build in `._testing.pytest`?
 pytest_plugins = ['pytester']
 
+
+def tractor_test(fn):
+    """
+    Use:
+
+    @tractor_test
+    async def test_whatever():
+        await ...
+
+    If fixtures:
+
+        - ``reg_addr`` (a socket addr tuple where arbiter is listening)
+        - ``loglevel`` (logging level passed to tractor internals)
+        - ``start_method`` (subprocess spawning backend)
+
+    are defined in the `pytest` fixture space they will be automatically
+    injected to tests declaring these funcargs.
+    """
+    @wraps(fn)
+    def wrapper(
+        *args,
+        loglevel=None,
+        reg_addr=None,
+        start_method: str|None = None,
+        debug_mode: bool = False,
+        **kwargs
+    ):
+        # __tracebackhide__ = True
+
+        # NOTE: inject ant test func declared fixture
+        # names by manually checking!
+        if 'reg_addr' in inspect.signature(fn).parameters:
+            # injects test suite fixture value to test as well
+            # as `run()`
+            kwargs['reg_addr'] = reg_addr
+
+        if 'loglevel' in inspect.signature(fn).parameters:
+            # allows test suites to define a 'loglevel' fixture
+            # that activates the internal logging
+            kwargs['loglevel'] = loglevel
+
+        if start_method is None:
+            if platform.system() == "Windows":
+                start_method = 'trio'
+
+        if 'start_method' in inspect.signature(fn).parameters:
+            # set of subprocess spawning backends
+            kwargs['start_method'] = start_method
+
+        if 'debug_mode' in inspect.signature(fn).parameters:
+            # set of subprocess spawning backends
+            kwargs['debug_mode'] = debug_mode
+
+
+        if kwargs:
+
+            # use explicit root actor start
+            async def _main():
+                async with tractor.open_root_actor(
+                    # **kwargs,
+                    registry_addrs=[reg_addr] if reg_addr else None,
+                    loglevel=loglevel,
+                    start_method=start_method,
+
+                    # TODO: only enable when pytest is passed --pdb
+                    debug_mode=debug_mode,
+
+                ):
+                    await fn(*args, **kwargs)
+
+            main = _main
+
+        else:
+            # use implicit root actor start
+            main = partial(fn, *args, **kwargs)
+
+        return trio.run(main)
+
+    return wrapper
+
+
 # Sending signal.SIGINT on subprocess fails on windows. Use CTRL_* alternatives
 if platform.system() == 'Windows':
     _KILL_SIGNAL = signal.CTRL_BREAK_EVENT
@@ -39,6 +118,23 @@ no_windows = pytest.mark.skipif(
 )
 
 
+def repodir() -> pathlib.Path:
+    '''
+    Return the abspath to the repo directory.
+
+    '''
+    # 2 parents up to step up through tests/<repo_dir>
+    return pathlib.Path(__file__).parent.parent.absolute()
+
+
+def examples_dir() -> pathlib.Path:
+    '''
+    Return the abspath to the examples directory as `pathlib.Path`.
+
+    '''
+    return repodir() / 'examples'
+
+
 def pytest_addoption(parser):
     parser.addoption(
         "--ll",
@@ -75,10 +171,7 @@ def pytest_configure(config):
 
 @pytest.fixture(scope='session')
 def debug_mode(request):
-    debug_mode: bool = request.config.option.tractor_debug_mode
-    # if debug_mode:
-    #     breakpoint()
-    return debug_mode
+    return request.config.option.tractor_debug_mode
 
 
 @pytest.fixture(scope='session', autouse=True)
@@ -95,29 +188,16 @@ def spawn_backend(request) -> str:
     return request.config.option.spawn_backend
 
 
-# @pytest.fixture(scope='function', autouse=True)
-# def debug_enabled(request) -> str:
-#     from tractor import _state
-#     if _state._runtime_vars['_debug_mode']:
-#         breakpoint()
-
 _ci_env: bool = os.environ.get('CI', False)
 
 
 @pytest.fixture(scope='session')
 def ci_env() -> bool:
-    '''
-    Detect CI envoirment.
-
-    '''
+    """Detect CI envoirment.
+    """
     return _ci_env
 
 
-# TODO: also move this to `._testing` for now?
-# -[ ] possibly generalize and re-use for multi-tree spawning
-#    along with the new stuff for multi-addrs in distribute_dis
-#    branch?
-#
 # choose randomly at import time
 _reg_addr: tuple[str, int] = (
     '127.0.0.1',
@@ -159,18 +239,6 @@ def pytest_generate_tests(metafunc):
         metafunc.parametrize("start_method", [spawn_backend], scope='module')
 
 
-# TODO: a way to let test scripts (like from `examples/`)
-# guarantee they won't registry addr collide!
-# @pytest.fixture
-# def open_test_runtime(
-#     reg_addr: tuple,
-# ) -> AsyncContextManager:
-#     return partial(
-#         tractor.open_nursery,
-#         registry_addrs=[reg_addr],
-#     )
-
-
 def sig_prog(proc, sig):
     "Kill the actor-process with ``sig``."
     proc.send_signal(sig)
@@ -183,7 +251,6 @@ def sig_prog(proc, sig):
     assert ret
 
 
-# TODO: factor into @cm and move to `._testing`?
 @pytest.fixture
 def daemon(
     loglevel: str,

tests/devx/conftest.py

@@ -1,243 +0,0 @@
-'''
-`tractor.devx.*` tooling sub-pkg test space.
-
-'''
-import time
-from typing import (
-    Callable,
-)
-
-import pytest
-from pexpect.exceptions import (
-    TIMEOUT,
-)
-from pexpect.spawnbase import SpawnBase
-
-from tractor._testing import (
-    mk_cmd,
-)
-from tractor.devx._debug import (
-    _pause_msg as _pause_msg,
-    _crash_msg as _crash_msg,
-    _repl_fail_msg as _repl_fail_msg,
-    _ctlc_ignore_header as _ctlc_ignore_header,
-)
-from ..conftest import (
-    _ci_env,
-)
-
-
-@pytest.fixture
-def spawn(
-    start_method,
-    testdir: pytest.Pytester,
-    reg_addr: tuple[str, int],
-
-) -> Callable[[str], None]:
-    '''
-    Use the `pexpect` module shipped via `testdir.spawn()` to
-    run an `./examples/..` script by name.
-
-    '''
-    if start_method != 'trio':
-        pytest.skip(
-            '`pexpect` based tests only supported on `trio` backend'
-        )
-
-    def unset_colors():
-        '''
-        Python 3.13 introduced colored tracebacks that break patt
-        matching,
-
-        https://docs.python.org/3/using/cmdline.html#envvar-PYTHON_COLORS
-        https://docs.python.org/3/using/cmdline.html#using-on-controlling-color
-
-        '''
-        import os
-        os.environ['PYTHON_COLORS'] = '0'
-
-    def _spawn(
-        cmd: str,
-        **mkcmd_kwargs,
-    ):
-        unset_colors()
-        return testdir.spawn(
-            cmd=mk_cmd(
-                cmd,
-                **mkcmd_kwargs,
-            ),
-            expect_timeout=3,
-            # preexec_fn=unset_colors,
-            # ^TODO? get `pytest` core to expose underlying
-            # `pexpect.spawn()` stuff?
-        )
-
-    # such that test-dep can pass input script name.
-    return _spawn
-
-
-@pytest.fixture(
-    params=[False, True],
-    ids='ctl-c={}'.format,
-)
-def ctlc(
-    request,
-    ci_env: bool,
-
-) -> bool:
-
-    use_ctlc = request.param
-
-    node = request.node
-    markers = node.own_markers
-    for mark in markers:
-        if mark.name == 'has_nested_actors':
-            pytest.skip(
-                f'Test {node} has nested actors and fails with Ctrl-C.\n'
-                f'The test can sometimes run fine locally but until'
-                ' we solve' 'this issue this CI test will be xfail:\n'
-                'https://github.com/goodboy/tractor/issues/320'
-            )
-
-        if mark.name == 'ctlcs_bish':
-            pytest.skip(
-                f'Test {node} prolly uses something from the stdlib (namely `asyncio`..)\n'
-                f'The test and/or underlying example script can *sometimes* run fine '
-                f'locally but more then likely until the cpython peeps get their sh#$ together, '
-                f'this test will definitely not behave like `trio` under SIGINT..\n'
-            )
-
-    if use_ctlc:
-        # XXX: disable pygments highlighting for auto-tests
-        # since some envs (like actions CI) will struggle
-        # the the added color-char encoding..
-        from tractor.devx._debug import TractorConfig
-        TractorConfig.use_pygements = False
-
-    yield use_ctlc
-
-
-def expect(
-    child,
-
-    # normally a `pdb` prompt by default
-    patt: str,
-
-    **kwargs,
-
-) -> None:
-    '''
-    Expect wrapper that prints last seen console
-    data before failing.
-
-    '''
-    try:
-        child.expect(
-            patt,
-            **kwargs,
-        )
-    except TIMEOUT:
-        before = str(child.before.decode())
-        print(before)
-        raise
-
-
-PROMPT = r"\(Pdb\+\)"
-
-
-def in_prompt_msg(
-    child: SpawnBase,
-    parts: list[str],
-
-    pause_on_false: bool = False,
-    err_on_false: bool = False,
-    print_prompt_on_false: bool = True,
-
-) -> bool:
-    '''
-    Predicate check if (the prompt's) std-streams output has all
-    `str`-parts in it.
-
-    Can be used in test asserts for bulk matching expected
-    log/REPL output for a given `pdb` interact point.
-
-    '''
-    __tracebackhide__: bool = False
-
-    before: str = str(child.before.decode())
-    for part in parts:
-        if part not in before:
-            if pause_on_false:
-                import pdbp
-                pdbp.set_trace()
-
-            if print_prompt_on_false:
-                print(before)
-
-            if err_on_false:
-                raise ValueError(
-                    f'Could not find pattern in `before` output?\n'
-                    f'part: {part!r}\n'
-                )
-            return False
-
-    return True
-
-
-# TODO: todo support terminal color-chars stripping so we can match
-# against call stack frame output from the the 'll' command the like!
-# -[ ] SO answer for stipping ANSI codes: https://stackoverflow.com/a/14693789
-def assert_before(
-    child: SpawnBase,
-    patts: list[str],
-
-    **kwargs,
-
-) -> None:
-    __tracebackhide__: bool = False
-
-    assert in_prompt_msg(
-        child=child,
-        parts=patts,
-
-        # since this is an "assert" helper ;)
-        err_on_false=True,
-        **kwargs
-    )
-
-
-def do_ctlc(
-    child,
-    count: int = 3,
-    delay: float = 0.1,
-    patt: str|None = None,
-
-    # expect repl UX to reprint the prompt after every
-    # ctrl-c send.
-    # XXX: no idea but, in CI this never seems to work even on 3.10 so
-    # needs some further investigation potentially...
-    expect_prompt: bool = not _ci_env,
-
-) -> str|None:
-
-    before: str|None = None
-
-    # make sure ctl-c sends don't do anything but repeat output
-    for _ in range(count):
-        time.sleep(delay)
-        child.sendcontrol('c')
-
-        # TODO: figure out why this makes CI fail..
-        # if you run this test manually it works just fine..
-        if expect_prompt:
-            time.sleep(delay)
-            child.expect(PROMPT)
-            before = str(child.before.decode())
-            time.sleep(delay)
-
-            if patt:
-                # should see the last line on console
-                assert patt in before
-
-    # return the console content up to the final prompt
-    return before

tests/devx/test_pause_from_non_trio.py

@@ -1,381 +0,0 @@
-'''
-That "foreign loop/thread" debug REPL support better ALSO WORK!
-
-Same as `test_native_pause.py`.
-All these tests can be understood (somewhat) by running the
-equivalent `examples/debugging/` scripts manually.
-
-'''
-from contextlib import (
-    contextmanager as cm,
-)
-# from functools import partial
-# import itertools
-import time
-# from typing import (
-#     Iterator,
-# )
-
-import pytest
-from pexpect.exceptions import (
-    TIMEOUT,
-    EOF,
-)
-
-from .conftest import (
-    # _ci_env,
-    do_ctlc,
-    PROMPT,
-    # expect,
-    in_prompt_msg,
-    assert_before,
-    _pause_msg,
-    _crash_msg,
-    _ctlc_ignore_header,
-    # _repl_fail_msg,
-)
-
-@cm
-def maybe_expect_timeout(
-    ctlc: bool = False,
-) -> None:
-    try:
-        yield
-    except TIMEOUT:
-        # breakpoint()
-        if ctlc:
-            pytest.xfail(
-                'Some kinda redic threading SIGINT bug i think?\n'
-                'See the notes in `examples/debugging/sync_bp.py`..\n'
-            )
-        raise
-
-
-@pytest.mark.ctlcs_bish
-def test_pause_from_sync(
-    spawn,
-    ctlc: bool,
-):
-    '''
-    Verify we can use the `pdbp` REPL from sync functions AND from
-    any thread spawned with `trio.to_thread.run_sync()`.
-
-    `examples/debugging/sync_bp.py`
-
-    '''
-    child = spawn('sync_bp')
-
-    # first `sync_pause()` after nurseries open
-    child.expect(PROMPT)
-    assert_before(
-        child,
-        [
-            # pre-prompt line
-            _pause_msg,
-            "<Task '__main__.main'",
-            "('root'",
-        ]
-    )
-    if ctlc:
-        do_ctlc(child)
-        # ^NOTE^ subactor not spawned yet; don't need extra delay.
-
-    child.sendline('c')
-
-    # first `await tractor.pause()` inside `p.open_context()` body
-    child.expect(PROMPT)
-
-    # XXX shouldn't see gb loaded message with PDB loglevel!
-    # assert not in_prompt_msg(
-    #     child,
-    #     ['`greenback` portal opened!'],
-    # )
-    # should be same root task
-    assert_before(
-        child,
-        [
-            _pause_msg,
-            "<Task '__main__.main'",
-            "('root'",
-        ]
-    )
-
-    if ctlc:
-        do_ctlc(
-            child,
-            # NOTE: setting this to 0 (or some other sufficient
-            # small val) can cause the test to fail since the
-            # `subactor` suffers a race where the root/parent
-            # sends an actor-cancel prior to it hitting its pause
-            # point; by def the value is 0.1
-            delay=0.4,
-        )
-
-    # XXX, fwiw without a brief sleep here the SIGINT might actually
-    # trigger "subactor" cancellation by its parent  before the
-    # shield-handler is engaged.
-    #
-    # => similar to the `delay` input to `do_ctlc()` below, setting
-    # this too low can cause the test to fail since the `subactor`
-    # suffers a race where the root/parent sends an actor-cancel
-    # prior to the context task hitting its pause point (and thus
-    # engaging the `sigint_shield()` handler in time); this value
-    # seems be good enuf?
-    time.sleep(0.6)
-
-    # one of the bg thread or subactor should have
-    # `Lock.acquire()`-ed
-    # (NOT both, which will result in REPL clobbering!)
-    attach_patts: dict[str, list[str]] = {
-        'subactor': [
-            "'start_n_sync_pause'",
-            "('subactor'",
-        ],
-        'inline_root_bg_thread': [
-            "<Thread(inline_root_bg_thread",
-            "('root'",
-        ],
-        'start_soon_root_bg_thread': [
-            "<Thread(start_soon_root_bg_thread",
-            "('root'",
-        ],
-    }
-    conts: int = 0  # for debugging below matching logic on failure
-    while attach_patts:
-        child.sendline('c')
-        conts += 1
-        child.expect(PROMPT)
-        before = str(child.before.decode())
-        for key in attach_patts:
-            if key in before:
-                attach_key: str = key
-                expected_patts: str = attach_patts.pop(key)
-                assert_before(
-                    child,
-                    [_pause_msg]
-                    +
-                    expected_patts
-                )
-                break
-        else:
-            pytest.fail(
-                f'No keys found?\n\n'
-                f'{attach_patts.keys()}\n\n'
-                f'{before}\n'
-            )
-
-        # ensure no other task/threads engaged a REPL
-        # at the same time as the one that was detected above.
-        for key, other_patts in attach_patts.copy().items():
-            assert not in_prompt_msg(
-                child,
-                other_patts,
-            )
-
-        if ctlc:
-            do_ctlc(
-                child,
-                patt=attach_key,
-                # NOTE same as comment above
-                delay=0.4,
-            )
-
-    child.sendline('c')
-
-    # XXX TODO, weird threading bug it seems despite the
-    # `abandon_on_cancel: bool` setting to
-    # `trio.to_thread.run_sync()`..
-    with maybe_expect_timeout(
-        ctlc=ctlc,
-    ):
-        child.expect(EOF)
-
-
-def expect_any_of(
-    attach_patts: dict[str, list[str]],
-    child,   # what type?
-    ctlc: bool = False,
-    prompt: str = _ctlc_ignore_header,
-    ctlc_delay: float = .4,
-
-) -> list[str]:
-    '''
-    Receive any of a `list[str]` of patterns provided in
-    `attach_patts`.
-
-    Used to test racing prompts from multiple actors and/or
-    tasks using a common root process' `pdbp` REPL.
-
-    '''
-    assert attach_patts
-
-    child.expect(PROMPT)
-    before = str(child.before.decode())
-
-    for attach_key in attach_patts:
-        if attach_key in before:
-            expected_patts: str = attach_patts.pop(attach_key)
-            assert_before(
-                child,
-                expected_patts
-            )
-            break  # from for
-    else:
-        pytest.fail(
-            f'No keys found?\n\n'
-            f'{attach_patts.keys()}\n\n'
-            f'{before}\n'
-        )
-
-    # ensure no other task/threads engaged a REPL
-    # at the same time as the one that was detected above.
-    for key, other_patts in attach_patts.copy().items():
-        assert not in_prompt_msg(
-            child,
-            other_patts,
-        )
-
-    if ctlc:
-        do_ctlc(
-            child,
-            patt=prompt,
-            # NOTE same as comment above
-            delay=ctlc_delay,
-        )
-
-    return expected_patts
-
-
-@pytest.mark.ctlcs_bish
-def test_sync_pause_from_aio_task(
-    spawn,
-
-    ctlc: bool
-    # ^TODO, fix for `asyncio`!!
-):
-    '''
-    Verify we can use the `pdbp` REPL from an `asyncio.Task` spawned using
-    APIs in `.to_asyncio`.
-
-    `examples/debugging/asycio_bp.py`
-
-    '''
-    child = spawn('asyncio_bp')
-
-    # RACE on whether trio/asyncio task bps first
-    attach_patts: dict[str, list[str]] = {
-
-        # first pause in guest-mode (aka "infecting")
-        # `trio.Task`.
-        'trio-side': [
-            _pause_msg,
-            "<Task 'trio_ctx'",
-            "('aio_daemon'",
-        ],
-
-        # `breakpoint()` from `asyncio.Task`.
-        'asyncio-side': [
-            _pause_msg,
-            "<Task pending name='Task-2' coro=<greenback_shim()",
-            "('aio_daemon'",
-        ],
-    }
-
-    while attach_patts:
-        expect_any_of(
-            attach_patts=attach_patts,
-            child=child,
-            ctlc=ctlc,
-        )
-        child.sendline('c')
-
-    # NOW in race order,
-    # - the asyncio-task will error
-    # - the root-actor parent task will pause
-    #
-    attach_patts: dict[str, list[str]] = {
-
-        # error raised in `asyncio.Task`
-        "raise ValueError('asyncio side error!')": [
-            _crash_msg,
-            "<Task 'trio_ctx'",
-            "@ ('aio_daemon'",
-            "ValueError: asyncio side error!",
-
-            # XXX, we no longer show this frame by default!
-            # 'return await chan.receive()',  # `.to_asyncio` impl internals in tb
-        ],
-
-        # parent-side propagation via actor-nursery/portal
-        # "tractor._exceptions.RemoteActorError: remote task raised a 'ValueError'": [
-        "remote task raised a 'ValueError'": [
-            _crash_msg,
-            "src_uid=('aio_daemon'",
-            "('aio_daemon'",
-        ],
-
-        # a final pause in root-actor
-        "<Task '__main__.main'": [
-            _pause_msg,
-            "<Task '__main__.main'",
-            "('root'",
-        ],
-    }
-    while attach_patts:
-        expect_any_of(
-            attach_patts=attach_patts,
-            child=child,
-            ctlc=ctlc,
-        )
-        child.sendline('c')
-
-    assert not attach_patts
-
-    # final boxed error propagates to root
-    assert_before(
-        child,
-        [
-            _crash_msg,
-            "<Task '__main__.main'",
-            "('root'",
-            "remote task raised a 'ValueError'",
-            "ValueError: asyncio side error!",
-        ]
-    )
-
-    if ctlc:
-        do_ctlc(
-            child,
-            # NOTE: setting this to 0 (or some other sufficient
-            # small val) can cause the test to fail since the
-            # `subactor` suffers a race where the root/parent
-            # sends an actor-cancel prior to it hitting its pause
-            # point; by def the value is 0.1
-            delay=0.4,
-        )
-
-    child.sendline('c')
-    # with maybe_expect_timeout():
-    child.expect(EOF)
-
-
-def test_sync_pause_from_non_greenbacked_aio_task():
-    '''
-    Where the `breakpoint()` caller task is NOT spawned by
-    `tractor.to_asyncio` and thus never activates
-    a `greenback.ensure_portal()` beforehand, presumably bc the task
-    was started by some lib/dep as in often seen in the field.
-
-    Ensure sync pausing works when the pause is in,
-
-    - the root actor running in infected-mode?
-      |_ since we don't need any IPC to acquire the debug lock?
-      |_ is there some way to handle this like the non-main-thread case?
-
-    All other cases need to error out appropriately right?
-
-    - for any subactor we can't avoid needing the repl lock..
-      |_ is there a way to hook into `asyncio.ensure_future(obj)`?
-
-    '''
-    pass

tests/devx/test_tooling.py

@@ -1,172 +0,0 @@
-'''
-That "native" runtime-hackin toolset better be dang useful!
-
-Verify the funtion of a variety of "developer-experience" tools we
-offer from the `.devx` sub-pkg:
-
-- use of the lovely `stackscope` for dumping actor `trio`-task trees
-  during operation and hangs.
-
-TODO:
-- demonstration of `CallerInfo` call stack frame filtering such that
-  for logging and REPL purposes a user sees exactly the layers needed
-  when debugging a problem inside the stack vs. in their app.
-
-'''
-import os
-import signal
-import time
-
-from .conftest import (
-    expect,
-    assert_before,
-    in_prompt_msg,
-    PROMPT,
-    _pause_msg,
-)
-from pexpect.exceptions import (
-    # TIMEOUT,
-    EOF,
-)
-
-
-def test_shield_pause(
-    spawn,
-):
-    '''
-    Verify the `tractor.pause()/.post_mortem()` API works inside an
-    already cancelled `trio.CancelScope` and that you can step to the
-    next checkpoint wherein the cancelled will get raised.
-
-    '''
-    child = spawn(
-        'shield_hang_in_sub'
-    )
-    expect(
-        child,
-        'Yo my child hanging..?',
-    )
-    assert_before(
-        child,
-        [
-            'Entering shield sleep..',
-            'Enabling trace-trees on `SIGUSR1` since `stackscope` is installed @',
-        ]
-    )
-
-    script_pid: int = child.pid
-    print(
-        f'Sending SIGUSR1 to {script_pid}\n'
-        f'(kill -s SIGUSR1 {script_pid})\n'
-    )
-    os.kill(
-        script_pid,
-        signal.SIGUSR1,
-    )
-    time.sleep(0.2)
-    expect(
-        child,
-        # end-of-tree delimiter
-        "end-of-\('root'",
-    )
-    assert_before(
-        child,
-        [
-            # 'Srying to dump `stackscope` tree..',
-            # 'Dumping `stackscope` tree for actor',
-            "('root'",  # uid line
-
-            # TODO!? this used to show?
-            # -[ ] mk reproducable for @oremanj?
-            #
-            # parent block point (non-shielded)
-            # 'await trio.sleep_forever()  # in root',
-        ]
-    )
-    expect(
-        child,
-        # end-of-tree delimiter
-        "end-of-\('hanger'",
-    )
-    assert_before(
-        child,
-        [
-            # relay to the sub should be reported
-            'Relaying `SIGUSR1`[10] to sub-actor',
-
-            "('hanger'",  # uid line
-
-            # TODO!? SEE ABOVE
-            # hanger LOC where it's shield-halted
-            # 'await trio.sleep_forever()  # in subactor',
-        ]
-    )
-
-    # simulate the user sending a ctl-c to the hanging program.
-    # this should result in the terminator kicking in since
-    # the sub is shield blocking and can't respond to SIGINT.
-    os.kill(
-        child.pid,
-        signal.SIGINT,
-    )
-    expect(
-        child,
-        'Shutting down actor runtime',
-        timeout=6,
-    )
-    assert_before(
-        child,
-        [
-            'raise KeyboardInterrupt',
-            # 'Shutting down actor runtime',
-            '#T-800 deployed to collect zombie B0',
-            "'--uid', \"('hanger',",
-        ]
-    )
-
-
-def test_breakpoint_hook_restored(
-    spawn,
-):
-    '''
-    Ensures our actor runtime sets a custom `breakpoint()` hook
-    on open then restores the stdlib's default on close.
-
-    The hook state validation is done via `assert`s inside the
-    invoked script with only `breakpoint()` (not `tractor.pause()`)
-    calls used.
-
-    '''
-    child = spawn('restore_builtin_breakpoint')
-
-    child.expect(PROMPT)
-    assert_before(
-        child,
-        [
-            _pause_msg,
-            "<Task '__main__.main'",
-            "('root'",
-            "first bp, tractor hook set",
-        ]
-    )
-    child.sendline('c')
-    child.expect(PROMPT)
-    assert_before(
-        child,
-        [
-            "last bp, stdlib hook restored",
-        ]
-    )
-
-    # since the stdlib hook was already restored there should be NO
-    # `tractor` `log.pdb()` content from console!
-    assert not in_prompt_msg(
-        child,
-        [
-            _pause_msg,
-            "<Task '__main__.main'",
-            "('root'",
-        ],
-    )
-    child.sendline('c')
-    child.expect(EOF)

tests/test_advanced_faults.py

@@ -4,28 +4,21 @@ cancelacion?..
 
 '''
 from functools import partial
-from types import ModuleType
 
 import pytest
 from _pytest.pathlib import import_path
 import trio
 import tractor
-from tractor._testing import (
+
+from conftest import (
     examples_dir,
-    break_ipc,
 )
 
 
 @pytest.mark.parametrize(
-    'pre_aclose_msgstream',
-    [
-        False,
-        True,
-    ],
-    ids=[
-        'no_msgstream_aclose',
-        'pre_aclose_msgstream',
-    ],
+    'debug_mode',
+    [False, True],
+    ids=['no_debug_mode', 'debug_mode'],
 )
 @pytest.mark.parametrize(
     'ipc_break',
@@ -70,10 +63,8 @@ from tractor._testing import (
 )
 def test_ipc_channel_break_during_stream(
     debug_mode: bool,
-    loglevel: str,
     spawn_backend: str,
-    ipc_break: dict|None,
-    pre_aclose_msgstream: bool,
+    ipc_break: dict | None,
 ):
     '''
     Ensure we can have an IPC channel break its connection during
@@ -90,149 +81,72 @@ def test_ipc_channel_break_during_stream(
 
         # non-`trio` spawners should never hit the hang condition that
         # requires the user to do ctl-c to cancel the actor tree.
-        # expect_final_exc = trio.ClosedResourceError
-        expect_final_exc = tractor.TransportClosed
+        expect_final_exc = trio.ClosedResourceError
 
-    mod: ModuleType = import_path(
-        examples_dir() / 'advanced_faults'
-        / 'ipc_failure_during_stream.py',
+    mod = import_path(
+        examples_dir() / 'advanced_faults' / 'ipc_failure_during_stream.py',
         root=examples_dir(),
-        consider_namespace_packages=False,
     )
 
-    # by def we expect KBI from user after a simulated "hang
-    # period" wherein the user eventually hits ctl-c to kill the
-    # root-actor tree.
-    expect_final_exc: BaseException = KeyboardInterrupt
+    expect_final_exc = KeyboardInterrupt
+
+    # when ONLY the child breaks we expect the parent to get a closed
+    # resource error on the next `MsgStream.receive()` and then fail out
+    # and cancel the child from there.
     if (
-        # only expect EoC if trans is broken on the child side,
-        ipc_break['break_child_ipc_after'] is not False
-        # AND we tell the child to call `MsgStream.aclose()`.
-        and pre_aclose_msgstream
-    ):
-        # expect_final_exc = trio.EndOfChannel
-        # ^XXX NOPE! XXX^ since now `.open_stream()` absorbs this
-        # gracefully!
-        expect_final_exc = KeyboardInterrupt
 
-    # NOTE when ONLY the child breaks or it breaks BEFORE the
-    # parent we expect the parent to get a closed resource error
-    # on the next `MsgStream.receive()` and then fail out and
-    # cancel the child from there.
-    #
-    # ONLY CHILD breaks
-    if (
-        ipc_break['break_child_ipc_after']
-        and
-        ipc_break['break_parent_ipc_after'] is False
-    ):
-        # NOTE: we DO NOT expect this any more since
-        # the child side's channel will be broken silently
-        # and nothing on the parent side will indicate this!
-        # expect_final_exc = trio.ClosedResourceError
-
-        # NOTE: child will send a 'stop' msg before it breaks
-        # the transport channel BUT, that will be absorbed by the
-        # `ctx.open_stream()` block and thus the `.open_context()`
-        # should hang, after which the test script simulates
-        # a user sending ctl-c by raising a KBI.
-        if pre_aclose_msgstream:
-            expect_final_exc = KeyboardInterrupt
-
-            # XXX OLD XXX
-            # if child calls `MsgStream.aclose()` then expect EoC.
-            # ^ XXX not any more ^ since eoc is always absorbed
-            # gracefully and NOT bubbled to the `.open_context()`
-            # block!
-            # expect_final_exc = trio.EndOfChannel
-
-    # BOTH but, CHILD breaks FIRST
-    elif (
-        ipc_break['break_child_ipc_after'] is not False
-        and (
-            ipc_break['break_parent_ipc_after']
-            > ipc_break['break_child_ipc_after']
-        )
-    ):
-        if pre_aclose_msgstream:
-            expect_final_exc = KeyboardInterrupt
-
-    # NOTE when the parent IPC side dies (even if the child does as well
-    # but the child fails BEFORE the parent) we always expect the
-    # IPC layer to raise a closed-resource, NEVER do we expect
-    # a stop msg since the parent-side ctx apis will error out
-    # IMMEDIATELY before the child ever sends any 'stop' msg.
-    #
-    # ONLY PARENT breaks
-    elif (
-        ipc_break['break_parent_ipc_after']
-        and
-        ipc_break['break_child_ipc_after'] is False
-    ):
-        # expect_final_exc = trio.ClosedResourceError
-        expect_final_exc = tractor.TransportClosed
-
-    # BOTH but, PARENT breaks FIRST
-    elif (
-        ipc_break['break_parent_ipc_after'] is not False
-        and (
+        # only child breaks
+        (
             ipc_break['break_child_ipc_after']
-            >
+            and ipc_break['break_parent_ipc_after'] is False
+        )
+
+        # both break but, parent breaks first
+        or (
+            ipc_break['break_child_ipc_after'] is not False
+            and (
+                ipc_break['break_parent_ipc_after']
+                > ipc_break['break_child_ipc_after']
+            )
+        )
+
+    ):
+        expect_final_exc = trio.ClosedResourceError
+
+    # when the parent IPC side dies (even if the child's does as well
+    # but the child fails BEFORE the parent) we expect the channel to be
+    # sent a stop msg from the child at some point which will signal the
+    # parent that the stream has been terminated.
+    # NOTE: when the parent breaks "after" the child you get this same
+    # case as well, the child breaks the IPC channel with a stop msg
+    # before any closure takes place.
+    elif (
+        # only parent breaks
+        (
             ipc_break['break_parent_ipc_after']
+            and ipc_break['break_child_ipc_after'] is False
+        )
+
+        # both break but, child breaks first
+        or (
+            ipc_break['break_parent_ipc_after'] is not False
+            and (
+                ipc_break['break_child_ipc_after']
+                > ipc_break['break_parent_ipc_after']
+            )
         )
     ):
-        # expect_final_exc = trio.ClosedResourceError
-        expect_final_exc = tractor.TransportClosed
+        expect_final_exc = trio.EndOfChannel
 
-    with pytest.raises(
-        expected_exception=(
-            expect_final_exc,
-            ExceptionGroup,
-        ),
-    ) as excinfo:
-        try:
-            trio.run(
-                partial(
-                    mod.main,
-                    debug_mode=debug_mode,
-                    start_method=spawn_backend,
-                    loglevel=loglevel,
-                    pre_close=pre_aclose_msgstream,
-                    **ipc_break,
-                )
+    with pytest.raises(expect_final_exc):
+        trio.run(
+            partial(
+                mod.main,
+                debug_mode=debug_mode,
+                start_method=spawn_backend,
+                **ipc_break,
             )
-        except KeyboardInterrupt as _kbi:
-            kbi = _kbi
-            if expect_final_exc is not KeyboardInterrupt:
-                pytest.fail(
-                    'Rxed unexpected KBI !?\n'
-                    f'{repr(kbi)}'
-                )
-
-            raise
-
-        except tractor.TransportClosed as _tc:
-            tc = _tc
-            if expect_final_exc is KeyboardInterrupt:
-                pytest.fail(
-                    'Unexpected transport failure !?\n'
-                    f'{repr(tc)}'
-                )
-            cause: Exception = tc.__cause__
-            assert (
-                type(cause) is trio.ClosedResourceError
-                and
-                cause.args[0] == 'another task closed this fd'
-            )
-            raise
-
-    # get raw instance from pytest wrapper
-    value = excinfo.value
-    if isinstance(value, ExceptionGroup):
-        excs = value.exceptions
-        assert len(excs) == 1
-        final_exc = excs[0]
-        assert isinstance(final_exc, expect_final_exc)
+        )
 
 
 @tractor.context
@@ -241,50 +155,39 @@ async def break_ipc_after_started(
 ) -> None:
     await ctx.started()
     async with ctx.open_stream() as stream:
-
-        # TODO: make a test which verifies the error
-        # for this, i.e. raises a `MsgTypeError`
-        # await ctx.chan.send(None)
-
-        await break_ipc(
-            stream=stream,
-            pre_close=True,
-        )
+        await stream.aclose()
+        await trio.sleep(0.2)
+        await ctx.chan.send(None)
         print('child broke IPC and terminating')
 
 
 def test_stream_closed_right_after_ipc_break_and_zombie_lord_engages():
     '''
-    Verify that is a subactor's IPC goes down just after bringing up
-    a stream the parent can trigger a SIGINT and the child will be
-    reaped out-of-IPC by the localhost process supervision machinery:
-    aka "zombie lord".
+    Verify that is a subactor's IPC goes down just after bringing up a stream
+    the parent can trigger a SIGINT and the child will be reaped out-of-IPC by
+    the localhost process supervision machinery: aka "zombie lord".
 
     '''
     async def main():
-        with trio.fail_after(3):
-            async with tractor.open_nursery() as an:
-                portal = await an.start_actor(
-                    'ipc_breaker',
-                    enable_modules=[__name__],
-                )
+        async with tractor.open_nursery() as n:
+            portal = await n.start_actor(
+                'ipc_breaker',
+                enable_modules=[__name__],
+            )
 
-                with trio.move_on_after(1):
-                    async with (
-                        portal.open_context(
-                            break_ipc_after_started
-                        ) as (ctx, sent),
-                    ):
-                        async with ctx.open_stream():
-                            await trio.sleep(0.5)
+            with trio.move_on_after(1):
+                async with (
+                    portal.open_context(
+                        break_ipc_after_started
+                    ) as (ctx, sent),
+                ):
+                    async with ctx.open_stream():
+                        await trio.sleep(0.5)
 
-                        print('parent waiting on context')
+                    print('parent waiting on context')
 
-                print(
-                    'parent exited context\n'
-                    'parent raising KBI..\n'
-                )
-                raise KeyboardInterrupt
+            print('parent exited context')
+            raise KeyboardInterrupt
 
     with pytest.raises(KeyboardInterrupt):
         trio.run(main)

tests/test_advanced_streaming.py

@@ -6,7 +6,6 @@ from collections import Counter
 import itertools
 import platform
 
-import pytest
 import trio
 import tractor
 
@@ -144,16 +143,8 @@ def test_dynamic_pub_sub():
 
     try:
         trio.run(main)
-    except (
-        trio.TooSlowError,
-        ExceptionGroup,
-    ) as err:
-        if isinstance(err, ExceptionGroup):
-            for suberr in err.exceptions:
-                if isinstance(suberr, trio.TooSlowError):
-                    break
-            else:
-                pytest.fail('Never got a `TooSlowError` ?')
+    except trio.TooSlowError:
+        pass
 
 
 @tractor.context
@@ -307,15 +298,7 @@ async def inf_streamer(
 
     async with (
         ctx.open_stream() as stream,
-
-        # XXX TODO, INTERESTING CASE!!
-        # - if we don't collapse the eg then the embedded
-        # `trio.EndOfChannel` doesn't propagate directly to the above
-        # .open_stream() parent, resulting in it also raising instead
-        # of gracefully absorbing as normal.. so how to handle?
-        trio.open_nursery(
-            strict_exception_groups=False,
-        ) as tn,
+        trio.open_nursery() as tn,
     ):
         async def close_stream_on_sentinel():
             async for msg in stream:

tests/test_cancellation.py

@@ -8,13 +8,15 @@ import platform
 import time
 from itertools import repeat
 
+from exceptiongroup import (
+    BaseExceptionGroup,
+    ExceptionGroup,
+)
 import pytest
 import trio
 import tractor
-from tractor._testing import (
-    tractor_test,
-)
-from .conftest import no_windows
+
+from conftest import tractor_test, no_windows
 
 
 def is_win():
@@ -77,7 +79,7 @@ def test_remote_error(reg_addr, args_err):
                 # of this actor nursery.
                 await portal.result()
             except tractor.RemoteActorError as err:
-                assert err.boxed_type == errtype
+                assert err.type == errtype
                 print("Look Maa that actor failed hard, hehh")
                 raise
 
@@ -86,33 +88,20 @@ def test_remote_error(reg_addr, args_err):
         with pytest.raises(tractor.RemoteActorError) as excinfo:
             trio.run(main)
 
-        assert excinfo.value.boxed_type == errtype
+        assert excinfo.value.type == errtype
 
     else:
-        # the root task will also error on the `Portal.result()`
-        # call so we expect an error from there AND the child.
-        # |_ tho seems like on new `trio` this doesn't always
-        #    happen?
-        with pytest.raises((
-            BaseExceptionGroup,
-            tractor.RemoteActorError,
-        )) as excinfo:
+        # the root task will also error on the `.result()` call
+        # so we expect an error from there AND the child.
+        with pytest.raises(BaseExceptionGroup) as excinfo:
             trio.run(main)
 
-        # ensure boxed errors are `errtype`
-        err: BaseException = excinfo.value
-        if isinstance(err, BaseExceptionGroup):
-            suberrs: list[BaseException] = err.exceptions
-        else:
-            suberrs: list[BaseException] = [err]
-
-        for exc in suberrs:
-            assert exc.boxed_type == errtype
+        # ensure boxed errors
+        for exc in excinfo.value.exceptions:
+            assert exc.type == errtype
 
 
-def test_multierror(
-    reg_addr: tuple[str, int],
-):
+def test_multierror(reg_addr):
     '''
     Verify we raise a ``BaseExceptionGroup`` out of a nursery where
     more then one actor errors.
@@ -130,7 +119,7 @@ def test_multierror(
             try:
                 await portal2.result()
             except tractor.RemoteActorError as err:
-                assert err.boxed_type is AssertionError
+                assert err.type == AssertionError
                 print("Look Maa that first actor failed hard, hehh")
                 raise
 
@@ -182,7 +171,7 @@ def test_multierror_fast_nursery(reg_addr, start_method, num_subactors, delay):
 
     for exc in exceptions:
         assert isinstance(exc, tractor.RemoteActorError)
-        assert exc.boxed_type is AssertionError
+        assert exc.type == AssertionError
 
 
 async def do_nothing():
@@ -323,7 +312,7 @@ async def test_some_cancels_all(num_actors_and_errs, start_method, loglevel):
                         await portal.run(func, **kwargs)
 
                     except tractor.RemoteActorError as err:
-                        assert err.boxed_type == err_type
+                        assert err.type == err_type
                         # we only expect this first error to propogate
                         # (all other daemons are cancelled before they
                         # can be scheduled)
@@ -342,11 +331,11 @@ async def test_some_cancels_all(num_actors_and_errs, start_method, loglevel):
             assert len(err.exceptions) == num_actors
             for exc in err.exceptions:
                 if isinstance(exc, tractor.RemoteActorError):
-                    assert exc.boxed_type == err_type
+                    assert exc.type == err_type
                 else:
                     assert isinstance(exc, trio.Cancelled)
         elif isinstance(err, tractor.RemoteActorError):
-            assert err.boxed_type == err_type
+            assert err.type == err_type
 
         assert n.cancelled is True
         assert not n._children
@@ -425,7 +414,7 @@ async def test_nested_multierrors(loglevel, start_method):
                     elif isinstance(subexc, tractor.RemoteActorError):
                         # on windows it seems we can't exactly be sure wtf
                         # will happen..
-                        assert subexc.boxed_type in (
+                        assert subexc.type in (
                             tractor.RemoteActorError,
                             trio.Cancelled,
                             BaseExceptionGroup,
@@ -435,7 +424,7 @@ async def test_nested_multierrors(loglevel, start_method):
                         for subsub in subexc.exceptions:
 
                             if subsub in (tractor.RemoteActorError,):
-                                subsub = subsub.boxed_type
+                                subsub = subsub.type
 
                             assert type(subsub) in (
                                 trio.Cancelled,
@@ -450,16 +439,16 @@ async def test_nested_multierrors(loglevel, start_method):
                     # we get back the (sent) cancel signal instead
                     if is_win():
                         if isinstance(subexc, tractor.RemoteActorError):
-                            assert subexc.boxed_type in (
+                            assert subexc.type in (
                                 BaseExceptionGroup,
                                 tractor.RemoteActorError
                             )
                         else:
                             assert isinstance(subexc, BaseExceptionGroup)
                     else:
-                        assert subexc.boxed_type is ExceptionGroup
+                        assert subexc.type is ExceptionGroup
                 else:
-                    assert subexc.boxed_type in (
+                    assert subexc.type in (
                         tractor.RemoteActorError,
                         trio.Cancelled
                     )
@@ -504,9 +493,7 @@ def test_cancel_via_SIGINT_other_task(
     if is_win():  # smh
         timeout += 1
 
-    async def spawn_and_sleep_forever(
-        task_status=trio.TASK_STATUS_IGNORED
-    ):
+    async def spawn_and_sleep_forever(task_status=trio.TASK_STATUS_IGNORED):
         async with tractor.open_nursery() as tn:
             for i in range(3):
                 await tn.run_in_actor(
@@ -519,9 +506,7 @@ def test_cancel_via_SIGINT_other_task(
     async def main():
         # should never timeout since SIGINT should cancel the current program
         with trio.fail_after(timeout):
-            async with trio.open_nursery(
-                strict_exception_groups=False,
-            ) as n:
+            async with trio.open_nursery() as n:
                 await n.start(spawn_and_sleep_forever)
                 if 'mp' in spawn_backend:
                     time.sleep(0.1)
@@ -614,12 +599,6 @@ def test_fast_graceful_cancel_when_spawn_task_in_soft_proc_wait_for_daemon(
                     nurse.start_soon(delayed_kbi)
 
                     await p.run(do_nuthin)
-
-        # need to explicitly re-raise the lone kbi..now
-        except* KeyboardInterrupt as kbi_eg:
-            assert (len(excs := kbi_eg.exceptions) == 1)
-            raise excs[0]
-
         finally:
             duration = time.time() - start
             if duration > timeout:

tests/test_child_manages_service_nursery.py

@@ -6,15 +6,13 @@ sub-sub-actor daemons.
 '''
 from typing import Optional
 import asyncio
-from contextlib import (
-    asynccontextmanager as acm,
-    aclosing,
-)
+from contextlib import asynccontextmanager as acm
 
 import pytest
 import trio
 import tractor
 from tractor import RemoteActorError
+from async_generator import aclosing
 
 
 async def aio_streamer(
@@ -95,8 +93,8 @@ async def trio_main(
 
     # stash a "service nursery" as "actor local" (aka a Python global)
     global _nursery
-    tn = _nursery
-    assert tn
+    n = _nursery
+    assert n
 
     async def consume_stream():
         async with wrapper_mngr() as stream:
@@ -104,10 +102,10 @@ async def trio_main(
                 print(msg)
 
     # run 2 tasks to ensure broadcaster chan use
-    tn.start_soon(consume_stream)
-    tn.start_soon(consume_stream)
+    n.start_soon(consume_stream)
+    n.start_soon(consume_stream)
 
-    tn.start_soon(trio_sleep_and_err)
+    n.start_soon(trio_sleep_and_err)
 
     await trio.sleep_forever()
 
@@ -117,10 +115,8 @@ async def open_actor_local_nursery(
     ctx: tractor.Context,
 ):
     global _nursery
-    async with trio.open_nursery(
-        strict_exception_groups=False,
-    ) as tn:
-        _nursery = tn
+    async with trio.open_nursery() as n:
+        _nursery = n
         await ctx.started()
         await trio.sleep(10)
         # await trio.sleep(1)
@@ -134,7 +130,7 @@ async def open_actor_local_nursery(
         # never yields back.. aka a scenario where the
         # ``tractor.context`` task IS NOT in the service n's cancel
         # scope.
-        tn.cancel_scope.cancel()
+        n.cancel_scope.cancel()
 
 
 @pytest.mark.parametrize(
@@ -159,7 +155,7 @@ def test_actor_managed_trio_nursery_task_error_cancels_aio(
         async with tractor.open_nursery() as n:
             p = await n.start_actor(
                 'nursery_mngr',
-                infect_asyncio=asyncio_mode,  # TODO, is this enabling debug mode?
+                infect_asyncio=asyncio_mode,
                 enable_modules=[__name__],
             )
             async with (
@@ -173,4 +169,4 @@ def test_actor_managed_trio_nursery_task_error_cancels_aio(
 
     # verify boxed error
     err = excinfo.value
-    assert err.boxed_type is NameError
+    assert isinstance(err.type(), NameError)

tests/test_clustering.py

@@ -5,7 +5,9 @@ import trio
 import tractor
 from tractor import open_actor_cluster
 from tractor.trionics import gather_contexts
-from tractor._testing import tractor_test
+
+from conftest import tractor_test
+
 
 MESSAGE = 'tractoring at full speed'
 

tests/test_context_stream_semantics.py

@@ -5,8 +5,8 @@ Verify the we raise errors when streams are opened prior to
 sync-opening a ``tractor.Context`` beforehand.
 
 '''
+from contextlib import asynccontextmanager as acm
 from itertools import count
-import math
 import platform
 from pprint import pformat
 from typing import (
@@ -25,12 +25,8 @@ from tractor._exceptions import (
     StreamOverrun,
     ContextCancelled,
 )
-from tractor._state import current_ipc_ctx
 
-from tractor._testing import (
-    tractor_test,
-    expect_ctxc,
-)
+from conftest import tractor_test
 
 # ``Context`` semantics are as follows,
 #  ------------------------------------
@@ -38,9 +34,9 @@ from tractor._testing import (
 # - standard setup/teardown:
 #   ``Portal.open_context()`` starts a new
 #   remote task context in another actor. The target actor's task must
-#   call ``Context.started()`` to unblock this entry on the parent side.
-#   the child task executes until complete and returns a final value
-#   which is delivered to the parent side and retreived via
+#   call ``Context.started()`` to unblock this entry on the caller side.
+#   the callee task executes until complete and returns a final value
+#   which is delivered to the caller side and retreived via
 #   ``Context.result()``.
 
 # - cancel termination:
@@ -145,8 +141,6 @@ async def simple_setup_teardown(
     global _state
     _state = True
 
-    assert current_ipc_ctx() is ctx
-
     # signal to parent that we're up
     await ctx.started(data + 1)
 
@@ -170,9 +164,9 @@ async def assert_state(value: bool):
     [False, ValueError, KeyboardInterrupt],
 )
 @pytest.mark.parametrize(
-    'child_blocks_forever',
+    'callee_blocks_forever',
     [False, True],
-    ids=lambda item: f'child_blocks_forever={item}'
+    ids=lambda item: f'callee_blocks_forever={item}'
 )
 @pytest.mark.parametrize(
     'pointlessly_open_stream',
@@ -181,7 +175,7 @@ async def assert_state(value: bool):
 )
 def test_simple_context(
     error_parent,
-    child_blocks_forever,
+    callee_blocks_forever,
     pointlessly_open_stream,
     debug_mode: bool,
 ):
@@ -200,17 +194,15 @@ def test_simple_context(
                 )
 
                 try:
-                    async with (
-                        portal.open_context(
-                            simple_setup_teardown,
-                            data=10,
-                            block_forever=child_blocks_forever,
-                        ) as (ctx, sent),
-                    ):
-                        assert current_ipc_ctx() is ctx
+                    async with portal.open_context(
+                        simple_setup_teardown,
+                        data=10,
+                        block_forever=callee_blocks_forever,
+                    ) as (ctx, sent):
+
                         assert sent == 11
 
-                        if child_blocks_forever:
+                        if callee_blocks_forever:
                             await portal.run(assert_state, value=True)
                         else:
                             assert await ctx.result() == 'yo'
@@ -220,7 +212,7 @@ def test_simple_context(
                                 if error_parent:
                                     raise error_parent
 
-                                if child_blocks_forever:
+                                if callee_blocks_forever:
                                     await ctx.cancel()
                                 else:
                                     # in this case the stream will send a
@@ -250,18 +242,29 @@ def test_simple_context(
             trio.run(main)
         except error_parent:
             pass
-        except BaseExceptionGroup as beg:
+        except trio.MultiError as me:
             # XXX: on windows it seems we may have to expect the group error
             from tractor._exceptions import is_multi_cancelled
-            assert is_multi_cancelled(beg)
+            assert is_multi_cancelled(me)
     else:
         trio.run(main)
 
 
+@acm
+async def expect_ctxc(yay: bool) -> None:
+    if yay:
+        try:
+            yield
+        except ContextCancelled:
+            return
+    else:
+        yield
+
+
 @pytest.mark.parametrize(
-    'child_returns_early',
+    'callee_returns_early',
     [True, False],
-    ids=lambda item: f'child_returns_early={item}'
+    ids=lambda item: f'callee_returns_early={item}'
 )
 @pytest.mark.parametrize(
     'cancel_method',
@@ -273,14 +276,14 @@ def test_simple_context(
     [True, False],
     ids=lambda item: f'chk_ctx_result_before_exit={item}'
 )
-def test_parent_cancels(
+def test_caller_cancels(
     cancel_method: str,
     chk_ctx_result_before_exit: bool,
-    child_returns_early: bool,
+    callee_returns_early: bool,
     debug_mode: bool,
 ):
     '''
-    Verify that when the opening side of a context (aka the parent)
+    Verify that when the opening side of a context (aka the caller)
     cancels that context, the ctx does not raise a cancelled when
     either calling `.result()` or on context exit.
 
@@ -290,20 +293,16 @@ def test_parent_cancels(
     ) -> None:
         actor: Actor = current_actor()
         uid: tuple = actor.uid
-        _ctxc: ContextCancelled|None = None
 
         if (
             cancel_method == 'portal'
-            and not child_returns_early
+            and not callee_returns_early
         ):
             try:
                 res = await ctx.result()
                 assert 0, 'Portal cancel should raise!'
 
             except ContextCancelled as ctxc:
-                # with trio.CancelScope(shield=True):
-                #     await tractor.pause()
-                _ctxc = ctxc
                 assert ctx.chan._cancel_called
                 assert ctxc.canceller == uid
                 assert ctxc is ctx.maybe_error
@@ -312,13 +311,10 @@ def test_parent_cancels(
         # case since self-cancellation should swallow the ctxc
         # silently!
         else:
-            try:
-                res = await ctx.result()
-            except ContextCancelled as ctxc:
-                pytest.fail(f'should not have raised ctxc\n{ctxc}')
+            res = await ctx.result()
 
         # we actually get a result
-        if child_returns_early:
+        if callee_returns_early:
             assert res == 'yo'
             assert ctx.outcome is res
             assert ctx.maybe_error is None
@@ -346,10 +342,6 @@ def test_parent_cancels(
                 # await tractor.pause()
                 # assert ctx._local_error is None
 
-        # TODO: don't need this right?
-        # if _ctxc:
-        #     raise _ctxc
-
 
     async def main():
 
@@ -360,45 +352,29 @@ def test_parent_cancels(
                 'simple_context',
                 enable_modules=[__name__],
             )
-            timeout: float = (
-                0.5
-                if not child_returns_early
-                else 2
-            )
+            timeout = 0.5 if not callee_returns_early else 2
             with trio.fail_after(timeout):
                 async with (
-                    expect_ctxc(
-                        yay=(
-                            not child_returns_early
-                            and cancel_method == 'portal'
-                        )
-                    ),
+
+                    expect_ctxc(yay=cancel_method == 'portal'),
 
                     portal.open_context(
                         simple_setup_teardown,
                         data=10,
-                        block_forever=not child_returns_early,
+                        block_forever=not callee_returns_early,
                     ) as (ctx, sent),
                 ):
 
-                    if child_returns_early:
+                    if callee_returns_early:
                         # ensure we block long enough before sending
-                        # a cancel such that the child has already
+                        # a cancel such that the callee has already
                         # returned it's result.
                         await trio.sleep(0.5)
 
                     if cancel_method == 'ctx':
-                        print('cancelling with `Context.cancel()`')
                         await ctx.cancel()
-
-                    elif cancel_method == 'portal':
-                        print('cancelling with `Portal.cancel_actor()`')
-                        await portal.cancel_actor()
-
                     else:
-                        pytest.fail(
-                            f'Unknown `cancel_method={cancel_method} ?'
-                        )
+                        await portal.cancel_actor()
 
                     if chk_ctx_result_before_exit:
                         await check_canceller(ctx)
@@ -409,32 +385,25 @@ def test_parent_cancels(
             if cancel_method != 'portal':
                 await portal.cancel_actor()
 
-            # XXX NOTE XXX: non-normal yet purposeful
-            # test-specific ctxc suppression is implemented!
-            #
-            # WHY: the `.cancel_actor()` case (cancel_method='portal')
-            # will cause both:
-            #  * the `ctx.result()` inside `.open_context().__aexit__()`
-            #  * AND the `ctx.result()` inside `check_canceller()`
-            # to raise ctxc.
-            #
-            #   which should in turn cause `ctx._scope` to
+            # since the `.cancel_actor()` call just above
+            # will cause the `.open_context().__aexit__()` raise
+            # a ctxc which should in turn cause `ctx._scope` to
             # catch any cancellation?
             if (
-                not child_returns_early
-                and cancel_method != 'portal'
+                not callee_returns_early
+                and cancel_method == 'portal'
             ):
-                assert not ctx._scope.cancelled_caught
+                assert ctx._scope.cancelled_caught
 
     trio.run(main)
 
 
 # basic stream terminations:
-# - child context closes without using stream
-# - parent context closes without using stream
-# - parent context calls `Context.cancel()` while streaming
-#   is ongoing resulting in child being cancelled
-# - child calls `Context.cancel()` while streaming and parent
+# - callee context closes without using stream
+# - caller context closes without using stream
+# - caller context calls `Context.cancel()` while streaming
+#   is ongoing resulting in callee being cancelled
+# - callee calls `Context.cancel()` while streaming and caller
 #   sees stream terminated in `RemoteActorError`
 
 # TODO: future possible features
@@ -443,6 +412,7 @@ def test_parent_cancels(
 
 @tractor.context
 async def close_ctx_immediately(
+
     ctx: Context,
 
 ) -> None:
@@ -453,24 +423,13 @@ async def close_ctx_immediately(
     async with ctx.open_stream():
         pass
 
-    print('child returning!')
 
-
-@pytest.mark.parametrize(
-    'parent_send_before_receive',
-    [
-        False,
-        True,
-    ],
-    ids=lambda item: f'child_send_before_receive={item}'
-)
 @tractor_test
-async def test_child_exits_ctx_after_stream_open(
+async def test_callee_closes_ctx_after_stream_open(
     debug_mode: bool,
-    parent_send_before_receive: bool,
 ):
     '''
-    child context closes without using stream.
+    callee context closes without using stream.
 
     This should result in a msg sequence
     |_<root>_
@@ -484,9 +443,6 @@ async def test_child_exits_ctx_after_stream_open(
     => {'stop': True, 'cid': <str>}
 
     '''
-    timeout: float = (
-        0.5 if not debug_mode else 999
-    )
     async with tractor.open_nursery(
         debug_mode=debug_mode,
     ) as an:
@@ -495,7 +451,7 @@ async def test_child_exits_ctx_after_stream_open(
             enable_modules=[__name__],
         )
 
-        with trio.fail_after(timeout):
+        with trio.fail_after(0.5):
             async with portal.open_context(
                 close_ctx_immediately,
 
@@ -507,56 +463,41 @@ async def test_child_exits_ctx_after_stream_open(
 
                 with trio.fail_after(0.4):
                     async with ctx.open_stream() as stream:
-                        if parent_send_before_receive:
-                            print('sending first msg from parent!')
-                            await stream.send('yo')
 
                         # should fall through since ``StopAsyncIteration``
                         # should be raised through translation of
                         # a ``trio.EndOfChannel`` by
                         # ``trio.abc.ReceiveChannel.__anext__()``
-                        msg = 10
-                        async for msg in stream:
+                        async for _ in stream:
                             # trigger failure if we DO NOT
                             # get an EOC!
                             assert 0
                         else:
-                            # never should get anythinig new from
-                            # the underlying stream
-                            assert msg == 10
 
                             # verify stream is now closed
                             try:
                                 with trio.fail_after(0.3):
-                                    print('parent trying to `.receive()` on EoC stream!')
                                     await stream.receive()
-                                    assert 0, 'should have raised eoc!?'
                             except trio.EndOfChannel:
-                                print('parent got EoC as expected!')
                                 pass
-                                # raise
 
                 # TODO: should be just raise the closed resource err
                 # directly here to enforce not allowing a re-open
                 # of a stream to the context (at least until a time of
                 # if/when we decide that's a good idea?)
                 try:
-                    with trio.fail_after(timeout):
+                    with trio.fail_after(0.5):
                         async with ctx.open_stream() as stream:
                             pass
                 except trio.ClosedResourceError:
                     pass
 
-                # if ctx._rx_chan._state.data:
-                #     await tractor.pause()
-
         await portal.cancel_actor()
 
 
 @tractor.context
 async def expect_cancelled(
     ctx: Context,
-    send_before_receive: bool = False,
 
 ) -> None:
     global _state
@@ -566,79 +507,35 @@ async def expect_cancelled(
 
     try:
         async with ctx.open_stream() as stream:
-
-            if send_before_receive:
-                await stream.send('yo')
-
             async for msg in stream:
                 await stream.send(msg)  # echo server
 
     except trio.Cancelled:
-
-        # on ctx.cancel() the internal RPC scope is cancelled but
-        # never caught until the func exits.
-        assert ctx._scope.cancel_called
-        assert not ctx._scope.cancelled_caught
-
-        # should be the RPC cmd request for `._cancel_task()`
-        assert ctx._cancel_msg
-        # which, has not yet resolved to an error outcome
-        # since this rpc func has not yet exited.
-        assert not ctx.maybe_error
-        assert not ctx._final_result_is_set()
-
-        # debug REPL if needed
-        # with trio.CancelScope(shield=True):
-        #     await tractor.pause()
-
         # expected case
         _state = False
         raise
 
     else:
-        assert 0, "child wasn't cancelled !?"
+        assert 0, "callee wasn't cancelled !?"
 
 
-@pytest.mark.parametrize(
-    'child_send_before_receive',
-    [
-        False,
-        True,
-    ],
-    ids=lambda item: f'child_send_before_receive={item}'
-)
-@pytest.mark.parametrize(
-    'rent_wait_for_msg',
-    [
-        False,
-        True,
-    ],
-    ids=lambda item: f'rent_wait_for_msg={item}'
-)
 @pytest.mark.parametrize(
     'use_ctx_cancel_method',
-    [
-        False,
-        'pre_stream',
-        'post_stream_open',
-        'post_stream_close',
-    ],
-    ids=lambda item: f'use_ctx_cancel_method={item}'
+    [False, True],
 )
 @tractor_test
-async def test_parent_exits_ctx_after_child_enters_stream(
-    use_ctx_cancel_method: bool|str,
+async def test_caller_closes_ctx_after_callee_opens_stream(
+    use_ctx_cancel_method: bool,
     debug_mode: bool,
-    rent_wait_for_msg: bool,
-    child_send_before_receive: bool,
 ):
     '''
-    Parent-side of IPC context closes without sending on `MsgStream`.
+    caller context closes without using/opening stream
 
     '''
     async with tractor.open_nursery(
         debug_mode=debug_mode,
     ) as an:
+
         root: Actor = current_actor()
         portal = await an.start_actor(
             'ctx_cancelled',
@@ -647,52 +544,41 @@ async def test_parent_exits_ctx_after_child_enters_stream(
 
         async with portal.open_context(
             expect_cancelled,
-            send_before_receive=child_send_before_receive,
         ) as (ctx, sent):
             assert sent is None
 
             await portal.run(assert_state, value=True)
 
             # call `ctx.cancel()` explicitly
-            if use_ctx_cancel_method == 'pre_stream':
+            if use_ctx_cancel_method:
                 await ctx.cancel()
 
                 # NOTE: means the local side `ctx._scope` will
                 # have been cancelled by an ctxc ack and thus
                 # `._scope.cancelled_caught` should be set.
-                async with (
-                    expect_ctxc(
-                        # XXX: the cause is US since we call
-                        # `Context.cancel()` just above!
-                        yay=True,
-
-                        # XXX: must be propagated to __aexit__
-                        # and should be silently absorbed there
-                        # since we called `.cancel()` just above ;)
-                        reraise=True,
-                    ) as maybe_ctxc,
-                ):
+                try:
                     async with ctx.open_stream() as stream:
+                        async for msg in stream:
+                            pass
 
-                        if rent_wait_for_msg:
-                            async for msg in stream:
-                                print(f'PARENT rx: {msg!r}\n')
-                                break
+                except tractor.ContextCancelled as ctxc:
+                    # XXX: the cause is US since we call
+                    # `Context.cancel()` just above!
+                    assert (
+                        ctxc.canceller
+                        ==
+                        current_actor().uid
+                        ==
+                        root.uid
+                    )
 
-                        if use_ctx_cancel_method == 'post_stream_open':
-                            await ctx.cancel()
+                    # XXX: must be propagated to __aexit__
+                    # and should be silently absorbed there
+                    # since we called `.cancel()` just above ;)
+                    raise
 
-                    if use_ctx_cancel_method == 'post_stream_close':
-                        await ctx.cancel()
-
-                ctxc: tractor.ContextCancelled = maybe_ctxc.value
-                assert (
-                    ctxc.canceller
-                    ==
-                    current_actor().uid
-                    ==
-                    root.uid
-                )
+                else:
+                    assert 0, "Should have context cancelled?"
 
                 # channel should still be up
                 assert portal.channel.connected()
@@ -703,28 +589,21 @@ async def test_parent_exits_ctx_after_child_enters_stream(
                     value=False,
                 )
 
-            # XXX CHILD-BLOCKS case, we SHOULD NOT exit from the
-            # `.open_context()` before the child has returned,
-            # errored or been cancelled!
             else:
                 try:
-                    with trio.fail_after(
-                        0.5  # if not debug_mode else 999
-                    ):
-                        res = await ctx.wait_for_result()
-                        assert res is not tractor._context.Unresolved
+                    with trio.fail_after(0.2):
+                        await ctx.result()
                         assert 0, "Callee should have blocked!?"
+
                 except trio.TooSlowError:
-                    # NO-OP -> since already triggered by
-                    # `trio.fail_after()` above!
+                    # NO-OP -> since already called above
                     await ctx.cancel()
 
         # NOTE: local scope should have absorbed the cancellation since
         # in this case we call `ctx.cancel()` and the local
-        # `._scope` does not get `.cancel_called` and thus
-        # `.cancelled_caught` neither will ever bet set.
+        # `._scope` gets `.cancel_called` on the ctxc ack.
         if use_ctx_cancel_method:
-            assert not ctx._scope.cancelled_caught
+            assert ctx._scope.cancelled_caught
 
         # rxed ctxc response from far end
         assert ctx.cancel_acked
@@ -756,7 +635,7 @@ async def test_parent_exits_ctx_after_child_enters_stream(
 
 
 @tractor_test
-async def test_multitask_parent_cancels_from_nonroot_task(
+async def test_multitask_caller_cancels_from_nonroot_task(
     debug_mode: bool,
 ):
     async with tractor.open_nursery(
@@ -808,6 +687,7 @@ async def test_multitask_parent_cancels_from_nonroot_task(
 
 @tractor.context
 async def cancel_self(
+
     ctx: Context,
 
 ) -> None:
@@ -847,11 +727,11 @@ async def cancel_self(
 
 
 @tractor_test
-async def test_child_cancels_before_started(
+async def test_callee_cancels_before_started(
     debug_mode: bool,
 ):
     '''
-    Callee calls `Context.cancel()` while streaming and parent
+    Callee calls `Context.cancel()` while streaming and caller
     sees stream terminated in `ContextCancelled`.
 
     '''
@@ -872,12 +752,10 @@ async def test_child_cancels_before_started(
 
         # raises a special cancel signal
         except tractor.ContextCancelled as ce:
-            _ce = ce  # for debug on crash
-            ce.boxed_type == trio.Cancelled
+            ce.type == trio.Cancelled
 
             # the traceback should be informative
-            assert 'itself' in ce.tb_str
-            assert ce.tb_str == ce.msgdata['tb_str']
+            assert 'itself' in ce.msgdata['tb_str']
 
         # teardown the actor
         await portal.cancel_actor()
@@ -898,13 +776,14 @@ async def never_open_stream(
 
 
 @tractor.context
-async def keep_sending_from_child(
+async def keep_sending_from_callee(
+
     ctx:  Context,
     msg_buffer_size: int|None = None,
 
 ) -> None:
     '''
-    Send endlessly on the child stream.
+    Send endlessly on the calleee stream.
 
     '''
     await ctx.started()
@@ -912,7 +791,7 @@ async def keep_sending_from_child(
         msg_buffer_size=msg_buffer_size,
     ) as stream:
         for msg in count():
-            print(f'child sending {msg}')
+            print(f'callee sending {msg}')
             await stream.send(msg)
             await trio.sleep(0.01)
 
@@ -920,13 +799,10 @@ async def keep_sending_from_child(
 @pytest.mark.parametrize(
     'overrun_by',
     [
-        ('parent', 1, never_open_stream),
-        ('child', 0, keep_sending_from_child),
+        ('caller', 1, never_open_stream),
+        ('callee', 0, keep_sending_from_callee),
     ],
-    ids=[
-         ('parent_1buf_never_open_stream'),
-         ('child_0buf_keep_sending_from_child'),
-    ]
+    ids='overrun_condition={}'.format,
 )
 def test_one_end_stream_not_opened(
     overrun_by: tuple[str, int, Callable],
@@ -950,50 +826,50 @@ def test_one_end_stream_not_opened(
                 enable_modules=[__name__],
             )
 
-            with trio.fail_after(1):
-                async with portal.open_context(
-                    entrypoint,
-                ) as (ctx, sent):
-                    assert sent is None
+            async with portal.open_context(
+                entrypoint,
+            ) as (ctx, sent):
+                assert sent is None
 
-                    if 'parent' in overrunner:
-                        async with ctx.open_stream() as stream:
+                if 'caller' in overrunner:
 
-                            # itersend +1 msg more then the buffer size
-                            # to cause the most basic overrun.
-                            for i in range(buf_size):
-                                print(f'sending {i}')
-                                await stream.send(i)
+                    async with ctx.open_stream() as stream:
 
-                            else:
-                                # expect overrun error to be relayed back
-                                # and this sleep interrupted
-                                await trio.sleep_forever()
+                        # itersend +1 msg more then the buffer size
+                        # to cause the most basic overrun.
+                        for i in range(buf_size):
+                            print(f'sending {i}')
+                            await stream.send(i)
 
-                    else:
-                        # child overruns parent case so we do nothing here
-                        await trio.sleep_forever()
+                        else:
+                            # expect overrun error to be relayed back
+                            # and this sleep interrupted
+                            await trio.sleep_forever()
+
+                else:
+                    # callee overruns caller case so we do nothing here
+                    await trio.sleep_forever()
 
             await portal.cancel_actor()
 
     # 2 overrun cases and the no overrun case (which pushes right up to
     # the msg limit)
     if (
-        overrunner == 'parent'
+        overrunner == 'caller'
     ):
         with pytest.raises(tractor.RemoteActorError) as excinfo:
             trio.run(main)
 
-        assert excinfo.value.boxed_type == StreamOverrun
+        assert excinfo.value.type == StreamOverrun
 
-    elif overrunner == 'child':
+    elif overrunner == 'callee':
         with pytest.raises(tractor.RemoteActorError) as excinfo:
             trio.run(main)
 
         # TODO: embedded remote errors so that we can verify the source
-        # error? the child delivers an error which is an overrun
+        # error? the callee delivers an error which is an overrun
         # wrapped in a remote actor error.
-        assert excinfo.value.boxed_type == tractor.RemoteActorError
+        assert excinfo.value.type == tractor.RemoteActorError
 
     else:
         trio.run(main)
@@ -1001,7 +877,8 @@ def test_one_end_stream_not_opened(
 
 @tractor.context
 async def echo_back_sequence(
-    ctx: Context,
+
+    ctx:  Context,
     seq: list[int],
     wait_for_cancel: bool,
     allow_overruns_side: str,
@@ -1010,12 +887,12 @@ async def echo_back_sequence(
 
 ) -> None:
     '''
-    Send endlessly on the child stream using a small buffer size
+    Send endlessly on the calleee stream using a small buffer size
     setting on the contex to simulate backlogging that would normally
     cause overruns.
 
     '''
-    # NOTE: ensure that if the parent is expecting to cancel this task
+    # NOTE: ensure that if the caller is expecting to cancel this task
     # that we stay echoing much longer then they are so we don't
     # return early instead of receive the cancel msg.
     total_batches: int = (
@@ -1024,7 +901,7 @@ async def echo_back_sequence(
     )
 
     await ctx.started()
-    # await tractor.pause()
+    # await tractor.breakpoint()
     async with ctx.open_stream(
         msg_buffer_size=msg_buffer_size,
 
@@ -1065,18 +942,18 @@ async def echo_back_sequence(
                 if be_slow:
                     await trio.sleep(0.05)
 
-                print('child waiting on next')
+                print('callee waiting on next')
 
-            print(f'child echoing back latest batch\n{batch}')
+            print(f'callee echoing back latest batch\n{batch}')
             for msg in batch:
-                print(f'child sending msg\n{msg}')
+                print(f'callee sending msg\n{msg}')
                 await stream.send(msg)
 
     try:
         return 'yo'
     finally:
         print(
-            'exiting child with context:\n'
+            'exiting callee with context:\n'
             f'{pformat(ctx)}\n'
         )
 
@@ -1104,8 +981,6 @@ def test_maybe_allow_overruns_stream(
     cancel_ctx: bool,
     slow_side: str,
     allow_overruns_side: str,
-
-    # conftest wide
     loglevel: str,
     debug_mode: bool,
 ):
@@ -1130,68 +1005,59 @@ def test_maybe_allow_overruns_stream(
             debug_mode=debug_mode,
         ) as an:
             portal = await an.start_actor(
-                'child_sends_forever',
+                'callee_sends_forever',
                 enable_modules=[__name__],
                 loglevel=loglevel,
                 debug_mode=debug_mode,
             )
+            seq = list(range(10))
+            async with portal.open_context(
+                echo_back_sequence,
+                seq=seq,
+                wait_for_cancel=cancel_ctx,
+                be_slow=(slow_side == 'child'),
+                allow_overruns_side=allow_overruns_side,
 
-            # stream-sequence batch info with send delay to determine
-            # approx timeout determining whether test has hung.
-            total_batches: int = 2
-            num_items: int = 10
-            seq = list(range(num_items))
-            parent_send_delay: float = 0.16
-            timeout: float = math.ceil(
-                total_batches * num_items * parent_send_delay
-            )
-            with trio.fail_after(timeout):
-                async with portal.open_context(
-                    echo_back_sequence,
-                    seq=seq,
-                    wait_for_cancel=cancel_ctx,
-                    be_slow=(slow_side == 'child'),
-                    allow_overruns_side=allow_overruns_side,
+            ) as (ctx, sent):
+                assert sent is None
 
-                ) as (ctx, sent):
-                    assert sent is None
+                async with ctx.open_stream(
+                    msg_buffer_size=1 if slow_side == 'parent' else None,
+                    allow_overruns=(allow_overruns_side in {'parent', 'both'}),
+                ) as stream:
 
-                    async with ctx.open_stream(
-                        msg_buffer_size=1 if slow_side == 'parent' else None,
-                        allow_overruns=(allow_overruns_side in {'parent', 'both'}),
-                    ) as stream:
+                    total_batches: int = 2
+                    for _ in range(total_batches):
+                        for msg in seq:
+                            # print(f'root tx {msg}')
+                            await stream.send(msg)
+                            if slow_side == 'parent':
+                                # NOTE: we make the parent slightly
+                                # slower, when it is slow, to make sure
+                                # that in the overruns everywhere case
+                                await trio.sleep(0.16)
 
-                        for _ in range(total_batches):
-                            for msg in seq:
-                                # print(f'root tx {msg}')
-                                await stream.send(msg)
-                                if slow_side == 'parent':
-                                    # NOTE: we make the parent slightly
-                                    # slower, when it is slow, to make sure
-                                    # that in the overruns everywhere case
-                                    await trio.sleep(parent_send_delay)
-
-                            batch = []
-                            async for msg in stream:
-                                print(f'root rx {msg}')
-                                batch.append(msg)
-                                if batch == seq:
-                                    break
-
-                    if cancel_ctx:
-                        # cancel the remote task
-                        print('Requesting `ctx.cancel()` in parent!')
-                        await ctx.cancel()
-
-                res: str|ContextCancelled = await ctx.result()
+                        batch = []
+                        async for msg in stream:
+                            print(f'root rx {msg}')
+                            batch.append(msg)
+                            if batch == seq:
+                                break
 
                 if cancel_ctx:
-                    assert isinstance(res, ContextCancelled)
-                    assert tuple(res.canceller) == current_actor().uid
+                    # cancel the remote task
+                    print('Requesting `ctx.cancel()` in parent!')
+                    await ctx.cancel()
 
-                else:
-                    print(f'RX ROOT SIDE RESULT {res}')
-                    assert res == 'yo'
+            res: str|ContextCancelled = await ctx.result()
+
+            if cancel_ctx:
+                assert isinstance(res, ContextCancelled)
+                assert tuple(res.canceller) == current_actor().uid
+
+            else:
+                print(f'RX ROOT SIDE RESULT {res}')
+                assert res == 'yo'
 
             # cancel the daemon
             await portal.cancel_actor()
@@ -1220,7 +1086,7 @@ def test_maybe_allow_overruns_stream(
             # NOTE: i tried to isolate to a deterministic case here
             # based on timeing, but i was kinda wasted, and i don't
             # think it's sane to catch them..
-            assert err.boxed_type in (
+            assert err.type in (
                 tractor.RemoteActorError,
                 StreamOverrun,
             )
@@ -1228,63 +1094,13 @@ def test_maybe_allow_overruns_stream(
         elif (
             slow_side == 'child'
         ):
-            assert err.boxed_type == StreamOverrun
+            assert err.type == StreamOverrun
 
         elif slow_side == 'parent':
-            assert err.boxed_type == tractor.RemoteActorError
-            assert 'StreamOverrun' in err.tb_str
-            assert err.tb_str == err.msgdata['tb_str']
+            assert err.type == tractor.RemoteActorError
+            assert 'StreamOverrun' in err.msgdata['tb_str']
 
     else:
         # if this hits the logic blocks from above are not
         # exhaustive..
         pytest.fail('PARAMETRIZED CASE GEN PROBLEM YO')
-
-
-def test_ctx_with_self_actor(
-    loglevel: str,
-    debug_mode: bool,
-):
-    '''
-    NOTE: for now this is an INVALID OP!
-
-    BUT, eventually presuming we add a "side" key to `Actor.get_context()`,
-    we might be able to get this working symmetrically, but should we??
-
-    Open a context back to the same actor and ensure all cancellation
-    and error semantics hold the same.
-
-    '''
-    async def main():
-        async with tractor.open_nursery(
-            debug_mode=debug_mode,
-            enable_modules=[__name__],
-        ) as an:
-            assert an
-            async with (
-                tractor.find_actor('root') as portal,
-                portal.open_context(
-                    expect_cancelled,
-                    # echo_back_sequence,
-                    # seq=seq,
-                    # wait_for_cancel=cancel_ctx,
-                    # be_slow=(slow_side == 'child'),
-                    # allow_overruns_side=allow_overruns_side,
-
-                ) as (ctx, sent),
-                ctx.open_stream() as ipc,
-            ):
-                assert sent is None
-
-                seq = list(range(10))
-                for i in seq:
-                    await ipc.send(i)
-                    rx: int = await ipc.receive()
-                    assert rx == i
-
-                await ctx.cancel()
-
-    with pytest.raises(RuntimeError) as excinfo:
-        trio.run(main)
-
-    assert 'Invalid Operation' in repr(excinfo.value)

tests/test_debugger.py

@@ -12,26 +12,27 @@ TODO:
 """
 from functools import partial
 import itertools
+# from os import path
+from typing import Optional
 import platform
+import pathlib
+# import sys
 import time
 
 import pytest
+import pexpect
 from pexpect.exceptions import (
     TIMEOUT,
     EOF,
 )
 
-from .conftest import (
-    do_ctlc,
-    PROMPT,
+from tractor.devx._debug import (
     _pause_msg,
     _crash_msg,
-    _repl_fail_msg,
 )
-from .conftest import (
-    expect,
-    in_prompt_msg,
-    assert_before,
+from conftest import (
+    examples_dir,
+    _ci_env,
 )
 
 # TODO: The next great debugger audit could be done by you!
@@ -51,6 +52,15 @@ if platform.system() == 'Windows':
     )
 
 
+def mk_cmd(ex_name: str) -> str:
+    '''
+    Generate a command suitable to pass to ``pexpect.spawn()``.
+
+    '''
+    script_path: pathlib.Path = examples_dir() / 'debugging' / f'{ex_name}.py'
+    return ' '.join(['python', str(script_path)])
+
+
 # TODO: was trying to this xfail style but some weird bug i see in CI
 # that's happening at collect time.. pretty soon gonna dump actions i'm
 # thinkin...
@@ -69,6 +79,136 @@ has_nested_actors = pytest.mark.has_nested_actors
 # )
 
 
+@pytest.fixture
+def spawn(
+    start_method,
+    testdir,
+    reg_addr,
+) -> 'pexpect.spawn':
+
+    if start_method != 'trio':
+        pytest.skip(
+            "Debugger tests are only supported on the trio backend"
+        )
+
+    def _spawn(cmd):
+        return testdir.spawn(
+            cmd=mk_cmd(cmd),
+            expect_timeout=3,
+        )
+
+    return _spawn
+
+
+PROMPT = r"\(Pdb\+\)"
+
+
+def expect(
+    child,
+
+    # prompt by default
+    patt: str = PROMPT,
+
+    **kwargs,
+
+) -> None:
+    '''
+    Expect wrapper that prints last seen console
+    data before failing.
+
+    '''
+    try:
+        child.expect(
+            patt,
+            **kwargs,
+        )
+    except TIMEOUT:
+        before = str(child.before.decode())
+        print(before)
+        raise
+
+
+def in_prompt_msg(
+    prompt: str,
+    parts: list[str],
+
+    pause_on_false: bool = False,
+    print_prompt_on_false: bool = True,
+
+) -> bool:
+    '''
+    Predicate check if (the prompt's) std-streams output has all
+    `str`-parts in it.
+
+    Can be used in test asserts for bulk matching expected
+    log/REPL output for a given `pdb` interact point.
+
+    '''
+    for part in parts:
+        if part not in prompt:
+
+            if pause_on_false:
+                import pdbp
+                pdbp.set_trace()
+
+            if print_prompt_on_false:
+                print(prompt)
+
+            return False
+
+    return True
+
+def assert_before(
+    child,
+    patts: list[str],
+
+    **kwargs,
+
+) -> None:
+
+    # as in before the prompt end
+    before: str = str(child.before.decode())
+    assert in_prompt_msg(
+        prompt=before,
+        parts=patts,
+
+        **kwargs
+    )
+
+
+@pytest.fixture(
+    params=[False, True],
+    ids='ctl-c={}'.format,
+)
+def ctlc(
+    request,
+    ci_env: bool,
+
+) -> bool:
+
+    use_ctlc = request.param
+
+    node = request.node
+    markers = node.own_markers
+    for mark in markers:
+        if mark.name == 'has_nested_actors':
+            pytest.skip(
+                f'Test {node} has nested actors and fails with Ctrl-C.\n'
+                f'The test can sometimes run fine locally but until'
+                ' we solve' 'this issue this CI test will be xfail:\n'
+                'https://github.com/goodboy/tractor/issues/320'
+            )
+
+    if use_ctlc:
+        # XXX: disable pygments highlighting for auto-tests
+        # since some envs (like actions CI) will struggle
+        # the the added color-char encoding..
+        from tractor.devx._debug import TractorConfig
+        TractorConfig.use_pygements = False
+
+    yield use_ctlc
+
+
 @pytest.mark.parametrize(
     'user_in_out',
     [
@@ -77,10 +217,7 @@ has_nested_actors = pytest.mark.has_nested_actors
     ],
     ids=lambda item: f'{item[0]} -> {item[1]}',
 )
-def test_root_actor_error(
-    spawn,
-    user_in_out,
-):
+def test_root_actor_error(spawn, user_in_out):
     '''
     Demonstrate crash handler entering pdb from basic error in root actor.
 
@@ -92,15 +229,14 @@ def test_root_actor_error(
     # scan for the prompt
     expect(child, PROMPT)
 
+    before = str(child.before.decode())
+
     # make sure expected logging and error arrives
     assert in_prompt_msg(
-        child,
-        [
-            _crash_msg,
-            "('root'",
-            'AssertionError',
-        ]
+        before,
+        [_crash_msg, "('root'"]
     )
+    assert 'AssertionError' in before
 
     # send user command
     child.sendline(user_input)
@@ -119,10 +255,8 @@ def test_root_actor_error(
     ids=lambda item: f'{item[0]} -> {item[1]}',
 )
 def test_root_actor_bp(spawn, user_in_out):
-    '''
-    Demonstrate breakpoint from in root actor.
-
-    '''
+    """Demonstrate breakpoint from in root actor.
+    """
     user_input, expect_err_str = user_in_out
     child = spawn('root_actor_breakpoint')
 
@@ -136,7 +270,7 @@ def test_root_actor_bp(spawn, user_in_out):
     child.expect('\r\n')
 
     # process should exit
-    child.expect(EOF)
+    child.expect(pexpect.EOF)
 
     if expect_err_str is None:
         assert 'Error' not in str(child.before)
@@ -144,6 +278,38 @@ def test_root_actor_bp(spawn, user_in_out):
         assert expect_err_str in str(child.before)
 
 
+def do_ctlc(
+    child,
+    count: int = 3,
+    delay: float = 0.1,
+    patt: Optional[str] = None,
+
+    # expect repl UX to reprint the prompt after every
+    # ctrl-c send.
+    # XXX: no idea but, in CI this never seems to work even on 3.10 so
+    # needs some further investigation potentially...
+    expect_prompt: bool = not _ci_env,
+
+) -> None:
+
+    # make sure ctl-c sends don't do anything but repeat output
+    for _ in range(count):
+        time.sleep(delay)
+        child.sendcontrol('c')
+
+        # TODO: figure out why this makes CI fail..
+        # if you run this test manually it works just fine..
+        if expect_prompt:
+            before = str(child.before.decode())
+            time.sleep(delay)
+            child.expect(PROMPT)
+            time.sleep(delay)
+
+            if patt:
+                # should see the last line on console
+                assert patt in before
+
+
 def test_root_actor_bp_forever(
     spawn,
     ctlc: bool,
@@ -183,7 +349,7 @@ def test_root_actor_bp_forever(
 
     # quit out of the loop
     child.sendline('q')
-    child.expect(EOF)
+    child.expect(pexpect.EOF)
 
 
 @pytest.mark.parametrize(
@@ -205,12 +371,10 @@ def test_subactor_error(
     # scan for the prompt
     child.expect(PROMPT)
 
+    before = str(child.before.decode())
     assert in_prompt_msg(
-        child,
-        [
-            _crash_msg,
-            "('name_error'",
-        ]
+        before,
+        [_crash_msg, "('name_error'"]
     )
 
     if do_next:
@@ -229,15 +393,17 @@ def test_subactor_error(
         child.sendline('continue')
 
     child.expect(PROMPT)
+    before = str(child.before.decode())
+
+    # root actor gets debugger engaged
     assert in_prompt_msg(
-        child,
-        [
-            _crash_msg,
-            # root actor gets debugger engaged
-            "('root'",
-            # error is a remote error propagated from the subactor
-            "('name_error'",
-        ]
+        before,
+        [_crash_msg, "('root'"]
+    )
+    # error is a remote error propagated from the subactor
+    assert in_prompt_msg(
+        before,
+        [_crash_msg, "('name_error'"]
     )
 
     # another round
@@ -248,7 +414,7 @@ def test_subactor_error(
     child.expect('\r\n')
 
     # process should exit
-    child.expect(EOF)
+    child.expect(pexpect.EOF)
 
 
 def test_subactor_breakpoint(
@@ -258,11 +424,14 @@ def test_subactor_breakpoint(
     "Single subactor with an infinite breakpoint loop"
 
     child = spawn('subactor_breakpoint')
+
+    # scan for the prompt
     child.expect(PROMPT)
+
+    before = str(child.before.decode())
     assert in_prompt_msg(
-        child,
-        [_pause_msg,
-         "('breakpoint_forever'",]
+        before,
+        [_pause_msg, "('breakpoint_forever'"]
     )
 
     # do some "next" commands to demonstrate recurrent breakpoint
@@ -278,8 +447,9 @@ def test_subactor_breakpoint(
     for _ in range(5):
         child.sendline('continue')
         child.expect(PROMPT)
+        before = str(child.before.decode())
         assert in_prompt_msg(
-            child,
+            before,
             [_pause_msg, "('breakpoint_forever'"]
         )
 
@@ -292,12 +462,9 @@ def test_subactor_breakpoint(
     # child process should exit but parent will capture pdb.BdbQuit
     child.expect(PROMPT)
 
-    assert in_prompt_msg(
-        child,
-        ['RemoteActorError:',
-         "('breakpoint_forever'",
-         'bdb.BdbQuit',]
-    )
+    before = str(child.before.decode())
+    assert "RemoteActorError: ('breakpoint_forever'" in before
+    assert 'bdb.BdbQuit' in before
 
     if ctlc:
         do_ctlc(child)
@@ -306,17 +473,11 @@ def test_subactor_breakpoint(
     child.sendline('c')
 
     # process should exit
-    child.expect(EOF)
+    child.expect(pexpect.EOF)
 
-    assert in_prompt_msg(
-        child, [
-        'MessagingError:',
-        'RemoteActorError:',
-         "('breakpoint_forever'",
-         'bdb.BdbQuit',
-        ],
-        pause_on_false=True,
-    )
+    before = str(child.before.decode())
+    assert "RemoteActorError: ('breakpoint_forever'" in before
+    assert 'bdb.BdbQuit' in before
 
 
 @has_nested_actors
@@ -336,7 +497,7 @@ def test_multi_subactors(
 
     before = str(child.before.decode())
     assert in_prompt_msg(
-        child,
+        before,
         [_pause_msg, "('breakpoint_forever'"]
     )
 
@@ -357,14 +518,12 @@ def test_multi_subactors(
 
     # first name_error failure
     child.expect(PROMPT)
+    before = str(child.before.decode())
     assert in_prompt_msg(
-        child,
-        [
-            _crash_msg,
-            "('name_error'",
-            "NameError",
-        ]
+        before,
+        [_crash_msg, "('name_error'"]
     )
+    assert "NameError" in before
 
     if ctlc:
         do_ctlc(child)
@@ -388,8 +547,9 @@ def test_multi_subactors(
     # breakpoint loop should re-engage
     child.sendline('c')
     child.expect(PROMPT)
+    before = str(child.before.decode())
     assert in_prompt_msg(
-        child,
+        before,
         [_pause_msg, "('breakpoint_forever'"]
     )
 
@@ -452,7 +612,7 @@ def test_multi_subactors(
 
     # process should exit
     child.sendline('c')
-    child.expect(EOF)
+    child.expect(pexpect.EOF)
 
     # repeat of previous multierror for final output
     assert_before(child, [
@@ -482,28 +642,25 @@ def test_multi_daemon_subactors(
     # the root's tty lock first so anticipate either crash
     # message on the first entry.
 
-    bp_forev_parts = [
-        _pause_msg,
-        "('bp_forever'",
-    ]
+    bp_forev_parts = [_pause_msg, "('bp_forever'"]
     bp_forev_in_msg = partial(
         in_prompt_msg,
         parts=bp_forev_parts,
     )
 
-    name_error_msg: str = "NameError: name 'doggypants' is not defined"
-    name_error_parts: list[str] = [name_error_msg]
+    name_error_msg = "NameError: name 'doggypants' is not defined"
+    name_error_parts = [name_error_msg]
 
     before = str(child.before.decode())
 
-    if bp_forev_in_msg(child=child):
+    if bp_forev_in_msg(prompt=before):
         next_parts = name_error_parts
 
     elif name_error_msg in before:
         next_parts = bp_forev_parts
 
     else:
-        raise ValueError('Neither log msg was found !?')
+        raise ValueError("Neither log msg was found !?")
 
     if ctlc:
         do_ctlc(child)
@@ -572,12 +729,14 @@ def test_multi_daemon_subactors(
     # wait for final error in root
     # where it crashs with boxed error
     while True:
-        child.sendline('c')
-        child.expect(PROMPT)
-        if not in_prompt_msg(
-            child,
-            bp_forev_parts
-        ):
+        try:
+            child.sendline('c')
+            child.expect(PROMPT)
+            assert_before(
+                child,
+                bp_forev_parts
+            )
+        except AssertionError:
             break
 
     assert_before(
@@ -586,14 +745,13 @@ def test_multi_daemon_subactors(
             # boxed error raised in root task
             # "Attaching to pdb in crashed actor: ('root'",
             _crash_msg,
-            "('root'",  # should attach in root
-            "_exceptions.RemoteActorError:",  # with an embedded RAE for..
-            "('name_error'",  # the src subactor which raised
+            "('root'",
+            "_exceptions.RemoteActorError: ('name_error'",
         ]
     )
 
     child.sendline('c')
-    child.expect(EOF)
+    child.expect(pexpect.EOF)
 
 
 @has_nested_actors
@@ -669,7 +827,7 @@ def test_multi_subactors_root_errors(
     ])
 
     child.sendline('c')
-    child.expect(EOF)
+    child.expect(pexpect.EOF)
 
     assert_before(child, [
         # "Attaching to pdb in crashed actor: ('root'",
@@ -689,11 +847,10 @@ def test_multi_nested_subactors_error_through_nurseries(
     # https://github.com/goodboy/tractor/issues/320
     # ctlc: bool,
 ):
-    '''
-    Verify deeply nested actors that error trigger debugger entries
+    """Verify deeply nested actors that error trigger debugger entries
     at each actor nurserly (level) all the way up the tree.
 
-    '''
+    """
     # NOTE: previously, inside this script was a bug where if the
     # parent errors before a 2-levels-lower actor has released the lock,
     # the parent tries to cancel it but it's stuck in the debugger?
@@ -713,31 +870,22 @@ def test_multi_nested_subactors_error_through_nurseries(
         except EOF:
             break
 
-    assert_before(
-        child,
-        [ # boxed source errors
-            "NameError: name 'doggypants' is not defined",
-            "tractor._exceptions.RemoteActorError:",
-            "('name_error'",
-            "bdb.BdbQuit",
+    assert_before(child, [
 
-            # first level subtrees
-            # "tractor._exceptions.RemoteActorError: ('spawner0'",
-            "src_uid=('spawner0'",
+        # boxed source errors
+        "NameError: name 'doggypants' is not defined",
+        "tractor._exceptions.RemoteActorError: ('name_error'",
+        "bdb.BdbQuit",
 
-            # "tractor._exceptions.RemoteActorError: ('spawner1'",
+        # first level subtrees
+        "tractor._exceptions.RemoteActorError: ('spawner0'",
+        # "tractor._exceptions.RemoteActorError: ('spawner1'",
 
-            # propagation of errors up through nested subtrees
-            # "tractor._exceptions.RemoteActorError: ('spawn_until_0'",
-            # "tractor._exceptions.RemoteActorError: ('spawn_until_1'",
-            # "tractor._exceptions.RemoteActorError: ('spawn_until_2'",
-            # ^-NOTE-^ old RAE repr, new one is below with a field
-            # showing the src actor's uid.
-            "src_uid=('spawn_until_0'",
-            "relay_uid=('spawn_until_1'",
-            "src_uid=('spawn_until_2'",
-        ]
-    )
+        # propagation of errors up through nested subtrees
+        "tractor._exceptions.RemoteActorError: ('spawn_until_0'",
+        "tractor._exceptions.RemoteActorError: ('spawn_until_1'",
+        "tractor._exceptions.RemoteActorError: ('spawn_until_2'",
+    ])
 
 
 @pytest.mark.timeout(15)
@@ -758,13 +906,10 @@ def test_root_nursery_cancels_before_child_releases_tty_lock(
     child = spawn('root_cancelled_but_child_is_in_tty_lock')
 
     child.expect(PROMPT)
-    assert_before(
-        child,
-        [
-            "NameError: name 'doggypants' is not defined",
-            "tractor._exceptions.RemoteActorError: ('name_error'",
-        ],
-    )
+
+    before = str(child.before.decode())
+    assert "NameError: name 'doggypants' is not defined" in before
+    assert "tractor._exceptions.RemoteActorError: ('name_error'" not in before
     time.sleep(0.5)
 
     if ctlc:
@@ -802,7 +947,7 @@ def test_root_nursery_cancels_before_child_releases_tty_lock(
 
     for i in range(3):
         try:
-            child.expect(EOF, timeout=0.5)
+            child.expect(pexpect.EOF, timeout=0.5)
             break
         except TIMEOUT:
             child.sendline('c')
@@ -844,7 +989,7 @@ def test_root_cancels_child_context_during_startup(
         do_ctlc(child)
 
     child.sendline('c')
-    child.expect(EOF)
+    child.expect(pexpect.EOF)
 
 
 def test_different_debug_mode_per_actor(
@@ -855,8 +1000,9 @@ def test_different_debug_mode_per_actor(
     child.expect(PROMPT)
 
     # only one actor should enter the debugger
+    before = str(child.before.decode())
     assert in_prompt_msg(
-        child,
+        before,
         [_crash_msg, "('debugged_boi'", "RuntimeError"],
     )
 
@@ -864,240 +1010,18 @@ def test_different_debug_mode_per_actor(
         do_ctlc(child)
 
     child.sendline('c')
-    child.expect(EOF)
+    child.expect(pexpect.EOF)
+
+    before = str(child.before.decode())
 
     # NOTE: this debugged actor error currently WON'T show up since the
     # root will actually cancel and terminate the nursery before the error
     # msg reported back from the debug mode actor is processed.
     # assert "tractor._exceptions.RemoteActorError: ('debugged_boi'" in before
 
+    assert "tractor._exceptions.RemoteActorError: ('crash_boi'" in before
+
     # the crash boi should not have made a debugger request but
     # instead crashed completely
-    assert_before(
-        child,
-        [
-            "tractor._exceptions.RemoteActorError:",
-            "src_uid=('crash_boi'",
-            "RuntimeError",
-        ]
-    )
-
-
-def test_post_mortem_api(
-    spawn,
-    ctlc: bool,
-):
-    '''
-    Verify the `tractor.post_mortem()` API works in an exception
-    handler block.
-
-    '''
-    child = spawn('pm_in_subactor')
-
-    # First entry is via manual `.post_mortem()`
-    child.expect(PROMPT)
-    assert_before(
-        child,
-        [
-            _crash_msg,
-            "<Task 'name_error'",
-            "NameError",
-            "('child'",
-            "tractor.post_mortem()",
-        ]
-    )
-    if ctlc:
-        do_ctlc(child)
-    child.sendline('c')
-
-    # 2nd is RPC crash handler
-    child.expect(PROMPT)
-    assert_before(
-        child,
-        [
-            _crash_msg,
-            "<Task 'name_error'",
-            "NameError",
-            "('child'",
-        ]
-    )
-    if ctlc:
-        do_ctlc(child)
-    child.sendline('c')
-
-    # 3rd is via RAE bubbled to root's parent ctx task and
-    # crash-handled via another manual pm call.
-    child.expect(PROMPT)
-    assert_before(
-        child,
-        [
-            _crash_msg,
-            "<Task '__main__.main'",
-            "('root'",
-            "NameError",
-            "tractor.post_mortem()",
-            "src_uid=('child'",
-        ]
-    )
-    if ctlc:
-        do_ctlc(child)
-    child.sendline('c')
-
-    # 4th and FINAL is via RAE bubbled to root's parent ctx task and
-    # crash-handled via another manual pm call.
-    child.expect(PROMPT)
-    assert_before(
-        child,
-        [
-            _crash_msg,
-            "<Task '__main__.main'",
-            "('root'",
-            "NameError",
-            "src_uid=('child'",
-        ]
-    )
-    if ctlc:
-        do_ctlc(child)
-
-
-    # TODO: ensure we're stopped and showing the right call stack frame
-    # -[ ] need a way to strip the terminal color chars in order to
-    #    pattern match... see TODO around `assert_before()` above!
-    # child.sendline('w')
-    # child.expect(PROMPT)
-    # assert_before(
-    #     child,
-    #     [
-    #         # error src block annot at ctx open
-    #         '-> async with p.open_context(name_error) as (ctx, first):',
-    #     ]
-    # )
-
-    # # step up a frame to ensure the it's the root's nursery
-    # child.sendline('u')
-    # child.expect(PROMPT)
-    # assert_before(
-    #     child,
-    #     [
-    #         # handler block annotation
-    #         '-> async with tractor.open_nursery(',
-    #     ]
-    # )
-
-    child.sendline('c')
-    child.expect(EOF)
-
-
-def test_shield_pause(
-    spawn,
-):
-    '''
-    Verify the `tractor.pause()/.post_mortem()` API works inside an
-    already cancelled `trio.CancelScope` and that you can step to the
-    next checkpoint wherein the cancelled will get raised.
-
-    '''
-    child = spawn('shielded_pause')
-
-    # First entry is via manual `.post_mortem()`
-    child.expect(PROMPT)
-    assert_before(
-        child,
-        [
-            _pause_msg,
-            "cancellable_pause_loop'",
-            "('cancelled_before_pause'",  # actor name
-        ]
-    )
-
-    # since 3 tries in ex. shield pause loop
-    for i in range(3):
-        child.sendline('c')
-        child.expect(PROMPT)
-        assert_before(
-            child,
-            [
-                _pause_msg,
-                "INSIDE SHIELDED PAUSE",
-                "('cancelled_before_pause'",  # actor name
-            ]
-        )
-
-    # back inside parent task that opened nursery
-    child.sendline('c')
-    child.expect(PROMPT)
-    assert_before(
-        child,
-        [
-            _crash_msg,
-            "('cancelled_before_pause'",  # actor name
-            _repl_fail_msg,
-            "trio.Cancelled",
-            "raise Cancelled._create()",
-
-            # we should be handling a taskc inside
-            # the first `.port_mortem()` sin-shield!
-            'await DebugStatus.req_finished.wait()',
-        ]
-    )
-
-    # same as above but in the root actor's task
-    child.sendline('c')
-    child.expect(PROMPT)
-    assert_before(
-        child,
-        [
-            _crash_msg,
-            "('root'",  # actor name
-            _repl_fail_msg,
-            "trio.Cancelled",
-            "raise Cancelled._create()",
-
-            # handling a taskc inside the first unshielded
-            # `.port_mortem()`.
-            # BUT in this case in the root-proc path ;)
-            'wait Lock._debug_lock.acquire()',
-        ]
-    )
-    child.sendline('c')
-    child.expect(EOF)
-
-
-# TODO: better error for "non-ideal" usage from the root actor.
-# -[ ] if called from an async scope emit a message that suggests
-#    using `await tractor.pause()` instead since it's less overhead
-#    (in terms of `greenback` and/or extra threads) and if it's from
-#    a sync scope suggest that usage must first call
-#    `ensure_portal()` in the (eventual parent) async calling scope?
-def test_sync_pause_from_bg_task_in_root_actor_():
-    '''
-    When used from the root actor, normally we can only implicitly
-    support `.pause_from_sync()` from the main-parent-task (that
-    opens the runtime via `open_root_actor()`) since `greenback`
-    requires a `.ensure_portal()` call per `trio.Task` where it is
-    used.
-
-    '''
-    ...
-
-# TODO: needs ANSI code stripping tho, see `assert_before()` # above!
-def test_correct_frames_below_hidden():
-    '''
-    Ensure that once a `tractor.pause()` enages, when the user
-    inputs a "next"/"n" command the actual next line steps
-    and that using a "step"/"s" into the next LOC, particuarly
-    `tractor` APIs, you can step down into that code.
-
-    '''
-    ...
-
-
-def test_cant_pause_from_paused_task():
-    '''
-    Pausing from with an already paused task should raise an error.
-
-    Normally this should only happen in practise while debugging the call stack of `tractor.pause()` itself, likely
-    by a `.pause()` line somewhere inside our runtime.
-
-    '''
-    ...
+    assert "tractor._exceptions.RemoteActorError: ('crash_boi'" in before
+    assert "RuntimeError" in before

tests/test_discovery.py

@@ -9,9 +9,10 @@ import itertools
 
 import pytest
 import tractor
-from tractor._testing import tractor_test
 import trio
 
+from conftest import tractor_test
+
 
 @tractor_test
 async def test_reg_then_unreg(reg_addr):
@@ -26,7 +27,7 @@ async def test_reg_then_unreg(reg_addr):
         portal = await n.start_actor('actor', enable_modules=[__name__])
         uid = portal.channel.uid
 
-        async with tractor.get_registry(*reg_addr) as aportal:
+        async with tractor.get_arbiter(*reg_addr) as aportal:
             # this local actor should be the arbiter
             assert actor is aportal.actor
 
@@ -160,7 +161,7 @@ async def spawn_and_check_registry(
     async with tractor.open_root_actor(
         registry_addrs=[reg_addr],
     ):
-        async with tractor.get_registry(*reg_addr) as portal:
+        async with tractor.get_arbiter(*reg_addr) as portal:
             # runtime needs to be up to call this
             actor = tractor.current_actor()
 
@@ -181,9 +182,7 @@ async def spawn_and_check_registry(
 
             try:
                 async with tractor.open_nursery() as n:
-                    async with trio.open_nursery(
-                        strict_exception_groups=False,
-                    ) as trion:
+                    async with trio.open_nursery() as trion:
 
                         portals = {}
                         for i in range(3):
@@ -300,7 +299,7 @@ async def close_chans_before_nursery(
     async with tractor.open_root_actor(
         registry_addrs=[reg_addr],
     ):
-        async with tractor.get_registry(*reg_addr) as aportal:
+        async with tractor.get_arbiter(*reg_addr) as aportal:
             try:
                 get_reg = partial(unpack_reg, aportal)
 
@@ -318,9 +317,7 @@ async def close_chans_before_nursery(
                         async with portal2.open_stream_from(
                             stream_forever
                         ) as agen2:
-                            async with trio.open_nursery(
-                                strict_exception_groups=False,
-                            ) as n:
+                            async with trio.open_nursery() as n:
                                 n.start_soon(streamer, agen1)
                                 n.start_soon(cancel, use_signal, .5)
                                 try:

tests/test_docs_examples.py

@@ -11,7 +11,8 @@ import platform
 import shutil
 
 import pytest
-from tractor._testing import (
+
+from conftest import (
     examples_dir,
 )
 
@@ -19,7 +20,7 @@ from tractor._testing import (
 @pytest.fixture
 def run_example_in_subproc(
     loglevel: str,
-    testdir: pytest.Pytester,
+    testdir,
     reg_addr: tuple[str, int],
 ):
 
@@ -81,36 +82,27 @@ def run_example_in_subproc(
 
     # walk yields: (dirpath, dirnames, filenames)
     [
-        (p[0], f)
-        for p in os.walk(examples_dir())
-        for f in p[2]
+        (p[0], f) for p in os.walk(examples_dir()) for f in p[2]
 
-        if (
-            '__' not in f
-            and f[0] != '_'
-            and 'debugging' not in p[0]
-            and 'integration' not in p[0]
-            and 'advanced_faults' not in p[0]
-            and 'multihost' not in p[0]
-        )
+        if '__' not in f
+        and f[0] != '_'
+        and 'debugging' not in p[0]
+        and 'integration' not in p[0]
+        and 'advanced_faults' not in p[0]
     ],
+
     ids=lambda t: t[1],
 )
-def test_example(
-    run_example_in_subproc,
-    example_script,
-):
-    '''
-    Load and run scripts from this repo's ``examples/`` dir as a user
+def test_example(run_example_in_subproc, example_script):
+    """Load and run scripts from this repo's ``examples/`` dir as a user
     would copy and pasing them into their editor.
 
     On windows a little more "finessing" is done to make
     ``multiprocessing`` play nice: we copy the ``__main__.py`` into the
     test directory and invoke the script as a module with ``python -m
     test_example``.
-
-    '''
-    ex_file: str = os.path.join(*example_script)
+    """
+    ex_file = os.path.join(*example_script)
 
     if 'rpc_bidir_streaming' in ex_file and sys.version_info < (3, 9):
         pytest.skip("2-way streaming example requires py3.9 async with syntax")
@@ -136,8 +128,7 @@ def test_example(
                     # shouldn't eventually once we figure out what's
                     # a better way to be explicit about aio side
                     # cancels?
-                    and
-                    'asyncio.exceptions.CancelledError' not in last_error
+                    and 'asyncio.exceptions.CancelledError' not in last_error
                 ):
                     raise Exception(errmsg)
 

tests/test_ext_types_msgspec.py

@@ -1,946 +0,0 @@
-'''
-Low-level functional audits for our
-"capability based messaging"-spec feats.
-
-B~)
-
-'''
-from contextlib import (
-    contextmanager as cm,
-    # nullcontext,
-)
-import importlib
-from typing import (
-    Any,
-    Type,
-    Union,
-)
-
-from msgspec import (
-    # structs,
-    # msgpack,
-    Raw,
-    # Struct,
-    ValidationError,
-)
-import pytest
-import trio
-
-import tractor
-from tractor import (
-    Actor,
-    # _state,
-    MsgTypeError,
-    Context,
-)
-from tractor.msg import (
-    _codec,
-    _ctxvar_MsgCodec,
-    _exts,
-
-    NamespacePath,
-    MsgCodec,
-    MsgDec,
-    mk_codec,
-    mk_dec,
-    apply_codec,
-    current_codec,
-)
-from tractor.msg.types import (
-    log,
-    Started,
-    # _payload_msgs,
-    # PayloadMsg,
-    # mk_msg_spec,
-)
-from tractor.msg._ops import (
-    limit_plds,
-)
-
-def enc_nsp(obj: Any) -> Any:
-    actor: Actor = tractor.current_actor(
-        err_on_no_runtime=False,
-    )
-    uid: tuple[str, str]|None = None if not actor else actor.uid
-    print(f'{uid} ENC HOOK')
-
-    match obj:
-        # case NamespacePath()|str():
-        case NamespacePath():
-            encoded: str = str(obj)
-            print(
-                f'----- ENCODING `NamespacePath` as `str` ------\n'
-                f'|_obj:{type(obj)!r} = {obj!r}\n'
-                f'|_encoded: str = {encoded!r}\n'
-            )
-            # if type(obj) != NamespacePath:
-            #     breakpoint()
-            return encoded
-        case _:
-            logmsg: str = (
-                f'{uid}\n'
-                'FAILED ENCODE\n'
-                f'obj-> `{obj}: {type(obj)}`\n'
-            )
-            raise NotImplementedError(logmsg)
-
-
-def dec_nsp(
-    obj_type: Type,
-    obj: Any,
-
-) -> Any:
-    # breakpoint()
-    actor: Actor = tractor.current_actor(
-        err_on_no_runtime=False,
-    )
-    uid: tuple[str, str]|None = None if not actor else actor.uid
-    print(
-        f'{uid}\n'
-        'CUSTOM DECODE\n'
-        f'type-arg-> {obj_type}\n'
-        f'obj-arg-> `{obj}`: {type(obj)}\n'
-    )
-    nsp = None
-    # XXX, never happens right?
-    if obj_type is Raw:
-        breakpoint()
-
-    if (
-        obj_type is NamespacePath
-        and isinstance(obj, str)
-        and ':' in obj
-    ):
-        nsp = NamespacePath(obj)
-        # TODO: we could built a generic handler using
-        # JUST matching the obj_type part?
-        # nsp = obj_type(obj)
-
-    if nsp:
-        print(f'Returning NSP instance: {nsp}')
-        return nsp
-
-    logmsg: str = (
-        f'{uid}\n'
-        'FAILED DECODE\n'
-        f'type-> {obj_type}\n'
-        f'obj-arg-> `{obj}`: {type(obj)}\n\n'
-        f'current codec:\n'
-        f'{current_codec()}\n'
-    )
-    # TODO: figure out the ignore subsys for this!
-    # -[ ] option whether to defense-relay backc the msg
-    #   inside an `Invalid`/`Ignore`
-    # -[ ] how to make this handling pluggable such that a
-    #   `Channel`/`MsgTransport` can intercept and process
-    #   back msgs either via exception handling or some other
-    #   signal?
-    log.warning(logmsg)
-    # NOTE: this delivers the invalid
-    # value up to `msgspec`'s decoding
-    # machinery for error raising.
-    return obj
-    # raise NotImplementedError(logmsg)
-
-
-def ex_func(*args):
-    '''
-    A mod level func we can ref and load via our `NamespacePath`
-    python-object pointer `str` subtype.
-
-    '''
-    print(f'ex_func({args})')
-
-
-@pytest.mark.parametrize(
-    'add_codec_hooks',
-    [
-        True,
-        False,
-    ],
-    ids=['use_codec_hooks', 'no_codec_hooks'],
-)
-def test_custom_extension_types(
-    debug_mode: bool,
-    add_codec_hooks: bool
-):
-    '''
-    Verify that a `MsgCodec` (used for encoding all outbound IPC msgs
-    and decoding all inbound `PayloadMsg`s) and a paired `MsgDec`
-    (used for decoding the `PayloadMsg.pld: Raw` received within a given
-    task's ipc `Context` scope) can both send and receive "extension types"
-    as supported via custom converter hooks passed to `msgspec`.
-
-    '''
-    nsp_pld_dec: MsgDec = mk_dec(
-        spec=None,  # ONLY support the ext type
-        dec_hook=dec_nsp if add_codec_hooks else None,
-        ext_types=[NamespacePath],
-    )
-    nsp_codec: MsgCodec = mk_codec(
-        # ipc_pld_spec=Raw,  # default!
-
-        # NOTE XXX: the encode hook MUST be used no matter what since
-        # our `NamespacePath` is not any of a `Any` native type nor
-        # a `msgspec.Struct` subtype - so `msgspec` has no way to know
-        # how to encode it unless we provide the custom hook.
-        #
-        # AGAIN that is, regardless of whether we spec an
-        # `Any`-decoded-pld the enc has no knowledge (by default)
-        # how to enc `NamespacePath` (nsp), so we add a custom
-        # hook to do that ALWAYS.
-        enc_hook=enc_nsp if add_codec_hooks else None,
-
-        # XXX NOTE: pretty sure this is mutex with the `type=` to
-        # `Decoder`? so it won't work in tandem with the
-        # `ipc_pld_spec` passed above?
-        ext_types=[NamespacePath],
-
-        # TODO? is it useful to have the `.pld` decoded *prior* to
-        # the `PldRx`?? like perf or mem related?
-        # ext_dec=nsp_pld_dec,
-    )
-    if add_codec_hooks:
-        assert nsp_codec.dec.dec_hook is None
-
-        # TODO? if we pass `ext_dec` above?
-        # assert nsp_codec.dec.dec_hook is dec_nsp
-
-        assert nsp_codec.enc.enc_hook is enc_nsp
-
-    nsp = NamespacePath.from_ref(ex_func)
-
-    try:
-        nsp_bytes: bytes = nsp_codec.encode(nsp)
-        nsp_rt_sin_msg = nsp_pld_dec.decode(nsp_bytes)
-        nsp_rt_sin_msg.load_ref() is ex_func
-    except TypeError:
-        if not add_codec_hooks:
-            pass
-
-    try:
-        msg_bytes: bytes = nsp_codec.encode(
-            Started(
-                cid='cid',
-                pld=nsp,
-            )
-        )
-        # since the ext-type obj should also be set as the msg.pld
-        assert nsp_bytes in msg_bytes
-        started_rt: Started = nsp_codec.decode(msg_bytes)
-        pld: Raw = started_rt.pld
-        assert isinstance(pld, Raw)
-        nsp_rt: NamespacePath = nsp_pld_dec.decode(pld)
-        assert isinstance(nsp_rt, NamespacePath)
-        # in obj comparison terms they should be the same
-        assert nsp_rt == nsp
-        # ensure we've decoded to ext type!
-        assert nsp_rt.load_ref() is ex_func
-
-    except TypeError:
-        if not add_codec_hooks:
-            pass
-
-@tractor.context
-async def sleep_forever_in_sub(
-    ctx: Context,
-) -> None:
-    await trio.sleep_forever()
-
-
-def mk_custom_codec(
-    add_hooks: bool,
-
-) -> tuple[
-    MsgCodec,  # encode to send
-    MsgDec,  # pld receive-n-decode
-]:
-    '''
-    Create custom `msgpack` enc/dec-hooks and set a `Decoder`
-    which only loads `pld_spec` (like `NamespacePath`) types.
-
-    '''
-
-    # XXX NOTE XXX: despite defining `NamespacePath` as a type
-    # field on our `PayloadMsg.pld`, we still need a enc/dec_hook() pair
-    # to cast to/from that type on the wire. See the docs:
-    # https://jcristharif.com/msgspec/extending.html#mapping-to-from-native-types
-
-    # if pld_spec is Any:
-    #     pld_spec = Raw
-
-    nsp_codec: MsgCodec = mk_codec(
-        # ipc_pld_spec=Raw,  # default!
-
-        # NOTE XXX: the encode hook MUST be used no matter what since
-        # our `NamespacePath` is not any of a `Any` native type nor
-        # a `msgspec.Struct` subtype - so `msgspec` has no way to know
-        # how to encode it unless we provide the custom hook.
-        #
-        # AGAIN that is, regardless of whether we spec an
-        # `Any`-decoded-pld the enc has no knowledge (by default)
-        # how to enc `NamespacePath` (nsp), so we add a custom
-        # hook to do that ALWAYS.
-        enc_hook=enc_nsp if add_hooks else None,
-
-        # XXX NOTE: pretty sure this is mutex with the `type=` to
-        # `Decoder`? so it won't work in tandem with the
-        # `ipc_pld_spec` passed above?
-        ext_types=[NamespacePath],
-    )
-    # dec_hook=dec_nsp if add_hooks else None,
-    return nsp_codec
-
-
-@pytest.mark.parametrize(
-    'limit_plds_args',
-    [
-        (
-            {'dec_hook': None, 'ext_types': None},
-            None,
-        ),
-        (
-            {'dec_hook': dec_nsp, 'ext_types': None},
-            TypeError,
-        ),
-        (
-            {'dec_hook': dec_nsp, 'ext_types': [NamespacePath]},
-            None,
-        ),
-        (
-            {'dec_hook': dec_nsp, 'ext_types': [NamespacePath|None]},
-            None,
-        ),
-    ],
-    ids=[
-        'no_hook_no_ext_types',
-        'only_hook',
-        'hook_and_ext_types',
-        'hook_and_ext_types_w_null',
-    ]
-)
-def test_pld_limiting_usage(
-    limit_plds_args: tuple[dict, Exception|None],
-):
-    '''
-    Verify `dec_hook()` and `ext_types` need to either both be
-    provided or we raise a explanator type-error.
-
-    '''
-    kwargs, maybe_err = limit_plds_args
-    async def main():
-        async with tractor.open_nursery() as an:  # just to open runtime
-
-            # XXX SHOULD NEVER WORK outside an ipc ctx scope!
-            try:
-                with limit_plds(**kwargs):
-                    pass
-            except RuntimeError:
-                pass
-
-            p: tractor.Portal = await an.start_actor(
-                'sub',
-                enable_modules=[__name__],
-            )
-            async with (
-                p.open_context(
-                    sleep_forever_in_sub
-                ) as (ctx, first),
-            ):
-                try:
-                    with limit_plds(**kwargs):
-                        pass
-                except maybe_err as exc:
-                    assert type(exc) is maybe_err
-                    pass
-
-
-def chk_codec_applied(
-    expect_codec: MsgCodec|None,
-    enter_value: MsgCodec|None = None,
-
-) -> MsgCodec:
-    '''
-    buncha sanity checks ensuring that the IPC channel's
-    context-vars are set to the expected codec and that are
-    ctx-var wrapper APIs match the same.
-
-    '''
-    # TODO: play with tricyle again, bc this is supposed to work
-    # the way we want?
-    #
-    # TreeVar
-    # task: trio.Task = trio.lowlevel.current_task()
-    # curr_codec = _ctxvar_MsgCodec.get_in(task)
-
-    # ContextVar
-    # task_ctx: Context = task.context
-    # assert _ctxvar_MsgCodec in task_ctx
-    # curr_codec: MsgCodec = task.context[_ctxvar_MsgCodec]
-    if expect_codec is None:
-        assert enter_value is None
-        return
-
-    # NOTE: currently we use this!
-    # RunVar
-    curr_codec: MsgCodec = current_codec()
-    last_read_codec = _ctxvar_MsgCodec.get()
-    # assert curr_codec is last_read_codec
-
-    assert (
-        (same_codec := expect_codec) is
-        # returned from `mk_codec()`
-
-        # yielded value from `apply_codec()`
-
-        # read from current task's `contextvars.Context`
-        curr_codec is
-        last_read_codec
-
-        # the default `msgspec` settings
-        is not _codec._def_msgspec_codec
-        is not _codec._def_tractor_codec
-    )
-
-    if enter_value:
-        assert enter_value is same_codec
-
-
-@tractor.context
-async def send_back_values(
-    ctx: Context,
-    rent_pld_spec_type_strs: list[str],
-    add_hooks: bool,
-
-) -> None:
-    '''
-    Setup up a custom codec to load instances of `NamespacePath`
-    and ensure we can round trip a func ref with our parent.
-
-    '''
-    uid: tuple = tractor.current_actor().uid
-
-    # init state in sub-actor should be default
-    chk_codec_applied(
-        expect_codec=_codec._def_tractor_codec,
-    )
-
-    # load pld spec from input str
-    rent_pld_spec = _exts.dec_type_union(
-        rent_pld_spec_type_strs,
-        mods=[
-            importlib.import_module(__name__),
-        ],
-    )
-    rent_pld_spec_types: set[Type] = _codec.unpack_spec_types(
-        rent_pld_spec,
-    )
-
-    # ONLY add ext-hooks if the rent specified a non-std type!
-    add_hooks: bool = (
-        NamespacePath in rent_pld_spec_types
-        and
-        add_hooks
-    )
-
-    # same as on parent side config.
-    nsp_codec: MsgCodec|None = None
-    if add_hooks:
-        nsp_codec = mk_codec(
-            enc_hook=enc_nsp,
-            ext_types=[NamespacePath],
-        )
-
-    with (
-        maybe_apply_codec(nsp_codec) as codec,
-        limit_plds(
-            rent_pld_spec,
-            dec_hook=dec_nsp if add_hooks else None,
-            ext_types=[NamespacePath]  if add_hooks else None,
-        ) as pld_dec,
-    ):
-        # ?XXX? SHOULD WE NOT be swapping the global codec since it
-        # breaks `Context.started()` roundtripping checks??
-        chk_codec_applied(
-            expect_codec=nsp_codec,
-            enter_value=codec,
-        )
-
-        # ?TODO, mismatch case(s)?
-        #
-        # ensure pld spec matches on both sides
-        ctx_pld_dec: MsgDec = ctx._pld_rx._pld_dec
-        assert pld_dec is ctx_pld_dec
-        child_pld_spec: Type = pld_dec.spec
-        child_pld_spec_types: set[Type] = _codec.unpack_spec_types(
-            child_pld_spec,
-        )
-        assert (
-            child_pld_spec_types.issuperset(
-                rent_pld_spec_types
-            )
-        )
-
-        # ?TODO, try loop for each of the types in pld-superset?
-        #
-        # for send_value in [
-        #     nsp,
-        #     str(nsp),
-        #     None,
-        # ]:
-        nsp = NamespacePath.from_ref(ex_func)
-        try:
-            print(
-                f'{uid}: attempting to `.started({nsp})`\n'
-                f'\n'
-                f'rent_pld_spec: {rent_pld_spec}\n'
-                f'child_pld_spec: {child_pld_spec}\n'
-                f'codec: {codec}\n'
-            )
-            # await tractor.pause()
-            await ctx.started(nsp)
-
-        except tractor.MsgTypeError as _mte:
-            mte = _mte
-
-            # false -ve case
-            if add_hooks:
-                raise RuntimeError(
-                    f'EXPECTED to `.started()` value given spec ??\n\n'
-                    f'child_pld_spec -> {child_pld_spec}\n'
-                    f'value = {nsp}: {type(nsp)}\n'
-                )
-
-            # true -ve case
-            raise mte
-
-        # TODO: maybe we should add our own wrapper error so as to
-        # be interchange-lib agnostic?
-        # -[ ] the error type is wtv is raised from the hook so we
-        #   could also require a type-class of errors for
-        #   indicating whether the hook-failure can be handled by
-        #   a nasty-dialog-unprot sub-sys?
-        except TypeError as typerr:
-            # false -ve
-            if add_hooks:
-                raise RuntimeError('Should have been able to send `nsp`??')
-
-            # true -ve
-            print('Failed to send `nsp` due to no ext hooks set!')
-            raise typerr
-
-        # now try sending a set of valid and invalid plds to ensure
-        # the pld spec is respected.
-        sent: list[Any] = []
-        async with ctx.open_stream() as ipc:
-            print(
-                f'{uid}: streaming all pld types to rent..'
-            )
-
-            # for send_value, expect_send in iter_send_val_items:
-            for send_value in [
-                nsp,
-                str(nsp),
-                None,
-            ]:
-                send_type: Type = type(send_value)
-                print(
-                    f'{uid}: SENDING NEXT pld\n'
-                    f'send_type: {send_type}\n'
-                    f'send_value: {send_value}\n'
-                )
-                try:
-                    await ipc.send(send_value)
-                    sent.append(send_value)
-
-                except ValidationError as valerr:
-                    print(f'{uid} FAILED TO SEND {send_value}!')
-
-                    # false -ve
-                    if add_hooks:
-                        raise RuntimeError(
-                            f'EXPECTED to roundtrip value given spec:\n'
-                            f'rent_pld_spec -> {rent_pld_spec}\n'
-                            f'child_pld_spec -> {child_pld_spec}\n'
-                            f'value = {send_value}: {send_type}\n'
-                        )
-
-                    # true -ve
-                    raise valerr
-                    # continue
-
-            else:
-                print(
-                    f'{uid}: finished sending all values\n'
-                    'Should be exiting stream block!\n'
-                )
-
-        print(f'{uid}: exited streaming block!')
-
-
-
-@cm
-def maybe_apply_codec(codec: MsgCodec|None) -> MsgCodec|None:
-    if codec is None:
-        yield None
-        return
-
-    with apply_codec(codec) as codec:
-        yield codec
-
-
-@pytest.mark.parametrize(
-    'pld_spec',
-    [
-        Any,
-        NamespacePath,
-        NamespacePath|None,  # the "maybe" spec Bo
-    ],
-    ids=[
-        'any_type',
-        'only_nsp_ext',
-        'maybe_nsp_ext',
-    ]
-)
-@pytest.mark.parametrize(
-    'add_hooks',
-    [
-        True,
-        False,
-    ],
-    ids=[
-        'use_codec_hooks',
-        'no_codec_hooks',
-    ],
-)
-def test_ext_types_over_ipc(
-    debug_mode: bool,
-    pld_spec: Union[Type],
-    add_hooks: bool,
-):
-    '''
-    Ensure we can support extension types coverted using
-    `enc/dec_hook()`s passed to the `.msg.limit_plds()` API
-    and that sane errors happen when we try do the same without
-    the codec hooks.
-
-    '''
-    pld_types: set[Type] = _codec.unpack_spec_types(pld_spec)
-
-    async def main():
-
-        # sanity check the default pld-spec beforehand
-        chk_codec_applied(
-            expect_codec=_codec._def_tractor_codec,
-        )
-
-        # extension type we want to send as msg payload
-        nsp = NamespacePath.from_ref(ex_func)
-
-        # ^NOTE, 2 cases:
-        # - codec hooks noto added -> decode nsp as `str`
-        # - codec with hooks -> decode nsp as `NamespacePath`
-        nsp_codec: MsgCodec|None = None
-        if (
-            NamespacePath in pld_types
-            and
-            add_hooks
-        ):
-            nsp_codec = mk_codec(
-                enc_hook=enc_nsp,
-                ext_types=[NamespacePath],
-            )
-
-        async with tractor.open_nursery(
-            debug_mode=debug_mode,
-        ) as an:
-            p: tractor.Portal = await an.start_actor(
-                'sub',
-                enable_modules=[__name__],
-            )
-            with (
-                maybe_apply_codec(nsp_codec) as codec,
-            ):
-                chk_codec_applied(
-                    expect_codec=nsp_codec,
-                    enter_value=codec,
-                )
-                rent_pld_spec_type_strs: list[str] = _exts.enc_type_union(pld_spec)
-
-                # XXX should raise an mte (`MsgTypeError`)
-                # when `add_hooks == False` bc the input
-                # `expect_ipc_send` kwarg has a nsp which can't be
-                # serialized!
-                #
-                # TODO:can we ensure this happens from the
-                # `Return`-side (aka the sub) as well?
-                try:
-                    ctx: tractor.Context
-                    ipc: tractor.MsgStream
-                    async with (
-
-                        # XXX should raise an mte (`MsgTypeError`)
-                        # when `add_hooks == False`..
-                        p.open_context(
-                            send_back_values,
-                            # expect_debug=debug_mode,
-                            rent_pld_spec_type_strs=rent_pld_spec_type_strs,
-                            add_hooks=add_hooks,
-                            # expect_ipc_send=expect_ipc_send,
-                        ) as (ctx, first),
-
-                        ctx.open_stream() as ipc,
-                    ):
-                        with (
-                            limit_plds(
-                                pld_spec,
-                                dec_hook=dec_nsp if add_hooks else None,
-                                ext_types=[NamespacePath]  if add_hooks else None,
-                            ) as pld_dec,
-                        ):
-                            ctx_pld_dec: MsgDec = ctx._pld_rx._pld_dec
-                            assert pld_dec is ctx_pld_dec
-
-                            # if (
-                            #     not add_hooks
-                            #     and
-                            #     NamespacePath in 
-                            # ):
-                            #     pytest.fail('ctx should fail to open without custom enc_hook!?')
-
-                            await ipc.send(nsp)
-                            nsp_rt = await ipc.receive()
-
-                            assert nsp_rt == nsp
-                            assert nsp_rt.load_ref() is ex_func
-
-                # this test passes bc we can go no further!
-                except MsgTypeError as mte:
-                    # if not add_hooks:
-                    #     # teardown nursery
-                    #     await p.cancel_actor()
-                        # return
-
-                    raise mte
-
-            await p.cancel_actor()
-
-    if (
-        NamespacePath in pld_types
-        and
-        add_hooks
-    ):
-        trio.run(main)
-
-    else:
-        with pytest.raises(
-            expected_exception=tractor.RemoteActorError,
-        ) as excinfo:
-            trio.run(main)
-
-        exc = excinfo.value
-        # bc `.started(nsp: NamespacePath)` will raise
-        assert exc.boxed_type is TypeError
-
-
-# def chk_pld_type(
-#     payload_spec: Type[Struct]|Any,
-#     pld: Any,
-
-#     expect_roundtrip: bool|None = None,
-
-# ) -> bool:
-
-#     pld_val_type: Type = type(pld)
-
-#     # TODO: verify that the overridden subtypes
-#     # DO NOT have modified type-annots from original!
-#     # 'Start',  .pld: FuncSpec
-#     # 'StartAck',  .pld: IpcCtxSpec
-#     # 'Stop',  .pld: UNSEt
-#     # 'Error',  .pld: ErrorData
-
-#     codec: MsgCodec = mk_codec(
-#         # NOTE: this ONLY accepts `PayloadMsg.pld` fields of a specified
-#         # type union.
-#         ipc_pld_spec=payload_spec,
-#     )
-
-#     # make a one-off dec to compare with our `MsgCodec` instance
-#     # which does the below `mk_msg_spec()` call internally
-#     ipc_msg_spec: Union[Type[Struct]]
-#     msg_types: list[PayloadMsg[payload_spec]]
-#     (
-#         ipc_msg_spec,
-#         msg_types,
-#     ) = mk_msg_spec(
-#         payload_type_union=payload_spec,
-#     )
-#     _enc = msgpack.Encoder()
-#     _dec = msgpack.Decoder(
-#         type=ipc_msg_spec or Any,  # like `PayloadMsg[Any]`
-#     )
-
-#     assert (
-#         payload_spec
-#         ==
-#         codec.pld_spec
-#     )
-
-#     # assert codec.dec == dec
-#     #
-#     # ^-XXX-^ not sure why these aren't "equal" but when cast
-#     # to `str` they seem to match ?? .. kk
-
-#     assert (
-#         str(ipc_msg_spec)
-#         ==
-#         str(codec.msg_spec)
-#         ==
-#         str(_dec.type)
-#         ==
-#         str(codec.dec.type)
-#     )
-
-#     # verify the boxed-type for all variable payload-type msgs.
-#     if not msg_types:
-#         breakpoint()
-
-#     roundtrip: bool|None = None
-#     pld_spec_msg_names: list[str] = [
-#         td.__name__ for td in _payload_msgs
-#     ]
-#     for typedef in msg_types:
-
-#         skip_runtime_msg: bool = typedef.__name__ not in pld_spec_msg_names
-#         if skip_runtime_msg:
-#             continue
-
-#         pld_field = structs.fields(typedef)[1]
-#         assert pld_field.type is payload_spec # TODO-^ does this need to work to get all subtypes to adhere?
-
-#         kwargs: dict[str, Any] = {
-#             'cid': '666',
-#             'pld': pld,
-#         }
-#         enc_msg: PayloadMsg = typedef(**kwargs)
-
-#         _wire_bytes: bytes = _enc.encode(enc_msg)
-#         wire_bytes: bytes = codec.enc.encode(enc_msg)
-#         assert _wire_bytes == wire_bytes
-
-#         ve: ValidationError|None = None
-#         try:
-#             dec_msg = codec.dec.decode(wire_bytes)
-#             _dec_msg = _dec.decode(wire_bytes)
-
-#             # decoded msg and thus payload should be exactly same!
-#             assert (roundtrip := (
-#                 _dec_msg
-#                 ==
-#                 dec_msg
-#                 ==
-#                 enc_msg
-#             ))
-
-#             if (
-#                 expect_roundtrip is not None
-#                 and expect_roundtrip != roundtrip
-#             ):
-#                 breakpoint()
-
-#             assert (
-#                 pld
-#                 ==
-#                 dec_msg.pld
-#                 ==
-#                 enc_msg.pld
-#             )
-#             # assert (roundtrip := (_dec_msg == enc_msg))
-
-#         except ValidationError as _ve:
-#             ve = _ve
-#             roundtrip: bool = False
-#             if pld_val_type is payload_spec:
-#                 raise ValueError(
-#                    'Got `ValidationError` despite type-var match!?\n'
-#                     f'pld_val_type: {pld_val_type}\n'
-#                     f'payload_type: {payload_spec}\n'
-#                 ) from ve
-
-#             else:
-#                 # ow we good cuz the pld spec mismatched.
-#                 print(
-#                     'Got expected `ValidationError` since,\n'
-#                     f'{pld_val_type} is not {payload_spec}\n'
-#                 )
-#         else:
-#             if (
-#                 payload_spec is not Any
-#                 and
-#                 pld_val_type is not payload_spec
-#             ):
-#                 raise ValueError(
-#                    'DID NOT `ValidationError` despite expected type match!?\n'
-#                     f'pld_val_type: {pld_val_type}\n'
-#                     f'payload_type: {payload_spec}\n'
-#                 )
-
-#     # full code decode should always be attempted!
-#     if roundtrip is None:
-#         breakpoint()
-
-#     return roundtrip
-
-
-# ?TODO? maybe remove since covered in the newer `test_pldrx_limiting`
-# via end-2-end testing of all this?
-# -[ ] IOW do we really NEED this lowlevel unit testing?
-#
-# def test_limit_msgspec(
-#     debug_mode: bool,
-# ):
-#     '''
-#     Internals unit testing to verify that type-limiting an IPC ctx's
-#     msg spec with `Pldrx.limit_plds()` results in various
-#     encapsulated `msgspec` object settings and state.
-
-#     '''
-#     async def main():
-#         async with tractor.open_root_actor(
-#             debug_mode=debug_mode,
-#         ):
-#             # ensure we can round-trip a boxing `PayloadMsg`
-#             assert chk_pld_type(
-#                 payload_spec=Any,
-#                 pld=None,
-#                 expect_roundtrip=True,
-#             )
-
-#             # verify that a mis-typed payload value won't decode
-#             assert not chk_pld_type(
-#                 payload_spec=int,
-#                 pld='doggy',
-#             )
-
-#             # parametrize the boxed `.pld` type as a custom-struct
-#             # and ensure that parametrization propagates
-#             # to all payload-msg-spec-able subtypes!
-#             class CustomPayload(Struct):
-#                 name: str
-#                 value: Any
-
-#             assert not chk_pld_type(
-#                 payload_spec=CustomPayload,
-#                 pld='doggy',
-#             )
-
-#             assert chk_pld_type(
-#                 payload_spec=CustomPayload,
-#                 pld=CustomPayload(name='doggy', value='urmom')
-#             )
-
-#             # yah, we can `.pause_from_sync()` now!
-#             # breakpoint()
-
-#     trio.run(main)

tests/test_inter_peer_cancellation.py

@@ -16,11 +16,6 @@ from tractor import (  # typing
     Portal,
     Context,
     ContextCancelled,
-    RemoteActorError,
-)
-from tractor._testing import (
-    # tractor_test,
-    expect_ctxc,
 )
 
 # XXX TODO cases:
@@ -55,10 +50,9 @@ from tractor._testing import (
 
 
 @tractor.context
-async def open_stream_then_sleep_forever(
+async def sleep_forever(
     ctx: Context,
     expect_ctxc: bool = False,
-
 ) -> None:
     '''
     Sync the context, open a stream then just sleep.
@@ -68,10 +62,6 @@ async def open_stream_then_sleep_forever(
     '''
     try:
         await ctx.started()
-
-        # NOTE: the below means this child will send a `Stop`
-        # to it's parent-side task despite that side never
-        # opening a stream itself.
         async with ctx.open_stream():
             await trio.sleep_forever()
 
@@ -105,7 +95,7 @@ async def error_before_started(
     '''
     async with tractor.wait_for_actor('sleeper') as p2:
         async with (
-            p2.open_context(open_stream_then_sleep_forever) as (peer_ctx, first),
+            p2.open_context(sleep_forever) as (peer_ctx, first),
             peer_ctx.open_stream(),
         ):
             # NOTE: this WAS inside an @acm body but i factored it
@@ -166,11 +156,10 @@ def test_do_not_swallow_error_before_started_by_remote_contextcancelled(
             ):
                 await trio.sleep_forever()
 
-    with pytest.raises(RemoteActorError) as excinfo:
+    with pytest.raises(tractor.RemoteActorError) as excinfo:
         trio.run(main)
 
-    rae = excinfo.value
-    assert rae.boxed_type is TypeError
+    assert excinfo.value.type == TypeError
 
 
 @tractor.context
@@ -190,10 +179,6 @@ async def sleep_a_bit_then_cancel_peer(
         await trio.sleep(cancel_after)
         await peer.cancel_actor()
 
-        # such that we're cancelled by our rent ctx-task
-        await trio.sleep(3)
-        print('CANCELLER RETURNING!')
-
 
 @tractor.context
 async def stream_ints(
@@ -209,13 +194,9 @@ async def stream_ints(
 @tractor.context
 async def stream_from_peer(
     ctx: Context,
-    debug_mode: bool,
     peer_name: str = 'sleeper',
 ) -> None:
 
-    # sanity
-    assert tractor._state.debug_mode() == debug_mode
-
     peer: Portal
     try:
         async with (
@@ -249,54 +230,15 @@ async def stream_from_peer(
                 assert msg is not None
                 print(msg)
 
-    # NOTE: cancellation of the (sleeper) peer should always cause
-    # a `ContextCancelled` raise in this streaming actor.
-    except ContextCancelled as _ctxc:
-        ctxc = _ctxc
+    # NOTE: cancellation of the (sleeper) peer should always
+    # cause a `ContextCancelled` raise in this streaming
+    # actor.
+    except ContextCancelled as ctxc:
+        ctxerr = ctxc
 
-        # print("TRYING TO ENTER PAUSSE!!!")
-        # await tractor.pause(shield=True)
-        re: ContextCancelled = peer_ctx._remote_error
-
-        # XXX YES XXX, remote error should be unpacked only once!
-        assert (
-            re
-            is
-            peer_ctx.maybe_error
-            is
-            ctxc
-            is
-            peer_ctx._local_error
-        )
-        # NOTE: these errors should all match!
-        #   ------ - ------
-        # XXX [2024-05-03] XXX
-        #   ------ - ------
-        # broke this due to a re-raise inside `.msg._ops.drain_to_final_msg()`
-        # where the `Error()` msg was directly raising the ctxc
-        # instead of just returning up to the caller inside
-        # `Context.return()` which would results in a diff instance of
-        # the same remote error bubbling out above vs what was
-        # already unpacked and set inside `Context.
-        assert (
-            peer_ctx._remote_error.msgdata
-            ==
-            ctxc.msgdata
-        )
-        # ^-XXX-^ notice the data is of course the exact same.. so
-        # the above larger assert makes sense to also always be true!
-
-        # XXX YES XXX, bc should be exact same msg instances
-        assert peer_ctx._remote_error._ipc_msg is ctxc._ipc_msg
-
-        # XXX NO XXX, bc new one always created for property accesss
-        assert peer_ctx._remote_error.ipc_msg != ctxc.ipc_msg
-
-        # the peer ctx is the canceller even though it's canceller
-        # is the "canceller" XD
-        assert peer_name in peer_ctx.canceller
-
-        assert "canceller" in ctxc.canceller
+        assert peer_ctx._remote_error is ctxerr
+        assert peer_ctx._remote_error.msgdata == ctxerr.msgdata
+        assert peer_ctx.canceller == ctxerr.canceller
 
         # caller peer should not be the cancel requester
         assert not ctx.cancel_called
@@ -320,17 +262,17 @@ async def stream_from_peer(
 
         # TODO / NOTE `.canceller` won't have been set yet
         # here because that machinery is inside
-        # `Portal.open_context().__aexit__()` BUT, if we had
+        # `.open_context().__aexit__()` BUT, if we had
         # a way to know immediately (from the last
         # checkpoint) that cancellation was due to
         # a remote, we COULD assert this here..see,
         # https://github.com/goodboy/tractor/issues/368
         #
-        # await tractor.pause()
         # assert 'canceller' in ctx.canceller
 
         # root/parent actor task should NEVER HAVE cancelled us!
         assert not ctx.canceller
+        assert 'canceller' in peer_ctx.canceller
 
         raise
         # TODO: IN THEORY we could have other cases depending on
@@ -350,7 +292,6 @@ async def stream_from_peer(
 @pytest.mark.parametrize(
     'error_during_ctxerr_handling',
     [False, True],
-    ids=lambda item: f'rte_during_ctxerr={item}',
 )
 def test_peer_canceller(
     error_during_ctxerr_handling: bool,
@@ -430,13 +371,12 @@ def test_peer_canceller(
             try:
                 async with (
                     sleeper.open_context(
-                        open_stream_then_sleep_forever,
+                        sleep_forever,
                         expect_ctxc=True,
                     ) as (sleeper_ctx, sent),
 
                     just_caller.open_context(
                         stream_from_peer,
-                        debug_mode=debug_mode,
                     ) as (caller_ctx, sent),
 
                     canceller.open_context(
@@ -462,11 +402,10 @@ def test_peer_canceller(
 
                     # should always raise since this root task does
                     # not request the sleeper cancellation ;)
-                    except ContextCancelled as _ctxc:
-                        ctxc = _ctxc
+                    except ContextCancelled as ctxerr:
                         print(
                             'CAUGHT REMOTE CONTEXT CANCEL\n\n'
-                            f'{ctxc}\n'
+                            f'{ctxerr}\n'
                         )
 
                         # canceller and caller peers should not
@@ -477,7 +416,7 @@ def test_peer_canceller(
                         # we were not the actor, our peer was
                         assert not sleeper_ctx.cancel_acked
 
-                        assert ctxc.canceller[0] == 'canceller'
+                        assert ctxerr.canceller[0] == 'canceller'
 
                         # XXX NOTE XXX: since THIS `ContextCancelled`
                         # HAS NOT YET bubbled up to the
@@ -488,7 +427,7 @@ def test_peer_canceller(
 
                         # CASE_1: error-during-ctxc-handling,
                         if error_during_ctxerr_handling:
-                            raise RuntimeError('Simulated RTE re-raise during ctxc handling')
+                            raise RuntimeError('Simulated error during teardown')
 
                         # CASE_2: standard teardown inside in `.open_context()` block
                         raise
@@ -553,17 +492,8 @@ def test_peer_canceller(
                 #   should be cancelled by US.
                 #
                 if error_during_ctxerr_handling:
-                    print(f'loc_err: {_loc_err}\n')
                     assert isinstance(loc_err, RuntimeError)
-
-                    # since we do a rte reraise above, the
-                    # `.open_context()` error handling should have
-                    # raised a local rte, thus the internal
-                    # `.open_context()` enterer task's
-                    # cancel-scope should have raised the RTE, NOT
-                    # a `trio.Cancelled` due to a local
-                    # `._scope.cancel()` call.
-                    assert not sleeper_ctx._scope.cancelled_caught
+                    print(f'_loc_err: {_loc_err}\n')
                     # assert sleeper_ctx._local_error is _loc_err
                     # assert sleeper_ctx._local_error is _loc_err
                     assert not (
@@ -587,25 +517,25 @@ def test_peer_canceller(
 
                         assert ctx.cancel_called
 
-                        if ctx is sleeper_ctx:
-                            assert 'canceller' in re.canceller
-                            assert 'sleeper' in ctx.canceller
-
-                        if ctx is canceller_ctx:
+                        if (
+                            ctx is sleeper_ctx
+                            or ctx is caller_ctx
+                        ):
                             assert (
                                 re.canceller
                                 ==
-                                root.uid
+                                ctx.canceller
+                                ==
+                                canceller.channel.uid
                             )
 
-                        else:  # the other 2 ctxs
+                        else:
                             assert (
-                                isinstance(re, ContextCancelled)
-                                and (
-                                    re.canceller
-                                    ==
-                                    canceller.channel.uid
-                                )
+                                re.canceller
+                                ==
+                                ctx.canceller
+                                ==
+                                root.uid
                             )
 
                     # since the sleeper errors while handling a
@@ -628,24 +558,15 @@ def test_peer_canceller(
                 #   propagated
                 #
                 else:
-                    # since sleeper_ctx.result() IS called above
-                    # we should have (silently) absorbed the
-                    # corresponding `ContextCancelled` for it and
-                    # `._scope.cancel()` should never have been
-                    # called.
-                    assert not sleeper_ctx._scope.cancelled_caught
-
                     assert isinstance(loc_err, ContextCancelled)
-
-                    # the received remote error's `.canceller`
-                    # will of course be the "canceller" actor BUT
-                    # the canceller set on the local handle to
-                    # `sleeper_ctx` will be the "sleeper" uid
-                    # since it's the actor that relayed us the
-                    # error which was **caused** by the
-                    # "canceller".
-                    assert 'sleeper' in sleeper_ctx.canceller
-                    assert 'canceller' == loc_err.canceller[0]
+                    assert loc_err.canceller == sleeper_ctx.canceller
+                    assert (
+                        loc_err.canceller[0]
+                        ==
+                        sleeper_ctx.canceller[0]
+                        ==
+                        'canceller'
+                    )
 
                     # the sleeper's remote error is the error bubbled
                     # out of the context-stack above!
@@ -704,31 +625,20 @@ def test_peer_canceller(
 
                             assert not ctx._remote_error
 
-                            # neither of the `caller/canceller_ctx` should
-                            # have called `ctx.cancel()` bc the
-                            # canceller's task internally issues
-                            # a `Portal.cancel_actor()` to the
-                            # sleeper and thus never should call
-                            # `ctx.cancel()` per say UNLESS the
-                            # sleeper's `.result()` call above
-                            # ctxc exception results in the
-                            # canceller's
-                            # `.open_context().__aexit__()` error
-                            # handling to kick in BEFORE a remote
-                            # error is delivered - which since
-                            # we're asserting what we are above,
-                            # that should normally be the case
-                            # right?
+                            # the `canceller_ctx` shouldn't
+                            # have called `ctx.cancel()` either!
                             #
+                            # since its context was remotely
+                            # cancelled, we never needed to call
+                            # `Context.cancel()` bc the far end
+                            # task already done by the peer and
+                            # also we never 
                             assert not ctx.cancel_called
-                            #
-                            # assert ctx.cancel_called
-                            # orig ^
 
                             # TODO: figure out the details of this..?
                             # if you look the `._local_error` here
                             # is a multi of ctxc + 2 Cancelleds?
-                            # assert not ctx._scope.cancelled_caught
+                            # assert not ctx.cancelled_caught
 
                             assert (
                                 not ctx.cancel_called
@@ -798,16 +708,14 @@ def test_peer_canceller(
         with pytest.raises(ContextCancelled) as excinfo:
             trio.run(main)
 
-        assert excinfo.value.boxed_type == ContextCancelled
+        assert excinfo.value.type == ContextCancelled
         assert excinfo.value.canceller[0] == 'canceller'
 
 
 @tractor.context
 async def basic_echo_server(
     ctx: Context,
-    peer_name: str = 'wittle_bruv',
-
-    err_after: int|None = None,
+    peer_name: str = 'stepbro',
 
 ) -> None:
     '''
@@ -835,30 +743,17 @@ async def basic_echo_server(
             # assert 0
             await ipc.send(resp)
 
-            if (
-                err_after
-                and i > err_after
-            ):
-                raise RuntimeError(
-                    f'Simulated error in `{peer_name}`'
-                )
-
 
 @tractor.context
 async def serve_subactors(
     ctx: Context,
     peer_name: str,
-    debug_mode: bool,
 
 ) -> None:
     async with open_nursery() as an:
-
-        # sanity
-        assert tractor._state.debug_mode() == debug_mode
-
         await ctx.started(peer_name)
-        async with ctx.open_stream() as ipc:
-            async for msg in ipc:
+        async with ctx.open_stream() as reqs:
+            async for msg in reqs:
                 peer_name: str = msg
                 peer: Portal = await an.start_actor(
                     name=peer_name,
@@ -869,7 +764,7 @@ async def serve_subactors(
                     f'{peer_name}\n'
                     f'|_{peer}\n'
                 )
-                await ipc.send((
+                await reqs.send((
                     peer.chan.uid,
                     peer.chan.raddr,
                 ))
@@ -881,20 +776,14 @@ async def serve_subactors(
 async def client_req_subactor(
     ctx: Context,
     peer_name: str,
-    debug_mode: bool,
 
     # used to simulate a user causing an error to be raised
     # directly in thread (like a KBI) to better replicate the
     # case where a `modden` CLI client would hang afer requesting
     # a `Context.cancel()` to `bigd`'s wks spawner.
     reraise_on_cancel: str|None = None,
-    sub_err_after: int|None = None,
 
 ) -> None:
-    # sanity
-    if debug_mode:
-        assert tractor._state.debug_mode()
-
     # TODO: other cases to do with sub lifetimes:
     # -[ ] test that we can have the server spawn a sub
     #   that lives longer then ctx with this client.
@@ -916,7 +805,6 @@ async def client_req_subactor(
         spawner.open_context(
             serve_subactors,
             peer_name=peer_name,
-            debug_mode=debug_mode,
         ) as (spawner_ctx, first),
     ):
         assert first == peer_name
@@ -938,7 +826,6 @@ async def client_req_subactor(
             await tell_little_bro(
                 actor_name=sub_uid[0],
                 caller='client',
-                err_after=sub_err_after,
             )
 
             # TODO: test different scope-layers of
@@ -950,7 +837,9 @@ async def client_req_subactor(
             # TODO: would be super nice to have a special injected
             # cancel type here (maybe just our ctxc) but using
             # some native mechanism in `trio` :p
-            except trio.Cancelled as err:
+            except (
+                trio.Cancelled
+            ) as err:
                 _err = err
                 if reraise_on_cancel:
                     errtype = globals()['__builtins__'][reraise_on_cancel]
@@ -977,9 +866,7 @@ async def client_req_subactor(
 
 async def tell_little_bro(
     actor_name: str,
-
-    caller: str = '',
-    err_after: int|None = None,
+    caller: str = ''
 ):
     # contact target actor, do a stream dialog.
     async with (
@@ -988,12 +875,10 @@ async def tell_little_bro(
         ) as lb,
         lb.open_context(
             basic_echo_server,
-
-            # XXX proxy any delayed err condition
-            err_after=err_after,
         ) as (sub_ctx, first),
-
-        sub_ctx.open_stream() as echo_ipc,
+        sub_ctx.open_stream(
+            basic_echo_server,
+        ) as echo_ipc,
     ):
         actor: Actor = current_actor()
         uid: tuple = actor.uid
@@ -1020,15 +905,9 @@ async def tell_little_bro(
     'raise_client_error',
     [None, 'KeyboardInterrupt'],
 )
-@pytest.mark.parametrize(
-    'raise_sub_spawn_error_after',
-    [None, 50],
-)
 def test_peer_spawns_and_cancels_service_subactor(
     debug_mode: bool,
     raise_client_error: str,
-    reg_addr: tuple[str, int],
-    raise_sub_spawn_error_after: int|None,
 ):
     # NOTE: this tests for the modden `mod wks open piker` bug
     # discovered as part of implementing workspace ctx
@@ -1042,21 +921,10 @@ def test_peer_spawns_and_cancels_service_subactor(
     #   and the server's spawned child should cancel and terminate!
     peer_name: str = 'little_bro'
 
-    def check_inner_rte(rae: RemoteActorError):
-        '''
-        Validate the little_bro's relayed inception!
-
-        '''
-        assert rae.boxed_type is RemoteActorError
-        assert rae.src_type is RuntimeError
-        assert 'client' in rae.relay_uid
-        assert peer_name in rae.src_uid
-
     async def main():
         async with tractor.open_nursery(
             # NOTE: to halt the peer tasks on ctxc, uncomment this.
             debug_mode=debug_mode,
-            registry_addrs=[reg_addr],
         ) as an:
             server: Portal = await an.start_actor(
                 (server_name := 'spawn_server'),
@@ -1075,24 +943,14 @@ def test_peer_spawns_and_cancels_service_subactor(
                     server.open_context(
                         serve_subactors,
                         peer_name=peer_name,
-                        debug_mode=debug_mode,
-
                     ) as (spawn_ctx, first),
 
                     client.open_context(
                         client_req_subactor,
                         peer_name=peer_name,
-                        debug_mode=debug_mode,
                         reraise_on_cancel=raise_client_error,
-
-                        # trigger for error condition in sub
-                        # during streaming.
-                        sub_err_after=raise_sub_spawn_error_after,
-
                     ) as (client_ctx, client_says),
                 ):
-                    root: Actor = current_actor()
-                    spawner_uid: tuple = spawn_ctx.chan.uid
                     print(
                         f'Server says: {first}\n'
                         f'Client says: {client_says}\n'
@@ -1102,7 +960,6 @@ def test_peer_spawns_and_cancels_service_subactor(
                     # (grandchild of this root actor) "little_bro"
                     # and ensure we can also use it as an echo
                     # server.
-                    sub: Portal
                     async with tractor.wait_for_actor(
                         name=peer_name,
                     ) as sub:
@@ -1114,138 +971,56 @@ def test_peer_spawns_and_cancels_service_subactor(
                         f'.uid: {sub.actor.uid}\n'
                         f'chan.raddr: {sub.chan.raddr}\n'
                     )
+                    await tell_little_bro(
+                        actor_name=peer_name,
+                        caller='root',
+                    )
 
-                    async with expect_ctxc(
-                        yay=raise_sub_spawn_error_after,
-                        reraise=False,
-                    ):
-                        await tell_little_bro(
-                            actor_name=peer_name,
-                            caller='root',
-                        )
+                    # signal client to raise a KBI
+                    await client_ctx.cancel()
+                    print('root cancelled client, checking that sub-spawn is down')
 
-                    if not raise_sub_spawn_error_after:
+                    async with tractor.find_actor(
+                        name=peer_name,
+                    ) as sub:
+                        assert not sub
 
-                        # signal client to cancel and maybe raise a KBI
-                        await client_ctx.cancel()
-                        print(
-                            '-> root cancelling client,\n'
-                            '-> root checking `client_ctx.result()`,\n'
-                            f'-> checking that sub-spawn {peer_name} is down\n'
-                        )
+                    print('root cancelling server/client sub-actors')
 
-                    try:
-                        res = await client_ctx.result(hide_tb=False)
-
-                        # in remote (relayed inception) error
-                        # case, we should error on the line above!
-                        if raise_sub_spawn_error_after:
-                            pytest.fail(
-                                'Never rxed proxied `RemoteActorError[RuntimeError]` !?'
-                            )
-
-                        assert isinstance(res, ContextCancelled)
-                        assert client_ctx.cancel_acked
-                        assert res.canceller == root.uid
-
-                    except RemoteActorError as rae:
-                        _err = rae
-                        assert raise_sub_spawn_error_after
-
-                        # since this is a "relayed error" via the client
-                        # sub-actor, it is expected to be
-                        # a `RemoteActorError` boxing another
-                        # `RemoteActorError` otherwise known as
-                        #  an "inception" (from `trio`'s parlance)
-                        # ((or maybe a "Matryoshka" and/or "matron"
-                        # in our own working parlance)) which
-                        # contains the source error from the
-                        # little_bro: a `RuntimeError`.
-                        #
-                        check_inner_rte(rae)
-                        assert rae.relay_uid == client.chan.uid
-                        assert rae.src_uid == sub.chan.uid
-
-                        assert not client_ctx.cancel_acked
-                        assert (
-                            client_ctx.maybe_error
-                            is client_ctx.outcome
-                            is rae
-                        )
-                        raise
-                        # await tractor.pause()
-
-                    else:
-                        assert not raise_sub_spawn_error_after
-
-                        # cancelling the spawner sub should
-                        # transitively cancel it's sub, the little
-                        # bruv.
-                        print('root cancelling server/client sub-actors')
-                        await spawn_ctx.cancel()
-                        async with tractor.find_actor(
-                            name=peer_name,
-                        ) as sub:
-                            assert not sub
+                    # await tractor.pause()
+                    res = await client_ctx.result(hide_tb=False)
+                    assert isinstance(res, ContextCancelled)
+                    assert client_ctx.cancel_acked
+                    assert res.canceller == current_actor().uid
 
+                    await spawn_ctx.cancel()
                     # await server.cancel_actor()
 
-            except RemoteActorError as rae:
-                # XXX more-or-less same as above handler
-                # this is just making sure the error bubbles out
-                # of the 
-                _err = rae
-                assert raise_sub_spawn_error_after
-                raise
-
             # since we called `.cancel_actor()`, `.cancel_ack`
             # will not be set on the ctx bc `ctx.cancel()` was not
             # called directly fot this confext.
             except ContextCancelled as ctxc:
-                _ctxc = ctxc
-                print(
-                    f'{root.uid} caught ctxc from ctx with {client_ctx.chan.uid}\n'
-                    f'{repr(ctxc)}\n'
-                )
-
-                if not raise_sub_spawn_error_after:
-                    assert ctxc.canceller == root.uid
-                else:
-                    assert ctxc.canceller == spawner_uid
-
+                print('caught ctxc from contexts!')
+                assert ctxc.canceller == current_actor().uid
                 assert ctxc is spawn_ctx.outcome
                 assert ctxc is spawn_ctx.maybe_error
                 raise
 
-            if raise_sub_spawn_error_after:
-                pytest.fail(
-                    'context block(s) in PARENT never raised?!?'
-                )
+            # assert spawn_ctx.cancel_acked
+            assert spawn_ctx.cancel_acked
+            assert client_ctx.cancel_acked
 
-            if not raise_sub_spawn_error_after:
-                # assert spawn_ctx.cancel_acked
-                assert spawn_ctx.cancel_acked
-                assert client_ctx.cancel_acked
+            await client.cancel_actor()
+            await server.cancel_actor()
 
-                await client.cancel_actor()
-                await server.cancel_actor()
+            # WOA WOA WOA! we need this to close..!!!??
+            # that's super bad XD
 
-                # WOA WOA WOA! we need this to close..!!!??
-                # that's super bad XD
+            # TODO: why isn't this working!?!?
+            # we're now outside the `.open_context()` block so
+            # the internal `Context._scope: CancelScope` should be
+            # gracefully "closed" ;)
 
-                # TODO: why isn't this working!?!?
-                # we're now outside the `.open_context()` block so
-                # the internal `Context._scope: CancelScope` should be
-                # gracefully "closed" ;)
+            # assert spawn_ctx.cancelled_caught
 
-                # assert spawn_ctx.cancelled_caught
-
-    if raise_sub_spawn_error_after:
-        with pytest.raises(RemoteActorError) as excinfo:
-            trio.run(main)
-
-        rae: RemoteActorError = excinfo.value
-        check_inner_rte(rae)
-
-    else:
-        trio.run(main)
+    trio.run(main)

tests/test_legacy_one_way_streaming.py

@@ -9,7 +9,7 @@ import trio
 import tractor
 import pytest
 
-from tractor._testing import tractor_test
+from conftest import tractor_test
 
 
 def test_must_define_ctx():
@@ -38,13 +38,10 @@ async def async_gen_stream(sequence):
     assert cs.cancelled_caught
 
 
-# TODO: deprecated either remove entirely
-# or re-impl in terms of `MsgStream` one-sides
-# wrapper, but at least remove `Portal.open_stream_from()`
 @tractor.stream
 async def context_stream(
     ctx: tractor.Context,
-    sequence: list[int],
+    sequence
 ):
     for i in sequence:
         await ctx.send_yield(i)

tests/test_local.py

@@ -7,7 +7,7 @@ import pytest
 import trio
 import tractor
 
-from tractor._testing import tractor_test
+from conftest import tractor_test
 
 
 @pytest.mark.trio
@@ -38,7 +38,7 @@ async def test_self_is_registered_localportal(reg_addr):
     "Verify waiting on the arbiter to register itself using a local portal."
     actor = tractor.current_actor()
     assert actor.is_arbiter
-    async with tractor.get_registry(*reg_addr) as portal:
+    async with tractor.get_arbiter(*reg_addr) as portal:
         assert isinstance(portal, tractor._portal.LocalPortal)
 
         with trio.fail_after(0.2):

tests/test_multi_program.py

@@ -7,10 +7,8 @@ import time
 import pytest
 import trio
 import tractor
-from tractor._testing import (
+from conftest import (
     tractor_test,
-)
-from .conftest import (
     sig_prog,
     _INT_SIGNAL,
     _INT_RETURN_CODE,
@@ -32,7 +30,7 @@ def test_abort_on_sigint(daemon):
 @tractor_test
 async def test_cancel_remote_arbiter(daemon, reg_addr):
     assert not tractor.current_actor().is_arbiter
-    async with tractor.get_registry(*reg_addr) as portal:
+    async with tractor.get_arbiter(*reg_addr) as portal:
         await portal.cancel_actor()
 
     time.sleep(0.1)
@@ -41,7 +39,7 @@ async def test_cancel_remote_arbiter(daemon, reg_addr):
 
     # no arbiter socket should exist
     with pytest.raises(OSError):
-        async with tractor.get_registry(*reg_addr) as portal:
+        async with tractor.get_arbiter(*reg_addr) as portal:
             pass
 
 

tests/test_pldrx_limiting.py

@@ -1,364 +0,0 @@
-'''
-Audit sub-sys APIs from `.msg._ops`
-mostly for ensuring correct `contextvars`
-related settings around IPC contexts.
-
-'''
-from contextlib import (
-    asynccontextmanager as acm,
-)
-
-from msgspec import (
-    Struct,
-)
-import pytest
-import trio
-
-import tractor
-from tractor import (
-    Context,
-    MsgTypeError,
-    current_ipc_ctx,
-    Portal,
-)
-from tractor.msg import (
-    _ops as msgops,
-    Return,
-)
-from tractor.msg import (
-    _codec,
-)
-from tractor.msg.types import (
-    log,
-)
-
-
-class PldMsg(
-    Struct,
-
-    # TODO: with multiple structs in-spec we need to tag them!
-    # -[ ] offer a built-in `PldMsg` type to inherit from which takes
-    #      case of these details?
-    #
-    # https://jcristharif.com/msgspec/structs.html#tagged-unions
-    # tag=True,
-    # tag_field='msg_type',
-):
-    field: str
-
-
-maybe_msg_spec = PldMsg|None
-
-
-@acm
-async def maybe_expect_raises(
-    raises: BaseException|None = None,
-    ensure_in_message: list[str]|None = None,
-    post_mortem: bool = False,
-    timeout: int = 3,
-) -> None:
-    '''
-    Async wrapper for ensuring errors propagate from the inner scope.
-
-    '''
-    if tractor._state.debug_mode():
-        timeout += 999
-
-    with trio.fail_after(timeout):
-        try:
-            yield
-        except BaseException as _inner_err:
-            inner_err = _inner_err
-            # wasn't-expected to error..
-            if raises is None:
-                raise
-
-            else:
-                assert type(inner_err) is raises
-
-                # maybe check for error txt content
-                if ensure_in_message:
-                    part: str
-                    err_repr: str = repr(inner_err)
-                    for part in ensure_in_message:
-                        for i, arg in enumerate(inner_err.args):
-                            if part in err_repr:
-                                break
-                        # if part never matches an arg, then we're
-                        # missing a match.
-                        else:
-                            raise ValueError(
-                                'Failed to find error message content?\n\n'
-                                f'expected: {ensure_in_message!r}\n'
-                                f'part: {part!r}\n\n'
-                                f'{inner_err.args}'
-                        )
-
-                if post_mortem:
-                    await tractor.post_mortem()
-
-        else:
-            if raises:
-                raise RuntimeError(
-                    f'Expected a {raises.__name__!r} to be raised?'
-                )
-
-
-@tractor.context(
-    pld_spec=maybe_msg_spec,
-)
-async def child(
-    ctx: Context,
-    started_value: int|PldMsg|None,
-    return_value: str|None,
-    validate_pld_spec: bool,
-    raise_on_started_mte: bool = True,
-
-) -> None:
-    '''
-    Call ``Context.started()`` more then once (an error).
-
-    '''
-    expect_started_mte: bool = started_value == 10
-
-    # sanaity check that child RPC context is the current one
-    curr_ctx: Context = current_ipc_ctx()
-    assert ctx is curr_ctx
-
-    rx: msgops.PldRx = ctx._pld_rx
-    curr_pldec: _codec.MsgDec = rx.pld_dec
-
-    ctx_meta: dict = getattr(
-        child,
-        '_tractor_context_meta',
-        None,
-    )
-    if ctx_meta:
-        assert (
-            ctx_meta['pld_spec']
-            is curr_pldec.spec
-            is curr_pldec.pld_spec
-        )
-
-    # 2 cases: hdndle send-side and recv-only validation
-    # - when `raise_on_started_mte == True`, send validate
-    # - else, parent-recv-side only validation
-    mte: MsgTypeError|None = None
-    try:
-        await ctx.started(
-            value=started_value,
-            validate_pld_spec=validate_pld_spec,
-        )
-
-    except MsgTypeError as _mte:
-        mte = _mte
-        log.exception('started()` raised an MTE!\n')
-        if not expect_started_mte:
-            raise RuntimeError(
-                'Child-ctx-task SHOULD NOT HAVE raised an MTE for\n\n'
-                f'{started_value!r}\n'
-            )
-
-        boxed_div: str = '------ - ------'
-        assert boxed_div not in mte._message
-        assert boxed_div not in mte.tb_str
-        assert boxed_div not in repr(mte)
-        assert boxed_div not in str(mte)
-        mte_repr: str = repr(mte)
-        for line in mte.message.splitlines():
-            assert line in mte_repr
-
-        # since this is a *local error* there should be no
-        # boxed traceback content!
-        assert not mte.tb_str
-
-        # propagate to parent?
-        if raise_on_started_mte:
-            raise
-
-    # no-send-side-error fallthrough
-    if (
-        validate_pld_spec
-        and
-        expect_started_mte
-    ):
-        raise RuntimeError(
-            'Child-ctx-task SHOULD HAVE raised an MTE for\n\n'
-            f'{started_value!r}\n'
-        )
-
-    assert (
-        not expect_started_mte
-        or
-        not validate_pld_spec
-    )
-
-    # if wait_for_parent_to_cancel:
-    #     ...
-    #
-    # ^-TODO-^ logic for diff validation policies on each side:
-    #
-    # -[ ] ensure that if we don't validate on the send
-    #   side, that we are eventually error-cancelled by our
-    #   parent due to the bad `Started` payload!
-    # -[ ] the boxed error should be srced from the parent's
-    #   runtime NOT ours!
-    # -[ ] we should still error on bad `return_value`s
-    #   despite the parent not yet error-cancelling us?
-    #   |_ how do we want the parent side to look in that
-    #     case?
-    #     -[ ] maybe the equiv of "during handling of the
-    #       above error another occurred" for the case where
-    #       the parent sends a MTE to this child and while
-    #       waiting for the child to terminate it gets back
-    #       the MTE for this case?
-    #
-
-    # XXX should always fail on recv side since we can't
-    # really do much else beside terminate and relay the
-    # msg-type-error from this RPC task ;)
-    return return_value
-
-
-@pytest.mark.parametrize(
-    'return_value',
-    [
-        'yo',
-        None,
-    ],
-    ids=[
-        'return[invalid-"yo"]',
-        'return[valid-None]',
-    ],
-)
-@pytest.mark.parametrize(
-    'started_value',
-    [
-        10,
-        PldMsg(field='yo'),
-    ],
-    ids=[
-        'Started[invalid-10]',
-        'Started[valid-PldMsg]',
-    ],
-)
-@pytest.mark.parametrize(
-    'pld_check_started_value',
-    [
-        True,
-        False,
-    ],
-    ids=[
-        'check-started-pld',
-        'no-started-pld-validate',
-    ],
-)
-def test_basic_payload_spec(
-    debug_mode: bool,
-    loglevel: str,
-    return_value: str|None,
-    started_value: int|PldMsg,
-    pld_check_started_value: bool,
-):
-    '''
-    Validate the most basic `PldRx` msg-type-spec semantics around
-    a IPC `Context` endpoint start, started-sync, and final return
-    value depending on set payload types and the currently applied
-    pld-spec.
-
-    '''
-    invalid_return: bool = return_value == 'yo'
-    invalid_started: bool = started_value == 10
-
-    async def main():
-        async with tractor.open_nursery(
-            debug_mode=debug_mode,
-            loglevel=loglevel,
-        ) as an:
-            p: Portal = await an.start_actor(
-                'child',
-                enable_modules=[__name__],
-            )
-
-            # since not opened yet.
-            assert current_ipc_ctx() is None
-
-            if invalid_started:
-                msg_type_str: str = 'Started'
-                bad_value: int = 10
-            elif invalid_return:
-                msg_type_str: str = 'Return'
-                bad_value: str = 'yo'
-            else:
-                # XXX but should never be used below then..
-                msg_type_str: str = ''
-                bad_value: str = ''
-
-            maybe_mte: MsgTypeError|None = None
-            should_raise: Exception|None = (
-                MsgTypeError if (
-                    invalid_return
-                    or
-                    invalid_started
-                ) else None
-            )
-            async with (
-                maybe_expect_raises(
-                    raises=should_raise,
-                    ensure_in_message=[
-                        f"invalid `{msg_type_str}` msg payload",
-                        f'{bad_value}',
-                        f'has type {type(bad_value)!r}',
-                        'not match type-spec',
-                        f'`{msg_type_str}.pld: PldMsg|NoneType`',
-                    ],
-                    # only for debug
-                    # post_mortem=True,
-                ),
-                p.open_context(
-                    child,
-                    return_value=return_value,
-                    started_value=started_value,
-                    validate_pld_spec=pld_check_started_value,
-                ) as (ctx, first),
-            ):
-                # now opened with 'child' sub
-                assert current_ipc_ctx() is ctx
-
-                assert type(first) is PldMsg
-                assert first.field == 'yo'
-
-                try:
-                    res: None|PldMsg = await ctx.result(hide_tb=False)
-                    assert res is None
-                except MsgTypeError as mte:
-                    maybe_mte = mte
-                    if not invalid_return:
-                        raise
-
-                    # expected this invalid `Return.pld` so audit
-                    # the error state + meta-data
-                    assert mte.expected_msg_type is Return
-                    assert mte.cid == ctx.cid
-                    mte_repr: str = repr(mte)
-                    for line in mte.message.splitlines():
-                        assert line in mte_repr
-
-                    assert mte.tb_str
-                    # await tractor.pause(shield=True)
-
-                    # verify expected remote mte deats
-                    assert ctx._local_error is None
-                    assert (
-                        mte is
-                        ctx._remote_error is
-                        ctx.maybe_error is
-                        ctx.outcome
-                    )
-
-            if should_raise is None:
-                assert maybe_mte is None
-
-            await p.cancel_actor()
-
-    trio.run(main)

tests/test_pubsub.py

@@ -5,7 +5,8 @@ import pytest
 import trio
 import tractor
 from tractor.experimental import msgpub
-from tractor._testing import tractor_test
+
+from conftest import tractor_test
 
 
 def test_type_checks():

tests/test_root_infect_asyncio.py

@@ -1,248 +0,0 @@
-'''
-Special attention cases for using "infect `asyncio`" mode from a root
-actor; i.e. not using a std `trio.run()` bootstrap.
-
-'''
-import asyncio
-from functools import partial
-
-import pytest
-import trio
-import tractor
-from tractor import (
-    to_asyncio,
-)
-from tests.test_infected_asyncio import (
-    aio_echo_server,
-)
-
-
-@pytest.mark.parametrize(
-    'raise_error_mid_stream',
-    [
-        False,
-        Exception,
-        KeyboardInterrupt,
-    ],
-    ids='raise_error={}'.format,
-)
-def test_infected_root_actor(
-    raise_error_mid_stream: bool|Exception,
-
-    # conftest wide
-    loglevel: str,
-    debug_mode: bool,
-):
-    '''
-    Verify you can run the `tractor` runtime with `Actor.is_infected_aio() == True`
-    in the root actor.
-
-    '''
-    async def _trio_main():
-        with trio.fail_after(2 if not debug_mode else 999):
-            first: str
-            chan: to_asyncio.LinkedTaskChannel
-            async with (
-                tractor.open_root_actor(
-                    debug_mode=debug_mode,
-                    loglevel=loglevel,
-                ),
-                to_asyncio.open_channel_from(
-                    aio_echo_server,
-                ) as (first, chan),
-            ):
-                assert first == 'start'
-
-                for i in range(1000):
-                    await chan.send(i)
-                    out = await chan.receive()
-                    assert out == i
-                    print(f'asyncio echoing {i}')
-
-                    if (
-                        raise_error_mid_stream
-                        and
-                        i == 500
-                    ):
-                        raise raise_error_mid_stream
-
-                    if out is None:
-                        try:
-                            out = await chan.receive()
-                        except trio.EndOfChannel:
-                            break
-                        else:
-                            raise RuntimeError(
-                                'aio channel never stopped?'
-                            )
-
-    if raise_error_mid_stream:
-        with pytest.raises(raise_error_mid_stream):
-            tractor.to_asyncio.run_as_asyncio_guest(
-                trio_main=_trio_main,
-            )
-    else:
-        tractor.to_asyncio.run_as_asyncio_guest(
-            trio_main=_trio_main,
-        )
-
-
-
-async def sync_and_err(
-    # just signature placeholders for compat with
-    # ``to_asyncio.open_channel_from()``
-    to_trio: trio.MemorySendChannel,
-    from_trio: asyncio.Queue,
-    ev: asyncio.Event,
-
-):
-    if to_trio:
-        to_trio.send_nowait('start')
-
-    await ev.wait()
-    raise RuntimeError('asyncio-side')
-
-
-@pytest.mark.parametrize(
-    'aio_err_trigger',
-    [
-        'before_start_point',
-        'after_trio_task_starts',
-        'after_start_point',
-    ],
-    ids='aio_err_triggered={}'.format
-)
-def test_trio_prestarted_task_bubbles(
-    aio_err_trigger: str,
-
-    # conftest wide
-    loglevel: str,
-    debug_mode: bool,
-):
-    async def pre_started_err(
-        raise_err: bool = False,
-        pre_sleep: float|None = None,
-        aio_trigger: asyncio.Event|None = None,
-        task_status=trio.TASK_STATUS_IGNORED,
-    ):
-        '''
-        Maybe pre-started error then sleep.
-
-        '''
-        if pre_sleep is not None:
-            print(f'Sleeping from trio for {pre_sleep!r}s !')
-            await trio.sleep(pre_sleep)
-
-        # signal aio-task to raise JUST AFTER this task
-        # starts but has not yet `.started()`
-        if aio_trigger:
-            print('Signalling aio-task to raise from `trio`!!')
-            aio_trigger.set()
-
-        if raise_err:
-            print('Raising from trio!')
-            raise TypeError('trio-side')
-
-        task_status.started()
-        await trio.sleep_forever()
-
-    async def _trio_main():
-        # with trio.fail_after(2):
-        with trio.fail_after(999):
-            first: str
-            chan: to_asyncio.LinkedTaskChannel
-            aio_ev = asyncio.Event()
-
-            async with (
-                tractor.open_root_actor(
-                    debug_mode=False,
-                    loglevel=loglevel,
-                ),
-            ):
-                # TODO, tests for this with 3.13 egs?
-                # from tractor.devx import open_crash_handler
-                # with open_crash_handler():
-                async with (
-                    # where we'll start a sub-task that errors BEFORE
-                    # calling `.started()` such that the error should
-                    # bubble before the guest run terminates!
-                    trio.open_nursery() as tn,
-
-                    # THEN start an infect task which should error just
-                    # after the trio-side's task does.
-                    to_asyncio.open_channel_from(
-                        partial(
-                            sync_and_err,
-                            ev=aio_ev,
-                        )
-                    ) as (first, chan),
-                ):
-
-                    for i in range(5):
-                        pre_sleep: float|None = None
-                        last_iter: bool = (i == 4)
-
-                        # TODO, missing cases?
-                        # -[ ] error as well on
-                        #    'after_start_point' case as well for
-                        #    another case?
-                        raise_err: bool = False
-
-                        if last_iter:
-                            raise_err: bool = True
-
-                            # trigger aio task to error on next loop
-                            # tick/checkpoint
-                            if aio_err_trigger == 'before_start_point':
-                                aio_ev.set()
-
-                            pre_sleep: float = 0
-
-                        await tn.start(
-                            pre_started_err,
-                            raise_err,
-                            pre_sleep,
-                            (aio_ev if (
-                                    aio_err_trigger == 'after_trio_task_starts'
-                                    and
-                                    last_iter
-                                ) else None
-                            ),
-                        )
-
-                        if (
-                            aio_err_trigger == 'after_start_point'
-                            and
-                            last_iter
-                        ):
-                            aio_ev.set()
-
-    with pytest.raises(
-        expected_exception=ExceptionGroup,
-    ) as excinfo:
-        tractor.to_asyncio.run_as_asyncio_guest(
-            trio_main=_trio_main,
-        )
-
-    eg = excinfo.value
-    rte_eg, rest_eg = eg.split(RuntimeError)
-
-    # ensure the trio-task's error bubbled despite the aio-side
-    # having (maybe) errored first.
-    if aio_err_trigger in (
-        'after_trio_task_starts',
-        'after_start_point',
-    ):
-        assert len(errs := rest_eg.exceptions) == 1
-        typerr = errs[0]
-        assert (
-            type(typerr) is TypeError
-            and
-            'trio-side' in typerr.args
-        )
-
-    # when aio errors BEFORE (last) trio task is scheduled, we should
-    # never see anythinb but the aio-side.
-    else:
-        assert len(rtes := rte_eg.exceptions) == 1
-        assert 'asyncio-side' in rtes[0].args[0]

tests/test_rpc.py

@@ -1,8 +1,6 @@
-'''
-RPC (or maybe better labelled as "RTS: remote task scheduling"?)
-related API and error checks.
-
-'''
+"""
+RPC related
+"""
 import itertools
 
 import pytest
@@ -36,7 +34,7 @@ async def sleep_back_actor(
                 if not exposed_mods:
                     expect = tractor.ModuleNotExposed
 
-                assert err.boxed_type is expect
+                assert err.type is expect
                 raise
     else:
         await trio.sleep(float('inf'))
@@ -54,13 +52,8 @@ async def short_sleep():
         (['tmp_mod'], 'import doggy', ModuleNotFoundError),
         (['tmp_mod'], '4doggy', SyntaxError),
     ],
-    ids=[
-        'no_mods',
-        'this_mod',
-        'this_mod_bad_func',
-        'fail_to_import',
-        'fail_on_syntax',
-    ],
+    ids=['no_mods', 'this_mod', 'this_mod_bad_func', 'fail_to_import',
+         'fail_on_syntax'],
 )
 def test_rpc_errors(
     reg_addr,
@@ -134,9 +127,7 @@ def test_rpc_errors(
         run()
     else:
         # underlying errors aren't propagated upwards (yet)
-        with pytest.raises(
-            expected_exception=(remote_err, ExceptionGroup),
-        ) as err:
+        with pytest.raises(remote_err) as err:
             run()
 
         # get raw instance from pytest wrapper
@@ -150,4 +141,4 @@ def test_rpc_errors(
             ))
 
         if getattr(value, 'type', None):
-            assert value.boxed_type is inside_err
+            assert value.type is inside_err

tests/test_runtime.py

@@ -8,7 +8,7 @@ import pytest
 import trio
 import tractor
 
-from tractor._testing import tractor_test
+from conftest import tractor_test
 
 
 _file_path: str = ''
@@ -64,8 +64,7 @@ async def test_lifetime_stack_wipes_tmpfile(
 
     except (
         tractor.RemoteActorError,
-        # tractor.BaseExceptionGroup,
-        BaseExceptionGroup,
+        tractor.BaseExceptionGroup,
     ):
         pass
 

tests/test_shm.py

@@ -0,0 +1,167 @@
+"""
+Shared mem primitives and APIs.
+
+"""
+import uuid
+
+# import numpy
+import pytest
+import trio
+import tractor
+from tractor._shm import (
+    open_shm_list,
+    attach_shm_list,
+)
+
+
+@tractor.context
+async def child_attach_shml_alot(
+    ctx: tractor.Context,
+    shm_key: str,
+) -> None:
+
+    await ctx.started(shm_key)
+
+    # now try to attach a boatload of times in a loop..
+    for _ in range(1000):
+        shml = attach_shm_list(
+            key=shm_key,
+            readonly=False,
+        )
+        assert shml.shm.name == shm_key
+        await trio.sleep(0.001)
+
+
+def test_child_attaches_alot():
+    async def main():
+        async with tractor.open_nursery() as an:
+
+            # allocate writeable list in parent
+            key = f'shml_{uuid.uuid4()}'
+            shml = open_shm_list(
+                key=key,
+            )
+
+            portal = await an.start_actor(
+                'shm_attacher',
+                enable_modules=[__name__],
+            )
+
+            async with (
+                portal.open_context(
+                    child_attach_shml_alot,
+                    shm_key=shml.key,
+                ) as (ctx, start_val),
+            ):
+                assert start_val == key
+                await ctx.result()
+
+            await portal.cancel_actor()
+
+    trio.run(main)
+
+
+@tractor.context
+async def child_read_shm_list(
+    ctx: tractor.Context,
+    shm_key: str,
+    use_str: bool,
+    frame_size: int,
+) -> None:
+
+    # attach in child
+    shml = attach_shm_list(
+        key=shm_key,
+        # dtype=str if use_str else float,
+    )
+    await ctx.started(shml.key)
+
+    async with ctx.open_stream() as stream:
+        async for i in stream:
+            print(f'(child): reading shm list index: {i}')
+
+            if use_str:
+                expect = str(float(i))
+            else:
+                expect = float(i)
+
+            if frame_size == 1:
+                val = shml[i]
+                assert expect == val
+                print(f'(child): reading value: {val}')
+            else:
+                frame = shml[i - frame_size:i]
+                print(f'(child): reading frame: {frame}')
+
+
+@pytest.mark.parametrize(
+    'use_str',
+    [False, True],
+    ids=lambda i: f'use_str_values={i}',
+)
+@pytest.mark.parametrize(
+    'frame_size',
+    [1, 2**6, 2**10],
+    ids=lambda i: f'frame_size={i}',
+)
+def test_parent_writer_child_reader(
+    use_str: bool,
+    frame_size: int,
+):
+
+    async def main():
+        async with tractor.open_nursery(
+            # debug_mode=True,
+        ) as an:
+
+            portal = await an.start_actor(
+                'shm_reader',
+                enable_modules=[__name__],
+                debug_mode=True,
+            )
+
+            # allocate writeable list in parent
+            key = 'shm_list'
+            seq_size = int(2 * 2 ** 10)
+            shml = open_shm_list(
+                key=key,
+                size=seq_size,
+                dtype=str if use_str else float,
+                readonly=False,
+            )
+
+            async with (
+                portal.open_context(
+                    child_read_shm_list,
+                    shm_key=key,
+                    use_str=use_str,
+                    frame_size=frame_size,
+                ) as (ctx, sent),
+
+                ctx.open_stream() as stream,
+            ):
+
+                assert sent == key
+
+                for i in range(seq_size):
+
+                    val = float(i)
+                    if use_str:
+                        val = str(val)
+
+                    # print(f'(parent): writing {val}')
+                    shml[i] = val
+
+                    # only on frame fills do we
+                    # signal to the child that a frame's
+                    # worth is ready.
+                    if (i % frame_size) == 0:
+                        print(f'(parent): signalling frame full on {val}')
+                        await stream.send(i)
+                else:
+                    print(f'(parent): signalling final frame on {val}')
+                    await stream.send(i)
+
+            await portal.cancel_actor()
+
+    trio.run(main)

tests/test_spawning.py

@@ -2,15 +2,13 @@
 Spawning basics
 
 """
-from typing import (
-    Any,
-)
+from typing import Optional
 
 import pytest
 import trio
 import tractor
 
-from tractor._testing import tractor_test
+from conftest import tractor_test
 
 data_to_pass_down = {'doggy': 10, 'kitty': 4}
 
@@ -27,11 +25,13 @@ async def spawn(
     async with tractor.open_root_actor(
         arbiter_addr=reg_addr,
     ):
+
         actor = tractor.current_actor()
         assert actor.is_arbiter == is_arbiter
         data = data_to_pass_down
 
         if actor.is_arbiter:
+
             async with tractor.open_nursery() as nursery:
 
                 # forks here
@@ -95,9 +95,7 @@ async def test_movie_theatre_convo(start_method):
         await portal.cancel_actor()
 
 
-async def cellar_door(
-    return_value: str|None,
-):
+async def cellar_door(return_value: Optional[str]):
     return return_value
 
 
@@ -107,18 +105,16 @@ async def cellar_door(
 )
 @tractor_test
 async def test_most_beautiful_word(
-    start_method: str,
-    return_value: Any,
-    debug_mode: bool,
+    start_method,
+    return_value
 ):
     '''
     The main ``tractor`` routine.
 
     '''
     with trio.fail_after(1):
-        async with tractor.open_nursery(
-            debug_mode=debug_mode,
-        ) as n:
+        async with tractor.open_nursery() as n:
+
             portal = await n.run_in_actor(
                 cellar_door,
                 return_value=return_value,

tests/test_task_broadcasting.py

@@ -2,9 +2,7 @@
 Broadcast channels for fan-out to local tasks.
 
 """
-from contextlib import (
-    asynccontextmanager as acm,
-)
+from contextlib import asynccontextmanager
 from functools import partial
 from itertools import cycle
 import time
@@ -17,7 +15,6 @@ import tractor
 from tractor.trionics import (
     broadcast_receiver,
     Lagged,
-    collapse_eg,
 )
 
 
@@ -65,7 +62,7 @@ async def ensure_sequence(
                 break
 
 
-@acm
+@asynccontextmanager
 async def open_sequence_streamer(
 
     sequence: list[int],
@@ -77,9 +74,9 @@ async def open_sequence_streamer(
     async with tractor.open_nursery(
         arbiter_addr=reg_addr,
         start_method=start_method,
-    ) as an:
+    ) as tn:
 
-        portal = await an.start_actor(
+        portal = await tn.start_actor(
             'sequence_echoer',
             enable_modules=[__name__],
         )
@@ -158,12 +155,9 @@ def test_consumer_and_parent_maybe_lag(
         ) as stream:
 
             try:
-                async with (
-                    collapse_eg(),
-                    trio.open_nursery() as tn,
-                ):
+                async with trio.open_nursery() as n:
 
-                    tn.start_soon(
+                    n.start_soon(
                         ensure_sequence,
                         stream,
                         sequence.copy(),
@@ -236,8 +230,8 @@ def test_faster_task_to_recv_is_cancelled_by_slower(
 
         ) as stream:
 
-            async with trio.open_nursery() as tn:
-                tn.start_soon(
+            async with trio.open_nursery() as n:
+                n.start_soon(
                     ensure_sequence,
                     stream,
                     sequence.copy(),
@@ -259,7 +253,7 @@ def test_faster_task_to_recv_is_cancelled_by_slower(
                         continue
 
                 print('cancelling faster subtask')
-                tn.cancel_scope.cancel()
+                n.cancel_scope.cancel()
 
             try:
                 value = await stream.receive()
@@ -277,7 +271,7 @@ def test_faster_task_to_recv_is_cancelled_by_slower(
                         # the faster subtask was cancelled
                         break
 
-                # await tractor.pause()
+                # await tractor.breakpoint()
                 # await stream.receive()
                 print(f'final value: {value}')
 
@@ -377,13 +371,13 @@ def test_ensure_slow_consumers_lag_out(
                                     f'on {lags}:{value}')
                                 return
 
-            async with trio.open_nursery() as tn:
+            async with trio.open_nursery() as nursery:
 
                 for i in range(1, num_laggers):
 
                     task_name = f'sub_{i}'
                     laggers[task_name] = 0
-                    tn.start_soon(
+                    nursery.start_soon(
                         partial(
                             sub_and_print,
                             delay=i*0.001,
@@ -503,7 +497,6 @@ def test_no_raise_on_lag():
                 # internals when the no raise flag is set.
                 loglevel='warning',
             ),
-            collapse_eg(),
             trio.open_nursery() as n,
         ):
             n.start_soon(slow)

tests/test_trioisms.py

@@ -3,13 +3,9 @@ Reminders for oddities in `trio` that we need to stay aware of and/or
 want to see changed.
 
 '''
-from contextlib import (
-    asynccontextmanager as acm,
-)
-
 import pytest
 import trio
-from trio import TaskStatus
+from trio_typing import TaskStatus
 
 
 @pytest.mark.parametrize(
@@ -64,9 +60,7 @@ def test_stashed_child_nursery(use_start_soon):
     async def main():
 
         async with (
-            trio.open_nursery(
-                strict_exception_groups=False,
-            ) as pn,
+            trio.open_nursery() as pn,
         ):
             cn = await pn.start(mk_child_nursery)
             assert cn
@@ -86,118 +80,3 @@ def test_stashed_child_nursery(use_start_soon):
 
     with pytest.raises(NameError):
         trio.run(main)
-
-
-@pytest.mark.parametrize(
-    ('unmask_from_canc', 'canc_from_finally'),
-    [
-        (True, False),
-        (True, True),
-        pytest.param(False, True,
-                     marks=pytest.mark.xfail(reason="never raises!")
-        ),
-    ],
-    # TODO, ask ronny how to impl this .. XD
-    # ids='unmask_from_canc={0}, canc_from_finally={1}',#.format,
-)
-def test_acm_embedded_nursery_propagates_enter_err(
-    canc_from_finally: bool,
-    unmask_from_canc: bool,
-    debug_mode: bool,
-):
-    '''
-    Demo how a masking `trio.Cancelled` could be handled by unmasking from the
-    `.__context__` field when a user (by accident) re-raises from a `finally:`.
-
-    '''
-    import tractor
-
-    @acm
-    async def maybe_raise_from_masking_exc(
-        tn: trio.Nursery,
-        unmask_from: BaseException|None = trio.Cancelled
-
-        # TODO, maybe offer a collection?
-        # unmask_from: set[BaseException] = {
-        #     trio.Cancelled,
-        # },
-    ):
-        if not unmask_from:
-            yield
-            return
-
-        try:
-            yield
-        except* unmask_from as be_eg:
-
-            # TODO, if we offer `unmask_from: set`
-            # for masker_exc_type in unmask_from:
-
-            matches, rest = be_eg.split(unmask_from)
-            if not matches:
-                raise
-
-            for exc_match in be_eg.exceptions:
-                if (
-                    (exc_ctx := exc_match.__context__)
-                    and
-                    type(exc_ctx) not in {
-                        # trio.Cancelled,  # always by default?
-                        unmask_from,
-                    }
-                ):
-                    exc_ctx.add_note(
-                        f'\n'
-                        f'WARNING: the above error was masked by a {unmask_from!r} !?!\n'
-                        f'Are you always cancelling? Say from a `finally:` ?\n\n'
-
-                        f'{tn!r}'
-                    )
-                    raise exc_ctx from exc_match
-
-
-    @acm
-    async def wraps_tn_that_always_cancels():
-        async with (
-            trio.open_nursery() as tn,
-            maybe_raise_from_masking_exc(
-                tn=tn,
-                unmask_from=(
-                    trio.Cancelled
-                    if unmask_from_canc
-                    else None
-                ),
-            )
-        ):
-            try:
-                yield tn
-            finally:
-                if canc_from_finally:
-                    tn.cancel_scope.cancel()
-                    await trio.lowlevel.checkpoint()
-
-    async def _main():
-        with tractor.devx.maybe_open_crash_handler(
-            pdb=debug_mode,
-        ) as bxerr:
-            assert not bxerr.value
-
-            async with (
-                wraps_tn_that_always_cancels() as tn,
-            ):
-                assert not tn.cancel_scope.cancel_called
-                assert 0
-
-        assert (
-            (err := bxerr.value)
-            and
-            type(err) is AssertionError
-        )
-
-    with pytest.raises(ExceptionGroup) as excinfo:
-        trio.run(_main)
-
-    eg: ExceptionGroup = excinfo.value
-    assert_eg, rest_eg = eg.split(AssertionError)
-
-    assert len(assert_eg.exceptions) == 1

tractor/__init__.py

@@ -18,6 +18,7 @@
 tractor: structured concurrent ``trio``-"actors".
 
 """
+from exceptiongroup import BaseExceptionGroup as BaseExceptionGroup
 
 from ._clustering import (
     open_actor_cluster as open_actor_cluster,
@@ -31,7 +32,7 @@ from ._streaming import (
     stream as stream,
 )
 from ._discovery import (
-    get_registry as get_registry,
+    get_arbiter as get_arbiter,
     find_actor as find_actor,
     wait_for_actor as wait_for_actor,
     query_actor as query_actor,
@@ -43,15 +44,11 @@ from ._supervise import (
 from ._state import (
     current_actor as current_actor,
     is_root_process as is_root_process,
-    current_ipc_ctx as current_ipc_ctx,
-    debug_mode as debug_mode
 )
 from ._exceptions import (
-    ContextCancelled as ContextCancelled,
-    ModuleNotExposed as ModuleNotExposed,
-    MsgTypeError as MsgTypeError,
     RemoteActorError as RemoteActorError,
-    TransportClosed as TransportClosed,
+    ModuleNotExposed as ModuleNotExposed,
+    ContextCancelled as ContextCancelled,
 )
 from .devx import (
     breakpoint as breakpoint,
@@ -67,4 +64,3 @@ from ._root import (
 from ._ipc import Channel as Channel
 from ._portal import Portal as Portal
 from ._runtime import Actor as Actor
-# from . import hilevel as hilevel

tractor/_child.py

@@ -36,7 +36,6 @@ def parse_ipaddr(arg):
 
 
 if __name__ == "__main__":
-    __tracebackhide__: bool = True
 
     parser = argparse.ArgumentParser()
     parser.add_argument("--uid", type=parse_uid)

tractor/_clustering.py

@@ -19,13 +19,10 @@ Actor cluster helpers.
 
 '''
 from __future__ import annotations
-from contextlib import (
-    asynccontextmanager as acm,
-)
+
+from contextlib import asynccontextmanager as acm
 from multiprocessing import cpu_count
-from typing import (
-    AsyncGenerator,
-)
+from typing import AsyncGenerator, Optional
 
 import trio
 import tractor

tractor/_discovery.py

@@ -26,8 +26,8 @@ from typing import (
     TYPE_CHECKING,
 )
 from contextlib import asynccontextmanager as acm
+import warnings
 
-from tractor.log import get_logger
 from .trionics import gather_contexts
 from ._ipc import _connect_chan, Channel
 from ._portal import (
@@ -35,18 +35,13 @@ from ._portal import (
     open_portal,
     LocalPortal,
 )
-from ._state import (
-    current_actor,
-    _runtime_vars,
-)
+from ._state import current_actor, _runtime_vars
+
 
 if TYPE_CHECKING:
     from ._runtime import Actor
 
 
-log = get_logger(__name__)
-
-
 @acm
 async def get_registry(
     host: str,
@@ -58,12 +53,14 @@ async def get_registry(
 ]:
     '''
     Return a portal instance connected to a local or remote
-    registry-service actor; if a connection already exists re-use it
-    (presumably to call a `.register_actor()` registry runtime RPC
-    ep).
+    arbiter.
 
     '''
-    actor: Actor = current_actor()
+    actor = current_actor()
+
+    if not actor:
+        raise RuntimeError("No actor instance has been defined yet?")
+
     if actor.is_registrar:
         # we're already the arbiter
         # (likely a re-entrant call from the arbiter actor)
@@ -72,8 +69,6 @@ async def get_registry(
             Channel((host, port))
         )
     else:
-        # TODO: try to look pre-existing connection from
-        # `Actor._peers` and use it instead?
         async with (
             _connect_chan(host, port) as chan,
             open_portal(chan) as regstr_ptl,
@@ -82,6 +77,19 @@ async def get_registry(
 
 
 
+# TODO: deprecate and this remove _arbiter form!
+@acm
+async def get_arbiter(*args, **kwargs):
+    warnings.warn(
+        '`tractor.get_arbiter()` is now deprecated!\n'
+        'Use `.get_registry()` instead!',
+        DeprecationWarning,
+        stacklevel=2,
+    )
+    async with get_registry(*args, **kwargs) as to_yield:
+        yield to_yield
+
+
 @acm
 async def get_root(
     **kwargs,
@@ -99,53 +107,22 @@ async def get_root(
         yield portal
 
 
-def get_peer_by_name(
-    name: str,
-    # uuid: str|None = None,
-
-) -> list[Channel]|None:  # at least 1
-    '''
-    Scan for an existing connection (set) to a named actor
-    and return any channels from `Actor._peers`.
-
-    This is an optimization method over querying the registrar for
-    the same info.
-
-    '''
-    actor: Actor = current_actor()
-    to_scan: dict[tuple, list[Channel]] = actor._peers.copy()
-    pchan: Channel|None = actor._parent_chan
-    if pchan:
-        to_scan[pchan.uid].append(pchan)
-
-    for aid, chans in to_scan.items():
-        _, peer_name = aid
-        if name == peer_name:
-            if not chans:
-                log.warning(
-                    'No IPC chans for matching peer {peer_name}\n'
-                )
-                continue
-            return chans
-
-    return None
-
-
 @acm
 async def query_actor(
     name: str,
-    regaddr: tuple[str, int]|None = None,
+    arbiter_sockaddr: tuple[str, int] | None = None,
+    regaddr: tuple[str, int] | None = None,
 
 ) -> AsyncGenerator[
-    tuple[str, int]|None,
+    tuple[str, int] | None,
     None,
 ]:
     '''
-    Lookup a transport address (by actor name) via querying a registrar
-    listening @ `regaddr`.
+    Make a transport address lookup for an actor name to a specific
+    registrar.
 
-    Returns the transport protocol (socket) address or `None` if no
-    entry under that name exists.
+    Returns the (socket) address or ``None`` if no entry under that
+    name exists for the given registrar listening @ `regaddr`.
 
     '''
     actor: Actor = current_actor()
@@ -157,10 +134,14 @@ async def query_actor(
             'The current actor IS the registry!?'
         )
 
-    maybe_peers: list[Channel]|None = get_peer_by_name(name)
-    if maybe_peers:
-        yield maybe_peers[0].raddr
-        return
+    if arbiter_sockaddr is not None:
+        warnings.warn(
+            '`tractor.query_actor(regaddr=<blah>)` is deprecated.\n'
+            'Use `registry_addrs: list[tuple]` instead!',
+            DeprecationWarning,
+            stacklevel=2,
+        )
+        regaddr: list[tuple[str, int]] = arbiter_sockaddr
 
     reg_portal: Portal
     regaddr: tuple[str, int] = regaddr or actor.reg_addrs[0]
@@ -175,29 +156,11 @@ async def query_actor(
         yield sockaddr
 
 
-@acm
-async def maybe_open_portal(
-    addr: tuple[str, int],
-    name: str,
-):
-    async with query_actor(
-        name=name,
-        regaddr=addr,
-    ) as sockaddr:
-        pass
-
-    if sockaddr:
-        async with _connect_chan(*sockaddr) as chan:
-            async with open_portal(chan) as portal:
-                yield portal
-    else:
-        yield None
-
-
 @acm
 async def find_actor(
     name: str,
-    registry_addrs: list[tuple[str, int]]|None = None,
+    arbiter_sockaddr: tuple[str, int] | None = None,
+    registry_addrs: list[tuple[str, int]] | None = None,
 
     only_first: bool = True,
     raise_on_none: bool = False,
@@ -213,34 +176,44 @@ async def find_actor(
     known to the arbiter.
 
     '''
-    # optimization path, use any pre-existing peer channel
-    maybe_peers: list[Channel]|None = get_peer_by_name(name)
-    if maybe_peers and only_first:
-        async with open_portal(maybe_peers[0]) as peer_portal:
-            yield peer_portal
-            return
+    if arbiter_sockaddr is not None:
+        warnings.warn(
+            '`tractor.find_actor(arbiter_sockaddr=<blah>)` is deprecated.\n'
+            'Use `registry_addrs: list[tuple]` instead!',
+            DeprecationWarning,
+            stacklevel=2,
+        )
+        registry_addrs: list[tuple[str, int]] = [arbiter_sockaddr]
+
+    @acm
+    async def maybe_open_portal_from_reg_addr(
+        addr: tuple[str, int],
+    ):
+        async with query_actor(
+            name=name,
+            regaddr=addr,
+        ) as sockaddr:
+            if sockaddr:
+                async with _connect_chan(*sockaddr) as chan:
+                    async with open_portal(chan) as portal:
+                        yield portal
+            else:
+                yield None
 
     if not registry_addrs:
         # XXX NOTE: make sure to dynamically read the value on
         # every call since something may change it globally (eg.
         # like in our discovery test suite)!
         from . import _root
-        registry_addrs = (
-            _runtime_vars['_registry_addrs']
-            or
-            _root._default_lo_addrs
-        )
+        registry_addrs = _root._default_lo_addrs
 
     maybe_portals: list[
         AsyncContextManager[tuple[str, int]]
     ] = list(
-        maybe_open_portal(
-            addr=addr,
-            name=name,
-        )
+        maybe_open_portal_from_reg_addr(addr)
         for addr in registry_addrs
     )
-    portals: list[Portal]
+
     async with gather_contexts(
         mngrs=maybe_portals,
     ) as portals:
@@ -274,31 +247,31 @@ async def find_actor(
 @acm
 async def wait_for_actor(
     name: str,
+    arbiter_sockaddr: tuple[str, int] | None = None,
     registry_addr: tuple[str, int] | None = None,
 
 ) -> AsyncGenerator[Portal, None]:
     '''
-    Wait on at least one peer actor to register `name` with the
-    registrar, yield a `Portal to the first registree.
+    Wait on an actor to register with the arbiter.
+
+    A portal to the first registered actor is returned.
 
     '''
     actor: Actor = current_actor()
 
-    # optimization path, use any pre-existing peer channel
-    maybe_peers: list[Channel]|None = get_peer_by_name(name)
-    if maybe_peers:
-        async with open_portal(maybe_peers[0]) as peer_portal:
-            yield peer_portal
-            return
+    if arbiter_sockaddr is not None:
+        warnings.warn(
+            '`tractor.wait_for_actor(arbiter_sockaddr=<foo>)` is deprecated.\n'
+            'Use `registry_addr: tuple` instead!',
+            DeprecationWarning,
+            stacklevel=2,
+        )
+        registry_addr: tuple[str, int] = arbiter_sockaddr
 
-    regaddr: tuple[str, int] = (
-        registry_addr
-        or
-        actor.reg_addrs[0]
-    )
     # TODO: use `.trionics.gather_contexts()` like
     # above in `find_actor()` as well?
     reg_portal: Portal
+    regaddr: tuple[str, int] = registry_addr or actor.reg_addrs[0]
     async with get_registry(*regaddr) as reg_portal:
         sockaddrs = await reg_portal.run_from_ns(
             'self',

tractor/_entry.py

@@ -20,9 +20,6 @@ Sub-process entry points.
 """
 from __future__ import annotations
 from functools import partial
-import multiprocessing as mp
-import os
-import textwrap
 from typing import (
     Any,
     TYPE_CHECKING,
@@ -35,7 +32,6 @@ from .log import (
     get_logger,
 )
 from . import _state
-from .devx import _debug
 from .to_asyncio import run_as_asyncio_guest
 from ._runtime import (
     async_main,
@@ -60,27 +56,25 @@ def _mp_main(
 
 ) -> None:
     '''
-    The routine called *after fork* which invokes a fresh `trio.run()`
+    The routine called *after fork* which invokes a fresh ``trio.run``
 
     '''
     actor._forkserver_info = forkserver_info
     from ._spawn import try_set_start_method
-    spawn_ctx: mp.context.BaseContext = try_set_start_method(start_method)
-    assert spawn_ctx
+    spawn_ctx = try_set_start_method(start_method)
 
     if actor.loglevel is not None:
         log.info(
-            f'Setting loglevel for {actor.uid} to {actor.loglevel}'
-        )
+            f"Setting loglevel for {actor.uid} to {actor.loglevel}")
         get_console_log(actor.loglevel)
 
-    # TODO: use scops headers like for `trio` below!
-    # (well after we libify it maybe..)
+    assert spawn_ctx
     log.info(
-        f'Started new {spawn_ctx.current_process()} for {actor.uid}'
-    #     f"parent_addr is {parent_addr}"
-    )
-    _state._current_actor: Actor = actor
+        f"Started new {spawn_ctx.current_process()} for {actor.uid}")
+
+    _state._current_actor = actor
+
+    log.debug(f"parent_addr is {parent_addr}")
     trio_main = partial(
         async_main,
         actor=actor,
@@ -97,113 +91,11 @@ def _mp_main(
         pass  # handle it the same way trio does?
 
     finally:
-        log.info(
-            f'`mp`-subactor {actor.uid} exited'
-        )
-
-
-# TODO: move this func to some kinda `.devx._conc_lang.py` eventually
-# as we work out our multi-domain state-flow-syntax!
-def nest_from_op(
-    input_op: str,
-    #
-    # ?TODO? an idea for a syntax to the state of concurrent systems
-    # as a "3-domain" (execution, scope, storage) model and using
-    # a minimal ascii/utf-8 operator-set.
-    #
-    # try not to take any of this seriously yet XD
-    #
-    # > is a "play operator" indicating (CPU bound)
-    #   exec/work/ops required at the "lowest level computing"
-    #
-    # execution primititves (tasks, threads, actors..) denote their
-    # lifetime with '(' and ')' since parentheses normally are used
-    # in many langs to denote function calls.
-    #
-    # starting = (
-    # >(  opening/starting; beginning of the thread-of-exec (toe?)
-    # (>  opened/started,  (finished spawning toe)
-    # |_<Task: blah blah..>  repr of toe, in py these look like <objs>
-    #
-    # >) closing/exiting/stopping,
-    # )> closed/exited/stopped,
-    # |_<Task: blah blah..>
-    #   [OR <), )< ?? ]
-    #
-    # ending = )
-    # >c) cancelling to close/exit
-    # c)> cancelled (caused close), OR?
-    #  |_<Actor: ..>
-    #   OR maybe "<c)" which better indicates the cancel being
-    #   "delivered/returned" / returned" to LHS?
-    #
-    # >x)  erroring to eventuall exit
-    # x)>  errored and terminated
-    #  |_<Actor: ...>
-    #
-    # scopes: supers/nurseries, IPC-ctxs, sessions, perms, etc.
-    # >{  opening
-    # {>  opened
-    # }>  closed
-    # >}  closing
-    #
-    # storage: like queues, shm-buffers, files, etc..
-    # >[  opening
-    # [>  opened
-    #  |_<FileObj: ..>
-    #
-    # >]  closing
-    # ]>  closed
-
-    # IPC ops: channels, transports, msging
-    # =>  req msg
-    # <=  resp msg
-    # <=> 2-way streaming (of msgs)
-    # <-  recv 1 msg
-    # ->  send 1 msg
-    #
-    # TODO: still not sure on R/L-HS approach..?
-    # =>(  send-req to exec start (task, actor, thread..)
-    # (<=  recv-req to ^
-    #
-    # (<=  recv-req ^
-    # <=(  recv-resp opened remote exec primitive
-    # <=)  recv-resp closed
-    #
-    # )<=c req to stop due to cancel
-    # c=>) req to stop due to cancel
-    #
-    # =>{  recv-req to open
-    # <={  send-status that it closed
-
-    tree_str: str,
-
-    # NOTE: so move back-from-the-left of the `input_op` by
-    # this amount.
-    back_from_op: int = 0,
-) -> str:
-    '''
-    Depth-increment the input (presumably hierarchy/supervision)
-    input "tree string" below the provided `input_op` execution
-    operator, so injecting a `"\n|_{input_op}\n"`and indenting the
-    `tree_str` to nest content aligned with the ops last char.
-
-    '''
-    return (
-        f'{input_op}\n'
-        +
-        textwrap.indent(
-            tree_str,
-            prefix=(
-                len(input_op)
-                -
-                (back_from_op + 1)
-            ) * ' ',
-        )
-    )
+        log.info(f"Actor {actor.uid} terminated")
 
 
 def _trio_main(
+
     actor: Actor,
     *,
     parent_addr: tuple[str, int] | None = None,
@@ -214,8 +106,6 @@ def _trio_main(
     Entry point for a `trio_run_in_process` subactor.
 
     '''
-    _debug.hide_runtime_frames()
-
     _state._current_actor = actor
     trio_main = partial(
         async_main,
@@ -225,6 +115,7 @@ def _trio_main(
 
     if actor.loglevel is not None:
         get_console_log(actor.loglevel)
+        import os
         actor_info: str = (
             f'|_{actor}\n'
             f'  uid: {actor.uid}\n'
@@ -233,54 +124,27 @@ def _trio_main(
             f'  loglevel: {actor.loglevel}\n'
         )
         log.info(
-            'Starting new `trio` subactor:\n'
+            'Started new trio process:\n'
             +
-            nest_from_op(
-                input_op='>(',  # see syntax ideas above
-                tree_str=actor_info,
-                back_from_op=2,  # since "complete"
-            )
+            actor_info
         )
-    logmeth = log.info
-    exit_status: str = (
-        'Subactor exited\n'
-        +
-        nest_from_op(
-            input_op=')>',  # like a "closed-to-play"-icon from super perspective
-            tree_str=actor_info,
-            back_from_op=1,
-        )
-    )
+
     try:
         if infect_asyncio:
             actor._infected_aio = True
             run_as_asyncio_guest(trio_main)
         else:
             trio.run(trio_main)
-
     except KeyboardInterrupt:
-        logmeth = log.cancel
-        exit_status: str = (
-            'Actor received KBI (aka an OS-cancel)\n'
+        log.cancel(
+            'Actor received KBI\n'
             +
-            nest_from_op(
-                input_op='c)>',  # closed due to cancel (see above)
-                tree_str=actor_info,
-            )
+            actor_info
         )
-    except BaseException as err:
-        logmeth = log.error
-        exit_status: str = (
-            'Main actor task exited due to crash?\n'
-            +
-            nest_from_op(
-                input_op='x)>',  # closed by error
-                tree_str=actor_info,
-            )
-        )
-        # NOTE since we raise a tb will already be shown on the
-        # console, thus we do NOT use `.exception()` above.
-        raise err
 
     finally:
-        logmeth(exit_status)
+        log.info(
+            'Actor terminated\n'
+            +
+            actor_info
+        )

tractor/_ipc.py

@@ -19,64 +19,38 @@ Inter-process comms abstractions
 
 """
 from __future__ import annotations
+import struct
+import platform
+from pprint import pformat
 from collections.abc import (
     AsyncGenerator,
     AsyncIterator,
 )
-from contextlib import (
-    asynccontextmanager as acm,
-    contextmanager as cm,
-)
-import platform
-from pprint import pformat
-import struct
 import typing
 from typing import (
     Any,
-    Callable,
     runtime_checkable,
     Protocol,
     Type,
     TypeVar,
 )
 
-import msgspec
 from tricycle import BufferedReceiveStream
+import msgspec
 import trio
+from async_generator import asynccontextmanager
 
-from tractor.log import get_logger
-from tractor._exceptions import (
-    MsgTypeError,
-    pack_from_raise,
-    TransportClosed,
-    _mk_send_mte,
-    _mk_recv_mte,
-)
-from tractor.msg import (
-    _ctxvar_MsgCodec,
-    # _codec,  XXX see `self._codec` sanity/debug checks
-    MsgCodec,
-    types as msgtypes,
-    pretty_struct,
-)
-
+from .log import get_logger
+from ._exceptions import TransportClosed
 log = get_logger(__name__)
 
+
 _is_windows = platform.system() == 'Windows'
+log = get_logger(__name__)
 
 
-def get_stream_addrs(
-    stream: trio.SocketStream
-) -> tuple[
-    tuple[str, int],  # local
-    tuple[str, int],  # remote
-]:
-    '''
-    Return the `trio` streaming transport prot's socket-addrs for
-    both the local and remote sides as a pair.
-
-    '''
-    # rn, should both be IP sockets
+def get_stream_addrs(stream: trio.SocketStream) -> tuple:
+    # should both be IP sockets
     lsockname = stream.socket.getsockname()
     rsockname = stream.socket.getpeername()
     return (
@@ -85,22 +59,16 @@ def get_stream_addrs(
     )
 
 
-# from tractor.msg.types import MsgType
-# ?TODO? this should be our `Union[*msgtypes.__spec__]` alias now right..?
-# => BLEH, except can't bc prots must inherit typevar or param-spec
-#   vars..
-MsgType = TypeVar('MsgType')
+MsgType = TypeVar("MsgType")
+
+# TODO: consider using a generic def and indexing with our eventual
+# msg definition/types?
+# - https://docs.python.org/3/library/typing.html#typing.Protocol
+# - https://jcristharif.com/msgspec/usage.html#structs
 
 
-# TODO: break up this mod into a subpkg so we can start adding new
-# backends and move this type stuff into a dedicated file.. Bo
-#
 @runtime_checkable
 class MsgTransport(Protocol[MsgType]):
-#
-# ^-TODO-^ consider using a generic def and indexing with our
-# eventual msg definition/types?
-# - https://docs.python.org/3/library/typing.html#typing.Protocol
 
     stream: trio.SocketStream
     drained: list[MsgType]
@@ -135,9 +103,9 @@ class MsgTransport(Protocol[MsgType]):
         ...
 
 
-# TODO: typing oddity.. not sure why we have to inherit here, but it
-# seems to be an issue with `get_msg_transport()` returning
-# a `Type[Protocol]`; probably should make a `mypy` issue?
+# TODO: not sure why we have to inherit here, but it seems to be an
+# issue with ``get_msg_transport()`` returning a ``Type[Protocol]``;
+# probably should make a `mypy` issue?
 class MsgpackTCPStream(MsgTransport):
     '''
     A ``trio.SocketStream`` delivering ``msgpack`` formatted data
@@ -156,16 +124,6 @@ class MsgpackTCPStream(MsgTransport):
         stream: trio.SocketStream,
         prefix_size: int = 4,
 
-        # XXX optionally provided codec pair for `msgspec`:
-        # https://jcristharif.com/msgspec/extending.html#mapping-to-from-native-types
-        #
-        # TODO: define this as a `Codec` struct which can be
-        # overriden dynamically by the application/runtime?
-        codec: tuple[
-            Callable[[Any], Any]|None,  # coder
-            Callable[[type, Any], Any]|None,  # decoder
-        ]|None = None,
-
     ) -> None:
 
         self.stream = stream
@@ -175,44 +133,30 @@ class MsgpackTCPStream(MsgTransport):
         self._laddr, self._raddr = get_stream_addrs(stream)
 
         # create read loop instance
-        self._aiter_pkts = self._iter_packets()
+        self._agen = self._iter_packets()
         self._send_lock = trio.StrictFIFOLock()
 
         # public i guess?
         self.drained: list[dict] = []
 
-        self.recv_stream = BufferedReceiveStream(
-            transport_stream=stream
-        )
+        self.recv_stream = BufferedReceiveStream(transport_stream=stream)
         self.prefix_size = prefix_size
 
-        # allow for custom IPC msg interchange format
-        # dynamic override Bo
-        self._task = trio.lowlevel.current_task()
-
-        # XXX for ctxvar debug only!
-        # self._codec: MsgCodec = (
-        #     codec
-        #     or
-        #     _codec._ctxvar_MsgCodec.get()
-        # )
+        # TODO: struct aware messaging coders
+        self.encode = msgspec.msgpack.Encoder().encode
+        self.decode = msgspec.msgpack.Decoder().decode  # dict[str, Any])
 
     async def _iter_packets(self) -> AsyncGenerator[dict, None]:
-        '''
-        Yield `bytes`-blob decoded packets from the underlying TCP
-        stream using the current task's `MsgCodec`.
-
-        This is a streaming routine implemented as an async generator
-        func (which was the original design, but could be changed?)
-        and is allocated by a `.__call__()` inside `.__init__()` where
-        it is assigned to the `._aiter_pkts` attr.
+        '''Yield packets from the underlying stream.
 
         '''
+        import msgspec  # noqa
         decodes_failed: int = 0
 
         while True:
             try:
-                header: bytes = await self.recv_stream.receive_exactly(4)
+                header = await self.recv_stream.receive_exactly(4)
+
             except (
                 ValueError,
                 ConnectionResetError,
@@ -221,122 +165,25 @@ class MsgpackTCPStream(MsgTransport):
                 # seem to be getting racy failures here on
                 # arbiter/registry name subs..
                 trio.BrokenResourceError,
-
-            ) as trans_err:
-
-                loglevel = 'transport'
-                match trans_err:
-                    # case (
-                    #     ConnectionResetError()
-                    # ):
-                    #     loglevel = 'transport'
-
-                    # peer actor (graceful??) TCP EOF but `tricycle`
-                    # seems to raise a 0-bytes-read?
-                    case ValueError() if (
-                        'unclean EOF' in trans_err.args[0]
-                    ):
-                        pass
-
-                    # peer actor (task) prolly shutdown quickly due
-                    # to cancellation
-                    case trio.BrokenResourceError() if (
-                        'Connection reset by peer' in trans_err.args[0]
-                    ):
-                        pass
-
-                    # unless the disconnect condition falls under "a
-                    # normal operation breakage" we usualy console warn
-                    # about it.
-                    case _:
-                        loglevel: str = 'warning'
-
-
+            ):
                 raise TransportClosed(
-                    message=(
-                        f'IPC transport already closed by peer\n'
-                        f'x]> {type(trans_err)}\n'
-                        f'  |_{self}\n'
-                    ),
-                    loglevel=loglevel,
-                ) from trans_err
-
-            # XXX definitely can happen if transport is closed
-            # manually by another `trio.lowlevel.Task` in the
-            # same actor; we use this in some simulated fault
-            # testing for ex, but generally should never happen
-            # under normal operation!
-            #
-            # NOTE: as such we always re-raise this error from the
-            #       RPC msg loop!
-            except trio.ClosedResourceError as closure_err:
-                raise TransportClosed(
-                    message=(
-                        f'IPC transport already manually closed locally?\n'
-                        f'x]> {type(closure_err)} \n'
-                        f'  |_{self}\n'
-                    ),
-                    loglevel='error',
-                    raise_on_report=(
-                        closure_err.args[0] == 'another task closed this fd'
-                        or
-                        closure_err.args[0] in ['another task closed this fd']
-                    ),
-                ) from closure_err
-
-            # graceful TCP EOF disconnect
-            if header == b'':
-                raise TransportClosed(
-                    message=(
-                        f'IPC transport already gracefully closed\n'
-                        f']>\n'
-                        f' |_{self}\n'
-                    ),
-                    loglevel='transport',
-                    # cause=???  # handy or no?
+                    f'transport {self} was already closed prior ro read'
+                )
+
+            if header == b'':
+                raise TransportClosed(
+                    f'transport {self} was already closed prior ro read'
                 )
 
-            size: int
             size, = struct.unpack("<I", header)
 
             log.transport(f'received header {size}')  # type: ignore
-            msg_bytes: bytes = await self.recv_stream.receive_exactly(size)
+
+            msg_bytes = await self.recv_stream.receive_exactly(size)
 
             log.transport(f"received {msg_bytes}")  # type: ignore
             try:
-                # NOTE: lookup the `trio.Task.context`'s var for
-                # the current `MsgCodec`.
-                codec: MsgCodec = _ctxvar_MsgCodec.get()
-
-                # XXX for ctxvar debug only!
-                # if self._codec.pld_spec != codec.pld_spec:
-                #     assert (
-                #         task := trio.lowlevel.current_task()
-                #     ) is not self._task
-                #     self._task = task
-                #     self._codec = codec
-                #     log.runtime(
-                #         f'Using new codec in {self}.recv()\n'
-                #         f'codec: {self._codec}\n\n'
-                #         f'msg_bytes: {msg_bytes}\n'
-                #     )
-                yield codec.decode(msg_bytes)
-
-            # XXX NOTE: since the below error derives from
-            # `DecodeError` we need to catch is specially
-            # and always raise such that spec violations
-            # are never allowed to be caught silently!
-            except msgspec.ValidationError as verr:
-                msgtyperr: MsgTypeError = _mk_recv_mte(
-                    msg=msg_bytes,
-                    codec=codec,
-                    src_validation_error=verr,
-                )
-                # XXX deliver up to `Channel.recv()` where
-                # a re-raise and `Error`-pack can inject the far
-                # end actor `.uid`.
-                yield msgtyperr
-
+                yield self.decode(msg_bytes)
             except (
                 msgspec.DecodeError,
                 UnicodeDecodeError,
@@ -346,95 +193,29 @@ class MsgpackTCPStream(MsgTransport):
                     # do with a channel drop - hope that receiving from the
                     # channel will raise an expected error and bubble up.
                     try:
-                        msg_str: str|bytes = msg_bytes.decode()
+                        msg_str: str | bytes = msg_bytes.decode()
                     except UnicodeDecodeError:
                         msg_str = msg_bytes
 
-                    log.exception(
-                        'Failed to decode msg?\n'
-                        f'{codec}\n\n'
-                        'Rxed bytes from wire:\n\n'
-                        f'{msg_str!r}\n'
+                    log.error(
+                        '`msgspec` failed to decode!?\n'
+                        'dumping bytes:\n'
+                        f'{msg_str!r}'
                     )
                     decodes_failed += 1
                 else:
                     raise
 
-    async def send(
-        self,
-        msg: msgtypes.MsgType,
-
-        strict_types: bool = True,
-        hide_tb: bool = False,
-
-    ) -> None:
-        '''
-        Send a msgpack encoded py-object-blob-as-msg over TCP.
-
-        If `strict_types == True` then a `MsgTypeError` will be raised on any
-        invalid msg type
-
-        '''
-        __tracebackhide__: bool = hide_tb
-
-        # XXX see `trio._sync.AsyncContextManagerMixin` for details
-        # on the `.acquire()`/`.release()` sequencing..
+    async def send(self, msg: Any) -> None:
         async with self._send_lock:
 
-            # NOTE: lookup the `trio.Task.context`'s var for
-            # the current `MsgCodec`.
-            codec: MsgCodec = _ctxvar_MsgCodec.get()
-
-            # XXX for ctxvar debug only!
-            # if self._codec.pld_spec != codec.pld_spec:
-            #     self._codec = codec
-            #     log.runtime(
-            #         f'Using new codec in {self}.send()\n'
-            #         f'codec: {self._codec}\n\n'
-            #         f'msg: {msg}\n'
-            #     )
-
-            if type(msg) not in msgtypes.__msg_types__:
-                if strict_types:
-                    raise _mk_send_mte(
-                        msg,
-                        codec=codec,
-                    )
-                else:
-                    log.warning(
-                        'Sending non-`Msg`-spec msg?\n\n'
-                        f'{msg}\n'
-                    )
-
-            try:
-                bytes_data: bytes = codec.encode(msg)
-            except TypeError as _err:
-                typerr = _err
-                msgtyperr: MsgTypeError = _mk_send_mte(
-                    msg,
-                    codec=codec,
-                    message=(
-                        f'IPC-msg-spec violation in\n\n'
-                        f'{pretty_struct.Struct.pformat(msg)}'
-                    ),
-                    src_type_error=typerr,
-                )
-                raise msgtyperr from typerr
+            bytes_data: bytes = self.encode(msg)
 
             # supposedly the fastest says,
             # https://stackoverflow.com/a/54027962
             size: bytes = struct.pack("<I", len(bytes_data))
-            return await self.stream.send_all(size + bytes_data)
 
-        # ?TODO? does it help ever to dynamically show this
-        # frame?
-        # try:
-        #     <the-above_code>
-        # except BaseException as _err:
-        #     err = _err
-        #     if not isinstance(err, MsgTypeError):
-        #         __tracebackhide__: bool = False
-        #     raise
+            return await self.stream.send_all(size + bytes_data)
 
     @property
     def laddr(self) -> tuple[str, int]:
@@ -445,7 +226,7 @@ class MsgpackTCPStream(MsgTransport):
         return self._raddr
 
     async def recv(self) -> Any:
-        return await self._aiter_pkts.asend(None)
+        return await self._agen.asend(None)
 
     async def drain(self) -> AsyncIterator[dict]:
         '''
@@ -462,7 +243,7 @@ class MsgpackTCPStream(MsgTransport):
                 yield msg
 
     def __aiter__(self):
-        return self._aiter_pkts
+        return self._agen
 
     def connected(self) -> bool:
         return self.stream.socket.fileno() != -1
@@ -517,7 +298,7 @@ class Channel:
         # set after handshake - always uid of far end
         self.uid: tuple[str, str]|None = None
 
-        self._aiter_msgs = self._iter_msgs()
+        self._agen = self._aiter_recv()
         self._exc: Exception|None = None  # set if far end actor errors
         self._closed: bool = False
 
@@ -528,9 +309,7 @@ class Channel:
 
     @property
     def msgstream(self) -> MsgTransport:
-        log.info(
-            '`Channel.msgstream` is an old name, use `._transport`'
-        )
+        log.info('`Channel.msgstream` is an old name, use `._transport`')
         return self._transport
 
     @property
@@ -561,45 +340,11 @@ class Channel:
         stream: trio.SocketStream,
         type_key: tuple[str, str]|None = None,
 
-        # XXX optionally provided codec pair for `msgspec`:
-        # https://jcristharif.com/msgspec/extending.html#mapping-to-from-native-types
-        codec: MsgCodec|None = None,
-
     ) -> MsgTransport:
-        type_key = (
-            type_key
-            or
-            self._transport_key
-        )
-        # get transport type, then
-        self._transport = get_msg_transport(
-            type_key
-        # instantiate an instance of the msg-transport
-        )(
-            stream,
-            codec=codec,
-        )
+        type_key = type_key or self._transport_key
+        self._transport = get_msg_transport(type_key)(stream)
         return self._transport
 
-    @cm
-    def apply_codec(
-        self,
-        codec: MsgCodec,
-
-    ) -> None:
-        '''
-        Temporarily override the underlying IPC msg codec for
-        dynamic enforcement of messaging schema.
-
-        '''
-        orig: MsgCodec = self._transport.codec
-        try:
-            self._transport.codec = codec
-            yield
-        finally:
-            self._transport.codec = orig
-
-    # TODO: do a .src/.dst: str for maddrs?
     def __repr__(self) -> str:
         if not self._transport:
             return '<Channel with inactive transport?>'
@@ -643,53 +388,20 @@ class Channel:
         )
         return transport
 
-    # TODO: something like,
-    # `pdbp.hideframe_on(errors=[MsgTypeError])`
-    # instead of the `try/except` hack we have rn..
-    # seems like a pretty useful thing to have in general
-    # along with being able to filter certain stack frame(s / sets)
-    # possibly based on the current log-level?
-    async def send(
-        self,
-        payload: Any,
+    async def send(self, item: Any) -> None:
 
-        hide_tb: bool = False,
+        log.transport(
+            '=> send IPC msg:\n\n'
+            f'{pformat(item)}\n'
+        )  # type: ignore
+        assert self._transport
 
-    ) -> None:
-        '''
-        Send a coded msg-blob over the transport.
-
-        '''
-        __tracebackhide__: bool = hide_tb
-        try:
-            log.transport(
-                '=> send IPC msg:\n\n'
-                f'{pformat(payload)}\n'
-            )
-            # assert self._transport  # but why typing?
-            await self._transport.send(
-                payload,
-                hide_tb=hide_tb,
-            )
-        except BaseException as _err:
-            err = _err  # bind for introspection
-            if not isinstance(_err, MsgTypeError):
-                # assert err
-                __tracebackhide__: bool = False
-            else:
-                assert err.cid
-
-            raise
+        await self._transport.send(item)
 
     async def recv(self) -> Any:
         assert self._transport
         return await self._transport.recv()
 
-        # TODO: auto-reconnect features like 0mq/nanomsg?
-        # -[ ] implement it manually with nods to SC prot
-        #      possibly on multiple transport backends?
-        #  -> seems like that might be re-inventing scalability
-        #     prots tho no?
         # try:
         #     return await self._transport.recv()
         # except trio.BrokenResourceError:
@@ -716,11 +428,8 @@ class Channel:
         await self.aclose(*args)
 
     def __aiter__(self):
-        return self._aiter_msgs
+        return self._agen
 
-    # ?TODO? run any reconnection sequence?
-    # -[ ] prolly should be impl-ed as deco-API?
-    #
     # async def _reconnect(self) -> None:
     #     """Handle connection failures by polling until a reconnect can be
     #     established.
@@ -738,6 +447,7 @@ class Channel:
     #             else:
     #                 log.transport("Stream connection re-established!")
 
+    #                 # TODO: run any reconnection sequence
     #                 # on_recon = self._recon_seq
     #                 # if on_recon:
     #                 #     await on_recon(self)
@@ -751,42 +461,23 @@ class Channel:
     #                     " for re-establishment")
     #             await trio.sleep(1)
 
-    async def _iter_msgs(
+    async def _aiter_recv(
         self
     ) -> AsyncGenerator[Any, None]:
         '''
-        Yield `MsgType` IPC msgs decoded and deliverd from
-        an underlying `MsgTransport` protocol.
-
-        This is a streaming routine alo implemented as an async-gen
-        func (same a `MsgTransport._iter_pkts()`) gets allocated by
-        a `.__call__()` inside `.__init__()` where it is assigned to
-        the `._aiter_msgs` attr.
+        Async iterate items from underlying stream.
 
         '''
         assert self._transport
         while True:
             try:
-                async for msg in self._transport:
-                    match msg:
-                        # NOTE: if transport/interchange delivers
-                        # a type error, we pack it with the far
-                        # end peer `Actor.uid` and relay the
-                        # `Error`-msg upward to the `._rpc` stack
-                        # for normal RAE handling.
-                        case MsgTypeError():
-                            yield pack_from_raise(
-                                local_err=msg,
-                                cid=msg.cid,
-
-                                # XXX we pack it here bc lower
-                                # layers have no notion of an
-                                # actor-id ;)
-                                src_uid=self.uid,
-                            )
-                        case _:
-                            yield msg
-
+                async for item in self._transport:
+                    yield item
+                    # sent = yield item
+                    # if sent is not None:
+                    #     # optimization, passing None through all the
+                    #     # time is pointless
+                    #     await self._transport.send(sent)
             except trio.BrokenResourceError:
 
                 # if not self._autorecon:
@@ -802,7 +493,7 @@ class Channel:
         return self._transport.connected() if self._transport else False
 
 
-@acm
+@asynccontextmanager
 async def _connect_chan(
     host: str,
     port: int
@@ -816,5 +507,4 @@ async def _connect_chan(
     chan = Channel((host, port))
     await chan.connect()
     yield chan
-    with trio.CancelScope(shield=True):
-        await chan.aclose()
+    await chan.aclose()

tractor/_portal.py

@@ -24,51 +24,84 @@ OS processes, possibly on different (hardware) hosts.
 
 '''
 from __future__ import annotations
-from contextlib import asynccontextmanager as acm
 import importlib
 import inspect
 from typing import (
     Any,
     Callable,
     AsyncGenerator,
-    TYPE_CHECKING,
+    Type,
 )
 from functools import partial
 from dataclasses import dataclass
 import warnings
 
 import trio
+from async_generator import asynccontextmanager
 
 from .trionics import maybe_open_nursery
+from .devx import (
+    # acquire_debug_lock,
+    pause,
+    maybe_wait_for_debugger,
+)
 from ._state import (
     current_actor,
+    debug_mode,
 )
 from ._ipc import Channel
 from .log import get_logger
-from .msg import (
-    # Error,
-    PayloadMsg,
-    NamespacePath,
-    Return,
-)
+from .msg import NamespacePath
 from ._exceptions import (
-    # unpack_error,
+    InternalError,
+    _raise_from_no_key_in_msg,
+    unpack_error,
     NoResult,
+    ContextCancelled,
+    RemoteActorError,
 )
 from ._context import (
     Context,
-    open_context_from_portal,
 )
 from ._streaming import (
     MsgStream,
 )
 
-if TYPE_CHECKING:
-    from ._runtime import Actor
 
 log = get_logger(__name__)
 
 
+# TODO: rename to `unwrap_result()` and use
+# `._raise_from_no_key_in_msg()` (after tweak to
+# accept a `chan: Channel` arg) in key block!
+def _unwrap_msg(
+    msg: dict[str, Any],
+    channel: Channel,
+
+    hide_tb: bool = True,
+
+) -> Any:
+    '''
+    Unwrap a final result from a `{return: <Any>}` IPC msg.
+
+    '''
+    __tracebackhide__: bool = hide_tb
+
+    try:
+        return msg['return']
+    except KeyError as ke:
+
+        # internal error should never get here
+        assert msg.get('cid'), (
+            "Received internal error at portal?"
+        )
+
+        raise unpack_error(
+            msg,
+            channel
+        ) from ke
+
+
 class Portal:
     '''
     A 'portal' to a memory-domain-separated `Actor`.
@@ -92,26 +125,17 @@ class Portal:
     # connected (peer) actors.
     cancel_timeout: float = 0.5
 
-    def __init__(
-        self,
-        channel: Channel,
-    ) -> None:
-
-        self._chan: Channel = channel
+    def __init__(self, channel: Channel) -> None:
+        self.chan = channel
         # during the portal's lifetime
-        self._final_result_pld: Any|None = None
-        self._final_result_msg: PayloadMsg|None = None
+        self._result_msg: dict|None = None
 
         # When set to a ``Context`` (when _submit_for_result is called)
         # it is expected that ``result()`` will be awaited at some
         # point.
-        self._expect_result_ctx: Context|None = None
+        self._expect_result: Context | None = None
         self._streams: set[MsgStream] = set()
-        self.actor: Actor = current_actor()
-
-    @property
-    def chan(self) -> Channel:
-        return self._chan
+        self.actor = current_actor()
 
     @property
     def channel(self) -> Channel:
@@ -125,8 +149,6 @@ class Portal:
         )
         return self.chan
 
-    # TODO: factor this out into a `.highlevel` API-wrapper that uses
-    # a single `.open_context()` call underneath.
     async def _submit_for_result(
         self,
         ns: str,
@@ -134,34 +156,32 @@ class Portal:
         **kwargs
     ) -> None:
 
-        if self._expect_result_ctx is not None:
-            raise RuntimeError(
-                'A pending main result has already been submitted'
-            )
-
-        self._expect_result_ctx: Context = await self.actor.start_remote_task(
-            self.channel,
-            nsf=NamespacePath(f'{ns}:{func}'),
-            kwargs=kwargs,
-            portal=self,
+        assert self._expect_result is None, (
+            "A pending main result has already been submitted"
         )
 
-    # TODO: we should deprecate this API right? since if we remove
-    # `.run_in_actor()` (and instead move it to a `.highlevel`
-    # wrapper api (around a single `.open_context()` call) we don't
-    # really have any notion of a "main" remote task any more?
-    #
-    # @api_frame
-    async def wait_for_result(
+        self._expect_result = await self.actor.start_remote_task(
+            self.channel,
+            nsf=NamespacePath(f'{ns}:{func}'),
+            kwargs=kwargs
+        )
+
+    async def _return_once(
         self,
-        hide_tb: bool = True,
-    ) -> Any:
+        ctx: Context,
+
+    ) -> dict[str, Any]:
+
+        assert ctx._remote_func_type == 'asyncfunc'  # single response
+        msg: dict = await ctx._recv_chan.receive()
+        return msg
+
+    async def result(self) -> Any:
         '''
-        Return the final result delivered by a `Return`-msg from the
-        remote peer actor's "main" task's `return` statement.
+        Return the result(s) from the remote actor's "main" task.
 
         '''
-        __tracebackhide__: bool = hide_tb
+        # __tracebackhide__ = True
         # Check for non-rpc errors slapped on the
         # channel for which we always raise
         exc = self.channel._exc
@@ -169,7 +189,7 @@ class Portal:
             raise exc
 
         # not expecting a "main" result
-        if self._expect_result_ctx is None:
+        if self._expect_result is None:
             log.warning(
                 f"Portal for {self.channel.uid} not expecting a final"
                 " result?\nresult() should only be called if subactor"
@@ -177,40 +197,16 @@ class Portal:
             return NoResult
 
         # expecting a "main" result
-        assert self._expect_result_ctx
+        assert self._expect_result
 
-        if self._final_result_msg is None:
-            try:
-                (
-                    self._final_result_msg,
-                    self._final_result_pld,
-                ) = await self._expect_result_ctx._pld_rx.recv_msg(
-                    ipc=self._expect_result_ctx,
-                    expect_msg=Return,
-                )
-            except BaseException as err:
-                # TODO: wrap this into `@api_frame` optionally with
-                # some kinda filtering mechanism like log levels?
-                __tracebackhide__: bool = False
-                raise err
+        if self._result_msg is None:
+            self._result_msg = await self._return_once(
+                self._expect_result
+            )
 
-        return self._final_result_pld
-
-    # TODO: factor this out into a `.highlevel` API-wrapper that uses
-    # a single `.open_context()` call underneath.
-    async def result(
-        self,
-        *args,
-        **kwargs,
-    ) -> Any|Exception:
-        typname: str = type(self).__name__
-        log.warning(
-            f'`{typname}.result()` is DEPRECATED!\n'
-            f'Use `{typname}.wait_for_result()` instead!\n'
-        )
-        return await self.wait_for_result(
-            *args,
-            **kwargs,
+        return _unwrap_msg(
+            self._result_msg,
+            self.channel,
         )
 
     async def _cancel_streams(self):
@@ -253,8 +249,6 @@ class Portal:
         purpose.
 
         '''
-        __runtimeframe__: int = 1  # noqa
-
         chan: Channel = self.channel
         if not chan.connected():
             log.runtime(
@@ -263,15 +257,14 @@ class Portal:
             return False
 
         reminfo: str = (
-            f'c)=> {self.channel.uid}\n'
-            f'  |_{chan}\n'
+            f'`Portal.cancel_actor()` => {self.channel.uid}\n'
+            f' |_{chan}\n'
         )
         log.cancel(
-            f'Requesting actor-runtime cancel for peer\n\n'
+            f'Sending runtime `.cancel()` request to peer\n\n'
             f'{reminfo}'
         )
 
-        # XXX the one spot we set it?
         self.channel._cancel_called: bool = True
         try:
             # send cancel cmd - might not get response
@@ -311,8 +304,6 @@ class Portal:
             )
             return False
 
-    # TODO: do we still need this for low level `Actor`-runtime
-    # method calls or can we also remove it?
     async def run_from_ns(
         self,
         namespace_path: str,
@@ -335,23 +326,21 @@ class Portal:
           internals!
 
         '''
-        __runtimeframe__: int = 1  # noqa
         nsf = NamespacePath(
             f'{namespace_path}:{function_name}'
         )
-        ctx: Context = await self.actor.start_remote_task(
+        ctx = await self.actor.start_remote_task(
             chan=self.channel,
             nsf=nsf,
             kwargs=kwargs,
-            portal=self,
         )
-        return await ctx._pld_rx.recv_pld(
-            ipc=ctx,
-            expect_msg=Return,
+        ctx._portal = self
+        msg = await self._return_once(ctx)
+        return _unwrap_msg(
+            msg,
+            self.channel,
         )
 
-    # TODO: factor this out into a `.highlevel` API-wrapper that uses
-    # a single `.open_context()` call underneath.
     async def run(
         self,
         func: str,
@@ -367,8 +356,6 @@ class Portal:
         remote rpc task or a local async generator instance.
 
         '''
-        __runtimeframe__: int = 1  # noqa
-
         if isinstance(func, str):
             warnings.warn(
                 "`Portal.run(namespace: str, funcname: str)` is now"
@@ -399,30 +386,20 @@ class Portal:
             self.channel,
             nsf=nsf,
             kwargs=kwargs,
-            portal=self,
         )
-        return await ctx._pld_rx.recv_pld(
-            ipc=ctx,
-            expect_msg=Return,
+        ctx._portal = self
+        return _unwrap_msg(
+            await self._return_once(ctx),
+            self.channel,
         )
 
-    # TODO: factor this out into a `.highlevel` API-wrapper that uses
-    # a single `.open_context()` call underneath.
-    @acm
+    @asynccontextmanager
     async def open_stream_from(
         self,
         async_gen_func: Callable,  # typing: ignore
         **kwargs,
 
     ) -> AsyncGenerator[MsgStream, None]:
-        '''
-        Legacy one-way streaming API.
-
-        TODO: re-impl on top `Portal.open_context()` + an async gen
-        around `Context.open_stream()`.
-
-        '''
-        __runtimeframe__: int = 1  # noqa
 
         if not inspect.isasyncgenfunction(async_gen_func):
             if not (
@@ -436,8 +413,8 @@ class Portal:
             self.channel,
             nsf=NamespacePath.from_ref(async_gen_func),
             kwargs=kwargs,
-            portal=self,
         )
+        ctx._portal = self
 
         # ensure receive-only stream entrypoint
         assert ctx._remote_func_type == 'asyncgen'
@@ -446,13 +423,13 @@ class Portal:
             # deliver receive only stream
             async with MsgStream(
                 ctx=ctx,
-                rx_chan=ctx._rx_chan,
-            ) as stream:
-                self._streams.add(stream)
-                ctx._stream = stream
-                yield stream
+                rx_chan=ctx._recv_chan,
+            ) as rchan:
+                self._streams.add(rchan)
+                yield rchan
 
         finally:
+
             # cancel the far end task on consumer close
             # NOTE: this is a special case since we assume that if using
             # this ``.open_fream_from()`` api, the stream is one a one
@@ -471,14 +448,533 @@ class Portal:
 
             # XXX: should this always be done?
             # await recv_chan.aclose()
-            self._streams.remove(stream)
+            self._streams.remove(rchan)
 
-    # NOTE: impl is found in `._context`` mod to make
-    # reading/groking the details simpler code-org-wise. This
-    # method does not have to be used over that `@acm` module func
-    # directly, it is for conventience and from the original API
-    # design.
-    open_context = open_context_from_portal
+    # TODO: move this impl to `._context` mod and
+    # instead just bind it here as a method so that the logic
+    # for ctx stuff stays all in one place (instead of frickin
+    # having to open this file in tandem every gd time!!! XD)
+    #
+    @asynccontextmanager
+    async def open_context(
+
+        self,
+        func: Callable,
+
+        allow_overruns: bool = False,
+
+        # TODO: if we set this the wrapping `@acm` body will
+        # still be shown (awkwardly) on pdb REPL entry. Ideally
+        # we can similarly annotate that frame to NOT show?
+        hide_tb: bool = False,
+
+        # proxied to RPC
+        **kwargs,
+
+    ) -> AsyncGenerator[tuple[Context, Any], None]:
+        '''
+        Open an inter-actor "task context"; a remote task is
+        scheduled and cancel-scope-state-linked to a `trio.run()` across
+        memory boundaries in another actor's runtime.
+
+        This is an `@acm` API which allows for deterministic setup
+        and teardown of a remotely scheduled task in another remote
+        actor. Once opened, the 2 now "linked" tasks run completely
+        in parallel in each actor's runtime with their enclosing
+        `trio.CancelScope`s kept in a synced state wherein if
+        either side errors or cancels an equivalent error is
+        relayed to the other side via an SC-compat IPC protocol.
+
+        The yielded `tuple` is a pair delivering a `tractor.Context`
+        and any first value "sent" by the "callee" task via a call
+        to `Context.started(<value: Any>)`; this side of the
+        context does not unblock until the "callee" task calls
+        `.started()` in similar style to `trio.Nursery.start()`.
+        When the "callee" (side that is "called"/started by a call
+        to *this* method) returns, the caller side (this) unblocks
+        and any final value delivered from the other end can be
+        retrieved using the `Contex.result()` api.
+
+        The yielded ``Context`` instance further allows for opening
+        bidirectional streams, explicit cancellation and
+        structurred-concurrency-synchronized final result-msg
+        collection. See ``tractor.Context`` for more details.
+
+        '''
+        __tracebackhide__: bool = hide_tb
+
+        # conduct target func method structural checks
+        if not inspect.iscoroutinefunction(func) and (
+            getattr(func, '_tractor_contex_function', False)
+        ):
+            raise TypeError(
+                f'{func} must be an async generator function!')
+
+        # TODO: i think from here onward should probably
+        # just be factored into an `@acm` inside a new
+        # a new `_context.py` mod.
+        nsf = NamespacePath.from_ref(func)
+
+        ctx: Context = await self.actor.start_remote_task(
+            self.channel,
+            nsf=nsf,
+            kwargs=kwargs,
+
+            # NOTE: it's imporant to expose this since you might
+            # get the case where the parent who opened the context does
+            # not open a stream until after some slow startup/init
+            # period, in which case when the first msg is read from
+            # the feeder mem chan, say when first calling
+            # `Context.open_stream(allow_overruns=True)`, the overrun condition will be
+            # raised before any ignoring of overflow msgs can take
+            # place..
+            allow_overruns=allow_overruns,
+        )
+
+        assert ctx._remote_func_type == 'context'
+        msg: dict = await ctx._recv_chan.receive()
+
+        try:
+            # the "first" value here is delivered by the callee's
+            # ``Context.started()`` call.
+            first: Any = msg['started']
+            ctx._started_called: bool = True
+
+        except KeyError as src_error:
+            _raise_from_no_key_in_msg(
+                ctx=ctx,
+                msg=msg,
+                src_err=src_error,
+                log=log,
+                expect_key='started',
+            )
+
+        ctx._portal: Portal = self
+        uid: tuple = self.channel.uid
+        cid: str = ctx.cid
+
+        # placeholder for any exception raised in the runtime
+        # or by user tasks which cause this context's closure.
+        scope_err: BaseException|None = None
+        ctxc_from_callee: ContextCancelled|bool = False
+        try:
+            async with trio.open_nursery() as nurse:
+
+                # NOTE: used to start overrun queuing tasks
+                ctx._scope_nursery: trio.Nursery = nurse
+                ctx._scope: trio.CancelScope = nurse.cancel_scope
+
+                # deliver context instance and .started() msg value
+                # in enter tuple.
+                yield ctx, first
+
+                # ??TODO??: do we still want to consider this or is
+                # the `else:` block handling via a `.result()`
+                # call below enough??
+                # -[ ] pretty sure `.result()` internals do the
+                # same as our ctxc handler below so it ended up
+                # being same (repeated?) behaviour, but ideally we
+                # wouldn't have that duplication either by somehow
+                # factoring the `.result()` handler impl in a way
+                # that we can re-use it around the `yield` ^ here
+                # or vice versa?
+                #
+                # NOTE: between the caller exiting and arriving
+                # here the far end may have sent a ctxc-msg or
+                # other error, so check for it here immediately
+                # and maybe raise so as to engage the ctxc
+                # handling block below!
+                #
+                # if re := ctx._remote_error:
+                #     maybe_ctxc: ContextCancelled|None = ctx._maybe_raise_remote_err(
+                #         re,
+                #         # TODO: do we want this to always raise?
+                #         # - means that on self-ctxc, if/when the
+                #         #   block is exited before the msg arrives
+                #         #   but then the msg during __exit__
+                #         #   calling we may not activate the
+                #         #   ctxc-handler block below? should we
+                #         #   be?
+                #         # - if there's a remote error that arrives
+                #         #   after the child has exited, we won't
+                #         #   handle until the `finally:` block
+                #         #   where `.result()` is always called,
+                #         #   again in which case we handle it
+                #         #   differently then in the handler block
+                #         #   that would normally engage from THIS
+                #         #   block?
+                #         raise_ctxc_from_self_call=True,
+                #     )
+                #     ctxc_from_callee = maybe_ctxc
+
+                # when in allow_overruns mode there may be
+                # lingering overflow sender tasks remaining?
+                if nurse.child_tasks:
+                    # XXX: ensure we are in overrun state
+                    # with ``._allow_overruns=True`` bc otherwise
+                    # there should be no tasks in this nursery!
+                    if (
+                        not ctx._allow_overruns
+                        or len(nurse.child_tasks) > 1
+                    ):
+                        raise InternalError(
+                            'Context has sub-tasks but is '
+                            'not in `allow_overruns=True` mode!?'
+                        )
+
+                    # ensure we cancel all overflow sender
+                    # tasks started in the nursery when
+                    # `._allow_overruns == True`.
+                    #
+                    # NOTE: this means `._scope.cancelled_caught`
+                    # will prolly be set! not sure if that's
+                    # non-ideal or not ???
+                    ctx._scope.cancel()
+
+        # XXX NOTE XXX: maybe shield against
+        # self-context-cancellation (which raises a local
+        # `ContextCancelled`) when requested (via
+        # `Context.cancel()`) by the same task (tree) which entered
+        # THIS `.open_context()`.
+        #
+        # NOTE: There are 2 operating cases for a "graceful cancel"
+        # of a `Context`. In both cases any `ContextCancelled`
+        # raised in this scope-block came from a transport msg
+        # relayed from some remote-actor-task which our runtime set
+        # as to `Context._remote_error`
+        #
+        # the CASES:
+        #
+        # - if that context IS THE SAME ONE that called
+        #   `Context.cancel()`, we want to absorb the error
+        #   silently and let this `.open_context()` block to exit
+        #   without raising, ideally eventually receiving the ctxc
+        #   ack msg thus resulting in `ctx.cancel_acked == True`.
+        #
+        # - if it is from some OTHER context (we did NOT call
+        #   `.cancel()`), we want to re-RAISE IT whilst also
+        #   setting our own ctx's "reason for cancel" to be that
+        #   other context's cancellation condition; we set our
+        #   `.canceller: tuple[str, str]` to be same value as
+        #   caught here in a `ContextCancelled.canceller`.
+        #
+        # AGAIN to restate the above, there are 2 cases:
+        #
+        # 1-some other context opened in this `.open_context()`
+        #   block cancelled due to a self or peer cancellation
+        #   request in which case we DO let the error bubble to the
+        #   opener.
+        #
+        # 2-THIS "caller" task somewhere invoked `Context.cancel()`
+        #   and received a `ContextCanclled` from the "callee"
+        #   task, in which case we mask the `ContextCancelled` from
+        #   bubbling to this "caller" (much like how `trio.Nursery`
+        #   swallows any `trio.Cancelled` bubbled by a call to
+        #   `Nursery.cancel_scope.cancel()`)
+        except ContextCancelled as ctxc:
+            scope_err = ctxc
+            # XXX TODO XXX: FIX THIS debug_mode BUGGGG!!!
+            # using this code and then resuming the REPL will
+            # cause a SIGINT-ignoring HANG!
+            # -> prolly due to a stale debug lock entry..
+            # -[ ] USE `.stackscope` to demonstrate that (possibly
+            #   documenting it as a definittive example of
+            #   debugging the tractor-runtime itself using it's
+            #   own `.devx.` tooling!
+            # 
+            # await pause()
+
+            # ctxc_from_callee: bool = (
+            #     ctxc.src_actor_uid == self.chan.uid
+            # )
+
+            # CASE 2: context was cancelled by local task calling
+            # `.cancel()`, we don't raise and the exit block should
+            # exit silently.
+            if (
+                ctx._cancel_called
+                and
+                ctxc is ctx._remote_error
+                and
+                ctxc.canceller == self.actor.uid
+            ):
+                ctxc_from_callee = True
+                log.cancel(
+                    f'Context (cid=[{ctx.cid[-6:]}..] cancelled gracefully with:\n'
+                    f'{ctxc}'
+                )
+            # CASE 1: this context was never cancelled via a local
+            # task (tree) having called `Context.cancel()`, raise
+            # the error since it was caused by someone else
+            # -> probably a remote peer!
+            else:
+                raise
+
+        # the above `._scope` can be cancelled due to:
+        # 1. an explicit self cancel via `Context.cancel()` or
+        #    `Actor.cancel()`,
+        # 2. any "callee"-side remote error, possibly also a cancellation
+        #    request by some peer,
+        # 3. any "caller" (aka THIS scope's) local error raised in the above `yield`
+        except (
+            # CASE 3: standard local error in this caller/yieldee
+            Exception,
+
+            # CASES 1 & 2: can manifest as a `ctx._scope_nursery`
+            # exception-group of,
+            #
+            # 1.-`trio.Cancelled`s, since
+            #   `._scope.cancel()` will have been called
+            #   (transitively by the runtime calling
+            #   `._deliver_msg()`) and any `ContextCancelled`
+            #   eventually absorbed and thus absorbed/supressed in
+            #   any `Context._maybe_raise_remote_err()` call.
+            #
+            # 2.-`BaseExceptionGroup[ContextCancelled | RemoteActorError]`
+            #    from any error delivered from the "callee" side
+            #    AND a group-exc is only raised if there was > 1
+            #    tasks started *here* in the "caller" / opener
+            #    block. If any one of those tasks calls
+            #    `.result()` or `MsgStream.receive()`
+            #    `._maybe_raise_remote_err()` will be transitively
+            #    called and the remote error raised causing all
+            #    tasks to be cancelled.
+            #    NOTE: ^ this case always can happen if any
+            #    overrun handler tasks were spawned!
+            BaseExceptionGroup,
+
+            trio.Cancelled,  # NOTE: NOT from inside the ctx._scope
+            KeyboardInterrupt,
+
+        ) as caller_err:
+            scope_err = caller_err
+
+            # XXX: ALWAYS request the context to CANCEL ON any ERROR.
+            # NOTE: `Context.cancel()` is conversely NEVER CALLED in
+            # the `ContextCancelled` "self cancellation absorbed" case
+            # handled in the block above ^^^ !!
+            log.cancel(
+                'Context terminated due to\n\n'
+                f'{caller_err}\n'
+            )
+
+            # TODO: does this need to be shielded?
+            if debug_mode():
+                # async with acquire_debug_lock(self.actor.uid):
+                #     pass
+                # TODO: factor ^ into below for non-root cases?
+                was_acquired: bool = await maybe_wait_for_debugger(
+                    header_msg=(
+                        'Delaying `ctx.cancel()` until debug lock '
+                        'acquired..\n'
+                    ),
+                )
+                if was_acquired:
+                    log.pdb(
+                        'Acquired debug lock! '
+                        'Calling `ctx.cancel()`!\n'
+                    )
+
+            # we don't need to cancel the callee if it already
+            # told us it's cancelled ;p
+            if not ctxc_from_callee:
+                log.critical('SENDING CANCEL')
+                try:
+                    # await pause(shield=True)
+                    with trio.CancelScope(shield=True):
+                        await ctx.cancel()
+                except (
+                    trio.BrokenResourceError,
+                    trio.ClosedResourceError,
+                ):
+                    log.warning(
+                        'IPC connection for context is broken?\n'
+                        f'task:{cid}\n'
+                        f'actor:{uid}'
+                    )
+
+            raise  # duh
+
+        # no local scope error, the "clean exit with a result" case.
+        else:
+            if ctx.chan.connected():
+                log.runtime(
+                    'Waiting on final context result for\n'
+                    f'peer: {uid}\n'
+                    f'|_{ctx._task}\n'
+                )
+                # XXX NOTE XXX: the below call to
+                # `Context.result()` will ALWAYS raise
+                # a `ContextCancelled` (via an embedded call to
+                # `Context._maybe_raise_remote_err()`) IFF
+                # a `Context._remote_error` was set by the runtime
+                # via a call to
+                # `Context._maybe_cancel_and_set_remote_error()`.
+                # As per `Context._deliver_msg()`, that error IS
+                # ALWAYS SET any time "callee" side fails and causes "caller
+                # side" cancellation via a `ContextCancelled` here.
+                try:
+                    result_or_err: Exception|Any = await ctx.result()
+                except BaseException as berr:
+                    # on normal teardown, if we get some error
+                    # raised in `Context.result()` we still want to
+                    # save that error on the ctx's state to
+                    # determine things like `.cancelled_caught` for
+                    # cases where there was remote cancellation but
+                    # this task didn't know until final teardown
+                    # / value collection.
+                    scope_err = berr
+                    raise
+
+                # yes! this worx Bp
+                # from .devx import _debug
+                # await _debug.pause()
+
+                # an exception type boxed in a `RemoteActorError`
+                # is returned (meaning it was obvi not raised)
+                # that we want to log-report on.
+                msgdata: str|None = getattr(
+                    result_or_err,
+                    'msgdata',
+                    None
+                )
+                match (msgdata, result_or_err):
+                    case (
+                        {'tb_str': tbstr},
+                        ContextCancelled(),
+                    ):
+                        log.cancel(tbstr)
+
+                    case (
+                        {'tb_str': tbstr},
+                        RemoteActorError(),
+                    ):
+                        log.exception(
+                            'Context remotely errored!\n'
+                            f'<= peer: {uid}\n'
+                            f'  |_ {nsf}()\n\n'
+
+                            f'{tbstr}'
+                        )
+                    case (None, _):
+                        log.runtime(
+                            'Context returned final result from callee task:\n'
+                            f'<= peer: {uid}\n'
+                            f'  |_ {nsf}()\n\n'
+
+                            f'`{result_or_err}`\n'
+                        )
+
+        finally:
+            # XXX: (MEGA IMPORTANT) if this is a root opened process we
+            # wait for any immediate child in debug before popping the
+            # context from the runtime msg loop otherwise inside
+            # ``Actor._push_result()`` the msg will be discarded and in
+            # the case where that msg is global debugger unlock (via
+            # a "stop" msg for a stream), this can result in a deadlock
+            # where the root is waiting on the lock to clear but the
+            # child has already cleared it and clobbered IPC.
+            await maybe_wait_for_debugger()
+
+            # though it should be impossible for any tasks
+            # operating *in* this scope to have survived
+            # we tear down the runtime feeder chan last
+            # to avoid premature stream clobbers.
+            if (
+                (rxchan := ctx._recv_chan)
+
+                # maybe TODO: yes i know the below check is
+                # touching `trio` memchan internals..BUT, there are
+                # only a couple ways to avoid a `trio.Cancelled`
+                # bubbling from the `.aclose()` call below:
+                #
+                # - catch and mask it via the cancel-scope-shielded call
+                #   as we are rn (manual and frowned upon) OR,
+                # - specially handle the case where `scope_err` is
+                #   one of {`BaseExceptionGroup`, `trio.Cancelled`}
+                #   and then presume that the `.aclose()` call will
+                #   raise a `trio.Cancelled` and just don't call it
+                #   in those cases..
+                #
+                # that latter approach is more logic, LOC, and more
+                # convoluted so for now stick with the first
+                # psuedo-hack-workaround where we just try to avoid
+                # the shielded call as much as we can detect from
+                # the memchan's `._closed` state..
+                #
+                # XXX MOTIVATION XXX-> we generally want to raise
+                # any underlying actor-runtime/internals error that
+                # surfaces from a bug in tractor itself so it can
+                # be easily detected/fixed AND, we also want to
+                # minimize noisy runtime tracebacks (normally due
+                # to the cross-actor linked task scope machinery
+                # teardown) displayed to user-code and instead only
+                # displaying `ContextCancelled` traces where the
+                # cause of crash/exit IS due to something in
+                # user/app code on either end of the context.
+                and not rxchan._closed
+            ):
+                # XXX NOTE XXX: and again as per above, we mask any
+                # `trio.Cancelled` raised here so as to NOT mask
+                # out any exception group or legit (remote) ctx
+                # error that sourced from the remote task or its
+                # runtime.
+                #
+                # NOTE: further, this should be the only place the
+                # underlying feeder channel is
+                # once-and-only-CLOSED!
+                with trio.CancelScope(shield=True):
+                    await ctx._recv_chan.aclose()
+
+            # XXX: we always raise remote errors locally and
+            # generally speaking mask runtime-machinery related
+            # multi-`trio.Cancelled`s. As such, any `scope_error`
+            # which was the underlying cause of this context's exit
+            # should be stored as the `Context._local_error` and
+            # used in determining `Context.cancelled_caught: bool`.
+            if scope_err is not None:
+                ctx._local_error: BaseException = scope_err
+                etype: Type[BaseException] = type(scope_err)
+
+                # CASE 2
+                if (
+                    ctx._cancel_called
+                    and ctx.cancel_acked
+                ):
+                    log.cancel(
+                        'Context cancelled by caller task\n'
+                        f'|_{ctx._task}\n\n'
+
+                        f'{repr(scope_err)}\n'
+                    )
+
+                # TODO: should we add a `._cancel_req_received`
+                # flag to determine if the callee manually called
+                # `ctx.cancel()`?
+                # -[ ] going to need a cid check no?
+
+                # CASE 1
+                else:
+                    outcome_str: str = ctx.repr_outcome(
+                        show_error_fields=True,
+                        # type_only=True,
+                    )
+                    log.cancel(
+                        f'Context terminated due to local scope error:\n\n'
+                        f'{ctx.chan.uid} => {outcome_str}\n'
+                    )
+
+            # FINALLY, remove the context from runtime tracking and
+            # exit!
+            log.runtime(
+                'Removing IPC ctx opened with peer\n'
+                f'{uid}\n'
+                f'|_{ctx}\n'
+            )
+            self.actor._contexts.pop(
+                (uid, cid),
+                None,
+            )
 
 
 @dataclass
@@ -509,11 +1005,11 @@ class LocalPortal:
         return await func(**kwargs)
 
 
-@acm
+@asynccontextmanager
 async def open_portal(
 
     channel: Channel,
-    tn: trio.Nursery|None = None,
+    nursery: trio.Nursery|None = None,
     start_msg_loop: bool = True,
     shield: bool = False,
 
@@ -521,23 +1017,15 @@ async def open_portal(
     '''
     Open a ``Portal`` through the provided ``channel``.
 
-    Spawns a background task to handle RPC processing, normally
-    done by the actor-runtime implicitly via a call to
-    `._rpc.process_messages()`. just after connection establishment.
+    Spawns a background task to handle message processing (normally
+    done by the actor-runtime implicitly).
 
     '''
     actor = current_actor()
     assert actor
-    was_connected: bool = False
+    was_connected = False
 
-    async with maybe_open_nursery(
-        tn,
-        shield=shield,
-        strict_exception_groups=False,
-        # ^XXX^ TODO? soo roll our own then ??
-        # -> since we kinda want the "if only one `.exception` then
-        # just raise that" interface?
-    ) as tn:
+    async with maybe_open_nursery(nursery, shield=shield) as nursery:
 
         if not channel.connected():
             await channel.connect()
@@ -549,7 +1037,7 @@ async def open_portal(
         msg_loop_cs: trio.CancelScope|None = None
         if start_msg_loop:
             from ._runtime import process_messages
-            msg_loop_cs = await tn.start(
+            msg_loop_cs = await nursery.start(
                 partial(
                     process_messages,
                     actor,
@@ -566,10 +1054,12 @@ async def open_portal(
             await portal.aclose()
 
             if was_connected:
-                await channel.aclose()
+                # gracefully signal remote channel-msg loop
+                await channel.send(None)
+                # await channel.aclose()
 
             # cancel background msg loop task
-            if msg_loop_cs is not None:
+            if msg_loop_cs:
                 msg_loop_cs.cancel()
 
-            tn.cancel_scope.cancel()
+            nursery.cancel_scope.cancel()

tractor/_root.py

@@ -18,25 +18,22 @@
 Root actor runtime ignition(s).
 
 '''
-from contextlib import asynccontextmanager as acm
+from contextlib import asynccontextmanager
 from functools import partial
 import importlib
-import inspect
 import logging
-import os
 import signal
 import sys
-from typing import Callable
+import os
 import warnings
 
 
+from exceptiongroup import BaseExceptionGroup
 import trio
 
 from ._runtime import (
     Actor,
     Arbiter,
-    # TODO: rename and make a non-actor subtype?
-    # Arbiter as Registry,
     async_main,
 )
 from .devx import _debug
@@ -61,118 +58,57 @@ _default_lo_addrs: list[tuple[str, int]] = [(
 logger = log.get_logger('tractor')
 
 
-@acm
+@asynccontextmanager
 async def open_root_actor(
 
     *,
     # defaults are above
-    registry_addrs: list[tuple[str, int]]|None = None,
+    registry_addrs: list[tuple[str, int]] | None = None,
 
     # defaults are above
-    arbiter_addr: tuple[str, int]|None = None,
+    arbiter_addr: tuple[str, int] | None = None,
 
-    name: str|None = 'root',
+    name: str | None = 'root',
 
     # either the `multiprocessing` start method:
     # https://docs.python.org/3/library/multiprocessing.html#contexts-and-start-methods
     # OR `trio` (the new default).
-    start_method: _spawn.SpawnMethodKey|None = None,
+    start_method: _spawn.SpawnMethodKey | None = None,
 
     # enables the multi-process debugger support
     debug_mode: bool = False,
-    maybe_enable_greenback: bool = True,  # `.pause_from_sync()/breakpoint()` support
-    enable_stack_on_sig: bool = False,
 
     # internal logging
-    loglevel: str|None = None,
+    loglevel: str | None = None,
 
-    enable_modules: list|None = None,
-    rpc_module_paths: list|None = None,
+    enable_modules: list | None = None,
+    rpc_module_paths: list | None = None,
 
     # NOTE: allow caller to ensure that only one registry exists
     # and that this call creates it.
     ensure_registry: bool = False,
 
-    hide_tb: bool = True,
-
-    # XXX, proxied directly to `.devx._debug._maybe_enter_pm()`
-    # for REPL-entry logic.
-    debug_filter: Callable[
-        [BaseException|BaseExceptionGroup],
-        bool,
-    ] = lambda err: not is_multi_cancelled(err),
-
-    # TODO, a way for actors to augment passing derived
-    # read-only state to sublayers?
-    # extra_rt_vars: dict|None = None,
-
 ) -> Actor:
     '''
     Runtime init entry point for ``tractor``.
 
     '''
-    _debug.hide_runtime_frames()
-    __tracebackhide__: bool = hide_tb
-
-    # TODO: stick this in a `@cm` defined in `devx._debug`?
-    #
     # Override the global debugger hook to make it play nice with
     # ``trio``, see much discussion in:
     # https://github.com/python-trio/trio/issues/1155#issuecomment-742964018
-    builtin_bp_handler: Callable = sys.breakpointhook
-    orig_bp_path: str|None = os.environ.get(
-        'PYTHONBREAKPOINT',
-        None,
-    )
-    if (
-        debug_mode
-        and maybe_enable_greenback
-        and (
-            maybe_mod := await _debug.maybe_init_greenback(
-                raise_not_found=False,
-            )
-        )
-    ):
-        logger.info(
-            f'Found `greenback` installed @ {maybe_mod}\n'
-            'Enabling `tractor.pause_from_sync()` support!\n'
-        )
-        os.environ['PYTHONBREAKPOINT'] = (
-            'tractor.devx._debug._sync_pause_from_builtin'
-        )
-        _state._runtime_vars['use_greenback'] = True
-
-    else:
-        # TODO: disable `breakpoint()` by default (without
-        # `greenback`) since it will break any multi-actor
-        # usage by a clobbered TTY's stdstreams!
-        def block_bps(*args, **kwargs):
-            raise RuntimeError(
-                'Trying to use `breakpoint()` eh?\n\n'
-                'Welp, `tractor` blocks `breakpoint()` built-in calls by default!\n'
-                'If you need to use it please install `greenback` and set '
-                '`debug_mode=True` when opening the runtime '
-                '(either via `.open_nursery()` or `open_root_actor()`)\n'
-            )
-
-        sys.breakpointhook = block_bps
-        # lol ok,
-        # https://docs.python.org/3/library/sys.html#sys.breakpointhook
-        os.environ['PYTHONBREAKPOINT'] = "0"
+    builtin_bp_handler = sys.breakpointhook
+    orig_bp_path: str | None = os.environ.get('PYTHONBREAKPOINT', None)
+    os.environ['PYTHONBREAKPOINT'] = 'tractor.devx._debug.pause_from_sync'
 
     # attempt to retreive ``trio``'s sigint handler and stash it
     # on our debugger lock state.
-    _debug.DebugStatus._trio_handler = signal.getsignal(signal.SIGINT)
+    _debug.Lock._trio_handler = signal.getsignal(signal.SIGINT)
 
     # mark top most level process as root actor
     _state._runtime_vars['_is_root'] = True
 
     # caps based rpc list
-    enable_modules = (
-        enable_modules
-        or
-        []
-    )
+    enable_modules = enable_modules or []
 
     if rpc_module_paths:
         warnings.warn(
@@ -228,7 +164,6 @@ async def open_root_actor(
         ):
             loglevel = 'PDB'
 
-
     elif debug_mode:
         raise RuntimeError(
             "Debug mode is only supported for the `trio` backend!"
@@ -239,13 +174,15 @@ async def open_root_actor(
     assert _log
 
     # TODO: factor this into `.devx._stackscope`!!
-    if (
-        debug_mode
-        and
-        enable_stack_on_sig
-    ):
-        from .devx._stackscope import enable_stack_on_sig
-        enable_stack_on_sig()
+    if debug_mode:
+        try:
+            logger.info('Enabling `stackscope` traces on SIGUSR1')
+            from .devx import enable_stack_on_sig
+            enable_stack_on_sig()
+        except ImportError:
+            logger.warning(
+                '`stackscope` not installed for use in debug mode!'
+            )
 
     # closed into below ping task-func
     ponged_addrs: list[tuple[str, int]] = []
@@ -276,9 +213,7 @@ async def open_root_actor(
 
         except OSError:
             # TODO: make this a "discovery" log level?
-            logger.info(
-                f'No actor registry found @ {addr}\n'
-            )
+            logger.warning(f'No actor registry found @ {addr}')
 
     async with trio.open_nursery() as tn:
         for addr in registry_addrs:
@@ -292,6 +227,7 @@ async def open_root_actor(
     # Create a new local root-actor instance which IS NOT THE
     # REGISTRAR
     if ponged_addrs:
+
         if ensure_registry:
             raise RuntimeError(
                  f'Failed to open `{name}`@{ponged_addrs}: '
@@ -341,10 +277,6 @@ async def open_root_actor(
             loglevel=loglevel,
             enable_modules=enable_modules,
         )
-        # XXX, in case the root actor runtime was actually run from
-        # `tractor.to_asyncio.run_as_asyncio_guest()` and NOt
-        # `.trio.run()`.
-        actor._infected_aio = _state._runtime_vars['_is_infected_aio']
 
     # Start up main task set via core actor-runtime nurseries.
     try:
@@ -362,10 +294,7 @@ async def open_root_actor(
         )
 
         # start the actor runtime in a new task
-        async with trio.open_nursery(
-            strict_exception_groups=False,
-            # ^XXX^ TODO? instead unpack any RAE as per "loose" style?
-        ) as nursery:
+        async with trio.open_nursery() as nursery:
 
             # ``_runtime.async_main()`` creates an internal nursery
             # and blocks here until any underlying actor(-process)
@@ -385,36 +314,19 @@ async def open_root_actor(
             )
             try:
                 yield actor
+
             except (
                 Exception,
                 BaseExceptionGroup,
             ) as err:
 
-                # TODO, in beginning to handle the subsubactor with
-                # crashed grandparent cases..
-                #
-                # was_locked: bool = await _debug.maybe_wait_for_debugger(
-                #     child_in_debug=True,
-                # )
-                # XXX NOTE XXX see equiv note inside
-                # `._runtime.Actor._stream_handler()` where in the
-                # non-root or root-that-opened-this-mahually case we
-                # wait for the local actor-nursery to exit before
-                # exiting the transport channel handler.
-                entered: bool = await _debug._maybe_enter_pm(
-                    err,
-                    api_frame=inspect.currentframe(),
-                    debug_filter=debug_filter,
-                )
+                entered: bool = await _debug._maybe_enter_pm(err)
 
                 if (
                     not entered
-                    and
-                    not is_multi_cancelled(
-                        err,
-                    )
+                    and not is_multi_cancelled(err)
                 ):
-                    logger.exception('Root actor crashed\n')
+                    logger.exception('Root actor crashed:\n')
 
                 # ALWAYS re-raise any error bubbled up from the
                 # runtime!
@@ -436,23 +348,14 @@ async def open_root_actor(
                 await actor.cancel(None)  # self cancel
     finally:
         _state._current_actor = None
-        _state._last_actor_terminated = actor
 
         # restore built-in `breakpoint()` hook state
-        if (
-            debug_mode
-            and
-            maybe_enable_greenback
-        ):
-            if builtin_bp_handler is not None:
-                sys.breakpointhook = builtin_bp_handler
-
-            if orig_bp_path is not None:
-                os.environ['PYTHONBREAKPOINT'] = orig_bp_path
-
-            else:
-                # clear env back to having no entry
-                os.environ.pop('PYTHONBREAKPOINT', None)
+        sys.breakpointhook = builtin_bp_handler
+        if orig_bp_path is not None:
+            os.environ['PYTHONBREAKPOINT'] = orig_bp_path
+        else:
+            # clear env back to having no entry
+            os.environ.pop('PYTHONBREAKPOINT')
 
         logger.runtime("Root actor terminated")
 
@@ -466,19 +369,12 @@ def run_daemon(
 
     start_method: str | None = None,
     debug_mode: bool = False,
-
-    # TODO, support `infected_aio=True` mode by,
-    # - calling the appropriate entrypoint-func from `.to_asyncio`
-    # - maybe init-ing `greenback` as done above in
-    #   `open_root_actor()`.
-
     **kwargs
 
 ) -> None:
     '''
-    Spawn a root (daemon) actor which will respond to RPC; the main
-    task simply starts the runtime and then blocks via embedded
-    `trio.sleep_forever()`.
+    Spawn daemon actor which will respond to RPC; the main task simply
+    starts the runtime and then sleeps forever.
 
     This is a very minimal convenience wrapper around starting
     a "run-until-cancelled" root actor which can be started with a set
@@ -491,6 +387,7 @@ def run_daemon(
         importlib.import_module(path)
 
     async def _main():
+
         async with open_root_actor(
             registry_addrs=registry_addrs,
             name=name,

tractor/_shm.py

@@ -0,0 +1,833 @@
+# tractor: structured concurrent "actors".
+# Copyright 2018-eternity Tyler Goodlet.
+
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU Affero General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU Affero General Public License for more details.
+
+# You should have received a copy of the GNU Affero General Public License
+# along with this program.  If not, see <https://www.gnu.org/licenses/>.
+
+"""
+SC friendly shared memory management geared at real-time
+processing.
+
+Support for ``numpy`` compatible array-buffers is provided but is
+considered optional within the context of this runtime-library.
+
+"""
+from __future__ import annotations
+from sys import byteorder
+import time
+from typing import Optional
+from multiprocessing import shared_memory as shm
+from multiprocessing.shared_memory import (
+    SharedMemory,
+    ShareableList,
+)
+
+from msgspec import Struct
+import tractor
+
+from .log import get_logger
+
+
+_USE_POSIX = getattr(shm, '_USE_POSIX', False)
+if _USE_POSIX:
+    from _posixshmem import shm_unlink
+
+
+try:
+    import numpy as np
+    from numpy.lib import recfunctions as rfn
+    import nptyping
+except ImportError:
+    pass
+
+
+log = get_logger(__name__)
+
+
+def disable_mantracker():
+    '''
+    Disable all ``multiprocessing``` "resource tracking" machinery since
+    it's an absolute multi-threaded mess of non-SC madness.
+
+    '''
+    from multiprocessing import resource_tracker as mantracker
+
+    # Tell the "resource tracker" thing to fuck off.
+    class ManTracker(mantracker.ResourceTracker):
+        def register(self, name, rtype):
+            pass
+
+        def unregister(self, name, rtype):
+            pass
+
+        def ensure_running(self):
+            pass
+
+    # "know your land and know your prey"
+    # https://www.dailymotion.com/video/x6ozzco
+    mantracker._resource_tracker = ManTracker()
+    mantracker.register = mantracker._resource_tracker.register
+    mantracker.ensure_running = mantracker._resource_tracker.ensure_running
+    mantracker.unregister = mantracker._resource_tracker.unregister
+    mantracker.getfd = mantracker._resource_tracker.getfd
+
+
+disable_mantracker()
+
+
+class SharedInt:
+    '''
+    Wrapper around a single entry shared memory array which
+    holds an ``int`` value used as an index counter.
+
+    '''
+    def __init__(
+        self,
+        shm: SharedMemory,
+    ) -> None:
+        self._shm = shm
+
+    @property
+    def value(self) -> int:
+        return int.from_bytes(self._shm.buf, byteorder)
+
+    @value.setter
+    def value(self, value) -> None:
+        self._shm.buf[:] = value.to_bytes(self._shm.size, byteorder)
+
+    def destroy(self) -> None:
+        if _USE_POSIX:
+            # We manually unlink to bypass all the "resource tracker"
+            # nonsense meant for non-SC systems.
+            name = self._shm.name
+            try:
+                shm_unlink(name)
+            except FileNotFoundError:
+                # might be a teardown race here?
+                log.warning(f'Shm for {name} already unlinked?')
+
+
+class NDToken(Struct, frozen=True):
+    '''
+    Internal represenation of a shared memory ``numpy`` array "token"
+    which can be used to key and load a system (OS) wide shm entry
+    and correctly read the array by type signature.
+
+    This type is msg safe.
+
+    '''
+    shm_name: str  # this servers as a "key" value
+    shm_first_index_name: str
+    shm_last_index_name: str
+    dtype_descr: tuple
+    size: int  # in struct-array index / row terms
+
+    # TODO: use nptyping here on dtypes
+    @property
+    def dtype(self) -> list[tuple[str, str, tuple[int, ...]]]:
+        return np.dtype(
+            list(
+                map(tuple, self.dtype_descr)
+            )
+        ).descr
+
+    def as_msg(self):
+        return self.to_dict()
+
+    @classmethod
+    def from_msg(cls, msg: dict) -> NDToken:
+        if isinstance(msg, NDToken):
+            return msg
+
+        # TODO: native struct decoding
+        # return _token_dec.decode(msg)
+
+        msg['dtype_descr'] = tuple(map(tuple, msg['dtype_descr']))
+        return NDToken(**msg)
+
+
+# _token_dec = msgspec.msgpack.Decoder(NDToken)
+
+# TODO: this api?
+# _known_tokens = tractor.ActorVar('_shm_tokens', {})
+# _known_tokens = tractor.ContextStack('_known_tokens', )
+# _known_tokens = trio.RunVar('shms', {})
+
+# TODO: this should maybe be provided via
+# a `.trionics.maybe_open_context()` wrapper factory?
+# process-local store of keys to tokens
+_known_tokens: dict[str, NDToken] = {}
+
+
+def get_shm_token(key: str) -> NDToken | None:
+    '''
+    Convenience func to check if a token
+    for the provided key is known by this process.
+
+    Returns either the ``numpy`` token or a string for a shared list.
+
+    '''
+    return _known_tokens.get(key)
+
+
+def _make_token(
+    key: str,
+    size: int,
+    dtype: np.dtype,
+
+) -> NDToken:
+    '''
+    Create a serializable token that can be used
+    to access a shared array.
+
+    '''
+    return NDToken(
+        shm_name=key,
+        shm_first_index_name=key + "_first",
+        shm_last_index_name=key + "_last",
+        dtype_descr=tuple(np.dtype(dtype).descr),
+        size=size,
+    )
+
+
+class ShmArray:
+    '''
+    A shared memory ``numpy.ndarray`` API.
+
+    An underlying shared memory buffer is allocated based on
+    a user specified ``numpy.ndarray``. This fixed size array
+    can be read and written to by pushing data both onto the "front"
+    or "back" of a set index range. The indexes for the "first" and
+    "last" index are themselves stored in shared memory (accessed via
+    ``SharedInt`` interfaces) values such that multiple processes can
+    interact with the same array using a synchronized-index.
+
+    '''
+    def __init__(
+        self,
+        shmarr: np.ndarray,
+        first: SharedInt,
+        last: SharedInt,
+        shm: SharedMemory,
+        # readonly: bool = True,
+    ) -> None:
+        self._array = shmarr
+
+        # indexes for first and last indices corresponding
+        # to fille data
+        self._first = first
+        self._last = last
+
+        self._len = len(shmarr)
+        self._shm = shm
+        self._post_init: bool = False
+
+        # pushing data does not write the index (aka primary key)
+        self._write_fields: list[str] | None = None
+        dtype = shmarr.dtype
+        if dtype.fields:
+            self._write_fields = list(shmarr.dtype.fields.keys())[1:]
+
+    # TODO: ringbuf api?
+
+    @property
+    def _token(self) -> NDToken:
+        return NDToken(
+            shm_name=self._shm.name,
+            shm_first_index_name=self._first._shm.name,
+            shm_last_index_name=self._last._shm.name,
+            dtype_descr=tuple(self._array.dtype.descr),
+            size=self._len,
+        )
+
+    @property
+    def token(self) -> dict:
+        """Shared memory token that can be serialized and used by
+        another process to attach to this array.
+        """
+        return self._token.as_msg()
+
+    @property
+    def index(self) -> int:
+        return self._last.value % self._len
+
+    @property
+    def array(self) -> np.ndarray:
+        '''
+        Return an up-to-date ``np.ndarray`` view of the
+        so-far-written data to the underlying shm buffer.
+
+        '''
+        a = self._array[self._first.value:self._last.value]
+
+        # first, last = self._first.value, self._last.value
+        # a = self._array[first:last]
+
+        # TODO: eventually comment this once we've not seen it in the
+        # wild in a long time..
+        # XXX: race where first/last indexes cause a reader
+        # to load an empty array..
+        if len(a) == 0 and self._post_init:
+            raise RuntimeError('Empty array race condition hit!?')
+            # breakpoint()
+
+        return a
+
+    def ustruct(
+        self,
+        fields: Optional[list[str]] = None,
+
+        # type that all field values will be cast to
+        # in the returned view.
+        common_dtype: np.dtype = float,
+
+    ) -> np.ndarray:
+
+        array = self._array
+
+        if fields:
+            selection = array[fields]
+            # fcount = len(fields)
+        else:
+            selection = array
+            # fcount = len(array.dtype.fields)
+
+        # XXX: manual ``.view()`` attempt that also doesn't work.
+        # uview = selection.view(
+        #     dtype='<f16',
+        # ).reshape(-1, 4, order='A')
+
+        # assert len(selection) == len(uview)
+
+        u = rfn.structured_to_unstructured(
+            selection,
+            # dtype=float,
+            copy=True,
+        )
+
+        # unstruct = np.ndarray(u.shape, dtype=a.dtype, buffer=shm.buf)
+        # array[:] = a[:]
+        return u
+        # return ShmArray(
+        #     shmarr=u,
+        #     first=self._first,
+        #     last=self._last,
+        #     shm=self._shm
+        # )
+
+    def last(
+        self,
+        length: int = 1,
+
+    ) -> np.ndarray:
+        '''
+        Return the last ``length``'s worth of ("row") entries from the
+        array.
+
+        '''
+        return self.array[-length:]
+
+    def push(
+        self,
+        data: np.ndarray,
+
+        field_map: Optional[dict[str, str]] = None,
+        prepend: bool = False,
+        update_first: bool = True,
+        start: int | None = None,
+
+    ) -> int:
+        '''
+        Ring buffer like "push" to append data
+        into the buffer and return updated "last" index.
+
+        NB: no actual ring logic yet to give a "loop around" on overflow
+        condition, lel.
+
+        '''
+        length = len(data)
+
+        if prepend:
+            index = (start or self._first.value) - length
+
+            if index < 0:
+                raise ValueError(
+                    f'Array size of {self._len} was overrun during prepend.\n'
+                    f'You have passed {abs(index)} too many datums.'
+                )
+
+        else:
+            index = start if start is not None else self._last.value
+
+        end = index + length
+
+        if field_map:
+            src_names, dst_names = zip(*field_map.items())
+        else:
+            dst_names = src_names = self._write_fields
+
+        try:
+            self._array[
+                list(dst_names)
+            ][index:end] = data[list(src_names)][:]
+
+            # NOTE: there was a race here between updating
+            # the first and last indices and when the next reader
+            # tries to access ``.array`` (which due to the index
+            # overlap will be empty). Pretty sure we've fixed it now
+            # but leaving this here as a reminder.
+            if (
+                prepend
+                and update_first
+                and length
+            ):
+                assert index < self._first.value
+
+            if (
+                index < self._first.value
+                and update_first
+            ):
+                assert prepend, 'prepend=True not passed but index decreased?'
+                self._first.value = index
+
+            elif not prepend:
+                self._last.value = end
+
+            self._post_init = True
+            return end
+
+        except ValueError as err:
+            if field_map:
+                raise
+
+            # should raise if diff detected
+            self.diff_err_fields(data)
+            raise err
+
+    def diff_err_fields(
+        self,
+        data: np.ndarray,
+    ) -> None:
+        # reraise with any field discrepancy
+        our_fields, their_fields = (
+            set(self._array.dtype.fields),
+            set(data.dtype.fields),
+        )
+
+        only_in_ours = our_fields - their_fields
+        only_in_theirs = their_fields - our_fields
+
+        if only_in_ours:
+            raise TypeError(
+                f"Input array is missing field(s): {only_in_ours}"
+            )
+        elif only_in_theirs:
+            raise TypeError(
+                f"Input array has unknown field(s): {only_in_theirs}"
+            )
+
+    # TODO: support "silent" prepends that don't update ._first.value?
+    def prepend(
+        self,
+        data: np.ndarray,
+    ) -> int:
+        end = self.push(data, prepend=True)
+        assert end
+
+    def close(self) -> None:
+        self._first._shm.close()
+        self._last._shm.close()
+        self._shm.close()
+
+    def destroy(self) -> None:
+        if _USE_POSIX:
+            # We manually unlink to bypass all the "resource tracker"
+            # nonsense meant for non-SC systems.
+            shm_unlink(self._shm.name)
+
+        self._first.destroy()
+        self._last.destroy()
+
+    def flush(self) -> None:
+        # TODO: flush to storage backend like markestore?
+        ...
+
+
+def open_shm_ndarray(
+    size: int,
+    key: str | None = None,
+    dtype: np.dtype | None = None,
+    append_start_index: int | None = None,
+    readonly: bool = False,
+
+) -> ShmArray:
+    '''
+    Open a memory shared ``numpy`` using the standard library.
+
+    This call unlinks (aka permanently destroys) the buffer on teardown
+    and thus should be used from the parent-most accessor (process).
+
+    '''
+    # create new shared mem segment for which we
+    # have write permission
+    a = np.zeros(size, dtype=dtype)
+    a['index'] = np.arange(len(a))
+
+    shm = SharedMemory(
+        name=key,
+        create=True,
+        size=a.nbytes
+    )
+    array = np.ndarray(
+        a.shape,
+        dtype=a.dtype,
+        buffer=shm.buf
+    )
+    array[:] = a[:]
+    array.setflags(write=int(not readonly))
+
+    token = _make_token(
+        key=key,
+        size=size,
+        dtype=dtype,
+    )
+
+    # create single entry arrays for storing an first and last indices
+    first = SharedInt(
+        shm=SharedMemory(
+            name=token.shm_first_index_name,
+            create=True,
+            size=4,  # std int
+        )
+    )
+
+    last = SharedInt(
+        shm=SharedMemory(
+            name=token.shm_last_index_name,
+            create=True,
+            size=4,  # std int
+        )
+    )
+
+    # Start the "real-time" append-updated (or "pushed-to") section
+    # after some start index: ``append_start_index``. This allows appending
+    # from a start point in the array which isn't the 0 index and looks
+    # something like,
+    # -------------------------
+    # |              |        i
+    # _________________________
+    # <-------------> <------->
+    #  history         real-time
+    #
+    # Once fully "prepended", the history section will leave the
+    # ``ShmArray._start.value: int = 0`` and the yet-to-be written
+    # real-time section will start at ``ShmArray.index: int``.
+
+    # this sets the index to nearly 2/3rds into the the length of
+    # the buffer leaving at least a "days worth of second samples"
+    # for the real-time section.
+    if append_start_index is None:
+        append_start_index = round(size * 0.616)
+
+    last.value = first.value = append_start_index
+
+    shmarr = ShmArray(
+        array,
+        first,
+        last,
+        shm,
+    )
+
+    assert shmarr._token == token
+    _known_tokens[key] = shmarr.token
+
+    # "unlink" created shm on process teardown by
+    # pushing teardown calls onto actor context stack
+    stack = tractor.current_actor().lifetime_stack
+    stack.callback(shmarr.close)
+    stack.callback(shmarr.destroy)
+
+    return shmarr
+
+
+def attach_shm_ndarray(
+    token: tuple[str, str, tuple[str, str]],
+    readonly: bool = True,
+
+) -> ShmArray:
+    '''
+    Attach to an existing shared memory array previously
+    created by another process using ``open_shared_array``.
+
+    No new shared mem is allocated but wrapper types for read/write
+    access are constructed.
+
+    '''
+    token = NDToken.from_msg(token)
+    key = token.shm_name
+
+    if key in _known_tokens:
+        assert NDToken.from_msg(_known_tokens[key]) == token, "WTF"
+
+    # XXX: ugh, looks like due to the ``shm_open()`` C api we can't
+    # actually place files in a subdir, see discussion here:
+    # https://stackoverflow.com/a/11103289
+
+    # attach to array buffer and view as per dtype
+    _err: Optional[Exception] = None
+    for _ in range(3):
+        try:
+            shm = SharedMemory(
+                name=key,
+                create=False,
+            )
+            break
+        except OSError as oserr:
+            _err = oserr
+            time.sleep(0.1)
+    else:
+        if _err:
+            raise _err
+
+    shmarr = np.ndarray(
+        (token.size,),
+        dtype=token.dtype,
+        buffer=shm.buf
+    )
+    shmarr.setflags(write=int(not readonly))
+
+    first = SharedInt(
+        shm=SharedMemory(
+            name=token.shm_first_index_name,
+            create=False,
+            size=4,  # std int
+        ),
+    )
+    last = SharedInt(
+        shm=SharedMemory(
+            name=token.shm_last_index_name,
+            create=False,
+            size=4,  # std int
+        ),
+    )
+
+    # make sure we can read
+    first.value
+
+    sha = ShmArray(
+        shmarr,
+        first,
+        last,
+        shm,
+    )
+    # read test
+    sha.array
+
+    # Stash key -> token knowledge for future queries
+    # via `maybe_opepn_shm_array()` but only after we know
+    # we can attach.
+    if key not in _known_tokens:
+        _known_tokens[key] = token
+
+    # "close" attached shm on actor teardown
+    tractor.current_actor().lifetime_stack.callback(sha.close)
+
+    return sha
+
+
+def maybe_open_shm_ndarray(
+    key: str,  # unique identifier for segment
+    size: int,
+    dtype: np.dtype | None = None,
+    append_start_index: int = 0,
+    readonly: bool = True,
+
+) -> tuple[ShmArray, bool]:
+    '''
+    Attempt to attach to a shared memory block using a "key" lookup
+    to registered blocks in the users overall "system" registry
+    (presumes you don't have the block's explicit token).
+
+    This function is meant to solve the problem of discovering whether
+    a shared array token has been allocated or discovered by the actor
+    running in **this** process. Systems where multiple actors may seek
+    to access a common block can use this function to attempt to acquire
+    a token as discovered by the actors who have previously stored
+    a "key" -> ``NDToken`` map in an actor local (aka python global)
+    variable.
+
+    If you know the explicit ``NDToken`` for your memory segment instead
+    use ``attach_shm_array``.
+
+    '''
+    try:
+        # see if we already know this key
+        token = _known_tokens[key]
+        return (
+            attach_shm_ndarray(
+                token=token,
+                readonly=readonly,
+            ),
+            False,  # not newly opened
+        )
+    except KeyError:
+        log.warning(f"Could not find {key} in shms cache")
+        if dtype:
+            token = _make_token(
+                key,
+                size=size,
+                dtype=dtype,
+            )
+        else:
+
+            try:
+                return (
+                    attach_shm_ndarray(
+                        token=token,
+                        readonly=readonly,
+                    ),
+                    False,
+                )
+            except FileNotFoundError:
+                log.warning(f"Could not attach to shm with token {token}")
+
+        # This actor does not know about memory
+        # associated with the provided "key".
+        # Attempt to open a block and expect
+        # to fail if a block has been allocated
+        # on the OS by someone else.
+        return (
+            open_shm_ndarray(
+                key=key,
+                size=size,
+                dtype=dtype,
+                append_start_index=append_start_index,
+                readonly=readonly,
+            ),
+            True,
+        )
+
+
+class ShmList(ShareableList):
+    '''
+    Carbon copy of ``.shared_memory.ShareableList`` with a few
+    enhancements:
+
+    - readonly mode via instance var flag  `._readonly: bool`
+    - ``.__getitem__()`` accepts ``slice`` inputs
+    - exposes the underlying buffer "name" as a ``.key: str``
+
+    '''
+    def __init__(
+        self,
+        sequence: list | None = None,
+        *,
+        name: str | None = None,
+        readonly: bool = True
+
+    ) -> None:
+        self._readonly = readonly
+        self._key = name
+        return super().__init__(
+            sequence=sequence,
+            name=name,
+        )
+
+    @property
+    def key(self) -> str:
+        return self._key
+
+    @property
+    def readonly(self) -> bool:
+        return self._readonly
+
+    def __setitem__(
+        self,
+        position,
+        value,
+
+    ) -> None:
+
+        # mimick ``numpy`` error
+        if self._readonly:
+            raise ValueError('assignment destination is read-only')
+
+        return super().__setitem__(position, value)
+
+    def __getitem__(
+        self,
+        indexish,
+    ) -> list:
+
+        # NOTE: this is a non-writeable view (copy?) of the buffer
+        # in a new list instance.
+        if isinstance(indexish, slice):
+            return list(self)[indexish]
+
+        return super().__getitem__(indexish)
+
+    # TODO: should we offer a `.array` and `.push()` equivalent
+    # to the `ShmArray`?
+    # currently we have the following limitations:
+    # - can't write slices of input using traditional slice-assign
+    #   syntax due to the ``ShareableList.__setitem__()`` implementation.
+    # - ``list(shmlist)`` returns a non-mutable copy instead of
+    #   a writeable view which would be handier numpy-style ops.
+
+
+def open_shm_list(
+    key: str,
+    sequence: list | None = None,
+    size: int = int(2 ** 10),
+    dtype: float | int | bool | str | bytes | None = float,
+    readonly: bool = True,
+
+) -> ShmList:
+
+    if sequence is None:
+        default = {
+            float: 0.,
+            int: 0,
+            bool: True,
+            str: 'doggy',
+            None: None,
+        }[dtype]
+        sequence = [default] * size
+
+    shml = ShmList(
+        sequence=sequence,
+        name=key,
+        readonly=readonly,
+    )
+
+    # "close" attached shm on actor teardown
+    try:
+        actor = tractor.current_actor()
+        actor.lifetime_stack.callback(shml.shm.close)
+        actor.lifetime_stack.callback(shml.shm.unlink)
+    except RuntimeError:
+        log.warning('tractor runtime not active, skipping teardown steps')
+
+    return shml
+
+
+def attach_shm_list(
+    key: str,
+    readonly: bool = False,
+
+) -> ShmList:
+
+    return ShmList(
+        name=key,
+        readonly=readonly,
+    )

tractor/_spawn.py

@@ -31,28 +31,25 @@ from typing import (
     TYPE_CHECKING,
 )
 
+from exceptiongroup import BaseExceptionGroup
 import trio
-from trio import TaskStatus
+from trio_typing import TaskStatus
 
-from .devx._debug import (
+from .devx import (
     maybe_wait_for_debugger,
     acquire_debug_lock,
 )
-from tractor._state import (
+from ._state import (
     current_actor,
     is_main_process,
     is_root_process,
     debug_mode,
-    _runtime_vars,
-)
-from tractor.log import get_logger
-from tractor._portal import Portal
-from tractor._runtime import Actor
-from tractor._entry import _mp_main
-from tractor._exceptions import ActorFailure
-from tractor.msg.types import (
-    SpawnSpec,
 )
+from .log import get_logger
+from ._portal import Portal
+from ._runtime import Actor
+from ._entry import _mp_main
+from ._exceptions import ActorFailure
 
 
 if TYPE_CHECKING:
@@ -143,13 +140,11 @@ async def exhaust_portal(
     '''
     __tracebackhide__ = True
     try:
-        log.debug(
-            f'Waiting on final result from {actor.uid}'
-        )
+        log.debug(f"Waiting on final result from {actor.uid}")
 
         # XXX: streams should never be reaped here since they should
         # always be established and shutdown using a context manager api
-        final: Any = await portal.wait_for_result()
+        final: Any = await portal.result()
 
     except (
         Exception,
@@ -198,10 +193,7 @@ async def cancel_on_completion(
     # if this call errors we store the exception for later
     # in ``errors`` which will be reraised inside
     # an exception group and we still send out a cancel request
-    result: Any|Exception = await exhaust_portal(
-        portal,
-        actor,
-    )
+    result: Any|Exception = await exhaust_portal(portal, actor)
     if isinstance(result, Exception):
         errors[actor.uid]: Exception = result
         log.cancel(
@@ -223,11 +215,7 @@ async def cancel_on_completion(
 
 async def hard_kill(
     proc: trio.Process,
-
-    terminate_after: int = 1.6,
-    # NOTE: for mucking with `.pause()`-ing inside the runtime
-    # whilst also hacking on it XD
-    # terminate_after: int = 99999,
+    terminate_after: int = 3,
 
     # NOTE: for mucking with `.pause()`-ing inside the runtime
     # whilst also hacking on it XD
@@ -250,9 +238,8 @@ async def hard_kill(
 
     '''
     log.cancel(
-        'Terminating sub-proc\n'
-        f'>x)\n'
-        f' |_{proc}\n'
+        'Terminating sub-proc:\n'
+        f'|_{proc}\n'
     )
     # NOTE: this timeout used to do nothing since we were shielding
     # the ``.wait()`` inside ``new_proc()`` which will pretty much
@@ -294,17 +281,15 @@ async def hard_kill(
     # zombies (as a feature) we ask the OS to do send in the
     # removal swad as the last resort.
     if cs.cancelled_caught:
-        # TODO: toss in the skynet-logo face as ascii art?
         log.critical(
-            # 'Well, the #ZOMBIE_LORD_IS_HERE# to collect\n'
-            '#T-800 deployed to collect zombie B0\n'
-            f'>x)\n'
-            f' |_{proc}\n'
+            'Well, the #ZOMBIE_LORD_IS_HERE# to collect\n'
+            f'|_{proc}\n'
         )
         proc.kill()
 
 
 async def soft_kill(
+
     proc: ProcessType,
     wait_func: Callable[
         [ProcessType],
@@ -327,27 +312,13 @@ async def soft_kill(
     uid: tuple[str, str] = portal.channel.uid
     try:
         log.cancel(
-            f'Soft killing sub-actor via portal request\n'
-            f'\n'
-            f'(c=> {portal.chan.uid}\n'
-            f'  |_{proc}\n'
+            'Soft killing sub-actor via `Portal.cancel_actor()`\n'
+            f'|_{proc}\n'
         )
         # wait on sub-proc to signal termination
         await wait_func(proc)
 
     except trio.Cancelled:
-        with trio.CancelScope(shield=True):
-            await maybe_wait_for_debugger(
-                child_in_debug=_runtime_vars.get(
-                    '_debug_mode', False
-                ),
-                header_msg=(
-                    'Delaying `soft_kill()` subproc reaper while debugger locked..\n'
-                ),
-                # TODO: need a diff value then default?
-                # poll_steps=9999999,
-            )
-
         # if cancelled during a soft wait, cancel the child
         # actor before entering the hard reap sequence
         # below. This means we try to do a graceful teardown
@@ -475,9 +446,10 @@ async def trio_proc(
     proc: trio.Process|None = None
     try:
         try:
-            proc: trio.Process = await trio.lowlevel.open_process(spawn_cmd)
+            # TODO: needs ``trio_typing`` patch?
+            proc = await trio.lowlevel.open_process(spawn_cmd)
             log.runtime(
-                'Started new child\n'
+                'Started new sub-proc\n'
                 f'|_{proc}\n'
             )
 
@@ -515,20 +487,17 @@ async def trio_proc(
             portal,
         )
 
-        # send a "spawning specification" which configures the
-        # initial runtime state of the child.
-        await chan.send(
-            SpawnSpec(
-                _parent_main_data=subactor._parent_main_data,
-                enable_modules=subactor.enable_modules,
-                reg_addrs=subactor.reg_addrs,
-                bind_addrs=bind_addrs,
-                _runtime_vars=_runtime_vars,
-            )
-        )
+        # send additional init params
+        await chan.send({
+            '_parent_main_data': subactor._parent_main_data,
+            'enable_modules': subactor.enable_modules,
+            'reg_addrs': subactor.reg_addrs,
+            'bind_addrs': bind_addrs,
+            '_runtime_vars': _runtime_vars,
+        })
 
         # track subactor in current nursery
-        curr_actor: Actor = current_actor()
+        curr_actor = current_actor()
         curr_actor._actoruid2nursery[subactor.uid] = actor_nursery
 
         # resume caller at next checkpoint now that child is up
@@ -559,9 +528,8 @@ async def trio_proc(
             # cancel result waiter that may have been spawned in
             # tandem if not done already
             log.cancel(
-                'Cancelling portal result reaper task\n'
-                f'>c)\n'
-                f' |_{subactor.uid}\n'
+                'Cancelling existing result waiter task for '
+                f'{subactor.uid}'
             )
             nursery.cancel_scope.cancel()
 
@@ -570,13 +538,9 @@ async def trio_proc(
         # allowed! Do this **after** cancellation/teardown to avoid
         # killing the process too early.
         if proc:
-            log.cancel(
-                f'Hard reap sequence starting for subactor\n'
-                f'>x)\n'
-                f' |_{subactor}@{subactor.uid}\n'
-            )
-
+            log.cancel(f'Hard reap sequence starting for {subactor.uid}')
             with trio.CancelScope(shield=True):
+
                 # don't clobber an ongoing pdb
                 if cancelled_during_spawn:
                     # Try again to avoid TTY clobbering.

tractor/_state.py

@@ -18,83 +18,27 @@
 Per process state
 
 """
-from __future__ import annotations
-from contextvars import (
-    ContextVar,
-)
 from typing import (
+    Optional,
     Any,
-    TYPE_CHECKING,
 )
 
-from trio.lowlevel import current_task
-
-if TYPE_CHECKING:
-    from ._runtime import Actor
-    from ._context import Context
-
-
-_current_actor: Actor|None = None  # type: ignore # noqa
-_last_actor_terminated: Actor|None = None
-
-# TODO: mk this a `msgspec.Struct`!
+_current_actor: Optional['Actor'] = None  # type: ignore # noqa
 _runtime_vars: dict[str, Any] = {
     '_debug_mode': False,
     '_is_root': False,
-    '_root_mailbox': (None, None),
-    '_registry_addrs': [],
-
-    '_is_infected_aio': False,
-
-    # for `tractor.pause_from_sync()` & `breakpoint()` support
-    'use_greenback': False,
+    '_root_mailbox': (None, None)
 }
 
 
-def last_actor() -> Actor|None:
-    '''
-    Try to return last active `Actor` singleton
-    for this process.
-
-    For case where runtime already exited but someone is asking
-    about the "last" actor probably to get its `.uid: tuple`.
-
-    '''
-    return _last_actor_terminated
-
-
-def current_actor(
-    err_on_no_runtime: bool = True,
-) -> Actor:
+def current_actor(err_on_no_runtime: bool = True) -> 'Actor':  # type: ignore # noqa
     '''
     Get the process-local actor instance.
 
     '''
-    if (
-        err_on_no_runtime
-        and
-        _current_actor is None
-    ):
-        msg: str = 'No local actor has been initialized yet?\n'
-        from ._exceptions import NoRuntime
-
-        if last := last_actor():
-            msg += (
-                f'Apparently the lact active actor was\n'
-                f'|_{last}\n'
-                f'|_{last.uid}\n'
-            )
-        # no actor runtime has (as of yet) ever been started for
-        # this process.
-        else:
-            msg += (
-                # 'No last actor found?\n'
-                '\nDid you forget to call one of,\n'
-                '- `tractor.open_root_actor()`\n'
-                '- `tractor.open_nursery()`\n'
-            )
-
-        raise NoRuntime(msg)
+    from ._exceptions import NoRuntime
+    if _current_actor is None and err_on_no_runtime:
+        raise NoRuntime("No local actor has been initialized yet")
 
     return _current_actor
 
@@ -108,7 +52,6 @@ def is_main_process() -> bool:
     return mp.current_process().name == 'MainProcess'
 
 
-# TODO, more verby name?
 def debug_mode() -> bool:
     '''
     Bool determining if "debug mode" is on which enables
@@ -120,26 +63,3 @@ def debug_mode() -> bool:
 
 def is_root_process() -> bool:
     return _runtime_vars['_is_root']
-
-
-_ctxvar_Context: ContextVar[Context] = ContextVar(
-    'ipc_context',
-    default=None,
-)
-
-
-def current_ipc_ctx(
-    error_on_not_set: bool = False,
-) -> Context|None:
-    ctx: Context = _ctxvar_Context.get()
-
-    if (
-        not ctx
-        and error_on_not_set
-    ):
-        from ._exceptions import InternalError
-        raise InternalError(
-            'No IPC context has been allocated for this task yet?\n'
-            f'|_{current_task()}\n'
-        )
-    return ctx

tractor/_streaming.py

@@ -26,7 +26,6 @@ import inspect
 from pprint import pformat
 from typing import (
     Any,
-    AsyncGenerator,
     Callable,
     AsyncIterator,
     TYPE_CHECKING,
@@ -36,27 +35,17 @@ import warnings
 import trio
 
 from ._exceptions import (
+    _raise_from_no_key_in_msg,
     ContextCancelled,
-    RemoteActorError,
 )
 from .log import get_logger
 from .trionics import (
     broadcast_receiver,
     BroadcastReceiver,
 )
-from tractor.msg import (
-    Error,
-    Return,
-    Stop,
-    MsgType,
-    PayloadT,
-    Yield,
-)
 
 if TYPE_CHECKING:
-    from ._runtime import Actor
     from ._context import Context
-    from ._ipc import Channel
 
 
 log = get_logger(__name__)
@@ -70,9 +59,10 @@ log = get_logger(__name__)
 class MsgStream(trio.abc.Channel):
     '''
     A bidirectional message stream for receiving logically sequenced
-    values over an inter-actor IPC `Channel`.
-
+    values over an inter-actor IPC ``Channel``.
 
+    This is the type returned to a local task which entered either
+    ``Portal.open_stream_from()`` or ``Context.open_stream()``.
 
     Termination rules:
 
@@ -88,128 +78,50 @@ class MsgStream(trio.abc.Channel):
         self,
         ctx: Context,  # typing: ignore # noqa
         rx_chan: trio.MemoryReceiveChannel,
-        _broadcaster: BroadcastReceiver|None = None,
+        _broadcaster: BroadcastReceiver | None = None,
 
     ) -> None:
         self._ctx = ctx
         self._rx_chan = rx_chan
         self._broadcaster = _broadcaster
 
-        # any actual IPC msg which is effectively an `EndOfStream`
-        self._stop_msg: bool|Stop = False
-
         # flag to denote end of stream
         self._eoc: bool|trio.EndOfChannel = False
         self._closed: bool|trio.ClosedResourceError = False
 
-    @property
-    def ctx(self) -> Context:
-        '''
-        A read-only ref to this stream's inter-actor-task `Context`.
-
-        '''
-        return self._ctx
-
-    @property
-    def chan(self) -> Channel:
-        '''
-        Ref to the containing `Context`'s transport `Channel`.
-
-        '''
-        return self._ctx.chan
-
-    # TODO: could we make this a direct method bind to `PldRx`?
-    # -> receive_nowait = PldRx.recv_pld
-    # |_ means latter would have to accept `MsgStream`-as-`self`?
-    #  => should be fine as long as,
-    #  -[ ] both define `._rx_chan`
-    #  -[ ] .ctx is bound into `PldRx` using a `@cm`?
-    #
     # delegate directly to underlying mem channel
-    def receive_nowait(
-        self,
-        expect_msg: MsgType = Yield,
-    ) -> PayloadT:
-        ctx: Context = self._ctx
-        (
-            msg,
-            pld,
-        ) = ctx._pld_rx.recv_msg_nowait(
-            ipc=self,
-            expect_msg=expect_msg,
-        )
+    def receive_nowait(self):
+        msg = self._rx_chan.receive_nowait()
+        try:
+            return msg['yield']
+        except KeyError as kerr:
+            _raise_from_no_key_in_msg(
+                ctx=self._ctx,
+                msg=msg,
+                src_err=kerr,
+                log=log,
+                expect_key='yield',
+                stream=self,
+            )
 
-        # ?TODO, maybe factor this into a hyper-common `unwrap_pld()`
-        #
-        match msg:
-
-            # XXX, these never seems to ever hit? cool?
-            case Stop():
-                log.cancel(
-                    f'Msg-stream was ended via stop msg\n'
-                    f'{msg}'
-                )
-            case Error():
-                log.error(
-                    f'Msg-stream was ended via error msg\n'
-                    f'{msg}'
-                )
-
-            # XXX NOTE, always set any final result on the ctx to
-            # avoid teardown race conditions where previously this msg
-            # would be consumed silently (by `.aclose()` doing its
-            # own "msg drain loop" but WITHOUT those `drained: lists[MsgType]`
-            # being post-close-processed!
-            #
-            # !!TODO, see the equiv todo-comment in `.receive()`
-            # around the `if drained:` where we should prolly
-            # ACTUALLY be doing this post-close processing??
-            #
-            case Return(pld=pld):
-                log.warning(
-                    f'Msg-stream final result msg for IPC ctx?\n'
-                    f'{msg}'
-                )
-                # XXX TODO, this **should be covered** by higher
-                # scoped runtime-side method calls such as
-                # `Context._deliver_msg()`, so you should never
-                # really see the warning above or else something
-                # racy/out-of-order is likely going on between
-                # actor-runtime-side push tasks and the user-app-side
-                # consume tasks!
-                # -[ ] figure out that set of race cases and fix!
-                # -[ ] possibly return the `msg` given an input
-                #     arg-flag is set so we can process the `Return`
-                #     from the `.aclose()` caller?
-                #
-                # breakpoint()  # to debug this RACE CASE!
-                ctx._result = pld
-                ctx._outcome_msg = msg
-
-        return pld
-
-    async def receive(
-        self,
-        hide_tb: bool = False,
-    ):
+    async def receive(self):
         '''
         Receive a single msg from the IPC transport, the next in
         sequence sent by the far end task (possibly in order as
         determined by the underlying protocol).
 
         '''
-        __tracebackhide__: bool = hide_tb
-
-        # NOTE FYI: `trio.ReceiveChannel` implements EOC handling as
-        # follows (aka uses it to gracefully exit async for loops):
+        # NOTE: `trio.ReceiveChannel` implements
+        # EOC handling as follows (aka uses it
+        # to gracefully exit async for loops):
         #
         # async def __anext__(self) -> ReceiveType:
         #     try:
         #         return await self.receive()
         #     except trio.EndOfChannel:
         #         raise StopAsyncIteration
-        #
-        # see `.aclose()` for notes on the old behaviour prior to
+
+        # see ``.aclose()`` for notes on the old behaviour prior to
         # introducing this
         if self._eoc:
             raise self._eoc
@@ -217,36 +129,73 @@ class MsgStream(trio.abc.Channel):
         if self._closed:
             raise self._closed
 
-        src_err: Exception|None = None  # orig tb
+        src_err: Exception|None = None
         try:
-            ctx: Context = self._ctx
-            pld = await ctx._pld_rx.recv_pld(
-                ipc=self,
-                expect_msg=Yield,
-            )
-            return pld
+            try:
+                msg = await self._rx_chan.receive()
+                return msg['yield']
 
-        # XXX: the stream terminates on either of:
-        # - `self._rx_chan.receive()` raising  after manual closure
-        #   by the rpc-runtime,
-        #   OR
-        # - via a `Stop`-msg received from remote peer task.
-        #   NOTE
-        #   |_ previously this was triggered by calling
-        #   `._rx_chan.aclose()` on the send side of the channel
-        #   inside `Actor._deliver_ctx_payload()`, but now the 'stop'
-        #   message handling gets delegated to `PldRFx.recv_pld()`
-        #   internals.
-        except trio.EndOfChannel as eoc:
-            # a graceful stream finished signal
-            self._eoc = eoc
+            except KeyError as kerr:
+                # log.exception('GOT KEYERROR')
+                src_err = kerr
+
+                # NOTE: may raise any of the below error types
+                # includg EoC when a 'stop' msg is found.
+                _raise_from_no_key_in_msg(
+                    ctx=self._ctx,
+                    msg=msg,
+                    src_err=kerr,
+                    log=log,
+                    expect_key='yield',
+                    stream=self,
+                )
+
+        # XXX: we close the stream on any of these error conditions:
+        except (
+            # trio.ClosedResourceError,  # by self._rx_chan
+            trio.EndOfChannel,  # by self._rx_chan or `stop` msg from far end
+        ) as eoc:
+            # log.exception('GOT EOC')
             src_err = eoc
+            self._eoc = eoc
 
-        # a `ClosedResourceError` indicates that the internal feeder
-        # memory receive channel was closed likely by the runtime
-        # after the associated transport-channel disconnected or
-        # broke.
-        except trio.ClosedResourceError as cre:  # by self._rx_chan.receive()
+            # a ``ClosedResourceError`` indicates that the internal
+            # feeder memory receive channel was closed likely by the
+            # runtime after the associated transport-channel
+            # disconnected or broke.
+
+            # an ``EndOfChannel`` indicates either the internal recv
+            # memchan exhausted **or** we raisesd it just above after
+            # receiving a `stop` message from the far end of the stream.
+
+            # Previously this was triggered by calling ``.aclose()`` on
+            # the send side of the channel inside
+            # ``Actor._push_result()`` (should still be commented code
+            # there - which should eventually get removed), but now the
+            # 'stop' message handling has been put just above.
+
+            # TODO: Locally, we want to close this stream gracefully, by
+            # terminating any local consumers tasks deterministically.
+            # One we have broadcast support, we **don't** want to be
+            # closing this stream and not flushing a final value to
+            # remaining (clone) consumers who may not have been
+            # scheduled to receive it yet.
+            # try:
+            #     maybe_err_msg_or_res: dict = self._rx_chan.receive_nowait()
+            #     if maybe_err_msg_or_res:
+            #         log.warning(
+            #             'Discarding un-processed msg:\n'
+            #             f'{maybe_err_msg_or_res}'
+            #         )
+            # except trio.WouldBlock:
+            #     # no queued msgs that might be another remote
+            #     # error, so just raise the original EoC
+            #     pass
+
+            # raise eoc
+
+        except trio.ClosedResourceError as cre:  # by self._rx_chan
+            # log.exception('GOT CRE')
             src_err = cre
             log.warning(
                 '`Context._rx_chan` was already closed?'
@@ -257,61 +206,28 @@ class MsgStream(trio.abc.Channel):
         # terminated and signal this local iterator to stop
         drained: list[Exception|dict] = await self.aclose()
         if drained:
-        #  ^^^^^^^^TODO? pass these to the `._ctx._drained_msgs:
-        #  deque` and then iterate them as part of any
-        #  `.wait_for_result()` call?
-        #
-        # -[ ] move the match-case processing from
-        #     `.receive_nowait()` instead to right here, use it from
-        #     a for msg in drained:` post-proc loop?
-        #
+            # from .devx import pause
+            # await pause()
             log.warning(
-                'Drained context msgs during closure\n\n'
+                'Drained context msgs during closure:\n'
                 f'{drained}'
             )
+        # TODO: pass these to the `._ctx._drained_msgs: deque`
+        # and then iterate them as part of any `.result()` call?
 
         # NOTE XXX: if the context was cancelled or remote-errored
         # but we received the stream close msg first, we
         # probably want to instead raise the remote error
         # over the end-of-stream connection error since likely
         # the remote error was the source cause?
-        # ctx: Context = self._ctx
-        ctx.maybe_raise(
-            raise_ctxc_from_self_call=True,
-            from_src_exc=src_err,
-        )
-
-        # propagate any error but hide low-level frame details from
-        # the caller by default for console/debug-REPL noise
-        # reduction.
-        if (
-            hide_tb
-            and (
-
-                # XXX NOTE special conditions: don't reraise on
-                # certain stream-specific internal error types like,
-                #
-                # - `trio.EoC` since we want to use the exact instance
-                #   to ensure that it is the error that bubbles upward
-                #   for silent absorption by `Context.open_stream()`.
-                not self._eoc
-
-                # - `RemoteActorError` (or subtypes like ctxc)
-                #    since we want to present the error as though it is
-                #    "sourced" directly from this `.receive()` call and
-                #    generally NOT include the stack frames raised from
-                #    inside the `PldRx` and/or the transport stack
-                #    layers.
-                or isinstance(src_err, RemoteActorError)
+        ctx: Context = self._ctx
+        if re := ctx._remote_error:
+            ctx._maybe_raise_remote_err(
+                re,
+                raise_ctxc_from_self_call=True,
             )
-        ):
-            raise type(src_err)(*src_err.args) from src_err
-        else:
-            # for any non-graceful-EOC we want to NOT hide this frame
-            if not self._eoc:
-                __tracebackhide__: bool = False
 
-            raise src_err
+        raise src_err  # propagate
 
     async def aclose(self) -> list[Exception|dict]:
         '''
@@ -327,6 +243,9 @@ class MsgStream(trio.abc.Channel):
          - more or less we try to maintain adherance to trio's `.aclose()` semantics:
            https://trio.readthedocs.io/en/stable/reference-io.html#trio.abc.AsyncResource.aclose
         '''
+
+        # rx_chan = self._rx_chan
+
         # XXX NOTE XXX
         # it's SUPER IMPORTANT that we ensure we don't DOUBLE
         # DRAIN msgs on closure so avoid getting stuck handing on
@@ -338,17 +257,13 @@ class MsgStream(trio.abc.Channel):
             # this stream has already been closed so silently succeed as
             # per ``trio.AsyncResource`` semantics.
             # https://trio.readthedocs.io/en/stable/reference-io.html#trio.abc.AsyncResource.aclose
-            # import tractor
-            # await tractor.pause()
             return []
 
         ctx: Context = self._ctx
         drained: list[Exception|dict] = []
         while not drained:
             try:
-                maybe_final_msg: Yield|Return = self.receive_nowait(
-                    expect_msg=Yield|Return,
-                )
+                maybe_final_msg = self.receive_nowait()
                 if maybe_final_msg:
                     log.debug(
                         'Drained un-processed stream msg:\n'
@@ -432,30 +347,13 @@ class MsgStream(trio.abc.Channel):
         #         await rx_chan.aclose()
 
         if not self._eoc:
-            this_side: str = self._ctx.side
-            peer_side: str = self._ctx.peer_side
-            message: str = (
-                f'Stream self-closed by {this_side!r}-side before EoC from {peer_side!r}\n'
-                # } bc a stream is a "scope"/msging-phase inside an IPC
-                f'x}}>\n'
-                f'  |_{self}\n'
+            self._eoc: bool = trio.EndOfChannel(
+                f'Context stream closed by {self._ctx.side}\n'
+                f'|_{self}\n'
             )
-            log.cancel(message)
-            self._eoc = trio.EndOfChannel(message)
-
-            if (
-                (rx_chan := self._rx_chan)
-                and
-                (stats := rx_chan.statistics()).tasks_waiting_receive
-            ):
-                log.cancel(
-                    f'Msg-stream is closing but there is still reader tasks,\n'
-                    f'{stats}\n'
-                )
-
         # ?XXX WAIT, why do we not close the local mem chan `._rx_chan` XXX?
         # => NO, DEFINITELY NOT! <=
-        # if we're a bi-dir `MsgStream` BECAUSE this same
+        # if we're a bi-dir ``MsgStream`` BECAUSE this same
         # core-msg-loop mem recv-chan is used to deliver the
         # potential final result from the surrounding inter-actor
         # `Context` so we don't want to close it until that
@@ -493,11 +391,13 @@ class MsgStream(trio.abc.Channel):
 
     @property
     def closed(self) -> bool:
-
-        rxc: bool = self._rx_chan._closed
-        _closed: bool|Exception = self._closed
-        _eoc: bool|trio.EndOfChannel = self._eoc
-        if rxc or _closed or _eoc:
+        if (
+            (rxc := self._rx_chan._closed)
+            or
+            (_closed := self._closed)
+            or
+            (_eoc := self._eoc)
+        ):
             log.runtime(
                 f'`MsgStream` is already closed\n'
                 f'{self}\n'
@@ -537,9 +437,6 @@ class MsgStream(trio.abc.Channel):
                 self,
                 # use memory channel size by default
                 self._rx_chan._state.max_buffer_size,  # type: ignore
-
-                # TODO: can remove this kwarg right since
-                # by default behaviour is to do this anyway?
                 receive_afunc=self.receive,
             )
 
@@ -566,260 +463,23 @@ class MsgStream(trio.abc.Channel):
 
     async def send(
         self,
-        data: Any,
-
-        hide_tb: bool = True,
+        data: Any
     ) -> None:
         '''
         Send a message over this stream to the far end.
 
         '''
-        __tracebackhide__: bool = hide_tb
-
-        # raise any alreay known error immediately
-        self._ctx.maybe_raise()
-        if self._eoc:
-            raise self._eoc
+        if self._ctx._remote_error:
+            raise self._ctx._remote_error  # from None
 
         if self._closed:
             raise self._closed
+            # raise trio.ClosedResourceError('This stream was already closed')
 
-        try:
-            await self._ctx.chan.send(
-                payload=Yield(
-                    cid=self._ctx.cid,
-                    pld=data,
-                ),
-            )
-        except (
-            trio.ClosedResourceError,
-            trio.BrokenResourceError,
-            BrokenPipeError,
-        ) as trans_err:
-            if hide_tb:
-                raise type(trans_err)(
-                    *trans_err.args
-                ) from trans_err
-            else:
-                raise
-
-    # TODO: msg capability context api1
-    # @acm
-    # async def enable_msg_caps(
-    #     self,
-    #     msg_subtypes: Union[
-    #         list[list[Struct]],
-    #         Protocol,   # hypothetical type that wraps a msg set
-    #     ],
-    # ) -> tuple[Callable, Callable]:  # payload enc, dec pair
-    #     ...
-
-
-@acm
-async def open_stream_from_ctx(
-    ctx: Context,
-    allow_overruns: bool|None = False,
-    msg_buffer_size: int|None = None,
-
-) -> AsyncGenerator[MsgStream, None]:
-    '''
-    Open a `MsgStream`, a bi-directional msg transport dialog
-    connected to the cross-actor peer task for an IPC `Context`.
-
-    This context manager must be entered in both the "parent" (task
-    which entered `Portal.open_context()`) and "child" (RPC task
-    which is decorated by `@context`) tasks for the stream to
-    logically be considered "open"; if one side begins sending to an
-    un-opened peer, depending on policy config, msgs will either be
-    queued until the other side opens and/or a `StreamOverrun` will
-    (eventually) be raised.
-
-                         ------ - ------
-
-    Runtime semantics design:
-
-    A `MsgStream` session adheres to "one-shot use" semantics,
-    meaning if you close the scope it **can not** be "re-opened".
-
-    Instead you must re-establish a new surrounding RPC `Context`
-    (RTC: remote task context?) using `Portal.open_context()`.
-
-    In the future this *design choice* may need to be changed but
-    currently there seems to be no obvious reason to support such
-    semantics..
-
-    - "pausing a stream" can be supported with a message implemented
-      by the `tractor` application dev.
-
-    - any remote error will normally require a restart of the entire
-      `trio.Task`'s scope due to the nature of `trio`'s cancellation
-      (`CancelScope`) system and semantics (level triggered).
-
-    '''
-    actor: Actor = ctx._actor
-
-    # If the surrounding context has been cancelled by some
-    # task with a handle to THIS, we error here immediately
-    # since it likely means the surrounding lexical-scope has
-    # errored, been `trio.Cancelled` or at the least
-    # `Context.cancel()` was called by some task.
-    if ctx._cancel_called:
-
-        # XXX NOTE: ALWAYS RAISE any remote error here even if
-        # it's an expected `ContextCancelled` due to a local
-        # task having called `.cancel()`!
-        #
-        # WHY: we expect the error to always bubble up to the
-        # surrounding `Portal.open_context()` call and be
-        # absorbed there (silently) and we DO NOT want to
-        # actually try to stream - a cancel msg was already
-        # sent to the other side!
-        ctx.maybe_raise(
-            raise_ctxc_from_self_call=True,
-        )
-        # NOTE: this is diff then calling
-        # `._maybe_raise_remote_err()` specifically
-        # because we want to raise a ctxc on any task entering this `.open_stream()`
-        # AFTER cancellation was already been requested,
-        # we DO NOT want to absorb any ctxc ACK silently!
-        # if ctx._remote_error:
-        #     raise ctx._remote_error
-
-        # XXX NOTE: if no `ContextCancelled` has been responded
-        # back from the other side (yet), we raise a different
-        # runtime error indicating that this task's usage of
-        # `Context.cancel()` and then `.open_stream()` is WRONG!
-        task: str = trio.lowlevel.current_task().name
-        raise RuntimeError(
-            'Stream opened after `Context.cancel()` called..?\n'
-            f'task: {actor.uid[0]}:{task}\n'
-            f'{ctx}'
-        )
-
-    if (
-        not ctx._portal
-        and not ctx._started_called
-    ):
-        raise RuntimeError(
-            'Context.started()` must be called before opening a stream'
-        )
-
-    # NOTE: in one way streaming this only happens on the
-    # parent-ctx-task side (on the side that calls
-    # `Actor.start_remote_task()`) so if you try to send
-    # a stop from the caller to the callee in the
-    # single-direction-stream case you'll get a lookup error
-    # currently.
-    ctx: Context = actor.get_context(
-        chan=ctx.chan,
-        cid=ctx.cid,
-        nsf=ctx._nsf,
-        # side=ctx.side,
-
-        msg_buffer_size=msg_buffer_size,
-        allow_overruns=allow_overruns,
-    )
-    ctx._allow_overruns: bool = allow_overruns
-    assert ctx is ctx
-
-    # XXX: If the underlying channel feeder receive mem chan has
-    # been closed then likely client code has already exited
-    # a ``.open_stream()`` block prior or there was some other
-    # unanticipated error or cancellation from ``trio``.
-
-    if ctx._rx_chan._closed:
-        raise trio.ClosedResourceError(
-            'The underlying channel for this stream was already closed!\n'
-        )
-
-    # NOTE: implicitly this will call `MsgStream.aclose()` on
-    # `.__aexit__()` due to stream's parent `Channel` type!
-    #
-    # XXX NOTE XXX: ensures the stream is "one-shot use",
-    # which specifically means that on exit,
-    # - signal ``trio.EndOfChannel``/``StopAsyncIteration`` to
-    #   the far end indicating that the caller exited
-    #   the streaming context purposefully by letting
-    #   the exit block exec.
-    # - this is diff from the cancel/error case where
-    #   a cancel request from this side or an error
-    #   should be sent to the far end indicating the
-    #   stream WAS NOT just closed normally/gracefully.
-    async with MsgStream(
-        ctx=ctx,
-        rx_chan=ctx._rx_chan,
-    ) as stream:
-
-        # NOTE: we track all existing streams per portal for
-        # the purposes of attempting graceful closes on runtime
-        # cancel requests.
-        if ctx._portal:
-            ctx._portal._streams.add(stream)
-
-        try:
-            ctx._stream_opened: bool = True
-            ctx._stream = stream
-
-            # XXX: do we need this?
-            # ensure we aren't cancelled before yielding the stream
-            # await trio.lowlevel.checkpoint()
-            yield stream
-
-            # XXX: (MEGA IMPORTANT) if this is a root opened process we
-            # wait for any immediate child in debug before popping the
-            # context from the runtime msg loop otherwise inside
-            # ``Actor._deliver_ctx_payload()`` the msg will be discarded and in
-            # the case where that msg is global debugger unlock (via
-            # a "stop" msg for a stream), this can result in a deadlock
-            # where the root is waiting on the lock to clear but the
-            # child has already cleared it and clobbered IPC.
-            #
-            # await maybe_wait_for_debugger()
-
-            # XXX TODO: pretty sure this isn't needed (see
-            # note above this block) AND will result in
-            # a double `.send_stop()` call. The only reason to
-            # put it here would be to due with "order" in
-            # terms of raising any remote error (as per
-            # directly below) or bc the stream's
-            # `.__aexit__()` block might not get run
-            # (doubtful)? Either way if we did put this back
-            # in we also need a state var to avoid the double
-            # stop-msg send..
-            #
-            # await stream.aclose()
-
-        # NOTE: absorb and do not raise any
-        # EoC received from the other side such that
-        # it is not raised inside the surrounding
-        # context block's scope!
-        except trio.EndOfChannel as eoc:
-            if (
-                eoc
-                and
-                stream.closed
-            ):
-                # sanity, can remove?
-                assert eoc is stream._eoc
-
-                log.warning(
-                    'Stream was terminated by EoC\n\n'
-                    # NOTE: won't show the error <Type> but
-                    # does show txt followed by IPC msg.
-                    f'{str(eoc)}\n'
-                )
-
-        finally:
-            if ctx._portal:
-                try:
-                    ctx._portal._streams.remove(stream)
-                except KeyError:
-                    log.warning(
-                        f'Stream was already destroyed?\n'
-                        f'actor: {ctx.chan.uid}\n'
-                        f'ctx id: {ctx.cid}'
-                    )
-
+        await self._ctx.chan.send({
+            'yield': data,
+            'cid': self._ctx.cid,
+        })
 
 
 def stream(func: Callable) -> Callable:
@@ -829,7 +489,7 @@ def stream(func: Callable) -> Callable:
     '''
     # TODO: apply whatever solution ``mypy`` ends up picking for this:
     # https://github.com/python/mypy/issues/2087#issuecomment-769266912
-    func._tractor_stream_function: bool = True  # type: ignore
+    func._tractor_stream_function = True  # type: ignore
 
     sig = inspect.signature(func)
     params = sig.parameters

tractor/_supervise.py

@@ -26,6 +26,7 @@ from typing import TYPE_CHECKING
 import typing
 import warnings
 
+from exceptiongroup import BaseExceptionGroup
 import trio
 
 from .devx._debug import maybe_wait_for_debugger
@@ -80,19 +81,15 @@ class ActorNursery:
     '''
     def __init__(
         self,
-        # TODO: maybe def these as fields of a struct looking type?
         actor: Actor,
         ria_nursery: trio.Nursery,
         da_nursery: trio.Nursery,
         errors: dict[tuple[str, str], BaseException],
-
     ) -> None:
         # self.supervisor = supervisor  # TODO
         self._actor: Actor = actor
-
-        # TODO: rename to `._tn` for our conventional "task-nursery"
+        self._ria_nursery = ria_nursery
         self._da_nursery = da_nursery
-
         self._children: dict[
             tuple[str, str],
             tuple[
@@ -101,12 +98,13 @@ class ActorNursery:
                 Portal | None,
             ]
         ] = {}
-
+        # portals spawned with ``run_in_actor()`` are
+        # cancelled when their "main" result arrives
+        self._cancel_after_result_on_exit: set = set()
         self.cancelled: bool = False
         self._join_procs = trio.Event()
         self._at_least_one_child_in_debug: bool = False
         self.errors = errors
-        self._scope_error: BaseException|None = None
         self.exited = trio.Event()
 
         # NOTE: when no explicit call is made to
@@ -117,48 +115,28 @@ class ActorNursery:
         # and syncing purposes to any actor opened nurseries.
         self._implicit_runtime_started: bool = False
 
-        # TODO: remove the `.run_in_actor()` API and thus this 2ndary
-        # nursery when that API get's moved outside this primitive!
-        self._ria_nursery = ria_nursery
-        # portals spawned with ``run_in_actor()`` are
-        # cancelled when their "main" result arrives
-        self._cancel_after_result_on_exit: set = set()
-
     async def start_actor(
         self,
         name: str,
-
         *,
-
         bind_addrs: list[tuple[str, int]] = [_default_bind_addr],
-        rpc_module_paths: list[str]|None = None,
-        enable_modules: list[str]|None = None,
-        loglevel: str|None = None,  # set log level per subactor
-        debug_mode: bool|None = None,
+        rpc_module_paths: list[str] | None = None,
+        enable_modules: list[str] | None = None,
+        loglevel: str | None = None,  # set log level per subactor
+        nursery: trio.Nursery | None = None,
+        debug_mode: bool | None = None,
         infect_asyncio: bool = False,
-
-        # TODO: ideally we can rm this once we no longer have
-        # a `._ria_nursery` since the dependent APIs have been
-        # removed!
-        nursery: trio.Nursery|None = None,
-
     ) -> Portal:
         '''
         Start a (daemon) actor: an process that has no designated
         "main task" besides the runtime.
 
         '''
-        __runtimeframe__: int = 1  # noqa
-        loglevel: str = (
-            loglevel
-            or self._actor.loglevel
-            or get_loglevel()
-        )
+        loglevel = loglevel or self._actor.loglevel or get_loglevel()
 
         # configure and pass runtime state
         _rtv = _state._runtime_vars.copy()
         _rtv['_is_root'] = False
-        _rtv['_is_infected_aio'] = infect_asyncio
 
         # allow setting debug policy per actor
         if debug_mode is not None:
@@ -207,14 +185,6 @@ class ActorNursery:
             )
         )
 
-    # TODO: DEPRECATE THIS:
-    # -[ ] impl instead as a hilevel wrapper on
-    #   top of a `@context` style invocation.
-    #  |_ dynamic @context decoration on child side
-    #  |_ implicit `Portal.open_context() as (ctx, first):`
-    #    and `return first` on parent side.
-    #  |_ mention how it's similar to `trio-parallel` API?
-    # -[ ] use @api_frame on the wrapper
     async def run_in_actor(
         self,
 
@@ -240,14 +210,13 @@ class ActorNursery:
         the actor is terminated.
 
         '''
-        __runtimeframe__: int = 1  # noqa
         mod_path: str = fn.__module__
 
         if name is None:
             # use the explicit function name if not provided
             name = fn.__name__
 
-        portal: Portal = await self.start_actor(
+        portal = await self.start_actor(
             name,
             enable_modules=[mod_path] + (
                 enable_modules or rpc_module_paths or []
@@ -276,24 +245,19 @@ class ActorNursery:
         )
         return portal
 
-    # @api_frame
     async def cancel(
         self,
         hard_kill: bool = False,
 
     ) -> None:
         '''
-        Cancel this actor-nursery by instructing each subactor's
-        runtime to cancel and wait for all underlying sub-processes
-        to terminate.
+        Cancel this nursery by instructing each subactor to cancel
+        itself and wait for all subactors to terminate.
 
-        If `hard_kill` is set then kill the processes directly using
-        the spawning-backend's API/OS-machinery without any attempt
-        at (graceful) `trio`-style cancellation using our
-        `Actor.cancel()`.
+        If ``hard_killl`` is set to ``True`` then kill the processes
+        directly without any far end graceful ``trio`` cancellation.
 
         '''
-        __runtimeframe__: int = 1  # noqa
         self.cancelled = True
 
         # TODO: impl a repr for spawn more compact
@@ -374,15 +338,11 @@ class ActorNursery:
 @acm
 async def _open_and_supervise_one_cancels_all_nursery(
     actor: Actor,
-    tb_hide: bool = False,
 
 ) -> typing.AsyncGenerator[ActorNursery, None]:
 
-    # normally don't need to show user by default
-    __tracebackhide__: bool = tb_hide
-
-    outer_err: BaseException|None = None
-    inner_err: BaseException|None = None
+    # TODO: yay or nay?
+    __tracebackhide__ = True
 
     # the collection of errors retreived from spawned sub-actors
     errors: dict[tuple[str, str], BaseException] = {}
@@ -392,26 +352,20 @@ async def _open_and_supervise_one_cancels_all_nursery(
     # handling errors that are generated by the inner nursery in
     # a supervisor strategy **before** blocking indefinitely to wait for
     # actors spawned in "daemon mode" (aka started using
-    # `ActorNursery.start_actor()`).
+    # ``ActorNursery.start_actor()``).
 
     # errors from this daemon actor nursery bubble up to caller
-    async with trio.open_nursery(
-        strict_exception_groups=False,
-        # ^XXX^ TODO? instead unpack any RAE as per "loose" style?
-    ) as da_nursery:
+    async with trio.open_nursery() as da_nursery:
         try:
             # This is the inner level "run in actor" nursery. It is
             # awaited first since actors spawned in this way (using
-            # `ActorNusery.run_in_actor()`) are expected to only
+            # ``ActorNusery.run_in_actor()``) are expected to only
             # return a single result and then complete (i.e. be canclled
             # gracefully). Errors collected from these actors are
             # immediately raised for handling by a supervisor strategy.
             # As such if the strategy propagates any error(s) upwards
             # the above "daemon actor" nursery will be notified.
-            async with trio.open_nursery(
-                strict_exception_groups=False,
-                # ^XXX^ TODO? instead unpack any RAE as per "loose" style?
-            ) as ria_nursery:
+            async with trio.open_nursery() as ria_nursery:
 
                 an = ActorNursery(
                     actor,
@@ -433,8 +387,7 @@ async def _open_and_supervise_one_cancels_all_nursery(
                     )
                     an._join_procs.set()
 
-                except BaseException as _inner_err:
-                    inner_err = _inner_err
+                except BaseException as inner_err:
                     errors[actor.uid] = inner_err
 
                     # If we error in the root but the debugger is
@@ -478,8 +431,8 @@ async def _open_and_supervise_one_cancels_all_nursery(
                             ContextCancelled,
                         }:
                             log.cancel(
-                                'Actor-nursery caught remote cancellation\n'
-                                '\n'
+                                'Actor-nursery caught remote cancellation\n\n'
+
                                 f'{inner_err.tb_str}'
                             )
                         else:
@@ -512,10 +465,8 @@ async def _open_and_supervise_one_cancels_all_nursery(
             Exception,
             BaseExceptionGroup,
             trio.Cancelled
-        ) as _outer_err:
-            outer_err = _outer_err
 
-            an._scope_error = outer_err or inner_err
+        ) as err:
 
             # XXX: yet another guard before allowing the cancel
             # sequence in case a (single) child is in debug.
@@ -530,7 +481,7 @@ async def _open_and_supervise_one_cancels_all_nursery(
             if an._children:
                 log.cancel(
                     'Actor-nursery cancelling due error type:\n'
-                    f'{outer_err}\n'
+                    f'{err}\n'
                 )
                 with trio.CancelScope(shield=True):
                     await an.cancel()
@@ -557,23 +508,13 @@ async def _open_and_supervise_one_cancels_all_nursery(
                 else:
                     raise list(errors.values())[0]
 
-            # show frame on any (likely) internal error
-            if (
-                not an.cancelled
-                and an._scope_error
-            ):
-                __tracebackhide__: bool = False
-
         # da_nursery scope end - nursery checkpoint
     # final exit
 
 
 @acm
-# @api_frame
 async def open_nursery(
-    hide_tb: bool = True,
     **kwargs,
-    # ^TODO, paramspec for `open_root_actor()`
 
 ) -> typing.AsyncGenerator[ActorNursery, None]:
     '''
@@ -591,15 +532,13 @@ async def open_nursery(
     which cancellation scopes correspond to each spawned subactor set.
 
     '''
-    __tracebackhide__: bool = hide_tb
     implicit_runtime: bool = False
-    actor: Actor = current_actor(err_on_no_runtime=False)
-    an: ActorNursery|None = None
+
+    actor = current_actor(err_on_no_runtime=False)
+
     try:
-        if (
-            actor is None
-            and is_main_process()
-        ):
+        if actor is None and is_main_process():
+
             # if we are the parent process start the
             # actor runtime implicitly
             log.info("Starting actor runtime!")
@@ -607,10 +546,7 @@ async def open_nursery(
             # mark us for teardown on exit
             implicit_runtime: bool = True
 
-            async with open_root_actor(
-                hide_tb=hide_tb,
-                **kwargs,
-            ) as actor:
+            async with open_root_actor(**kwargs) as actor:
                 assert actor is current_actor()
 
                 try:
@@ -645,27 +581,13 @@ async def open_nursery(
                 an.exited.set()
 
     finally:
-        # show frame on any internal runtime-scope error
-        if (
-            an
-            and
-            not an.cancelled
-            and
-            an._scope_error
-        ):
-            __tracebackhide__: bool = False
-
         msg: str = (
             'Actor-nursery exited\n'
-            f'|_{an}\n'
+            f'|_{an}\n\n'
         )
 
+        # shutdown runtime if it was started
         if implicit_runtime:
-            # shutdown runtime if it was started and report noisly
-            # that we're did so.
             msg += '=> Shutting down actor runtime <=\n'
-            log.info(msg)
 
-        else:
-            # keep noise low during std operation.
-            log.runtime(msg)
+        log.info(msg)

tractor/_testing/__init__.py

@@ -1,113 +0,0 @@
-# tractor: structured concurrent "actors".
-# Copyright 2018-eternity Tyler Goodlet.
-
-# This program is free software: you can redistribute it and/or modify
-# it under the terms of the GNU Affero General Public License as published by
-# the Free Software Foundation, either version 3 of the License, or
-# (at your option) any later version.
-
-# This program is distributed in the hope that it will be useful,
-# but WITHOUT ANY WARRANTY; without even the implied warranty of
-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
-# GNU Affero General Public License for more details.
-
-# You should have received a copy of the GNU Affero General Public License
-# along with this program.  If not, see <https://www.gnu.org/licenses/>.
-
-'''
-Various helpers/utils for auditing your `tractor` app and/or the
-core runtime.
-
-'''
-from contextlib import (
-    asynccontextmanager as acm,
-)
-import os
-import pathlib
-
-import tractor
-from tractor.devx._debug import (
-    BoxedMaybeException,
-)
-from .pytest import (
-    tractor_test as tractor_test
-)
-from .fault_simulation import (
-    break_ipc as break_ipc,
-)
-
-
-def repodir() -> pathlib.Path:
-    '''
-    Return the abspath to the repo directory.
-
-    '''
-    # 2 parents up to step up through tests/<repo_dir>
-    return pathlib.Path(
-        __file__
-
-    # 3 .parents bc:
-    # <._testing-pkg>.<tractor-pkg>.<git-repo-dir>
-    # /$HOME/../<tractor-repo-dir>/tractor/_testing/__init__.py
-    ).parent.parent.parent.absolute()
-
-
-def examples_dir() -> pathlib.Path:
-    '''
-    Return the abspath to the examples directory as `pathlib.Path`.
-
-    '''
-    return repodir() / 'examples'
-
-
-def mk_cmd(
-    ex_name: str,
-    exs_subpath: str = 'debugging',
-) -> str:
-    '''
-    Generate a shell command suitable to pass to `pexpect.spawn()`
-    which runs the script as a python program's entrypoint.
-
-    In particular ensure we disable the new tb coloring via unsetting
-    `$PYTHON_COLORS` so that `pexpect` can pattern match without
-    color-escape-codes.
-
-    '''
-    script_path: pathlib.Path = (
-        examples_dir()
-        / exs_subpath
-        / f'{ex_name}.py'
-    )
-    py_cmd: str = ' '.join([
-        'python',
-        str(script_path)
-    ])
-    # XXX, required for py 3.13+
-    # https://docs.python.org/3/using/cmdline.html#using-on-controlling-color
-    # https://docs.python.org/3/using/cmdline.html#envvar-PYTHON_COLORS
-    os.environ['PYTHON_COLORS'] = '0'
-    return py_cmd
-
-
-@acm
-async def expect_ctxc(
-    yay: bool,
-    reraise: bool = False,
-) -> None:
-    '''
-    Small acm to catch `ContextCancelled` errors when expected
-    below it in a `async with ()` block.
-
-    '''
-    if yay:
-        try:
-            yield (maybe_exc := BoxedMaybeException())
-            raise RuntimeError('Never raised ctxc?')
-        except tractor.ContextCancelled as ctxc:
-            maybe_exc.value = ctxc
-            if reraise:
-                raise
-            else:
-                return
-    else:
-        yield (maybe_exc := BoxedMaybeException())

tractor/_testing/fault_simulation.py

@@ -1,92 +0,0 @@
-# tractor: structured concurrent "actors".
-# Copyright 2018-eternity Tyler Goodlet.
-
-# This program is free software: you can redistribute it and/or modify
-# it under the terms of the GNU Affero General Public License as published by
-# the Free Software Foundation, either version 3 of the License, or
-# (at your option) any later version.
-
-# This program is distributed in the hope that it will be useful,
-# but WITHOUT ANY WARRANTY; without even the implied warranty of
-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
-# GNU Affero General Public License for more details.
-
-# You should have received a copy of the GNU Affero General Public License
-# along with this program.  If not, see <https://www.gnu.org/licenses/>.
-
-'''
-`pytest` utils helpers and plugins for testing `tractor`'s runtime
-and applications.
-
-'''
-
-from tractor import (
-    MsgStream,
-)
-
-async def break_ipc(
-    stream: MsgStream,
-    method: str|None = None,
-    pre_close: bool = False,
-
-    def_method: str = 'socket_close',
-
-) -> None:
-    '''
-    XXX: close the channel right after an error is raised
-    purposely breaking the IPC transport to make sure the parent
-    doesn't get stuck in debug or hang on the connection join.
-    this more or less simulates an infinite msg-receive hang on
-    the other end.
-
-    '''
-    # close channel via IPC prot msging before
-    # any transport breakage
-    if pre_close:
-        await stream.aclose()
-
-    method: str = method or def_method
-    print(
-        '#################################\n'
-        'Simulating CHILD-side IPC BREAK!\n'
-        f'method: {method}\n'
-        f'pre `.aclose()`: {pre_close}\n'
-        '#################################\n'
-    )
-
-    match method:
-        case 'socket_close':
-            await stream._ctx.chan.transport.stream.aclose()
-
-        case 'socket_eof':
-            # NOTE: `trio` does the following underneath this
-            # call in `src/trio/_highlevel_socket.py`:
-            # `Stream.socket.shutdown(tsocket.SHUT_WR)`
-            await stream._ctx.chan.transport.stream.send_eof()
-
-        # TODO: remove since now this will be invalid with our
-        # new typed msg spec?
-        # case 'msg':
-        #     await stream._ctx.chan.send(None)
-
-        # TODO: the actual real-world simulated cases like
-        # transport layer hangs and/or lower layer 2-gens type
-        # scenarios..
-        #
-        # -[ ] already have some issues for this general testing
-        # area:
-        #  - https://github.com/goodboy/tractor/issues/97
-        #  - https://github.com/goodboy/tractor/issues/124
-        #   - PR from @guille:
-        #     https://github.com/goodboy/tractor/pull/149
-        # case 'hang':
-        # TODO: framework research:
-        #
-        # - https://github.com/GuoTengda1993/pynetem
-        # - https://github.com/shopify/toxiproxy
-        # - https://manpages.ubuntu.com/manpages/trusty/man1/wirefilter.1.html
-
-        case _:
-            raise RuntimeError(
-                f'IPC break method unsupported: {method}'
-            )

tractor/_testing/pytest.py

@@ -1,113 +0,0 @@
-# tractor: structured concurrent "actors".
-# Copyright 2018-eternity Tyler Goodlet.
-
-# This program is free software: you can redistribute it and/or modify
-# it under the terms of the GNU Affero General Public License as published by
-# the Free Software Foundation, either version 3 of the License, or
-# (at your option) any later version.
-
-# This program is distributed in the hope that it will be useful,
-# but WITHOUT ANY WARRANTY; without even the implied warranty of
-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
-# GNU Affero General Public License for more details.
-
-# You should have received a copy of the GNU Affero General Public License
-# along with this program.  If not, see <https://www.gnu.org/licenses/>.
-
-'''
-`pytest` utils helpers and plugins for testing `tractor`'s runtime
-and applications.
-
-'''
-from functools import (
-    partial,
-    wraps,
-)
-import inspect
-import platform
-
-import tractor
-import trio
-
-
-def tractor_test(fn):
-    '''
-    Decorator for async test funcs to present them as "native"
-    looking sync funcs runnable by `pytest` using `trio.run()`.
-
-    Use:
-
-    @tractor_test
-    async def test_whatever():
-        await ...
-
-    If fixtures:
-
-        - ``reg_addr`` (a socket addr tuple where arbiter is listening)
-        - ``loglevel`` (logging level passed to tractor internals)
-        - ``start_method`` (subprocess spawning backend)
-
-    are defined in the `pytest` fixture space they will be automatically
-    injected to tests declaring these funcargs.
-    '''
-    @wraps(fn)
-    def wrapper(
-        *args,
-        loglevel=None,
-        reg_addr=None,
-        start_method: str|None = None,
-        debug_mode: bool = False,
-        **kwargs
-    ):
-        # __tracebackhide__ = True
-
-        # NOTE: inject ant test func declared fixture
-        # names by manually checking!
-        if 'reg_addr' in inspect.signature(fn).parameters:
-            # injects test suite fixture value to test as well
-            # as `run()`
-            kwargs['reg_addr'] = reg_addr
-
-        if 'loglevel' in inspect.signature(fn).parameters:
-            # allows test suites to define a 'loglevel' fixture
-            # that activates the internal logging
-            kwargs['loglevel'] = loglevel
-
-        if start_method is None:
-            if platform.system() == "Windows":
-                start_method = 'trio'
-
-        if 'start_method' in inspect.signature(fn).parameters:
-            # set of subprocess spawning backends
-            kwargs['start_method'] = start_method
-
-        if 'debug_mode' in inspect.signature(fn).parameters:
-            # set of subprocess spawning backends
-            kwargs['debug_mode'] = debug_mode
-
-
-        if kwargs:
-
-            # use explicit root actor start
-            async def _main():
-                async with tractor.open_root_actor(
-                    # **kwargs,
-                    registry_addrs=[reg_addr] if reg_addr else None,
-                    loglevel=loglevel,
-                    start_method=start_method,
-
-                    # TODO: only enable when pytest is passed --pdb
-                    debug_mode=debug_mode,
-
-                ):
-                    await fn(*args, **kwargs)
-
-            main = _main
-
-        else:
-            # use implicit root actor start
-            main = partial(fn, *args, **kwargs)
-
-        return trio.run(main)
-
-    return wrapper

tractor/devx/__init__.py

@@ -26,53 +26,12 @@ from ._debug import (
     breakpoint as breakpoint,
     pause as pause,
     pause_from_sync as pause_from_sync,
-    sigint_shield as sigint_shield,
+    shield_sigint_handler as shield_sigint_handler,
+    MultiActorPdb as MultiActorPdb,
     open_crash_handler as open_crash_handler,
     maybe_open_crash_handler as maybe_open_crash_handler,
-    maybe_init_greenback as maybe_init_greenback,
     post_mortem as post_mortem,
-    mk_pdb as mk_pdb,
 )
 from ._stackscope import (
     enable_stack_on_sig as enable_stack_on_sig,
 )
-from .pformat import (
-    add_div as add_div,
-    pformat_caller_frame as pformat_caller_frame,
-    pformat_boxed_tb as pformat_boxed_tb,
-)
-
-
-# TODO, move this to a new `.devx._pdbp` mod?
-def _enable_readline_feats() -> str:
-    '''
-    Handle `readline` when compiled with `libedit` to avoid breaking
-    tab completion in `pdbp` (and its dep `tabcompleter`)
-    particularly since `uv` cpython distis are compiled this way..
-
-    See docs for deats,
-    https://docs.python.org/3/library/readline.html#module-readline
-
-    Originally discovered soln via SO answer,
-    https://stackoverflow.com/q/49287102
-
-    '''
-    import readline
-    if (
-        # 3.13+ attr
-        # https://docs.python.org/3/library/readline.html#readline.backend
-        (getattr(readline, 'backend', False) == 'libedit')
-        or
-        'libedit' in readline.__doc__
-    ):
-        readline.parse_and_bind("python:bind -v")
-        readline.parse_and_bind("python:bind ^I rl_complete")
-        return 'libedit'
-    else:
-        readline.parse_and_bind("tab: complete")
-        readline.parse_and_bind("set editing-mode vi")
-        readline.parse_and_bind("set keymap vi")
-        return 'readline'
-
-
-_enable_readline_feats()

tractor/devx/_frame_stack.py

@@ -1,303 +0,0 @@
-# tractor: structured concurrent "actors".
-# Copyright 2018-eternity Tyler Goodlet.
-
-# This program is free software: you can redistribute it and/or modify
-# it under the terms of the GNU Affero General Public License as published by
-# the Free Software Foundation, either version 3 of the License, or
-# (at your option) any later version.
-
-# This program is distributed in the hope that it will be useful,
-# but WITHOUT ANY WARRANTY; without even the implied warranty of
-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
-# GNU Affero General Public License for more details.
-
-# You should have received a copy of the GNU Affero General Public License
-# along with this program.  If not, see <https://www.gnu.org/licenses/>.
-
-'''
-Tools for code-object annotation, introspection and mutation
-as it pertains to improving the grok-ability of our runtime!
-
-'''
-from __future__ import annotations
-from functools import partial
-import inspect
-from types import (
-    FrameType,
-    FunctionType,
-    MethodType,
-    # CodeType,
-)
-from typing import (
-    Any,
-    Callable,
-    Type,
-)
-
-from tractor.msg import (
-    pretty_struct,
-    NamespacePath,
-)
-import wrapt
-
-
-# TODO: yeah, i don't love this and we should prolly just
-# write a decorator that actually keeps a stupid ref to the func
-# obj..
-def get_class_from_frame(fr: FrameType) -> (
-    FunctionType
-    |MethodType
-):
-    '''
-    Attempt to get the function (or method) reference
-    from a given `FrameType`.
-
-    Verbatim from an SO:
-    https://stackoverflow.com/a/2220759
-
-    '''
-    args, _, _, value_dict = inspect.getargvalues(fr)
-
-    # we check the first parameter for the frame function is
-    # named 'self'
-    if (
-        len(args)
-        and
-        # TODO: other cases for `@classmethod` etc..?)
-        args[0] == 'self'
-    ):
-        # in that case, 'self' will be referenced in value_dict
-        instance: object = value_dict.get('self')
-        if instance:
-          # return its class
-          return getattr(
-              instance,
-              '__class__',
-              None,
-          )
-
-    # return None otherwise
-    return None
-
-
-def get_ns_and_func_from_frame(
-    frame: FrameType,
-) -> Callable:
-    '''
-    Return the corresponding function object reference from
-    a `FrameType`, and return it and it's parent namespace `dict`.
-
-    '''
-    ns: dict[str, Any]
-
-    # for a method, go up a frame and lookup the name in locals()
-    if '.' in (qualname := frame.f_code.co_qualname):
-        cls_name, _, func_name = qualname.partition('.')
-        ns = frame.f_back.f_locals[cls_name].__dict__
-
-    else:
-        func_name: str = frame.f_code.co_name
-        ns = frame.f_globals
-
-    return (
-        ns,
-        ns[func_name],
-    )
-
-
-def func_ref_from_frame(
-    frame: FrameType,
-) -> Callable:
-    func_name: str = frame.f_code.co_name
-    try:
-        return frame.f_globals[func_name]
-    except KeyError:
-        cls: Type|None = get_class_from_frame(frame)
-        if cls:
-            return getattr(
-                cls,
-                func_name,
-            )
-
-
-class CallerInfo(pretty_struct.Struct):
-    # https://docs.python.org/dev/reference/datamodel.html#frame-objects
-    # https://docs.python.org/dev/library/inspect.html#the-interpreter-stack
-    _api_frame: FrameType
-
-    @property
-    def api_frame(self) -> FrameType:
-        try:
-            self._api_frame.clear()
-        except RuntimeError:
-            # log.warning(
-            print(
-                f'Frame {self._api_frame} for {self.api_func} is still active!'
-            )
-
-        return self._api_frame
-
-    _api_func: Callable
-
-    @property
-    def api_func(self) -> Callable:
-        return self._api_func
-
-    _caller_frames_up: int|None = 1
-    _caller_frame: FrameType|None = None  # cached after first stack scan
-
-    @property
-    def api_nsp(self) -> NamespacePath|None:
-        func: FunctionType = self.api_func
-        if func:
-            return NamespacePath.from_ref(func)
-
-        return '<unknown>'
-
-    @property
-    def caller_frame(self) -> FrameType:
-
-        # if not already cached, scan up stack explicitly by
-        # configured count.
-        if not self._caller_frame:
-            if self._caller_frames_up:
-                for _ in range(self._caller_frames_up):
-                    caller_frame: FrameType|None = self.api_frame.f_back
-
-                if not caller_frame:
-                    raise ValueError(
-                        'No frame exists {self._caller_frames_up} up from\n'
-                        f'{self.api_frame} @ {self.api_nsp}\n'
-                    )
-
-            self._caller_frame = caller_frame
-
-        return self._caller_frame
-
-    @property
-    def caller_nsp(self) -> NamespacePath|None:
-        func: FunctionType = self.api_func
-        if func:
-            return NamespacePath.from_ref(func)
-
-        return '<unknown>'
-
-
-def find_caller_info(
-    dunder_var: str = '__runtimeframe__',
-    iframes:int = 1,
-    check_frame_depth: bool = True,
-
-) -> CallerInfo|None:
-    '''
-    Scan up the callstack for a frame with a `dunder_var: str` variable
-    and return the `iframes` frames above it.
-
-    By default we scan for a `__runtimeframe__` scope var which
-    denotes a `tractor` API above which (one frame up) is "user
-    app code" which "called into" the `tractor` method or func.
-
-    TODO: ex with `Portal.open_context()`
-
-    '''
-    # TODO: use this instead?
-    # https://docs.python.org/3/library/inspect.html#inspect.getouterframes
-    frames: list[inspect.FrameInfo] = inspect.stack()
-    for fi in frames:
-        assert (
-            fi.function
-            ==
-            fi.frame.f_code.co_name
-        )
-        this_frame: FrameType = fi.frame
-        dunder_val: int|None = this_frame.f_locals.get(dunder_var)
-        if dunder_val:
-            go_up_iframes: int = (
-                dunder_val  # could be 0 or `True` i guess?
-                or
-                iframes
-            )
-            rt_frame: FrameType = fi.frame
-            call_frame = rt_frame
-            for i in range(go_up_iframes):
-                call_frame = call_frame.f_back
-
-            return CallerInfo(
-                _api_frame=rt_frame,
-                _api_func=func_ref_from_frame(rt_frame),
-                _caller_frames_up=go_up_iframes,
-            )
-
-    return None
-
-
-_frame2callerinfo_cache: dict[FrameType, CallerInfo] = {}
-
-
-# TODO: -[x] move all this into new `.devx._frame_stack`!
-# -[ ] consider rename to _callstack?
-# -[ ] prolly create a `@runtime_api` dec?
-#   |_ @api_frame seems better?
-# -[ ] ^- make it capture and/or accept buncha optional
-#     meta-data like a fancier version of `@pdbp.hideframe`.
-#
-def api_frame(
-    wrapped: Callable|None = None,
-    *,
-    caller_frames_up: int = 1,
-
-) -> Callable:
-
-    # handle the decorator called WITHOUT () case,
-    # i.e. just @api_frame, NOT @api_frame(extra=<blah>)
-    if wrapped is None:
-        return partial(
-            api_frame,
-            caller_frames_up=caller_frames_up,
-        )
-
-    @wrapt.decorator
-    async def wrapper(
-        wrapped: Callable,
-        instance: object,
-        args: tuple,
-        kwargs: dict,
-    ):
-        # maybe cache the API frame for this call
-        global _frame2callerinfo_cache
-        this_frame: FrameType = inspect.currentframe()
-        api_frame: FrameType = this_frame.f_back
-
-        if not _frame2callerinfo_cache.get(api_frame):
-            _frame2callerinfo_cache[api_frame] = CallerInfo(
-                _api_frame=api_frame,
-                _api_func=wrapped,
-                _caller_frames_up=caller_frames_up,
-            )
-
-        return wrapped(*args, **kwargs)
-
-    # annotate the function as a "api function", meaning it is
-    # a function for which the function above it in the call stack should be
-    # non-`tractor` code aka "user code".
-    #
-    # in the global frame cache for easy lookup from a given
-    # func-instance
-    wrapped._call_infos: dict[FrameType, CallerInfo] = _frame2callerinfo_cache
-    wrapped.__api_func__: bool = True
-    return wrapper(wrapped)
-
-
-# TODO: something like this instead of the adhoc frame-unhiding
-# blocks all over the runtime!! XD
-# -[ ] ideally we can expect a certain error (set) and if something
-#     else is raised then all frames below the wrapped one will be
-#     un-hidden via `__tracebackhide__: bool = False`.
-# |_ might need to dynamically mutate the code objs like
-#    `pdbp.hideframe()` does?
-# -[ ] use this as a `@acm` decorator as introed in 3.10?
-# @acm
-# async def unhide_frame_when_not(
-#     error_set: set[BaseException],
-# ) -> TracebackType:
-#     ...

tractor/devx/_stackscope.py

@@ -23,109 +23,29 @@ into each ``trio.Nursery`` except it links the lifetimes of memory space
 disjoint, parallel executing tasks in separate actors.
 
 '''
-from __future__ import annotations
-# from functools import partial
-from threading import (
-    current_thread,
-    Thread,
-    RLock,
-)
-import multiprocessing as mp
 from signal import (
     signal,
-    getsignal,
     SIGUSR1,
-    SIGINT,
-)
-# import traceback
-from types import ModuleType
-from typing import (
-    Callable,
-    TYPE_CHECKING,
 )
 
 import trio
-from tractor import (
-    _state,
-    log as logmod,
-)
-from tractor.devx import _debug
-
-log = logmod.get_logger(__name__)
-
-
-if TYPE_CHECKING:
-    from tractor._spawn import ProcessType
-    from tractor import (
-        Actor,
-        ActorNursery,
-    )
-
 
 @trio.lowlevel.disable_ki_protection
 def dump_task_tree() -> None:
-    '''
-    Do a classic `stackscope.extract()` task-tree dump to console at
-    `.devx()` level.
-
-    '''
     import stackscope
+    from tractor.log import get_console_log
+
     tree_str: str = str(
         stackscope.extract(
             trio.lowlevel.current_root_task(),
             recurse_child_tasks=True
         )
     )
-    actor: Actor = _state.current_actor()
-    thr: Thread = current_thread()
-    current_sigint_handler: Callable = getsignal(SIGINT)
-    if (
-        current_sigint_handler
-        is not
-        _debug.DebugStatus._trio_handler
-    ):
-        sigint_handler_report: str = (
-            'The default `trio` SIGINT handler was replaced?!'
-        )
-    else:
-        sigint_handler_report: str = (
-            'The default `trio` SIGINT handler is in use?!'
-        )
-
-    # sclang symbology
-    # |_<object>
-    # |_(Task/Thread/Process/Actor
-    # |_{Supervisor/Scope
-    # |_[Storage/Memory/IPC-Stream/Data-Struct
-
-    log.devx(
-        f'Dumping `stackscope` tree for actor\n'
-        f'(>: {actor.uid!r}\n'
-        f' |_{mp.current_process()}\n'
-        f'   |_{thr}\n'
-        f'     |_{actor}\n'
-        f'\n'
-        f'{sigint_handler_report}\n'
-        f'signal.getsignal(SIGINT) -> {current_sigint_handler!r}\n'
-        # f'\n'
-        # start-of-trace-tree delimiter (mostly for testing)
-        # f'------ {actor.uid!r} ------\n'
-        f'\n'
-        f'------ start-of-{actor.uid!r} ------\n'
-        f'|\n'
-        f'{tree_str}'
-        # end-of-trace-tree delimiter (mostly for testing)
-        f'|\n'
-        f'|_____ end-of-{actor.uid!r} ______\n'
+    log = get_console_log('cancel')
+    log.pdb(
+        f'Dumping `stackscope` tree:\n\n'
+        f'{tree_str}\n'
     )
-    # TODO: can remove this right?
-    # -[ ] was original code from author
-    #
-    # print(
-    #     'DUMPING FROM PRINT\n'
-    #     +
-    #     content
-    # )
     # import logging
     # try:
     #     with open("/dev/tty", "w") as tty:
@@ -135,130 +55,30 @@ def dump_task_tree() -> None:
     #         "task_tree"
     #     ).exception("Error printing task tree")
 
-_handler_lock = RLock()
-_tree_dumped: bool = False
 
+def signal_handler(sig: int, frame: object) -> None:
+    import traceback
+    try:
+        trio.lowlevel.current_trio_token(
+        ).run_sync_soon(dump_task_tree)
+    except RuntimeError:
+        # not in async context -- print a normal traceback
+        traceback.print_stack()
 
-def dump_tree_on_sig(
-    sig: int,
-    frame: object,
-
-    relay_to_subs: bool = True,
-
-) -> None:
-    global _tree_dumped, _handler_lock
-    with _handler_lock:
-        # if _tree_dumped:
-        #     log.warning(
-        #         'Already dumped for this actor...??'
-        #     )
-        #     return
-
-        _tree_dumped = True
-
-        # actor: Actor = _state.current_actor()
-        log.devx(
-            'Trying to dump `stackscope` tree..\n'
-        )
-        try:
-            dump_task_tree()
-            # await actor._service_n.start_soon(
-            #     partial(
-            #         trio.to_thread.run_sync,
-            #         dump_task_tree,
-            #     )
-            # )
-            # trio.lowlevel.current_trio_token().run_sync_soon(
-            #     dump_task_tree
-            # )
-
-        except RuntimeError:
-            log.exception(
-                'Failed to dump `stackscope` tree..\n'
-            )
-            # not in async context -- print a normal traceback
-            # traceback.print_stack()
-            raise
-
-        except BaseException:
-            log.exception(
-                'Failed to dump `stackscope` tree..\n'
-            )
-            raise
-
-        # log.devx(
-        #     'Supposedly we dumped just fine..?'
-        # )
-
-    if not relay_to_subs:
-        return
-
-    an: ActorNursery
-    for an in _state.current_actor()._actoruid2nursery.values():
-        subproc: ProcessType
-        subactor: Actor
-        for subactor, subproc, _ in an._children.values():
-            log.warning(
-                f'Relaying `SIGUSR1`[{sig}] to sub-actor\n'
-                f'{subactor}\n'
-                f' |_{subproc}\n'
-            )
-
-            # bc of course stdlib can't have a std API.. XD
-            match subproc:
-                case trio.Process():
-                    subproc.send_signal(sig)
-
-                case mp.Process():
-                    subproc._send_signal(sig)
 
 
 def enable_stack_on_sig(
-    sig: int = SIGUSR1,
-) -> ModuleType:
+    sig: int = SIGUSR1
+) -> None:
     '''
     Enable `stackscope` tracing on reception of a signal; by
     default this is SIGUSR1.
 
-    HOT TIP: a task/ctx-tree dump can be triggered from a shell with
-    fancy cmds.
-
-    For ex. from `bash` using `pgrep` and cmd-sustitution
-    (https://www.gnu.org/software/bash/manual/bash.html#Command-Substitution)
-    you could use:
-
-    >> kill -SIGUSR1 $(pgrep -f <part-of-cmd: str>)
-
-    OR without a sub-shell,
-
-    >> pkill --signal SIGUSR1 -f <part-of-cmd: str>
-
     '''
-    try:
-        import stackscope
-    except ImportError:
-        log.warning(
-            '`stackscope` not installed for use in debug mode!'
-        )
-        return None
-
-    handler: Callable|int = getsignal(sig)
-    if handler is dump_tree_on_sig:
-        log.devx(
-            'A `SIGUSR1` handler already exists?\n'
-            f'|_ {handler!r}\n'
-        )
-        return
-
     signal(
         sig,
-        dump_tree_on_sig,
+        signal_handler,
     )
-    log.devx(
-        'Enabling trace-trees on `SIGUSR1` '
-        'since `stackscope` is installed @ \n'
-        f'{stackscope!r}\n\n'
-        f'With `SIGUSR1` handler\n'
-        f'|_{dump_tree_on_sig}\n'
-    )
-    return stackscope
+    # NOTE: not the above can be triggered from
+    # a (xonsh) shell using:
+    # kill -SIGUSR1 @$(pgrep -f '<cmd>')

tractor/devx/pformat.py

@@ -1,169 +0,0 @@
-# tractor: structured concurrent "actors".
-# Copyright 2018-eternity Tyler Goodlet.
-
-# This program is free software: you can redistribute it and/or modify
-# it under the terms of the GNU Affero General Public License as published by
-# the Free Software Foundation, either version 3 of the License, or
-# (at your option) any later version.
-
-# This program is distributed in the hope that it will be useful,
-# but WITHOUT ANY WARRANTY; without even the implied warranty of
-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
-# GNU Affero General Public License for more details.
-
-# You should have received a copy of the GNU Affero General Public License
-# along with this program.  If not, see <https://www.gnu.org/licenses/>.
-
-'''
-Pretty formatters for use throughout the code base.
-Mostly handy for logging and exception message content.
-
-'''
-import textwrap
-import traceback
-
-from trio import CancelScope
-
-
-def add_div(
-    message: str,
-    div_str: str = '------ - ------',
-
-) -> str:
-    '''
-    Add a "divider string" to the input `message` with
-    a little math to center it underneath.
-
-    '''
-    div_offset: int = (
-        round(len(message)/2)+1
-        -
-        round(len(div_str)/2)+1
-    )
-    div_str: str = (
-        '\n' + ' '*div_offset + f'{div_str}\n'
-    )
-    return div_str
-
-
-def pformat_boxed_tb(
-    tb_str: str,
-    fields_str: str|None = None,
-    field_prefix: str = ' |_',
-
-    tb_box_indent: int|None = None,
-    tb_body_indent: int = 1,
-    boxer_header: str = '-'
-
-) -> str:
-    '''
-    Create a "boxed" looking traceback string.
-
-    Useful for emphasizing traceback text content as being an
-    embedded attribute of some other object (like
-    a `RemoteActorError` or other boxing remote error shuttle
-    container).
-
-    Any other parent/container "fields" can be passed in the
-    `fields_str` input along with other prefix/indent settings.
-
-    '''
-    if (
-        fields_str
-        and
-        field_prefix
-    ):
-        fields: str = textwrap.indent(
-            fields_str,
-            prefix=field_prefix,
-        )
-    else:
-        fields = fields_str or ''
-
-    tb_body = tb_str
-    if tb_body_indent:
-        tb_body: str = textwrap.indent(
-            tb_str,
-            prefix=tb_body_indent * ' ',
-        )
-
-    tb_box: str = (
-        f'|\n'
-        f' ------ {boxer_header} ------\n'
-        f'{tb_body}'
-        f' ------ {boxer_header}- ------\n'
-        f'_|'
-    )
-    tb_box_indent: str = (
-        tb_box_indent
-        or
-        1
-
-        # (len(field_prefix))
-        # ? ^-TODO-^ ? if you wanted another indent level
-    )
-    if tb_box_indent > 0:
-        tb_box: str = textwrap.indent(
-            tb_box,
-            prefix=tb_box_indent * ' ',
-        )
-
-    return (
-        fields
-        +
-        tb_box
-    )
-
-
-def pformat_caller_frame(
-    stack_limit: int = 1,
-    box_tb: bool = True,
-) -> str:
-    '''
-    Capture and return the traceback text content from
-    `stack_limit` call frames up.
-
-    '''
-    tb_str: str = (
-        '\n'.join(
-            traceback.format_stack(limit=stack_limit)
-        )
-    )
-    if box_tb:
-        tb_str: str = pformat_boxed_tb(
-            tb_str=tb_str,
-            field_prefix='  ',
-            indent='',
-        )
-    return tb_str
-
-
-def pformat_cs(
-    cs: CancelScope,
-    var_name: str = 'cs',
-    field_prefix: str = ' |_',
-) -> str:
-    '''
-    Pretty format info about a `trio.CancelScope` including most
-    of its public state and `._cancel_status`.
-
-    The output can be modified to show a "var name" for the
-    instance as a field prefix, just a simple str before each
-    line more or less.
-
-    '''
-
-    fields: str = textwrap.indent(
-        (
-            f'cancel_called = {cs.cancel_called}\n'
-            f'cancelled_caught = {cs.cancelled_caught}\n'
-            f'_cancel_status = {cs._cancel_status}\n'
-            f'shield = {cs.shield}\n'
-        ),
-        prefix=field_prefix,
-    )
-    return (
-        f'{var_name}: {cs}\n'
-        +
-        fields
-    )

tractor/experimental/_pubsub.py

@@ -31,7 +31,7 @@ from typing import (
     Callable,
 )
 from functools import partial
-from contextlib import aclosing
+from async_generator import aclosing
 
 import trio
 import wrapt

tractor/log.py

@@ -21,11 +21,6 @@ Log like a forester!
 from collections.abc import Mapping
 import sys
 import logging
-from logging import (
-    LoggerAdapter,
-    Logger,
-    StreamHandler,
-)
 import colorlog  # type: ignore
 
 import trio
@@ -53,20 +48,20 @@ LOG_FORMAT = (
 
 DATE_FORMAT = '%b %d %H:%M:%S'
 
-# FYI, ERROR is 40
-# TODO: use a `bidict` to avoid the :155 check?
-CUSTOM_LEVELS: dict[str, int] = {
+LEVELS: dict[str, int] = {
     'TRANSPORT': 5,
     'RUNTIME': 15,
-    'DEVX': 17,
-    'CANCEL': 22,
+    'CANCEL': 16,
     'PDB': 500,
 }
+# _custom_levels: set[str] = {
+#     lvlname.lower for lvlname in LEVELS.keys()
+# }
+
 STD_PALETTE = {
     'CRITICAL': 'red',
     'ERROR': 'red',
     'PDB': 'white',
-    'DEVX': 'cyan',
     'WARNING': 'yellow',
     'INFO': 'green',
     'CANCEL': 'yellow',
@@ -83,7 +78,7 @@ BOLD_PALETTE = {
 
 # TODO: this isn't showing the correct '{filename}'
 # as it did before..
-class StackLevelAdapter(LoggerAdapter):
+class StackLevelAdapter(logging.LoggerAdapter):
 
     def transport(
         self,
@@ -91,8 +86,7 @@ class StackLevelAdapter(LoggerAdapter):
 
     ) -> None:
         '''
-        IPC transport level msg IO; generally anything below
-        `._ipc.Channel` and friends.
+        IPC level msg-ing.
 
         '''
         return self.log(5, msg)
@@ -108,11 +102,11 @@ class StackLevelAdapter(LoggerAdapter):
         msg: str,
     ) -> None:
         '''
-        Cancellation sequencing, mostly for runtime reporting.
+        Cancellation logging, mostly for runtime reporting.
 
         '''
         return self.log(
-            level=22,
+            level=16,
             msg=msg,
             # stacklevel=4,
         )
@@ -122,21 +116,11 @@ class StackLevelAdapter(LoggerAdapter):
         msg: str,
     ) -> None:
         '''
-        `pdb`-REPL (debugger) related statuses.
+        Debugger logging.
 
         '''
         return self.log(500, msg)
 
-    def devx(
-        self,
-        msg: str,
-    ) -> None:
-        '''
-        "Developer experience" sub-sys statuses.
-
-        '''
-        return self.log(17, msg)
-
     def log(
         self,
         level,
@@ -148,13 +132,12 @@ class StackLevelAdapter(LoggerAdapter):
         Delegate a log call to the underlying logger, after adding
         contextual information from this adapter instance.
 
-        NOTE: all custom level methods (above) delegate to this!
-
         '''
         if self.isEnabledFor(level):
             stacklevel: int = 3
             if (
-                level in CUSTOM_LEVELS.values()
+                level in LEVELS.values()
+                # or level in _custom_levels
             ):
                 stacklevel: int = 4
 
@@ -201,30 +184,8 @@ class StackLevelAdapter(LoggerAdapter):
         )
 
 
-# TODO IDEAs:
-# -[ ] move to `.devx.pformat`?
-# -[ ] do per task-name and actor-name color coding
-# -[ ] unique color per task-id and actor-uuid
-def pformat_task_uid(
-    id_part: str = 'tail'
-):
-    '''
-    Return `str`-ified unique for a `trio.Task` via a combo of its
-    `.name: str` and `id()` truncated output.
-
-    '''
-    task: trio.Task = trio.lowlevel.current_task()
-    tid: str = str(id(task))
-    if id_part == 'tail':
-        tid_part: str = tid[-6:]
-    else:
-        tid_part: str = tid[:6]
-
-    return f'{task.name}[{tid_part}]'
-
-
 _conc_name_getters = {
-    'task': pformat_task_uid,
+    'task': lambda: trio.lowlevel.current_task().name,
     'actor': lambda: current_actor(),
     'actor_name': lambda: current_actor().name,
     'actor_uid': lambda: current_actor().uid[1][:6],
@@ -232,10 +193,7 @@ _conc_name_getters = {
 
 
 class ActorContextInfo(Mapping):
-    '''
-    Dyanmic lookup for local actor and task names.
-
-    '''
+    "Dyanmic lookup for local actor and task names"
     _context_keys = (
         'task',
         'actor',
@@ -258,28 +216,19 @@ class ActorContextInfo(Mapping):
 
 
 def get_logger(
-    name: str|None = None,
+
+    name: str | None = None,
     _root_name: str = _proj_name,
 
-    logger: Logger|None = None,
-
-    # TODO, using `.config.dictConfig()` api?
-    # -[ ] SO answer with docs links
-    #  |_https://stackoverflow.com/questions/7507825/where-is-a-complete-example-of-logging-config-dictconfig
-    #  |_https://docs.python.org/3/library/logging.config.html#configuration-dictionary-schema
-    subsys_spec: str|None = None,
-
 ) -> StackLevelAdapter:
     '''Return the package log or a sub-logger for ``name`` if provided.
 
     '''
-    log: Logger
-    log = rlog = logger or logging.getLogger(_root_name)
+    log = rlog = logging.getLogger(_root_name)
 
     if (
         name
-        and
-        name != _proj_name
+        and name != _proj_name
     ):
 
         # NOTE: for handling for modules that use ``get_logger(__name__)``
@@ -291,7 +240,7 @@ def get_logger(
         #   since in python the {filename} is always this same
         #   module-file.
 
-        sub_name: None|str = None
+        sub_name: None | str = None
         rname, _, sub_name = name.partition('.')
         pkgpath, _, modfilename = sub_name.rpartition('.')
 
@@ -314,13 +263,10 @@ def get_logger(
 
     # add our actor-task aware adapter which will dynamically look up
     # the actor and task names at each log emit
-    logger = StackLevelAdapter(
-        log,
-        ActorContextInfo(),
-    )
+    logger = StackLevelAdapter(log, ActorContextInfo())
 
     # additional levels
-    for name, val in CUSTOM_LEVELS.items():
+    for name, val in LEVELS.items():
         logging.addLevelName(val, name)
 
         # ensure customs levels exist as methods
@@ -330,25 +276,15 @@ def get_logger(
 
 
 def get_console_log(
-    level: str|None = None,
-    logger: Logger|None = None,
+    level: str | None = None,
     **kwargs,
+) -> logging.LoggerAdapter:
+    '''Get the package logger and enable a handler which writes to stderr.
 
-) -> LoggerAdapter:
+    Yeah yeah, i know we can use ``DictConfig``. You do it.
     '''
-    Get a `tractor`-style logging instance: a `Logger` wrapped in
-    a `StackLevelAdapter` which injects various concurrency-primitive
-    (process, thread, task) fields and enables a `StreamHandler` that
-    writes on stderr using `colorlog` formatting.
-
-    Yeah yeah, i know we can use `logging.config.dictConfig()`. You do it.
-
-    '''
-    log = get_logger(
-        logger=logger,
-        **kwargs
-    )  # set a root logger
-    logger: Logger = log.logger
+    log = get_logger(**kwargs)  # our root logger
+    logger = log.logger
 
     if not level:
         return log
@@ -367,13 +303,9 @@ def get_console_log(
             None,
         )
     ):
-        fmt = LOG_FORMAT
-        # if logger:
-        #     fmt = None
-
-        handler = StreamHandler()
+        handler = logging.StreamHandler()
         formatter = colorlog.ColoredFormatter(
-            fmt=fmt,
+            LOG_FORMAT,
             datefmt=DATE_FORMAT,
             log_colors=STD_PALETTE,
             secondary_log_colors=BOLD_PALETTE,
@@ -390,20 +322,4 @@ def get_loglevel() -> str:
 
 
 # global module logger for tractor itself
-log: StackLevelAdapter = get_logger('tractor')
-
-
-def at_least_level(
-    log: Logger|LoggerAdapter,
-    level: int|str,
-) -> bool:
-    '''
-    Predicate to test if a given level is active.
-
-    '''
-    if isinstance(level, str):
-        level: int = CUSTOM_LEVELS[level.upper()]
-
-    if log.getEffectiveLevel() <= level:
-        return True
-    return False
+log = get_logger('tractor')

tractor/msg/__init__.py

@@ -18,57 +18,9 @@
 Built-in messaging patterns, types, APIs and helpers.
 
 '''
-from typing import (
-    TypeAlias,
-)
 from .ptr import (
     NamespacePath as NamespacePath,
 )
-from .pretty_struct import (
+from .types import (
     Struct as Struct,
 )
-from ._codec import (
-    _def_msgspec_codec as _def_msgspec_codec,
-    _ctxvar_MsgCodec as _ctxvar_MsgCodec,
-
-    apply_codec as apply_codec,
-    mk_codec as mk_codec,
-    mk_dec as mk_dec,
-    MsgCodec as MsgCodec,
-    MsgDec as MsgDec,
-    current_codec as current_codec,
-)
-# currently can't bc circular with `._context`
-# from ._ops import (
-#     PldRx as PldRx,
-#     _drain_to_final_msg as _drain_to_final_msg,
-# )
-
-from .types import (
-    PayloadMsg as PayloadMsg,
-
-    Aid as Aid,
-    SpawnSpec as SpawnSpec,
-
-    Start as Start,
-    StartAck as StartAck,
-
-    Started as Started,
-    Yield as Yield,
-    Stop as Stop,
-    Return as Return,
-    CancelAck as CancelAck,
-
-    Error as Error,
-
-    # type-var for `.pld` field
-    PayloadT as PayloadT,
-
-    # full msg class set from above as list
-    __msg_types__ as __msg_types__,
-
-    # type-alias for union of all msgs
-    MsgType as MsgType,
-)
-
-__msg_spec__: TypeAlias = MsgType

tractor/msg/_codec.py

@@ -1,886 +0,0 @@
-# tractor: structured concurrent "actors".
-# Copyright 2018-eternity Tyler Goodlet.
-
-# This program is free software: you can redistribute it and/or modify
-# it under the terms of the GNU Affero General Public License as published by
-# the Free Software Foundation, either version 3 of the License, or
-# (at your option) any later version.
-
-# This program is distributed in the hope that it will be useful,
-# but WITHOUT ANY WARRANTY; without even the implied warranty of
-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
-# GNU Affero General Public License for more details.
-
-# You should have received a copy of the GNU Affero General Public License
-# along with this program.  If not, see <https://www.gnu.org/licenses/>.
-
-'''
-IPC msg interchange codec management.
-
-Supported backend libs:
-- `msgspec.msgpack`
-
-ToDo: backends we prolly should offer:
-
-- see project/lib list throughout GH issue discussion comments:
-  https://github.com/goodboy/tractor/issues/196
-
-- `capnproto`: https://capnproto.org/rpc.html
-   - https://capnproto.org/language.html#language-reference
-
-'''
-from __future__ import annotations
-from contextlib import (
-    contextmanager as cm,
-)
-from contextvars import (
-    ContextVar,
-    Token,
-)
-import textwrap
-from typing import (
-    Any,
-    Callable,
-    Protocol,
-    Type,
-    TYPE_CHECKING,
-    TypeVar,
-    Union,
-)
-from types import ModuleType
-
-import msgspec
-from msgspec import (
-    msgpack,
-    Raw,
-)
-# TODO: see notes below from @mikenerone..
-# from tricycle import TreeVar
-
-from tractor.msg.pretty_struct import Struct
-from tractor.msg.types import (
-    mk_msg_spec,
-    MsgType,
-    PayloadMsg,
-)
-from tractor.log import get_logger
-
-if TYPE_CHECKING:
-    from tractor._context import Context
-
-log = get_logger(__name__)
-
-
-# TODO: unify with `MsgCodec` by making `._dec` part this?
-class MsgDec(Struct):
-    '''
-    An IPC msg (payload) decoder.
-
-    Normally used to decode only a payload: `MsgType.pld:
-    PayloadT` field before delivery to IPC consumer code.
-
-    '''
-    _dec: msgpack.Decoder
-    # _ext_types_box: Struct|None = None
-
-    @property
-    def dec(self) -> msgpack.Decoder:
-        return self._dec
-
-    def __repr__(self) -> str:
-
-        speclines: str = self.spec_str
-
-        # in multi-typed spec case we stick the list
-        # all on newlines after the |__pld_spec__:,
-        # OW it's prolly single type spec-value
-        # so just leave it on same line.
-        if '\n' in speclines:
-            speclines: str = '\n' + textwrap.indent(
-                speclines,
-                prefix=' '*3,
-            )
-
-        body: str = textwrap.indent(
-            f'|_dec_hook: {self.dec.dec_hook}\n'
-            f'|__pld_spec__: {speclines}\n',
-            prefix=' '*2,
-        )
-        return (
-            f'<{type(self).__name__}(\n'
-            f'{body}'
-            ')>'
-        )
-
-    # struct type unions
-    # https://jcristharif.com/msgspec/structs.html#tagged-unions
-    #
-    # ^-TODO-^: make a wrapper type for this such that alt
-    # backends can be represented easily without a `Union` needed,
-    # AND so that we have better support for wire transport.
-    #
-    # -[ ] maybe `FieldSpec` is a good name since msg-spec
-    #   better applies to a `MsgType[FieldSpec]`?
-    #
-    # -[ ] both as part of the `.open_context()` call AND as part of the
-    #     immediate ack-reponse (see similar below)
-    #     we should do spec matching and fail if anything is awry?
-    #
-    # -[ ] eventually spec should be generated/parsed from the
-    #     type-annots as # desired in GH issue:
-    #     https://github.com/goodboy/tractor/issues/365
-    #
-    # -[ ] semantics of the mismatch case
-    #   - when caller-callee specs we should raise
-    #    a `MsgTypeError` or `MsgSpecError` or similar?
-    #
-    # -[ ] wrapper types for both spec types such that we can easily
-    #     IPC transport them?
-    #     - `TypeSpec: Union[Type]`
-    #      * also a `.__contains__()` for doing `None in
-    #      TypeSpec[None|int]` since rn you need to do it on
-    #      `.__args__` for unions..
-    #     - `MsgSpec: Union[MsgType]
-    #
-    # -[ ] auto-genning this from new (in 3.12) type parameter lists Bo
-    # |_ https://docs.python.org/3/reference/compound_stmts.html#type-params
-    # |_ historical pep 695: https://peps.python.org/pep-0695/
-    # |_ full lang spec: https://typing.readthedocs.io/en/latest/spec/
-    # |_ on annotation scopes:
-    #    https://docs.python.org/3/reference/executionmodel.html#annotation-scopes
-    # |_ 3.13 will have subscriptable funcs Bo
-    #    https://peps.python.org/pep-0718/
-    @property
-    def spec(self) -> Union[Type[Struct]]:
-        # NOTE: defined and applied inside `mk_codec()`
-        return self._dec.type
-
-    # no difference, as compared to a `MsgCodec` which defines the
-    # `MsgType.pld: PayloadT` part of its spec separately
-    pld_spec = spec
-
-    # TODO: would get moved into `FieldSpec.__str__()` right?
-    @property
-    def spec_str(self) -> str:
-        return pformat_msgspec(
-            codec=self,
-            join_char='|',
-        )
-
-    pld_spec_str = spec_str
-
-    def decode(
-        self,
-        raw: Raw|bytes,
-    ) -> Any:
-        return self._dec.decode(raw)
-
-    @property
-    def hook(self) -> Callable|None:
-        return self._dec.dec_hook
-
-
-def mk_dec(
-    spec: Union[Type[Struct]]|Type|None,
-
-    # NOTE, required for ad-hoc type extensions to the underlying
-    # serialization proto (which is default `msgpack`),
-    # https://jcristharif.com/msgspec/extending.html#mapping-to-from-native-types
-    dec_hook: Callable|None = None,
-    ext_types: list[Type]|None = None,
-
-) -> MsgDec:
-    '''
-    Create an IPC msg decoder, a slightly higher level wrapper around
-    a `msgspec.msgpack.Decoder` which provides,
-
-    - easier introspection of the underlying type spec via
-      the `.spec` and `.spec_str` attrs,
-    - `.hook` access to the `Decoder.dec_hook()`,
-    - automatic custom extension-types decode support when
-      `dec_hook()` is provided such that any `PayloadMsg.pld` tagged
-      as a type from from `ext_types` (presuming the `MsgCodec.encode()` also used
-      a `.enc_hook()`) is processed and constructed by a `PldRx` implicitily.
-
-    NOTE, as mentioned a `MsgDec` is normally used for `PayloadMsg.pld: PayloadT` field
-    decoding inside an IPC-ctx-oriented `PldRx`.
-
-    '''
-    if (
-        spec is None
-        and
-        ext_types is None
-    ):
-        raise TypeError(
-            f'MIssing type-`spec` for msg decoder!\n'
-            f'\n'
-            f'`spec=None` is **only** permitted is if custom extension types '
-            f'are provided via `ext_types`, meaning it must be non-`None`.\n'
-            f'\n'
-            f'In this case it is presumed that only the `ext_types`, '
-            f'which much be handled by a paired `dec_hook()`, '
-            f'will be permitted within the payload type-`spec`!\n'
-            f'\n'
-            f'spec = {spec!r}\n'
-            f'dec_hook = {dec_hook!r}\n'
-            f'ext_types = {ext_types!r}\n'
-        )
-
-    if dec_hook:
-        if ext_types is None:
-            raise TypeError(
-                f'If extending the serializable types with a custom decode hook (`dec_hook()`), '
-                f'you must also provide the expected type set that the hook will handle '
-                f'via a `ext_types: Union[Type]|None = None` argument!\n'
-                f'\n'
-                f'dec_hook = {dec_hook!r}\n'
-                f'ext_types = {ext_types!r}\n'
-            )
-
-        # XXX, i *thought* we would require a boxing struct as per docs,
-        # https://jcristharif.com/msgspec/extending.html#mapping-to-from-native-types
-        # |_ see comment,
-        #  > Note that typed deserialization is required for
-        #  > successful roundtripping here, so we pass `MyMessage` to
-        #  > `Decoder`.
-        #
-        # BUT, turns out as long as you spec a union with `Raw` it
-        # will work? kk B)
-        #
-        # maybe_box_struct = mk_boxed_ext_struct(ext_types)
-        spec = Raw | Union[*ext_types]
-
-    return MsgDec(
-        _dec=msgpack.Decoder(
-            type=spec,  # like `MsgType[Any]`
-            dec_hook=dec_hook,
-        ),
-    )
-
-
-# TODO? remove since didn't end up needing this?
-def mk_boxed_ext_struct(
-    ext_types: list[Type],
-) -> Struct:
-    # NOTE, originally was to wrap non-msgpack-supported "extension
-    # types" in a field-typed boxing struct, see notes around the
-    # `dec_hook()` branch in `mk_dec()`.
-    ext_types_union = Union[*ext_types]
-    repr_ext_types_union: str = (
-        str(ext_types_union)
-        or
-        "|".join(ext_types)
-    )
-    BoxedExtType = msgspec.defstruct(
-        f'BoxedExts[{repr_ext_types_union}]',
-        fields=[
-            ('boxed', ext_types_union),
-        ],
-    )
-    return BoxedExtType
-
-
-def unpack_spec_types(
-    spec: Union[Type]|Type,
-) -> set[Type]:
-    '''
-    Given an input type-`spec`, either a lone type
-    or a `Union` of types (like `str|int|MyThing`),
-    return a set of individual types.
-
-    When `spec` is not a type-union returns `{spec,}`.
-
-    '''
-    spec_subtypes: set[Union[Type]] = set(
-         getattr(
-             spec,
-             '__args__',
-             {spec,},
-         )
-    )
-    return spec_subtypes
-
-
-def mk_msgspec_table(
-    dec: msgpack.Decoder,
-    msg: MsgType|None = None,
-
-) -> dict[str, MsgType]|str:
-    '''
-    Fill out a `dict` of `MsgType`s keyed by name
-    for a given input `msgspec.msgpack.Decoder`
-    as defined by its `.type: Union[Type]` setting.
-
-    If `msg` is provided, only deliver a `dict` with a single
-    entry for that type.
-
-    '''
-    msgspec: Union[Type]|Type = dec.type
-
-    if not (msgtypes := getattr(msgspec, '__args__', False)):
-        msgtypes = [msgspec]
-
-    msgt_table: dict[str, MsgType] = {
-        msgt: str(msgt.__name__)
-        for msgt in msgtypes
-    }
-    if msg:
-        msgt: MsgType = type(msg)
-        str_repr: str = msgt_table[msgt]
-        return {msgt: str_repr}
-
-    return msgt_table
-
-
-def pformat_msgspec(
-    codec: MsgCodec|MsgDec,
-    msg: MsgType|None = None,
-    join_char: str = '\n',
-
-) -> str:
-    '''
-    Pretty `str` format the `msgspec.msgpack.Decoder.type` attribute
-    for display in (console) log messages as a nice (maybe multiline)
-    presentation of all supported `Struct`s (subtypes) available for
-    typed decoding.
-
-    '''
-    dec: msgpack.Decoder = getattr(codec, 'dec', codec)
-    return join_char.join(
-        mk_msgspec_table(
-            dec=dec,
-            msg=msg,
-        ).values()
-    )
-
-# TODO: overall IPC msg-spec features (i.e. in this mod)!
-#
-# -[ ] API changes towards being interchange lib agnostic!
-#   -[ ] capnproto has pre-compiled schema for eg..
-#    * https://capnproto.org/language.html
-#    * http://capnproto.github.io/pycapnp/quickstart.html
-#     * https://github.com/capnproto/pycapnp/blob/master/examples/addressbook.capnp
-#
-# -[ ] struct aware messaging coders as per:
-#   -[x] https://github.com/goodboy/tractor/issues/36
-#   -[ ] https://github.com/goodboy/tractor/issues/196
-#   -[ ] https://github.com/goodboy/tractor/issues/365
-#
-class MsgCodec(Struct):
-    '''
-    A IPC msg interchange format lib's encoder + decoder pair.
-
-    Pretty much nothing more then delegation to underlying
-    `msgspec.<interchange-protocol>.Encoder/Decoder`s for now.
-
-    '''
-    _enc: msgpack.Encoder
-    _dec: msgpack.Decoder
-    _pld_spec: Type[Struct]|Raw|Any
-
-    # _ext_types_box: Struct|None = None
-
-    def __repr__(self) -> str:
-        speclines: str = textwrap.indent(
-            pformat_msgspec(codec=self),
-            prefix=' '*3,
-        )
-        body: str = textwrap.indent(
-            f'|_lib = {self.lib.__name__!r}\n'
-            f'|_enc_hook: {self.enc.enc_hook}\n'
-            f'|_dec_hook: {self.dec.dec_hook}\n'
-            f'|_pld_spec: {self.pld_spec_str}\n'
-            # f'|\n'
-            f'|__msg_spec__:\n'
-            f'{speclines}\n',
-            prefix=' '*2,
-        )
-        return (
-            f'<{type(self).__name__}(\n'
-            f'{body}'
-            ')>'
-        )
-
-    @property
-    def pld_spec(self) -> Type[Struct]|Raw|Any:
-        return self._pld_spec
-
-    @property
-    def pld_spec_str(self) -> str:
-
-        # TODO: could also use match: instead?
-        spec: Union[Type]|Type = self.pld_spec
-
-        # `typing.Union` case
-        if getattr(spec, '__args__', False):
-            return str(spec)
-
-        # just a single type
-        else:
-            return spec.__name__
-
-    # struct type unions
-    # https://jcristharif.com/msgspec/structs.html#tagged-unions
-    @property
-    def msg_spec(self) -> Union[Type[Struct]]:
-        # NOTE: defined and applied inside `mk_codec()`
-        return self._dec.type
-
-    # TODO: some way to make `pretty_struct.Struct` use this
-    # wrapped field over the `.msg_spec` one?
-    @property
-    def msg_spec_str(self) -> str:
-        return pformat_msgspec(self.msg_spec)
-
-    lib: ModuleType = msgspec
-
-    # TODO: use `functools.cached_property` for these ?
-    # https://docs.python.org/3/library/functools.html#functools.cached_property
-    @property
-    def enc(self) -> msgpack.Encoder:
-        return self._enc
-
-    # TODO: reusing encode buffer for perf?
-    # https://jcristharif.com/msgspec/perf-tips.html#reusing-an-output-buffer
-    _buf: bytearray = bytearray()
-
-    def encode(
-        self,
-        py_obj: Any|PayloadMsg,
-
-        use_buf: bool = False,
-        # ^-XXX-^ uhh why am i getting this?
-        # |_BufferError: Existing exports of data: object cannot be re-sized
-
-        as_ext_type: bool = False,
-        hide_tb: bool = True,
-
-    ) -> bytes:
-        '''
-        Encode input python objects to `msgpack` bytes for
-        transfer on a tranport protocol connection.
-
-        When `use_buf == True` use the output buffer optimization:
-        https://jcristharif.com/msgspec/perf-tips.html#reusing-an-output-buffer
-
-        '''
-        __tracebackhide__: bool = hide_tb
-        if use_buf:
-            self._enc.encode_into(py_obj, self._buf)
-            return self._buf
-
-        return self._enc.encode(py_obj)
-        # try:
-        #     return self._enc.encode(py_obj)
-        # except TypeError as typerr:
-        #     typerr.add_note(
-        #         '|_src error from `msgspec`'
-        #         # f'|_{self._enc.encode!r}'
-        #     )
-        #     raise typerr
-
-        # TODO! REMOVE once i'm confident we won't ever need it!
-        #
-        # box: Struct = self._ext_types_box
-        # if (
-        #     as_ext_type
-        #     or
-        #     (
-        #         # XXX NOTE, auto-detect if the input type
-        #         box
-        #         and
-        #         (ext_types := unpack_spec_types(
-        #             spec=box.__annotations__['boxed'])
-        #         )
-        #     )
-        # ):
-        #     match py_obj:
-        #         # case PayloadMsg(pld=pld) if (
-        #         #     type(pld) in ext_types
-        #         # ):
-        #         #     py_obj.pld = box(boxed=py_obj)
-        #         #     breakpoint()
-        #         case _ if (
-        #             type(py_obj) in ext_types
-        #         ):
-        #             py_obj = box(boxed=py_obj)
-
-    @property
-    def dec(self) -> msgpack.Decoder:
-        return self._dec
-
-    def decode(
-        self,
-        msg: bytes,
-    ) -> Any:
-        '''
-        Decode received `msgpack` bytes into a local python object
-        with special `msgspec.Struct` (or other type) handling
-        determined by the 
-
-        '''
-        # https://jcristharif.com/msgspec/usage.html#typed-decoding
-        return self._dec.decode(msg)
-
-
-# ?TODO? time to remove this finally?
-#
-# -[x] TODO: a sub-decoder system as well?
-# => No! already re-architected to include a "payload-receiver"
-#   now found in `._ops`.
-#
-# -[x] do we still want to try and support the sub-decoder with
-# `.Raw` technique in the case that the `Generic` approach gives
-# future grief?
-# => well YES but NO, since we went with the `PldRx` approach
-#   instead!
-#
-# IF however you want to see the code that was staged for this
-# from wayyy back, see the pure removal commit.
-
-
-def mk_codec(
-    ipc_pld_spec: Union[Type[Struct]]|Any|Raw = Raw,
-    # tagged-struct-types-union set for `Decoder`ing of payloads, as
-    # per https://jcristharif.com/msgspec/structs.html#tagged-unions.
-    # NOTE that the default `Raw` here **is very intentional** since
-    # the `PldRx._pld_dec: MsgDec` is responsible for per ipc-ctx-task
-    # decoding of msg-specs defined by the user as part of **their**
-    # `tractor` "app's" type-limited IPC msg-spec.
-
-    # TODO: offering a per-msg(-field) type-spec such that
-    # the fields can be dynamically NOT decoded and left as `Raw`
-    # values which are later loaded by a sub-decoder specified
-    # by `tag_field: str` value key?
-    # payload_msg_specs: dict[
-    #     str,  # tag_field value as sub-decoder key
-    #     Union[Type[Struct]]  # `MsgType.pld` type spec
-    # ]|None = None,
-
-    libname: str = 'msgspec',
-
-    # settings for encoding-to-send extension-types,
-    # https://jcristharif.com/msgspec/extending.html#mapping-to-from-native-types
-    # dec_hook: Callable|None = None,
-    enc_hook: Callable|None = None,
-    ext_types: list[Type]|None = None,
-
-    # optionally provided msg-decoder from which we pull its,
-    # |_.dec_hook()
-    # |_.type
-    ext_dec: MsgDec|None = None
-    #
-    # ?TODO? other params we might want to support
-    # Encoder:
-    # write_buffer_size=write_buffer_size,
-    #
-    # Decoder:
-    # ext_hook: ext_hook_sig
-
-) -> MsgCodec:
-    '''
-    Convenience factory for creating codecs eventually meant
-    to be interchange lib agnostic (i.e. once we support more then just
-    `msgspec` ;).
-
-    '''
-    pld_spec = ipc_pld_spec
-    if enc_hook:
-        if not ext_types:
-            raise TypeError(
-                f'If extending the serializable types with a custom encode hook (`enc_hook()`), '
-                f'you must also provide the expected type set that the hook will handle '
-                f'via a `ext_types: Union[Type]|None = None` argument!\n'
-                f'\n'
-                f'enc_hook = {enc_hook!r}\n'
-                f'ext_types = {ext_types!r}\n'
-            )
-
-    dec_hook: Callable|None = None
-    if ext_dec:
-        dec: msgspec.Decoder = ext_dec.dec
-        dec_hook = dec.dec_hook
-        pld_spec |= dec.type
-        if ext_types:
-            pld_spec |= Union[*ext_types]
-
-    # (manually) generate a msg-spec (how appropes) for all relevant
-    # payload-boxing-struct-msg-types, parameterizing the
-    # `PayloadMsg.pld: PayloadT` for the decoder such that all msgs
-    # in our SC-RPC-protocol will automatically decode to
-    # a type-"limited" payload (`Struct`) object (set).
-    (
-        ipc_msg_spec,
-        msg_types,
-    ) = mk_msg_spec(
-        payload_type_union=pld_spec,
-    )
-
-    msg_spec_types: set[Type] = unpack_spec_types(ipc_msg_spec)
-    assert (
-        len(ipc_msg_spec.__args__) == len(msg_types)
-        and
-        len(msg_spec_types) == len(msg_types)
-    )
-
-    dec = msgpack.Decoder(
-        type=ipc_msg_spec,
-        dec_hook=dec_hook,
-    )
-    enc = msgpack.Encoder(
-       enc_hook=enc_hook,
-    )
-    codec = MsgCodec(
-        _enc=enc,
-        _dec=dec,
-        _pld_spec=pld_spec,
-    )
-    # sanity on expected backend support
-    assert codec.lib.__name__ == libname
-    return codec
-
-
-# instance of the default `msgspec.msgpack` codec settings, i.e.
-# no custom structs, hooks or other special types.
-#
-# XXX NOTE XXX, this will break our `Context.start()` call!
-#
-# * by default we roundtrip the started pld-`value` and if you apply
-#   this codec (globally anyway with `apply_codec()`) then the
-#   `roundtripped` value will include a non-`.pld: Raw` which will
-#   then type-error on the consequent `._ops.validte_payload_msg()`..
-#
-_def_msgspec_codec: MsgCodec = mk_codec(
-    ipc_pld_spec=Any,
-)
-
-# The built-in IPC `Msg` spec.
-# Our composing "shuttle" protocol which allows `tractor`-app code
-# to use any `msgspec` supported type as the `PayloadMsg.pld` payload,
-# https://jcristharif.com/msgspec/supported-types.html
-#
-_def_tractor_codec: MsgCodec = mk_codec(
-    ipc_pld_spec=Raw,  # XXX should be default righ!?
-)
-
-# -[x] TODO, IDEALLY provides for per-`trio.Task` specificity of the
-# IPC msging codec used by the transport layer when doing
-# `Channel.send()/.recv()` of wire data.
-# => impled as our `PldRx` which is `Context` scoped B)
-
-# ContextVar-TODO: DIDN'T WORK, kept resetting in every new task to default!?
-# _ctxvar_MsgCodec: ContextVar[MsgCodec] = ContextVar(
-
-# TreeVar-TODO: DIDN'T WORK, kept resetting in every new embedded nursery
-# even though it's supposed to inherit from a parent context ???
-#
-# _ctxvar_MsgCodec: TreeVar[MsgCodec] = TreeVar(
-#
-# ^-NOTE-^: for this to work see the mods by @mikenerone from `trio` gitter:
-#
-# 22:02:54 <mikenerone> even for regular contextvars, all you have to do is:
-#    `task: Task = trio.lowlevel.current_task()`
-#    `task.parent_nursery.parent_task.context.run(my_ctx_var.set, new_value)`
-#
-# From a comment in his prop code he couldn't share outright:
-# 1. For every TreeVar set in the current task (which covers what
-#    we need from SynchronizerFacade), walk up the tree until the
-#    root or finding one where the TreeVar is already set, setting
-#    it in all of the contexts along the way.
-# 2. For each of those, we also forcibly set the values that are
-#    pending for child nurseries that have not yet accessed the
-#    TreeVar.
-# 3. We similarly set the pending values for the child nurseries
-#    of the *current* task.
-#
-_ctxvar_MsgCodec: ContextVar[MsgCodec] = ContextVar(
-    'msgspec_codec',
-    default=_def_tractor_codec,
-)
-
-
-@cm
-def apply_codec(
-    codec: MsgCodec,
-
-    ctx: Context|None = None,
-
-) -> MsgCodec:
-    '''
-    Dynamically apply a `MsgCodec` to the current task's runtime
-    context such that all (of a certain class of payload
-    containing i.e. `MsgType.pld: PayloadT`) IPC msgs are
-    processed with it for that task.
-
-    Uses a `contextvars.ContextVar` to ensure the scope of any
-    codec setting matches the current `Context` or
-    `._rpc.process_messages()` feeder task's prior setting without
-    mutating any surrounding scope.
-
-    When a `ctx` is supplied, only mod its `Context.pld_codec`.
-
-    matches the `@cm` block and DOES NOT change to the original
-    (default) value in new tasks (as it does for `ContextVar`).
-
-    '''
-    __tracebackhide__: bool = True
-
-    if ctx is not None:
-        var: ContextVar = ctx._var_pld_codec
-    else:
-        # use IPC channel-connection "global" codec
-        var: ContextVar = _ctxvar_MsgCodec
-
-    orig: MsgCodec = var.get()
-
-    assert orig is not codec
-    if codec.pld_spec is None:
-        breakpoint()
-
-    log.info(
-        'Applying new msg-spec codec\n\n'
-        f'{codec}\n'
-    )
-    token: Token = var.set(codec)
-
-    try:
-        yield var.get()
-    finally:
-        var.reset(token)
-        log.info(
-            'Reverted to last msg-spec codec\n\n'
-            f'{orig}\n'
-        )
-        assert var.get() is orig
-
-    # ?TODO? for TreeVar approach which copies from the
-    # cancel-scope of the prior value, NOT the prior task
-    #
-    # See the docs:
-    # - https://tricycle.readthedocs.io/en/latest/reference.html#tree-variables
-    # - https://github.com/oremanj/tricycle/blob/master/tricycle/_tests/test_tree_var.py
-    #   ^- see docs for @cm `.being()` API
-    #
-    # with _ctxvar_MsgCodec.being(codec):
-    #     new = _ctxvar_MsgCodec.get()
-    #     assert new is codec
-    #     yield codec
-
-
-def current_codec() -> MsgCodec:
-    '''
-    Return the current `trio.Task.context`'s value
-    for `msgspec_codec` used by `Channel.send/.recv()`
-    for wire serialization.
-
-    '''
-    return _ctxvar_MsgCodec.get()
-
-
-@cm
-def limit_msg_spec(
-    payload_spec: Union[Type[Struct]],
-
-    # TODO: don't need this approach right?
-    # -> related to the `MsgCodec._payload_decs` stuff above..
-    # tagged_structs: list[Struct]|None = None,
-
-    hide_tb: bool = True,
-    **codec_kwargs,
-
-) -> MsgCodec:
-    '''
-    Apply a `MsgCodec` that will natively decode the SC-msg set's
-    `PayloadMsg.pld: Union[Type[Struct]]` payload fields using
-    tagged-unions of `msgspec.Struct`s from the `payload_types`
-    for all IPC contexts in use by the current `trio.Task`.
-
-    '''
-    __tracebackhide__: bool = hide_tb
-    curr_codec: MsgCodec = current_codec()
-    msgspec_codec: MsgCodec = mk_codec(
-        ipc_pld_spec=payload_spec,
-        **codec_kwargs,
-    )
-    with apply_codec(msgspec_codec) as applied_codec:
-        assert applied_codec is msgspec_codec
-        yield msgspec_codec
-
-    assert curr_codec is current_codec()
-
-
-# XXX: msgspec won't allow this with non-struct custom types
-# like `NamespacePath`!@!
-# @cm
-# def extend_msg_spec(
-#     payload_spec: Union[Type[Struct]],
-
-# ) -> MsgCodec:
-#     '''
-#     Extend the current `MsgCodec.pld_spec` (type set) by extending
-#     the payload spec to **include** the types specified by
-#     `payload_spec`.
-
-#     '''
-#     codec: MsgCodec = current_codec()
-#     pld_spec: Union[Type] = codec.pld_spec
-#     extended_spec: Union[Type] = pld_spec|payload_spec
-
-#     with limit_msg_spec(payload_types=extended_spec) as ext_codec:
-#         # import pdbp; pdbp.set_trace()
-#         assert ext_codec.pld_spec == extended_spec
-#         yield ext_codec
-#
-# ^-TODO-^ is it impossible to make something like this orr!?
-
-# TODO: make an auto-custom hook generator from a set of input custom
-# types?
-# -[ ] below is a proto design using a `TypeCodec` idea?
-#
-# type var for the expected interchange-lib's
-# IPC-transport type when not available as a built-in
-# serialization output.
-WireT = TypeVar('WireT')
-
-
-# TODO: some kinda (decorator) API for built-in subtypes
-# that builds this implicitly by inspecting the `mro()`?
-class TypeCodec(Protocol):
-    '''
-    A per-custom-type wire-transport serialization translator
-    description type.
-
-    '''
-    src_type: Type
-    wire_type: WireT
-
-    def encode(obj: Type) -> WireT:
-        ...
-
-    def decode(
-        obj_type: Type[WireT],
-        obj: WireT,
-    ) -> Type:
-        ...
-
-
-class MsgpackTypeCodec(TypeCodec):
-    ...
-
-
-def mk_codec_hooks(
-    type_codecs: list[TypeCodec],
-
-) -> tuple[Callable, Callable]:
-    '''
-    Deliver a `enc_hook()`/`dec_hook()` pair which handle
-    manual convertion from an input `Type` set such that whenever
-    the `TypeCodec.filter()` predicate matches the
-    `TypeCodec.decode()` is called on the input native object by
-    the `dec_hook()` and whenever the
-    `isiinstance(obj, TypeCodec.type)` matches against an
-    `enc_hook(obj=obj)` the return value is taken from a
-    `TypeCodec.encode(obj)` callback.
-
-    '''
-    ...

tractor/msg/_exts.py

@@ -1,94 +0,0 @@
-# tractor: structured concurrent "actors".
-# Copyright 2018-eternity Tyler Goodlet.
-
-# This program is free software: you can redistribute it and/or modify
-# it under the terms of the GNU Affero General Public License as published by
-# the Free Software Foundation, either version 3 of the License, or
-# (at your option) any later version.
-
-# This program is distributed in the hope that it will be useful,
-# but WITHOUT ANY WARRANTY; without even the implied warranty of
-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
-# GNU Affero General Public License for more details.
-
-# You should have received a copy of the GNU Affero General Public License
-# along with this program.  If not, see <https://www.gnu.org/licenses/>.
-
-'''
-Type-extension-utils for codec-ing (python) objects not
-covered by the `msgspec.msgpack` protocol.
-
-See the various API docs from `msgspec`.
-
-extending from native types,
-- https://jcristharif.com/msgspec/extending.html#mapping-to-from-native-types
-
-converters,
-- https://jcristharif.com/msgspec/converters.html
-- https://jcristharif.com/msgspec/api.html#msgspec.convert
-
-`Raw` fields,
-- https://jcristharif.com/msgspec/api.html#raw
-- support for `.convert()` and `Raw`,
-  |_ https://jcristharif.com/msgspec/changelog.html
-
-'''
-from types import (
-    ModuleType,
-)
-import typing
-from typing import (
-    Type,
-    Union,
-)
-
-def dec_type_union(
-    type_names: list[str],
-    mods: list[ModuleType] = []
-) -> Type|Union[Type]:
-    '''
-    Look up types by name, compile into a list and then create and
-    return a `typing.Union` from the full set.
-
-    '''
-    # import importlib
-    types: list[Type] = []
-    for type_name in type_names:
-        for mod in [
-            typing,
-            # importlib.import_module(__name__),
-        ] + mods:
-            if type_ref := getattr(
-                mod,
-                type_name,
-                False,
-            ):
-                types.append(type_ref)
-
-    # special case handling only..
-    # ipc_pld_spec: Union[Type] = eval(
-    #     pld_spec_str,
-    #     {},  # globals
-    #     {'typing': typing},  # locals
-    # )
-
-    return Union[*types]
-
-
-def enc_type_union(
-    union_or_type: Union[Type]|Type,
-) -> list[str]:
-    '''
-    Encode a type-union or single type to a list of type-name-strings
-    ready for IPC interchange.
-
-    '''
-    type_strs: list[str] = []
-    for typ in getattr(
-        union_or_type,
-        '__args__',
-        {union_or_type,},
-    ):
-        type_strs.append(typ.__qualname__)
-
-    return type_strs

tractor/msg/_ops.py

@@ -1,905 +0,0 @@
-# tractor: structured concurrent "actors".
-# Copyright 2018-eternity Tyler Goodlet.
-
-# This program is free software: you can redistribute it and/or modify
-# it under the terms of the GNU Affero General Public License as published by
-# the Free Software Foundation, either version 3 of the License, or
-# (at your option) any later version.
-
-# This program is distributed in the hope that it will be useful,
-# but WITHOUT ANY WARRANTY; without even the implied warranty of
-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
-# GNU Affero General Public License for more details.
-
-# You should have received a copy of the GNU Affero General Public License
-# along with this program.  If not, see <https://www.gnu.org/licenses/>.
-
-'''
-Near-application abstractions for `MsgType.pld: PayloadT|Raw`
-delivery, filtering and type checking as well as generic
-operational helpers for processing transaction flows.
-
-'''
-from __future__ import annotations
-from contextlib import (
-    asynccontextmanager as acm,
-    contextmanager as cm,
-)
-from typing import (
-    Any,
-    Callable,
-    Type,
-    TYPE_CHECKING,
-    Union,
-)
-# ------ - ------
-from msgspec import (
-    msgpack,
-    Raw,
-    Struct,
-    ValidationError,
-)
-import trio
-# ------ - ------
-from tractor.log import get_logger
-from tractor._exceptions import (
-    MessagingError,
-    InternalError,
-    _raise_from_unexpected_msg,
-    MsgTypeError,
-    _mk_recv_mte,
-    pack_error,
-)
-from tractor._state import (
-    current_ipc_ctx,
-)
-from ._codec import (
-    mk_dec,
-    MsgDec,
-    MsgCodec,
-    current_codec,
-)
-from .types import (
-    CancelAck,
-    Error,
-    MsgType,
-    PayloadT,
-    Return,
-    Started,
-    Stop,
-    Yield,
-    pretty_struct,
-)
-
-
-if TYPE_CHECKING:
-    from tractor._context import Context
-    from tractor._streaming import MsgStream
-
-
-log = get_logger(__name__)
-
-
-_def_any_pldec: MsgDec[Any] = mk_dec(spec=Any)
-
-
-class PldRx(Struct):
-    '''
-    A "msg payload receiver".
-
-    The pairing of a "feeder" `trio.abc.ReceiveChannel` and an
-    interchange-specific (eg. msgpack) payload field decoder. The
-    validation/type-filtering rules are runtime mutable and allow
-    type constraining the set of `MsgType.pld: Raw|PayloadT`
-    values at runtime, per IPC task-context.
-
-    This abstraction, being just below "user application code",
-    allows for the equivalent of our `MsgCodec` (used for
-    typer-filtering IPC dialog protocol msgs against a msg-spec)
-    but with granular control around payload delivery (i.e. the
-    data-values user code actually sees and uses (the blobs that
-    are "shuttled" by the wrapping dialog prot) such that invalid
-    `.pld: Raw` can be decoded and handled by IPC-primitive user
-    code (i.e. that operates on `Context` and `Msgstream` APIs)
-    without knowledge of the lower level `Channel`/`MsgTransport`
-    primitives nor the `MsgCodec` in use. Further, lazily decoding
-    payload blobs allows for topical (and maybe intentionally
-    "partial") encryption of msg field subsets.
-
-    '''
-    # TODO: better to bind it here?
-    # _rx_mc: trio.MemoryReceiveChannel
-    _pld_dec: MsgDec
-
-    @property
-    def pld_dec(self) -> MsgDec:
-        return self._pld_dec
-
-    @cm
-    def limit_plds(
-        self,
-        spec: Union[Type[Struct]],
-        **dec_kwargs,
-
-    ) -> MsgDec:
-        '''
-        Type-limit the loadable msg payloads via an applied
-        `MsgDec` given an input spec, revert to prior decoder on
-        exit.
-
-        '''
-        # TODO, ensure we pull the current `MsgCodec`'s custom
-        # dec/enc_hook settings as well ?
-        # -[ ] see `._codec.mk_codec()` inputs
-        #
-        orig_dec: MsgDec = self._pld_dec
-        limit_dec: MsgDec = mk_dec(
-            spec=spec,
-            **dec_kwargs,
-        )
-        try:
-            self._pld_dec = limit_dec
-            yield limit_dec
-        finally:
-            self._pld_dec = orig_dec
-
-    @property
-    def dec(self) -> msgpack.Decoder:
-        return self._pld_dec.dec
-
-    def recv_msg_nowait(
-        self,
-        # TODO: make this `MsgStream` compat as well, see above^
-        # ipc_prim: Context|MsgStream,
-        ipc: Context|MsgStream,
-
-        ipc_msg: MsgType|None = None,
-        expect_msg: Type[MsgType]|None = None,
-        hide_tb: bool = False,
-        **dec_pld_kwargs,
-
-    ) -> tuple[
-        MsgType[PayloadT],
-        PayloadT,
-    ]:
-        '''
-        Attempt to non-blocking receive a message from the `._rx_chan` and
-        unwrap it's payload delivering the pair to the caller.
-
-        '''
-        __tracebackhide__: bool = hide_tb
-
-        msg: MsgType = (
-            ipc_msg
-            or
-            # sync-rx msg from underlying IPC feeder (mem-)chan
-            ipc._rx_chan.receive_nowait()
-        )
-        pld: PayloadT = self.decode_pld(
-            msg,
-            ipc=ipc,
-            expect_msg=expect_msg,
-            hide_tb=hide_tb,
-            **dec_pld_kwargs,
-        )
-        return (
-            msg,
-            pld,
-        )
-
-    async def recv_msg(
-        self,
-        ipc: Context|MsgStream,
-        expect_msg: MsgType,
-
-        # NOTE: ONLY for handling `Stop`-msgs that arrive during
-        # a call to `drain_to_final_msg()` above!
-        passthrough_non_pld_msgs: bool = True,
-        hide_tb: bool = True,
-
-        **decode_pld_kwargs,
-
-    ) -> tuple[MsgType, PayloadT]:
-        '''
-        Retrieve the next avail IPC msg, decode its payload, and
-        return the (msg, pld) pair.
-
-        '''
-        __tracebackhide__: bool = hide_tb
-        msg: MsgType = await ipc._rx_chan.receive()
-        match msg:
-            case Return()|Error():
-                log.runtime(
-                    f'Rxed final outcome msg\n'
-                    f'{msg}\n'
-                )
-            case Stop():
-                log.runtime(
-                    f'Rxed stream stopped msg\n'
-                    f'{msg}\n'
-                )
-                if passthrough_non_pld_msgs:
-                    return msg, None
-
-        # TODO: is there some way we can inject the decoded
-        # payload into an existing output buffer for the original
-        # msg instance?
-        pld: PayloadT = self.decode_pld(
-            msg,
-            ipc=ipc,
-            expect_msg=expect_msg,
-            hide_tb=hide_tb,
-
-            **decode_pld_kwargs,
-        )
-        return (
-            msg,
-            pld,
-        )
-
-    async def recv_pld(
-        self,
-        ipc: Context|MsgStream,
-        ipc_msg: MsgType[PayloadT]|None = None,
-        expect_msg: Type[MsgType]|None = None,
-        hide_tb: bool = True,
-
-        **dec_pld_kwargs,
-
-    ) -> PayloadT:
-        '''
-        Receive a `MsgType`, then decode and return its `.pld` field.
-
-        '''
-        __tracebackhide__: bool = hide_tb
-        msg: MsgType = (
-            ipc_msg
-            or
-            # async-rx msg from underlying IPC feeder (mem-)chan
-            await ipc._rx_chan.receive()
-        )
-        if (
-            type(msg) is Return
-        ):
-            log.info(
-                f'Rxed final result msg\n'
-                f'{msg}\n'
-            )
-        return self.decode_pld(
-            msg=msg,
-            ipc=ipc,
-            expect_msg=expect_msg,
-            **dec_pld_kwargs,
-        )
-
-    def decode_pld(
-        self,
-        msg: MsgType,
-        ipc: Context|MsgStream,
-        expect_msg: Type[MsgType]|None,
-
-        raise_error: bool = True,
-        hide_tb: bool = True,
-
-        # XXX for special (default?) case of send side call with
-        # `Context.started(validate_pld_spec=True)`
-        is_started_send_side: bool = False,
-
-    ) -> PayloadT|Raw:
-        '''
-        Decode a msg's payload field: `MsgType.pld: PayloadT|Raw` and
-        return the value or raise an appropriate error.
-
-        '''
-        __tracebackhide__: bool = hide_tb
-        src_err: BaseException|None = None
-        match msg:
-            # payload-data shuttle msg; deliver the `.pld` value
-            # directly to IPC (primitive) client-consumer code.
-            case (
-                Started(pld=pld)  # sync phase
-                |Yield(pld=pld)  # streaming phase
-                |Return(pld=pld)  # termination phase
-            ):
-                try:
-                    pld: PayloadT = self._pld_dec.decode(pld)
-                    log.runtime(
-                        'Decoded msg payload\n\n'
-                        f'{msg}\n'
-                        f'where payload decoded as\n'
-                        f'|_pld={pld!r}\n'
-                    )
-                    return pld
-                except TypeError as typerr:
-                    __tracebackhide__: bool = False
-                    raise typerr
-
-                # XXX pld-value type failure
-                except ValidationError as valerr:
-                    # pack mgterr into error-msg for
-                    # reraise below; ensure remote-actor-err
-                    # info is displayed nicely?
-                    mte: MsgTypeError = _mk_recv_mte(
-                        msg=msg,
-                        codec=self.pld_dec,
-                        src_validation_error=valerr,
-                        is_invalid_payload=True,
-                        expected_msg=expect_msg,
-                    )
-                    # NOTE: just raise the MTE inline instead of all
-                    # the pack-unpack-repack non-sense when this is
-                    # a "send side" validation error.
-                    if is_started_send_side:
-                        raise mte
-
-                    # NOTE: the `.message` is automatically
-                    # transferred into the message as long as we
-                    # define it as a `Error.message` field.
-                    err_msg: Error = pack_error(
-                        exc=mte,
-                        cid=msg.cid,
-                        src_uid=(
-                            ipc.chan.uid
-                            if not is_started_send_side
-                            else ipc._actor.uid
-                        ),
-                    )
-                    mte._ipc_msg = err_msg
-
-                    # XXX override the `msg` passed to
-                    # `_raise_from_unexpected_msg()` (below) so so
-                    # that we're effectively able to use that same
-                    # func to unpack and raise an "emulated remote
-                    # `Error`" of this local MTE.
-                    msg = err_msg
-                    # XXX NOTE: so when the `_raise_from_unexpected_msg()`
-                    # raises the boxed `err_msg` from above it raises
-                    # it from the above caught interchange-lib
-                    # validation error.
-                    src_err = valerr
-
-            # a runtime-internal RPC endpoint response.
-            # always passthrough since (internal) runtime
-            # responses are generally never exposed to consumer
-            # code.
-            case CancelAck(
-                pld=bool(cancelled)
-            ):
-                return cancelled
-
-            case Error():
-                src_err = MessagingError(
-                    'IPC ctx dialog terminated without `Return`-ing a result\n'
-                    f'Instead it raised {msg.boxed_type_str!r}!'
-                )
-                # XXX NOTE XXX another super subtle runtime-y thing..
-                #
-                # - when user code (transitively) calls into this
-                #   func (usually via a `Context/MsgStream` API) we
-                #   generally want errors to propagate immediately
-                #   and directly so that the user can define how it
-                #   wants to handle them.
-                #
-                #  HOWEVER,
-                #
-                # - for certain runtime calling cases, we don't want to
-                #   directly raise since the calling code might have
-                #   special logic around whether to raise the error
-                #   or supress it silently (eg. a `ContextCancelled`
-                #   received from the far end which was requested by
-                #   this side, aka a self-cancel).
-                #
-                # SO, we offer a flag to control this.
-                if not raise_error:
-                    return src_err
-
-            case Stop(cid=cid):
-                ctx: Context = getattr(ipc, 'ctx', ipc)
-                message: str = (
-                    f'{ctx.side!r}-side of ctx received stream-`Stop` from '
-                    f'{ctx.peer_side!r} peer ?\n'
-                    f'|_cid: {cid}\n\n'
-
-                    f'{pretty_struct.pformat(msg)}\n'
-                )
-                if ctx._stream is None:
-                    explain: str = (
-                        f'BUT, no `MsgStream` (was) open(ed) on this '
-                        f'{ctx.side!r}-side of the IPC ctx?\n'
-                        f'Maybe check your code for streaming phase race conditions?\n'
-                    )
-                    log.warning(
-                        message
-                        +
-                        explain
-                    )
-                    # let caller decide what to do when only one
-                    # side opened a stream, don't raise.
-                    return msg
-
-                else:
-                    explain: str = (
-                        'Received a `Stop` when it should NEVER be possible!?!?\n'
-                    )
-                    # TODO: this is constructed inside
-                    # `_raise_from_unexpected_msg()` but maybe we
-                    # should pass it in?
-                    # src_err = trio.EndOfChannel(explain)
-                    src_err = None
-
-            case _:
-                src_err = InternalError(
-                    'Invalid IPC msg ??\n\n'
-                    f'{msg}\n'
-                )
-
-        # TODO: maybe use the new `.add_note()` from 3.11?
-        # |_https://docs.python.org/3.11/library/exceptions.html#BaseException.add_note
-        #
-        # fallthrough and raise from `src_err`
-        try:
-            _raise_from_unexpected_msg(
-                ctx=getattr(ipc, 'ctx', ipc),
-                msg=msg,
-                src_err=src_err,
-                log=log,
-                expect_msg=expect_msg,
-                hide_tb=hide_tb,
-            )
-        except UnboundLocalError:
-            # XXX if there's an internal lookup error in the above
-            # code (prolly on `src_err`) we want to show this frame
-            # in the tb!
-            __tracebackhide__: bool = False
-            raise
-
-
-@cm
-def limit_plds(
-    spec: Union[Type[Struct]],
-    **dec_kwargs,
-
-) -> MsgDec:
-    '''
-    Apply a `MsgCodec` that will natively decode the SC-msg set's
-    `PayloadMsg.pld: Union[Type[Struct]]` payload fields using
-    tagged-unions of `msgspec.Struct`s from the `payload_types`
-    for all IPC contexts in use by the current `trio.Task`.
-
-    '''
-    __tracebackhide__: bool = True
-    curr_ctx: Context|None = current_ipc_ctx()
-    if curr_ctx is None:
-        raise RuntimeError(
-            'No IPC `Context` is active !?\n'
-            'Did you open `limit_plds()` from outside '
-            'a `Portal.open_context()` scope-block?'
-        )
-    try:
-        rx: PldRx = curr_ctx._pld_rx
-        orig_pldec: MsgDec = rx.pld_dec
-        with rx.limit_plds(
-            spec=spec,
-            **dec_kwargs,
-        ) as pldec:
-            log.runtime(
-                'Applying payload-decoder\n\n'
-                f'{pldec}\n'
-            )
-            yield pldec
-
-    except BaseException:
-        __tracebackhide__: bool = False
-        raise
-
-    finally:
-        log.runtime(
-            'Reverted to previous payload-decoder\n\n'
-            f'{orig_pldec}\n'
-        )
-        # sanity on orig settings
-        assert rx.pld_dec is orig_pldec
-
-
-@acm
-async def maybe_limit_plds(
-    ctx: Context,
-    spec: Union[Type[Struct]]|None = None,
-    dec_hook: Callable|None = None,
-    **kwargs,
-
-) -> MsgDec|None:
-    '''
-    Async compat maybe-payload type limiter.
-
-    Mostly for use inside other internal `@acm`s such that a separate
-    indent block isn't needed when an async one is already being
-    used.
-
-    '''
-    if (
-        spec is None
-        and
-        dec_hook is None
-    ):
-        yield None
-        return
-
-    # sanity check on IPC scoping
-    curr_ctx: Context = current_ipc_ctx()
-    assert ctx is curr_ctx
-
-    with ctx._pld_rx.limit_plds(
-        spec=spec,
-        dec_hook=dec_hook,
-        **kwargs,
-    ) as msgdec:
-        yield msgdec
-
-    # when the applied spec is unwound/removed, the same IPC-ctx
-    # should still be in scope.
-    curr_ctx: Context = current_ipc_ctx()
-    assert ctx is curr_ctx
-
-
-async def drain_to_final_msg(
-    ctx: Context,
-
-    msg_limit: int = 6,
-    hide_tb: bool = True,
-
-) -> tuple[
-    Return|None,
-    list[MsgType]
-]:
-    '''
-    Drain IPC msgs delivered to the underlying IPC context's
-    rx-mem-chan (i.e. from `Context._rx_chan`) in search for a final
-    `Return` or `Error` msg.
-
-    Deliver the `Return` + preceding drained msgs (`list[MsgType]`)
-    as a pair unless an `Error` is found, in which unpack and raise
-    it.
-
-    The motivation here is to always capture any remote error relayed
-    by the remote peer task during a ctxc condition.
-
-    For eg. a ctxc-request may be sent to the peer as part of the
-    local task's (request for) cancellation but then that same task
-    **also errors** before executing the teardown in the
-    `Portal.open_context().__aexit__()` block. In such error-on-exit
-    cases we want to always capture and raise any delivered remote
-    error (like an expected ctxc-ACK) as part of the final
-    `ctx.wait_for_result()` teardown sequence such that the
-    `Context.outcome` related state always reflect what transpired
-    even after ctx closure and the `.open_context()` block exit.
-
-    '''
-    raise_overrun: bool = not ctx._allow_overruns
-    parent_never_opened_stream: bool = ctx._stream is None
-
-    # wait for a final context result by collecting (but
-    # basically ignoring) any bi-dir-stream msgs still in transit
-    # from the far end.
-    pre_result_drained: list[MsgType] = []
-    result_msg: Return|Error|None = None
-    while not (
-        ctx.maybe_error
-        and
-        not ctx._final_result_is_set()
-    ):
-        try:
-            # receive all msgs, scanning for either a final result
-            # or error; the underlying call should never raise any
-            # remote error directly!
-            msg, pld = await ctx._pld_rx.recv_msg(
-                ipc=ctx,
-                expect_msg=Return,
-                raise_error=False,
-                hide_tb=hide_tb,
-            )
-            # ^-TODO-^ some bad ideas?
-            # -[ ] wrap final outcome .receive() in a scope so
-            #     it can be cancelled out of band if needed?
-            # |_with trio.CancelScope() as res_cs:
-            #       ctx._res_scope = res_cs
-            #       msg: dict = await ctx._rx_chan.receive()
-            #   if res_cs.cancelled_caught:
-            #
-            # -[ ] make sure pause points work here for REPLing
-            #   the runtime itself; i.e. ensure there's no hangs!
-            # |_from tractor.devx._debug import pause
-            #   await pause()
-
-        # NOTE: we get here if the far end was
-        # `ContextCancelled` in 2 cases:
-        # 1. we requested the cancellation and thus
-        #    SHOULD NOT raise that far end error,
-        # 2. WE DID NOT REQUEST that cancel and thus
-        #    SHOULD RAISE HERE!
-        except trio.Cancelled as _taskc:
-            taskc: trio.Cancelled = _taskc
-
-            # report when the cancellation wasn't (ostensibly) due to
-            # RPC operation, some surrounding parent cancel-scope.
-            if not ctx._scope.cancel_called:
-                task: trio.lowlevel.Task = trio.lowlevel.current_task()
-                rent_n: trio.Nursery = task.parent_nursery
-                if (
-                    (local_cs := rent_n.cancel_scope).cancel_called
-                ):
-                    log.cancel(
-                        'RPC-ctx cancelled by local-parent scope during drain!\n\n'
-                        f'c}}>\n'
-                        f' |_{rent_n}\n'
-                        f'   |_.cancel_scope = {local_cs}\n'
-                        f'   |_>c}}\n'
-                        f'      |_{ctx.pformat(indent=" "*9)}'
-                        # ^TODO, some (other) simpler repr here?
-                    )
-                    __tracebackhide__: bool = False
-
-            else:
-                log.cancel(
-                    f'IPC ctx cancelled externally during result drain ?\n'
-                    f'{ctx}'
-                )
-            # CASE 2: mask the local cancelled-error(s)
-            # only when we are sure the remote error is
-            # the source cause of this local task's
-            # cancellation.
-            ctx.maybe_raise(
-                hide_tb=hide_tb,
-                from_src_exc=taskc,
-                # ?TODO? when *should* we use this?
-            )
-
-            # CASE 1: we DID request the cancel we simply
-            # continue to bubble up as normal.
-            raise taskc
-
-        match msg:
-
-            # final result arrived!
-            case Return():
-                log.runtime(
-                    'Context delivered final draining msg:\n'
-                    f'{pretty_struct.pformat(msg)}'
-                )
-                ctx._result: Any = pld
-                result_msg = msg
-                break
-
-            # far end task is still streaming to us so discard
-            # and report depending on local ctx state.
-            case Yield():
-                pre_result_drained.append(msg)
-                if (
-                    not parent_never_opened_stream
-                    and (
-                        (ctx._stream.closed
-                         and
-                         (reason := 'stream was already closed')
-                        ) or
-                        (ctx.cancel_acked
-                            and
-                            (reason := 'ctx cancelled other side')
-                        )
-                        or (ctx._cancel_called
-                            and
-                            (reason := 'ctx called `.cancel()`')
-                        )
-                        or (len(pre_result_drained) > msg_limit
-                            and
-                            (reason := f'"yield" limit={msg_limit}')
-                        )
-                    )
-                ):
-                    log.cancel(
-                        'Cancelling `MsgStream` drain since '
-                        f'{reason}\n\n'
-                        f'<= {ctx.chan.uid}\n'
-                        f'  |_{ctx._nsf}()\n\n'
-                        f'=> {ctx._task}\n'
-                        f'  |_{ctx._stream}\n\n'
-
-                        f'{pretty_struct.pformat(msg)}\n'
-                    )
-                    break
-
-                # drain up to the `msg_limit` hoping to get
-                # a final result or error/ctxc.
-                else:
-                    report: str = (
-                        'Ignoring "yield" msg during `ctx.result()` drain..\n'
-                        f'<= {ctx.chan.uid}\n'
-                        f'  |_{ctx._nsf}()\n\n'
-                        f'=> {ctx._task}\n'
-                        f'  |_{ctx._stream}\n\n'
-
-                        f'{pretty_struct.pformat(msg)}\n'
-                    )
-                    if parent_never_opened_stream:
-                        report = (
-                            f'IPC ctx never opened stream on {ctx.side!r}-side!\n'
-                            f'\n'
-                            # f'{ctx}\n'
-                        ) + report
-
-                    log.warning(report)
-                    continue
-
-            # stream terminated, but no result yet..
-            #
-            # TODO: work out edge cases here where
-            # a stream is open but the task also calls
-            # this?
-            # -[ ] should be a runtime error if a stream is open right?
-            # Stop()
-            case Stop():
-                pre_result_drained.append(msg)
-                log.runtime(  # normal/expected shutdown transaction
-                    'Remote stream terminated due to "stop" msg:\n\n'
-                    f'{pretty_struct.pformat(msg)}\n'
-                )
-                continue
-
-            # remote error msg, likely already handled inside
-            # `Context._deliver_msg()`
-            case Error():
-                # TODO: can we replace this with `ctx.maybe_raise()`?
-                # -[ ]  would this be handier for this case maybe?
-                # |_async with maybe_raise_on_exit() as raises:
-                #       if raises:
-                #           log.error('some msg about raising..')
-                #
-                re: Exception|None = ctx._remote_error
-                if re:
-                    assert msg is ctx._cancel_msg
-                    # NOTE: this solved a super duper edge case XD
-                    # this was THE super duper edge case of:
-                    # - local task opens a remote task,
-                    # - requests remote cancellation of far end
-                    #   ctx/tasks,
-                    # - needs to wait for the cancel ack msg
-                    #   (ctxc) or some result in the race case
-                    #   where the other side's task returns
-                    #   before the cancel request msg is ever
-                    #   rxed and processed,
-                    # - here this surrounding drain loop (which
-                    #   iterates all ipc msgs until the ack or
-                    #   an early result arrives) was NOT exiting
-                    #   since we are the edge case: local task
-                    #   does not re-raise any ctxc it receives
-                    #   IFF **it** was the cancellation
-                    #   requester..
-                    #
-                    # XXX will raise if necessary but ow break
-                    # from loop presuming any supressed error
-                    # (ctxc) should terminate the context!
-                    ctx._maybe_raise_remote_err(
-                        re,
-                        # NOTE: obvi we don't care if we
-                        # overran the far end if we're already
-                        # waiting on a final result (msg).
-                        # raise_overrun_from_self=False,
-                        raise_overrun_from_self=raise_overrun,
-                    )
-                    result_msg = msg
-                    break  # OOOOOF, yeah obvi we need this..
-
-                else:
-                    # bubble the original src key error
-                    raise
-
-            # XXX should pretty much never get here unless someone
-            # overrides the default `MsgType` spec.
-            case _:
-                pre_result_drained.append(msg)
-                # It's definitely an internal error if any other
-                # msg type without a`'cid'` field arrives here!
-                report: str = (
-                    f'Invalid or unknown msg type {type(msg)!r}!?\n'
-                )
-                if not msg.cid:
-                    report += (
-                        '\nWhich also has no `.cid` field?\n'
-                    )
-
-                raise MessagingError(
-                    report
-                    +
-                    f'\n{msg}\n'
-                )
-
-    else:
-        log.cancel(
-            'Skipping `MsgStream` drain since final outcome is set\n\n'
-            f'{ctx.outcome}\n'
-        )
-
-    __tracebackhide__: bool = hide_tb
-    return (
-        result_msg,
-        pre_result_drained,
-    )
-
-
-def validate_payload_msg(
-    pld_msg: Started|Yield|Return,
-    pld_value: PayloadT,
-    ipc: Context|MsgStream,
-
-    raise_mte: bool = True,
-    strict_pld_parity: bool = False,
-    hide_tb: bool = True,
-
-) -> MsgTypeError|None:
-    '''
-    Validate a `PayloadMsg.pld` value with the current
-    IPC ctx's `PldRx` and raise an appropriate `MsgTypeError`
-    on failure.
-
-    '''
-    __tracebackhide__: bool = hide_tb
-    codec: MsgCodec = current_codec()
-    msg_bytes: bytes = codec.encode(pld_msg)
-    roundtripped: Started|None = None
-    try:
-        roundtripped: Started = codec.decode(msg_bytes)
-    except TypeError as typerr:
-        __tracebackhide__: bool = False
-        raise typerr
-
-    try:
-        ctx: Context = getattr(ipc, 'ctx', ipc)
-        pld: PayloadT = ctx.pld_rx.decode_pld(
-            msg=roundtripped,
-            ipc=ipc,
-            expect_msg=Started,
-            hide_tb=hide_tb,
-            is_started_send_side=True,
-        )
-        if (
-            strict_pld_parity
-            and
-            pld != pld_value
-        ):
-            # TODO: make that one a mod func too..
-            diff = pretty_struct.Struct.__sub__(
-                roundtripped,
-                pld_msg,
-            )
-            complaint: str = (
-                'Started value does not match after roundtrip?\n\n'
-                f'{diff}'
-            )
-            raise ValidationError(complaint)
-
-    # usually due to `.decode()` input type
-    except TypeError as typerr:
-        __tracebackhide__: bool = False
-        raise typerr
-
-    # raise any msg type error NO MATTER WHAT!
-    except ValidationError as verr:
-        try:
-            mte: MsgTypeError = _mk_recv_mte(
-                msg=roundtripped,
-                codec=codec,
-                src_validation_error=verr,
-                verb_header='Trying to send ',
-                is_invalid_payload=True,
-            )
-        except BaseException as _be:
-            if not roundtripped:
-                raise verr
-
-            be = _be
-            __tracebackhide__: bool = False
-            raise be
-
-        if not raise_mte:
-            return mte
-
-        raise mte from verr

tractor/msg/pretty_struct.py

@@ -1,342 +0,0 @@
-# tractor: structured concurrent "actors".
-# Copyright 2018-eternity Tyler Goodlet.
-
-# This program is free software: you can redistribute it and/or modify
-# it under the terms of the GNU Affero General Public License as published by
-# the Free Software Foundation, either version 3 of the License, or
-# (at your option) any later version.
-
-# This program is distributed in the hope that it will be useful,
-# but WITHOUT ANY WARRANTY; without even the implied warranty of
-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
-# GNU Affero General Public License for more details.
-
-# You should have received a copy of the GNU Affero General Public License
-# along with this program.  If not, see <https://www.gnu.org/licenses/>.
-
-'''
-Prettified version of `msgspec.Struct` for easier console grokin.
-
-'''
-from __future__ import annotations
-from collections import UserList
-from typing import (
-    Any,
-    Iterator,
-)
-
-from msgspec import (
-    msgpack,
-    Struct as _Struct,
-    structs,
-)
-# from pprint import (
-#     saferepr,
-# )
-
-from tractor.log import get_logger
-
-log = get_logger()
-# TODO: auto-gen type sig for input func both for
-# type-msgs and logging of RPC tasks?
-# taken and modified from:
-# https://stackoverflow.com/a/57110117
-# import inspect
-# from typing import List
-
-# def my_function(input_1: str, input_2: int) -> list[int]:
-#     pass
-
-# def types_of(func):
-#     specs = inspect.getfullargspec(func)
-#     return_type = specs.annotations['return']
-#     input_types = [t.__name__ for s, t in specs.annotations.items() if s != 'return']
-#     return f'{func.__name__}({": ".join(input_types)}) -> {return_type}'
-
-# types_of(my_function)
-
-
-class DiffDump(UserList):
-    '''
-    Very simple list delegator that repr() dumps (presumed) tuple
-    elements of the form `tuple[str, Any, Any]` in a nice
-    multi-line readable form for analyzing `Struct` diffs.
-
-    '''
-    def __repr__(self) -> str:
-        if not len(self):
-            return super().__repr__()
-
-        # format by displaying item pair's ``repr()`` on multiple,
-        # indented lines such that they are more easily visually
-        # comparable when printed to console when printed to
-        # console.
-        repstr: str = '[\n'
-        for k, left, right in self:
-            repstr += (
-                f'({k},\n'
-                f' |_{repr(left)},\n'
-                f' |_{repr(right)},\n'
-                ')\n'
-            )
-        repstr += ']\n'
-        return repstr
-
-
-def iter_fields(struct: Struct) -> Iterator[
-    tuple[
-        structs.FieldIinfo,
-        str,
-        Any,
-    ]
-]:
-    '''
-    Iterate over all non-@property fields of this struct.
-
-    '''
-    fi: structs.FieldInfo
-    for fi in structs.fields(struct):
-        key: str = fi.name
-        val: Any = getattr(struct, key)
-        yield (
-            fi,
-            key,
-            val,
-        )
-
-
-def pformat(
-    struct: Struct,
-    field_indent: int = 2,
-    indent: int = 0,
-
-) -> str:
-    '''
-    Recursion-safe `pprint.pformat()` style formatting of
-    a `msgspec.Struct` for sane reading by a human using a REPL.
-
-    '''
-    # global whitespace indent
-    ws: str = ' '*indent
-
-    # field whitespace indent
-    field_ws: str = ' '*(field_indent + indent)
-
-    # qtn: str = ws + struct.__class__.__qualname__
-    qtn: str = struct.__class__.__qualname__
-
-    obj_str: str = ''  # accumulator
-    fi: structs.FieldInfo
-    k: str
-    v: Any
-    for fi, k, v in iter_fields(struct):
-
-        # TODO: how can we prefer `Literal['option1',  'option2,
-        # ..]` over .__name__ == `Literal` but still get only the
-        # latter for simple types like `str | int | None` etc..?
-        ft: type = fi.type
-        typ_name: str = getattr(ft, '__name__', str(ft))
-
-        # recurse to get sub-struct's `.pformat()` output Bo
-        if isinstance(v, Struct):
-            val_str: str =  v.pformat(
-                indent=field_indent + indent,
-                field_indent=indent + field_indent,
-            )
-
-        else:
-            val_str: str = repr(v)
-
-            # XXX LOL, below just seems to be f#$%in causing
-            # recursion errs..
-            #
-            # the `pprint` recursion-safe format:
-            # https://docs.python.org/3.11/library/pprint.html#pprint.saferepr
-            # try:
-            #     val_str: str = saferepr(v)
-            # except Exception:
-            #     log.exception(
-            #         'Failed to `saferepr({type(struct)})` !?\n'
-            #     )
-                # raise
-                # return _Struct.__repr__(struct)
-
-        # TODO: LOLOL use `textwrap.indent()` instead dawwwwwg!
-        obj_str += (field_ws + f'{k}: {typ_name} = {val_str},\n')
-
-    return (
-        f'{qtn}(\n'
-        f'{obj_str}'
-        f'{ws})'
-    )
-
-
-class Struct(
-    _Struct,
-
-    # https://jcristharif.com/msgspec/structs.html#tagged-unions
-    # tag='pikerstruct',
-    # tag=True,
-):
-    '''
-    A "human friendlier" (aka repl buddy) struct subtype.
-
-    '''
-    def to_dict(
-        self,
-        include_non_members: bool = True,
-
-    ) -> dict:
-        '''
-        Like it sounds.. direct delegation to:
-        https://jcristharif.com/msgspec/api.html#msgspec.structs.asdict
-
-        BUT, by default we pop all non-member (aka not defined as
-        struct fields) fields by default.
-
-        '''
-        asdict: dict = structs.asdict(self)
-        if include_non_members:
-            return asdict
-
-        # only return a dict of the struct members
-        # which were provided as input, NOT anything
-        # added as type-defined `@property` methods!
-        sin_props: dict = {}
-        fi: structs.FieldInfo
-        for fi, k, v in iter_fields(self):
-            sin_props[k] = asdict[k]
-
-        return sin_props
-
-    pformat = pformat
-
-    def __repr__(self) -> str:
-        try:
-            return pformat(self)
-        except Exception:
-            log.exception(
-                f'Failed to `pformat({type(self)})` !?\n'
-            )
-            return _Struct.__repr__(self)
-
-    # __repr__ = pformat
-    # __str__ = __repr__ = pformat
-    # TODO: use a pprint.PrettyPrinter instance around ONLY rendering
-    # inside a known tty?
-    # def __repr__(self) -> str:
-    #     ...
-
-    def copy(
-        self,
-        update: dict | None = None,
-
-    ) -> Struct:
-        '''
-        Validate-typecast all self defined fields, return a copy of
-        us with all such fields.
-
-        NOTE: This is kinda like the default behaviour in
-        `pydantic.BaseModel` except a copy of the object is
-        returned making it compat with `frozen=True`.
-
-        '''
-        if update:
-            for k, v in update.items():
-                setattr(self, k, v)
-
-        # NOTE: roundtrip serialize to validate
-        # - enode to msgpack binary format,
-        # - decode that back to a struct.
-        return msgpack.Decoder(type=type(self)).decode(
-            msgpack.Encoder().encode(self)
-        )
-
-    def typecast(
-        self,
-
-        # TODO: allow only casting a named subset?
-        # fields: set[str] | None = None,
-
-    ) -> None:
-        '''
-        Cast all fields using their declared type annotations
-        (kinda like what `pydantic` does by default).
-
-        NOTE: this of course won't work on frozen types, use
-        ``.copy()`` above in such cases.
-
-        '''
-        # https://jcristharif.com/msgspec/api.html#msgspec.structs.fields
-        fi: structs.FieldInfo
-        for fi in structs.fields(self):
-            setattr(
-                self,
-                fi.name,
-                fi.type(getattr(self, fi.name)),
-            )
-
-    # TODO: make a mod func instead and just point to it here for
-    # method impl?
-    def __sub__(
-        self,
-        other: Struct,
-
-    ) -> DiffDump[tuple[str, Any, Any]]:
-        '''
-        Compare fields/items key-wise and return a `DiffDump`
-        for easy visual REPL comparison B)
-
-        '''
-        diffs: DiffDump[tuple[str, Any, Any]] = DiffDump()
-        for fi in structs.fields(self):
-            attr_name: str = fi.name
-            ours: Any = getattr(self, attr_name)
-            theirs: Any = getattr(other, attr_name)
-            if ours != theirs:
-                diffs.append((
-                    attr_name,
-                    ours,
-                    theirs,
-                ))
-
-        return diffs
-
-    @classmethod
-    def fields_diff(
-        cls,
-        other: dict|Struct,
-
-    ) -> DiffDump[tuple[str, Any, Any]]:
-        '''
-        Very similar to `PrettyStruct.__sub__()` except accepts an
-        input `other: dict` (presumably that would normally be called
-        like `Struct(**other)`) which returns a `DiffDump` of the
-        fields of the struct and the `dict`'s fields.
-
-        '''
-        nullish = object()
-        consumed: dict = other.copy()
-        diffs: DiffDump[tuple[str, Any, Any]] = DiffDump()
-        for fi in structs.fields(cls):
-            field_name: str = fi.name
-            # ours: Any = getattr(self, field_name)
-            theirs: Any = consumed.pop(field_name, nullish)
-            if theirs is nullish:
-                diffs.append((
-                    field_name,
-                    f'{fi.type!r}',
-                    'NOT-DEFINED in `other: dict`',
-                ))
-
-        # when there are lingering fields in `other` that this struct
-        # DOES NOT define we also append those.
-        if consumed:
-            for k, v in consumed.items():
-                diffs.append((
-                    k,
-                    f'NOT-DEFINED for `{cls.__name__}`',
-                    f'`other: dict` has value = {v!r}',
-                ))
-
-        return diffs

tractor/msg/ptr.py

@@ -76,11 +76,9 @@ class NamespacePath(str):
         return self._ref
 
     @staticmethod
-    def _mk_fqnp(
-        ref: type|object,
-    ) -> tuple[str, str]:
+    def _mk_fqnp(ref: type | object) -> tuple[str, str]:
         '''
-        Generate a minial `str` pair which describes a python
+        Generate a minial ``str`` pair which describes a python
         object's namespace path and object/type name.
 
         In more precise terms something like:
@@ -89,9 +87,10 @@ class NamespacePath(str):
             of THIS type XD
 
         '''
-        if isfunction(ref):
+        if (
+            isfunction(ref)
+        ):
             name: str = getattr(ref, '__name__')
-            mod_name: str = ref.__module__
 
         elif ismethod(ref):
             # build out the path manually i guess..?
@@ -100,19 +99,15 @@ class NamespacePath(str):
                 type(ref.__self__).__name__,
                 ref.__func__.__name__,
             ])
-            mod_name: str = ref.__self__.__module__
 
         else:  # object or other?
             # isinstance(ref, object)
             # and not isfunction(ref)
             name: str = type(ref).__name__
-            mod_name: str = ref.__module__
 
-        # TODO: return static value direactly?
-        #
         # fully qualified namespace path, tuple.
         fqnp: tuple[str, str] = (
-            mod_name,
+            ref.__module__,
             name,
         )
         return fqnp
@@ -120,7 +115,7 @@ class NamespacePath(str):
     @classmethod
     def from_ref(
         cls,
-        ref: type|object,
+        ref: type | object,
 
     ) -> NamespacePath:
 

tractor/msg/types.py

@@ -15,713 +15,256 @@
 # along with this program.  If not, see <https://www.gnu.org/licenses/>.
 
 '''
-Define our strictly typed IPC message spec for the SCIPP:
-
-that is,
-
-the "Structurred-Concurrency-Inter-Process-(dialog)-(un)Protocol".
+Extensions to built-in or (heavily used but 3rd party) friend-lib
+types.
 
 '''
 from __future__ import annotations
-import types
+from collections import UserList
+from pprint import (
+    saferepr,
+)
 from typing import (
     Any,
-    Generic,
-    Literal,
-    Type,
-    TypeVar,
-    TypeAlias,
-    Union,
+    Iterator,
 )
 
 from msgspec import (
-    defstruct,
-    # field,
-    Raw,
-    Struct,
-    # UNSET,
-    # UnsetType,
+    msgpack,
+    Struct as _Struct,
+    structs,
 )
 
-from tractor.msg import (
-    pretty_struct,
-)
-from tractor.log import get_logger
+# TODO: auto-gen type sig for input func both for
+# type-msgs and logging of RPC tasks?
+# taken and modified from:
+# https://stackoverflow.com/a/57110117
+# import inspect
+# from typing import List
+
+# def my_function(input_1: str, input_2: int) -> list[int]:
+#     pass
+
+# def types_of(func):
+#     specs = inspect.getfullargspec(func)
+#     return_type = specs.annotations['return']
+#     input_types = [t.__name__ for s, t in specs.annotations.items() if s != 'return']
+#     return f'{func.__name__}({": ".join(input_types)}) -> {return_type}'
+
+# types_of(my_function)
 
 
-log = get_logger('tractor.msgspec')
+class DiffDump(UserList):
+    '''
+    Very simple list delegator that repr() dumps (presumed) tuple
+    elements of the form `tuple[str, Any, Any]` in a nice
+    multi-line readable form for analyzing `Struct` diffs.
 
-# type variable for the boxed payload field `.pld`
-PayloadT = TypeVar('PayloadT')
+    '''
+    def __repr__(self) -> str:
+        if not len(self):
+            return super().__repr__()
+
+        # format by displaying item pair's ``repr()`` on multiple,
+        # indented lines such that they are more easily visually
+        # comparable when printed to console when printed to
+        # console.
+        repstr: str = '[\n'
+        for k, left, right in self:
+            repstr += (
+                f'({k},\n'
+                f'\t{repr(left)},\n'
+                f'\t{repr(right)},\n'
+                ')\n'
+            )
+        repstr += ']\n'
+        return repstr
 
 
-class PayloadMsg(
-    Struct,
-    Generic[PayloadT],
+class Struct(
+    _Struct,
 
     # https://jcristharif.com/msgspec/structs.html#tagged-unions
-    tag=True,
-    tag_field='msg_type',
-
-    # https://jcristharif.com/msgspec/structs.html#field-ordering
-    # kw_only=True,
-
-    # https://jcristharif.com/msgspec/structs.html#equality-and-order
-    # order=True,
-
-    # https://jcristharif.com/msgspec/structs.html#encoding-decoding-as-arrays
-    # as_array=True,
+    # tag='pikerstruct',
+    # tag=True,
 ):
     '''
-    An abstract payload boxing/shuttling IPC msg type.
-
-    Boxes data-values passed to/from user code
-
-    (i.e. any values passed by `tractor` application code using any of
-
-      |_ `._streaming.MsgStream.send/receive()`
-      |_ `._context.Context.started/result()`
-      |_ `._ipc.Channel.send/recv()`
-
-     aka our "IPC primitive APIs")
-
-    as message "payloads" set to the `.pld` field and uses
-    `msgspec`'s "tagged unions" feature to support a subset of our
-    "SC-transitive shuttle protocol" specification with
-    a `msgspec.Struct` inheritance tree.
+    A "human friendlier" (aka repl buddy) struct subtype.
 
     '''
-    cid: str  # call/context-id
-    # ^-TODO-^: more explicit type?
-    # -[ ] use UNSET here?
-    #  https://jcristharif.com/msgspec/supported-types.html#unset
-    #
-    # -[ ] `uuid.UUID` which has multi-protocol support
-    #  https://jcristharif.com/msgspec/supported-types.html#uuid
-
-    # The msg's "payload" (spelled without vowels):
-    # https://en.wikipedia.org/wiki/Payload_(computing)
-    pld: Raw
-
-    # ^-NOTE-^ inherited from any `PayloadMsg` (and maybe type
-    # overriden via the `._ops.limit_plds()` API), but by default is
-    # parameterized to be `Any`.
-    #
-    # XXX this `Union` must strictly NOT contain `Any` if
-    # a limited msg-type-spec is intended, such that when
-    # creating and applying a new `MsgCodec` its 
-    # `.decoder: Decoder` is configured with a `Union[Type[Struct]]` which
-    # restricts the allowed payload content (this `.pld` field) 
-    # by type system defined loading constraints B)
-    #
-    # TODO: could also be set to `msgspec.Raw` if the sub-decoders
-    # approach is preferred over the generic parameterization 
-    # approach as take by `mk_msg_spec()` below.
-
-
-# TODO: complete rename
-Msg = PayloadMsg
-
-
-class Aid(
-    Struct,
-    tag=True,
-    tag_field='msg_type',
-):
-    '''
-    Actor-identity msg.
-
-    Initial contact exchange enabling an actor "mailbox handshake"
-    delivering the peer identity (and maybe eventually contact)
-    info.
-
-    Used by discovery protocol to register actors as well as
-    conduct the initial comms (capability) filtering.
-
-    '''
-    name: str
-    uuid: str
-    # TODO: use built-in support for UUIDs?
-    # -[ ] `uuid.UUID` which has multi-protocol support
-    #  https://jcristharif.com/msgspec/supported-types.html#uuid
-
-
-class SpawnSpec(
-    pretty_struct.Struct,
-    tag=True,
-    tag_field='msg_type',
-):
-    '''
-    Initial runtime spec handed down from a spawning parent to its
-    child subactor immediately following first contact via an
-    `Aid` msg.
-
-    '''
-    # TODO: similar to the `Start` kwargs spec needed below, we need
-    # a hard `Struct` def for all of these fields!
-    _parent_main_data: dict
-    _runtime_vars: dict[str, Any]
-
-    # module import capability
-    enable_modules: dict[str, str]
-
-    # TODO: not just sockaddr pairs?
-    # -[ ] abstract into a `TransportAddr` type?
-    reg_addrs: list[tuple[str, int]]
-    bind_addrs: list[tuple[str, int]]
-
-
-# TODO: caps based RPC support in the payload?
-#
-# -[ ] integration with our ``enable_modules: list[str]`` caps sys.
-#   ``pkgutil.resolve_name()`` internally uses
-#   ``importlib.import_module()`` which can be filtered by
-#   inserting a ``MetaPathFinder`` into ``sys.meta_path`` (which
-#   we could do before entering the ``Actor._process_messages()``
-#   loop)?
-#   - https://github.com/python/cpython/blob/main/Lib/pkgutil.py#L645
-#   - https://stackoverflow.com/questions/1350466/preventing-python-code-from-importing-certain-modules
-#   - https://stackoverflow.com/a/63320902
-#   - https://docs.python.org/3/library/sys.html#sys.meta_path
-#
-# -[ ] can we combine .ns + .func into a native `NamespacePath` field?
-#
-# -[ ] better name, like `Call/TaskInput`?
-#
-# -[ ] XXX a debugger lock msg transaction with payloads like,
-#   child -> `.pld: DebugLock` -> root
-#   child <- `.pld: DebugLocked` <- root
-#   child -> `.pld: DebugRelease` -> root
-#
-#   WHY => when a pld spec is provided it might not allow for
-#   debug mode msgs as they currently are (using plain old `pld.
-#   str` payloads) so we only when debug_mode=True we need to
-#   union in this debugger payload set?
-#
-#   mk_msg_spec(
-#       MyPldSpec,
-#       debug_mode=True,
-#   ) -> (
-#       Union[MyPldSpec]
-#      | Union[DebugLock, DebugLocked, DebugRelease]
-#   )
-
-# class Params(
-#     Struct,
-#     Generic[PayloadT],
-# ):
-#     spec: PayloadT|ParamSpec
-#     inputs: InputsT|dict[str, Any]
-
-    # TODO: for eg. we could stringently check the target
-    # task-func's type sig and enforce it?
-    # as an example for an IPTC,
-    # @tractor.context
-    # async def send_back_nsp(
-    #     ctx: Context,
-    #     expect_debug: bool,
-    #     pld_spec_str: str,
-    #     add_hooks: bool,
-    #     started_msg_dict: dict,
-    # ) -> <WhatHere!>:
-
-    # TODO: figure out which of the `typing` feats we want to
-    # support:
-    # - plain ol `ParamSpec`:
-    #   https://docs.python.org/3/library/typing.html#typing.ParamSpec
-    # - new in 3.12 type parameter lists Bo
-    # |_ https://docs.python.org/3/reference/compound_stmts.html#type-params
-    # |_ historical pep 695: https://peps.python.org/pep-0695/
-    # |_ full lang spec: https://typing.readthedocs.io/en/latest/spec/
-    # |_ on annotation scopes:
-    #    https://docs.python.org/3/reference/executionmodel.html#annotation-scopes
-    # spec: ParamSpec[
-    #     expect_debug: bool,
-    #     pld_spec_str: str,
-    #     add_hooks: bool,
-    #     started_msg_dict: dict,
-    # ]
-
-
-# TODO: possibly sub-type for runtime method requests?
-# -[ ] `Runtime(Start)` with a `.ns: str = 'self' or
-#     we can just enforce any such method as having a strict
-#     ns for calling funcs, namely the `Actor` instance?
-class Start(
-    Struct,
-    tag=True,
-    tag_field='msg_type',
-):
-    '''
-    Initial request to remotely schedule an RPC `trio.Task` via
-    `Actor.start_remote_task()`.
-
-    It is called by all the following public APIs:
-
-    - `ActorNursery.run_in_actor()`
-
-    - `Portal.run()`
-          `|_.run_from_ns()`
-          `|_.open_stream_from()`
-          `|_._submit_for_result()`
-
-    - `Context.open_context()`
-
-    '''
-    cid: str
-
-    ns: str
-    func: str
-
-    # TODO: make this a sub-struct which can be further
-    # type-limited, maybe `Inputs`?
-    # => SEE ABOVE <=
-    kwargs: dict[str, Any]
-    uid: tuple[str, str]  # (calling) actor-id
-
-    # TODO: enforcing a msg-spec in terms `Msg.pld`
-    # parameterizable msgs to be used in the appls IPC dialog.
-    # => SEE `._codec.MsgDec` for more <=
-    pld_spec: str = str(Any)
-
-
-class StartAck(
-    Struct,
-    tag=True,
-    tag_field='msg_type',
-):
-    '''
-    Init response to a `Cmd` request indicating the far
-    end's RPC spec, namely its callable "type".
-
-    '''
-    cid: str
-    # TODO: maybe better names for all these?
-    # -[ ] obvi ^ would need sync with `._rpc`
-    functype: Literal[
-        'asyncfunc',
-        'asyncgen',
-        'context',  # TODO: the only one eventually?
-    ]
-
-    # import typing
-    # eval(str(Any), {}, {'typing': typing})
-    # started_spec: str = str(Any)
-    # return_spec
-
-
-class Started(
-    PayloadMsg,
-    Generic[PayloadT],
-):
-    '''
-    Packet to shuttle the "first value" delivered by
-    `Context.started(value: Any)` from a `@tractor.context`
-    decorated IPC endpoint.
-
-    '''
-    pld: PayloadT|Raw
-
-
-# TODO: cancel request dedicated msg?
-# -[ ] instead of using our existing `Start`?
-#
-# class Cancel:
-#     cid: str
-
-
-class Yield(
-    PayloadMsg,
-    Generic[PayloadT],
-):
-    '''
-    Per IPC transmission of a value from `await MsgStream.send(<value>)`.
-
-    '''
-    pld: PayloadT|Raw
-
-
-class Stop(
-    Struct,
-    tag=True,
-    tag_field='msg_type',
-):
-    '''
-    Stream termination signal much like an IPC version 
-    of `StopAsyncIteration`.
-
-    '''
-    cid: str
-    # TODO: do we want to support a payload on stop?
-    # pld: UnsetType = UNSET
-
-
-# TODO: is `Result` or `Out[come]` a better name?
-class Return(
-    PayloadMsg,
-    Generic[PayloadT],
-):
-    '''
-    Final `return <value>` from a remotely scheduled
-    func-as-`trio.Task`.
-
-    '''
-    pld: PayloadT|Raw
-
-
-class CancelAck(
-    PayloadMsg,
-    Generic[PayloadT],
-):
-    '''
-    Deliver the `bool` return-value from a cancellation `Actor`
-    method scheduled via and prior RPC request.
-
-    - `Actor.cancel()`
-       `|_.cancel_soon()`
-       `|_.cancel_rpc_tasks()`
-       `|_._cancel_task()`
-       `|_.cancel_server()`
-
-    RPCs to these methods must **always** be able to deliver a result
-    despite the currently configured IPC msg spec such that graceful
-    cancellation is always functional in the runtime.
-
-    '''
-    pld: bool
-
-
-# TODO: unify this with `._exceptions.RemoteActorError`
-# such that we can have a msg which is both raisable and
-# IPC-wire ready?
-# B~o
-class Error(
-    Struct,
-    tag=True,
-    tag_field='msg_type',
-
-    # TODO may omit defaults?
-    # https://jcristharif.com/msgspec/structs.html#omitting-default-values
-    # omit_defaults=True,
-):
-    '''
-    A pkt that wraps `RemoteActorError`s for relay and raising.
-
-    Fields are 1-to-1 meta-data as needed originally by
-    `RemoteActorError.msgdata: dict` but now are defined here.
-
-    Note: this msg shuttles `ContextCancelled` and `StreamOverrun`
-    as well is used to rewrap any `MsgTypeError` for relay-reponse
-    to bad `Yield.pld` senders during an IPC ctx's streaming dialog
-    phase.
-
-    '''
-    src_uid: tuple[str, str]
-    src_type_str: str
-    boxed_type_str: str
-    relay_path: list[tuple[str, str]]
-
-    # normally either both are provided or just
-    # a message for certain special cases where
-    # we pack a message for a locally raised
-    # mte or ctxc.
-    message: str|None = None
-    tb_str: str = ''
-
-    # TODO: only optionally include sub-type specfic fields?
-    # -[ ] use UNSET or don't include them via `omit_defaults` (see
-    #      inheritance-line options above)
-    #
-    # `ContextCancelled` reports the src cancelling `Actor.uid`
-    canceller: tuple[str, str]|None = None
-
-    # `StreamOverrun`-specific src `Actor.uid`
-    sender: tuple[str, str]|None = None
-
-    # `MsgTypeError` meta-data
-    cid: str|None = None
-    # when the receiver side fails to decode a delivered
-    # `PayloadMsg`-subtype; one and/or both the msg-struct instance
-    # and `Any`-decoded to `dict` of the msg are set and relayed
-    # (back to the sender) for introspection.
-    _bad_msg: Started|Yield|Return|None = None
-    _bad_msg_as_dict: dict|None = None
-
-
-def from_dict_msg(
-    dict_msg: dict,
-
-    msgT: MsgType|None = None,
-    tag_field: str = 'msg_type',
-    use_pretty: bool = False,
-
-) -> MsgType:
-    '''
-    Helper to build a specific `MsgType` struct from a "vanilla"
-    decoded `dict`-ified equivalent of the msg: i.e. if the
-    `msgpack.Decoder.type == Any`, the default when using
-    `msgspec.msgpack` and not "typed decoding" using
-    `msgspec.Struct`.
-
-    '''
-    msg_type_tag_field: str = (
-        msgT.__struct_config__.tag_field
-        if msgT is not None
-        else tag_field
-    )
-    # XXX ensure tag field is removed
-    msgT_name: str = dict_msg.pop(msg_type_tag_field)
-    msgT: MsgType = _msg_table[msgT_name]
-    if use_pretty:
-        msgT = defstruct(
-            name=msgT_name,
-            fields=[
-                (key, fi.type)
-                for fi, key, _
-                in pretty_struct.iter_fields(msgT)
-            ],
-            bases=(
-                pretty_struct.Struct,
-                msgT,
-            ),
-        )
-    return msgT(**dict_msg)
-
-# TODO: should be make a set of cancel msgs?
-# -[ ] a version of `ContextCancelled`?
-#     |_ and/or with a scope field?
-# -[ ] or, a full `ActorCancelled`?
-#
-# class Cancelled(MsgType):
-#     cid: str
-#
-# -[ ] what about overruns?
-#
-# class Overrun(MsgType):
-#     cid: str
-
-_runtime_msgs: list[Struct] = [
-
-    # identity handshake on first IPC `Channel` contact.
-    Aid,
-
-    # parent-to-child spawn specification passed as 2nd msg after
-    # handshake ONLY after child connects back to parent.
-    SpawnSpec,
-
-    # inter-actor RPC initiation
-    Start,  # schedule remote task-as-func
-    StartAck,  # ack the schedule request
-
-    # emission from `MsgStream.aclose()`
-    Stop,
-
-    # `Return` sub-type that we always accept from
-    # runtime-internal cancel endpoints
-    CancelAck,
-
-    # box remote errors, normally subtypes
-    # of `RemoteActorError`.
-    Error,
-]
-
-# the no-outcome-yet IAC (inter-actor-communication) sub-set which
-# can be `PayloadMsg.pld` payload field type-limited by application code
-# using `apply_codec()` and `limit_msg_spec()`.
-_payload_msgs: list[PayloadMsg] = [
-    # first <value> from `Context.started(<value>)`
-    Started,
-
-    # any <value> sent via `MsgStream.send(<value>)`
-    Yield,
-
-    # the final value returned from a `@context` decorated
-    # IPC endpoint.
-    Return,
-]
-
-# built-in SC shuttle protocol msg type set in
-# approx order of the IPC txn-state spaces.
-__msg_types__: list[MsgType] = (
-    _runtime_msgs
-    +
-    _payload_msgs
-)
-
-
-_msg_table: dict[str, MsgType] = {
-    msgT.__name__: msgT
-    for msgT in __msg_types__
-}
-
-# TODO: use new type declaration syntax for msg-type-spec
-# https://docs.python.org/3/library/typing.html#type-aliases
-# https://docs.python.org/3/reference/simple_stmts.html#type
-MsgType: TypeAlias = Union[*__msg_types__]
-
-
-def mk_msg_spec(
-    payload_type_union: Union[Type] = Any,
-
-    spec_build_method: Literal[
-        'indexed_generics',  # works
-        'defstruct',
-        'types_new_class',
-
-    ] = 'indexed_generics',
-
-) -> tuple[
-    Union[MsgType],
-    list[MsgType],
-]:
-    '''
-    Create a payload-(data-)type-parameterized IPC message specification.
-
-    Allows generating IPC msg types from the above builtin set
-    with a payload (field) restricted data-type, the `Msg.pld: PayloadT`.
-
-    This allows runtime-task contexts to use the python type system
-    to limit/filter payload values as determined by the input
-    `payload_type_union: Union[Type]`.
-
-    Notes: originally multiple approaches for constructing the
-    type-union passed to `msgspec` were attempted as selected via the
-    `spec_build_method`, but it turns out only the defaul method
-    'indexed_generics' seems to work reliably in all use cases. As
-    such, the others will likely be removed in the near future.
-
-    '''
-    submsg_types: list[MsgType] = Msg.__subclasses__()
-    bases: tuple = (
-        # XXX NOTE XXX the below generic-parameterization seems to
-        # be THE ONLY way to get this to work correctly in terms
-        # of getting ValidationError on a roundtrip?
-        Msg[payload_type_union],
-        Generic[PayloadT],
-    )
-    # defstruct_bases: tuple = (
-    #     Msg, # [payload_type_union],
-    #     # Generic[PayloadT],
-    #     # ^-XXX-^: not allowed? lul..
-    # )
-    ipc_msg_types: list[Msg] = []
-
-    idx_msg_types: list[Msg] = []
-    # defs_msg_types: list[Msg] = []
-    nc_msg_types: list[Msg] = []
-
-    for msgtype in __msg_types__:
-
-        # for the NON-payload (user api) type specify-able
-        # msgs types, we simply aggregate the def as is
-        # for inclusion in the output type `Union`.
-        if msgtype not in _payload_msgs:
-            ipc_msg_types.append(msgtype)
-            continue
-
-        # check inheritance sanity
-        assert msgtype in submsg_types
-
-        # TODO: wait why do we need the dynamic version here?
-        # XXX ANSWER XXX -> BC INHERITANCE.. don't work w generics..
-        #
-        # NOTE previously bc msgtypes WERE NOT inheriting
-        # directly the `Generic[PayloadT]` type, the manual method
-        # of generic-paraming with `.__class_getitem__()` wasn't
-        # working..
-        #
-        # XXX but bc i changed that to make every subtype inherit
-        # it, this manual "indexed parameterization" method seems
-        # to work?
-        #
-        # -[x] paraming the `PayloadT` values via `Generic[T]`
-        #   does work it seems but WITHOUT inheritance of generics
-        #
-        # -[-] is there a way to get it to work at module level
-        #   just using inheritance or maybe a metaclass?
-        #  => thot that `defstruct` might work, but NOPE, see
-        #   below..
-        #
-        idxed_msg_type: Msg = msgtype[payload_type_union]
-        idx_msg_types.append(idxed_msg_type)
-
-        # TODO: WHY do we need to dynamically generate the
-        # subtype-msgs here to ensure the `.pld` parameterization
-        # propagates as well as works at all in terms of the
-        # `msgpack.Decoder()`..?
-        #
-        # dynamically create the payload type-spec-limited msg set.
-        newclass_msgtype: Type = types.new_class(
-            name=msgtype.__name__,
-            bases=bases,
-            kwds={},
-        )
-        nc_msg_types.append(
-            newclass_msgtype[payload_type_union]
+    def _sin_props(self) -> Iterator[
+        tuple[
+            structs.FieldIinfo,
+            str,
+            Any,
+        ]
+    ]:
+        '''
+        Iterate over all non-@property fields of this struct.
+
+        '''
+        fi: structs.FieldInfo
+        for fi in structs.fields(self):
+            key: str = fi.name
+            val: Any = getattr(self, key)
+            yield fi, key, val
+
+    def to_dict(
+        self,
+        include_non_members: bool = True,
+
+    ) -> dict:
+        '''
+        Like it sounds.. direct delegation to:
+        https://jcristharif.com/msgspec/api.html#msgspec.structs.asdict
+
+        BUT, by default we pop all non-member (aka not defined as
+        struct fields) fields by default.
+
+        '''
+        asdict: dict = structs.asdict(self)
+        if include_non_members:
+            return asdict
+
+        # only return a dict of the struct members
+        # which were provided as input, NOT anything
+        # added as type-defined `@property` methods!
+        sin_props: dict = {}
+        fi: structs.FieldInfo
+        for fi, k, v in self._sin_props():
+            sin_props[k] = asdict[k]
+
+        return sin_props
+
+    def pformat(
+        self,
+        field_indent: int = 2,
+        indent: int = 0,
+
+    ) -> str:
+        '''
+        Recursion-safe `pprint.pformat()` style formatting of
+        a `msgspec.Struct` for sane reading by a human using a REPL.
+
+        '''
+        # global whitespace indent
+        ws: str = ' '*indent
+
+        # field whitespace indent
+        field_ws: str = ' '*(field_indent + indent)
+
+        # qtn: str = ws + self.__class__.__qualname__
+        qtn: str = self.__class__.__qualname__
+
+        obj_str: str = ''  # accumulator
+        fi: structs.FieldInfo
+        k: str
+        v: Any
+        for fi, k, v in self._sin_props():
+
+            # TODO: how can we prefer `Literal['option1',  'option2,
+            # ..]` over .__name__ == `Literal` but still get only the
+            # latter for simple types like `str | int | None` etc..?
+            ft: type = fi.type
+            typ_name: str = getattr(ft, '__name__', str(ft))
+
+            # recurse to get sub-struct's `.pformat()` output Bo
+            if isinstance(v, Struct):
+                val_str: str =  v.pformat(
+                    indent=field_indent + indent,
+                    field_indent=indent + field_indent,
+                )
+
+            else:  # the `pprint` recursion-safe format:
+                # https://docs.python.org/3.11/library/pprint.html#pprint.saferepr
+                val_str: str = saferepr(v)
+
+            # TODO: LOLOL use `textwrap.indent()` instead dawwwwwg!
+            obj_str += (field_ws + f'{k}: {typ_name} = {val_str},\n')
+
+        return (
+            f'{qtn}(\n'
+            f'{obj_str}'
+            f'{ws})'
         )
 
-        # with `msgspec.structs.defstruct`
-        # XXX ALSO DOESN'T WORK
-        # defstruct_msgtype = defstruct(
-        #     name=msgtype.__name__,
-        #     fields=[
-        #         ('cid', str),
+    # TODO: use a pprint.PrettyPrinter instance around ONLY rendering
+    # inside a known tty?
+    # def __repr__(self) -> str:
+    #     ...
 
-        #         # XXX doesn't seem to work..
-        #         # ('pld', PayloadT),
+    # __str__ = __repr__ = pformat
+    __repr__ = pformat
 
-        #         ('pld', payload_type_union),
-        #     ],
-        #     bases=defstruct_bases,
-        # )
-        # defs_msg_types.append(defstruct_msgtype)
-        # assert index_paramed_msg_type == manual_paramed_msg_subtype
-        # paramed_msg_type = manual_paramed_msg_subtype
-        # ipc_payload_msgs_type_union |= index_paramed_msg_type
+    def copy(
+        self,
+        update: dict | None = None,
 
-    idx_spec: Union[Type[Msg]] = Union[*idx_msg_types]
-    # def_spec: Union[Type[Msg]] = Union[*defs_msg_types]
-    nc_spec: Union[Type[Msg]] = Union[*nc_msg_types]
+    ) -> Struct:
+        '''
+        Validate-typecast all self defined fields, return a copy of
+        us with all such fields.
 
-    specs: dict[str, Union[Type[Msg]]] = {
-        'indexed_generics': idx_spec,
-        # 'defstruct': def_spec,
-        'types_new_class': nc_spec,
-    }
-    msgtypes_table: dict[str, list[Msg]] = {
-        'indexed_generics': idx_msg_types,
-        # 'defstruct': defs_msg_types,
-        'types_new_class': nc_msg_types,
-    }
+        NOTE: This is kinda like the default behaviour in
+        `pydantic.BaseModel` except a copy of the object is
+        returned making it compat with `frozen=True`.
 
-    # XXX lol apparently type unions can't ever
-    # be equal eh?
-    # TODO: grok the diff here better..
-    #
-    # assert (
-    #     idx_spec
-    #     ==
-    #     nc_spec
-    #     ==
-    #     def_spec
-    # )
-    # breakpoint()
+        '''
+        if update:
+            for k, v in update.items():
+                setattr(self, k, v)
 
-    pld_spec: Union[Type] = specs[spec_build_method]
-    runtime_spec: Union[Type] = Union[*ipc_msg_types]
-    ipc_spec = pld_spec | runtime_spec
-    log.runtime(
-        'Generating new IPC msg-spec\n'
-        f'{ipc_spec}\n'
-    )
-    assert (
-        ipc_spec
-        and
-        ipc_spec is not Any
-    )
-    return (
-        ipc_spec,
-        msgtypes_table[spec_build_method]
-        +
-        ipc_msg_types,
-    )
+        # NOTE: roundtrip serialize to validate
+        # - enode to msgpack binary format,
+        # - decode that back to a struct.
+        return msgpack.Decoder(type=type(self)).decode(
+            msgpack.Encoder().encode(self)
+        )
+
+    def typecast(
+        self,
+
+        # TODO: allow only casting a named subset?
+        # fields: set[str] | None = None,
+
+    ) -> None:
+        '''
+        Cast all fields using their declared type annotations
+        (kinda like what `pydantic` does by default).
+
+        NOTE: this of course won't work on frozen types, use
+        ``.copy()`` above in such cases.
+
+        '''
+        # https://jcristharif.com/msgspec/api.html#msgspec.structs.fields
+        fi: structs.FieldInfo
+        for fi in structs.fields(self):
+            setattr(
+                self,
+                fi.name,
+                fi.type(getattr(self, fi.name)),
+            )
+
+    def __sub__(
+        self,
+        other: Struct,
+
+    ) -> DiffDump[tuple[str, Any, Any]]:
+        '''
+        Compare fields/items key-wise and return a ``DiffDump``
+        for easy visual REPL comparison B)
+
+        '''
+        diffs: DiffDump[tuple[str, Any, Any]] = DiffDump()
+        for fi in structs.fields(self):
+            attr_name: str = fi.name
+            ours: Any = getattr(self, attr_name)
+            theirs: Any = getattr(other, attr_name)
+            if ours != theirs:
+                diffs.append((
+                    attr_name,
+                    ours,
+                    theirs,
+                ))
+
+        return diffs

tractor/trionics/__init__.py

@@ -29,6 +29,3 @@ from ._broadcast import (
     BroadcastReceiver as BroadcastReceiver,
     Lagged as Lagged,
 )
-from ._beg import (
-    collapse_eg as collapse_eg,
-)

tractor/trionics/_beg.py

@@ -1,58 +0,0 @@
-# tractor: structured concurrent "actors".
-# Copyright 2018-eternity Tyler Goodlet.
-
-# This program is free software: you can redistribute it and/or modify
-# it under the terms of the GNU Affero General Public License as published by
-# the Free Software Foundation, either version 3 of the License, or
-# (at your option) any later version.
-
-# This program is distributed in the hope that it will be useful,
-# but WITHOUT ANY WARRANTY; without even the implied warranty of
-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
-# GNU Affero General Public License for more details.
-
-# You should have received a copy of the GNU Affero General Public License
-# along with this program.  If not, see <https://www.gnu.org/licenses/>.
-
-'''
-`BaseExceptionGroup` related utils and helpers pertaining to
-first-class-`trio` from a historical perspective B)
-
-'''
-from contextlib import (
-    asynccontextmanager as acm,
-)
-
-
-def maybe_collapse_eg(
-    beg: BaseExceptionGroup,
-) -> BaseException:
-    '''
-    If the input beg can collapse to a single non-eg sub-exception,
-    return it instead.
-
-    '''
-    if len(excs := beg.exceptions) == 1:
-        return excs[0]
-
-    return beg
-
-
-@acm
-async def collapse_eg():
-    '''
-    If `BaseExceptionGroup` raised in the body scope is
-    "collapse-able" (in the same way that
-    `trio.open_nursery(strict_exception_groups=False)` works) then
-    only raise the lone emedded non-eg in in place.
-
-    '''
-    try:
-        yield
-    except* BaseException as beg:
-        if (
-            exc := maybe_collapse_eg(beg)
-        ) is not beg:
-            raise exc
-
-        raise beg

tractor/trionics/_broadcast.py

@@ -15,7 +15,7 @@
 # along with this program.  If not, see <https://www.gnu.org/licenses/>.
 
 '''
-`tokio` style broadcast channel.
+``tokio`` style broadcast channel.
 https://docs.rs/tokio/1.11.0/tokio/sync/broadcast/index.html
 
 '''
@@ -26,6 +26,7 @@ from contextlib import asynccontextmanager
 from functools import partial
 from operator import ne
 from typing import (
+    Optional,
     Callable,
     Awaitable,
     Any,
@@ -44,11 +45,6 @@ from tractor.log import get_logger
 
 log = get_logger(__name__)
 
-# TODO: use new type-vars syntax from 3.12
-# https://realpython.com/python312-new-features/#dedicated-type-variable-syntax
-# https://docs.python.org/3/whatsnew/3.12.html#whatsnew312-pep695
-# https://docs.python.org/3/reference/simple_stmts.html#type
-#
 # A regular invariant generic type
 T = TypeVar("T")
 
@@ -114,7 +110,7 @@ class BroadcastState(Struct):
 
     # broadcast event to wake up all sleeping consumer tasks
     # on a newly produced value from the sender.
-    recv_ready: tuple[int, trio.Event]|None = None
+    recv_ready: Optional[tuple[int, trio.Event]] = None
 
     # if a ``trio.EndOfChannel`` is received on any
     # consumer all consumers should be placed in this state
@@ -156,12 +152,11 @@ class BroadcastState(Struct):
 
 class BroadcastReceiver(ReceiveChannel):
     '''
-    A memory receive channel broadcaster which is non-lossy for
-    the fastest consumer.
+    A memory receive channel broadcaster which is non-lossy for the
+    fastest consumer.
 
-    Additional consumer tasks can receive all produced values by
-    registering with ``.subscribe()`` and receiving from the new
-    instance it delivers.
+    Additional consumer tasks can receive all produced values by registering
+    with ``.subscribe()`` and receiving from the new instance it delivers.
 
     '''
     def __init__(
@@ -169,7 +164,7 @@ class BroadcastReceiver(ReceiveChannel):
 
         rx_chan: AsyncReceiver,
         state: BroadcastState,
-        receive_afunc: Callable[[], Awaitable[Any]]|None = None,
+        receive_afunc: Optional[Callable[[], Awaitable[Any]]] = None,
         raise_on_lag: bool = True,
 
     ) -> None:
@@ -382,7 +377,7 @@ class BroadcastReceiver(ReceiveChannel):
                         # likely it makes sense to unwind back to the
                         # underlying?
                         # import tractor
-                        # await tractor.pause()
+                        # await tractor.breakpoint()
                         log.warning(
                             f'Only one sub left for {self}?\n'
                             'We can probably unwind from breceiver?'
@@ -457,7 +452,7 @@ def broadcast_receiver(
 
     recv_chan: AsyncReceiver,
     max_buffer_size: int,
-    receive_afunc: Callable[[], Awaitable[Any]]|None = None,
+    receive_afunc: Optional[Callable[[], Awaitable[Any]]] = None,
     raise_on_lag: bool = True,
 
 ) -> BroadcastReceiver: