Update readme for `uv`, refine feats list
Also make all code example snippets a sub-section of a new `Example codez` section in prep to reduce the amount of code in the readme which instead will be simply linked to in the repo in the future.pkg_tidying
parent
db31bbfee2
commit
0272936fdc
124
docs/README.rst
124
docs/README.rst
|
@ -3,18 +3,22 @@
|
||||||
|gh_actions|
|
|gh_actions|
|
||||||
|docs|
|
|docs|
|
||||||
|
|
||||||
``tractor`` is a `structured concurrent`_, (optionally
|
``tractor`` is a `structured concurrency`_ (SC), (optionally
|
||||||
distributed_) multi-processing_ runtime built on trio_.
|
distributed_) multi-processing_ runtime built on trio_.
|
||||||
|
|
||||||
Fundamentally, ``tractor`` gives you parallelism via
|
Fundamentally, ``tractor`` provides parallelism via
|
||||||
``trio``-"*actors*": independent Python processes (aka
|
``trio``-"*actors*": independent Python *processes* (currently
|
||||||
non-shared-memory threads) which maintain structured
|
*non-shared-memory threads*) which schedule ``trio`` tasks that
|
||||||
concurrency (SC) *end-to-end* inside a *supervision tree*.
|
maintain SC *end-to-end* inside a *distributed supervision tree*.
|
||||||
|
|
||||||
Cross-process (and thus cross-host) SC is accomplished through the
|
Cross-process (and thus cross-host) SC is accomplished through the
|
||||||
combined use of our "actor nurseries_" and an "SC-transitive IPC
|
combined use of our,
|
||||||
protocol" constructed on top of multiple Pythons each running a ``trio``
|
|
||||||
scheduled runtime - a call to ``trio.run()``.
|
- "actor nurseries_" which provide for spawning multiple, and
|
||||||
|
possibly nested, Python process trees each running a ``trio``
|
||||||
|
scheduled runtime - a call to ``trio.run()``,
|
||||||
|
- an "SC-transitive supervision protocol" implemented as an
|
||||||
|
IPC-message-spec enforced around each RPC-dialog.
|
||||||
|
|
||||||
We believe the system adheres to the `3 axioms`_ of an "`actor model`_"
|
We believe the system adheres to the `3 axioms`_ of an "`actor model`_"
|
||||||
but likely **does not** look like what **you** probably *think* an "actor
|
but likely **does not** look like what **you** probably *think* an "actor
|
||||||
|
@ -35,22 +39,73 @@ Some great places to start are,
|
||||||
|
|
||||||
Features
|
Features
|
||||||
--------
|
--------
|
||||||
- **It's just** a ``trio`` API
|
- **It's just** a ``trio`` API!
|
||||||
- *Infinitely nesteable* process trees
|
- *Infinitely nesteable* process running embedded ``trio.Task`` trees.
|
||||||
- Builtin IPC streaming APIs with task fan-out broadcasting
|
- Support for a swappable, OS-specific, process spawning via
|
||||||
- A "native" multi-core debugger REPL using `pdbp`_ (a fork & fix of
|
multiple backends.
|
||||||
`pdb++`_ thanks to @mdmintz!)
|
- A modular transport stack, allowing for custom interchange formats (eg.
|
||||||
- Support for a swappable, OS specific, process spawning layer
|
as offered from `msgspec`_), varied transport protocols (TCP, RUDP), and
|
||||||
- A modular transport stack, allowing for custom serialization (eg. with
|
OS-env specific higher perf IPC primitives (like shared-mem buffers).
|
||||||
`msgspec`_), communications protocols, and environment specific IPC
|
- Builtin streaming API with task fan-out `broadcasting`_.
|
||||||
primitives
|
- A "native" and multi-core-safe debugger REPL using `pdbp`_ (a fork
|
||||||
- Support for spawning process-level-SC, inter-loop one-to-one-task oriented
|
& fix of `pdb++`_ thanks to @mdmintz!)
|
||||||
``asyncio`` actors via "infected ``asyncio``" mode
|
- "infected ``asyncio``" mode: support for starting each
|
||||||
- `structured chadcurrency`_ from the ground up
|
``tractor.Actor`` to run as a guest on the ``asyncio`` loop
|
||||||
|
allowing us to provide stringent SC-style ``trio.Task``-supervision
|
||||||
|
around any ``asyncio.Task`` spawned via our ``tractor.to_asyncio``
|
||||||
|
APIs.
|
||||||
|
|
||||||
|
|
||||||
|
Install
|
||||||
|
-------
|
||||||
|
``tractor`` is still in a *alpha-near-beta-stage* for many
|
||||||
|
of its subsystems, however we are very close to having a stable
|
||||||
|
lowlevel runtime and API.
|
||||||
|
|
||||||
|
As such, it's currently recommended that you clone and install the
|
||||||
|
repo from source::
|
||||||
|
|
||||||
|
pip install git+git://github.com/goodboy/tractor.git
|
||||||
|
|
||||||
|
|
||||||
|
We use the very hip `uv`_ for ::
|
||||||
|
|
||||||
|
git clone https://github.com/goodboy/tractor.git
|
||||||
|
cd tractor
|
||||||
|
uv sync --dev
|
||||||
|
uv run python examples/rpc_bidir_streaming.py
|
||||||
|
|
||||||
|
Consider activating a virtual/project-env before starting to hack on
|
||||||
|
the code base::
|
||||||
|
|
||||||
|
# you could use plain ol' venvs
|
||||||
|
# https://docs.astral.sh/uv/pip/environments/
|
||||||
|
uv venv tractor_py313 --python 3.13
|
||||||
|
|
||||||
|
# but @goodboy prefers the more explicit (and shell agnostic)
|
||||||
|
# https://docs.astral.sh/uv/configuration/environment/#uv_project_environment
|
||||||
|
UV_PROJECT_ENVIRONMENT="tractor_py313
|
||||||
|
|
||||||
|
uv run --dev xonsh # HINT, @goodboy's fave shell B)
|
||||||
|
|
||||||
|
Alongside all this we ofc offer "releases" on PyPi::
|
||||||
|
|
||||||
|
pip install tractor
|
||||||
|
|
||||||
|
Just note that YMMV since the main git branch is often much further
|
||||||
|
ahead then any latest release.
|
||||||
|
|
||||||
|
|
||||||
|
Example codez
|
||||||
|
-------------
|
||||||
|
In ``tractor``'s (very lacking) documention we prefer to point to
|
||||||
|
example scripts in the repo over duplicating their in docs, with that
|
||||||
|
in mind here are some definitive snippets to help hook you into
|
||||||
|
looking deeper.
|
||||||
|
|
||||||
|
|
||||||
Run a func in a process
|
Run a func in a process
|
||||||
-----------------------
|
***********************
|
||||||
Use ``trio``'s style of focussing on *tasks as functions*:
|
Use ``trio``'s style of focussing on *tasks as functions*:
|
||||||
|
|
||||||
.. code:: python
|
.. code:: python
|
||||||
|
@ -108,7 +163,7 @@ might want to check out `trio-parallel`_.
|
||||||
|
|
||||||
|
|
||||||
Zombie safe: self-destruct a process tree
|
Zombie safe: self-destruct a process tree
|
||||||
-----------------------------------------
|
*****************************************
|
||||||
``tractor`` tries to protect you from zombies, no matter what.
|
``tractor`` tries to protect you from zombies, no matter what.
|
||||||
|
|
||||||
.. code:: python
|
.. code:: python
|
||||||
|
@ -164,7 +219,7 @@ it **is a bug**.
|
||||||
|
|
||||||
|
|
||||||
"Native" multi-process debugging
|
"Native" multi-process debugging
|
||||||
--------------------------------
|
********************************
|
||||||
Using the magic of `pdbp`_ and our internal IPC, we've
|
Using the magic of `pdbp`_ and our internal IPC, we've
|
||||||
been able to create a native feeling debugging experience for
|
been able to create a native feeling debugging experience for
|
||||||
any (sub-)process in your ``tractor`` tree.
|
any (sub-)process in your ``tractor`` tree.
|
||||||
|
@ -219,7 +274,7 @@ We're hoping to add a respawn-from-repl system soon!
|
||||||
|
|
||||||
|
|
||||||
SC compatible bi-directional streaming
|
SC compatible bi-directional streaming
|
||||||
--------------------------------------
|
**************************************
|
||||||
Yes, you saw it here first; we provide 2-way streams
|
Yes, you saw it here first; we provide 2-way streams
|
||||||
with reliable, transitive setup/teardown semantics.
|
with reliable, transitive setup/teardown semantics.
|
||||||
|
|
||||||
|
@ -311,7 +366,7 @@ hear your thoughts on!
|
||||||
|
|
||||||
|
|
||||||
Worker poolz are easy peasy
|
Worker poolz are easy peasy
|
||||||
---------------------------
|
***************************
|
||||||
The initial ask from most new users is *"how do I make a worker
|
The initial ask from most new users is *"how do I make a worker
|
||||||
pool thing?"*.
|
pool thing?"*.
|
||||||
|
|
||||||
|
@ -333,7 +388,7 @@ This uses no extra threads, fancy semaphores or futures; all we need
|
||||||
is ``tractor``'s IPC!
|
is ``tractor``'s IPC!
|
||||||
|
|
||||||
"Infected ``asyncio``" mode
|
"Infected ``asyncio``" mode
|
||||||
---------------------------
|
***************************
|
||||||
Have a bunch of ``asyncio`` code you want to force to be SC at the process level?
|
Have a bunch of ``asyncio`` code you want to force to be SC at the process level?
|
||||||
|
|
||||||
Check out our experimental system for `guest-mode`_ controlled
|
Check out our experimental system for `guest-mode`_ controlled
|
||||||
|
@ -442,7 +497,7 @@ We need help refining the `asyncio`-side channel API to be more
|
||||||
|
|
||||||
|
|
||||||
Higher level "cluster" APIs
|
Higher level "cluster" APIs
|
||||||
---------------------------
|
***************************
|
||||||
To be extra terse the ``tractor`` devs have started hacking some "higher
|
To be extra terse the ``tractor`` devs have started hacking some "higher
|
||||||
level" APIs for managing actor trees/clusters. These interfaces should
|
level" APIs for managing actor trees/clusters. These interfaces should
|
||||||
generally be condsidered provisional for now but we encourage you to try
|
generally be condsidered provisional for now but we encourage you to try
|
||||||
|
@ -499,18 +554,6 @@ spawn a flat cluster:
|
||||||
.. _full worker pool re-implementation: https://github.com/goodboy/tractor/blob/master/examples/parallelism/concurrent_actors_primes.py
|
.. _full worker pool re-implementation: https://github.com/goodboy/tractor/blob/master/examples/parallelism/concurrent_actors_primes.py
|
||||||
|
|
||||||
|
|
||||||
Install
|
|
||||||
-------
|
|
||||||
From PyPi::
|
|
||||||
|
|
||||||
pip install tractor
|
|
||||||
|
|
||||||
|
|
||||||
From git::
|
|
||||||
|
|
||||||
pip install git+git://github.com/goodboy/tractor.git
|
|
||||||
|
|
||||||
|
|
||||||
Under the hood
|
Under the hood
|
||||||
--------------
|
--------------
|
||||||
``tractor`` is an attempt to pair trionic_ `structured concurrency`_ with
|
``tractor`` is an attempt to pair trionic_ `structured concurrency`_ with
|
||||||
|
@ -614,6 +657,7 @@ channel`_!
|
||||||
.. _adherance to: https://www.youtube.com/watch?v=7erJ1DV_Tlo&t=1821s
|
.. _adherance to: https://www.youtube.com/watch?v=7erJ1DV_Tlo&t=1821s
|
||||||
.. _trio gitter channel: https://gitter.im/python-trio/general
|
.. _trio gitter channel: https://gitter.im/python-trio/general
|
||||||
.. _matrix channel: https://matrix.to/#/!tractor:matrix.org
|
.. _matrix channel: https://matrix.to/#/!tractor:matrix.org
|
||||||
|
.. _broadcasting: https://github.com/goodboy/tractor/pull/229
|
||||||
.. _pdbp: https://github.com/mdmintz/pdbp
|
.. _pdbp: https://github.com/mdmintz/pdbp
|
||||||
.. _pdb++: https://github.com/pdbpp/pdbpp
|
.. _pdb++: https://github.com/pdbpp/pdbpp
|
||||||
.. _guest mode: https://trio.readthedocs.io/en/stable/reference-lowlevel.html?highlight=guest%20mode#using-guest-mode-to-run-trio-on-top-of-other-event-loops
|
.. _guest mode: https://trio.readthedocs.io/en/stable/reference-lowlevel.html?highlight=guest%20mode#using-guest-mode-to-run-trio-on-top-of-other-event-loops
|
||||||
|
@ -623,10 +667,10 @@ channel`_!
|
||||||
.. _structured concurrency: https://en.wikipedia.org/wiki/Structured_concurrency
|
.. _structured concurrency: https://en.wikipedia.org/wiki/Structured_concurrency
|
||||||
.. _SC: https://en.wikipedia.org/wiki/Structured_concurrency
|
.. _SC: https://en.wikipedia.org/wiki/Structured_concurrency
|
||||||
.. _libdill-docs: https://sustrik.github.io/libdill/structured-concurrency.html
|
.. _libdill-docs: https://sustrik.github.io/libdill/structured-concurrency.html
|
||||||
.. _structured chadcurrency: https://en.wikipedia.org/wiki/Structured_concurrency
|
|
||||||
.. _unrequirements: https://en.wikipedia.org/wiki/Actor_model#Direct_communication_and_asynchrony
|
.. _unrequirements: https://en.wikipedia.org/wiki/Actor_model#Direct_communication_and_asynchrony
|
||||||
.. _async generators: https://www.python.org/dev/peps/pep-0525/
|
.. _async generators: https://www.python.org/dev/peps/pep-0525/
|
||||||
.. _trio-parallel: https://github.com/richardsheridan/trio-parallel
|
.. _trio-parallel: https://github.com/richardsheridan/trio-parallel
|
||||||
|
.. _uv: https://docs.astral.sh/uv/
|
||||||
.. _msgspec: https://jcristharif.com/msgspec/
|
.. _msgspec: https://jcristharif.com/msgspec/
|
||||||
.. _guest-mode: https://trio.readthedocs.io/en/stable/reference-lowlevel.html?highlight=guest%20mode#using-guest-mode-to-run-trio-on-top-of-other-event-loops
|
.. _guest-mode: https://trio.readthedocs.io/en/stable/reference-lowlevel.html?highlight=guest%20mode#using-guest-mode-to-run-trio-on-top-of-other-event-loops
|
||||||
|
|
||||||
|
|
Loading…
Reference in New Issue