140 lines
4.5 KiB
ReStructuredText
140 lines
4.5 KiB
ReStructuredText
|
||
tractor
|
||
*******
|
||
|
||
The Python async-native multi-core system *you always wanted*.
|
||
|
||
https://actions-badge.atrox.dev/goodboy/tractor/goto
|
||
https://tractor.readthedocs.io/en/latest/?badge=latest
|
||
|
||
``tractor`` is a `structured concurrent
|
||
<https://trio.discourse.group/t/concise-definition-of-structured-concurrency/228>`_
|
||
“`actor model <https://en.wikipedia.org/wiki/Actor_model>`_” built on
|
||
`trio <https://github.com/python-trio/trio>`_ and `multi-processing
|
||
<https://en.wikipedia.org/wiki/Multiprocessing>`_.
|
||
|
||
It is an attempt to pair `trionic
|
||
<https://trio.readthedocs.io/en/latest/design.html#high-level-design-principles>`_
|
||
`structured concurrency
|
||
<https://vorpus.org/blog/notes-on-structured-concurrency-or-go-statement-considered-harmful/>`_
|
||
with distributed Python. You can think of it as a ``trio``
|
||
*-across-processes* or simply as an opinionated replacement for the
|
||
stdlib’s ``multiprocessing`` but built on async programming primitives
|
||
from the ground up.
|
||
|
||
Don’t be scared off by this description. ``tractor`` **is just
|
||
``trio``** but with nurseries for process management and cancel-able
|
||
IPC. If you understand how to work with ``trio``, ``tractor`` will
|
||
give you the parallelism you’ve been missing.
|
||
|
||
``tractor``’s nurseries let you spawn ``trio`` *“actors”*: new Python
|
||
processes which each run a ``trio`` scheduled task tree (also known as
|
||
an `async sandwich
|
||
<https://trio.readthedocs.io/en/latest/tutorial.html#async-sandwich>`_
|
||
- a call to ``trio.run()``). That is, each “*Actor*” is a new process
|
||
plus a ``trio`` runtime.
|
||
|
||
“Actors” communicate by exchanging asynchronous `messages
|
||
<https://en.wikipedia.org/wiki/Message_passing>`_ and avoid sharing
|
||
state. The intention of this model is to allow for highly distributed
|
||
software that, through the adherence to *structured concurrency*,
|
||
results in systems which fail in predictable and recoverable ways.
|
||
|
||
The first step to grok ``tractor`` is to get the basics of ``trio``
|
||
down. A great place to start is the `trio docs
|
||
<https://trio.readthedocs.io/en/latest/>`_ and this `blog post
|
||
<https://vorpus.org/blog/notes-on-structured-concurrency-or-go-statement-considered-harmful/>`_.
|
||
|
||
|
||
Install
|
||
=======
|
||
|
||
No PyPi release yet!
|
||
|
||
::
|
||
|
||
pip install git+git://github.com/goodboy/tractor.git
|
||
|
||
|
||
Alluring Features
|
||
=================
|
||
|
||
* **It’s just** ``trio``, but with SC applied to processes (aka
|
||
“actors”)
|
||
|
||
* Infinitely nesteable process trees
|
||
|
||
* Built-in API for inter-process streaming
|
||
|
||
* A (first ever?) “native” multi-core debugger for Python using
|
||
`pdb++ <https://github.com/pdbpp/pdbpp>`_
|
||
|
||
* (Soon to land) ``asyncio`` support allowing for “infected” actors
|
||
where *trio* drives the *asyncio* scheduler via the astounding
|
||
“`guest mode
|
||
<https://trio.readthedocs.io/en/stable/reference-lowlevel.html?highlight=guest%20mode#using-guest-mode-to-run-trio-on-top-of-other-event-loops>`_”
|
||
|
||
|
||
The example you’re probably after…
|
||
==================================
|
||
|
||
It seems the initial query from most new users is “how do I make a
|
||
worker pool thing?”.
|
||
|
||
``tractor`` is built to handle any SC process tree you can imagine;
|
||
the “worker pool” pattern is a trivial special case:
|
||
|
||
# TODO: workerpool example
|
||
|
||
.. code::
|
||
|
||
"""
|
||
Run with a process monitor from a terminal using:
|
||
$TERM -e watch -n 0.1 "pstree -a $$" & python examples/parallelism/we_are_processes.py && kill $!
|
||
|
||
"""
|
||
from multiprocessing import cpu_count
|
||
import os
|
||
|
||
import tractor
|
||
import trio
|
||
|
||
|
||
async def target():
|
||
print(f"Yo, i'm '{tractor.current_actor().name}' "
|
||
f"running in pid {os.getpid()}")
|
||
await trio.sleep_forever()
|
||
|
||
|
||
async def main():
|
||
|
||
async with tractor.open_nursery() as n:
|
||
|
||
for i in range(cpu_count()):
|
||
await n.run_in_actor(target, name=f'worker_{i}')
|
||
|
||
print('This process tree will self-destruct in 1 sec...')
|
||
await trio.sleep(1)
|
||
|
||
# you could have done this yourself
|
||
raise Exception('Self Destructed')
|
||
|
||
|
||
if __name__ == '__main__':
|
||
try:
|
||
trio.run(main)
|
||
except Exception:
|
||
print('Zombies Contained')
|
||
|
||
|
||
Feel like saying hi?
|
||
====================
|
||
|
||
This project is very much coupled to the ongoing development of
|
||
``trio`` (i.e. ``tractor`` gets most of its ideas from that brilliant
|
||
community). If you want to help, have suggestions or just want to say
|
||
hi, please feel free to reach us in our `matrix channel
|
||
<https://matrix.to/#/!tractor:matrix.org>`_. If matrix seems too hip,
|
||
we’re also mostly all in the the `trio gitter channel
|
||
<https://gitter.im/python-trio/general>`_!
|