tractor/docs/README.rst

tractor
*******

The Python async-native multi-core system *you always wanted*.

https://actions-badge.atrox.dev/goodboy/tractor/goto
https://tractor.readthedocs.io/en/latest/?badge=latest

``tractor`` is a `structured concurrent
<https://trio.discourse.group/t/concise-definition-of-structured-concurrency/228>`_
“`actor model <https://en.wikipedia.org/wiki/Actor_model>`_” built on
`trio <https://github.com/python-trio/trio>`_ and `multi-processing
<https://en.wikipedia.org/wiki/Multiprocessing>`_.

It is an attempt to pair `trionic
<https://trio.readthedocs.io/en/latest/design.html#high-level-design-principles>`_
`structured concurrency
<https://vorpus.org/blog/notes-on-structured-concurrency-or-go-statement-considered-harmful/>`_
with distributed Python. You can think of it as a ``trio``
*-across-processes* or simply as an opinionated replacement for the
stdlib’s ``multiprocessing`` but built on async programming primitives
from the ground up.

Don’t be scared off by this description. ``tractor`` **is just
``trio``** but with nurseries for process management and cancel-able
IPC. If you understand how to work with ``trio``, ``tractor`` will
give you the parallelism you’ve been missing.

``tractor``’s nurseries let you spawn ``trio`` *“actors”*: new Python
processes which each run a ``trio`` scheduled task tree (also known as
an `async sandwich
<https://trio.readthedocs.io/en/latest/tutorial.html#async-sandwich>`_
- a call to ``trio.run()``). That is, each “*Actor*” is a new process
plus a ``trio`` runtime.

“Actors” communicate by exchanging asynchronous `messages
<https://en.wikipedia.org/wiki/Message_passing>`_ and avoid sharing
state. The intention of this model is to allow for highly distributed
software that, through the adherence to *structured concurrency*,
results in systems which fail in predictable and recoverable ways.

The first step to grok ``tractor`` is to get the basics of ``trio``
down. A great place to start is the `trio docs
<https://trio.readthedocs.io/en/latest/>`_ and this `blog post
<https://vorpus.org/blog/notes-on-structured-concurrency-or-go-statement-considered-harmful/>`_.


Install
=======

No PyPi release yet!

::

   pip install git+git://github.com/goodboy/tractor.git


Alluring Features
=================

* **It’s just** ``trio``, but with SC applied to processes (aka
   “actors”)

* Infinitely nesteable process trees

* Built-in API for inter-process streaming

* A (first ever?) “native” multi-core debugger for Python using
   `pdb++ <https://github.com/pdbpp/pdbpp>`_

* (Soon to land) ``asyncio`` support allowing for “infected” actors
   where *trio* drives the *asyncio* scheduler via the astounding
   “`guest mode
   <https://trio.readthedocs.io/en/stable/reference-lowlevel.html?highlight=guest%20mode#using-guest-mode-to-run-trio-on-top-of-other-event-loops>`_”


The example you’re probably after…
==================================

It seems the initial query from most new users is “how do I make a
worker pool thing?”.

``tractor`` is built to handle any SC process tree you can imagine;
the “worker pool” pattern is a trivial special case:

# TODO: workerpool example

.. code::

   """
   Run with a process monitor from a terminal using:
   $TERM -e watch -n 0.1  "pstree -a $$" & python examples/parallelism/we_are_processes.py && kill $!

   """
   from multiprocessing import cpu_count
   import os

   import tractor
   import trio


   async def target():
       print(f"Yo, i'm '{tractor.current_actor().name}' "
             f"running in pid {os.getpid()}")
       await trio.sleep_forever()


   async def main():

       async with tractor.open_nursery() as n:

           for i in range(cpu_count()):
               await n.run_in_actor(target, name=f'worker_{i}')

           print('This process tree will self-destruct in 1 sec...')
           await trio.sleep(1)

           # you could have done this yourself
           raise Exception('Self Destructed')


   if __name__ == '__main__':
       try:
           trio.run(main)
       except Exception:
           print('Zombies Contained')


Feel like saying hi?
====================

This project is very much coupled to the ongoing development of
``trio`` (i.e. ``tractor`` gets most of its ideas from that brilliant
community). If you want to help, have suggestions or just want to say
hi, please feel free to reach us in our `matrix channel
<https://matrix.to/#/!tractor:matrix.org>`_.  If matrix seems too hip,
we’re also mostly all in the the `trio gitter channel
<https://gitter.im/python-trio/general>`_!