forked from goodboy/tractor
1
0
Fork 0

Draft use sphinx-restbuilder to gen readme

readme_pump
Tyler Goodlet 2021-02-21 23:13:51 -05:00
parent 90c987d0ae
commit 92f4b402ad
5 changed files with 368 additions and 70 deletions

View File

@ -1,57 +1,54 @@
tractor tractor
======= *******
The Python async-native multi-core system *you always wanted*. The Python async-native multi-core system *you always wanted*.
https://actions-badge.atrox.dev/goodboy/tractor/goto
https://tractor.readthedocs.io/en/latest/?badge=latest
|gh_actions| ``tractor`` is a `structured concurrent
|docs| <https://trio.discourse.group/t/concise-definition-of-structured-concurrency/228>`_
`actor model <https://en.wikipedia.org/wiki/Actor_model>`_” built on
`trio <https://github.com/python-trio/trio>`_ and `multi-processing
<https://en.wikipedia.org/wiki/Multiprocessing>`_.
.. _actor model: https://en.wikipedia.org/wiki/Actor_model It is an attempt to pair `trionic
.. _trio: https://github.com/python-trio/trio <https://trio.readthedocs.io/en/latest/design.html#high-level-design-principles>`_
.. _multi-processing: https://en.wikipedia.org/wiki/Multiprocessing `structured concurrency
.. _trionic: https://trio.readthedocs.io/en/latest/design.html#high-level-design-principles <https://vorpus.org/blog/notes-on-structured-concurrency-or-go-statement-considered-harmful/>`_
.. _async sandwich: https://trio.readthedocs.io/en/latest/tutorial.html#async-sandwich with distributed Python. You can think of it as a ``trio``
.. _structured concurrent: https://trio.discourse.group/t/concise-definition-of-structured-concurrency/228
``tractor`` is a `structured concurrent`_ "`actor model`_" built on trio_ and multi-processing_.
It is an attempt to pair trionic_ `structured concurrency`_ with
distributed Python. You can think of it as a ``trio``
*-across-processes* or simply as an opinionated replacement for the *-across-processes* or simply as an opinionated replacement for the
stdlib's ``multiprocessing`` but built on async programming primitives stdlibs ``multiprocessing`` but built on async programming primitives
from the ground up. from the ground up.
Don't be scared off by this description. ``tractor`` **is just ``trio``** Dont be scared off by this description. ``tractor`` **is just
but with nurseries for process management and cancel-able IPC. ``trio``** but with nurseries for process management and cancel-able
If you understand how to work with ``trio``, ``tractor`` will give you IPC. If you understand how to work with ``trio``, ``tractor`` will
the parallelism you've been missing. give you the parallelism youve been missing.
``tractor``'s nurseries let you spawn ``trio`` *"actors"*: new Python ``tractor``s nurseries let you spawn ``trio`` *“actors”*: new Python
processes which each run a ``trio`` scheduled task tree (also known as processes which each run a ``trio`` scheduled task tree (also known as
an `async sandwich`_ - a call to ``trio.run()``). That is, each an `async sandwich
"*Actor*" is a new process plus a ``trio`` runtime. <https://trio.readthedocs.io/en/latest/tutorial.html#async-sandwich>`_
- a call to ``trio.run()``). That is, each “*Actor*” is a new process
plus a ``trio`` runtime.
"Actors" communicate by exchanging asynchronous messages_ and avoid “Actors” communicate by exchanging asynchronous `messages
sharing state. The intention of this model is to allow for highly <https://en.wikipedia.org/wiki/Message_passing>`_ and avoid sharing
distributed software that, through the adherence to *structured state. The intention of this model is to allow for highly distributed
concurrency*, results in systems which fail in predictable and software that, through the adherence to *structured concurrency*,
recoverable ways. results in systems which fail in predictable and recoverable ways.
The first step to grok ``tractor`` is to get the basics of ``trio`` down. The first step to grok ``tractor`` is to get the basics of ``trio``
A great place to start is the `trio docs`_ and this `blog post`_. down. A great place to start is the `trio docs
<https://trio.readthedocs.io/en/latest/>`_ and this `blog post
.. _messages: https://en.wikipedia.org/wiki/Message_passing <https://vorpus.org/blog/notes-on-structured-concurrency-or-go-statement-considered-harmful/>`_.
.. _trio docs: https://trio.readthedocs.io/en/latest/
.. _blog post: https://vorpus.org/blog/notes-on-structured-concurrency-or-go-statement-considered-harmful/
.. _structured concurrency: https://vorpus.org/blog/notes-on-structured-concurrency-or-go-statement-considered-harmful/
.. _3 axioms: https://en.wikipedia.org/wiki/Actor_model#Fundamental_concepts
.. _unrequirements: https://en.wikipedia.org/wiki/Actor_model#Direct_communication_and_asynchrony
.. _async generators: https://www.python.org/dev/peps/pep-0525/
Install Install
------- =======
No PyPi release yet! No PyPi release yet!
:: ::
@ -60,44 +57,83 @@ No PyPi release yet!
Alluring Features Alluring Features
----------------- =================
- **It's just** ``trio``, but with SC applied to processes (aka "actors")
- Infinitely nesteable process trees * **Its just** ``trio``, but with SC applied to processes (aka
- Built-in API for inter-process streaming “actors”)
- A (first ever?) "native" multi-core debugger for Python using `pdb++`_
- (Soon to land) ``asyncio`` support allowing for "infected" actors where * Infinitely nesteable process trees
`trio` drives the `asyncio` scheduler via the astounding "`guest mode`_"
* Built-in API for inter-process streaming
* A (first ever?) “native” multi-core debugger for Python using
`pdb++ <https://github.com/pdbpp/pdbpp>`_
* (Soon to land) ``asyncio`` support allowing for “infected” actors
where *trio* drives the *asyncio* scheduler via the astounding
“`guest mode
<https://trio.readthedocs.io/en/stable/reference-lowlevel.html?highlight=guest%20mode#using-guest-mode-to-run-trio-on-top-of-other-event-loops>`_”
The example you're probably after... The example youre probably after…
------------------------------------ ==================================
It seems the initial query from most new users is "how do I make a worker
pool thing?".
``tractor`` is built to handle any SC process tree you can It seems the initial query from most new users is “how do I make a
imagine; the "worker pool" pattern is a trivial special case: worker pool thing?”.
``tractor`` is built to handle any SC process tree you can imagine;
the “worker pool” pattern is a trivial special case:
# TODO: workerpool example # TODO: workerpool example
.. code::
"""
Run with a process monitor from a terminal using:
$TERM -e watch -n 0.1 "pstree -a $$" & python examples/parallelism/we_are_processes.py && kill $!
"""
from multiprocessing import cpu_count
import os
import tractor
import trio
async def target():
print(f"Yo, i'm '{tractor.current_actor().name}' "
f"running in pid {os.getpid()}")
await trio.sleep_forever()
async def main():
async with tractor.open_nursery() as n:
for i in range(cpu_count()):
await n.run_in_actor(target, name=f'worker_{i}')
print('This process tree will self-destruct in 1 sec...')
await trio.sleep(1)
# you could have done this yourself
raise Exception('Self Destructed')
if __name__ == '__main__':
try:
trio.run(main)
except Exception:
print('Zombies Contained')
Feel like saying hi? Feel like saying hi?
-------------------- ====================
This project is very much coupled to the ongoing development of This project is very much coupled to the ongoing development of
``trio`` (i.e. ``tractor`` gets most of its ideas from that brilliant ``trio`` (i.e. ``tractor`` gets most of its ideas from that brilliant
community). If you want to help, have suggestions or just want to community). If you want to help, have suggestions or just want to say
say hi, please feel free to reach us in our `matrix channel`_. If hi, please feel free to reach us in our `matrix channel
matrix seems too hip, we're also mostly all in the the `trio gitter <https://matrix.to/#/!tractor:matrix.org>`_. If matrix seems too hip,
channel`_! were also mostly all in the the `trio gitter channel
<https://gitter.im/python-trio/general>`_!
.. _trio gitter channel: https://gitter.im/python-trio/general
.. _matrix channel: https://matrix.to/#/!tractor:matrix.org
.. _pdb++: https://github.com/pdbpp/pdbpp
.. _guest mode: https://trio.readthedocs.io/en/stable/reference-lowlevel.html?highlight=guest%20mode#using-guest-mode-to-run-trio-on-top-of-other-event-loops
.. |gh_actions| image:: https://img.shields.io/endpoint.svg?url=https%3A%2F%2Factions-badge.atrox.dev%2Fgoodboy%2Ftractor%2Fbadge&style=popout-square
:target: https://actions-badge.atrox.dev/goodboy/tractor/goto
.. |docs| image:: https://readthedocs.org/projects/tractor/badge/?version=latest
:target: https://tractor.readthedocs.io/en/latest/?badge=latest
:alt: Documentation Status

103
docs/_README.rst 100644
View File

@ -0,0 +1,103 @@
tractor
=======
The Python async-native multi-core system *you always wanted*.
|gh_actions|
|docs|
.. _actor model: https://en.wikipedia.org/wiki/Actor_model
.. _trio: https://github.com/python-trio/trio
.. _multi-processing: https://en.wikipedia.org/wiki/Multiprocessing
.. _trionic: https://trio.readthedocs.io/en/latest/design.html#high-level-design-principles
.. _async sandwich: https://trio.readthedocs.io/en/latest/tutorial.html#async-sandwich
.. _structured concurrent: https://trio.discourse.group/t/concise-definition-of-structured-concurrency/228
``tractor`` is a `structured concurrent`_ "`actor model`_" built on trio_ and multi-processing_.
It is an attempt to pair trionic_ `structured concurrency`_ with
distributed Python. You can think of it as a ``trio``
*-across-processes* or simply as an opinionated replacement for the
stdlib's ``multiprocessing`` but built on async programming primitives
from the ground up.
Don't be scared off by this description. ``tractor`` **is just ``trio``**
but with nurseries for process management and cancel-able IPC.
If you understand how to work with ``trio``, ``tractor`` will give you
the parallelism you've been missing.
``tractor``'s nurseries let you spawn ``trio`` *"actors"*: new Python
processes which each run a ``trio`` scheduled task tree (also known as
an `async sandwich`_ - a call to ``trio.run()``). That is, each
"*Actor*" is a new process plus a ``trio`` runtime.
"Actors" communicate by exchanging asynchronous messages_ and avoid
sharing state. The intention of this model is to allow for highly
distributed software that, through the adherence to *structured
concurrency*, results in systems which fail in predictable and
recoverable ways.
The first step to grok ``tractor`` is to get the basics of ``trio`` down.
A great place to start is the `trio docs`_ and this `blog post`_.
.. _messages: https://en.wikipedia.org/wiki/Message_passing
.. _trio docs: https://trio.readthedocs.io/en/latest/
.. _blog post: https://vorpus.org/blog/notes-on-structured-concurrency-or-go-statement-considered-harmful/
.. _structured concurrency: https://vorpus.org/blog/notes-on-structured-concurrency-or-go-statement-considered-harmful/
.. _3 axioms: https://en.wikipedia.org/wiki/Actor_model#Fundamental_concepts
.. _unrequirements: https://en.wikipedia.org/wiki/Actor_model#Direct_communication_and_asynchrony
.. _async generators: https://www.python.org/dev/peps/pep-0525/
Install
-------
No PyPi release yet!
::
pip install git+git://github.com/goodboy/tractor.git
Alluring Features
-----------------
- **It's just** ``trio``, but with SC applied to processes (aka "actors")
- Infinitely nesteable process trees
- Built-in API for inter-process streaming
- A (first ever?) "native" multi-core debugger for Python using `pdb++`_
- (Soon to land) ``asyncio`` support allowing for "infected" actors where
`trio` drives the `asyncio` scheduler via the astounding "`guest mode`_"
The example you're probably after...
------------------------------------
It seems the initial query from most new users is "how do I make a worker
pool thing?".
``tractor`` is built to handle any SC process tree you can
imagine; the "worker pool" pattern is a trivial special case:
# TODO: workerpool example
Feel like saying hi?
--------------------
This project is very much coupled to the ongoing development of
``trio`` (i.e. ``tractor`` gets most of its ideas from that brilliant
community). If you want to help, have suggestions or just want to
say hi, please feel free to reach us in our `matrix channel`_. If
matrix seems too hip, we're also mostly all in the the `trio gitter
channel`_!
.. _trio gitter channel: https://gitter.im/python-trio/general
.. _matrix channel: https://matrix.to/#/!tractor:matrix.org
.. _pdb++: https://github.com/pdbpp/pdbpp
.. _guest mode: https://trio.readthedocs.io/en/stable/reference-lowlevel.html?highlight=guest%20mode#using-guest-mode-to-run-trio-on-top-of-other-event-loops
.. |gh_actions| image:: https://img.shields.io/endpoint.svg?url=https%3A%2F%2Factions-badge.atrox.dev%2Fgoodboy%2Ftractor%2Fbadge&style=popout-square
:target: https://actions-badge.atrox.dev/goodboy/tractor/goto
.. |docs| image:: https://readthedocs.org/projects/tractor/badge/?version=latest
:target: https://tractor.readthedocs.io/en/latest/?badge=latest
:alt: Documentation Status

View File

@ -0,0 +1,104 @@
tractor
=======
The Python async-native multi-core system *you always wanted*.
|gh_actions|
|docs|
.. _actor model: https://en.wikipedia.org/wiki/Actor_model
.. _trio: https://github.com/python-trio/trio
.. _multi-processing: https://en.wikipedia.org/wiki/Multiprocessing
.. _trionic: https://trio.readthedocs.io/en/latest/design.html#high-level-design-principles
.. _async sandwich: https://trio.readthedocs.io/en/latest/tutorial.html#async-sandwich
.. _structured concurrent: https://trio.discourse.group/t/concise-definition-of-structured-concurrency/228
``tractor`` is a `structured concurrent`_ "`actor model`_" built on trio_ and multi-processing_.
It is an attempt to pair trionic_ `structured concurrency`_ with
distributed Python. You can think of it as a ``trio``
*-across-processes* or simply as an opinionated replacement for the
stdlib's ``multiprocessing`` but built on async programming primitives
from the ground up.
Don't be scared off by this description. ``tractor`` **is just ``trio``**
but with nurseries for process management and cancel-able IPC.
If you understand how to work with ``trio``, ``tractor`` will give you
the parallelism you've been missing.
``tractor``'s nurseries let you spawn ``trio`` *"actors"*: new Python
processes which each run a ``trio`` scheduled task tree (also known as
an `async sandwich`_ - a call to ``trio.run()``). That is, each
"*Actor*" is a new process plus a ``trio`` runtime.
"Actors" communicate by exchanging asynchronous messages_ and avoid
sharing state. The intention of this model is to allow for highly
distributed software that, through the adherence to *structured
concurrency*, results in systems which fail in predictable and
recoverable ways.
The first step to grok ``tractor`` is to get the basics of ``trio`` down.
A great place to start is the `trio docs`_ and this `blog post`_.
.. _messages: https://en.wikipedia.org/wiki/Message_passing
.. _trio docs: https://trio.readthedocs.io/en/latest/
.. _blog post: https://vorpus.org/blog/notes-on-structured-concurrency-or-go-statement-considered-harmful/
.. _structured concurrency: https://vorpus.org/blog/notes-on-structured-concurrency-or-go-statement-considered-harmful/
.. _3 axioms: https://en.wikipedia.org/wiki/Actor_model#Fundamental_concepts
.. _unrequirements: https://en.wikipedia.org/wiki/Actor_model#Direct_communication_and_asynchrony
.. _async generators: https://www.python.org/dev/peps/pep-0525/
Install
-------
No PyPi release yet!
::
pip install git+git://github.com/goodboy/tractor.git
Alluring Features
-----------------
- **It's just** ``trio``, but with SC applied to processes (aka "actors")
- Infinitely nesteable process trees
- Built-in API for inter-process streaming
- A (first ever?) "native" multi-core debugger for Python using `pdb++`_
- (Soon to land) ``asyncio`` support allowing for "infected" actors where
`trio` drives the `asyncio` scheduler via the astounding "`guest mode`_"
The example you're probably after...
------------------------------------
It seems the initial query from most new users is "how do I make a worker
pool thing?".
``tractor`` is built to handle any SC process tree you can
imagine; the "worker pool" pattern is a trivial special case:
# TODO: workerpool example
.. literalinclude:: ../../examples/parallelism/we_are_processes.py
Feel like saying hi?
--------------------
This project is very much coupled to the ongoing development of
``trio`` (i.e. ``tractor`` gets most of its ideas from that brilliant
community). If you want to help, have suggestions or just want to
say hi, please feel free to reach us in our `matrix channel`_. If
matrix seems too hip, we're also mostly all in the the `trio gitter
channel`_!
.. _trio gitter channel: https://gitter.im/python-trio/general
.. _matrix channel: https://matrix.to/#/!tractor:matrix.org
.. _pdb++: https://github.com/pdbpp/pdbpp
.. _guest mode: https://trio.readthedocs.io/en/stable/reference-lowlevel.html?highlight=guest%20mode#using-guest-mode-to-run-trio-on-top-of-other-event-loops
.. |gh_actions| image:: https://img.shields.io/endpoint.svg?url=https%3A%2F%2Factions-badge.atrox.dev%2Fgoodboy%2Ftractor%2Fbadge&style=popout-square
:target: https://actions-badge.atrox.dev/goodboy/tractor/goto
.. |docs| image:: https://readthedocs.org/projects/tractor/badge/?version=latest
:target: https://tractor.readthedocs.io/en/latest/?badge=latest
:alt: Documentation Status

View File

@ -0,0 +1,51 @@
# Configuration file for the Sphinx documentation builder.
# this config is for the rst generation extension and thus
# requires only basic settings:
# https://github.com/sphinx-contrib/restbuilder
# -- Path setup --------------------------------------------------------------
# If extensions (or modules to document with autodoc) are in another directory,
# add these directories to sys.path here. If the directory is relative to the
# documentation root, use os.path.abspath to make it absolute, like shown here.
#
# import os
# import sys
# sys.path.insert(0, os.path.abspath('.'))
# Warn about all references to unknown targets
nitpicky = True
# The master toctree document.
master_doc = '_sphinx_readme'
# -- Project information -----------------------------------------------------
project = 'tractor'
copyright = '2018, Tyler Goodlet'
author = 'Tyler Goodlet'
# The full version, including alpha/beta/rc tags
release = '0.0.0a0.dev0'
# -- General configuration ---------------------------------------------------
# Add any Sphinx extension module names here, as strings. They can be
# extensions coming with Sphinx (named 'sphinx.ext.*') or your custom
# ones.
extensions = [
# 'sphinx.ext.autodoc',
# 'sphinx.ext.intersphinx',
# 'sphinx.ext.todo',
'sphinxcontrib.restbuilder',
]
# Add any paths that contain templates here, relative to this directory.
templates_path = ['_templates']
# List of patterns, relative to source directory, that match files and
# directories to ignore when looking for source files.
# This pattern also affects html_static_path and html_extra_path.
exclude_patterns = ['_build', 'Thumbs.db', '.DS_Store']

View File

@ -0,0 +1,4 @@
#!/bin/bash
sphinx-build -b rst ./github_readme ./
mv _sphinx_readme.rst README.rst