From 92f4b402ad2d45f108fc442a5edcd368938f24d3 Mon Sep 17 00:00:00 2001 From: Tyler Goodlet Date: Sun, 21 Feb 2021 23:13:51 -0500 Subject: [PATCH] Draft use sphinx-restbuilder to gen readme --- docs/README.rst | 176 ++++++++++++++++---------- docs/_README.rst | 103 +++++++++++++++ docs/github_readme/_sphinx_readme.rst | 104 +++++++++++++++ docs/github_readme/conf.py | 51 ++++++++ docs/mk_gh_readme.sh | 4 + 5 files changed, 368 insertions(+), 70 deletions(-) create mode 100644 docs/_README.rst create mode 100644 docs/github_readme/_sphinx_readme.rst create mode 100644 docs/github_readme/conf.py create mode 100755 docs/mk_gh_readme.sh diff --git a/docs/README.rst b/docs/README.rst index 4300a84..150bbed 100644 --- a/docs/README.rst +++ b/docs/README.rst @@ -1,103 +1,139 @@ + tractor -======= +******* + The Python async-native multi-core system *you always wanted*. +https://actions-badge.atrox.dev/goodboy/tractor/goto +https://tractor.readthedocs.io/en/latest/?badge=latest -|gh_actions| -|docs| +``tractor`` is a `structured concurrent +`_ +“`actor model `_” built on +`trio `_ and `multi-processing +`_. -.. _actor model: https://en.wikipedia.org/wiki/Actor_model -.. _trio: https://github.com/python-trio/trio -.. _multi-processing: https://en.wikipedia.org/wiki/Multiprocessing -.. _trionic: https://trio.readthedocs.io/en/latest/design.html#high-level-design-principles -.. _async sandwich: https://trio.readthedocs.io/en/latest/tutorial.html#async-sandwich -.. _structured concurrent: https://trio.discourse.group/t/concise-definition-of-structured-concurrency/228 - - -``tractor`` is a `structured concurrent`_ "`actor model`_" built on trio_ and multi-processing_. - -It is an attempt to pair trionic_ `structured concurrency`_ with -distributed Python. You can think of it as a ``trio`` +It is an attempt to pair `trionic +`_ +`structured concurrency +`_ +with distributed Python. You can think of it as a ``trio`` *-across-processes* or simply as an opinionated replacement for the -stdlib's ``multiprocessing`` but built on async programming primitives +stdlib’s ``multiprocessing`` but built on async programming primitives from the ground up. -Don't be scared off by this description. ``tractor`` **is just ``trio``** -but with nurseries for process management and cancel-able IPC. -If you understand how to work with ``trio``, ``tractor`` will give you -the parallelism you've been missing. +Don’t be scared off by this description. ``tractor`` **is just +``trio``** but with nurseries for process management and cancel-able +IPC. If you understand how to work with ``trio``, ``tractor`` will +give you the parallelism you’ve been missing. -``tractor``'s nurseries let you spawn ``trio`` *"actors"*: new Python +``tractor``’s nurseries let you spawn ``trio`` *“actors”*: new Python processes which each run a ``trio`` scheduled task tree (also known as -an `async sandwich`_ - a call to ``trio.run()``). That is, each -"*Actor*" is a new process plus a ``trio`` runtime. +an `async sandwich +`_ +- a call to ``trio.run()``). That is, each “*Actor*” is a new process +plus a ``trio`` runtime. -"Actors" communicate by exchanging asynchronous messages_ and avoid -sharing state. The intention of this model is to allow for highly -distributed software that, through the adherence to *structured -concurrency*, results in systems which fail in predictable and -recoverable ways. +“Actors” communicate by exchanging asynchronous `messages +`_ and avoid sharing +state. The intention of this model is to allow for highly distributed +software that, through the adherence to *structured concurrency*, +results in systems which fail in predictable and recoverable ways. -The first step to grok ``tractor`` is to get the basics of ``trio`` down. -A great place to start is the `trio docs`_ and this `blog post`_. - -.. _messages: https://en.wikipedia.org/wiki/Message_passing -.. _trio docs: https://trio.readthedocs.io/en/latest/ -.. _blog post: https://vorpus.org/blog/notes-on-structured-concurrency-or-go-statement-considered-harmful/ -.. _structured concurrency: https://vorpus.org/blog/notes-on-structured-concurrency-or-go-statement-considered-harmful/ -.. _3 axioms: https://en.wikipedia.org/wiki/Actor_model#Fundamental_concepts -.. _unrequirements: https://en.wikipedia.org/wiki/Actor_model#Direct_communication_and_asynchrony -.. _async generators: https://www.python.org/dev/peps/pep-0525/ +The first step to grok ``tractor`` is to get the basics of ``trio`` +down. A great place to start is the `trio docs +`_ and this `blog post +`_. Install -------- +======= + No PyPi release yet! :: - pip install git+git://github.com/goodboy/tractor.git + pip install git+git://github.com/goodboy/tractor.git Alluring Features ------------------ -- **It's just** ``trio``, but with SC applied to processes (aka "actors") -- Infinitely nesteable process trees -- Built-in API for inter-process streaming -- A (first ever?) "native" multi-core debugger for Python using `pdb++`_ -- (Soon to land) ``asyncio`` support allowing for "infected" actors where - `trio` drives the `asyncio` scheduler via the astounding "`guest mode`_" +================= + +* **It’s just** ``trio``, but with SC applied to processes (aka + “actors”) + +* Infinitely nesteable process trees + +* Built-in API for inter-process streaming + +* A (first ever?) “native” multi-core debugger for Python using + `pdb++ `_ + +* (Soon to land) ``asyncio`` support allowing for “infected” actors + where *trio* drives the *asyncio* scheduler via the astounding + “`guest mode + `_” -The example you're probably after... ------------------------------------- -It seems the initial query from most new users is "how do I make a worker -pool thing?". +The example you’re probably after… +================================== -``tractor`` is built to handle any SC process tree you can -imagine; the "worker pool" pattern is a trivial special case: +It seems the initial query from most new users is “how do I make a +worker pool thing?”. +``tractor`` is built to handle any SC process tree you can imagine; +the “worker pool” pattern is a trivial special case: # TODO: workerpool example +.. code:: + + """ + Run with a process monitor from a terminal using: + $TERM -e watch -n 0.1 "pstree -a $$" & python examples/parallelism/we_are_processes.py && kill $! + + """ + from multiprocessing import cpu_count + import os + + import tractor + import trio + + + async def target(): + print(f"Yo, i'm '{tractor.current_actor().name}' " + f"running in pid {os.getpid()}") + await trio.sleep_forever() + + + async def main(): + + async with tractor.open_nursery() as n: + + for i in range(cpu_count()): + await n.run_in_actor(target, name=f'worker_{i}') + + print('This process tree will self-destruct in 1 sec...') + await trio.sleep(1) + + # you could have done this yourself + raise Exception('Self Destructed') + + + if __name__ == '__main__': + try: + trio.run(main) + except Exception: + print('Zombies Contained') + Feel like saying hi? --------------------- +==================== + This project is very much coupled to the ongoing development of ``trio`` (i.e. ``tractor`` gets most of its ideas from that brilliant -community). If you want to help, have suggestions or just want to -say hi, please feel free to reach us in our `matrix channel`_. If -matrix seems too hip, we're also mostly all in the the `trio gitter -channel`_! - -.. _trio gitter channel: https://gitter.im/python-trio/general -.. _matrix channel: https://matrix.to/#/!tractor:matrix.org -.. _pdb++: https://github.com/pdbpp/pdbpp -.. _guest mode: https://trio.readthedocs.io/en/stable/reference-lowlevel.html?highlight=guest%20mode#using-guest-mode-to-run-trio-on-top-of-other-event-loops - - -.. |gh_actions| image:: https://img.shields.io/endpoint.svg?url=https%3A%2F%2Factions-badge.atrox.dev%2Fgoodboy%2Ftractor%2Fbadge&style=popout-square - :target: https://actions-badge.atrox.dev/goodboy/tractor/goto -.. |docs| image:: https://readthedocs.org/projects/tractor/badge/?version=latest - :target: https://tractor.readthedocs.io/en/latest/?badge=latest - :alt: Documentation Status +community). If you want to help, have suggestions or just want to say +hi, please feel free to reach us in our `matrix channel +`_. If matrix seems too hip, +we’re also mostly all in the the `trio gitter channel +`_! diff --git a/docs/_README.rst b/docs/_README.rst new file mode 100644 index 0000000..4300a84 --- /dev/null +++ b/docs/_README.rst @@ -0,0 +1,103 @@ +tractor +======= +The Python async-native multi-core system *you always wanted*. + + +|gh_actions| +|docs| + +.. _actor model: https://en.wikipedia.org/wiki/Actor_model +.. _trio: https://github.com/python-trio/trio +.. _multi-processing: https://en.wikipedia.org/wiki/Multiprocessing +.. _trionic: https://trio.readthedocs.io/en/latest/design.html#high-level-design-principles +.. _async sandwich: https://trio.readthedocs.io/en/latest/tutorial.html#async-sandwich +.. _structured concurrent: https://trio.discourse.group/t/concise-definition-of-structured-concurrency/228 + + +``tractor`` is a `structured concurrent`_ "`actor model`_" built on trio_ and multi-processing_. + +It is an attempt to pair trionic_ `structured concurrency`_ with +distributed Python. You can think of it as a ``trio`` +*-across-processes* or simply as an opinionated replacement for the +stdlib's ``multiprocessing`` but built on async programming primitives +from the ground up. + +Don't be scared off by this description. ``tractor`` **is just ``trio``** +but with nurseries for process management and cancel-able IPC. +If you understand how to work with ``trio``, ``tractor`` will give you +the parallelism you've been missing. + +``tractor``'s nurseries let you spawn ``trio`` *"actors"*: new Python +processes which each run a ``trio`` scheduled task tree (also known as +an `async sandwich`_ - a call to ``trio.run()``). That is, each +"*Actor*" is a new process plus a ``trio`` runtime. + +"Actors" communicate by exchanging asynchronous messages_ and avoid +sharing state. The intention of this model is to allow for highly +distributed software that, through the adherence to *structured +concurrency*, results in systems which fail in predictable and +recoverable ways. + +The first step to grok ``tractor`` is to get the basics of ``trio`` down. +A great place to start is the `trio docs`_ and this `blog post`_. + +.. _messages: https://en.wikipedia.org/wiki/Message_passing +.. _trio docs: https://trio.readthedocs.io/en/latest/ +.. _blog post: https://vorpus.org/blog/notes-on-structured-concurrency-or-go-statement-considered-harmful/ +.. _structured concurrency: https://vorpus.org/blog/notes-on-structured-concurrency-or-go-statement-considered-harmful/ +.. _3 axioms: https://en.wikipedia.org/wiki/Actor_model#Fundamental_concepts +.. _unrequirements: https://en.wikipedia.org/wiki/Actor_model#Direct_communication_and_asynchrony +.. _async generators: https://www.python.org/dev/peps/pep-0525/ + + +Install +------- +No PyPi release yet! + +:: + + pip install git+git://github.com/goodboy/tractor.git + + +Alluring Features +----------------- +- **It's just** ``trio``, but with SC applied to processes (aka "actors") +- Infinitely nesteable process trees +- Built-in API for inter-process streaming +- A (first ever?) "native" multi-core debugger for Python using `pdb++`_ +- (Soon to land) ``asyncio`` support allowing for "infected" actors where + `trio` drives the `asyncio` scheduler via the astounding "`guest mode`_" + + +The example you're probably after... +------------------------------------ +It seems the initial query from most new users is "how do I make a worker +pool thing?". + +``tractor`` is built to handle any SC process tree you can +imagine; the "worker pool" pattern is a trivial special case: + + +# TODO: workerpool example + + +Feel like saying hi? +-------------------- +This project is very much coupled to the ongoing development of +``trio`` (i.e. ``tractor`` gets most of its ideas from that brilliant +community). If you want to help, have suggestions or just want to +say hi, please feel free to reach us in our `matrix channel`_. If +matrix seems too hip, we're also mostly all in the the `trio gitter +channel`_! + +.. _trio gitter channel: https://gitter.im/python-trio/general +.. _matrix channel: https://matrix.to/#/!tractor:matrix.org +.. _pdb++: https://github.com/pdbpp/pdbpp +.. _guest mode: https://trio.readthedocs.io/en/stable/reference-lowlevel.html?highlight=guest%20mode#using-guest-mode-to-run-trio-on-top-of-other-event-loops + + +.. |gh_actions| image:: https://img.shields.io/endpoint.svg?url=https%3A%2F%2Factions-badge.atrox.dev%2Fgoodboy%2Ftractor%2Fbadge&style=popout-square + :target: https://actions-badge.atrox.dev/goodboy/tractor/goto +.. |docs| image:: https://readthedocs.org/projects/tractor/badge/?version=latest + :target: https://tractor.readthedocs.io/en/latest/?badge=latest + :alt: Documentation Status diff --git a/docs/github_readme/_sphinx_readme.rst b/docs/github_readme/_sphinx_readme.rst new file mode 100644 index 0000000..03b09d5 --- /dev/null +++ b/docs/github_readme/_sphinx_readme.rst @@ -0,0 +1,104 @@ +tractor +======= +The Python async-native multi-core system *you always wanted*. + + +|gh_actions| +|docs| + +.. _actor model: https://en.wikipedia.org/wiki/Actor_model +.. _trio: https://github.com/python-trio/trio +.. _multi-processing: https://en.wikipedia.org/wiki/Multiprocessing +.. _trionic: https://trio.readthedocs.io/en/latest/design.html#high-level-design-principles +.. _async sandwich: https://trio.readthedocs.io/en/latest/tutorial.html#async-sandwich +.. _structured concurrent: https://trio.discourse.group/t/concise-definition-of-structured-concurrency/228 + + +``tractor`` is a `structured concurrent`_ "`actor model`_" built on trio_ and multi-processing_. + +It is an attempt to pair trionic_ `structured concurrency`_ with +distributed Python. You can think of it as a ``trio`` +*-across-processes* or simply as an opinionated replacement for the +stdlib's ``multiprocessing`` but built on async programming primitives +from the ground up. + +Don't be scared off by this description. ``tractor`` **is just ``trio``** +but with nurseries for process management and cancel-able IPC. +If you understand how to work with ``trio``, ``tractor`` will give you +the parallelism you've been missing. + +``tractor``'s nurseries let you spawn ``trio`` *"actors"*: new Python +processes which each run a ``trio`` scheduled task tree (also known as +an `async sandwich`_ - a call to ``trio.run()``). That is, each +"*Actor*" is a new process plus a ``trio`` runtime. + +"Actors" communicate by exchanging asynchronous messages_ and avoid +sharing state. The intention of this model is to allow for highly +distributed software that, through the adherence to *structured +concurrency*, results in systems which fail in predictable and +recoverable ways. + +The first step to grok ``tractor`` is to get the basics of ``trio`` down. +A great place to start is the `trio docs`_ and this `blog post`_. + +.. _messages: https://en.wikipedia.org/wiki/Message_passing +.. _trio docs: https://trio.readthedocs.io/en/latest/ +.. _blog post: https://vorpus.org/blog/notes-on-structured-concurrency-or-go-statement-considered-harmful/ +.. _structured concurrency: https://vorpus.org/blog/notes-on-structured-concurrency-or-go-statement-considered-harmful/ +.. _3 axioms: https://en.wikipedia.org/wiki/Actor_model#Fundamental_concepts +.. _unrequirements: https://en.wikipedia.org/wiki/Actor_model#Direct_communication_and_asynchrony +.. _async generators: https://www.python.org/dev/peps/pep-0525/ + + +Install +------- +No PyPi release yet! + +:: + + pip install git+git://github.com/goodboy/tractor.git + + +Alluring Features +----------------- +- **It's just** ``trio``, but with SC applied to processes (aka "actors") +- Infinitely nesteable process trees +- Built-in API for inter-process streaming +- A (first ever?) "native" multi-core debugger for Python using `pdb++`_ +- (Soon to land) ``asyncio`` support allowing for "infected" actors where + `trio` drives the `asyncio` scheduler via the astounding "`guest mode`_" + + +The example you're probably after... +------------------------------------ +It seems the initial query from most new users is "how do I make a worker +pool thing?". + +``tractor`` is built to handle any SC process tree you can +imagine; the "worker pool" pattern is a trivial special case: + +# TODO: workerpool example + +.. literalinclude:: ../../examples/parallelism/we_are_processes.py + + +Feel like saying hi? +-------------------- +This project is very much coupled to the ongoing development of +``trio`` (i.e. ``tractor`` gets most of its ideas from that brilliant +community). If you want to help, have suggestions or just want to +say hi, please feel free to reach us in our `matrix channel`_. If +matrix seems too hip, we're also mostly all in the the `trio gitter +channel`_! + +.. _trio gitter channel: https://gitter.im/python-trio/general +.. _matrix channel: https://matrix.to/#/!tractor:matrix.org +.. _pdb++: https://github.com/pdbpp/pdbpp +.. _guest mode: https://trio.readthedocs.io/en/stable/reference-lowlevel.html?highlight=guest%20mode#using-guest-mode-to-run-trio-on-top-of-other-event-loops + + +.. |gh_actions| image:: https://img.shields.io/endpoint.svg?url=https%3A%2F%2Factions-badge.atrox.dev%2Fgoodboy%2Ftractor%2Fbadge&style=popout-square + :target: https://actions-badge.atrox.dev/goodboy/tractor/goto +.. |docs| image:: https://readthedocs.org/projects/tractor/badge/?version=latest + :target: https://tractor.readthedocs.io/en/latest/?badge=latest + :alt: Documentation Status diff --git a/docs/github_readme/conf.py b/docs/github_readme/conf.py new file mode 100644 index 0000000..04d288e --- /dev/null +++ b/docs/github_readme/conf.py @@ -0,0 +1,51 @@ +# Configuration file for the Sphinx documentation builder. + +# this config is for the rst generation extension and thus +# requires only basic settings: +# https://github.com/sphinx-contrib/restbuilder + +# -- Path setup -------------------------------------------------------------- + +# If extensions (or modules to document with autodoc) are in another directory, +# add these directories to sys.path here. If the directory is relative to the +# documentation root, use os.path.abspath to make it absolute, like shown here. +# +# import os +# import sys +# sys.path.insert(0, os.path.abspath('.')) + +# Warn about all references to unknown targets +nitpicky = True + +# The master toctree document. +master_doc = '_sphinx_readme' + +# -- Project information ----------------------------------------------------- + +project = 'tractor' +copyright = '2018, Tyler Goodlet' +author = 'Tyler Goodlet' + +# The full version, including alpha/beta/rc tags +release = '0.0.0a0.dev0' + +# -- General configuration --------------------------------------------------- + +# Add any Sphinx extension module names here, as strings. They can be +# extensions coming with Sphinx (named 'sphinx.ext.*') or your custom +# ones. +extensions = [ + # 'sphinx.ext.autodoc', + # 'sphinx.ext.intersphinx', + # 'sphinx.ext.todo', + 'sphinxcontrib.restbuilder', + +] + +# Add any paths that contain templates here, relative to this directory. +templates_path = ['_templates'] + +# List of patterns, relative to source directory, that match files and +# directories to ignore when looking for source files. +# This pattern also affects html_static_path and html_extra_path. +exclude_patterns = ['_build', 'Thumbs.db', '.DS_Store'] diff --git a/docs/mk_gh_readme.sh b/docs/mk_gh_readme.sh new file mode 100755 index 0000000..3836e2f --- /dev/null +++ b/docs/mk_gh_readme.sh @@ -0,0 +1,4 @@ +#!/bin/bash +sphinx-build -b rst ./github_readme ./ + +mv _sphinx_readme.rst README.rst