Commit Graph

28 Commits (663373cb044ec6dd51ef3e27461c0e5d66a02498)

Author SHA1 Message Date
Tyler Goodlet 7b6e34aaf4 Better formalize `pikerd` service semantics
An async exit stack around the new `@tractor.context` is problematic
since a pushed context can't bubble errors unless the exit stack has
been closed. But in that case why do you need the exit stack if you're
going to push it and wait it right away; it seems more correct to use
a nursery and spawn a task in `pikerd` that waits on the both the
target context completion first (thus being able to bubble up any errors
from the remote, and top level service task) and the sub-actor portal.
(Sub)service Daemons are spawned with `.start_actor()` and thus will
block forever until cancelled so, add a way to cancel them explicitly
which we'll need eventually for restarts and dynamic feed management.

The big lesson here is that async exit stacks are not conducive to
spawning and monitoring service tasks, and especially so if
a `@tractor.context` is used since if the `.open_context()` call isn't
exited (only possible by the stack being closed), then there will be no
way for `trio` to cancel the task that pushed that context (since it
can't run a checkpoint while yielded inside the stack) without also
cancelling all other contexts pushed on that stack. Presuming one
`pikerd` task is used to do the original pushing (which it was) then
any error would have to kill all service daemon tasks which obviously
won't work.

I see this mostly as the painz of tinkering out an SC service manager
with `tractor` / `trio` for the first time, so try to go easy on the
process ;P
2021-07-06 16:41:41 -04:00
Tyler Goodlet cc9a720af7 Don't use a context stack for contexts 2021-07-06 16:41:10 -04:00
Tyler Goodlet b06cb5bb5a Comments clean and improvments 2021-06-10 08:24:10 -04:00
Tyler Goodlet 9931accc52 Port clearing systems to new tractor context api
This avoids somewhat convoluted "hackery" making 2 one-way streams
between the order client and the EMS and instead uses the new
bi-directional streaming and context API from `tractor`. Add a router
type to the EMS that gets setup by the initial service tree and which
we'll eventually use to work toward multi-provider executions and
order-trigger monitoring. Move to py3.9 style where possible throughout.
2021-06-01 10:48:23 -04:00
Tyler Goodlet 7fb2c95ef1 Factor daemon spawning logic, use it to spawn emsd 2021-06-01 10:48:23 -04:00
Tyler Goodlet 7fa9f3f542 Add `Client.search_symbols()` to all backends, use it in `piker search` 2021-05-28 12:29:58 -04:00
Tyler Goodlet c9cf72d554 Add remote context allocation api to service daemon
This allows for more deterministically managing long running sub-daemon
services under `pikerd` using the new context api from `tractor`.
The contexts are allocated in an async exit stack and torn down at root
daemon termination. Spawn brokerds using this method by changing the
persistence entry point to be a `@tractor.context`.
2021-05-27 13:05:23 -04:00
Tyler Goodlet 74b4b4b130 Expose registry port as dict 2021-05-24 12:26:28 -04:00
Tyler Goodlet dcc60524cb Add remote context allocation api to service daemon
This allows for more deterministically managing long running sub-daemon
services under `pikerd` using the new context api from `tractor`.
The contexts are allocated in an async exit stack and torn down at root
daemon termination. Spawn brokerds using this method by changing the
persistence entry point to be a `@tractor.context`.
2021-05-24 12:26:11 -04:00
Tyler Goodlet 9b59471dc1 Pass debug flag down to any daemons 2021-05-24 12:20:20 -04:00
Tyler Goodlet 6e70bc4fa7 New tractor non-default port 2021-05-24 12:19:38 -04:00
Tyler Goodlet 7d6bc4d856 Move feed api(s) into new submodule
Also add a --pdb flag to chart app.
2021-04-15 10:43:29 -04:00
Tyler Goodlet 0b10c8d6d6 Make standalone mode work again 2021-04-15 08:23:08 -04:00
Tyler Goodlet feeabb3d28 Add --pdb flag to pikerd to enable service debug mode 2021-04-14 12:57:46 -04:00
Tyler Goodlet 7f580ad356 Pass through kwargs 2021-04-14 12:56:08 -04:00
Tyler Goodlet 71d02db126 Rename "buffer" to "sampling" 2021-04-10 14:18:40 -04:00
Tyler Goodlet 28a961ebc0 Add a maybe-startup-runtime manager 2021-04-10 14:18:40 -04:00
Tyler Goodlet 14c5fc24ec Port to new setup routine name 2021-04-10 14:18:40 -04:00
Tyler Goodlet 549f81e85d Fix lints 2021-04-03 12:29:56 -04:00
Tyler Goodlet 2b7cecc78e Drop upcoming cached feed usage 2021-04-03 12:22:14 -04:00
Tyler Goodlet b1a1b32315 Better tractor startup, runtime detection 2021-04-03 12:22:14 -04:00
Tyler Goodlet 877db52162 Add license header 2021-04-03 12:22:14 -04:00
Tyler Goodlet 2a51582ec0 Start forming an services api
Add a `Services` nurseries container singleton for spawning sub-daemons
inside the long running `pikerd` tree. Bring in `brokerd` spawning util
funcs to start getting eyes on what can be factored into a service api.
2021-04-03 12:22:14 -04:00
Tyler Goodlet 4ab8545e87 Return early on remote actor lookups instead of branching 2021-04-03 12:22:14 -04:00
Tyler Goodlet 1931da97f4 Fix spawn attr typo, merge tractor kwargs 2021-04-03 12:22:14 -04:00
Tyler Goodlet 79d37646a2 Add `open_pikerd()` and `spawn_emsd()`
The direct open is needed for running `pikerd` cmd and
the ems spawn point is the first step toward detaching UI based order
entry from the engine itself.
2021-04-03 12:22:14 -04:00
Tyler Goodlet 2b9ac8d8ec Port daemon(s) to new clearing system apis 2021-04-03 12:22:14 -04:00
Guillermo Rodriguez 189c56c012 Add piker root daemon spawning machinery
Refactor maybe_spawn_brokerd to adapt to new process tree structure
and add a ``maybe_open_pikerd``.
2021-04-03 12:22:14 -04:00