Commit Graph

1356 Commits (3eabe93d54bc7d7c28c195da109e391c0626a739)

Author SHA1 Message Date
Tyler Goodlet 5383dd6446 Add a working arbiter registry system
Every actor now registers (and unregisters) with the arbiter at
startup/teardown. For now the registry is stored in a plain `dict` in
the arbiter's memory. This makes it possible to easily coordinate actors
started as plain Python processes or via `multiprocessing`.

A whole smörgåsbord of changes was required to accomplish this:
- factor handshake steps into a func
- track *every* channel connected to an actor including multiples to the
  same remote peer (may want to optimize this later)
- handle `trio.ClosedStreamError` gracefully in the message loop
- add an `open_portal` asynccontextmanager which handles channel
  creation, handshaking, and spawning a bg task for msg processing
- add a `start_actor()` for starting in-process actors directly
- add working `get_arbiter()` and `find_actor()` public routines
- `_main` now tries an anonymous channel connect to the stated
  arbiter sockaddr and uses that to determine whether to crown itself
2018-07-04 14:34:25 -04:00
Tyler Goodlet 1d3fde4a4d Add StreamQueue.connected() 2018-07-04 03:16:00 -04:00
Tyler Goodlet ddf27e5e7f Cancel GUI updates on exit 2018-07-04 03:14:54 -04:00
Tyler Goodlet 5d6b9d138c Update lock file 2018-06-27 15:38:06 -04:00
Tyler Goodlet 23830d51d1 Log either kwarg 2018-06-27 11:59:02 -04:00
Tyler Goodlet 4ecfcdc354 Port `piker watch` to tractor api 2018-06-27 11:52:56 -04:00
Tyler Goodlet e22f17bfe9 Port watchlist app to tractor api 2018-06-27 11:50:02 -04:00
Tyler Goodlet f32ecb3d65 Arbiter now supports non-empty statespace 2018-06-27 11:45:21 -04:00
Tyler Goodlet 266518a734 Handle kb interrupt gracefully in sub-actors
Fail gracefully (by "aborting") the same way `trio` does. This avoids
ugly sub-proc tracebacks thrown at the console. Unset the local actor
when `tractor._main` completes. Cancel all tasks for a peer channel on
disconnect.
2018-06-27 11:34:22 -04:00
Tyler Goodlet 8019296c67 Port broker daemon to tractor
Drop all channel/connection handling from the core and break up all the
start up steps into compact and useful functions. The main difference is
the daemon now only needs to worry about spawning per broker streaming
tasks and handling symbol list subscription requests.
2018-06-26 17:55:52 -04:00
Tyler Goodlet c0d8d4fd99 Support re-entrant calls to `get_arbiter()`
It gets called more then once when using `tractor.run()` and
then `find_actor()`. Also, allow passing in the log level to each
new spawned subactor.
2018-06-25 17:41:30 -04:00
Tyler Goodlet 8c5af7fd97 Drop console logging - messes with other tests 2018-06-23 15:33:33 -04:00
Tyler Goodlet c062b01b7b Add an actor spawning test
Test that the actor nursery API and ``tractor.run`` entrypoint work when
the sub-actor's main task is all that is run (i.e. no rpc requests).
2018-06-23 14:59:31 -04:00
Tyler Goodlet fa8418f97f Port stream_quotes to tractor ipc protocol 2018-06-23 14:57:02 -04:00
Tyler Goodlet 37eb8a8552 Fix actor nursery __exit__ handling
When an error is raised inside a nursery block (in the local actor)
cancel all spawned children and ensure the error is properly
unsuppressed.

Also,
- change `invoke_cmd` to `send_cmd` and expect the caller to use
  `result_from_q` (avoids implicit blocking for responses that might
  never arrive)
- `nursery.start()` the channel server block such that we wait for the
  underlying listener to spawn before making outbound connections
- cancel the channel server when an actor's main task completes
  (given that `outlive_main == False`)
- raise subactor errors directly in the local actors's msg loop
- enforce that `treat_as_gen` async functions respond with a caller id
  (`cid`) in each yield packet
2018-06-23 14:57:02 -04:00
Tyler Goodlet 84cd29644e Add reliable subactor lifetime control 2018-06-23 14:57:02 -04:00
Tyler Goodlet ef90d7f106 Add remote actor error handling and parent re-raising
Command requests are sent out and responses are handled in a "message
loop" where each command is associated with a "caller id" and multiple
cmds and results are multiplexed on the came inter-actor channel. When
a cmd result arrives it is pushed into a local queue and delivered to the
appropriate calling actor's task. Errors from a remote actor are always shipped
in an "error" packet back to their spawning-parent actor such that any error
in a subactor is always raised directly in the parent. Based on the
first response to a cmd (either a 'return' or 'yield' packet) the caller
side portal will retrieve values by wrapping the local response queue in
either of an async function or generator as appropriate.
2018-06-23 14:41:10 -04:00
Tyler Goodlet 75996fed0d Use trace level for packet contents 2018-06-23 14:40:49 -04:00
Tyler Goodlet b908bd0b9d Add uid,event attrs to `Channel` 2018-06-23 14:40:49 -04:00
Tyler Goodlet fa0aefff4d Take that QT nulls 2018-06-12 15:33:11 -04:00
Tyler Goodlet acb5dc398e Add a basic `tractor.run()` test 2018-06-12 15:23:58 -04:00
Tyler Goodlet bf2f6769fc Uhhh make everything better 2018-06-12 15:17:48 -04:00
Tyler Goodlet 5a997d67d4 Add an initial `tractor` price streaming test 2018-06-07 09:18:40 -04:00
Tyler Goodlet 58f0182d8f Add a very rough, minimal actor model system
I'm calling it `tractor` (as in trio-actor) for now.
Lots of work to do still as per the many comments!

Relates to #27
2018-06-07 00:26:49 -04:00
Tyler Goodlet 28eff7122d Use `Channel` throughout cli entry point 2018-06-07 00:26:21 -04:00
Tyler Goodlet f71f986dae Use new IPC apis throughout core
Move to using `Channel` throughout instead of the `StreamQueue`
and add a very basic function for upcoming actor model testing.
2018-06-07 00:23:11 -04:00
Tyler Goodlet 2f82db33f4 IPC primitives improvements
- Rename the `Client` to `Channel`
- Add better `__repr__()`
- use laddr, raddr instead of sockaddr, peer
- don't allow re-entrant `Channel.connect()` calls
- Make `Channel` an async iterable
2018-06-07 00:19:31 -04:00
Tyler Goodlet 485aa76ff6 Move ipc types into separate module 2018-05-30 12:36:23 -04:00
Tyler Goodlet 2e5cdbcb7c Adjust to new modules 2018-05-30 09:37:53 -04:00
goodboy 623291cc69
Merge pull request #48 from pikers/reconnect_fixes
Reconnect fixes
2018-05-28 19:54:06 -04:00
Tyler Goodlet 7e5e3c4cc6 Adjust reconnect coro to swallow symbol data resp
Couple fixes here:
- if no tickers for a watchlist name -> bail
- swallow the symbol data response in the reconnect handler coro
- don't sleep 5 seconds before connecting to subproc daemon...

Resolves #43
2018-05-17 13:20:21 -04:00
Tyler Goodlet a05a8cc557 Include process name in log messages 2018-05-16 20:39:47 -04:00
Tyler Goodlet 84fadf7ac4 Explicitly subscribe for tickers at wl startup 2018-05-16 20:33:44 -04:00
Tyler Goodlet 186befc704 Only run 'startup sequence' on reconnect
When a client loses a connection it will currently need to re-subscribe
for symbols and receive a symbol data summary as a first quote response.
Only run the provided coroutine on reconnect and call the kwarg
`on_reconnect`. The client consuming code is entirely expected at this
point to know how the symbol registration protocol works.
2018-05-16 19:15:43 -04:00
goodboy 09ae9f5ef1
Merge pull request #47 from pikers/remote_connect
Remote connect
2018-05-09 18:18:35 -04:00
Tyler Goodlet 9b34aac0fd Build columns only for dataframe output 2018-05-09 18:09:04 -04:00
Tyler Goodlet bcaef70612 Pack null results without raising 2018-05-09 18:09:04 -04:00
Tyler Goodlet fd1fe0816e Don't call formatting func on None values 2018-05-09 18:09:04 -04:00
Tyler Goodlet 3646fb4a23 Filter out bad symbols before adding client subscription
Event if a broker client is already spawned new clients should still
receive a detailed symbol data packet as the first response. Avoid
exposing the new client's queue to the broker (i.e. subscribing it for
quotes) until after first pushing this packet with all bad symbols
filtered out.
2018-05-09 18:09:04 -04:00
Tyler Goodlet 3a40c2f8fe Zero bad fields 2018-05-09 18:09:04 -04:00
Tyler Goodlet 995851360d Rx symbol data from daemon as first response 2018-05-09 18:09:04 -04:00
Tyler Goodlet fcaeeae618 Acquire symbol data with daemon; push as first response 2018-05-09 18:09:04 -04:00
Tyler Goodlet 5a9c079c10 Support specifying daemon host address 2018-05-09 18:09:00 -04:00
Tyler Goodlet 3d6b14ec3f Pass in the host addr 2018-05-08 15:07:13 -04:00
Tyler Goodlet 4e711395e0 Link to manifesto 2018-05-08 15:06:18 -04:00
goodboy 735fcb94c2
Merge pull request #42 from pikers/multi_ticker_add
Multi ticker add
2018-04-25 15:20:05 -04:00
Tyler Goodlet c8539b7373 Test adding multiple tickers on cli 2018-04-25 09:11:21 -04:00
Tyler Goodlet 7aa99019cb Allow adding multiple tickers via CLI 2018-04-25 09:10:57 -04:00
goodboy ec2884a556
Merge pull request #41 from pikers/env_update
Update pipenv files
2018-04-25 08:08:11 -04:00
Tyler Goodlet 91519bbb9e Update pipenv files 2018-04-24 15:48:31 -04:00