Commit Graph

1143 Commits (web_utils)

Author SHA1 Message Date
Tyler Goodlet fa8418f97f Port stream_quotes to tractor ipc protocol 2018-06-23 14:57:02 -04:00
Tyler Goodlet 37eb8a8552 Fix actor nursery __exit__ handling
When an error is raised inside a nursery block (in the local actor)
cancel all spawned children and ensure the error is properly
unsuppressed.

Also,
- change `invoke_cmd` to `send_cmd` and expect the caller to use
  `result_from_q` (avoids implicit blocking for responses that might
  never arrive)
- `nursery.start()` the channel server block such that we wait for the
  underlying listener to spawn before making outbound connections
- cancel the channel server when an actor's main task completes
  (given that `outlive_main == False`)
- raise subactor errors directly in the local actors's msg loop
- enforce that `treat_as_gen` async functions respond with a caller id
  (`cid`) in each yield packet
2018-06-23 14:57:02 -04:00
Tyler Goodlet 84cd29644e Add reliable subactor lifetime control 2018-06-23 14:57:02 -04:00
Tyler Goodlet ef90d7f106 Add remote actor error handling and parent re-raising
Command requests are sent out and responses are handled in a "message
loop" where each command is associated with a "caller id" and multiple
cmds and results are multiplexed on the came inter-actor channel. When
a cmd result arrives it is pushed into a local queue and delivered to the
appropriate calling actor's task. Errors from a remote actor are always shipped
in an "error" packet back to their spawning-parent actor such that any error
in a subactor is always raised directly in the parent. Based on the
first response to a cmd (either a 'return' or 'yield' packet) the caller
side portal will retrieve values by wrapping the local response queue in
either of an async function or generator as appropriate.
2018-06-23 14:41:10 -04:00
Tyler Goodlet 75996fed0d Use trace level for packet contents 2018-06-23 14:40:49 -04:00
Tyler Goodlet b908bd0b9d Add uid,event attrs to `Channel` 2018-06-23 14:40:49 -04:00
Tyler Goodlet fa0aefff4d Take that QT nulls 2018-06-12 15:33:11 -04:00
Tyler Goodlet acb5dc398e Add a basic `tractor.run()` test 2018-06-12 15:23:58 -04:00
Tyler Goodlet bf2f6769fc Uhhh make everything better 2018-06-12 15:17:48 -04:00
Tyler Goodlet 5a997d67d4 Add an initial `tractor` price streaming test 2018-06-07 09:18:40 -04:00
Tyler Goodlet 58f0182d8f Add a very rough, minimal actor model system
I'm calling it `tractor` (as in trio-actor) for now.
Lots of work to do still as per the many comments!

Relates to #27
2018-06-07 00:26:49 -04:00
Tyler Goodlet 28eff7122d Use `Channel` throughout cli entry point 2018-06-07 00:26:21 -04:00
Tyler Goodlet f71f986dae Use new IPC apis throughout core
Move to using `Channel` throughout instead of the `StreamQueue`
and add a very basic function for upcoming actor model testing.
2018-06-07 00:23:11 -04:00
Tyler Goodlet 2f82db33f4 IPC primitives improvements
- Rename the `Client` to `Channel`
- Add better `__repr__()`
- use laddr, raddr instead of sockaddr, peer
- don't allow re-entrant `Channel.connect()` calls
- Make `Channel` an async iterable
2018-06-07 00:19:31 -04:00
Tyler Goodlet 485aa76ff6 Move ipc types into separate module 2018-05-30 12:36:23 -04:00
Tyler Goodlet 2e5cdbcb7c Adjust to new modules 2018-05-30 09:37:53 -04:00
goodboy 623291cc69
Merge pull request #48 from pikers/reconnect_fixes
Reconnect fixes
2018-05-28 19:54:06 -04:00
Tyler Goodlet 7e5e3c4cc6 Adjust reconnect coro to swallow symbol data resp
Couple fixes here:
- if no tickers for a watchlist name -> bail
- swallow the symbol data response in the reconnect handler coro
- don't sleep 5 seconds before connecting to subproc daemon...

Resolves #43
2018-05-17 13:20:21 -04:00
Tyler Goodlet a05a8cc557 Include process name in log messages 2018-05-16 20:39:47 -04:00
Tyler Goodlet 84fadf7ac4 Explicitly subscribe for tickers at wl startup 2018-05-16 20:33:44 -04:00
Tyler Goodlet 186befc704 Only run 'startup sequence' on reconnect
When a client loses a connection it will currently need to re-subscribe
for symbols and receive a symbol data summary as a first quote response.
Only run the provided coroutine on reconnect and call the kwarg
`on_reconnect`. The client consuming code is entirely expected at this
point to know how the symbol registration protocol works.
2018-05-16 19:15:43 -04:00
goodboy 09ae9f5ef1
Merge pull request #47 from pikers/remote_connect
Remote connect
2018-05-09 18:18:35 -04:00
Tyler Goodlet 9b34aac0fd Build columns only for dataframe output 2018-05-09 18:09:04 -04:00
Tyler Goodlet bcaef70612 Pack null results without raising 2018-05-09 18:09:04 -04:00
Tyler Goodlet fd1fe0816e Don't call formatting func on None values 2018-05-09 18:09:04 -04:00
Tyler Goodlet 3646fb4a23 Filter out bad symbols before adding client subscription
Event if a broker client is already spawned new clients should still
receive a detailed symbol data packet as the first response. Avoid
exposing the new client's queue to the broker (i.e. subscribing it for
quotes) until after first pushing this packet with all bad symbols
filtered out.
2018-05-09 18:09:04 -04:00
Tyler Goodlet 3a40c2f8fe Zero bad fields 2018-05-09 18:09:04 -04:00
Tyler Goodlet 995851360d Rx symbol data from daemon as first response 2018-05-09 18:09:04 -04:00
Tyler Goodlet fcaeeae618 Acquire symbol data with daemon; push as first response 2018-05-09 18:09:04 -04:00
Tyler Goodlet 5a9c079c10 Support specifying daemon host address 2018-05-09 18:09:00 -04:00
Tyler Goodlet 3d6b14ec3f Pass in the host addr 2018-05-08 15:07:13 -04:00
Tyler Goodlet 4e711395e0 Link to manifesto 2018-05-08 15:06:18 -04:00
goodboy 735fcb94c2
Merge pull request #42 from pikers/multi_ticker_add
Multi ticker add
2018-04-25 15:20:05 -04:00
Tyler Goodlet c8539b7373 Test adding multiple tickers on cli 2018-04-25 09:11:21 -04:00
Tyler Goodlet 7aa99019cb Allow adding multiple tickers via CLI 2018-04-25 09:10:57 -04:00
goodboy ec2884a556
Merge pull request #41 from pikers/env_update
Update pipenv files
2018-04-25 08:08:11 -04:00
Tyler Goodlet 91519bbb9e Update pipenv files 2018-04-24 15:48:31 -04:00
goodboy b00b872414
Merge pull request #36 from pikers/daemonize
Daemonize broker quote engine
2018-04-23 00:47:51 -04:00
Tyler Goodlet 482f9531ca Try to connect to daemon once on startup; don't poll 2018-04-22 13:27:41 -04:00
Tyler Goodlet a2c4f0c80b Don't recurse in Client.aiter_recv() 2018-04-22 12:48:35 -04:00
Tyler Goodlet 6a6f773477 Adjust some log levels 2018-04-20 13:18:35 -04:00
Tyler Goodlet 063dfad5b4 Make daemon registry cross-task 2018-04-20 13:18:35 -04:00
Tyler Goodlet 4f387ea2be Fix subscriptions and connection handling
Oh boy where to start.

- Handle broken streams in the `StreamQueue` gracefully; terminate the
  async generator.
- When a stream queue connection is unwritable discard its subscriptions
  inside the quoter task
- If all subscriptions are discarded for a broker then tear down its
  quoter task
- Use listener parent nursery for spawning quoter tasks
- Make broker subs data structures global/shared between conn
  handler tasks
- Register the `tickers2qs` entry *after* instantiating broker client(s)
  (avoids race condition when mulitple client connections are coming
  online simultaneously)
- Push smoke quotes to every client not just the first that connects
- Track quoter tasks in a cross-task set
- Handle unsubscriptions more correctly
2018-04-20 13:18:35 -04:00
Tyler Goodlet 0add443e8b Spawn broker-daemon without asking 2018-04-20 13:18:35 -04:00
Tyler Goodlet 2973b40946 Allow wl app to spawn a broker daemon in a subprocess 2018-04-20 13:18:35 -04:00
Tyler Goodlet 90e8dd911c Daemon main doesn't require brokermod anymore 2018-04-20 13:18:35 -04:00
Tyler Goodlet 4123139750 Use `Client` in watchlist app 2018-04-20 13:18:35 -04:00
Tyler Goodlet 17feb17535 Add a reliable `Client` API
In order to start working toward a HA distributed
architecture make apps use a `Client` type to talk to daemons.
The `Client` provides fault-tolerance for connection failures such
that the app will continue running until a connection to the original
service can be made or the process is killed. This will make it easier
to simply spawn up new daemon child processes when faults are detected.
2018-04-20 13:18:35 -04:00
Tyler Goodlet 4d4c04cd11 Document daemon usage 2018-04-20 13:18:35 -04:00
Tyler Goodlet 51b44cf236 Use msgpack for quote-packet serialization 2018-04-20 11:43:14 -04:00