Commit Graph

2481 Commits (cb7a9b94490d2b462cf191352223cd8e8f59d390)

Author SHA1 Message Date
Tyler Goodlet f71f986dae Use new IPC apis throughout core
Move to using `Channel` throughout instead of the `StreamQueue`
and add a very basic function for upcoming actor model testing.
2018-06-07 00:23:11 -04:00
Tyler Goodlet 2f82db33f4 IPC primitives improvements
- Rename the `Client` to `Channel`
- Add better `__repr__()`
- use laddr, raddr instead of sockaddr, peer
- don't allow re-entrant `Channel.connect()` calls
- Make `Channel` an async iterable
2018-06-07 00:19:31 -04:00
Tyler Goodlet 485aa76ff6 Move ipc types into separate module 2018-05-30 12:36:23 -04:00
Tyler Goodlet 2e5cdbcb7c Adjust to new modules 2018-05-30 09:37:53 -04:00
goodboy 623291cc69
Merge pull request #48 from pikers/reconnect_fixes
Reconnect fixes
2018-05-28 19:54:06 -04:00
Tyler Goodlet 7e5e3c4cc6 Adjust reconnect coro to swallow symbol data resp
Couple fixes here:
- if no tickers for a watchlist name -> bail
- swallow the symbol data response in the reconnect handler coro
- don't sleep 5 seconds before connecting to subproc daemon...

Resolves #43
2018-05-17 13:20:21 -04:00
Tyler Goodlet a05a8cc557 Include process name in log messages 2018-05-16 20:39:47 -04:00
Tyler Goodlet 84fadf7ac4 Explicitly subscribe for tickers at wl startup 2018-05-16 20:33:44 -04:00
Tyler Goodlet 186befc704 Only run 'startup sequence' on reconnect
When a client loses a connection it will currently need to re-subscribe
for symbols and receive a symbol data summary as a first quote response.
Only run the provided coroutine on reconnect and call the kwarg
`on_reconnect`. The client consuming code is entirely expected at this
point to know how the symbol registration protocol works.
2018-05-16 19:15:43 -04:00
goodboy 09ae9f5ef1
Merge pull request #47 from pikers/remote_connect
Remote connect
2018-05-09 18:18:35 -04:00
Tyler Goodlet 9b34aac0fd Build columns only for dataframe output 2018-05-09 18:09:04 -04:00
Tyler Goodlet bcaef70612 Pack null results without raising 2018-05-09 18:09:04 -04:00
Tyler Goodlet fd1fe0816e Don't call formatting func on None values 2018-05-09 18:09:04 -04:00
Tyler Goodlet 3646fb4a23 Filter out bad symbols before adding client subscription
Event if a broker client is already spawned new clients should still
receive a detailed symbol data packet as the first response. Avoid
exposing the new client's queue to the broker (i.e. subscribing it for
quotes) until after first pushing this packet with all bad symbols
filtered out.
2018-05-09 18:09:04 -04:00
Tyler Goodlet 3a40c2f8fe Zero bad fields 2018-05-09 18:09:04 -04:00
Tyler Goodlet 995851360d Rx symbol data from daemon as first response 2018-05-09 18:09:04 -04:00
Tyler Goodlet fcaeeae618 Acquire symbol data with daemon; push as first response 2018-05-09 18:09:04 -04:00
Tyler Goodlet 5a9c079c10 Support specifying daemon host address 2018-05-09 18:09:00 -04:00
Tyler Goodlet 3d6b14ec3f Pass in the host addr 2018-05-08 15:07:13 -04:00
Tyler Goodlet 4e711395e0 Link to manifesto 2018-05-08 15:06:18 -04:00
goodboy 735fcb94c2
Merge pull request #42 from pikers/multi_ticker_add
Multi ticker add
2018-04-25 15:20:05 -04:00
Tyler Goodlet c8539b7373 Test adding multiple tickers on cli 2018-04-25 09:11:21 -04:00
Tyler Goodlet 7aa99019cb Allow adding multiple tickers via CLI 2018-04-25 09:10:57 -04:00
goodboy ec2884a556
Merge pull request #41 from pikers/env_update
Update pipenv files
2018-04-25 08:08:11 -04:00
Tyler Goodlet 91519bbb9e Update pipenv files 2018-04-24 15:48:31 -04:00
goodboy b00b872414
Merge pull request #36 from pikers/daemonize
Daemonize broker quote engine
2018-04-23 00:47:51 -04:00
Tyler Goodlet 482f9531ca Try to connect to daemon once on startup; don't poll 2018-04-22 13:27:41 -04:00
Tyler Goodlet a2c4f0c80b Don't recurse in Client.aiter_recv() 2018-04-22 12:48:35 -04:00
Tyler Goodlet 6a6f773477 Adjust some log levels 2018-04-20 13:18:35 -04:00
Tyler Goodlet 063dfad5b4 Make daemon registry cross-task 2018-04-20 13:18:35 -04:00
Tyler Goodlet 4f387ea2be Fix subscriptions and connection handling
Oh boy where to start.

- Handle broken streams in the `StreamQueue` gracefully; terminate the
  async generator.
- When a stream queue connection is unwritable discard its subscriptions
  inside the quoter task
- If all subscriptions are discarded for a broker then tear down its
  quoter task
- Use listener parent nursery for spawning quoter tasks
- Make broker subs data structures global/shared between conn
  handler tasks
- Register the `tickers2qs` entry *after* instantiating broker client(s)
  (avoids race condition when mulitple client connections are coming
  online simultaneously)
- Push smoke quotes to every client not just the first that connects
- Track quoter tasks in a cross-task set
- Handle unsubscriptions more correctly
2018-04-20 13:18:35 -04:00
Tyler Goodlet 0add443e8b Spawn broker-daemon without asking 2018-04-20 13:18:35 -04:00
Tyler Goodlet 2973b40946 Allow wl app to spawn a broker daemon in a subprocess 2018-04-20 13:18:35 -04:00
Tyler Goodlet 90e8dd911c Daemon main doesn't require brokermod anymore 2018-04-20 13:18:35 -04:00
Tyler Goodlet 4123139750 Use `Client` in watchlist app 2018-04-20 13:18:35 -04:00
Tyler Goodlet 17feb17535 Add a reliable `Client` API
In order to start working toward a HA distributed
architecture make apps use a `Client` type to talk to daemons.
The `Client` provides fault-tolerance for connection failures such
that the app will continue running until a connection to the original
service can be made or the process is killed. This will make it easier
to simply spawn up new daemon child processes when faults are detected.
2018-04-20 13:18:35 -04:00
Tyler Goodlet 4d4c04cd11 Document daemon usage 2018-04-20 13:18:35 -04:00
Tyler Goodlet 51b44cf236 Use msgpack for quote-packet serialization 2018-04-20 11:43:14 -04:00
Tyler Goodlet dd5e1e7ea7 Doh, set sleeptime after adjusting the rate limit 2018-04-20 11:43:14 -04:00
Tyler Goodlet 030ecdcce8 Filter symbols and push initial quote in stream handler
Filter out bad symbols by processing an initial batch quote and
pushing to the subscribing client before spawning a quoter task.
This also avoids exposing the quoter task to anything but the
broker module and a `get_quotes()` routine.
2018-04-20 11:43:14 -04:00
Tyler Goodlet 02a71c51ba Make <brokermod>.quoter() a simple factory func 2018-04-20 11:43:14 -04:00
Tyler Goodlet a6dc697327 Move watchlist app to new daemon-socket api 2018-04-20 11:43:14 -04:00
Tyler Goodlet 0c7ecd383b Monkey patch broker mods with a name attr 2018-04-20 11:43:14 -04:00
Tyler Goodlet 6359623019 Allow broker specific subscriptions
Allow client connections to subscribe for quote streams from specific
brokers and spawn broker-client quoter tasks on-demand according
to client connection demands. Support multiple subscribers to a
single daemon process.
2018-04-20 11:43:14 -04:00
Tyler Goodlet f80735121c Use an async generator inside `StreamQueue`
Async generators are faster and less code. Handle segmented packets
which can happen during periods of high quote volume. Move per-broker
rate limit logic into daemon task.
2018-04-20 11:43:14 -04:00
Tyler Goodlet 4898459bcd Make watchlist app retrieve quotes from the broker daemon 2018-04-20 11:43:14 -04:00
Tyler Goodlet 73ef95f42a Add `pikerd` entry point 2018-04-20 11:43:14 -04:00
Tyler Goodlet 23ae71089f Handle dynamic symbol subscriptions in QT backend 2018-04-20 11:42:59 -04:00
Tyler Goodlet d65bd78f5d Add a quote stream server task
Add a daemon-server task for delivering subscription based
quote streams via json serialized packets wrapped in a queue
interface.
2018-04-16 02:03:22 -04:00
goodboy 07c95e4f3f
Merge pull request #34 from pikers/travis_ci
Travis CI
2018-04-11 14:27:16 -04:00