Commit Graph

1381 Commits (6825ad4804e2cdad8333a5b5863e5940032c90b8)

Author SHA1 Message Date
Tyler Goodlet 3646fb4a23 Filter out bad symbols before adding client subscription
Event if a broker client is already spawned new clients should still
receive a detailed symbol data packet as the first response. Avoid
exposing the new client's queue to the broker (i.e. subscribing it for
quotes) until after first pushing this packet with all bad symbols
filtered out.
2018-05-09 18:09:04 -04:00
Tyler Goodlet 3a40c2f8fe Zero bad fields 2018-05-09 18:09:04 -04:00
Tyler Goodlet 995851360d Rx symbol data from daemon as first response 2018-05-09 18:09:04 -04:00
Tyler Goodlet fcaeeae618 Acquire symbol data with daemon; push as first response 2018-05-09 18:09:04 -04:00
Tyler Goodlet 5a9c079c10 Support specifying daemon host address 2018-05-09 18:09:00 -04:00
Tyler Goodlet 3d6b14ec3f Pass in the host addr 2018-05-08 15:07:13 -04:00
Tyler Goodlet 7aa99019cb Allow adding multiple tickers via CLI 2018-04-25 09:10:57 -04:00
Tyler Goodlet 482f9531ca Try to connect to daemon once on startup; don't poll 2018-04-22 13:27:41 -04:00
Tyler Goodlet a2c4f0c80b Don't recurse in Client.aiter_recv() 2018-04-22 12:48:35 -04:00
Tyler Goodlet 6a6f773477 Adjust some log levels 2018-04-20 13:18:35 -04:00
Tyler Goodlet 063dfad5b4 Make daemon registry cross-task 2018-04-20 13:18:35 -04:00
Tyler Goodlet 4f387ea2be Fix subscriptions and connection handling
Oh boy where to start.

- Handle broken streams in the `StreamQueue` gracefully; terminate the
  async generator.
- When a stream queue connection is unwritable discard its subscriptions
  inside the quoter task
- If all subscriptions are discarded for a broker then tear down its
  quoter task
- Use listener parent nursery for spawning quoter tasks
- Make broker subs data structures global/shared between conn
  handler tasks
- Register the `tickers2qs` entry *after* instantiating broker client(s)
  (avoids race condition when mulitple client connections are coming
  online simultaneously)
- Push smoke quotes to every client not just the first that connects
- Track quoter tasks in a cross-task set
- Handle unsubscriptions more correctly
2018-04-20 13:18:35 -04:00
Tyler Goodlet 0add443e8b Spawn broker-daemon without asking 2018-04-20 13:18:35 -04:00
Tyler Goodlet 2973b40946 Allow wl app to spawn a broker daemon in a subprocess 2018-04-20 13:18:35 -04:00
Tyler Goodlet 90e8dd911c Daemon main doesn't require brokermod anymore 2018-04-20 13:18:35 -04:00
Tyler Goodlet 4123139750 Use `Client` in watchlist app 2018-04-20 13:18:35 -04:00
Tyler Goodlet 17feb17535 Add a reliable `Client` API
In order to start working toward a HA distributed
architecture make apps use a `Client` type to talk to daemons.
The `Client` provides fault-tolerance for connection failures such
that the app will continue running until a connection to the original
service can be made or the process is killed. This will make it easier
to simply spawn up new daemon child processes when faults are detected.
2018-04-20 13:18:35 -04:00
Tyler Goodlet 51b44cf236 Use msgpack for quote-packet serialization 2018-04-20 11:43:14 -04:00
Tyler Goodlet dd5e1e7ea7 Doh, set sleeptime after adjusting the rate limit 2018-04-20 11:43:14 -04:00
Tyler Goodlet 030ecdcce8 Filter symbols and push initial quote in stream handler
Filter out bad symbols by processing an initial batch quote and
pushing to the subscribing client before spawning a quoter task.
This also avoids exposing the quoter task to anything but the
broker module and a `get_quotes()` routine.
2018-04-20 11:43:14 -04:00
Tyler Goodlet 02a71c51ba Make <brokermod>.quoter() a simple factory func 2018-04-20 11:43:14 -04:00
Tyler Goodlet a6dc697327 Move watchlist app to new daemon-socket api 2018-04-20 11:43:14 -04:00
Tyler Goodlet 0c7ecd383b Monkey patch broker mods with a name attr 2018-04-20 11:43:14 -04:00
Tyler Goodlet 6359623019 Allow broker specific subscriptions
Allow client connections to subscribe for quote streams from specific
brokers and spawn broker-client quoter tasks on-demand according
to client connection demands. Support multiple subscribers to a
single daemon process.
2018-04-20 11:43:14 -04:00
Tyler Goodlet f80735121c Use an async generator inside `StreamQueue`
Async generators are faster and less code. Handle segmented packets
which can happen during periods of high quote volume. Move per-broker
rate limit logic into daemon task.
2018-04-20 11:43:14 -04:00
Tyler Goodlet 4898459bcd Make watchlist app retrieve quotes from the broker daemon 2018-04-20 11:43:14 -04:00
Tyler Goodlet 73ef95f42a Add `pikerd` entry point 2018-04-20 11:43:14 -04:00
Tyler Goodlet 23ae71089f Handle dynamic symbol subscriptions in QT backend 2018-04-20 11:42:59 -04:00
Tyler Goodlet d65bd78f5d Add a quote stream server task
Add a daemon-server task for delivering subscription based
quote streams via json serialized packets wrapped in a queue
interface.
2018-04-16 02:03:22 -04:00
Tyler Goodlet 577ca43c02 Include built-in lists in show, log errors from remove 2018-04-10 14:13:00 -04:00
Tyler Goodlet 381df2815d Expose remove errors to caller 2018-04-10 14:12:06 -04:00
Tyler Goodlet 6b72d04427 Change watchlist write function name 2018-04-06 15:07:47 -04:00
Tyler Goodlet 7e92df4352 Move built-in lists to respective module 2018-04-06 15:00:11 -04:00
Tyler Goodlet 0cccdd01b5 Only log when the network first goes down 2018-04-05 23:15:24 -04:00
K0nstantine 918133f265 Finalize WL Management and fix merge func 2018-04-04 00:35:12 -04:00
K0nstantine 49b760673e Remove write to file from API and move to CLI 2018-04-04 00:35:12 -04:00
K0nstantine cd69c30143 CLI tests for watchlist commands 2018-04-04 00:35:12 -04:00
K0nstantine 11f25958ba Move watchlist api tests to seperate module 2018-04-04 00:34:42 -04:00
K0nstantine e859222df4 Add initial API test, need relocation 2018-04-04 00:25:34 -04:00
K0nstantine d28a3dc461 Add initial config dir test 2018-04-04 00:25:34 -04:00
K0nstantine ce75bd8f6f Update CLI and create watchlists module
Moved the watchlists management implementation to a seperate module
wrapped in an api.

Resolves: #5
2018-04-04 00:25:34 -04:00
K0nstantine 43fbea5722 Sorted values and keys in watchlist dictionary and added a purge function 2018-04-04 00:25:34 -04:00
K0nstantine 7ada8a291e Initial watchlist management cli 2018-04-04 00:25:34 -04:00
Tyler Goodlet e246823f2d Add timeout handling to network poll loop 2018-04-02 14:51:38 -04:00
Tyler Goodlet 01cfbbdd64 Handle network outages
Quote queries will hang indefinitely when the network goes down.
Instead poll for network reestablishment such that roaming on
wifi is supported and real-time feeds will resume once the network is
back.
2018-03-31 12:02:22 -04:00
Tyler Goodlet 2b51e84a3c Expect quotes dict in watchlist update loop 2018-03-29 13:02:03 -04:00
Tyler Goodlet 164d636c67 Handle defunct QT tickers and delays 2018-03-29 13:01:13 -04:00
Tyler Goodlet 823bd2ea29 Make payload a ticker dict 2018-03-29 13:00:48 -04:00
Tyler Goodlet 9667042fe2 Drop NUU it's now just APH 2018-03-28 14:00:17 -04:00
Tyler Goodlet 801b9f860b Use `symbol_data()` in watclist UI 2018-03-27 16:27:55 -04:00
Tyler Goodlet 19303cf0ad Use brokermod util functions in cli 2018-03-27 16:27:30 -04:00
Tyler Goodlet 8544c22cdc Change `Client.symbols()` to `symbol_data()`
Make `symbol_data()` a common backend API method for looking up
auxiliary ticker/symbol data from brokers. It seems most have such
a call/endpoint.
2018-03-27 16:24:57 -04:00
Tyler Goodlet 8fc74c5085 Drop QT error 2018-03-27 16:16:31 -04:00
Tyler Goodlet 49d290d036 Add brokermod loader utils 2018-03-27 16:03:01 -04:00
Tyler Goodlet 29ddfe017c Use total time calc 2018-03-23 16:15:56 -04:00
Tyler Goodlet 178e091f41 Make robinhood the default broker backend
- Add a rate limit cli option
- Allow broker backends to define a max quote query limit
- Add an index ETF list to demonstrate robinhood's real-time prices
2018-03-21 17:28:40 -04:00
Tyler Goodlet 200526da8a Query QT at most 3 quotes/sec to avoid rate limits 2018-03-21 17:28:26 -04:00
Tyler Goodlet 933fe980c1 Set default quote rate to 5/sec 2018-03-21 17:27:04 -04:00
Tyler Goodlet 1ea784558c Pass through rate arg to `poll_tickers` 2018-03-21 17:26:30 -04:00
Tyler Goodlet 456e86990f Make watchlist app broker agnostic 2018-03-21 10:44:05 -04:00
Tyler Goodlet 6b47130c77 Move quote formatting to broker backends 2018-03-21 10:30:43 -04:00
Tyler Goodlet f0149118e1 Specify bid-ask "stacked" cells by argument 2018-03-20 21:20:55 -04:00
Tyler Goodlet 6c0f1fbdfc Fill failed symbol lookup df row with NaNs 2018-03-20 21:02:59 -04:00
Tyler Goodlet 48fe280e0c Return None on failed symbol lookups 2018-03-20 21:01:55 -04:00
Tyler Goodlet 04fa3c7ca4 Factor out QT quoting specifics into the backend 2018-03-20 15:39:49 -04:00
Tyler Goodlet 42e9296b36 Adjust cli to new backend api 2018-03-20 13:28:24 -04:00
Tyler Goodlet e75f0718a5 Add a basic quotes-only robinhood backend
We need a yank to test the order system and other end points that
require auth.

Resolves #2
2018-03-20 13:26:12 -04:00
Tyler Goodlet bd7eb16ab2 Move core tasks to separate module
Begin abstracting out broker backends by moving core data query tasks
into a module which requires and calls a broker backend API.
2018-03-20 13:13:07 -04:00
Tyler Goodlet 56f38263be Stack VWAP over daily high/low 2018-03-14 14:00:24 -04:00
goodboy d2e1605bf0
Merge pull request #14 from pikers/searchbar
Searchbar
2018-03-07 23:52:32 -05:00
Tyler Goodlet 937285d35c Add example watchlist your dad would want 2018-03-07 23:50:47 -05:00
Tyler Goodlet 2f34d982bc Require symbol info for quote processing 2018-03-06 20:54:37 -05:00
K0nstantine 61206f5903 Fix issue where config dir was not created by default
Fixes #15
2018-03-05 19:15:10 -05:00
Tyler Goodlet 7f3ee8c728 Add really basic ticker search bar 2018-03-05 09:26:52 -05:00
Tyler Goodlet 1e7dc38a50 Drop row factories; add ticker search API 2018-03-05 09:26:52 -05:00
Tyler Goodlet fc45e1215d Rename as pager mod 2018-03-05 09:26:52 -05:00
goodboy 09b16bf3e7
Merge pull request #13 from pikers/kb_ctls
Pager support
2018-03-04 21:58:16 -05:00
Tyler Goodlet 80ee1bfc19 Poll at 3 r/s and add more detailed timing logs 2018-03-02 21:24:09 -05:00
Tyler Goodlet 13e4389c9f Handle expired tokens more reliably; trace log json data 2018-02-22 18:44:46 -05:00
Tyler Goodlet 4b59b87f98 Add convenience methods for all log levels 2018-02-22 18:44:00 -05:00
Tyler Goodlet 6e0209ac6b Add pager support to watchlist 2018-02-22 18:39:21 -05:00
Tyler Goodlet 385f1b9607 Add a pager widget
It's a `ScrollView` but with keyboard controls that allow for paging
just like the classic unix `less` program. Add a search bar widget too!
2018-02-22 18:38:50 -05:00
Tyler Goodlet 9e0f58ea6b Add a keyboard input handling task 2018-02-17 15:09:22 -05:00
Tyler Goodlet ff437ce9e1 Handle out of process token refreshes 2018-02-15 16:40:33 -05:00
Tyler Goodlet b41b262b7d Raise any unexpected error 2018-02-14 18:49:58 -05:00
Tyler Goodlet d50aa17a83 Add real-time market caps 2018-02-14 12:06:54 -05:00
Tyler Goodlet 722d294915 Handle non-numbers in `humanize()` 2018-02-14 12:06:29 -05:00
Tyler Goodlet 722b515246 Limit humanize output to 2 decimal places 2018-02-14 02:43:55 -05:00
Tyler Goodlet e464898210 Combine last,bid,ask in a StackLayout-cell 2018-02-14 02:37:26 -05:00
Tyler Goodlet 29be7f58c9 Push every new change; not just the last trade 2018-02-13 15:57:12 -05:00
Tyler Goodlet 62bac7b2cd Don't sleep on less than zero delay 2018-02-13 12:13:27 -05:00
Tyler Goodlet 472ff68811 Fix row header highlight typo 2018-02-13 12:13:27 -05:00
Tyler Goodlet f31ebe6fcd Handle numbers of magnitude 2 2018-02-13 12:13:19 -05:00
Tyler Goodlet e4ff113dfc Sort rows by column on click 2018-02-13 10:34:24 -05:00
Tyler Goodlet 8e577cd0d0 Use lighter red; sort rows on startup 2018-02-12 16:11:31 -05:00
Tyler Goodlet 274a5a728a Fix row header borders using a `BorderImage` 2018-02-12 16:02:11 -05:00
Tyler Goodlet 1bbf212ad0 Work when only newly time stamped quotes are received 2018-02-12 14:05:57 -05:00
Tyler Goodlet b7b7abbc1f Only push new quotes to the queue at 5 per sec 2018-02-12 14:03:31 -05:00
Tyler Goodlet 9f3efd2a6a Import client for now until we make a proper shim 2018-02-12 10:35:54 -05:00
Tyler Goodlet f4fd35fa21 Separate sortable and dislplayable quote values 2018-02-10 19:54:09 -05:00
Tyler Goodlet 488f3988ea Handle weekend errors 2018-02-10 19:44:41 -05:00
Tyler Goodlet 1af14bc46f Add watchlist sort-by-column and row header highlighting 2018-02-09 22:04:53 -05:00
Tyler Goodlet 0997418a47 Be PC 2018-02-09 22:01:14 -05:00
Tyler Goodlet 37a4d2e5f8 Expose watchlist app via `piker watch` for now 2018-02-09 03:29:30 -05:00
Tyler Goodlet e220e9b658 Move cli mod to top level package 2018-02-09 03:01:58 -05:00
Tyler Goodlet f71391252d Tighten up the UI to be super sleek 2018-02-09 02:44:43 -05:00
Tyler Goodlet e45c07dce7 Watchlist fixes
- make the % daily change use the previous days close as the reference
  price
- color each cell on every change (results in "pulsed" colors on changes)
- tweak some quote fields
- redraw and sort all rows on every quotes update cycle
- error when the QT api is returning None values
2018-02-08 19:30:09 -05:00
Tyler Goodlet 17c4ac3b8c Adjust cli to new api 2018-02-08 19:15:21 -05:00
Tyler Goodlet 224451f44a Make ticker stream caching optional
Push all ticker quotes to the queue regardless of duplicate
content. That is, don't worry about only pushing new quote changes
(turns out it is useful when coloring a watchlist where multiple
of the same quote may indicate multiple similar trades and we only
want to quickly "pulse" color changes on value changes).
If it is desired to only push new changes, the ``cache`` flag enables
the old behaviour.

Also add `Client.symbols()` for returning symbol data from a sequence of
tickers.
2018-02-08 19:10:17 -05:00
Tyler Goodlet 13342c459a Our first real-time watchlist! 2018-02-08 02:39:18 -05:00
Tyler Goodlet b8a3fb67a1 Add a quote-from-json-file streamer for testing 2018-02-08 02:39:18 -05:00
Tyler Goodlet 6781a23850 Add a `quote` subcommand
Add `piker quote <tickerA> <tickerB> <tickerC>` command for easily
dumping quote data to the console. With `-df` will dump as a pandas data
frame. Add key filtering to `piker api` calls.
2018-02-08 02:39:18 -05:00
Tyler Goodlet adecc082ac Save `brokers.ini` in the user config dir using click 2018-02-08 02:39:18 -05:00
Tyler Goodlet 151e7bf4fa More client enhancements
- Extend the qt api to include candles (not working yet), balances, positions.
- Add a `quote()` method to the `Client` for batch ticker quotes and expose
  it through a CLI subcommand.
- Make `poll_tickers` push new quotes to a `trio.Queue`
2018-02-08 02:39:18 -05:00
Tyler Goodlet 797efedf6a Add quote polling; pseudo-streaming
Add a ``poll_tickers`` coro which can be used to "stream" quotes at
a requested rate. Expose through a cli subcommand `piker stream`.
Drop the `pikerd` command for now.
2018-01-29 12:45:48 -05:00
Tyler Goodlet 66441d15e8 Complain when kwargs are missing but required 2018-01-27 01:52:00 -05:00
Tyler Goodlet 1b93a4c02a Add an `api` cli subcommand for console testing
Add `piker api <method> <kwargs>` for easy testing of the
underlying broker api from the console.
2018-01-26 14:31:15 -05:00
Tyler Goodlet 27a39ac3ad More client improvements
- colorize json response data in logs
- support ``refresh_token`` retrieval from user if the token for some
  reason expires while the client is live
- extend api method support for markets, search, symbols, and quotes
- support "proxying" through api calls via an ``api`` coro for one off
  client queries (useful for cli testing)
2018-01-26 14:25:53 -05:00
Tyler Goodlet 534ba0b698 Add json highlighting; make debug msgs white 2018-01-26 11:31:40 -05:00
Tyler Goodlet 9e8ed392d4 Add token refresher task 2018-01-25 21:53:55 -05:00
Tyler Goodlet 4e1c64a7fb Import broker backend by name 2018-01-25 21:08:49 -05:00
Tyler Goodlet c6cff5a432 Swap debug-garbage log colours 2018-01-25 20:59:56 -05:00
Tyler Goodlet 5c4996873a Start using click for cli 2018-01-25 20:56:57 -05:00
Tyler Goodlet 1b0269e51a Drop `Client.from_config()` factory - more cleanups 2018-01-25 20:56:57 -05:00
Tyler Goodlet e45cdf92f0 Log entire access config on exit 2018-01-25 20:56:57 -05:00
Tyler Goodlet c7258f3fb2 Build bold palette automatically 2018-01-22 22:05:42 -05:00
Tyler Goodlet 570d879146 Save tokens locally for use across runs
Store tokens in a local config file avoiding any refresh delay
unless necessary when the current access token expires.

Summary:
- move draft main routine into the `brokers` package mod
- start an api wrapper type
- always write the current access tokens to the config on teardown
2018-01-22 22:05:42 -05:00
Tyler Goodlet e312fb6525 Add config for saving access creds between runs 2018-01-22 22:05:42 -05:00
Tyler Goodlet 9745e16cf2 Drop userdata; use a dict 2018-01-22 22:05:42 -05:00
Tyler Goodlet 5a6b7510f8 Initial user account access using trio + asks 2018-01-22 22:05:42 -05:00
Tyler Goodlet 15fe580326 Super hot hipster logging 2018-01-22 22:05:42 -05:00