Commit Graph

399 Commits (2dec32e41f600393a259eab52b9c6e0729b33041)

Author SHA1 Message Date
Tyler Goodlet 4ce99e62e0 Override annoying stuff in ib_insync 2020-09-29 17:06:28 -04:00
Tyler Goodlet 41c6517a23 Port to new streaming api, yield whole tickers 2020-09-29 17:06:28 -04:00
Tyler Goodlet 450a39ce1c Add better contract search/lookup
Add a `Client.find_contract()` which internally takes
a <symbol>.<exchange> str as input and uses `IB.qualifyContractsAsync()`
internally to try and validate the most likely contract. Make the module
script call this using `asyncio.run()` for console testing.
2020-09-29 17:06:28 -04:00
Tyler Goodlet b8209cd506 Add a mostly actor aware API to IB backend
Infected `asyncio` support is being added to `tractor` in
goodboy/tractor#121 so delegate to all that new machinery.

Start building out an "actor-aware" api which takes care of all the
`trio`-`asyncio` interaction for data streaming and request handling.
Add a little (shudder) method proxy system which can be used to invoke
client methods from another actor. Start on a streaming api in
preparation for real-time charting.
2020-09-29 17:06:28 -04:00
Tyler Goodlet 1abadeb506 Add initial IB broker backend using ib_insync
Start working towards meeting the backend client api.
Infect `asyncio` using `trio`'s new guest mode and demonstrate
real-time ticker streaming to console.
2020-09-29 17:06:28 -04:00
Tyler Goodlet 14a5d047c3 Copy forward stupid kraken zeroed vwaps 2020-09-29 16:42:28 -04:00
Tyler Goodlet b13da849d0 Include vwap in kraken historical bars 2020-09-29 16:42:06 -04:00
Tyler Goodlet 0b42ac1420 Normalize kraken quotes for latency tracking 2020-09-26 11:31:57 -04:00
Tyler Goodlet ea8205968c Begin to use `@tractor.msg.pub` throughout streaming API
Since the new FSP system will require time aligned data amongst actors,
it makes sense to share broker data feeds as much as possible on a local
system. There doesn't seem to be downside to this approach either since
if not fanning-out in our code, the broker (server) has to do it anyway
(and who knows how junk their implementation is) though with more
clients, sockets etc. in memory on our end. It also preps the code for
introducing a more "serious" pub-sub systems like zeromq/nanomessage.
2020-09-26 11:31:57 -04:00
Tyler Goodlet 44010abf4d Handle (far end forced) disconnects 2020-09-26 11:31:57 -04:00
Tyler Goodlet 03c5c7d2ba Trigger connection reset on slowed heartbeat 2020-09-26 11:31:57 -04:00
Tyler Goodlet e92abd376a Trace log the heartbeat 2020-09-26 11:31:57 -04:00
Tyler Goodlet bf9a0136df Make ws loop restart on connection failures 2020-09-26 11:31:57 -04:00
Tyler Goodlet d976f3d074 Generate tick data correctly using .etime 2020-09-26 11:31:57 -04:00
Tyler Goodlet ad92188703 Support new normalized ticks format with kraken
Generate tick datums in a list under a `ticks` field in each quote
kinda like how IB does it.
2020-09-26 11:31:57 -04:00
Tyler Goodlet 9976bc3a3b Fix typo 2020-09-26 11:31:57 -04:00
Tyler Goodlet 3655e449d6 Raise errors, fix module script entry 2020-09-26 11:31:57 -04:00
Tyler Goodlet ffe47acf1d Add historical bars retreival 2020-09-26 11:31:57 -04:00
Tyler Goodlet 7bccfc7b10 Convert to stream, parse into dataclass 2020-09-26 11:31:57 -04:00
Tyler Goodlet 2738b54851 Start kraken backend 2020-09-26 11:31:57 -04:00
goodboy 67a75c3080
Merge pull request #114 from pikers/facepalm
Add marketstore cli stuff; not sure how this got missed?!
2020-09-26 11:29:36 -04:00
Tyler Goodlet 53eb564f06 Add marketstore cli stuff; not sure how this got missed?! 2020-09-25 16:08:15 -04:00
Tyler Goodlet 05d2985f5f Clarify some odd spots 2020-09-02 11:32:54 -04:00
Tyler Goodlet 316137fdf2 Begin to wrap marketstore as a data feed
Wrap the sync client in an async interface in anticipation of an actual
async client. This starts support for the `open_fee()`/`stream_quotes()`
api though the tick normalization isn't correct yet.
2020-09-02 00:40:35 -04:00
Tyler Goodlet 702c63f607 Define "packetizer" in specific broker mod
Allows for formatting published quotes using a broker specific
formatting callback.
2020-09-02 00:36:19 -04:00
Tyler Goodlet 312169e790 Support the `stream_quotes()` api in questrade backend 2020-09-01 20:55:24 -04:00
Tyler Goodlet 60b74ad7d1 Use new method name 2020-09-01 20:53:45 -04:00
Tyler Goodlet a6de623147 Add search command to cli 2020-09-01 20:53:27 -04:00
Tyler Goodlet 75f98276cc Add symbol search to broker api 2020-09-01 20:53:04 -04:00
Tyler Goodlet 519712e128 Add stocks search to qt client 2020-09-01 20:52:01 -04:00
Tyler Goodlet 934108a024 Add symbol-info command 2020-09-01 20:51:09 -04:00
Tyler Goodlet dcb0a30ad6 Move UI spawning cmds to new module 2020-09-01 20:47:24 -04:00
Tyler Goodlet 3c4699abef Pass broker name 2020-09-01 18:31:52 -04:00
Tyler Goodlet 78784a4bf3 Port to new data apis 2020-09-01 18:30:55 -04:00
Tyler Goodlet 57a8db8cba Start enforcing a common stream setup api
Add routines for brokerd spawning and quote stream creation.
2020-09-01 18:18:48 -04:00
Tyler Goodlet f6f6d98a95 Allow passing in tbk keys to query 2020-09-01 18:14:11 -04:00
Tyler Goodlet bc9af977a4 Update quote cache on each loop 2020-09-01 13:25:41 -04:00
Tyler Goodlet 436e4d2df4 Add tbk tick streaming with trio-websocket 2020-09-01 13:25:41 -04:00
Tyler Goodlet acd32341e2 Fix assignment out of order 2020-09-01 13:25:41 -04:00
Tyler Goodlet 3cfb15ed6e Make monitor handle non-full quote messages 2020-09-01 13:25:41 -04:00
Tyler Goodlet d66cfb8fa0 Push only new key value pairs over quote streams
This is something I've been meaning to try for a while and will likely
make writing tick data to a db more straight forward (filling in NaN
values is more matter of fact) plus it should minimize bandwidth usage.
Note, it'll require stream consumers to be considerate of non-full
quotes arriving and thus using the first "full" quote message to fill
out dynamically formatted systems or displays.
2020-09-01 13:25:41 -04:00
Tyler Goodlet 335cee63b2 Make stock quote formatter work with diff streams 2020-09-01 13:25:41 -04:00
Tyler Goodlet a6c692fb8b Add support for TICK ingest to marketstore 2020-09-01 13:25:09 -04:00
Tyler Goodlet b58820e297 Relicense to AGPLv3 2020-06-16 00:12:19 -04:00
Tyler Goodlet 2c93bb0302 Drop use of deprecated `trio.Event.clear()` 2020-06-02 10:33:57 -04:00
Tyler Goodlet 882f4a28d1 Add privacy note 2020-06-01 14:43:39 -04:00
Tyler Goodlet 64993666fa Pull 100 bars by default 2020-05-26 14:49:50 -04:00
Tyler Goodlet 5f200b8db1 Always delay bars 15m; rename to volume 2020-05-26 14:49:50 -04:00
Tyler Goodlet a7f7de32b4 Docstring tweaks 2020-05-26 14:49:50 -04:00
Tyler Goodlet ffd8bdac5f Fix imports 2020-05-26 14:49:50 -04:00
Tyler Goodlet 9b91ecd408 Pass confdir and watchlist path to cli 2020-05-26 14:49:50 -04:00
Tyler Goodlet 057086efdb Pass through test file to monitor 2020-05-26 14:49:50 -04:00
Tyler Goodlet f798d13038 Reord watchlist cmds into new package 2020-05-26 14:49:50 -04:00
Tyler Goodlet ff843372a1 Add `piker bars` command
For easy testing of questrade historical data from cli.
Re-org the common cli components into a new package to avoid having all
commands defined in a top-level module.
2020-05-26 14:49:50 -04:00
Tyler Goodlet c11946988e Implement candles retrieval from Questrade
There's some expected limitations with the number of sticks allowed in
a single query (they say 2k but I've been able to pull 20k). Also note
without a paid data sub there's a 15m delay on 1m sticks (we'll hack
around that shortly, don't worry).
2020-05-26 14:49:50 -04:00
Tyler Goodlet 595f79c632 Add symbol not found 2020-05-26 14:49:50 -04:00
Tyler Goodlet 732638ae2d Delegate logging to `tractor`
Gives us PIDs and task ids in logs which is much more sane for
debugging and cuts down on superfluous code.
2020-02-29 12:29:20 -05:00
Tyler Goodlet 837e30ff68 use >1 asks session connections
Gets us better throughput when polling multiple endpoints (eg. option
and stock quotes simultaneously) since slower round trip request won't
block faster ones when using multiple connections.
2020-02-18 16:13:20 -05:00
Tyler Goodlet a2e5d07b2c Port to mainline kivy and Python 3.8
This required some copy-paste of code from @matham's branch:
https://github.com/kivy/kivy/pull/5241

namely, the stuff in the `utils_async.py` module. I've added all that as
a standalone file for now.

Update the pipfile to use `kivy`'s master branch (since there seems to
be some lingering cython issues in the current release wheels).
2019-11-21 10:21:43 -05:00
Tyler Goodlet 49d612de49 Port to new @tractor.stream API 2019-04-26 00:27:01 -04:00
Tyler Goodlet 1cc33abca0 Don't bail when a sub-optschain crashes 2019-03-24 12:09:17 -04:00
Tyler Goodlet b2252e5762 Fix bad import 2019-03-21 22:18:09 -04:00
Tyler Goodlet 8bf82a2f76 Allow spawning an optschain UI from monitor using the `o` keybinding 2019-03-21 22:15:08 -04:00
Tyler Goodlet d59283d36d Update optschain cli entrypoint to pass primitive arg types 2019-03-21 22:14:22 -04:00
Tyler Goodlet 884fcaa88e A slew of tiny pager improvements
- stop displaying search bar widget on <ctrl-c>
- if there's existing search bar content highlight it automatically
  to allow user to start typing new content right away
- when activated allow search bar to insert its own set of keybinding
  controls; restore prior bindings on exit
2019-03-21 22:01:50 -04:00
Tyler Goodlet 3bbb1db2b4 Make option chain spawnable as a subactor
Look up the broker module and set up the loglevel locally.
Ask the arbiter for a portal to the data daemon since we can't
pass one to a subactor by reference.
2019-03-21 21:50:55 -04:00
Tyler Goodlet fc1e63b2c1 Spawn keyboard functions in new tasks 2019-03-18 22:39:21 -04:00
Tyler Goodlet c8da096126 Track table's last clicked row 2019-03-18 22:38:32 -04:00
Tyler Goodlet 3a439fc99d Fix ask use logic for testing/CI 2019-03-17 23:03:45 -04:00
Tyler Goodlet 6a50049af7 Use toml instead of ini for broker config 2019-03-15 19:37:04 -04:00
Tyler Goodlet 5e9c38039c Move brokerd spawner to `brokers.core` 2019-02-25 22:29:48 -05:00
Tyler Goodlet d3fae00e74 Add a `configdir` cli option 2019-02-25 20:22:29 -05:00
Tyler Goodlet 747d703d92 Add hack-fix to avoid leaking cmdline flags to kivy 2019-02-25 20:14:36 -05:00
Tyler Goodlet 77548d2ee6 Add token-from-user toggles to token auth methods 2019-02-25 20:11:45 -05:00
Tyler Goodlet 130553b8df Accept a path arg to `write()` 2019-02-25 19:29:54 -05:00
Tyler Goodlet 295ccbbe64 Name the data what it is 2019-02-24 10:55:52 -05:00
Tyler Goodlet c1a398d826 Use click context to factor cmd options 2019-02-23 16:13:04 -05:00
Tyler Goodlet 7ee731faac Use trio memory channels throughout UIs 2019-02-21 23:09:19 -05:00
Tyler Goodlet cbb973ae9d Drop internal nursery from option chain 2019-02-21 23:07:44 -05:00
Tyler Goodlet 462c419970 Add basic practice account support 2019-02-21 17:24:50 -05:00
Tyler Goodlet 435b2a56e8 Remove stream opening lock on `DataFeed`
Fixes to `tractor` that resolve issues with async generators being
non-task safe make the need for the mutex lock in
`DataFeed.open_stream()` unnecessary. Also, don't bother pushing empty
quotes from the publisher; avoids hitting the network when possible.
2019-02-20 21:39:57 -05:00
Tyler Goodlet 3a6efd451d Don't bother ensuring all symbols in data 2019-02-10 19:09:54 -05:00
Tyler Goodlet 3ab9e28ddb Use brokerd's client to get all contracts 2019-02-09 21:58:27 -05:00
Tyler Goodlet 395f0c8e4a Synchronize Questrade token refreshing per client
Questrade's API is half baked and can't handle concurrency.
It allows multiple concurrent requests to most endpoints *except*
for the auth endpoint used to refresh tokens:

    https://www.questrade.com/api/documentation/security

I've gone through extensive dialogue with their API team and despite
making what I think are very good arguments for doing the request
serialization on the server side, they decided that I should instead
do the "locking" on the client side. Frankly it doesn't seem like they
have that competent an engineering department as it took me a long time
to explain the issue even though it's rather trivial and probably not
that hard to fix; maybe it's better this way.

This adds a few things to ensure more reliable token refreshes on
expiry:

- add a `@refresh_token_on_err` decorator which can be used on `_API`
  methods that should refresh tokens on failure
- decorate most endpoints with this *except* for the auth ep
- add locking logic for the troublesome scenario as follows:
  * every time a request is sent out set a "request in progress" event
    variable that can be used to determine when no requests are currently
    outstanding
  * every time the auth end point is hit in order to refresh tokens set
    an event that locks out other tasks from making requests
  * only allow hitting the auth endpoint when there are no "requests in
    progress" using the first event
  * mutex all auth endpoint requests; there can only be one outstanding

- don't hit the accounts endpoint at client startup; we want to
  eventually support keys from multiple accounts and you can disable
  account info per key and just share the market data function
2019-02-09 21:39:22 -05:00
Tyler Goodlet f6230dd6df Add a `DataFeed.call_client()` method
Allows for calling an actor local broker client's methods from a remote
actor.
2019-02-09 21:38:00 -05:00
Tyler Goodlet 026b015627 Allow passing a config path for broker testing in CI 2019-02-04 00:17:11 -05:00
Tyler Goodlet 5dac8fa44d Note the RH auth/account requirements for usage 2019-02-04 00:15:10 -05:00
Tyler Goodlet e91a50a1ba Make `get_cached_feed()` an asynccontextmanager
Adjust feed locking around internal manager `yields` to make this work.

Also, change quote publisher to deliver a list of quotes for each
retrieved batch. This was actually broken for option streaming since
each quote was being overwritten due to a common `key` value for all
expiries. Asjust the `packetizer` function accordingly to work for
both options and stocks.
2019-02-03 23:40:51 -05:00
Tyler Goodlet 2514843fc1 Port to the new `@tractor.msg.pub` decorator API
The pub-sub data feed system was factored into `tractor` as an
experimental api / subsystem. Move to using that which greatly
simplifies the data feed architecture.
2019-01-27 14:50:04 -05:00
Tyler Goodlet 22670afe58 Generalize the publisher/fan-out system
Start working toward a more general (on-demand) pub-sub system which
can be brought into ``tractor``. Right now this just means making
the code in the `fan_out_to_ctxs()` less specific but, eventually
I think this function should be coupled with a decorator and shipped
as a standard "message pattern".

Additionally,
- try out making `BrokerFeed` a `@dataclass`
- strip out all the `trio.Event` / uneeded nursery / extra task crap
  from `start_quote_stream()`
2019-01-14 21:23:49 -05:00
Tyler Goodlet c94ce47aa6 Always set contract sub state 2019-01-14 21:13:22 -05:00
Tyler Goodlet 4753dc2db8 Alway teardown quote gen on exit 2019-01-14 21:12:35 -05:00
Tyler Goodlet 36d0c2ed68 Port monitor app to `DataFeed` api 2019-01-05 19:08:31 -05:00
Tyler Goodlet a4501bb0e0 Factor `DataFeed` client API into `brokers.data` 2019-01-05 19:08:27 -05:00
Tyler Goodlet 7f8c88be0c Drop open/close prices for now; never really use them 2019-01-02 21:24:42 -05:00
Tyler Goodlet fa6bae1f5c Reorg table widgets into a new module 2019-01-02 21:12:42 -05:00
Tyler Goodlet 4895690642 Display a message when no contracts exist 2019-01-01 23:42:49 -05:00
Tyler Goodlet 0cffa4b97a Font size shrinks 2019-01-01 23:36:46 -05:00
Tyler Goodlet 32a7f4cbd3 Right, gotta expose widgets to other actors 2018-12-31 11:51:04 -05:00
Tyler Goodlet 72f417b9c2 Support monitor linked symbol selection
This allows for using a monitor to select the current option chain
symbol!

The deats:
- start a bg task which streams the monitor selected symbol
- dynamically repopulate expiry buttons on a newly published symbol
- move static widget creation into a chain method to avoid multiple
  quotes requests at startup
- rename a bunch of methods
2018-12-30 15:00:46 -05:00