Commit Graph

1984 Commits (0cb05ef868c82cc1b2c21871f387e0ce385b03b3)

Author SHA1 Message Date
Tyler Goodlet 5775c5fe71 WIP get `pikerd` working with and without `--tsdb` flag 2022-03-11 19:50:58 -05:00
Tyler Goodlet 820dfff08a Add context-styled `asyncio` client proxy for ib
This adds a new client manager-factory: `open_client_proxy()` which uses
the newer `tractor.to_asyncio.open_channel_from()` (and thus the
inter-loop-task-channel style) a `aio_client_method_relay()` and
a re-implemented `MethodProxy` wrapper to allow transparently calling
`asyncio` client methods from `trio` tasks. Use this proxy in the
history backfiller task and add a new (prototype)
`open_history_client()` which will be used in the new storage management
layer. Drop `get_client()` which was the portal wrapping equivalent of
the same proxy but with a one-task-per-call approach. Oh, and
`Client.bars()` can take `datetime`, so let's use it B)
2022-03-11 19:50:58 -05:00
Tyler Goodlet cf589c840d Move ib data reset script into a new `scripts/` dir 2022-03-11 19:50:58 -05:00
Tyler Goodlet bbaba71465 Use new `tractor.query_actor()` for service checking 2022-03-11 19:50:58 -05:00
Tyler Goodlet 73aebdfa16 Return all timeframe arrays if `timeframe` not passed as input 2022-03-11 19:50:58 -05:00
Tyler Goodlet d9862a4962 Convert `iter_ohlc_periods()` to a `@tractor.context` 2022-03-11 19:50:58 -05:00
Tyler Goodlet de599233af Make `pikerd` work again without `--tsdb` flag 2022-03-11 19:50:58 -05:00
Tyler Goodlet 855d02ef5a Add a service checker predicate 2022-03-11 19:50:58 -05:00
Tyler Goodlet 7fbd4a95e3 Allow kill-child-proc-with-root-perms to fail silently in `tractor` reaping 2022-03-11 19:50:58 -05:00
Tyler Goodlet 847c95d277 Proxy `marketstore` container log level to our own 2022-03-11 19:50:58 -05:00
Tyler Goodlet 8af76322c9 Prototype a high level `Storage` api
Starts a wrapper around the `marketstore` client to do basic ohlcv query
and retrieval and prototypes out write methods for ohlc and tick.
Try to connect to `marketstore` automatically (which will fail if not
started currently) but we will eventually first do a service query.
2022-03-11 19:50:58 -05:00
Tyler Goodlet eb5a4f7eeb Move factor helper to a classmethod 2022-03-11 19:50:58 -05:00
Tyler Goodlet e008f69505 Doc str formatting 2022-03-11 19:50:58 -05:00
Tyler Goodlet 6c8b79906b Make linux timeout the same 2022-03-11 19:50:58 -05:00
Tyler Goodlet 40e62c1a38 Add latency measures around diffs/writes to mkts 2022-03-11 19:50:58 -05:00
Tyler Goodlet bed47d3ae6 Add flag to avoid logging json to console 2022-03-11 19:50:58 -05:00
Tyler Goodlet f60d9dd79c Prototype out writing `1Sec` OHLCV data 2022-03-11 19:50:58 -05:00
Tyler Goodlet 4402b2dc73 Better doc string 2022-03-11 19:50:58 -05:00
Tyler Goodlet 6e37ab6bf9 Use `asyncio` in `Client.get_quote()` 2022-03-11 19:50:58 -05:00
Tyler Goodlet 88411a6a26 Persist backing `/data/` filesystem across container runs 2022-03-11 19:50:58 -05:00
Tyler Goodlet a0c3d5f32f Get basic OHLCV writes working with `anyio` client 2022-03-11 19:50:58 -05:00
Tyler Goodlet 236df4b6d6 Pass in daemon name to `start_ahab()` 2022-03-11 19:50:58 -05:00
Tyler Goodlet a3ec0c16c6 Map the grpc port and add graceful container teardown
Not sure how I missed mapping the 5995 grpc port 🤦; done now.
Also adds graceful teardown using SIGINT with included container
logging relayed to the piker console B).
2022-03-11 19:50:58 -05:00
Tyler Goodlet 51ced95962 Revive `ms-shell` sub-cmd 2022-03-11 19:50:58 -05:00
Tyler Goodlet 3487f76147 Add WIP backfiller from data feed helper 2022-03-11 19:50:58 -05:00
Tyler Goodlet fa69fca311 Better handle nested erros from docker client 2022-03-11 19:50:58 -05:00
Tyler Goodlet 57b3d2f7e4 Add back in legacy write loop for reference 2022-03-11 19:50:58 -05:00
Tyler Goodlet f9b799b53d Add back in OHLCV dtype template and client side ws streamer 2022-03-11 19:50:58 -05:00
Tyler Goodlet 35f7c3409a Drop ununsed `Services` ref 2022-03-11 19:50:58 -05:00
Tyler Goodlet 9c5f7a6bb9 Py3.9+ type updates 2022-03-11 19:50:58 -05:00
Tyler Goodlet 86337430d8 Add `--tsdb` flag to start `marketstore` with `pikerd` 2022-03-11 19:50:58 -05:00
Tyler Goodlet 8d09d63095 De-escalate sudo perms in `pikerd` once docker spawns 2022-03-11 19:50:58 -05:00
Tyler Goodlet df04ccb845 Handle the non-root perms case specifically too 2022-03-11 19:50:58 -05:00
Tyler Goodlet ad0ace2528 Add explicit no-docker error and supervisor start task-func 2022-03-11 19:50:58 -05:00
Tyler Goodlet edd273d5d8 Type annot updates 2022-03-11 19:50:58 -05:00
Tyler Goodlet cfc77a0a66 Drop old client instantiate line 2022-03-11 19:50:58 -05:00
Tyler Goodlet 69b3120444 Drop import, it's got madness with and SIGINT? 2022-03-11 19:50:58 -05:00
Tyler Goodlet 8662cde7ca Add a super simple `marketstore` container supervisor 2022-03-11 19:50:58 -05:00
Tyler Goodlet 73b3f7ead8 Extract non-sudo user for config dir path 2022-03-11 19:50:58 -05:00
Tyler Goodlet c3509e7f93 Basic module-script for spawning `marketstore`, needs correct bind mount usage 2022-03-11 19:50:58 -05:00
Tyler Goodlet fea645423e Add `anyio-marketstore` as requirements.txt dep 2022-03-11 19:50:58 -05:00
Guillermo Rodriguez d215a69049 Still WIP, switch to using new marketstore client, missing streaming from marketstore 2022-03-11 19:50:58 -05:00
Guillermo Rodriguez a11cee82d0 Simplify and optimize tick format, similar to techtonicdb's 2022-03-11 19:50:58 -05:00
Guillermo Rodriguez aba50515df Add multi ingestor support and update to new feed API 2022-03-11 19:50:58 -05:00
Tyler Goodlet 1b1bf07f54 Array diff lengths must be int 2022-03-11 16:49:58 -05:00
Tyler Goodlet 77a7b73260 Open feeds using `Portal.open_context()` 2022-03-11 16:49:37 -05:00
Tyler Goodlet 4ad06e4cc0 Limit real-time chart updates in "big data" cases
- the chart's uppx (units-per-pixel) is > 4 (i.e. zoomed out a lot)
- don't shift the chart (to keep the most recent step in view) if the
  last datum isn't in view (aka the user is probably looking at history)
2022-03-11 16:31:28 -05:00
Tyler Goodlet 3da081c67a Only trigger downsampling on manual changes, add a uppx method 2022-03-11 16:31:20 -05:00
Tyler Goodlet d56d1fc4c1 Add for a `BarItems` to display a line on high uppx
When a bars graphic is zoomed out enough you get a high uppx, datum
units-per-pixel, and there is no point in drawing the 6-lines in each
bar element-graphic if you can't see them on the screen/display device.

Instead here we offer converting to a `FastAppendCurve` which traces
the high-low outline and instead display that when it's impossible to see the
details of bars - approximately when the uppx >= 2.

There is also some draft-commented code in here for downsampling the
outlines as zoom level increases but it's not fully working and should
likely be factored out into a higher level api anyway.
2022-03-11 16:31:14 -05:00
Tyler Goodlet 544578c67d Original index offset was right 2022-03-11 16:31:07 -05:00