Commit Graph

1951 Commits (990417b172ca23b4a88fee39ec3c70f289ddacc9)

Author SHA1 Message Date
Tyler Goodlet 990417b172 Maybe spawn `brokerd` in `asyncio` mode if declared in backend mod 2022-03-17 09:03:44 -04:00
Tyler Goodlet 5d09d8258f WIP add non-working m4 ds code to ohlc graphic 2022-03-17 09:00:59 -04:00
Tyler Goodlet 3e72b59658 Use service cancel method for graceful teardown 2022-03-16 09:54:52 -04:00
Tyler Goodlet a3b282dffe Add curve px width getter
`ChartPlotWidget.curve_width_pxs()` now can be used to get the total
horizontal (x) pixels on screen that are occupied by the current curve
graphics for a given chart. This will be used for downsampling large
data sets to the pixel domain using M4.
2022-03-16 07:28:03 -04:00
Tyler Goodlet 23a368b5e5 Add display loop profiling
Probably the best place to root the profiler since we can get a better
top down view of bottlenecks in the graphics stack.

More,
- add in draft M4 downsampling code (commented) after getting it mostly
  working; next step is to move this processing into an FSP subactor.
- always update the vlm chart last y-axis sticky
- set call `.default_view()` just before inf sleep on startup
2022-03-16 07:24:14 -04:00
Tyler Goodlet a4dd6c81dc Profiler format, code stretch 2022-03-15 14:13:06 -04:00
Tyler Goodlet a2ef955690 Fix x-range -> # of frames calculation
Obviously determining the x-range from indices was wrong and was the
reason for the incorrect (downsampled) output size XD. Instead correctly
determine the x range and start value from the *values of* the input
x-array. Pretty sure this makes the implementation nearly production
ready.

Relates to #109
2022-03-15 14:03:44 -04:00
Tyler Goodlet 6d9a94065d Be mega-tolerant to feed consumer disconnects 2022-03-15 10:45:51 -04:00
Tyler Goodlet c976bff40c Add `ChartPlotWidget.in_view()` shm-compatible array slicer 2022-03-15 09:11:12 -04:00
Tyler Goodlet 11bda4f9b4 Add (ostensibly) working first attempt at M4 algo
All the refs are in the comments and original sample code from infinite
has been reworked to expect the input x/y arrays to already be sliced
(though we can later support passing in the start-end indexes if
desired).

The new routines are `ds_m4()` the python top level API and `_m4()` the
fast `numba` implementation.
2022-03-15 09:06:35 -04:00
Tyler Goodlet 803c65bc88 Add no-path guard now that we can use a poly 2022-03-14 06:04:18 -04:00
Tyler Goodlet cf7163194c Try downsampling mkts data 2022-03-11 19:50:58 -05:00
Tyler Goodlet afe41236ff Drop old type annot 2022-03-11 19:50:58 -05:00
Tyler Goodlet b4d35496f7 Comment each special key combo 2022-03-11 19:50:58 -05:00
Tyler Goodlet c5be35dad4 Load any symbol-matching shm array if no `marketstored` found 2022-03-11 19:50:58 -05:00
Tyler Goodlet e33d0aac15 Support no spawning `brokerd` with no real-time quotes 2022-03-11 19:50:58 -05:00
Tyler Goodlet 02ba7b6b96 Get ib key hack script to work with reconnect 2022-03-11 19:50:58 -05:00
Tyler Goodlet 5775c5fe71 WIP get `pikerd` working with and without `--tsdb` flag 2022-03-11 19:50:58 -05:00
Tyler Goodlet 820dfff08a Add context-styled `asyncio` client proxy for ib
This adds a new client manager-factory: `open_client_proxy()` which uses
the newer `tractor.to_asyncio.open_channel_from()` (and thus the
inter-loop-task-channel style) a `aio_client_method_relay()` and
a re-implemented `MethodProxy` wrapper to allow transparently calling
`asyncio` client methods from `trio` tasks. Use this proxy in the
history backfiller task and add a new (prototype)
`open_history_client()` which will be used in the new storage management
layer. Drop `get_client()` which was the portal wrapping equivalent of
the same proxy but with a one-task-per-call approach. Oh, and
`Client.bars()` can take `datetime`, so let's use it B)
2022-03-11 19:50:58 -05:00
Tyler Goodlet cf589c840d Move ib data reset script into a new `scripts/` dir 2022-03-11 19:50:58 -05:00
Tyler Goodlet bbaba71465 Use new `tractor.query_actor()` for service checking 2022-03-11 19:50:58 -05:00
Tyler Goodlet 73aebdfa16 Return all timeframe arrays if `timeframe` not passed as input 2022-03-11 19:50:58 -05:00
Tyler Goodlet d9862a4962 Convert `iter_ohlc_periods()` to a `@tractor.context` 2022-03-11 19:50:58 -05:00
Tyler Goodlet de599233af Make `pikerd` work again without `--tsdb` flag 2022-03-11 19:50:58 -05:00
Tyler Goodlet 855d02ef5a Add a service checker predicate 2022-03-11 19:50:58 -05:00
Tyler Goodlet 7fbd4a95e3 Allow kill-child-proc-with-root-perms to fail silently in `tractor` reaping 2022-03-11 19:50:58 -05:00
Tyler Goodlet 847c95d277 Proxy `marketstore` container log level to our own 2022-03-11 19:50:58 -05:00
Tyler Goodlet 8af76322c9 Prototype a high level `Storage` api
Starts a wrapper around the `marketstore` client to do basic ohlcv query
and retrieval and prototypes out write methods for ohlc and tick.
Try to connect to `marketstore` automatically (which will fail if not
started currently) but we will eventually first do a service query.
2022-03-11 19:50:58 -05:00
Tyler Goodlet eb5a4f7eeb Move factor helper to a classmethod 2022-03-11 19:50:58 -05:00
Tyler Goodlet e008f69505 Doc str formatting 2022-03-11 19:50:58 -05:00
Tyler Goodlet 6c8b79906b Make linux timeout the same 2022-03-11 19:50:58 -05:00
Tyler Goodlet 40e62c1a38 Add latency measures around diffs/writes to mkts 2022-03-11 19:50:58 -05:00
Tyler Goodlet bed47d3ae6 Add flag to avoid logging json to console 2022-03-11 19:50:58 -05:00
Tyler Goodlet f60d9dd79c Prototype out writing `1Sec` OHLCV data 2022-03-11 19:50:58 -05:00
Tyler Goodlet 4402b2dc73 Better doc string 2022-03-11 19:50:58 -05:00
Tyler Goodlet 6e37ab6bf9 Use `asyncio` in `Client.get_quote()` 2022-03-11 19:50:58 -05:00
Tyler Goodlet 88411a6a26 Persist backing `/data/` filesystem across container runs 2022-03-11 19:50:58 -05:00
Tyler Goodlet a0c3d5f32f Get basic OHLCV writes working with `anyio` client 2022-03-11 19:50:58 -05:00
Tyler Goodlet 236df4b6d6 Pass in daemon name to `start_ahab()` 2022-03-11 19:50:58 -05:00
Tyler Goodlet a3ec0c16c6 Map the grpc port and add graceful container teardown
Not sure how I missed mapping the 5995 grpc port 🤦; done now.
Also adds graceful teardown using SIGINT with included container
logging relayed to the piker console B).
2022-03-11 19:50:58 -05:00
Tyler Goodlet 51ced95962 Revive `ms-shell` sub-cmd 2022-03-11 19:50:58 -05:00
Tyler Goodlet 3487f76147 Add WIP backfiller from data feed helper 2022-03-11 19:50:58 -05:00
Tyler Goodlet fa69fca311 Better handle nested erros from docker client 2022-03-11 19:50:58 -05:00
Tyler Goodlet 57b3d2f7e4 Add back in legacy write loop for reference 2022-03-11 19:50:58 -05:00
Tyler Goodlet f9b799b53d Add back in OHLCV dtype template and client side ws streamer 2022-03-11 19:50:58 -05:00
Tyler Goodlet 35f7c3409a Drop ununsed `Services` ref 2022-03-11 19:50:58 -05:00
Tyler Goodlet 9c5f7a6bb9 Py3.9+ type updates 2022-03-11 19:50:58 -05:00
Tyler Goodlet 86337430d8 Add `--tsdb` flag to start `marketstore` with `pikerd` 2022-03-11 19:50:58 -05:00
Tyler Goodlet 8d09d63095 De-escalate sudo perms in `pikerd` once docker spawns 2022-03-11 19:50:58 -05:00
Tyler Goodlet df04ccb845 Handle the non-root perms case specifically too 2022-03-11 19:50:58 -05:00