Commit Graph

96 Commits (ce074544eee9ab9530c1f1568b7cb42992f49d0d)

Author SHA1 Message Date
Tyler Goodlet e07b91ec73 Copy timestamps from source to FSP dest buffer 2023-01-13 13:23:01 -05:00
Tyler Goodlet d792fed099 Move sync log msg back to info 2023-01-13 13:21:49 -05:00
Tyler Goodlet a746258f99 `binance`: always request an extra 1min OHLC bar
Seems that by default their history indexing rounds down/back to the
previous time step, so make sure we add a minute inside `Client.bars()`
when the `end_dt=None`, indicating "get the latest bar". Add
a breakpoint block that should trigger whenever the latest bar vs. the
latest epoch time is mismatched; we'll remove this after some testing
verifying the history bars issue is resolved.

Further this drops the legacy `backfill_bars()` endpoint which has been
deprecated and unused for a while.
2023-01-13 13:21:49 -05:00
Tyler Goodlet e0ca5d5200 Use `open_sample_stream()` to increment fsp buffers 2023-01-13 13:21:47 -05:00
Tyler Goodlet b5f2ff854c Drop meaning the clearing rate, use per step count 2023-01-13 13:21:15 -05:00
Tyler Goodlet 89095d4e9f Ensure FSPs last 2 times are synced with its source 2023-01-13 13:21:15 -05:00
Tyler Goodlet d1b07c625f Copy timestamps from source to FSP dest buffer
Slice up to history's length worth of (latest) time stamps from source
series read at the start of the history init phase.
2023-01-13 13:21:15 -05:00
Tyler Goodlet 7da5c2b238 Add epoch time index to fsp buffers 2023-01-13 13:21:15 -05:00
Tyler Goodlet 43717c92d9 Type annot-declare fsp-engine data `Feed` 2023-01-13 13:21:15 -05:00
Tyler Goodlet b89fd9652c `binance`: always request an extra 1min OHLC bar
Seems that by default their history indexing rounds down/back to the
previous time step, so make sure we add a minute inside `Client.bars()`
when the `end_dt=None`, indicating "get the latest bar". Add
a breakpoint block that should trigger whenever the latest bar vs. the
latest epoch time is mismatched; we'll remove this after some testing
verifying the history bars issue is resolved.

Further this drops the legacy `backfill_bars()` endpoint which has been
deprecated and unused for a while.
2023-01-13 13:14:35 -05:00
Tyler Goodlet 2c4daf08e0 Adjust to per-fqsn-oriented `Flume` lookups throughout 2023-01-10 11:09:19 -05:00
Tyler Goodlet d46945cb09 Move profiler imports to internal version 2022-10-31 09:26:36 -04:00
Tyler Goodlet f79c3617d6 Always load FSPs with the default (fast) sampling period 2022-09-12 20:25:15 -04:00
Tyler Goodlet e0491cf2e7 Cache fsp ``ShmArrays`` where possible
Minimize calling `.data._shmarray.attach_shm_array()` as much as is
possible to avoid the crash from #332. This is the suggested hack from
issue #359.

Resolves https://github.com/pikers/piker/issues/359
2022-07-19 09:07:40 -04:00
Tyler Goodlet 920a394539 Use new `anext()` builtin 2022-06-27 13:59:08 -04:00
Tyler Goodlet 958f53d8e9 Lower re-syncing log msgs to debug level 2022-06-16 15:50:21 -04:00
Tyler Goodlet 4f36743f64 Only udpate prepended graphics when actually in view 2022-06-05 22:13:08 -04:00
Tyler Goodlet 423af37389 Truncate trade rate wma window sizes 2022-05-09 11:15:14 -04:00
Tyler Goodlet f1f257d4a2 Profiler format, code stretch 2022-04-30 11:36:23 -04:00
Tyler Goodlet d8db9233c9 Establish stream before `fsp_compute` so that backfill updates work again.. 2022-04-16 13:25:14 -04:00
Tyler Goodlet 2564acea1b Facepalm**2: only update on special "update" msg 2022-04-16 13:25:14 -04:00
Tyler Goodlet 761b823939 Always fire a "step/update message" on every fsp history update 2022-04-16 13:25:14 -04:00
Tyler Goodlet 6ac60fbe22 Expect fqsns through fsp machinery 2022-04-10 17:33:02 -04:00
Tyler Goodlet 3e7d4f8717 Detect and request sample period in fsp engine 2022-02-28 08:30:32 -05:00
Tyler Goodlet 326b2c089a Drop dvlm 'rates' (they're just means), add default params, period -> 6 2022-02-08 12:04:01 -05:00
Tyler Goodlet 4a7b2d835b Yield 0 initial values from `flow_rates` fsp 2022-02-08 07:46:36 -05:00
Tyler Goodlet ee4ad32d3b Fix `dvlm` to actually yield trade count, add instantaneous support 2022-02-07 12:53:30 -05:00
Tyler Goodlet 73faafcfc1 Add trade "rates" (i.e. trade counts) support B)
Though it's not per-tick accurate, accumulate the number of "trades"
(i.e. the "clearing rate" - maybe this is a better name?) per bar
inside the `dolla_vlm` fsp and average and report wmas of this in the
`flow_rates` fsp.
2022-02-07 12:53:30 -05:00
Tyler Goodlet 5274eb538c Add 16 period dollar vlm rates, drop ib rates for now 2022-02-07 12:53:30 -05:00
Tyler Goodlet b358b8e874 Move `wma` fsp earlier in module 2022-02-07 12:53:30 -05:00
Tyler Goodlet 1fc6429f75 Prep for manual rate calcs, handle non-ib backends XD 2022-02-07 12:53:30 -05:00
Tyler Goodlet ebf3e00438 Add `Fsp._flow_registry` as actor-local table
Define the flows table as a class var (thus making it a "global" and/or
actor-local state) which can be accessed by any in process task. Add
`Fsp.get_shm()` to allow accessing output streams by source-token + fsp
routine reference and thus providing inter-fsp low level access to
real-time flows.
2022-02-07 12:53:30 -05:00
Tyler Goodlet d130f0449f Expect registry of fsp "flows" to each engine task
In order for fsp routines to be able to look up other "flows" in the
cascade, we need a small registry-table which gives access to a map of
a source stream + an fsp -> an output stream. Eventually we'll also
likely want a dependency (injection) mechanism so that any fsp demanded
can either be dynamically allocated or at the least waited upon before
a consumer tries to access it.
2022-02-07 12:53:30 -05:00
Tyler Goodlet d4f79a6245 Comment flow rates fsp prints 2022-02-07 12:53:30 -05:00
Tyler Goodlet 97c2f86092 TOSQUASH, fix separate vlm vs trade rate 2022-02-07 12:53:30 -05:00
Tyler Goodlet f3289c1977 Create source length zeroed arrays from yielded `None` fsp history 2022-02-07 12:53:30 -05:00
Tyler Goodlet b81209e78e Ensure `sym` arg is a `str` 2022-02-07 12:53:30 -05:00
Tyler Goodlet dfe4473c9a Yield history `dict`s, add a `flow_rates` fsp 2022-02-07 12:53:30 -05:00
Tyler Goodlet 1aae40cdeb Expect multi-output fsps to yield a `dict` of history arrays 2022-02-07 12:53:30 -05:00
Tyler Goodlet 296863348d Update imports to `tractor.msg.NamespacePath` 2022-01-30 12:46:20 -05:00
Tyler Goodlet 28b5be0719 Accumulate dark vlm ticks independently per sample step 2022-01-28 08:46:04 -05:00
Tyler Goodlet 67de8f24b9 Init history output with `np.zeros()` 2022-01-28 08:45:28 -05:00
Tyler Goodlet bd2460846e Decorate momo routines 2022-01-28 07:43:49 -05:00
Tyler Goodlet be93ded0e5 Log fsp registy entries in `cascade` startup 2022-01-28 07:43:23 -05:00
Tyler Goodlet 9d9929fb89 Drop old `wrapt` cruft, add `Fsp.name` 2022-01-28 07:18:14 -05:00
Tyler Goodlet cc5390376c Use `Fsp` abstration layer through engine and UI
Instead of referencing the remote processing funcs by a `str` name start
embracing the new `@fsp`/`Fsp` API such that wrapped processing
functions are first class APIs.

Summary of the changeset:
- move and load the fsp built-in set in the new `.fsp._api` module
- handle processors ("fsps") which want to yield multiple keyed-values
  (interleaved in time) by expecting both history that is keyed and
  assigned to the appropriate struct-array field, *and* real-time
  `yield`ed value in tuples of the form `tuple[str, float]` such that
  any one (async) processing function can deliver multiple outputs from
  the same base calculation.
- drop `maybe_mk_fsp_shm()` from UI module
- expect and manage `Fsp` instances (`@fsp` decorated funcs) throughout
  the UI code, particularly the `FspAdmin` layer.
2022-01-27 18:57:16 -05:00
Tyler Goodlet 72f4474273 Use new decorator on volume fsp routines 2022-01-27 09:08:03 -05:00
Tyler Goodlet c6a3c66e7e WIP start a `@piker.fsp` API for registering processors 2022-01-26 14:38:49 -05:00
Tyler Goodlet 13b8807f1f Print dark trades to console for the moment 2022-01-26 13:48:21 -05:00
Tyler Goodlet d351fe14a8 Annoying doc string(s) 2022-01-25 08:30:00 -05:00