Commit Graph

1752 Commits (9c88b26d85c46ca6016886c7cd2bcd7b65819400)

Author SHA1 Message Date
Tyler Goodlet ad0ace2528 Add explicit no-docker error and supervisor start task-func 2022-03-11 19:50:58 -05:00
Tyler Goodlet edd273d5d8 Type annot updates 2022-03-11 19:50:58 -05:00
Tyler Goodlet cfc77a0a66 Drop old client instantiate line 2022-03-11 19:50:58 -05:00
Tyler Goodlet 69b3120444 Drop import, it's got madness with and SIGINT? 2022-03-11 19:50:58 -05:00
Tyler Goodlet 8662cde7ca Add a super simple `marketstore` container supervisor 2022-03-11 19:50:58 -05:00
Tyler Goodlet 73b3f7ead8 Extract non-sudo user for config dir path 2022-03-11 19:50:58 -05:00
Tyler Goodlet c3509e7f93 Basic module-script for spawning `marketstore`, needs correct bind mount usage 2022-03-11 19:50:58 -05:00
Guillermo Rodriguez d215a69049 Still WIP, switch to using new marketstore client, missing streaming from marketstore 2022-03-11 19:50:58 -05:00
Guillermo Rodriguez a11cee82d0 Simplify and optimize tick format, similar to techtonicdb's 2022-03-11 19:50:58 -05:00
Guillermo Rodriguez aba50515df Add multi ingestor support and update to new feed API 2022-03-11 19:50:58 -05:00
Tyler Goodlet 1b1bf07f54 Array diff lengths must be int 2022-03-11 16:49:58 -05:00
Tyler Goodlet 77a7b73260 Open feeds using `Portal.open_context()` 2022-03-11 16:49:37 -05:00
Tyler Goodlet 4ad06e4cc0 Limit real-time chart updates in "big data" cases
- the chart's uppx (units-per-pixel) is > 4 (i.e. zoomed out a lot)
- don't shift the chart (to keep the most recent step in view) if the
  last datum isn't in view (aka the user is probably looking at history)
2022-03-11 16:31:28 -05:00
Tyler Goodlet 3da081c67a Only trigger downsampling on manual changes, add a uppx method 2022-03-11 16:31:20 -05:00
Tyler Goodlet d56d1fc4c1 Add for a `BarItems` to display a line on high uppx
When a bars graphic is zoomed out enough you get a high uppx, datum
units-per-pixel, and there is no point in drawing the 6-lines in each
bar element-graphic if you can't see them on the screen/display device.

Instead here we offer converting to a `FastAppendCurve` which traces
the high-low outline and instead display that when it's impossible to see the
details of bars - approximately when the uppx >= 2.

There is also some draft-commented code in here for downsampling the
outlines as zoom level increases but it's not fully working and should
likely be factored out into a higher level api anyway.
2022-03-11 16:31:14 -05:00
Tyler Goodlet 544578c67d Original index offset was right 2022-03-11 16:31:07 -05:00
Tyler Goodlet 01ea2b3110 Add an ohlcv high/low tracer with optional downsampling 2022-03-11 16:30:52 -05:00
Tyler Goodlet 2f02f71610 Pass linked charts into `BarItems` so that graphics can be cycled on downsample 2022-03-11 16:24:43 -05:00
Tyler Goodlet b318ebc221 Move graphics compression routines to new module 2022-03-11 16:24:30 -05:00
Tyler Goodlet d737adb1b8 Use 12Hz as default fps throttle 2022-03-11 16:24:22 -05:00
Tyler Goodlet 75d7314493 Add comment on how to enable `pyqtgraph` profiling 2022-03-11 16:24:15 -05:00
Tyler Goodlet d7d824030d Add basic optional polyline support, draft out downsampling routine 2022-03-11 16:24:02 -05:00
Tyler Goodlet 28436bcb2b Add a downsampled line-curve support to `BarItems`
In effort to start getting some graphics speedups as detailed in #109,
this adds a `FastAppendCurve`to every `BarItems` as a `._ds_line` which
is only displayed (instead of the normal mult-line bars curve) when the
"width" of a bar is indistinguishable on screen from a line -> so once
the view coordinates map to > 2 pixels on the display device.
`BarItems.maybe_paint_line()` takes care of this scaling detection logic and is
called by the associated view's `.sigXRangeChanged` signal handler.
2022-03-11 16:23:55 -05:00
Tyler Goodlet 692e310a98 Use a `DisplayState` in the graphics update loop
The graphics update loop is much easier to grok when all the UI
components which potentially need to be updated on a cycle are arranged
together in a high-level composite namespace, thus this new
`DisplayState` addition. Create and set this state on each
`LinkedSplits` chart set and add a new method `.graphics_cycle()` which
let's a caller trigger a graphics loop update manually. Use this method
in the fsp graphics manager such that a chain can update new history
output even if there is no real-time feed driving the display loop (eg.
when a market is "closed").
2022-03-11 16:23:28 -05:00
Tyler Goodlet c60d523428 Support an array field map to `ShmArray.push()`, start index 3days in 2022-03-11 16:23:19 -05:00
Tyler Goodlet 00d7bb089f Convert `iter_ohlc_periods()` to a `@tractor.context` 2022-03-11 16:23:00 -05:00
Tyler Goodlet 3dc87e0426 Facepalm**2: only update on special "update" msg 2022-03-11 16:22:49 -05:00
Tyler Goodlet 49531a2da6 Facepalm: display state must be linked charts specific 2022-03-11 16:22:39 -05:00
Tyler Goodlet 53641abc4b Add detailed `.addItem()`` comment 2022-03-11 16:22:24 -05:00
Tyler Goodlet b0e236fadf Manually trigger graphics loops updates on msgs from the fsp chain 2022-03-11 16:21:06 -05:00
Tyler Goodlet ef0516a84b Always fire a "step/update message" on every fsp history update 2022-03-11 16:20:55 -05:00
Tyler Goodlet bcd0895a12 Factor sync part of graphics update into func, add `trigger_update()`` 2022-03-11 16:20:42 -05:00
Tyler Goodlet 81c69c54ec Add guard for real-time-not-active last line is `None` case 2022-03-11 16:13:13 -05:00
Tyler Goodlet 01f5f2d015 Don't require a rt quote, increase client connect timeout 2022-03-03 17:49:21 -05:00
Tyler Goodlet af3d624281 Just give up on discretized pp bar for now 2022-03-03 17:15:55 -05:00
Tyler Goodlet 2c9612ebd8 Force exact pp bar size 2022-03-03 10:46:30 -05:00
Tyler Goodlet 16b9e39e11 Dis-allow an allocator limit less then the current pp size 2022-03-02 10:05:33 -05:00
Tyler Goodlet 6889a25926 Drop pp bar clipping, hopefully fix slot sizing 2022-03-02 10:05:33 -05:00
Tyler Goodlet f7d03489d8 Drop `marketstore` loading cruft (will come later) 2022-03-01 12:39:12 -05:00
Tyler Goodlet 09079b61fc Comment task canceller method prototype 2022-03-01 12:37:31 -05:00
Tyler Goodlet adccb687fe Fix `piker services` cmd 2022-03-01 12:36:32 -05:00
Tyler Goodlet c239faf4e5 Add a `._sampling.sampler` registry composite type 2022-03-01 12:36:32 -05:00
Tyler Goodlet 6f3d78b729 Handle "no data" case in ranger calcs and avoid crashes 2022-02-28 08:30:44 -05:00
Tyler Goodlet 3e7d4f8717 Detect and request sample period in fsp engine 2022-02-28 08:30:32 -05:00
Tyler Goodlet b1cce8f9cf Adjust and add notes for python-trio/trio#2258 2022-02-28 08:30:22 -05:00
Tyler Goodlet 89a98c4aa2 Fix portal result `await`, comment some unused code 2022-02-28 08:30:15 -05:00
Tyler Goodlet 7a943f0e1e Always transmit index event even when no shm is registered 2022-02-28 08:29:56 -05:00
Tyler Goodlet 786ffde4e6 Use 3.9+ annots 2022-02-28 08:27:59 -05:00
Tyler Goodlet 11d4ebd0b5 Just warn on double-remove of a sub 2022-02-28 08:27:37 -05:00
Tyler Goodlet 81f8b4e145 Don't zero clearing rates on sample steps 2022-02-28 08:26:48 -05:00
Tyler Goodlet cc55e1f4bb Drop task-driven sample step graphics updates
Since moving to a "god loop" for graphics, we don't really need to have
a dedicated task for updating graphics on new sample increments. The
only UX difference will be that curves won't be updated until an actual new
rt-quote-event triggers the graphics loop -> so we'll have the chart
"jump" to a new position and new curve segments generated only when new
data arrives. This is imo fine since it's just less "idle" updates
where the chart would sit printing the same (last) value every step.
Instead only update the view increment if a new index is detected by
reading shm.

If we ever want this dedicated task update again this commit can be
easily reverted B)
2022-02-28 08:26:26 -05:00
Tyler Goodlet 412c9ee6cf Support view increment with a steps size 2022-02-28 08:26:20 -05:00
Tyler Goodlet bf3b58e861 Async load data history, allow "offline" feed use
Break up real-time quote feed and history loading into 2 separate tasks
and deliver a client side `data.Feed` as soon as history is loaded
(instead of waiting for a rt quote - the previous logic). If
a symbol doesn't have history then likely the feed shouldn't be loaded
(since presumably client code will need at least "some" datums history
to do anything) and waiting on a real-time quote is dumb, since it'll
hang if the market isn't open XD. If a symbol doesn't have history we
can always write a zero/null array when we run into that case. This also
greatly speeds up feed loading when both history and quotes are available.

TL;DR summary:
- add a `_Feedsbus.start_task()` one-cancel-scope-per-task method for
  assisting with (re-)starting and stopping long running persistent
  feeds (basically a "one cancels one" style nursery API).
- add a `manage_history()` task which does all history loading (and
  eventually real-time writing) which has an independent signal and
  start it in a separate task.
- drop the "sample rate per symbol" stuff since client code doesn't really
  care when it can just inspect shm indexing/time-steps itself.
- run throttle tasks in the bus nursery thus avoiding cancelling the
  underlying sampler task on feed client disconnects.
- don't store a repeated ref the bus nursery's cancel scope..
2022-02-28 08:26:13 -05:00
Tyler Goodlet 1d3ed6c333 Add `mk_` prefix since assignments will use `fqsn` 2022-02-28 08:23:57 -05:00
Tyler Goodlet 832e4c97d2 Drop shm: ShmArray` to `stream_quotes()` endpoint 2022-02-28 08:23:16 -05:00
Tyler Goodlet 23aa7eb31c Stick time step in window header 2022-02-28 08:22:47 -05:00
Tyler Goodlet c2a13c474c Support no realtime stream sending with feed bus 2022-02-28 08:22:40 -05:00
Tyler Goodlet 7252094f90 Add `open_piker_runtime()` to setup actor runtime correctly from non-daemons 2022-02-28 08:16:30 -05:00
Tyler Goodlet b1dd24d1f7 Only throttle warn on rate >= display rate 2022-02-28 08:15:39 -05:00
Tyler Goodlet a073039b30 Drop dependence on `msgpack` and `msgpack_numpy` 2022-02-28 08:15:18 -05:00
Tyler Goodlet 5c343aa748 Misc curve doc strings 2022-02-28 08:14:11 -05:00
Tyler Goodlet 7b13124dd4 Keep clear loop price pedantically up to date
To avoid the "trigger finger" issue (darks execing before they should
due to a stale last price state, normally when generating a trigger
predicate..) always iterate the loop and update the last known book
price even when no execs/triggered orders are registered.
2022-02-11 10:30:30 -05:00
Tyler Goodlet ca1c1cf415 Annoying doc strings 2022-02-11 10:30:30 -05:00
Tyler Goodlet 92c63988bc Bleh, just fill the available window space 2022-02-11 10:07:43 -05:00
Tyler Goodlet 9ed153bcb6 Less gap below results view 2022-02-11 08:45:57 -05:00
Tyler Goodlet 412c34eba0 Drop width check logic; only do height 2022-02-11 08:32:28 -05:00
Tyler Goodlet 68e1db27f8 Drop old null window size 2022-02-10 14:35:28 -05:00
Tyler Goodlet 86b1316691 Handle no-rows-yet case 2022-02-10 14:35:11 -05:00
Tyler Goodlet 890ffc76cf Dynamically re-size the search results view 2022-02-10 14:22:46 -05:00
Tyler Goodlet 51d94a301a Support resize event relaying from the god widget 2022-02-10 14:21:17 -05:00
Tyler Goodlet c54c9ae3d3 Add doc string to DE sizing method 2022-02-10 14:20:15 -05:00
Tyler Goodlet 5a4c155798 Add detailed comment around DE scaling 2022-02-10 13:04:13 -05:00
wattygetlood a5ad24f770 Additionally apply DPI scaling to font size if detected 2022-02-10 10:26:52 -05:00
Tyler Goodlet a0034e2948 If the DE (like windohz) already scales DPI, just use that scale for font size 2022-02-10 10:26:52 -05:00
wattygetlood fc3c0741b8 Set isn't serializable on std msgpack 2022-02-10 10:26:52 -05:00
wattygetlood cc87508fd9 Only load 4 ib requests worth of bars on windows... 2022-02-10 10:26:52 -05:00
wattygetlood d069481f1d Hack search view on windows to 1/2 window height; needs a better solution 2022-02-10 10:26:52 -05:00
wattygetlood c411a244f6 Size the window to aproximately 1/3 the screen space 2022-02-10 10:26:52 -05:00
wattygetlood 15556e40f0 No support for notifications (yet) on windows 2022-02-10 10:26:52 -05:00
wattygetlood c0082e15bc Fix default `brokers.toml` copying since module move 2022-02-10 10:26:52 -05:00
wattygetlood 2ebdf008da Configure window size based on screen dims on windows 2022-02-10 10:26:52 -05:00
Tyler Goodlet 71f9b5c000 Don't enable curve coord cache unless in step mode
You can get a weird "last line segment" artifact if *only* that segment
is drawn and the cache is enabled, so just disable unless in step mode
at startup and re-flash as normal when new path data is appended. Add
a `.disable_cache()` method for the multi-use in the update method. Use
line style on the `._last_line: QLineF` segment as well.
2022-02-10 08:12:15 -05:00
Tyler Goodlet 228f21d7b0 Zero trade rates each step 2022-02-09 22:16:33 -05:00
Tyler Goodlet 45464a5465 Drop graphics throttle to 22Hz, add a `.maxmin` to our view box 2022-02-09 22:15:57 -05:00
Tyler Goodlet 723eef3fd6 🤦 assign `Flow` *after* type check... 2022-02-09 16:00:10 -05:00
Tyler Goodlet e0462f0a8c Type and formatting fixes 2022-02-08 15:57:32 -05:00
Tyler Goodlet 1c49f7f47f Tweak dash pattern to be less sparse 2022-02-08 15:57:02 -05:00
Tyler Goodlet ef04781a2b Expect new flow type through display and fsp UI code 2022-02-08 15:56:20 -05:00
Tyler Goodlet e3a3fd2d39 Add a `Flow` compound type for coupling graphics with backing data-streams 2022-02-08 15:52:50 -05:00
Tyler Goodlet 860ed99757 Drop dvlm "rates" curves from flows chart 2022-02-08 12:05:56 -05:00
Tyler Goodlet 326b2c089a Drop dvlm 'rates' (they're just means), add default params, period -> 6 2022-02-08 12:04:01 -05:00
Tyler Goodlet 8f467bf4f0 Factor batch curve plotting into helper func 2022-02-08 08:21:08 -05:00
Tyler Goodlet 4a7b2d835b Yield 0 initial values from `flow_rates` fsp 2022-02-08 07:46:36 -05:00
Tyler Goodlet 30cf54480d Add more appropriate default params 2022-02-07 13:59:26 -05:00
Tyler Goodlet ee4ad32d3b Fix `dvlm` to actually yield trade count, add instantaneous support 2022-02-07 12:53:30 -05:00
Tyler Goodlet e7516447df Better rate axis title? 2022-02-07 12:53:30 -05:00
Tyler Goodlet a006b87546 Exit `.maxmin()` early on non-yet-registered array lookup 2022-02-07 12:53:30 -05:00
Tyler Goodlet 9490129a74 Add overlays to end of layout grid (aka append) by default 2022-02-07 12:53:30 -05:00
Tyler Goodlet 2f2aef28dd Adjust x-axis label from summed left axes widths 2022-02-07 12:53:30 -05:00
Tyler Goodlet 0271841412 Add `PlotItemOverlay.get_axes()`
Enables retrieving all "named axes" on a particular "side" of the
overlayed plot items. This is useful for calculating how much space
needs to be allocated for the axes before the view box area starts.
2022-02-07 12:53:30 -05:00
Tyler Goodlet e8d7709358 Drop notification display time to piker seconds worth 2022-02-07 12:53:30 -05:00
Tyler Goodlet e3c46a5d4d Add draft, commented tickbytick for ib 2022-02-07 12:53:30 -05:00
Tyler Goodlet 8d432e1988 Shorter clear rate axis title 2022-02-07 12:53:30 -05:00
Tyler Goodlet 87653ddca2 Simplify to only needed one LHS axis for clearing rates 2022-02-07 12:53:30 -05:00
Tyler Goodlet 73faafcfc1 Add trade "rates" (i.e. trade counts) support B)
Though it's not per-tick accurate, accumulate the number of "trades"
(i.e. the "clearing rate" - maybe this is a better name?) per bar
inside the `dolla_vlm` fsp and average and report wmas of this in the
`flow_rates` fsp.
2022-02-07 12:53:30 -05:00
Tyler Goodlet e4244e96a9 Fix var name typo 2022-02-07 12:53:30 -05:00
Tyler Goodlet 5274eb538c Add 16 period dollar vlm rates, drop ib rates for now 2022-02-07 12:53:30 -05:00
Tyler Goodlet b358b8e874 Move `wma` fsp earlier in module 2022-02-07 12:53:30 -05:00
Tyler Goodlet 2d3c685e19 Typecast np dtype description to a tuple 2022-02-07 12:53:30 -05:00
Tyler Goodlet 4570b06c26 Handle no y-range maxmin output (seems like bug?) 2022-02-07 12:53:30 -05:00
Tyler Goodlet 26b0071471 Subscribe for rate calcs from IB in default tick set 2022-02-07 12:53:30 -05:00
Tyler Goodlet 1fc6429f75 Prep for manual rate calcs, handle non-ib backends XD 2022-02-07 12:53:30 -05:00
Tyler Goodlet ebf3e00438 Add `Fsp._flow_registry` as actor-local table
Define the flows table as a class var (thus making it a "global" and/or
actor-local state) which can be accessed by any in process task. Add
`Fsp.get_shm()` to allow accessing output streams by source-token + fsp
routine reference and thus providing inter-fsp low level access to
real-time flows.
2022-02-07 12:53:30 -05:00
Tyler Goodlet df6afe24a4 Define a flow registry on `FspAdmin`, use it to update fsp engine clusters 2022-02-07 12:53:30 -05:00
Tyler Goodlet d130f0449f Expect registry of fsp "flows" to each engine task
In order for fsp routines to be able to look up other "flows" in the
cascade, we need a small registry-table which gives access to a map of
a source stream + an fsp -> an output stream. Eventually we'll also
likely want a dependency (injection) mechanism so that any fsp demanded
can either be dynamically allocated or at the least waited upon before
a consumer tries to access it.
2022-02-07 12:53:30 -05:00
Tyler Goodlet efb743fd85 Flip to using `pydantic` for shm tokens 2022-02-07 12:53:30 -05:00
Tyler Goodlet 615bf3a55a Use solid line for vlm rate and dashed for trades rate 2022-02-07 12:53:30 -05:00
Tyler Goodlet d4f79a6245 Comment flow rates fsp prints 2022-02-07 12:53:30 -05:00
Tyler Goodlet 4b7d1fb35b Add line style via `str` style name to our fast curve 2022-02-07 12:53:30 -05:00
Tyler Goodlet 0b5250d5e3 Plot the vlm rate (per min) taken verbatim from ib 2022-02-07 12:53:30 -05:00
Tyler Goodlet 97c2f86092 TOSQUASH, fix separate vlm vs trade rate 2022-02-07 12:53:30 -05:00
Tyler Goodlet f3289c1977 Create source length zeroed arrays from yielded `None` fsp history 2022-02-07 12:53:30 -05:00
Tyler Goodlet 4e96dd09e3 Add a `.text_color` property to our axes types 2022-02-07 12:53:30 -05:00
Tyler Goodlet b81209e78e Ensure `sym` arg is a `str` 2022-02-07 12:53:30 -05:00
Tyler Goodlet dfe4473c9a Yield history `dict`s, add a `flow_rates` fsp 2022-02-07 12:53:30 -05:00
Tyler Goodlet 1aae40cdeb Expect multi-output fsps to yield a `dict` of history arrays 2022-02-07 12:53:30 -05:00
Tyler Goodlet 8118a57b9a Guard against no time field in some provider quotes 2022-02-07 12:53:30 -05:00
Tyler Goodlet 5952e7f538 Add dark vlm deduplication support via flag 2022-02-07 12:53:30 -05:00
Tyler Goodlet 16c04e11e4 Comment out nan-price assert previously for `ib` in ems clear tasks 2022-02-07 09:49:45 -05:00
Tyler Goodlet 9bfad86c29 Drop timeout-cancel block 2022-02-07 09:49:45 -05:00
Tyler Goodlet a9d42b374f ib: Allow history backfilling even when markets are closed 2022-02-07 09:49:45 -05:00
goodboy 43b39d3b6b
Merge pull request #275 from pikers/py3.10_support
Py3.10 support
2022-02-07 09:48:54 -05:00
Tyler Goodlet 00a90e7390 Change dpi log msg back to debug 2022-02-07 09:36:07 -05:00
Tyler Goodlet 1aaa382036 Avoid null index race-error during startup 2022-02-07 09:36:07 -05:00
Tyler Goodlet 999d3efdd7 Another `int` required 2022-02-07 09:36:07 -05:00
Tyler Goodlet f63a7c497d Latest `PyQt5` expects `ints` for widget sizings 2022-02-07 09:36:07 -05:00
Guillermo Rodriguez 82d1b85b09
Add default for multiplier var 2022-02-02 20:46:45 -03:00
Tyler Goodlet bb8fade16f Update `msgpub` import from `tractor.experimental` 2022-01-30 12:46:54 -05:00
Tyler Goodlet 296863348d Update imports to `tractor.msg.NamespacePath` 2022-01-30 12:46:20 -05:00
Tyler Goodlet 95b31cbc0f Drop references to deprecated `tractor.msg.pub` 2022-01-29 12:44:45 -05:00
Tyler Goodlet 6a0fba1eb3 Support maxmin over multiple arrays; Keep dark vlm in view 2022-01-28 11:45:47 -05:00
Tyler Goodlet 06934be047 Overlay dark $volume B) 2022-01-28 08:46:24 -05:00
Tyler Goodlet 28b5be0719 Accumulate dark vlm ticks independently per sample step 2022-01-28 08:46:04 -05:00
Tyler Goodlet 67de8f24b9 Init history output with `np.zeros()` 2022-01-28 08:45:28 -05:00
Tyler Goodlet b112f24e7e Drop old commented cruft, use `Fsp.name` 2022-01-28 07:51:13 -05:00
Tyler Goodlet bd2460846e Decorate momo routines 2022-01-28 07:43:49 -05:00
Tyler Goodlet be93ded0e5 Log fsp registy entries in `cascade` startup 2022-01-28 07:43:23 -05:00
Tyler Goodlet 9d9929fb89 Drop old `wrapt` cruft, add `Fsp.name` 2022-01-28 07:18:14 -05:00
Tyler Goodlet cc5390376c Use `Fsp` abstration layer through engine and UI
Instead of referencing the remote processing funcs by a `str` name start
embracing the new `@fsp`/`Fsp` API such that wrapped processing
functions are first class APIs.

Summary of the changeset:
- move and load the fsp built-in set in the new `.fsp._api` module
- handle processors ("fsps") which want to yield multiple keyed-values
  (interleaved in time) by expecting both history that is keyed and
  assigned to the appropriate struct-array field, *and* real-time
  `yield`ed value in tuples of the form `tuple[str, float]` such that
  any one (async) processing function can deliver multiple outputs from
  the same base calculation.
- drop `maybe_mk_fsp_shm()` from UI module
- expect and manage `Fsp` instances (`@fsp` decorated funcs) throughout
  the UI code, particularly the `FspAdmin` layer.
2022-01-27 18:57:16 -05:00
Tyler Goodlet 72f4474273 Use new decorator on volume fsp routines 2022-01-27 09:08:03 -05:00