Commit Graph

3634 Commits (335e8d10d4bf83e2f812e07d0f1e05bc622cc8c0)

Author SHA1 Message Date
Tyler Goodlet 94f0ef13ef Repair x-label datetime labels when in array-index mode 2023-03-10 18:20:22 -05:00
Tyler Goodlet 7579863068 Skip overlay handling when `N < 2` are detected 2023-03-10 18:20:22 -05:00
Tyler Goodlet 993bb47138 Drop passing overlay method from viewbox to view-mode handler 2023-03-10 18:20:22 -05:00
Tyler Goodlet 8c392fda60 Drop a bunch of commented/uneeded cruft 2023-03-10 18:20:22 -05:00
Tyler Goodlet 45e97dd4c8 Solve a final minor-should-rescale edge case
When the minor has the same scaling as the major in a given direction we
should still do back-scaling against the major-target and previous
minors to avoid strange edge cases where only the target-major might not
be shifted correctly to show an matched intersect point? More or less
this just meant making the y-mxmn checks interval-inclusive with
`>=`/`<=` operators.

Also adds a shite ton of detailed comments throughout the pin-to-target
method blocks and moves the final major y-range call outside the final
`scaled: dict` loop.
2023-03-10 18:20:22 -05:00
Tyler Goodlet 01ea706644 Better doc string, use `Viz.vs: ViewState` 2023-03-10 18:20:22 -05:00
Tyler Goodlet c1ea8552ac Back-rescale previous (minor) curves from latest
For the "pin to target major/target curve" overlay method, this finally
solves the longstanding issue of ensuring that any new minor curve,
which requires and increase in the major/target curve y-range, also
re-scales all previously scaled minor curves retroactively. Thus we now
guarantee that all minor curves are correctly "pinned" to their
target/major on their earliest available datum **and** are all kept in
view.
2023-03-10 18:20:22 -05:00
Tyler Goodlet 6601dea8cc Support "pin-to-target-curve" overlay method again
Yah yah, i know it's the same as before (the N > 2 curves case with
out-of-range-minor rescaling the previously scaled curves isn't fixed
yet...) but, this is a much better and optional implementation in less
code. Further we're now better leveraging various new cached properties
and methods on `Viz`.

We now handle different `overlay_technique: str` options using `match:`
syntax in the 2ndary scaling loop, stash the returns scalars per curve
in `overlay_table`, and store and iterate the curves by dispersion
measure sort order.

Further wrt "pin-to-target-curve" mode, which currently still pins to
the largest measured dispersion curve in the overlay set:

- pop major Ci overlay table entries at start for sub-calcs usage when
  handling the "minor requires major rescale after pin" case.
- (finally) correctly rescale the major curve y-mxmn to whatever the
  latest minor overlay curve by calcing the inverse transform from the
  minor *at that point*:
  - the intersect point being that which the minor has starts support on
    the major's x-domain* using the new `Viz.scalars_from_index()` and,
  - checking that the minor is not out of range (versus what the major's
    transform calcs it to be, in which case,
  - calc the inverse transform from the current out-of-range minor and
    use it to project the new y-mxmn for the major/target based on the
    same intersect-reference point in the x-domain used by the minor.
  - always handle the target-major Ci specially by only setting the
    `mx_ymn` / `mx_ymn` value when iterating that entry in the overlay
    table.
  - add todos around also doing the last sub-sub bullet for all previously
    major-transform scaled minor overlays (this is coming next..i hope).
- add a final 3rd overlay loop which goes through a final `scaled: dict`
  to apply all range values to each view; this is where we will
  eventually solve that last edge case of an out-of-range minor's
  scaling needing to be used to rescale already scaled minors XD
2023-03-10 18:20:22 -05:00
Tyler Goodlet 4d11c5c89c Add cached dispersion methods to `Viz`
In an effort to make overlay calcs cleaner and leverage caching of view
range -> dispersion measures, this adds the following new methods:

- `._dispersion()` an lru cached returns scalar calculator given input
  y-range and y-ref values.
- `.disp_from_range()` which calls the above method and returns variable
  output depending on requested calc `method: str`.
- `.i_from_t()` a currently unused cached method for slicing the
  in-view's array index from time stamp (though not working yet due to
  needing to parameterize the cache by the input `.vs.xrange`).

Further refinements/adjustments:
- rename `.view_state: ViewState` -> `.vs`.
- drop the `.bars_range()` method as it's no longer used anywhere else
  in the code base.
- always set the `ViewState.in_view: np.ndarray` inside `.read()`.
- return the start array index (from slice) and `yref` value @ `xref`
  from `.scalars_from_index()` to aid with "pin to curve" rescaling
  caused by out-of-range pinned-minor curves.
2023-03-10 18:20:22 -05:00
Tyler Goodlet 29418e9655 Avoid index-from-time slicing including gaps
Not sure why this was ever allowed but, for slicing to the sample
*before* whatever target time stamp is passed in we should definitely
not return the prior index as for the slice start since that might
include a very large gap prior to whatever sample is scanned to have
the earliest matching time stamp.

This was essential to fixing overlay intersect points searching in our
``ui.view_mode`` machinery..
2023-03-10 18:20:22 -05:00
Tyler Goodlet 8fd5c67f2a Drop last lingering usage of `Viz.bars_range()` 2023-03-10 18:20:22 -05:00
Tyler Goodlet 62e0889bf5 Add `Viz.view_state: ViewState`
Adds a small struct which is used to track the most recently viewed
data's x/y ranges as well as the last `Viz.read()` "in view" array data
for fast access by chart related graphics processing code, namely view
mode overlay handling.

Also adds new `Viz` interfaces:
- `Viz.ds_yrange: tuple[float, float]' which replaces the previous
  `.yrange` (now set by `.datums_range()` on manual y-range calcs) so
  that the m4 downsampler can set this field specifically and then it
  get used (when available) by `Viz.maxmin()`.
- `Viz.scalars_from_index()` a new returns-scalar generator which can be
  used to calc the up and down returns values (used for scaling overlay
  y-ranges) from an input `xref` x-domain index which maps to some
  `Ci(xref) = yref`.
2023-03-10 18:20:22 -05:00
Tyler Goodlet 26690b061b Make slow chart a teensie bit smaller 2023-03-10 18:20:22 -05:00
Tyler Goodlet 98b7d78476 Drop (now) unused major curve mx/mn variables 2023-03-10 18:20:22 -05:00
Tyler Goodlet 35c40e825a Move overlay transform logic to new `.ui.view_mode`
It was getting waayy to long to be jammed in a method XD

This moves all the chart-viz iteration and transform logic into a new
`piker.ui.view_mode.overlay_viewlists()` core routine which will make it
a lot nicer for,

- AOT compilation via `numba` / `cython` / `mypyc`.
- decoupling from the `pyqtgraph.ViewBox` APIs if we ever decide to get
  crazy and go without another graphics engine.
- keeping your head clear when trying to rework the code B)
2023-03-10 18:20:22 -05:00
Tyler Goodlet 753e991dae Adjust `.ui` modules to new set-style "optional" annots 2023-03-10 18:20:22 -05:00
Tyler Goodlet 54ecb0990f Remove vlm chart again, drop lotsa fsp cruft 2023-03-10 18:20:22 -05:00
Tyler Goodlet 5f470d6122 Rework overlay pin technique: "align to first"
As part of solving a final bullet-issue in #455, which is specifically
a case:
- with N > 2 curves, one of which is the "major" dispersion curve" and
  the others are "minors",
- we can run into a scenario where some minor curve which gets pinned to
  the major (due to the original "pinning technique" -> "align to
  major") at some `P(t)` which is *not* the major's minimum / maximum
  due to the minor having a smaller/shorter support and thus,
- requires that in order to show then max/min on the minor curve we have
  to expand the range of the major curve as well but,
- that also means any previously scaled (to the major) minor curves need
  to be adjusted as well or they'll not be pinned to the major the same
  way!

I originally was trying to avoid doing the recursive iteration back
through all previously scaled minor curves and instead decided to try
implementing the "per side" curve dispersion detection (as was
originally attempted when first starting this work). The idea is to
decide which curve's up or down "swing in % returns" would determine the
global y-range *on that side*. Turns out I stumbled on the "align to
first" technique in the process: "for each overlay curve we align its
earliest sample (in time) to the same level of the earliest such sample
for whatever is deemed the major (directionally disperse) curve in
view".

I decided (with help) that this "pin to first" approach/style is equally
as useful and maybe often more so when wanting to view support-disjoint
time series:

- instead of compressing the y-range on "longer series which have lesser
  sigma" to make whatever "shorter but larger-sigma series" pin to it at
  an intersect time step, this instead will expand the price ranges
  based on the earliest time step in each series.
- the output global-returns-overlay-range for any N-set of series is equal to
  the same in the previous "pin to intersect time" technique.
- the only time this technique seems less useful is for overlaying
  market feeds which have the same destination asset but different
  source assets (eg. btceur and btcusd on the same chart since if one
  of the series is shorter it will always be aligned to the earliest
  datum on the longer instead of more naturally to the intersect sample
  level as was in the previous approach).

As such I'm going to keep this technique as discovered and will later
add back optional support for the "align to intersect" approach from
previous (which will again require detecting the highest dispersion
curve direction-agnostic) and pin all minors to the price level at which
they start on the major.

Further details of the implementation rework in
`.interact_graphics_cycle()` include:

- add `intersect_from_longer()` to detect and deliver a common datum
  from 2 series which are different in length: the first time-index
  sample in the longer.
- Rewrite the drafted `OverlayT` to only compute (inversed log-returns)
  transforms for a single direction and use 2 instances, one for each
  direction inside the `Viz`-overlay iteration loop.
- do all dispersion-per-side major curve detection in the first pass of
  all `Viz`s on a plot, instead updating the `OverlayT` instances for
  each side and compensating for any length mismatch and
  rescale-to-minor cases in each loop cycle.
2023-03-10 18:20:22 -05:00
Tyler Goodlet d5ba26cfaf Try to hide all axes even when removed 2023-03-10 18:20:22 -05:00
Tyler Goodlet cb5e2d48e2 Add hack-zone UI REPL access via `ctl-u` 2023-03-10 18:20:22 -05:00
Tyler Goodlet a6d1053c50 Facepalm, align overlay plot view exactly to parent
Previously we were aligning the child's `PlotItem` to the "root" (top
most) overlays `ViewBox`..smh. This is why there was a weird gap on the
LHS next to the 'left' price axes: something weird in the implied axes
offsets was getting jammed in that rect.

Also comments out "the-skipping-of" moving axes from the overlay's
`PlotItem.layout` to the root's linear layout(s) when an overlay's axis
is read as not visible; this isn't really necessary nor useful and if we
want to remove the axes entirely we should do it explicitly and/or
provide a way through the `ComposeGridLayout` API.
2023-03-10 18:20:22 -05:00
Tyler Goodlet 3dc1f66ff6 Go back to caching on all curves
Despite there being artifacts when interacting, the speedups when
cross-hair-ing are just too good to ignore. We can always play with
disabling caches when interaction takes place much like we do with feed
pausing.
2023-03-10 18:20:22 -05:00
Tyler Goodlet 091afccb72 Dynamically adjust y-range margin in display loop
When zoomed in alot, and thus a quote driven y-range resize takes place,
it makes more sense to increase the `range_margin: float` input to
`._set_yrange()` to ensure all L1 labels stay in view; generally the
more zoomed in,
- the smaller the y-range is and thus the larger the needed margin (on
  that range's dispersion diff) would be,
- it's more likely to get a last datum move outside the previous range.

Also, always do overlayT style scaling on the slow chart whenever it
treads.
2023-03-10 18:20:22 -05:00
Tyler Goodlet cda3bcc1f6 Expose `._set_yrange()` kwargs via `yrange_kwargs: dict`
Since it can be desirable to dynamically adjust inputs to the y-ranging
method (such as in the display loop when a chart is very zoomed in), this
adds such support through a new `yrange_kwargs: dict[Viz, dict]` which
replaces the `yrange` tuple we were passing through prior. Also, adjusts
the y-range margin back to the original 0.09 of the diff now that we can
support dynamic control.
2023-03-10 18:20:22 -05:00
Tyler Goodlet 2d7359851f Go back to no-cache on OHLC downsample line 2023-03-10 18:20:22 -05:00
Tyler Goodlet db1e0a04f8 Only use last `ChartView._yrange` if set 2023-03-10 18:20:22 -05:00
Tyler Goodlet 972b723a5d Skip overlay transform calcs on common-pi curves
If there is a common `PlotItem` used for a set of `Viz`/curves (on
a given view) we don't need to do overlay scaling and thus can also
short circuit the viz iteration loop early.
2023-03-10 18:20:22 -05:00
Tyler Goodlet 74c215d5b2 Lel, always meant to no-cache the step curve.. 2023-03-10 18:20:22 -05:00
Tyler Goodlet c646b435bf Incrementally set vlm chart yrange per quote 2023-03-10 18:20:22 -05:00
Tyler Goodlet 0a939311fe Only set the specific view's yrange per quote
Somewhat of a facepalm but, for incremental update of the auto-yrange
from quotes in the display loop obviously we only want to update the
associated `Viz`/viewbox for *that* fqsn. Further we don't need to worry
about the whole "tick margin" stuff since `._set_yrange()` already adds
margin to the yrange by default; thus we remove all of that.
2023-03-10 18:20:22 -05:00
Tyler Goodlet a7db6adc2e Always set the `ChartView._viz` for each plot 2023-03-10 18:20:22 -05:00
Tyler Goodlet c57567ab0d No-overlays, y-ranging optimizations
When the caller passes `do_overlay_scaling=False` we skip the given
chart's `Viz` iteration loop, and set the yrange immediately, then
continue to the next chart (if `do_linked_charts` is set) instead of
a `continue` short circuit within the viz sub-loop.

Deats:
- add a `_maybe_calc_yrange()` helper which makes the `yranges`
  provided-or-not case logic more terse (factored).
- add a `do_linked_charts=False` short circuit.
- drop the legacy commented swing % calcs stuff.
- use the `ChartView._viz` when `do_overlay_scaling=False` thus
  presuming that we want to handle the viz mapped to *this* view box.
- add a `._yrange` "last set yrange" tracking var which keeps record of
  the last ymn/ymx value set in `._set_yrange()` BEFORE doing range
  margins; this will be used for incremental update in the display loop.
2023-03-10 18:20:22 -05:00
Tyler Goodlet 3daee0caa9 Disable overlay scaling on per-symbol-feed updates
Since each symbol's feed is multiplexed by quote key (an fqsn), we can
avoid scaling overlay curves on any single update, presuming each quote
driven cycle will trigger **only** the specific symbol's curve.

Also disables fsp `.interact_graphics_cycle()` calls for now since it
seems they aren't really that critical to and we should be using the
same technique as above (doing incremental y-range checks/updates) for
FSPs as well.
2023-03-10 18:20:22 -05:00
Tyler Goodlet 6ea64a7d2e Iterate all charts (widgets) when only one overlay
The reason (fsp) subcharts were not linked-updating correctly was
because of the early termination of the interact update loop when only
one "overlay" (aka no other overlays then the main curve) is detected.
Obviously in this case we still need to iterate all linked charts in the
set (presuming the user doesn't disable this).

Also tweaks a few internals:
- rename `start_datums: dict` -> `overlay_table`.
- compact all "single curve" checks to one logic block.
- don't collect curve info into the `overlay_table: dict` when
  `do_overlay_scaling=True`.
2023-03-10 18:20:22 -05:00
Tyler Goodlet 25cf8df367 Pass windowed y-mxmn to `.interact_graphics_cycle()` calls in display loop 2023-03-10 18:20:22 -05:00
Tyler Goodlet 91d41ebf76 Allow y-range input via a `yranges: dict[Viz, tuple[float, float]]` 2023-03-10 18:20:22 -05:00
Tyler Goodlet c690e141e1 Don't unset `Viz.render` for unit vlm
Such that we still y-range auto-sort inside
`ChartView.interact_graphics_cycle()` still runs on the unit vlm axis
and we always size such that the y-label stays in view.
2023-03-10 18:20:22 -05:00
Tyler Goodlet 2ed43c0758 Fix profiler f-string 2023-03-10 18:20:22 -05:00
Tyler Goodlet 7a83a7288c Update profile msgs to new apis 2023-03-10 18:20:22 -05:00
Tyler Goodlet 9930f25ad3 Move axis hiding into `.overlay_plotitem()`
Since we pretty much always want the 'bottom' and any side that is not
declared by the caller move the axis hides into this method. Lets us
drop the same calls in `.ui._fsp` and `._display`.

This also disables the auto-ranging back-linking for now since it
doesn't seem to be working quite yet?
2023-03-10 18:20:22 -05:00
Tyler Goodlet 5dd69b2295 Better handle dynamic registry sampler broadcasts
In situations where clients are (dynamically) subscribing *while*
broadcasts are starting to taking place we need to handle the
`set`-modified-during-iteration case. This scenario seems to be more
common during races on concurrent startup of multiple symbols. The
solution here is to use another set to take note of subscribers which
are successfully sent-to and then skipping them on re-try.

This also contains an attempt to exception-handle throttled stream
overruns caused by higher frequency feeds (like binance) pushing more
quotes then can be handled during (UI) client startup.
2023-03-10 18:20:22 -05:00
Tyler Goodlet 246d07021e Drop old loop and wait on fsp engine tasks startups 2023-03-10 18:20:22 -05:00
Tyler Goodlet 7ebcd6d734 Comment out all median usage, turns out it's uneeded.. 2023-03-10 18:20:22 -05:00
Tyler Goodlet 5a8fd42c0c Lul, actually scaled main chart from linked set
This was a subtle logic error when building the `plots: dict` we weren't
adding the "main (ohlc or other source) chart" from the `LinkedSplits`
set when interacting with some sub-chart from `.subplots`..

Further this tries out bypassing `numpy.median()` altogether by just
using `median = (ymx - ymn) / 2` which should be nearly the same?
2023-03-10 18:20:22 -05:00
Tyler Goodlet 517c68f3ad Use `._pathops.slice_from_time()` for overlay intersects
It's way faster since it uses a uniform time arithmetic to narrow the
`numpy.searchsorted()` range before actually doing the index search B)
2023-03-10 18:20:22 -05:00
Tyler Goodlet ea84505682 Don't scale overlays on linked from display loop
In the (incrementally updated) display loop we have range logic that is
incrementally updated in real-time by streams, as such we don't really
need to update all linked chart's (for any given, currently updated
chart) y-ranges on calls of each separate (sub-)chart's
`ChartView.interact_graphics_cycle()`. In practise there are plenty of
cases where resizing in one chart (say the vlm fsps sub-plot) requires
a y-range re-calc but not in the OHLC price chart. Therefore
we always avoid doing more resizing then necessary despite it resulting
in potentially more method call overhead (which will later be justified
by better leveraging incrementally updated `Viz.maxmin()` and
`media_from_range()` calcs).
2023-03-10 18:20:22 -05:00
Tyler Goodlet 5eaca18ee0 Don't skip overlay scaling in disp-loop for now 2023-03-10 18:20:22 -05:00
Tyler Goodlet e06d4b405d Add linked charts guard-flag for use in display loop 2023-03-10 18:20:22 -05:00
Tyler Goodlet cf67c790e5 Use new cached median method in overlay scaling
Massively speeds up scaling transform cycles (duh).

Also includes a draft for an "overlay transform" type/api; obviously
still a WIP 🏄..
2023-03-10 18:20:22 -05:00
Tyler Goodlet ec8679ad74 Add `Viz.median_from_range()`
A super snappy `numpy.median()` calculator (per input range) which we
slap an `lru_cache` on thanks to handy dunder method hacks for such
things on mutable types XD
2023-03-10 18:20:22 -05:00