Commit Graph

3084 Commits (7ef6219d0190d2a15d1a2b9d2a6a6fbd922fae3d)

Author SHA1 Message Date
Tyler Goodlet 7ef6219d01 Delegate to `Viz.default_view()` on chart
Also add a rage print to not forget about the global index
tracking/diffing in the display loop we still need to change.
2023-01-10 12:42:25 -05:00
Tyler Goodlet 5976d68bb2 Re-implement `.default_view()` on `Viz`
Since we don't really need it defined on the "chart widget" move it to
a viz method and rework it to hell:

- always discard the invalid view l > r case.
- use the graphic's UPPX to determine UI-to-scene coordinate scaling for
  the L1-label collision detection, if there is no L1 just offset by
  a few (index step scaled) datums; this allows us to drop the 2x
  x-range calls as was hacked previous.
- handle no-data-in-view cases explicitly and error if we get any
  ostensibly impossible cases.
- expect caller to trigger a graphics cycle if needed.

Further support this includes a rework a slew of other important
details:

- add `Viz.index_step`, an idempotent computed, index (presumably uniform)
  step value which is needed for variable sample rate graphics displayed
  on an epoch (second) time index.
- rework `Viz.datums_range()` to pass view x-endpoints as first and last
  elements in return `tuple`; tighten up snap-to-data edge case logic
  using `max()`/`min()` calls and better internal var naming.
- adjust all calls to `slice_from_time()` to not expect an "abs" slice.
- drop all `.yrange` resetting since we can just have the `Renderer` do
  it when necessary.
2023-01-10 12:42:25 -05:00
Tyler Goodlet 3a0cbe518e Add gap detection for `stop_t`, though only report atm 2023-01-10 12:42:25 -05:00
Tyler Goodlet aaa1bccd60 Add `.x_last()` meth to flow graphics 2023-01-10 12:42:25 -05:00
Tyler Goodlet 14bcba367e Drop `Flume.view_data()` 2023-01-10 12:42:25 -05:00
Tyler Goodlet 1b95668309 Drop old breakpoint 2023-01-10 12:42:25 -05:00
Tyler Goodlet 688d7d7f2f Drop `_slice_from_time()` 2023-01-10 12:42:25 -05:00
Tyler Goodlet 5c417fe815 Use uniform step arithmetic in `slice_from_time()`
If we presume that time indexing using a uniform step we can calculate
the exact index (using `//`) for the input time presuming the data
set has zero gaps. This gives a massive speedup over `numpy` fancy
indexing and (naive) `numba` iteration. Further in the case where time
gaps are detected, we can use `numpy.searchsorted()` to binary search
for the nearest expected index at lower latency.

Deatz,
- comment-disable the call to the naive `numba` scan impl.
- add a optional `step: int` input (calced if not provided).
- add todos for caching binary search results in the gap detection
  cases.
- drop returning the "absolute buffer indexing" slice since the caller
  can always just use the read-relative slice to acquire it.
2023-01-10 12:42:25 -05:00
Tyler Goodlet 94e0f48f39 Make `.default_view()` time step aware
When we use an epoch index and any sample rate > 1s we need to scale the
"number of bars" to that step in order to place the view correctly in
x-domain terms. For now we're calcing the step in-method but likely,
longer run, we'll pull this from elsewhere (like a ``Viz`` attr).
2023-01-10 12:42:25 -05:00
Tyler Goodlet aa404ab18b Flip over to epoch-time based x-domain indexing 2023-01-10 12:42:25 -05:00
Tyler Goodlet 3e62832580 Adjust all `slice_from_time()` calls to not expect mask 2023-01-10 12:42:25 -05:00
Tyler Goodlet 352bd4a1f7 Rewrite `slice_from_time()` using `numba`
Gives approx a 3-4x speedup using plain old iterate-with-for-loop style
though still not really happy with this .5 to 1 ms latency..

Move the core `@njit` part to a `_slice_from_time()` with a pure python
func with orig name around it. Also, drop the output `mask` array since
we can generally just use the slices in the caller to accomplish the
same input array slicing, duh..
2023-01-10 12:42:25 -05:00
Tyler Goodlet f18247b855 Use index (time) step to calc OHLC bar/line uppx threshold 2023-01-10 12:42:25 -05:00
Tyler Goodlet 92a71293ac Use step size to determine bar gaps 2023-01-10 12:42:25 -05:00
Tyler Goodlet 6829daa79c Use step size to determine last datum bar gap 2023-01-10 12:42:25 -05:00
Tyler Goodlet cd58bfb8cf Move `Flume.slice_from_time()` to `.data._pathops` mod func 2023-01-10 12:42:25 -05:00
Tyler Goodlet 49ea4e1ef6 Drop `index_field` input to renders, add `.read()` profiling 2023-01-10 12:42:25 -05:00
Tyler Goodlet d8f325ddd9 Delegate formatter `.index_field` to the parent `Viz` 2023-01-10 12:42:25 -05:00
Tyler Goodlet 2e6f14afb3 Facepalm**2: fix array-read-slice, like actually..
We need to subtract the first index in the array segment read, not the
first index value in the time-sliced output, to get the correct offset
into the non-absolute (`ShmArray.array` read) array..

Further we **do** need the `&` between the advance indexing conditions
and this adds profiling to see that it is indeed real slow (like 20ms
ish even when using `np.where()`).
2023-01-10 12:42:25 -05:00
Tyler Goodlet a2f75a83b6 TOSQUASH 4eb5fe0dd96 (FSP copy time from src -> dst)
Slice up to history's length worth of (latest) time stamps from source
series read at the start of the history init phase.
2023-01-10 12:42:25 -05:00
Tyler Goodlet faecd6f0e0 Markup OHLC->path gen with `numba` issue # 2023-01-10 12:42:25 -05:00
Tyler Goodlet 69d4fe9fef Facepalm: put graphics cycle in `do_ds: bool` block.. 2023-01-10 12:42:25 -05:00
Tyler Goodlet 6ce9872530 TOSQUASH: 552a8c298cd (return index for arrow..) 2023-01-10 12:42:25 -05:00
Tyler Goodlet c5a352bc64 Facepalm: actually return latest index on time slice fail.. 2023-01-10 12:42:25 -05:00
Tyler Goodlet 787fa53aa9 Go with explicit `.data._m4` mod name
Since it's a notable and self-contained graphics compression algo, might
as well give it a dedicated module B)
2023-01-10 12:42:25 -05:00
Tyler Goodlet 8de8a40a1e Move (unused) path gen routines to `.ui._pathops` 2023-01-10 12:42:25 -05:00
Tyler Goodlet a2d23244e7 Move qpath-ops routines back to separate mod 2023-01-10 12:42:25 -05:00
Tyler Goodlet 4ca8e23b5b Rename `.ui._pathops.py` -> `.ui._formatters.py 2023-01-10 12:42:25 -05:00
Tyler Goodlet 95ee69c119 Look up "index field" in display cycles
Again, to make epoch indexing a flip-of-switch for testing look up the
`Viz.index_field: str` value when updating labels.

Also, drops the legacy tick-type set tracking which we no longer use
thanks to the new throttler subsys and it's framing msgs.
2023-01-10 12:42:25 -05:00
Tyler Goodlet cab75217dd Fix from-time index slicing?
Apparently we want an `|` for the advanced indexing logic?
Also, fix `read_slc` start to not always be 0 XD
2023-01-10 12:42:25 -05:00
Tyler Goodlet 59766f53cf Move old label sizing cruft to label mod 2023-01-10 12:42:25 -05:00
Tyler Goodlet 3098d12221 Move path ops routines to top of mod
Planning to put the formatters into a new mod and aggregate all path
gen/op helpers into this module.

Further tweak include:
- moving `path_arrays_from_ohlc()` back to module level
- slice out the last xy datum for `OHLCBarsAsCurveFmtr` 1d formatting
- always copy the new x-value from the source to `.x_nd`
2023-01-10 12:42:25 -05:00
Tyler Goodlet b1ad1f2af1 Drop diff state tracking in formatter
This was a major cause of error (particularly trying to get epoch
indexing working) and really isn't necessary; instead just have
`.diff()` always read from the underlying source array for current
index-step diffing and append/prepend slice construction.

Allows us to,
- drop `._last_read` state management and thus usage.
- better handle startup indexing by setting `.xy_nd_start/stop` to
  `None` initially so that the first update can be done in one large
  prepend.
- better understand and document the step curve "slice back to previous
  level" logic which is now heavily commented B)
- drop all the `slice_to_head` stuff from and instead allow each
  formatter to choose it's 1d segmenting.
2023-01-10 12:42:25 -05:00
Tyler Goodlet 1bee6e3150 Explicitly enable chart widget yranging in display init 2023-01-10 12:42:25 -05:00
Tyler Goodlet ede2edc85c Enable/disable vlm chart yranging (TO SQUASH) 2023-01-10 12:42:25 -05:00
Tyler Goodlet 27b2daa448 Don't disable non-enabled vlm chart y-autoranging 2023-01-10 12:42:25 -05:00
Tyler Goodlet 620152d783 Comment out bps for time indexing 2023-01-10 12:42:25 -05:00
Tyler Goodlet a13b7aab7c Call `Viz.bars_range()` from display loop 2023-01-10 12:42:25 -05:00
Tyler Goodlet 5cff7a7193 TOSQUASH: f5dcf1dc (viz index field) 2023-01-10 12:42:25 -05:00
Tyler Goodlet 83b3cac807 Fix `.default_view()` to view-left-of-data 2023-01-10 12:42:25 -05:00
Tyler Goodlet 166f97c8af Add `Viz.index_field: str`, pass to graphics objs
In an effort to make it easy to override the indexing scheme.

Further, this repairs the `.datums_range()` special case to handle when
the view box is to-the-right-of the data set (i.e. l > datum_start).
2023-01-10 12:42:25 -05:00
Tyler Goodlet 0751f51cfa Expect `index_field: str` in all graphics objects 2023-01-10 12:42:25 -05:00
Tyler Goodlet 3096b206d9 TOSQUASH: 2dc706aa (.default_view w time) 2023-01-10 12:42:25 -05:00
Tyler Goodlet 16d5ea5b33 Frame ticks in helper routine
Wow, turns out tick framing was totally borked since we weren't framing
on "greater then throttle period long waits" XD

This moves all the framing logic into a common func and calls it in
every case:
- every (normal) "pre throttle period expires" quote receive
- each "no new quote before throttle period expires" (slow case)
- each "no clearing tick yet received" / only burst on clears case
2023-01-10 12:42:25 -05:00
Tyler Goodlet ac0166f936 Facepalm: pass correct flume to each FSP chart group.. 2023-01-10 12:42:25 -05:00
Tyler Goodlet 925849b5e4 Attempt to make `.default_view()` time-index ready
As in make the call to `Flume.slice_from_time()` to try and convert any
time index values from the view range to array-indices; all untested
atm.

Also drop some old/unused/moved methods:
- `._set_xlimits()`
- `.bars_range()`
- `.curve_width_pxs()`

and fix some `flow` -> `viz` var naming.
2023-01-10 12:42:25 -05:00
Tyler Goodlet bd2abcb91f Simplify formatter update methodology
Don't expect values (array + slice) to be returned and applied by
`.incr_update_xy_nd()` and instead presume this will implemented
internally in each (sub)formatter.

Attempt to simplify some incr-update routines, (particularly in the step
curve formatter, though most of it was reverted to just a simpler form
of the original implementation XD) including:
- dropping the need for the `slice_to_head: int` control.
- using the `xy_nd_start/stop` index counters over custom lookups.
2023-01-10 12:42:25 -05:00
Tyler Goodlet 97feb195e6 TOSQUASH: f3d757c2 (flow->viz) 2023-01-10 12:42:25 -05:00
Tyler Goodlet c084a1122a First attempt, field-index agnostic formatting
Remove harcoded `'index'` field refs from all formatters in a first
attempt at moving towards epoch-time alignment (though don't actually
use it it yet).

Adjustments to the formatter interface:
- property for `.xy_nd` the x/y nd arrays.
- property for and `.xy_slice` the nd format array(s) start->stop index
  slice.

Internal routine tweaks:
- drop `read_src_from_key` and always pass full source array on updates
  and adjust handlers to expect to have to index the data field of
  interest.
- set `.last_read` right after update calls instead of after 1d
  conversion.
- drop `slice_to_head` array read slicing.
- add some debug points for testing 'time' indexing (though not used
  here yet).
- add `.x_nd` array update logic for when the `.index_field` is not
  'index' - i.e. when we begin to try and support epoch time.
- simplify some new y_nd updates to not require use of `np.broadcast()`
  where possible.
2023-01-10 12:42:25 -05:00
Tyler Goodlet 1b9f6a7152 Pepper render routines with time-slice calls 2023-01-10 12:42:25 -05:00