pikers
/
piker
4
1
Fork 0
Code Issues 10 Pull Requests 14 Packages Projects Releases Wiki Activity
piker/piker/ui/_render.py

343 lines
10 KiB
Python
Raw Normal View History

Add new `ui._flows` module This begins the removal of data processing / analysis methods from the chart widget and instead moving them to our new `Flow` API (in the new module introduce here) and delegating the old chart methods to the respective internal flow. Most importantly is no longer storing the "last read" of an array from shm in an internal chart table (was `._arrays`) and instead the `ShmArray` instance is passed as input and stored in the `Flow` instance. This greatly simplifies lookup logic such that the display loop now doesn't have to worry about reading shm, it can be done by internal graphics logic as desired. Generally speaking, all previous `._arrays`/`._graphics` lookups are now delegated to the entries in the chart's `._flows` table. The new `Flow` methods are generally better factored and provide more detailed output regarding data-stream <-> graphics inter-relations for the future purpose of allowing much more efficient update calls in the display loop as well as supporting low latency interaction UX. The concept here is that we're introducing an intermediary layer that ties together graphics and real-time data flows such that widget code is oriented around plot layout and the flow apis are oriented around real-time low latency updates and providing an efficient high level metric layer for the UX. The summary api transition is something like: - `update_graphics_from_array()` -> `.update_graphics_from_flow()` - `.bars_range()` -> `Flow.datums_range()` - `.bars_range()` -> `Flow.datums_range()`
2022-04-14 13:38:25 +00:00
# piker: trading gear for hackers
# Copyright (C) Tyler Goodlet (in stewardship for pikers)
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU Affero General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU Affero General Public License for more details.
# You should have received a copy of the GNU Affero General Public License
# along with this program. If not, see <https://www.gnu.org/licenses/>.
'''
High level streaming graphics primitives.
This is an intermediate layer which associates real-time low latency
Move `Viz` layer to new `.ui` mod
2022-12-14 17:05:35 +00:00
graphics primitives with underlying stream/flow related data structures
for fast incremental update.
Add new `ui._flows` module This begins the removal of data processing / analysis methods from the chart widget and instead moving them to our new `Flow` API (in the new module introduce here) and delegating the old chart methods to the respective internal flow. Most importantly is no longer storing the "last read" of an array from shm in an internal chart table (was `._arrays`) and instead the `ShmArray` instance is passed as input and stored in the `Flow` instance. This greatly simplifies lookup logic such that the display loop now doesn't have to worry about reading shm, it can be done by internal graphics logic as desired. Generally speaking, all previous `._arrays`/`._graphics` lookups are now delegated to the entries in the chart's `._flows` table. The new `Flow` methods are generally better factored and provide more detailed output regarding data-stream <-> graphics inter-relations for the future purpose of allowing much more efficient update calls in the display loop as well as supporting low latency interaction UX. The concept here is that we're introducing an intermediary layer that ties together graphics and real-time data flows such that widget code is oriented around plot layout and the flow apis are oriented around real-time low latency updates and providing an efficient high level metric layer for the UX. The summary api transition is something like: - `update_graphics_from_array()` -> `.update_graphics_from_flow()` - `.bars_range()` -> `Flow.datums_range()` - `.bars_range()` -> `Flow.datums_range()`
2022-04-14 13:38:25 +00:00
'''
WIP incremental render apis
2022-04-16 19:22:11 +00:00
from __future__ import annotations
Add new `ui._flows` module This begins the removal of data processing / analysis methods from the chart widget and instead moving them to our new `Flow` API (in the new module introduce here) and delegating the old chart methods to the respective internal flow. Most importantly is no longer storing the "last read" of an array from shm in an internal chart table (was `._arrays`) and instead the `ShmArray` instance is passed as input and stored in the `Flow` instance. This greatly simplifies lookup logic such that the display loop now doesn't have to worry about reading shm, it can be done by internal graphics logic as desired. Generally speaking, all previous `._arrays`/`._graphics` lookups are now delegated to the entries in the chart's `._flows` table. The new `Flow` methods are generally better factored and provide more detailed output regarding data-stream <-> graphics inter-relations for the future purpose of allowing much more efficient update calls in the display loop as well as supporting low latency interaction UX. The concept here is that we're introducing an intermediary layer that ties together graphics and real-time data flows such that widget code is oriented around plot layout and the flow apis are oriented around real-time low latency updates and providing an efficient high level metric layer for the UX. The summary api transition is something like: - `update_graphics_from_array()` -> `.update_graphics_from_flow()` - `.bars_range()` -> `Flow.datums_range()` - `.bars_range()` -> `Flow.datums_range()`
2022-04-14 13:38:25 +00:00
from typing import (
Re-implement `.default_view()` on `Viz` Since we don't really need it defined on the "chart widget" move it to a viz method and rework it to hell: - always discard the invalid view l > r case. - use the graphic's UPPX to determine UI-to-scene coordinate scaling for the L1-label collision detection, if there is no L1 just offset by a few (index step scaled) datums; this allows us to drop the 2x x-range calls as was hacked previous. - handle no-data-in-view cases explicitly and error if we get any ostensibly impossible cases. - expect caller to trigger a graphics cycle if needed. Further support this includes a rework a slew of other important details: - add `Viz.index_step`, an idempotent computed, index (presumably uniform) step value which is needed for variable sample rate graphics displayed on an epoch (second) time index. - rework `Viz.datums_range()` to pass view x-endpoints as first and last elements in return `tuple`; tighten up snap-to-data edge case logic using `max()`/`min()` calls and better internal var naming. - adjust all calls to `slice_from_time()` to not expect an "abs" slice. - drop all `.yrange` resetting since we can just have the `Renderer` do it when necessary.
2022-12-07 21:31:32 +00:00
TYPE_CHECKING,
Add new `ui._flows` module This begins the removal of data processing / analysis methods from the chart widget and instead moving them to our new `Flow` API (in the new module introduce here) and delegating the old chart methods to the respective internal flow. Most importantly is no longer storing the "last read" of an array from shm in an internal chart table (was `._arrays`) and instead the `ShmArray` instance is passed as input and stored in the `Flow` instance. This greatly simplifies lookup logic such that the display loop now doesn't have to worry about reading shm, it can be done by internal graphics logic as desired. Generally speaking, all previous `._arrays`/`._graphics` lookups are now delegated to the entries in the chart's `._flows` table. The new `Flow` methods are generally better factored and provide more detailed output regarding data-stream <-> graphics inter-relations for the future purpose of allowing much more efficient update calls in the display loop as well as supporting low latency interaction UX. The concept here is that we're introducing an intermediary layer that ties together graphics and real-time data flows such that widget code is oriented around plot layout and the flow apis are oriented around real-time low latency updates and providing an efficient high level metric layer for the UX. The summary api transition is something like: - `update_graphics_from_array()` -> `.update_graphics_from_flow()` - `.bars_range()` -> `Flow.datums_range()` - `.bars_range()` -> `Flow.datums_range()`
2022-04-14 13:38:25 +00:00
)
import msgspec
import numpy as np
import pyqtgraph as pg
from PyQt5.QtGui import QPainterPath
Rename `.ui._pathops.py` -> `.ui._formatters.py
2022-11-30 23:48:04 +00:00
from ..data._formatters import (
Add graphics incr-updated "formatter" subsys After trying to hack epoch indexed time series and failing miserably, decided to properly factor out all formatting routines into a common subsystem API: ``IncrementalFormatter`` which provides the interface for incrementally updating and tracking pre-path-graphics formatted data. Previously this functionality was mangled into our `Renderer` (which also does the work of `QPath` generation and update) but splitting it out also preps for being able to do graphics-buffer downsampling and caching on a remote host B) The ``IncrementalFormatter`` (parent type) has the default behaviour of tracking a single field-array on some source `ShmArray`, updating a flattened `numpy.ndarray` in-mem allocation, and providing a default 1d conversion for pre-downsampling and path generation. Changed out of `Renderer`, - `.allocate_xy()`, `update_xy()` and `format_xy()` all are moved to more explicitly named formatter methods. - all `.x/y_data` nd array management and update - "last view range" tracking - `.last_read`, `.diff()` - now calls `IncrementalFormatter.format_to_1d()` inside `.render()` The new API gets, - `.diff()`, `.last_read` - all view range diff tracking through `.track_inview_range()`. - better nd format array names: `.x/y_nd`, `xy_nd_start/stop`. - `.format_to_1d()` which renders pre-path formatted arrays ready for both m4 sampling and path gen. - better explicit overloadable formatting method names: * `.allocate_xy()` -> `.allocate_xy_nd()` * `.update_xy()` -> `.incr_update_xy_nd()` * `.format_xy()` -> `.format_xy_nd_to_1d()` Finally this implements per-graphics-type formatters which define each set up related formatting routines: - `OHLCBarsFmtr`: std multi-line style bars - `OHLCBarsAsCurveFmtr`: draws an interpolated line for ohlc sampled data - `StepCurveFmtr`: handles vlm style curves
2022-11-22 22:28:58 +00:00
IncrementalFormatter,
Use new profiler arg name, add more marks throughout flow update
2022-05-13 20:16:17 +00:00
)
Move `Flume.slice_from_time()` to `.data._pathops` mod func
2022-12-03 01:13:17 +00:00
from ..data._pathops import (
xy_downsample,
WIP incremental render apis
2022-04-16 19:22:11 +00:00
)
More WIP, implement `BarItems` rendering in `Flow.update_graphics()`
2022-04-20 15:43:47 +00:00
from ..log import get_logger
Start `piker.toolz` subpkg for all our tooling B) Since there's a growing list of top level mods which are more or less utils/tools for working with the runtime; begin to move them into a new subpkg starting with a new `.toolz.debug`. Start with, - a new `open_crash_handller()` for doing breakpoints around blocks that might error. - move in what was `piker._profile` into `.toolz.profile` and adjust all importing appropriately.
2023-07-20 19:23:01 +00:00
from ..toolz import (
Drop `index_field` input to renders, add `.read()` profiling
2022-12-03 01:05:49 +00:00
Profiler,
)
More WIP, implement `BarItems` rendering in `Flow.update_graphics()`
2022-04-20 15:43:47 +00:00
Re-implement `.default_view()` on `Viz` Since we don't really need it defined on the "chart widget" move it to a viz method and rework it to hell: - always discard the invalid view l > r case. - use the graphic's UPPX to determine UI-to-scene coordinate scaling for the L1-label collision detection, if there is no L1 just offset by a few (index step scaled) datums; this allows us to drop the 2x x-range calls as was hacked previous. - handle no-data-in-view cases explicitly and error if we get any ostensibly impossible cases. - expect caller to trigger a graphics cycle if needed. Further support this includes a rework a slew of other important details: - add `Viz.index_step`, an idempotent computed, index (presumably uniform) step value which is needed for variable sample rate graphics displayed on an epoch (second) time index. - rework `Viz.datums_range()` to pass view x-endpoints as first and last elements in return `tuple`; tighten up snap-to-data edge case logic using `max()`/`min()` calls and better internal var naming. - adjust all calls to `slice_from_time()` to not expect an "abs" slice. - drop all `.yrange` resetting since we can just have the `Renderer` do it when necessary.
2022-12-07 21:31:32 +00:00
if TYPE_CHECKING:
Move `Viz` layer to new `.ui` mod
2022-12-14 17:05:35 +00:00
from ._dataviz import Viz
Re-implement `.default_view()` on `Viz` Since we don't really need it defined on the "chart widget" move it to a viz method and rework it to hell: - always discard the invalid view l > r case. - use the graphic's UPPX to determine UI-to-scene coordinate scaling for the L1-label collision detection, if there is no L1 just offset by a few (index step scaled) datums; this allows us to drop the 2x x-range calls as was hacked previous. - handle no-data-in-view cases explicitly and error if we get any ostensibly impossible cases. - expect caller to trigger a graphics cycle if needed. Further support this includes a rework a slew of other important details: - add `Viz.index_step`, an idempotent computed, index (presumably uniform) step value which is needed for variable sample rate graphics displayed on an epoch (second) time index. - rework `Viz.datums_range()` to pass view x-endpoints as first and last elements in return `tuple`; tighten up snap-to-data edge case logic using `max()`/`min()` calls and better internal var naming. - adjust all calls to `slice_from_time()` to not expect an "abs" slice. - drop all `.yrange` resetting since we can just have the `Renderer` do it when necessary.
2022-12-07 21:31:32 +00:00
More WIP, implement `BarItems` rendering in `Flow.update_graphics()`
2022-04-20 15:43:47 +00:00
log = get_logger(__name__)
Add new `ui._flows` module This begins the removal of data processing / analysis methods from the chart widget and instead moving them to our new `Flow` API (in the new module introduce here) and delegating the old chart methods to the respective internal flow. Most importantly is no longer storing the "last read" of an array from shm in an internal chart table (was `._arrays`) and instead the `ShmArray` instance is passed as input and stored in the `Flow` instance. This greatly simplifies lookup logic such that the display loop now doesn't have to worry about reading shm, it can be done by internal graphics logic as desired. Generally speaking, all previous `._arrays`/`._graphics` lookups are now delegated to the entries in the chart's `._flows` table. The new `Flow` methods are generally better factored and provide more detailed output regarding data-stream <-> graphics inter-relations for the future purpose of allowing much more efficient update calls in the display loop as well as supporting low latency interaction UX. The concept here is that we're introducing an intermediary layer that ties together graphics and real-time data flows such that widget code is oriented around plot layout and the flow apis are oriented around real-time low latency updates and providing an efficient high level metric layer for the UX. The summary api transition is something like: - `update_graphics_from_array()` -> `.update_graphics_from_flow()` - `.bars_range()` -> `Flow.datums_range()` - `.bars_range()` -> `Flow.datums_range()`
2022-04-14 13:38:25 +00:00
Implement OHLC downsampled curve via renderer, drop old bypass code
2022-05-25 23:45:09 +00:00
Move graphics update logic into `Renderer.render()` Finally this gets us much closer to a generic incremental update system for graphics wherein the input array diffing, pre-graphical format data processing, downsampler activation and incremental update and storage of any of these data flow stages can be managed in one modular sub-system :surfer_boi:. Dirty deatz: - reorg and move all path logic into `Renderer.render()` and have it take in pretty much the same flags as the old `FastAppendCurve.update_from_array()` and instead storing all update state vars (even copies of the downsampler related ones) on the renderer instance: - new state vars: `._last_uppx, ._in_ds, ._vr, ._avr` - `.render()` input bools: `new_sample_rate, should_redraw, should_ds, showing_src_data` - add a hack-around for passing in incremental update data (for now) via a `input_data: tuple` of numpy arrays - a default `uppx: float = 1` - add new render interface attrs: - `.format_xy()` which takes in the source data array and produces out x, y arrays (and maybe a `connect` array) that can be passed to `.draw_path()` (the default for this is just to slice out the index and `array_key: str` columns from the input struct array), - `.draw_path()` which takes in the x, y, connect arrays and generates a `QPainterPath` - `.fast_path`, for "appendable" updates like there was on the fast append curve - move redraw (aka `.clear()` calls) into `.draw_path()` and trigger via `redraw: bool` flag. - our graphics objects no longer set their own `.path` state, it's done by the `Flow.update_graphics()` method using output from `Renderer.render()` (and it's state if necessary)
2022-05-18 19:23:14 +00:00
class Renderer(msgspec.Struct):
Better doc string, use `Viz.vs: ViewState`
2023-02-28 14:43:16 +00:00
'''
Low(er) level interface for converting a source, real-time updated,
data buffer (usually held in a ``ShmArray``) to a graphics data
format usable by `Qt`.
A renderer reads in context-specific source data using a ``Viz``,
formats that data to a 2D-xy pre-graphics format using
a ``IncrementalFormatter``, then renders that data to a set of
output graphics objects normally a ``.ui._curve.FlowGraphics``
sub-type to which the ``Renderer.path`` is applied and further "last
datum" graphics are updated from the source buffer's latest
sample(s).
'''
Rename `Flow` -> `Viz` The type is better described as a "data visualization": https://en.wikipedia.org/wiki/Data_and_information_visualization Add `ChartPlotWidget.get_viz()` to start working towards not accessing the private table directly XD We'll probably end up using the name `Flow` for a type that tracks a collection of composed/cascaded `Flume`s: https://en.wikipedia.org/wiki/Two-port_network#Cascade_connection
2022-11-24 20:33:58 +00:00
viz: Viz
Add graphics incr-updated "formatter" subsys After trying to hack epoch indexed time series and failing miserably, decided to properly factor out all formatting routines into a common subsystem API: ``IncrementalFormatter`` which provides the interface for incrementally updating and tracking pre-path-graphics formatted data. Previously this functionality was mangled into our `Renderer` (which also does the work of `QPath` generation and update) but splitting it out also preps for being able to do graphics-buffer downsampling and caching on a remote host B) The ``IncrementalFormatter`` (parent type) has the default behaviour of tracking a single field-array on some source `ShmArray`, updating a flattened `numpy.ndarray` in-mem allocation, and providing a default 1d conversion for pre-downsampling and path generation. Changed out of `Renderer`, - `.allocate_xy()`, `update_xy()` and `format_xy()` all are moved to more explicitly named formatter methods. - all `.x/y_data` nd array management and update - "last view range" tracking - `.last_read`, `.diff()` - now calls `IncrementalFormatter.format_to_1d()` inside `.render()` The new API gets, - `.diff()`, `.last_read` - all view range diff tracking through `.track_inview_range()`. - better nd format array names: `.x/y_nd`, `xy_nd_start/stop`. - `.format_to_1d()` which renders pre-path formatted arrays ready for both m4 sampling and path gen. - better explicit overloadable formatting method names: * `.allocate_xy()` -> `.allocate_xy_nd()` * `.update_xy()` -> `.incr_update_xy_nd()` * `.format_xy()` -> `.format_xy_nd_to_1d()` Finally this implements per-graphics-type formatters which define each set up related formatting routines: - `OHLCBarsFmtr`: std multi-line style bars - `OHLCBarsAsCurveFmtr`: draws an interpolated line for ohlc sampled data - `StepCurveFmtr`: handles vlm style curves
2022-11-22 22:28:58 +00:00
fmtr: IncrementalFormatter
Implement pre-graphics format incremental update Adds a new pre-graphics data-format callback incremental update api to our `Renderer`. `Renderer` instance can now overload these custom routines: - `.update_xy()` a routine which accepts the latest [pre/a]pended data sliced out from shm and returns it in a format suitable to store in the optional `.[x/y]_data` arrays. - `.allocate_xy()` which initially does the work of pre-allocating the `.[x/y]_data` arrays based on the source shm sizing such that new data can be filled in (to memory). - `._xy_[first/last]: int` attrs to track index diffs between src shm and the xy format data updates. Implement the step curve data format with 3 super simple routines: - `.allocate_xy()` -> `._pathops.to_step_format()` - `.update_xy()` -> `._flows.update_step_xy()` - `.format_xy()` -> `._flows.step_to_xy()` Further, adjust `._pathops.gen_ohlc_qpath()` to adhere to the new call signature.
2022-05-25 15:15:46 +00:00
Move graphics update logic into `Renderer.render()` Finally this gets us much closer to a generic incremental update system for graphics wherein the input array diffing, pre-graphical format data processing, downsampler activation and incremental update and storage of any of these data flow stages can be managed in one modular sub-system :surfer_boi:. Dirty deatz: - reorg and move all path logic into `Renderer.render()` and have it take in pretty much the same flags as the old `FastAppendCurve.update_from_array()` and instead storing all update state vars (even copies of the downsampler related ones) on the renderer instance: - new state vars: `._last_uppx, ._in_ds, ._vr, ._avr` - `.render()` input bools: `new_sample_rate, should_redraw, should_ds, showing_src_data` - add a hack-around for passing in incremental update data (for now) via a `input_data: tuple` of numpy arrays - a default `uppx: float = 1` - add new render interface attrs: - `.format_xy()` which takes in the source data array and produces out x, y arrays (and maybe a `connect` array) that can be passed to `.draw_path()` (the default for this is just to slice out the index and `array_key: str` columns from the input struct array), - `.draw_path()` which takes in the x, y, connect arrays and generates a `QPainterPath` - `.fast_path`, for "appendable" updates like there was on the fast append curve - move redraw (aka `.clear()` calls) into `.draw_path()` and trigger via `redraw: bool` flag. - our graphics objects no longer set their own `.path` state, it's done by the `Flow.update_graphics()` method using output from `Renderer.render()` (and it's state if necessary)
2022-05-18 19:23:14 +00:00
# output graphics rendering, the main object
# processed in ``QGraphicsObject.paint()``
Modernize optional path variable type annots
2022-12-28 07:04:06 +00:00
path: QPainterPath | None = None
fast_path: QPainterPath | None = None
Move graphics update logic into `Renderer.render()` Finally this gets us much closer to a generic incremental update system for graphics wherein the input array diffing, pre-graphical format data processing, downsampler activation and incremental update and storage of any of these data flow stages can be managed in one modular sub-system :surfer_boi:. Dirty deatz: - reorg and move all path logic into `Renderer.render()` and have it take in pretty much the same flags as the old `FastAppendCurve.update_from_array()` and instead storing all update state vars (even copies of the downsampler related ones) on the renderer instance: - new state vars: `._last_uppx, ._in_ds, ._vr, ._avr` - `.render()` input bools: `new_sample_rate, should_redraw, should_ds, showing_src_data` - add a hack-around for passing in incremental update data (for now) via a `input_data: tuple` of numpy arrays - a default `uppx: float = 1` - add new render interface attrs: - `.format_xy()` which takes in the source data array and produces out x, y arrays (and maybe a `connect` array) that can be passed to `.draw_path()` (the default for this is just to slice out the index and `array_key: str` columns from the input struct array), - `.draw_path()` which takes in the x, y, connect arrays and generates a `QPainterPath` - `.fast_path`, for "appendable" updates like there was on the fast append curve - move redraw (aka `.clear()` calls) into `.draw_path()` and trigger via `redraw: bool` flag. - our graphics objects no longer set their own `.path` state, it's done by the `Flow.update_graphics()` method using output from `Renderer.render()` (and it's state if necessary)
2022-05-18 19:23:14 +00:00
# downsampling state
_last_uppx: float = 0
_in_ds: bool = False
def draw_path(
self,
x: np.ndarray,
y: np.ndarray,
Add graphics incr-updated "formatter" subsys After trying to hack epoch indexed time series and failing miserably, decided to properly factor out all formatting routines into a common subsystem API: ``IncrementalFormatter`` which provides the interface for incrementally updating and tracking pre-path-graphics formatted data. Previously this functionality was mangled into our `Renderer` (which also does the work of `QPath` generation and update) but splitting it out also preps for being able to do graphics-buffer downsampling and caching on a remote host B) The ``IncrementalFormatter`` (parent type) has the default behaviour of tracking a single field-array on some source `ShmArray`, updating a flattened `numpy.ndarray` in-mem allocation, and providing a default 1d conversion for pre-downsampling and path generation. Changed out of `Renderer`, - `.allocate_xy()`, `update_xy()` and `format_xy()` all are moved to more explicitly named formatter methods. - all `.x/y_data` nd array management and update - "last view range" tracking - `.last_read`, `.diff()` - now calls `IncrementalFormatter.format_to_1d()` inside `.render()` The new API gets, - `.diff()`, `.last_read` - all view range diff tracking through `.track_inview_range()`. - better nd format array names: `.x/y_nd`, `xy_nd_start/stop`. - `.format_to_1d()` which renders pre-path formatted arrays ready for both m4 sampling and path gen. - better explicit overloadable formatting method names: * `.allocate_xy()` -> `.allocate_xy_nd()` * `.update_xy()` -> `.incr_update_xy_nd()` * `.format_xy()` -> `.format_xy_nd_to_1d()` Finally this implements per-graphics-type formatters which define each set up related formatting routines: - `OHLCBarsFmtr`: std multi-line style bars - `OHLCBarsAsCurveFmtr`: draws an interpolated line for ohlc sampled data - `StepCurveFmtr`: handles vlm style curves
2022-11-22 22:28:58 +00:00
connect: str | np.ndarray = 'all',
Modernize optional path variable type annots
2022-12-28 07:04:06 +00:00
path: QPainterPath | None = None,
Move graphics update logic into `Renderer.render()` Finally this gets us much closer to a generic incremental update system for graphics wherein the input array diffing, pre-graphical format data processing, downsampler activation and incremental update and storage of any of these data flow stages can be managed in one modular sub-system :surfer_boi:. Dirty deatz: - reorg and move all path logic into `Renderer.render()` and have it take in pretty much the same flags as the old `FastAppendCurve.update_from_array()` and instead storing all update state vars (even copies of the downsampler related ones) on the renderer instance: - new state vars: `._last_uppx, ._in_ds, ._vr, ._avr` - `.render()` input bools: `new_sample_rate, should_redraw, should_ds, showing_src_data` - add a hack-around for passing in incremental update data (for now) via a `input_data: tuple` of numpy arrays - a default `uppx: float = 1` - add new render interface attrs: - `.format_xy()` which takes in the source data array and produces out x, y arrays (and maybe a `connect` array) that can be passed to `.draw_path()` (the default for this is just to slice out the index and `array_key: str` columns from the input struct array), - `.draw_path()` which takes in the x, y, connect arrays and generates a `QPainterPath` - `.fast_path`, for "appendable" updates like there was on the fast append curve - move redraw (aka `.clear()` calls) into `.draw_path()` and trigger via `redraw: bool` flag. - our graphics objects no longer set their own `.path` state, it's done by the `Flow.update_graphics()` method using output from `Renderer.render()` (and it's state if necessary)
2022-05-18 19:23:14 +00:00
redraw: bool = False,
) -> QPainterPath:
path_was_none = path is None
if redraw and path:
path.clear()
# TODO: avoid this?
if self.fast_path:
self.fast_path.clear()
path = pg.functions.arrayToQPath(
x,
y,
connect=connect,
finiteCheck=False,
# reserve mem allocs see:
# - https://doc.qt.io/qt-5/qpainterpath.html#reserve
# - https://doc.qt.io/qt-5/qpainterpath.html#capacity
# - https://doc.qt.io/qt-5/qpainterpath.html#clear
Modernize optional path variable type annots
2022-12-28 07:04:06 +00:00
# XXX: right now this is based on ad-hoc checks on a
Move graphics update logic into `Renderer.render()` Finally this gets us much closer to a generic incremental update system for graphics wherein the input array diffing, pre-graphical format data processing, downsampler activation and incremental update and storage of any of these data flow stages can be managed in one modular sub-system :surfer_boi:. Dirty deatz: - reorg and move all path logic into `Renderer.render()` and have it take in pretty much the same flags as the old `FastAppendCurve.update_from_array()` and instead storing all update state vars (even copies of the downsampler related ones) on the renderer instance: - new state vars: `._last_uppx, ._in_ds, ._vr, ._avr` - `.render()` input bools: `new_sample_rate, should_redraw, should_ds, showing_src_data` - add a hack-around for passing in incremental update data (for now) via a `input_data: tuple` of numpy arrays - a default `uppx: float = 1` - add new render interface attrs: - `.format_xy()` which takes in the source data array and produces out x, y arrays (and maybe a `connect` array) that can be passed to `.draw_path()` (the default for this is just to slice out the index and `array_key: str` columns from the input struct array), - `.draw_path()` which takes in the x, y, connect arrays and generates a `QPainterPath` - `.fast_path`, for "appendable" updates like there was on the fast append curve - move redraw (aka `.clear()` calls) into `.draw_path()` and trigger via `redraw: bool` flag. - our graphics objects no longer set their own `.path` state, it's done by the `Flow.update_graphics()` method using output from `Renderer.render()` (and it's state if necessary)
2022-05-18 19:23:14 +00:00
# hidpi 3840x2160 4k monitor but we should optimize for
# the target display(s) on the sys.
# if no_path_yet:
# graphics.path.reserve(int(500e3))
Guard against empty source history slice output
2022-06-01 16:14:15 +00:00
# path=path, # path re-use / reserving
Move graphics update logic into `Renderer.render()` Finally this gets us much closer to a generic incremental update system for graphics wherein the input array diffing, pre-graphical format data processing, downsampler activation and incremental update and storage of any of these data flow stages can be managed in one modular sub-system :surfer_boi:. Dirty deatz: - reorg and move all path logic into `Renderer.render()` and have it take in pretty much the same flags as the old `FastAppendCurve.update_from_array()` and instead storing all update state vars (even copies of the downsampler related ones) on the renderer instance: - new state vars: `._last_uppx, ._in_ds, ._vr, ._avr` - `.render()` input bools: `new_sample_rate, should_redraw, should_ds, showing_src_data` - add a hack-around for passing in incremental update data (for now) via a `input_data: tuple` of numpy arrays - a default `uppx: float = 1` - add new render interface attrs: - `.format_xy()` which takes in the source data array and produces out x, y arrays (and maybe a `connect` array) that can be passed to `.draw_path()` (the default for this is just to slice out the index and `array_key: str` columns from the input struct array), - `.draw_path()` which takes in the x, y, connect arrays and generates a `QPainterPath` - `.fast_path`, for "appendable" updates like there was on the fast append curve - move redraw (aka `.clear()` calls) into `.draw_path()` and trigger via `redraw: bool` flag. - our graphics objects no longer set their own `.path` state, it's done by the `Flow.update_graphics()` method using output from `Renderer.render()` (and it's state if necessary)
2022-05-18 19:23:14 +00:00
)
# avoid mem allocs if possible
if path_was_none:
path.reserve(path.capacity())
return path
def render(
self,
new_read,
array_key: str,
Move profiler imports to internal version
2022-10-31 13:26:36 +00:00
profiler: Profiler,
Move graphics update logic into `Renderer.render()` Finally this gets us much closer to a generic incremental update system for graphics wherein the input array diffing, pre-graphical format data processing, downsampler activation and incremental update and storage of any of these data flow stages can be managed in one modular sub-system :surfer_boi:. Dirty deatz: - reorg and move all path logic into `Renderer.render()` and have it take in pretty much the same flags as the old `FastAppendCurve.update_from_array()` and instead storing all update state vars (even copies of the downsampler related ones) on the renderer instance: - new state vars: `._last_uppx, ._in_ds, ._vr, ._avr` - `.render()` input bools: `new_sample_rate, should_redraw, should_ds, showing_src_data` - add a hack-around for passing in incremental update data (for now) via a `input_data: tuple` of numpy arrays - a default `uppx: float = 1` - add new render interface attrs: - `.format_xy()` which takes in the source data array and produces out x, y arrays (and maybe a `connect` array) that can be passed to `.draw_path()` (the default for this is just to slice out the index and `array_key: str` columns from the input struct array), - `.draw_path()` which takes in the x, y, connect arrays and generates a `QPainterPath` - `.fast_path`, for "appendable" updates like there was on the fast append curve - move redraw (aka `.clear()` calls) into `.draw_path()` and trigger via `redraw: bool` flag. - our graphics objects no longer set their own `.path` state, it's done by the `Flow.update_graphics()` method using output from `Renderer.render()` (and it's state if necessary)
2022-05-18 19:23:14 +00:00
uppx: float = 1,
# redraw and ds flags
Implement OHLC downsampled curve via renderer, drop old bypass code
2022-05-25 23:45:09 +00:00
should_redraw: bool = False,
Move graphics update logic into `Renderer.render()` Finally this gets us much closer to a generic incremental update system for graphics wherein the input array diffing, pre-graphical format data processing, downsampler activation and incremental update and storage of any of these data flow stages can be managed in one modular sub-system :surfer_boi:. Dirty deatz: - reorg and move all path logic into `Renderer.render()` and have it take in pretty much the same flags as the old `FastAppendCurve.update_from_array()` and instead storing all update state vars (even copies of the downsampler related ones) on the renderer instance: - new state vars: `._last_uppx, ._in_ds, ._vr, ._avr` - `.render()` input bools: `new_sample_rate, should_redraw, should_ds, showing_src_data` - add a hack-around for passing in incremental update data (for now) via a `input_data: tuple` of numpy arrays - a default `uppx: float = 1` - add new render interface attrs: - `.format_xy()` which takes in the source data array and produces out x, y arrays (and maybe a `connect` array) that can be passed to `.draw_path()` (the default for this is just to slice out the index and `array_key: str` columns from the input struct array), - `.draw_path()` which takes in the x, y, connect arrays and generates a `QPainterPath` - `.fast_path`, for "appendable" updates like there was on the fast append curve - move redraw (aka `.clear()` calls) into `.draw_path()` and trigger via `redraw: bool` flag. - our graphics objects no longer set their own `.path` state, it's done by the `Flow.update_graphics()` method using output from `Renderer.render()` (and it's state if necessary)
2022-05-18 19:23:14 +00:00
new_sample_rate: bool = False,
should_ds: bool = False,
showing_src_data: bool = True,
do_append: bool = True,
use_fpath: bool = True,
Expose a `force_reformat: bool` up through graphics stack
2023-06-06 21:22:50 +00:00
force_reformat: bool = False,
Move graphics update logic into `Renderer.render()` Finally this gets us much closer to a generic incremental update system for graphics wherein the input array diffing, pre-graphical format data processing, downsampler activation and incremental update and storage of any of these data flow stages can be managed in one modular sub-system :surfer_boi:. Dirty deatz: - reorg and move all path logic into `Renderer.render()` and have it take in pretty much the same flags as the old `FastAppendCurve.update_from_array()` and instead storing all update state vars (even copies of the downsampler related ones) on the renderer instance: - new state vars: `._last_uppx, ._in_ds, ._vr, ._avr` - `.render()` input bools: `new_sample_rate, should_redraw, should_ds, showing_src_data` - add a hack-around for passing in incremental update data (for now) via a `input_data: tuple` of numpy arrays - a default `uppx: float = 1` - add new render interface attrs: - `.format_xy()` which takes in the source data array and produces out x, y arrays (and maybe a `connect` array) that can be passed to `.draw_path()` (the default for this is just to slice out the index and `array_key: str` columns from the input struct array), - `.draw_path()` which takes in the x, y, connect arrays and generates a `QPainterPath` - `.fast_path`, for "appendable" updates like there was on the fast append curve - move redraw (aka `.clear()` calls) into `.draw_path()` and trigger via `redraw: bool` flag. - our graphics objects no longer set their own `.path` state, it's done by the `Flow.update_graphics()` method using output from `Renderer.render()` (and it's state if necessary)
2022-05-18 19:23:14 +00:00
# only render datums "in view" of the ``ChartView``
use_vr: bool = True,
Fix line -> bars on 6x UPPX Read the `Viz.index_step()` directly to avoid always reading 1 on the slow chart; this was completely broken before and resulting in not rendering the bars graphic on the slow chart until at a true uppx of 1 which obviously doesn't work for 60 width bars XD Further cleanups to `._render` module: - drop `array` output from `Renderer.render()`, `read_from_key` input and fix type annot. - drop `should_line`, `changed_to_line` and `render_kwargs` from `render_baritems()` outputs and instead calc `should_redraw` logic inside the func body and return as output.
2022-12-14 14:49:43 +00:00
) -> tuple[QPainterPath, bool]:
Move graphics update logic into `Renderer.render()` Finally this gets us much closer to a generic incremental update system for graphics wherein the input array diffing, pre-graphical format data processing, downsampler activation and incremental update and storage of any of these data flow stages can be managed in one modular sub-system :surfer_boi:. Dirty deatz: - reorg and move all path logic into `Renderer.render()` and have it take in pretty much the same flags as the old `FastAppendCurve.update_from_array()` and instead storing all update state vars (even copies of the downsampler related ones) on the renderer instance: - new state vars: `._last_uppx, ._in_ds, ._vr, ._avr` - `.render()` input bools: `new_sample_rate, should_redraw, should_ds, showing_src_data` - add a hack-around for passing in incremental update data (for now) via a `input_data: tuple` of numpy arrays - a default `uppx: float = 1` - add new render interface attrs: - `.format_xy()` which takes in the source data array and produces out x, y arrays (and maybe a `connect` array) that can be passed to `.draw_path()` (the default for this is just to slice out the index and `array_key: str` columns from the input struct array), - `.draw_path()` which takes in the x, y, connect arrays and generates a `QPainterPath` - `.fast_path`, for "appendable" updates like there was on the fast append curve - move redraw (aka `.clear()` calls) into `.draw_path()` and trigger via `redraw: bool` flag. - our graphics objects no longer set their own `.path` state, it's done by the `Flow.update_graphics()` method using output from `Renderer.render()` (and it's state if necessary)
2022-05-18 19:23:14 +00:00
'''
Render the current graphics path(s)
There are (at least) 3 stages from source data to graphics data:
- a data transform (which can be stored in additional shm)
- a graphics transform which converts discrete basis data to
a `float`-basis view-coords graphics basis. (eg. ``ohlc_flatten()``,
``step_path_arrays_from_1d()``, etc.)
- blah blah blah (from notes)
'''
Rename `Flow` -> `Viz` The type is better described as a "data visualization": https://en.wikipedia.org/wiki/Data_and_information_visualization Add `ChartPlotWidget.get_viz()` to start working towards not accessing the private table directly XD We'll probably end up using the name `Flow` for a type that tracks a collection of composed/cascaded `Flume`s: https://en.wikipedia.org/wiki/Two-port_network#Cascade_connection
2022-11-24 20:33:58 +00:00
# TODO: can the renderer just call ``Viz.read()`` directly?
Move graphics update logic into `Renderer.render()` Finally this gets us much closer to a generic incremental update system for graphics wherein the input array diffing, pre-graphical format data processing, downsampler activation and incremental update and storage of any of these data flow stages can be managed in one modular sub-system :surfer_boi:. Dirty deatz: - reorg and move all path logic into `Renderer.render()` and have it take in pretty much the same flags as the old `FastAppendCurve.update_from_array()` and instead storing all update state vars (even copies of the downsampler related ones) on the renderer instance: - new state vars: `._last_uppx, ._in_ds, ._vr, ._avr` - `.render()` input bools: `new_sample_rate, should_redraw, should_ds, showing_src_data` - add a hack-around for passing in incremental update data (for now) via a `input_data: tuple` of numpy arrays - a default `uppx: float = 1` - add new render interface attrs: - `.format_xy()` which takes in the source data array and produces out x, y arrays (and maybe a `connect` array) that can be passed to `.draw_path()` (the default for this is just to slice out the index and `array_key: str` columns from the input struct array), - `.draw_path()` which takes in the x, y, connect arrays and generates a `QPainterPath` - `.fast_path`, for "appendable" updates like there was on the fast append curve - move redraw (aka `.clear()` calls) into `.draw_path()` and trigger via `redraw: bool` flag. - our graphics objects no longer set their own `.path` state, it's done by the `Flow.update_graphics()` method using output from `Renderer.render()` (and it's state if necessary)
2022-05-18 19:23:14 +00:00
# unpack latest source data read
Add graphics incr-updated "formatter" subsys After trying to hack epoch indexed time series and failing miserably, decided to properly factor out all formatting routines into a common subsystem API: ``IncrementalFormatter`` which provides the interface for incrementally updating and tracking pre-path-graphics formatted data. Previously this functionality was mangled into our `Renderer` (which also does the work of `QPath` generation and update) but splitting it out also preps for being able to do graphics-buffer downsampling and caching on a remote host B) The ``IncrementalFormatter`` (parent type) has the default behaviour of tracking a single field-array on some source `ShmArray`, updating a flattened `numpy.ndarray` in-mem allocation, and providing a default 1d conversion for pre-downsampling and path generation. Changed out of `Renderer`, - `.allocate_xy()`, `update_xy()` and `format_xy()` all are moved to more explicitly named formatter methods. - all `.x/y_data` nd array management and update - "last view range" tracking - `.last_read`, `.diff()` - now calls `IncrementalFormatter.format_to_1d()` inside `.render()` The new API gets, - `.diff()`, `.last_read` - all view range diff tracking through `.track_inview_range()`. - better nd format array names: `.x/y_nd`, `xy_nd_start/stop`. - `.format_to_1d()` which renders pre-path formatted arrays ready for both m4 sampling and path gen. - better explicit overloadable formatting method names: * `.allocate_xy()` -> `.allocate_xy_nd()` * `.update_xy()` -> `.incr_update_xy_nd()` * `.format_xy()` -> `.format_xy_nd_to_1d()` Finally this implements per-graphics-type formatters which define each set up related formatting routines: - `OHLCBarsFmtr`: std multi-line style bars - `OHLCBarsAsCurveFmtr`: draws an interpolated line for ohlc sampled data - `StepCurveFmtr`: handles vlm style curves
2022-11-22 22:28:58 +00:00
fmtr = self.fmtr
Move graphics update logic into `Renderer.render()` Finally this gets us much closer to a generic incremental update system for graphics wherein the input array diffing, pre-graphical format data processing, downsampler activation and incremental update and storage of any of these data flow stages can be managed in one modular sub-system :surfer_boi:. Dirty deatz: - reorg and move all path logic into `Renderer.render()` and have it take in pretty much the same flags as the old `FastAppendCurve.update_from_array()` and instead storing all update state vars (even copies of the downsampler related ones) on the renderer instance: - new state vars: `._last_uppx, ._in_ds, ._vr, ._avr` - `.render()` input bools: `new_sample_rate, should_redraw, should_ds, showing_src_data` - add a hack-around for passing in incremental update data (for now) via a `input_data: tuple` of numpy arrays - a default `uppx: float = 1` - add new render interface attrs: - `.format_xy()` which takes in the source data array and produces out x, y arrays (and maybe a `connect` array) that can be passed to `.draw_path()` (the default for this is just to slice out the index and `array_key: str` columns from the input struct array), - `.draw_path()` which takes in the x, y, connect arrays and generates a `QPainterPath` - `.fast_path`, for "appendable" updates like there was on the fast append curve - move redraw (aka `.clear()` calls) into `.draw_path()` and trigger via `redraw: bool` flag. - our graphics objects no longer set their own `.path` state, it's done by the `Flow.update_graphics()` method using output from `Renderer.render()` (and it's state if necessary)
2022-05-18 19:23:14 +00:00
(
Add graphics incr-updated "formatter" subsys After trying to hack epoch indexed time series and failing miserably, decided to properly factor out all formatting routines into a common subsystem API: ``IncrementalFormatter`` which provides the interface for incrementally updating and tracking pre-path-graphics formatted data. Previously this functionality was mangled into our `Renderer` (which also does the work of `QPath` generation and update) but splitting it out also preps for being able to do graphics-buffer downsampling and caching on a remote host B) The ``IncrementalFormatter`` (parent type) has the default behaviour of tracking a single field-array on some source `ShmArray`, updating a flattened `numpy.ndarray` in-mem allocation, and providing a default 1d conversion for pre-downsampling and path generation. Changed out of `Renderer`, - `.allocate_xy()`, `update_xy()` and `format_xy()` all are moved to more explicitly named formatter methods. - all `.x/y_data` nd array management and update - "last view range" tracking - `.last_read`, `.diff()` - now calls `IncrementalFormatter.format_to_1d()` inside `.render()` The new API gets, - `.diff()`, `.last_read` - all view range diff tracking through `.track_inview_range()`. - better nd format array names: `.x/y_nd`, `xy_nd_start/stop`. - `.format_to_1d()` which renders pre-path formatted arrays ready for both m4 sampling and path gen. - better explicit overloadable formatting method names: * `.allocate_xy()` -> `.allocate_xy_nd()` * `.update_xy()` -> `.incr_update_xy_nd()` * `.format_xy()` -> `.format_xy_nd_to_1d()` Finally this implements per-graphics-type formatters which define each set up related formatting routines: - `OHLCBarsFmtr`: std multi-line style bars - `OHLCBarsAsCurveFmtr`: draws an interpolated line for ohlc sampled data - `StepCurveFmtr`: handles vlm style curves
2022-11-22 22:28:58 +00:00
_,
_,
Move graphics update logic into `Renderer.render()` Finally this gets us much closer to a generic incremental update system for graphics wherein the input array diffing, pre-graphical format data processing, downsampler activation and incremental update and storage of any of these data flow stages can be managed in one modular sub-system :surfer_boi:. Dirty deatz: - reorg and move all path logic into `Renderer.render()` and have it take in pretty much the same flags as the old `FastAppendCurve.update_from_array()` and instead storing all update state vars (even copies of the downsampler related ones) on the renderer instance: - new state vars: `._last_uppx, ._in_ds, ._vr, ._avr` - `.render()` input bools: `new_sample_rate, should_redraw, should_ds, showing_src_data` - add a hack-around for passing in incremental update data (for now) via a `input_data: tuple` of numpy arrays - a default `uppx: float = 1` - add new render interface attrs: - `.format_xy()` which takes in the source data array and produces out x, y arrays (and maybe a `connect` array) that can be passed to `.draw_path()` (the default for this is just to slice out the index and `array_key: str` columns from the input struct array), - `.draw_path()` which takes in the x, y, connect arrays and generates a `QPainterPath` - `.fast_path`, for "appendable" updates like there was on the fast append curve - move redraw (aka `.clear()` calls) into `.draw_path()` and trigger via `redraw: bool` flag. - our graphics objects no longer set their own `.path` state, it's done by the `Flow.update_graphics()` method using output from `Renderer.render()` (and it's state if necessary)
2022-05-18 19:23:14 +00:00
array,
ivl,
ivr,
in_view,
) = new_read
Implement OHLC downsampled curve via renderer, drop old bypass code
2022-05-25 23:45:09 +00:00
# xy-path data transform: convert source data to a format
# able to be passed to a `QPainterPath` rendering routine.
Add graphics incr-updated "formatter" subsys After trying to hack epoch indexed time series and failing miserably, decided to properly factor out all formatting routines into a common subsystem API: ``IncrementalFormatter`` which provides the interface for incrementally updating and tracking pre-path-graphics formatted data. Previously this functionality was mangled into our `Renderer` (which also does the work of `QPath` generation and update) but splitting it out also preps for being able to do graphics-buffer downsampling and caching on a remote host B) The ``IncrementalFormatter`` (parent type) has the default behaviour of tracking a single field-array on some source `ShmArray`, updating a flattened `numpy.ndarray` in-mem allocation, and providing a default 1d conversion for pre-downsampling and path generation. Changed out of `Renderer`, - `.allocate_xy()`, `update_xy()` and `format_xy()` all are moved to more explicitly named formatter methods. - all `.x/y_data` nd array management and update - "last view range" tracking - `.last_read`, `.diff()` - now calls `IncrementalFormatter.format_to_1d()` inside `.render()` The new API gets, - `.diff()`, `.last_read` - all view range diff tracking through `.track_inview_range()`. - better nd format array names: `.x/y_nd`, `xy_nd_start/stop`. - `.format_to_1d()` which renders pre-path formatted arrays ready for both m4 sampling and path gen. - better explicit overloadable formatting method names: * `.allocate_xy()` -> `.allocate_xy_nd()` * `.update_xy()` -> `.incr_update_xy_nd()` * `.format_xy()` -> `.format_xy_nd_to_1d()` Finally this implements per-graphics-type formatters which define each set up related formatting routines: - `OHLCBarsFmtr`: std multi-line style bars - `OHLCBarsAsCurveFmtr`: draws an interpolated line for ohlc sampled data - `StepCurveFmtr`: handles vlm style curves
2022-11-22 22:28:58 +00:00
fmt_out = fmtr.format_to_1d(
new_read,
Implement OHLC downsampled curve via renderer, drop old bypass code
2022-05-25 23:45:09 +00:00
array_key,
Add graphics incr-updated "formatter" subsys After trying to hack epoch indexed time series and failing miserably, decided to properly factor out all formatting routines into a common subsystem API: ``IncrementalFormatter`` which provides the interface for incrementally updating and tracking pre-path-graphics formatted data. Previously this functionality was mangled into our `Renderer` (which also does the work of `QPath` generation and update) but splitting it out also preps for being able to do graphics-buffer downsampling and caching on a remote host B) The ``IncrementalFormatter`` (parent type) has the default behaviour of tracking a single field-array on some source `ShmArray`, updating a flattened `numpy.ndarray` in-mem allocation, and providing a default 1d conversion for pre-downsampling and path generation. Changed out of `Renderer`, - `.allocate_xy()`, `update_xy()` and `format_xy()` all are moved to more explicitly named formatter methods. - all `.x/y_data` nd array management and update - "last view range" tracking - `.last_read`, `.diff()` - now calls `IncrementalFormatter.format_to_1d()` inside `.render()` The new API gets, - `.diff()`, `.last_read` - all view range diff tracking through `.track_inview_range()`. - better nd format array names: `.x/y_nd`, `xy_nd_start/stop`. - `.format_to_1d()` which renders pre-path formatted arrays ready for both m4 sampling and path gen. - better explicit overloadable formatting method names: * `.allocate_xy()` -> `.allocate_xy_nd()` * `.update_xy()` -> `.incr_update_xy_nd()` * `.format_xy()` -> `.format_xy_nd_to_1d()` Finally this implements per-graphics-type formatters which define each set up related formatting routines: - `OHLCBarsFmtr`: std multi-line style bars - `OHLCBarsAsCurveFmtr`: draws an interpolated line for ohlc sampled data - `StepCurveFmtr`: handles vlm style curves
2022-11-22 22:28:58 +00:00
profiler,
slice_to_inview=use_vr,
Expose a `force_reformat: bool` up through graphics stack
2023-06-06 21:22:50 +00:00
force_full_realloc=force_reformat,
Implement OHLC downsampled curve via renderer, drop old bypass code
2022-05-25 23:45:09 +00:00
)
Move graphics update logic into `Renderer.render()` Finally this gets us much closer to a generic incremental update system for graphics wherein the input array diffing, pre-graphical format data processing, downsampler activation and incremental update and storage of any of these data flow stages can be managed in one modular sub-system :surfer_boi:. Dirty deatz: - reorg and move all path logic into `Renderer.render()` and have it take in pretty much the same flags as the old `FastAppendCurve.update_from_array()` and instead storing all update state vars (even copies of the downsampler related ones) on the renderer instance: - new state vars: `._last_uppx, ._in_ds, ._vr, ._avr` - `.render()` input bools: `new_sample_rate, should_redraw, should_ds, showing_src_data` - add a hack-around for passing in incremental update data (for now) via a `input_data: tuple` of numpy arrays - a default `uppx: float = 1` - add new render interface attrs: - `.format_xy()` which takes in the source data array and produces out x, y arrays (and maybe a `connect` array) that can be passed to `.draw_path()` (the default for this is just to slice out the index and `array_key: str` columns from the input struct array), - `.draw_path()` which takes in the x, y, connect arrays and generates a `QPainterPath` - `.fast_path`, for "appendable" updates like there was on the fast append curve - move redraw (aka `.clear()` calls) into `.draw_path()` and trigger via `redraw: bool` flag. - our graphics objects no longer set their own `.path` state, it's done by the `Flow.update_graphics()` method using output from `Renderer.render()` (and it's state if necessary)
2022-05-18 19:23:14 +00:00
Add graphics incr-updated "formatter" subsys After trying to hack epoch indexed time series and failing miserably, decided to properly factor out all formatting routines into a common subsystem API: ``IncrementalFormatter`` which provides the interface for incrementally updating and tracking pre-path-graphics formatted data. Previously this functionality was mangled into our `Renderer` (which also does the work of `QPath` generation and update) but splitting it out also preps for being able to do graphics-buffer downsampling and caching on a remote host B) The ``IncrementalFormatter`` (parent type) has the default behaviour of tracking a single field-array on some source `ShmArray`, updating a flattened `numpy.ndarray` in-mem allocation, and providing a default 1d conversion for pre-downsampling and path generation. Changed out of `Renderer`, - `.allocate_xy()`, `update_xy()` and `format_xy()` all are moved to more explicitly named formatter methods. - all `.x/y_data` nd array management and update - "last view range" tracking - `.last_read`, `.diff()` - now calls `IncrementalFormatter.format_to_1d()` inside `.render()` The new API gets, - `.diff()`, `.last_read` - all view range diff tracking through `.track_inview_range()`. - better nd format array names: `.x/y_nd`, `xy_nd_start/stop`. - `.format_to_1d()` which renders pre-path formatted arrays ready for both m4 sampling and path gen. - better explicit overloadable formatting method names: * `.allocate_xy()` -> `.allocate_xy_nd()` * `.update_xy()` -> `.incr_update_xy_nd()` * `.format_xy()` -> `.format_xy_nd_to_1d()` Finally this implements per-graphics-type formatters which define each set up related formatting routines: - `OHLCBarsFmtr`: std multi-line style bars - `OHLCBarsAsCurveFmtr`: draws an interpolated line for ohlc sampled data - `StepCurveFmtr`: handles vlm style curves
2022-11-22 22:28:58 +00:00
# no history in view case
if not fmt_out:
# XXX: this might be why the profiler only has exits?
return
Move graphics update logic into `Renderer.render()` Finally this gets us much closer to a generic incremental update system for graphics wherein the input array diffing, pre-graphical format data processing, downsampler activation and incremental update and storage of any of these data flow stages can be managed in one modular sub-system :surfer_boi:. Dirty deatz: - reorg and move all path logic into `Renderer.render()` and have it take in pretty much the same flags as the old `FastAppendCurve.update_from_array()` and instead storing all update state vars (even copies of the downsampler related ones) on the renderer instance: - new state vars: `._last_uppx, ._in_ds, ._vr, ._avr` - `.render()` input bools: `new_sample_rate, should_redraw, should_ds, showing_src_data` - add a hack-around for passing in incremental update data (for now) via a `input_data: tuple` of numpy arrays - a default `uppx: float = 1` - add new render interface attrs: - `.format_xy()` which takes in the source data array and produces out x, y arrays (and maybe a `connect` array) that can be passed to `.draw_path()` (the default for this is just to slice out the index and `array_key: str` columns from the input struct array), - `.draw_path()` which takes in the x, y, connect arrays and generates a `QPainterPath` - `.fast_path`, for "appendable" updates like there was on the fast append curve - move redraw (aka `.clear()` calls) into `.draw_path()` and trigger via `redraw: bool` flag. - our graphics objects no longer set their own `.path` state, it's done by the `Flow.update_graphics()` method using output from `Renderer.render()` (and it's state if necessary)
2022-05-18 19:23:14 +00:00
Add graphics incr-updated "formatter" subsys After trying to hack epoch indexed time series and failing miserably, decided to properly factor out all formatting routines into a common subsystem API: ``IncrementalFormatter`` which provides the interface for incrementally updating and tracking pre-path-graphics formatted data. Previously this functionality was mangled into our `Renderer` (which also does the work of `QPath` generation and update) but splitting it out also preps for being able to do graphics-buffer downsampling and caching on a remote host B) The ``IncrementalFormatter`` (parent type) has the default behaviour of tracking a single field-array on some source `ShmArray`, updating a flattened `numpy.ndarray` in-mem allocation, and providing a default 1d conversion for pre-downsampling and path generation. Changed out of `Renderer`, - `.allocate_xy()`, `update_xy()` and `format_xy()` all are moved to more explicitly named formatter methods. - all `.x/y_data` nd array management and update - "last view range" tracking - `.last_read`, `.diff()` - now calls `IncrementalFormatter.format_to_1d()` inside `.render()` The new API gets, - `.diff()`, `.last_read` - all view range diff tracking through `.track_inview_range()`. - better nd format array names: `.x/y_nd`, `xy_nd_start/stop`. - `.format_to_1d()` which renders pre-path formatted arrays ready for both m4 sampling and path gen. - better explicit overloadable formatting method names: * `.allocate_xy()` -> `.allocate_xy_nd()` * `.update_xy()` -> `.incr_update_xy_nd()` * `.format_xy()` -> `.format_xy_nd_to_1d()` Finally this implements per-graphics-type formatters which define each set up related formatting routines: - `OHLCBarsFmtr`: std multi-line style bars - `OHLCBarsAsCurveFmtr`: draws an interpolated line for ohlc sampled data - `StepCurveFmtr`: handles vlm style curves
2022-11-22 22:28:58 +00:00
(
x_1d,
y_1d,
connect,
prepend_length,
append_length,
view_changed,
Drop px-cache-resets, failed try at path appends Comments out the pixel-cache resetting since it doesn't seem we need it any more to avoid draw oddities? For `.fast_path` appends, this nearly got it working except the new path segments are either not being connected correctly (step curve) or not being drawn in full since the history path (plain line). Leaving the attempted code commented in for a retry in the future; my best guesses are that maybe, - `.connectPath()` call is being done with incorrect segment length and/or start point. - the "appended" data: `appended = array[-append_len-1:slice_to_head]` (done inside the formatter) isn't correct (i.e. endpoint handling considering a path append) and needs special handling for different curve types?
2022-11-23 19:17:18 +00:00
# append_tres,
Add graphics incr-updated "formatter" subsys After trying to hack epoch indexed time series and failing miserably, decided to properly factor out all formatting routines into a common subsystem API: ``IncrementalFormatter`` which provides the interface for incrementally updating and tracking pre-path-graphics formatted data. Previously this functionality was mangled into our `Renderer` (which also does the work of `QPath` generation and update) but splitting it out also preps for being able to do graphics-buffer downsampling and caching on a remote host B) The ``IncrementalFormatter`` (parent type) has the default behaviour of tracking a single field-array on some source `ShmArray`, updating a flattened `numpy.ndarray` in-mem allocation, and providing a default 1d conversion for pre-downsampling and path generation. Changed out of `Renderer`, - `.allocate_xy()`, `update_xy()` and `format_xy()` all are moved to more explicitly named formatter methods. - all `.x/y_data` nd array management and update - "last view range" tracking - `.last_read`, `.diff()` - now calls `IncrementalFormatter.format_to_1d()` inside `.render()` The new API gets, - `.diff()`, `.last_read` - all view range diff tracking through `.track_inview_range()`. - better nd format array names: `.x/y_nd`, `xy_nd_start/stop`. - `.format_to_1d()` which renders pre-path formatted arrays ready for both m4 sampling and path gen. - better explicit overloadable formatting method names: * `.allocate_xy()` -> `.allocate_xy_nd()` * `.update_xy()` -> `.incr_update_xy_nd()` * `.format_xy()` -> `.format_xy_nd_to_1d()` Finally this implements per-graphics-type formatters which define each set up related formatting routines: - `OHLCBarsFmtr`: std multi-line style bars - `OHLCBarsAsCurveFmtr`: draws an interpolated line for ohlc sampled data - `StepCurveFmtr`: handles vlm style curves
2022-11-22 22:28:58 +00:00
) = fmt_out
WIP get incremental step curve updates working This took longer then i care to admit XD but it definitely adds a huge speedup and with only a few outstanding correctness bugs: - panning from left to right causes strange trailing artifacts in the flows fsp (vlm) sub-plot but only when some data is off-screen on the left but doesn't appear to be an issue if we keep the `._set_yrange()` handler hooked up to the `.sigXRangeChanged` signal (but we aren't going to because this makes panning way slower). i've got a feeling this is a bug todo with the device coordinate cache stuff and we may need to report to Qt core? - factoring out the step curve logic from `FastAppendCurve.update_from_array()` (un)fortunately required some logic branch uncoupling but also meant we needed special input controls to avoid things like redraws and curve appends for special cases, this will hopefully all be better rectified in code when the core of this method is moved into a renderer type/implementation. - the `tina_vwap` fsp curve now somehow causes hangs when doing erratic scrolling on downsampled graphics data. i have no idea why or how but disabling it makes the issue go away (ui will literally just freeze and gobble CPU on a `.paint()` call until you ctrl-c the hell out of it). my guess is that something in the logic for standard line curves and appends on large data sets is the issue? Code related changes/hacks: - drop use of `step_path_arrays_from_1d()`, it was always a bit hacky (being based on `pyqtgraph` internals) and was generally hard to understand since it returns 1d data instead of the more expected (N,2) array of "step levels"; instead this is now implemented (uglily) in the `Flow.update_graphics()` block for step curves (which will obviously get cleaned up and factored elsewhere). - add a bunch of new flags to the update method on the fast append curve: `draw_last: bool`, `slice_to_head: int`, `do_append: bool`, `should_redraw: bool` which are all controls to aid with previously mentioned issues specific to getting step curve updates working correctly. - add a ton of commented tinkering related code (that we may end up using) to both the flow and append curve methods that was written as part of the effort to get this all working. - implement all step curve updating inline in `Flow.update_graphics()` including prepend and append logic for pre-graphics incremental step data maintenance and in-view slicing as well as "last step" graphics updating. Obviously clean up commits coming stat B)
2022-04-26 12:34:53 +00:00
Short-circuit rendering on no 1d-data; avoid m4 layer crash
2023-03-13 16:18:54 +00:00
if not x_1d.size:
log.warning(f'{array_key} has no `.size`?')
return
Reset line graphics on downsample step.. This was a bit of a nightmare to figure out but, it seems that the coordinate caching system will really be a dick (like the nickname for richard for you serious types) about leaving stale graphics if we don't reset the cache on downsample full-redraw updates...Sooo, instead we do this manual reset to avoid such artifacts and consequently (for now) return a `reset: bool` flag in the return tuple from `Renderer.render()` to indicate as such. Some further shite: - move the step mode `.draw_last()` equivalent graphics updates down with the rest.. - drop some superfluous `should_redraw` logic from `Renderer.render()` and compound it in the full path redraw block.
2022-05-30 23:02:06 +00:00
# redraw conditions
if (
prepend_length > 0
or new_sample_rate
Add graphics incr-updated "formatter" subsys After trying to hack epoch indexed time series and failing miserably, decided to properly factor out all formatting routines into a common subsystem API: ``IncrementalFormatter`` which provides the interface for incrementally updating and tracking pre-path-graphics formatted data. Previously this functionality was mangled into our `Renderer` (which also does the work of `QPath` generation and update) but splitting it out also preps for being able to do graphics-buffer downsampling and caching on a remote host B) The ``IncrementalFormatter`` (parent type) has the default behaviour of tracking a single field-array on some source `ShmArray`, updating a flattened `numpy.ndarray` in-mem allocation, and providing a default 1d conversion for pre-downsampling and path generation. Changed out of `Renderer`, - `.allocate_xy()`, `update_xy()` and `format_xy()` all are moved to more explicitly named formatter methods. - all `.x/y_data` nd array management and update - "last view range" tracking - `.last_read`, `.diff()` - now calls `IncrementalFormatter.format_to_1d()` inside `.render()` The new API gets, - `.diff()`, `.last_read` - all view range diff tracking through `.track_inview_range()`. - better nd format array names: `.x/y_nd`, `xy_nd_start/stop`. - `.format_to_1d()` which renders pre-path formatted arrays ready for both m4 sampling and path gen. - better explicit overloadable formatting method names: * `.allocate_xy()` -> `.allocate_xy_nd()` * `.update_xy()` -> `.incr_update_xy_nd()` * `.format_xy()` -> `.format_xy_nd_to_1d()` Finally this implements per-graphics-type formatters which define each set up related formatting routines: - `OHLCBarsFmtr`: std multi-line style bars - `OHLCBarsAsCurveFmtr`: draws an interpolated line for ohlc sampled data - `StepCurveFmtr`: handles vlm style curves
2022-11-22 22:28:58 +00:00
or view_changed
Drop px-cache-resets, failed try at path appends Comments out the pixel-cache resetting since it doesn't seem we need it any more to avoid draw oddities? For `.fast_path` appends, this nearly got it working except the new path segments are either not being connected correctly (step curve) or not being drawn in full since the history path (plain line). Leaving the attempted code commented in for a retry in the future; my best guesses are that maybe, - `.connectPath()` call is being done with incorrect segment length and/or start point. - the "appended" data: `appended = array[-append_len-1:slice_to_head]` (done inside the formatter) isn't correct (i.e. endpoint handling considering a path append) and needs special handling for different curve types?
2022-11-23 19:17:18 +00:00
# NOTE: comment this to try and make "append paths"
# work below..
or append_length > 0
Reset line graphics on downsample step.. This was a bit of a nightmare to figure out but, it seems that the coordinate caching system will really be a dick (like the nickname for richard for you serious types) about leaving stale graphics if we don't reset the cache on downsample full-redraw updates...Sooo, instead we do this manual reset to avoid such artifacts and consequently (for now) return a `reset: bool` flag in the return tuple from `Renderer.render()` to indicate as such. Some further shite: - move the step mode `.draw_last()` equivalent graphics updates down with the rest.. - drop some superfluous `should_redraw` logic from `Renderer.render()` and compound it in the full path redraw block.
2022-05-30 23:02:06 +00:00
):
Factor `.update_from_array()` into `Flow.update_graphics()` A bit hacky to get all graphics types working but this is hopefully the first step toward moving all the generic update logic into `Renderer` types which can be themselves managed more compactly and cached per uppx-m4 level.
2022-05-18 12:24:12 +00:00
should_redraw = True
WIP get incremental step curve updates working This took longer then i care to admit XD but it definitely adds a huge speedup and with only a few outstanding correctness bugs: - panning from left to right causes strange trailing artifacts in the flows fsp (vlm) sub-plot but only when some data is off-screen on the left but doesn't appear to be an issue if we keep the `._set_yrange()` handler hooked up to the `.sigXRangeChanged` signal (but we aren't going to because this makes panning way slower). i've got a feeling this is a bug todo with the device coordinate cache stuff and we may need to report to Qt core? - factoring out the step curve logic from `FastAppendCurve.update_from_array()` (un)fortunately required some logic branch uncoupling but also meant we needed special input controls to avoid things like redraws and curve appends for special cases, this will hopefully all be better rectified in code when the core of this method is moved into a renderer type/implementation. - the `tina_vwap` fsp curve now somehow causes hangs when doing erratic scrolling on downsampled graphics data. i have no idea why or how but disabling it makes the issue go away (ui will literally just freeze and gobble CPU on a `.paint()` call until you ctrl-c the hell out of it). my guess is that something in the logic for standard line curves and appends on large data sets is the issue? Code related changes/hacks: - drop use of `step_path_arrays_from_1d()`, it was always a bit hacky (being based on `pyqtgraph` internals) and was generally hard to understand since it returns 1d data instead of the more expected (N,2) array of "step levels"; instead this is now implemented (uglily) in the `Flow.update_graphics()` block for step curves (which will obviously get cleaned up and factored elsewhere). - add a bunch of new flags to the update method on the fast append curve: `draw_last: bool`, `slice_to_head: int`, `do_append: bool`, `should_redraw: bool` which are all controls to aid with previously mentioned issues specific to getting step curve updates working correctly. - add a ton of commented tinkering related code (that we may end up using) to both the flow and append curve methods that was written as part of the effort to get this all working. - implement all step curve updating inline in `Flow.update_graphics()` including prepend and append logic for pre-graphics incremental step data maintenance and in-view slicing as well as "last step" graphics updating. Obviously clean up commits coming stat B)
2022-04-26 12:34:53 +00:00
Add type annots to vars inside `Render.render()`
2022-12-19 00:50:41 +00:00
path: QPainterPath = self.path
fast_path: QPainterPath = self.fast_path
reset: bool = False
Factor `.update_from_array()` into `Flow.update_graphics()` A bit hacky to get all graphics types working but this is hopefully the first step toward moving all the generic update logic into `Renderer` types which can be themselves managed more compactly and cached per uppx-m4 level.
2022-05-18 12:24:12 +00:00
Better doc string, use `Viz.vs: ViewState`
2023-02-28 14:43:16 +00:00
self.viz.ds_yrange = None
Re-implement `.default_view()` on `Viz` Since we don't really need it defined on the "chart widget" move it to a viz method and rework it to hell: - always discard the invalid view l > r case. - use the graphic's UPPX to determine UI-to-scene coordinate scaling for the L1-label collision detection, if there is no L1 just offset by a few (index step scaled) datums; this allows us to drop the 2x x-range calls as was hacked previous. - handle no-data-in-view cases explicitly and error if we get any ostensibly impossible cases. - expect caller to trigger a graphics cycle if needed. Further support this includes a rework a slew of other important details: - add `Viz.index_step`, an idempotent computed, index (presumably uniform) step value which is needed for variable sample rate graphics displayed on an epoch (second) time index. - rework `Viz.datums_range()` to pass view x-endpoints as first and last elements in return `tuple`; tighten up snap-to-data edge case logic using `max()`/`min()` calls and better internal var naming. - adjust all calls to `slice_from_time()` to not expect an "abs" slice. - drop all `.yrange` resetting since we can just have the `Renderer` do it when necessary.
2022-12-07 21:31:32 +00:00
Drop tons of old cruft, move around some commented ideas
2022-05-19 14:23:59 +00:00
# redraw the entire source data if we have either of:
# - no prior path graphic rendered or,
# - we always intend to re-render the data only in view
Factor `.update_from_array()` into `Flow.update_graphics()` A bit hacky to get all graphics types working but this is hopefully the first step toward moving all the generic update logic into `Renderer` types which can be themselves managed more compactly and cached per uppx-m4 level.
2022-05-18 12:24:12 +00:00
if (
Treat paths like input/output vars
2022-05-18 12:53:35 +00:00
path is None
Factor `.update_from_array()` into `Flow.update_graphics()` A bit hacky to get all graphics types working but this is hopefully the first step toward moving all the generic update logic into `Renderer` types which can be themselves managed more compactly and cached per uppx-m4 level.
2022-05-18 12:24:12 +00:00
or should_redraw
):
Rename `Flow` -> `Viz` The type is better described as a "data visualization": https://en.wikipedia.org/wiki/Data_and_information_visualization Add `ChartPlotWidget.get_viz()` to start working towards not accessing the private table directly XD We'll probably end up using the name `Flow` for a type that tracks a collection of composed/cascaded `Flume`s: https://en.wikipedia.org/wiki/Two-port_network#Cascade_connection
2022-11-24 20:33:58 +00:00
# print(f"{self.viz.name} -> REDRAWING BRUH")
Expose a `force_reformat: bool` up through graphics stack
2023-06-06 21:22:50 +00:00
if (
new_sample_rate
and showing_src_data
):
Clean a buncha cruft from render mod
2023-01-05 23:04:40 +00:00
log.info(f'DE-downsampling -> {array_key}')
Factor `.update_from_array()` into `Flow.update_graphics()` A bit hacky to get all graphics types working but this is hopefully the first step toward moving all the generic update logic into `Renderer` types which can be themselves managed more compactly and cached per uppx-m4 level.
2022-05-18 12:24:12 +00:00
self._in_ds = False
elif should_ds and uppx > 1:
Breakpoint bad (-ve or too large) x-ranges to m4 This should never really happen but when it does it appears to be a race with writing startup pre-graphics-formatter array data where we get `x_end` epoch value subtracting some really small offset value (like `-/+0.5`) or the opposite where the `x_start` is epoch and `x_end` is small. This adds a warning msg and `breakpoint()` as well as guards around the entire code downsampling code path so that when resumed the downsampling cycle should just be skipped and avoid a crash.
2023-01-12 21:03:19 +00:00
ds_out = xy_downsample(
Add graphics incr-updated "formatter" subsys After trying to hack epoch indexed time series and failing miserably, decided to properly factor out all formatting routines into a common subsystem API: ``IncrementalFormatter`` which provides the interface for incrementally updating and tracking pre-path-graphics formatted data. Previously this functionality was mangled into our `Renderer` (which also does the work of `QPath` generation and update) but splitting it out also preps for being able to do graphics-buffer downsampling and caching on a remote host B) The ``IncrementalFormatter`` (parent type) has the default behaviour of tracking a single field-array on some source `ShmArray`, updating a flattened `numpy.ndarray` in-mem allocation, and providing a default 1d conversion for pre-downsampling and path generation. Changed out of `Renderer`, - `.allocate_xy()`, `update_xy()` and `format_xy()` all are moved to more explicitly named formatter methods. - all `.x/y_data` nd array management and update - "last view range" tracking - `.last_read`, `.diff()` - now calls `IncrementalFormatter.format_to_1d()` inside `.render()` The new API gets, - `.diff()`, `.last_read` - all view range diff tracking through `.track_inview_range()`. - better nd format array names: `.x/y_nd`, `xy_nd_start/stop`. - `.format_to_1d()` which renders pre-path formatted arrays ready for both m4 sampling and path gen. - better explicit overloadable formatting method names: * `.allocate_xy()` -> `.allocate_xy_nd()` * `.update_xy()` -> `.incr_update_xy_nd()` * `.format_xy()` -> `.format_xy_nd_to_1d()` Finally this implements per-graphics-type formatters which define each set up related formatting routines: - `OHLCBarsFmtr`: std multi-line style bars - `OHLCBarsAsCurveFmtr`: draws an interpolated line for ohlc sampled data - `StepCurveFmtr`: handles vlm style curves
2022-11-22 22:28:58 +00:00
x_1d,
y_1d,
Factor `.update_from_array()` into `Flow.update_graphics()` A bit hacky to get all graphics types working but this is hopefully the first step toward moving all the generic update logic into `Renderer` types which can be themselves managed more compactly and cached per uppx-m4 level.
2022-05-18 12:24:12 +00:00
uppx,
WIP get incremental step curve updates working This took longer then i care to admit XD but it definitely adds a huge speedup and with only a few outstanding correctness bugs: - panning from left to right causes strange trailing artifacts in the flows fsp (vlm) sub-plot but only when some data is off-screen on the left but doesn't appear to be an issue if we keep the `._set_yrange()` handler hooked up to the `.sigXRangeChanged` signal (but we aren't going to because this makes panning way slower). i've got a feeling this is a bug todo with the device coordinate cache stuff and we may need to report to Qt core? - factoring out the step curve logic from `FastAppendCurve.update_from_array()` (un)fortunately required some logic branch uncoupling but also meant we needed special input controls to avoid things like redraws and curve appends for special cases, this will hopefully all be better rectified in code when the core of this method is moved into a renderer type/implementation. - the `tina_vwap` fsp curve now somehow causes hangs when doing erratic scrolling on downsampled graphics data. i have no idea why or how but disabling it makes the issue go away (ui will literally just freeze and gobble CPU on a `.paint()` call until you ctrl-c the hell out of it). my guess is that something in the logic for standard line curves and appends on large data sets is the issue? Code related changes/hacks: - drop use of `step_path_arrays_from_1d()`, it was always a bit hacky (being based on `pyqtgraph` internals) and was generally hard to understand since it returns 1d data instead of the more expected (N,2) array of "step levels"; instead this is now implemented (uglily) in the `Flow.update_graphics()` block for step curves (which will obviously get cleaned up and factored elsewhere). - add a bunch of new flags to the update method on the fast append curve: `draw_last: bool`, `slice_to_head: int`, `do_append: bool`, `should_redraw: bool` which are all controls to aid with previously mentioned issues specific to getting step curve updates working correctly. - add a ton of commented tinkering related code (that we may end up using) to both the flow and append curve methods that was written as part of the effort to get this all working. - implement all step curve updating inline in `Flow.update_graphics()` including prepend and append logic for pre-graphics incremental step data maintenance and in-view slicing as well as "last step" graphics updating. Obviously clean up commits coming stat B)
2022-04-26 12:34:53 +00:00
)
Breakpoint bad (-ve or too large) x-ranges to m4 This should never really happen but when it does it appears to be a race with writing startup pre-graphics-formatter array data where we get `x_end` epoch value subtracting some really small offset value (like `-/+0.5`) or the opposite where the `x_start` is epoch and `x_end` is small. This adds a warning msg and `breakpoint()` as well as guards around the entire code downsampling code path so that when resumed the downsampling cycle should just be skipped and avoid a crash.
2023-01-12 21:03:19 +00:00
if ds_out is not None:
x_1d, y_1d, ymn, ymx = ds_out
Better doc string, use `Viz.vs: ViewState`
2023-02-28 14:43:16 +00:00
self.viz.ds_yrange = ymn, ymx
Breakpoint bad (-ve or too large) x-ranges to m4 This should never really happen but when it does it appears to be a race with writing startup pre-graphics-formatter array data where we get `x_end` epoch value subtracting some really small offset value (like `-/+0.5`) or the opposite where the `x_start` is epoch and `x_end` is small. This adds a warning msg and `breakpoint()` as well as guards around the entire code downsampling code path so that when resumed the downsampling cycle should just be skipped and avoid a crash.
2023-01-12 21:03:19 +00:00
# print(f'{self.viz.name} post ds: ymn, ymx: {ymn},{ymx}')
reset = True
profiler(f'FULL PATH downsample redraw={should_ds}')
self._in_ds = True
Factor `.update_from_array()` into `Flow.update_graphics()` A bit hacky to get all graphics types working but this is hopefully the first step toward moving all the generic update logic into `Renderer` types which can be themselves managed more compactly and cached per uppx-m4 level.
2022-05-18 12:24:12 +00:00
Move graphics update logic into `Renderer.render()` Finally this gets us much closer to a generic incremental update system for graphics wherein the input array diffing, pre-graphical format data processing, downsampler activation and incremental update and storage of any of these data flow stages can be managed in one modular sub-system :surfer_boi:. Dirty deatz: - reorg and move all path logic into `Renderer.render()` and have it take in pretty much the same flags as the old `FastAppendCurve.update_from_array()` and instead storing all update state vars (even copies of the downsampler related ones) on the renderer instance: - new state vars: `._last_uppx, ._in_ds, ._vr, ._avr` - `.render()` input bools: `new_sample_rate, should_redraw, should_ds, showing_src_data` - add a hack-around for passing in incremental update data (for now) via a `input_data: tuple` of numpy arrays - a default `uppx: float = 1` - add new render interface attrs: - `.format_xy()` which takes in the source data array and produces out x, y arrays (and maybe a `connect` array) that can be passed to `.draw_path()` (the default for this is just to slice out the index and `array_key: str` columns from the input struct array), - `.draw_path()` which takes in the x, y, connect arrays and generates a `QPainterPath` - `.fast_path`, for "appendable" updates like there was on the fast append curve - move redraw (aka `.clear()` calls) into `.draw_path()` and trigger via `redraw: bool` flag. - our graphics objects no longer set their own `.path` state, it's done by the `Flow.update_graphics()` method using output from `Renderer.render()` (and it's state if necessary)
2022-05-18 19:23:14 +00:00
path = self.draw_path(
Add graphics incr-updated "formatter" subsys After trying to hack epoch indexed time series and failing miserably, decided to properly factor out all formatting routines into a common subsystem API: ``IncrementalFormatter`` which provides the interface for incrementally updating and tracking pre-path-graphics formatted data. Previously this functionality was mangled into our `Renderer` (which also does the work of `QPath` generation and update) but splitting it out also preps for being able to do graphics-buffer downsampling and caching on a remote host B) The ``IncrementalFormatter`` (parent type) has the default behaviour of tracking a single field-array on some source `ShmArray`, updating a flattened `numpy.ndarray` in-mem allocation, and providing a default 1d conversion for pre-downsampling and path generation. Changed out of `Renderer`, - `.allocate_xy()`, `update_xy()` and `format_xy()` all are moved to more explicitly named formatter methods. - all `.x/y_data` nd array management and update - "last view range" tracking - `.last_read`, `.diff()` - now calls `IncrementalFormatter.format_to_1d()` inside `.render()` The new API gets, - `.diff()`, `.last_read` - all view range diff tracking through `.track_inview_range()`. - better nd format array names: `.x/y_nd`, `xy_nd_start/stop`. - `.format_to_1d()` which renders pre-path formatted arrays ready for both m4 sampling and path gen. - better explicit overloadable formatting method names: * `.allocate_xy()` -> `.allocate_xy_nd()` * `.update_xy()` -> `.incr_update_xy_nd()` * `.format_xy()` -> `.format_xy_nd_to_1d()` Finally this implements per-graphics-type formatters which define each set up related formatting routines: - `OHLCBarsFmtr`: std multi-line style bars - `OHLCBarsAsCurveFmtr`: draws an interpolated line for ohlc sampled data - `StepCurveFmtr`: handles vlm style curves
2022-11-22 22:28:58 +00:00
x=x_1d,
y=y_1d,
Move graphics update logic into `Renderer.render()` Finally this gets us much closer to a generic incremental update system for graphics wherein the input array diffing, pre-graphical format data processing, downsampler activation and incremental update and storage of any of these data flow stages can be managed in one modular sub-system :surfer_boi:. Dirty deatz: - reorg and move all path logic into `Renderer.render()` and have it take in pretty much the same flags as the old `FastAppendCurve.update_from_array()` and instead storing all update state vars (even copies of the downsampler related ones) on the renderer instance: - new state vars: `._last_uppx, ._in_ds, ._vr, ._avr` - `.render()` input bools: `new_sample_rate, should_redraw, should_ds, showing_src_data` - add a hack-around for passing in incremental update data (for now) via a `input_data: tuple` of numpy arrays - a default `uppx: float = 1` - add new render interface attrs: - `.format_xy()` which takes in the source data array and produces out x, y arrays (and maybe a `connect` array) that can be passed to `.draw_path()` (the default for this is just to slice out the index and `array_key: str` columns from the input struct array), - `.draw_path()` which takes in the x, y, connect arrays and generates a `QPainterPath` - `.fast_path`, for "appendable" updates like there was on the fast append curve - move redraw (aka `.clear()` calls) into `.draw_path()` and trigger via `redraw: bool` flag. - our graphics objects no longer set their own `.path` state, it's done by the `Flow.update_graphics()` method using output from `Renderer.render()` (and it's state if necessary)
2022-05-18 19:23:14 +00:00
connect=connect,
Treat paths like input/output vars
2022-05-18 12:53:35 +00:00
path=path,
Move graphics update logic into `Renderer.render()` Finally this gets us much closer to a generic incremental update system for graphics wherein the input array diffing, pre-graphical format data processing, downsampler activation and incremental update and storage of any of these data flow stages can be managed in one modular sub-system :surfer_boi:. Dirty deatz: - reorg and move all path logic into `Renderer.render()` and have it take in pretty much the same flags as the old `FastAppendCurve.update_from_array()` and instead storing all update state vars (even copies of the downsampler related ones) on the renderer instance: - new state vars: `._last_uppx, ._in_ds, ._vr, ._avr` - `.render()` input bools: `new_sample_rate, should_redraw, should_ds, showing_src_data` - add a hack-around for passing in incremental update data (for now) via a `input_data: tuple` of numpy arrays - a default `uppx: float = 1` - add new render interface attrs: - `.format_xy()` which takes in the source data array and produces out x, y arrays (and maybe a `connect` array) that can be passed to `.draw_path()` (the default for this is just to slice out the index and `array_key: str` columns from the input struct array), - `.draw_path()` which takes in the x, y, connect arrays and generates a `QPainterPath` - `.fast_path`, for "appendable" updates like there was on the fast append curve - move redraw (aka `.clear()` calls) into `.draw_path()` and trigger via `redraw: bool` flag. - our graphics objects no longer set their own `.path` state, it's done by the `Flow.update_graphics()` method using output from `Renderer.render()` (and it's state if necessary)
2022-05-18 19:23:14 +00:00
redraw=True,
Factor `.update_from_array()` into `Flow.update_graphics()` A bit hacky to get all graphics types working but this is hopefully the first step toward moving all the generic update logic into `Renderer` types which can be themselves managed more compactly and cached per uppx-m4 level.
2022-05-18 12:24:12 +00:00
)
Move graphics update logic into `Renderer.render()` Finally this gets us much closer to a generic incremental update system for graphics wherein the input array diffing, pre-graphical format data processing, downsampler activation and incremental update and storage of any of these data flow stages can be managed in one modular sub-system :surfer_boi:. Dirty deatz: - reorg and move all path logic into `Renderer.render()` and have it take in pretty much the same flags as the old `FastAppendCurve.update_from_array()` and instead storing all update state vars (even copies of the downsampler related ones) on the renderer instance: - new state vars: `._last_uppx, ._in_ds, ._vr, ._avr` - `.render()` input bools: `new_sample_rate, should_redraw, should_ds, showing_src_data` - add a hack-around for passing in incremental update data (for now) via a `input_data: tuple` of numpy arrays - a default `uppx: float = 1` - add new render interface attrs: - `.format_xy()` which takes in the source data array and produces out x, y arrays (and maybe a `connect` array) that can be passed to `.draw_path()` (the default for this is just to slice out the index and `array_key: str` columns from the input struct array), - `.draw_path()` which takes in the x, y, connect arrays and generates a `QPainterPath` - `.fast_path`, for "appendable" updates like there was on the fast append curve - move redraw (aka `.clear()` calls) into `.draw_path()` and trigger via `redraw: bool` flag. - our graphics objects no longer set their own `.path` state, it's done by the `Flow.update_graphics()` method using output from `Renderer.render()` (and it's state if necessary)
2022-05-18 19:23:14 +00:00
Factor `.update_from_array()` into `Flow.update_graphics()` A bit hacky to get all graphics types working but this is hopefully the first step toward moving all the generic update logic into `Renderer` types which can be themselves managed more compactly and cached per uppx-m4 level.
2022-05-18 12:24:12 +00:00
profiler(
'generated fresh path. '
f'(should_redraw: {should_redraw} '
f'should_ds: {should_ds} new_sample_rate: {new_sample_rate})'
)
# TODO: get this piecewise prepend working - right now it's
# giving heck on vwap...
# elif prepend_length:
# prepend_path = pg.functions.arrayToQPath(
# x[0:prepend_length],
# y[0:prepend_length],
# connect='all'
# )
# # swap prepend path in "front"
# old_path = graphics.path
# graphics.path = prepend_path
# # graphics.path.moveTo(new_x[0], new_y[0])
# graphics.path.connectPath(old_path)
elif (
append_length > 0
and do_append
):
Clean a buncha cruft from render mod
2023-01-05 23:04:40 +00:00
profiler(f'sliced append path {append_length}')
Drop px-cache-resets, failed try at path appends Comments out the pixel-cache resetting since it doesn't seem we need it any more to avoid draw oddities? For `.fast_path` appends, this nearly got it working except the new path segments are either not being connected correctly (step curve) or not being drawn in full since the history path (plain line). Leaving the attempted code commented in for a retry in the future; my best guesses are that maybe, - `.connectPath()` call is being done with incorrect segment length and/or start point. - the "appended" data: `appended = array[-append_len-1:slice_to_head]` (done inside the formatter) isn't correct (i.e. endpoint handling considering a path append) and needs special handling for different curve types?
2022-11-23 19:17:18 +00:00
# (
# x_1d,
# y_1d,
# connect,
# ) = append_tres
Factor `.update_from_array()` into `Flow.update_graphics()` A bit hacky to get all graphics types working but this is hopefully the first step toward moving all the generic update logic into `Renderer` types which can be themselves managed more compactly and cached per uppx-m4 level.
2022-05-18 12:24:12 +00:00
profiler(
f'diffed array input, append_length={append_length}'
)
WIP get incremental step curve updates working This took longer then i care to admit XD but it definitely adds a huge speedup and with only a few outstanding correctness bugs: - panning from left to right causes strange trailing artifacts in the flows fsp (vlm) sub-plot but only when some data is off-screen on the left but doesn't appear to be an issue if we keep the `._set_yrange()` handler hooked up to the `.sigXRangeChanged` signal (but we aren't going to because this makes panning way slower). i've got a feeling this is a bug todo with the device coordinate cache stuff and we may need to report to Qt core? - factoring out the step curve logic from `FastAppendCurve.update_from_array()` (un)fortunately required some logic branch uncoupling but also meant we needed special input controls to avoid things like redraws and curve appends for special cases, this will hopefully all be better rectified in code when the core of this method is moved into a renderer type/implementation. - the `tina_vwap` fsp curve now somehow causes hangs when doing erratic scrolling on downsampled graphics data. i have no idea why or how but disabling it makes the issue go away (ui will literally just freeze and gobble CPU on a `.paint()` call until you ctrl-c the hell out of it). my guess is that something in the logic for standard line curves and appends on large data sets is the issue? Code related changes/hacks: - drop use of `step_path_arrays_from_1d()`, it was always a bit hacky (being based on `pyqtgraph` internals) and was generally hard to understand since it returns 1d data instead of the more expected (N,2) array of "step levels"; instead this is now implemented (uglily) in the `Flow.update_graphics()` block for step curves (which will obviously get cleaned up and factored elsewhere). - add a bunch of new flags to the update method on the fast append curve: `draw_last: bool`, `slice_to_head: int`, `do_append: bool`, `should_redraw: bool` which are all controls to aid with previously mentioned issues specific to getting step curve updates working correctly. - add a ton of commented tinkering related code (that we may end up using) to both the flow and append curve methods that was written as part of the effort to get this all working. - implement all step curve updating inline in `Flow.update_graphics()` including prepend and append logic for pre-graphics incremental step data maintenance and in-view slicing as well as "last step" graphics updating. Obviously clean up commits coming stat B)
2022-04-26 12:34:53 +00:00
Drop px-cache-resets, failed try at path appends Comments out the pixel-cache resetting since it doesn't seem we need it any more to avoid draw oddities? For `.fast_path` appends, this nearly got it working except the new path segments are either not being connected correctly (step curve) or not being drawn in full since the history path (plain line). Leaving the attempted code commented in for a retry in the future; my best guesses are that maybe, - `.connectPath()` call is being done with incorrect segment length and/or start point. - the "appended" data: `appended = array[-append_len-1:slice_to_head]` (done inside the formatter) isn't correct (i.e. endpoint handling considering a path append) and needs special handling for different curve types?
2022-11-23 19:17:18 +00:00
# if should_ds and uppx > 1:
Factor `.update_from_array()` into `Flow.update_graphics()` A bit hacky to get all graphics types working but this is hopefully the first step toward moving all the generic update logic into `Renderer` types which can be themselves managed more compactly and cached per uppx-m4 level.
2022-05-18 12:24:12 +00:00
# new_x, new_y = xy_downsample(
# new_x,
# new_y,
# uppx,
# )
# profiler(f'fast path downsample redraw={should_ds}')
Move graphics update logic into `Renderer.render()` Finally this gets us much closer to a generic incremental update system for graphics wherein the input array diffing, pre-graphical format data processing, downsampler activation and incremental update and storage of any of these data flow stages can be managed in one modular sub-system :surfer_boi:. Dirty deatz: - reorg and move all path logic into `Renderer.render()` and have it take in pretty much the same flags as the old `FastAppendCurve.update_from_array()` and instead storing all update state vars (even copies of the downsampler related ones) on the renderer instance: - new state vars: `._last_uppx, ._in_ds, ._vr, ._avr` - `.render()` input bools: `new_sample_rate, should_redraw, should_ds, showing_src_data` - add a hack-around for passing in incremental update data (for now) via a `input_data: tuple` of numpy arrays - a default `uppx: float = 1` - add new render interface attrs: - `.format_xy()` which takes in the source data array and produces out x, y arrays (and maybe a `connect` array) that can be passed to `.draw_path()` (the default for this is just to slice out the index and `array_key: str` columns from the input struct array), - `.draw_path()` which takes in the x, y, connect arrays and generates a `QPainterPath` - `.fast_path`, for "appendable" updates like there was on the fast append curve - move redraw (aka `.clear()` calls) into `.draw_path()` and trigger via `redraw: bool` flag. - our graphics objects no longer set their own `.path` state, it's done by the `Flow.update_graphics()` method using output from `Renderer.render()` (and it's state if necessary)
2022-05-18 19:23:14 +00:00
append_path = self.draw_path(
Drop px-cache-resets, failed try at path appends Comments out the pixel-cache resetting since it doesn't seem we need it any more to avoid draw oddities? For `.fast_path` appends, this nearly got it working except the new path segments are either not being connected correctly (step curve) or not being drawn in full since the history path (plain line). Leaving the attempted code commented in for a retry in the future; my best guesses are that maybe, - `.connectPath()` call is being done with incorrect segment length and/or start point. - the "appended" data: `appended = array[-append_len-1:slice_to_head]` (done inside the formatter) isn't correct (i.e. endpoint handling considering a path append) and needs special handling for different curve types?
2022-11-23 19:17:18 +00:00
x=x_1d,
y=y_1d,
Move graphics update logic into `Renderer.render()` Finally this gets us much closer to a generic incremental update system for graphics wherein the input array diffing, pre-graphical format data processing, downsampler activation and incremental update and storage of any of these data flow stages can be managed in one modular sub-system :surfer_boi:. Dirty deatz: - reorg and move all path logic into `Renderer.render()` and have it take in pretty much the same flags as the old `FastAppendCurve.update_from_array()` and instead storing all update state vars (even copies of the downsampler related ones) on the renderer instance: - new state vars: `._last_uppx, ._in_ds, ._vr, ._avr` - `.render()` input bools: `new_sample_rate, should_redraw, should_ds, showing_src_data` - add a hack-around for passing in incremental update data (for now) via a `input_data: tuple` of numpy arrays - a default `uppx: float = 1` - add new render interface attrs: - `.format_xy()` which takes in the source data array and produces out x, y arrays (and maybe a `connect` array) that can be passed to `.draw_path()` (the default for this is just to slice out the index and `array_key: str` columns from the input struct array), - `.draw_path()` which takes in the x, y, connect arrays and generates a `QPainterPath` - `.fast_path`, for "appendable" updates like there was on the fast append curve - move redraw (aka `.clear()` calls) into `.draw_path()` and trigger via `redraw: bool` flag. - our graphics objects no longer set their own `.path` state, it's done by the `Flow.update_graphics()` method using output from `Renderer.render()` (and it's state if necessary)
2022-05-18 19:23:14 +00:00
connect=connect,
Implement OHLC downsampled curve via renderer, drop old bypass code
2022-05-25 23:45:09 +00:00
path=fast_path,
Factor `.update_from_array()` into `Flow.update_graphics()` A bit hacky to get all graphics types working but this is hopefully the first step toward moving all the generic update logic into `Renderer` types which can be themselves managed more compactly and cached per uppx-m4 level.
2022-05-18 12:24:12 +00:00
)
profiler('generated append qpath')
Move graphics update logic into `Renderer.render()` Finally this gets us much closer to a generic incremental update system for graphics wherein the input array diffing, pre-graphical format data processing, downsampler activation and incremental update and storage of any of these data flow stages can be managed in one modular sub-system :surfer_boi:. Dirty deatz: - reorg and move all path logic into `Renderer.render()` and have it take in pretty much the same flags as the old `FastAppendCurve.update_from_array()` and instead storing all update state vars (even copies of the downsampler related ones) on the renderer instance: - new state vars: `._last_uppx, ._in_ds, ._vr, ._avr` - `.render()` input bools: `new_sample_rate, should_redraw, should_ds, showing_src_data` - add a hack-around for passing in incremental update data (for now) via a `input_data: tuple` of numpy arrays - a default `uppx: float = 1` - add new render interface attrs: - `.format_xy()` which takes in the source data array and produces out x, y arrays (and maybe a `connect` array) that can be passed to `.draw_path()` (the default for this is just to slice out the index and `array_key: str` columns from the input struct array), - `.draw_path()` which takes in the x, y, connect arrays and generates a `QPainterPath` - `.fast_path`, for "appendable" updates like there was on the fast append curve - move redraw (aka `.clear()` calls) into `.draw_path()` and trigger via `redraw: bool` flag. - our graphics objects no longer set their own `.path` state, it's done by the `Flow.update_graphics()` method using output from `Renderer.render()` (and it's state if necessary)
2022-05-18 19:23:14 +00:00
if use_fpath:
Factor `.update_from_array()` into `Flow.update_graphics()` A bit hacky to get all graphics types working but this is hopefully the first step toward moving all the generic update logic into `Renderer` types which can be themselves managed more compactly and cached per uppx-m4 level.
2022-05-18 12:24:12 +00:00
# an attempt at trying to make append-updates faster..
if fast_path is None:
Treat paths like input/output vars
2022-05-18 12:53:35 +00:00
fast_path = append_path
# fast_path.reserve(int(6e3))
Factor `.update_from_array()` into `Flow.update_graphics()` A bit hacky to get all graphics types working but this is hopefully the first step toward moving all the generic update logic into `Renderer` types which can be themselves managed more compactly and cached per uppx-m4 level.
2022-05-18 12:24:12 +00:00
else:
Clean a buncha cruft from render mod
2023-01-05 23:04:40 +00:00
# print(
# f'{self.viz.name}: FAST PATH\n'
# f"append_path br: {append_path.boundingRect()}\n"
# f"path size: {size}\n"
# f"append_path len: {append_path.length()}\n"
# f"fast_path len: {fast_path.length()}\n"
# )
Factor `.update_from_array()` into `Flow.update_graphics()` A bit hacky to get all graphics types working but this is hopefully the first step toward moving all the generic update logic into `Renderer` types which can be themselves managed more compactly and cached per uppx-m4 level.
2022-05-18 12:24:12 +00:00
fast_path.connectPath(append_path)
size = fast_path.capacity()
profiler(f'connected fast path w size: {size}')
# graphics.path.moveTo(new_x[0], new_y[0])
# path.connectPath(append_path)
# XXX: lol this causes a hang..
# graphics.path = graphics.path.simplified()
WIP incrementally update step array format
2022-04-24 16:33:25 +00:00
else:
Move graphics update logic into `Renderer.render()` Finally this gets us much closer to a generic incremental update system for graphics wherein the input array diffing, pre-graphical format data processing, downsampler activation and incremental update and storage of any of these data flow stages can be managed in one modular sub-system :surfer_boi:. Dirty deatz: - reorg and move all path logic into `Renderer.render()` and have it take in pretty much the same flags as the old `FastAppendCurve.update_from_array()` and instead storing all update state vars (even copies of the downsampler related ones) on the renderer instance: - new state vars: `._last_uppx, ._in_ds, ._vr, ._avr` - `.render()` input bools: `new_sample_rate, should_redraw, should_ds, showing_src_data` - add a hack-around for passing in incremental update data (for now) via a `input_data: tuple` of numpy arrays - a default `uppx: float = 1` - add new render interface attrs: - `.format_xy()` which takes in the source data array and produces out x, y arrays (and maybe a `connect` array) that can be passed to `.draw_path()` (the default for this is just to slice out the index and `array_key: str` columns from the input struct array), - `.draw_path()` which takes in the x, y, connect arrays and generates a `QPainterPath` - `.fast_path`, for "appendable" updates like there was on the fast append curve - move redraw (aka `.clear()` calls) into `.draw_path()` and trigger via `redraw: bool` flag. - our graphics objects no longer set their own `.path` state, it's done by the `Flow.update_graphics()` method using output from `Renderer.render()` (and it's state if necessary)
2022-05-18 19:23:14 +00:00
size = path.capacity()
Factor `.update_from_array()` into `Flow.update_graphics()` A bit hacky to get all graphics types working but this is hopefully the first step toward moving all the generic update logic into `Renderer` types which can be themselves managed more compactly and cached per uppx-m4 level.
2022-05-18 12:24:12 +00:00
profiler(f'connected history path w size: {size}')
Move graphics update logic into `Renderer.render()` Finally this gets us much closer to a generic incremental update system for graphics wherein the input array diffing, pre-graphical format data processing, downsampler activation and incremental update and storage of any of these data flow stages can be managed in one modular sub-system :surfer_boi:. Dirty deatz: - reorg and move all path logic into `Renderer.render()` and have it take in pretty much the same flags as the old `FastAppendCurve.update_from_array()` and instead storing all update state vars (even copies of the downsampler related ones) on the renderer instance: - new state vars: `._last_uppx, ._in_ds, ._vr, ._avr` - `.render()` input bools: `new_sample_rate, should_redraw, should_ds, showing_src_data` - add a hack-around for passing in incremental update data (for now) via a `input_data: tuple` of numpy arrays - a default `uppx: float = 1` - add new render interface attrs: - `.format_xy()` which takes in the source data array and produces out x, y arrays (and maybe a `connect` array) that can be passed to `.draw_path()` (the default for this is just to slice out the index and `array_key: str` columns from the input struct array), - `.draw_path()` which takes in the x, y, connect arrays and generates a `QPainterPath` - `.fast_path`, for "appendable" updates like there was on the fast append curve - move redraw (aka `.clear()` calls) into `.draw_path()` and trigger via `redraw: bool` flag. - our graphics objects no longer set their own `.path` state, it's done by the `Flow.update_graphics()` method using output from `Renderer.render()` (and it's state if necessary)
2022-05-18 19:23:14 +00:00
path.connectPath(append_path)
WIP starting architecture doc str writeup..
2022-04-18 12:30:28 +00:00
Move graphics update logic into `Renderer.render()` Finally this gets us much closer to a generic incremental update system for graphics wherein the input array diffing, pre-graphical format data processing, downsampler activation and incremental update and storage of any of these data flow stages can be managed in one modular sub-system :surfer_boi:. Dirty deatz: - reorg and move all path logic into `Renderer.render()` and have it take in pretty much the same flags as the old `FastAppendCurve.update_from_array()` and instead storing all update state vars (even copies of the downsampler related ones) on the renderer instance: - new state vars: `._last_uppx, ._in_ds, ._vr, ._avr` - `.render()` input bools: `new_sample_rate, should_redraw, should_ds, showing_src_data` - add a hack-around for passing in incremental update data (for now) via a `input_data: tuple` of numpy arrays - a default `uppx: float = 1` - add new render interface attrs: - `.format_xy()` which takes in the source data array and produces out x, y arrays (and maybe a `connect` array) that can be passed to `.draw_path()` (the default for this is just to slice out the index and `array_key: str` columns from the input struct array), - `.draw_path()` which takes in the x, y, connect arrays and generates a `QPainterPath` - `.fast_path`, for "appendable" updates like there was on the fast append curve - move redraw (aka `.clear()` calls) into `.draw_path()` and trigger via `redraw: bool` flag. - our graphics objects no longer set their own `.path` state, it's done by the `Flow.update_graphics()` method using output from `Renderer.render()` (and it's state if necessary)
2022-05-18 19:23:14 +00:00
self.path = path
self.fast_path = fast_path
More WIP, implement `BarItems` rendering in `Flow.update_graphics()`
2022-04-20 15:43:47 +00:00
Fix line -> bars on 6x UPPX Read the `Viz.index_step()` directly to avoid always reading 1 on the slow chart; this was completely broken before and resulting in not rendering the bars graphic on the slow chart until at a true uppx of 1 which obviously doesn't work for 60 width bars XD Further cleanups to `._render` module: - drop `array` output from `Renderer.render()`, `read_from_key` input and fix type annot. - drop `should_line`, `changed_to_line` and `render_kwargs` from `render_baritems()` outputs and instead calc `should_redraw` logic inside the func body and return as output.
2022-12-14 14:49:43 +00:00
return self.path, reset