Not really sure there's much we can do besides dump Grpc stuff when we
detect an "error" `str` for the moment..
Either way leave a buncha complaints (como siempre) and do linting
fixups..
Previously we would make the `ahabd` supervisor-actor sync to docker
container startup using pseudo-blocking log message processing.
This has issues,
- we're forced to do a hacky "yield back to `trio`" in order to be
"fake async" when reading the log stream and further,
- blocking on a message is fragile and often slow.
Instead, run the log processor in a background task and in the parent
task poll for the container to be in the client list using a similar
pseudo-async poll pattern. This allows the super to `Context.started()`
sooner (when the container is actually registered as "up") and thus
unblock its (remote) caller faster whilst still doing full log msg
proxying!
Deatz:
- adds `Container.cuid: str` a unique container id for logging.
- correctly proxy through the `loglevel: str` from `pikerd` caller task.
- shield around `Container.cancel()` in the teardown block and use
cancel level logging in that method.
With the addition of a new `elastixsearch` docker support in
https://github.com/pikers/piker/pull/464, adjustments were made
to container startup sync logic (particularly the `trio` checkpoint
sleep period - which itself is a hack around a sync client api) which
caused a regression in upstream startup logic wherein container error
logs were not being bubbled up correctly causing a silent failure mode:
- `marketstore` container started with corrupt input config
- `ahabd` super code timed out on startup phase due to a larger log
polling period, skipped processing startup logs from the container,
and continued on as though the container was started
- history client fails on grpc connection with no clear error on why the
connection failed.
Here we revert to the old poll period (1ms) to avoid any more silent
failures and further extend supervisor control through a configuration
override mechanism. To address the underlying design issue, this patch
adds support for container-endpoint-callbacks to override supervisor
startup configuration parameters via the 2nd value in their returned
tuple: the already delivered configuration `dict` value.
The current exposed values include:
{
'startup_timeout': 1.0,
'startup_query_period': 0.001,
'log_msg_key': 'msg',
},
This allows for container specific control over the startup-sync query
period (the hack mentioned above) as well as the expected log msg key
and of course the startup timeout.
Adds a `piker storage` subcmd with a `-d` flag to wipe a particular
fqsn's time series (both 1s and 60s). Obviously this needs to be
extended much more but provides a start point.
Since apparently the container we were using is totally borked on new
kernels and/or latest jvm, this move our old manual local-X-desktop script
back for use in `brokerd` backend code.
Adds a new `.brokers.ib._util` which contains the 2 methods and fails
over to this one when we can't connect to a VNC server. Also adjusts the
original in `scripts/ib_data_reset.py` to import and run the module code
as a script-program.
Also includes a retyping of `Client._pair: dict[str, Pair]` to look up
pair structs and map all alt-key-name-sets to each for easy precision
info lookup to set the `.sym` field for each transaction including for
on-chain transfers which kraken provides as an "asset decimals" field,
presumably pulled from the particular block-token's limitation info.
In order to support existing `pps.toml` files in the wild which don't
have the `asset_type, price_tick_size, lot_tick_size` fields, we need to
only optionally read them and instead expect that backends will write
the fields going forward (coming in follow patches).
Further this makes some small asset-size (vlm accounting) quantization
related adjustments:
- rename `Symbol.decimal_quant()` -> `.quantize_size()` since that is
explicitly what this method is doing.
- and expect an input `size: float` which we cast to decimal instead of
doing it inside the `.calc_size()` caller code.
- drop `Symbol.iterfqsns()` which wasn't being used anywhere at all..
Additionally, this drafts out a new replacement market-trading-pair data
type to eventually replace `.data._source.Symbol` -> `MktPair` which we
aren't using yet, but serves as the documentation-driven motivator ;)
and, it relates to https://github.com/pikers/piker/issues/467.
Add decimal quantize API to Symbol to simplify by-broker truncation
Add symbol info to `pps.toml`
Move _assert call to outside the _async_main context manager
Minor indentation and styling changes, also convert a few prints to log calls
Fix multi write / race condition on open_pps call
Switch open_pps to not write by default
Fix integer math kraken syminfo _tick_size initialization
ensure not to write pp header on startup
Comment out pytest settings
Add comments explaining delete_testing_dir fixture
use nonlocal instead of global for test state
Add unpacking get_fqsn method
Format test_paper
Add comments explaining sync/async book.send calls
Curve-path colouring and cache mode settings are used (and can thus be
factored out of) all child types; this moves them into the parent type's
`.__init__()` and adjusts all sub-types match:
- the bulk was moved out of the `Curve.__init__()` including all
previous commentary around cache settings.
- adjust `BarItems` to use a `NoCache` mode and instead use the
`last_step_pen: pg.Pen` and `._pen` inside it's `.pain()` instead of
defining functionally duplicate vars.
- adjust all (transitive) calls to `BarItems` to use the new kwargs
names.
Allows callers to know if they should care about a particular viz
rendering call by immediately knowing if the graphics are in view. This
turns out super useful particularly when doing dynamic y-ranging overlay
calcs.
First, we rename what was `chart_maxmin()` -> `multi_maxmin()` and don't
`partial` it in to the `DisplayState`, just call it with correct `Viz`
ref inputs.
Second, as we've done with `ChartView.maybe_downsample_graphics()` use
the output from the main `Viz.update_graphics()` and feed it to the
`.maxmin()` calls for the ohlc and vlm chart but still deliver the same
output signature as prior. Also accept and use an optional profiler
input, drop `DisplayState.maxmin()` and add `.vlm_viz`.
Further perf related tweak to do with more efficient incremental
updates:
- only call `multi_maxmin()` if the main fast chart viz does a pixel
column step.
- mask out hist viz and vlm viz and all linked fsp `._set_yrange()`
calls for now until we figure out how to best optimize these updates
when considering the new group-scaled-by-% style for multicharts.
- drop `.enable_auto_yrange()` calls during startup.
Acts as short cut when pipe-lining from `Viz.update_graphics()` (which
now returns the needed in-view array-relative-read-slice as output) such
that `Viz.read()` and `.datums_range()` doesn't need to be called
internally multiple times. In this case where `i_read_range` is provided
we of course skip doing time index translations and consequently lookup
the appropriate (epoch-time) index indices for caching.
Removes the multi-maxmin usage as well as ensures appropriate `Viz` refs
are passed into the view methods now requiring it. Also drops the "back
linking" of the vlm chart view to the source OHLC chart since we're
going to add this as a default to the charting API.
The max min for a given data range is defined on the lowest level
through the `Viz` api intermingling it with the view is a layering
issue. Instead make `._set_yrange()` call the appropriate view's viz
(since they should be one-to-one) directly and thus avoid any callback
monkey patching nonsense.
Requires that we now make `._set_yrange()` require either one of an
explicit `yrange: tuple[float, float]` min/max pair or the `Viz` ref (so
that maxmin can be called) as input. Adjust
`enable/disable_auto_yrange()` to bind in a new `._yranger()` partial
that's (solely) needed for signal reg/unreg which binds in the now
required input `Viz` to these methods.
Comment the `autoscale_overlays` block in `.maybe_downsample_graphics()`
for now until we figure out the most sane way to auto-range all linked
overlays and subplots (with their own overlays).
In an effort to ensure uniform and uppx-optimized last datum graphics
updates call this method directly instead of the equivalent graphics
object thus ensuring we only update the last pixel column according with
the appropriate max/min computed from the last uppx's worth of data.
Fixes / improvements to enable `.draw_last()` usage include,
- change `Viz._render_table` -> `._alt_r: tuple[Renderer, pg.GraphicsItem] | None`
which holds an alternative (usually downsampled) render and graphics
obj.
- extend the `.draw_last()` signature to include:
- `last_read` to allow passing in the already read data from
`.update_graphics()`, if it isn't passed then a manual read is done
internally.
- `reset_cache: bool` which is passed through to the graphics obj.
- use the new `Formatter.flat_index_ratio: float` when indexing into xy
1d data to compute the max/min for that px column.
Other,
- drop `bars_range` input from `maxmin()` since it's unused.
For the purposes of avoiding another full format call we can stash the
last rendered 1d xy pre-graphics formats as
`IncrementalFormatter.x/y_1d: np.ndarray`s and allow readers in the viz
and render machinery to use this data easily for things like "only
drawing the last uppx's worth of data as a line". Also add
a `.flat_index_ratio: float` which can be used similarly as a scalar
applied to indexes into the src array but instead when indexing
(flattened) 1d xy formatted outputs. Finally, this drops the way
overdone/noisy `.__repr__()` meth we had XD
When a new tick comes in but no new time step / bar is yet needed (to be
appended) we can simply adjust **only** the last bar datum
lines-graphic(s) to avoid a redraw of the preceding `QPainterPath` on
every tick. Do this by calling `Viz.draw_last()` on the fast and slow
chart and adjusting the guards around calls to `Viz.update_graphics()`
(which *does* update paths) to only enter when there's a `do_px_step`
condition. We can stop calling `main_viz.plot.vb._set_yrange()` on view
treading cases since the range should have already been adjusted by the
clearing-tick processing mxmn updates.
Further this changes,
- the `chart_maxmin()` helper (which we should eventually just get rid
of) to take bound in `Viz`s for the ohlc and vlm chart instead of the
chart widget handles.
- extend the guard around hist viz yranging to only enter when not in
"axis mode" - the same as for the fast viz.
Since we removed the `Viz.update_graphics()` call from the main rt loop
we have to be sure to call it in the history chart incr-loop to avoid
a gap between the last bar and prior history since startup. We only
need to update on tread since that should be the only time a full redraw
is ever necessary, ow only the last datum is needed.
Further this moves the graphics cycle func's profiler init to the top in
an effort to get more correct latency measures.
Since `ChartPlotWidget.update_graphics_from_flow()` is more or less just
a call to `Viz.update_graphics()` try to call that directly where
possible.
Changes include:
- calling the viz in the display state specific `maxmin()`.
- passing a viz instance to each `ChartView._set_yrange()` call (in prep
of explicit group auto-ranging); not that this input is unused in the
method for now.
- drop `bars_range` var passing since we don't use it.
Inside `._interaction` routines we need access to `Viz` instances.
Instead of doing `CharPlotWidget._vizs: dict` lookups this ensures each
plot can lookup it's (parent) viz without error.
Also, adjusts `Viz.maxmin()` output parsing to new signature.
Move the `Viz.datums_range()` call into `Viz.maxmin()` itself thus
minimizing the chart `.maxmin()` method to an ultra light wrapper around
the viz call. Also move all profiling into the `Viz` method.
Adjust `Viz.maxmin()` to return both the (rounded) x-range values which
correspond to the range containing the y-domain min and max so that
it can be used for up and coming overlay group maxmin calcs.
We obviously don't want to be debugging a sample-index issue if/when the
market for the asset is closed (since we'll be guaranteed to have
a mismatch, lul). Pass in the `feed_is_live: trio.Event` throughout the
backfilling routines to allow first checking for the live feed being active
so as to avoid breakpointing on false +ves. Also, add a detailed warning
log message for when *actually* investigating a mismatch.
This should never really happen but when it does it appears to be a race
with writing startup pre-graphics-formatter array data where we get
`x_end` epoch value subtracting some really small offset value (like
`-/+0.5`) or the opposite where the `x_start` is epoch and `x_end` is
small.
This adds a warning msg and `breakpoint()` as well as guards around the
entire code downsampling code path so that when resumed the downsampling
cycle should just be skipped and avoid a crash.
This attempt was unsuccessful since trying to (re)select the last
highlighted item on both an "enter" or "click" of that item causes
a hang and then segfault in `Qt`; no clue why..
Adds a `keep_current_item_selected: bool` flag to
`CompleterView.show_cache_entries()` but using it seems to always cause
a hang and crash; we keep all potential use spots commented for now
obviously to avoid this. Also included is a bunch of tidying to logic
blocks in the kb-control loop for readability.
Whenever the last datum is in view `slice_from_time()` need to always
spec the final array index (i.e. the len - 1 value we set as
`read_i_max`) to avoid a uniform-step arithmetic error where gaps in the
underlying time series causes an index that's too low to be returned.
- adjust zoom focal to be min of the view-right coord or the right-most
point on the flow graphic in view and drop all the legacy l1-in-view
focal point cruft.
- flip to not auto-scaling overlays by default.
- change the `._set_yrange()` margin to `0.09`.
- drop `use_vr: bool` usage.
Before this axes were being stacked from the outside in (for `'right'`
and 'bottom'` axes) which is somewhat non-intuitive for an `.append()`
operation. As such this change makes a symbol list stack a set of
`'right'` axes from left-to-right.
Details:
- rename `ComposeGridLayout.items` -> `.pitems`
- return `(int, list[AxisItem])` pairs from `.insert/append_plotitem()`
and the down stream `PlotItemOverlay.add_plotitem()`.
- drop `PlotItemOverlay.overlays` and add it back as `@property` around
the underlying `.layout.pitems`.
Factor and fix dst <- src pair parsing into a new func
`get_likely_pair()` and use throughout initial position loading; solves
a parsing bug for src asset balances which aren't only 3 chars long..
a terrible assumption.
Goes back to always adjusting the y-axis range to include the L1 spread
and clearing label in view whenever the last datum is also in view,
previously this was broken after reworking the display loop for
multi-feeds.
Drops a bunch of old commented tick looping cruft from before we started
using tick-type framing. Also adds more stringent guards for ignoring
but error logging quote values that are more then 25% out of range; it
seems particularly our `ib` feed has some issues with strange `price`
values that are way off here and there?
Instead of having the l1 lines be inside the view space, move them to be
inside their respective axis (with only a 16 unit portion inside the
view) such that the clear price label can overlay with them nicely
without obscuring; this is much better suited to multiple adjacent
y-axes and in general is simpler and less noisy.
Further `L1Labels` + `LevelLabel` style tweaks:
- adjust `.rect` positioning to be "right" (i.e. inside the parent
y-axis) with a slight 16 unit shift toward the viewbox (using the new
`._x_br_offset`) to allow seeing each level label's line even when the
clearing price label is positioned at that same level.
- add a newline's worth of vertical space to each of the bid/ask labels
so that L1 labels' text content isn't ever obscured by the clear price
label.
- set a low (10) z-value to ensure l1 labels are always placed
underneath the clear price label.
- always fill the label rect with the chosen background color.
- make labels fully opaque so as to always make them hide the parent
axes' `.tickStrings()` contents.
- make default color the "default" from the global scheme.
- drop the "price" part from the l1 label text contents, just show the
book-queue's amount (in dst asset's units, aka the potential clearing vlm).
In the case where the last-datum-graphic hasn't been created yet, simply
return a `None` from this method so the caller can choose to ignore the
output. Further, drop `.px_width()` since it makes more sense defined on
`Viz` as well as the previously commented `BarItems.x_uppx()` method.
Also, don't round the `.x_uppx()` output since it can then be used when
< 1 to do x-domain scaling during high zoom usage.
Factor some common methods into the parent type:
- `.x_uppx()` for reading the horizontal units-per-pixel.
- `.x_last()` for reading the "closest to y-axis" last datum coordinate
for zooming "around" during mouse interaction.
- `.px_width()` for computing the max width of any curve in view in
pixels.
Adjust all previous derived `pg.GraphicsObject` child types to now
inherit from this new parent and in particular enable proper `.x_uppx()`
support to `BarItems`.
Use proper uppx scaling when either of scaling the data to the x-domain
index-range or when the uppx is < 1 (now that we support it) such that
both the fast and slow chart always appropriately scale and offset to
the y-axis with the last datum graphic just adjacent to the order line
arrow markers.
Further this fixes the `.index_step()` calc to use the "earliest" 16
values to compute the expected sample step diff since the last set often
contained gaps due to start up race conditions and generated
unexpected/incorrect output.
Further this drops the `.curve_width_pxs()` method and replaces it with
`.px_width()`, taken from the graphics object API and instead returns
the pixel account for the whole view width instead of the
x-domain-data-range within the view.
Doesn't seem like we really need to handle the situation where the start
or stop input time stamps are outside the index range of the data since
the new binary search handling via `numpy.searchsorted()` covers this
case at minimal runtime cost and with an equally correct output. Allows
us to drop some other indexing endpoint internal variables as well.
We want the fast and slow chart to behave the same on calls to
`Viz.default_view()` so adjust the offset calc to make both work:
- just offset by the line len regardless of step / uppx
- add back the `should_line: bool` output from `render_bar_items()` (and
use it to set a new `ds_allowed: bool` guard variable) so that we can
bypass calling the m4 downsampler unless the bars have been switched
to the interpolation line graphic (which we normally required before
any downsampling of OHLC graphics data).
Further, this drops use of the `use_vr: bool` flag from all rendering
since we pretty much always use it by default.
Previously with array-int indexing we had to map the input x-domain
"indexes" passed to `DynamicDateAxis._indexes_to_timestr()`. In the
epoch-time indexing case we obviously don't need to lookup time stamps
from the underlying shm array and can instead just cast to `int` and
relay the values verbatim.
Further, this patch includes some style adjustments to `AxisLabel` to
better enable multi-feed chart overlays by avoiding L1 label clutter
when multiple y-axes are stacked adjacent:
- adjust the `Axis` typical max string to include a couple spaces suffix
providing for a bit more margin between side-by-side y-axes.
- make the default label (fill) color the "default" from the global
color scheme and drop it's opacity to .9
- add some new label placement options and use them in the
`.boundingRect()` method:
* `._x/y_br_offset` for relatively shifting the overall label relative
to it's parent axis.
* `._y_txt_h_scaling` for increasing the bounding rect's height
without including more whitespace in the label's text content.
- ensure labels have a high z-value such that by default they are always
placed "on top" such that when we adjust the l1 labels they can be set
to a lower value and thus never obscure the last-price label.
Turns out we were updating the wrong ``Viz``/``DisplayState`` inside the
closure style `increment_history_view()`` (probably due to looping
through the flumes and dynamically closing in that task-func).. Instead
define the history incrementer at module level and pass in the
`DisplayState` explicitly. Further rework the `DisplayState` attrs to be
more focused around the `Viz` associated with the fast and slow chart
and be sure to adjust output from each `Viz.incr_info()` call to latest
update. Oh, and just tweaked the line palette for the moment.
FYI "treading" here is referring to the x-shifting of the curve when
the last datum is in view such that on new sampled appends the "last"
datum is kept in the same x-location in UI terms.
Mainly it was the global (should we )increment logic that needs to be
independent for the fast vs. slow chart such that the slow isn't
update-shifted by the fast and vice versa. We do this using a new
`'i_last_slow'` key in the `DisplayState.globalz: dict` which is
singleton for each sample-rate-specific chart and works for both time
and array indexing.
Also, we drop some old commented `graphics.draw_last_datum()` code that
never ended up being needed again inside the coordinate cache reset
bloc.
Might as well since it makes the chart look less gappy and we can easily
flip the index switch now B)
Also adds a new `'i_slow_last'` key to `DisplayState` for a singleton
across all slow charts and thus no more need for special case logic in
`viz.incr_info()`.
Define the x-domain coords "offset" (determining the curve graphics
per-datum placement) for each formatter such that there's only on place
to change it when needed. Obviously each graphics type has it's own
dimensionality and this is reflected by the array shapes on each
subtype.
Previously we were drawing with the middle of the bar on each index with
arms to either side: +/- some arm length. Instead this changes so that
each bar is drawn *after* each index/timestamp such that in graphics
coords the bar span more correctly matches the time span in the
x-domain. This makes the linked region between slow and fast chart
directly match (without any transform) for epoch-time indexing such that
the last x-coord in view on the fast chart is no more then the
next time step in (downsampled) slow view.
Deats:
- adjust in `._pathops.path_arrays_from_ohlc()` and take an `bar_w` bar
width input (normally taken from the data step size).
- change `.ui._ohlc.bar_from_ohlc_row()` and
`BarItems.draw_last_datum()` to match.
Allows easily switching between normal array `int` indexing and time
indexing by just flipping the `Viz._index_field: str`.
Also, guard all the x-data audit breakpoints with a time indexing
condition.
Turned out to be super simple to get the first draft to work since the
fast and slow chart now use the same domain, however, it seems like
maybe there's an offset issue still where the fast may be a couple
minutes ahead of the slow?
Need to dig in a bit..
Using a global "last index step" (via module var) obviously has problems
when working with multiple feed sets in a single global app instance:
any separate feed-set will be incremented according to an app-global
index-step and thus won't correctly calc per-feed-set-step update info.
Impl deatz:
- drop `DisplayState.incr_info()` (since previously moved to `Viz`) and
call that method on each appropriate `Viz` instance where necessary;
further ensure the appropriate `DisplayState` instance is passed in to
each call and make sure to pass a `state: DisplayState`.
- add `DisplayState.hist_vars: dict` for history chart (sets) to
determine the per-feed (not set) current slow chart (time) step.
- add `DisplayState.globalz: dict` to house a common per-feed-set state
and use it inside the new `Viz.incr_info()` such that
a `should_increment: bool` can be returned and used by the display
loop to determine whether to x-shift the current chart.
Read the `Viz.index_step()` directly to avoid always reading 1 on the
slow chart; this was completely broken before and resulting in not
rendering the bars graphic on the slow chart until at a true uppx of
1 which obviously doesn't work for 60 width bars XD
Further cleanups to `._render` module:
- drop `array` output from `Renderer.render()`, `read_from_key` input
and fix type annot.
- drop `should_line`, `changed_to_line` and `render_kwargs` from
`render_baritems()` outputs and instead calc `should_redraw` logic
inside the func body and return as output.
First allocation vs. first "prepend" of source data to an xy `ndarray`
format **must be mutex** in order to avoid a double prepend.
Previously when both blocks were executed we'd end up with
a `.xy_nd_start` that was decremented (at least) twice as much as it
should be on the first `.format_to_1d()` call which is obviously
incorrect (and causes problems for m4 downsampling as discussed below).
Further, since the underlying `ShmArray` buffer indexing is managed
(i.e. write-updated) completely independently from the incremental
formatter updates and internal xy indexing, we can't use
`ShmArray._first.value` and instead need to use the particular `.diff()`
output's prepend length value to decrement the `.xy_nd_start` on updates
after initial alloc.
Problems this resolves with m4:
- m4 uses a x-domain diff to calculate the number of "frames" to
downsample to, this is normally based on the ratio of pixel columns on
screen vs. the size of the input xy data.
- previously using an int-index (not epoch time) the max diff between
first and last index would be the size of the input buffer and thus
would never cause a large mem allocation issue (though it may have
been inefficient in terms of needed size).
- with an epoch time index this max diff could explode if you had some
near-now epoch time stamp **minus** an x-allocation value: generally
some value in `[0.5, -0.5]` which would result in a massive frames and
thus internal `np.ndarray()` allocation causing either a crash in
`numba` code or actual system mem over allocation.
Further, put in some more x value checks that trigger breakpoints if we
detect values that caused this issue - we'll remove em after this has
been tested enough.
Turns out we can't seem to avoid the artefacts when click-drag-scrolling
(results in weird repeated "smeared" curve segments) so just go back to
the original code.
Ensures that a "last datum" graphics object exists so that zooming can
read it using `.x_last()`. Also, disable the linked region stuff for now
since it's totally borked after flipping to the time indexing.
Since we don't really need it defined on the "chart widget" move it to
a viz method and rework it to hell:
- always discard the invalid view l > r case.
- use the graphic's UPPX to determine UI-to-scene coordinate scaling for
the L1-label collision detection, if there is no L1 just offset by
a few (index step scaled) datums; this allows us to drop the 2x
x-range calls as was hacked previous.
- handle no-data-in-view cases explicitly and error if we get any
ostensibly impossible cases.
- expect caller to trigger a graphics cycle if needed.
Further support this includes a rework a slew of other important
details:
- add `Viz.index_step`, an idempotent computed, index (presumably uniform)
step value which is needed for variable sample rate graphics displayed
on an epoch (second) time index.
- rework `Viz.datums_range()` to pass view x-endpoints as first and last
elements in return `tuple`; tighten up snap-to-data edge case logic
using `max()`/`min()` calls and better internal var naming.
- adjust all calls to `slice_from_time()` to not expect an "abs" slice.
- drop all `.yrange` resetting since we can just have the `Renderer` do
it when necessary.
If we presume that time indexing using a uniform step we can calculate
the exact index (using `//`) for the input time presuming the data
set has zero gaps. This gives a massive speedup over `numpy` fancy
indexing and (naive) `numba` iteration. Further in the case where time
gaps are detected, we can use `numpy.searchsorted()` to binary search
for the nearest expected index at lower latency.
Deatz,
- comment-disable the call to the naive `numba` scan impl.
- add a optional `step: int` input (calced if not provided).
- add todos for caching binary search results in the gap detection
cases.
- drop returning the "absolute buffer indexing" slice since the caller
can always just use the read-relative slice to acquire it.
When we use an epoch index and any sample rate > 1s we need to scale the
"number of bars" to that step in order to place the view correctly in
x-domain terms. For now we're calcing the step in-method but likely,
longer run, we'll pull this from elsewhere (like a ``Viz`` attr).
Gives approx a 3-4x speedup using plain old iterate-with-for-loop style
though still not really happy with this .5 to 1 ms latency..
Move the core `@njit` part to a `_slice_from_time()` with a pure python
func with orig name around it. Also, drop the output `mask` array since
we can generally just use the slices in the caller to accomplish the
same input array slicing, duh..
We need to subtract the first index in the array segment read, not the
first index value in the time-sliced output, to get the correct offset
into the non-absolute (`ShmArray.array` read) array..
Further we **do** need the `&` between the advance indexing conditions
and this adds profiling to see that it is indeed real slow (like 20ms
ish even when using `np.where()`).
Again, to make epoch indexing a flip-of-switch for testing look up the
`Viz.index_field: str` value when updating labels.
Also, drops the legacy tick-type set tracking which we no longer use
thanks to the new throttler subsys and it's framing msgs.
Planning to put the formatters into a new mod and aggregate all path
gen/op helpers into this module.
Further tweak include:
- moving `path_arrays_from_ohlc()` back to module level
- slice out the last xy datum for `OHLCBarsAsCurveFmtr` 1d formatting
- always copy the new x-value from the source to `.x_nd`
This was a major cause of error (particularly trying to get epoch
indexing working) and really isn't necessary; instead just have
`.diff()` always read from the underlying source array for current
index-step diffing and append/prepend slice construction.
Allows us to,
- drop `._last_read` state management and thus usage.
- better handle startup indexing by setting `.xy_nd_start/stop` to
`None` initially so that the first update can be done in one large
prepend.
- better understand and document the step curve "slice back to previous
level" logic which is now heavily commented B)
- drop all the `slice_to_head` stuff from and instead allow each
formatter to choose it's 1d segmenting.
In an effort to make it easy to override the indexing scheme.
Further, this repairs the `.datums_range()` special case to handle when
the view box is to-the-right-of the data set (i.e. l > datum_start).
Tyler Goodlet
Guillermo Rodriguez
jaredgoldman
algorandpa
Guillermo Rodriguez
Esmeralda Gallardo
goodboy