128 lines
5.1 KiB
Markdown
128 lines
5.1 KiB
Markdown
# Chart-Local Gap Overlays
|
|
|
|
## Goal
|
|
|
|
Make gap overlays a chart-actor-owned display service while preserving
|
|
remote annotation as a first-class API. Local startup and UI input avoid
|
|
an unnecessary IPC round trip; remote tools use a strict `msgspec` schema
|
|
and the same renderer/controller.
|
|
|
|
## Ownership Model
|
|
|
|
`GapOverlayMngr` is created once in the chart actor and references the
|
|
actor-global `_dss` and `_annots` registries. Each layer key is
|
|
`(owner, id(chart), fqme, timeframe)`. The chart identity keeps duplicate
|
|
FQMEs in separate cached displays independent. The reserved `chart-local`
|
|
owner is also independent from each remote `Context.cid`, so remote context
|
|
teardown cannot remove locally owned graphics.
|
|
|
|
The controller has one rendering entry point:
|
|
|
|
1. `GapOverlayMngr.apply(SetGapOverlay)` replaces an owner layer.
|
|
2. `GapOverlayMngr.refresh()` detects gaps from chart-local SHM and calls
|
|
`apply()` directly.
|
|
3. `AnnotClient.set_gap_overlay()` sends the same request through the
|
|
strict `remote_gap_overlays()` context.
|
|
|
|
The generic `remote_annotate()` context and its rectangle, arrow, text,
|
|
batch, remove and redraw commands remain unchanged.
|
|
|
|
## Startup And UI
|
|
|
|
`graphics_update_loop()` registers each new `DisplayState` in `_dss`, then
|
|
builds the default 60-second local overlay. The focused `ChartView` handles
|
|
`Ctrl+G` through its existing asynchronous keyboard loop. It chooses the
|
|
focused chart's 1-second or 60-second timeframe and calls
|
|
`GapOverlayMngr.toggle()` directly.
|
|
|
|
Gap detection reads the structured OHLCV array, excludes invalid timestamps,
|
|
uses one `numpy.diff()` pass, and creates Python `GapSpec` objects only for
|
|
actual positive timestamp gaps.
|
|
|
|
## Typed IPC
|
|
|
|
`remote_gap_overlays()` is separate from `remote_annotate()` because
|
|
`msgspec` cannot strictly decode a union containing both arbitrary legacy
|
|
dicts and several struct types. Its Tractor payload specification is:
|
|
|
|
```python
|
|
SetGapOverlay | GapOverlay | list[str] | None
|
|
```
|
|
|
|
`SetGapOverlay` and `GapOverlay` are tagged structs. Tractor validates
|
|
started, stream and return payloads against this union. A lock shared by
|
|
all FQMEs on one typed stream serializes each send/receive exchange. Each
|
|
side narrows its public `Context.pld_rx` decoder to the directional request
|
|
or response type while receiving. Request IDs let the client discard a late
|
|
response after an earlier timeout instead of skewing later request/reply
|
|
pairs.
|
|
|
|
## Refresh Semantics
|
|
|
|
Startup and every UI transition from hidden to visible recompute the layer
|
|
from current SHM. Existing redraw handling repositions matching overlays by
|
|
both FQME and timeframe. Rectangle and arrow geometry update together.
|
|
|
|
The next production step is to trigger `refresh()` from an explicit history
|
|
revision/publication event after a repair or prepend. Polling from the quote
|
|
display loop is intentionally excluded.
|
|
|
|
Request/reply serialization and stale-response correlation can move into a
|
|
generic IPC dialog helper after a second typed consumer establishes the
|
|
reusable interface. The legacy annotation dict protocol should first migrate
|
|
to tagged messages before sharing the strict gap context.
|
|
|
|
## Landing Order
|
|
|
|
The existing ancestry is:
|
|
|
|
```text
|
|
main
|
|
gap_annotator
|
|
datad_service
|
|
backfiller_deep_fixes
|
|
wkt/fix_broadcast_consumers
|
|
```
|
|
|
|
The recommended integration order is:
|
|
|
|
1. Review and land the complete 23-commit `main..gap_annotator` range.
|
|
2. Land Tractor PR 490 after its PR 475 dependency, then update the piker
|
|
Tractor reference used by `datad_service`.
|
|
3. Land `datad_service` on the new mainline.
|
|
4. Land the smaller four-commit broadcast-consumer PR 92.
|
|
5. Rebase `backfiller_deep_fixes` onto the broadcast result and run the
|
|
NativeDB/SHM/manual chart qualification sequence.
|
|
6. Replay the chart-local overlay patch above the qualified backfiller tip.
|
|
|
|
This order puts the already-reviewed ownership fix ahead of the broader
|
|
backfiller qualification. The only known overlapping production path is
|
|
`piker/brokers/ib/feed.py`; `git merge-tree --write-tree` reports no current
|
|
textual conflict, but behavior must still be rechecked.
|
|
|
|
No branch movement, rebase, merge, commit or push is part of this prototype.
|
|
|
|
## Verification
|
|
|
|
Run from the prototype worktree with the source-resolving Python environment:
|
|
|
|
```text
|
|
python -m pytest -p no:xonsh -q -x --tb=short --no-header \
|
|
tests/test_gap_overlays.py
|
|
python -m ruff check piker/ui/_gaps.py \
|
|
tests/test_gap_overlays.py piker/ui/_remote_ctl.py \
|
|
piker/ui/_display.py piker/ui/_interaction.py \
|
|
piker/ui/_widget.py piker/ui/_annotate.py
|
|
```
|
|
|
|
Manual chart qualification must confirm startup rendering, focused-timeframe
|
|
`Ctrl+G`, remote typed replacement, remote disconnect cleanup, cached symbol
|
|
switching and refresh after backfill repair.
|
|
|
|
Automated integration coverage uses real offscreen Qt plots and a real
|
|
Tractor child actor. It verifies manager insertion, replacement, hiding and
|
|
owner cleanup; prepend repositioning; duplicate-FQME chart identity; startup
|
|
registration; Qt Ctrl-G event routing; typed endpoint/client exchange; and
|
|
cancellation with stale-reply recovery. A broker-backed qtractor session and
|
|
history-revision-triggered refresh remain manual qualification boundaries.
|