Own gap overlays in chart-local `GapOverlayMngr`

Detect positive timestamp gaps from chart OHLCV SHM with one
vectorized pass. Route local UI and remote callers through one
`GapOverlayMngr` renderer, and key each layer by owner, chart,
FQME and timeframe so remote ctx teardown cannot remove local
graphics.

Deats,
- add tagged `SetGapOverlay` requests and typed
  `GapOverlay` responses;
- keep `remote_annotate()` intact and serve its typed dialog from
  a dedicated `remote_gap_overlays()` Tractor ctx;
- admit `SetGapOverlay`, `GapOverlay`, `list[str]` and `None` in
  the Tractor payload spec, then strictly narrow requests to
  `SetGapOverlay` and replies to `GapOverlay`;
- serialize shared-stream dialogs and correlate request IDs so
  stale replies cannot skew later exchanges;
- rename `AnnotCtl` / `open_annot_ctl()` to `AnnotClient` /
  `open_annot_client()` across storage and annotation callers;
- enable 60s history overlays at startup and toggle the focused
  1s or 60s chart directly with `Ctrl+G`;
- guard redraws by chart, FQME and timeframe; tolerate empty
  arrays and out-of-range lookups; update rect and arrow geometry;
- cover detector geometry, msgpack roundtrips, schema rejection,
  typed stale-FQME errors and renamed storage seams;
- document ownership, refresh, stack landing and live
  qualification.

Prompt-IO: ai/prompt-io/opencode/20260814T032716Z_2aa66c2c_prompt_io.md

(this patch was generated in some part by `opencode` using `gpt-5.6-sol` (`openai`))

ai/prompt-io/opencode/20260814T032716Z_2aa66c2c_prompt_io.md

@@ -0,0 +1,43 @@
+---
+model: openai/gpt-5.6-sol
+service: opencode
+session: unavailable
+timestamp: 2026-08-14T03:27:16Z
+git_ref: 2aa66c2c
+scope: code
+substantive: true
+raw_file: 20260814T032716Z_2aa66c2c_prompt_io.raw.md
+---
+
+## Prompt
+
+Prototype chart-actor-owned gap overlays from the current backfiller tip,
+with actor-local startup and UI toggling. Preserve remote annotation IPC and
+make the gap-specific path strict and `msgspec`-typed.
+
+## Response summary
+
+Adds a chart-local controller and vectorized SHM detector, starts historical
+gap overlays with display state, toggles the focused timeframe through the
+existing input loop and exposes the same renderer through a strict Tractor
+payload context. Keeps all legacy remote annotation commands and separates
+local graphics from each remote context's teardown ownership.
+
+## Files changed
+
+- `piker/ui/_gaps.py` - typed messages, detection and local controller.
+- `piker/ui/_remote_ctl.py` - IPC clients and chart-actor endpoints.
+- `piker/ui/_display.py` - actor-local startup integration.
+- `piker/ui/_interaction.py` - focused `Ctrl+G` toggle.
+- `piker/ui/_widget.py` - controller reference.
+- `piker/ui/_annotate.py` - safe FQME-aware repositioning.
+- `piker/tsp/_annotate.py` - typed annotation-client reference.
+- `piker/storage/cli.py` - renamed annotation client context.
+- `tests/test_gap_overlays.py` - detector and schema regression.
+- `tests/test_ldshm.py` - renamed annotation-client test seams.
+- `plans/opencode/chart-local-gap-overlays.md` - architecture and landing plan.
+
+## Human edits
+
+None - the work remains an uncommitted disposable-worktree prototype for
+human review and live chart qualification.

ai/prompt-io/opencode/20260814T032716Z_2aa66c2c_prompt_io.raw.md

@@ -0,0 +1,72 @@
+---
+model: openai/gpt-5.6-sol
+service: opencode
+timestamp: 2026-08-14T03:27:16Z
+git_ref: 2aa66c2c
+diff_cmd: git diff HEAD~1..HEAD
+---
+
+## Prompt
+
+The user asked what work had been done so far, then authorized autonomous
+work while AFK. The target was to scheme and prototype an actor-local gap
+annotator from the `backfiller_deep_fixes` tip without mutating existing
+branches. The user then clarified twice that the annotation IPC API must
+remain available and should become more rigorous and `msgspec`-typed rather
+than being removed.
+
+## Generated code
+
+> `git diff HEAD~1..HEAD -- piker/ui/_gaps.py`
+
+Adds tagged gap request/reply structs, render-ready gap specs, vectorized
+OHLCV timestamp-gap detection and chart-local overlay ownership.
+
+> `git diff HEAD~1..HEAD -- piker/ui/_remote_ctl.py`
+
+Adds strict Tractor gap IPC, `AnnotClient` stream mapping/locking and
+owner-scoped endpoint cleanup while retaining the legacy annotation endpoint.
+
+> `git diff HEAD~1..HEAD -- piker/ui/_display.py`
+
+Starts the local 60-second overlay after display-state registration.
+
+> `git diff HEAD~1..HEAD -- piker/ui/_interaction.py`
+
+Routes `Ctrl+G` from the focused chart's existing async input loop directly
+to the local controller.
+
+> `git diff HEAD~1..HEAD -- piker/ui/_widget.py`
+
+Stores the chart actor's local controller reference.
+
+> `git diff HEAD~1..HEAD -- piker/ui/_annotate.py`
+
+Keeps redraw repositioning isolated by FQME and synchronizes arrow geometry.
+
+> `git diff HEAD~1..HEAD -- piker/tsp/_annotate.py piker/storage/cli.py`
+
+Renames the remote annotation control type and context manager to the
+client-oriented `AnnotClient` / `open_annot_client` interface.
+
+> `git diff HEAD~1..HEAD -- tests/test_gap_overlays.py`
+
+Adds deterministic detector, directional payload-schema, typed error and
+msgpack roundtrip regressions.
+
+> `git diff HEAD~1..HEAD -- tests/test_ldshm.py`
+
+Updates storage CLI test seams for the renamed annotation client context.
+
+## Generated architecture output
+
+> `git diff HEAD~1..HEAD -- plans/opencode/chart-local-gap-overlays.md`
+
+Documents ownership, startup, UI, strict IPC, refresh and branch landing
+semantics.
+
+## Verification output
+
+Source resolution points to this worktree. The targeted regression passes,
+Ruff reports no changed-file findings, `compileall` succeeds, `git diff
+--check` succeeds and Tractor's strict gap payload decoder initializes.

piker/storage/cli.py

@@ -62,7 +62,7 @@ from .nativedb import (
 )
 
 if TYPE_CHECKING:
-    from piker.ui._remote_ctl import AnnotCtl
+    from piker.ui._remote_ctl import AnnotClient
 
 
 store = typer.Typer(no_args_is_help=True)
@@ -940,9 +940,9 @@ def shm(
 
     async def main() -> bool:
         from piker.ui._remote_ctl import (
-            open_annot_ctl,
+            open_annot_client,
         )
-        actl: AnnotCtl
+        actl: AnnotClient
         mod: ModuleType
         client: StorageClient
         async with (
@@ -955,7 +955,7 @@ def shm(
                 mod,
                 client,
             ),
-            open_annot_ctl() as actl,
+            open_annot_client() as actl,
         ):
             shm_df: pl.DataFrame | None = None
             tf2aids: dict[float, dict] = {}

piker/tsp/_annotate.py

@@ -38,7 +38,7 @@ from piker.toolz.profile import (
 from piker.ui._style import get_fonts
 
 if TYPE_CHECKING:
-    from piker.ui._remote_ctl import AnnotCtl
+    from piker.ui._remote_ctl import AnnotClient
 
 
 def humanize_duration(
@@ -89,7 +89,7 @@ def humanize_duration(
 async def markup_gaps(
     fqme: str,
     timeframe: float,
-    actl: AnnotCtl,
+    actl: AnnotClient,
     wdts: pl.DataFrame,
     gaps: pl.DataFrame,
 

piker/ui/_annotate.py

@@ -581,6 +581,15 @@ class GapAnnotations(GraphicsObject):
         '''
         # skip reposition if timeframe doesn't match
         # (e.g., 1s gaps being repositioned with 60s array)
+        if (
+            fqme is not None
+            and
+            self._fqme is not None
+            and
+            fqme != self._fqme
+        ):
+            return
+
         if (
             timeframe is not None
             and
@@ -605,6 +614,13 @@ class GapAnnotations(GraphicsObject):
             )
             return
 
+        if not len(array):
+            log.warning(
+                'GapAnnotations.reposition() called with an empty '
+                'array'
+            )
+            return
+
         # collect all unique timestamps we need to lookup
         timestamps: set[float] = set()
         for spec in self._gap_specs:
@@ -628,9 +644,14 @@ class GapAnnotations(GraphicsObject):
             )
 
             # vectorized bounds check and exact match verification
-            valid_mask = (
+            in_bounds = search_indices < len(array)
-                (search_indices < len(array))
+            valid_mask = np.zeros(
-                & (time_arr[search_indices] == ts_array)
+                len(search_indices),
+                dtype=bool,
+            )
+            valid_mask[in_bounds] = (
+                time_arr[search_indices[in_bounds]]
+                == ts_array[in_bounds]
             )
 
             valid_indices = search_indices[valid_mask]
@@ -702,18 +723,30 @@ class GapAnnotations(GraphicsObject):
         # rebuild arrow path with new indices
         self._arrow_path.clear()
 
+        spec: dict
         for spec in self._gap_specs:
-            time_val = spec.get('time')
+            time_val: float|None = spec.get('time')
             if time_val is None:
                 continue
 
-            arrow_row = time_to_row.get(time_val)
+            arrow_row: dict[str, float]|None = time_to_row.get(
+                time_val,
+            )
             if arrow_row is None:
                 continue
 
-            arrow_x = arrow_row['index']
+            arrow_x: float = arrow_row['index']
-            arrow_y = arrow_row['close']
+            start_row: dict[str, float]|None = time_to_row.get(
-            pointing = spec['pointing']
+                spec.get('start_time'),
+            )
+            arrow_y: float = (
+                start_row['close']
+                if start_row is not None
+                else spec['arrow_y']
+            )
+            pointing: str = spec['pointing']
+            spec['arrow_x'] = arrow_x
+            spec['arrow_y'] = arrow_y
 
             # create arrow polygon
             if pointing == 'down':

piker/ui/_display.py

@@ -78,6 +78,11 @@ from ._forms import (
     FieldsForm,
     mk_order_pane_layout,
 )
+from ._gaps import (
+    GapOverlay,
+    GapOverlayMngr,
+    HIST_GAP_TIMEFRAME,
+)
 from . import _pg_overrides as pgo
 from .order_mode import (
     open_order_mode,
@@ -487,6 +492,25 @@ async def graphics_update_loop(
     # prior view..
     from . import _remote_ctl
     _remote_ctl._dss.update(dss)
+    gapman: GapOverlayMngr = _remote_ctl._gapman
+    godwidget.gapman = gapman
+    gap_timeframe: float = HIST_GAP_TIMEFRAME
+
+    # TODO: expose startup visibility and enabled timeframes through
+    # the UI config once chart periods are configurable.
+    fqme: str
+    ds: DisplayState
+    for fqme, ds in dss.items():
+        gap: GapOverlay = gapman.refresh(
+            ds=ds,
+            timeframe=gap_timeframe,
+        )
+        log.info(
+            'Chart-local gap overlay ready:\n'
+            f'fqme: {fqme}\n'
+            f'timeframe: {gap_timeframe}s\n'
+            f'gaps: {gap.gap_count}\n'
+        )
 
     # main real-time quotes update loop
     stream: tractor.MsgStream

piker/ui/_gaps.py

@@ -0,0 +1,413 @@
+# piker: trading gear for hackers
+# Copyright (C) Tyler Goodlet (in stewardship for pikers)
+
+# This program is free software: you can redistribute it and/or
+# modify it under the terms of the GNU Affero General Public License
+# as published by the Free Software Foundation, either version 3
+# of the License, or (at your option) any later version.
+
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU Affero General Public License for more details.
+
+# You should have received a copy of the GNU Affero General Public
+# License along with this program. If not, see
+# <https://www.gnu.org/licenses/>.
+
+'''
+Chart-local gap-overlay detection, rendering and IPC messages.
+
+'''
+from __future__ import annotations
+from typing import (
+    Literal,
+    TYPE_CHECKING,
+)
+
+import numpy as np
+
+from piker.types import Struct
+from ._annotate import GapAnnotations
+
+if TYPE_CHECKING:
+    from piker.ui.qt import QGraphicsItem
+    from PyQt6.QtWidgets import QGraphicsScene
+    from ._chart import ChartPlotWidget
+    from ._dataviz import Viz
+    from ._display import DisplayState
+
+
+# `DisplayState` currently exposes exactly one realtime and one
+# historical OHLC chart at these sampling periods.
+RT_GAP_TIMEFRAME: float = 1.
+HIST_GAP_TIMEFRAME: float = 60.
+
+
+class GapSpec(Struct, frozen=True):
+    '''
+    Render-ready geometry for one OHLC time gap.
+
+    '''
+    start_pos: tuple[float, float]
+    end_pos: tuple[float, float]
+    arrow_x: float
+    arrow_y: float
+    pointing: Literal['up', 'down']
+    start_time: float
+    end_time: float
+    time: float
+    color: str = 'dad_blue'
+    alpha: int = 169
+
+
+class SetGapOverlay(Struct, frozen=True, tag=True):
+    '''
+    Request replacement of one named gap-overlay layer.
+
+    `GapOverlayMngr.apply()` accepts this type locally and through
+    the chart actor's typed IPC endpoint. The receiver assigns
+    ownership so remote teardown cannot remove chart-local overlays.
+
+    '''
+    fqme: str
+    timeframe: float
+    specs: list[GapSpec]
+    visible: bool = True
+    request_id: str = ''
+
+
+class GapOverlay(Struct, frozen=True, tag=True):
+    '''
+    Gap-overlay state returned after applying `SetGapOverlay`.
+
+    '''
+    fqme: str
+    timeframe: float
+    visible: bool
+    gap_count: int
+    aid: int|None = None
+    error: str|None = None
+    request_id: str = ''
+
+
+GapOverlayPayload = (
+    SetGapOverlay
+    | GapOverlay
+    | list[str]
+    | None
+)
+
+
+def gap_specs_from_ohlcv(
+    array: np.ndarray,
+    period_s: float,
+
+) -> list[GapSpec]:
+    '''
+    Build render specs for positive timestamp gaps in sorted OHLCV.
+
+    Invalid and non-positive timestamps are excluded. Timestamp steps
+    are detected in one vectorized pass; only actual gaps enter the
+    small Python object-construction loop.
+
+    '''
+    if len(array) < 2:
+        return []
+
+    times: np.ndarray = array['time'].astype(float, copy=False)
+    valid: np.ndarray = (
+        np.isfinite(times)
+        & (times > 0)
+    )
+    rows: np.ndarray = array[valid]
+    if len(rows) < 2:
+        return []
+
+    times = rows['time'].astype(float, copy=False)
+    gap_ends: np.ndarray = np.flatnonzero(
+        np.diff(times) > period_s
+    ) + 1
+    if not len(gap_ends):
+        return []
+
+    gap_starts: np.ndarray = gap_ends - 1
+    left: np.ndarray = rows[gap_starts]
+    right: np.ndarray = rows[gap_ends]
+
+    start_xs: np.ndarray = (
+        left['index'].astype(float, copy=False)
+        + 0.9
+    )
+    end_xs: np.ndarray = (
+        right['index'].astype(float, copy=False)
+        + 0.1
+    )
+    start_ys: np.ndarray = left['close'].astype(
+        float,
+        copy=False,
+    )
+    end_ys: np.ndarray = right['open'].astype(
+        float,
+        copy=False,
+    )
+
+    specs: list[GapSpec] = []
+    start_x: np.floating
+    end_x: np.floating
+    start_y: np.floating
+    end_y: np.floating
+    start_time: np.floating
+    end_time: np.floating
+    for (
+        start_x,
+        end_x,
+        start_y,
+        end_y,
+        start_time,
+        end_time,
+    ) in zip(
+        start_xs,
+        end_xs,
+        start_ys,
+        end_ys,
+        left['time'],
+        right['time'],
+        strict=True,
+    ):
+        specs.append(GapSpec(
+            start_pos=(float(start_x), float(start_y)),
+            end_pos=(float(end_x), float(end_y)),
+            arrow_x=float(end_x - 0.1),
+            arrow_y=float(start_y),
+            pointing=(
+                'down'
+                if start_y > end_y
+                else 'up'
+            ),
+            start_time=float(start_time),
+            end_time=float(end_time),
+            time=float(end_time),
+        ))
+
+    return specs
+
+
+class GapOverlayMngr:
+    '''
+    Own named gap-overlay layers inside one chart actor.
+
+    Local UI calls and remote annotation IPC both dispatch typed
+    `SetGapOverlay` messages through `GapOverlayMngr.apply()`.
+
+    '''
+    def __init__(
+        self,
+        dss: dict[str, DisplayState],
+        annots: dict[int, QGraphicsItem],
+
+    ) -> None:
+        self.dss: dict[str, DisplayState] = dss
+        self.annots: dict[int, QGraphicsItem] = annots
+        self._layers: dict[
+            tuple[  # owner-scoped layer key
+                str,  # owner (`chart-local` or `Context.cid`)
+                int,  # `id(ChartPlotWidget)`
+                str,  # fqme
+                float,  # timeframe
+            ],
+            int,  # `id(GapAnnotations)`
+        ] = {}
+        self._visible: dict[
+            tuple[  # actor-local `SetGapOverlay` key
+                int,  # `id(ChartPlotWidget)`
+                str,  # fqme
+                float,  # timeframe
+            ],
+            bool,  # `SetGapOverlay.visible`
+        ] = {}
+
+    def apply(
+        self,
+        req: SetGapOverlay,
+        owner: str = 'chart-local',
+        aids: set[int]|None = None,
+        ds: DisplayState|None = None,
+
+    ) -> GapOverlay:
+        '''
+        Replace one owner-scoped batch layer from a typed request.
+
+        '''
+        if ds is None:
+            ds = self.dss.get(req.fqme)
+            if ds is None:
+                return GapOverlay(
+                    fqme=req.fqme,
+                    timeframe=req.timeframe,
+                    visible=False,
+                    gap_count=0,
+                    request_id=req.request_id,
+                    error=(
+                        f'No display state for fqme={req.fqme}'
+                    ),
+                )
+        try:
+            chart: ChartPlotWidget = {
+                HIST_GAP_TIMEFRAME: ds.hist_chart,
+                RT_GAP_TIMEFRAME: ds.chart,
+            }[req.timeframe]
+        except KeyError:
+            return GapOverlay(
+                fqme=req.fqme,
+                timeframe=req.timeframe,
+                visible=False,
+                gap_count=0,
+                request_id=req.request_id,
+                error=(
+                    f'No chart for timeframe={req.timeframe}s'
+                ),
+            )
+
+        layer_key: tuple[str, int, str, float] = (
+            owner,
+            id(chart),
+            req.fqme,
+            req.timeframe,
+        )
+        aid: int|None = self._layers.pop(layer_key, None)
+        if aid is not None:
+            annot: QGraphicsItem|None = self.annots.pop(
+                aid,
+                None,
+            )
+            if aids is not None:
+                aids.discard(aid)
+            if annot is not None:
+                scene: QGraphicsScene|None = annot.scene()
+                if scene:
+                    scene.removeItem(annot)
+
+        if owner == 'chart-local':
+            self._visible[
+                (id(chart), req.fqme, req.timeframe)
+            ] = req.visible
+        if (
+            not req.visible
+            or not req.specs
+        ):
+            return GapOverlay(
+                fqme=req.fqme,
+                timeframe=req.timeframe,
+                visible=req.visible,
+                gap_count=len(req.specs),
+                request_id=req.request_id,
+            )
+
+        viz: Viz = chart.get_viz(req.fqme)
+        gap_specs: list[dict] = []
+        spec: GapSpec
+        for spec in req.specs:
+            gap_specs.append(spec.to_dict())
+        gaps_item: GapAnnotations = GapAnnotations(
+            gap_specs=gap_specs,
+            array=viz.shm.array,
+            color=req.specs[0].color,
+            alpha=req.specs[0].alpha,
+            arrow_size=10,
+            fqme=req.fqme,
+            timeframe=req.timeframe,
+        )
+        viz.plot.addItem(gaps_item)
+        gaps_item._chart = chart
+
+        new_aid: int = id(gaps_item)
+        self.annots[new_aid] = gaps_item
+        self._layers[layer_key] = new_aid
+        if aids is not None:
+            aids.add(new_aid)
+        return GapOverlay(
+            fqme=req.fqme,
+            timeframe=req.timeframe,
+            visible=True,
+            gap_count=len(req.specs),
+            aid=new_aid,
+            request_id=req.request_id,
+        )
+
+    def refresh(
+        self,
+        ds: DisplayState,
+        timeframe: float,
+        visible: bool = True,
+
+    ) -> GapOverlay:
+        '''
+        Detect gaps from chart-local SHM and replace the local layer.
+
+        '''
+        viz: Viz = {
+            HIST_GAP_TIMEFRAME: ds.hist_viz,
+            RT_GAP_TIMEFRAME: ds.viz,
+        }[timeframe]
+        specs: list[GapSpec] = gap_specs_from_ohlcv(
+            viz.shm.array,
+            period_s=timeframe,
+        )
+        return self.apply(SetGapOverlay(
+            fqme=ds.fqme,
+            timeframe=timeframe,
+            specs=specs,
+            visible=visible,
+        ), ds=ds)
+
+    def toggle(
+        self,
+        ds: DisplayState,
+        timeframe: float,
+
+    ) -> GapOverlay:
+        '''
+        Toggle and refresh one chart-local gap-overlay layer.
+
+        '''
+        chart: ChartPlotWidget = {
+            HIST_GAP_TIMEFRAME: ds.hist_chart,
+            RT_GAP_TIMEFRAME: ds.chart,
+        }[timeframe]
+        visible: bool = not self._visible.get(
+            (id(chart), ds.fqme, timeframe),
+            False,
+        )
+        return self.refresh(
+            ds=ds,
+            timeframe=timeframe,
+            visible=visible,
+        )
+
+    def remove_owner(
+        self,
+        owner: str,
+
+    ) -> None:
+        '''
+        Remove all graphics and bookkeeping for one IPC owner.
+
+        '''
+        owner_keys: list[tuple[str, int, str, float]] = []
+        key: tuple[str, int, str, float]
+        for key in self._layers:
+            if key[0] == owner:
+                owner_keys.append(key)
+
+        layer_key: tuple[str, int, str, float]
+        for layer_key in owner_keys:
+            aid: int = self._layers.pop(layer_key)
+            annot: QGraphicsItem|None = self.annots.pop(
+                aid,
+                None,
+            )
+            if annot is not None:
+                scene: QGraphicsScene|None = annot.scene()
+                if scene:
+                    scene.removeItem(annot)

piker/ui/_interaction.py

@@ -61,6 +61,12 @@ from ..toolz import (
 from .view_mode import overlay_viewlists
 # from ._style import _min_points_to_show
 from ._editors import SelectRect
+from ._gaps import (
+    GapOverlay,
+    GapOverlayMngr,
+    HIST_GAP_TIMEFRAME,
+    RT_GAP_TIMEFRAME,
+)
 from . import _event
 
 if TYPE_CHECKING:
@@ -225,6 +231,35 @@ async def handle_viewmode_kb_inputs(
                         # path necessary?
                         viz.reset_graphics()
 
+            # Ctrl-G is the default chart-local gap-overlay toggle.
+            # TODO: expose this binding through the UI config.
+            if (
+                ctrl
+                and
+                key == Qt.Key_G
+            ):
+                gapman: GapOverlayMngr|None = godw.gapman
+                if gapman is not None:
+                    fqme: str
+                    ds: DisplayState
+                    for fqme, ds in dss.items():
+                        timeframe: float = (
+                            HIST_GAP_TIMEFRAME
+                            if view._chart is ds.hist_chart
+                            else RT_GAP_TIMEFRAME
+                        )
+                        gap: GapOverlay = gapman.toggle(
+                            ds=ds,
+                            timeframe=timeframe,
+                        )
+                        log.info(
+                            'Toggled chart-local gap overlay:\n'
+                            f'fqme: {fqme}\n'
+                            f'timeframe: {timeframe}s\n'
+                            f'visible: {gap.visible}\n'
+                            f'gaps: {gap.gap_count}\n'
+                        )
+
             # ------ - ------
             # SEARCH MODE
             # ------ - ------

piker/ui/_remote_ctl.py

@@ -61,6 +61,12 @@ from ._editors import (
 from ._chart import ChartPlotWidget
 from ._dataviz import Viz
 from ._style import hcolor
+from ._gaps import (
+    GapOverlay,
+    GapOverlayMngr,
+    GapOverlayPayload,
+    SetGapOverlay,
+)
 
 log = get_logger(__name__)
 
@@ -96,6 +102,11 @@ EditorsTable = dict[int, ArrowEditor]
 _annots: AnnotsTable  = {}
 _editors: EditorsTable = {}
 
+_gapman = GapOverlayMngr(
+    dss=_dss,
+    annots=_annots,
+)
+
 def rm_annot(
     annot: ArrowEditor|SelectRect|pg.TextItem
 ) -> bool:
@@ -155,6 +166,13 @@ def no_qt_updates(*items):
                 item.setUpdatesEnabled(True)
 
 
+# TODO XXX: TYPE THE ENTIRE ANNOTATION IPC API.
+# `serve_rc_annots()` still dispatches arbitrary dicts from one large
+# match block. Move each operation to tagged request/response
+# structs, narrow each receive phase with
+# `Context.pld_rx.limit_plds()`, and then reassess whether
+# `remote_gap_overlays()` should fold back into the general annotation
+# endpoint.
 async def serve_rc_annots(
     ipc_key: str,
     annot_req_stream: MsgStream,
@@ -169,6 +187,7 @@ async def serve_rc_annots(
 
     '''
     global _editors
+    msg: dict
     async for msg in annot_req_stream:
         match msg:
             case {
@@ -684,7 +703,8 @@ async def serve_rc_annots(
                         fqme=fqme,
                         timeframe=timeframe,
                     )
-                    chart.plotItem.addItem(gaps_item)
+                    gaps_item._chart = chart
+                    viz.plot.addItem(gaps_item)
 
                     # register single item for repositioning
                     aid: int = id(gaps_item)
@@ -839,6 +859,8 @@ async def serve_rc_annots(
                 for aid, annot in annots.items():
                     # GapAnnotations batch items have .reposition()
                     if hasattr(annot, 'reposition'):
+                        if annot._chart is not chart:
+                            continue
                         annot.reposition(
                             array=arr,
                             fqme=fqme,
@@ -928,17 +950,57 @@ async def remote_annotate(
             profiler(f'removed all {len(aids)} annotations')
 
 
-class AnnotCtl(Struct):
+@tractor.context(pld_spec=GapOverlayPayload)
+async def remote_gap_overlays(
+    ctx: Context,
+
+) -> None:
     '''
-    A control for remote "data annotations".
+    Strictly typed IPC endpoint for batched gap-overlay control.
+
+    The generic `remote_annotate()` endpoint remains available for
+    legacy rectangle, arrow and text commands.
+
+    '''
+    if not _dss:
+        raise RuntimeError(
+            'Chart display state is not initialized'
+        )
+
+    owner: str = ctx.cid
+    aids: set[int] = set()
+    await ctx.started(list(_dss))
+    try:
+        stream: MsgStream
+        async with ctx.open_stream() as stream:
+            with ctx.pld_rx.limit_plds(
+                spec=SetGapOverlay,
+            ):
+                req: SetGapOverlay
+                async for req in stream:
+                    resp: GapOverlay = _gapman.apply(
+                        req=req,
+                        owner=owner,
+                        aids=aids,
+                    )
+                    await stream.send(resp)
+    finally:
+        _gapman.remove_owner(owner)
+
+
+class AnnotClient(Struct):
+    '''
+    IPC client for remote chart annotations.
 
     You know those "squares they always show in machine vision
     UIs.." this API allows you to remotely control stuff like that
     in some other graphics actor.
 
     '''
-    ctx2fqmes: dict[str, str]
+    ctx2fqmes: dict[str, set[str]]
     fqme2ipc: dict[str, MsgStream]
+    fqme2gap_ipc: dict[str, MsgStream]
+    _gap_locks: dict[str, trio.Lock]
     _annot_stack: AsyncExitStack
 
     # runtime-populated mapping of all annotation
@@ -957,6 +1019,55 @@ class AnnotCtl(Struct):
             )
         return ipc
 
+    async def set_gap_overlay(
+        self,
+        req: SetGapOverlay,
+
+    ) -> GapOverlay:
+        '''
+        Replace one remote gap-overlay layer using typed IPC.
+
+        '''
+        ipc: MsgStream|None = self.fqme2gap_ipc.get(req.fqme)
+        if ipc is None:
+            raise SymbolNotFound(
+                'No chart actor exposes typed gap-overlay IPC for:\n'
+                f'{req.fqme}'
+            )
+        request_id: str = str(uuid4())
+        req = SetGapOverlay(
+            fqme=req.fqme,
+            timeframe=req.timeframe,
+            specs=req.specs,
+            visible=req.visible,
+            request_id=request_id,
+        )
+
+        # TODO: factor this keyed dialog transaction into a generic
+        # IPC helper once another req/resp client needs stale reply
+        # correlation and per-stream serialization.
+        async with self._gap_locks[req.fqme]:
+            with trio.fail_after(3):
+                await ipc.send(req)
+                with ipc.ctx.pld_rx.limit_plds(
+                    spec=GapOverlay,
+                ):
+                    while True:
+                        resp: GapOverlay = await ipc.receive()
+                        if resp.request_id == request_id:
+                            break
+                        log.warning(
+                            'Discarding stale gap-overlay '
+                            'response:\n'
+                            f'expected: {request_id}\n'
+                            f'received: {resp.request_id}\n'
+                        )
+
+        if resp.error:
+            raise RuntimeError(resp.error)
+
+        return resp
+
     async def add_rect(
         self,
         fqme: str,
@@ -1243,10 +1354,10 @@ class AnnotCtl(Struct):
 
 
 @acm
-async def open_annot_ctl(
+async def open_annot_client(
     uid: tuple[str, str] | None = None,
 
-) -> AnnotCtl:
+) -> AnnotClient:
     # TODO: load connetion to a specific chart actor
     # -[ ] pull from either service scan or config
     # -[ ] return some kinda client/proxy thinger?
@@ -1262,6 +1373,7 @@ async def open_annot_ctl(
     ) as maybe_portals:
 
         ctx_mngrs: list[AsyncContextManager] = []
+        gap_ctx_mngrs: list[AsyncContextManager] = []
 
         # TODO: print the current discoverable actor UID set
         # here as well?
@@ -1274,13 +1386,26 @@ async def open_annot_ctl(
             ctx_mngrs.append(
                 portal.open_context(remote_annotate)
             )
+            gap_ctx_mngrs.append(
+                portal.open_context(remote_gap_overlays)
+            )
+
+        # Keep the typed gap protocol on its own ctx while legacy
+        # annotation commands remain arbitrary dicts. A dynamic
+        # `tractor.msg.limit_plds()` cannot choose the right decoder
+        # before a mixed stream payload has itself been decoded.
 
         ctx2fqmes: dict[str, set[str]] = {}
         fqme2ipc: dict[str, MsgStream] = {}
         stream_ctxs: list[AsyncContextManager] = []
+        gap_ctx2fqmes: dict[str, set[str]] = {}
+        fqme2gap_ipc: dict[str, MsgStream] = {}
+        gap_locks: dict[str, trio.Lock] = {}
+        gap_stream_ctxs: list[AsyncContextManager] = []
 
         async with (
             trionics.gather_contexts(ctx_mngrs) as ctxs,
+            trionics.gather_contexts(gap_ctx_mngrs) as gap_ctxs,
         ):
             for (ctx, fqmes) in ctxs:
                 stream_ctxs.append(ctx.open_stream())
@@ -1291,9 +1416,9 @@ async def open_annot_ctl(
                         raise ValueError(
                             f'More then one chart displays {fqme}!?\n'
                             'Other UI actor info:\n'
-                            f'channel: {other._ctx.chan}]\n'
+                            f'channel: {other.ctx.chan}]\n'
-                            f'actor uid: {other._ctx.chan.uid}]\n'
+                            f'actor uid: {other.ctx.chan.uid}]\n'
-                            f'ctx id: {other._ctx.cid}]\n'
+                            f'ctx id: {other.ctx.cid}]\n'
                         )
 
                     ctx2fqmes.setdefault(
@@ -1301,12 +1426,28 @@ async def open_annot_ctl(
                         set(),
                     ).add(fqme)
 
-            async with trionics.gather_contexts(stream_ctxs) as streams:
+            for (ctx, fqmes) in gap_ctxs:
+                gap_stream_ctxs.append(ctx.open_stream())
+                gap_ctx2fqmes[ctx.cid] = set(fqmes)
+
+            async with (
+                trionics.gather_contexts(stream_ctxs) as streams,
+                trionics.gather_contexts(
+                    gap_stream_ctxs,
+                ) as gap_streams,
+            ):
                 for stream in streams:
-                    fqmes: set[str] = ctx2fqmes[stream._ctx.cid]
+                    fqmes: set[str] = ctx2fqmes[stream.ctx.cid]
                     for fqme in fqmes:
                         fqme2ipc[fqme] = stream
 
+                for stream in gap_streams:
+                    fqmes: set[str] = gap_ctx2fqmes[stream.ctx.cid]
+                    stream_lock: trio.Lock = trio.Lock()
+                    for fqme in fqmes:
+                        fqme2gap_ipc[fqme] = stream
+                        gap_locks[fqme] = stream_lock
+
                 # NOTE: on graceful teardown we always attempt to
                 # remove all annots that were created by the
                 # entering client.
@@ -1315,16 +1456,18 @@ async def open_annot_ctl(
                 # disconnects we always delete all annotations by
                 # default instaead of expecting the client to?
                 async with AsyncExitStack() as annots_stack:
-                    client = AnnotCtl(
+                    client: AnnotClient = AnnotClient(
                         ctx2fqmes=ctx2fqmes,
                         fqme2ipc=fqme2ipc,
+                        fqme2gap_ipc=fqme2gap_ipc,
+                        _gap_locks=gap_locks,
                         _annot_stack=annots_stack,
                     )
                     yield client
 
                     # client exited, measure teardown time
                     teardown_profiler = Profiler(
-                        msg='Client AnnotCtl teardown',
+                        msg='`AnnotClient` teardown',
                         disabled=False,
                         ms_threshold=0.0,
                     )

piker/ui/_widget.py

@@ -37,6 +37,7 @@ from piker.ui.qt import (
 from ..log import get_logger
 
 if TYPE_CHECKING:
+    from ._gaps import GapOverlayMngr
     from ._search import SearchWidget
     from ._chart import (
         LinkedSplits,
@@ -114,6 +115,7 @@ class GodWidget(QWidget):
 
         # assigned in the startup func `_async_main()`
         self._root_n: trio.Nursery = None
+        self.gapman: GapOverlayMngr|None = None
 
         self._widgets: dict[str, QWidget] = {}
         self._resizing: bool = False
@@ -348,5 +350,3 @@ class GodWidget(QWidget):
         self.rt_linked.resize_sidepanes()
         self.hist_linked.resize_sidepanes(from_linked=rt_linked)
         self.search.on_resize()
-
-

plans/opencode/chart-local-gap-overlays.md

@@ -0,0 +1,120 @@
+# Chart-Local Gap Overlays
+
+## Goal
+
+Make gap overlays a chart-actor-owned display service while preserving
+remote annotation as a first-class API. Local startup and UI input avoid
+an unnecessary IPC round trip; remote tools use a strict `msgspec` schema
+and the same renderer/controller.
+
+## Ownership Model
+
+`GapOverlayMngr` is created once in the chart actor and references the
+actor-global `_dss` and `_annots` registries. Each layer key is
+`(owner, id(chart), fqme, timeframe)`. The chart identity keeps duplicate
+FQMEs in separate cached displays independent. The reserved `chart-local`
+owner is also independent from each remote `Context.cid`, so remote context
+teardown cannot remove locally owned graphics.
+
+The controller has one rendering entry point:
+
+1. `GapOverlayMngr.apply(SetGapOverlay)` replaces an owner layer.
+2. `GapOverlayMngr.refresh()` detects gaps from chart-local SHM and calls
+   `apply()` directly.
+3. `AnnotClient.set_gap_overlay()` sends the same request through the
+   strict `remote_gap_overlays()` context.
+
+The generic `remote_annotate()` context and its rectangle, arrow, text,
+batch, remove and redraw commands remain unchanged.
+
+## Startup And UI
+
+`graphics_update_loop()` registers each new `DisplayState` in `_dss`, then
+builds the default 60-second local overlay. The focused `ChartView` handles
+`Ctrl+G` through its existing asynchronous keyboard loop. It chooses the
+focused chart's 1-second or 60-second timeframe and calls
+`GapOverlayMngr.toggle()` directly.
+
+Gap detection reads the structured OHLCV array, excludes invalid timestamps,
+uses one `numpy.diff()` pass, and creates Python `GapSpec` objects only for
+actual positive timestamp gaps.
+
+## Typed IPC
+
+`remote_gap_overlays()` is separate from `remote_annotate()` because
+`msgspec` cannot strictly decode a union containing both arbitrary legacy
+dicts and several struct types. Its Tractor payload specification is:
+
+```python
+SetGapOverlay | GapOverlay | list[str] | None
+```
+
+`SetGapOverlay` and `GapOverlay` are tagged structs. Tractor validates
+started, stream and return payloads against this union. A lock shared by
+all FQMEs on one typed stream serializes each send/receive exchange. Each
+side narrows its public `Context.pld_rx` decoder to the directional request
+or response type while receiving. Request IDs let the client discard a late
+response after an earlier timeout instead of skewing later request/reply
+pairs.
+
+## Refresh Semantics
+
+Startup and every UI transition from hidden to visible recompute the layer
+from current SHM. Existing redraw handling repositions matching overlays by
+both FQME and timeframe. Rectangle and arrow geometry update together.
+
+The next production step is to trigger `refresh()` from an explicit history
+revision/publication event after a repair or prepend. Polling from the quote
+display loop is intentionally excluded.
+
+Request/reply serialization and stale-response correlation can move into a
+generic IPC dialog helper after a second typed consumer establishes the
+reusable interface. The legacy annotation dict protocol should first migrate
+to tagged messages before sharing the strict gap context.
+
+## Landing Order
+
+The existing ancestry is:
+
+```text
+main
+  gap_annotator
+    datad_service
+      backfiller_deep_fixes
+      wkt/fix_broadcast_consumers
+```
+
+The recommended integration order is:
+
+1. Review and land the complete 23-commit `main..gap_annotator` range.
+2. Land Tractor PR 490 after its PR 475 dependency, then update the piker
+   Tractor reference used by `datad_service`.
+3. Land `datad_service` on the new mainline.
+4. Land the smaller four-commit broadcast-consumer PR 92.
+5. Rebase `backfiller_deep_fixes` onto the broadcast result and run the
+   NativeDB/SHM/manual chart qualification sequence.
+6. Replay the chart-local overlay patch above the qualified backfiller tip.
+
+This order puts the already-reviewed ownership fix ahead of the broader
+backfiller qualification. The only known overlapping production path is
+`piker/brokers/ib/feed.py`; `git merge-tree --write-tree` reports no current
+textual conflict, but behavior must still be rechecked.
+
+No branch movement, rebase, merge, commit or push is part of this prototype.
+
+## Verification
+
+Run from the prototype worktree with the source-resolving Python environment:
+
+```text
+python -m pytest -p no:xonsh -q -x --tb=short --no-header \
+    tests/test_gap_overlays.py
+python -m ruff check piker/ui/_gaps.py \
+    tests/test_gap_overlays.py piker/ui/_remote_ctl.py \
+    piker/ui/_display.py piker/ui/_interaction.py \
+    piker/ui/_widget.py piker/ui/_annotate.py
+```
+
+Manual chart qualification must confirm startup rendering, focused-timeframe
+`Ctrl+G`, remote typed replacement, remote disconnect cleanup, cached symbol
+switching and refresh after backfill repair.

plans/opencode/chart-local-gap-overlays.summary.md

@@ -0,0 +1,27 @@
+Own gap overlays inside the chart actor
+
+- Add typed `SetGapOverlay` and `GapOverlay` payloads
+- Detect gaps directly from chart-local OHLCV SHM
+- Share `GapOverlayMngr` between local UI and remote callers
+- Enable 60-second overlays at display startup
+- Toggle the focused timeframe directly with `Ctrl+G`
+- Preserve legacy `remote_annotate()` commands unchanged
+- Isolate local and remote graphics ownership and teardown
+- Document the stack landing and manual qualification order
+
+## Deferred
+
+- Trigger refresh from an explicit history revision event
+- Run live Qt chart and remote actor qualification
+- Review and land the full `gap_annotator` commit range
+
+## Stats
+
+- 10 production/test files changed
+- 2 architecture plan files added
+- 19 focused regressions passing
+- 0 existing branches moved
+
+(this patch was generated in some part by
+[`opencode`][opencode-gh])
+[opencode-gh]: https://github.com/sst/opencode

tests/test_gap_overlays.py

@@ -0,0 +1,148 @@
+'''
+Typed chart-local gap-overlay regressions.
+
+'''
+import msgspec
+import numpy as np
+import pytest
+
+from piker.ui._gaps import (
+    GapOverlay,
+    GapOverlayMngr,
+    GapOverlayPayload,
+    GapSpec,
+    SetGapOverlay,
+    gap_specs_from_ohlcv,
+)
+
+
+def test_gap_specs_and_wire_roundtrip() -> None:
+    '''
+    Local detection and remote IPC share one typed request model.
+
+    Build a small sorted OHLCV array with one positive timestamp gap,
+    derive its geometry without a per-row dataframe scan, and
+    roundtrip the request/reply through msgpack. This proves callers
+    invoke one chart renderer with one concrete schema.
+
+    '''
+    dtype: np.dtype = np.dtype([
+        ('index', 'i8'),
+        ('time', 'f8'),
+        ('open', 'f8'),
+        ('close', 'f8'),
+    ])
+    array: np.ndarray = np.array([
+        (10, 60, 100, 101),
+        (11, 120, 101, 102),
+        (12, 300, 95, 96),
+        (13, 360, 96, 97),
+    ], dtype=dtype)
+
+    specs: list[GapSpec] = gap_specs_from_ohlcv(
+        array,
+        period_s=60,
+    )
+    assert len(specs) == 1
+    spec: GapSpec = specs[0]
+    assert spec.start_time == 120
+    assert spec.end_time == 300
+    assert spec.start_pos == (11.9, 102)
+    assert spec.end_pos == (12.1, 95)
+    assert spec.pointing == 'down'
+
+    req: SetGapOverlay = SetGapOverlay(
+        fqme='mnq.cme.ib',
+        timeframe=60,
+        specs=specs,
+        request_id='req-616',
+    )
+    req_wire: bytes = msgspec.msgpack.encode(req)
+    assert msgspec.msgpack.decode(
+        req_wire,
+        type=SetGapOverlay,
+    ) == req
+
+    resp: GapOverlay = GapOverlay(
+        fqme=req.fqme,
+        timeframe=req.timeframe,
+        visible=True,
+        gap_count=1,
+        aid=616,
+        request_id=req.request_id,
+    )
+    resp_wire: bytes = msgspec.msgpack.encode(resp)
+    assert msgspec.msgpack.decode(
+        resp_wire,
+        type=GapOverlay,
+    ) == resp
+
+
+def test_gap_overlay_payload_rejects_other_msgs() -> None:
+    '''
+    Reject payloads outside the strict gap-overlay dialog protocol.
+
+    The sibling Tractor context installs `GapOverlayPayload` so
+    legacy annotation dicts cannot enter its stream as partially
+    validated commands. Encode one such dict and prove the same
+    msgspec decoder rejects it before endpoint dispatch.
+
+    '''
+    invalid_wire: bytes = msgspec.msgpack.encode({
+        'cmd': 'SelectRect',
+    })
+
+    with pytest.raises(msgspec.ValidationError):
+        msgspec.msgpack.decode(
+            invalid_wire,
+            type=GapOverlayPayload,
+        )
+
+    req: SetGapOverlay = SetGapOverlay(
+        fqme='mnq.cme.ib',
+        timeframe=60,
+        specs=[],
+    )
+    resp: GapOverlay = GapOverlay(
+        fqme=req.fqme,
+        timeframe=req.timeframe,
+        visible=False,
+        gap_count=0,
+    )
+    with pytest.raises(msgspec.ValidationError):
+        msgspec.msgpack.decode(
+            msgspec.msgpack.encode(resp),
+            type=SetGapOverlay,
+        )
+    with pytest.raises(msgspec.ValidationError):
+        msgspec.msgpack.decode(
+            msgspec.msgpack.encode(req),
+            type=GapOverlay,
+        )
+
+
+def test_gap_overlay_unknown_fqme_returns_typed_error() -> None:
+    '''
+    Keep a stale remote FQME request inside the typed dialog.
+
+    A chart can unload or replace display state after a client learns
+    its FQME set. Submit a valid request to an empty controller and
+    prove it returns `GapOverlay.error` instead of leaking a
+    `KeyError` which would terminate the whole Tractor context.
+
+    '''
+    gapman: GapOverlayMngr = GapOverlayMngr(
+        dss={},
+        annots={},
+    )
+    req: SetGapOverlay = SetGapOverlay(
+        fqme='gone.test',
+        timeframe=60,
+        specs=[],
+        request_id='req-stale',
+    )
+
+    resp: GapOverlay = gapman.apply(req)
+
+    assert resp.request_id == req.request_id
+    assert resp.error == 'No display state for fqme=gone.test'

tests/test_ldshm.py

@@ -204,7 +204,7 @@ def test_shm_invalid_snapshot_never_reaches_storage(
     from piker.ui import _remote_ctl
     monkeypatch.setattr(
         _remote_ctl,
-        'open_annot_ctl',
+        'open_annot_client',
         open_annotations,
     )
 
@@ -345,7 +345,7 @@ def test_shm_write_without_reload_uses_deduped_markup_frame(
     from piker.ui import _remote_ctl
     monkeypatch.setattr(
         _remote_ctl,
-        'open_annot_ctl',
+        'open_annot_client',
         open_annotations,
     )