Introduce `AnnotCtl.add_batch()` and `serve_rc_annots()` batch
handler to submit 1000s of gaps in single IPC msg instead of
per-annot round-trips. Server builds `GapAnnotations` from specs
and handles vectorized timestamp-to-index lookups.
Deats,
- add `'cmd': 'batch'` handler in `serve_rc_annots()`
- vectorized timestamp lookup via `np.searchsorted()` + masking
- build `gap_specs: list[dict]` from rect+arrow specs client-side
- create single `GapAnnotations` item for all gaps server-side
- handle `GapAnnotations.reposition()` in redraw handler
- add profiling to batch path for perf measurement
- support optional individual arrows for A/B comparison
Also,
- refactor `markup_gaps()` to collect specs + single batch call
- add `no_qt_updates()` context mgr for batch render ops
- add profiling to annotation teardown path
- add `GapAnnotations` case to `rm_annot()` match block
(this patch was generated in some part by [`claude-code`][claude-code-gh])
[claude-code-gh]: https://github.com/anthropics/claude-code
For a ~1000x perf gain says ol' claudy, our boi who wrote this entire
patch! Bo
Introduce `GapAnnotations` in `.ui._annotate` for batch-rendering gap
rects/arrows instead of individual `QGraphicsItem` instances. Uses
upstream's `pyqtgraph.Qt.internals.PrimitiveArray` for rects and
a `QPainterPath` for arrows. This API-replicates our prior annotator's
in view shape-graphics but now using (what we're dubbing)
"single-array-multiple-graphics" tech much like our `.ui._curve`
extensions to `pg` B)
Impl deats,
- batch draw ~1000 gaps in single paint call vs 1000 items
- arrows render in scene coords to maintain pixel size on zoom
- add vectorized timestamp-to-index lookup for repositioning
- cache bounding rect, rebuild on `reposition()` calls
- match `SelectRect` + `ArrowItem` visual style/colors
- skip reposition when timeframe doesn't match gap's period
Other,
- fix typo in `LevelMarker` docstring: "graphich" -> "graphic"
- reflow docstring in `qgo_draw_markers()` to 67 char limit
(this patch was generated in some part by [`claude-code`][claude-code-gh])
[claude-code-gh]: https://github.com/anthropics/claude-code
Log shm file name and detected period before null segment
processing to aid debugging.
(this patch was generated in some part by [`claude-code`][claude-code-gh])
[claude-code-gh]: https://github.com/anthropics/claude-code
Refine zoom methods in `MainWindow` and font helpers
in `_style` to return `px_size` up the call chain and
log detailed zoom state on each change.
Deats,
- make `_set_qfont_px_size()` return `self.px_size`.
- make `configure_to_dpi()` and `_config_fonts_to_screen()`
return the new `px_size` up through the call chain.
- add `font_size` to `log.info()` in `zoom_in()`,
`zoom_out()`, and `reset_zoom()` alongside
`zoom_step` and `zoom_level(%)`.
- reformat `has_ctrl`/`_has_shift` bitwise checks and
key-match tuples to multiline style.
- comment out `Shift` modifier requirement for zoom
hotkeys (now `Ctrl`-only).
- comment out unused `mn_dpi` and `dpi` locals.
Also,
- convert all single-line docstrings to `'''` multiline
style across zoom and font methods.
- rewrap `configure_to_dpi()` docstring to 67 chars.
- move `from . import _style` to module-level import
in `_window.py`.
- drop unused `screen` binding in `boundingRect()`.
(this commit msg was generated in some part by [`claude-code`][claude-code-gh])
[claude-code-gh]: https://github.com/anthropics/claude-code
Again a patch (vibed) from our very own @dnks
(just a commit msg reworking using his new `/commit-msg` skill added by
@goodboy B)
Deats,
- add `Axis.update_fonts()` to recalculate tick font, text offset,
bounding rect and `pyqtgraph`'s internal text-width/height tracking
after a zoom change; store `_typical_max_str` at init for later reuse.
- rework `PriceAxis.size_to_values()` and
`DynamicDateAxis.size_to_values()` to use pyqtgraph's
`_updateWidth()`/`_updateHeight()` with `updateGeometry()` instead of
raw `setWidth()`/ `setHeight()` so auto-expand constraints are
respected.
- fix `GlobalZoomEventFilter` to mask out `KeypadModifier` and
explicitly require both Ctrl+Shift, letting plain Ctrl+Plus/Minus pass
through to chart zoom.
- add `_update_chart_axes()` to walk all plot-item axes during
`_apply_zoom()` and call `splits.resize_sidepanes()` to sync subplot
widths.
(this commit msg, and likely patch, was generated in some part by
[`claude-code`][claude-code-gh])
[claude-code-gh]: https://github.com/anthropics/claude-code
Add `Ctrl+Shift+Plus/Minus/0` shortcuts for zooming all
UI widget font sizes via a `GlobalZoomEventFilter`
installed at the `QApplication` level.
Deats,
- `.ui._window`: add `GlobalZoomEventFilter` event
filter class and `MainWindow.zoom_in/out/reset_zoom()`
methods that reconfigure `DpiAwareFont` with a
`zoom_level` multiplier then propagate to all child
widgets.
- `.ui._style`: extend `DpiAwareFont.configure_to_dpi()`
and `_config_fonts_to_screen()` to accept a
`zoom_level` float multiplier; cast `px_size` to `int`.
- `.ui._forms`: add `update_fonts()` to `Edit`,
`Selection`, `FieldsForm` and `FillStatusBar` for
stylesheet regen.
- `.ui._label`: add `FormatLabel.update_font()` method.
- `.ui._position`: add `SettingsPane.update_fonts()`.
- `.ui._search`: add `update_fonts()` to `CompleterView`
and `SearchWidget`.
- `.ui._exec`: install the zoom filter on window show.
- `.ui.qt`: import `QObject` from `PyQt6`.
(this commit msg was generated in some part by [`claude-code`][claude-code-gh])
[claude-code-gh]: https://github.com/anthropics/claude-code
Note since it's actually `xonsh` code run with either,
- most pedantically: `xonsh ./snippets/calc_ppi.xsh`
- or relying on how shebang: `./snippets/calc_ppi.xsh`
* an sheboom.
For wtv reason on nixos importing `pyqtgraph` first is causing `numpy`
to fail import?? No idea, but likely something to do with recent
`flake.nix`'s ld-lib-linking with `<nixpkgs>` marlarky?
- set `QT_USE_PHYSICAL_DPI='1'` env var for Qt6 high-DPI
* we likely want to do this in `piker.ui` as well!
- move `pxr` calc from widget to per-screen in loop.
- add `unscaled_size` calc using `pxr * size`.
- switch from `.availableGeometry()` to `.geometry()` for full
bounds.
- shorten output labels, add `!r` repr formatting
- add Qt6 DPI rounding policy TODO with doc links
(this patch was generated in some part by [`claude-code`][claude-code-gh])
[claude-code-gh]: https://github.com/anthropics/claude-code
Port all 16 internal import sites from re-exporting
via `piker.data._sharedmem` shim to importing core
shm types directly from `tractor.ipc._shm`.
Deats,
- `ShmArray` now imported from tractor in 10 files.
- `_Token` renamed to `NDToken` everywhere (5 files).
- `attach_shm_array` → `attach_shm_ndarray` at all
call sites.
- `data/__init__.py` sources `ShmArray`,
`get_shm_token` from tractor; keeps
`open/attach_shm_array` as public API aliases.
- Trim shim to only piker-specific wrappers:
`_make_token()`, `maybe_open_shm_array()`,
`try_read()`.
- Drop `Optional` usage in shim, use `|None`.
(this patch was generated in some part by [`claude-code`][claude-code-gh])
[claude-code-gh]: https://github.com/anthropics/claude-code
Replace the ~716 line `piker.data._sharedmem` mod with a thin re-export
shim consuming `tractor.ipc._shm` types directly, since the `tractor`
version is the refined factoring of piker's original impl.
Deats,
- Re-export `SharedInt`, `ShmArray`, `ShmList`, `get_shm_token`,
`_known_tokens` directly
- Alias renames: `NDToken as _Token`, `open_shm_ndarray as
open_shm_array`, `attach_shm_ndarray as attach_shm_array`
- Keep `_make_token()` wrapper for piker's default dtype fallback to
`def_iohlcv_fields`
- Keep `maybe_open_shm_array()` wrapper preserving piker's historical
defaults (`readonly=False`, `append_start_index=None`)
- Keep `try_read()` race-condition guard (not in `tractor`)
All 13 import sites across piker continue to work unchanged with no
modifications needed.
(this patch was generated in some part by [`claude-code`][claude-code-gh])
[claude-code-gh]: https://github.com/anthropics/claude-code
Match upstream `tractor` API change where
`open_channel_from()` now yields `(chan, first)`
instead of `(first, chan)` — i.e.
`tuple[LinkedTaskChannel, Any]`.
- `brokers/ib/api.py`
- `brokers/ib/broker.py`
- `brokers/ib/feed.py`
- `brokers/deribit/api.py` (2 sites)
(this patch was generated in some part by [`claude-code`][claude-code-gh])
[claude-code-gh]: https://github.com/anthropics/claude-code
Replace hand-rolled `click`-based platform branching using
the much saner `platformdirs.user_config_path()`.
Deats,
- remove Windows/macOS/Unix `if/elif` platform dispatch
(~25 lines) in favour of single `user_config_path()` call.
- move `_posixify()` inside `force_posix` branch since it's
only used there.
- add `log.info()` reporting platform name and resolved dirs.
Also,
- drop now unneeded `sys` import.
- reformat `assert` in `repodir()` to multiline style.
- convert docstring from `r"""..."""` to `'''...'''` style.
(this commit msg was generated in some part by [`claude-code`][claude-code-gh])
[claude-code-gh]: https://github.com/anthropics/claude-code
Add TAB + ENTER key presses after the `Ctrl+Alt+<key>` hotkey
combo to auto-confirm the "simulate a reset?" dialog that IB
gateway sometimes shows.
Deats,
- press `ISO_Enter` before click to dismiss any prior active
dialog window.
- add post-hotkey loop sending `Tab` then `KP_Enter` with
`asyncio.sleep()` delays to handle the confirmation dialog.
- add `asyncio` import.
Also,
- capture VNC connect error as `vnc_err` and log it instead of
falling through to `try_xdo_manual()`.
- comment-out `try_xdo_manual()` fallback in VNC error path.
- reformat `client.press()` call to multiline style.
- reformat `RuntimeError` raise to multiline style with `!r`.
(this commit msg was generated in some part by [`claude-code`][claude-code-gh])
[claude-code-gh]: https://github.com/anthropics/claude-code
Refactor `open_symbol_search()` to use `partial()` for nursery task
spawning and add detailed query->results logging via `ppfmt()`.
Deats,
- change `extend_results()` to accept `target` callable +
`pattern` + `**kwargs` and invoke inside, instead of receiving
a pre-called awaitable; use `partial()` to pass args.
- add `ppfmt()` formatted logging of search query params and
results including client class + method repr.
- change `print()` -> `log.exception()` for `Lagged` overrun.
- bump `upto=5` -> `upto=10` for `search_symbols()` call.
Also for styling,
- add type some missing type annots.
- add multiline style to `or` conditionals in pattern check.
- reformat log msgs to multiline style throughout.
- use `ppfmt()` for fuzzy match debug log.
- rename nursery `sn` -> `tn`.
- add TODO comment about `assert 0` hang.
(this commit msg was generated in some part by [`claude-code`][claude-code-gh])
[claude-code-gh]: https://github.com/anthropics/claude-code
Add `isinstance()` dispatch for the `'error'` event case in
`deliver_trade_events()` to handle `ib_async` sometimes emitting plain
`str` error items instead of the previously expected `dict`.
Deats,
- add `isinstance(err, dict)` branch for the standard case with
`error_code`, `reason`, and `reqid` fields.
- add `isinstance(err, str)` branch to parse error strings of the
form `'[code 104] connection failed'` into `code` and `reason`.
- set `reqid: str = '<unknown>'` for string-form errors since
there's no request ID available.
- update `err` type annot to `dict|str`.
(this commit msg was generated in some part by [`claude-code`][claude-code-gh])
[claude-code-gh]: https://github.com/anthropics/claude-code
Add guards for empty-array and zero-bar-diff cases in the TSP backfill
loops to avoid crashes and allow graceful loop termination.
In `maybe_fill_null_segments()`,
- add `array.size == 0` guard in `maybe_fill_null_segments()` to detect
valid (venue closure) gaps from the backend; add a warning + bp
+ break for this case.
- add TODO that we should likely be filling nulls with the close price
for the gap's duration.
In `start_backfill()`,
- expand the "0 bars after diff" warning msg with
`backfill_until_dt` and `end_dt_param` context.
- mask the `await tractor.pause()` and add a `break` to avoid blocking
the backfill loop.
(this commit msg was generated in some part by
[`claude-code`][claude-code-gh])
[claude-code-gh]: https://github.com/anthropics/claude-code
Downgrade the `start_dt`-trimming check in `open_history_client()`
from a `RuntimeError` raise to a warning log, allowing the caller
to still receive a (shorter) frame of bars (though we may need to still
specially handle such cases in the backfiller's biz logic layer).
Deats,
- add `trimmed_bars.size` guard to skip check on empty results.
- change condition to `>=` and log a warning with the short-frame
size instead of raising.
- comment-out `raise RuntimeError` and breakpoint for future
removal once confident.
- add docstring-style comment on `start_dt=` kwarg noting that
`Client.bars()` doesn't truly support it (uses duration-style
queries internally).
(this commit msg was generated in some part by [`claude-code`][claude-code-gh])
[claude-code-gh]: https://github.com/anthropics/claude-code
Use (the only available in `ib_async`) `returnAll=True` in
`qualifyContractsAsync()` calls within `get_fute()` and handle the case
where IB returns a list of ambiguous contract matches instead of
a single result.
Deats,
- add `returnAll=True` to both `ContFuture` and `Future`
qualification calls.
- add `isinstance(con, list)` check after unpacking first result
to detect ambiguous contract sets.
- log warning with input params and matched contracts when
ambiguous.
- update return type annot to `Contract|list[Contract]`.
Also,
- handle list-of-contracts case in `find_contracts()` by unpacking
`*contracts` into the `qualifyContractsAsync()` call.
- reformat `qualifyContractsAsync()` calls to multiline style.
(this commit msg was generated in some part by [`claude-code`][claude-code-gh])
[claude-code-gh]: https://github.com/anthropics/claude-code
Use `con.exchange` as fallback when `con.primaryExchange` is empty
in `has_holiday()` to handle contracts like futures that don't
always set a `primaryExchange`.
Deats,
- extract `con: Contract` from `con_deats.contract` for reuse.
- use `con.primaryExchange or con.exchange` to ensure a valid
exchange code is always passed to the calendar lookup.
- add `Contract` to `TYPE_CHECKING` imports.
(this commit msg was generated in some part by [`claude-code`][claude-code-gh])
[claude-code-gh]: https://github.com/anthropics/claude-code
Add exchange name translation in `.ib.venues.has_holiday()` to handle
non-standard exchange codes when looking up holiday gaps..
Deats,
- add an ad-hoc lookup dict to remap an IB `Contract.primaryExchange` val
which doesn't exist in the `exchange_calendars`'s alias set.
- use `.get()` with fallback to map `exch` to new `std_exch` and pass
that to `xcals.get_calendar()`.
- add the case i just caught, `'ARCA'` -> `'ARCX'` to the table when i loaded
the `gld.arca.ib` market..
(this commit msg was generated in some part by [`claude-code`][claude-code-gh])
[claude-code-gh]: https://github.com/anthropics/claude-code
Add fallback handling for unknown IB order status strings to
avoid crashes when IB returns unexpected status values.
Deats,
- add `'ValidationError': 'error'` mapping to `_statuses` dict.
- use `.get()` with `'error'` default instead of direct dict
lookup for `status.status`.
- add `elif status_str == 'error'` block to log unknown status
values.
- add type annots to `event_name` and `item` in
`deliver_trade_events()` loop.
Also,
- reformat log msg in `deliver_trade_events()` to multiline.
- drop extra conditional in `if status_str == 'filled'` check.
(this commit msg was generated in some part by [`claude-code`][claude-code-gh])
[claude-code-gh]: https://github.com/anthropics/claude-code
Migrate the IB broker backend to use `ib_async` (the actively maintained
fork) instead of the now stale, original `ib_insync` lib.
Deats,
- update `pyproject.toml` dep: drop `ib-insync` pin, add
`ib-async>=2.1.0`.
- update lock file with `ib-async` and its new `aeventkit` dep (which
i guess replaces `eventkit`).
- obvi, change all `ib_insync` imports to `ib_async` across `.ib.*`.
- update docs and select internal comments referencing the original lib.
Also,
- drop unused `ledger_dict` init in `_flex_reports.load_flex_trades()`.
- fix union type annot style: `dict | None` -> `dict|None`.
- strip `.tzinfo` from `lastTimeStamp` in `normalize()` to avoid
IPC codec issues with `ib_async`'s `timezone.utc` injection.
- pop `'defaults'` from ticker data dict in `normalize()` to avoid
non-serializable `timezone` objects and warning-log in such
cases.
(this commit msg was generated in some part by [`claude-code`][claude-code-gh])
[claude-code-gh]: https://github.com/anthropics/claude-code
That is to be able to (eventually) introspect "ambiguous" contract sets
once we move to `ib_async` and its `returnAll: bool` now offered by
`IB.qualifyContractsAsync()`,
https://github.com/ib-api-reloaded/ib_async/blob/main/ib_async/ib.py#L2115
Also, tweak some type type annots to multline style in sibling mods.
Seems to be required to get certain widgets to size correctly on macos
like the order-mode "health bar" among other types of labels?
(this commit msg was generated in some part by [`claude-code`][claude-code-gh])
[claude-code-gh]: https://github.com/anthropics/claude-code
Originally from @dnks with a weird(er) commit msg XP
Shorten SHM buffer key names to fit Darwin's 31
char filename limit by hashing the `fqme`/content
portion with `md5` and truncating to 8 hex chars.
Deats,
- `.fsp._api`: replace `piker.{actor}[{uuid}].{sym}`
format with `{uuid[:8]}_{hash}.fsp`
- `.tsp._history`: add `platform.system()` check to
conditionally shorten `.hist`/`.rt` keys on Darwin
while keeping the full key format on Linux
(this commit msg was generated in some part by [`claude-code`][claude-code-gh])
[claude-code-gh]: https://github.com/anthropics/claude-code
NOTE that this patch was orig by @dnks and broken out from a larger
commit which added unnecessary/out-of-scope changes we didn't end up
requiring.
That is, in-mem history buffer files should have a `.hist` suffix (which
was added back instead of the weird `.h` change from the rebased
8352d89e, initially providing the macos support fix).
I'm pretty sure we can afford the 3 extra chars if we're already
truncating to a max 16 char `key: str = f'{uuid[:8]}_{fqme_hash}'`, per
@goodboy's edit from the original key-truncation approach in the
original original @dnks patch: 75d8582b).
Gud Boi
di1ara
Tyler Goodlet