Compare commits

..

16 Commits

Author SHA1 Message Date
Gud Boi 1c22bbe016 Exclude crypto futes from `without_src` sym key
Extend the `col_sym_key` asset-type check in `start_backfill()`
to also exclude crypto-denominated futures (where `src` is
`'crypto_currency'` and `dst` is `'future'`) from the
`without_src=True` fqme path.

Also in `.brokers.binance` backend (it being the guilty culprit in the
discovery of this bug; and why i touched styling this code),

- reformat `make_sub()` fn sig to multiline style in
  `.binance.feed`.
- add backtick around `dict` in `make_sub()` docstring.
- reformat `or` conditionals to multiline style in
  `.binance.feed.get_mkt_info()`.

(this commit msg was generated in some part by [`claude-code`][claude-code-gh])
[claude-code-gh]: https://github.com/anthropics/claude-code
2026-03-11 10:32:01 -04:00
Gud Boi 2fadc75314 Enable console log (from passed down `loglevel`) in `.tsp._history.manage_history()` 2026-03-11 10:32:01 -04:00
Gud Boi bbca498bb0 Drop `Flume.feed`, it's unused yet causes import cycles.. 2026-03-11 10:32:01 -04:00
Gud Boi 9002b55c28 Just warn on single-bar nulls instead of bping
Replace the debug breakpoint with a warning-log when a single-bar
null-segment is detected in `get_null_segs()`. This lets the gap
analysis continue while still alerting about the anomaly.

Deats,
- extract the 3-bar window (before, null, after) and calculate
  a `gap: pendulum.Interval` for the warning msg.
- comment-out the old breakpoint block for optional debugging as needed.

(this commit msg was generated in some part by [`claude-code`][claude-code-gh])
[claude-code-gh]: https://github.com/anthropics/claude-code
2026-03-11 10:32:01 -04:00
Gud Boi 7c95146bf7 Lul, drop long unused poetry lock file 2026-03-11 10:32:01 -04:00
Gud Boi 71c974b5e9 Pin `pg` at latest official `0.14.0` release
Keep in masked GH sources lines for easy hackin against upstream
`master` branch when needed as well!
2026-03-11 10:32:01 -04:00
Gud Boi a7480c7fbe .ui._editors: log multiline styling and re-leveling 2026-03-11 10:32:01 -04:00
Gud Boi f931d6ebb6 .ui._lines: drop unused graphics-item import 2026-03-11 10:32:01 -04:00
Gud Boi a8cdc05b5b Add batch-submit API for gap annotations
Introduce `AnnotCtl.add_batch()` and `serve_rc_annots()` batch
handler to submit 1000s of gaps in single IPC msg instead of
per-annot round-trips. Server builds `GapAnnotations` from specs
and handles vectorized timestamp-to-index lookups.

Deats,
- add `'cmd': 'batch'` handler in `serve_rc_annots()`
- vectorized timestamp lookup via `np.searchsorted()` + masking
- build `gap_specs: list[dict]` from rect+arrow specs client-side
- create single `GapAnnotations` item for all gaps server-side
- handle `GapAnnotations.reposition()` in redraw handler
- add profiling to batch path for perf measurement
- support optional individual arrows for A/B comparison

Also,
- refactor `markup_gaps()` to collect specs + single batch call
- add `no_qt_updates()` context mgr for batch render ops
- add profiling to annotation teardown path
- add `GapAnnotations` case to `rm_annot()` match block

(this patch was generated in some part by [`claude-code`][claude-code-gh])
[claude-code-gh]: https://github.com/anthropics/claude-code
2026-03-11 10:32:01 -04:00
Gud Boi 786cf7d7c2 Add a `GapAnnotations` path-renderer
For a ~1000x perf gain says ol' claudy, our boi who wrote this entire
patch! Bo

Introduce `GapAnnotations` in `.ui._annotate` for batch-rendering gap
rects/arrows instead of individual `QGraphicsItem` instances. Uses
upstream's `pyqtgraph.Qt.internals.PrimitiveArray` for rects and
a `QPainterPath` for arrows. This API-replicates our prior annotator's
in view shape-graphics but now using (what we're dubbing)
"single-array-multiple-graphics" tech much like our `.ui._curve`
extensions to `pg` B)

Impl deats,
- batch draw ~1000 gaps in single paint call vs 1000 items
- arrows render in scene coords to maintain pixel size on zoom
- add vectorized timestamp-to-index lookup for repositioning
- cache bounding rect, rebuild on `reposition()` calls
- match `SelectRect` + `ArrowItem` visual style/colors
- skip reposition when timeframe doesn't match gap's period

Other,
- fix typo in `LevelMarker` docstring: "graphich" -> "graphic"
- reflow docstring in `qgo_draw_markers()` to 67 char limit

(this patch was generated in some part by [`claude-code`][claude-code-gh])
[claude-code-gh]: https://github.com/anthropics/claude-code
2026-03-11 10:32:01 -04:00
Gud Boi 2d9dd406e7 Add info log for shm processing in `ldshm` CLI cmd
Log shm file name and detected period before null segment
processing to aid debugging.

(this patch was generated in some part by [`claude-code`][claude-code-gh])
[claude-code-gh]: https://github.com/anthropics/claude-code
2026-03-11 10:32:01 -04:00
Gud Boi 0345296991 Bump to latest official `pyqtgraph` release 2026-03-11 10:32:01 -04:00
Gud Boi e2cbc4ac27 Tweak y-move to `400` to align better with reset-dialog box 2026-03-11 10:01:37 -04:00
Gud Boi e591b63985 Use `platformdirs` for `.config.get_app_dir()`
Replace hand-rolled `click`-based platform branching using
the much saner `platformdirs.user_config_path()`.

Deats,
- remove Windows/macOS/Unix `if/elif` platform dispatch
  (~25 lines) in favour of single `user_config_path()` call.
- move `_posixify()` inside `force_posix` branch since it's
  only used there.
- add `log.info()` reporting platform name and resolved dirs.

Also,
- drop now unneeded `sys` import.
- reformat `assert` in `repodir()` to multiline style.
- convert docstring from `r"""..."""` to `'''...'''` style.

(this commit msg was generated in some part by [`claude-code`][claude-code-gh])
[claude-code-gh]: https://github.com/anthropics/claude-code
2026-03-11 10:01:37 -04:00
Gud Boi d38bec2871 Drop bp from duration mismatch branch in `.ib.api.Client.bars()` 2026-03-11 10:01:37 -04:00
Gud Boi 9405d88234 Handle VNC reset-dialog in `vnc_click_hack()`
Add TAB + ENTER key presses after the `Ctrl+Alt+<key>` hotkey
combo to auto-confirm the "simulate a reset?" dialog that IB
gateway sometimes shows.

Deats,
- press `ISO_Enter` before click to dismiss any prior active
  dialog window.
- add post-hotkey loop sending `Tab` then `KP_Enter` with
  `asyncio.sleep()` delays to handle the confirmation dialog.
- add `asyncio` import.

Also,
- capture VNC connect error as `vnc_err` and log it instead of
  falling through to `try_xdo_manual()`.
- comment-out `try_xdo_manual()` fallback in VNC error path.
- reformat `client.press()` call to multiline style.
- reformat `RuntimeError` raise to multiline style with `!r`.

(this commit msg was generated in some part by [`claude-code`][claude-code-gh])
[claude-code-gh]: https://github.com/anthropics/claude-code
2026-03-11 10:01:37 -04:00
3 changed files with 73 additions and 52 deletions

View File

@ -20,6 +20,7 @@ runnable script-programs.
''' '''
from __future__ import annotations from __future__ import annotations
import asyncio
from datetime import ( # noqa from datetime import ( # noqa
datetime, datetime,
date, date,
@ -140,7 +141,8 @@ async def data_reset_hack(
except ( except (
OSError, # no VNC server avail.. OSError, # no VNC server avail..
PermissionError, # asyncvnc pw fail.. PermissionError, # asyncvnc pw fail..
): ) as _vnc_err:
vnc_err = _vnc_err
try: try:
import i3ipc # noqa (since a deps dynamic check) import i3ipc # noqa (since a deps dynamic check)
except ModuleNotFoundError: except ModuleNotFoundError:
@ -166,14 +168,22 @@ async def data_reset_hack(
# localhost but no vnc-client or it borked.. # localhost but no vnc-client or it borked..
else: else:
try_xdo_manual(client) log.error(
'VNC CLICK HACK FAILE with,\n'
f'{vnc_err!r}\n'
)
# breakpoint()
# try_xdo_manual(client)
case 'i3ipc_xdotool': case 'i3ipc_xdotool':
try_xdo_manual(client) try_xdo_manual(client)
# i3ipc_xdotool_manual_click_hack() # i3ipc_xdotool_manual_click_hack()
case _ as tech: case _ as tech:
raise RuntimeError(f'{tech} is not supported for reset tech!?') raise RuntimeError(
f'{tech!r} is not supported for reset tech!?'
)
# we don't really need the ``xdotool`` approach any more B) # we don't really need the ``xdotool`` approach any more B)
return True return True
@ -265,14 +275,39 @@ async def vnc_click_hack(
# 640x1800 # 640x1800
await client.move( await client.move(
Point( Point(
500, 500, # x from left
500, 400, # y from top
) )
) )
# in case a prior dialog win is open/active.
await client.press('ISO_Enter')
# ensure the ib-gw window is active # ensure the ib-gw window is active
await client.click(MOUSE_BUTTON_LEFT) await client.click(MOUSE_BUTTON_LEFT)
# send the hotkeys combo B) # send the hotkeys combo B)
await client.press('Ctrl', 'Alt', key) # keys are stacked await client.press(
'Ctrl',
'Alt',
key,
) # NOTE, keys are stacked
# XXX, sometimes a dialog asking if you want to "simulate
# a reset" will show, in which case we want to select
# "Yes" (by tabbing) and then hit enter.
iters: int = 1
delay: float = 0.3
await asyncio.sleep(delay)
for i in range(iters):
log.info(f'Sending TAB {i}')
await client.press('Tab')
await asyncio.sleep(delay)
for i in range(iters):
log.info(f'Sending ENTER {i}')
await client.press('KP_Enter')
await asyncio.sleep(delay)
def i3ipc_fin_wins_titled( def i3ipc_fin_wins_titled(

View File

@ -561,7 +561,7 @@ class Client:
# f'Recursing for more bars:\n' # f'Recursing for more bars:\n'
) )
# XXX, debug! # XXX, debug!
breakpoint() # breakpoint()
# XXX ? TODO? recursively try to re-request? # XXX ? TODO? recursively try to re-request?
# => i think *NO* right? # => i think *NO* right?
# #

View File

@ -19,7 +19,6 @@ Platform configuration (files) mgmt.
""" """
import platform import platform
import sys
import os import os
import shutil import shutil
from typing import ( from typing import (
@ -29,6 +28,7 @@ from typing import (
from pathlib import Path from pathlib import Path
from bidict import bidict from bidict import bidict
import platformdirs
import tomlkit import tomlkit
try: try:
import tomllib import tomllib
@ -41,7 +41,7 @@ from .log import get_logger
log = get_logger('broker-config') log = get_logger('broker-config')
# XXX NOTE: taken from `click` # XXX NOTE: orig impl was taken from `click`
# |_https://github.com/pallets/click/blob/main/src/click/utils.py#L449 # |_https://github.com/pallets/click/blob/main/src/click/utils.py#L449
# #
# (since apparently they have some super weirdness with SIGINT and # (since apparently they have some super weirdness with SIGINT and
@ -54,44 +54,21 @@ def get_app_dir(
force_posix: bool = False, force_posix: bool = False,
) -> str: ) -> str:
r"""Returns the config folder for the application. The default behavior '''
Returns the config folder for the application. The default behavior
is to return whatever is most appropriate for the operating system. is to return whatever is most appropriate for the operating system.
To give you an idea, for an app called ``"Foo Bar"``, something like ----
the following folders could be returned: NOTE, below is originally from `click` impl fn, we can prolly remove?
----
Mac OS X:
``~/Library/Application Support/Foo Bar``
Mac OS X (POSIX):
``~/.foo-bar``
Unix:
``~/.config/foo-bar``
Unix (POSIX):
``~/.foo-bar``
Win XP (roaming):
``C:\Documents and Settings\<user>\Local Settings\Application Data\Foo``
Win XP (not roaming):
``C:\Documents and Settings\<user>\Application Data\Foo Bar``
Win 7 (roaming):
``C:\Users\<user>\AppData\Roaming\Foo Bar``
Win 7 (not roaming):
``C:\Users\<user>\AppData\Local\Foo Bar``
.. versionadded:: 2.0
:param app_name: the application name. This should be properly capitalized
and can contain whitespace.
:param roaming: controls if the folder should be roaming or not on Windows. :param roaming: controls if the folder should be roaming or not on Windows.
Has no affect otherwise. Has no affect otherwise.
:param force_posix: if this is set to `True` then on any POSIX system the :param force_posix: if this is set to `True` then on any POSIX system the
folder will be stored in the home folder with a leading folder will be stored in the home folder with a leading
dot instead of the XDG config home or darwin's dot instead of the XDG config home or darwin's
application support folder. application support folder.
""" '''
def _posixify(name):
return "-".join(name.split()).lower()
# NOTE: for testing with `pytest` we leverage the `tmp_dir` # NOTE: for testing with `pytest` we leverage the `tmp_dir`
# fixture to generate (and clean up) a test-request-specific # fixture to generate (and clean up) a test-request-specific
# directory for isolated configuration files such that, # directory for isolated configuration files such that,
@ -117,23 +94,30 @@ def get_app_dir(
# assert testdirpath.exists(), 'piker test harness might be borked!?' # assert testdirpath.exists(), 'piker test harness might be borked!?'
# app_name = str(testdirpath) # app_name = str(testdirpath)
if platform.system() == 'Windows': os_name: str = platform.system()
key = "APPDATA" if roaming else "LOCALAPPDATA" conf_dir: Path = platformdirs.user_config_path()
folder = os.environ.get(key) app_dir: Path = conf_dir / app_name
if folder is None:
folder = os.path.expanduser("~") # ?TODO, from `click`; can remove?
return os.path.join(folder, app_name)
if force_posix: if force_posix:
def _posixify(name):
return "-".join(name.split()).lower()
return os.path.join( return os.path.join(
os.path.expanduser("~/.{}".format(_posixify(app_name)))) os.path.expanduser(
if sys.platform == "darwin": "~/.{}".format(
return os.path.join( _posixify(app_name)
os.path.expanduser("~/Library/Application Support"), app_name )
)
) )
return os.path.join(
os.environ.get("XDG_CONFIG_HOME", os.path.expanduser("~/.config")), log.info(
_posixify(app_name), f'Using user config directory,\n'
f'platform.system(): {os_name!r}\n'
f'conf_dir: {conf_dir!r}\n'
f'app_dir: {conf_dir!r}\n'
) )
return app_dir
_click_config_dir: Path = Path(get_app_dir('piker')) _click_config_dir: Path = Path(get_app_dir('piker'))
@ -250,7 +234,9 @@ def repodir() -> Path:
repodir: Path = Path(os.environ.get('GITHUB_WORKSPACE')) repodir: Path = Path(os.environ.get('GITHUB_WORKSPACE'))
confdir: Path = repodir / 'config' confdir: Path = repodir / 'config'
assert confdir.is_dir(), f'{confdir} DNE, {repodir} is likely incorrect!' assert confdir.is_dir(), (
f'{confdir} DNE, {repodir} is likely incorrect!'
)
return repodir return repodir