Commit Graph

1077 Commits (19fc9667659878d5d2e9932fa27e486ec6939bc6)

Author SHA1 Message Date
Gud Boi b22c59e7a0 Swap `open_channel_from()` yield-pair order
Match upstream `tractor` API change where
`open_channel_from()` now yields `(chan, first)`
instead of `(first, chan)` — i.e.
`tuple[LinkedTaskChannel, Any]`.

- `brokers/ib/api.py`
- `brokers/ib/broker.py`
- `brokers/ib/feed.py`
- `brokers/deribit/api.py` (2 sites)

(this patch was generated in some part by [`claude-code`][claude-code-gh])
[claude-code-gh]: https://github.com/anthropics/claude-code
2026-03-17 15:06:30 -04:00
Gud Boi d5da993b0a Tweak y-move to `400` to align better with reset-dialog box 2026-03-11 15:51:41 -04:00
Gud Boi 4d394a1897 Drop bp from duration mismatch branch in `.ib.api.Client.bars()` 2026-03-11 15:51:41 -04:00
Gud Boi 8620c0fd45 Handle VNC reset-dialog in `vnc_click_hack()`
Add TAB + ENTER key presses after the `Ctrl+Alt+<key>` hotkey
combo to auto-confirm the "simulate a reset?" dialog that IB
gateway sometimes shows.

Deats,
- press `ISO_Enter` before click to dismiss any prior active
  dialog window.
- add post-hotkey loop sending `Tab` then `KP_Enter` with
  `asyncio.sleep()` delays to handle the confirmation dialog.
- add `asyncio` import.

Also,
- capture VNC connect error as `vnc_err` and log it instead of
  falling through to `try_xdo_manual()`.
- comment-out `try_xdo_manual()` fallback in VNC error path.
- reformat `client.press()` call to multiline style.
- reformat `RuntimeError` raise to multiline style with `!r`.

(this commit msg was generated in some part by [`claude-code`][claude-code-gh])
[claude-code-gh]: https://github.com/anthropics/claude-code
2026-03-11 15:51:41 -04:00
Gud Boi 31bf5f55d0 Augment `.ib.symbols` search with more logging
Refactor `open_symbol_search()` to use `partial()` for nursery task
spawning and add detailed query->results logging via `ppfmt()`.

Deats,
- change `extend_results()` to accept `target` callable +
  `pattern` + `**kwargs` and invoke inside, instead of receiving
  a pre-called awaitable; use `partial()` to pass args.
- add `ppfmt()` formatted logging of search query params and
  results including client class + method repr.
- change `print()` -> `log.exception()` for `Lagged` overrun.
- bump `upto=5` -> `upto=10` for `search_symbols()` call.

Also for styling,
- add type some missing type annots.
- add multiline style to `or` conditionals in pattern check.
- reformat log msgs to multiline style throughout.
- use `ppfmt()` for fuzzy match debug log.
- rename nursery `sn` -> `tn`.
- add TODO comment about `assert 0` hang.

(this commit msg was generated in some part by [`claude-code`][claude-code-gh])
[claude-code-gh]: https://github.com/anthropics/claude-code
2026-03-11 15:51:41 -04:00
Gud Boi adf1b18d82 Handle `str`-errors in `.ib.broker` trade events
Add `isinstance()` dispatch for the `'error'` event case in
`deliver_trade_events()` to handle `ib_async` sometimes emitting plain
`str` error items instead of the previously expected `dict`.

Deats,
- add `isinstance(err, dict)` branch for the standard case with
  `error_code`, `reason`, and `reqid` fields.
- add `isinstance(err, str)` branch to parse error strings of the
  form `'[code 104] connection failed'` into `code` and `reason`.
- set `reqid: str = '<unknown>'` for string-form errors since
  there's no request ID available.
- update `err` type annot to `dict|str`.

(this commit msg was generated in some part by [`claude-code`][claude-code-gh])
[claude-code-gh]: https://github.com/anthropics/claude-code
2026-03-11 15:51:41 -04:00
Gud Boi 1fa68181df Warn instead of raise on `start_dt`-trimmed frames
Downgrade the `start_dt`-trimming check in `open_history_client()`
from a `RuntimeError` raise to a warning log, allowing the caller
to still receive a (shorter) frame of bars (though we may need to still
specially handle such cases in the backfiller's biz logic layer).

Deats,
- add `trimmed_bars.size` guard to skip check on empty results.
- change condition to `>=` and log a warning with the short-frame
  size instead of raising.
- comment-out `raise RuntimeError` and breakpoint for future
  removal once confident.
- add docstring-style comment on `start_dt=` kwarg noting that
  `Client.bars()` doesn't truly support it (uses duration-style
  queries internally).

(this commit msg was generated in some part by [`claude-code`][claude-code-gh])
[claude-code-gh]: https://github.com/anthropics/claude-code
2026-03-11 15:51:41 -04:00
Gud Boi f78362b2e4 Handle ambiguous futes contracts in `get_fute()`
Use (the only available in `ib_async`) `returnAll=True` in
`qualifyContractsAsync()` calls within `get_fute()` and handle the case
where IB returns a list of ambiguous contract matches instead of
a single result.

Deats,
- add `returnAll=True` to both `ContFuture` and `Future`
  qualification calls.
- add `isinstance(con, list)` check after unpacking first result
  to detect ambiguous contract sets.
- log warning with input params and matched contracts when
  ambiguous.
- update return type annot to `Contract|list[Contract]`.

Also,
- handle list-of-contracts case in `find_contracts()` by unpacking
  `*contracts` into the `qualifyContractsAsync()` call.
- reformat `qualifyContractsAsync()` calls to multiline style.

(this commit msg was generated in some part by [`claude-code`][claude-code-gh])
[claude-code-gh]: https://github.com/anthropics/claude-code
2026-03-11 15:51:41 -04:00
Gud Boi 5ff1a9ca45 Fall back to `Contract.exchange` in `has_holiday()`
Use `con.exchange` as fallback when `con.primaryExchange` is empty
in `has_holiday()` to handle contracts like futures that don't
always set a `primaryExchange`.

Deats,
- extract `con: Contract` from `con_deats.contract` for reuse.
- use `con.primaryExchange or con.exchange` to ensure a valid
  exchange code is always passed to the calendar lookup.
- add `Contract` to `TYPE_CHECKING` imports.

(this commit msg was generated in some part by [`claude-code`][claude-code-gh])
[claude-code-gh]: https://github.com/anthropics/claude-code
2026-03-11 15:51:41 -04:00
Gud Boi ca9b1f80fe Remap non-std IB exchange values
Add exchange name translation in `.ib.venues.has_holiday()` to handle
non-standard exchange codes when looking up holiday gaps..

Deats,
- add an ad-hoc lookup dict to remap an IB `Contract.primaryExchange` val
  which doesn't exist in the `exchange_calendars`'s alias set.
- use `.get()` with fallback to map `exch` to new `std_exch` and pass
  that to `xcals.get_calendar()`.
- add the case i just caught, `'ARCA'` -> `'ARCX'` to the table when i loaded
  the `gld.arca.ib` market..

(this commit msg was generated in some part by [`claude-code`][claude-code-gh])
[claude-code-gh]: https://github.com/anthropics/claude-code
2026-03-11 15:51:41 -04:00
Gud Boi 6f828ca379 Handle unknown order statuses in `.ib.broker`
Add fallback handling for unknown IB order status strings to
avoid crashes when IB returns unexpected status values.

Deats,
- add `'ValidationError': 'error'` mapping to `_statuses` dict.
- use `.get()` with `'error'` default instead of direct dict
  lookup for `status.status`.
- add `elif status_str == 'error'` block to log unknown status
  values.
- add type annots to `event_name` and `item` in
  `deliver_trade_events()` loop.

Also,
- reformat log msg in `deliver_trade_events()` to multiline.
- drop extra conditional in `if status_str == 'filled'` check.

(this commit msg was generated in some part by [`claude-code`][claude-code-gh])
[claude-code-gh]: https://github.com/anthropics/claude-code
2026-03-11 15:51:41 -04:00
Gud Boi 758251e252 Port `.ib` backend from `ib_insync` to `ib_async`
Migrate the IB broker backend to use `ib_async` (the actively maintained
fork) instead of the now stale, original `ib_insync` lib.

Deats,
- update `pyproject.toml` dep: drop `ib-insync` pin, add
  `ib-async>=2.1.0`.
- update lock file with `ib-async` and its new `aeventkit` dep (which
  i guess replaces `eventkit`).
- obvi, change all `ib_insync` imports to `ib_async` across `.ib.*`.
- update docs and select internal comments referencing the original lib.

Also,
- drop unused `ledger_dict` init in `_flex_reports.load_flex_trades()`.
- fix union type annot style: `dict | None` -> `dict|None`.
- strip `.tzinfo` from `lastTimeStamp` in `normalize()` to avoid
  IPC codec issues with `ib_async`'s `timezone.utc` injection.
- pop `'defaults'` from ticker data dict in `normalize()` to avoid
  non-serializable `timezone` objects and warning-log in such
  cases.

(this commit msg was generated in some part by [`claude-code`][claude-code-gh])
[claude-code-gh]: https://github.com/anthropics/claude-code
2026-03-11 15:51:41 -04:00
Gud Boi 6834dd4ce6 Capture `cons` in `Client.get_fute()`
That is to be able to (eventually) introspect "ambiguous" contract sets
once we move to `ib_async` and its `returnAll: bool` now offered by
`IB.qualifyContractsAsync()`,

https://github.com/ib-api-reloaded/ib_async/blob/main/ib_async/ib.py#L2115

Also, tweak some type type annots to multline style in sibling mods.
2026-03-11 15:51:41 -04:00
Gud Boi d17e6ab5d9 Add holiday-gap detection via `exchange_calendars`
Integrate `exchange_calendars` lib to detect market holidays in
gap-checking logic via new `.ib.venues.has_holiday()` helper!

The `.ib.venues` impl deats,
- add  a new `has_holiday()` using `xcals.get_calendar()` and friends
  for sanity checking a venue's holiday closure-gaps.
  * final holiday detection-check is basically,
   `(cash_gap := (next_open - prev_close)) > period`
- include `time_step_s` param to `is_venue_closure()` for boundary
  tolerance checks.
  * let's us expand closure-time checks to include `+/-time_step_s`
    "off-by-one-`timeframe`-sample" edge case ranges.
- add real docstring to `has_weekend()`.

In `.ib.api` refine usage for ^ changes,
- move `is_venue_open()` call + tz-convert outside gap check
- use a walrus to capture `has_closure_gap` from `is_venue_closure()`
- add a `not has_closure_gap` condition to the
  mismatched-duration/short-frame warning block to avoid needless warns.
- keep duration-based "short-frame" log as `.error()` but toss in a bp
  so (somone can) umask to figure out wtf is going on..
  * we should **never** really hit this path unless there's a valid bug
    or data issue with IB/GFIS!
  * keep recursion path masked-out just leave a `breakpoint()` for now.

Also some logger updates,
- import `get_logger()` from top-level `piker.log` vs `.ib._util` which
  was always kinda wrong..
- change `NonShittyIB._logger` to use `__name__` vs literal.

(this commit msg was generated in some part by [`claude-code`][claude-code-gh])
[claude-code-gh]: https://github.com/anthropics/claude-code
2026-02-23 13:41:00 -05:00
Gud Boi 1417c49051 Adjust type annots in binance and IB symbol mods
Namely, again switching `|`-union syntax to rm adjacent white space.

Also, flip to multiline style for threshold comparison in
`.binance.feed` and change gap-check threshold to `timeframe` (vs
a hardcoded `60`s) in the `get_ohlc()` closure.

(this commit msg was generated in some part by [`claude-code`][claude-code-gh])
[claude-code-gh]: https://github.com/anthropics/claude-code
2026-02-23 13:41:00 -05:00
Gud Boi 044afb0f6e Use `.ib.venues.is_venue_open()` in `.feed`
In `.ib.feed.stream_quotes()` specifically that is since time-range
checking code was moved to the new sub-mod.

Deats,
- drop import of old `is_current_time_in_range()` from `._util`
- change `get_bars()` sig: `end_dt`/`start_dt` to `datetime|None`
- comment-out `breakpoint()` in `open_history_client()`

Styling,
- add multiline style to conditionals and tuple unpacks
- fix type annotation: `Contract|None` vs `Contract | None`
- fix backticks in comment: `ib_insync` vs `ib_async`

(this commit msg was generated in some part by [`claude-code`][claude-code-gh])
[claude-code-gh]: https://github.com/anthropics/claude-code
2026-02-23 13:41:00 -05:00
Gud Boi c96ecdab75 Add venue-closure gap-detection in `.ib.api.Client.bars()`
With all detection logic coming from our new `.ib.venues` helpers
allowing use to verify IB's OHLC bars frames don't contain unexpected
time-gaps.

`Client.bars()` new checking deats,
- add `is_venue_open()`, `has_weekend()`, `sesh_times()`, and
  `is_venue_closure()` checks when `last_dt < end_dt`
- always calc gap-period in local tz via `ContractDetails.timeZoneId`.
- log warnings on invalid non-closure gaps, debug on closures for now.
- change recursion case to just `log.error()` + `breakpoint()`; we might end
  up tossing it since i don't think i could ever get it to be reliable..
  * mask-out recursive `.bars()` call (likely unnecessary).
- flip `start_dt`/`end_dt` param defaults to `None` vs epoch `str`.
- update docstring to clarify no `start_dt` support by IB
- add mod level `_iso8601_epoch_in_est` const to keep track of orig
  param default value.
- add multiline style to return type-annot, type all `pendulum` objects.

Also,
- uppercase `Crypto.symbol` for PAXOS contracts in `.find_contracts()`,
  tho now we're getting a weird new API error i left in a todo-comment..

(this commit msg was generated in some part by [`claude-code`][claude-code-gh])
[claude-code-gh]: https://github.com/anthropics/claude-code
2026-02-23 13:41:00 -05:00
Gud Boi e1e59453a9 Mv `parse_trading_hours()` from `._util` to `.venues`
It was an AI-model draft that we can prolly toss but figured might as
well org it appropriately.

(this commit msg was generated in some part by [`claude-code`][claude-code-gh])
[claude-code-gh]: https://github.com/anthropics/claude-code
2026-02-23 13:41:00 -05:00
Gud Boi d784af9df9 Add `.ib.venues` for mkt-venue-closure checkers
Introduce set of helper-fns for detecting venue open/close status,
session start/end times, and related time-gap detection using
`pendulum`.

Deats,
- add `iter_sessions()` to yield `pendulum.Interval`s from
  a `ContractDetails` instance.
- add `is_venue_open()` to check if active at a given time.
- add `is_venue_closure()` to detect valid closure gaps.
- add `sesh_times()` to extract weekday-agnostic open/close times.
- add `has_weekend()` to check for Sat/Sun in interval.
- move in lowlevel `is_current_time_in_range()` for checking a
  datetime within a `sesh: pendulum.Interval`.

(this commit msg was generated in some part by [`claude-code`][claude-code-gh])
[claude-code-gh]: https://github.com/anthropics/claude-code
2026-02-23 13:41:00 -05:00
Gud Boi 2d0005ce48 Drop info-level `.pause()`-es used while devving 2026-02-23 13:26:46 -05:00
Gud Boi 9a97c477e2 Use `name=__name__` for logs throughout `.service`
Change all `.service` sub-modules to use `get_logger(name=__name__)`
for per-submod instances vs a shared `._util.log`.

Deats,
- import `get_logger()` and `get_console_log()` from top-level
  `piker.log` instead of `._util` for all.
- drop `log` and `get_console_log()` partial from `._util`.
- add `name=subsys` kwarg to `get_console_log()` call in
  `_actor_runtime.maybe_open_pikerd()`.
- add `name='piker.service'` to `get_console_log()` in
  `_ahab.open_ahabd()`.
- change default `loglevel` from `None` to `'cancel'` in
  `_ahab.open_ahabd()`.
- add sanity check: `assert log.name == 'piker.service'` in
  `_daemon.maybe_spawn_daemon()`.
- change `print()` -> `log.info()` in `_registry.find_service()`.
- drop stray `from piker.service._util import log` import in
  `brokers._daemon.spawn_brokerd()`.

Styling/cleanups,
- drop blank lines from various fn sigs.
- do more sin-ws union type annots.
- add more multiline style to `or` expressions in `_actor_runtime` and
  `_registry`.
- update `._util` docstring with TODO about `import`-time console
  log setup.
- add TODO comments in `_registry` about UDS registry support.
- use `.aid.uid` from actor in `_registry.open_registry()`.
- add intermediate var `reg_addrs` in `_registry.open_registry()` (bc
  i was tracing rtvs value issues in `tractor`).
- add `pformat` import to `.elastic` (code path is currently
  not used but figured might as well appease the linter..)

(this commit msg was generated in some part by [`claude-code`][claude-code-gh])
[claude-code-gh]: https://github.com/anthropics/claude-code
2026-02-23 12:22:48 -05:00
Gud Boi aa403bd390 Pass `loglevel` down through `.data` callstack
Add `loglevel` param propagation across the data feed and sampling
subsystems to enable proper console log setup in downstream (distibuted)
subactor tasks. This ensures sampler and history-mgmt tasks receive the
same loglevel as their parent `.data.feed` tasks.

Deats,
- add `loglevel: str|None` param to `register_with_sampler()`,
  `maybe_open_samplerd()`, and `open_sample_stream()`.
- pass `loglevel` through to `get_console_log()` in
  `register_with_sampler()` with fallback to actor `loglevel`.
- use `partial()` in `allocate_persistent_feed()` to pass
  `loglevel` to `manage_history()` at task-start.
- add `loglevel` param to `manage_history()` with default
  `'warning'` and pass through to `open_sample_stream()` from there.
- capture `loglevel` var in `brokers.cli.search()` and pass to
  `symbol_search()` call.

Also,
- drop blank lines in fn sigs for consistency with piker style.
- add debug bp in `open_feed()` when `loglevel != 'info'`.

(this commit msg was generated in some part by [`claude-code`][claude-code-gh])
[claude-code-gh]: https://github.com/anthropics/claude-code
2026-02-23 12:22:48 -05:00
Gud Boi 50ffc1095b Use `__name__` for loggers across most sub-mods
Change most sub-modules to use `get_logger(name=__name__)` for
per-leaf-module `log` instances vs previous subpkg-level/shared refs.

Primary changes,
- import `get_[console_]logger()` from top-level `piker.log` across leaf
  mods.
- change any `<subsys>._util.log` logger-instances as well (though this
  approach should no longer be used since it masks the endpoint module's
  emissions.

Also,
- add a defaulted `loglevel: str` param to all `open_trade_dialog()`
  endpoints, anticipating it being passed in by `.clearing`-engine.
- call `get_console_log(level=loglevel, name=__name__)` in each trade
  dialog ep to enable per-`brokerd`-backend console writing.
- drop `get_logger` from `.brokers.__all__` exports
- fix type annotations: `str|None` vs `str | None`
- add TODOs for,
  * comments in `._util` about multi-subsys logging
  * `.accounting.__init__` about console log setup

(this commit msg was generated in some part by [`claude-code`][claude-code-gh])
[claude-code-gh]: https://github.com/anthropics/claude-code
2026-02-23 12:22:48 -05:00
Gud Boi 437d87ab5f Use `__name__` for loggers across `.ib` sub-mods
Change all `.ib` sub-modules to use `get_logger(name=__name__)`
for per-module logger instances vs shared `._util.log`.

Deats,
- change `._util` to use `__name__` vs literal string.
- change `.broker`, `.feed`, `.ledger`, `.symbols` to import
  `get_logger()` from top-level `.log` and call with `__name__`.
- drop `log` imports from `._util` in all affected mods.

Also,
- drop trailing comma in `.cli.services()` conditional for `loglevel`
  passthrough -> fixes an actual kwargs bug!!

(this commit msg was generated in some part by [`claude-code`][claude-code-gh])
[claude-code-gh]: https://github.com/anthropics/claude-code
2026-02-23 12:22:48 -05:00
Gud Boi 6d896eeed2 .brokers._daemon: enable `tractor` log in `brokerd`
Also,
- capture `Actor.loglevel` in `tll` var for reuse (when `loglevel` is
  null) and pass `bool`-ed as new `with_tractor_log`-flag.
- add `with_tractor_log=bool(tll)` to `.get_console_log()`
- add assertion check for logger name.
- comment-out `tractor.trionics.collapse_eg()` context for now, pretty
  sure we don't need it and it just ends up adding extra logging
  overhead for no good reason (warnings on various `trio` internal
  cancelled-maskings, etc).
- change type annotation: `str|None` vs `str | None`.

(this commit msg was generated in some part by [`claude-code`][claude-code-gh])
[claude-code-gh]: https://github.com/anthropics/claude-code
2026-02-23 12:22:48 -05:00
Gud Boi a940018721 Adjust binance stale-bar detection to 2x tolerance
Change the stale-bar check in `.binance.feed` from `timeframe` to
`timeframe * 2` tolerance to avoid false-positive pauses when bars
are slightly delayed but still within acceptable bounds.

Styling,
- add walrus operator to capture `_time_step` for debugger
  inspection.
- add comment explaining the debug purpose of this check.

(this commit msg was generated in some part by [`claude-code`][claude-code-gh])
[claude-code-gh]: https://github.com/anthropics/claude-code
2026-02-22 22:08:35 -05:00
Gud Boi b90edf95a7 .ib.feed: only set `feed_is_live` after first quote
Move `feed_is_live.set()` to after receiving the first valid
quote instead of setting early on venue-closed path. Prevents
sampler registration when no live data expected.

Also,
- drop redundant `.set()` call in quote iteration loop
- add TODO note about sleeping until venue opens vs forever
- init `first_quote: dict` early for consistency

(this commit msg was generated in some part by [`claude-code`][claude-code-gh])
[claude-code-gh]: https://github.com/anthropics/claude-code
2026-02-22 22:08:35 -05:00
Gud Boi be4adfc202 ib.feed: drop legacy "quote-with-vlm" polling
Since now we explicitly check each mkt's venue hours now we don't need
this mega hacky "waiting on a quote with real vlm" stuff to determing
whether historical data should be loaded immediately. This approach also
had the added complexity that we needed to handle edge cases for tickers
(like xauusd.cmdty) which never have vlm.. so it's nice to be rid of it
all ;p
2026-02-22 22:08:35 -05:00
Gud Boi fd92cd99c2 .ib._util: ignore attr err on click-hack twm wakeups? 2026-02-22 22:08:35 -05:00
Gud Boi 40cbc8546d .ib.feed: trim bars frame to `start_dt` 2026-02-22 22:08:35 -05:00
Gud Boi e3d7077f18 ib._util: ignore timeout-errs when crash-handling `pyvnc` connects 2026-02-22 22:08:35 -05:00
Gud Boi 7ddcf5893e Lul, woops compare against first-dt in `.ib.feed` bars frame.. 2026-02-22 22:08:35 -05:00
Gud Boi 11e95d9cbf Catch too-early ib hist frames
For now by REPLing them and raising an RTE inside `.ib.feed` as well as
tracing any such cases that make it (from other providers) up to the
`.tsp._history` layer during null-segment backfilling.
2026-02-22 22:08:34 -05:00
Gud Boi bd418078ca ib: up API timeout default for remote host conns 2026-02-22 22:08:34 -05:00
Gud Boi fff9de9aec binance: add API changelog link 2026-01-21 22:19:13 -05:00
Gud Boi b7cdbd89d4 binance: mk `AggTrade.nq` optional..
Oof! my bad.
Turns out spot pairs don't provide the `.nq` field looks like..
i guess i should not just test `.perp.` pairs all the time!

Bp
2026-01-21 19:59:07 -05:00
Tyler Goodlet 58654915ac Set `.bs_mktid` on all IB position-msg emissions.. 2026-01-07 13:41:07 -05:00
Tyler Goodlet ccb4f79170 Bump various `.brokers.core` doc string content/style 2026-01-07 13:39:34 -05:00
Tyler Goodlet 1089de024a ib: multiline stylings, typing, timeout report 2026-01-07 13:39:34 -05:00
Tyler Goodlet 05bdac5542 Woops, fix to read `.api_port` ref from the `Client.ib.client`.. 2026-01-07 13:39:34 -05:00
Tyler Goodlet a392185d2f Support per-`ib.vnc_addrs` vnc passwords
Such that the `brokers.toml` can contain any of the following
<port> = dict|tuple styles,

```toml
    [ib.vnc_addrs]
    4002 = {host = 'localhost', port = 5900, pw = 'doggy'}  # host, port, pw
    4002 = {host = 'localhost', port = 5900}  # host, port, pw
    4002 = ['localhost', 5900]  # host, port, pw
```

With the first line demonstrating a vnc-server password (as normally set
via a `.env` file in the `dockering/ib/` subdir) with the `pw =` field.
This obviously removes the hardcoded `'doggy'` password from prior.

Impl details in `.brokers.ib._util`:
- pass the `ib.api.Client` down into `vnc_click_hack()` doing all config
 reading within and removing host, port unpacking in the callingn
 `data_reset_hack()`.
- also pass the client `try_xdo_manual()` and comment (with plans to
  remove) the recently added localhost-only fallback section since
  we now have a fully working py vnc client again with `pyvnc` B)
- in `vnc_click_hack()` match for all the possible config line styles
  and,
  * pass any `pw` field to `pyvncVNCConfig`,
  * continue matching host, port without password,
  * fallthrough to raising a val-err when neither ^ match.
2026-01-07 13:39:34 -05:00
Tyler Goodlet 6ff9ba2e78 ib.feed: better no-bars error-log message format 2026-01-07 13:39:34 -05:00
Tyler Goodlet c1fbf70c62 Switch to `pyvnc` for IB reset hackz
It actually works for vncAuth(2) (thank god!) which the previous
`asyncvnc` **did not**, and seems to be mostly based on the work
from the `asyncvnc` author anyway (so all my past efforts don't seem to
have been in vain XD).

NOTE, the below deats ended up being factored in earlier into the
`pyproject.toml` alongside nix(os) support needed for testing and
landing this history. As the such, the comments are the originals but
the changes are not.

Deats,
- switch to `pyvnc` async API (using `asyncio` again obvi) in
  `.ib._util._vnc_click_hack()`.
- add `pyvnc` as src installed dep from GH.
- drop `asyncvnc` as dep.

Other,
- update `pytest` version range to avoid weird auto-load plugin exposed
  by `xonsh`?
- add a `tool.pytest.ini_options` to project file with vars to,
  - disable that^ `xonsh` plug using `addopts = '-p no:xonsh'`.
  - set a `testpaths` to avoid running anything but that subdir.
  - try out the `'progress'` style console output (does it work?).
2026-01-07 13:23:41 -05:00
Tyler Goodlet 269b8158e6 Convert remaining `.to_asyncio.open_channel_from()` to `chan` fn-sig usage 2026-01-07 13:23:41 -05:00
Tyler Goodlet 728a6f428e `ib.feed`: finally solve `push()` exc propagation
Such that if/when the `push()` ticker callback (closure) errors
internally, we actually eventually bubble the error out-and-up from the
`asyncio.Task` and from there out the `.to_asyncio.open_channel_from()` to
the parent `trio.Task`..

It ended up being much more subtle to solve then i would have liked
thanks to,

- whatever `Ticker.updateEvent.connect()` does behind the scenes in
  terms of (clearly) swallowing with only log reporting any exc raised
  in the registered callback (in our case `push()`),

- `asyncio.Task.set_excepion()` never working and instead needing to
  resort to `Task.cancel()`, catching `CancelledError` and re-raising
  the stashed `maybe_exc` from `push()` when set..

Further this ports `.to_asyncio.open_channel_from()` usage to use
the new `chan: tractor.to_asyncio.LinkedTaskChannel` fn-sig API, namely
for `_setup_quote_stream()` task. Requires the latest `tractor` updates
to the inter-eventloop-chan iface providing a `.set_nowait()` and
`.get()` for the `asyncio`-side.

Impl deats within `_setup_quote_stream()`,
- implement `push()` error-bubbling by adding a `maybe_exc` which can be
  set by that callback itself or by its registering task; when set it is
  both,
  * reported on by the `teardown()` cb,
  * re-raised by the terminated (via `.cancel()`) `asyncio.Task` after
    woken from its sleep, aka "cancelled" (since that's apparently one
    of the only options.. see big rant further todo comments).
- add explicit error-tolerance-tuning via a `handler_tries: int` counter
  and `tries_before_raise: int` limit such that we only bubble
  a `push()` raised exc once enough tries have consecutively failed.
- as mentioned, use the new `chan` fn-sig support and thus the new
  method API for `asyncio` -> `trio` comms.
- a big TODO XXX around the need to use a better sys for terminating
  `asyncio.Task`s whether it's by delegating to some `.to_asyncio`
  internals after a factor-out OR by potentially going full bore `anyio`
  throughout `.to_asyncio`'s impl in general..
- mk `teardown()` use appropriate `log.<level>()`s based on outcome.

Surroundingly,
- add a ton of doc-strings to mod fns previously missing them.
- improved / added-new comments to `wait_on_data_reset()` internals and
  anything changed per ^above.

NOTE, resolved conflicts on `piker/brokers/ib/feed.py` due to
`brokers_refinery` commit:

d809c797 `.brokers.ib.feed`: better `tractor.to_asyncio` typing and var naming throughout!
2026-01-07 13:23:41 -05:00
Tyler Goodlet 323840fdfc `ib`: various type-annot, multiline styling and todos updates 2026-01-07 13:23:41 -05:00
Tyler Goodlet 27c83fae0c ib: add venue-hours checking
Such that we can avoid other (pretty unreliable) "alternative" checks to
determine whether a real-time quote should be waited on or (when venue
is closed) we should just signal that historical backfilling can
commence immediately.

This has been a todo for a very long time and it turned out to be much
easier to accomplish than anticipated..

Deats,
- add a new `is_current_time_in_range()` dt range checker to predicate
  whether an input range contains `datetime.now(start_dt.tzinfo)`.
- in `.ib.feed.stream_quotes()` add a `venue_is_open: bool` which uses
  all of the new ^^ to determine whether to branch for the
  short-circuit-and-do-history-now-case or the std real-time-quotes
  should-be-awaited-since-venue-is-open, case; drop all the old hacks
  trying to workaround not figuring that venue state stuff..

Other,
- also add a gpt5 composed parser to `._util` for the
  `ib_insync.ContractDetails.tradingHours: str` for before i realized
  there was a `.tradingSessions` property XD
- in `.ib_feed`,
  * add various EG-collapsings per recent tractor/trio updates.
  * better logging / exc-handling around ticker quote pushes.
  * stop clearing `Ticker.ticks` each quote iteration; not sure if this
    is needed/correct tho?
  * add masked `Ticker.ticks` poll loop that logs.
- fix some `str.format()` usage in `._util.try_xdo_manual()`

NOTE, resolved conflicts on `piker/brokers/ib/feed.py` due to
rebasing onto up stream `brokers_refinery` commit,

d809c797 `.brokers.ib.feed`: better `tractor.to_asyncio` typing and var naming throughout
2026-01-07 13:23:41 -05:00
Tyler Goodlet e92d5baf99 ib: never relay "Warning:" errors to EMS..
You'd think they could be bothered to make either a "log" or "warning"
msg type instead of a `type='error'`.. but alas, this attempts to detect
all such "warning"-errors and never proxy them to the clearing engine
thus avoiding the cancellation of any associated (by `reqid`)
pre-existing orders (control dialogs).

Also update all surrounding log messages to a more multiline style.
2026-01-07 13:23:41 -05:00
Tyler Goodlet b1111bf9b0 ib: jig `.data_reset_hack()` with vnc-client failover
Since apparently porting to the new docker container enforces using
a vnc password and `asyncvnc` seems to have a bug/mis-config whenever
i've tried a pw over a wg tunnel..?

Soo, this tries out the old `i3ipc`-win-focus + `xdo` click hack when
the above fails.

Deats,
- add a mod-level `try_xdo_manual()` to wrap calling
  `i3ipc_xdotool_manual_click_hack()` with an oserr handler, ensure we
  don't bother trying if `i3ipc` import fails beforehand tho.
- call ^ from both the orig case block and the failover from the
  vnc-client case.
- factor the `+no_setup_msg: str` out to mod level and expect it to be
  `.format()`-ed.
- refresh todo around `asyncvnc` pw ish..
- add a new `i3ipc_fin_wins_titled()` window-title scanner which
  predicates input `titles` and delivers any matches alongside the orig
  focused win at call time.
- tweak `i3ipc_xdotool_manual_click_hack()` to call ^ and remove prior
  unfactored window scanning logic.
2026-01-07 13:23:41 -05:00
Tyler Goodlet 9be8ca6097 binance: add `AggTrade.nq: float`: "normal quantity" field.. 2026-01-06 23:43:44 -05:00