Such that the `brokers.toml` can contain any of the following
<port> = dict|tuple styles,
```toml
[ib.vnc_addrs]
4002 = {host = 'localhost', port = 5900, pw = 'doggy'} # host, port, pw
4002 = {host = 'localhost', port = 5900} # host, port, pw
4002 = ['localhost', 5900] # host, port, pw
```
With the first line demonstrating a vnc-server password (as normally set
via a `.env` file in the `dockering/ib/` subdir) with the `pw =` field.
This obviously removes the hardcoded `'doggy'` password from prior.
Impl details in `.brokers.ib._util`:
- pass the `ib.api.Client` down into `vnc_click_hack()` doing all config
reading within and removing host, port unpacking in the callingn
`data_reset_hack()`.
- also pass the client `try_xdo_manual()` and comment (with plans to
remove) the recently added localhost-only fallback section since
we now have a fully working py vnc client again with `pyvnc` B)
- in `vnc_click_hack()` match for all the possible config line styles
and,
* pass any `pw` field to `pyvncVNCConfig`,
* continue matching host, port without password,
* fallthrough to raising a val-err when neither ^ match.
It actually works for vncAuth(2) (thank god!) which the previous
`asyncvnc` **did not**, and seems to be mostly based on the work
from the `asyncvnc` author anyway (so all my past efforts don't seem to
have been in vain XD).
NOTE, the below deats ended up being factored in earlier into the
`pyproject.toml` alongside nix(os) support needed for testing and
landing this history. As the such, the comments are the originals but
the changes are not.
Deats,
- switch to `pyvnc` async API (using `asyncio` again obvi) in
`.ib._util._vnc_click_hack()`.
- add `pyvnc` as src installed dep from GH.
- drop `asyncvnc` as dep.
Other,
- update `pytest` version range to avoid weird auto-load plugin exposed
by `xonsh`?
- add a `tool.pytest.ini_options` to project file with vars to,
- disable that^ `xonsh` plug using `addopts = '-p no:xonsh'`.
- set a `testpaths` to avoid running anything but that subdir.
- try out the `'progress'` style console output (does it work?).
Such that if/when the `push()` ticker callback (closure) errors
internally, we actually eventually bubble the error out-and-up from the
`asyncio.Task` and from there out the `.to_asyncio.open_channel_from()` to
the parent `trio.Task`..
It ended up being much more subtle to solve then i would have liked
thanks to,
- whatever `Ticker.updateEvent.connect()` does behind the scenes in
terms of (clearly) swallowing with only log reporting any exc raised
in the registered callback (in our case `push()`),
- `asyncio.Task.set_excepion()` never working and instead needing to
resort to `Task.cancel()`, catching `CancelledError` and re-raising
the stashed `maybe_exc` from `push()` when set..
Further this ports `.to_asyncio.open_channel_from()` usage to use
the new `chan: tractor.to_asyncio.LinkedTaskChannel` fn-sig API, namely
for `_setup_quote_stream()` task. Requires the latest `tractor` updates
to the inter-eventloop-chan iface providing a `.set_nowait()` and
`.get()` for the `asyncio`-side.
Impl deats within `_setup_quote_stream()`,
- implement `push()` error-bubbling by adding a `maybe_exc` which can be
set by that callback itself or by its registering task; when set it is
both,
* reported on by the `teardown()` cb,
* re-raised by the terminated (via `.cancel()`) `asyncio.Task` after
woken from its sleep, aka "cancelled" (since that's apparently one
of the only options.. see big rant further todo comments).
- add explicit error-tolerance-tuning via a `handler_tries: int` counter
and `tries_before_raise: int` limit such that we only bubble
a `push()` raised exc once enough tries have consecutively failed.
- as mentioned, use the new `chan` fn-sig support and thus the new
method API for `asyncio` -> `trio` comms.
- a big TODO XXX around the need to use a better sys for terminating
`asyncio.Task`s whether it's by delegating to some `.to_asyncio`
internals after a factor-out OR by potentially going full bore `anyio`
throughout `.to_asyncio`'s impl in general..
- mk `teardown()` use appropriate `log.<level>()`s based on outcome.
Surroundingly,
- add a ton of doc-strings to mod fns previously missing them.
- improved / added-new comments to `wait_on_data_reset()` internals and
anything changed per ^above.
NOTE, resolved conflicts on `piker/brokers/ib/feed.py` due to
`brokers_refinery` commit:
d809c797 `.brokers.ib.feed`: better `tractor.to_asyncio` typing and var naming throughout!
Such that we can avoid other (pretty unreliable) "alternative" checks to
determine whether a real-time quote should be waited on or (when venue
is closed) we should just signal that historical backfilling can
commence immediately.
This has been a todo for a very long time and it turned out to be much
easier to accomplish than anticipated..
Deats,
- add a new `is_current_time_in_range()` dt range checker to predicate
whether an input range contains `datetime.now(start_dt.tzinfo)`.
- in `.ib.feed.stream_quotes()` add a `venue_is_open: bool` which uses
all of the new ^^ to determine whether to branch for the
short-circuit-and-do-history-now-case or the std real-time-quotes
should-be-awaited-since-venue-is-open, case; drop all the old hacks
trying to workaround not figuring that venue state stuff..
Other,
- also add a gpt5 composed parser to `._util` for the
`ib_insync.ContractDetails.tradingHours: str` for before i realized
there was a `.tradingSessions` property XD
- in `.ib_feed`,
* add various EG-collapsings per recent tractor/trio updates.
* better logging / exc-handling around ticker quote pushes.
* stop clearing `Ticker.ticks` each quote iteration; not sure if this
is needed/correct tho?
* add masked `Ticker.ticks` poll loop that logs.
- fix some `str.format()` usage in `._util.try_xdo_manual()`
NOTE, resolved conflicts on `piker/brokers/ib/feed.py` due to
rebasing onto up stream `brokers_refinery` commit,
d809c797 `.brokers.ib.feed`: better `tractor.to_asyncio` typing and var naming throughout
You'd think they could be bothered to make either a "log" or "warning"
msg type instead of a `type='error'`.. but alas, this attempts to detect
all such "warning"-errors and never proxy them to the clearing engine
thus avoiding the cancellation of any associated (by `reqid`)
pre-existing orders (control dialogs).
Also update all surrounding log messages to a more multiline style.
Since apparently porting to the new docker container enforces using
a vnc password and `asyncvnc` seems to have a bug/mis-config whenever
i've tried a pw over a wg tunnel..?
Soo, this tries out the old `i3ipc`-win-focus + `xdo` click hack when
the above fails.
Deats,
- add a mod-level `try_xdo_manual()` to wrap calling
`i3ipc_xdotool_manual_click_hack()` with an oserr handler, ensure we
don't bother trying if `i3ipc` import fails beforehand tho.
- call ^ from both the orig case block and the failover from the
vnc-client case.
- factor the `+no_setup_msg: str` out to mod level and expect it to be
`.format()`-ed.
- refresh todo around `asyncvnc` pw ish..
- add a new `i3ipc_fin_wins_titled()` window-title scanner which
predicates input `titles` and delivers any matches alongside the orig
focused win at call time.
- tweak `i3ipc_xdotool_manual_click_hack()` to call ^ and remove prior
unfactored window scanning logic.
Handle new API field per 2025-12-02 update.
(this patch was generated in some part by [`claude-code`][claude-code-gh])
[claude-code-gh]: https://github.com/anthropics/claude-code
Lol, a cheeky unforeseen bug due to TOML's lack of a null type and
thinking i can render an `Optional` field on a `msgspec.Struct`
(defaulted to `None`) the `binance.symcache.toml` cache file..
I didn't catch this when i first updated to the 3.1 API in f7caa75228
because i never did a cache-files flush.. lesson learned and we **really
need tests for this**!!
Such that the next time i inevitably must debug the some order-request
error status or precision discrepancy, i have the mkt-symbol branch
ready to go. Also, switch to `'action': 'buy'|'sell' as action,` style
`case` matching instead of the post-`if` predicate style.
Topically, throughout various (seemingly) console-UX-affecting or benign
spots in the code base; nothing that required more intervention beyond
things superficial. A few spots also include `trio.Nursery` ref renames
(always to something with a `tn` in it) and log-level reductions to
quiet (benign) console noise oriented around issues meant to be solved
long..
Note there's still a couple spots i left with the loose-ify flag because
i haven't fully tested them without using the latest version of
`tractor.trionics.collapse_eg()`, but more then likely they should flip
over fine.
Commit this change separate from the (original) broader set applied to
the entire code base since the `.deribit.api` mod contained changes from
upstream max-pain work (from our very own @nt) which caused a noticeable
conflict and intros un-required changes from his work to re-enable
`deribit` support.
Note the original commit, "69eac7bb Spurious first-draft of EG
collapsing", applied similar changes through the rest of the code base.
AGAIN, this mod's change is only being broken out to minimize upstream
change conflicts due to updates to the `deribit` backend done earlier in
time-history.
Using `tractor.trionics.collapse_eg()` as needed to avoid, at the least,
crash-worthy (in debug-mode REPL-ing terms) nested cancellation egs that
exhibit on SIGINT/ctl-c of each "app" (chart & daemon).
Also a bit of renaming of all `trio.Nursery`s to `tn`, the new "task
nursery" shorthand-var-name being used in all our other `tractor`
related projects.
This must have broke at some point during the new `MktPair` and thus
`.fqme: str` updates; mas-o-menos the symbol key in the quote-msg-`dict`
was NOT set to the `MktPair.bs_fqme: str` value and thus wasn't being
processed by the downstream sampling and feed subsys.
So fix that as well as a few other refinements,
- set the `topic: mkt.bs_fqme` in quote msgs obvi.
- drop the "wait for first clearing vlm" quote poll loop; going to fix
the sampler to handle a `first_quote` without a `'last'` key.
- add some typing around calls to `get_mkt_info()`.
- rename `stream_messages()` -> `iter_normed_quotes()`.
Like other backends use the `AsyncClient` for all venue specific
client-sessions but change to allocating them inside `get_client()`
using an `AsyncExitStack` and inserting directly in the
`Client.venue_sesh: dict` table during init.
Supporting impl tweaks:
- remove most of the API client session building logic and instead make
`Client.__init__()` take in a `venue_sessions: dict` (set it to
`.venue_sesh`) and `conf: dict`, instead opting to do the http client
configuration inside `get_client()` since all that code only needs to
be run once.
|_load config inside `get_client()` once.
|_move session token creation into a new util func `init_api_keys()` and
also call it from `get_client()` factory; toss in an ex. toml section
config to the doc string.
- define `_venue_urls: dict[str, str]` (content taken from old static
`.venue_sesh` dict) at module level and feed them as `base_url: str`
inputs to the client create loop.
- adjust all call sigs in httpx-sesh-using methods, namely just
`._api()`.
- do a `.exch_info()` call in `get_client()` to cache the symbology
set.
Unrelated changes for various other outstanding buggers:
- to get futures feeds correctly loading when selected
from search (like 'XMRUSDT.USDTM.PERP'), expect a `MktPair` input to
`Client.bars()` such that the exact venue-key can be looked up (via
a new `.pair2venuekey()` meth) and then passed to `._api()`.
- adjust `.broker.open_trade_dialog()` to failover to paper engine when
there's no `api_key` key set for the `subconf` venue-key.
Like we do with other history backends to indicate lack of a data set.
This avoids any raise that will will bring down the backloader task with
some downstream error.
Raise a `ValueError` on no time index for now.
Gud Boi
Tyler Goodlet
Nelson Torres