The `proxy_msgs()` deadline covers both `ws.get_message()` and
`snd.send()`. Repeated socket-ready reports with no first relay
could therefore mean either a silent source or a blocked local
consumer; the old generic reset log could not distinguish them.
Track the currently awaited operation and report its phase, URL,
deadline, accumulated timeout count and reset threshold on each
timeout. Snapshot `NoBsWs._timeout` once per iteration so the log
reflects the deadline that actually elapsed. Keep the existing
shared deadline, counter behavior and reconnect policy intact.
Add two real loopback regressions: a silent websocket server and
one payload blocked on an unconsumed zero-capacity Trio relay.
Require the matching timeout phase and URL before the reset budget
is exhausted. The 15 selected Binance/websocket cases pass.
The human observes this after invalid `.usdm.` startup followed
by corrected `.usdtm.` input. Metadata accepts the typo but later
history lookup expects canonical keys. Record that investigation
as context; this diagnostic patch does not fix FQME validation,
failed-startup cleanup or prove the cause of corrected-feed stalls.
Prompt-IO: ai/prompt-io/opencode/20261028T180856Z_032580bf_timeout_phase.md
(this patch was generated in some part by `opencode` using
`gpt-6.1-sol` (`openai`))
An INFO pikerd still hid websocket connection-success messages.
The shared ._util.log emitter is piker.data, but open_feed_bus()
configured piker.data.feed. Errors reached bare stderr while
INFO diagnostics remained disabled on the actual emitter.
Configure the shared logger by its exact log.name and prefer
the actor's loglevel over a quieter per-client request for these
actor-wide transport diagnostics. Preserve the selected client
level passed to provider feed startup.
Also,
- retain the existing handshake-success report and add URL-tagged
readiness after fixture setup and first-message relay once per
connection, including replacement sockets.
- include the URL in receive-timeout reset errors so Binance's
independent bookTicker and aggTrade routes are distinguishable.
- prove real console INFO visibility through production feed
startup, stopping before provider lookup, then restore logging
state; the original code fails with the emitter at WARNING.
- extend the loopback recovery test to verify both readiness
reports and the first relay from the replacement connection.
First-message relay is raw websocket traffic, not proof that
parsed market data or chart updates have recovered. The 18
logging/Binance/websocket cases pass; live reporting qualification
remains separate.
This is a local follow-up, not a downstream cherry-pick. The
54da2973 websocket cleanup and 5adf77a9 namespace fix do not
configure this shared emitter. Relevant local histories and
dirty IB-history/TIPC deltas contain no equivalent wiring fix.
Prompt-IO: ai/prompt-io/opencode/20261028T165638Z_b95e7c79_reconnect_logging.md
(this patch was generated in some part by `opencode` using
`gpt-6.1-sol` (`openai`))
A Binance Ping racing local socket closure makes trio-websocket's
reader send a Pong while wsproto is LOCAL_CLOSING. That reader
nursery error escapes proxy_msgs(), then tears down brokerd and
pikerd instead of restoring quote delivery.
Classify NoBsWs.recon_errors around the entire websocket context,
with collapse_eg() outside it so a singleton reader exit group
reaches the reconnect handler. Invalidate the stale connected
event and socket reference before retrying. Keep retry sleep
unshielded so parent cancellation stops recovery promptly.
Deats,
- Add real loopback websocket coverage: TCP, handshake, Ping,
reader dispatch and wsproto's actual Pong validation.
- Control only the first client Ping's protocol-state ordering,
withholding its Close frame to prevent a peer response racing
the deterministic LOCAL_CLOSING reproducer.
- Require fresh data on a replacement connection, plus exactly
one connection when cancelling at the reader or retry boundary.
- Use RuntimeError for the unexpected-failure control:
LocalProtocolError is now a recoverable connection failure.
- Preserve the original unexpected exception with bare raise,
and assert its identity in the test. Log recoverable errors
with the websocket URL and the underlying traceback.
This is a local follow-up, not another downstream cherry-pick.
The earlier wrapper suppressed escaped communication failures;
97eb8347 deliberately re-raised them. Retain that outer error
visibility rather than restoring suppression without recovery.
The isolated baseline reproduces the live Pong failure; all 13
selected Binance/websocket cases pass with this fix. Human live
qualification remains separate from these loopback regressions.
Follow-up-to: 97eb834799
Original-equivalent: d50ab6124c
Related-history: 3d83b61f3f
Prompt-IO: ai/prompt-io/opencode/20261028T160846Z_7242dcb9_websocket_close.md
(this patch was generated in some part by `opencode` using
`gpt-6.1-sol` (`openai`))
- reject non-single-symbol feeds with `FeedInitializationError`
- route book and trade data across the correct USD-M endpoints
- accept the first matching L1 or trade quote as feed readiness
- preserve transport and startup-timeout failure context
- cover provider msg schemas and both readiness classes
Prompt-IO: ai/prompt-io/opencode/20260829T175542Z_e09e306a_prompt_io.md
(this patch was generated in some part by `opencode` using `gpt-5.6-sol` (`openai`))
- restart handled handshake failures before touching cancel state
- propagate `trio.Cancelled` without shielded retry throttling
- bound initial connection waits and expose transport failures
- cover real reconnect task paths in `test_websocket.py`
Prompt-IO: ai/prompt-io/opencode/20260901T024235Z_e09e306a_prompt_io.md
(this patch was generated in some part by `opencode` using `gpt-5.6-sol` (`openai`))
Binance retired its legacy futures websocket and split quote
traffic across `/public` and `/market`. Route `bookTicker` and
`aggTrade` through dedicated URLs, merge both sockets, and accept
the new `st` stream-type field.
Bound connection and first-quote startup so failed or silent
streams raise `DataUnavailable` instead of hanging `datad` and
chart consumers.
Also,
- derive feed mode from resolved `FutesPair` metadata
- keep direct URL subscriptions stable across reconnects
- cover USD-M routes, payloads, and startup timeouts
Prompt-IO: ai/prompt-io/opencode/20260826T222600Z_5f631e19_prompt_io.md
(this patch was generated in some part by `opencode` using `gpt-5.6-sol` (`openai`))
Import the source-equivalent EMS code and its original wrapper
regression. `clear_dark_triggers()` owns a non-strict child from
`quote_stream.subscribe()` for the lifetime of its clearing loop,
instead of changing a shared receiver's private lag policy.
LOCAL ADDITION,
Add a second regression using the production clearing loop and a
real broadcaster. Deliver a tick to EMS and a sibling, cancel EMS,
and prove the sibling remains usable with strict lag policy.
No order trigger or live broker is involved in this test.
Downstream replay,
The EMS code has the source patch's stable patch ID. The test file
also contains the local real-broadcaster case, so the complete
commit is not patch-identical. Preserve that extra coverage when
reconciling an already-applied EMS delta.
Backport-of: 12b0d66e85
Source-author: goodboy <goodboy_foss@protonmail.com>
Prompt-IO: ai/prompt-io/opencode/20261022T212248Z_dc618229_ib_recovery_prompt_io.md
(this patch was generated in some part by `opencode` using
`gpt-6-astra` (`openai`))
PARTIAL BACKPORT -- IB adapter and six boundary cases only.
The source also fixes history-layer endpoint/filtering behavior;
those changes and its SHM/NativeDB regressions are NOT imported.
Import the exact `piker/brokers/ib/feed.py` delta. Complete HMDS
no-data replies at the requested date instead of silently walking
back a day. Wake the waiter and let `open_history_client()` turn
the internal empty sentinel into `NoData` or `DataUnavailable`.
Deats,
- Retain the original parametrized request-boundary test body and
its six cases, adding only its required module import.
- Add LOCAL real-bridge history recovery coverage after the reset
backport: production proxy/relay/reset tasks, replayed external
IB/VNC input, and a real farm event before retrying 1s/60s bars.
- Leave `request_end_dt` versus `oldest_shm_dt`, storage cutoff
filtering, `diff_history()` removal and full-bar retention
coverage to the downstream history-layer commit.
Downstream replay,
Retain those omitted changes even if the adapter hunks vanish.
Preserve the local real-bridge regression when reconciling tests.
The resulting adapter file matches the source byte-for-byte;
that equivalence does not apply to the whole source commit.
Partial-backport-of: 1da9b760a3
Source-author: goodboy <goodboy_foss@protonmail.com>
Prompt-IO: ai/prompt-io/opencode/20261022T212248Z_dc618229_ib_recovery_prompt_io.md
(this patch was generated in some part by `opencode` using
`gpt-6-astra` (`openai`))
PARTIAL BACKPORT -- IB adapter and seven adapter regressions only.
The source commit also repairs startup gaps in `ShmArray`; this
patch does NOT deliver that complete history/storage fix.
Import the exact `piker/brokers/ib/feed.py` delta: serialize farm
resets, publish completion and result separately, keep failure
counters per request, bound IB cancellation retries, and complete
empty replies without leaving the parent waiting forever.
Deats,
- Extract seven existing cases into `tests/test_ib_history.py`.
Their function bodies match the source; the module header and
imports are reduced for this older landing tree.
- Omit `piker/tsp/_history.py`: inclusive endpoint handling, SHM
layout, final null-sweep ordering and synthetic gap boundaries.
- Omit the source's history-layer regressions, harness update and
historical prompt files. The source SHA retains that provenance.
Downstream replay,
Only the adapter hunks and these seven cases are already present.
Retain the omitted history/SHM changes and their regressions; do
NOT skip the whole source commit when its adapter delta is empty.
Partial-backport-of: 0846cbd4a4
Source-author: goodboy <goodboy_foss@protonmail.com>
Prompt-IO: ai/prompt-io/opencode/20261022T212248Z_dc618229_ib_recovery_prompt_io.md
(this patch was generated in some part by `opencode` using
`gpt-6-astra` (`openai`))
Reverse the quote-owner code and mocked-cache test from the
landing replay below. Keep its historical prompt records and
restore the direct-owner `_quote_streams` implementation for #94.
With the pinned `tractor`, the cached owner turns quote restart
teardown into a normal bridge exit, then propagates its injected
`TrioTaskExited` into the consumer and kills the broker actor.
The evaluated downstream branch did not contain this cache patch.
LOCAL DIVERGENCE,
- This is a targeted backout, not a replacement broadcast design.
- Add a real infected-actor quote restart regression, including a
second RPC on the same actor and subscription/task cleanup.
- Keep the strict callback-cleanliness case visibly red: actor
survival does not establish clean asyncio completion callbacks.
Downstream replay,
Reintroducing the original cache patch requires qualifying its
owner teardown with the matching `tractor` fix. Do not interpret
this backout as rejecting lexical broadcast subscriptions or as
proving arbitrary concurrent direct-owner callers are safe.
Reverts-code-and-test: 635314c757
Original-equivalent: 4af885d90f
Source-author: goodboy <goodboy_foss@protonmail.com>
Prompt-IO: ai/prompt-io/opencode/20261022T212248Z_dc618229_ib_recovery_prompt_io.md
(this patch was generated in some part by `opencode` using
`gpt-6-astra` (`openai`))
Keep provider root streams private to the cached `Feed` and give every
`maybe_open_feed()` caller, including the first owner, a local
subscription view with copied `Flume` descriptors.
Also,
- key cached resources by the normalized full symbol set and stream
policy instead of only the first FQME.
- subscribe each `Feed.open_multi_stream()` provider relay and wait for
all relays to start before yielding the merged channel.
- give each provider relay an independent send-channel clone so one
provider EOC cannot terminate its siblings.
- document `Feed.pause()` and `Feed.resume()` as shared remote-provider
controls rather than caller-local cursor operations.
- cover miss/hit ownership, multi-market provider mapping, relay
fan-out, lexical teardown and cache-key separation.
Prompt-IO: ai/prompt-io/opencode/20260813T202402Z_12b0d66e_prompt_io.md
(this patch was generated in some part by `opencode` using `gpt-5.6-sol` (`openai`))
Replace the manual `_quote_streams` raw-channel registry with a
symbol-keyed `maybe_open_context()` resource whose lifetime extends to
the final quote consumer.
Give every `open_aio_quote_stream()` caller a lexical child from the
cached `LinkedTaskChannel` broadcaster so concurrent users receive the
same quotes and close independently.
Prompt-IO: ai/prompt-io/opencode/20260813T230744Z_ddac3553_prompt_io.md
(this patch was generated in some part by `opencode` using `gpt-5.6-sol` (`openai`))
`open_client_proxy()` broadcast its one-slot linked channel to an
always-running status relay and an idle method receiver. Status
traffic advanced only the relay, so the first lazy contract lookup
for an uncached order raised `Lagged` before receiving its
response.
Give `relay_client_proxy_messages()` sole receive ownership. Route
tagged responses from concurrent caller tasks into their per-`mid`
slots while each proxy context retains its own channel and relay.
Deats,
- log stale responses with their original method name;
- warn when unresolved-request metadata exceeds its bounded limit;
- drop late responses after caller cancellation;
- preserve the reconnect design note and message-case rationale;
- accept clean channel EOF during proxy teardown;
- cover idle status, concurrency, cancellation and EOF paths.
Prompt-IO: ai/prompt-io/opencode/20260811T205826Z_06d5ea5d_prompt_io.md
(this patch was generated in some part by `opencode` using `gpt-5.6-sol` (`openai`))
Keep the shipped `conf.toml` defaults TCP-only for this `tractor`
pin, with an explicit single-transport note and commented UDS
alternatives. Preserve repeated Click `--maddrs` options and treat
a direct string as one endpoint.
Deats,
- Validate daemon bind requests against actual listeners, not
registrar discovery endpoints; normalize tagged addresses and
allow OS-assigned TCP ports. Explain registrar ownership and
automatic client binding at the validation site.
- Resolve explicit or inherited registrar addresses independently
of other daemon kwargs, consuming overrides before forwarding
them and keeping discovery and startup on the same registrar.
- Default chart binding to `None` for absent or empty network
config, and remove the unconditional Binance debugger pause.
- Defer the prematurely cherry-picked `test_datad_spawn` until
its service API lands. Add offline endpoint/startup regressions
and a real distinct-registrar/daemon lifecycle check.
Include the human-directed config and registrar/bind explanations
and the direct prompt provenance pair from the review fix pass.
Prompt-IO: ai/prompt-io/20261007T165453Z_bde2457b_prompt_io.md
(this patch was generated in some part by `opencode` using
`gpt-6.1-sol` (`openai`))
Use `str.removeprefix()` instead of character-set stripping so
subloggers such as `piker.paper` retain their requested namespace.
Also,
- drop stale parent-logger assertions from daemon startup paths
- keep logger setup calls without assigning assertion-only locals
- cover both logger factories and prefix-sensitive module names
Prompt-IO: ai/prompt-io/opencode/20260821T042058Z_b7aca4c0_prompt_io.md
(this patch was generated in some part by `opencode` using `gpt-5.6-sol` (`openai`))
Switch all `test_feeds.py` parametrized fqmes from
legacy `<pair>.<broker>` to the current
`<pair>.<venue>.<broker>` schema (e.g.
`btcusdt.spot.binance`).
Deats,
- update binance fqmes: `btcusdt.binance` ->
`btcusdt.spot.binance`, same for `ethusdt`.
- update kraken fqmes: `ethusdt.kraken` ->
`ethusdt.spot.kraken`, `xbtusd.kraken` ->
`xbtusd.spot.kraken`.
- update cross-broker set similarly.
- comment out old fqmes with `!TODO` to later
validate raising on bad/legacy formats.
Also,
- reformat `if ci_env and not run_in_ci` to
multiline style.
- reformat `pytest.skip()` msg to multiline.
- add `?TODO` for symbology helper fn.
- drop stray `await tractor.pause()` in
`conftest.py`.
(this commit msg was generated in some part by [`claude-code`][claude-code-gh])
[claude-code-gh]: https://github.com/anthropics/claude-code
Canonicalize legacy socket pairs and tagged tuples before storing
or comparing `Registry.addrs`, keeping existing config readable
when tractor emits protocol-tagged forms.
Also,
- broaden registry/runtime addr annotations for both wire shapes
- compare service fixture addrs through tractor's address codec
- use tractor's shared `--ll` pytest option instead of
registering it twice
- cover mixed legacy/tagged registry state and `pikerd` startup
Prompt-IO: ai/prompt-io/opencode/20260820T163519Z_5f631e19_prompt_io.md
(this patch was generated in some part by `opencode` using `gpt-5.6-sol` (`openai`))
Continue the `repair_tests`-branch mission (already merged
in this stack's ancestry, see f4c4f1e2 which ported
`conftest.py`) by fixing the remaining drift breakage vs.
`tractor` git `main`; without these NOTHING boots since the
`tractor.Address` port in 604e5fcf.
Deats,
- normalize reg addrs via `wrap_address()` in
`open_pikerd()` before `.unwrap()`-ing; entries may be
raw `tuple`s when passed in from (test) client code.
- port `check_for_service()` to `query_actor(regaddr=)`
(was `arbiter_sockaddr=`) incl. its 2-tuple yield and
the now-required `open_registry(addrs=)` arg.
- `wait_for_actor(registry_addr=)` + `.chan.raddr.unwrap()`
raw-tuple compares in `test_runtime_boot` and
`ensure_service()`.
- update `run_test_w_cancel_method()` for modern `tractor`
cancel semantics: self-requested sub-service cancels are
absorbed (no `ContextCancelled` raised to the opener) and
single-exc groups collapse to a bare KBI.
- `RemoteActorError.boxed_type` (was `.type`) and
`Position.cumsize` (was `.size`) renames in tests.
- bump the paper-EMS startup budget 9 -> 19s; it includes
a live (kraken) symbology fetch so needs net headroom.
- woops, add the missing comma in `.deribit.api`'s
`tractor.trionics` import tuple..
(this patch was generated in some part by [`claude-code`][claude-code-gh])
[claude-code-gh]: https://github.com/anthropics/claude-code
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Prompt-IO: ai/prompt-io/claude/20260610T171022Z_4485f2b9_prompt_io.md
Port test fixtures to match `tractor`'s updated
registry and channel APIs.
Deats,
- use `registry_addrs=[reg_addr]` (list) instead of
`registry_addr=reg_addr` in `maybe_open_pikerd()`.
- `wait_for_actor()` now takes `registry_addr=`
kwarg instead of `arbiter_sockaddr=`.
- access `portal.chan` (not `.channel`) and unwrap
remote addr via `raddr.unwrap()`.
- yield `raddr._host`/`raddr._port` instead of
tuple-indexing.
- drop random port generation; accept `reg_addr`
fixture from `tractor`'s builtin pytest plugin.
Also,
- add `reg_addr: tuple` param to `open_test_pikerd()`
fixture (sourced from `tractor._testing.pytest`).
- type-narrow `reg_addr` to `tuple[str, int|str]`.
- drop unused `import random`.
(this commit msg was generated in some part by [`claude-code`][claude-code-gh])
[claude-code-gh]: https://github.com/anthropics/claude-code
To start this is just a shell for the test, there's no checking logic
yet.. put it as `test_accounting.test_ib_account_with_duplicated_mktids()`.
The test is composed for now to be completely runtime-free using only
the offline txn-ledger / symcache / account loading APIs, ideally we
fill in the activated symbology-data-runtime cases once we figure a sane
way to handle incremental symcache updates for backends like IB..
To actually fill the test out with real checks we still need to,
- extract the problem account file from my ib.algopape into the test
harness data.
- pick some contracts with multiple fqmes despite a single bs_mktid and
ensure they're aggregated as a single `Position` as well as,
* ideally de-duplicating txns from the account file section for the
mkt..
* warning appropriately about greater-then-one fqme for the bs_mktid
and providing a way for the ledger re-writing to choose the
appropriate `<venue>` as the "primary" when the
data-symbology-runtime is up and possibly use it to incrementally
update the IB symcache and store offline for next use?
Topically, throughout various (seemingly) console-UX-affecting or benign
spots in the code base; nothing that required more intervention beyond
things superficial. A few spots also include `trio.Nursery` ref renames
(always to something with a `tn` in it) and log-level reductions to
quiet (benign) console noise oriented around issues meant to be solved
long..
Note there's still a couple spots i left with the loose-ify flag because
i haven't fully tested them without using the latest version of
`tractor.trionics.collapse_eg()`, but more then likely they should flip
over fine.
Avoids the really sloppy flag passing to `open_pikerd()` and allows for
separation of the individual docker daemon starts.
Also add a new `root_conf() -> Path` fixture which will open and load
the `dict` for the new root `conf.toml` file.
Since `.config.load()` was changed to not touch conf files by default
(without explicitly setting `touch_if_dne: bool`), this ensures both the
global module value is set and the `brokers.toml` file exists before
every test.
More or less a complete rework which allows passing a detailed
clearing/fills input and allows for *not* rebooting the runtime / ems
between each position check.
Some further enhancements:
- use (unit) fractional sizes to simulate both the more realistic and
more "complex position calculation" case; since this is crypto.
- add a no-fqme-found test.
- factor cross-session/offline pos storage (pps.toml) checks into
a `load_and_check_pos()` helper which does all entry loading directly
from a provided `BrokerdPosition` msg.
- use the new `OrderClient.send()` async api.
Gud Boi
Tyler Goodlet