Import the source-equivalent EMS code and its original wrapper
regression. `clear_dark_triggers()` owns a non-strict child from
`quote_stream.subscribe()` for the lifetime of its clearing loop,
instead of changing a shared receiver's private lag policy.
LOCAL ADDITION,
Add a second regression using the production clearing loop and a
real broadcaster. Deliver a tick to EMS and a sibling, cancel EMS,
and prove the sibling remains usable with strict lag policy.
No order trigger or live broker is involved in this test.
Downstream replay,
The EMS code has the source patch's stable patch ID. The test file
also contains the local real-broadcaster case, so the complete
commit is not patch-identical. Preserve that extra coverage when
reconciling an already-applied EMS delta.
Backport-of: 12b0d66e85
Source-author: goodboy <goodboy_foss@protonmail.com>
Prompt-IO: ai/prompt-io/opencode/20261022T212248Z_dc618229_ib_recovery_prompt_io.md
(this patch was generated in some part by `opencode` using
`gpt-6-astra` (`openai`))
PARTIAL BACKPORT -- IB adapter and six boundary cases only.
The source also fixes history-layer endpoint/filtering behavior;
those changes and its SHM/NativeDB regressions are NOT imported.
Import the exact `piker/brokers/ib/feed.py` delta. Complete HMDS
no-data replies at the requested date instead of silently walking
back a day. Wake the waiter and let `open_history_client()` turn
the internal empty sentinel into `NoData` or `DataUnavailable`.
Deats,
- Retain the original parametrized request-boundary test body and
its six cases, adding only its required module import.
- Add LOCAL real-bridge history recovery coverage after the reset
backport: production proxy/relay/reset tasks, replayed external
IB/VNC input, and a real farm event before retrying 1s/60s bars.
- Leave `request_end_dt` versus `oldest_shm_dt`, storage cutoff
filtering, `diff_history()` removal and full-bar retention
coverage to the downstream history-layer commit.
Downstream replay,
Retain those omitted changes even if the adapter hunks vanish.
Preserve the local real-bridge regression when reconciling tests.
The resulting adapter file matches the source byte-for-byte;
that equivalence does not apply to the whole source commit.
Partial-backport-of: 1da9b760a3
Source-author: goodboy <goodboy_foss@protonmail.com>
Prompt-IO: ai/prompt-io/opencode/20261022T212248Z_dc618229_ib_recovery_prompt_io.md
(this patch was generated in some part by `opencode` using
`gpt-6-astra` (`openai`))
PARTIAL BACKPORT -- IB adapter and seven adapter regressions only.
The source commit also repairs startup gaps in `ShmArray`; this
patch does NOT deliver that complete history/storage fix.
Import the exact `piker/brokers/ib/feed.py` delta: serialize farm
resets, publish completion and result separately, keep failure
counters per request, bound IB cancellation retries, and complete
empty replies without leaving the parent waiting forever.
Deats,
- Extract seven existing cases into `tests/test_ib_history.py`.
Their function bodies match the source; the module header and
imports are reduced for this older landing tree.
- Omit `piker/tsp/_history.py`: inclusive endpoint handling, SHM
layout, final null-sweep ordering and synthetic gap boundaries.
- Omit the source's history-layer regressions, harness update and
historical prompt files. The source SHA retains that provenance.
Downstream replay,
Only the adapter hunks and these seven cases are already present.
Retain the omitted history/SHM changes and their regressions; do
NOT skip the whole source commit when its adapter delta is empty.
Partial-backport-of: 0846cbd4a4
Source-author: goodboy <goodboy_foss@protonmail.com>
Prompt-IO: ai/prompt-io/opencode/20261022T212248Z_dc618229_ib_recovery_prompt_io.md
(this patch was generated in some part by `opencode` using
`gpt-6-astra` (`openai`))
Reverse the quote-owner code and mocked-cache test from the
landing replay below. Keep its historical prompt records and
restore the direct-owner `_quote_streams` implementation for #94.
With the pinned `tractor`, the cached owner turns quote restart
teardown into a normal bridge exit, then propagates its injected
`TrioTaskExited` into the consumer and kills the broker actor.
The evaluated downstream branch did not contain this cache patch.
LOCAL DIVERGENCE,
- This is a targeted backout, not a replacement broadcast design.
- Add a real infected-actor quote restart regression, including a
second RPC on the same actor and subscription/task cleanup.
- Keep the strict callback-cleanliness case visibly red: actor
survival does not establish clean asyncio completion callbacks.
Downstream replay,
Reintroducing the original cache patch requires qualifying its
owner teardown with the matching `tractor` fix. Do not interpret
this backout as rejecting lexical broadcast subscriptions or as
proving arbitrary concurrent direct-owner callers are safe.
Reverts-code-and-test: 635314c757
Original-equivalent: 4af885d90f
Source-author: goodboy <goodboy_foss@protonmail.com>
Prompt-IO: ai/prompt-io/opencode/20261022T212248Z_dc618229_ib_recovery_prompt_io.md
(this patch was generated in some part by `opencode` using
`gpt-6-astra` (`openai`))
Keep provider root streams private to the cached `Feed` and give every
`maybe_open_feed()` caller, including the first owner, a local
subscription view with copied `Flume` descriptors.
Also,
- key cached resources by the normalized full symbol set and stream
policy instead of only the first FQME.
- subscribe each `Feed.open_multi_stream()` provider relay and wait for
all relays to start before yielding the merged channel.
- give each provider relay an independent send-channel clone so one
provider EOC cannot terminate its siblings.
- document `Feed.pause()` and `Feed.resume()` as shared remote-provider
controls rather than caller-local cursor operations.
- cover miss/hit ownership, multi-market provider mapping, relay
fan-out, lexical teardown and cache-key separation.
Prompt-IO: ai/prompt-io/opencode/20260813T202402Z_12b0d66e_prompt_io.md
(this patch was generated in some part by `opencode` using `gpt-5.6-sol` (`openai`))
Replace the manual `_quote_streams` raw-channel registry with a
symbol-keyed `maybe_open_context()` resource whose lifetime extends to
the final quote consumer.
Give every `open_aio_quote_stream()` caller a lexical child from the
cached `LinkedTaskChannel` broadcaster so concurrent users receive the
same quotes and close independently.
Prompt-IO: ai/prompt-io/opencode/20260813T230744Z_ddac3553_prompt_io.md
(this patch was generated in some part by `opencode` using `gpt-5.6-sol` (`openai`))
`open_client_proxy()` broadcast its one-slot linked channel to an
always-running status relay and an idle method receiver. Status
traffic advanced only the relay, so the first lazy contract lookup
for an uncached order raised `Lagged` before receiving its
response.
Give `relay_client_proxy_messages()` sole receive ownership. Route
tagged responses from concurrent caller tasks into their per-`mid`
slots while each proxy context retains its own channel and relay.
Deats,
- log stale responses with their original method name;
- warn when unresolved-request metadata exceeds its bounded limit;
- drop late responses after caller cancellation;
- preserve the reconnect design note and message-case rationale;
- accept clean channel EOF during proxy teardown;
- cover idle status, concurrency, cancellation and EOF paths.
Prompt-IO: ai/prompt-io/opencode/20260811T205826Z_06d5ea5d_prompt_io.md
(this patch was generated in some part by `opencode` using `gpt-5.6-sol` (`openai`))
Keep the shipped `conf.toml` defaults TCP-only for this `tractor`
pin, with an explicit single-transport note and commented UDS
alternatives. Preserve repeated Click `--maddrs` options and treat
a direct string as one endpoint.
Deats,
- Validate daemon bind requests against actual listeners, not
registrar discovery endpoints; normalize tagged addresses and
allow OS-assigned TCP ports. Explain registrar ownership and
automatic client binding at the validation site.
- Resolve explicit or inherited registrar addresses independently
of other daemon kwargs, consuming overrides before forwarding
them and keeping discovery and startup on the same registrar.
- Default chart binding to `None` for absent or empty network
config, and remove the unconditional Binance debugger pause.
- Defer the prematurely cherry-picked `test_datad_spawn` until
its service API lands. Add offline endpoint/startup regressions
and a real distinct-registrar/daemon lifecycle check.
Include the human-directed config and registrar/bind explanations
and the direct prompt provenance pair from the review fix pass.
Prompt-IO: ai/prompt-io/20261007T165453Z_bde2457b_prompt_io.md
(this patch was generated in some part by `opencode` using
`gpt-6.1-sol` (`openai`))
Use `str.removeprefix()` instead of character-set stripping so
subloggers such as `piker.paper` retain their requested namespace.
Also,
- drop stale parent-logger assertions from daemon startup paths
- keep logger setup calls without assigning assertion-only locals
- cover both logger factories and prefix-sensitive module names
Prompt-IO: ai/prompt-io/opencode/20260821T042058Z_b7aca4c0_prompt_io.md
(this patch was generated in some part by `opencode` using `gpt-5.6-sol` (`openai`))
Switch all `test_feeds.py` parametrized fqmes from
legacy `<pair>.<broker>` to the current
`<pair>.<venue>.<broker>` schema (e.g.
`btcusdt.spot.binance`).
Deats,
- update binance fqmes: `btcusdt.binance` ->
`btcusdt.spot.binance`, same for `ethusdt`.
- update kraken fqmes: `ethusdt.kraken` ->
`ethusdt.spot.kraken`, `xbtusd.kraken` ->
`xbtusd.spot.kraken`.
- update cross-broker set similarly.
- comment out old fqmes with `!TODO` to later
validate raising on bad/legacy formats.
Also,
- reformat `if ci_env and not run_in_ci` to
multiline style.
- reformat `pytest.skip()` msg to multiline.
- add `?TODO` for symbology helper fn.
- drop stray `await tractor.pause()` in
`conftest.py`.
(this commit msg was generated in some part by [`claude-code`][claude-code-gh])
[claude-code-gh]: https://github.com/anthropics/claude-code
Canonicalize legacy socket pairs and tagged tuples before storing
or comparing `Registry.addrs`, keeping existing config readable
when tractor emits protocol-tagged forms.
Also,
- broaden registry/runtime addr annotations for both wire shapes
- compare service fixture addrs through tractor's address codec
- use tractor's shared `--ll` pytest option instead of
registering it twice
- cover mixed legacy/tagged registry state and `pikerd` startup
Prompt-IO: ai/prompt-io/opencode/20260820T163519Z_5f631e19_prompt_io.md
(this patch was generated in some part by `opencode` using `gpt-5.6-sol` (`openai`))
Continue the `repair_tests`-branch mission (already merged
in this stack's ancestry, see f4c4f1e2 which ported
`conftest.py`) by fixing the remaining drift breakage vs.
`tractor` git `main`; without these NOTHING boots since the
`tractor.Address` port in 604e5fcf.
Deats,
- normalize reg addrs via `wrap_address()` in
`open_pikerd()` before `.unwrap()`-ing; entries may be
raw `tuple`s when passed in from (test) client code.
- port `check_for_service()` to `query_actor(regaddr=)`
(was `arbiter_sockaddr=`) incl. its 2-tuple yield and
the now-required `open_registry(addrs=)` arg.
- `wait_for_actor(registry_addr=)` + `.chan.raddr.unwrap()`
raw-tuple compares in `test_runtime_boot` and
`ensure_service()`.
- update `run_test_w_cancel_method()` for modern `tractor`
cancel semantics: self-requested sub-service cancels are
absorbed (no `ContextCancelled` raised to the opener) and
single-exc groups collapse to a bare KBI.
- `RemoteActorError.boxed_type` (was `.type`) and
`Position.cumsize` (was `.size`) renames in tests.
- bump the paper-EMS startup budget 9 -> 19s; it includes
a live (kraken) symbology fetch so needs net headroom.
- woops, add the missing comma in `.deribit.api`'s
`tractor.trionics` import tuple..
(this patch was generated in some part by [`claude-code`][claude-code-gh])
[claude-code-gh]: https://github.com/anthropics/claude-code
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Prompt-IO: ai/prompt-io/claude/20260610T171022Z_4485f2b9_prompt_io.md
Port test fixtures to match `tractor`'s updated
registry and channel APIs.
Deats,
- use `registry_addrs=[reg_addr]` (list) instead of
`registry_addr=reg_addr` in `maybe_open_pikerd()`.
- `wait_for_actor()` now takes `registry_addr=`
kwarg instead of `arbiter_sockaddr=`.
- access `portal.chan` (not `.channel`) and unwrap
remote addr via `raddr.unwrap()`.
- yield `raddr._host`/`raddr._port` instead of
tuple-indexing.
- drop random port generation; accept `reg_addr`
fixture from `tractor`'s builtin pytest plugin.
Also,
- add `reg_addr: tuple` param to `open_test_pikerd()`
fixture (sourced from `tractor._testing.pytest`).
- type-narrow `reg_addr` to `tuple[str, int|str]`.
- drop unused `import random`.
(this commit msg was generated in some part by [`claude-code`][claude-code-gh])
[claude-code-gh]: https://github.com/anthropics/claude-code
To start this is just a shell for the test, there's no checking logic
yet.. put it as `test_accounting.test_ib_account_with_duplicated_mktids()`.
The test is composed for now to be completely runtime-free using only
the offline txn-ledger / symcache / account loading APIs, ideally we
fill in the activated symbology-data-runtime cases once we figure a sane
way to handle incremental symcache updates for backends like IB..
To actually fill the test out with real checks we still need to,
- extract the problem account file from my ib.algopape into the test
harness data.
- pick some contracts with multiple fqmes despite a single bs_mktid and
ensure they're aggregated as a single `Position` as well as,
* ideally de-duplicating txns from the account file section for the
mkt..
* warning appropriately about greater-then-one fqme for the bs_mktid
and providing a way for the ledger re-writing to choose the
appropriate `<venue>` as the "primary" when the
data-symbology-runtime is up and possibly use it to incrementally
update the IB symcache and store offline for next use?
Topically, throughout various (seemingly) console-UX-affecting or benign
spots in the code base; nothing that required more intervention beyond
things superficial. A few spots also include `trio.Nursery` ref renames
(always to something with a `tn` in it) and log-level reductions to
quiet (benign) console noise oriented around issues meant to be solved
long..
Note there's still a couple spots i left with the loose-ify flag because
i haven't fully tested them without using the latest version of
`tractor.trionics.collapse_eg()`, but more then likely they should flip
over fine.
Avoids the really sloppy flag passing to `open_pikerd()` and allows for
separation of the individual docker daemon starts.
Also add a new `root_conf() -> Path` fixture which will open and load
the `dict` for the new root `conf.toml` file.
Since `.config.load()` was changed to not touch conf files by default
(without explicitly setting `touch_if_dne: bool`), this ensures both the
global module value is set and the `brokers.toml` file exists before
every test.
More or less a complete rework which allows passing a detailed
clearing/fills input and allows for *not* rebooting the runtime / ems
between each position check.
Some further enhancements:
- use (unit) fractional sizes to simulate both the more realistic and
more "complex position calculation" case; since this is crypto.
- add a no-fqme-found test.
- factor cross-session/offline pos storage (pps.toml) checks into
a `load_and_check_pos()` helper which does all entry loading directly
from a provided `BrokerdPosition` msg.
- use the new `OrderClient.send()` async api.
More or less we need to be able to audit not only simple "make trades
check pps.toml files" tests (which btw were great to get started!).
We also need more sophisticated and granular order mgmt and service
config scenarios,
- full e2e EMS msg flow verification
- multi-client (dis)connection scenarios and/or monitoring
- dark order clearing and offline storage
- accounting schema and position calcs detailing
As such, this is the beginning to "modularlizingz" the components needed
in the test harness to this end by breaking up the `OrderClient` control
flows vs. position checking logic so as to allow for more flexible test
scenario cases and likely `pytest` parametrizations over different
transaction sequences.
Not sure how this worked before but we need to also override the
`piker._config_dir: Path` in the root actor when running in `pytest`; my
guess is something in the old test suite was masking this problem after
the change to passing the dir path down through the runtime vars via
`tractor`?
Also this drops the ems related fixtures/factories since they're
specific enough to define in the clearing engine tests directly.
Gud Boi
Tyler Goodlet