diff --git a/ai/ib-recovery-candidate.md b/ai/ib-recovery-candidate.md new file mode 100644 index 00000000..ee2c3a77 --- /dev/null +++ b/ai/ib-recovery-candidate.md @@ -0,0 +1,233 @@ +# Bounded IB recovery candidate + +Prepared 2026-10-22 on `wkt/land_ib_recovery`, in the main checkout +`/home/goodboy/repos/piker`. Base: +`dc618229bd8c9c90aa3160e26677b553a7d472c7`. +Changes are unstaged and uncommitted. No acceptance is implied. + +## Patch selection and provenance + +Keep these existing landing commits: + +| Commit | Reason | +| --- | --- | +| `4133f073` and its ancestors | Packaging, Nix, runtime API and pin work | +| `e8e0730a` | Correlate MethodProxy requests and responses | +| `7bef50b0` | One response reader, per-request waiters | +| `d1571ddf` | Propagate daemon-handler exceptions | +| `dc618229` | Isolate cached feed stream consumers | + +Proposed separate implementation boundaries, in this order: + +1. Back out the code and original test from + `635314c757c74a1cd95c5cf3dc7dfd03be984b00`. + This restores the direct-owner `_quote_streams` implementation. + Retain the historical prompt records. Replace the removed + mocked-cache regression with the real-bridge restart reproducer. + This backout is not a claim that arbitrary concurrent direct + callers have safe broadcast ownership. +2. Backport **only** `piker/brokers/ib/feed.py` from + `0846cbd4a4197e47415689cf5e9d146ba3174c32`. + Retain its seven history-adapter regressions in + `tests/test_ib_history.py`. This includes reset serialization, + completion/result signalling and bounded cancellation/failure. + It does not include the original SHM/storage orchestration patch. +3. Backport **only** `piker/brokers/ib/feed.py` from + `1da9b760a3bc6b96501110cab2b5d09846d42a1d`. + Retain `test_empty_history_preserves_request_boundary` and its + six cases. This preserves no-data request boundaries and wakes + the waiter. It does not include the original history-layer + endpoint/filtering changes. Add the real-bridge history recovery + case after these adapter changes are present. +4. Import the code and original regression from + `12b0d66e854247e92cb5a8668be71d816c72d856`. + Add a separate real-broadcaster/production-loop regression. +5. Remove the single-null-sample debugger stop in `get_null_segs()`. + This is a local UX cleanup retaining the existing `None` result. + It does not backport the null-grouping/deduplication rewrite in + `976db6c33c4b8ed07eee3047226417d3928d9cca`. +6. Record cross-boundary regression results and live qualification + evidence, distinguishing the known callback failure from passes. + +All three imported source commits were authored by +`goodboy `; preserve that authorship +and the full source SHA when committing each partial backport. +Use explicit partial-backport wording, not a whole-commit cherry +claim. New real-runtime tests are this session's additional work. + +The current aggregate working diff contains these boundaries in +the same feed file. Staging that entire file would combine the +first three boundaries. Reconstruct/stage the original per-source +patches in the above order when preparing commits. + +## Quote-cache history correction + +`4af885d90f6e4bcfa963f2f7a1d40436d3c928f3` and landing commit +`635314c757c74a1cd95c5cf3dc7dfd03be984b00` have the same stable +patch ID: `bc54892c25e957be112b135bb9e383ec4dd123c5`. +They are equivalent replays, not different implementations. + +`wkt/ib_downstream_pin_eval@e929fdc2` lacks that quote-cache +change and retains direct ownership. The original qualification +did not validate the combination assembled for the landing. + +After this candidate's edits, `piker/brokers/ib/feed.py` matches +the source at `1da9b760` byte-for-byte. Compared with the evaluated +tip, only the later schedule/reservation APIs are absent. +The EMS code patch has the same stable patch ID as the original +`12b0d66e` EMS code patch: +`50fee047e94fd67c8d9feab66859fbe3ab25b64c`. + +## Real-runtime evidence + +`tests/test_ib_quote_restart.py` uses real TCP Tractor actors, +an asyncio-infected child, the installed pinned Tractor bridge, +production Piker quote setup, and real IB/eventkit ticker events. +Only the external IB connection/input is replaced with replay data. + +Before the quote-cache backout, cancellation after ticker delivery +raised `RemoteActorError[TrioTaskExited]` through +`maybe_open_context()` / `_Cache.run_ctx` teardown, matching the +reported fatal path. + +After the backout: + +- Two quote generations finish; each subscription is cancelled, + each asyncio producer is done and each ticker handler removed. +- A second RPC repeats the sequence in the same actor successfully. +- The strict callback-cleanliness case still fails: Tractor's + `signal_trio_when_done()` raises `TrioTaskExited` into asyncio's + exception handler even though the Trio consumer recovers. + +The history integration case keeps `Client.bars()`, MethodProxy, +both relays, `get_bars()`, `wait_on_data_reset()` and +`data_reset_hack()` real. It replaces external contract discovery, +IB history replies and VNC input. A real IB error event announces +farm recovery. The replay rejects retries before recovery and +proves one reset followed by completion for both 1s and 60s data. + +The new EMS case runs the production dark-clearing loop over a +real Tractor broadcaster: both EMS and a sibling receive a tick, +EMS cancellation removes only its subscription, and the sibling +continues with an unchanged strict lag policy. + +## Verification + +Unique selected cases: **38 passed, 1 failed**. +The failure is intentionally visible, not skipped or xfailed. + +Run from the checkout's working Nix environment: + +```xsh +timeout -k 5 45 py313/bin/python -m pytest -p no:xonsh -q -x --tb=short tests/test_ib_history.py tests/test_ems_broadcast.py tests/test_ib_method_proxy.py tests/test_feed_broadcast.py +timeout -k 5 45 py313/bin/python -m pytest -p no:xonsh -q --tb=short tests/test_ib_quote_restart.py +timeout -k 5 45 py313/bin/python -m pytest -p no:xonsh -q -x --tb=short tests/test_logging.py tests/test_startup.py +timeout -k 5 45 py313/bin/python -m pytest -p no:xonsh -q -x --tb=short tests/test_services.py::test_runtime_boot +git diff --check +``` + +The adapter/proxy/feed/EMS selection initially passed 21 cases; +the added real EMS case then passed with its imported case. +The bridge file passed two cases and failed callback cleanliness. +Logging/startup passed 13 cases; runtime boot passed one with the +existing `Registrar.uid` deprecation warning. + +The agent shell inherited another project's environment and lacked +native libraries. Tests used the existing Piker `py313` via +`/tmp/nix-shell.zfKbeB/opencode/piker_test_env.py`, which supplies +libraries from the pinned flake's existing Qt dependency closure, +excluding Nix Qt itself and glibc. No environment sync or shellHook +ran. Piker resolved to this checkout; Tractor resolved to its +installed package in `py313`. + +Workflow-state preparation initially reported changed legacy +files/modes after migration. On 2026-10-23 the human authorized +reconciliation. Inspection found only two missing legacy guidance +snapshots; the neutral copies matched their saved hashes and modes. +Restoring those local snapshots resolved preparation without +changing archived messages or the original migration receipt. + +## Human live qualification, 2026-10-23 + +The human ran these journeys with the installed Tractor pin, +the existing IB Gateway and pre-existing NativeDB. The agent did +not operate the broker connection, VNC, orders or database. + +- Initial QQQ startup/backfill recovered after pacing error 162 + with one reset attempt, then displayed live updates. The reset + waiter appears to have timed out; the log did not establish an + acknowledged farm reset or live quote-stream restart. +- Sequential chart exits and reattachment preserved all four + daemon PIDs; a read-only process/listener check confirmed this. +- The overlapping two-chart journey used `pikerd@961108`, + `brokerd.ib@964765`, `samplerd@965900` and `emsd@969015`. + B started at 08:16:31; C at 08:17:33. B closed at 08:18:11, + replacement B started at 08:18:24 and had FSPs up by 08:18:28, + then C closed at 08:18:37. The human reported the exercise worked; + survivor updates are a manual observation, while timestamps and + unchanged daemon identities corroborate overlap and service reuse. +- Startup in that journey hit the explicit single-null-sample + breakpoint several times before the charts overlapped. The small + debugger cleanup above was applied afterward; this live run does + not itself verify that later edit. +- Departing charts still report cancellation groups and an FSP + receiver-close `ClosedResourceError`, followed by the Qt close + handler's self-SIGINT. Peer-disconnected final result warnings + remain contained in the persistent services. +- Daemon SIGINT at 08:19:43 reproduced cached shared-client teardown: + `load_clients_for_trio()` / `_Cache.run_ctx` promoted + `TrioTaskExited` through `open_trade_dialog()` and entered PDB. + After the human continued, all three children were reported + exited with status 0 and the root terminated at 08:19:45. + Cancellation-only RPC error reporting and late cancel-ack + diagnostics also remain. + +The human explicitly accepted deferring this daemon teardown fix +from #94 and requested loud attribution for the bounded backports. +This is an operational smoke result, not clean-shutdown coverage. + +## Downstream reconciliation requirements + +Before landing #94, the human wants an easy replay of the downstream +histories intended to follow it. Rehearse against the agreed #94 +tree in disposable worktrees, then inspect range-diffs, final code +and relevant regressions. This note does not authorize rewriting +the original branches or changing acceptance markers. + +- `0846cbd4` and `1da9b760` are partial backports. Their imported + adapter hunks may already be present, but their history/SHM/storage + changes and omitted integration tests remain necessary downstream. + Never drop either whole source commit merely because `feed.py` + matches or its adapter hunk becomes empty. +- `12b0d66e` has source-equivalent EMS code and its imported test, + plus an additional local real-broadcaster test. Preserve that extra + regression when resolving test-file overlap. +- The quote-cache backout intentionally restores direct ownership + for this landing. Reintroducing original `4af885d9` downstream + requires qualifying cached-owner teardown with the matching + Tractor revision; the backout is not a permanent ownership design. +- The local single-null breakpoint removal retains legacy behavior. + Prefer the full downstream `976db6c3` algorithm when it is replayed; + preserving this `return None` guard would lose that later fix. +- Real-bridge tests and manual qualification notes are local work, + not evidence that omitted portions of the source commits landed. + +Commit messages must name full source SHAs, original authorship, +imported paths/tests, omissions, local additions and replay guidance. +Use explicit partial-backport language for adapter subsets; do not +claim a full cherry-pick or reuse the source's broader fix claims. + +## Remaining decision + +This candidate removes the reproduced fatal quote-restart +propagation and imports existing reset fixes. The human exercised +live QQQ startup, backfill and concurrent chart consumer lifetimes. +Completion-callback and teardown errors remain; the human accepts +them as follow-ups outside the bounded #94 landing. +The storage/SHM portions of the downstream fixes are not covered by +the adapter backport or its integration tests. + +`ddac3553` is Deribit-specific and omitted. Qt lifecycle commit +`1a2d4893` stays a separate follow-up. Further Tractor teardown +changes or storage imports require reassessing the bounded landing +rather than silently expanding this candidate. diff --git a/ai/prompt-io/opencode/20261022T212248Z_dc618229_ib_recovery_prompt_io.md b/ai/prompt-io/opencode/20261022T212248Z_dc618229_ib_recovery_prompt_io.md new file mode 100644 index 00000000..63f92a42 --- /dev/null +++ b/ai/prompt-io/opencode/20261022T212248Z_dc618229_ib_recovery_prompt_io.md @@ -0,0 +1,67 @@ +--- +created_at: 2026-10-22T21:22:48Z +generated_by: + harness: opencode + provider: openai + model: openai/gpt-6-astra +git_ref: dc618229bd8c9c90aa3160e26677b553a7d472c7 +scope: code +substantive: true +raw_file: 20261022T212248Z_dc618229_ib_recovery_prompt_io.raw.md +--- + +## Prompt + +The human approved trying the bounded candidate: back out the IB +quote-context cache, import adapter-only reset/boundary fixes and +the missing EMS subscription fix, preserve source authorship and +commit boundaries, and exercise the real Tractor bridge. They +required an explicit keep/backout audit and clarification that +the evaluated downstream lineage lacked the broadcast fix. + +## Response summary + +Verified identical stable patch IDs for original 4af885d9 and +landing replay 635314c7. Created `wkt/land_ib_recovery` at dc618229 +in the main checkout. Applied source-equivalent adapter backports +from 0846cbd4 and 1da9b760 and the EMS code from 12b0d66e, with +source attribution recorded in `ai/ib-recovery-candidate.md`. + +The real-runtime quote test first reproduced the reported fatal +TrioTaskExited propagation with caching enabled. The direct-owner +backout restores restart and subsequent RPC survival, but a strict +callback-cleanliness test remains red. Real bridge history pacing +recovery and real broadcaster dark clearing pass. No live broker, +VNC, orders or TSDB operations were performed by the agent. + +On 2026-10-23 the human supplied successful live startup/backfill, +sequential reattachment and overlapping chart-close/reopen results, +plus the still-failing daemon SIGINT teardown. The human accepted +deferring teardown from #94, requested loud partial-backport +attribution and required downstream rebase rehearsals before landing. +The agent removed only the explicit single-null debugger stop, +retaining its legacy result. Downstream `976db6c3` instead rewrites +null grouping; the local cleanup does not claim that algorithmic fix. + +## Files changed + +- `piker/brokers/ib/feed.py`: source-equivalent backout/backports. +- `piker/clearing/_ems.py`: original EMS subscription patch. +- `tests/test_ib_broadcast.py`: removed with the reverted behavior. +- `tests/test_ib_history.py`: original adapter regressions only. +- `tests/test_ems_broadcast.py`: imported and real-broadcaster tests. +- `tests/test_ib_quote_restart.py`: real Tractor boundary tests. +- `piker/tsp/_anal.py`: local single-null breakpoint removal only. +- `ai/ib-recovery-candidate.md`: provenance and qualification limits. + +## Human edits + +The human required original fixes and authorship, rejected broad +speculative integration work, required real concurrency boundaries +in tests, and corrected the interpretation of live qualification. +They performed the live tests, chose to defer known teardown defects +from #94, permitted the minimal debugger cleanup, and requested +explicit downstream discrepancies and rebase compatibility checks. +The agent applied those directed edits; no direct manual source edits +are known in this candidate. The raw partner retains the originally +captured output; this structured record includes later contributions. diff --git a/ai/prompt-io/opencode/20261022T212248Z_dc618229_ib_recovery_prompt_io.raw.md b/ai/prompt-io/opencode/20261022T212248Z_dc618229_ib_recovery_prompt_io.raw.md new file mode 100644 index 00000000..9ff03a36 --- /dev/null +++ b/ai/prompt-io/opencode/20261022T212248Z_dc618229_ib_recovery_prompt_io.raw.md @@ -0,0 +1,69 @@ +--- +created_at: 2026-10-22T21:22:48Z +generated_by: + harness: opencode + provider: openai + model: openai/gpt-6-astra +git_ref: dc618229bd8c9c90aa3160e26677b553a7d472c7 +diff_cmd: git diff dc618229 -- piker/brokers/ib/feed.py piker/clearing/_ems.py tests +--- + +## Human direction + +> yeah i guess this is our best approach strat for sure. + +> so you're saying these 2 are not the same patch? +> please tell me no lol + +> so detail which commits to drop vs. keep please in detail so i can +> audit your choices this time.. + +> let's try it and consider bailing and re-trying if this leads us into +> a further (and deeper) sinkhole, bc as it stands right now this whole +> endevour has turned into a complete shitfest XD + +> you didn't respond to my ask about the wkt/ib_downstream_pin_eval stuff and it missing the bcast rx commit either + +## Unedited diagnostic response + +The real-bridge test reproduced the exact fatal path: +`maybe_open_context()` teardown raises `TrioTaskExited` back into +the quote consumer. + +With only the quote-cache backout, both quote generations complete +and their asyncio tasks finish. Tractor still emits +`TrioTaskExited` callback errors, so the strict test remains red. +That narrows the result: the backout removes the fatal propagation +in this reproducer, but doesn’t make bridge teardown clean. + +## Code output pointers + +> `git diff dc618229 -- piker/brokers/ib/feed.py` + +Back out 635314c7's quote owner and import the original adapter-only +deltas from 0846cbd4 and 1da9b760. + +> `git diff dc618229 -- piker/clearing/_ems.py` + +Import 12b0d66e's dedicated dark-clearing quote subscription. + +> `git diff dc618229 -- tests/test_ib_broadcast.py tests/test_ib_quote_restart.py tests/test_ib_history.py tests/test_ems_broadcast.py` + +Replace the reverted mock-cache test with a real-runtime restart +reproducer; retain original history/EMS tests and add real bridge +history recovery and real broadcaster EMS coverage. Newly added +test files remain untracked until the human stages them, so the +diff pointers include them only after staging or committing. + +## Verification output + +```text +21 passed in 0.53s +1 failed, 2 passed in 5.53s +13 passed in 1.41s +1 passed, 1 warning in 0.04s +2 passed in 0.03s +``` + +The last run includes the original EMS case again and one additional +real-broadcaster case: 38 unique passes and one strict callback failure.