Document bounded IB recovery and downstream replay

Record exact keep/backout/import provenance for the #94 candidate,
including the quote-cache lineage correction and the distinction
between file-equivalent adapter deltas and partial source commits.

Deats,
- Preserve the original raw prompt record and add the human's
  later scope decisions, corrections and live-test contributions
  to its structured partner.
- Record 38 prior unique offline passes and the known-red bridge
  callback-cleanliness case, without claiming clean teardown.
- Record live QQQ backfill, reattachment and overlapping chart
  close/reopen results. Startup needed debugger continuation;
  the local breakpoint cleanup was applied after that run.
- Retain the cached-client daemon shutdown failure as a human-
  accepted follow-up outside #94, with its exact propagation path.
- Require disposable downstream rebase rehearsals before landing,
  preserving omitted SHM/storage fixes and local regressions.

The commit package records exact per-boundary project checks.
Its structural replay/preflight does not substitute for those
checks, the prior live observations or rebase qualification.

Prompt-IO: ai/prompt-io/opencode/20261022T212248Z_dc618229_ib_recovery_prompt_io.md

(this patch was generated in some part by `opencode` using
`gpt-6-astra` (`openai`))

ai/ib-recovery-candidate.md

@@ -0,0 +1,233 @@
+# Bounded IB recovery candidate
+
+Prepared 2026-10-22 on `wkt/land_ib_recovery`, in the main checkout
+`/home/goodboy/repos/piker`. Base:
+`dc618229bd8c9c90aa3160e26677b553a7d472c7`.
+Changes are unstaged and uncommitted. No acceptance is implied.
+
+## Patch selection and provenance
+
+Keep these existing landing commits:
+
+| Commit | Reason |
+| --- | --- |
+| `4133f073` and its ancestors | Packaging, Nix, runtime API and pin work |
+| `e8e0730a` | Correlate MethodProxy requests and responses |
+| `7bef50b0` | One response reader, per-request waiters |
+| `d1571ddf` | Propagate daemon-handler exceptions |
+| `dc618229` | Isolate cached feed stream consumers |
+
+Proposed separate implementation boundaries, in this order:
+
+1. Back out the code and original test from
+   `635314c757c74a1cd95c5cf3dc7dfd03be984b00`.
+   This restores the direct-owner `_quote_streams` implementation.
+   Retain the historical prompt records. Replace the removed
+   mocked-cache regression with the real-bridge restart reproducer.
+   This backout is not a claim that arbitrary concurrent direct
+   callers have safe broadcast ownership.
+2. Backport **only** `piker/brokers/ib/feed.py` from
+   `0846cbd4a4197e47415689cf5e9d146ba3174c32`.
+   Retain its seven history-adapter regressions in
+   `tests/test_ib_history.py`. This includes reset serialization,
+   completion/result signalling and bounded cancellation/failure.
+   It does not include the original SHM/storage orchestration patch.
+3. Backport **only** `piker/brokers/ib/feed.py` from
+   `1da9b760a3bc6b96501110cab2b5d09846d42a1d`.
+   Retain `test_empty_history_preserves_request_boundary` and its
+   six cases. This preserves no-data request boundaries and wakes
+   the waiter. It does not include the original history-layer
+   endpoint/filtering changes. Add the real-bridge history recovery
+   case after these adapter changes are present.
+4. Import the code and original regression from
+   `12b0d66e854247e92cb5a8668be71d816c72d856`.
+   Add a separate real-broadcaster/production-loop regression.
+5. Remove the single-null-sample debugger stop in `get_null_segs()`.
+   This is a local UX cleanup retaining the existing `None` result.
+   It does not backport the null-grouping/deduplication rewrite in
+   `976db6c33c4b8ed07eee3047226417d3928d9cca`.
+6. Record cross-boundary regression results and live qualification
+   evidence, distinguishing the known callback failure from passes.
+
+All three imported source commits were authored by
+`goodboy <goodboy_foss@protonmail.com>`; preserve that authorship
+and the full source SHA when committing each partial backport.
+Use explicit partial-backport wording, not a whole-commit cherry
+claim. New real-runtime tests are this session's additional work.
+
+The current aggregate working diff contains these boundaries in
+the same feed file. Staging that entire file would combine the
+first three boundaries. Reconstruct/stage the original per-source
+patches in the above order when preparing commits.
+
+## Quote-cache history correction
+
+`4af885d90f6e4bcfa963f2f7a1d40436d3c928f3` and landing commit
+`635314c757c74a1cd95c5cf3dc7dfd03be984b00` have the same stable
+patch ID: `bc54892c25e957be112b135bb9e383ec4dd123c5`.
+They are equivalent replays, not different implementations.
+
+`wkt/ib_downstream_pin_eval@e929fdc2` lacks that quote-cache
+change and retains direct ownership. The original qualification
+did not validate the combination assembled for the landing.
+
+After this candidate's edits, `piker/brokers/ib/feed.py` matches
+the source at `1da9b760` byte-for-byte. Compared with the evaluated
+tip, only the later schedule/reservation APIs are absent.
+The EMS code patch has the same stable patch ID as the original
+`12b0d66e` EMS code patch:
+`50fee047e94fd67c8d9feab66859fbe3ab25b64c`.
+
+## Real-runtime evidence
+
+`tests/test_ib_quote_restart.py` uses real TCP Tractor actors,
+an asyncio-infected child, the installed pinned Tractor bridge,
+production Piker quote setup, and real IB/eventkit ticker events.
+Only the external IB connection/input is replaced with replay data.
+
+Before the quote-cache backout, cancellation after ticker delivery
+raised `RemoteActorError[TrioTaskExited]` through
+`maybe_open_context()` / `_Cache.run_ctx` teardown, matching the
+reported fatal path.
+
+After the backout:
+
+- Two quote generations finish; each subscription is cancelled,
+  each asyncio producer is done and each ticker handler removed.
+- A second RPC repeats the sequence in the same actor successfully.
+- The strict callback-cleanliness case still fails: Tractor's
+  `signal_trio_when_done()` raises `TrioTaskExited` into asyncio's
+  exception handler even though the Trio consumer recovers.
+
+The history integration case keeps `Client.bars()`, MethodProxy,
+both relays, `get_bars()`, `wait_on_data_reset()` and
+`data_reset_hack()` real. It replaces external contract discovery,
+IB history replies and VNC input. A real IB error event announces
+farm recovery. The replay rejects retries before recovery and
+proves one reset followed by completion for both 1s and 60s data.
+
+The new EMS case runs the production dark-clearing loop over a
+real Tractor broadcaster: both EMS and a sibling receive a tick,
+EMS cancellation removes only its subscription, and the sibling
+continues with an unchanged strict lag policy.
+
+## Verification
+
+Unique selected cases: **38 passed, 1 failed**.
+The failure is intentionally visible, not skipped or xfailed.
+
+Run from the checkout's working Nix environment:
+
+```xsh
+timeout -k 5 45 py313/bin/python -m pytest -p no:xonsh -q -x --tb=short tests/test_ib_history.py tests/test_ems_broadcast.py tests/test_ib_method_proxy.py tests/test_feed_broadcast.py
+timeout -k 5 45 py313/bin/python -m pytest -p no:xonsh -q --tb=short tests/test_ib_quote_restart.py
+timeout -k 5 45 py313/bin/python -m pytest -p no:xonsh -q -x --tb=short tests/test_logging.py tests/test_startup.py
+timeout -k 5 45 py313/bin/python -m pytest -p no:xonsh -q -x --tb=short tests/test_services.py::test_runtime_boot
+git diff --check
+```
+
+The adapter/proxy/feed/EMS selection initially passed 21 cases;
+the added real EMS case then passed with its imported case.
+The bridge file passed two cases and failed callback cleanliness.
+Logging/startup passed 13 cases; runtime boot passed one with the
+existing `Registrar.uid` deprecation warning.
+
+The agent shell inherited another project's environment and lacked
+native libraries. Tests used the existing Piker `py313` via
+`/tmp/nix-shell.zfKbeB/opencode/piker_test_env.py`, which supplies
+libraries from the pinned flake's existing Qt dependency closure,
+excluding Nix Qt itself and glibc. No environment sync or shellHook
+ran. Piker resolved to this checkout; Tractor resolved to its
+installed package in `py313`.
+
+Workflow-state preparation initially reported changed legacy
+files/modes after migration. On 2026-10-23 the human authorized
+reconciliation. Inspection found only two missing legacy guidance
+snapshots; the neutral copies matched their saved hashes and modes.
+Restoring those local snapshots resolved preparation without
+changing archived messages or the original migration receipt.
+
+## Human live qualification, 2026-10-23
+
+The human ran these journeys with the installed Tractor pin,
+the existing IB Gateway and pre-existing NativeDB. The agent did
+not operate the broker connection, VNC, orders or database.
+
+- Initial QQQ startup/backfill recovered after pacing error 162
+  with one reset attempt, then displayed live updates. The reset
+  waiter appears to have timed out; the log did not establish an
+  acknowledged farm reset or live quote-stream restart.
+- Sequential chart exits and reattachment preserved all four
+  daemon PIDs; a read-only process/listener check confirmed this.
+- The overlapping two-chart journey used `pikerd@961108`,
+  `brokerd.ib@964765`, `samplerd@965900` and `emsd@969015`.
+  B started at 08:16:31; C at 08:17:33. B closed at 08:18:11,
+  replacement B started at 08:18:24 and had FSPs up by 08:18:28,
+  then C closed at 08:18:37. The human reported the exercise worked;
+  survivor updates are a manual observation, while timestamps and
+  unchanged daemon identities corroborate overlap and service reuse.
+- Startup in that journey hit the explicit single-null-sample
+  breakpoint several times before the charts overlapped. The small
+  debugger cleanup above was applied afterward; this live run does
+  not itself verify that later edit.
+- Departing charts still report cancellation groups and an FSP
+  receiver-close `ClosedResourceError`, followed by the Qt close
+  handler's self-SIGINT. Peer-disconnected final result warnings
+  remain contained in the persistent services.
+- Daemon SIGINT at 08:19:43 reproduced cached shared-client teardown:
+  `load_clients_for_trio()` / `_Cache.run_ctx` promoted
+  `TrioTaskExited` through `open_trade_dialog()` and entered PDB.
+  After the human continued, all three children were reported
+  exited with status 0 and the root terminated at 08:19:45.
+  Cancellation-only RPC error reporting and late cancel-ack
+  diagnostics also remain.
+
+The human explicitly accepted deferring this daemon teardown fix
+from #94 and requested loud attribution for the bounded backports.
+This is an operational smoke result, not clean-shutdown coverage.
+
+## Downstream reconciliation requirements
+
+Before landing #94, the human wants an easy replay of the downstream
+histories intended to follow it. Rehearse against the agreed #94
+tree in disposable worktrees, then inspect range-diffs, final code
+and relevant regressions. This note does not authorize rewriting
+the original branches or changing acceptance markers.
+
+- `0846cbd4` and `1da9b760` are partial backports. Their imported
+  adapter hunks may already be present, but their history/SHM/storage
+  changes and omitted integration tests remain necessary downstream.
+  Never drop either whole source commit merely because `feed.py`
+  matches or its adapter hunk becomes empty.
+- `12b0d66e` has source-equivalent EMS code and its imported test,
+  plus an additional local real-broadcaster test. Preserve that extra
+  regression when resolving test-file overlap.
+- The quote-cache backout intentionally restores direct ownership
+  for this landing. Reintroducing original `4af885d9` downstream
+  requires qualifying cached-owner teardown with the matching
+  Tractor revision; the backout is not a permanent ownership design.
+- The local single-null breakpoint removal retains legacy behavior.
+  Prefer the full downstream `976db6c3` algorithm when it is replayed;
+  preserving this `return None` guard would lose that later fix.
+- Real-bridge tests and manual qualification notes are local work,
+  not evidence that omitted portions of the source commits landed.
+
+Commit messages must name full source SHAs, original authorship,
+imported paths/tests, omissions, local additions and replay guidance.
+Use explicit partial-backport language for adapter subsets; do not
+claim a full cherry-pick or reuse the source's broader fix claims.
+
+## Remaining decision
+
+This candidate removes the reproduced fatal quote-restart
+propagation and imports existing reset fixes. The human exercised
+live QQQ startup, backfill and concurrent chart consumer lifetimes.
+Completion-callback and teardown errors remain; the human accepts
+them as follow-ups outside the bounded #94 landing.
+The storage/SHM portions of the downstream fixes are not covered by
+the adapter backport or its integration tests.
+
+`ddac3553` is Deribit-specific and omitted. Qt lifecycle commit
+`1a2d4893` stays a separate follow-up. Further Tractor teardown
+changes or storage imports require reassessing the bounded landing
+rather than silently expanding this candidate.

ai/prompt-io/opencode/20261022T212248Z_dc618229_ib_recovery_prompt_io.md

@@ -0,0 +1,67 @@
+---
+created_at: 2026-10-22T21:22:48Z
+generated_by:
+  harness: opencode
+  provider: openai
+  model: openai/gpt-6-astra
+git_ref: dc618229bd8c9c90aa3160e26677b553a7d472c7
+scope: code
+substantive: true
+raw_file: 20261022T212248Z_dc618229_ib_recovery_prompt_io.raw.md
+---
+
+## Prompt
+
+The human approved trying the bounded candidate: back out the IB
+quote-context cache, import adapter-only reset/boundary fixes and
+the missing EMS subscription fix, preserve source authorship and
+commit boundaries, and exercise the real Tractor bridge. They
+required an explicit keep/backout audit and clarification that
+the evaluated downstream lineage lacked the broadcast fix.
+
+## Response summary
+
+Verified identical stable patch IDs for original 4af885d9 and
+landing replay 635314c7. Created `wkt/land_ib_recovery` at dc618229
+in the main checkout. Applied source-equivalent adapter backports
+from 0846cbd4 and 1da9b760 and the EMS code from 12b0d66e, with
+source attribution recorded in `ai/ib-recovery-candidate.md`.
+
+The real-runtime quote test first reproduced the reported fatal
+TrioTaskExited propagation with caching enabled. The direct-owner
+backout restores restart and subsequent RPC survival, but a strict
+callback-cleanliness test remains red. Real bridge history pacing
+recovery and real broadcaster dark clearing pass. No live broker,
+VNC, orders or TSDB operations were performed by the agent.
+
+On 2026-10-23 the human supplied successful live startup/backfill,
+sequential reattachment and overlapping chart-close/reopen results,
+plus the still-failing daemon SIGINT teardown. The human accepted
+deferring teardown from #94, requested loud partial-backport
+attribution and required downstream rebase rehearsals before landing.
+The agent removed only the explicit single-null debugger stop,
+retaining its legacy result. Downstream `976db6c3` instead rewrites
+null grouping; the local cleanup does not claim that algorithmic fix.
+
+## Files changed
+
+- `piker/brokers/ib/feed.py`: source-equivalent backout/backports.
+- `piker/clearing/_ems.py`: original EMS subscription patch.
+- `tests/test_ib_broadcast.py`: removed with the reverted behavior.
+- `tests/test_ib_history.py`: original adapter regressions only.
+- `tests/test_ems_broadcast.py`: imported and real-broadcaster tests.
+- `tests/test_ib_quote_restart.py`: real Tractor boundary tests.
+- `piker/tsp/_anal.py`: local single-null breakpoint removal only.
+- `ai/ib-recovery-candidate.md`: provenance and qualification limits.
+
+## Human edits
+
+The human required original fixes and authorship, rejected broad
+speculative integration work, required real concurrency boundaries
+in tests, and corrected the interpretation of live qualification.
+They performed the live tests, chose to defer known teardown defects
+from #94, permitted the minimal debugger cleanup, and requested
+explicit downstream discrepancies and rebase compatibility checks.
+The agent applied those directed edits; no direct manual source edits
+are known in this candidate. The raw partner retains the originally
+captured output; this structured record includes later contributions.

ai/prompt-io/opencode/20261022T212248Z_dc618229_ib_recovery_prompt_io.raw.md

@@ -0,0 +1,69 @@
+---
+created_at: 2026-10-22T21:22:48Z
+generated_by:
+  harness: opencode
+  provider: openai
+  model: openai/gpt-6-astra
+git_ref: dc618229bd8c9c90aa3160e26677b553a7d472c7
+diff_cmd: git diff dc618229 -- piker/brokers/ib/feed.py piker/clearing/_ems.py tests
+---
+
+## Human direction
+
+> yeah i guess this is our best approach strat for sure.
+
+> so you're saying these 2 are not the same patch?
+> please tell me no lol
+
+> so detail which commits to drop vs. keep please in detail so i can
+> audit your choices this time..
+
+> let's try it and consider bailing and re-trying if this leads us into
+> a further (and deeper) sinkhole, bc as it stands right now this whole
+> endevour has turned into a complete shitfest XD
+
+> you didn't respond to my ask about the wkt/ib_downstream_pin_eval stuff and it missing the bcast rx commit either
+
+## Unedited diagnostic response
+
+The real-bridge test reproduced the exact fatal path:
+`maybe_open_context()` teardown raises `TrioTaskExited` back into
+the quote consumer.
+
+With only the quote-cache backout, both quote generations complete
+and their asyncio tasks finish. Tractor still emits
+`TrioTaskExited` callback errors, so the strict test remains red.
+That narrows the result: the backout removes the fatal propagation
+in this reproducer, but doesn’t make bridge teardown clean.
+
+## Code output pointers
+
+> `git diff dc618229 -- piker/brokers/ib/feed.py`
+
+Back out 635314c7's quote owner and import the original adapter-only
+deltas from 0846cbd4 and 1da9b760.
+
+> `git diff dc618229 -- piker/clearing/_ems.py`
+
+Import 12b0d66e's dedicated dark-clearing quote subscription.
+
+> `git diff dc618229 -- tests/test_ib_broadcast.py tests/test_ib_quote_restart.py tests/test_ib_history.py tests/test_ems_broadcast.py`
+
+Replace the reverted mock-cache test with a real-runtime restart
+reproducer; retain original history/EMS tests and add real bridge
+history recovery and real broadcaster EMS coverage. Newly added
+test files remain untracked until the human stages them, so the
+diff pointers include them only after staging or committing.
+
+## Verification output
+
+```text
+21 passed in 0.53s
+1 failed, 2 passed in 5.53s
+13 passed in 1.41s
+1 passed, 1 warning in 0.04s
+2 passed in 0.03s
+```
+
+The last run includes the original EMS case again and one additional
+real-broadcaster case: 38 unique passes and one strict callback failure.