Reorient E2E plan toward live charts
Deats, - park replay until a concrete regression needs controlled data - prioritize nonblocking `QtractorSession` lifecycle ownership - define isolated real Kraken chart and QtBot qualification - record entry-point plugin migration for replay resumption Prompt-IO: ai/prompt-io/opencode/20260903T192744Z_aadfd1cf_prompt_io.md (this patch was generated in some part by `opencode` using `gpt-5.6-sol` (`openai`))wkt/replay_provider_e2e
parent
ee33492c81
commit
06fee7000b
|
|
@ -0,0 +1,46 @@
|
||||||
|
---
|
||||||
|
model: openai/gpt-5.6-sol
|
||||||
|
service: opencode
|
||||||
|
session: 8737d0a9-98da-4fa1-ba35-ca8a3a42bbc9
|
||||||
|
timestamp: 2026-09-03T19:27:44Z
|
||||||
|
git_ref: wkt/replay_provider_e2e
|
||||||
|
scope: docs
|
||||||
|
substantive: true
|
||||||
|
raw_file: 20260903T192744Z_aadfd1cf_prompt_io.raw.md
|
||||||
|
---
|
||||||
|
|
||||||
|
## Prompt
|
||||||
|
|
||||||
|
Perform a deep review of the human-facing E2E plan against real broker
|
||||||
|
and application machinery. Prioritize booting production charts against
|
||||||
|
mostly continuous live backends, isolate tests from any production Piker
|
||||||
|
instance on the host, put replay entirely on hold, and provide immediate
|
||||||
|
practical next steps for Qt plus Trio application testing.
|
||||||
|
|
||||||
|
Record the package-entry-point replay move as a follow-up if replay is
|
||||||
|
ever resumed.
|
||||||
|
|
||||||
|
## Response summary
|
||||||
|
|
||||||
|
Reorients the roadmap around the actual application blockers and a live
|
||||||
|
Kraken vertical slice. The review separates test boundary from data
|
||||||
|
source, defines exact runtime isolation, specifies a nonblocking
|
||||||
|
`QtractorSession`, and defers replay until a real regression supplies a
|
||||||
|
concrete reason for controlled input.
|
||||||
|
|
||||||
|
## Files changed
|
||||||
|
|
||||||
|
- `plans/opencode/live-backend-chart-e2e-review.md` - record the revised
|
||||||
|
live-backend-first roadmap, lifecycle seam, isolation contract, and
|
||||||
|
replay/plugin resumption design.
|
||||||
|
|
||||||
|
## Human edits
|
||||||
|
|
||||||
|
The human rejected deterministic replay as the organizing prerequisite
|
||||||
|
for E2E, prioritized failures observable through real broker and chart
|
||||||
|
machinery, selected the Qt/Trio lifecycle as immediate work, approved a
|
||||||
|
live Kraken chart journey followed by real interactions and installed
|
||||||
|
process coverage, and required replay to remain parked unless motivated
|
||||||
|
by a concrete regression. The human also required the future replay
|
||||||
|
move to use general package-entry-point backend support and explicitly
|
||||||
|
approved provenance capture for this design document.
|
||||||
|
|
@ -0,0 +1,30 @@
|
||||||
|
---
|
||||||
|
model: openai/gpt-5.6-sol
|
||||||
|
service: opencode
|
||||||
|
timestamp: 2026-09-03T19:27:44Z
|
||||||
|
git_ref: wkt/replay_provider_e2e
|
||||||
|
diff_cmd: git diff HEAD~1..HEAD
|
||||||
|
---
|
||||||
|
|
||||||
|
## Generated document
|
||||||
|
|
||||||
|
> `git diff HEAD~1..HEAD -- plans/opencode/live-backend-chart-e2e-review.md`
|
||||||
|
|
||||||
|
Reviews the original replay-centered phase order against the actual
|
||||||
|
Qt/Trio host, chart startup, real backend, runtime isolation, and test
|
||||||
|
harness machinery. It parks replay, promotes the nonblocking production
|
||||||
|
application seam and live Kraken chart journey, defines exact isolation
|
||||||
|
and teardown expectations, and records package-entry-point migration for
|
||||||
|
any future replay resumption.
|
||||||
|
|
||||||
|
## Review conclusions
|
||||||
|
|
||||||
|
- Replay is not a technical prerequisite for Qt/Trio lifecycle work.
|
||||||
|
- `run_qtractor()` blocking, discarded guest outcomes, pytest-qt app
|
||||||
|
shutdown, process-wide close SIGINT, and missing readiness are the
|
||||||
|
immediate blockers.
|
||||||
|
- Existing XDG, config, Qt, and registry isolation is reusable.
|
||||||
|
- A public Kraken chart can exercise real REST, WebSocket, datad,
|
||||||
|
samplerd, SHM, paper EMS, rendering, input, and teardown machinery.
|
||||||
|
- Replay should return only for a concrete captured payload, race,
|
||||||
|
network-free CI requirement, or fault-control regression.
|
||||||
|
|
@ -0,0 +1,348 @@
|
||||||
|
# Live-backend chart E2E plan review
|
||||||
|
|
||||||
|
## Decision
|
||||||
|
|
||||||
|
Reorient the human-facing test effort around the production Qt chart
|
||||||
|
running against real public backends. Put replay implementation,
|
||||||
|
scenario design, provider failure injection, and replay-backed CI on
|
||||||
|
hold until a real application or backend failure demonstrates a need
|
||||||
|
for controlled data.
|
||||||
|
|
||||||
|
The first valuable target is an in-process application journey using
|
||||||
|
pytest-qt, Trio guest mode, an isolated Piker actor tree, and public
|
||||||
|
Kraken data. Installed-process and paper-order journeys follow after
|
||||||
|
that path can start, report readiness, and stop without process-wide
|
||||||
|
signals.
|
||||||
|
|
||||||
|
Entry-point backend discovery remains independently useful, but it is
|
||||||
|
not a prerequisite for chart E2E. Treat it as a short standalone patch
|
||||||
|
or defer it with replay.
|
||||||
|
|
||||||
|
## Review of the original plan
|
||||||
|
|
||||||
|
The original plan correctly identifies the end goal: user-facing QtBot
|
||||||
|
journeys paired with installed-process tests. It also correctly requires
|
||||||
|
real Tractor, IPC, SHM, feed, search, and EMS boundaries.
|
||||||
|
|
||||||
|
Its central ordering assumption is now rejected. It makes a deterministic
|
||||||
|
offline provider the organizing dependency for chart boot, search, paper
|
||||||
|
orders, public APIs, process tests, and failure handling. It schedules
|
||||||
|
that provider and a general subprocess harness before the production
|
||||||
|
Qt/Trio lifecycle seam.
|
||||||
|
|
||||||
|
That ordering optimizes for deterministic default CI. It does not optimize
|
||||||
|
for finding failures users currently encounter with real venues, real
|
||||||
|
history payloads, real quote timing, real symbology, and the composed chart.
|
||||||
|
|
||||||
|
The replay phase is not technically required by the later lifecycle work:
|
||||||
|
|
||||||
|
1. Qt/Trio startup and shutdown are provider-independent.
|
||||||
|
2. A chart can use Kraken's public history and quote APIs without credentials.
|
||||||
|
3. Search can use the real Kraken symbol service.
|
||||||
|
4. Paper EMS can consume a real public feed without spawning live brokerd.
|
||||||
|
5. Installed chart startup can use an isolated registry and real provider.
|
||||||
|
6. Guest errors, child failures, SIGINT behavior, and repeated-session leaks
|
||||||
|
do not require synthetic provider failures.
|
||||||
|
|
||||||
|
Replay remains potentially useful for forcing exact races, reproducing a
|
||||||
|
captured venue payload, denying network in required CI, or proving a fix for
|
||||||
|
a specific data-dependent regression. Those are follow-up uses, not current
|
||||||
|
prerequisites.
|
||||||
|
|
||||||
|
## What replay would and would not audit
|
||||||
|
|
||||||
|
A replay-backed chart journey would prove that the composed application can
|
||||||
|
consume one protocol-conforming source, render known bars, process known
|
||||||
|
quotes, and shut down. It would be useful interface and lifecycle coverage.
|
||||||
|
|
||||||
|
It would not prove the behavior most likely to drift outside the repository:
|
||||||
|
|
||||||
|
- exchange REST and WebSocket availability;
|
||||||
|
- TLS, DNS, geo-routing, and endpoint changes;
|
||||||
|
- authentication behavior accidentally applied to public requests;
|
||||||
|
- real symbology and market metadata changes;
|
||||||
|
- incomplete, duplicated, delayed, or out-of-order history;
|
||||||
|
- backend normalization against current venue payloads;
|
||||||
|
- reconnect and rate-limit behavior;
|
||||||
|
- the timing and volume of actual chart updates.
|
||||||
|
|
||||||
|
The current priority is the second list. Therefore real-provider application
|
||||||
|
qualification should precede further replay work.
|
||||||
|
|
||||||
|
## Current practical blockers
|
||||||
|
|
||||||
|
### Blocking application host
|
||||||
|
|
||||||
|
`run_qtractor()` currently creates or reuses `QApplication`, constructs the
|
||||||
|
production `MainWindow` and `GodWidget`, starts Trio guest mode, shows the
|
||||||
|
window, and then blocks in `app.exec_()`.
|
||||||
|
|
||||||
|
This prevents a synchronous pytest-qt test from receiving a session handle
|
||||||
|
while Qt owns the host loop.
|
||||||
|
|
||||||
|
### Unobservable Trio completion
|
||||||
|
|
||||||
|
The Trio guest `done_callback` prints unexpected errors and unconditionally
|
||||||
|
calls `app.quit()`. It does not retain the `outcome.Outcome`, expose a Qt
|
||||||
|
completion signal, or let pytest fail on the original exception.
|
||||||
|
|
||||||
|
Calling `app.quit()` also violates pytest-qt ownership of its session-scoped
|
||||||
|
application.
|
||||||
|
|
||||||
|
### Process-wide chart shutdown
|
||||||
|
|
||||||
|
`MainWindow.closeEvent()` saves geometry and sends SIGINT to the entire test
|
||||||
|
process. A QtBot-owned window cannot exercise normal close behavior without
|
||||||
|
interrupting pytest.
|
||||||
|
|
||||||
|
### No application readiness contract
|
||||||
|
|
||||||
|
`_async_main()` has a meaningful readiness point after the initial chart,
|
||||||
|
paper order mode, search handlers, and status cleanup are active, but it does
|
||||||
|
not publish that state outside its Trio task.
|
||||||
|
|
||||||
|
### Incomplete child-resource accounting
|
||||||
|
|
||||||
|
The test harness isolates XDG paths, config paths, registry addresses, and
|
||||||
|
current-process SHM. A chart application test must additionally record the
|
||||||
|
specific child actor IDs and SHM names created by datad so teardown can prove
|
||||||
|
those exact resources disappeared without broad cleanup.
|
||||||
|
|
||||||
|
## Existing foundation to keep
|
||||||
|
|
||||||
|
The completed harness foundation already provides:
|
||||||
|
|
||||||
|
- pytest-qt ownership of one `QApplication`;
|
||||||
|
- import-time PyQt6 and offscreen selection with `--headless`;
|
||||||
|
- process-level XDG isolation before Piker and Qt imports;
|
||||||
|
- function-scoped restoration of Piker, Qt, QSettings, and PyQtGraph state;
|
||||||
|
- real QtBot key delivery and widget registration;
|
||||||
|
- unique Tractor registry addresses from the Tractor pytest plugin;
|
||||||
|
- test-owned config directories propagated to child actors through
|
||||||
|
`piker_test_dir` runtime variables;
|
||||||
|
- exact current-process SHM ownership checks.
|
||||||
|
|
||||||
|
This is sufficient foundation for the lifecycle extraction. Replay is not
|
||||||
|
needed to begin it.
|
||||||
|
|
||||||
|
## Revised test model
|
||||||
|
|
||||||
|
Keep independent dimensions instead of defining application coverage by its
|
||||||
|
data source:
|
||||||
|
|
||||||
|
| Dimension | Initial selection | Later selections |
|
||||||
|
|---|---|---|
|
||||||
|
| Boundary | In-process production application | Installed chart process |
|
||||||
|
| Data source | Live Kraken public data | Binance, other qualified backends |
|
||||||
|
| Display | Qt offscreen through pytest-qt | Real Wayland/X11 compositor |
|
||||||
|
| Clearing | Real paper EMS | Credentialed live broker qualification |
|
||||||
|
| Cadence | Explicit local/CI qualification | Repeated soak and release gate |
|
||||||
|
|
||||||
|
The first test is therefore a headless, live-provider, in-process application
|
||||||
|
journey. It is not a deterministic default unit test, and it should not be
|
||||||
|
presented as one.
|
||||||
|
|
||||||
|
## Isolation contract
|
||||||
|
|
||||||
|
Real provider access does not require attaching to a developer's production
|
||||||
|
Piker runtime. Each application test must:
|
||||||
|
|
||||||
|
1. Use the Tractor pytest plugin's unique registry address.
|
||||||
|
2. Pass that address explicitly through `run_qtractor()` into
|
||||||
|
`maybe_open_pikerd()`; never allow fallback to the default registry.
|
||||||
|
3. Propagate the test-owned config directory through
|
||||||
|
`tractor_runtime_overrides['piker_test_dir']`.
|
||||||
|
4. Start a fresh local `pikerd`, `datad.kraken`, and `samplerd` under the
|
||||||
|
test's registry.
|
||||||
|
5. Use public Kraken data only; do not load user credentials.
|
||||||
|
6. Record every actor identity and each chart feed's exact SHM names.
|
||||||
|
7. Request structured shutdown through the application session.
|
||||||
|
8. Assert those actors, streams, registry connections, and SHM names are gone.
|
||||||
|
9. Never use process-name matching, broad SHM scans, `pkill`, or global ports.
|
||||||
|
10. Treat the external network as intentionally shared qualification input.
|
||||||
|
|
||||||
|
An explicitly selected live test must fail on connection or provider errors,
|
||||||
|
not silently skip after startup. Tests skip only when live qualification was
|
||||||
|
not requested.
|
||||||
|
|
||||||
|
## Immediate vertical slice
|
||||||
|
|
||||||
|
### Production lifecycle seam
|
||||||
|
|
||||||
|
Extract a nonblocking `start_qtractor()` from `run_qtractor()`.
|
||||||
|
|
||||||
|
It should return a `QtractorSession` containing:
|
||||||
|
|
||||||
|
- the production `MainWindow` and `GodWidget`;
|
||||||
|
- a Trio token and owned cancellation scope;
|
||||||
|
- an application-ready Qt signal;
|
||||||
|
- a completion Qt signal;
|
||||||
|
- the final `outcome.Outcome`;
|
||||||
|
- a structured `request_shutdown()` method;
|
||||||
|
- whether the adapter or caller owns `QApplication.exec()`.
|
||||||
|
|
||||||
|
`start_qtractor()` must configure and show the production objects but must not
|
||||||
|
call `app.exec()` or `app.quit()`. `run_qtractor()` remains the blocking CLI
|
||||||
|
adapter: it calls `start_qtractor()`, owns `app.exec()`, and unwraps the final
|
||||||
|
outcome into a process result.
|
||||||
|
|
||||||
|
Change `MainWindow.closeEvent()` to request session shutdown through an
|
||||||
|
injected callback. Keep geometry persistence. Remove process-wide SIGINT from
|
||||||
|
normal window closure; terminal SIGINT remains an outer CLI concern.
|
||||||
|
|
||||||
|
Publish application readiness from `_async_main()` only after the initial
|
||||||
|
feed, charts, paper order mode, search handlers, and startup status have all
|
||||||
|
entered their live scopes.
|
||||||
|
|
||||||
|
### First live chart journey
|
||||||
|
|
||||||
|
Add an explicitly selected test such as:
|
||||||
|
|
||||||
|
`tests/app/test_live_chart.py::test_kraken_chart_boots_and_closes`
|
||||||
|
|
||||||
|
Use `xbtusd.spot.kraken`, because Kraken public REST/WebSocket data is
|
||||||
|
credential-free and the pair is continuously active.
|
||||||
|
|
||||||
|
The test should:
|
||||||
|
|
||||||
|
1. Start `QtractorSession` with pytest-qt's `QApplication`.
|
||||||
|
2. Pass the unique test registry and child config root.
|
||||||
|
3. Register `session.window` with `qtbot` immediately.
|
||||||
|
4. Wait for the application-ready signal with a bounded timeout.
|
||||||
|
5. Assert the displayed FQME and production window title.
|
||||||
|
6. Assert both historical and real-time chart visualizations have nonempty
|
||||||
|
data from their real SHM arrays.
|
||||||
|
7. Observe at least one post-start quote/display update from Kraken.
|
||||||
|
8. Drive Ctrl-L with QtBot and assert the real search bar receives focus.
|
||||||
|
9. Drive the existing search-dismiss interaction and assert chart focus.
|
||||||
|
10. Request structured close and wait for the completion signal.
|
||||||
|
11. Unwrap a successful Trio outcome.
|
||||||
|
12. Prove exact actor and SHM teardown.
|
||||||
|
|
||||||
|
Do not assert pixels, exact prices, exact bar counts, result ordering, actor
|
||||||
|
PIDs, generated SHM names, or quote arrival within an unrealistically short
|
||||||
|
interval. Those values vary while the semantic behavior remains correct.
|
||||||
|
|
||||||
|
## Live-provider selection
|
||||||
|
|
||||||
|
Use Kraken as the initial baseline:
|
||||||
|
|
||||||
|
- public market metadata, OHLC, and WebSocket data need no credentials;
|
||||||
|
- `xbtusd.spot.kraken` and `ethusdt.spot.kraken` already appear in live feed
|
||||||
|
tests;
|
||||||
|
- the existing suite allows Kraken feed coverage in CI;
|
||||||
|
- crypto trading is continuous outside venue maintenance and outages.
|
||||||
|
|
||||||
|
Use Binance spot as a second qualification only where network and geographic
|
||||||
|
access are known. Keep Kucoin informational until its public configuration and
|
||||||
|
startup behavior receive direct repair. Exclude IB, Questrade, and current
|
||||||
|
Deribit from credential-free chart qualification.
|
||||||
|
|
||||||
|
Add one explicit selection mechanism, preferably `--live-provider=kraken`.
|
||||||
|
Without it, live application tests skip. With it, backend startup failure is a
|
||||||
|
test failure and retains the original Trio/Tractor exception and diagnostics.
|
||||||
|
|
||||||
|
Do not add a blanket retry. Record cold-start duration and failure signatures.
|
||||||
|
A separate repeated-run or soak command can measure operational reliability
|
||||||
|
after the first journey is stable.
|
||||||
|
|
||||||
|
## Reordered implementation phases
|
||||||
|
|
||||||
|
### Phase A: Qt/Trio lifecycle
|
||||||
|
|
||||||
|
Implement `QtractorSession`, nonblocking startup, structured close, observable
|
||||||
|
outcome, and repeated-session cleanup. Verify first with a provider-independent
|
||||||
|
guest coroutine so failures are localized to lifecycle ownership.
|
||||||
|
|
||||||
|
### Phase B: real Kraken chart
|
||||||
|
|
||||||
|
Boot the production `_async_main()` against `xbtusd.spot.kraken` using the
|
||||||
|
isolated runtime contract. Prove chart readiness, live data, one QtBot focus
|
||||||
|
journey, and complete teardown.
|
||||||
|
|
||||||
|
### Phase C: real chart behavior
|
||||||
|
|
||||||
|
Add A-to-B-to-A Kraken symbol switching, chart navigation, timeframe changes,
|
||||||
|
and paper order open/cancel behavior. Prefer stable semantic assertions and
|
||||||
|
record real failures before extracting narrow regression tests.
|
||||||
|
|
||||||
|
### Phase D: installed-process coverage
|
||||||
|
|
||||||
|
Run the installed `piker chart` command with the same isolated registry/config
|
||||||
|
contract. Prove cold startup, readiness, graceful close, exit status, and exact
|
||||||
|
descendant cleanup. Add standalone `pikerd` attachment only after cold start
|
||||||
|
works.
|
||||||
|
|
||||||
|
### Phase E: qualification breadth
|
||||||
|
|
||||||
|
Add Binance where available, real compositor runs, repeated cold starts, and
|
||||||
|
bounded soak sessions. Keep provider outages visible as qualification results
|
||||||
|
rather than rewriting them into synthetic success.
|
||||||
|
|
||||||
|
### Deferred work
|
||||||
|
|
||||||
|
Defer replay scenarios, provider-conformance tests, deterministic quote
|
||||||
|
sequencing, synthetic provider failures, and replay-backed required CI.
|
||||||
|
|
||||||
|
Reconsider replay only when one of these concrete needs appears:
|
||||||
|
|
||||||
|
- a real payload must be preserved as a regression fixture;
|
||||||
|
- an ordering race cannot be reproduced reliably against a live venue;
|
||||||
|
- required network-free CI needs a minimal known-good chart source;
|
||||||
|
- a reconnect or malformed-data defect needs deterministic fault control.
|
||||||
|
|
||||||
|
Backend entry-point discovery may land separately if its implementation stays
|
||||||
|
small and useful to external providers. Do not make it block Phase A or B, and
|
||||||
|
do not use plugin support as a reason to continue replay work.
|
||||||
|
|
||||||
|
## Replay resumption record
|
||||||
|
|
||||||
|
If replay work resumes, first move its implementation out of the production
|
||||||
|
backend namespace and use it to introduce general external backend discovery:
|
||||||
|
|
||||||
|
1. Move the implementation to `piker.testing.brokers.replay`.
|
||||||
|
2. Add a `piker.brokers` Python entry-point group which resolves installed
|
||||||
|
external backend modules after conventional built-in imports.
|
||||||
|
3. Register replay through that entry point in Piker's package metadata.
|
||||||
|
4. Preserve a backend's actual module path when constructing datad and brokerd
|
||||||
|
`enable_modules` lists so Tractor authorizes the external endpoint module.
|
||||||
|
5. Reject duplicate names, built-in shadowing, non-module entry points, name
|
||||||
|
mismatches, and unsupported backend API versions.
|
||||||
|
6. Preserve dependency import failures from built-in backends instead of
|
||||||
|
misclassifying them as absent modules.
|
||||||
|
7. Cover resolver behavior with focused unit tests and prove one installed
|
||||||
|
entry-point backend through a spawned datad actor.
|
||||||
|
|
||||||
|
Do not use pytest-only `sys.modules` or package-attribute monkeypatching: those
|
||||||
|
registrations do not reliably cross spawn or forkserver actor boundaries.
|
||||||
|
External backends under active development can use editable installation in
|
||||||
|
the same Python environment as Piker.
|
||||||
|
|
||||||
|
## Practical sequence from the current worktree
|
||||||
|
|
||||||
|
The replay worktree contains the completed replay experiment. The unfinished
|
||||||
|
entry-point resolver experiment was deliberately excluded. Preserve this
|
||||||
|
branch as parked work rather than mixing it into the live chart patch.
|
||||||
|
|
||||||
|
Create the live-chart work on a fresh branch/worktree from the completed
|
||||||
|
Phase 1 head. The first implementation boundary should touch only the Qt/Trio
|
||||||
|
lifecycle and its provider-independent lifecycle tests. The second boundary
|
||||||
|
should add the opt-in real Kraken chart journey and any defects that journey
|
||||||
|
demonstrates.
|
||||||
|
|
||||||
|
If entry-point discovery is finished first, cap it at one independent resolver
|
||||||
|
commit with focused unit tests. Do not move or expand replay as part of that
|
||||||
|
commit.
|
||||||
|
|
||||||
|
## Effort estimate
|
||||||
|
|
||||||
|
- Entry-point resolver and unit tests: roughly half a day if kept independent.
|
||||||
|
- Qt/Trio lifecycle extraction and tests: one to two focused days.
|
||||||
|
- First isolated real Kraken chart journey: one to two days, depending on
|
||||||
|
defects revealed during startup and teardown.
|
||||||
|
- First QtBot search/focus behavior journey: roughly one additional day after
|
||||||
|
lifecycle stability.
|
||||||
|
- Installed-process chart coverage: one to two days after the in-process path.
|
||||||
|
|
||||||
|
The first real chart should therefore be reachable in roughly two to four
|
||||||
|
focused days without further replay work.
|
||||||
Loading…
Reference in New Issue