piker/plans/opencode/live-backend-chart-e2e-review.md

Live-backend chart E2E plan review

Decision

Reorient the human-facing test effort around the production Qt chart running against real public backends. Put replay implementation, scenario design, provider failure injection, and replay-backed CI on hold until a real application or backend failure demonstrates a need for controlled data.

The first valuable target is an in-process application journey using pytest-qt, Trio guest mode, an isolated Piker actor tree, and public Kraken data. Installed-process and paper-order journeys follow after that path can start, report readiness, and stop without process-wide signals.

Entry-point backend discovery remains independently useful, but it is not a prerequisite for chart E2E. Treat it as a short standalone patch or defer it with replay.

Review of the original plan

The original plan correctly identifies the end goal: user-facing QtBot journeys paired with installed-process tests. It also correctly requires real Tractor, IPC, SHM, feed, search, and EMS boundaries.

Its central ordering assumption is now rejected. It makes a deterministic offline provider the organizing dependency for chart boot, search, paper orders, public APIs, process tests, and failure handling. It schedules that provider and a general subprocess harness before the production Qt/Trio lifecycle seam.

That ordering optimizes for deterministic default CI. It does not optimize for finding failures users currently encounter with real venues, real history payloads, real quote timing, real symbology, and the composed chart.

The replay phase is not technically required by the later lifecycle work:

1. Qt/Trio startup and shutdown are provider-independent.
2. A chart can use Kraken’s public history and quote APIs without credentials.
3. Search can use the real Kraken symbol service.
4. Paper EMS can consume a real public feed without spawning live brokerd.
5. Installed chart startup can use an isolated registry and real provider.
6. Guest errors, child failures, SIGINT behavior, and repeated-session leaks do not require synthetic provider failures.

Replay remains potentially useful for forcing exact races, reproducing a captured venue payload, denying network in required CI, or proving a fix for a specific data-dependent regression. Those are follow-up uses, not current prerequisites.

What replay would and would not audit

A replay-backed chart journey would prove that the composed application can consume one protocol-conforming source, render known bars, process known quotes, and shut down. It would be useful interface and lifecycle coverage.

It would not prove the behavior most likely to drift outside the repository:

• exchange REST and WebSocket availability;
• TLS, DNS, geo-routing, and endpoint changes;
• authentication behavior accidentally applied to public requests;
• real symbology and market metadata changes;
• incomplete, duplicated, delayed, or out-of-order history;
• backend normalization against current venue payloads;
• reconnect and rate-limit behavior;
• the timing and volume of actual chart updates.

The current priority is the second list. Therefore real-provider application qualification should precede further replay work.

Current practical blockers

Blocking application host

run_qtractor() currently creates or reuses QApplication, constructs the production MainWindow and GodWidget, starts Trio guest mode, shows the window, and then blocks in app.exec_().

This prevents a synchronous pytest-qt test from receiving a session handle while Qt owns the host loop.

Unobservable Trio completion

The Trio guest done_callback prints unexpected errors and unconditionally calls app.quit(). It does not retain the outcome.Outcome, expose a Qt completion signal, or let pytest fail on the original exception.

Calling app.quit() also violates pytest-qt ownership of its session-scoped application.

Process-wide chart shutdown

MainWindow.closeEvent() saves geometry and sends SIGINT to the entire test process. A QtBot-owned window cannot exercise normal close behavior without interrupting pytest.

No application readiness contract

_async_main() has a meaningful readiness point after the initial chart, paper order mode, search handlers, and status cleanup are active, but it does not publish that state outside its Trio task.

Incomplete child-resource accounting

The test harness isolates XDG paths, config paths, registry addresses, and current-process SHM. A chart application test must additionally record the specific child actor IDs and SHM names created by datad so teardown can prove those exact resources disappeared without broad cleanup.

Existing foundation to keep

The completed harness foundation already provides:

• pytest-qt ownership of one QApplication;
• import-time PyQt6 and offscreen selection with --headless;
• process-level XDG isolation before Piker and Qt imports;
• function-scoped restoration of Piker, Qt, QSettings, and PyQtGraph state;
• real QtBot key delivery and widget registration;
• unique Tractor registry addresses from the Tractor pytest plugin;
• test-owned config directories propagated to child actors through piker_test_dir runtime variables;
• exact current-process SHM ownership checks.

This is sufficient foundation for the lifecycle extraction. Replay is not needed to begin it.

Revised test model

Keep independent dimensions instead of defining application coverage by its data source:

Dimension	Initial selection	Later selections
Boundary	In-process production application	Installed chart process
Data source	Live Kraken public data	Binance, other qualified backends
Display	Qt offscreen through pytest-qt	Real Wayland/X11 compositor
Clearing	Real paper EMS	Credentialed live broker qualification
Cadence	Explicit local/CI qualification	Repeated soak and release gate

The first test is therefore a headless, live-provider, in-process application journey. It is not a deterministic default unit test, and it should not be presented as one.

Isolation contract

Real provider access does not require attaching to a developer’s production Piker runtime. Each application test must:

1. Use the Tractor pytest plugin’s unique registry address.
2. Pass that address explicitly through run_qtractor() into maybe_open_pikerd(); never allow fallback to the default registry.
3. Propagate the test-owned config directory through tractor_runtime_overrides['piker_test_dir'].
4. Start a fresh local pikerd, datad.kraken, and samplerd under the test’s registry.
5. Use public Kraken data only; do not load user credentials.
6. Record every actor identity and each chart feed’s exact SHM names.
7. Request structured shutdown through the application session.
8. Assert those actors, streams, registry connections, and SHM names are gone.
9. Never use process-name matching, broad SHM scans, pkill, or global ports.
10. Treat the external network as intentionally shared qualification input.

An explicitly selected live test must fail on connection or provider errors, not silently skip after startup. Tests skip only when live qualification was not requested.

Immediate vertical slice

Production lifecycle seam

Extract a nonblocking start_qtractor() from run_qtractor().

It should return a QtractorSession containing:

• the production MainWindow and GodWidget;
• a Trio token and owned cancellation scope;
• an application-ready Qt signal;
• a completion Qt signal;
• the final outcome.Outcome;
• a structured request_shutdown() method;
• whether the adapter or caller owns QApplication.exec().

start_qtractor() must configure and show the production objects but must not call app.exec() or app.quit(). run_qtractor() remains the blocking CLI adapter: it calls start_qtractor(), owns app.exec(), and unwraps the final outcome into a process result.

Change MainWindow.closeEvent() to request session shutdown through an injected callback. Keep geometry persistence. Remove process-wide SIGINT from normal window closure; terminal SIGINT remains an outer CLI concern.

Publish application readiness from _async_main() only after the initial feed, charts, paper order mode, search handlers, and startup status have all entered their live scopes.

First live chart journey

Add an explicitly selected test such as:

tests/app/test_live_chart.py::test_kraken_chart_boots_and_closes

Use xbtusd.spot.kraken, because Kraken public REST/WebSocket data is credential-free and the pair is continuously active.

The test should:

1. Start QtractorSession with pytest-qt’s QApplication.
2. Pass the unique test registry and child config root.
3. Register session.window with qtbot immediately.
4. Wait for the application-ready signal with a bounded timeout.
5. Assert the displayed FQME and production window title.
6. Assert both historical and real-time chart visualizations have nonempty data from their real SHM arrays.
7. Observe at least one post-start quote/display update from Kraken.
8. Drive Ctrl-L with QtBot and assert the real search bar receives focus.
9. Drive the existing search-dismiss interaction and assert chart focus.
10. Request structured close and wait for the completion signal.
11. Unwrap a successful Trio outcome.
12. Prove exact actor and SHM teardown.

Do not assert pixels, exact prices, exact bar counts, result ordering, actor PIDs, generated SHM names, or quote arrival within an unrealistically short interval. Those values vary while the semantic behavior remains correct.

Live-provider selection

Use Kraken as the initial baseline:

• public market metadata, OHLC, and WebSocket data need no credentials;
• xbtusd.spot.kraken and ethusdt.spot.kraken already appear in live feed tests;
• the existing suite allows Kraken feed coverage in CI;
• crypto trading is continuous outside venue maintenance and outages.

Use Binance spot as a second qualification only where network and geographic access are known. Keep Kucoin informational until its public configuration and startup behavior receive direct repair. Exclude IB, Questrade, and current Deribit from credential-free chart qualification.

Add one explicit selection mechanism, preferably --live-provider=kraken. Without it, live application tests skip. With it, backend startup failure is a test failure and retains the original Trio/Tractor exception and diagnostics.

Do not add a blanket retry. Record cold-start duration and failure signatures. A separate repeated-run or soak command can measure operational reliability after the first journey is stable.

Reordered implementation phases

Phase A: Qt/Trio lifecycle

Implement QtractorSession, nonblocking startup, structured close, observable outcome, and repeated-session cleanup. Verify first with a provider-independent guest coroutine so failures are localized to lifecycle ownership.

Phase B: real Kraken chart

Boot the production _async_main() against xbtusd.spot.kraken using the isolated runtime contract. Prove chart readiness, live data, one QtBot focus journey, and complete teardown.

Phase C: real chart behavior

Add A-to-B-to-A Kraken symbol switching, chart navigation, timeframe changes, and paper order open/cancel behavior. Prefer stable semantic assertions and record real failures before extracting narrow regression tests.

Phase D: installed-process coverage

Run the installed piker chart command with the same isolated registry/config contract. Prove cold startup, readiness, graceful close, exit status, and exact descendant cleanup. Add standalone pikerd attachment only after cold start works.

Phase E: qualification breadth

Add Binance where available, real compositor runs, repeated cold starts, and bounded soak sessions. Keep provider outages visible as qualification results rather than rewriting them into synthetic success.

Deferred work

Defer replay scenarios, provider-conformance tests, deterministic quote sequencing, synthetic provider failures, and replay-backed required CI.

Reconsider replay only when one of these concrete needs appears:

• a real payload must be preserved as a regression fixture;
• an ordering race cannot be reproduced reliably against a live venue;
• required network-free CI needs a minimal known-good chart source;
• a reconnect or malformed-data defect needs deterministic fault control.

Backend entry-point discovery may land separately if its implementation stays small and useful to external providers. Do not make it block Phase A or B, and do not use plugin support as a reason to continue replay work.

Replay resumption record

If replay work resumes, first move its implementation out of the production backend namespace and use it to introduce general external backend discovery:

1. Move the implementation to piker.testing.brokers.replay.
2. Add a piker.brokers Python entry-point group which resolves installed external backend modules after conventional built-in imports.
3. Register replay through that entry point in Piker’s package metadata.
4. Preserve a backend’s actual module path when constructing datad and brokerd enable_modules lists so Tractor authorizes the external endpoint module.
5. Reject duplicate names, built-in shadowing, non-module entry points, name mismatches, and unsupported backend API versions.
6. Preserve dependency import failures from built-in backends instead of misclassifying them as absent modules.
7. Cover resolver behavior with focused unit tests and prove one installed entry-point backend through a spawned datad actor.

Do not use pytest-only sys.modules or package-attribute monkeypatching: those registrations do not reliably cross spawn or forkserver actor boundaries. External backends under active development can use editable installation in the same Python environment as Piker.

Practical sequence from the current worktree

The replay worktree contains the completed replay experiment. The unfinished entry-point resolver experiment was deliberately excluded. Preserve this branch as parked work rather than mixing it into the live chart patch.

Create the live-chart work on a fresh branch/worktree from the completed Phase 1 head. The first implementation boundary should touch only the Qt/Trio lifecycle and its provider-independent lifecycle tests. The second boundary should add the opt-in real Kraken chart journey and any defects that journey demonstrates.

If entry-point discovery is finished first, cap it at one independent resolver commit with focused unit tests. Do not move or expand replay as part of that commit.

Effort estimate

• Entry-point resolver and unit tests: roughly half a day if kept independent.
• Qt/Trio lifecycle extraction and tests: one to two focused days.
• First isolated real Kraken chart journey: one to two days, depending on defects revealed during startup and teardown.
• First QtBot search/focus behavior journey: roughly one additional day after lifecycle stability.
• Installed-process chart coverage: one to two days after the in-process path.

The first real chart should therefore be reachable in roughly two to four focused days without further replay work.