piker/.claude/skills/run-tests/test-harness-reference.md

# Test Harness Reference: piker

This repository-local file supplements the canonical `/run-tests` skill.
Keep shared execution, worktree, failure-inspection, and cleanup policy in the
canonical `SKILL.md`; keep Piker commands, paths, fixtures, and known outcomes
here.

## Project And Environment

- Project/import: `piker`
- Test root: `tests/`
- Supported Python: `>=3.12,<3.14`
- Preferred verified environment: worktree-local `.venv`.
- The testing group includes pytest and the locked `pytest-qt==4.5.0`.
- Verify the interpreter, local package resolution, and dependency versions
  before running tests.

This worktree's environment was provisioned with:

```text
env UV_PROJECT_ENVIRONMENT=.venv uv sync --group testing --frozen
```

Do not rerun provisioning without approval. Use the already-provisioned
environment directly:

```text
.venv/bin/python -m pytest
```

If direct environment paths are unavailable, use uv without changing the
environment, subject to the same import check:

```text
env UV_PROJECT_ENVIRONMENT=.venv uv run --frozen --no-sync python -m pytest
```

Ask before entering or provisioning alternate environments such as:

```text
nix develop
nix-shell default.nix
```

`nix develop` is the current Wayland/Qt 6 shell. `default.nix` is the current
X11 shell. Do not use `develop.nix` for current testing; it retains the old
Python 3.11, Poetry, and Qt 5 stack.

Do not enter `nix develop` without approval: its shell hook may provision its
own configured environment. Plain `uv sync` does not include the testing
group; use `--group testing` when provisioning is explicitly authorized. The
`dbs` dependency group is also absent from normal dev-shell provisioning.

## Commands

Base command in the preferred environment:

```text
.venv/bin/python -m pytest
```

The comments-only `pytest.ini` has been deleted. The authoritative
`[tool.pytest.ini_options]` in `pyproject.toml` sets `testpaths = ["tests"]`,
`addopts = "-p no:xonsh"`, and `qt_api = "pyqt6"`; do not repeat
`-p no:xonsh` in ordinary commands. Still pass a test path or node ID when a
deterministic scope is required.

Package-resolution check that does not import Piker's dependencies:

```text
.venv/bin/python -c 'import importlib.util, pathlib, sys; root = pathlib.Path.cwd().resolve(); spec = importlib.util.find_spec("piker"); mod = pathlib.Path(spec.origin).resolve(); print(sys.executable); print(mod); assert mod.is_relative_to(root)'
```

Dependency import check, required before collection or execution:

```text
.venv/bin/python -c 'from importlib.metadata import version; import pathlib, piker, pytestqt, sys; root = pathlib.Path.cwd().resolve(); mod = pathlib.Path(piker.__file__).resolve(); qt_ver = version("pytest-qt"); print(sys.executable); print(mod); print(qt_ver); assert mod.is_relative_to(root); assert qt_ver == "4.5.0"'
```

Safe core collection check:

```text
.venv/bin/python -m pytest -q --collect-only tests/test_watchlists.py tests/test_accounting.py tests/test_services.py tests/test_ems.py tests/test_feeds.py tests/test_cli.py
```

Default first-pass flags are `-q -x --tb=short --no-header` unless the user
requests otherwise. For actor-heavy tests, use one file or node per process
with an outer timeout:

```text
timeout -k 5 300 .venv/bin/python -m pytest -q <one-file-or-node>
```

If an actor-heavy command exits `124` or `143`, retry that exact command once
and report both attempts. Never convert a retry pass into an unconditional
pass, and do not retry assertion, import, collection, or configuration
failures.

## Scope And Path Resolution

Resolve bare test filenames beneath `tests/`. Preserve complete node IDs and
apply `-k` only within explicitly selected paths. There is no marker-based
offline/full-suite split, so never use `pytest tests` as a deterministic
default.

Deterministic or local first-pass targets:

- `tests/test_watchlists.py`
- `tests/test_storage_audit.py`
- `tests/test_store_cli.py`
- `tests/test_backfill_audit_snippet.py`
- `tests/test_ib_history.py`
- `tests/test_ib_method_proxy.py`
- `tests/test_history_backfill.py`
- `tests/test_ldshm.py`
- `tests/test_dpi_font.py`
- `tests/test_gap_overlays.py`
- `tests/test_accounting.py::test_account_file_default_empty`
- `tests/test_services.py::test_runtime_boot`
- `tests/test_services.py::test_datad_spawn`
- `tests/test_ems.py::test_ems_err_on_bad_broker`

Require explicit authorization before running:

- `tests/test_feeds.py` - live Binance/Kraken feeds;
- `tests/test_services.py::test_ensure_datafeed_actors` - live Kraken feed;
- `tests/test_services.py::test_ensure_ems_in_paper_actors` - paper EMS with
  live Kraken symbology/feed access;
- `tests/test_ems.py::test_multi_fill_positions` - live backend setup and
  persisted state;
- `tests/test_accounting.py::test_paper_ledger_position_calcs` - tracked
  fixtures plus possible live symcache generation;
- `tests/test_accounting.py::test_ib_account_with_duplicated_mktids` - active
  broker/account configuration and state writes;
- `tests/test_docker_services.py` - optional dependencies and containers;
- `tests/test_questrade.py` - obsolete credentialed imports.

`tests/test_cli.py` is currently hard-skipped. It is not an active CLI
regression gate.

Never execute `piker store anal` or `piker store shm --write-parquet`
as tests. They are mutating or interactive operational commands.

## Project-Specific Flags And Backend Matrix

| Flag | Purpose |
|---|---|
| `--ll LEVEL` | Piker log level |
| `--confdir PATH` | Override `piker.config._config_dir` |
| `--spawn-backend trio|mp_spawn|mp_forkserver` | Tractor process backend |
| `--tpt-proto PROTO` | Tractor transport; one protocol per session |
| `--tpdb` / `--debug-mode` | Tractor plugin debug mode |
| `--pdb` | Standard pytest debugger |
| `-s` | No capture; required with `--pdb` and `open_test_pikerd` |

Do not invent `network`, `offline`, `docker`, `gui`, or `broker` markers; none
currently exists. `CI=1` is not an offline selector: feed tests still leave a
live Kraken case enabled. Current Piker tests exercise TCP; do not assume the
whole suite supports UDS merely because the Tractor plugin exposes it.

## Fixture Invariants

- The session `confdir` fixture does nothing unless `--confdir` is passed.
  Its claimed `tests/data` fallback is not implemented and that directory is
  absent.
- Function-scoped `tmpconfdir` changes process-global config state and does
  not restore the previous path. Use separate pytest processes when
  diagnosing state leakage.
- `open_test_pikerd` passes the temporary config path to child actors through
  `tractor_runtime_overrides`.
- `tests/_inputs/trades_binance_paper.toml` and
  `tests/_inputs/account.binance.paper.toml` are used in place. Accounting
  contexts can write them on exit. Inspect `git diff -- tests/_inputs` after
  any selected accounting case.
- `tests/conftest.py` selects offscreen PyQt6 defaults before test modules
  import Qt, and its `qapp_args` fixture isolates `XDG_CONFIG_HOME` before
  pytest-qt creates the application.
- Piker and the installed Tractor pytest plugin do not provide a
  repository-local process or socket reaper. Never apply historical broad
  `pkill -f tractor._child` guidance automatically.
- The function-scoped autouse `shm_leak_tracker` fixture wraps Tractor's
  current-process `SharedMemory` factory. It tracks only successful
  `create=True` calls, restores the pre-test `_known_tokens` cache, and
  unlinks exact surviving names before failing the leaking test. It never
  scans `/dev/shm` or unlinks attachments created by another process.

## Qt/UI Tier

The default pytest process environment is `QT_QPA_PLATFORM=offscreen` and
`PYTEST_QT_API=pyqt6`. `tests/conftest.py` installs both with `setdefault`
before Qt imports, so an explicit caller environment still wins. The
authoritative pytest config also selects `qt_api = "pyqt6"`, and
`pytest-qt==4.5.0` is locked.

The session `qapp_args` fixture assigns a temporary `XDG_CONFIG_HOME` before
pytest-qt creates its `QApplication` and uses `piker-tests` as the application
argument. Deterministic offscreen Qt tests are normal local targets; they are
not opt-in merely because they import Qt or might otherwise read user config.

Exact verified gap-overlay commands:

```text
.venv/bin/python -m pytest -q --collect-only tests/test_gap_overlays.py
.venv/bin/python -m pytest -q -x --tb=short --no-header tests/test_gap_overlays.py
```

The first command collected 9 tests. The second passed all 9 in 2.00s with
one Tractor `trio.Event` boolean-use deprecation warning.

- Let `qtbot` own every widget registered with `qtbot.addWidget()` through
  teardown. Do not manually close or delete the same widget a second time.
- Keep production `MainWindow` out of this tier: its `closeEvent()` sends
  `SIGINT` to the pytest process.
- A real compositor is an explicit process override, for example:

```text
QT_QPA_PLATFORM=wayland .venv/bin/python -m pytest -q -x --tb=short --no-header tests/test_gap_overlays.py
QT_QPA_PLATFORM=xcb .venv/bin/python -m pytest -q -x --tb=short --no-header tests/test_gap_overlays.py
```

Real-compositor, visual, manual, and live UI tests remain opt-in. Do not
replace the offscreen default or infer compositor availability without
explicit authorization.

## Test Layout

```text
tests/
  conftest.py               Piker options, config fixtures, Tractor plugin
  _inputs/                  tracked ledger/account fixtures
  test_accounting.py        config, ledgers, accounts, and position math
  test_cli.py               legacy CLI suite; hard-skipped
  test_docker_services.py   container integrations; optional deps, skipped
  test_dpi_font.py          Qt DPI/font behavior
  test_ems.py               actor, EMS, and paper-position behavior
  test_feeds.py             live Binance/Kraken feeds and shared memory
  test_gap_overlays.py      typed gap logic, offscreen Qt, and Tractor IPC
  test_ib_history.py        deterministic IB history request formatting
  test_ib_method_proxy.py   deterministic IB asyncio proxy routing
  test_history_backfill.py  deterministic history/SHM orchestration
  test_ldshm.py             SHM unpublished-slot guard
  test_questrade.py         obsolete credentialed tests; skipped
  test_services.py          pikerd/datad/feed/EMS actor lifecycle
  test_store_cli.py         storage command help and diagnostics UX
  test_storage_audit.py     read-only NativeDB audit and JSON CLI
  test_backfill_audit_snippet.py
                            disposable xonsh qualification helpers
  test_watchlists.py        deterministic watchlist JSON operations
```

## Change-To-Test Mapping

| Changed area | Run first | Caveat |
|---|---|---|
| `piker/watchlists/` | `tests/test_watchlists.py` | CLI suite is skipped |
| `piker/storage/_audit.py`, `piker/storage/cli.py` | `tests/test_storage_audit.py` | direct Typer app, no actor |
| `piker/storage/cli.py` command UX | `tests/test_store_cli.py` | fake SHM/runtime, no mutation |
| `snippets/nativedb_backfill_audit.xsh` | `tests/test_backfill_audit_snippet.py` | disposable paths only |
| `piker/brokers/ib/api.py`, `feed.py` history | `tests/test_ib_history.py` | fake client, no network |
| `piker/brokers/ib/api.py` method proxy | `tests/test_ib_method_proxy.py` | fake channel, no network |
| `piker/tsp/_history.py` | `tests/test_history_backfill.py` | fake provider/storage/SHM |
| `piker/storage/cli.py` SHM null-slot guard | `tests/test_ldshm.py` | synthetic timestamps, no SHM mutation |
| `piker/config.py` | `test_account_file_default_empty` | root-network test has a known mismatch |
| `piker/accounting/` | targeted accounting node | some cases use live/configured state |
| `piker/ui/_gaps.py`, `_annotate.py`, `_display.py`, `_interaction.py`, `_remote_ctl.py` | `tests/test_gap_overlays.py` | offscreen PyQt6 plus one local Tractor actor |
| `piker/ui/_style.py`, `piker/ui/qt.py` | `tests/test_dpi_font.py` | deterministic offscreen Qt |
| `piker/service/_actor_runtime.py`, `_registry.py`, `_mngr.py` | `test_runtime_boot` | then `test_datad_spawn` |
| `piker/service/`, `piker/data/_daemon.py` | `test_datad_spawn` | feed lifecycle cases are live |
| `piker/data/feed.py`, `flows.py`, `_sharedmem.py`, `_sampling.py` | collect first | feed execution needs live permission |
| `piker/clearing/` | `test_ems_err_on_bad_broker` | multi-fill case is live/persisted |
| `piker/brokers/binance/`, `kraken/` | selected feed/accounting node | live network |
| `piker/brokers/ib/` | duplicated-market-ID node | controlled account config required |
| Docker/service adapters | `tests/test_docker_services.py` | optional deps and containers |
| project, lock, or Nix files | import check and safe collection | full collection is not safe by default |

Prefer deterministic filesystem/config and offscreen Qt tests, then local
actor-runtime nodes, then explicitly approved live broker, visual UI, or
container coverage.

## Quick Checks

```text
.venv/bin/python -c 'import importlib.util, pathlib, sys; root = pathlib.Path.cwd().resolve(); spec = importlib.util.find_spec("piker"); mod = pathlib.Path(spec.origin).resolve(); print(sys.executable); print(mod); assert mod.is_relative_to(root)'
.venv/bin/python -c 'from importlib.metadata import version; import pathlib, piker, pytestqt, sys; root = pathlib.Path.cwd().resolve(); mod = pathlib.Path(piker.__file__).resolve(); qt_ver = version("pytest-qt"); print(sys.executable); print(mod); print(qt_ver); assert mod.is_relative_to(root); assert qt_ver == "4.5.0"'
.venv/bin/python -m pytest -q --collect-only tests/test_gap_overlays.py
.venv/bin/python -m pytest -q -x --tb=short --no-header tests/test_gap_overlays.py
.venv/bin/python -m pytest -q tests/test_watchlists.py
.venv/bin/python -m pytest -q tests/test_accounting.py::test_account_file_default_empty
timeout -k 5 300 .venv/bin/python -m pytest -q tests/test_services.py::test_runtime_boot
timeout -k 5 300 .venv/bin/python -m pytest -q tests/test_services.py::test_datad_spawn
timeout -k 5 300 .venv/bin/python -m pytest -q tests/test_ems.py::test_ems_err_on_bad_broker
```

## Known Outcomes

- The verified `.venv` import check resolves `piker` beneath this worktree and
  imports the locked `pytest-qt` 4.5.0.
- `tests/test_accounting.py::test_root_conf_networking_section` currently
  expects `network.tsdb`, which is absent from the tracked config template.
  Match the current `KeyError: 'tsdb'` before classifying it as the known
  repository mismatch.
- `tests/test_docker_services.py` is marked skipped but imports
  `elasticsearch` first. Without the `dbs` group it fails collection rather
  than skipping.
- `tests/test_questrade.py` is marked skipped but imports undeclared `asks`
  through the legacy broker module before marks apply.
- `test_open_orders_reloaded` and `test_dark_order_clearing` in
  `tests/test_ems.py` contain only ellipsis bodies. A pass does not verify the
  named behavior.
- Treat `.pytest_cache` feed IDs using old FQME forms as stale historical
  cache, not current known failures.

Do not classify every `TooSlowError`, timeout, or child survivor as the known
Tractor teardown wedge. Match the selected node and the documented
second-runtime/lingering-child signature.

## Tractor Runtime Notes

The suite loads `tractor._testing.pytest`. Defaults are the `trio` spawn
backend and TCP transport. Registry addresses are normally session-unique,
except `tests/test_services.py::test_runtime_boot`, which binds
`127.0.0.1:6666`. Check that fixed port only for that node; ordinary tests do
not require Piker's production `127.0.0.1:6116` registry address.

Current repository concurrency notes document an intermittent second
in-process `pikerd` boot wedge with a lingering broker child and unread parent
IPC bytes. Use the outer timeout for actor-heavy nodes. Retry an exact command
once only after status `124` or `143`, and report both attempts.

Ordinary actor tests support normal capture. Diagnose a capture-dependent hang
by retrying only the exact node with `-s`. Standard `--pdb` with
`open_test_pikerd` requires `-s`; Tractor's `--tpdb` is a separate option.

After abnormal exit, inspect only descendants, sockets, and shared-memory
objects attributable to the exact pytest session. Ask before signaling or
unlinking anything.