pikers
/
piker
4
1
Fork 0
Code Issues 20 Pull Requests 12 Packages Projects Releases Wiki Activity
piker/.claude/skills/run-tests/test-harness-reference.md

16 KiB
Raw Blame History

Test Harness Reference: piker

This repository-local file supplements the canonical /run-tests skill. Keep shared execution, worktree, failure-inspection, and cleanup policy in the canonical SKILL.md; keep Piker commands, paths, fixtures, and known outcomes here.

Project And Environment

  • Project/import: piker
  • Test root: tests/
  • Supported Python: >=3.12,<3.14
  • Preferred verified environment: worktree-local .venv.
  • The testing group includes pytest and the locked pytest-qt==4.5.0.
  • Verify the interpreter, local package resolution, and dependency versions before running tests.

This worktree’s environment was provisioned with:

env UV_PROJECT_ENVIRONMENT=.venv uv sync --group testing --frozen

Do not rerun provisioning without approval. Use the already-provisioned environment directly:

.venv/bin/python -m pytest

If direct environment paths are unavailable, use uv without changing the environment, subject to the same import check:

env UV_PROJECT_ENVIRONMENT=.venv uv run --frozen --no-sync python -m pytest

Ask before entering or provisioning alternate environments such as:

nix develop
nix-shell default.nix

nix develop is the current Wayland/Qt 6 shell. default.nix is the current X11 shell. Do not use develop.nix for current testing; it retains the old Python 3.11, Poetry, and Qt 5 stack.

Do not enter nix develop without approval: its shell hook may provision its own configured environment. Plain uv sync does not include the testing group; use --group testing when provisioning is explicitly authorized. The dbs dependency group is also absent from normal dev-shell provisioning.

Commands

Base command in the preferred environment:

.venv/bin/python -m pytest

The comments-only pytest.ini has been deleted. The authoritative [tool.pytest.ini_options] in pyproject.toml sets testpaths = ["tests"], addopts = "-p no:xonsh", and qt_api = "pyqt6"; do not repeat -p no:xonsh in ordinary commands. Still pass a test path or node ID when a deterministic scope is required.

Package-resolution check that does not import Piker’s dependencies:

.venv/bin/python -c 'import importlib.util, pathlib, sys; root = pathlib.Path.cwd().resolve(); spec = importlib.util.find_spec("piker"); mod = pathlib.Path(spec.origin).resolve(); print(sys.executable); print(mod); assert mod.is_relative_to(root)'

Dependency import check, required before collection or execution:

.venv/bin/python -c 'from importlib.metadata import version; import pathlib, piker, pytestqt, sys; root = pathlib.Path.cwd().resolve(); mod = pathlib.Path(piker.__file__).resolve(); qt_ver = version("pytest-qt"); print(sys.executable); print(mod); print(qt_ver); assert mod.is_relative_to(root); assert qt_ver == "4.5.0"'

Safe core collection check:

.venv/bin/python -m pytest -q --collect-only tests/test_watchlists.py tests/test_accounting.py tests/test_services.py tests/test_ems.py tests/test_feeds.py tests/test_cli.py

Default first-pass flags are -q -x --tb=short --no-header unless the user requests otherwise. For actor-heavy tests, use one file or node per process with an outer timeout:

timeout -k 5 300 .venv/bin/python -m pytest -q <one-file-or-node>

If an actor-heavy command exits 124 or 143, retry that exact command once and report both attempts. Never convert a retry pass into an unconditional pass, and do not retry assertion, import, collection, or configuration failures.

Scope And Path Resolution

Resolve bare test filenames beneath tests/. Preserve complete node IDs and apply -k only within explicitly selected paths. There is no marker-based offline/full-suite split, so never use pytest tests as a deterministic default.

Deterministic or local first-pass targets:

  • tests/test_watchlists.py
  • tests/test_storage_audit.py
  • tests/test_store_cli.py
  • tests/test_backfill_audit_snippet.py
  • tests/test_ib_history.py
  • tests/test_ib_method_proxy.py
  • tests/test_history_backfill.py
  • tests/test_ldshm.py
  • tests/test_dpi_font.py
  • tests/test_gap_overlays.py
  • tests/test_accounting.py::test_account_file_default_empty
  • tests/test_services.py::test_runtime_boot
  • tests/test_services.py::test_datad_spawn
  • tests/test_ems.py::test_ems_err_on_bad_broker

Require explicit authorization before running:

  • tests/test_feeds.py - live Binance/Kraken feeds;
  • tests/test_services.py::test_ensure_datafeed_actors - live Kraken feed;
  • tests/test_services.py::test_ensure_ems_in_paper_actors - paper EMS with live Kraken symbology/feed access;
  • tests/test_ems.py::test_multi_fill_positions - live backend setup and persisted state;
  • tests/test_accounting.py::test_paper_ledger_position_calcs - tracked fixtures plus possible live symcache generation;
  • tests/test_accounting.py::test_ib_account_with_duplicated_mktids - active broker/account configuration and state writes;
  • tests/test_docker_services.py - optional dependencies and containers;
  • tests/test_questrade.py - obsolete credentialed imports.

tests/test_cli.py is currently hard-skipped. It is not an active CLI regression gate.

Never execute piker store anal or piker store shm --write-parquet as tests. They are mutating or interactive operational commands.

Project-Specific Flags And Backend Matrix

Flag Purpose
--ll LEVEL Piker log level
--confdir PATH Override piker.config._config_dir
--spawn-backend trio|mp_spawn|mp_forkserver Tractor process backend
--tpt-proto PROTO Tractor transport; one protocol per session
--tpdb / --debug-mode Tractor plugin debug mode
--pdb Standard pytest debugger
-s No capture; required with --pdb and open_test_pikerd

Do not invent network, offline, docker, gui, or broker markers; none currently exists. CI=1 is not an offline selector: feed tests still leave a live Kraken case enabled. Current Piker tests exercise TCP; do not assume the whole suite supports UDS merely because the Tractor plugin exposes it.

Fixture Invariants

  • The session confdir fixture does nothing unless --confdir is passed. Its claimed tests/data fallback is not implemented and that directory is absent.
  • Function-scoped tmpconfdir changes process-global config state and does not restore the previous path. Use separate pytest processes when diagnosing state leakage.
  • open_test_pikerd passes the temporary config path to child actors through tractor_runtime_overrides.
  • tests/_inputs/trades_binance_paper.toml and tests/_inputs/account.binance.paper.toml are used in place. Accounting contexts can write them on exit. Inspect git diff -- tests/_inputs after any selected accounting case.
  • tests/conftest.py selects offscreen PyQt6 defaults before test modules import Qt, and its qapp_args fixture isolates XDG_CONFIG_HOME before pytest-qt creates the application.
  • Piker and the installed Tractor pytest plugin do not provide a repository-local process or socket reaper. Never apply historical broad pkill -f tractor._child guidance automatically.
  • The function-scoped autouse shm_leak_tracker fixture wraps Tractor’s current-process SharedMemory factory. It tracks only successful create=True calls, restores the pre-test _known_tokens cache, and unlinks exact surviving names before failing the leaking test. It never scans /dev/shm or unlinks attachments created by another process.

Qt/UI Tier

The default pytest process environment is QT_QPA_PLATFORM=offscreen and PYTEST_QT_API=pyqt6. tests/conftest.py installs both with setdefault before Qt imports, so an explicit caller environment still wins. The authoritative pytest config also selects qt_api = "pyqt6", and pytest-qt==4.5.0 is locked.

The session qapp_args fixture assigns a temporary XDG_CONFIG_HOME before pytest-qt creates its QApplication and uses piker-tests as the application argument. Deterministic offscreen Qt tests are normal local targets; they are not opt-in merely because they import Qt or might otherwise read user config.

Exact verified gap-overlay commands:

.venv/bin/python -m pytest -q --collect-only tests/test_gap_overlays.py
.venv/bin/python -m pytest -q -x --tb=short --no-header tests/test_gap_overlays.py

The first command collected 9 tests. The second passed all 9 in 2.00s with one Tractor trio.Event boolean-use deprecation warning.

  • Let qtbot own every widget registered with qtbot.addWidget() through teardown. Do not manually close or delete the same widget a second time.
  • Keep production MainWindow out of this tier: its closeEvent() sends SIGINT to the pytest process.
  • A real compositor is an explicit process override, for example:
QT_QPA_PLATFORM=wayland .venv/bin/python -m pytest -q -x --tb=short --no-header tests/test_gap_overlays.py
QT_QPA_PLATFORM=xcb .venv/bin/python -m pytest -q -x --tb=short --no-header tests/test_gap_overlays.py

Real-compositor, visual, manual, and live UI tests remain opt-in. Do not replace the offscreen default or infer compositor availability without explicit authorization.

Test Layout

tests/
  conftest.py               Piker options, config fixtures, Tractor plugin
  _inputs/                  tracked ledger/account fixtures
  test_accounting.py        config, ledgers, accounts, and position math
  test_cli.py               legacy CLI suite; hard-skipped
  test_docker_services.py   container integrations; optional deps, skipped
  test_dpi_font.py          Qt DPI/font behavior
  test_ems.py               actor, EMS, and paper-position behavior
  test_feeds.py             live Binance/Kraken feeds and shared memory
  test_gap_overlays.py      typed gap logic, offscreen Qt, and Tractor IPC
  test_ib_history.py        deterministic IB history request formatting
  test_ib_method_proxy.py   deterministic IB asyncio proxy routing
  test_history_backfill.py  deterministic history/SHM orchestration
  test_ldshm.py             SHM unpublished-slot guard
  test_questrade.py         obsolete credentialed tests; skipped
  test_services.py          pikerd/datad/feed/EMS actor lifecycle
  test_store_cli.py         storage command help and diagnostics UX
  test_storage_audit.py     read-only NativeDB audit and JSON CLI
  test_backfill_audit_snippet.py
                            disposable xonsh qualification helpers
  test_watchlists.py        deterministic watchlist JSON operations

Change-To-Test Mapping

Changed area Run first Caveat
piker/watchlists/ tests/test_watchlists.py CLI suite is skipped
piker/storage/_audit.py, piker/storage/cli.py tests/test_storage_audit.py direct Typer app, no actor
piker/storage/cli.py command UX tests/test_store_cli.py fake SHM/runtime, no mutation
snippets/nativedb_backfill_audit.xsh tests/test_backfill_audit_snippet.py disposable paths only
piker/brokers/ib/api.py, feed.py history tests/test_ib_history.py fake client, no network
piker/brokers/ib/api.py method proxy tests/test_ib_method_proxy.py fake channel, no network
piker/tsp/_history.py tests/test_history_backfill.py fake provider/storage/SHM
piker/storage/cli.py SHM null-slot guard tests/test_ldshm.py synthetic timestamps, no SHM mutation
piker/config.py test_account_file_default_empty root-network test has a known mismatch
piker/accounting/ targeted accounting node some cases use live/configured state
piker/ui/_gaps.py, _annotate.py, _display.py, _interaction.py, _remote_ctl.py tests/test_gap_overlays.py offscreen PyQt6 plus one local Tractor actor
piker/ui/_style.py, piker/ui/qt.py tests/test_dpi_font.py deterministic offscreen Qt
piker/service/_actor_runtime.py, _registry.py, _mngr.py test_runtime_boot then test_datad_spawn
piker/service/, piker/data/_daemon.py test_datad_spawn feed lifecycle cases are live
piker/data/feed.py, flows.py, _sharedmem.py, _sampling.py collect first feed execution needs live permission
piker/clearing/ test_ems_err_on_bad_broker multi-fill case is live/persisted
piker/brokers/binance/, kraken/ selected feed/accounting node live network
piker/brokers/ib/ duplicated-market-ID node controlled account config required
Docker/service adapters tests/test_docker_services.py optional deps and containers
project, lock, or Nix files import check and safe collection full collection is not safe by default

Prefer deterministic filesystem/config and offscreen Qt tests, then local actor-runtime nodes, then explicitly approved live broker, visual UI, or container coverage.

Quick Checks

.venv/bin/python -c 'import importlib.util, pathlib, sys; root = pathlib.Path.cwd().resolve(); spec = importlib.util.find_spec("piker"); mod = pathlib.Path(spec.origin).resolve(); print(sys.executable); print(mod); assert mod.is_relative_to(root)'
.venv/bin/python -c 'from importlib.metadata import version; import pathlib, piker, pytestqt, sys; root = pathlib.Path.cwd().resolve(); mod = pathlib.Path(piker.__file__).resolve(); qt_ver = version("pytest-qt"); print(sys.executable); print(mod); print(qt_ver); assert mod.is_relative_to(root); assert qt_ver == "4.5.0"'
.venv/bin/python -m pytest -q --collect-only tests/test_gap_overlays.py
.venv/bin/python -m pytest -q -x --tb=short --no-header tests/test_gap_overlays.py
.venv/bin/python -m pytest -q tests/test_watchlists.py
.venv/bin/python -m pytest -q tests/test_accounting.py::test_account_file_default_empty
timeout -k 5 300 .venv/bin/python -m pytest -q tests/test_services.py::test_runtime_boot
timeout -k 5 300 .venv/bin/python -m pytest -q tests/test_services.py::test_datad_spawn
timeout -k 5 300 .venv/bin/python -m pytest -q tests/test_ems.py::test_ems_err_on_bad_broker

Known Outcomes

  • The verified .venv import check resolves piker beneath this worktree and imports the locked pytest-qt 4.5.0.
  • tests/test_accounting.py::test_root_conf_networking_section currently expects network.tsdb, which is absent from the tracked config template. Match the current KeyError: 'tsdb' before classifying it as the known repository mismatch.
  • tests/test_docker_services.py is marked skipped but imports elasticsearch first. Without the dbs group it fails collection rather than skipping.
  • tests/test_questrade.py is marked skipped but imports undeclared asks through the legacy broker module before marks apply.
  • test_open_orders_reloaded and test_dark_order_clearing in tests/test_ems.py contain only ellipsis bodies. A pass does not verify the named behavior.
  • Treat .pytest_cache feed IDs using old FQME forms as stale historical cache, not current known failures.

Do not classify every TooSlowError, timeout, or child survivor as the known Tractor teardown wedge. Match the selected node and the documented second-runtime/lingering-child signature.

Tractor Runtime Notes

The suite loads tractor._testing.pytest. Defaults are the trio spawn backend and TCP transport. Registry addresses are normally session-unique, except tests/test_services.py::test_runtime_boot, which binds 127.0.0.1:6666. Check that fixed port only for that node; ordinary tests do not require Piker’s production 127.0.0.1:6116 registry address.

Current repository concurrency notes document an intermittent second in-process pikerd boot wedge with a lingering broker child and unread parent IPC bytes. Use the outer timeout for actor-heavy nodes. Retry an exact command once only after status 124 or 143, and report both attempts.

Ordinary actor tests support normal capture. Diagnose a capture-dependent hang by retrying only the exact node with -s. Standard --pdb with open_test_pikerd requires -s; Tractor’s --tpdb is a separate option.

After abnormal exit, inspect only descendants, sockets, and shared-memory objects attributable to the exact pytest session. Ask before signaling or unlinking anything.