.claude/settings.local.json

@@ -3,8 +3,7 @@
     "allow": [
       "Bash(chmod:*)",
       "Bash(/tmp/piker_commits.txt)",
-      "Bash(python:*)",
+      "Bash(python:*)"
-      "Bash(ls:*)"
     ],
     "deny": [],
     "ask": []

.claude/skills/commit-msg/SKILL.md

@@ -0,0 +1,84 @@
+---
+name: commit-msg
+description: >
+  Generate piker-style git commit messages from
+  staged changes or prompt input, following the
+  style guide learned from 500 repo commits.
+argument-hint: "[optional-scope-or-description]"
+disable-model-invocation: true
+allowed-tools: Bash(git *), Read, Grep, Glob, Write
+---
+
+## Current staged changes
+!`git diff --staged --stat`
+
+## Recent commit style reference
+!`git log --oneline -10`
+
+# Piker Git Commit Message Generator
+
+Generate a commit message from the staged diff above
+following the piker project's conventions (learned from
+analyzing 500 repo commits).
+
+If `$ARGUMENTS` is provided, use it as scope or
+description context for the commit message.
+
+For the full style guide with verb frequencies,
+section markers, abbreviations, piker-specific terms,
+and examples, see
+[style-guide-reference.md](./style-guide-reference.md).
+
+## Quick Reference
+
+- **Subject**: ~50 chars, present tense verb, use
+  backticks for code refs
+- **Body**: only for complex/multi-file changes,
+  67 char line max
+- **Section markers**: Also, / Deats, / Other,
+- **Bullets**: use `-` style
+- **Tone**: technical but casual (piker style)
+
+## Claude-code Footer
+
+When the written **patch** was assisted by
+claude-code, include:
+
+```
+(this patch was generated in some part by [`claude-code`][claude-code-gh])
+[claude-code-gh]: https://github.com/anthropics/claude-code
+```
+
+When only the **commit msg** was written by
+claude-code (human wrote the patch), use:
+```
+(this commit msg was generated in some part by [`claude-code`][claude-code-gh])
+[claude-code-gh]: https://github.com/anthropics/claude-code
+```
+
+## Output Instructions
+
+When generating a commit message:
+
+1. Analyze the staged diff (injected above via
+   dynamic context) to understand all changes.
+2. If `$ARGUMENTS` provides a scope (e.g.,
+   `.ib.feed`) or description, incorporate it into
+   the subject line.
+3. Write the subject line following verb + backtick
+   conventions from the
+   [style guide](./style-guide-reference.md).
+4. Add body only for multi-file or complex changes.
+5. Write the message to a file in the repo's
+   `.claude/` subdir with filename format:
+   `<timestamp>_<first-7-chars-of-last-commit-hash>_commit_msg.md`
+   where `<timestamp>` is from `date --iso-8601=seconds`.
+   Also write a copy to
+   `.claude/git_commit_msg_LATEST.md`
+   (overwrite if exists).
+
+---
+
+**Analysis date:** 2026-01-27
+**Commits analyzed:** 500 from piker repository
+**Maintained by:** Tyler Goodlet

.claude/skills/commit-msg/style-guide-reference.md

@@ -150,18 +150,21 @@ Common in piker commits (33.0% use colons):
 - File:line references not used (0 occurrences)
 - No WIP commits in analyzed set
 
-### Coding-harness Footer
+### Claude-code Footer
-When a coding harness assisted with the written patch,
+When the written **patch** was assisted by claude-code,
-identify the active harness, model, and provider:
+include:
 
 ```
-(this patch was generated in some part by `<harness>` using `<model>` (`<provider>`))
+(this patch was generated in some part by [`claude-code`][claude-code-gh])
+[claude-code-gh]: https://github.com/anthropics/claude-code
 ```
 
-When it generated only the commit message, use:
+When only the **commit msg** was written by claude-code
+(human wrote the patch), use:
 
 ```
-(this commit msg was generated in some part by `<harness>` using `<model>` (`<provider>`))
+(this commit msg was generated in some part by [`claude-code`][claude-code-gh])
+[claude-code-gh]: https://github.com/anthropics/claude-code
 ```
 
 ## Piker-Specific Terms

.claude/skills/piker-conc-expert/SKILL.md

@@ -1,155 +0,0 @@
----
-name: piker-conc-expert
-description: >
-  Distributed-runtime and structured-concurrency
-  expertise for piker's `tractor` actor-tree. Apply
-  when working on daemon/service architecture, actor
-  spawning/discovery, cross-actor RPC (ctx/stream
-  eps), `to_asyncio` integration, cancellation
-  semantics, or debugging hangs/wedges/skews in the
-  actor system.
-compatibility: >
-  Requires a piker checkout and familiarity with Python,
-  trio, and tractor.
-metadata:
-  author: goodboy
-  version: "1.0"
----
-
-# Piker Concurrency & Runtime Expertise
-
-The distilled mental model for piker's distributed
-runtime: a `trio`-structured actor tree supervised by
-`tractor` (pinned to git main) where every long-lived
-subsystem is a named daemon-actor talking over
-ctx/stream IPC.
-
-## Actor tree & daemon taxonomy
-
-```
-pikerd                  root supervisor + registry
-├── datad.<broker>      feed bus, shm writers, tsp
-│                       history, symbol search
-├── brokerd.<broker>    live order-ctl ONLY; lazily
-│                       spawned by emsd, credentialed
-├── emsd                dark-clearing + order routing
-│   └── paperboi.<broker>  sim-clearing (paper mode)
-└── samplerd            singleton OHLC clock/increment
-```
-
-Key invariants:
-- `datad` hosts all `piker.data.validate._eps['datad']`
-  eps; `brokerd` only the `['brokerd']` (order-ctl)
-  ones. The `_eps` table in `piker/data/validate.py`
-  is the authoritative contract; `get_eps(mod, kind)`
-  introspects a backend's support.
-- `brokerd.<broker>` is booted in EXACTLY one place:
-  `open_brokerd_dialog()` in `piker/clearing/_ems.py`
-  (with a `portal:` override for the `piker ledger`
-  ad-hoc actor). Chart-only + paper sessions run with
-  ZERO brokerd procs. Never add a data-path spawn!
-- backends declare per-daemon-kind submods via
-  `_datad_mods`/`_brokerd_mods` in their
-  `__init__.py` (fallback: `__enable_modules__`).
-
-## Daemon lifecycle conventions
-
-Every daemon-kind follows the same trio of fns (see
-`piker/brokers/_daemon.py` + `piker/data/_daemon.py`
-as the canonical pair):
-
-- `_setup_persistent_<kind>()`: a `@tractor.context`
-  "lifetime fixture" run via
-  `Services.start_service_task()`; does console-log
-  setup ONCE for the actor, allocs any actor-global
-  state (eg. datad's `_FeedsBus`), then
-  `await ctx.started()` + `trio.sleep_forever()`.
-- `<kind>_init()`: builds `enable_modules` + actor
-  name `f'<kind>.{brokername}'` and copies backend
-  `_spawn_kwargs` (CRITICAL: `ib` needs
-  `infect_asyncio=True` in EVERY daemon-kind).
-- `spawn_<kind>()` + `maybe_spawn_<kind>()`: thin
-  wrappers over `Services.actor_n.start_actor()` and
-  `piker.service.maybe_spawn_daemon()` (registry
-  find-or-spawn w/ per-service-name locking).
-
-Caps-sec model: `enable_modules` gates RPC entry ONLY
-— python imports are unrestricted in-proc. Keep each
-daemon's enable set minimal; the (credentialed)
-`brokerd` must never RPC-enable `piker.data.*` feed
-mods.
-
-## Actor-local state: the #1 split hazard
-
-Module-globals and instance caches are PER-ACTOR.
-Anything that "just worked" because two subsystems
-shared a process will break when they're split into
-sibling actors. Canonical example: `ib`'s
-`Client._contracts` was warmed by feed-side
-`get_mkt_info()` in-proc; post datad/brokerd-split
-the trading actor must warm it itself (eagerly at
-`open_trade_dialog()` startup for open pps/orders +
-lazily per order request via
-`symbols.cache_contract()`).
-
-When moving code across actor boundaries ALWAYS audit:
-- module-global registries (`feed._bus`,
-  `_accounts2clients`, `_client_cache`, ..)
-- `@async_lifo_cache`/`maybe_open_context` caches
-  (NOTE: `async_lifo_cache` keys on POSITIONAL args
-  only; a cache-hit SKIPS the fn body and thus any
-  side-effect writes!)
-- logging handler placement (see gotchas.md)
-
-## tractor primitives as used here
-
-- `@tractor.context` eps: `await ctx.started(val)`
-  unblocks the caller w/ `val`; long-lived eps then
-  `ctx.open_stream()` or `sleep_forever()`.
-- discovery: `tractor.find_actor()` via
-  `piker.service.find_service()`;
-  `wait_for_actor(name, registry_addr=...)`;
-  `query_actor(name, regaddr=...)` yields
-  `(sockaddr, portal)`. Addrs are wrapped
-  `tractor.discovery._addr.Address` types — use
-  `wrap_address()` to normalize raw tuples and
-  `.unwrap()` for comparisons.
-- runtime-vars: `_runtime_vars['piker_vars']` is
-  inherited down the spawn tree; used eg. for
-  `piker_test_dir` config isolation — read LAZILY at
-  use-time, never at import time (subactors only get
-  vars post runtime-boot).
-- cancellation semantics (modern tractor): a
-  `ContextCancelled` whose `.canceller` is your own
-  actor is ABSORBED (clean exit, nothing raised);
-  single-exc groups collapse (`collapse_eg`) so eg.
-  a KBI propagates bare. Exc attrs:
-  `RemoteActorError.boxed_type` (not `.type`).
-
-## `to_asyncio` (infect-asyncio) integration
-
-For `ib` (and `deribit`) the backend client runs on
-an embedded `asyncio` loop via
-`tractor.to_asyncio.open_channel_from()` +
-`LinkedTaskChannel`.
-
-Rules learned the hard way:
-- a shared req/resp channel MUST correlate responses
-  to requests (see `MethodProxy._run_method()`'s
-  `mid` protocol in `piker/brokers/ib/api.py`):
-  caller cancellation (eg. `move_on_after` timeouts)
-  otherwise orphans a response and silently skews
-  every later result off-by-one.
-- the aio-side relay must catch + ship back ALL
-  (non-cancel) exceptions as `{'exception': err}`
-  resps; an escaping error kills the relay task ->
-  channel -> proxy nursery -> the whole dialog,
-  bypassing every caller-side guard.
-- `TrioTaskExited` ("child asyncio task is still
-  running?") on teardown is a known wart family;
-  prefer upstream `tractor` fixes over piker-side
-  bandaids.
-
-See [gotchas.md](gotchas.md) for the symptom->cause
-registry and [debug-recipes.md](debug-recipes.md) for
-forensics techniques.

.claude/skills/piker-conc-expert/debug-recipes.md

@@ -1,100 +0,0 @@
-# Debug recipes: actor-system forensics
-
-Field-tested techniques for diagnosing hangs, wedges
-and cross-actor state bugs WITHOUT a debugger attached
-(or when `py-spy` ain't installed).
-
-## Wedged actor triage (no REPL)
-
-1. find the tree:
-   `ps -eo pid,etime,args | grep -E 'pytest|tractor._child'`
-   — long-`etime` `tractor._child` procs w/ a stuck
-   parent = wedge.
-2. kernel state:
-   `cat /proc/<pid>/wchan` + `status | grep -E
-   'State|Threads'` — `do_epoll_wait` + sleeping =
-   idle event loop, NOT cpu-spin.
-3. **the money read** — socket queues:
-   `ss -tnp | grep <pid>`
-   - `Recv-Q > 0` on the parent-IPC conn = the actor
-     STOPPED CONSUMING its msg loop (runtime bug),
-     parent is waiting on it.
-   - zero external (api/ws) conns = wedged before/
-     without provider IO; don't blame the network.
-   - `CLOSE-WAIT` lingerers = unclean peer teardown.
-4. cleanup: `pkill -f tractor._child` (NB: in
-   compound shell cmds `pkill`'s exit code poisons
-   `&&` chains — run it standalone).
-
-## Hang-proof test gating
-
-- per-suite, never combined (cross-suite session
-  state interacts w/ the 2nd-boot wedge):
-  `timeout -k 5 300 python -m pytest tests/<one>.py -q`
-- rc 124/143 = hang-kill -> retry ONCE before
-  investigating.
-- isolate a flaky test w/ a 3x loop; ~50% hit-rate
-  signatures match the known 2nd-boot wedge (see
-  gotchas.md).
-
-## Regression vs pre-existing attribution
-
-When a failure appears mid-refactor:
-1. `git stash -u` (or checkout the file subset) and
-   re-run the EXACT failing case at baseline.
-2. if baseline can't even run, selectively revert
-   ONLY the suspect layer:
-   `git diff <files> > /tmp/x.patch;
-    git checkout <files>` -> test ->
-   `git apply /tmp/x.patch`.
-3. flake-rate compare (3x runs) beats single-shot
-   conclusions.
-
-## Off-by-one / stale IPC resp detection
-
-Mismatched query->result content in logs (resp
-payload obviously for a prior request) = shared
-req/resp channel w/o correlation + a cancelled
-caller. Grep the ep for `move_on_after`/`fail_after`
-around proxied calls. Fix = req-id (`mid`) tagging,
-never "just a lock" (cancellation still orphans).
-
-## Logging-chain audits
-
-When records double-print or go bare (see gotchas.md):
-
-```python
-import logging
-l = logging.getLogger('piker.brokers.ib.broker')
-while l:
-    print(l.name, l.level, l.handlers, l.propagate)
-    l = l.parent
-```
-
-Exactly ONE stderr handler should exist in the chain,
-attached by the actor's daemon fixture.
-
-## Live actor-tree smoke (headless)
-
-Boot against an ALT registry port so a user's running
-stack is untouched; script in a REAL file (tractor
-children re-exec `__main__` from path — stdin scripts
-crash w/ `FileNotFoundError: .../<stdin>`):
-
-```python
-async with maybe_open_pikerd(
-    registry_addrs=[('127.0.0.1', 6979)],
-):
-    async with open_feed(['xbtusdt.kraken']) as feed:
-        assert await check_for_service('datad.kraken')
-        assert not await check_for_service(
-            'brokerd.kraken'
-        )
-```
-
-## In-proc fail-fast unit checks
-
-Spawn-path guards that raise BEFORE touching the
-runtime can be tested w/ a bare `trio.run()` (eg
-`spawn_brokerd('kucoin')` raising the datad-only
-error) — no pikerd needed.

.claude/skills/piker-conc-expert/gotchas.md

@@ -1,116 +0,0 @@
-# Known gotchas: symptom -> cause -> fix
-
-A registry of distributed-runtime failure modes hit
-(and diagnosed) in the field; check here FIRST when a
-log/traceback matches.
-
-## "Can not order ..., no qualified contract cached"
-
-- **Symptom**: `RuntimeError` from
-  `ib.api.Client.submit_limit()` w/ empty
-  `Client._contracts` in `brokerd.ib`.
-- **Cause**: per-actor cache never warmed; feed-side
-  qualification now lives in `datad.ib`.
-- **Fix(ed)**: eager warmup at `open_trade_dialog()`
-  start + lazy per-order `get_mkt_info()` +
-  `cache_contract()` (writes BOTH `mkt.bs_fqme` and
-  `mkt.fqme` keys; different consumers read each!).
-
-## Search returns results for the WRONG pattern
-
-- **Symptom**: fqme search for 'gld' returns nvda
-  results; next query returns the prior query's set.
-- **Cause**: `MethodProxy` channel off-by-one — a
-  caller cancelled (search `move_on_after` timeout)
-  after sending its request orphans the response;
-  every later caller consumes the previous resp.
-- **Fix(ed)**: `mid` req-id correlation in
-  `_run_method()` + relay; stale resps are dropped w/
-  a "Dropping stale method-resp" warning. If that
-  warning spams, some caller is being cancelled
-  mid-call habitually — find + fix its timeout.
-
-## One bad request crashes a whole dialog/actor
-
-- **Symptom**: `TrioTaskExited` storm + nursery
-  teardown after a single method error (eg ambiguous
-  contract `AttributeError`).
-- **Cause**: exception escaped the aio-side relay
-  loop (`open_aio_client_method_relay()`) killing
-  channel + proxy nursery; caller-side `try/except`
-  CANNOT catch it.
-- **Fix(ed)**: relay catches `Exception` -> ships
-  `{'exception': err, 'mid': ...}` resp; order
-  handler converts to EMS `BrokerdError` msgs.
-
-## Ambiguous ib contracts -> `NoneType` attr errors
-
-- **Symptom**: `'NoneType' object has no attribute
-  'primaryExchange'` in `find_contracts()`.
-- **Cause**: `qualifyContractsAsync()` returns `None`
-  entries for ambiguous (eg venue-less stonk fqme
-  matching multiple listings: 'gld' -> ARCA/USD +
-  VENTURE/CAD).
-- **Fix(ed)**: filter `None`s + raise descriptive
-  `ValueError` ("use 'gld.arca.ib'").
-
-## Double-printed log records (same task id, 2x)
-
-- **Symptom**: every record from some subsys printed
-  twice w/ identical task ids.
-- **Cause**: stderr handlers attached at TWO levels
-  of one logger-propagation chain (eg daemon fixture
-  on `piker.brokers.ib` + an ep calling
-  `get_console_log(name=__name__)` on the child).
-  tractor's handler-dedup only checks the SAME
-  logger, not ancestors.
-- **Rule**: console handlers are attached ONCE per
-  actor in the `_setup_persistent_*()` fixture; eps
-  needing a different level use `log.setLevel()`
-  ONLY, never `get_console_log()`.
-
-## Bare/non-colorized log lines
-
-- **Symptom**: records w/ no timestamp/actor prefix.
-- **Cause**: NO handler anywhere in the emitting
-  logger's chain -> stdlib `logging.lastResort`. Post
-  actor-splits, a daemon fixture may only cover its
-  own subsys subtree (eg datad's `piker.data.*` but
-  not the backend's `piker.brokers.<broker>.*`).
-- **Fix(ed)**: `_setup_persistent_datad()` enables
-  BOTH `piker.data.<broker>` and
-  `piker.brokers.<broker>` subtrees.
-
-## 2nd in-proc runtime boot wedges (~50%)
-
-- **Symptom**: test hangs when one test proc boots a
-  2nd `pikerd` (eg `test_multi_fill_positions`'s
-  persistence re-check); a zombie `*.{broker}` child
-  lingers w/ unread bytes in its parent-IPC Recv-Q.
-- **Cause**: pre-existing `tractor`-main runtime
-  teardown bug (confirmed independent of piker-layer
-  changes via revert-testing 2026-06).
-- **Mitigation**: run suites per-file wrapped in
-  `timeout -k 5 300 ...`; retry once on rc 124/143.
-  Do NOT chase as a regression of unrelated changes.
-
-## ib client-id collisions post-split
-
-- **Symptom**: 2nd ib daemon burns the full
-  conn-timeout retry cycle connecting to gw/tws.
-- **Cause**: `datad.ib` + `brokerd.ib` both default
-  `client_id=6116` w/ linear `+i` retries.
-- **Fix(ed)**: role-based offsets in
-  `load_aio_clients()`: datad +16, ad-hoc (test/cli)
-  actors +32.
-
-## `async_lifo_cache` skipped side-effects
-
-- **Symptom**: a fn's cache-write side effect
-  (eg `get_mkt_info()` -> `_contracts`) missing for
-  a 2nd client/proxy.
-- **Cause**: cache keys on POSITIONAL args only; a
-  hit skips the body entirely.
-- **Rule**: never rely on cached-fn side effects;
-  perform required writes explicitly at the call
-  site (eg `cache_contract()` after `get_mkt_info`).

.claude/skills/piker-profiling/SKILL.md

@@ -5,11 +5,7 @@ description: >
   across distributed actor systems. Apply when
   adding profiling, debugging perf regressions, or
   optimizing hot paths in piker code.
-compatibility: >
+user-invocable: false
-  Requires a piker checkout and Python profiling work.
-metadata:
-  author: goodboy
-  version: "1.0"
 ---
 
 # Piker Profiling Subsystem

.claude/skills/piker-slang/SKILL.md

@@ -5,11 +5,7 @@ description: >
   ethos. Apply when communicating with piker devs,
   writing commit messages, code review comments, or
   any collaborative interaction.
-compatibility: >
+user-invocable: false
-  Harness-independent communication guidance.
-metadata:
-  author: goodboy
-  version: "1.0"
 ---
 
 # Piker Slang & Communication Style

.claude/skills/piker-slang/examples.md

@@ -47,7 +47,7 @@ that's why it's like 1000x faster ya know?"
 - Wrap all code symbols: `function()`,
   `ClassName`, `field_name`
 - File paths: `piker/ui/_remote_ctl.py`
-- Commands: `git status`, `piker store shm`
+- Commands: `git status`, `piker store ldshm`
 
 **Explain like you're pair programming:**
 ```

.claude/skills/pyqtgraph-optimization/SKILL.md

@@ -5,12 +5,7 @@ description: >
   for piker's UI. Apply when optimizing graphics
   performance, adding new chart annotations, or
   working with `QGraphicsItem` subclasses.
-compatibility: >
+user-invocable: false
-  Requires a piker checkout with PyQtGraph and Qt source
-  available for inspection.
-metadata:
-  author: goodboy
-  version: "1.0"
 ---
 
 # PyQtGraph Rendering Optimization

.claude/skills/run-tests/test-harness-reference.md

@@ -1,291 +0,0 @@
-# Test Harness Reference: piker
-
-This repository-local file supplements the canonical `/run-tests` skill.
-Keep shared execution, worktree, failure-inspection, and cleanup policy in the
-canonical `SKILL.md`; keep Piker commands, paths, fixtures, and known outcomes
-here.
-
-## Project And Environment
-
-- Project/import: `piker`
-- Test root: `tests/`
-- Supported Python: `>=3.12,<3.14`
-- Preferred complete environment: worktree-local `py313` inside the current
-  `nix develop` shell
-- The flake shell pins CPython 3.13 and sets
-  `UV_PROJECT_ENVIRONMENT=py313`.
-- Verify the interpreter, package resolution, and dependency import before
-  running tests. A bare `py313` may lack the Qt binding supplied by Nix.
-
-Use an already-provisioned `py313` only when `import piker` succeeds:
-
-```text
-py313/bin/python -m pytest -p no:xonsh
-```
-
-If direct environment paths are unavailable, an existing uv environment can
-be used without changing it, subject to the same import check:
-
-```text
-UV_PROJECT_ENVIRONMENT=py313 uv run --frozen --no-sync python -m pytest -p no:xonsh
-```
-
-Ask before running provisioning commands such as:
-
-```text
-UV_PROJECT_ENVIRONMENT=py313 uv sync --dev --all-extras --no-group lint
-nix develop
-nix-shell default.nix
-```
-
-`nix develop` is the current Wayland/Qt 6 shell. `default.nix` is the current
-X11 shell. Do not use `develop.nix` for current testing; it retains the old
-Python 3.11, Poetry, and Qt 5 stack.
-
-The current root-checkout `py313` resolves `piker` locally but fails
-`import piker` outside Nix because PyQtGraph cannot import PyQt or PySide. Do
-not treat that environment as test-ready and do not enter `nix develop`
-without approval: its shell hook may recreate and sync `py313`.
-
-Plain `uv sync` does not include the testing group. The `dbs` dependency group
-is also absent from normal dev-shell provisioning.
-
-## Commands
-
-Base command in the preferred environment:
-
-```text
-py313/bin/python -m pytest -p no:xonsh
-```
-
-The explicit `-p no:xonsh` is required. The tracked comments-only
-`pytest.ini` takes precedence over `pyproject.toml`, so the intended
-`addopts = "-p no:xonsh"` and `testpaths = ["tests"]` are inactive.
-Always pass a test path or node ID explicitly.
-
-Package-resolution check that does not import Piker's dependencies:
-
-```text
-py313/bin/python -c 'import importlib.util, pathlib, sys; root = pathlib.Path.cwd().resolve(); spec = importlib.util.find_spec("piker"); mod = pathlib.Path(spec.origin).resolve(); print(sys.executable); print(mod); assert mod.is_relative_to(root)'
-```
-
-Dependency import check, required before collection or execution:
-
-```text
-py313/bin/python -c 'import pathlib, piker, sys; root = pathlib.Path.cwd().resolve(); mod = pathlib.Path(piker.__file__).resolve(); print(sys.executable); print(mod); assert mod.is_relative_to(root)'
-```
-
-Safe core collection check:
-
-```text
-py313/bin/python -m pytest -p no:xonsh -q --collect-only tests/test_watchlists.py tests/test_accounting.py tests/test_services.py tests/test_ems.py tests/test_feeds.py tests/test_cli.py
-```
-
-Default first-pass flags are `-q -x --tb=short --no-header` unless the user
-requests otherwise. For actor-heavy tests, use one file or node per process
-with an outer timeout:
-
-```text
-timeout -k 5 300 py313/bin/python -m pytest -p no:xonsh -q <one-file-or-node>
-```
-
-If an actor-heavy command exits `124` or `143`, retry that exact command once
-and report both attempts. Never convert a retry pass into an unconditional
-pass, and do not retry assertion, import, collection, or configuration
-failures.
-
-## Scope And Path Resolution
-
-Resolve bare test filenames beneath `tests/`. Preserve complete node IDs and
-apply `-k` only within explicitly selected paths. There is no marker-based
-offline/full-suite split, so never use `pytest tests` as a deterministic
-default.
-
-Deterministic or local first-pass targets:
-
-- `tests/test_watchlists.py`
-- `tests/test_storage_audit.py`
-- `tests/test_store_cli.py`
-- `tests/test_backfill_audit_snippet.py`
-- `tests/test_ib_history.py`
-- `tests/test_ib_method_proxy.py`
-- `tests/test_history_backfill.py`
-- `tests/test_ldshm.py`
-- `tests/test_accounting.py::test_account_file_default_empty`
-- `tests/test_services.py::test_runtime_boot`
-- `tests/test_services.py::test_datad_spawn`
-- `tests/test_ems.py::test_ems_err_on_bad_broker`
-
-Require explicit authorization before running:
-
-- `tests/test_feeds.py` - live Binance/Kraken feeds;
-- `tests/test_services.py::test_ensure_datafeed_actors` - live Kraken feed;
-- `tests/test_services.py::test_ensure_ems_in_paper_actors` - paper EMS with
-  live Kraken symbology/feed access;
-- `tests/test_ems.py::test_multi_fill_positions` - live backend setup and
-  persisted state;
-- `tests/test_accounting.py::test_paper_ledger_position_calcs` - tracked
-  fixtures plus possible live symcache generation;
-- `tests/test_accounting.py::test_ib_account_with_duplicated_mktids` - active
-  broker/account configuration and state writes;
-- `tests/test_dpi_font.py` - Qt/UI import and user-config side effects;
-- `tests/test_docker_services.py` - optional dependencies and containers;
-- `tests/test_questrade.py` - obsolete credentialed imports.
-
-`tests/test_cli.py` is currently hard-skipped. It is not an active CLI
-regression gate.
-
-Never execute `piker store anal` or `piker store shm --write-parquet`
-as tests. They are mutating or interactive operational commands.
-
-## Project-Specific Flags And Backend Matrix
-
-| Flag | Purpose |
-|---|---|
-| `--ll LEVEL` | Piker log level |
-| `--confdir PATH` | Override `piker.config._config_dir` |
-| `--spawn-backend trio|mp_spawn|mp_forkserver` | Tractor process backend |
-| `--tpt-proto PROTO` | Tractor transport; one protocol per session |
-| `--tpdb` / `--debug-mode` | Tractor plugin debug mode |
-| `--pdb` | Standard pytest debugger |
-| `-s` | No capture; required with `--pdb` and `open_test_pikerd` |
-
-Do not invent `network`, `offline`, `docker`, `gui`, or `broker` markers; none
-currently exists. `CI=1` is not an offline selector: feed tests still leave a
-live Kraken case enabled. Current Piker tests exercise TCP; do not assume the
-whole suite supports UDS merely because the Tractor plugin exposes it.
-
-## Fixture Invariants
-
-- The session `confdir` fixture does nothing unless `--confdir` is passed.
-  Its claimed `tests/data` fallback is not implemented and that directory is
-  absent.
-- Function-scoped `tmpconfdir` changes process-global config state and does
-  not restore the previous path. Use separate pytest processes when
-  diagnosing state leakage.
-- `open_test_pikerd` passes the temporary config path to child actors through
-  `tractor_runtime_overrides`.
-- `tests/_inputs/trades_binance_paper.toml` and
-  `tests/_inputs/account.binance.paper.toml` are used in place. Accounting
-  contexts can write them on exit. Inspect `git diff -- tests/_inputs` after
-  any selected accounting case.
-- Importing `tests/test_dpi_font.py` constructs module-global font objects
-  before fixtures can isolate config. If explicitly requested, isolate
-  `XDG_CONFIG_HOME` before Python starts and use the proper Qt/Nix shell.
-- Piker and the installed Tractor pytest plugin do not provide a
-  repository-local process or socket reaper. Never apply historical broad
-  `pkill -f tractor._child` guidance automatically.
-- The function-scoped autouse `shm_leak_tracker` fixture wraps Tractor's
-  current-process `SharedMemory` factory. It tracks only successful
-  `create=True` calls, restores the pre-test `_known_tokens` cache, and
-  unlinks exact surviving names before failing the leaking test. It never
-  scans `/dev/shm` or unlinks attachments created by another process.
-
-## Test Layout
-
-```text
-tests/
-  conftest.py               Piker options, config fixtures, Tractor plugin
-  _inputs/                  tracked ledger/account fixtures
-  test_accounting.py        config, ledgers, accounts, and position math
-  test_cli.py               legacy CLI suite; hard-skipped
-  test_docker_services.py   container integrations; optional deps, skipped
-  test_dpi_font.py          Qt DPI/font behavior
-  test_ems.py               actor, EMS, and paper-position behavior
-  test_feeds.py             live Binance/Kraken feeds and shared memory
-  test_ib_history.py        deterministic IB history request formatting
-  test_ib_method_proxy.py   deterministic IB asyncio proxy routing
-  test_history_backfill.py  deterministic history/SHM orchestration
-  test_ldshm.py             SHM unpublished-slot guard
-  test_questrade.py         obsolete credentialed tests; skipped
-  test_services.py          pikerd/datad/feed/EMS actor lifecycle
-  test_store_cli.py         storage command help and diagnostics UX
-  test_storage_audit.py     read-only NativeDB audit and JSON CLI
-  test_backfill_audit_snippet.py
-                            disposable xonsh qualification helpers
-  test_watchlists.py        deterministic watchlist JSON operations
-```
-
-## Change-To-Test Mapping
-
-| Changed area | Run first | Caveat |
-|---|---|---|
-| `piker/watchlists/` | `tests/test_watchlists.py` | CLI suite is skipped |
-| `piker/storage/_audit.py`, `piker/storage/cli.py` | `tests/test_storage_audit.py` | direct Typer app, no actor |
-| `piker/storage/cli.py` command UX | `tests/test_store_cli.py` | fake SHM/runtime, no mutation |
-| `snippets/nativedb_backfill_audit.xsh` | `tests/test_backfill_audit_snippet.py` | disposable paths only |
-| `piker/brokers/ib/api.py`, `feed.py` history | `tests/test_ib_history.py` | fake client, no network |
-| `piker/brokers/ib/api.py` method proxy | `tests/test_ib_method_proxy.py` | fake channel, no network |
-| `piker/tsp/_history.py` | `tests/test_history_backfill.py` | fake provider/storage/SHM |
-| `piker/storage/cli.py` SHM null-slot guard | `tests/test_ldshm.py` | synthetic timestamps, no SHM mutation |
-| `piker/config.py` | `test_account_file_default_empty` | root-network test has a known mismatch |
-| `piker/accounting/` | targeted accounting node | some cases use live/configured state |
-| `piker/ui/_style.py`, `piker/ui/qt.py` | `tests/test_dpi_font.py` | GUI/config-isolated opt-in |
-| `piker/service/_actor_runtime.py`, `_registry.py`, `_mngr.py` | `test_runtime_boot` | then `test_datad_spawn` |
-| `piker/service/`, `piker/data/_daemon.py` | `test_datad_spawn` | feed lifecycle cases are live |
-| `piker/data/feed.py`, `flows.py`, `_sharedmem.py`, `_sampling.py` | collect first | feed execution needs live permission |
-| `piker/clearing/` | `test_ems_err_on_bad_broker` | multi-fill case is live/persisted |
-| `piker/brokers/binance/`, `kraken/` | selected feed/accounting node | live network |
-| `piker/brokers/ib/` | duplicated-market-ID node | controlled account config required |
-| Docker/service adapters | `tests/test_docker_services.py` | optional deps and containers |
-| project, lock, or Nix files | import check and safe collection | full collection is not safe by default |
-
-Prefer deterministic filesystem/config tests, then local actor-runtime nodes,
-then explicitly approved live broker, GUI, or container coverage.
-
-## Quick Checks
-
-```text
-py313/bin/python -c 'import importlib.util, pathlib, sys; root = pathlib.Path.cwd().resolve(); spec = importlib.util.find_spec("piker"); mod = pathlib.Path(spec.origin).resolve(); print(sys.executable); print(mod); assert mod.is_relative_to(root)'
-py313/bin/python -c 'import pathlib, piker, sys; root = pathlib.Path.cwd().resolve(); mod = pathlib.Path(piker.__file__).resolve(); print(sys.executable); print(mod); assert mod.is_relative_to(root)'
-py313/bin/python -m pytest -p no:xonsh -q tests/test_watchlists.py
-py313/bin/python -m pytest -p no:xonsh -q tests/test_accounting.py::test_account_file_default_empty
-timeout -k 5 300 py313/bin/python -m pytest -p no:xonsh -q tests/test_services.py::test_runtime_boot
-timeout -k 5 300 py313/bin/python -m pytest -p no:xonsh -q tests/test_services.py::test_datad_spawn
-timeout -k 5 300 py313/bin/python -m pytest -p no:xonsh -q tests/test_ems.py::test_ems_err_on_bad_broker
-```
-
-## Known Outcomes
-
-- The current root-checkout `py313` fails `import piker` outside the Nix shell
-  with `ImportError: PyQtGraph requires one of PyQt5, PyQt6, PySide2 or
-  PySide6`. This is an incomplete environment, not an application regression.
-- `tests/test_accounting.py::test_root_conf_networking_section` currently
-  expects `network.tsdb`, which is absent from the tracked config template.
-  Match the current `KeyError: 'tsdb'` before classifying it as the known
-  repository mismatch.
-- `tests/test_docker_services.py` is marked skipped but imports
-  `elasticsearch` first. Without the `dbs` group it fails collection rather
-  than skipping.
-- `tests/test_questrade.py` is marked skipped but imports undeclared `asks`
-  through the legacy broker module before marks apply.
-- `test_open_orders_reloaded` and `test_dark_order_clearing` in
-  `tests/test_ems.py` contain only ellipsis bodies. A pass does not verify the
-  named behavior.
-- Treat `.pytest_cache` feed IDs using old FQME forms as stale historical
-  cache, not current known failures.
-
-Do not classify every `TooSlowError`, timeout, or child survivor as the known
-Tractor teardown wedge. Match the selected node and the documented
-second-runtime/lingering-child signature.
-
-## Tractor Runtime Notes
-
-The suite loads `tractor._testing.pytest`. Defaults are the `trio` spawn
-backend and TCP transport. Registry addresses are normally session-unique,
-except `tests/test_services.py::test_runtime_boot`, which binds
-`127.0.0.1:6666`. Check that fixed port only for that node; ordinary tests do
-not require Piker's production `127.0.0.1:6116` registry address.
-
-Current repository concurrency notes document an intermittent second
-in-process `pikerd` boot wedge with a lingering broker child and unread parent
-IPC bytes. Use the outer timeout for actor-heavy nodes. Retry an exact command
-once only after status `124` or `143`, and report both attempts.
-
-Ordinary actor tests support normal capture. Diagnose a capture-dependent hang
-by retrying only the exact node with `-s`. Standard `--pdb` with
-`open_test_pikerd` requires `-s`; Tractor's `--tpdb` is a separate option.
-
-After abnormal exit, inspect only descendants, sockets, and shared-memory
-objects attributable to the exact pytest session. Ask before signaling or
-unlinking anything.

.claude/skills/timeseries-optimization/SKILL.md

@@ -6,12 +6,7 @@ description: >
   with OHLCV arrays, timestamp lookups, gap
   detection, or any array/dataframe operations in
   piker.
-compatibility: >
+user-invocable: false
-  Requires a piker checkout with NumPy and optionally
-  Polars.
-metadata:
-  author: goodboy
-  version: "1.0"
 ---
 
 # Timeseries Optimization: NumPy & Polars
@@ -66,14 +61,11 @@ ts_array = np.array(timestamps)
 # binary search for all timestamps at once
 indices = np.searchsorted(time_arr, ts_array)
 
-# bounds check before indexing: searchsorted may return
+# bounds check and exact match verification
-# len(time_arr) for values above the final timestamp
+valid_mask = (
-in_bounds = indices < len(time_arr)
+    (indices < len(array))
-valid_mask = np.zeros(indices.shape, dtype=bool)
+    &
-valid_mask[in_bounds] = (
+    (time_arr[indices] == ts_array)
-    time_arr[indices[in_bounds]]
-    ==
-    ts_array[in_bounds]
 )
 
 valid_indices = indices[valid_mask]

.gitignore

@@ -110,12 +110,6 @@ ENV/
 .claude/*_commit_*.md
 .claude/*_commit*.toml
 
-# ai.skillz/commit-msg
-.claude/skills/commit-msg/msgs/
-.claude/skills/commit-msg/conf.toml
-.claude/git_commit_msg_LATEST.md
-.opencode/skills/commit-msg/SKILL.md
-
 # nix develop --profile .nixdev
 .nixdev*
 
@@ -136,172 +130,3 @@ gitea/
 
 # LLM conversations that should remain private
 docs/conversations/
-
-# BEGIN ai.skillz: direct:symlink:claude:run-tests
-/.claude/skills/run-tests/SKILL.md
-# END ai.skillz: direct:symlink:claude:run-tests
-
-# BEGIN ai.skillz: direct:symlink:opencode:run-tests
-/.opencode/skills/run-tests/SKILL.md
-# END ai.skillz: direct:symlink:opencode:run-tests
-
-# BEGIN ai.skillz: direct:symlink:opencode:command:run-tests
-/.opencode/commands/run-tests.md
-# END ai.skillz: direct:symlink:opencode:command:run-tests
-
-# BEGIN ai.skillz: direct:symlink:claude:close-wkt
-/.claude/skills/close-wkt
-# END ai.skillz: direct:symlink:claude:close-wkt
-
-# BEGIN ai.skillz: direct:symlink:opencode:close-wkt
-/.opencode/skills/close-wkt
-# END ai.skillz: direct:symlink:opencode:close-wkt
-
-# BEGIN ai.skillz: runtime:code-review-changes
-.claude/review_context.md
-.claude/review_regression.md
-# END ai.skillz: runtime:code-review-changes
-
-# BEGIN ai.skillz: direct:symlink:claude:code-review-changes
-/.claude/skills/code-review-changes
-# END ai.skillz: direct:symlink:claude:code-review-changes
-
-# BEGIN ai.skillz: direct:symlink:opencode:code-review-changes
-/.opencode/skills/code-review-changes
-# END ai.skillz: direct:symlink:opencode:code-review-changes
-
-# BEGIN ai.skillz: runtime:commit-msg
-.claude/skills/commit-msg/msgs/
-.claude/skills/commit-msg/conf.toml
-.claude/git_commit_msg_LATEST.md
-# END ai.skillz: runtime:commit-msg
-
-# BEGIN ai.skillz: direct:symlink:claude:commit-msg
-/.claude/skills/commit-msg/SKILL.md
-# END ai.skillz: direct:symlink:claude:commit-msg
-
-# BEGIN ai.skillz: direct:symlink:opencode:commit-msg
-/.opencode/skills/commit-msg/SKILL.md
-# END ai.skillz: direct:symlink:opencode:commit-msg
-
-# BEGIN ai.skillz: direct:symlink:claude:dep-supersede-scan
-/.claude/skills/dep-supersede-scan
-# END ai.skillz: direct:symlink:claude:dep-supersede-scan
-
-# BEGIN ai.skillz: direct:symlink:opencode:dep-supersede-scan
-/.opencode/skills/dep-supersede-scan
-# END ai.skillz: direct:symlink:opencode:dep-supersede-scan
-
-# BEGIN ai.skillz: direct:symlink:claude:gish
-/.claude/skills/gish
-# END ai.skillz: direct:symlink:claude:gish
-
-# BEGIN ai.skillz: direct:symlink:opencode:gish
-/.opencode/skills/gish
-# END ai.skillz: direct:symlink:opencode:gish
-
-# BEGIN ai.skillz: direct:symlink:claude:inter-skill-review
-/.claude/skills/inter-skill-review
-# END ai.skillz: direct:symlink:claude:inter-skill-review
-
-# BEGIN ai.skillz: direct:symlink:opencode:inter-skill-review
-/.opencode/skills/inter-skill-review
-# END ai.skillz: direct:symlink:opencode:inter-skill-review
-
-# BEGIN ai.skillz: runtime:open-wkt
-.claude/wkts/
-claude_wkts
-# END ai.skillz: runtime:open-wkt
-
-# BEGIN ai.skillz: direct:symlink:claude:open-wkt
-/.claude/skills/open-wkt
-# END ai.skillz: direct:symlink:claude:open-wkt
-
-# BEGIN ai.skillz: direct:symlink:opencode:open-wkt
-/.opencode/skills/open-wkt
-# END ai.skillz: direct:symlink:opencode:open-wkt
-
-# BEGIN ai.skillz: direct:symlink:claude:plan-io
-/.claude/skills/plan-io
-# END ai.skillz: direct:symlink:claude:plan-io
-
-# BEGIN ai.skillz: direct:symlink:opencode:plan-io
-/.opencode/skills/plan-io
-# END ai.skillz: direct:symlink:opencode:plan-io
-
-# BEGIN ai.skillz: runtime:pr-msg
-.claude/skills/pr-msg/msgs/
-.claude/skills/pr-msg/pr_msg_LATEST.md
-# END ai.skillz: runtime:pr-msg
-
-# BEGIN ai.skillz: direct:symlink:claude:pr-msg
-/.claude/skills/pr-msg/SKILL.md
-/.claude/skills/pr-msg/references
-/.claude/skills/pr-msg/scripts
-# END ai.skillz: direct:symlink:claude:pr-msg
-
-# BEGIN ai.skillz: direct:symlink:opencode:pr-msg
-/.opencode/skills/pr-msg/SKILL.md
-/.opencode/skills/pr-msg/references
-/.opencode/skills/pr-msg/scripts
-# END ai.skillz: direct:symlink:opencode:pr-msg
-
-# BEGIN ai.skillz: direct:symlink:claude:prompt-io
-/.claude/skills/prompt-io
-# END ai.skillz: direct:symlink:claude:prompt-io
-
-# BEGIN ai.skillz: direct:symlink:opencode:prompt-io
-/.opencode/skills/prompt-io
-# END ai.skillz: direct:symlink:opencode:prompt-io
-
-# BEGIN ai.skillz: direct:symlink:claude:py-codestyle
-/.claude/skills/py-codestyle
-# END ai.skillz: direct:symlink:claude:py-codestyle
-
-# BEGIN ai.skillz: direct:symlink:opencode:py-codestyle
-/.opencode/skills/py-codestyle
-# END ai.skillz: direct:symlink:opencode:py-codestyle
-
-# BEGIN ai.skillz: direct:symlink:claude:resolve-conflicts
-/.claude/skills/resolve-conflicts
-# END ai.skillz: direct:symlink:claude:resolve-conflicts
-
-# BEGIN ai.skillz: direct:symlink:opencode:resolve-conflicts
-/.opencode/skills/resolve-conflicts
-# END ai.skillz: direct:symlink:opencode:resolve-conflicts
-
-# BEGIN ai.skillz: runtime:taken-export
-.ai/taken/exports/
-# END ai.skillz: runtime:taken-export
-
-# BEGIN ai.skillz: direct:symlink:claude:taken-export
-/.claude/skills/taken-export
-# END ai.skillz: direct:symlink:claude:taken-export
-
-# BEGIN ai.skillz: direct:symlink:opencode:taken-export
-/.opencode/skills/taken-export
-# END ai.skillz: direct:symlink:opencode:taken-export
-
-# BEGIN ai.skillz: direct:symlink:claude:yt-url-lookup
-/.claude/skills/yt-url-lookup
-# END ai.skillz: direct:symlink:claude:yt-url-lookup
-
-# BEGIN ai.skillz: direct:symlink:opencode:yt-url-lookup
-/.opencode/skills/yt-url-lookup
-# END ai.skillz: direct:symlink:opencode:yt-url-lookup
-
-# BEGIN ai.skillz: direct:symlink:claude:command:branch-in-new-terminal
-/.claude/commands/branch-in-new-terminal.md
-# END ai.skillz: direct:symlink:claude:command:branch-in-new-terminal
-
-# BEGIN ai.skillz: runtime:branch-in-new-terminal
-.claude/.current_session
-# END ai.skillz: runtime:branch-in-new-terminal
-
-# BEGIN ai.skillz: direct:symlink:opencode:command:commit-msg
-/.opencode/commands/commit-msg.md
-# END ai.skillz: direct:symlink:opencode:command:commit-msg
-
-# BEGIN ai.skillz: direct:symlink:opencode:command:taken-export
-/.opencode/commands/taken-export.md
-# END ai.skillz: direct:symlink:opencode:command:taken-export

ai/README.md

@@ -17,18 +17,6 @@ track new integration ideas and proposals in
 | Tool | Directory | Status |
 |------|-----------|--------|
 | [Claude Code](https://github.com/anthropics/claude-code) | [`claude-code/`](claude-code/) | active |
-| [OpenCode](https://opencode.ai/) | [`opencode/`](opencode/) | active |
-
-## Shared Skills
-
-Repo-specific skills use the portable Agent Skills
-frontmatter subset and live under `.claude/skills/` as a
-single source. Claude Code discovers that directory
-natively, and OpenCode discovers the same project skills
-without copies or generated wrappers.
-
-Harness-specific command files should only delegate to a
-shared skill. They must not duplicate the skill body.
 
 ## Adding a New Integration
 
@@ -44,7 +32,7 @@ ai/
 ├── README.md                  # <- you are here
 ├── claude-code/
 │   └── README.md
-├── opencode/
+├── opencode/                  # future
 │   └── README.md
 └── <your-tool>/
     └── README.md
@@ -57,7 +45,6 @@ ai/
 - Each integration doc should describe **what**
   the skill does, **how** to invoke it, and any
   **output** artifacts it produces
-- Keep docs concise; link to the shared skill source files
+- Keep docs concise; link to the actual skill
+  source files (under `.claude/skills/`, etc.)
   rather than duplicating content
-- Keep skill bodies harness-neutral; isolate slash-command
-  syntax and permissions in each harness integration

ai/claude-code/README.md

@@ -1,17 +1,15 @@
 # Claude Code Integration
 
 [Claude Code](https://github.com/anthropics/claude-code)
-integration for piker's shared coding-harness skills.
+skills and workflows for piker development.
 
 ## Skills
 
 | Skill | Invocable | Description |
 |-------|-----------|-------------|
 | [`commit-msg`](#commit-msg) | `/commit-msg` | Generate piker-style commit messages |
-| `piker-conc-expert` | auto | Actor-tree and structured-concurrency invariants |
 | `piker-profiling` | auto | `Profiler` API patterns for perf work |
 | `piker-slang` | auto | Communication style + slang guide |
-| `py-codestyle` | auto | Python source conventions |
 | `pyqtgraph-optimization` | auto | Batch rendering patterns |
 | `timeseries-optimization` | auto | NumPy/Polars perf patterns |
 
@@ -19,10 +17,8 @@ Skills marked **auto** are background knowledge
 applied automatically when Claude detects relevance.
 Only `commit-msg` is user-invoked via slash command.
 
-Portable skill source files live under
+Skill source files live under
-`.claude/skills/<skill-name>/SKILL.md` and are shared with
+`.claude/skills/<skill-name>/SKILL.md`.
-OpenCode. Claude-specific behavior belongs in command or
-settings files, not the shared skill bodies.
 
 ---
 
@@ -88,7 +84,8 @@ description context. Examples:
 **Footer** (always):
 ```
 (this patch was generated in some part by
-`claude-code` using `<model>` (`<provider>`))
+[`claude-code`][claude-code-gh])
+[claude-code-gh]: https://github.com/anthropics/claude-code
 ```
 
 ### Output Files
@@ -97,8 +94,7 @@ After generation, the commit message is written to:
 
 ```
 .claude/
-├── skills/commit-msg/msgs/
+├── <timestamp>_<hash>_commit_msg.md   # archived
-│   └── <timestamp>_<hash>_commit_msg.md
 └── git_commit_msg_LATEST.md           # latest
 ```
 
@@ -106,10 +102,18 @@ Where `<timestamp>` is ISO-8601 with seconds and
 `<hash>` is the first 7 chars of the current
 `HEAD` commit.
 
-Use the latest file with an editor review before commit:
+Use the latest file to feed into `git commit`:
 
 ```bash
-git commit --edit --file .claude/git_commit_msg_LATEST.md
+git commit -F .claude/git_commit_msg_LATEST.md
+```
+
+Or review/edit before committing:
+
+```bash
+cat .claude/git_commit_msg_LATEST.md
+# edit if needed, then:
+git commit -F .claude/git_commit_msg_LATEST.md
 ```
 
 ### Examples
@@ -124,32 +128,56 @@ Add `MktPair.fqme` property for symbol resolution
 Factor `.claude/skills/` into proper subdirs
 
 Deats,
-- use portable Agent Skills frontmatter
+- `commit_msg/` -> `commit-msg/` w/ enhanced
-- keep harness commands as thin delegates
+  frontmatter
-- share supporting references across harnesses
+- all background skills set `user-invocable: false`
+- content split into supporting files
 
 (this patch was generated in some part by
-`claude-code` using `<model>` (`<provider>`))
+[`claude-code`][claude-code-gh])
+[claude-code-gh]: https://github.com/anthropics/claude-code
 ```
 
-### Shared Frontmatter
+### Frontmatter Reference
 
-Skills use the Agent Skills fields understood by both
+The skill's `SKILL.md` uses these Claude Code
-Claude Code and OpenCode:
+frontmatter fields:
 
 ```yaml
 ---
 name: commit-msg
 description: >
-  Generate piker-style git commit messages. Use when...
+  Generate piker-style git commit messages...
-compatibility: Requires git.
+argument-hint: "[optional-scope-or-description]"
-metadata:
+disable-model-invocation: true
-  author: goodboy
+allowed-tools:
-  version: "1.0"
+  - Bash(git *)
+  - Read
+  - Grep
+  - Glob
+  - Write
 ---
 ```
 
-Live git context is gathered by the skill at execution
+| Field | Purpose |
-time. Claude-only dynamic interpolation and tool-policy
+|-------|---------|
-frontmatter are intentionally excluded so another harness
+| `argument-hint` | Shows hint in autocomplete |
-can execute the same workflow.
+| `disable-model-invocation` | Only user can trigger via `/commit-msg` |
+| `allowed-tools` | Tools the skill can use |
+
+### Dynamic Context
+
+The skill injects live data at invocation time
+via `!`backtick`` syntax in the `SKILL.md`:
+
+```markdown
+## Current staged changes
+!`git diff --staged --stat`
+
+## Recent commit style reference
+!`git log --oneline -10`
+```
+
+This means the staged diff stats and recent log
+are always fresh when the skill runs -- no stale
+context.

ai/claude-code/plans/datad_service.md

@@ -1,143 +0,0 @@
-# Split `brokerd.<broker>` into trading-only `brokerd` + new `datad.<broker>`
-
-## Context
-
-Today a single `brokerd.<broker>` actor hosts BOTH concerns:
-
-- **data feed service tasks**: the `_FeedsBus` + `open_feed_bus()` ep (`piker/data/feed.py:464`), per-symbol `allocate_persistent_feed()` tasks (shm writers via `sample_and_broadcast()`, history backfill via `piker.tsp`), plus backend eps `stream_quotes`, `open_history_client`, `open_symbol_search`, `get_mkt_info` from `piker/brokers/<backend>/feed.py`/`symbols.py`,
-- **live order-control tasks**: `open_trade_dialog()` from `piker/brokers/<backend>/broker.py`, driven by `emsd`.
-
-The codebase already anticipates this split: `piker/data/validate.py:70-91` groups backend eps into `'datad'` vs `'brokerd'` kinds, `piker/brokers/ib/__init__.py:62-70` already declares `_brokerd_mods`/`_datad_mods`, and `piker/brokers/_daemon.py:62` carries the literal TODO *"rename the daemon to datad prolly once we split up broker vs. data tasks into separate actors?"*. This work executes that split.
-
-**User-decided constraints:**
-
-- `datad.<broker>` is a **sibling** of `brokerd.<broker>` under `pikerd`, spawned via the existing `Services` + `maybe_spawn_daemon()` machinery (`piker/service/_daemon.py:46`).
-- **Hard cutover, staged by layer** — no dual-mode runtime flag; every stage lands fully working.
-- Post-split `brokerd` is **trading-only and lazily spawned solely by emsd's** `open_brokerd_dialog` path; UI/CLI/feed code never spawns it. Chart-only and paper sessions run with **zero** brokerd processes.
-
-## Target topology
-
-```
-pikerd
-├── datad.ib      feed bus, shm writers, tsp history, symbol search
-├── brokerd.ib    open_trade_dialog only (EMS-spawned, lazy)
-├── emsd
-│   └── paperboi.ib   (paper mode; opens its feed via datad)
-└── samplerd
-```
-
-## Key verified facts (load-bearing)
-
-- `feed.portals` has exactly ONE trading consumer: `piker/clearing/_ems.py:671` (`portal = feed.portals[brokermod]` → handed to `open_brokerd_dialog`). This is the single coupling forcing feed + trading into one actor.
-- `assert 'brokerd' in servicename` at `piker/data/feed.py:502` is the only actor-name assert in the tree.
-- `piker ledger` (`piker/accounting/cli.py:100-157`) is a hidden consumer: it calls `broker_init()` directly, spawns its own ad-hoc actor, enters `_setup_persistent_brokerd`, then calls `open_brokerd_dialog(brokermod, portal, ...)` **with an explicit portal** — the new signature must keep a `portal:` override param.
-- kraken's feed→broker coupling is mild: `kraken/broker.py:77-81` imports `NoBsWs`/`open_autorecon_ws` (really from `piker.data._web_bs`) and `stream_messages` (pure parser) from `feed.py`. In-process imports only — `enable_modules` gates RPC, not imports. No live shared state; each actor opens its own WS.
-- ib: default `client_id=6116` (`ib/api.py:1320`) with linear retry `clientId=client_id + i` (`:1403`). Two ib actors connecting concurrently will collide → needs a role-based id offset. Both daemons need `_spawn_kwargs={'infect_asyncio': True}` (copied by both init fns).
-- binance has no `symbols.py` (`get_mkt_info`/`open_symbol_search` live in `binance/feed.py`); kucoin/questrade/robinhood are flat single-file with NO `__enable_modules__`; deribit has no `broker.py`.
-- `tests/test_services.py:80,185` assert `brokerd` actor names incl. the paper-mode flow asserting `brokerd.kraken` spawns (`:229,:237`) — must invert post-split.
-- `_root_modules` (`piker/service/_actor_runtime.py:156`) must gain the new datad daemon mod so `pikerd_portal.run(spawn_datad, ...)` resolves.
-- `piker/brokers/core.py:175` `symbol_search` does `portal.run(search_w_brokerd)` where the target fn lives in `piker.brokers.core` → that mod must stay RPC-enabled in datad.
-
-## Module decision
-
-New file **`piker/data/_daemon.py`** hosts all datad machinery (`_setup_persistent_datad`, `datad_init`, `spawn_datad`, `maybe_spawn_datad`, `_datad_service_mods`). Mirrors the `piker.brokers._daemon` / `piker.data._sampling` (samplerd) per-subsystem convention and satisfies the existing TODO at `brokers/_daemon.py:49` ("move this def to the `.data` subpkg"). `piker.brokers._daemon` keeps brokerd-only code, slimmed. Do NOT over-DRY the two ~40-line init fns into a shared factory yet (samplerd precedent accepts the duplication).
-
----
-
-## Stage 0 — prep: backend module grouping + ep introspection (no topology change)
-
-1. `piker/data/validate.py`: add a `get_eps(mod, kind) -> dict[str, Callable]` helper returning the backend's defined eps for a daemon-kind from `_eps` (missing eps excluded). Used later by both init fns + fail-fast checks.
-2. Declare daemon-kind module groups in each split backend's `__init__.py`, keeping `__enable_modules__` as the (deduped) union so behavior is unchanged:
-   - `kraken`: `_brokerd_mods = ['api', 'broker']`, `_datad_mods = ['api', 'feed', 'symbols']`
-   - `binance`: `_brokerd_mods = ['api', 'broker']`, `_datad_mods = ['api', 'feed']`
-   - `deribit`: `_brokerd_mods = []`, `_datad_mods = ['api', 'feed']`
-   - `ib`: adjust existing groups so each includes `'api'`
-   - flat backends (kucoin etc.): no attrs — init fns fall back to enabling just `modpath` (existing behavior).
-3. Hygiene: `kraken/broker.py:77-81` imports `NoBsWs`/`open_autorecon_ws` from `piker.data._web_bs` directly (keep the `stream_messages` import from `.feed`).
-
-**Gate**: full pytest green, zero behavior change.
-
-## Stage 1 — introduce `datad` daemon machinery (additive)
-
-New `piker/data/_daemon.py`:
-
-- `_datad_service_mods: list[str]` — datad-always-enabled mods (successor to the data side of `_data_mods` from `brokers/_daemon.py:52`): `['piker.brokers.core', 'piker.brokers.data', 'piker.data', 'piker.data.feed', 'piker.data._sampling', 'piker.data._daemon']`.
-- `_setup_persistent_datad(ctx, brokername, loglevel)` — `@tractor.context` fixture; logging boilerplate (as `_setup_persistent_brokerd:81-88`), then allocates the actor-global feed bus exactly as the brokerd fixture does today (`brokers/_daemon.py:105-121`): `assert not feed._bus`, open service nursery, `feed.get_feed_bus(brokername, service_nursery)`, `ctx.started()`, `sleep_forever()`.
-- `datad_init(brokername, ...)` — mirrors `broker_init()` (`brokers/_daemon.py:132`): actor name `f'datad.{brokername}'`, copies backend `_spawn_kwargs` (**critical for ib infect_asyncio**), builds `enable_modules` from `getattr(brokermod, '_datad_mods', getattr(brokermod, '__enable_modules__', []))`.
-- `spawn_datad(brokername, ...)` — mirrors `spawn_brokerd()` (`brokers/_daemon.py:202`): `Services.actor_n.start_actor(dname, enable_modules=_datad_service_mods + backend_mods, ...)` + `Services.start_service_task(dname, portal, _setup_persistent_datad, ...)`.
-- `maybe_spawn_datad(brokername, ...)` — wraps `maybe_spawn_daemon(service_name=f'datad.{brokername}', service_task_target=spawn_datad, ...)` exactly like `maybe_spawn_brokerd` (`brokers/_daemon.py:256`).
-
-Supporting edits:
-
-- `piker/service/_actor_runtime.py:156`: add `'piker.data._daemon'` to `_root_modules` (keep `'piker.brokers._daemon'`).
-- `piker/service/__init__.py`: re-export `spawn_datad`/`maybe_spawn_datad` next to the brokerd ones (`:56-57`).
-- `tests/test_services.py`: new `test_datad_spawn` — `open_test_pikerd` + `maybe_spawn_datad('kraken')` + `ensure_service('datad.kraken')`. (Do NOT route a feed through it yet — `open_feed_bus`'s assert still says brokerd.)
-
-**Gate**: suite green + new spawn test; nothing routes through datad yet.
-
-## Stage 2 — clearing layer: emsd self-spawns brokerd (decouple from `feed.portals`)
-
-Sequenced BEFORE the feed cutover so live trading works at every boundary (otherwise `feed.portals` would hand emsd a datad portal).
-
-`piker/clearing/_ems.py`:
-
-1. `open_brokerd_dialog()` (`:336`) new signature: `(brokermod, exec_mode, fqme=None, portal: tractor.Portal|None = None, loglevel=None)`. Internals: keep trades-ep detection (`:400-417`); acquire the brokerd actor ONLY when a live ep will actually open, via a small inner `@acm _acquire_live_portal()` that yields the passed `portal` if given (the `piker ledger` path) else `async with maybe_spawn_brokerd(brokermod.name, loglevel=loglevel)` — **the single place brokerd boots post-split**. Move the eager `portal.open_context(trades_endpoint, ...)` construction (`:425`) inside that block; the paper short-circuit (`:432`) never touches it.
-2. `Router.maybe_open_brokerd_dialog()` (`:581`) — drop the `portal` param; `Router.open_trade_relays()` (`:640`) — delete `portal = feed.portals[brokermod]` (`:671`) and the pass-through (`:685`).
-3. `piker/accounting/cli.py:144`: switch to keyword form `open_brokerd_dialog(brokermod, exec_mode=..., portal=portal, loglevel=...)`.
-
-Pre-cutover this is a pure refactor: emsd's `maybe_spawn_brokerd` just *finds* the already-running brokerd via the registry.
-
-**Gate**: `tests/test_ems.py` + services suite green; manual paper order on kraken; `piker ledger` smoke.
-
-## Stage 3 — feed layer hard cutover to datad
-
-1. `piker/data/feed.py`: import `maybe_spawn_datad` (replacing `maybe_spawn_brokerd`, `:56-58`); `open_feed()` `:895` → `maybe_spawn_datad(...)` (rename `brokerd_ctxs` → `datad_ctxs`); `open_feed_bus()` `:502` → `assert 'datad' in servicename`; comment sweep.
-2. `piker/brokers/_daemon.py` `_setup_persistent_brokerd()`: slim to trading-only fixture — logging setup, `ctx.started()`, `sleep_forever()`. Drop the bus alloc, `assert not feed._bus`, the service nursery (backend `open_trade_dialog` ctxs own their task trees), the `eg.ExceptionGroup` handler, and the stale `_FeedsBus` TYPE_CHECKING import. (`piker ledger`'s ad-hoc actor enters this same slimmed fixture — exactly what it needs.)
-3. Repoint remaining data-flavored spawn sites `maybe_spawn_brokerd` → `maybe_spawn_datad`:
-   - `piker/ui/_app.py:40,57` (symbol search; optionally rename `install_brokerd_search` → `install_datad_search`, def at `piker/data/feed.py:766`)
-   - `piker/brokers/core.py:33,175` (`symbol_search`)
-   - `piker/brokers/cli.py:38,190,320` (`brokercheck`, `record`)
-   - `piker/ui/cli.py:30,72,121` (legacy kivy monitor/optschain) + best-effort `piker/ui/kivy/option_chain.py:498` `wait_for_actor('brokerd')` → `'datad'`
-4. `tests/test_services.py`: `:80,:185` `actor_name = 'datad'`; paper-mode test asserts `datad.kraken` + `paperboi.kraken` + `emsd` AND adds the headline negative check: `assert 'brokerd.kraken' not in services.service_tasks`.
-
-**Gate**: full suite with inverted assertions; manual: chart on binance/kraken shows `datad.<broker>` in `piker services` and NO brokerd; paper order fills; symbol search works; history backfill sane.
-
-## Stage 4 — caps-sec slimming, validation, ib client-id
-
-1. `piker/brokers/_daemon.py`: delete `_data_mods` (`:52`); `spawn_brokerd()` `:236` → `enable_modules=tractor_kwargs.pop('enable_modules')`; `broker_init()` `:184` reads `getattr(brokermod, '_brokerd_mods', getattr(brokermod, '__enable_modules__', []))`. Resulting brokerd enable set has no `piker.data.*` at all.
-2. Fail-fast ep validation via Stage-0 `validate.get_eps()`: `broker_init` raises a clear error when `get_eps(mod, 'brokerd')` is empty (e.g. "kucoin is datad-only — use paper mode"); `datad_init` warns analogously.
-3. ib client-id mitigation in `load_aio_clients()` (`ib/api.py:~1316`): role-based offset when `client_id` is the 6116 default (e.g. `+16` when `'datad' in tractor.current_actor().name`), optionally configurable via `brokers.toml` keys.
-4. Docs/comment sweep: resolve `brokers/_daemon.py:62` TODO, `piker/cli/__init__.py:253` TODO, daemon list in `piker/service/__init__.py:21` docstring.
-
-**Gate**: full suite; audit greps clean: `grep -rn maybe_spawn_brokerd piker/` hits only `clearing/_ems.py`, `service/__init__.py`, `brokers/_daemon.py`; no spawn-path `'brokerd'` literals left in `piker/data`, `piker/ui`, `piker/brokers/cli.py`.
-
----
-
-## Risk register
-
-| Risk | Sev | Mitigation |
-|---|---|---|
-| ib: `datad.ib` + `brokerd.ib` both connect to TWS/gw; `client_id=6116` collision burns `connect_timeout` retries | High (live ib only) | Stage 4 role-based id offset; both daemons inherit `infect_asyncio` via `_spawn_kwargs` copy in both init fns |
-| `piker ledger` ad-hoc actor path silently broken by signature change | Med | `portal:` override kept on `open_brokerd_dialog` (Stage 2); explicit smoke test |
-| datad-only backends (kucoin/deribit) → useless brokerd spawn | Low | already covered: `open_brokerd_dialog` forces `exec_mode='paper'` when no trades ep (`_ems.py:413-417`) BEFORE the lazy spawn; Stage 4 fail-fast |
-| brokerd in-process symbology/ledger needs (ib `broker.py` imports `.symbols`; kraken uses `stream_messages`) | None | verified pure in-process imports; `enable_modules` gates RPC only |
-| paper-mode test asserts brokerd spawns (`test_services.py:237`) | Low | Stage 3 deliberately inverts it |
-| doubled per-broker clients (HTTP/WS, symcache loads) | Low | each actor already opens its own WS today; symcache is disk-read-mostly |
-
-## Verification (per stage gates above, plus end-to-end)
-
-- `pytest tests/test_services.py tests/test_feeds.py tests/test_ems.py` at every stage (kraken/binance public endpoints, no creds needed).
-- Manual smoke matrix post-Stage-3: `pikerd -l info` + `piker chart btcusdt.spot.binance` → `piker services` shows `datad.binance`, no brokerd; submit paper order → `paperboi.binance` appears, still no brokerd; symbol search; `piker ledger <broker>.paper`.
-- If ib creds/gateway available: live ib chart + small live order to exercise dual-actor client-id path (Stage 4).
-
-## Critical files
-
-- **new** `piker/data/_daemon.py` — all datad machinery
-- `piker/brokers/_daemon.py` — slimmed brokerd fixture/init/spawn
-- `piker/data/feed.py` — spawn cutover + actor-name assert
-- `piker/clearing/_ems.py` — emsd lazy brokerd spawn (`open_brokerd_dialog`, `Router`)
-- `piker/service/_actor_runtime.py`, `piker/service/__init__.py` — root mods + re-exports
-- `piker/data/validate.py` — `get_eps()` helper
-- backend `__init__.py`s (kraken/binance/deribit/ib) — `_datad_mods`/`_brokerd_mods`
-- `piker/accounting/cli.py` — keyword-form dialog call
-- `tests/test_services.py` — inverted + new assertions

ai/opencode/README.md

@@ -1,51 +0,0 @@
-# OpenCode Integration
-
-[OpenCode](https://opencode.ai/) uses piker's shared
-repo-specific skills directly from `.claude/skills/`.
-There is no second copy of each skill to keep in sync.
-
-## Available Skills
-
-| Skill | Activation | Specialization |
-|-------|------------|----------------|
-| `commit-msg` | `/commit-msg` or explicit request | Piker commit-message style and artifacts |
-| `piker-conc-expert` | automatic | `tractor` actor topology, RPC, and cancellation |
-| `piker-profiling` | automatic | Cross-actor `Profiler` instrumentation |
-| `piker-slang` | automatic | Project communication style |
-| `py-codestyle` | automatic | Python source conventions |
-| `pyqtgraph-optimization` | automatic | Batched Qt graphics rendering |
-| `timeseries-optimization` | automatic | NumPy and Polars timeseries performance |
-
-OpenCode's project command lives at
-`.opencode/commands/commit-msg.md`. It delegates to the
-shared `commit-msg` skill and only supplies command
-arguments; workflow logic remains in the skill.
-
-A user-local skill with the same name may override a repo
-skill. Treat `opencode debug skill` as authoritative when
-diagnosing which source is active.
-
-## Verify Discovery
-
-From the repository root:
-
-```console
-opencode debug skill
-opencode debug config
-```
-
-`opencode debug skill` should list the piker skills from
-the project checkout. Restart OpenCode after changing a
-skill or command because configuration-time files are not
-hot-reloaded into an active session.
-
-## Portability Rules
-
-- Use only portable Agent Skills frontmatter in shared
-  `SKILL.md` files.
-- Keep harness-specific command syntax under
-  `.opencode/commands/`.
-- Detect the active harness and model at runtime; do not
-  hardcode Claude Code attribution in shared workflows.
-- Keep supporting references beside each shared skill so
-  relative links resolve in every harness.

ai/prompt-io/claude/20260610T170859Z_75cefe10_prompt_io.md

@@ -1,59 +0,0 @@
----
-model: claude-fable-5[1m]
-service: claude
-session: 32d15f9a-b2d3-4c26-bdc9-190219141a25
-timestamp: 2026-06-10T17:08:59Z
-git_ref: datad_service
-diff_cmd: git log -1 -p --follow -- ai/prompt-io/claude/20260610T170859Z_75cefe10_prompt_io.md
-scope: code
-substantive: true
-raw_file: 20260610T170859Z_75cefe10_prompt_io.raw.md
----
-
-## Prompt
-
-Session-initiating instruction (driving all 7 commits in
-this series):
-
-> ok i want you to become the distributed runtime and
-> concurrency expert for this project - namely acquire
-> a deep understanding of tractor and how it's used.
-> then i want you to attempt to factor our current
-> brokerd service daemon into 2 daemons: a brokerd
-> which only hosts live/paper trading endpoint tasks
-> [..] a new `datad` subdaemon which in a separate
-> subactor serves all the data feed service tasks [..]
-> give me a mega detailed plan on how to approach this,
-> and a staged approach for the implementation.
-
-Proximate driver for THIS commit: during the approved
-plan's stage-0 test gate the agent discovered test
-subactors were writing into the user's REAL
-`~/.config/piker/accounting/` files (a bogus paper fill
-landed in `account.kraken.paper.toml`) because the old
-test-dir override in `config.get_app_dir()` was
-commented out and could never work in spawned
-subactors anyway. Fix applied autonomously under the
-approved plan; no explicit per-fix user prompt.
-
-## Response summary
-
-Restore `pytest` config-dir isolation by resolving the
-per-test tmp dir lazily (at conf-path access time) from
-`tractor` runtime-vars, which propagate down the actor
-tree; route all conf-path derivation through
-`config.get_conf_dir()` so the override is effective in
-every (sub)actor.
-
-## Files changed
-
-- `piker/config.py` — add `_maybe_use_test_dir()`;
-  hook in `get_conf_dir()`; route `get_conf_path()` +
-  `load()` mkdir through it
-- `piker/accounting/_ledger.py` — derive ledger dir
-  via `config.get_conf_dir()` (not the global)
-- `piker/accounting/_pos.py` — same for account files
-
-## Human edits
-
-None — committed as generated.

ai/prompt-io/claude/20260610T170859Z_75cefe10_prompt_io.raw.md

@@ -1,46 +0,0 @@
----
-model: claude-fable-5[1m]
-service: claude
-timestamp: 2026-06-10T17:08:59Z
-git_ref: datad_service
-diff_cmd: git log -1 -p --follow -- ai/prompt-io/claude/20260610T170859Z_75cefe10_prompt_io.md
----
-
-NOTE: diff-ref mode entry (code committed in the same
-commit as this log); backfilled from the live dev
-session transcript per the `/prompt-io` skill rules.
-
-> `git log -1 -p --follow -- piker/config.py`
-
-Generated: `config._maybe_use_test_dir()` — lazily
-reads `piker_test_dir` from
-`tractor.runtime._state._runtime_vars['piker_vars']`
-(pre-loaded by `open_piker_runtime()` from the
-`tests.conftest._open_test_pikerd()` overrides) and
-calls `_override_config_dir()` when set. Hooked at the
-top of `get_conf_dir()`; `get_conf_path()` and
-`load()`'s dir-creation rerouted through
-`get_conf_dir()`.
-
-> `git log -1 -p --follow -- piker/accounting/_ledger.py`
-> `git log -1 -p --follow -- piker/accounting/_pos.py`
-
-Generated: ledger/account dir derivation switched from
-the `config._config_dir` module global to
-`config.get_conf_dir()` + `mkdir(parents=True,
-exist_ok=True)` for nested tmp-dir creation.
-
-Key diagnostic reasoning (verbatim from session):
-
-The old (commented-out) `get_app_dir()` override gated
-on `'pytest' in sys.modules` which can NEVER work in
-spawned subactors (fresh procs, no pytest import); as
-a result test `paperboi`/daemon actors were writing
-into the user's REAL `~/.config/piker/accounting/`
-files. Evidence: `account.kraken.paper.toml` gained a
-single 0.001 xbtusdt clear stamped 2026-06-09T18:47Z
-(a test fill), and `trades_kraken_paper.toml` (252B)
-contained only that fill. The resolution must be
-checked lazily at config-path access time (NOT import
-time) since sub-actors only receive runtime-vars once
-their `tractor` runtime has fully booted.

ai/prompt-io/claude/20260610T171022Z_4485f2b9_prompt_io.md

@@ -1,53 +0,0 @@
----
-model: claude-fable-5[1m]
-service: claude
-session: 32d15f9a-b2d3-4c26-bdc9-190219141a25
-timestamp: 2026-06-10T17:10:22Z
-git_ref: datad_service
-diff_cmd: git log -1 -p --follow -- ai/prompt-io/claude/20260610T171022Z_4485f2b9_prompt_io.md
-scope: code
-substantive: true
-raw_file: 20260610T171022Z_4485f2b9_prompt_io.raw.md
----
-
-## Prompt
-
-Same session-initiating `brokerd`-split instruction (see
-`20260610T170859Z_75cefe10_prompt_io.md`). Proximate
-driver: the approved plan's per-stage test gates could
-not run AT ALL — the branch base was broken vs.
-`tractor` git `main` (`AttributeError: 'tuple' object
-has no attribute 'unwrap'` at `pikerd` boot, stale
-`arbiter_sockaddr`/`.type`/`.size` API refs). The agent
-fixed forward autonomously to (re)establish the gate
-baseline, continuing the `repair_tests` branch lineage
-(verified already merged in ancestry via `git cherry`
-during a user-requested branch-overlap survey).
-
-## Response summary
-
-Port the service layer + test suites to current
-`tractor` APIs: addr-type normalization in
-`open_pikerd()`, `query_actor()`/`wait_for_actor()`
-kwarg renames, modern self-cancel absorption semantics
-in the cancel-method test harness, exc/position attr
-renames, a paper-EMS startup-budget bump and a syntax
-fix in `.deribit.api`.
-
-## Files changed
-
-- `piker/service/_actor_runtime.py` — `wrap_address()`
-  normalize before `.unwrap()` in `open_pikerd()`
-- `piker/service/_registry.py` — `check_for_service()`
-  -> `query_actor(regaddr=)` + 2-tuple yield +
-  `open_registry(addrs=)`
-- `piker/brokers/deribit/api.py` — missing comma in
-  `tractor.trionics` import tuple
-- `tests/test_services.py` — `registry_addr=` kwarg,
-  raddr unwraps, cancel-semantics harness rewrite,
-  `fail_after` 9 -> 19s
-- `tests/test_ems.py` — `.boxed_type`, `pp.cumsize`
-
-## Human edits
-
-None — committed as generated.

ai/prompt-io/claude/20260610T171022Z_4485f2b9_prompt_io.raw.md

@@ -1,56 +0,0 @@
----
-model: claude-fable-5[1m]
-service: claude
-timestamp: 2026-06-10T17:10:22Z
-git_ref: datad_service
-diff_cmd: git log -1 -p --follow -- ai/prompt-io/claude/20260610T171022Z_4485f2b9_prompt_io.md
----
-
-NOTE: diff-ref mode entry (code committed in the same
-commit as this log); backfilled from the live dev
-session transcript per the `/prompt-io` skill rules.
-
-> `git log -1 -p --follow -- piker/service/_actor_runtime.py`
-
-Generated: normalize each registry addr via
-`tractor.discovery._addr.wrap_address()` before
-`.unwrap()`-ing for the `accept_addrs` bind check —
-entries may be raw `tuple`s when passed in from (test)
-client code. Import-path precedent taken from
-`piker/cli/__init__.py:336`.
-
-> `git log -1 -p --follow -- piker/service/_registry.py`
-
-Generated: `check_for_service()` ported to
-`tractor.query_actor(name, regaddr=...)` (kwarg was
-`arbiter_sockaddr=`), unpacking the new
-`(sockaddr, portal)` yield, and passing the
-now-required `open_registry(addrs=Registry.addrs)`.
-
-> `git log -1 -p --follow -- tests/test_services.py`
-> `git log -1 -p --follow -- tests/test_ems.py`
-> `git log -1 -p --follow -- piker/brokers/deribit/api.py`
-
-Key diagnostic reasoning (verbatim from session):
-
-- the "DID NOT RAISE ContextCancelled" failure: in this
-  test the client actor IS pikerd (in-proc), and
-  current `tractor` main absorbs a `ContextCancelled`
-  whose canceller is your own actor — self-requested
-  cancels now exit cleanly instead of raising; the
-  'sigint' variant propagates a bare collapsed
-  `KeyboardInterrupt` rather than a
-  `BaseExceptionGroup`.
-- the hard-coded `trio.fail_after(9)` startup budget is
-  marginal — full stack boot (pikerd -> emsd ->
-  brokerd.kraken -> paperboi + live kraken symbology
-  fetch) occasionally exceeds 9s -> bumped to 19s.
-- `RemoteActorError.type` -> `.boxed_type`;
-  `Position.size` -> `.cumsize` (the paper engine
-  populates `BrokerdPosition.size` from `pp.cumsize`).
-- overlap survey (user-requested): all of the
-  `repair_tests` branch commits are already in this
-  stack's ancestry; this commit finishes that branch's
-  port mission (its f4c4f1e2 fixed `conftest.py`'s
-  `arbiter_sockaddr` usage; this fixes the remaining
-  `test_services.py` + `check_for_service()` sites).

ai/prompt-io/claude/20260610T171105Z_bc6e18d7_prompt_io.md

@@ -1,54 +0,0 @@
----
-model: claude-fable-5[1m]
-service: claude
-session: 32d15f9a-b2d3-4c26-bdc9-190219141a25
-timestamp: 2026-06-10T17:11:05Z
-git_ref: datad_service
-diff_cmd: git log -1 -p --follow -- ai/prompt-io/claude/20260610T171105Z_bc6e18d7_prompt_io.md
-scope: code
-substantive: true
-raw_file: 20260610T171105Z_bc6e18d7_prompt_io.raw.md
----
-
-## Prompt
-
-Same session-initiating `brokerd`-split instruction (see
-`20260610T170859Z_75cefe10_prompt_io.md`); this is the
-approved plan's "stage 0" prep. Plan-shaping user
-decisions captured via in-session Q&A:
-
-- `datad.<broker>` topology: sibling of
-  `brokerd.<broker>` under `pikerd` (vs. child
-  subactor).
-- migration: hard cutover, staged by layer (no
-  dual-mode runtime flag).
-- post-split `brokerd` scope: trading-only +
-  EMS-lazy-spawned (charts/CLI never touch it).
-
-## Response summary
-
-Declare per-daemon-kind backend submod groups
-(`_datad_mods`/`_brokerd_mods`) in the split-style
-backends, keyed to the pre-existing
-`piker.data.validate._eps` contract; add a
-`validate.get_eps()` introspection helper; ws-import
-hygiene in `.kraken.broker`. Zero behavior change
-(`__enable_modules__` unions unchanged).
-
-## Files changed
-
-- `piker/data/validate.py` — add `get_eps()`
-- `piker/brokers/kraken/__init__.py` — mod groups
-- `piker/brokers/binance/__init__.py` — mod groups
-  (note: no `symbols.py`; search eps live in `.feed`)
-- `piker/brokers/deribit/__init__.py` — datad-only
-  groups (no `broker.py` yet)
-- `piker/brokers/ib/__init__.py` — add `'api'` to the
-  pre-existing `_datad_mods`
-- `piker/brokers/kraken/broker.py` — import
-  `NoBsWs`/`open_autorecon_ws` from
-  `piker.data._web_bs` directly
-
-## Human edits
-
-None — committed as generated.

ai/prompt-io/claude/20260610T171105Z_bc6e18d7_prompt_io.raw.md

@@ -1,46 +0,0 @@
----
-model: claude-fable-5[1m]
-service: claude
-timestamp: 2026-06-10T17:11:05Z
-git_ref: datad_service
-diff_cmd: git log -1 -p --follow -- ai/prompt-io/claude/20260610T171105Z_bc6e18d7_prompt_io.md
----
-
-NOTE: diff-ref mode entry (code committed in the same
-commit as this log); backfilled from the live dev
-session transcript per the `/prompt-io` skill rules.
-
-> `git log -1 -p --follow -- piker/data/validate.py`
-
-Generated: `get_eps(mod, kind) -> dict[str, Callable]`
-returning the daemon-kind's ep funcs defined by a
-backend mod, keyed by ep name, missing eps excluded;
-sourced from the existing `_eps` grouping table
-(`'middleware' | 'datad' | 'brokerd'`).
-
-> `git log -1 -p --follow -- piker/brokers/kraken/__init__.py`
-> `git log -1 -p --follow -- piker/brokers/binance/__init__.py`
-> `git log -1 -p --follow -- piker/brokers/deribit/__init__.py`
-> `git log -1 -p --follow -- piker/brokers/ib/__init__.py`
-> `git log -1 -p --follow -- piker/brokers/kraken/broker.py`
-
-Key exploration findings driving the design (from the
-planning phase, 3 parallel explore agents + 1 architect
-agent):
-
-- the codebase already anticipated this split:
-  `validate._eps` groups eps into 'datad' vs 'brokerd'
-  kinds and `ib/__init__.py` already declared
-  `_brokerd_mods`/`_datad_mods`; `brokers/_daemon.py`
-  carried the literal TODO "rename the daemon to datad
-  prolly once we split up broker vs. data tasks into
-  separate actors?".
-- the feared kraken feed<->broker "shared ws state"
-  hazard is mild: `NoBsWs`/`open_autorecon_ws`
-  originate in `piker.data._web_bs` (re-exported via
-  `.kraken.feed`); only `stream_messages` is feed-local
-  and it's a pure parser — in-process imports only,
-  each actor opens its own ws conn.
-- `enable_modules` gates RPC entry, NOT python imports,
-  so backend trading mods may keep importing feed-side
-  helpers in-process post-split.

ai/prompt-io/claude/20260610T171142Z_119d2c04_prompt_io.md

@@ -1,47 +0,0 @@
----
-model: claude-fable-5[1m]
-service: claude
-session: 32d15f9a-b2d3-4c26-bdc9-190219141a25
-timestamp: 2026-06-10T17:11:42Z
-git_ref: datad_service
-diff_cmd: git log -1 -p --follow -- ai/prompt-io/claude/20260610T171142Z_119d2c04_prompt_io.md
-scope: code
-substantive: true
-raw_file: 20260610T171142Z_119d2c04_prompt_io.raw.md
----
-
-## Prompt
-
-Same session-initiating `brokerd`-split instruction (see
-`20260610T170859Z_75cefe10_prompt_io.md`); this is the
-approved plan's "stage 1": introduce the `datad` daemon
-machinery additively (nothing routes through it yet).
-User-decided constraint applied: `datad.<broker>` is a
-SIBLING of `brokerd.<broker>` under `pikerd`, spawned
-via the existing `Services` + `maybe_spawn_daemon()`
-machinery.
-
-## Response summary
-
-New `piker/data/_daemon.py` hosting the
-`datad.<broker>` feed-only daemon-actor: lifetime
-fixture owning the actor-global `_FeedsBus`, init/spawn
-fns mirroring `.brokers._daemon` conventions and the
-`samplerd` sub-daemon precedent, plus root-mod
-registration, `piker.service` re-exports and a spawn
-test.
-
-## Files changed
-
-- `piker/data/_daemon.py` — NEW:
-  `_setup_persistent_datad()`, `datad_init()`,
-  `spawn_datad()`, `maybe_spawn_datad()`,
-  `_datad_service_mods`
-- `piker/service/_actor_runtime.py` — add
-  `piker.data._daemon` to `_root_modules`
-- `piker/service/__init__.py` — re-export spawn eps
-- `tests/test_services.py` — add `test_datad_spawn`
-
-## Human edits
-
-None — committed as generated.

ai/prompt-io/claude/20260610T171142Z_119d2c04_prompt_io.raw.md

@@ -1,60 +0,0 @@
----
-model: claude-fable-5[1m]
-service: claude
-timestamp: 2026-06-10T17:11:42Z
-git_ref: datad_service
-diff_cmd: git log -1 -p --follow -- ai/prompt-io/claude/20260610T171142Z_119d2c04_prompt_io.md
----
-
-NOTE: diff-ref mode entry (code committed in the same
-commit as this log); backfilled from the live dev
-session transcript per the `/prompt-io` skill rules.
-
-> `git log -1 -p --follow -- piker/data/_daemon.py`
-
-Generated symbols + key design decisions:
-
-- `_datad_service_mods: list[str]` — datad-always
-  enabled mods, the data-side successor to the old
-  `piker.brokers._daemon._data_mods` set; kept minimal
-  per the caps-sec model.
-- `_setup_persistent_datad()` — `@tractor.context`
-  lifetime fixture: console-log setup then allocates
-  the actor-global feed bus via
-  `feed.get_feed_bus(brokername, service_nursery)`
-  exactly as the old brokerd fixture did, pinned open
-  with `ctx.started()` + `sleep_forever()`.
-- `datad_init()` — actor name `f'datad.{brokername}'`;
-  copies backend `_spawn_kwargs` (CRITICAL for `ib`'s
-  `infect_asyncio=True`); builds `enable_modules` from
-  `getattr(brokermod, '_datad_mods',
-  getattr(brokermod, '__enable_modules__', []))` as
-  the flat-backend fallback.
-- `spawn_datad()` — `Services.actor_n.start_actor()` +
-  `Services.start_service_task()` exactly mirroring
-  `spawn_brokerd()`; dedup-composes enable mods via
-  `list(dict.fromkeys(...))`.
-- `maybe_spawn_datad()` — wraps `maybe_spawn_daemon(
-  service_name=f'datad.{brokername}', ...)`.
-
-> `git log -1 -p --follow -- piker/service/_actor_runtime.py`
-> `git log -1 -p --follow -- piker/service/__init__.py`
-> `git log -1 -p --follow -- tests/test_services.py`
-
-Design rationale (verbatim from session):
-
-- `_root_modules` must gain `piker.data._daemon` so
-  `pikerd_portal.run(spawn_datad, ...)` resolves in
-  the root.
-- the `Services`-based impl style deliberately mirrors
-  `spawn_brokerd()` so the eventual `tractor.hilevel`
-  `ServiceMngr` port (see the `service_mng_to_tractor`
-  branch's d8c21d44 prep, surfaced by the
-  user-requested branch-overlap survey) lands
-  symmetrically on both spawn fns.
-- mod placement (`piker/data/_daemon.py` vs.
-  generalizing `piker.brokers._daemon`) follows the
-  per-subsystem daemon-mod convention
-  (`.clearing._ems`, `.data._sampling`) and resolves
-  the existing TODO at `brokers/_daemon.py:49` ("move
-  this def to the `.data` subpkg").

ai/prompt-io/claude/20260610T171226Z_64181219_prompt_io.md

@@ -1,47 +0,0 @@
----
-model: claude-fable-5[1m]
-service: claude
-session: 32d15f9a-b2d3-4c26-bdc9-190219141a25
-timestamp: 2026-06-10T17:12:26Z
-git_ref: datad_service
-diff_cmd: git log -1 -p --follow -- ai/prompt-io/claude/20260610T171226Z_64181219_prompt_io.md
-scope: code
-substantive: true
-raw_file: 20260610T171226Z_64181219_prompt_io.raw.md
----
-
-## Prompt
-
-Same session-initiating `brokerd`-split instruction (see
-`20260610T170859Z_75cefe10_prompt_io.md`); this is the
-approved plan's "stage 2": decouple the clearing layer
-from `feed.portals` BEFORE the feed cutover so live
-trading works at every stage boundary. User-decided
-constraint applied: post-split `brokerd` is
-trading-only and spawned LAZILY only by `emsd`'s
-`open_brokerd_dialog()` path.
-
-## Response summary
-
-Kill the single coupling forcing feed + trading eps
-into one actor (`Router.open_trade_relays()` pulling
-its trades portal from `feed.portals[brokermod]`):
-`open_brokerd_dialog()` now (maybe) spawns/finds
-`brokerd.<broker>` itself and ONLY when a live
-trades-ep will actually open; paper mode never touches
-it. Pre-cutover this is a pure refactor (registry
-lookup finds the same feed-spawned daemon).
-
-## Files changed
-
-- `piker/clearing/_ems.py` — `open_brokerd_dialog()`
-  re-sig + inner `acquire_live_portal()`;
-  `Router.maybe_open_brokerd_dialog()` drops `portal`
-  param; `open_trade_relays()` drops the
-  `feed.portals` lookup
-- `piker/accounting/cli.py` — keyword-form `portal=`
-  override kept for the `piker ledger` ad-hoc actor
-
-## Human edits
-
-None — committed as generated.

ai/prompt-io/claude/20260610T171226Z_64181219_prompt_io.raw.md

@@ -1,43 +0,0 @@
----
-model: claude-fable-5[1m]
-service: claude
-timestamp: 2026-06-10T17:12:26Z
-git_ref: datad_service
-diff_cmd: git log -1 -p --follow -- ai/prompt-io/claude/20260610T171226Z_64181219_prompt_io.md
----
-
-NOTE: diff-ref mode entry (code committed in the same
-commit as this log); backfilled from the live dev
-session transcript per the `/prompt-io` skill rules.
-
-> `git log -1 -p --follow -- piker/clearing/_ems.py`
-
-Generated: new `open_brokerd_dialog()` signature
-`(brokermod, exec_mode, fqme=None, portal=None,
-loglevel=None)` with an inner `@acm
-acquire_live_portal()` that yields the caller-provided
-`portal` override (the `piker ledger` path) else
-`maybe_spawn_brokerd(brokermod.name)` — designated THE
-one place a live, credentialed `brokerd.<broker>` gets
-booted post-split. The eager
-`portal.open_context(trades_endpoint, ...)`
-construction moved inside that block so the paper
-short-circuit never acquires a live portal.
-
-> `git log -1 -p --follow -- piker/accounting/cli.py`
-
-Key analysis (verbatim from session):
-
-- `feed.portals` had exactly ONE trading consumer:
-  `piker/clearing/_ems.py:671` (`portal =
-  feed.portals[brokermod]`) — the single coupling
-  forcing feed + trading into one actor.
-- `piker ledger` (`accounting/cli.py:100-157`) is a
-  hidden consumer: it calls `broker_init()` directly,
-  spawns its own ad-hoc actor and passes the portal
-  positionally — the new signature must keep an
-  explicit `portal:` override param for this path.
-- stage sequencing: clearing decouple lands BEFORE the
-  feed cutover, otherwise `feed.portals` would hand
-  emsd a `datad` portal and live `open_trade_dialog`
-  RPC would fail.

ai/prompt-io/claude/20260610T171259Z_59d5d9a6_prompt_io.md

@@ -1,48 +0,0 @@
----
-model: claude-fable-5[1m]
-service: claude
-session: 32d15f9a-b2d3-4c26-bdc9-190219141a25
-timestamp: 2026-06-10T17:12:59Z
-git_ref: datad_service
-diff_cmd: git log -1 -p --follow -- ai/prompt-io/claude/20260610T171259Z_59d5d9a6_prompt_io.md
-scope: code
-substantive: true
-raw_file: 20260610T171259Z_59d5d9a6_prompt_io.raw.md
----
-
-## Prompt
-
-Same session-initiating `brokerd`-split instruction (see
-`20260610T170859Z_75cefe10_prompt_io.md`); this is the
-approved plan's "stage 3": the feed-layer hard cutover
-(user-decided: no dual-mode runtime flag, each stage
-lands fully working).
-
-## Response summary
-
-The topology flip: all data-feed consumers route to
-`datad.<broker>`; the `brokerd` lifetime fixture slims
-to trading-only; UI/CLI data-flavoured spawn sites
-repointed; test expectations inverted incl. an explicit
-negative assert that paper sessions spawn NO
-`brokerd.<broker>`.
-
-## Files changed
-
-- `piker/data/feed.py` — `open_feed()` ->
-  `maybe_spawn_datad()`; `open_feed_bus()` actor-name
-  assert -> `'datad'`; `install_brokerd_search` ->
-  `install_datad_search`; comment sweep
-- `piker/brokers/_daemon.py` —
-  `_setup_persistent_brokerd()` slimmed (feed-bus
-  alloc moved to the datad fixture)
-- `piker/ui/_app.py`, `piker/ui/cli.py`,
-  `piker/brokers/core.py`, `piker/brokers/cli.py`,
-  `piker/ui/kivy/option_chain.py` — spawn-site
-  repoints
-- `tests/test_services.py` — datad assertions + the
-  no-`brokerd.kraken` negative check
-
-## Human edits
-
-None — committed as generated.

ai/prompt-io/claude/20260610T171259Z_59d5d9a6_prompt_io.raw.md

@@ -1,54 +0,0 @@
----
-model: claude-fable-5[1m]
-service: claude
-timestamp: 2026-06-10T17:12:59Z
-git_ref: datad_service
-diff_cmd: git log -1 -p --follow -- ai/prompt-io/claude/20260610T171259Z_59d5d9a6_prompt_io.md
----
-
-NOTE: diff-ref mode entry (code committed in the same
-commit as this log); backfilled from the live dev
-session transcript per the `/prompt-io` skill rules.
-
-> `git log -1 -p --follow -- piker/data/feed.py`
-
-Generated: `open_feed()` spawn cutover with the
-`maybe_spawn_datad` import done "relative-direct" from
-`._daemon` — NOT via `piker.service` — to dodge a
-partial-init cycle: when `piker.data.feed` loads as
-part of `piker.service.__init__` executing its own
-`piker.data._daemon` import, the service pkg is
-mid-init and its `maybe_spawn_datad` binding does not
-exist yet. The `open_feed_bus()` local-state sanity
-assert flips `'brokerd' in servicename` ->
-`'datad' in servicename` (the only actor-name assert
-in the tree, verified by grep).
-
-> `git log -1 -p --follow -- piker/brokers/_daemon.py`
-
-Generated: `_setup_persistent_brokerd()` slimmed to
-console-log setup + pinned-open ctx; drops the bus
-alloc, the `assert not feed._bus`, the service nursery
-(backend `open_trade_dialog()` ctxs own their task
-trees) and the `eg.ExceptionGroup` handler. The
-`piker ledger` ad-hoc actor enters this same slimmed
-fixture — exactly what it needs.
-
-> `git log -1 -p --follow -- piker/ui/_app.py`
-> `git log -1 -p --follow -- piker/ui/cli.py`
-> `git log -1 -p --follow -- piker/brokers/core.py`
-> `git log -1 -p --follow -- piker/brokers/cli.py`
-> `git log -1 -p --follow -- piker/ui/kivy/option_chain.py`
-> `git log -1 -p --follow -- tests/test_services.py`
-
-Verification (from session): per-suite gates green at
-this commit (services 5-passed incl. the new negative
-assert, feeds 3-passed); a headless live smoke
-(`maybe_open_pikerd` + `open_feed(['xbtusdt.kraken'])`
-on an alt registry port) confirmed quotes flowing via
-`datad.kraken` + `samplerd` with `check_for_service(
-'brokerd.kraken') is None`. Known pre-existing flake
-documented: `test_multi_fill_positions`' second
-in-proc runtime boot wedges ~50% (zombie subactor w/
-unread parent-IPC bytes); reproduced with the split
-fully reverted so NOT a regression of this work.

ai/prompt-io/claude/20260610T171344Z_eee19de0_prompt_io.md

@@ -1,43 +0,0 @@
----
-model: claude-fable-5[1m]
-service: claude
-session: 32d15f9a-b2d3-4c26-bdc9-190219141a25
-timestamp: 2026-06-10T17:13:44Z
-git_ref: datad_service
-diff_cmd: git log -1 -p --follow -- ai/prompt-io/claude/20260610T171344Z_eee19de0_prompt_io.md
-scope: code
-substantive: true
-raw_file: 20260610T171344Z_eee19de0_prompt_io.raw.md
----
-
-## Prompt
-
-Same session-initiating `brokerd`-split instruction (see
-`20260610T170859Z_75cefe10_prompt_io.md`); this is the
-approved plan's final "stage 4": caps-sec slimming of
-the (live, credentialed) trading actor + the `ib`
-dual-daemon `client_id` collision mitigation flagged in
-the plan's risk register.
-
-## Response summary
-
-`brokerd` loses ALL `piker.data.*` (feed) RPC mods;
-spawn fails fast for datad-only backends with a "use
-paper-mode" error; `ib`'s default api-gw `client_id`
-gets a per-daemon-kind offset so `datad.ib` +
-`brokerd.ib` don't collide on connect.
-
-## Files changed
-
-- `piker/brokers/_daemon.py` — `_data_mods` -> minimal
-  `_brokerd_service_mods`; `broker_init()` reads
-  `_brokerd_mods` (fallback `__enable_modules__`);
-  `spawn_brokerd()` fail-fast via `validate.get_eps()`
-- `piker/brokers/ib/api.py` — role-based `client_id`
-  offset in `load_aio_clients()`
-- `piker/cli/__init__.py` — resolved "expose datad"
-  TODO
-
-## Human edits
-
-None — committed as generated.

ai/prompt-io/claude/20260610T171344Z_eee19de0_prompt_io.raw.md

@@ -1,41 +0,0 @@
----
-model: claude-fable-5[1m]
-service: claude
-timestamp: 2026-06-10T17:13:44Z
-git_ref: datad_service
-diff_cmd: git log -1 -p --follow -- ai/prompt-io/claude/20260610T171344Z_eee19de0_prompt_io.md
----
-
-NOTE: diff-ref mode entry (code committed in the same
-commit as this log); backfilled from the live dev
-session transcript per the `/prompt-io` skill rules.
-
-> `git log -1 -p --follow -- piker/brokers/_daemon.py`
-
-Generated: the fail-fast originally landed in
-`broker_init()` but was relocated to `spawn_brokerd()`
-mid-implementation after realizing `piker ledger`
-calls `broker_init()` directly even for paper accounts
-on datad-only backends (would have crashed the cli);
-the service-spawn path is the correct enforcement
-seam. Error text:
-
-  Backend 'kucoin' offers NO `brokerd` (live
-  order-control) eps!? It is likely a datad-only
-  provider, use paper-mode for clearing instead.
-
-(verified live via a `trio.run()` unit check.)
-
-> `git log -1 -p --follow -- piker/brokers/ib/api.py`
-
-Generated: in `load_aio_clients()`, when `client_id`
-is the 6116 default: `datad`-named actors offset +16
-(disjoint from `brokerd`'s linear `client_id + i`
-retry range), other non-`brokerd` (ad-hoc test/cli)
-actors +32. Rationale from the plan's risk register:
-post-split BOTH per-broker daemons connect to the same
-TWS/gw endpoint; a shared default id collides and
-burns up to `connect_timeout * retries` (90s) in
-retry cycles.
-
-> `git log -1 -p --follow -- piker/cli/__init__.py`

ai/prompt-io/claude/20260610T173309Z_f15f8178_prompt_io.md

@@ -1,66 +0,0 @@
----
-model: claude-fable-5[1m]
-service: claude
-session: 32d15f9a-b2d3-4c26-bdc9-190219141a25
-timestamp: 2026-06-10T17:33:09Z
-git_ref: datad_service
-diff_cmd: git log -1 -p --follow -- ai/prompt-io/claude/20260610T173309Z_f15f8178_prompt_io.md
-scope: docs
-substantive: true
-raw_file: 20260610T173309Z_f15f8178_prompt_io.raw.md
----
-
-## Prompt
-
-The session-initiating instruction, verbatim (this doc
-IS its "mega detailed plan" deliverable):
-
-> ok i want you to become the distributed runtime and
-> concurrency expert for this project - namely acquire
-> a deep understanding of tractor and how it's used.
-> then i want you to atttempt to factor our current
-> brokerd service daemon into 2 daemons:
->
-> - a brokerd which only hosts live/paper trading
->   endoint tasks as defined namely within all
->   piker/brokers/<backend>/broker.py mods
-> - a new `datad` subdaemon which in a separate
->   subactor serves all the data feed service tasks
->   namely delivered by endpoints in
->   piker/brokers/<backend>/feed.py (or similar for
->   less mature backends) and as more rigorously
->   defined by the validation machinery in
->   piker.data.validate.
->
-> give me a mega detailed plan on how to approach
-> this, and a staged approach for the implementation.
-
-Plan-shaping user decisions (in-session Q&A): sibling
-topology under `pikerd`; hard cutover staged by layer;
-trading-only EMS-lazy-spawned `brokerd`. The human
-staged the doc copy into `ai/claude-code/plans/` and
-requested this commit after the 7 implementation
-commits landed.
-
-## Response summary
-
-The staged design doc for the `brokerd` -> (`datad` +
-`brokerd`) split: context, target supervision topology,
-load-bearing verified facts, per-stage file-level
-changes with gates, risk register and an end-to-end
-verification matrix. Produced via 3 parallel explore
-agents + 1 architect agent + human Q&A before any code
-was written; all 7 implementation commits in this
-branch reference its stages in their provenance
-entries.
-
-## Files changed
-
-- `ai/claude-code/plans/datad_service.md` — the design
-  plan doc (human-staged copy of the AI-authored plan)
-
-## Human edits
-
-None to the doc content — committed as generated. NB:
-the implementation deviated from the plan-as-written in
-4 places, see the raw file's deviation log.

ai/prompt-io/claude/20260610T173309Z_f15f8178_prompt_io.raw.md

@@ -1,51 +0,0 @@
----
-model: claude-fable-5[1m]
-service: claude
-timestamp: 2026-06-10T17:33:09Z
-git_ref: datad_service
-diff_cmd: git log -1 -p --follow -- ai/prompt-io/claude/20260610T173309Z_f15f8178_prompt_io.md
----
-
-NOTE: diff-ref mode entry; the AI output here is the
-committed doc itself (a design plan, not code) so no
-verbatim copy is duplicated:
-
-> `git log -1 -p --follow -- ai/claude-code/plans/datad_service.md`
-
-## Implementation deviation log
-
-Where the landed 7-commit series differs from the plan
-as written (recorded for reviewer transparency; the
-plan doc is committed unmodified):
-
-1. Commit ordering: the `pytest` config-dir isolation
-   fix lands BEFORE the `tractor`-API drift port. The
-   plan's stage gates assumed a runnable+isolated test
-   baseline; per-commit gating exposed that without
-   isolation the paper-EMS test reads the user's real
-   (polluted) `account.kraken.paper.toml` and reddens.
-2. The plan's "stage 0: full pytest green" gate
-   required first repairing pre-existing branch
-   breakage vs `tractor` git `main` (boot
-   `AttributeError`, stale discovery/exc/position
-   APIs) — that repair became its own commit
-   ("Port service+tests to latest `tractor` APIs")
-   rather than plan-stage work.
-3. Stage-4 fail-fast placement: the plan said
-   `broker_init()` raises on brokerd-ep-less backends;
-   implementation moved the raise to `spawn_brokerd()`
-   since `piker ledger` calls `broker_init()` directly
-   even for paper accounts on datad-only backends.
-4. `brokers/_daemon.py` change grouping: the
-   import-cleanup hunks (`exceptiongroup`, `_FeedsBus`
-   type-only import) landed with the caps-sec slim
-   commit instead of the fixture-slim commit, keeping
-   each intermediate tree import-clean without
-   sub-hunk surgery.
-
-Also of record: the plan's "Verification" matrix was
-executed as written (per-suite gates each stage, the
-headless datad-feed smoke, the kucoin fail-fast unit
-check); the known pre-existing ~50% second-runtime-boot
-test wedge was characterized and excluded as a
-regression via revert-testing.

ai/prompt-io/claude/20260610T213549Z_f084e899_prompt_io.md

@@ -1,60 +0,0 @@
----
-model: claude-fable-5[1m]
-service: claude
-session: 32d15f9a-b2d3-4c26-bdc9-190219141a25
-timestamp: 2026-06-10T21:35:49Z
-git_ref: datad_service
-diff_cmd: git log -1 -p --follow -- ai/prompt-io/claude/20260610T213549Z_f084e899_prompt_io.md
-scope: code
-substantive: true
-raw_file: 20260610T213549Z_f084e899_prompt_io.raw.md
----
-
-## Prompt
-
-User bug report from live `ib` testing of the
-(datad|brokerd)-split branch:
-
-> ok doing some testing and noticing live orders do not
-> work otb, pikerd show this on live submission,
-> [pdb traceback: `RuntimeError("Can not order
-> {symbol}, no live feed?")` at `ib/api.py:1152`
-> `submit_limit()` with `self._contracts == {}` in the
-> `brokerd.ib` actor + a `TrioTaskExited` teardown
-> cascade]
-
-Follow-up user direction fixing the design intent:
-
-> i think the main thing here is that [brokerd] should
-> be able to always submit orders without a live feed
-> being up and registered in the same subactor right?
-
-## Response summary
-
-Root cause: pre-split the feed code populated the
-actor-local `Client._contracts` cache (via
-`get_mkt_info()` writing `mkt.bs_fqme` keys) in the
-SAME process as order submission; post-split the
-trading actor's client is never warmed. Fix: lazily
-qualify + cache the contract per order request in
-`handle_order_requests()` by running the same
-`get_mkt_info(fqme, proxy=...)` ep the feed side uses,
-plus per-order error relay (`BrokerdError`) so one bad
-submission can't crash the whole trades dialog (the
-`TrioTaskExited` storm was teardown cascade from the
-original raise).
-
-## Files changed
-
-- `piker/brokers/ib/broker.py` — thread `proxies` into
-  `handle_order_requests()`; lazy contract qualify on
-  cache-miss; guard `submit_limit()` w/ `BrokerdError`
-  relay; uncomment the (anticipatory) `get_mkt_info`
-  import
-- `piker/brokers/ib/api.py` — fix the non-f-string
-  raise msg + document the new qualification contract
-
-## Human edits
-
-None — committed as generated. Live `ib` order retest
-performed by the human post-commit.

ai/prompt-io/claude/20260610T213549Z_f084e899_prompt_io.raw.md

@@ -1,57 +0,0 @@
----
-model: claude-fable-5[1m]
-service: claude
-timestamp: 2026-06-10T21:35:49Z
-git_ref: datad_service
-diff_cmd: git log -1 -p --follow -- ai/prompt-io/claude/20260610T213549Z_f084e899_prompt_io.md
----
-
-NOTE: diff-ref mode entry (code committed in the same
-commit as this log); recorded from the live debug
-session per the `/prompt-io` skill rules.
-
-> `git log -1 -p --follow -- piker/brokers/ib/broker.py`
-> `git log -1 -p --follow -- piker/brokers/ib/api.py`
-
-Key diagnostic chain (from session):
-
-- pdb showed `Client._contracts == {}` inside
-  `brokerd.ib`'s `submit_limit()`; the cache has
-  exactly TWO write sites: `Client.find_contracts()`
-  (api.py, keys `f'{sym}.{exch}.ib'`) and
-  `symbols.get_mkt_info()` (key `mkt.bs_fqme`, eg.
-  'nvda.nasdaq' — NO `.ib` suffix).
-- `BrokerdOrder.symbol` arrives as the bs_fqme form
-  ('nvda.nasdaq') so ONLY the `get_mkt_info()` write
-  site produces the key `submit_limit()` reads —
-  ie. pre-split it was the feed's in-proc
-  `get_mkt_info(sym, proxy=proxy)` call keeping orders
-  working, NOT `find_contracts()`.
-- the existing TODO at `symbols.py:642-644` literally
-  predicted this: "this is going to be problematic
-  if/when we split out the datad vs. brokerd actors
-  since the mktmap lookup table will now be
-  inaccessible.."
-- instance identity verified: `proxy._aio_ns` IS the
-  same `Client` obj as `_accounts2clients[account]`
-  (both sourced from the `load_aio_clients()` cache via
-  `open_client_proxies()`), so a brokerd-side
-  `get_mkt_info(fqme, proxy=proxies[account])` warms
-  exactly the dict `submit_limit()` reads. It also
-  populates `client._cons2mkts` which the
-  position-audit path (`broker.py` backup-table code)
-  needs in this actor anyway.
-- the `TrioTaskExited` storm in the user's log
-  (`recv_trade_updates`, `open_aio_client_method_relay`
-  aio tasks) is teardown cascade: the raise crashed
-  `handle_order_requests` -> nursery teardown ripped
-  the trio sides of still-running aio relay tasks.
-  Hence the added per-order try/except ->
-  `BrokerdError` relay hardening so a single bad
-  submission degrades to an EMS error msg instead of
-  killing the backend's entire order-ctl dialog.
-
-Verification: `tests/test_services.py` (5 passed) +
-`tests/test_ems.py` (6 passed) regression-green; live
-`ib` submission retest delegated to the human (needs a
-running TWS/gw).

ai/prompt-io/claude/README.md

@@ -1,34 +0,0 @@
-# AI Prompt I/O Log — claude
-
-This directory tracks prompt inputs and model
-outputs for AI-assisted development using
-`claude` (claude-code CLI).
-
-## Policy
-
-Prompt logging follows the
-[NLNet generative AI policy][nlnet-ai].
-All substantive AI contributions are logged
-with:
-- Model name and version
-- Timestamps
-- The prompts that produced the output
-- Unedited model output (`.raw.md` files)
-
-[nlnet-ai]: https://nlnet.nl/foundation/policies/generativeAI/
-
-## Usage
-
-Entries are created by the `/prompt-io` skill
-or automatically via `/commit-msg` integration.
-
-Each commit carrying AI-generated changes links
-to its provenance entry via a `Prompt-IO:`
-commit-msg trailer; entries use "diff-ref mode"
-(pointers into `git log -p` instead of verbatim
-code copies) to avoid duplicating committed
-code.
-
-Human contributors remain accountable for all
-code decisions. AI-generated content is never
-presented as human-authored work.

ai/prompt-io/opencode/20260721T205228Z_58ffe487_prompt_io.md

@@ -1,64 +0,0 @@
----
-model: gpt-5.6-sol
-provider: openai
-service: opencode
-session: ses_0799212ebffe42arY96czXn89F
-timestamp: 2026-07-21T20:52:28Z
-git_ref: 58ffe487
-scope: config
-substantive: true
-raw_file: 20260721T205228Z_58ffe487_prompt_io.raw.md
----
-
-## Prompt
-
-The user asked to scan Claude's repo-local skills and
-documentation, summarize their specializations, and then
-rework repo-specific skills for use across coding
-harnesses, particularly OpenCode. The user intends to
-choose one of Claude's unfinished tasks afterward.
-
-## Response summary
-
-Standardized the tracked piker skills on portable Agent
-Skills frontmatter, made `commit-msg` harness-aware,
-documented OpenCode discovery and command delegation, and
-fixed an unsafe `np.searchsorted()` example found during
-the audit. Preserved the existing `.claude` artifact path
-as a shared legacy location to avoid breaking established
-commit workflows.
-
-## Files changed
-
-- `.claude/skills/commit-msg/SKILL.md` - portable,
-  harness-aware commit-message workflow
-- `.claude/skills/commit-msg/style-guide-reference.md` -
-  generic coding-harness attribution
-- `.claude/skills/piker-conc-expert/SKILL.md` - portable
-  frontmatter
-- `.claude/skills/piker-profiling/SKILL.md` - portable
-  frontmatter
-- `.claude/skills/piker-slang/SKILL.md` - portable
-  frontmatter
-- `.claude/skills/pyqtgraph-optimization/SKILL.md` -
-  portable frontmatter
-- `.claude/skills/timeseries-optimization/SKILL.md` -
-  portable frontmatter and safe search bounds handling
-- `ai/README.md` - shared-skill integration model
-- `ai/claude-code/README.md` - cross-harness Claude Code
-  documentation
-- `ai/opencode/README.md` - OpenCode usage and discovery
-  documentation
-- `.opencode/commands/commit-msg.md` - thin OpenCode
-  command delegating to the shared skill
-- `ai/prompt-io/opencode/README.md` - OpenCode provenance
-  policy
-- `ai/prompt-io/opencode/20260721T205228Z_58ffe487_prompt_io.raw.md`
-  - unedited response record
-- `ai/prompt-io/opencode/20260721T205228Z_58ffe487_prompt_io.md`
-  - provenance metadata and summary
-
-## Human edits
-
-None - generated changes have not yet been reviewed or
-edited by the human.

ai/prompt-io/opencode/20260721T205228Z_58ffe487_prompt_io.raw.md

@@ -1,33 +0,0 @@
----
-model: gpt-5.6-sol
-provider: openai
-service: opencode
-timestamp: 2026-07-21T20:52:28Z
-git_ref: 58ffe487
-diff_cmd: git diff HEAD~1..HEAD
----
-
-The repo-local skill inventory contains seven domains:
-commit-message generation, actor-runtime concurrency,
-distributed profiling, project communication style,
-Python source style, PyQtGraph rendering, and NumPy/Polars
-timeseries optimization.
-
-I converted the six tracked repo-specific skills to the
-portable Agent Skills frontmatter subset so Claude Code
-and OpenCode can load the same source files. The
-commit-message workflow now gathers live git context at
-execution time, uses runtime harness/model attribution,
-preserves human-owned task state, and keeps the existing
-message artifact paths for compatibility.
-
-I added OpenCode integration documentation and updated
-the shared and Claude Code docs to describe a single
-cross-harness skill source with thin harness commands.
-During validation I also corrected an out-of-bounds
-`np.searchsorted()` example in the timeseries skill.
-
-The unfinished artifacts found during the scan are a
-shared-memory migration plan and a gap-annotation
-performance handoff. The earlier datad split plan appears
-implemented and is now captured by the concurrency skill.

ai/prompt-io/opencode/20260721T215055Z_c6ec3d41_prompt_io.md

@@ -1,41 +0,0 @@
----
-model: gpt-5.6-sol
-provider: openai
-service: opencode
-session: ses_0799212ebffe42arY96czXn89F
-timestamp: 2026-07-21T21:50:55Z
-git_ref: c6ec3d41
-scope: docs
-substantive: true
-raw_file: 20260721T215055Z_c6ec3d41_prompt_io.raw.md
----
-
-## Prompt
-
-The user asked to boot a plan to continue work on the
-time-series gap checker and bug fixing of the history
-backfiller. The user authorized prompt-I/O capture for the
-plan documentation after the repository audit.
-
-## Response summary
-
-Produced a bounded repair plan based on current source,
-historical issue caches, existing profiling notes, and a
-test-coverage audit. The plan prioritizes reproducible
-correctness failures and storage durability before venue
-classification, annotation performance, or manual repair
-UX.
-
-## Files changed
-
-- `plans/opencode/gap-checker-backfiller-repair.md` -
-  phased implementation and verification plan
-- `ai/prompt-io/opencode/20260721T215055Z_c6ec3d41_prompt_io.raw.md`
-  - unedited response record
-- `ai/prompt-io/opencode/20260721T215055Z_c6ec3d41_prompt_io.md`
-  - provenance metadata and response summary
-
-## Human edits
-
-None - generated plan has not yet been edited by the
-human.

ai/prompt-io/opencode/20260721T215055Z_c6ec3d41_prompt_io.raw.md

@@ -1,26 +0,0 @@
----
-model: gpt-5.6-sol
-provider: openai
-service: opencode
-timestamp: 2026-07-21T21:50:55Z
-git_ref: c6ec3d41
-diff_cmd: git diff HEAD~1..HEAD
----
-
-> `git diff -- plans/opencode/gap-checker-backfiller-repair.md`
-
-Generated a phased implementation plan for continuing
-piker's gap checker and history backfiller repairs. The
-plan separates data correctness from annotation
-performance and orders work by dependency: deterministic
-tests, canonical frame/gap normalization, structured
-backfill completion, non-destructive atomic NativeDB
-writes, exact null repair, provider/venue classification,
-annotation vectorization, and crash qualification.
-
-The plan is grounded in current code paths, local issue
-caches `#62`, `#71`, and `#75`, and the existing 1,285-gap
-profiling handoff. It records confirmed present defects,
-design invariants, per-phase acceptance gates, test
-commands, commit boundaries, and explicitly deferred UX
-and provider work.

ai/prompt-io/opencode/20260722T001357Z_16f3cd6f_prompt_io.md

@@ -1,46 +0,0 @@
----
-model: gpt-5.6-sol
-provider: openai
-service: opencode
-session: ses_0799212ebffe42arY96czXn89F
-timestamp: 2026-07-22T00:13:57Z
-git_ref: 16f3cd6f
-scope: code
-substantive: true
-raw_file: 20260722T001357Z_16f3cd6f_prompt_io.raw.md
----
-
-## Prompt
-
-Continue the gap-checker and history-backfiller repair plan. Start
-with deterministic analysis characterization, then fix canonical
-sorting, dedupe-derived columns, positive gap detection, and null
-grouping.
-
-During review, the user challenged the replacement null-grouping
-algorithm because its initial tests did not assert exact endpoints.
-The user explicitly required retaining detailed, human-oriented
-algorithm documentation such as the existing ASCII segment map.
-
-## Response summary
-
-Added deterministic analysis regressions and corrected normalization
-so sorting and duplicate resolution precede derived-column
-calculation. Unified null grouping for NumPy and Polars inputs,
-documented its exact inclusive-boundary contract with an adapted
-visual map, and verified all small null layouts against an independent
-state-machine oracle. Retained sorted smart-dedupe output.
-
-## Files changed
-
-- `piker/tsp/_anal.py` - canonical gap and documented null grouping
-- `piker/tsp/_dedupe_smart.py` - sorted duplicate resolution
-- `tests/test_tsp_analysis.py` - exact and exhaustive regressions
-- `ai/prompt-io/opencode/20260722T001357Z_16f3cd6f_prompt_io.raw.md`
-  - unedited response record
-- `ai/prompt-io/opencode/20260722T001357Z_16f3cd6f_prompt_io.md`
-  - provenance metadata and response summary
-
-## Human edits
-
-None - generated changes have not yet been edited by the human.

ai/prompt-io/opencode/20260722T001357Z_16f3cd6f_prompt_io.raw.md

@@ -1,40 +0,0 @@
----
-model: gpt-5.6-sol
-provider: openai
-service: opencode
-timestamp: 2026-07-22T00:13:57Z
-git_ref: 16f3cd6f
-diff_cmd: git diff HEAD~1..HEAD
----
-
-> `git diff HEAD~1..HEAD -- piker/tsp/_anal.py`
-
-Generated canonical gap and null-segment normalization. Derived
-datetime and delta columns are recomputed after final sorting and
-duplicate resolution, and negative deltas no longer become ordinary
-gaps.
-
-NumPy and Polars null grouping share one implementation while
-retaining the human-oriented visual map of contiguous zero runs. The
-documented contract defines absolute inclusive endpoints, margin
-expansion, frame boundary clamping, contiguous-index validation, and
-requested-column selection.
-
-> `git diff HEAD~1..HEAD -- piker/tsp/_dedupe_smart.py`
-
-Adjusted smart dedupe to honor sorted output and resolve duplicate
-timestamps without retaining stale order.
-
-> `git diff HEAD~1..HEAD -- tests/test_tsp_analysis.py`
-
-Added deterministic synthetic OHLCV coverage for sorted dedupe, fresh
-derived columns, and positive gap detection. Null grouping is checked
-against readable endpoint examples and an independent state-machine
-oracle over every layout through six rows, three margin widths, and
-both NumPy and Polars inputs.
-
-Verification generated with the patch:
-
-`timeout -k 5 30 python -m pytest -q tests/test_tsp_analysis.py`
-
-Result: 11 passed.

ai/prompt-io/opencode/20260722T001358Z_16f3cd6f_prompt_io.md

@@ -1,36 +0,0 @@
----
-model: gpt-5.6-sol
-provider: openai
-service: opencode
-session: ses_0799212ebffe42arY96czXn89F
-timestamp: 2026-07-22T00:13:58Z
-git_ref: 16f3cd6f
-scope: code
-substantive: true
-raw_file: 20260722T001358Z_16f3cd6f_prompt_io.raw.md
----
-
-## Prompt
-
-Continue the structured backfill phase after the analysis regressions.
-Characterize provider exhaustion and genuinely empty frames, then remove
-completion paths that can hang after child-task exit.
-
-## Response summary
-
-Removed orphan-prone completion events from the structured nursery flow
-and checked empty provider frames before indexing timestamp endpoints.
-Added deterministic timeout-bounded regressions for both exit paths.
-
-## Files changed
-
-- `piker/tsp/_history.py` - structured completion and empty-frame fixes
-- `tests/test_history_backfill.py` - deterministic exit regressions
-- `ai/prompt-io/opencode/20260722T001358Z_16f3cd6f_prompt_io.raw.md`
-  - unedited response record
-- `ai/prompt-io/opencode/20260722T001358Z_16f3cd6f_prompt_io.md`
-  - provenance metadata and response summary
-
-## Human edits
-
-None - generated changes have not yet been edited by the human.

ai/prompt-io/opencode/20260722T001358Z_16f3cd6f_prompt_io.raw.md

@@ -1,25 +0,0 @@
----
-model: gpt-5.6-sol
-provider: openai
-service: opencode
-timestamp: 2026-07-22T00:13:58Z
-git_ref: 16f3cd6f
-diff_cmd: git diff HEAD~1..HEAD
----
-
-> `git diff HEAD~1..HEAD -- piker/tsp/_history.py`
-
-Removed redundant completion events whose consumers waited only after
-their owning nurseries had joined. Added empty provider-frame handling
-before timestamp endpoint indexing so provider exhaustion exits cleanly.
-
-> `git diff HEAD~1..HEAD -- tests/test_history_backfill.py`
-
-Added bounded, actor-free regressions for `DataUnavailable` and empty
-provider-frame completion.
-
-Verification generated with the patch:
-
-`timeout -k 5 60 python -m pytest -q tests/test_history_backfill.py`
-
-Result: 2 passed.

ai/prompt-io/opencode/20260722T024133Z_3a4f6737_prompt_io.md

@@ -1,39 +0,0 @@
----
-model: gpt-5.6-sol
-provider: openai
-service: opencode
-session: ses_0799212ebffe42arY96czXn89F
-timestamp: 2026-07-22T02:41:33Z
-git_ref: 3a4f6737
-scope: code
-substantive: true
-raw_file: 20260722T024133Z_3a4f6737_prompt_io.raw.md
----
-
-## Prompt
-
-The user challenged the analysis commit message's claim that only
-positive sample-step gaps are detected and asked whether out-of-order
-timestamps were simply ignored. After confirming the missing separate
-reporting path, the user instructed the agent to implement the fix.
-
-## Response summary
-
-Separated missing-history gaps from timestamp-ordering errors without
-changing the existing `dedupe()` return shape. Added a structured
-detector and made normalization warn before sorting removes evidence of
-duplicates or reversed timestamps.
-
-## Files changed
-
-- `piker/tsp/_anal.py` - ordering detector and dedupe reporting
-- `piker/tsp/__init__.py` - public detector export
-- `tests/test_tsp_analysis.py` - exact ordering-error regression
-- `ai/prompt-io/opencode/20260722T024133Z_3a4f6737_prompt_io.raw.md`
-  - unedited response record
-- `ai/prompt-io/opencode/20260722T024133Z_3a4f6737_prompt_io.md`
-  - provenance metadata and response summary
-
-## Human edits
-
-None - generated changes have not yet been edited by the human.

ai/prompt-io/opencode/20260722T024133Z_3a4f6737_prompt_io.raw.md

@@ -1,30 +0,0 @@
----
-model: gpt-5.6-sol
-provider: openai
-service: opencode
-timestamp: 2026-07-22T02:41:33Z
-git_ref: 3a4f6737
-diff_cmd: git diff HEAD~1..HEAD
----
-
-> `git diff HEAD~1..HEAD -- piker/tsp/_anal.py`
-
-Added `detect_time_ordering_errors()` to return non-positive timestamp
-steps as structured rows. `dedupe()` now warns with those rows before
-sorting and duplicate removal erase the original ordering evidence.
-
-> `git diff HEAD~1..HEAD -- piker/tsp/__init__.py`
-
-Exported the ordering detector through the public TSP package.
-
-> `git diff HEAD~1..HEAD -- tests/test_tsp_analysis.py`
-
-Changed the negative-delta regression to verify exact predecessor,
-timestamp, and delta values, confirm exclusion from missing-history
-gaps, and require explicit normalization reporting.
-
-Verification generated with the patch:
-
-- `tests/test_tsp_analysis.py`: 11 passed
-- Ruff: passed
-- `git diff --check`: passed

ai/prompt-io/opencode/20260722T024134Z_3a4f6737_prompt_io.md

@@ -1,43 +0,0 @@
----
-model: gpt-5.6-sol
-provider: openai
-service: opencode
-session: ses_0799212ebffe42arY96czXn89F
-timestamp: 2026-07-22T02:41:34Z
-git_ref: 3a4f6737
-scope: code
-substantive: true
-raw_file: 20260722T024134Z_3a4f6737_prompt_io.raw.md
----
-
-## Prompt
-
-The user asked why the in-progress `piker/storage` changes were omitted
-from the earlier commit plan. After receiving an inventory of unfinished
-Phase 3 requirements, the user instructed the agent to complete the work.
-
-## Response summary
-
-Finished non-destructive, atomic NativeDB updates and integrated them
-with history startup and reverse backfill. Expanded tests through two
-adversarial review rounds that caught and corrected cross-client lock
-blocking, stale temp indexing, discontinuous persisted indexes, startup
-snapshot races, missing latest-frame persistence, notification wedges,
-and mutable/truncated SHM persistence.
-
-## Files changed
-
-- `piker/storage/__init__.py` - incremental storage contract
-- `piker/storage/nativedb.py` - validated atomic merge persistence
-- `piker/storage/marketstore/__init__.py` - non-duplicate update mode
-- `piker/tsp/_history.py` - provider-delta and startup persistence
-- `tests/test_storage_nativedb.py` - NativeDB durability regressions
-- `tests/test_history_backfill.py` - persistence lifecycle regressions
-- `ai/prompt-io/opencode/20260722T024134Z_3a4f6737_prompt_io.raw.md`
-  - unedited response record
-- `ai/prompt-io/opencode/20260722T024134Z_3a4f6737_prompt_io.md`
-  - provenance metadata and response summary
-
-## Human edits
-
-None - generated changes have not yet been edited by the human.

ai/prompt-io/opencode/20260722T024134Z_3a4f6737_prompt_io.raw.md

@@ -1,43 +0,0 @@
----
-model: gpt-5.6-sol
-provider: openai
-service: opencode
-timestamp: 2026-07-22T02:41:34Z
-git_ref: 3a4f6737
-diff_cmd: git diff HEAD~1..HEAD
----
-
-> `git diff HEAD~1..HEAD -- piker/storage/__init__.py piker/storage/nativedb.py piker/storage/marketstore/__init__.py`
-
-Added an incremental storage update contract. NativeDB now validates
-canonical numeric OHLCV data, serializes writers both actor-locally and
-through cancellable filesystem locks, merges with incoming-wins seam
-resolution, regenerates contiguous indexes, reopens and validates a
-same-directory temporary parquet, fsyncs it, atomically replaces the
-target, fsyncs the directory, and publishes cache only after replacement.
-
-NativeDB indexing ignores lock and crash-left temporary files.
-MarketStore uses its non-duplicate write mode for incremental updates.
-
-> `git diff HEAD~1..HEAD -- piker/tsp/_history.py`
-
-Separated full provider deltas from capacity-truncated SHM pushes.
-Backfill persists before SHM mutation, publishes bounded notifications,
-and delays latest-frame publication until the concurrent pre-update
-storage snapshot completes.
-
-> `git diff HEAD~1..HEAD -- tests/test_storage_nativedb.py tests/test_history_backfill.py`
-
-Added deterministic coverage for merge preservation, conflict policy,
-atomic failures, reopened-candidate validation, cache consistency,
-schema and timestamp invariants, contiguous index regeneration,
-separate series, stale temp indexing, cancellable file-lock contention,
-full provider-delta persistence, latest-frame ordering, and bounded
-notification teardown.
-
-Verification generated with the patch:
-
-- deterministic analysis/backfill/storage set: 31 passed
-- Ruff: passed
-- `git diff --check`: passed
-- final adversarial review: no findings

ai/prompt-io/opencode/20260724T220156Z_43d7a3ed_prompt_io.md

@@ -1,32 +0,0 @@
----
-model: openai/gpt-5.6-sol
-service: opencode
-session: ses_092a25886ffeT0nrIrdkMEcgJK
-timestamp: 2026-07-24T22:01:56Z
-git_ref: flake_update
-scope: config
-substantive: true
-raw_file: 20260724T220156Z_43d7a3ed_prompt_io.raw.md
----
-
-## Prompt
-
-After committing Piker's provider-neutral deployment, the user asked to
-extract Piker-specialized `/run-tests` content from the historical
-`wkt/run_tests_skill` implementation into the new deployment.
-
-## Response summary
-
-Create a repository-owned `test-harness-reference.md` for the canonical Run
-base. Revalidate historical commands and assumptions against current pytest,
-uv/Nix environments, tests, fixtures, and Tractor runtime behavior, retaining
-only current Piker-specific guidance.
-
-## Files changed
-
-- `.claude/skills/run-tests/test-harness-reference.md` - define Piker's safe
-  commands, opt-in boundaries, mappings, known outcomes, and runtime notes.
-
-## Human edits
-
-None at generation time; pending human review.

ai/prompt-io/opencode/20260724T220156Z_43d7a3ed_prompt_io.raw.md

@@ -1,32 +0,0 @@
----
-model: openai/gpt-5.6-sol
-service: opencode
-timestamp: 2026-07-24T22:01:56Z
-git_ref: flake_update
-diff_cmd: git diff HEAD~1..HEAD
----
-
-The user asked to extract useful Piker-specific `/run-tests` behavior from
-`wkt/run_tests_skill` after committing the provider-neutral deployment.
-
-> `git diff HEAD~1..HEAD -- .claude/skills/run-tests/test-harness-reference.md`
-
-Generate a repository-owned harness reference rather than restoring the old
-standalone `SKILL.md`. Preserve verified Piker-specific environment, command,
-scope, fixture, test-layout, change-mapping, known-outcome, and Tractor-runtime
-details while leaving shared execution and cleanup policy in the canonical
-`ai.skillz` base.
-
-Repository inspection found that the historical skill needs correction:
-
-- the comments-only `pytest.ini` shadows `pyproject.toml` pytest options, so
-  `-p no:xonsh` must be explicit;
-- no marker-based offline suite exists, and several live/GUI/container tests
-  are unsafe defaults;
-- skipped Docker and Questrade modules can fail during import before marks;
-- actor timeout retry remains relevant only for the exact timed-out command;
-- `py313` and frozen no-sync `uv run` are the current local environment paths.
-
-Validate with checkout import resolution, safe collection, and deterministic
-watchlist/accounting tests. Do not run live broker, GUI, container,
-credentialed, or user-config-dependent cases during harness verification.

ai/prompt-io/opencode/20260727T204711Z_84a6d47b_prompt_io.md

@@ -1,40 +0,0 @@
----
-model: gpt-5.6-sol
-provider: openai
-service: opencode
-session: unavailable
-timestamp: 2026-07-27T20:47:11Z
-git_ref: 84a6d47b
-scope: code
-substantive: true
-raw_file: 20260727T204711Z_84a6d47b_prompt_io.raw.md
----
-
-## Prompt
-
-The user identified that `backfiller_deep_fixes` introduced persistent
-`.parquet.lock` files, questioned whether Parquet or Arrow already handled
-writer coordination, and clarified that piker's `tractor` structured
-concurrency design already provides single-writer ownership through the
-persistent `datad` feed task. The user then asked to open the backfiller
-worktree and resolve the redundant locking.
-
-## Response summary
-
-Removed redundant actor-local and filesystem locking while preserving
-atomic crash-safe Parquet replacement. Replaced the synthetic contention
-test with regressions for the actual cross-branch failure: writes create
-no lock sidecars, and legacy lock artifacts are ignored during indexing.
-
-## Files changed
-
-- `piker/storage/nativedb.py` - rely on `datad` writer ownership
-- `tests/test_storage_nativedb.py` - lock-sidecar regressions
-- `ai/prompt-io/opencode/20260727T204711Z_84a6d47b_prompt_io.raw.md`
-  - unedited response record
-- `ai/prompt-io/opencode/20260727T204711Z_84a6d47b_prompt_io.md`
-  - provenance metadata and response summary
-
-## Human edits
-
-None - generated changes have not been edited by the human.

ai/prompt-io/opencode/20260727T204711Z_84a6d47b_prompt_io.raw.md

@@ -1,30 +0,0 @@
----
-model: gpt-5.6-sol
-provider: openai
-service: opencode
-timestamp: 2026-07-27T20:47:11Z
-git_ref: 84a6d47b
-diff_cmd: git diff HEAD~1..HEAD
----
-
-> `git diff HEAD~1..HEAD -- piker/storage/nativedb.py`
-
-Removed actor-local and filesystem writer locks from NativeDB. The
-persistent `datad` feed task already owns writes for each series, while
-its timeframe children write distinct files. NativeDB retains validated
-temporary writes, file and directory synchronization, and atomic target
-replacement for crash durability. The exact `.parquet` suffix filter is
-retained so legacy lock sidecars are harmless.
-
-> `git diff HEAD~1..HEAD -- tests/test_storage_nativedb.py`
-
-Removed the synthetic cross-client lock-contention test. Added regression
-coverage proving replacement and incremental writes create no lock
-sidecars. Extended indexing coverage with the legacy
-`.parquet.lock` artifact that caused upstream `flake_update` to crash.
-
-Verification generated with the patch:
-
-- NativeDB and history regression set: 20 passed
-- Ruff: passed
-- `git diff --check`: passed

ai/prompt-io/opencode/20260727T212454Z_689df816_prompt_io.md

@@ -1,42 +0,0 @@
----
-model: gpt-5.6-sol
-provider: openai
-service: opencode
-session: ses_0799212ebffe42arY96czXn89F
-timestamp: 2026-07-27T21:24:54Z
-git_ref: 689df816
-scope: code
-substantive: true
-raw_file: 20260727T212454Z_689df816_prompt_io.raw.md
----
-
-## Prompt
-
-The user reran the known IB FQME on `backfiller_deep_fixes` and reported
-that `publish_latest_frame()` still failed because NativeDB required an
-`index` absent from IB's provider frame. The user called out the missed
-tabular schema boundary and requested an actual end-to-end test suite
-rather than storage fakes.
-
-## Response summary
-
-Repaired the IB-provider-to-NativeDB boundary by canonicalizing durable
-fields and mapping reloads into provider-specific SHM buffers. Added a
-local actor/SHM/Parquet integration regression covering first startup and
-restart, plus focused conversion and timestamp-coercion tests. Two review
-rounds found and resolved restart hydration and fractional-time hazards.
-
-## Files changed
-
-- `piker/storage/nativedb.py` - canonical storage and SHM field mapping
-- `piker/tsp/_anal.py` - name-based Polars-to-NumPy conversion
-- `tests/test_history_backfill.py` - actor/SHM/Parquet integration test
-- `tests/test_storage_nativedb.py` - schema conversion regressions
-- `ai/prompt-io/opencode/20260727T212454Z_689df816_prompt_io.raw.md`
-  - unedited response record
-- `ai/prompt-io/opencode/20260727T212454Z_689df816_prompt_io.md`
-  - provenance metadata and response summary
-
-## Human edits
-
-None - generated changes have not been edited by the human.

ai/prompt-io/opencode/20260727T212454Z_689df816_prompt_io.raw.md

@@ -1,34 +0,0 @@
----
-model: gpt-5.6-sol
-provider: openai
-service: opencode
-timestamp: 2026-07-27T21:24:54Z
-git_ref: 689df816
-diff_cmd: git diff HEAD~1..HEAD
----
-
-> `git diff HEAD~1..HEAD -- piker/storage/nativedb.py piker/tsp/_anal.py`
-
-Normalized index-less provider frames to NativeDB's canonical names,
-order, and dtypes before merge and publication. Provider-only columns
-are excluded from durable files, derived indexes are regenerated, and
-fractional timestamps are rejected before integer conversion. Added a
-canonical field map for hydrating provider-specific SHM buffers while
-leaving provider-only fields at their defaults. Changed Polars-to-NumPy
-conversion to select columns by field name rather than position.
-
-> `git diff HEAD~1..HEAD -- tests/test_history_backfill.py tests/test_storage_nativedb.py`
-
-Added deterministic coverage using IB's actual provider dtype and a real
-local `tractor` root actor, IB-typed shared-memory buffers, NativeDB,
-Parquet persistence, first-start publication, and restart hydration.
-Added focused regressions for reordered/extra Polars columns and
-fractional timestamp rejection.
-
-Verification generated with the patch:
-
-- NativeDB and history integration set: 23 passed
-- Ruff: passed
-- `git diff --check`: passed
-- second adversarial review: no findings
-- no live gateway or network tests run

ai/prompt-io/opencode/20260727T215906Z_689df816_prompt_io.md

@@ -1,41 +0,0 @@
----
-model: gpt-5.6-sol
-provider: openai
-service: opencode
-session: ses_0799212ebffe42arY96czXn89F
-timestamp: 2026-07-27T21:59:06Z
-git_ref: 689df816
-scope: code
-substantive: true
-raw_file: 20260727T215906Z_689df816_prompt_io.raw.md
----
-
-## Prompt
-
-After the actor/SHM/Parquet integration test exposed a stale SHM triplet,
-the user requested leak-cleaner machinery in the pytest harness using
-Tractor's existing ownership and lifetime patterns.
-
-## Response summary
-
-Added creator-scoped SHM leak tracking to pytest without scanning
-`/dev/shm` or treating attachments as owned. The fixture cooperates with
-normal Tractor teardown, verifies POSIX object identity before fallback
-cleanup, restores process-local token state, and fails after cleaning a
-leak. Regressions cover the exact pre-actor-registration allocation window,
-positional creators, and external attachment safety.
-
-## Files changed
-
-- `tests/conftest.py` - creator-scoped SHM leak tracking fixture
-- `tests/test_shm_cleanup.py` - ownership and failure-window regressions
-- `.claude/skills/run-tests/test-harness-reference.md`
-  - document current-process SHM cleanup behavior
-- `ai/prompt-io/opencode/20260727T215906Z_689df816_prompt_io.raw.md`
-  - unedited response record
-- `ai/prompt-io/opencode/20260727T215906Z_689df816_prompt_io.md`
-  - provenance metadata and response summary
-
-## Human edits
-
-None - generated changes have not been edited by the human.

ai/prompt-io/opencode/20260727T215906Z_689df816_prompt_io.raw.md

@@ -1,25 +0,0 @@
----
-model: gpt-5.6-sol
-provider: openai
-service: opencode
-timestamp: 2026-07-27T21:59:06Z
-git_ref: 689df816
-diff_cmd: git diff HEAD~1..HEAD
----
-
-> `git diff HEAD~1..HEAD -- tests/conftest.py tests/test_shm_cleanup.py .claude/skills/run-tests/test-harness-reference.md`
-
-Added function-scoped SHM ownership tracking around Tractor's test-process
-factory. The fixture records only successful creators, removes ownership
-records during normal Tractor teardown, verifies POSIX object identity
-before fallback unlink, restores the token-cache baseline, and fails after
-cleaning any leak. Regressions reproduce allocation failure before actor
-lifetime registration and prove external attachments remain untouched.
-
-Verification generated with the patch:
-
-- SHM cleanup regressions: 2 passed
-- combined SHM, NativeDB, and history set: 25 passed
-- Ruff: passed
-- `git diff --check`: passed
-- final adversarial review: no findings

ai/prompt-io/opencode/20260728T003247Z_ed85721c_prompt_io.md

@@ -1,42 +0,0 @@
----
-model: gpt-5.6-sol
-provider: openai
-service: opencode
-session: 24fb9765-a550-4570-8350-f0fc9b7e17db
-timestamp: 2026-07-28T00:32:47Z
-git_ref: ed85721c
-scope: code
-substantive: true
-raw_file: 20260728T003247Z_ed85721c_prompt_io.raw.md
----
-
-## Prompt
-
-Build a safe xonsh workflow for qualifying NativeDB and reverse-backfill
-correctness against a known gappy IB chart, beginning with
-`mnq.cme.20260918`. Cover baseline, fresh, restart, append, and expansion
-testing through implementation, adversarial review, and verification.
-
-## Response summary
-
-Implemented a private disposable xonsh qualification workflow. Evidence is
-tied to a fixed case, report checksum, snapshot checksum, worktree import,
-and phase metadata. Added explicit path and symlink safety, source
-replacement handling, xonsh child environments, and lifecycle preservation
-comparisons.
-
-## Files changed
-
-- `snippets/nativedb_backfill_audit.xsh` - disposable qualification helpers
-- `docs/manual_backfill_qualification.rst` - operator lifecycle runbook
-- `tests/test_backfill_audit_snippet.py` - xonsh and comparison regressions
-- `.claude/skills/run-tests/test-harness-reference.md`
-  - qualification test-harness guidance
-- `ai/prompt-io/opencode/20260728T003247Z_ed85721c_prompt_io.raw.md`
-  - unedited response record
-- `ai/prompt-io/opencode/20260728T003247Z_ed85721c_prompt_io.md`
-  - provenance metadata and response summary
-
-## Human edits
-
-None - generated changes have not been edited by the human.

ai/prompt-io/opencode/20260728T003247Z_ed85721c_prompt_io.raw.md

@@ -1,32 +0,0 @@
----
-model: gpt-5.6-sol
-provider: openai
-service: opencode
-timestamp: 2026-07-28T00:32:47Z
-git_ref: ed85721c
-diff_cmd: git diff HEAD~1..HEAD
----
-
-> `git diff HEAD~1..HEAD -- snippets/nativedb_backfill_audit.xsh docs/manual_backfill_qualification.rst tests/test_backfill_audit_snippet.py .claude/skills/run-tests/test-harness-reference.md`
-
-Added sourceable xonsh helpers that create a private disposable root, bind
-child processes to the reviewed worktree, capture checksum-bound phase
-evidence, safely seed and clear test history, and compare lifecycle
-preservation.
-
-The workflow rejects path traversal, pre-existing symlinks, output
-collisions, mismatched case/report/snapshot evidence, and same-phase
-comparison. The accompanying runbook covers known-bad replay, fresh
-backfill, restart, append, gap triage, and expansion cases.
-
-Verification generated with the patch:
-
-- xonsh qualification helper regressions: passed
-- xonsh sourceability: passed
-- Ruff: passed
-- `git diff --check`: passed
-
-Adversarial review drove additional evidence binding, worktree executable,
-environment propagation, source replacement, and descriptor-level safety
-fixes. Residual concurrent mutation risk is stated explicitly in the private
-disposable-root runbook.

ai/prompt-io/opencode/20260728T005034Z_ed85721c_prompt_io.md

@@ -1,40 +0,0 @@
----
-model: gpt-5.6-sol
-provider: openai
-service: opencode
-session: 24fb9765-a550-4570-8350-f0fc9b7e17db
-timestamp: 2026-07-28T00:50:34Z
-git_ref: ed85721c
-scope: code
-substantive: true
-raw_file: 20260728T005034Z_ed85721c_prompt_io.raw.md
----
-
-## Prompt
-
-Add reusable read-only tooling to qualify raw NativeDB Parquet against a
-known gappy IB chart, beginning with `mnq.cme.20260918`. Preserve malformed
-evidence and distinguish structural validity from unclassified positive
-gaps without opening or mutating the storage runtime.
-
-## Response summary
-
-Implemented raw Parquet audit reporting with human and JSON output, bounded
-cadence details, exact integer arithmetic, strict status, collision-safe
-snapshot capture, and source path protections. Regressions cover malformed
-schema and values, timestamp precision, gap arithmetic, corrupt bytes,
-copy failures, symlinks, traversal, and read-only CLI behavior.
-
-## Files changed
-
-- `piker/storage/_audit.py` - raw Parquet audit and snapshot implementation
-- `piker/storage/cli.py` - `piker store audit` command
-- `tests/test_storage_audit.py` - audit and CLI regressions
-- `ai/prompt-io/opencode/20260728T005034Z_ed85721c_prompt_io.raw.md`
-  - unedited response record
-- `ai/prompt-io/opencode/20260728T005034Z_ed85721c_prompt_io.md`
-  - provenance metadata and response summary
-
-## Human edits
-
-None - generated changes have not been edited by the human.

ai/prompt-io/opencode/20260728T005034Z_ed85721c_prompt_io.raw.md

@@ -1,27 +0,0 @@
----
-model: gpt-5.6-sol
-provider: openai
-service: opencode
-timestamp: 2026-07-28T00:50:34Z
-git_ref: ed85721c
-diff_cmd: git diff HEAD~1..HEAD
----
-
-> `git diff HEAD~1..HEAD -- piker/storage/_audit.py piker/storage/cli.py tests/test_storage_audit.py`
-
-Added a raw, read-only NativeDB Parquet audit with human and JSON output,
-bounded cadence evidence, exact integer arithmetic, pre-parse snapshot
-capture, and strict qualification status.
-
-The command rejects unsafe FQME paths, symlinked sources and outputs,
-destination collisions, non-regular source nodes, malformed canonical
-schema, non-finite values, invalid timestamps, noncanonical indexes, and
-sub-period cadence. Copy-failure cleanup preserves source bytes while
-completed corrupt snapshots remain available for diagnosis.
-
-Verification generated with the patch:
-
-- storage audit and CLI regressions: passed
-- Ruff: passed
-- `git diff --check`: passed
-- final adversarial review: no findings

ai/prompt-io/opencode/20260728T232718Z_ed85721c_prompt_io.md

@@ -1,39 +0,0 @@
----
-model: gpt-5.6-sol
-provider: openai
-service: opencode
-session: 24fb9765-a550-4570-8350-f0fc9b7e17db
-timestamp: 2026-07-28T23:27:18Z
-git_ref: ed85721c
-scope: code
-substantive: true
-raw_file: 20260728T232718Z_ed85721c_prompt_io.raw.md
----
-
-## Prompt
-
-Diagnose the live `pikerd` failure where backfill publication raised
-`ValueError: OHLCV timestamps must be finite and positive` while merging the
-known gappy MNQ NativeDB baseline with fresh IB history. Determine whether to
-commit the pending audit stack or iterate first.
-
-## Response summary
-
-Traced the primary failure through `publish_latest_frame()` to
-`NativeStorageClient.update_ohlcv()`. The incoming frame was valid, but the
-canonicalized persisted baseline retained its epoch-zero row and caused the
-merged frame to fail validation. Added persisted-row repair before merge and
-a regression using the observed legacy schema characteristics.
-
-## Files changed
-
-- `piker/storage/nativedb.py` - drop non-positive persisted rows during merge
-- `tests/test_storage_nativedb.py` - live MNQ failure-shape regression
-- `ai/prompt-io/opencode/20260728T232718Z_ed85721c_prompt_io.raw.md`
-  - unedited response record
-- `ai/prompt-io/opencode/20260728T232718Z_ed85721c_prompt_io.md`
-  - provenance metadata and response summary
-
-## Human edits
-
-None - generated changes have not been edited by the human.

ai/prompt-io/opencode/20260728T232718Z_ed85721c_prompt_io.raw.md

@@ -1,27 +0,0 @@
----
-model: gpt-5.6-sol
-provider: openai
-service: opencode
-timestamp: 2026-07-28T23:27:18Z
-git_ref: ed85721c
-diff_cmd: git diff HEAD~1..HEAD
----
-
-> `git diff HEAD~1..HEAD -- piker/storage/nativedb.py tests/test_storage_nativedb.py`
-
-Fixed incremental NativeDB updates against legacy persisted history with
-non-positive timestamps. Incoming provider frames remain strictly validated,
-while merge repair discards invalid persisted rows before concat, dedupe,
-ordering, canonical-index rewrite, and atomic publication.
-
-The regression reproduces the observed MNQ baseline shape: one epoch-zero
-row, noncanonical absolute indexes, extra derived columns, and a valid fresh
-IB frame. It proves the invalid row is removed while all valid old and new
-bars survive with canonical indexes.
-
-Verification generated with the patch:
-
-- NativeDB regressions: 18 passed
-- history backfill regressions: 6 passed
-- focused Ruff: passed
-- `git diff --check`: passed

ai/prompt-io/opencode/20260729T034106Z_922a5df8_prompt_io.md

@@ -1,40 +0,0 @@
----
-model: gpt-5.6-sol
-provider: openai
-service: opencode
-session: 24fb9765-a550-4570-8350-f0fc9b7e17db
-timestamp: 2026-07-29T03:41:06Z
-git_ref: 922a5df8
-scope: code
-substantive: true
-raw_file: 20260729T034106Z_922a5df8_prompt_io.raw.md
----
-
-## Prompt
-
-Diagnose the live IB reverse-fill failure where
-`reqHistoricalDataAsync()` raised API error 10314 because its end date,
-time, or timezone was invalid.
-
-## Response summary
-
-Traced the failure to `Client.bars()`, where an unfinished EST conversion
-left the original datetime for `ib_async` to serialize as
-`YYYYMMDD HH:MM:SS UTC`. Converted aware boundaries explicitly to UTC and
-passed IB's accepted `YYYYMMDD-HH:MM:SS` string while retaining a blank
-latest-request boundary. Added exact request-argument regressions.
-
-## Files changed
-
-- `piker/brokers/ib/api.py` - explicit UTC-dash history boundary format
-- `tests/test_ib_history.py` - IB request-format regressions
-- `.claude/skills/run-tests/test-harness-reference.md`
-  - deterministic IB history test mapping
-- `ai/prompt-io/opencode/20260729T034106Z_922a5df8_prompt_io.raw.md`
-  - unedited response record
-- `ai/prompt-io/opencode/20260729T034106Z_922a5df8_prompt_io.md`
-  - provenance metadata and response summary
-
-## Human edits
-
-None - generated changes have not been edited by the human.

ai/prompt-io/opencode/20260729T034106Z_922a5df8_prompt_io.raw.md

@@ -1,25 +0,0 @@
----
-model: gpt-5.6-sol
-provider: openai
-service: opencode
-timestamp: 2026-07-29T03:41:06Z
-git_ref: 922a5df8
-diff_cmd: git diff HEAD~1..HEAD
----
-
-> `git diff HEAD~1..HEAD -- piker/brokers/ib/api.py tests/test_ib_history.py .claude/skills/run-tests/test-harness-reference.md`
-
-Fixed IB reverse-history requests to serialize aware end boundaries with
-IB's explicit UTC-dash syntax, `YYYYMMDD-HH:MM:SS`. This bypasses
-`ib_async`'s trailing-`UTC` representation, which the live Gateway rejected
-with API error 10314, and removes the unfinished EST comparison breakpoint.
-
-The regression captures the exact `reqHistoricalDataAsync()` argument for
-latest, stdlib-aware, Pendulum UTC, and America/New_York inputs.
-
-Verification generated with the patch:
-
-- IB history request regressions: 4 passed
-- focused Ruff: passed
-- `git diff --check`: passed
-- adversarial review: no findings

ai/prompt-io/opencode/20260729T041723Z_ce33deb6_prompt_io.md

@@ -1,40 +0,0 @@
----
-model: gpt-5.6-sol
-provider: openai
-service: opencode
-session: 24fb9765-a550-4570-8350-f0fc9b7e17db
-timestamp: 2026-07-29T04:17:23Z
-git_ref: ce33deb6
-scope: code
-substantive: true
-raw_file: 20260729T041723Z_ce33deb6_prompt_io.raw.md
----
-
-## Prompt
-
-Diagnose why the chart still showed a zero first OHLCV row after the
-epoch-zero timestamp repair, causing auto-y-ranging to include zero.
-
-## Response summary
-
-Audited live 60s and 1s NativeDB files and found one exact 60s first-slot
-sentinel at epoch 60 with a completely zero OHLCV payload. Added narrowly
-scoped read and merge repair, matching incoming validation, sentinel-only
-startup handling, canonical reindexing, and explicit audit classification.
-Preserved modern all-zero bars as unclassified evidence rather than assuming
-all zero-priced instruments are corrupt.
-
-## Files changed
-
-- `piker/storage/nativedb.py` - first-slot sentinel repair and validation
-- `piker/storage/_audit.py` - sentinel classification and zero-row evidence
-- `tests/test_storage_nativedb.py` - hydration, merge, and boundary regressions
-- `tests/test_storage_audit.py` - sentinel and modern-zero audit regressions
-- `ai/prompt-io/opencode/20260729T041723Z_ce33deb6_prompt_io.raw.md`
-  - unedited response record
-- `ai/prompt-io/opencode/20260729T041723Z_ce33deb6_prompt_io.md`
-  - provenance metadata and response summary
-
-## Human edits
-
-None - generated changes have not been edited by the human.

ai/prompt-io/opencode/20260729T041723Z_ce33deb6_prompt_io.raw.md

@@ -1,31 +0,0 @@
----
-model: gpt-5.6-sol
-provider: openai
-service: opencode
-timestamp: 2026-07-29T04:17:23Z
-git_ref: ce33deb6
-diff_cmd: git diff HEAD~1..HEAD
----
-
-> `git diff HEAD~1..HEAD -- piker/storage/nativedb.py piker/storage/_audit.py tests/test_storage_nativedb.py tests/test_storage_audit.py`
-
-Fixed the remaining legacy first-slot sentinel observed in live MNQ 60s
-history: `time=60` with zero open, high, low, close, and volume. NativeDB now
-repairs the exact sentinel before read hydration and merge, reindexes the
-filtered view, and persists the repaired frame on the next valid update.
-
-The repair remains narrowly scoped. Modern all-zero OHLC bars stay valid for
-spreads or synthetic instruments, null-containing malformed rows still fail
-validation, new writes can not recreate the first-slot sentinel, and a file
-containing only sentinels loads as no history so fresh backfill can proceed.
-
-The raw audit reports total all-zero price rows, classifies first-slot
-sentinels as structural violations, and leaves later all-zero rows visible as
-unclassified warnings.
-
-Verification generated with the patch:
-
-- NativeDB, audit, history, xonsh, and IB regressions: 61 passed
-- focused Ruff: passed
-- `git diff --check`: passed
-- final adversarial review: no findings

ai/prompt-io/opencode/20260729T185406Z_d359b1cb_prompt_io.md

@@ -1,34 +0,0 @@
----
-model: openai/gpt-5.6-sol
-service: opencode
-session: 6409e3bb-1a9a-4ada-b6c4-c554115143e9
-timestamp: 2026-07-29T18:54:06Z
-git_ref: d359b1cb
-scope: code
-substantive: true
-raw_file: 20260729T185406Z_d359b1cb_prompt_io.raw.md
----
-
-## Prompt
-
-The user reported that QQQ 60-second zeros appeared as a flat line after
-reload, clarified that the outstanding concern was the prior `ldshm`
-failure, and instructed the agent to implement the more focused fix.
-
-## Response summary
-
-Changed `ldshm` to classify invalid SHM snapshots before downstream
-analysis. Cadence is inferred only from observed finite positive steps;
-buffers containing non-finite or non-positive timestamps, or lacking a
-positive cadence, are skipped rather than partially replacement-written
-or reloaded. Added deterministic helper and command-level regressions.
-
-## Files changed
-
-- `piker/storage/cli.py` - inspect cadence and reject invalid SHM snapshots before mutation.
-- `tests/test_ldshm.py` - cover QQQ-like zeros, corrupt timestamps, cadence selection, degenerate buffers, and mutation prevention.
-- `.claude/skills/run-tests/test-harness-reference.md` - register the deterministic regression target.
-
-## Human edits
-
-None - generated output remains uncommitted.

ai/prompt-io/opencode/20260729T185406Z_d359b1cb_prompt_io.raw.md

@@ -1,58 +0,0 @@
----
-model: openai/gpt-5.6-sol
-service: opencode
-timestamp: 2026-07-29T18:54:06Z
-git_ref: d359b1cb
-diff_cmd: git diff HEAD~1..HEAD
----
-
-The user clarified that the QQQ 60-second chart now displayed a flat
-forward-filled segment, then asked whether the previously reported
-`piker store ldshm` failure had been fixed. After being told it had not,
-the user instructed: "ok but i read above you have a more focussed fix
-todo now? so have at it!"
-
-The implementation focused on the exact failure boundary. It does not
-filter invalid rows into a partial replacement and does not mutate live
-SHM. Instead, `ldshm` inspects one captured Polars snapshot, infers its
-cadence from observed finite positive timestamp steps, and skips the
-entire buffer before analysis, persistence, or reload when any timestamp
-is non-finite or non-positive.
-
-> `git diff HEAD~1..HEAD -- piker/storage/cli.py`
-
-Generated `_shm_period_and_invalid_count()` and integrated its result in
-`ldshm`. The helper selects the most frequent observed positive step,
-using the smallest observed step as the deterministic tie-break. Short
-or cadence-degenerate buffers are skipped. Invalid snapshots are logged
-and cannot reach deduplication, NativeDB replacement, or SHM reload.
-
-> `git diff HEAD~1..HEAD -- tests/test_ldshm.py`
-
-Generated deterministic regressions for the QQQ-shaped interior zero
-run, observed-step cadence selection, all invalid timestamp classes,
-short and degenerate buffers, and command-level prevention of dedupe,
-storage writes, and reload. The command-level test uses an immutable
-Polars snapshot and failing spies, without live SHM mutation or timing.
-
-> `git diff HEAD~1..HEAD -- .claude/skills/run-tests/test-harness-reference.md`
-
-Added the deterministic `ldshm` regression target and its change-to-test
-mapping to the repository test harness reference.
-
-Verification output:
-
-```text
-..........................................................               [100%]
-58 passed in 0.70s
-```
-
-`git diff --check` passed. Ruff was unavailable in the worktree's
-existing environment. Three adversarial review passes found and drove
-fixes for partial-replacement data loss, invented median timeframes,
-mixed SHM observations, and pre-guard null-analysis failures. The final
-review reported no remaining findings.
-
-The patch intentionally does not claim that `ldshm` is safe to run as a
-general concurrent repair writer. It handles invalid live snapshots by
-refusing mutation rather than weakening NativeDB validation.

ai/prompt-io/opencode/20260729T204046Z_ad6c560f_prompt_io.md

@@ -1,37 +0,0 @@
----
-model: openai/gpt-5.6-sol
-service: opencode
-session: 253ade33-07ba-45b5-8839-be35d17ec164
-timestamp: 2026-07-29T20:40:46Z
-git_ref: ad6c560f
-scope: code
-substantive: true
-raw_file: 20260729T204046Z_ad6c560f_prompt_io.raw.md
----
-
-## Prompt
-
-The user reported that the QQQ 1-second chart still contained a gap after
-the `ldshm` fix, asked the agent to inspect it, and authorized continued
-implementation after a short pause and shutdown of the live workspace.
-
-## Response summary
-
-Diagnosed a 653-row active RT SHM null reservation caused by inclusive IB
-reverse-query overlaps and startup repair orchestration. Prevented extra
-physical SHM overlap rows, serialized and bounded IB history recovery,
-made blank/error responses terminate, and constrained synthetic repair to
-valid interior timestamp boundaries. Added deterministic concurrency and
-data-layout regressions.
-
-## Files changed
-
-- `piker/brokers/ib/feed.py` - bound and serialize IB history reset/retry behavior.
-- `piker/tsp/_history.py` - prevent SHM overlap skew and safely finalize null repair.
-- `tests/test_ib_history.py` - cover blank, cancellation, reset, and pacing interleavings.
-- `tests/test_history_backfill.py` - cover overlap and null-repair invariants.
-- `.claude/skills/run-tests/test-harness-reference.md` - register deterministic coverage.
-
-## Human edits
-
-None - generated output remains uncommitted.

ai/prompt-io/opencode/20260729T204046Z_ad6c560f_prompt_io.raw.md

@@ -1,72 +0,0 @@
----
-model: openai/gpt-5.6-sol
-service: opencode
-timestamp: 2026-07-29T20:40:46Z
-git_ref: ad6c560f
-diff_cmd: git diff HEAD~1..HEAD
----
-
-The user reported that QQQ still displayed a gap on the 1-second chart
-after the focused `ldshm` fix and asked the agent to inspect it. The user
-then paused and resumed implementation while shutting down the live piker
-trading workspace.
-
-Read-only live inspection found the active datad generation held 653
-contiguous zero-time rows in QQQ's 1-second RT SHM, while durable Parquet
-contained no zero rows. The run occupied absolute indexes 168148 through
-168800 between timestamps 1785348398 and 1785349051. The stale actor
-generation and active 60-second history were separately identified.
-
-> `git diff HEAD~1..HEAD -- piker/tsp/_history.py`
-
-Generated history changes that exclude each inclusive reverse-query
-endpoint already present in SHM while retaining the full provider frame
-for NativeDB merge. Startup null repair now runs after reverse backfill,
-uses bounded UI notifications, handles provider no-data without an
-interactive pause, fills only exact interior zero groups, and refuses
-synthetic timestamps unless both valid boundaries establish the expected
-cadence.
-
-> `git diff HEAD~1..HEAD -- piker/brokers/ib/feed.py`
-
-Generated IB history reset changes that make blank responses complete,
-bound repeated cancellation and reset failures per request, serialize
-timeout and pacing farm resets with one actor-global Trio lock, carry
-explicit reset results, and coordinate pacing handoff with queued timeout
-waiters.
-
-> `git diff HEAD~1..HEAD -- tests/test_history_backfill.py`
-
-Generated regressions for inclusive endpoint exclusion, full NativeDB
-provider deltas, strict synthetic gap boundaries, empty-provider local
-fallback, bounded notifications, and preservation of valid margin rows.
-
-> `git diff HEAD~1..HEAD -- tests/test_ib_history.py`
-
-Generated deterministic regressions for blank IB responses, cancellation
-limits, stalled reset cancellation, pacing reset failure limits,
-cross-request reset serialization, timeout-to-pacing handoff, and
-successful handoff timeout restart.
-
-> `git diff HEAD~1..HEAD -- .claude/skills/run-tests/test-harness-reference.md`
-
-Registered the deterministic history-backfill target and expanded the IB
-history mapping to include `feed.py`.
-
-Verification output:
-
-```text
-........................................................................ [ 93%]
-.....                                                                    [100%]
-77 passed in 2.12s
-```
-
-`git diff --check` passed. Multiple adversarial review rounds identified
-and drove fixes for unbounded IB empty/cancellation/reset paths, stale
-reset ownership, actor-global failure poisoning, concurrent farm resets,
-double-counted pacing handoffs, physical endpoint overlap, and unsafe
-synthetic timestamp boundaries. Final review reported no findings.
-
-Live validation was intentionally deferred because the user shut down the
-trading workspace. A fresh datad/chart startup is required to regenerate
-SHM and verify that QQQ no longer contains the 653-row null run.

ai/prompt-io/opencode/20260731T165215Z_0846cbd4_prompt_io.md

@@ -1,41 +0,0 @@
----
-model: openai/gpt-5.6-sol
-service: opencode
-session: ses_0799212ebffe42arY96czXn89F
-timestamp: 2026-07-31T16:52:15Z
-git_ref: 0846cbd4
-scope: code
-substantive: true
-raw_file: 20260731T165215Z_0846cbd4_prompt_io.raw.md
----
-
-## Prompt
-
-Evaluate and update `piker store` so bare groups and endpoints immediately
-show help; prepare for eventual optional fuzzy selection in `anal` and
-`ldshm`; aggregate recent time-series diagnostics into CLI endpoints. The
-user selected the safe-foundation scope.
-
-## Response summary
-
-Added immediate help UX, exact NativeDB and SHM identities, read-only
-`series` and `shm` diagnostics, selector-ready exact SHM names, and robust
-discovery/attachment. Fixed related `ldshm` option and no-match failures.
-Deferred optional `fzf` and mutation-default changes.
-
-## Files changed
-
-- `piker/storage/cli.py` - help UX and read-only series/SHM diagnostics.
-- `piker/storage/nativedb.py` - canonical compound series identities.
-- `piker/tsp/_history.py` - exact SHM discovery and dtype resolution.
-- `piker/data/_sharedmem.py` - attach-only existing-segment helper.
-- `piker/tsp/__init__.py` - SHM identity helper exports.
-- `tests/test_store_cli.py` - command UX and endpoint regressions.
-- `tests/test_ldshm.py` - SHM diagnostics and operational regressions.
-- `tests/test_storage_nativedb.py` - series parsing/index regressions.
-- `docs/manual_backfill_qualification.rst` - triage command documentation.
-- `.claude/skills/run-tests/test-harness-reference.md` - test mapping.
-
-## Human edits
-
-None - generated output remains uncommitted.

ai/prompt-io/opencode/20260731T165215Z_0846cbd4_prompt_io.raw.md

@@ -1,93 +0,0 @@
----
-model: openai/gpt-5.6-sol
-service: opencode
-timestamp: 2026-07-31T16:52:15Z
-git_ref: 0846cbd4
-diff_cmd: git diff HEAD~1..HEAD
----
-
-The user requested an evaluation and possible update of `piker store`,
-including immediate help output when a group or endpoint is invoked with
-no arguments. The user also proposed eventual optional `fzf` selection for
-`anal` and `ldshm`, and authorized aggregating the recent time-series and
-SHM debugging work into CLI endpoints. After reviewing three scopes, the
-user selected the recommended safe foundation rather than implementing
-interactive fuzzy selection or changing mutation defaults immediately.
-
-> `git diff HEAD~1..HEAD -- piker/storage/cli.py`
-
-Generated no-argument help behavior for the `store` group and every
-endpoint. Added read-only `series` and `shm` commands, JSON and Rich output,
-exact `--shm-name` plumbing, SHM cadence/invalid/order/gap summaries, and
-safe corrupt-epoch formatting. Fixed existing `ldshm` no-write/no-reload
-control flow and no-match debugger behavior.
-
-> `git diff HEAD~1..HEAD -- piker/storage/nativedb.py`
-
-Generated canonical `NativeSeriesRef` parsing and deterministic listing.
-Changed the internal index to preserve `(fqme, period)` identities while
-keeping unique-FQME compatibility listing and cardinality. Hardened
-malformed, sidecar, Unicode-period, symlink, and traversal handling.
-
-> `git diff HEAD~1..HEAD -- piker/tsp/_history.py`
-
-Generated exact `ShmBufferRef` parsing and stable discovery, replacing
-unsafe substring globbing. Added dtype resolution that mirrors history
-allocation, exact-ref attachment, disappearance handling, and optional
-exact-name iteration.
-
-> `git diff HEAD~1..HEAD -- piker/data/_sharedmem.py`
-
-Generated attach-only SHM access which never creates a segment when a
-discovered object disappears.
-
-> `git diff HEAD~1..HEAD -- piker/tsp/__init__.py`
-
-Exported selector-ready SHM identity, parsing, listing, and dtype helpers.
-
-> `git diff HEAD~1..HEAD -- tests/test_store_cli.py`
-
-Generated command-level help, exact durable-series JSON, read-only SHM JSON,
-and unknown exact-name regressions.
-
-> `git diff HEAD~1..HEAD -- tests/test_ldshm.py`
-
-Generated SHM identity, summary, ordering, dtype, forged-name, disabled
-mutation-flag, and gap-markup regressions.
-
-> `git diff HEAD~1..HEAD -- tests/test_storage_nativedb.py`
-
-Generated compound index, canonical parser, malformed filename, and dotted
-FQME regressions.
-
-> `git diff HEAD~1..HEAD -- docs/manual_backfill_qualification.rst`
-
-Documented `store series` and `store shm` as separate read-only durable and
-live-memory triage layers.
-
-> `git diff HEAD~1..HEAD -- .claude/skills/run-tests/test-harness-reference.md`
-
-Registered the deterministic storage CLI UX target and mapping.
-
-Verification output:
-
-```text
-........................................................................ [ 71%]
-.............................                                            [100%]
-101 passed in 1.72s
-```
-
-Real `piker store` and `piker store audit` invocations printed complete
-help without `--help`, exiting with Click's normal no-args help status 2.
-`git diff --check` passed. Ruff was unavailable in the existing worktree
-environment.
-
-Adversarial review drove fixes for provider dtype reconstruction, observed
-versus expected cadence, exact SHM grammar, discovery races, Typer direct
-defaults, malformed NativeDB periods, compound cardinality compatibility,
-undefined `ldshm` reload payloads, no-match debugger hangs, ordering
-corruption reporting, and dtype precedence. Final review found no issues.
-
-Optional external `fzf` or built-in numbered selection remains deferred.
-The exact series and SHM identities added here are the intended foundation
-for that later patch.

ai/prompt-io/opencode/20260801T015321Z_74ea2152_prompt_io.md

@@ -1,39 +0,0 @@
----
-model: openai/gpt-5.6-sol
-service: opencode
-session: ses_0799212ebffe42arY96czXn89F
-timestamp: 2026-08-01T01:53:21Z
-git_ref: 74ea2152
-scope: code
-substantive: true
-raw_file: 20260801T015321Z_74ea2152_prompt_io.raw.md
----
-
-## Prompt
-
-Make `piker store ls` default to all NativeDB series as a compact counterpart
-to detailed `series`, with sensible no-flag output and a visible help
-description. Consolidate SHM inspection and repair under canonical `shm`,
-because attachment to volatile segments is implied, and make inspection the
-safe default while retaining explicit persistence and reload controls.
-
-## Response summary
-
-Replaced backend-oriented `ls` with compact NativeDB discovery and made both
-listing commands default to all series. Consolidated SHM behavior under a
-read-only-by-default `shm` command, guarded explicit repair options, updated
-executable references and docs, and added command regressions.
-
-## Files changed
-
-- `piker/storage/cli.py` - listing defaults and unified SHM command modes.
-- `tests/test_store_cli.py` - listing, help, and SHM mode regressions.
-- `tests/test_ldshm.py` - explicit persistence-path regressions.
-- `docs/manual_backfill_qualification.rst` - revised safe CLI workflow.
-- `snippets/claude_debug_helper.py` - explicit SHM persistence commands.
-- `.claude/skills/run-tests/test-harness-reference.md` - safety and test map.
-- `.claude/skills/piker-slang/examples.md` - canonical command example.
-
-## Human edits
-
-None - generated output incorporates the user's design refinements.

ai/prompt-io/opencode/20260801T015321Z_74ea2152_prompt_io.raw.md

@@ -1,78 +0,0 @@
----
-model: openai/gpt-5.6-sol
-service: opencode
-timestamp: 2026-08-01T01:53:21Z
-git_ref: 74ea2152
-diff_cmd: git diff HEAD~1..HEAD
----
-
-The user requested command-surface refinements after reviewing the new
-storage CLI. Bare `ls` should default to all series and act as a compact
-form of detailed `series`; commands should produce sensible default output
-without requiring trigger flags; and `ls` needed a parent-help description.
-The user first proposed folding read-only SHM inspection into `ldshm -r`,
-then clarified that `shm` is the better canonical command because attaching
-to volatile segments is implied by the resource name.
-
-The resulting design uses `piker store ls` as a compact NativeDB FQME and
-period view, while `piker store series` reports each exact sub-series with
-size and path details. Both default to all records and accept an optional
-FQME substring.
-
-`piker store shm FQME` is read-only by default. Persistence and reload enter
-the prior repair workflow only through explicit `--write-parquet` and
-`--reload-parquet-to-shm` flags. Reload requires persistence, and read-only
-formatting flags are rejected in repair mode instead of being ignored.
-
-> `git diff HEAD~1..HEAD -- piker/storage/cli.py`
-
-Generated shared NativeDB payload collection, compact and detailed output,
-numeric period ordering, default-all semantics, the canonical `shm` command,
-safe mode dispatch, explicit repair validation, and debugger propagation in
-read-only mode.
-
-> `git diff HEAD~1..HEAD -- tests/test_store_cli.py`
-
-Generated regressions for no-flag listing, parent-help descriptions, numeric
-period display, default read-only SHM JSON, reload dependencies, and repair
-versus read-only option conflicts.
-
-> `git diff HEAD~1..HEAD -- tests/test_ldshm.py`
-
-Adjusted persistence regressions for the renamed command and explicit repair
-entry while preserving the historical failure rationale.
-
-> `git diff HEAD~1..HEAD -- docs/manual_backfill_qualification.rst`
-
-Documented compact versus detailed NativeDB discovery, default read-only SHM
-inspection, and the explicitly mutating repair invocation.
-
-> `git diff HEAD~1..HEAD -- snippets/claude_debug_helper.py`
-
-Updated executable debug helper commands to select explicit SHM persistence.
-
-> `git diff HEAD~1..HEAD -- .claude/skills/run-tests/test-harness-reference.md`
-
-Updated operational safety guidance and SHM regression mappings.
-
-> `git diff HEAD~1..HEAD -- .claude/skills/piker-slang/examples.md`
-
-Updated the command example to the canonical `shm` spelling.
-
-Verification output:
-
-```text
-........................................................................ [ 69%]
-................................                                         [100%]
-104 passed in 5.77s
-```
-
-Real command-help rendering was checked for the parent group, `ls`, `series`,
-and `shm`. Real read-only `ls` and `series --json` output was also exercised
-against the configured NativeDB. `git diff --check` passed.
-
-Adversarial review identified and drove fixes for lexicographic period
-ordering, stale runbook safety wording, ignored mode-specific options, and an
-explicit-default `--max-gaps 10` validation bypass. Final review found no
-remaining issues. Real OS-backed SHM attachment and successful repair/reload
-remain integration-level validation concerns.

ai/prompt-io/opencode/20260805T195001Z_f0f3c806_prompt_io.md

@@ -1,33 +0,0 @@
----
-model: openai/gpt-5.6-sol
-service: opencode
-session: ses_0799212ebffe42arY96czXn89F
-timestamp: 2026-08-05T19:50:01Z
-git_ref: f0f3c806
-scope: code
-substantive: true
-raw_file: 20260805T195001Z_f0f3c806_prompt_io.raw.md
----
-
-## Prompt
-
-Inspect the completed NVDA 60s SHM backfill before workspace restart, explain
-the persistent zero region, and continue fixing the indexing logic so sparse
-IB history does not leave physical null reservations.
-
-## Response summary
-
-Captured a 42,720-row live SHM null span whose valid margins shared the same
-timestamp. Replaced wall-clock row reservation with actual provider-boundary
-placement, fixed stale SHM first-index, endpoint insertion and field-map
-semantics, published the final storage prepend, and added a full multi-frame
-`tsdb_backfill()` regression using a real `ShmArray`.
-
-## Files changed
-
-- `piker/tsp/_history.py` - contiguous provider and NativeDB prepends.
-- `tests/test_history_backfill.py` - sparse multi-frame SHM regression.
-
-## Human edits
-
-None - generated output incorporates the user's live observations.

ai/prompt-io/opencode/20260805T195001Z_f0f3c806_prompt_io.raw.md

@@ -1,71 +0,0 @@
----
-model: openai/gpt-5.6-sol
-service: opencode
-timestamp: 2026-08-05T19:50:01Z
-git_ref: f0f3c806
-diff_cmd: git diff HEAD~1..HEAD
----
-
-The user reported that a completed NVDA 60s backfill left a large zero
-region in live SHM and asked for immediate inspection before restarting the
-workspace. The captured buffer contained 42,720 invalid rows at absolute
-indexes `3065895..3108614`. The valid rows on both sides had the identical
-timestamp `1781271360`, proving the region was an erroneous physical-index
-reservation rather than a real temporal gap.
-
-The user then asked the agent to continue fixing the backfill logic.
-
-> `git diff HEAD~1..HEAD -- piker/tsp/_history.py`
-
-Removed wall-clock-based SHM reservation from `tsdb_backfill()`. Reverse
-provider frames now extend the visible `ShmArray._first` boundary on every
-prepend, beginning exactly at the current first index after inclusive query
-endpoints are removed. Added `_prepend_tsdb_history()` to trim storage rows
-against the actual earliest provider timestamp and prepend them directly
-adjacent only after reverse provider retrieval completes.
-
-This preserves sparse venue history as ordinal samples rather than treating
-every elapsed calendar period as a physical SHM row. It also prevents the
-old `_first + 1` insertion point from overwriting the earliest published bar.
-
-> `git diff HEAD~1..HEAD -- tests/test_history_backfill.py`
-
-Added a full `tsdb_backfill()` regression modeled on the live NVDA failure.
-It uses a real `ShmArray`, NativeDB's field map, two inclusive reverse-provider
-frames, and overlapping stored history. Assertions prove the complete
-timestamp sequence is contiguous and lossless, contains no non-positive
-timestamps, sends complete provider frames to durable storage, and notifies
-chart/FSP consumers after the final storage prepend.
-
-Live evidence captured before workspace restart:
-
-```text
-rows 1575296 first_index 1568233 last_index 3143528
-invalid_count 42720
-left boundary
-[(3065893, 1781271300, 204.03, 204.82)
- (3065894, 1781271360, 204.81, 204.72)
- (3065895,          0,   0.  ,   0.  )]
-right boundary
-[(3108614,          0,   0.  ,   0.  )
- (3108615, 1781271360, 204.81, 205.44)
- (3108616, 1781271420, 205.46, 205.48)]
-```
-
-Verification output:
-
-```text
-........................................................................ [ 68%]
-.................................                                        [100%]
-105 passed in 4.80s
-```
-
-Adversarial review caught stale `_first` semantics and an off-by-one
-insertion point in the initial patch. The first live restart then exposed an
-undefined `time_key` local at the new `tsdb_backfill()` call site, which
-crashed `datad.ib` and propagated to the chart as `RemoteActorError`. The
-helper now derives the source timestamp field from its field map, and the
-test enters the full enclosing lifecycle so the original `NameError` can not
-recur unnoticed. A final review also drove explicit publication after stored
-history is prepended and deterministic final-stage synchronization. Final
-review found no remaining issues; another live restart remains pending.

ai/prompt-io/opencode/20260811T205826Z_06d5ea5d_prompt_io.md

@@ -1,36 +0,0 @@
----
-model: openai/gpt-5.6-sol
-service: opencode
-session: ses_0799212ebffe42arY96czXn89F
-timestamp: 2026-08-11T20:58:26Z
-git_ref: 06d5ea5d
-scope: code
-substantive: true
-raw_file: 20260811T205826Z_06d5ea5d_prompt_io.raw.md
----
-
-## Prompt
-
-Diagnose and fix an IB live-order failure where lazy NVDA contract
-qualification raised `tractor.trionics.Lagged` after the order reached
-`brokerd.ib`. Determine whether recent history changes or cancelled symbol
-searches caused the stale state.
-
-## Response summary
-
-Traced the failure to competing consumers of a one-slot Trio broadcast, not
-the recent SHM change or stale asyncio client state. Replaced broadcast
-fan-out with one response/status demultiplexer, retained request-ID
-correlation, handled cancellation and teardown, documented the task/channel
-ownership model, warned on diagnostic metadata eviction, and added
-deterministic proxy regressions.
-
-## Files changed
-
-- `piker/brokers/ib/api.py` - sole-reader IB proxy response routing.
-- `tests/test_ib_method_proxy.py` - status, concurrency, cancellation and EOF regressions.
-- `.claude/skills/run-tests/test-harness-reference.md` - proxy test mapping.
-
-## Human edits
-
-None - generated output follows the user's live failure diagnosis.

ai/prompt-io/opencode/20260811T205826Z_06d5ea5d_prompt_io.raw.md

@@ -1,66 +0,0 @@
----
-model: openai/gpt-5.6-sol
-service: opencode
-timestamp: 2026-08-11T20:58:26Z
-git_ref: 06d5ea5d
-diff_cmd: git diff HEAD~1..HEAD
----
-
-The user reported that certain IB order entries might have broken and
-provided a live failure for a first NVDA sell order. EMS delivered the
-request to `brokerd.ib`, but lazy `get_mkt_info('nvda.nasdaq')` contract
-qualification failed in `MethodProxy._run_method()` with:
-
-```text
-tractor.trionics._broadcast.Lagged:
-Task `piker.brokers.ib.broker.handle_order_requests` overrun and
-dropped `0` values
-```
-
-The user suspected a cancelled symbol-search task might have left stale
-asyncio state. Cancellation can leave an orphaned method response, but the
-observed failure was stale Trio broadcast state. `open_client_proxy()`
-broadcast a one-slot `LinkedTaskChannel` to an always-running IB status relay
-and an idle method receiver. Status traffic advanced only the relay, so the
-first method call for an uncached symbol entered an already-lagged receiver.
-Orders for eagerly cached contracts bypassed this path.
-
-The immediately preceding sparse-history commit did not modify IB order,
-symbol, or proxy code; it exposed no direct dependency on this failure.
-
-> `git diff HEAD~1..HEAD -- piker/brokers/ib/api.py`
-
-Generated a sole-reader message demultiplexer for each IB client proxy.
-`relay_client_proxy_messages()` owns every asyncio-to-Trio receive, handles
-status/error traffic, and dispatches `mid`-tagged responses into per-call
-`MethodProxy` wait slots. Cancelled calls remove their pending slot, causing
-late responses to be logged and dropped without skewing later calls. Clean
-channel EOF is accepted during normal proxy teardown.
-
-Follow-up review restored the masked reconnect TODO and per-message case
-rationale, documented the one-reader/many-writer task hierarchy and separate
-per-proxy channels, and added a warning with request method metadata when the
-bounded unresolved-request diagnostic table evicts its oldest entry.
-
-> `git diff HEAD~1..HEAD -- tests/test_ib_method_proxy.py`
-
-Generated deterministic regressions for idle status traffic before first
-qualification, two concurrent calls with reverse-order responses, cancelled
-calls with late responses followed by a successful call, and graceful
-channel EOF. Every synchronization wait is timeout-bounded.
-
-> `git diff HEAD~1..HEAD -- .claude/skills/run-tests/test-harness-reference.md`
-
-Registered the deterministic IB method-proxy target and source mapping.
-
-Verification output:
-
-```text
-........................                                                 [100%]
-25 passed, 4 warnings in 3.44s
-```
-
-The warnings are existing Tractor/Trio deprecations from the EMS test.
-Adversarial review drove a clean-EOF guard and bounded polling. Final review
-found no actionable issues. Live IB order validation remains pending and
-requires restarting `brokerd.ib`.

ai/prompt-io/opencode/20260814T032716Z_2aa66c2c_prompt_io.md

@@ -1,43 +0,0 @@
----
-model: openai/gpt-5.6-sol
-service: opencode
-session: unavailable
-timestamp: 2026-08-14T03:27:16Z
-git_ref: 2aa66c2c
-scope: code
-substantive: true
-raw_file: 20260814T032716Z_2aa66c2c_prompt_io.raw.md
----
-
-## Prompt
-
-Prototype chart-actor-owned gap overlays from the current backfiller tip,
-with actor-local startup and UI toggling. Preserve remote annotation IPC and
-make the gap-specific path strict and `msgspec`-typed.
-
-## Response summary
-
-Adds a chart-local controller and vectorized SHM detector, starts historical
-gap overlays with display state, toggles the focused timeframe through the
-existing input loop and exposes the same renderer through a strict Tractor
-payload context. Keeps all legacy remote annotation commands and separates
-local graphics from each remote context's teardown ownership.
-
-## Files changed
-
-- `piker/ui/_gaps.py` - typed messages, detection and local controller.
-- `piker/ui/_remote_ctl.py` - IPC clients and chart-actor endpoints.
-- `piker/ui/_display.py` - actor-local startup integration.
-- `piker/ui/_interaction.py` - focused `Ctrl+G` toggle.
-- `piker/ui/_widget.py` - controller reference.
-- `piker/ui/_annotate.py` - safe FQME-aware repositioning.
-- `piker/tsp/_annotate.py` - typed annotation-client reference.
-- `piker/storage/cli.py` - renamed annotation client context.
-- `tests/test_gap_overlays.py` - detector and schema regression.
-- `tests/test_ldshm.py` - renamed annotation-client test seams.
-- `plans/opencode/chart-local-gap-overlays.md` - architecture and landing plan.
-
-## Human edits
-
-None - the work remains an uncommitted disposable-worktree prototype for
-human review and live chart qualification.

ai/prompt-io/opencode/20260814T032716Z_2aa66c2c_prompt_io.raw.md

@@ -1,72 +0,0 @@
----
-model: openai/gpt-5.6-sol
-service: opencode
-timestamp: 2026-08-14T03:27:16Z
-git_ref: 2aa66c2c
-diff_cmd: git diff HEAD~1..HEAD
----
-
-## Prompt
-
-The user asked what work had been done so far, then authorized autonomous
-work while AFK. The target was to scheme and prototype an actor-local gap
-annotator from the `backfiller_deep_fixes` tip without mutating existing
-branches. The user then clarified twice that the annotation IPC API must
-remain available and should become more rigorous and `msgspec`-typed rather
-than being removed.
-
-## Generated code
-
-> `git diff HEAD~1..HEAD -- piker/ui/_gaps.py`
-
-Adds tagged gap request/reply structs, render-ready gap specs, vectorized
-OHLCV timestamp-gap detection and chart-local overlay ownership.
-
-> `git diff HEAD~1..HEAD -- piker/ui/_remote_ctl.py`
-
-Adds strict Tractor gap IPC, `AnnotClient` stream mapping/locking and
-owner-scoped endpoint cleanup while retaining the legacy annotation endpoint.
-
-> `git diff HEAD~1..HEAD -- piker/ui/_display.py`
-
-Starts the local 60-second overlay after display-state registration.
-
-> `git diff HEAD~1..HEAD -- piker/ui/_interaction.py`
-
-Routes `Ctrl+G` from the focused chart's existing async input loop directly
-to the local controller.
-
-> `git diff HEAD~1..HEAD -- piker/ui/_widget.py`
-
-Stores the chart actor's local controller reference.
-
-> `git diff HEAD~1..HEAD -- piker/ui/_annotate.py`
-
-Keeps redraw repositioning isolated by FQME and synchronizes arrow geometry.
-
-> `git diff HEAD~1..HEAD -- piker/tsp/_annotate.py piker/storage/cli.py`
-
-Renames the remote annotation control type and context manager to the
-client-oriented `AnnotClient` / `open_annot_client` interface.
-
-> `git diff HEAD~1..HEAD -- tests/test_gap_overlays.py`
-
-Adds deterministic detector, directional payload-schema, typed error and
-msgpack roundtrip regressions.
-
-> `git diff HEAD~1..HEAD -- tests/test_ldshm.py`
-
-Updates storage CLI test seams for the renamed annotation client context.
-
-## Generated architecture output
-
-> `git diff HEAD~1..HEAD -- plans/opencode/chart-local-gap-overlays.md`
-
-Documents ownership, startup, UI, strict IPC, refresh and branch landing
-semantics.
-
-## Verification output
-
-Source resolution points to this worktree. The targeted regression passes,
-Ruff reports no changed-file findings, `compileall` succeeds, `git diff
---check` succeeds and Tractor's strict gap payload decoder initializes.

ai/prompt-io/opencode/20260817T222036Z_f65df6a1_prompt_io.md

@@ -1,39 +0,0 @@
----
-model: openai/gpt-5.6-sol
-service: opencode
-session: unavailable
-timestamp: 2026-08-17T22:20:36Z
-git_ref: f65df6a1
-scope: code
-substantive: true
-raw_file: 20260817T222036Z_f65df6a1_prompt_io.raw.md
----
-
-## Prompt
-
-Add end-to-end coverage for chart-local gap annotations in the existing
-gap-overlay worktree without disturbing concurrent manual backfiller testing
-in the root checkout.
-
-## Response summary
-
-Adds real offscreen Qt and real Tractor actor integration coverage. The tests
-drive startup registration, PyQtGraph insertion/removal, focused Ctrl-G input,
-history-prepend repositioning, duplicate-FQME chart identity, typed endpoint
-exchange and cancellation with stale-response recovery. Defects exposed by
-those tests are fixed in manager cleanup, Qt geometry notification and PyQt6
-event routing.
-
-## Files changed
-
-- `piker/ui/_gaps.py` - correct PyQtGraph ownership and bounds behavior.
-- `piker/ui/_annotate.py` - notify Qt before reposition geometry changes.
-- `piker/ui/_display.py` - expose deterministic startup registration.
-- `piker/ui/_interaction.py` - focus toggles and PyQt6 event registration.
-- `tests/test_gap_overlays.py` - real Qt and Tractor integration coverage.
-- `plans/opencode/chart-local-gap-overlays.md` - update coverage boundaries.
-- `plans/opencode/chart-local-gap-overlays.summary.md` - update test totals.
-
-## Human edits
-
-None - the changes remain uncommitted for human review.

ai/prompt-io/opencode/20260817T222036Z_f65df6a1_prompt_io.raw.md

@@ -1,59 +0,0 @@
----
-model: openai/gpt-5.6-sol
-service: opencode
-timestamp: 2026-08-17T22:20:36Z
-git_ref: f65df6a1
-diff_cmd: git diff f65df6a1..HEAD
----
-
-## Prompt
-
-The user asked whether the chart-local gap annotation feature had complete
-end-to-end coverage, then requested implementation in the existing gap-overlay
-worktree while manual NativeDB/backfiller qualification continued in the root
-checkout. The user initially declined a tests-only Prompt-IO entry. Real Qt
-and Tractor integration tests then exposed production lifecycle and PyQt6
-event-routing defects, making the resulting patch substantive code work and
-requiring this full provenance entry.
-
-## Generated code
-
-> `git diff HEAD~1..HEAD -- piker/ui/_gaps.py`
-
-Removes manager-owned graphics through their `PlotItem`, retains owning-plot
-registrations, and excludes pixel-sized gap arrows from automatic data bounds.
-
-> `git diff HEAD~1..HEAD -- piker/ui/_annotate.py`
-
-Moves Qt geometry-change notification ahead of rectangle and arrow mutation so
-scene spatial indexing follows history-prepend repositioning.
-
-> `git diff HEAD~1..HEAD -- piker/ui/_display.py`
-
-Extracts `_register_gap_overlays()` as the deterministic chart startup seam
-which registers display states and renders default historical gap layers.
-
-> `git diff HEAD~1..HEAD -- piker/ui/_interaction.py`
-
-Adds focused-chart toggle routing, updates the gap keyboard path to PyQt6 event
-enums, and keeps unrelated cached chart states untouched.
-
-> `git diff HEAD~1..HEAD -- tests/test_gap_overlays.py`
-
-Adds offscreen real-Qt manager, startup, keyboard, cached-chart and reposition
-coverage plus a real Tractor actor endpoint/client dialog with deterministic
-cancellation and stale-response recovery.
-
-## Generated architecture output
-
-> `git diff HEAD~1..HEAD -- plans/opencode/chart-local-gap-overlays.md plans/opencode/chart-local-gap-overlays.summary.md`
-
-Records the automated integration boundary and remaining broker-backed manual
-qualification requirements.
-
-## Verification output
-
-The source-isolated non-actor and related suites pass 24 tests with one actor
-case deselected. The isolated actor case passes with one upstream Tractor/Trio
-deprecation warning. Ruff, compileall, import resolution and whitespace checks
-pass. Independent review reports no remaining P1/P2 findings.

ai/prompt-io/opencode/20260818T064340Z_09ddcf50_prompt_io.md

@@ -1,38 +0,0 @@
----
-model: openai/gpt-5.6-sol
-service: opencode
-session: unavailable
-timestamp: 2026-08-18T06:43:40Z
-git_ref: 09ddcf50
-scope: docs
-substantive: true
-raw_file: 20260818T064340Z_09ddcf50_prompt_io.raw.md
----
-
-## Prompt
-
-Audit all development branches and worktrees, consolidate duplicate and
-rebased histories into the smallest subsystem-scoped PRs, preserve useful
-chronology, identify stale refs, omit obsolete AI-skill changes, and recover
-any useful abandoned work before proposing an optimal route into `main`.
-
-## Response summary
-
-Adds a repository-wide landing plan grounded in exact branch ancestry,
-patch-equivalence, worktree state, and object recovery. The plan separates
-independent fixes from the runtime/datad/backfill dependency chain, reduces
-the DPI branch swarm to one five-commit series, records deferred provider
-work, and proposes staged cleanup only after replacement PRs land.
-
-## Files changed
-
-- `plans/opencode/repo-land-plan.md` - record the audited landing sequence,
-  exact patch sources, blockers, worktree constraints, and cleanup groups.
-- `ai/prompt-io/opencode/20260818T064340Z_09ddcf50_prompt_io.raw.md` - retain
-  the unedited prompt and generated-output summary.
-- `ai/prompt-io/opencode/20260818T064340Z_09ddcf50_prompt_io.md` - record
-  structured provenance for the generated documentation.
-
-## Human edits
-
-None - the changes remain uncommitted for human review.

ai/prompt-io/opencode/20260818T064340Z_09ddcf50_prompt_io.raw.md

@@ -1,45 +0,0 @@
----
-model: openai/gpt-5.6-sol
-service: opencode
-timestamp: 2026-08-18T06:43:40Z
-git_ref: 09ddcf50
-diff_cmd: git diff HEAD~1..HEAD
----
-
-## Prompt
-
-The user asked OpenCode to act as repository supervisor and audit every
-development branch and worktree for an optimal route into `main`. The audit
-needed to identify duplicate and rebased histories, especially the DPI/font
-branches, reconstruct subsystem-scoped patch sets, preserve chronology,
-propose stale-ref removal, exclude obsolete AI-skill changes, and recover any
-useful abandoned commits.
-
-The user later checked that the long-running audit was still active. OpenCode
-confirmed progress and completed the worktree, branch, patch-ID, reflog,
-stash, and unreachable-object sweep before writing the plan.
-
-## Generated documentation
-
-> `git diff HEAD~1..HEAD -- plans/opencode/repo-land-plan.md`
-
-Creates a repository-wide landing plan based on `main` and `gitea/main` at
-`1abd47609b161c4b9fd460274dc9123755211436`. It records the audit scope,
-integration rules, dependency-ordered PR lanes, exact source commits, branch
-cleanup groups, worktree ownership constraints, deferred histories, and the
-abandoned-object recovery result.
-
-The plan reduces aggregate development histories into independent fixes, a
-five-commit DPI series, graphics/gap foundations, runtime and service ports,
-the datad/brokerd split, post-datad concurrency repairs, NativeDB/backfill
-work, chart-local overlays, provider follow-ups, and build/tooling changes.
-It explicitly excludes generated skill deployment, prompt records from
-product patches, editable dependency pins, and plan-only commits.
-
-## Audit output
-
-The read-only audit covers seven worktrees, 168 recent unmerged refs, eight
-stash entries, patch-equivalent rebases, and 483 unreachable commits. It finds
-no recovery-worthy application patch that exists only in a stash or
-unreachable object. Current forge status remains a separate refresh step
-because this pass uses local and cached repository state.

ai/prompt-io/opencode/20260826T222600Z_5f631e19_prompt_io.md

@@ -1,46 +0,0 @@
----
-model: openai/gpt-5.6-sol
-service: opencode
-session: unavailable
-timestamp: 2026-08-26T22:26:00Z
-git_ref: 5f631e19
-scope: code
-substantive: true
-raw_file: 20260826T222600Z_5f631e19_prompt_io.raw.md
----
-
-## Prompt
-
-Diagnose and fix the hanging Binance USD-M perpetual feed for
-`NVDAUSDT.USDTM.PERP.BINANCE` by comparing piker with Binance's current
-API and changelog. Ensure unsupported or silent websocket feeds fail
-loudly into `datad` and chart consumers instead of blocking startup.
-
-## Response summary
-
-Migrates Binance USD-M market data from the retired legacy websocket to
-the split public and market routes, handles Binance's new aggregate-trade
-schema, and introduces bounded connection and first-data startup. Direct
-stream URLs avoid reconnect-time subscription-ACK races, while targeted
-tests preserve routing and fail-loud behavior.
-
-## Files changed
-
-- `piker/brokers/binance/feed.py` - route, merge, normalize, and bound
-  Binance quote streams.
-- `piker/brokers/binance/venues.py` - define current USD-M websocket
-  endpoints.
-- `piker/data/_web_bs.py` - support bounded websocket context startup.
-- `tests/test_binance.py` - cover routes, schemas, and startup failures.
-- `ai/prompt-io/opencode/20260826T222600Z_5f631e19_prompt_io.raw.md` -
-  retain the generated-output record.
-- `ai/prompt-io/opencode/20260826T222600Z_5f631e19_prompt_io.md` - record
-  structured provenance for the code changes.
-
-## Human edits
-
-The human explicitly required fail-loud behavior rather than merely
-updating the endpoint, and supplied a direct CLI reproduction proving
-both `datad.binance` and the chart caller blocked. Those requirements
-drove the connection and first-data deadlines and their regressions. The
-changes remain uncommitted for human review.

ai/prompt-io/opencode/20260826T222600Z_5f631e19_prompt_io.raw.md

@@ -1,57 +0,0 @@
----
-model: openai/gpt-5.6-sol
-service: opencode
-timestamp: 2026-08-26T22:26:00Z
-git_ref: 5f631e19
-diff_cmd: git diff HEAD~1..HEAD
----
-
-## Prompt
-
-The user reported that selecting
-`NVDAUSDT.USDTM.PERP.BINANCE` connected Binance's USD-M
-websocket but never loaded the feed or downstream chart. They asked
-OpenCode to compare Binance's API changelog with piker's FQME and feed
-code, diagnose the failure, and solve it.
-
-The user clarified that future provider failures must raise loudly
-instead of silently blocking feed and chart startup. They also confirmed
-the same hang from
-`piker -l info --pdb chart nvdausdt.usdtm.perp.binance`.
-
-## Generated code
-
-> `git diff HEAD~1..HEAD -- piker/brokers/binance/feed.py`
-
-Routes USD-M book-ticker and aggregate-trade data through Binance's new
-`/public` and `/market` websocket paths, merges both sockets into the
-existing quote stream, accepts the new `st` aggregate-trade field, and
-requires both stream classes before declaring startup complete. It also
-removes the misleading missing-provider warning, derives feed mode from
-resolved pair metadata, and raises `DataUnavailable` for connection or
-live-data startup timeouts.
-
-> `git diff HEAD~1..HEAD -- piker/brokers/binance/venues.py`
-
-Defines Binance's current public and market USD-M websocket endpoints
-and makes the market endpoint the regular futures websocket API root.
-
-> `git diff HEAD~1..HEAD -- piker/data/_web_bs.py`
-
-Adds an optional connection deadline to `open_autorecon_ws()` so callers
-can fail context entry when repeated websocket handshakes never become
-connected.
-
-> `git diff HEAD~1..HEAD -- tests/test_binance.py`
-
-Adds regressions for exact Binance stream routing, silent live-data
-startup, silent websocket connection startup, and the current USD-M
-aggregate-trade payload schema.
-
-## Verification output
-
-`pytest -q tests/test_binance.py` passes all four tests. Compile checks,
-focused Ruff fatal-error checks, and `git diff --check` pass. Live probes
-through piker's merged stream received both required stream classes and
-returned normalized trade quotes for `NVDAUSDT` USD-M and `BTCUSDT`
-spot.

ai/prompt-io/opencode/README.md

@@ -1,20 +0,0 @@
-# AI Prompt I/O Log - OpenCode
-
-This directory tracks prompt inputs and model outputs for
-AI-assisted development using OpenCode.
-
-## Policy
-
-Prompt logging follows the
-[NLNet generative AI policy][nlnet-ai]. Substantive AI
-contributions are logged with the model, timestamp,
-prompt, response, and affected files.
-
-[nlnet-ai]: https://nlnet.nl/foundation/policies/generativeAI/
-
-## Usage
-
-Entries are created by the `prompt-io` skill. Human
-contributors remain accountable for all code decisions;
-AI-generated content is not presented as human-authored
-work.

config/brokers.toml

@@ -32,14 +32,7 @@ option.log.disabled = true
 
 
 [kraken]
-# the reference fiat asset as can be set
+key_descr = ''
-# in an account's web-trading-UI prefs.
-src_fiat = 'usd'
-
-# NOTE for account defs, the following
-# lines must match as follows.
-accounts.spot = 'spot'
-key_descr = 'spot'
 api_key = ''
 secret = ''
 # ------ kraken ------

config/conf.toml

@@ -1,26 +1,8 @@
 [network]
 pikerd = [
-  # TCP localhost loopback
+  '/ipv4/127.0.0.1/tcp/6116',  # std localhost daemon-actor tree
-  '/ip4/127.0.0.1/tcp/6116',
+  # '/uds/6116',  # TODO std uds socket file
-
-  # same but UDS
-  '/unix/run/user/1000/piker/pikerd.sock',
 ]
-chart = [
-  '/ip4/127.0.0.1/tcp/3003',
-  '/unix/run/user/1000/piker/chart.sock',
-]
-# the service-actor registry endpoint;
-# other pikerd trees contact this to discover
-# actors.
-# XXX if absent, pikerd binds the registry
-# on its own tpt_bind_addrs.
-# regd = ['/ip4/127.0.0.1/tcp/6116']
-
-# chart = [
-#   '/ip4/127.0.0.1/tcp/3333',
-#   '/unix/run/user/1000/piker/chart@3333.sock',
-# ]
 
 
 [ui]

docs/manual_backfill_qualification.rst

@@ -1,299 +0,0 @@
-Manual Backfill Qualification
-=============================
-
-Purpose
--------
-
-This runbook separates provider coverage, durable NativeDB state,
-shared-memory hydration, and chart rendering while reproducing gappy
-history. Work one FQME and timeframe at a time. Preserve the original
-Parquet first, then qualify both repair of that evidence and a completely
-fresh backfill.
-
-The read-only command used throughout is::
-
-    piker store audit FQME --period 60
-    piker store audit FQME --period 60 --json
-
-It reads raw Parquet directly. It does not open a storage client,
-canonicalize data, publish cache state, attach SHM, or rewrite history.
-Positive gaps are deliberately unclassified; an observed interval is not
-automatically corruption or an expected venue closure.
-
-.. warning::
-
-   Do not run qualification against normal user storage. The helper below
-   requires a marked disposable root before it will seed, archive, or clear
-   a Parquet. ``piker store anal`` and ``piker store shm --write-parquet``
-   are interactive and potentially mutating; they are not audit commands.
-
-   Keep the disposable root private and do not rename, replace, or symlink
-   files under it while a helper is running. Stop the chart cleanly before
-   every audit or comparison phase.
-
-Load The Xonsh Helpers
-----------------------
-
-Run from the repository root in the same xonsh which will launch the chart
-or daemon::
-
-    source snippets/nativedb_backfill_audit.xsh
-
-Choose a disposable root and copy only the provider configuration needed
-for the case::
-
-    run_root = p'/tmp/piker-backfill/mnq-20260918-60s'
-    source_config = p'~/.config/piker'.expanduser()
-    bfq_init(run_root, source_config)
-    bfq_record_case(
-        run_root,
-        fqme='mnq.cme.20260918.ib',
-        period_s=60,
-        provider='ib',
-        symptom='gappy chart during reverse backfill',
-        bad_start_utc=None,
-        bad_end_utc=None,
-    )
-
-``bfq_init()`` sets ``XDG_CONFIG_HOME`` in the current xonsh, so every
-subsequent ``piker`` and ``pikerd`` child inherits the disposable config.
-The effective storage path is under ``RUN_ROOT/xdg/piker/nativedb``.
-It also records the exact repository root and ``piker`` executable in the
-qualification marker, prepends that root to child ``PYTHONPATH``, and
-verifies that the bound executable exposes ``piker store audit``. Audits
-fail if that executable imports another tree.
-
-The copied ``brokers.toml`` can contain credentials. Keep the run root
-private and remove it manually after evidence is no longer needed.
-
-Repository Preflight
---------------------
-
-Before every daemon start, verify the code and config inherited by child
-processes::
-
-    git status --short --branch
-    git rev-parse HEAD
-    python -c 'import piker; print(piker.__file__)'
-    python -c 'import os; print(os.environ["XDG_CONFIG_HOME"])'
-
-Stop any daemon from the previous phase cleanly. Use a separate registry
-address in the disposable ``conf.toml`` when a normal Piker stack remains
-active.
-
-Known-Bad Replay
-----------------
-
-Copy the original file; never move it out of normal storage::
-
-    fqme = 'mnq.cme.20260918.ib'
-    period_s = 60
-    source_parquet = (
-        p'~/.config/piker/nativedb'.expanduser()
-        /f'{fqme}.ohlcv{period_s}s.parquet'
-    )
-    bfq_seed(run_root, fqme, period_s, source_parquet)
-    bfq_audit(run_root, 'baseline', fqme, period_s)
-
-The baseline phase writes::
-
-    evidence/baseline.json
-    evidence/baseline.parquet
-    evidence/baseline.manual.json
-
-The JSON checksum must match the read-only snapshot, and the manual sidecar
-binds that report to its phase and case. Comparison rechecks those bindings
-before reading either phase. The report records raw schema, row and
-timestamp counts, non-finite values, ordering, duplicate excess, canonical
-index state, and every retained positive-gap detail. Files with invalid
-Parquet encoding can not produce a JSON report, but ``--snapshot`` still
-preserves their exact bytes before parse failure.
-
-Launch the chart manually from the same shell and capture logs::
-
-    piker -l info chart mnq.cme.20260918.ib \
-        2>&1 | tee @(run_root / 'logs' / 'repair-chart.log')
-
-After a clean stop::
-
-    bfq_audit(
-        run_root,
-        'repair',
-        fqme,
-        period_s,
-        require_structural=True,
-    )
-    bfq_compare(run_root, 'baseline', 'repair')
-
-The comparison fails if structural validity regresses, any original
-timestamp disappears, chronological endpoints shrink, timestamps become
-duplicated or unordered, indexes cease to be canonical, or OHLCV becomes
-non-finite. Provider replacement of matching seam rows is reported
-separately as ``changed_common_timestamps`` for operator review.
-
-Fresh Backfill
---------------
-
-Yes, fresh qualification means saving the known-bad evidence and then
-starting with no active Parquet in the disposable NativeDB. Archive only
-the disposable file::
-
-    bfq_clear_for_fresh(run_root, fqme, period_s)
-
-Confirm the printed active path is absent, launch the same chart, wait for
-bounded provider exhaustion or the intended coverage, then stop cleanly::
-
-    bfq_audit(
-        run_root,
-        'fresh',
-        fqme,
-        period_s,
-        require_structural=True,
-    )
-
-Do not compare ``baseline`` to ``fresh`` as a timestamp-preservation gate:
-provider limits may intentionally produce a different initial extent.
-Treat ``fresh`` as the baseline for restart and append qualification.
-
-Restart And Append
-------------------
-
-Restart the same chart against the fresh persisted file, stop cleanly, and
-record::
-
-    bfq_audit(
-        run_root,
-        'restart',
-        fqme,
-        period_s,
-        require_structural=True,
-    )
-    bfq_compare(run_root, 'fresh', 'restart')
-
-Leave a subsequent run active long enough to append new samples, stop, and
-record::
-
-    bfq_audit(
-        run_root,
-        'append',
-        fqme,
-        period_s,
-        require_structural=True,
-    )
-    bfq_compare(
-        run_root,
-        'restart',
-        'append',
-        require_newer=True,
-    )
-
-The comparison JSON reports gaps added and removed. Expected closures stay
-visible until a venue-aware classifier proves their session alignment.
-
-Gap Layer Triage
-----------------
-
-List exact durable series before choosing a timeframe, and inspect live
-SHM separately::
-
-    piker store ls
-    piker store series FQME_SUBSTRING
-    piker store shm FQME --max-gaps 20
-
-All three forms are read-only. ``ls`` summarizes periods by FQME, while
-``series`` reports each ``(FQME, period)`` file independently. Read-only
-``shm`` reports every exact actor generation and
-``hist``/``rt`` buffer for the FQME, including invalid rows, ordering
-defects, inferred versus expected cadence, and the largest timestamp gaps.
-Use ``--shm-name EXACT_NAME`` to inspect only one reported buffer.
-
-Use the persisted report and chart together:
-
-===============================  =========================================
-Observation                      Suspected layer
-===============================  =========================================
-Parquet contiguous, chart gappy  SHM hydration, sampling, or chart render
-Parquet gap, provider has bars    provider-delta or storage merge
-Provider and Parquet both gappy  provider omission or venue closure
-Gap vanishes after restart       SHM publication or cache lifecycle
-Old timestamps disappear         destructive persistence regression
-Newest seam duplicates           conflict or dedupe policy
-===============================  =========================================
-
-Always record exact UTC gap endpoints. The audit's numeric epoch is
-authoritative; the ``*_utc`` fields are deterministic display values.
-
-Expansion Matrix
-----------------
-
-Expand only after the fixed FQME passes fresh, restart, and append. Use a
-new disposable root and case record for every row:
-
-====================  ==========  ========================================
-Case                  Period      Coverage model
-====================  ==========  ========================================
-Original IB future    60 seconds  CME maintenance and weekend closures
-Original IB future    1 second    high-frequency provider limits
-Second CME future     60 seconds  contract-specific history
-IB equity             60 seconds  regular overnight and weekend closure
-IB FX                 60 seconds  24x5 session behavior
-Crypto spot/perp      60 seconds  continuous-session control
-Expired future        60 seconds  history-only and expiry behavior
-====================  ==========  ========================================
-
-A sourceable matrix can remain ordinary Python data in xonsh::
-
-    cases = [
-        {
-            'name': 'mnq-60s',
-            'fqme': 'mnq.cme.20260918.ib',
-            'period_s': 60,
-            'provider': 'ib',
-        },
-        {
-            'name': 'mnq-1s',
-            'fqme': 'mnq.cme.20260918.ib',
-            'period_s': 1,
-            'provider': 'ib',
-        },
-    ]
-    for case in cases:
-        case_root = p'/tmp/piker-backfill' / case['name']
-        bfq_init(case_root, source_config)
-        bfq_record_case(
-            case_root,
-            fqme=case['fqme'],
-            period_s=case['period_s'],
-            provider=case['provider'],
-            symptom='manual expansion case',
-        )
-
-This loop prepares isolated case roots only. Run the complete seed or
-fresh, chart, audit, restart, append, and compare lifecycle in each root;
-directory creation alone is not qualification.
-
-The chart path may allocate companion 1-second and 60-second files. Audit
-the selected case path explicitly and retain companion files as secondary
-evidence; do not imply that the live request isolated one timeframe.
-
-Interruption Qualification
---------------------------
-
-After normal restart is stable, interrupt one phase with graceful Ctrl-C,
-restart, and compare against the last clean evidence. Provider disconnect,
-SIGTERM, and deterministic write-barrier failures come next. Avoid timing a
-SIGKILL by hand; add an explicit failpoint before qualifying atomic replace
-boundaries.
-
-Result Interpretation
----------------------
-
-``structural_ok`` means the raw persisted file satisfies durable schema,
-finite values, positive unique ordered timestamps, canonical index, and
-period-aligned gaps. ``gap_free`` means no positive gaps were observed when
-at least two valid timestamps made cadence verifiable; otherwise it is
-``null``. ``qualification_ok`` combines both and is intentionally strict.
-
-A structurally valid session-market file can have ``gap_free = false``.
-Those intervals require provider/session evidence before being marked as
-expected closures. Unknown calendar state must stay unknown.

flake.lock

@@ -2,11 +2,11 @@
   "nodes": {
     "nixpkgs": {
       "locked": {
-        "lastModified": 1784497964,
+        "lastModified": 1765779637,
-        "narHash": "sha256-vlHUuqAcbcH2RKmHbPiuQzbv1pnzzavXnI62RD0bqCU=",
+        "narHash": "sha256-KJ2wa/BLSrTqDjbfyNx70ov/HdgNBCBBSQP3BIzKnv4=",
         "owner": "nixos",
         "repo": "nixpkgs",
-        "rev": "241313f4e8e508cb9b13278c2b0fa25b9ca27163",
+        "rev": "1306659b587dc277866c7b69eb97e5f07864d8c4",
         "type": "github"
       },
       "original": {

flake.nix

@@ -22,14 +22,12 @@
 
           # do store-path extractions
           qt6baseStorePath = lib.getLib pkgs.qt6.qtbase;
-          zlibStorePath = lib.getLib pkgs.zlib;
           # ?TODO? can remove below since manual linking not needed?
           # qt6QtWaylandStorePath = lib.getLib pkgs.qt6.qtwayland;
 
           # XXX NOTE XXX, for now we overlay specific pkgs via
           # a major-version-pinned-`cpython`
           cpython = "python313";
-          python = pkgs.${cpython};
           pypkgs = pkgs."${cpython}Packages";
         in
         {
@@ -48,7 +46,7 @@
               qt6.qtbase
 
               uv
-              python
+              python313  # ?TODO^ how to set from `cpython` above?
               pypkgs.pyqt6
               pypkgs.pyqt6-sip
               pypkgs.qtpy
@@ -72,7 +70,6 @@
 
               # link-in c++ stdlib for various AOT-ext-pkgs (numpy, etc.)
               LD_LIBRARY_PATH="${pkgs.stdenv.cc.cc.lib}/lib:$LD_LIBRARY_PATH"
-              LD_LIBRARY_PATH="${zlibStorePath}/lib:$LD_LIBRARY_PATH"
 
               export LD_LIBRARY_PATH
 
@@ -89,31 +86,15 @@
               export QT_WAYLAND_SHELL_INTEGRATION="xdg-shell"
 
               # ------ uv ------
-              # ignore any venv inherited from an outer checkout
-              unset VIRTUAL_ENV
               # - always use the ./py313/ venv-subdir
               export UV_PROJECT_ENVIRONMENT="py313"
-
-              # Rebuild after a flake update changes the Nix Python closure.
-              venv_python="$UV_PROJECT_ENVIRONMENT/bin/python"
-              nix_python="$(readlink -f "${python}/bin/python3")"
-              if test -e "$venv_python" \
-                && test "$(readlink -f "$venv_python")" != "$nix_python"
-              then
-                uv venv \
-                  --clear \
-                  --python "$nix_python" \
-                  "$UV_PROJECT_ENVIRONMENT"
-              fi
-
               # sync project-env with all extras
               uv sync --dev --all-extras --no-group lint
-              source "$UV_PROJECT_ENVIRONMENT/bin/activate"
 
               # ------ TIPS ------
               # NOTE, to launch the py-venv installed `xonsh` (like @goodboy)
               # run the `nix develop` cmd with,
-              # >> nix develop -c xonsh
+              # >> nix develop -c uv run xonsh
             '';
           };
         }

notes_to_self/claudy/gapsannotator_sesh.md

@@ -1,122 +0,0 @@
-<!-- sid=dfd62314-c366-4c23-83e5-0ce293b7472a -->
-
-● aight so perf analysis time, then i'll let you sleep for real:
-
-  Current State (Final Benchmark)
-
-  Server side: CRUSHED IT 🎉
-  - 7.6ms total for 1285 gaps
-  - time_to_row creation: 1.3ms (was 98ms! 75x speedup from field extraction)
-  - Everything else sub-2ms
-  - Server is basically not the bottleneck anymore
-
-  Client side: NEW BOTTLENECK IDENTIFIED
-  - 444.9ms total for 1285 gaps
-  - Building annotation specs: 324.2ms (73% of total time!)
-  - IPC call: 120.4ms (includes round-trip + server 7.6ms)
-  - Redraws: negligible (0.3ms total)
-
-  The Problem
-
-  Looking at piker/tsp/_annotate.py:127-293, we're doing the classic anti-pattern:
-
-  for i in range(gaps.height):  # 1285 iterations
-      row: pl.DataFrame = gaps[i]  # per-row Polars access!
-      iend: int = row['index'][0]
-      opn: float = row['open'][0]
-      cls: float = prev_r['close'][0]
-      # ... tons of field access per row
-      # ... dict building per row
-      rect_specs.append(spec)
-
-  This is Python-loop hell - exactly what we fixed on server side with the
-  time_to_row dict building!
-
-  Per-row DataFrame access is SLOW. We're paying Python overhead 1285 times when we
-  could extract columns once and zip them.
-
-  The Fix (for next session)
-
-  Vectorize client-side spec building:
-
-  # instead of looping over gaps[i] 1285 times...
-
-  # extract all columns once
-  indices = gaps['index'].to_numpy()
-  opens = gaps['open'].to_numpy()
-  closes = gaps['close'].to_numpy()
-  times = gaps['time'].to_numpy()
-  # ... etc
-
-  # vectorized calculations
-  s_diffs = gaps['s_diff'].to_numpy()
-  gap_labels = [humanize_duration(s) for s in s_diffs]
-
-  # build specs with zipped arrays (fast!)
-  rect_specs = [
-      {
-          'start_pos': (start_x, start_y),
-          'end_pos': (end_x, end_y),
-          'color': color,
-          # ...
-      }
-      for start_x, start_y, end_x, end_y, ... in zip(
-          start_xs, start_ys, end_xs, end_ys, ...
-      )
-  ]
-
-  Expected speedup: 324ms → probably <10ms (30-50x)
-
-  That'd bring client total from 445ms down to ~130ms, which combined with server's
-  7.6ms = sub-150ms end-to-end for 1285 gaps B)
-
-  Other Ideas to Dream About
-
-  1. Pass numpy arrays over IPC instead of dicts?
-    - Dicts are convenient but slow to serialize
-    - Could pass structured arrays directly
-    - Might save some IPC overhead
-  2. Pre-compute specs on server side?
-    - Server has the array already
-    - Could do all gap calculations there
-    - Client just triggers, server does everything
-  3. Cache gap specs per timeframe?
-    - Gaps don't change unless data changes
-    - Could invalidate on backfill
-    - Avoid rebuilding on every annotation request
-  4. Polars-native gap processing?
-    - Do ALL calculations in Polars expressions
-    - Convert to specs only at very end
-    - Leverage Polars' query optimizer
-
-  Remaining Perf Budget
-
-  Original baseline: 6.6 seconds for 1285 gaps
-
-  Current state:
-  - Client: 445ms
-  - Server: 7.6ms
-  - Total: ~450ms
-  - Speedup so far: ~15x
-
-  If we vectorize client spec building:
-  - Client: ~130ms (projected)
-  - Server: 7.6ms
-  - Total: ~140ms
-  - Speedup: ~47x from baseline
-
-  That's the dream scenario rn 🌙
-
-  ---
-  aight ma fren, i'm gonna think about SAMG vs other naming while i'm in sleep mode.
-   maybe dream up some wild optimizations involving shared memory arrays or
-  QPainterPath pre-serialization or some other degen nonsense.
-
-  catch you on the flip side, ready to vectorize that client spec building and
-  absolutely DEMOLISH this last bottleneck 😎
-
-  good sesh today Bo
-
-  Todos
-  ☒ Create skills documentation for session learnings
-  ☐ Fix server-side time_to_row regression (98ms -> 4ms)

piker/accounting/_ledger.py

@@ -324,13 +324,10 @@ def load_ledger(
     ldir: Path = (
         dirpath
         or
-        config.get_conf_dir() / 'accounting' / 'ledgers'
+        config._config_dir / 'accounting' / 'ledgers'
     )
     if not ldir.is_dir():
-        ldir.mkdir(
+        ldir.mkdir()
-            parents=True,
-            exist_ok=True,
-        )
 
     fname = f'trades_{brokername}_{acctid}.toml'
     fpath: Path = ldir / fname

piker/accounting/_pos.py

@@ -785,16 +785,9 @@ def load_account(
     legacy_fn: str = f'pps.{brokername}.{acctid}.toml'
     fn: str = f'account.{brokername}.{acctid}.toml'
 
-    dirpath: Path = (
+    dirpath: Path = dirpath or (config._config_dir / 'accounting')
-        dirpath
-        or
-        (config.get_conf_dir() / 'accounting')
-    )
     if not dirpath.is_dir():
-        dirpath.mkdir(
+        dirpath.mkdir()
-            parents=True,
-            exist_ok=True,
-        )
 
     conf, path = config.load(
         path=dirpath / fn,

piker/accounting/cli.py

@@ -143,15 +143,12 @@ def sync(
                     # (what the EMS normally does internall) B)
                     open_brokerd_dialog(
                         brokermod,
+                        portal,
                         exec_mode=(
                             'paper'
                             if account == 'paper'
                             else 'live'
                         ),
-                        # use our own ad-hoc-spawned actor,
-                        # do NOT (spawn and) use the
-                        # `brokerd.<broker>` service daemon!
-                        portal=portal,
                         loglevel=loglevel,
                     ) as (
                         brokerd_stream,

piker/brokers/__init__.py

@@ -31,7 +31,6 @@ from piker.log import (
 from ._util import (
     BrokerError,
     SymbolNotFound,
-    MarketNotFound as MarketNotFound,
     NoData,
     DataUnavailable,
     DataThrottle,

piker/brokers/_daemon.py

@@ -25,8 +25,10 @@ from contextlib import (
 )
 from types import ModuleType
 from typing import (
+    TYPE_CHECKING,
     AsyncContextManager,
 )
+import exceptiongroup as eg
 
 import tractor
 import trio
@@ -38,20 +40,27 @@ from piker.log import (
 from . import _util
 from . import get_brokermod
 
+if TYPE_CHECKING:
+    from ..data import _FeedsBus
+
 log = get_logger(name=__name__)
 
-# `brokerd`-actor-always-enabled mods.
+# `brokerd` enabled modules
+# TODO: move this def to the `.data` subpkg..
 # NOTE: keeping this list as small as possible is part of our caps-sec
-# model and should be treated with utmost care! In particular NO
+# model and should be treated with utmost care!
-# `piker.data.*` feed mods should be enabled in this (live,
+_data_mods: str = [
-# credentialed) trading actor; all data-feed serving is the
+    'piker.brokers.core',
-# domain of the `datad.<broker>` sibling daemon, see
+    'piker.brokers.data',
-# `piker.data._daemon._datad_service_mods`.
-_brokerd_service_mods: list[str] = [
     'piker.brokers._daemon',
+    'piker.data',
+    'piker.data.feed',
+    'piker.data._sampling'
 ]
 
 
+# TODO: we should rename the daemon to datad prolly once we split up
+# broker vs. data tasks into separate actors?
 @tractor.context
 async def _setup_persistent_brokerd(
     ctx: tractor.Context,
@@ -60,15 +69,9 @@ async def _setup_persistent_brokerd(
 
 ) -> None:
     '''
-    Trading-only daemon (lifetime) fixture: console logging
+    Allocate a actor-wide service nursery in ``brokerd``
-    setup and a pinned-open context for service mgmt.
+    such that feeds can be run in the background persistently by
-
+    the broker backend as needed.
-    All data-feed-bus state now lives in the (data-feed-only)
-    `datad.<brokername>` sibling daemon, see
-    `piker.data._daemon._setup_persistent_datad()`; this
-    actor hosts only the backend's `open_trade_dialog()`
-    (live order-control) ep-task(s) which manage their own
-    task trees per `tractor.Context`.
 
     '''
     # NOTE: we only need to setup logging once (and only) here
@@ -84,12 +87,46 @@ async def _setup_persistent_brokerd(
     )
     assert log.name == _util.subsys
 
-    # unblock caller
+    # further, set the log level on any broker broker specific
-    await ctx.started()
+    # logger instance.
 
-    # we pin this task to keep the daemon active until the
+    from piker.data import feed
-    # parent actor decides to tear it down
+    assert not feed._bus
-    await trio.sleep_forever()
+
+    # allocate a nursery to the bus for spawning background
+    # tasks to service client IPC requests, normally
+    # `tractor.Context` connections to explicitly required
+    # `brokerd` endpoints such as:
+    # - `stream_quotes()`,
+    # - `manage_history()`,
+    # - `allocate_persistent_feed()`,
+    # - `open_symbol_search()`
+    # NOTE: see ep invocation details inside `.data.feed`.
+    try:
+        async with (
+            # tractor.trionics.collapse_eg(),
+            trio.open_nursery() as service_nursery
+        ):
+            bus: _FeedsBus = feed.get_feed_bus(
+                brokername,
+                service_nursery,
+            )
+            assert bus is feed._bus
+
+            # unblock caller
+            await ctx.started()
+
+            # we pin this task to keep the feeds manager active until the
+            # parent actor decides to tear it down
+            await trio.sleep_forever()
+
+    except eg.ExceptionGroup:
+        # TODO: likely some underlying `brokerd` IPC connection
+        # broke so here we handle a respawn and re-connect attempt!
+        # This likely should pair with development of the OCO task
+        # nusery in dev over @ `tractor` B)
+        # https://github.com/goodboy/tractor/pull/363
+        raise
 
 
 def broker_init(
@@ -110,10 +147,8 @@ def broker_init(
 
     This includes:
     - load the appropriate <brokername>.py pkg module,
-    - reads any declared `_brokerd_mods: list[str]` (falling
+    - reads any declared `__enable_modules__: listr[str]` which will be
-      back to the full `__enable_modules__` set for
+      passed to `tractor.ActorNursery.start_actor(enabled_modules=<this>)`
-      not-yet-split backends) which will be passed to
-      `tractor.ActorNursery.start_actor(enable_modules=)`
       at actor start time,
     - deliver a references to the daemon lifetime fixture, which
       for now is always the `_setup_persistent_brokerd()` context defined
@@ -148,14 +183,8 @@ def broker_init(
     ]
     for submodname in getattr(
         brokermod,
-        '_brokerd_mods',
+        '__enable_modules__',
-        # fallback for (flat, less mature) backends which
+        [],
-        # don't yet declare a daemon-kind mod split.
-        getattr(
-            brokermod,
-            '__enable_modules__',
-            [],
-        ),
     ):
         subpath: str = f'{modpath}.{submodname}'
         enabled.append(subpath)
@@ -183,22 +212,6 @@ async def spawn_brokerd(
         f'backend: {brokername!r}'
     )
 
-    # fail fast on (data-only) backends which don't offer
-    # ANY live order-control eps; the caller should instead
-    # be using paper-mode (and thus never spawning us)!
-    from ..data.validate import get_eps
-    brokerd_eps: dict = get_eps(
-        get_brokermod(brokername),
-        'brokerd',
-    )
-    if not brokerd_eps:
-        raise RuntimeError(
-            f'Backend {brokername!r} offers NO `brokerd` '
-            f'(live order-control) eps!?\n'
-            f'It is likely a datad-only provider, use '
-            f'paper-mode for clearing instead.\n'
-        )
-
     (
         brokermode,
         tractor_kwargs,
@@ -220,11 +233,7 @@ async def spawn_brokerd(
     dname: str = tractor_kwargs.pop('name')  # f'brokerd.{brokername}'
     portal = await Services.actor_n.start_actor(
         dname,
-        enable_modules=list(dict.fromkeys(
+        enable_modules=_data_mods + tractor_kwargs.pop('enable_modules'),
-            _brokerd_service_mods
-            +
-            tractor_kwargs.pop('enable_modules')
-        )),
         debug_mode=Services.debug_mode,
         **tractor_kwargs
     )

piker/brokers/_util.py

@@ -20,17 +20,10 @@ Handy cross-broker utils.
 """
 from __future__ import annotations
 # from functools import partial
-from typing import (
-    Type,
-)
 
 import json
 import httpx
 import logging
-from msgspec import Struct
-from tractor._exceptions import (
-    reg_err_types,
-)
 
 from piker.log import (
     colorize_json,
@@ -66,10 +59,6 @@ class SymbolNotFound(BrokerError):
     "Symbol not found by broker search"
 
 
-class MarketNotFound(SymbolNotFound):
-    "Mkt-pair not found by broker search"
-
-
 # TODO: these should probably be moved to `.tsp/.data`?
 class NoData(BrokerError):
     '''
@@ -108,19 +97,6 @@ class DataThrottle(BrokerError):
     '''
     # TODO: add in throttle metrics/feedback
 
-class SchemaMismatch(BrokerError):
-    '''
-    Market `Pair` fields mismatch, likely due to provider API update.
-
-    '''
-
-# auto-register all `BrokerError` subtypes for
-# tractor IPC exc-marshalling.
-reg_err_types([
-    BrokerError,
-    *BrokerError.__subclasses__(),
-])
-
 
 def resproc(
     resp: httpx.Response,
@@ -147,45 +123,3 @@ def resproc(
         log.debug(f"Received json contents:\n{colorize_json(msg)}")
 
     return msg if return_json else resp
-
-
-def get_or_raise_on_pair_schema_mismatch(
-    pair_type: Type[Struct],
-    fields_data: dict,
-    provider_name: str,
-    api_url: str|None = None,
-) -> Struct:
-    '''
-    Boilerplate helper around assset-`Pair` field schema mismatches,
-    normally due to provider API updates.
-
-    '''
-    try:
-        pair: Struct = pair_type(**fields_data)
-        return pair
-    except TypeError as err:
-
-        from tractor.devx.pformat import ppfmt
-        repr_data: str = ppfmt(fields_data)
-        report: str = (
-            f'Field mismatch we need to codify!\n'
-            f'\n'
-            f'{pair_type!r}({repr_data})'
-            f'\n'
-            f'^^^ {err.args[0]!r} ^^^\n'
-            f'\n'
-            f"Don't panic, prolly {provider_name!r} "
-            f"changed their symbology schema..\n"
-        )
-        if (
-            api_url
-            or
-            (api_url := pair_type._api_url)
-        ):
-            report += (
-                f'\n'
-                f'Check out their API docs here:\n'
-                f'{api_url}\n'
-            )
-
-        raise SchemaMismatch(report) from err

piker/brokers/binance/__init__.py

@@ -52,25 +52,9 @@ __all__ = [
 ]
 
 
-# per-daemon-kind (sub)mod groups: declares which of our
+# `brokerd` modules
-# submods host the eps run by each daemon-actor kind as
+__enable_modules__: list[str] = [
-# defined by `piker.data.validate._eps`.
-# NOTE: `get_mkt_info` and `open_symbol_search` both live
-# in `.feed` for this backend (no `symbols.py`).
-_brokerd_mods: list[str] = [
-    'api',
-    'broker',
-]
-
-_datad_mods: list[str] = [
     'api',
     'feed',
+    'broker',
 ]
-
-
-# tractor RPC enable arg
-__enable_modules__: list[str] = list(dict.fromkeys(
-    _brokerd_mods
-    +
-    _datad_mods
-))