Compare commits
No commits in common. "wkt/land_gap_overlays" and "main" have entirely different histories.
wkt/land_g
...
main
|
|
@ -47,7 +47,7 @@ that's why it's like 1000x faster ya know?"
|
||||||
- Wrap all code symbols: `function()`,
|
- Wrap all code symbols: `function()`,
|
||||||
`ClassName`, `field_name`
|
`ClassName`, `field_name`
|
||||||
- File paths: `piker/ui/_remote_ctl.py`
|
- File paths: `piker/ui/_remote_ctl.py`
|
||||||
- Commands: `git status`, `piker store shm`
|
- Commands: `git status`, `piker store ldshm`
|
||||||
|
|
||||||
**Explain like you're pair programming:**
|
**Explain like you're pair programming:**
|
||||||
```
|
```
|
||||||
|
|
|
||||||
|
|
@ -1,143 +0,0 @@
|
||||||
# Split `brokerd.<broker>` into trading-only `brokerd` + new `datad.<broker>`
|
|
||||||
|
|
||||||
## Context
|
|
||||||
|
|
||||||
Today a single `brokerd.<broker>` actor hosts BOTH concerns:
|
|
||||||
|
|
||||||
- **data feed service tasks**: the `_FeedsBus` + `open_feed_bus()` ep (`piker/data/feed.py:464`), per-symbol `allocate_persistent_feed()` tasks (shm writers via `sample_and_broadcast()`, history backfill via `piker.tsp`), plus backend eps `stream_quotes`, `open_history_client`, `open_symbol_search`, `get_mkt_info` from `piker/brokers/<backend>/feed.py`/`symbols.py`,
|
|
||||||
- **live order-control tasks**: `open_trade_dialog()` from `piker/brokers/<backend>/broker.py`, driven by `emsd`.
|
|
||||||
|
|
||||||
The codebase already anticipates this split: `piker/data/validate.py:70-91` groups backend eps into `'datad'` vs `'brokerd'` kinds, `piker/brokers/ib/__init__.py:62-70` already declares `_brokerd_mods`/`_datad_mods`, and `piker/brokers/_daemon.py:62` carries the literal TODO *"rename the daemon to datad prolly once we split up broker vs. data tasks into separate actors?"*. This work executes that split.
|
|
||||||
|
|
||||||
**User-decided constraints:**
|
|
||||||
|
|
||||||
- `datad.<broker>` is a **sibling** of `brokerd.<broker>` under `pikerd`, spawned via the existing `Services` + `maybe_spawn_daemon()` machinery (`piker/service/_daemon.py:46`).
|
|
||||||
- **Hard cutover, staged by layer** — no dual-mode runtime flag; every stage lands fully working.
|
|
||||||
- Post-split `brokerd` is **trading-only and lazily spawned solely by emsd's** `open_brokerd_dialog` path; UI/CLI/feed code never spawns it. Chart-only and paper sessions run with **zero** brokerd processes.
|
|
||||||
|
|
||||||
## Target topology
|
|
||||||
|
|
||||||
```
|
|
||||||
pikerd
|
|
||||||
├── datad.ib feed bus, shm writers, tsp history, symbol search
|
|
||||||
├── brokerd.ib open_trade_dialog only (EMS-spawned, lazy)
|
|
||||||
├── emsd
|
|
||||||
│ └── paperboi.ib (paper mode; opens its feed via datad)
|
|
||||||
└── samplerd
|
|
||||||
```
|
|
||||||
|
|
||||||
## Key verified facts (load-bearing)
|
|
||||||
|
|
||||||
- `feed.portals` has exactly ONE trading consumer: `piker/clearing/_ems.py:671` (`portal = feed.portals[brokermod]` → handed to `open_brokerd_dialog`). This is the single coupling forcing feed + trading into one actor.
|
|
||||||
- `assert 'brokerd' in servicename` at `piker/data/feed.py:502` is the only actor-name assert in the tree.
|
|
||||||
- `piker ledger` (`piker/accounting/cli.py:100-157`) is a hidden consumer: it calls `broker_init()` directly, spawns its own ad-hoc actor, enters `_setup_persistent_brokerd`, then calls `open_brokerd_dialog(brokermod, portal, ...)` **with an explicit portal** — the new signature must keep a `portal:` override param.
|
|
||||||
- kraken's feed→broker coupling is mild: `kraken/broker.py:77-81` imports `NoBsWs`/`open_autorecon_ws` (really from `piker.data._web_bs`) and `stream_messages` (pure parser) from `feed.py`. In-process imports only — `enable_modules` gates RPC, not imports. No live shared state; each actor opens its own WS.
|
|
||||||
- ib: default `client_id=6116` (`ib/api.py:1320`) with linear retry `clientId=client_id + i` (`:1403`). Two ib actors connecting concurrently will collide → needs a role-based id offset. Both daemons need `_spawn_kwargs={'infect_asyncio': True}` (copied by both init fns).
|
|
||||||
- binance has no `symbols.py` (`get_mkt_info`/`open_symbol_search` live in `binance/feed.py`); kucoin/questrade/robinhood are flat single-file with NO `__enable_modules__`; deribit has no `broker.py`.
|
|
||||||
- `tests/test_services.py:80,185` assert `brokerd` actor names incl. the paper-mode flow asserting `brokerd.kraken` spawns (`:229,:237`) — must invert post-split.
|
|
||||||
- `_root_modules` (`piker/service/_actor_runtime.py:156`) must gain the new datad daemon mod so `pikerd_portal.run(spawn_datad, ...)` resolves.
|
|
||||||
- `piker/brokers/core.py:175` `symbol_search` does `portal.run(search_w_brokerd)` where the target fn lives in `piker.brokers.core` → that mod must stay RPC-enabled in datad.
|
|
||||||
|
|
||||||
## Module decision
|
|
||||||
|
|
||||||
New file **`piker/data/_daemon.py`** hosts all datad machinery (`_setup_persistent_datad`, `datad_init`, `spawn_datad`, `maybe_spawn_datad`, `_datad_service_mods`). Mirrors the `piker.brokers._daemon` / `piker.data._sampling` (samplerd) per-subsystem convention and satisfies the existing TODO at `brokers/_daemon.py:49` ("move this def to the `.data` subpkg"). `piker.brokers._daemon` keeps brokerd-only code, slimmed. Do NOT over-DRY the two ~40-line init fns into a shared factory yet (samplerd precedent accepts the duplication).
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Stage 0 — prep: backend module grouping + ep introspection (no topology change)
|
|
||||||
|
|
||||||
1. `piker/data/validate.py`: add a `get_eps(mod, kind) -> dict[str, Callable]` helper returning the backend's defined eps for a daemon-kind from `_eps` (missing eps excluded). Used later by both init fns + fail-fast checks.
|
|
||||||
2. Declare daemon-kind module groups in each split backend's `__init__.py`, keeping `__enable_modules__` as the (deduped) union so behavior is unchanged:
|
|
||||||
- `kraken`: `_brokerd_mods = ['api', 'broker']`, `_datad_mods = ['api', 'feed', 'symbols']`
|
|
||||||
- `binance`: `_brokerd_mods = ['api', 'broker']`, `_datad_mods = ['api', 'feed']`
|
|
||||||
- `deribit`: `_brokerd_mods = []`, `_datad_mods = ['api', 'feed']`
|
|
||||||
- `ib`: adjust existing groups so each includes `'api'`
|
|
||||||
- flat backends (kucoin etc.): no attrs — init fns fall back to enabling just `modpath` (existing behavior).
|
|
||||||
3. Hygiene: `kraken/broker.py:77-81` imports `NoBsWs`/`open_autorecon_ws` from `piker.data._web_bs` directly (keep the `stream_messages` import from `.feed`).
|
|
||||||
|
|
||||||
**Gate**: full pytest green, zero behavior change.
|
|
||||||
|
|
||||||
## Stage 1 — introduce `datad` daemon machinery (additive)
|
|
||||||
|
|
||||||
New `piker/data/_daemon.py`:
|
|
||||||
|
|
||||||
- `_datad_service_mods: list[str]` — datad-always-enabled mods (successor to the data side of `_data_mods` from `brokers/_daemon.py:52`): `['piker.brokers.core', 'piker.brokers.data', 'piker.data', 'piker.data.feed', 'piker.data._sampling', 'piker.data._daemon']`.
|
|
||||||
- `_setup_persistent_datad(ctx, brokername, loglevel)` — `@tractor.context` fixture; logging boilerplate (as `_setup_persistent_brokerd:81-88`), then allocates the actor-global feed bus exactly as the brokerd fixture does today (`brokers/_daemon.py:105-121`): `assert not feed._bus`, open service nursery, `feed.get_feed_bus(brokername, service_nursery)`, `ctx.started()`, `sleep_forever()`.
|
|
||||||
- `datad_init(brokername, ...)` — mirrors `broker_init()` (`brokers/_daemon.py:132`): actor name `f'datad.{brokername}'`, copies backend `_spawn_kwargs` (**critical for ib infect_asyncio**), builds `enable_modules` from `getattr(brokermod, '_datad_mods', getattr(brokermod, '__enable_modules__', []))`.
|
|
||||||
- `spawn_datad(brokername, ...)` — mirrors `spawn_brokerd()` (`brokers/_daemon.py:202`): `Services.actor_n.start_actor(dname, enable_modules=_datad_service_mods + backend_mods, ...)` + `Services.start_service_task(dname, portal, _setup_persistent_datad, ...)`.
|
|
||||||
- `maybe_spawn_datad(brokername, ...)` — wraps `maybe_spawn_daemon(service_name=f'datad.{brokername}', service_task_target=spawn_datad, ...)` exactly like `maybe_spawn_brokerd` (`brokers/_daemon.py:256`).
|
|
||||||
|
|
||||||
Supporting edits:
|
|
||||||
|
|
||||||
- `piker/service/_actor_runtime.py:156`: add `'piker.data._daemon'` to `_root_modules` (keep `'piker.brokers._daemon'`).
|
|
||||||
- `piker/service/__init__.py`: re-export `spawn_datad`/`maybe_spawn_datad` next to the brokerd ones (`:56-57`).
|
|
||||||
- `tests/test_services.py`: new `test_datad_spawn` — `open_test_pikerd` + `maybe_spawn_datad('kraken')` + `ensure_service('datad.kraken')`. (Do NOT route a feed through it yet — `open_feed_bus`'s assert still says brokerd.)
|
|
||||||
|
|
||||||
**Gate**: suite green + new spawn test; nothing routes through datad yet.
|
|
||||||
|
|
||||||
## Stage 2 — clearing layer: emsd self-spawns brokerd (decouple from `feed.portals`)
|
|
||||||
|
|
||||||
Sequenced BEFORE the feed cutover so live trading works at every boundary (otherwise `feed.portals` would hand emsd a datad portal).
|
|
||||||
|
|
||||||
`piker/clearing/_ems.py`:
|
|
||||||
|
|
||||||
1. `open_brokerd_dialog()` (`:336`) new signature: `(brokermod, exec_mode, fqme=None, portal: tractor.Portal|None = None, loglevel=None)`. Internals: keep trades-ep detection (`:400-417`); acquire the brokerd actor ONLY when a live ep will actually open, via a small inner `@acm _acquire_live_portal()` that yields the passed `portal` if given (the `piker ledger` path) else `async with maybe_spawn_brokerd(brokermod.name, loglevel=loglevel)` — **the single place brokerd boots post-split**. Move the eager `portal.open_context(trades_endpoint, ...)` construction (`:425`) inside that block; the paper short-circuit (`:432`) never touches it.
|
|
||||||
2. `Router.maybe_open_brokerd_dialog()` (`:581`) — drop the `portal` param; `Router.open_trade_relays()` (`:640`) — delete `portal = feed.portals[brokermod]` (`:671`) and the pass-through (`:685`).
|
|
||||||
3. `piker/accounting/cli.py:144`: switch to keyword form `open_brokerd_dialog(brokermod, exec_mode=..., portal=portal, loglevel=...)`.
|
|
||||||
|
|
||||||
Pre-cutover this is a pure refactor: emsd's `maybe_spawn_brokerd` just *finds* the already-running brokerd via the registry.
|
|
||||||
|
|
||||||
**Gate**: `tests/test_ems.py` + services suite green; manual paper order on kraken; `piker ledger` smoke.
|
|
||||||
|
|
||||||
## Stage 3 — feed layer hard cutover to datad
|
|
||||||
|
|
||||||
1. `piker/data/feed.py`: import `maybe_spawn_datad` (replacing `maybe_spawn_brokerd`, `:56-58`); `open_feed()` `:895` → `maybe_spawn_datad(...)` (rename `brokerd_ctxs` → `datad_ctxs`); `open_feed_bus()` `:502` → `assert 'datad' in servicename`; comment sweep.
|
|
||||||
2. `piker/brokers/_daemon.py` `_setup_persistent_brokerd()`: slim to trading-only fixture — logging setup, `ctx.started()`, `sleep_forever()`. Drop the bus alloc, `assert not feed._bus`, the service nursery (backend `open_trade_dialog` ctxs own their task trees), the `eg.ExceptionGroup` handler, and the stale `_FeedsBus` TYPE_CHECKING import. (`piker ledger`'s ad-hoc actor enters this same slimmed fixture — exactly what it needs.)
|
|
||||||
3. Repoint remaining data-flavored spawn sites `maybe_spawn_brokerd` → `maybe_spawn_datad`:
|
|
||||||
- `piker/ui/_app.py:40,57` (symbol search; optionally rename `install_brokerd_search` → `install_datad_search`, def at `piker/data/feed.py:766`)
|
|
||||||
- `piker/brokers/core.py:33,175` (`symbol_search`)
|
|
||||||
- `piker/brokers/cli.py:38,190,320` (`brokercheck`, `record`)
|
|
||||||
- `piker/ui/cli.py:30,72,121` (legacy kivy monitor/optschain) + best-effort `piker/ui/kivy/option_chain.py:498` `wait_for_actor('brokerd')` → `'datad'`
|
|
||||||
4. `tests/test_services.py`: `:80,:185` `actor_name = 'datad'`; paper-mode test asserts `datad.kraken` + `paperboi.kraken` + `emsd` AND adds the headline negative check: `assert 'brokerd.kraken' not in services.service_tasks`.
|
|
||||||
|
|
||||||
**Gate**: full suite with inverted assertions; manual: chart on binance/kraken shows `datad.<broker>` in `piker services` and NO brokerd; paper order fills; symbol search works; history backfill sane.
|
|
||||||
|
|
||||||
## Stage 4 — caps-sec slimming, validation, ib client-id
|
|
||||||
|
|
||||||
1. `piker/brokers/_daemon.py`: delete `_data_mods` (`:52`); `spawn_brokerd()` `:236` → `enable_modules=tractor_kwargs.pop('enable_modules')`; `broker_init()` `:184` reads `getattr(brokermod, '_brokerd_mods', getattr(brokermod, '__enable_modules__', []))`. Resulting brokerd enable set has no `piker.data.*` at all.
|
|
||||||
2. Fail-fast ep validation via Stage-0 `validate.get_eps()`: `broker_init` raises a clear error when `get_eps(mod, 'brokerd')` is empty (e.g. "kucoin is datad-only — use paper mode"); `datad_init` warns analogously.
|
|
||||||
3. ib client-id mitigation in `load_aio_clients()` (`ib/api.py:~1316`): role-based offset when `client_id` is the 6116 default (e.g. `+16` when `'datad' in tractor.current_actor().name`), optionally configurable via `brokers.toml` keys.
|
|
||||||
4. Docs/comment sweep: resolve `brokers/_daemon.py:62` TODO, `piker/cli/__init__.py:253` TODO, daemon list in `piker/service/__init__.py:21` docstring.
|
|
||||||
|
|
||||||
**Gate**: full suite; audit greps clean: `grep -rn maybe_spawn_brokerd piker/` hits only `clearing/_ems.py`, `service/__init__.py`, `brokers/_daemon.py`; no spawn-path `'brokerd'` literals left in `piker/data`, `piker/ui`, `piker/brokers/cli.py`.
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
## Risk register
|
|
||||||
|
|
||||||
| Risk | Sev | Mitigation |
|
|
||||||
|---|---|---|
|
|
||||||
| ib: `datad.ib` + `brokerd.ib` both connect to TWS/gw; `client_id=6116` collision burns `connect_timeout` retries | High (live ib only) | Stage 4 role-based id offset; both daemons inherit `infect_asyncio` via `_spawn_kwargs` copy in both init fns |
|
|
||||||
| `piker ledger` ad-hoc actor path silently broken by signature change | Med | `portal:` override kept on `open_brokerd_dialog` (Stage 2); explicit smoke test |
|
|
||||||
| datad-only backends (kucoin/deribit) → useless brokerd spawn | Low | already covered: `open_brokerd_dialog` forces `exec_mode='paper'` when no trades ep (`_ems.py:413-417`) BEFORE the lazy spawn; Stage 4 fail-fast |
|
|
||||||
| brokerd in-process symbology/ledger needs (ib `broker.py` imports `.symbols`; kraken uses `stream_messages`) | None | verified pure in-process imports; `enable_modules` gates RPC only |
|
|
||||||
| paper-mode test asserts brokerd spawns (`test_services.py:237`) | Low | Stage 3 deliberately inverts it |
|
|
||||||
| doubled per-broker clients (HTTP/WS, symcache loads) | Low | each actor already opens its own WS today; symcache is disk-read-mostly |
|
|
||||||
|
|
||||||
## Verification (per stage gates above, plus end-to-end)
|
|
||||||
|
|
||||||
- `pytest tests/test_services.py tests/test_feeds.py tests/test_ems.py` at every stage (kraken/binance public endpoints, no creds needed).
|
|
||||||
- Manual smoke matrix post-Stage-3: `pikerd -l info` + `piker chart btcusdt.spot.binance` → `piker services` shows `datad.binance`, no brokerd; submit paper order → `paperboi.binance` appears, still no brokerd; symbol search; `piker ledger <broker>.paper`.
|
|
||||||
- If ib creds/gateway available: live ib chart + small live order to exercise dual-actor client-id path (Stage 4).
|
|
||||||
|
|
||||||
## Critical files
|
|
||||||
|
|
||||||
- **new** `piker/data/_daemon.py` — all datad machinery
|
|
||||||
- `piker/brokers/_daemon.py` — slimmed brokerd fixture/init/spawn
|
|
||||||
- `piker/data/feed.py` — spawn cutover + actor-name assert
|
|
||||||
- `piker/clearing/_ems.py` — emsd lazy brokerd spawn (`open_brokerd_dialog`, `Router`)
|
|
||||||
- `piker/service/_actor_runtime.py`, `piker/service/__init__.py` — root mods + re-exports
|
|
||||||
- `piker/data/validate.py` — `get_eps()` helper
|
|
||||||
- backend `__init__.py`s (kraken/binance/deribit/ib) — `_datad_mods`/`_brokerd_mods`
|
|
||||||
- `piker/accounting/cli.py` — keyword-form dialog call
|
|
||||||
- `tests/test_services.py` — inverted + new assertions
|
|
||||||
|
|
@ -1,233 +0,0 @@
|
||||||
# Bounded IB recovery candidate
|
|
||||||
|
|
||||||
Prepared 2026-10-22 on `wkt/land_ib_recovery`, in the main checkout
|
|
||||||
`/home/goodboy/repos/piker`. Base:
|
|
||||||
`dc618229bd8c9c90aa3160e26677b553a7d472c7`.
|
|
||||||
Changes are unstaged and uncommitted. No acceptance is implied.
|
|
||||||
|
|
||||||
## Patch selection and provenance
|
|
||||||
|
|
||||||
Keep these existing landing commits:
|
|
||||||
|
|
||||||
| Commit | Reason |
|
|
||||||
| --- | --- |
|
|
||||||
| `4133f073` and its ancestors | Packaging, Nix, runtime API and pin work |
|
|
||||||
| `e8e0730a` | Correlate MethodProxy requests and responses |
|
|
||||||
| `7bef50b0` | One response reader, per-request waiters |
|
|
||||||
| `d1571ddf` | Propagate daemon-handler exceptions |
|
|
||||||
| `dc618229` | Isolate cached feed stream consumers |
|
|
||||||
|
|
||||||
Proposed separate implementation boundaries, in this order:
|
|
||||||
|
|
||||||
1. Back out the code and original test from
|
|
||||||
`635314c757c74a1cd95c5cf3dc7dfd03be984b00`.
|
|
||||||
This restores the direct-owner `_quote_streams` implementation.
|
|
||||||
Retain the historical prompt records. Replace the removed
|
|
||||||
mocked-cache regression with the real-bridge restart reproducer.
|
|
||||||
This backout is not a claim that arbitrary concurrent direct
|
|
||||||
callers have safe broadcast ownership.
|
|
||||||
2. Backport **only** `piker/brokers/ib/feed.py` from
|
|
||||||
`0846cbd4a4197e47415689cf5e9d146ba3174c32`.
|
|
||||||
Retain its seven history-adapter regressions in
|
|
||||||
`tests/test_ib_history.py`. This includes reset serialization,
|
|
||||||
completion/result signalling and bounded cancellation/failure.
|
|
||||||
It does not include the original SHM/storage orchestration patch.
|
|
||||||
3. Backport **only** `piker/brokers/ib/feed.py` from
|
|
||||||
`1da9b760a3bc6b96501110cab2b5d09846d42a1d`.
|
|
||||||
Retain `test_empty_history_preserves_request_boundary` and its
|
|
||||||
six cases. This preserves no-data request boundaries and wakes
|
|
||||||
the waiter. It does not include the original history-layer
|
|
||||||
endpoint/filtering changes. Add the real-bridge history recovery
|
|
||||||
case after these adapter changes are present.
|
|
||||||
4. Import the code and original regression from
|
|
||||||
`12b0d66e854247e92cb5a8668be71d816c72d856`.
|
|
||||||
Add a separate real-broadcaster/production-loop regression.
|
|
||||||
5. Remove the single-null-sample debugger stop in `get_null_segs()`.
|
|
||||||
This is a local UX cleanup retaining the existing `None` result.
|
|
||||||
It does not backport the null-grouping/deduplication rewrite in
|
|
||||||
`976db6c33c4b8ed07eee3047226417d3928d9cca`.
|
|
||||||
6. Record cross-boundary regression results and live qualification
|
|
||||||
evidence, distinguishing the known callback failure from passes.
|
|
||||||
|
|
||||||
All three imported source commits were authored by
|
|
||||||
`goodboy <goodboy_foss@protonmail.com>`; preserve that authorship
|
|
||||||
and the full source SHA when committing each partial backport.
|
|
||||||
Use explicit partial-backport wording, not a whole-commit cherry
|
|
||||||
claim. New real-runtime tests are this session's additional work.
|
|
||||||
|
|
||||||
The current aggregate working diff contains these boundaries in
|
|
||||||
the same feed file. Staging that entire file would combine the
|
|
||||||
first three boundaries. Reconstruct/stage the original per-source
|
|
||||||
patches in the above order when preparing commits.
|
|
||||||
|
|
||||||
## Quote-cache history correction
|
|
||||||
|
|
||||||
`4af885d90f6e4bcfa963f2f7a1d40436d3c928f3` and landing commit
|
|
||||||
`635314c757c74a1cd95c5cf3dc7dfd03be984b00` have the same stable
|
|
||||||
patch ID: `bc54892c25e957be112b135bb9e383ec4dd123c5`.
|
|
||||||
They are equivalent replays, not different implementations.
|
|
||||||
|
|
||||||
`wkt/ib_downstream_pin_eval@e929fdc2` lacks that quote-cache
|
|
||||||
change and retains direct ownership. The original qualification
|
|
||||||
did not validate the combination assembled for the landing.
|
|
||||||
|
|
||||||
After this candidate's edits, `piker/brokers/ib/feed.py` matches
|
|
||||||
the source at `1da9b760` byte-for-byte. Compared with the evaluated
|
|
||||||
tip, only the later schedule/reservation APIs are absent.
|
|
||||||
The EMS code patch has the same stable patch ID as the original
|
|
||||||
`12b0d66e` EMS code patch:
|
|
||||||
`50fee047e94fd67c8d9feab66859fbe3ab25b64c`.
|
|
||||||
|
|
||||||
## Real-runtime evidence
|
|
||||||
|
|
||||||
`tests/test_ib_quote_restart.py` uses real TCP Tractor actors,
|
|
||||||
an asyncio-infected child, the installed pinned Tractor bridge,
|
|
||||||
production Piker quote setup, and real IB/eventkit ticker events.
|
|
||||||
Only the external IB connection/input is replaced with replay data.
|
|
||||||
|
|
||||||
Before the quote-cache backout, cancellation after ticker delivery
|
|
||||||
raised `RemoteActorError[TrioTaskExited]` through
|
|
||||||
`maybe_open_context()` / `_Cache.run_ctx` teardown, matching the
|
|
||||||
reported fatal path.
|
|
||||||
|
|
||||||
After the backout:
|
|
||||||
|
|
||||||
- Two quote generations finish; each subscription is cancelled,
|
|
||||||
each asyncio producer is done and each ticker handler removed.
|
|
||||||
- A second RPC repeats the sequence in the same actor successfully.
|
|
||||||
- The strict callback-cleanliness case still fails: Tractor's
|
|
||||||
`signal_trio_when_done()` raises `TrioTaskExited` into asyncio's
|
|
||||||
exception handler even though the Trio consumer recovers.
|
|
||||||
|
|
||||||
The history integration case keeps `Client.bars()`, MethodProxy,
|
|
||||||
both relays, `get_bars()`, `wait_on_data_reset()` and
|
|
||||||
`data_reset_hack()` real. It replaces external contract discovery,
|
|
||||||
IB history replies and VNC input. A real IB error event announces
|
|
||||||
farm recovery. The replay rejects retries before recovery and
|
|
||||||
proves one reset followed by completion for both 1s and 60s data.
|
|
||||||
|
|
||||||
The new EMS case runs the production dark-clearing loop over a
|
|
||||||
real Tractor broadcaster: both EMS and a sibling receive a tick,
|
|
||||||
EMS cancellation removes only its subscription, and the sibling
|
|
||||||
continues with an unchanged strict lag policy.
|
|
||||||
|
|
||||||
## Verification
|
|
||||||
|
|
||||||
Unique selected cases: **38 passed, 1 failed**.
|
|
||||||
The failure is intentionally visible, not skipped or xfailed.
|
|
||||||
|
|
||||||
Run from the checkout's working Nix environment:
|
|
||||||
|
|
||||||
```xsh
|
|
||||||
timeout -k 5 45 py313/bin/python -m pytest -p no:xonsh -q -x --tb=short tests/test_ib_history.py tests/test_ems_broadcast.py tests/test_ib_method_proxy.py tests/test_feed_broadcast.py
|
|
||||||
timeout -k 5 45 py313/bin/python -m pytest -p no:xonsh -q --tb=short tests/test_ib_quote_restart.py
|
|
||||||
timeout -k 5 45 py313/bin/python -m pytest -p no:xonsh -q -x --tb=short tests/test_logging.py tests/test_startup.py
|
|
||||||
timeout -k 5 45 py313/bin/python -m pytest -p no:xonsh -q -x --tb=short tests/test_services.py::test_runtime_boot
|
|
||||||
git diff --check
|
|
||||||
```
|
|
||||||
|
|
||||||
The adapter/proxy/feed/EMS selection initially passed 21 cases;
|
|
||||||
the added real EMS case then passed with its imported case.
|
|
||||||
The bridge file passed two cases and failed callback cleanliness.
|
|
||||||
Logging/startup passed 13 cases; runtime boot passed one with the
|
|
||||||
existing `Registrar.uid` deprecation warning.
|
|
||||||
|
|
||||||
The agent shell inherited another project's environment and lacked
|
|
||||||
native libraries. Tests used the existing Piker `py313` via
|
|
||||||
`/tmp/nix-shell.zfKbeB/opencode/piker_test_env.py`, which supplies
|
|
||||||
libraries from the pinned flake's existing Qt dependency closure,
|
|
||||||
excluding Nix Qt itself and glibc. No environment sync or shellHook
|
|
||||||
ran. Piker resolved to this checkout; Tractor resolved to its
|
|
||||||
installed package in `py313`.
|
|
||||||
|
|
||||||
Workflow-state preparation initially reported changed legacy
|
|
||||||
files/modes after migration. On 2026-10-23 the human authorized
|
|
||||||
reconciliation. Inspection found only two missing legacy guidance
|
|
||||||
snapshots; the neutral copies matched their saved hashes and modes.
|
|
||||||
Restoring those local snapshots resolved preparation without
|
|
||||||
changing archived messages or the original migration receipt.
|
|
||||||
|
|
||||||
## Human live qualification, 2026-10-23
|
|
||||||
|
|
||||||
The human ran these journeys with the installed Tractor pin,
|
|
||||||
the existing IB Gateway and pre-existing NativeDB. The agent did
|
|
||||||
not operate the broker connection, VNC, orders or database.
|
|
||||||
|
|
||||||
- Initial QQQ startup/backfill recovered after pacing error 162
|
|
||||||
with one reset attempt, then displayed live updates. The reset
|
|
||||||
waiter appears to have timed out; the log did not establish an
|
|
||||||
acknowledged farm reset or live quote-stream restart.
|
|
||||||
- Sequential chart exits and reattachment preserved all four
|
|
||||||
daemon PIDs; a read-only process/listener check confirmed this.
|
|
||||||
- The overlapping two-chart journey used `pikerd@961108`,
|
|
||||||
`brokerd.ib@964765`, `samplerd@965900` and `emsd@969015`.
|
|
||||||
B started at 08:16:31; C at 08:17:33. B closed at 08:18:11,
|
|
||||||
replacement B started at 08:18:24 and had FSPs up by 08:18:28,
|
|
||||||
then C closed at 08:18:37. The human reported the exercise worked;
|
|
||||||
survivor updates are a manual observation, while timestamps and
|
|
||||||
unchanged daemon identities corroborate overlap and service reuse.
|
|
||||||
- Startup in that journey hit the explicit single-null-sample
|
|
||||||
breakpoint several times before the charts overlapped. The small
|
|
||||||
debugger cleanup above was applied afterward; this live run does
|
|
||||||
not itself verify that later edit.
|
|
||||||
- Departing charts still report cancellation groups and an FSP
|
|
||||||
receiver-close `ClosedResourceError`, followed by the Qt close
|
|
||||||
handler's self-SIGINT. Peer-disconnected final result warnings
|
|
||||||
remain contained in the persistent services.
|
|
||||||
- Daemon SIGINT at 08:19:43 reproduced cached shared-client teardown:
|
|
||||||
`load_clients_for_trio()` / `_Cache.run_ctx` promoted
|
|
||||||
`TrioTaskExited` through `open_trade_dialog()` and entered PDB.
|
|
||||||
After the human continued, all three children were reported
|
|
||||||
exited with status 0 and the root terminated at 08:19:45.
|
|
||||||
Cancellation-only RPC error reporting and late cancel-ack
|
|
||||||
diagnostics also remain.
|
|
||||||
|
|
||||||
The human explicitly accepted deferring this daemon teardown fix
|
|
||||||
from #94 and requested loud attribution for the bounded backports.
|
|
||||||
This is an operational smoke result, not clean-shutdown coverage.
|
|
||||||
|
|
||||||
## Downstream reconciliation requirements
|
|
||||||
|
|
||||||
Before landing #94, the human wants an easy replay of the downstream
|
|
||||||
histories intended to follow it. Rehearse against the agreed #94
|
|
||||||
tree in disposable worktrees, then inspect range-diffs, final code
|
|
||||||
and relevant regressions. This note does not authorize rewriting
|
|
||||||
the original branches or changing acceptance markers.
|
|
||||||
|
|
||||||
- `0846cbd4` and `1da9b760` are partial backports. Their imported
|
|
||||||
adapter hunks may already be present, but their history/SHM/storage
|
|
||||||
changes and omitted integration tests remain necessary downstream.
|
|
||||||
Never drop either whole source commit merely because `feed.py`
|
|
||||||
matches or its adapter hunk becomes empty.
|
|
||||||
- `12b0d66e` has source-equivalent EMS code and its imported test,
|
|
||||||
plus an additional local real-broadcaster test. Preserve that extra
|
|
||||||
regression when resolving test-file overlap.
|
|
||||||
- The quote-cache backout intentionally restores direct ownership
|
|
||||||
for this landing. Reintroducing original `4af885d9` downstream
|
|
||||||
requires qualifying cached-owner teardown with the matching
|
|
||||||
Tractor revision; the backout is not a permanent ownership design.
|
|
||||||
- The local single-null breakpoint removal retains legacy behavior.
|
|
||||||
Prefer the full downstream `976db6c3` algorithm when it is replayed;
|
|
||||||
preserving this `return None` guard would lose that later fix.
|
|
||||||
- Real-bridge tests and manual qualification notes are local work,
|
|
||||||
not evidence that omitted portions of the source commits landed.
|
|
||||||
|
|
||||||
Commit messages must name full source SHAs, original authorship,
|
|
||||||
imported paths/tests, omissions, local additions and replay guidance.
|
|
||||||
Use explicit partial-backport language for adapter subsets; do not
|
|
||||||
claim a full cherry-pick or reuse the source's broader fix claims.
|
|
||||||
|
|
||||||
## Remaining decision
|
|
||||||
|
|
||||||
This candidate removes the reproduced fatal quote-restart
|
|
||||||
propagation and imports existing reset fixes. The human exercised
|
|
||||||
live QQQ startup, backfill and concurrent chart consumer lifetimes.
|
|
||||||
Completion-callback and teardown errors remain; the human accepts
|
|
||||||
them as follow-ups outside the bounded #94 landing.
|
|
||||||
The storage/SHM portions of the downstream fixes are not covered by
|
|
||||||
the adapter backport or its integration tests.
|
|
||||||
|
|
||||||
`ddac3553` is Deribit-specific and omitted. Qt lifecycle commit
|
|
||||||
`1a2d4893` stays a separate follow-up. Further Tractor teardown
|
|
||||||
changes or storage imports require reassessing the bounded landing
|
|
||||||
rather than silently expanding this candidate.
|
|
||||||
|
|
@ -1,71 +0,0 @@
|
||||||
# IB recovery qualification, 2026-11-06
|
|
||||||
|
|
||||||
This records human-operated qualification of the bounded #94
|
|
||||||
landing on `wkt/land_ib_recovery`, starting from `495c0915` plus
|
|
||||||
the narrow history catch described below. It supplements the
|
|
||||||
historical evidence in `ai/ib-recovery-candidate.md`.
|
|
||||||
|
|
||||||
## Runtime and dependency source
|
|
||||||
|
|
||||||
The evaluated environment uses Python 3.13, Trio 0.34.0 and Tractor
|
|
||||||
`8fe574c1e447358dafd8ebf5ada3b0714c688f78`. Tractor was an editable
|
|
||||||
local checkout during evaluation. The human subsequently published
|
|
||||||
that exact commit as GitHub's `goodboy/tractor:piker_pin`; a read-only
|
|
||||||
remote query confirmed the published OID before restoring Piker's
|
|
||||||
Git source and recording the same OID in `uv.lock`.
|
|
||||||
|
|
||||||
Offline lock validation resolves 126 packages successfully. No
|
|
||||||
environment sync was performed. This validates the lock metadata;
|
|
||||||
it does not claim a fresh installation from the published source.
|
|
||||||
|
|
||||||
## Manual observations
|
|
||||||
|
|
||||||
The human operated Gateway, chart windows and paper orders. The
|
|
||||||
agent did not operate live services or the account.
|
|
||||||
|
|
||||||
- Same-feed overlap, close/reopen and paper order submit/cancel
|
|
||||||
worked after the existing order-dialog guard was imported.
|
|
||||||
- Different-feed overlap originally exposed a timed-out one-second
|
|
||||||
MNQ March 2027 gap query. PDB showed `timedout=True` and
|
|
||||||
`out is None`. Its uncaught `NoData` canceled sibling history and
|
|
||||||
quote tasks even though `pikerd` remained alive.
|
|
||||||
- After the narrow catch, the human reported the overlap journeys
|
|
||||||
working, then confirmed cases 5 and 6 with Ctrl+Alt+F: overlapping
|
|
||||||
live feeds survived the Gateway data reset; closing all charts
|
|
||||||
and reopening with `pikerd` left running restored live updates.
|
|
||||||
- Earlier Ctrl+Shift+R attempts left feeds live, but do not prove
|
|
||||||
that the intended Gateway data reset was triggered.
|
|
||||||
|
|
||||||
These are manual observations, not automated timing or latency
|
|
||||||
measurements. Ctrl+Alt+R was not qualified here; Piker's utility
|
|
||||||
labels it a connection reset.
|
|
||||||
|
|
||||||
## Narrow history backport
|
|
||||||
|
|
||||||
Only the `NoData`/`DataUnavailable` catch around `get_hist()` in
|
|
||||||
`maybe_fill_null_segments()` is copied from
|
|
||||||
`0846cbd4a4197e47415689cf5e9d146ba3174c32`, authored by
|
|
||||||
`goodboy <goodboy_foss@protonmail.com>`. The existing gap fallback
|
|
||||||
remains in place; this does not import broader history repair.
|
|
||||||
|
|
||||||
The new regression uses real null detection and checks that a
|
|
||||||
sibling in the same nursery still runs after either provider
|
|
||||||
exception. Both cases failed before the catch. With it, the two
|
|
||||||
focused history files passed: 17 tests. Earlier bounded-recovery
|
|
||||||
and Tractor qualification results remain separate evidence, not
|
|
||||||
fresh test runs of the final dependency commit tree.
|
|
||||||
|
|
||||||
## Remaining recovery limits
|
|
||||||
|
|
||||||
A full Gateway restart was triggered accidentally through its
|
|
||||||
auto-restart settings dialog. The disconnect delivered the string
|
|
||||||
`Peer closed connection.` to `deliver_trade_events()`. The string
|
|
||||||
branch then reached `err['reqid']`, raising `TypeError`; its `code`
|
|
||||||
local can also retain an earlier event's value. Full Gateway restart
|
|
||||||
recovery is outside this bounded landing.
|
|
||||||
|
|
||||||
The earlier history-failure teardown also exposed a separate
|
|
||||||
Tractor completion-callback assertion after `BrokenResourceError`.
|
|
||||||
The history catch does not establish that this callback race is
|
|
||||||
fixed. Broader history recovery and these disconnect paths remain
|
|
||||||
downstream work.
|
|
||||||
|
|
@ -1,52 +0,0 @@
|
||||||
---
|
|
||||||
created_at: 2026-10-07T16:54:53Z
|
|
||||||
generated_by:
|
|
||||||
harness: opencode
|
|
||||||
provider: openai
|
|
||||||
model: gpt-6.1-sol
|
|
||||||
git_ref: bde2457b
|
|
||||||
scope: code
|
|
||||||
substantive: true
|
|
||||||
raw_file: 20261007T165453Z_bde2457b_prompt_io.raw.md
|
|
||||||
---
|
|
||||||
|
|
||||||
## Prompt
|
|
||||||
|
|
||||||
The user requested fixing all seven local PR94 review findings and
|
|
||||||
exporting the review for human inspection in Tuicr. They asked that a
|
|
||||||
Tractor pin adjustment be considered before adding Piker workarounds.
|
|
||||||
|
|
||||||
## Response summary
|
|
||||||
|
|
||||||
Correct the endpoint defaults, Click argument shape, daemon binding and
|
|
||||||
registrar option handling, chart fallback, and stray debugger pause.
|
|
||||||
Keep the absent datad implementation's test outside this landing
|
|
||||||
branch. Add offline startup coverage and export original findings
|
|
||||||
without modifying review states. The checked newer Tractor address
|
|
||||||
branch still rejects mixed transports, so leave the dependency pin.
|
|
||||||
|
|
||||||
## Files changed
|
|
||||||
|
|
||||||
- `config/conf.toml`: TCP-only default endpoints.
|
|
||||||
- `piker/cli/__init__.py`: repeatable and intact multiaddresses.
|
|
||||||
- `piker/service/_actor_runtime.py`: validate daemon listeners.
|
|
||||||
- `piker/service/_daemon.py`: initialize and consume registrar options.
|
|
||||||
- `piker/ui/cli.py`: fallback to automatic chart binding.
|
|
||||||
- `piker/brokers/binance/feed.py`: remove unconditional debugger entry.
|
|
||||||
- `tests/test_services.py`: defer test requiring missing datad API.
|
|
||||||
- `tests/test_startup.py`: CLI/config and service lifecycle regressions.
|
|
||||||
|
|
||||||
## Human edits
|
|
||||||
|
|
||||||
The human identified that datad tests arrived through the agent's
|
|
||||||
mixed cherry-picks and explicitly requested deferring that test. They
|
|
||||||
requested a chart default for existing configs, expanded the task to
|
|
||||||
all review findings, and directed checking whether a newer Tractor pin
|
|
||||||
could reduce the fix scope. No direct manual source edits were observed
|
|
||||||
during these changes; acceptance remains with the human.
|
|
||||||
|
|
||||||
During the human review pass, the human confirmed removing the Binance
|
|
||||||
pause and correcting CLI endpoint parsing. They requested an explicit
|
|
||||||
template note about this pin's single-transport limitation and detailed
|
|
||||||
comments distinguishing registrar discovery endpoints from daemon bind
|
|
||||||
endpoints. Those explanations were added at the human's direction.
|
|
||||||
|
|
@ -1,61 +0,0 @@
|
||||||
---
|
|
||||||
created_at: 2026-10-07T16:54:53Z
|
|
||||||
generated_by:
|
|
||||||
harness: opencode
|
|
||||||
provider: openai
|
|
||||||
model: gpt-6.1-sol
|
|
||||||
git_ref: bde2457b
|
|
||||||
diff_cmd: git diff HEAD~1..HEAD
|
|
||||||
---
|
|
||||||
|
|
||||||
## Prompt
|
|
||||||
|
|
||||||
Fix all findings from the local PR94 review, prefer a Tractor pin
|
|
||||||
change if it reduces work, and export the original review for Tuicr.
|
|
||||||
|
|
||||||
## Generated changes
|
|
||||||
|
|
||||||
> `git diff HEAD~1..HEAD -- config/conf.toml`
|
|
||||||
|
|
||||||
Use TCP-only defaults, retaining commented UDS alternatives. Both
|
|
||||||
checked Tractor address branch tips still reject mixed transports.
|
|
||||||
|
|
||||||
> `git diff HEAD~1..HEAD -- piker/cli/__init__.py`
|
|
||||||
|
|
||||||
Make Click multiaddresses repeatable and normalize single strings.
|
|
||||||
|
|
||||||
> `git diff HEAD~1..HEAD -- piker/service/_actor_runtime.py`
|
|
||||||
|
|
||||||
Validate requested daemon bind addresses instead of remote registrars.
|
|
||||||
Normalize accepted endpoints and allow TCP ephemeral port requests.
|
|
||||||
|
|
||||||
> `git diff HEAD~1..HEAD -- piker/service/_daemon.py`
|
|
||||||
|
|
||||||
Resolve explicit or inherited registrars independently of other daemon
|
|
||||||
options and consume the explicit keyword before forwarding kwargs.
|
|
||||||
|
|
||||||
> `git diff HEAD~1..HEAD -- piker/ui/cli.py`
|
|
||||||
|
|
||||||
Default chart bind endpoints to None when network config is absent.
|
|
||||||
|
|
||||||
> `git diff HEAD~1..HEAD -- piker/brokers/binance/feed.py`
|
|
||||||
|
|
||||||
Remove the unconditional market-info debugger pause.
|
|
||||||
|
|
||||||
> `git diff HEAD~1..HEAD -- tests/test_services.py tests/test_startup.py`
|
|
||||||
|
|
||||||
Remove the prematurely cherry-picked datad test and add offline startup
|
|
||||||
regressions, including a real registrar/child daemon lifecycle check.
|
|
||||||
|
|
||||||
## Verification
|
|
||||||
|
|
||||||
- Startup and logging tests: 13 passed.
|
|
||||||
- Existing runtime boot and tagged registrar tests: 2 passed.
|
|
||||||
- Scoped native Ruff and diff checks pass.
|
|
||||||
- The first new actor test used a removed nursery API; it was corrected
|
|
||||||
to the installed start_actor/Portal.run interface before passing.
|
|
||||||
- Import initially failed without native libraries. Tests use the
|
|
||||||
existing flake's GCC, zlib, Qt and PyQt paths without executing its
|
|
||||||
environment-rebuilding shell hook.
|
|
||||||
- Original seven findings exported locally as Tuicr review-add payloads
|
|
||||||
bound to bde2457b; no session comments or remote PR were changed.
|
|
||||||
|
|
@ -1,45 +0,0 @@
|
||||||
---
|
|
||||||
created_at: 2026-10-10T23:28:52Z
|
|
||||||
generated_by:
|
|
||||||
harness: opencode
|
|
||||||
provider: openai
|
|
||||||
model: openai/gpt-6.1-sol
|
|
||||||
git_ref: 1fa45dfa
|
|
||||||
scope: config
|
|
||||||
substantive: true
|
|
||||||
raw_file: 20261010T232852Z_1fa45dfa_prompt_io.raw.md
|
|
||||||
---
|
|
||||||
|
|
||||||
## Prompt
|
|
||||||
|
|
||||||
The human rejected keeping the April Tractor pin as a landing baseline.
|
|
||||||
They requested the reviewed `wkt/addr_unpacking` branch, removal of the
|
|
||||||
new rollback adaptations, reuse of original Piker histories where
|
|
||||||
possible, and a candidate prepared within their gateway relocation
|
|
||||||
window. They asked specifically whether the runtime validation review
|
|
||||||
hunk duplicated older endpoint history.
|
|
||||||
|
|
||||||
## Response summary
|
|
||||||
|
|
||||||
Select the reviewed address branch at an immutable locked revision and
|
|
||||||
preserve all committed application/test compatibility code. Verify the
|
|
||||||
minimal dependency boundary offline and preview existing consumer
|
|
||||||
fixes without importing unrelated datad/backfill changes. Distinguish
|
|
||||||
original-source test results from live teardown acceptance, which is
|
|
||||||
not available in this preparation window.
|
|
||||||
|
|
||||||
## Files changed
|
|
||||||
|
|
||||||
- `pyproject.toml`: select Tractor `wkt/addr_unpacking`.
|
|
||||||
- `uv.lock`: resolve reviewed `a966928d` and setproctitle 1.3.8 while
|
|
||||||
preserving Polars 1.36.0 and the rest of the locked graph.
|
|
||||||
|
|
||||||
## Human edits
|
|
||||||
|
|
||||||
The human redirected the task from an older known-good runtime to a
|
|
||||||
modern landing stack and insisted on reusing original history rather
|
|
||||||
than building new compatibility adapters. They proposed stricter
|
|
||||||
exception-group behavior as a hypothesis and set the live-test/time
|
|
||||||
constraints. Those directions materially changed the dependency
|
|
||||||
boundary. No direct manual edits to this final boundary were observed;
|
|
||||||
runtime acceptance and landing remain human-owned.
|
|
||||||
|
|
@ -1,52 +0,0 @@
|
||||||
---
|
|
||||||
created_at: 2026-10-10T23:28:52Z
|
|
||||||
generated_by:
|
|
||||||
harness: opencode
|
|
||||||
provider: openai
|
|
||||||
model: openai/gpt-6.1-sol
|
|
||||||
git_ref: 1fa45dfa
|
|
||||||
diff_cmd: git diff HEAD~1..HEAD
|
|
||||||
---
|
|
||||||
|
|
||||||
## Prompt
|
|
||||||
|
|
||||||
Undo the April Tractor rollback and select the reviewed
|
|
||||||
`wkt/addr_unpacking` branch. Prefer original downstream compatibility
|
|
||||||
commits to new adaptations, and prepare an offline-verified candidate
|
|
||||||
while live IB verification is deferred for gateway relocation.
|
|
||||||
|
|
||||||
## Generated changes
|
|
||||||
|
|
||||||
> `git diff HEAD~1..HEAD -- pyproject.toml uv.lock`
|
|
||||||
|
|
||||||
Change Tractor's source from `piker_pin` to `wkt/addr_unpacking` and
|
|
||||||
lock it at `a966928dead4cbf369ad244834ce9dbe988ac8ca`. Restore application
|
|
||||||
and test sources byte-for-byte to the committed PR94 tree. Keep the
|
|
||||||
rejected rollback journal only in ignored local experiment reports.
|
|
||||||
Preserve Polars 1.36.0 by retaining its exact existing lock blocks;
|
|
||||||
setproctitle re-resolves from 1.3.7 to 1.3.8. `uv sync --locked` accepts
|
|
||||||
the resulting graph. Actual imports confirm the reviewed Tractor pin
|
|
||||||
and Trio 0.32.0, not the hypothesized Trio 0.33.
|
|
||||||
|
|
||||||
## Original-history previews
|
|
||||||
|
|
||||||
The existing MethodProxy stack is retained. Read-only merge previews
|
|
||||||
show original `4af885d9` and `b691422b` can be applied without the datad
|
|
||||||
cutover. Original-source regressions run through temporary in-memory
|
|
||||||
overlays against the real pinned runtime: seven cases pass and one
|
|
||||||
cache-hit case fails because `log.at_least_level('info')` looks up INFO
|
|
||||||
in a custom-level mapping. A separate standard logging correction
|
|
||||||
produces eight passing cases. Its patch and execution handoff are saved
|
|
||||||
under ignored `.ai/code-review/reports/pr94_modern_stack/`; no consumer
|
|
||||||
cherry-picks or source edits are performed by this boundary.
|
|
||||||
|
|
||||||
## Verification
|
|
||||||
|
|
||||||
- Startup and logging: 13 passed.
|
|
||||||
- Runtime boot and tagged registrar normalization: 2 passed.
|
|
||||||
- Locked sync and source whitespace checks pass.
|
|
||||||
- Original application/test trees are unchanged from PR94 HEAD.
|
|
||||||
- Consumer overlay with explicit logging correction: 8 passed.
|
|
||||||
- Correction patch validates against its exact preview tree using a
|
|
||||||
private temporary index; the real index is untouched.
|
|
||||||
- Live IB startup, SIGINT and surviving-pikerd reattachment are pending.
|
|
||||||
|
|
@ -1,45 +0,0 @@
|
||||||
---
|
|
||||||
created_at: 2026-11-06T18:56:28.621577+00:00
|
|
||||||
generated_by:
|
|
||||||
harness: codex
|
|
||||||
provider: OpenAI
|
|
||||||
git_ref: 495c0915
|
|
||||||
scope: code
|
|
||||||
substantive: true
|
|
||||||
raw_file: 20261106T185628Z_495c0915_215e9b_prompt_io.raw.md
|
|
||||||
---
|
|
||||||
|
|
||||||
## Prompt
|
|
||||||
|
|
||||||
User reported a different-feed overlap failure and asked whether
|
|
||||||
a fix could be extracted without importing broader history work.
|
|
||||||
PDB confirmed MNQ March 2027 one-second history timed out and
|
|
||||||
returned None, producing NoData. After continuing PDB, sibling
|
|
||||||
history and quote tasks terminated while pikerd remained alive.
|
|
||||||
User requested continuing with the narrow backport.
|
|
||||||
|
|
||||||
## Response summary
|
|
||||||
|
|
||||||
Partial backport of the NoData/DataUnavailable catch from
|
|
||||||
0846cbd4a4197e47415689cf5e9d146ba3174c32 into
|
|
||||||
maybe_fill_null_segments(). The existing gap fallback remains in use.
|
|
||||||
A new parametrized regression uses real null detection and verifies
|
|
||||||
that a sibling in the same nursery survives the failed repair.
|
|
||||||
|
|
||||||
Both new cases failed before the catch. The focused history suites
|
|
||||||
passed after it: 17 passed. This does not qualify live overlap or fix
|
|
||||||
IB reset churn or the separate Tractor completion-callback assertion.
|
|
||||||
|
|
||||||
## Files changed
|
|
||||||
|
|
||||||
- `piker/tsp/_history.py`: existing downstream exception catch.
|
|
||||||
- `tests/test_history_backfill.py`: sibling survival regression.
|
|
||||||
|
|
||||||
## Human edits
|
|
||||||
|
|
||||||
User supplied live reproduction, PDB locals and cancellation logs,
|
|
||||||
and explicitly constrained scope to a small history extraction.
|
|
||||||
The production catch is copied from existing downstream commit
|
|
||||||
0846cbd4, not newly designed here. New regression is agent-authored;
|
|
||||||
no direct human source edits observed during this patch. Review and
|
|
||||||
live qualification remain with the user.
|
|
||||||
|
|
@ -1,22 +0,0 @@
|
||||||
---
|
|
||||||
created_at: 2026-11-06T18:56:28.621577+00:00
|
|
||||||
generated_by:
|
|
||||||
harness: codex
|
|
||||||
provider: OpenAI
|
|
||||||
git_ref: 495c0915
|
|
||||||
diff_cmd: git diff HEAD~1..HEAD
|
|
||||||
---
|
|
||||||
|
|
||||||
> `git diff HEAD~1..HEAD -- piker/tsp/_history.py`
|
|
||||||
|
|
||||||
> `git diff HEAD~1..HEAD -- tests/test_history_backfill.py`
|
|
||||||
|
|
||||||
Partial backport of the NoData/DataUnavailable catch from
|
|
||||||
0846cbd4a4197e47415689cf5e9d146ba3174c32 into
|
|
||||||
maybe_fill_null_segments(). The existing gap fallback remains in use.
|
|
||||||
A new parametrized regression uses real null detection and verifies
|
|
||||||
that a sibling in the same nursery survives the failed repair.
|
|
||||||
|
|
||||||
Both new cases failed before the catch. The focused history suites
|
|
||||||
passed after it: 17 passed. This does not qualify live overlap or fix
|
|
||||||
IB reset churn or the separate Tractor completion-callback assertion.
|
|
||||||
|
|
@ -1,41 +0,0 @@
|
||||||
---
|
|
||||||
created_at: 2026-11-06T20:17:46.622316+00:00
|
|
||||||
generated_by:
|
|
||||||
harness: codex
|
|
||||||
provider: OpenAI
|
|
||||||
git_ref: 495c0915
|
|
||||||
scope: config
|
|
||||||
substantive: true
|
|
||||||
raw_file: 20261106T201746Z_495c0915_612dfc_prompt_io.raw.md
|
|
||||||
---
|
|
||||||
|
|
||||||
## Prompt
|
|
||||||
|
|
||||||
The human requested a commit plan, then a fast-forward landing of
|
|
||||||
the #94 branch and PR updates. They agreed the temporary local
|
|
||||||
Tractor source must be replaced, published piker_pin themselves,
|
|
||||||
and reported "pushed it". They also supplied the manual recovery
|
|
||||||
results and full Gateway restart failure diagnostics.
|
|
||||||
|
|
||||||
## Response summary
|
|
||||||
|
|
||||||
Restore the Git Tractor source after local qualification and lock
|
|
||||||
its published piker_pin at 8fe574c1e447358dafd8ebf5ada3b0714c688f78.
|
|
||||||
Retain the evaluated Trio 0.34.0 lock and require Trio >=0.34.0.
|
|
||||||
Record the human's overlap, data-reset and reopen observations,
|
|
||||||
the narrow history extraction, and full-restart recovery limits.
|
|
||||||
Offline uv lock validation passed without an environment sync.
|
|
||||||
|
|
||||||
## Files changed
|
|
||||||
|
|
||||||
- `pyproject.toml`: Trio minimum; restore existing Tractor Git source.
|
|
||||||
- `uv.lock`: published Tractor OID and evaluated Trio 0.34.0.
|
|
||||||
- `ai/ib-recovery-qualification-20261106.md`: scoped evidence.
|
|
||||||
|
|
||||||
## Human edits
|
|
||||||
|
|
||||||
The human selected the bounded landing scope, performed live tests,
|
|
||||||
identified the more aggressive reset distinction, and published
|
|
||||||
the tested Tractor ref. The agent applied the metadata edits and
|
|
||||||
wrote the qualification record from those observations. No human
|
|
||||||
acceptance or task-state transitions are inferred by this record.
|
|
||||||
|
|
@ -1,21 +0,0 @@
|
||||||
---
|
|
||||||
created_at: 2026-11-06T20:17:46.622316+00:00
|
|
||||||
generated_by:
|
|
||||||
harness: codex
|
|
||||||
provider: OpenAI
|
|
||||||
git_ref: 495c0915
|
|
||||||
diff_cmd: git diff HEAD~1..HEAD
|
|
||||||
---
|
|
||||||
|
|
||||||
> `git diff HEAD~1..HEAD -- pyproject.toml`
|
|
||||||
|
|
||||||
> `git diff HEAD~1..HEAD -- uv.lock`
|
|
||||||
|
|
||||||
> `git diff HEAD~1..HEAD -- ai/ib-recovery-qualification-20261106.md`
|
|
||||||
|
|
||||||
Restore the Git Tractor source after local qualification and lock
|
|
||||||
its published piker_pin at 8fe574c1e447358dafd8ebf5ada3b0714c688f78.
|
|
||||||
Retain the evaluated Trio 0.34.0 lock and require Trio >=0.34.0.
|
|
||||||
Record the human's overlap, data-reset and reopen observations,
|
|
||||||
the narrow history extraction, and full-restart recovery limits.
|
|
||||||
Offline uv lock validation passed without an environment sync.
|
|
||||||
|
|
@ -1,11 +0,0 @@
|
||||||
# AI Prompt I/O Log
|
|
||||||
|
|
||||||
This directory records substantive AI-assisted changes, prompts,
|
|
||||||
original harness identity, and unedited outputs. Human contributors
|
|
||||||
remain accountable for code decisions.
|
|
||||||
|
|
||||||
Records follow the [NLNet generative AI policy][policy]. Each
|
|
||||||
structured entry links its immutable `.raw.md` partner. Code output
|
|
||||||
uses Git diff references to avoid duplication.
|
|
||||||
|
|
||||||
[policy]: https://nlnet.nl/foundation/policies/generativeAI/
|
|
||||||
|
|
@ -1,59 +0,0 @@
|
||||||
---
|
|
||||||
model: claude-fable-5[1m]
|
|
||||||
service: claude
|
|
||||||
session: 32d15f9a-b2d3-4c26-bdc9-190219141a25
|
|
||||||
timestamp: 2026-06-10T17:08:59Z
|
|
||||||
git_ref: datad_service
|
|
||||||
diff_cmd: git log -1 -p --follow -- ai/prompt-io/claude/20260610T170859Z_75cefe10_prompt_io.md
|
|
||||||
scope: code
|
|
||||||
substantive: true
|
|
||||||
raw_file: 20260610T170859Z_75cefe10_prompt_io.raw.md
|
|
||||||
---
|
|
||||||
|
|
||||||
## Prompt
|
|
||||||
|
|
||||||
Session-initiating instruction (driving all 7 commits in
|
|
||||||
this series):
|
|
||||||
|
|
||||||
> ok i want you to become the distributed runtime and
|
|
||||||
> concurrency expert for this project - namely acquire
|
|
||||||
> a deep understanding of tractor and how it's used.
|
|
||||||
> then i want you to attempt to factor our current
|
|
||||||
> brokerd service daemon into 2 daemons: a brokerd
|
|
||||||
> which only hosts live/paper trading endpoint tasks
|
|
||||||
> [..] a new `datad` subdaemon which in a separate
|
|
||||||
> subactor serves all the data feed service tasks [..]
|
|
||||||
> give me a mega detailed plan on how to approach this,
|
|
||||||
> and a staged approach for the implementation.
|
|
||||||
|
|
||||||
Proximate driver for THIS commit: during the approved
|
|
||||||
plan's stage-0 test gate the agent discovered test
|
|
||||||
subactors were writing into the user's REAL
|
|
||||||
`~/.config/piker/accounting/` files (a bogus paper fill
|
|
||||||
landed in `account.kraken.paper.toml`) because the old
|
|
||||||
test-dir override in `config.get_app_dir()` was
|
|
||||||
commented out and could never work in spawned
|
|
||||||
subactors anyway. Fix applied autonomously under the
|
|
||||||
approved plan; no explicit per-fix user prompt.
|
|
||||||
|
|
||||||
## Response summary
|
|
||||||
|
|
||||||
Restore `pytest` config-dir isolation by resolving the
|
|
||||||
per-test tmp dir lazily (at conf-path access time) from
|
|
||||||
`tractor` runtime-vars, which propagate down the actor
|
|
||||||
tree; route all conf-path derivation through
|
|
||||||
`config.get_conf_dir()` so the override is effective in
|
|
||||||
every (sub)actor.
|
|
||||||
|
|
||||||
## Files changed
|
|
||||||
|
|
||||||
- `piker/config.py` — add `_maybe_use_test_dir()`;
|
|
||||||
hook in `get_conf_dir()`; route `get_conf_path()` +
|
|
||||||
`load()` mkdir through it
|
|
||||||
- `piker/accounting/_ledger.py` — derive ledger dir
|
|
||||||
via `config.get_conf_dir()` (not the global)
|
|
||||||
- `piker/accounting/_pos.py` — same for account files
|
|
||||||
|
|
||||||
## Human edits
|
|
||||||
|
|
||||||
None — committed as generated.
|
|
||||||
|
|
@ -1,46 +0,0 @@
|
||||||
---
|
|
||||||
model: claude-fable-5[1m]
|
|
||||||
service: claude
|
|
||||||
timestamp: 2026-06-10T17:08:59Z
|
|
||||||
git_ref: datad_service
|
|
||||||
diff_cmd: git log -1 -p --follow -- ai/prompt-io/claude/20260610T170859Z_75cefe10_prompt_io.md
|
|
||||||
---
|
|
||||||
|
|
||||||
NOTE: diff-ref mode entry (code committed in the same
|
|
||||||
commit as this log); backfilled from the live dev
|
|
||||||
session transcript per the `/prompt-io` skill rules.
|
|
||||||
|
|
||||||
> `git log -1 -p --follow -- piker/config.py`
|
|
||||||
|
|
||||||
Generated: `config._maybe_use_test_dir()` — lazily
|
|
||||||
reads `piker_test_dir` from
|
|
||||||
`tractor.runtime._state._runtime_vars['piker_vars']`
|
|
||||||
(pre-loaded by `open_piker_runtime()` from the
|
|
||||||
`tests.conftest._open_test_pikerd()` overrides) and
|
|
||||||
calls `_override_config_dir()` when set. Hooked at the
|
|
||||||
top of `get_conf_dir()`; `get_conf_path()` and
|
|
||||||
`load()`'s dir-creation rerouted through
|
|
||||||
`get_conf_dir()`.
|
|
||||||
|
|
||||||
> `git log -1 -p --follow -- piker/accounting/_ledger.py`
|
|
||||||
> `git log -1 -p --follow -- piker/accounting/_pos.py`
|
|
||||||
|
|
||||||
Generated: ledger/account dir derivation switched from
|
|
||||||
the `config._config_dir` module global to
|
|
||||||
`config.get_conf_dir()` + `mkdir(parents=True,
|
|
||||||
exist_ok=True)` for nested tmp-dir creation.
|
|
||||||
|
|
||||||
Key diagnostic reasoning (verbatim from session):
|
|
||||||
|
|
||||||
The old (commented-out) `get_app_dir()` override gated
|
|
||||||
on `'pytest' in sys.modules` which can NEVER work in
|
|
||||||
spawned subactors (fresh procs, no pytest import); as
|
|
||||||
a result test `paperboi`/daemon actors were writing
|
|
||||||
into the user's REAL `~/.config/piker/accounting/`
|
|
||||||
files. Evidence: `account.kraken.paper.toml` gained a
|
|
||||||
single 0.001 xbtusdt clear stamped 2026-06-09T18:47Z
|
|
||||||
(a test fill), and `trades_kraken_paper.toml` (252B)
|
|
||||||
contained only that fill. The resolution must be
|
|
||||||
checked lazily at config-path access time (NOT import
|
|
||||||
time) since sub-actors only receive runtime-vars once
|
|
||||||
their `tractor` runtime has fully booted.
|
|
||||||
|
|
@ -1,53 +0,0 @@
|
||||||
---
|
|
||||||
model: claude-fable-5[1m]
|
|
||||||
service: claude
|
|
||||||
session: 32d15f9a-b2d3-4c26-bdc9-190219141a25
|
|
||||||
timestamp: 2026-06-10T17:10:22Z
|
|
||||||
git_ref: datad_service
|
|
||||||
diff_cmd: git log -1 -p --follow -- ai/prompt-io/claude/20260610T171022Z_4485f2b9_prompt_io.md
|
|
||||||
scope: code
|
|
||||||
substantive: true
|
|
||||||
raw_file: 20260610T171022Z_4485f2b9_prompt_io.raw.md
|
|
||||||
---
|
|
||||||
|
|
||||||
## Prompt
|
|
||||||
|
|
||||||
Same session-initiating `brokerd`-split instruction (see
|
|
||||||
`20260610T170859Z_75cefe10_prompt_io.md`). Proximate
|
|
||||||
driver: the approved plan's per-stage test gates could
|
|
||||||
not run AT ALL — the branch base was broken vs.
|
|
||||||
`tractor` git `main` (`AttributeError: 'tuple' object
|
|
||||||
has no attribute 'unwrap'` at `pikerd` boot, stale
|
|
||||||
`arbiter_sockaddr`/`.type`/`.size` API refs). The agent
|
|
||||||
fixed forward autonomously to (re)establish the gate
|
|
||||||
baseline, continuing the `repair_tests` branch lineage
|
|
||||||
(verified already merged in ancestry via `git cherry`
|
|
||||||
during a user-requested branch-overlap survey).
|
|
||||||
|
|
||||||
## Response summary
|
|
||||||
|
|
||||||
Port the service layer + test suites to current
|
|
||||||
`tractor` APIs: addr-type normalization in
|
|
||||||
`open_pikerd()`, `query_actor()`/`wait_for_actor()`
|
|
||||||
kwarg renames, modern self-cancel absorption semantics
|
|
||||||
in the cancel-method test harness, exc/position attr
|
|
||||||
renames, a paper-EMS startup-budget bump and a syntax
|
|
||||||
fix in `.deribit.api`.
|
|
||||||
|
|
||||||
## Files changed
|
|
||||||
|
|
||||||
- `piker/service/_actor_runtime.py` — `wrap_address()`
|
|
||||||
normalize before `.unwrap()` in `open_pikerd()`
|
|
||||||
- `piker/service/_registry.py` — `check_for_service()`
|
|
||||||
-> `query_actor(regaddr=)` + 2-tuple yield +
|
|
||||||
`open_registry(addrs=)`
|
|
||||||
- `piker/brokers/deribit/api.py` — missing comma in
|
|
||||||
`tractor.trionics` import tuple
|
|
||||||
- `tests/test_services.py` — `registry_addr=` kwarg,
|
|
||||||
raddr unwraps, cancel-semantics harness rewrite,
|
|
||||||
`fail_after` 9 -> 19s
|
|
||||||
- `tests/test_ems.py` — `.boxed_type`, `pp.cumsize`
|
|
||||||
|
|
||||||
## Human edits
|
|
||||||
|
|
||||||
None — committed as generated.
|
|
||||||
|
|
@ -1,56 +0,0 @@
|
||||||
---
|
|
||||||
model: claude-fable-5[1m]
|
|
||||||
service: claude
|
|
||||||
timestamp: 2026-06-10T17:10:22Z
|
|
||||||
git_ref: datad_service
|
|
||||||
diff_cmd: git log -1 -p --follow -- ai/prompt-io/claude/20260610T171022Z_4485f2b9_prompt_io.md
|
|
||||||
---
|
|
||||||
|
|
||||||
NOTE: diff-ref mode entry (code committed in the same
|
|
||||||
commit as this log); backfilled from the live dev
|
|
||||||
session transcript per the `/prompt-io` skill rules.
|
|
||||||
|
|
||||||
> `git log -1 -p --follow -- piker/service/_actor_runtime.py`
|
|
||||||
|
|
||||||
Generated: normalize each registry addr via
|
|
||||||
`tractor.discovery._addr.wrap_address()` before
|
|
||||||
`.unwrap()`-ing for the `accept_addrs` bind check —
|
|
||||||
entries may be raw `tuple`s when passed in from (test)
|
|
||||||
client code. Import-path precedent taken from
|
|
||||||
`piker/cli/__init__.py:336`.
|
|
||||||
|
|
||||||
> `git log -1 -p --follow -- piker/service/_registry.py`
|
|
||||||
|
|
||||||
Generated: `check_for_service()` ported to
|
|
||||||
`tractor.query_actor(name, regaddr=...)` (kwarg was
|
|
||||||
`arbiter_sockaddr=`), unpacking the new
|
|
||||||
`(sockaddr, portal)` yield, and passing the
|
|
||||||
now-required `open_registry(addrs=Registry.addrs)`.
|
|
||||||
|
|
||||||
> `git log -1 -p --follow -- tests/test_services.py`
|
|
||||||
> `git log -1 -p --follow -- tests/test_ems.py`
|
|
||||||
> `git log -1 -p --follow -- piker/brokers/deribit/api.py`
|
|
||||||
|
|
||||||
Key diagnostic reasoning (verbatim from session):
|
|
||||||
|
|
||||||
- the "DID NOT RAISE ContextCancelled" failure: in this
|
|
||||||
test the client actor IS pikerd (in-proc), and
|
|
||||||
current `tractor` main absorbs a `ContextCancelled`
|
|
||||||
whose canceller is your own actor — self-requested
|
|
||||||
cancels now exit cleanly instead of raising; the
|
|
||||||
'sigint' variant propagates a bare collapsed
|
|
||||||
`KeyboardInterrupt` rather than a
|
|
||||||
`BaseExceptionGroup`.
|
|
||||||
- the hard-coded `trio.fail_after(9)` startup budget is
|
|
||||||
marginal — full stack boot (pikerd -> emsd ->
|
|
||||||
brokerd.kraken -> paperboi + live kraken symbology
|
|
||||||
fetch) occasionally exceeds 9s -> bumped to 19s.
|
|
||||||
- `RemoteActorError.type` -> `.boxed_type`;
|
|
||||||
`Position.size` -> `.cumsize` (the paper engine
|
|
||||||
populates `BrokerdPosition.size` from `pp.cumsize`).
|
|
||||||
- overlap survey (user-requested): all of the
|
|
||||||
`repair_tests` branch commits are already in this
|
|
||||||
stack's ancestry; this commit finishes that branch's
|
|
||||||
port mission (its f4c4f1e2 fixed `conftest.py`'s
|
|
||||||
`arbiter_sockaddr` usage; this fixes the remaining
|
|
||||||
`test_services.py` + `check_for_service()` sites).
|
|
||||||
|
|
@ -1,54 +0,0 @@
|
||||||
---
|
|
||||||
model: claude-fable-5[1m]
|
|
||||||
service: claude
|
|
||||||
session: 32d15f9a-b2d3-4c26-bdc9-190219141a25
|
|
||||||
timestamp: 2026-06-10T17:11:05Z
|
|
||||||
git_ref: datad_service
|
|
||||||
diff_cmd: git log -1 -p --follow -- ai/prompt-io/claude/20260610T171105Z_bc6e18d7_prompt_io.md
|
|
||||||
scope: code
|
|
||||||
substantive: true
|
|
||||||
raw_file: 20260610T171105Z_bc6e18d7_prompt_io.raw.md
|
|
||||||
---
|
|
||||||
|
|
||||||
## Prompt
|
|
||||||
|
|
||||||
Same session-initiating `brokerd`-split instruction (see
|
|
||||||
`20260610T170859Z_75cefe10_prompt_io.md`); this is the
|
|
||||||
approved plan's "stage 0" prep. Plan-shaping user
|
|
||||||
decisions captured via in-session Q&A:
|
|
||||||
|
|
||||||
- `datad.<broker>` topology: sibling of
|
|
||||||
`brokerd.<broker>` under `pikerd` (vs. child
|
|
||||||
subactor).
|
|
||||||
- migration: hard cutover, staged by layer (no
|
|
||||||
dual-mode runtime flag).
|
|
||||||
- post-split `brokerd` scope: trading-only +
|
|
||||||
EMS-lazy-spawned (charts/CLI never touch it).
|
|
||||||
|
|
||||||
## Response summary
|
|
||||||
|
|
||||||
Declare per-daemon-kind backend submod groups
|
|
||||||
(`_datad_mods`/`_brokerd_mods`) in the split-style
|
|
||||||
backends, keyed to the pre-existing
|
|
||||||
`piker.data.validate._eps` contract; add a
|
|
||||||
`validate.get_eps()` introspection helper; ws-import
|
|
||||||
hygiene in `.kraken.broker`. Zero behavior change
|
|
||||||
(`__enable_modules__` unions unchanged).
|
|
||||||
|
|
||||||
## Files changed
|
|
||||||
|
|
||||||
- `piker/data/validate.py` — add `get_eps()`
|
|
||||||
- `piker/brokers/kraken/__init__.py` — mod groups
|
|
||||||
- `piker/brokers/binance/__init__.py` — mod groups
|
|
||||||
(note: no `symbols.py`; search eps live in `.feed`)
|
|
||||||
- `piker/brokers/deribit/__init__.py` — datad-only
|
|
||||||
groups (no `broker.py` yet)
|
|
||||||
- `piker/brokers/ib/__init__.py` — add `'api'` to the
|
|
||||||
pre-existing `_datad_mods`
|
|
||||||
- `piker/brokers/kraken/broker.py` — import
|
|
||||||
`NoBsWs`/`open_autorecon_ws` from
|
|
||||||
`piker.data._web_bs` directly
|
|
||||||
|
|
||||||
## Human edits
|
|
||||||
|
|
||||||
None — committed as generated.
|
|
||||||
|
|
@ -1,46 +0,0 @@
|
||||||
---
|
|
||||||
model: claude-fable-5[1m]
|
|
||||||
service: claude
|
|
||||||
timestamp: 2026-06-10T17:11:05Z
|
|
||||||
git_ref: datad_service
|
|
||||||
diff_cmd: git log -1 -p --follow -- ai/prompt-io/claude/20260610T171105Z_bc6e18d7_prompt_io.md
|
|
||||||
---
|
|
||||||
|
|
||||||
NOTE: diff-ref mode entry (code committed in the same
|
|
||||||
commit as this log); backfilled from the live dev
|
|
||||||
session transcript per the `/prompt-io` skill rules.
|
|
||||||
|
|
||||||
> `git log -1 -p --follow -- piker/data/validate.py`
|
|
||||||
|
|
||||||
Generated: `get_eps(mod, kind) -> dict[str, Callable]`
|
|
||||||
returning the daemon-kind's ep funcs defined by a
|
|
||||||
backend mod, keyed by ep name, missing eps excluded;
|
|
||||||
sourced from the existing `_eps` grouping table
|
|
||||||
(`'middleware' | 'datad' | 'brokerd'`).
|
|
||||||
|
|
||||||
> `git log -1 -p --follow -- piker/brokers/kraken/__init__.py`
|
|
||||||
> `git log -1 -p --follow -- piker/brokers/binance/__init__.py`
|
|
||||||
> `git log -1 -p --follow -- piker/brokers/deribit/__init__.py`
|
|
||||||
> `git log -1 -p --follow -- piker/brokers/ib/__init__.py`
|
|
||||||
> `git log -1 -p --follow -- piker/brokers/kraken/broker.py`
|
|
||||||
|
|
||||||
Key exploration findings driving the design (from the
|
|
||||||
planning phase, 3 parallel explore agents + 1 architect
|
|
||||||
agent):
|
|
||||||
|
|
||||||
- the codebase already anticipated this split:
|
|
||||||
`validate._eps` groups eps into 'datad' vs 'brokerd'
|
|
||||||
kinds and `ib/__init__.py` already declared
|
|
||||||
`_brokerd_mods`/`_datad_mods`; `brokers/_daemon.py`
|
|
||||||
carried the literal TODO "rename the daemon to datad
|
|
||||||
prolly once we split up broker vs. data tasks into
|
|
||||||
separate actors?".
|
|
||||||
- the feared kraken feed<->broker "shared ws state"
|
|
||||||
hazard is mild: `NoBsWs`/`open_autorecon_ws`
|
|
||||||
originate in `piker.data._web_bs` (re-exported via
|
|
||||||
`.kraken.feed`); only `stream_messages` is feed-local
|
|
||||||
and it's a pure parser — in-process imports only,
|
|
||||||
each actor opens its own ws conn.
|
|
||||||
- `enable_modules` gates RPC entry, NOT python imports,
|
|
||||||
so backend trading mods may keep importing feed-side
|
|
||||||
helpers in-process post-split.
|
|
||||||
|
|
@ -1,47 +0,0 @@
|
||||||
---
|
|
||||||
model: claude-fable-5[1m]
|
|
||||||
service: claude
|
|
||||||
session: 32d15f9a-b2d3-4c26-bdc9-190219141a25
|
|
||||||
timestamp: 2026-06-10T17:11:42Z
|
|
||||||
git_ref: datad_service
|
|
||||||
diff_cmd: git log -1 -p --follow -- ai/prompt-io/claude/20260610T171142Z_119d2c04_prompt_io.md
|
|
||||||
scope: code
|
|
||||||
substantive: true
|
|
||||||
raw_file: 20260610T171142Z_119d2c04_prompt_io.raw.md
|
|
||||||
---
|
|
||||||
|
|
||||||
## Prompt
|
|
||||||
|
|
||||||
Same session-initiating `brokerd`-split instruction (see
|
|
||||||
`20260610T170859Z_75cefe10_prompt_io.md`); this is the
|
|
||||||
approved plan's "stage 1": introduce the `datad` daemon
|
|
||||||
machinery additively (nothing routes through it yet).
|
|
||||||
User-decided constraint applied: `datad.<broker>` is a
|
|
||||||
SIBLING of `brokerd.<broker>` under `pikerd`, spawned
|
|
||||||
via the existing `Services` + `maybe_spawn_daemon()`
|
|
||||||
machinery.
|
|
||||||
|
|
||||||
## Response summary
|
|
||||||
|
|
||||||
New `piker/data/_daemon.py` hosting the
|
|
||||||
`datad.<broker>` feed-only daemon-actor: lifetime
|
|
||||||
fixture owning the actor-global `_FeedsBus`, init/spawn
|
|
||||||
fns mirroring `.brokers._daemon` conventions and the
|
|
||||||
`samplerd` sub-daemon precedent, plus root-mod
|
|
||||||
registration, `piker.service` re-exports and a spawn
|
|
||||||
test.
|
|
||||||
|
|
||||||
## Files changed
|
|
||||||
|
|
||||||
- `piker/data/_daemon.py` — NEW:
|
|
||||||
`_setup_persistent_datad()`, `datad_init()`,
|
|
||||||
`spawn_datad()`, `maybe_spawn_datad()`,
|
|
||||||
`_datad_service_mods`
|
|
||||||
- `piker/service/_actor_runtime.py` — add
|
|
||||||
`piker.data._daemon` to `_root_modules`
|
|
||||||
- `piker/service/__init__.py` — re-export spawn eps
|
|
||||||
- `tests/test_services.py` — add `test_datad_spawn`
|
|
||||||
|
|
||||||
## Human edits
|
|
||||||
|
|
||||||
None — committed as generated.
|
|
||||||
|
|
@ -1,60 +0,0 @@
|
||||||
---
|
|
||||||
model: claude-fable-5[1m]
|
|
||||||
service: claude
|
|
||||||
timestamp: 2026-06-10T17:11:42Z
|
|
||||||
git_ref: datad_service
|
|
||||||
diff_cmd: git log -1 -p --follow -- ai/prompt-io/claude/20260610T171142Z_119d2c04_prompt_io.md
|
|
||||||
---
|
|
||||||
|
|
||||||
NOTE: diff-ref mode entry (code committed in the same
|
|
||||||
commit as this log); backfilled from the live dev
|
|
||||||
session transcript per the `/prompt-io` skill rules.
|
|
||||||
|
|
||||||
> `git log -1 -p --follow -- piker/data/_daemon.py`
|
|
||||||
|
|
||||||
Generated symbols + key design decisions:
|
|
||||||
|
|
||||||
- `_datad_service_mods: list[str]` — datad-always
|
|
||||||
enabled mods, the data-side successor to the old
|
|
||||||
`piker.brokers._daemon._data_mods` set; kept minimal
|
|
||||||
per the caps-sec model.
|
|
||||||
- `_setup_persistent_datad()` — `@tractor.context`
|
|
||||||
lifetime fixture: console-log setup then allocates
|
|
||||||
the actor-global feed bus via
|
|
||||||
`feed.get_feed_bus(brokername, service_nursery)`
|
|
||||||
exactly as the old brokerd fixture did, pinned open
|
|
||||||
with `ctx.started()` + `sleep_forever()`.
|
|
||||||
- `datad_init()` — actor name `f'datad.{brokername}'`;
|
|
||||||
copies backend `_spawn_kwargs` (CRITICAL for `ib`'s
|
|
||||||
`infect_asyncio=True`); builds `enable_modules` from
|
|
||||||
`getattr(brokermod, '_datad_mods',
|
|
||||||
getattr(brokermod, '__enable_modules__', []))` as
|
|
||||||
the flat-backend fallback.
|
|
||||||
- `spawn_datad()` — `Services.actor_n.start_actor()` +
|
|
||||||
`Services.start_service_task()` exactly mirroring
|
|
||||||
`spawn_brokerd()`; dedup-composes enable mods via
|
|
||||||
`list(dict.fromkeys(...))`.
|
|
||||||
- `maybe_spawn_datad()` — wraps `maybe_spawn_daemon(
|
|
||||||
service_name=f'datad.{brokername}', ...)`.
|
|
||||||
|
|
||||||
> `git log -1 -p --follow -- piker/service/_actor_runtime.py`
|
|
||||||
> `git log -1 -p --follow -- piker/service/__init__.py`
|
|
||||||
> `git log -1 -p --follow -- tests/test_services.py`
|
|
||||||
|
|
||||||
Design rationale (verbatim from session):
|
|
||||||
|
|
||||||
- `_root_modules` must gain `piker.data._daemon` so
|
|
||||||
`pikerd_portal.run(spawn_datad, ...)` resolves in
|
|
||||||
the root.
|
|
||||||
- the `Services`-based impl style deliberately mirrors
|
|
||||||
`spawn_brokerd()` so the eventual `tractor.hilevel`
|
|
||||||
`ServiceMngr` port (see the `service_mng_to_tractor`
|
|
||||||
branch's d8c21d44 prep, surfaced by the
|
|
||||||
user-requested branch-overlap survey) lands
|
|
||||||
symmetrically on both spawn fns.
|
|
||||||
- mod placement (`piker/data/_daemon.py` vs.
|
|
||||||
generalizing `piker.brokers._daemon`) follows the
|
|
||||||
per-subsystem daemon-mod convention
|
|
||||||
(`.clearing._ems`, `.data._sampling`) and resolves
|
|
||||||
the existing TODO at `brokers/_daemon.py:49` ("move
|
|
||||||
this def to the `.data` subpkg").
|
|
||||||
|
|
@ -1,47 +0,0 @@
|
||||||
---
|
|
||||||
model: claude-fable-5[1m]
|
|
||||||
service: claude
|
|
||||||
session: 32d15f9a-b2d3-4c26-bdc9-190219141a25
|
|
||||||
timestamp: 2026-06-10T17:12:26Z
|
|
||||||
git_ref: datad_service
|
|
||||||
diff_cmd: git log -1 -p --follow -- ai/prompt-io/claude/20260610T171226Z_64181219_prompt_io.md
|
|
||||||
scope: code
|
|
||||||
substantive: true
|
|
||||||
raw_file: 20260610T171226Z_64181219_prompt_io.raw.md
|
|
||||||
---
|
|
||||||
|
|
||||||
## Prompt
|
|
||||||
|
|
||||||
Same session-initiating `brokerd`-split instruction (see
|
|
||||||
`20260610T170859Z_75cefe10_prompt_io.md`); this is the
|
|
||||||
approved plan's "stage 2": decouple the clearing layer
|
|
||||||
from `feed.portals` BEFORE the feed cutover so live
|
|
||||||
trading works at every stage boundary. User-decided
|
|
||||||
constraint applied: post-split `brokerd` is
|
|
||||||
trading-only and spawned LAZILY only by `emsd`'s
|
|
||||||
`open_brokerd_dialog()` path.
|
|
||||||
|
|
||||||
## Response summary
|
|
||||||
|
|
||||||
Kill the single coupling forcing feed + trading eps
|
|
||||||
into one actor (`Router.open_trade_relays()` pulling
|
|
||||||
its trades portal from `feed.portals[brokermod]`):
|
|
||||||
`open_brokerd_dialog()` now (maybe) spawns/finds
|
|
||||||
`brokerd.<broker>` itself and ONLY when a live
|
|
||||||
trades-ep will actually open; paper mode never touches
|
|
||||||
it. Pre-cutover this is a pure refactor (registry
|
|
||||||
lookup finds the same feed-spawned daemon).
|
|
||||||
|
|
||||||
## Files changed
|
|
||||||
|
|
||||||
- `piker/clearing/_ems.py` — `open_brokerd_dialog()`
|
|
||||||
re-sig + inner `acquire_live_portal()`;
|
|
||||||
`Router.maybe_open_brokerd_dialog()` drops `portal`
|
|
||||||
param; `open_trade_relays()` drops the
|
|
||||||
`feed.portals` lookup
|
|
||||||
- `piker/accounting/cli.py` — keyword-form `portal=`
|
|
||||||
override kept for the `piker ledger` ad-hoc actor
|
|
||||||
|
|
||||||
## Human edits
|
|
||||||
|
|
||||||
None — committed as generated.
|
|
||||||
|
|
@ -1,43 +0,0 @@
|
||||||
---
|
|
||||||
model: claude-fable-5[1m]
|
|
||||||
service: claude
|
|
||||||
timestamp: 2026-06-10T17:12:26Z
|
|
||||||
git_ref: datad_service
|
|
||||||
diff_cmd: git log -1 -p --follow -- ai/prompt-io/claude/20260610T171226Z_64181219_prompt_io.md
|
|
||||||
---
|
|
||||||
|
|
||||||
NOTE: diff-ref mode entry (code committed in the same
|
|
||||||
commit as this log); backfilled from the live dev
|
|
||||||
session transcript per the `/prompt-io` skill rules.
|
|
||||||
|
|
||||||
> `git log -1 -p --follow -- piker/clearing/_ems.py`
|
|
||||||
|
|
||||||
Generated: new `open_brokerd_dialog()` signature
|
|
||||||
`(brokermod, exec_mode, fqme=None, portal=None,
|
|
||||||
loglevel=None)` with an inner `@acm
|
|
||||||
acquire_live_portal()` that yields the caller-provided
|
|
||||||
`portal` override (the `piker ledger` path) else
|
|
||||||
`maybe_spawn_brokerd(brokermod.name)` — designated THE
|
|
||||||
one place a live, credentialed `brokerd.<broker>` gets
|
|
||||||
booted post-split. The eager
|
|
||||||
`portal.open_context(trades_endpoint, ...)`
|
|
||||||
construction moved inside that block so the paper
|
|
||||||
short-circuit never acquires a live portal.
|
|
||||||
|
|
||||||
> `git log -1 -p --follow -- piker/accounting/cli.py`
|
|
||||||
|
|
||||||
Key analysis (verbatim from session):
|
|
||||||
|
|
||||||
- `feed.portals` had exactly ONE trading consumer:
|
|
||||||
`piker/clearing/_ems.py:671` (`portal =
|
|
||||||
feed.portals[brokermod]`) — the single coupling
|
|
||||||
forcing feed + trading into one actor.
|
|
||||||
- `piker ledger` (`accounting/cli.py:100-157`) is a
|
|
||||||
hidden consumer: it calls `broker_init()` directly,
|
|
||||||
spawns its own ad-hoc actor and passes the portal
|
|
||||||
positionally — the new signature must keep an
|
|
||||||
explicit `portal:` override param for this path.
|
|
||||||
- stage sequencing: clearing decouple lands BEFORE the
|
|
||||||
feed cutover, otherwise `feed.portals` would hand
|
|
||||||
emsd a `datad` portal and live `open_trade_dialog`
|
|
||||||
RPC would fail.
|
|
||||||
|
|
@ -1,48 +0,0 @@
|
||||||
---
|
|
||||||
model: claude-fable-5[1m]
|
|
||||||
service: claude
|
|
||||||
session: 32d15f9a-b2d3-4c26-bdc9-190219141a25
|
|
||||||
timestamp: 2026-06-10T17:12:59Z
|
|
||||||
git_ref: datad_service
|
|
||||||
diff_cmd: git log -1 -p --follow -- ai/prompt-io/claude/20260610T171259Z_59d5d9a6_prompt_io.md
|
|
||||||
scope: code
|
|
||||||
substantive: true
|
|
||||||
raw_file: 20260610T171259Z_59d5d9a6_prompt_io.raw.md
|
|
||||||
---
|
|
||||||
|
|
||||||
## Prompt
|
|
||||||
|
|
||||||
Same session-initiating `brokerd`-split instruction (see
|
|
||||||
`20260610T170859Z_75cefe10_prompt_io.md`); this is the
|
|
||||||
approved plan's "stage 3": the feed-layer hard cutover
|
|
||||||
(user-decided: no dual-mode runtime flag, each stage
|
|
||||||
lands fully working).
|
|
||||||
|
|
||||||
## Response summary
|
|
||||||
|
|
||||||
The topology flip: all data-feed consumers route to
|
|
||||||
`datad.<broker>`; the `brokerd` lifetime fixture slims
|
|
||||||
to trading-only; UI/CLI data-flavoured spawn sites
|
|
||||||
repointed; test expectations inverted incl. an explicit
|
|
||||||
negative assert that paper sessions spawn NO
|
|
||||||
`brokerd.<broker>`.
|
|
||||||
|
|
||||||
## Files changed
|
|
||||||
|
|
||||||
- `piker/data/feed.py` — `open_feed()` ->
|
|
||||||
`maybe_spawn_datad()`; `open_feed_bus()` actor-name
|
|
||||||
assert -> `'datad'`; `install_brokerd_search` ->
|
|
||||||
`install_datad_search`; comment sweep
|
|
||||||
- `piker/brokers/_daemon.py` —
|
|
||||||
`_setup_persistent_brokerd()` slimmed (feed-bus
|
|
||||||
alloc moved to the datad fixture)
|
|
||||||
- `piker/ui/_app.py`, `piker/ui/cli.py`,
|
|
||||||
`piker/brokers/core.py`, `piker/brokers/cli.py`,
|
|
||||||
`piker/ui/kivy/option_chain.py` — spawn-site
|
|
||||||
repoints
|
|
||||||
- `tests/test_services.py` — datad assertions + the
|
|
||||||
no-`brokerd.kraken` negative check
|
|
||||||
|
|
||||||
## Human edits
|
|
||||||
|
|
||||||
None — committed as generated.
|
|
||||||
|
|
@ -1,54 +0,0 @@
|
||||||
---
|
|
||||||
model: claude-fable-5[1m]
|
|
||||||
service: claude
|
|
||||||
timestamp: 2026-06-10T17:12:59Z
|
|
||||||
git_ref: datad_service
|
|
||||||
diff_cmd: git log -1 -p --follow -- ai/prompt-io/claude/20260610T171259Z_59d5d9a6_prompt_io.md
|
|
||||||
---
|
|
||||||
|
|
||||||
NOTE: diff-ref mode entry (code committed in the same
|
|
||||||
commit as this log); backfilled from the live dev
|
|
||||||
session transcript per the `/prompt-io` skill rules.
|
|
||||||
|
|
||||||
> `git log -1 -p --follow -- piker/data/feed.py`
|
|
||||||
|
|
||||||
Generated: `open_feed()` spawn cutover with the
|
|
||||||
`maybe_spawn_datad` import done "relative-direct" from
|
|
||||||
`._daemon` — NOT via `piker.service` — to dodge a
|
|
||||||
partial-init cycle: when `piker.data.feed` loads as
|
|
||||||
part of `piker.service.__init__` executing its own
|
|
||||||
`piker.data._daemon` import, the service pkg is
|
|
||||||
mid-init and its `maybe_spawn_datad` binding does not
|
|
||||||
exist yet. The `open_feed_bus()` local-state sanity
|
|
||||||
assert flips `'brokerd' in servicename` ->
|
|
||||||
`'datad' in servicename` (the only actor-name assert
|
|
||||||
in the tree, verified by grep).
|
|
||||||
|
|
||||||
> `git log -1 -p --follow -- piker/brokers/_daemon.py`
|
|
||||||
|
|
||||||
Generated: `_setup_persistent_brokerd()` slimmed to
|
|
||||||
console-log setup + pinned-open ctx; drops the bus
|
|
||||||
alloc, the `assert not feed._bus`, the service nursery
|
|
||||||
(backend `open_trade_dialog()` ctxs own their task
|
|
||||||
trees) and the `eg.ExceptionGroup` handler. The
|
|
||||||
`piker ledger` ad-hoc actor enters this same slimmed
|
|
||||||
fixture — exactly what it needs.
|
|
||||||
|
|
||||||
> `git log -1 -p --follow -- piker/ui/_app.py`
|
|
||||||
> `git log -1 -p --follow -- piker/ui/cli.py`
|
|
||||||
> `git log -1 -p --follow -- piker/brokers/core.py`
|
|
||||||
> `git log -1 -p --follow -- piker/brokers/cli.py`
|
|
||||||
> `git log -1 -p --follow -- piker/ui/kivy/option_chain.py`
|
|
||||||
> `git log -1 -p --follow -- tests/test_services.py`
|
|
||||||
|
|
||||||
Verification (from session): per-suite gates green at
|
|
||||||
this commit (services 5-passed incl. the new negative
|
|
||||||
assert, feeds 3-passed); a headless live smoke
|
|
||||||
(`maybe_open_pikerd` + `open_feed(['xbtusdt.kraken'])`
|
|
||||||
on an alt registry port) confirmed quotes flowing via
|
|
||||||
`datad.kraken` + `samplerd` with `check_for_service(
|
|
||||||
'brokerd.kraken') is None`. Known pre-existing flake
|
|
||||||
documented: `test_multi_fill_positions`' second
|
|
||||||
in-proc runtime boot wedges ~50% (zombie subactor w/
|
|
||||||
unread parent-IPC bytes); reproduced with the split
|
|
||||||
fully reverted so NOT a regression of this work.
|
|
||||||
|
|
@ -1,43 +0,0 @@
|
||||||
---
|
|
||||||
model: claude-fable-5[1m]
|
|
||||||
service: claude
|
|
||||||
session: 32d15f9a-b2d3-4c26-bdc9-190219141a25
|
|
||||||
timestamp: 2026-06-10T17:13:44Z
|
|
||||||
git_ref: datad_service
|
|
||||||
diff_cmd: git log -1 -p --follow -- ai/prompt-io/claude/20260610T171344Z_eee19de0_prompt_io.md
|
|
||||||
scope: code
|
|
||||||
substantive: true
|
|
||||||
raw_file: 20260610T171344Z_eee19de0_prompt_io.raw.md
|
|
||||||
---
|
|
||||||
|
|
||||||
## Prompt
|
|
||||||
|
|
||||||
Same session-initiating `brokerd`-split instruction (see
|
|
||||||
`20260610T170859Z_75cefe10_prompt_io.md`); this is the
|
|
||||||
approved plan's final "stage 4": caps-sec slimming of
|
|
||||||
the (live, credentialed) trading actor + the `ib`
|
|
||||||
dual-daemon `client_id` collision mitigation flagged in
|
|
||||||
the plan's risk register.
|
|
||||||
|
|
||||||
## Response summary
|
|
||||||
|
|
||||||
`brokerd` loses ALL `piker.data.*` (feed) RPC mods;
|
|
||||||
spawn fails fast for datad-only backends with a "use
|
|
||||||
paper-mode" error; `ib`'s default api-gw `client_id`
|
|
||||||
gets a per-daemon-kind offset so `datad.ib` +
|
|
||||||
`brokerd.ib` don't collide on connect.
|
|
||||||
|
|
||||||
## Files changed
|
|
||||||
|
|
||||||
- `piker/brokers/_daemon.py` — `_data_mods` -> minimal
|
|
||||||
`_brokerd_service_mods`; `broker_init()` reads
|
|
||||||
`_brokerd_mods` (fallback `__enable_modules__`);
|
|
||||||
`spawn_brokerd()` fail-fast via `validate.get_eps()`
|
|
||||||
- `piker/brokers/ib/api.py` — role-based `client_id`
|
|
||||||
offset in `load_aio_clients()`
|
|
||||||
- `piker/cli/__init__.py` — resolved "expose datad"
|
|
||||||
TODO
|
|
||||||
|
|
||||||
## Human edits
|
|
||||||
|
|
||||||
None — committed as generated.
|
|
||||||
|
|
@ -1,41 +0,0 @@
|
||||||
---
|
|
||||||
model: claude-fable-5[1m]
|
|
||||||
service: claude
|
|
||||||
timestamp: 2026-06-10T17:13:44Z
|
|
||||||
git_ref: datad_service
|
|
||||||
diff_cmd: git log -1 -p --follow -- ai/prompt-io/claude/20260610T171344Z_eee19de0_prompt_io.md
|
|
||||||
---
|
|
||||||
|
|
||||||
NOTE: diff-ref mode entry (code committed in the same
|
|
||||||
commit as this log); backfilled from the live dev
|
|
||||||
session transcript per the `/prompt-io` skill rules.
|
|
||||||
|
|
||||||
> `git log -1 -p --follow -- piker/brokers/_daemon.py`
|
|
||||||
|
|
||||||
Generated: the fail-fast originally landed in
|
|
||||||
`broker_init()` but was relocated to `spawn_brokerd()`
|
|
||||||
mid-implementation after realizing `piker ledger`
|
|
||||||
calls `broker_init()` directly even for paper accounts
|
|
||||||
on datad-only backends (would have crashed the cli);
|
|
||||||
the service-spawn path is the correct enforcement
|
|
||||||
seam. Error text:
|
|
||||||
|
|
||||||
Backend 'kucoin' offers NO `brokerd` (live
|
|
||||||
order-control) eps!? It is likely a datad-only
|
|
||||||
provider, use paper-mode for clearing instead.
|
|
||||||
|
|
||||||
(verified live via a `trio.run()` unit check.)
|
|
||||||
|
|
||||||
> `git log -1 -p --follow -- piker/brokers/ib/api.py`
|
|
||||||
|
|
||||||
Generated: in `load_aio_clients()`, when `client_id`
|
|
||||||
is the 6116 default: `datad`-named actors offset +16
|
|
||||||
(disjoint from `brokerd`'s linear `client_id + i`
|
|
||||||
retry range), other non-`brokerd` (ad-hoc test/cli)
|
|
||||||
actors +32. Rationale from the plan's risk register:
|
|
||||||
post-split BOTH per-broker daemons connect to the same
|
|
||||||
TWS/gw endpoint; a shared default id collides and
|
|
||||||
burns up to `connect_timeout * retries` (90s) in
|
|
||||||
retry cycles.
|
|
||||||
|
|
||||||
> `git log -1 -p --follow -- piker/cli/__init__.py`
|
|
||||||
|
|
@ -1,66 +0,0 @@
|
||||||
---
|
|
||||||
model: claude-fable-5[1m]
|
|
||||||
service: claude
|
|
||||||
session: 32d15f9a-b2d3-4c26-bdc9-190219141a25
|
|
||||||
timestamp: 2026-06-10T17:33:09Z
|
|
||||||
git_ref: datad_service
|
|
||||||
diff_cmd: git log -1 -p --follow -- ai/prompt-io/claude/20260610T173309Z_f15f8178_prompt_io.md
|
|
||||||
scope: docs
|
|
||||||
substantive: true
|
|
||||||
raw_file: 20260610T173309Z_f15f8178_prompt_io.raw.md
|
|
||||||
---
|
|
||||||
|
|
||||||
## Prompt
|
|
||||||
|
|
||||||
The session-initiating instruction, verbatim (this doc
|
|
||||||
IS its "mega detailed plan" deliverable):
|
|
||||||
|
|
||||||
> ok i want you to become the distributed runtime and
|
|
||||||
> concurrency expert for this project - namely acquire
|
|
||||||
> a deep understanding of tractor and how it's used.
|
|
||||||
> then i want you to atttempt to factor our current
|
|
||||||
> brokerd service daemon into 2 daemons:
|
|
||||||
>
|
|
||||||
> - a brokerd which only hosts live/paper trading
|
|
||||||
> endoint tasks as defined namely within all
|
|
||||||
> piker/brokers/<backend>/broker.py mods
|
|
||||||
> - a new `datad` subdaemon which in a separate
|
|
||||||
> subactor serves all the data feed service tasks
|
|
||||||
> namely delivered by endpoints in
|
|
||||||
> piker/brokers/<backend>/feed.py (or similar for
|
|
||||||
> less mature backends) and as more rigorously
|
|
||||||
> defined by the validation machinery in
|
|
||||||
> piker.data.validate.
|
|
||||||
>
|
|
||||||
> give me a mega detailed plan on how to approach
|
|
||||||
> this, and a staged approach for the implementation.
|
|
||||||
|
|
||||||
Plan-shaping user decisions (in-session Q&A): sibling
|
|
||||||
topology under `pikerd`; hard cutover staged by layer;
|
|
||||||
trading-only EMS-lazy-spawned `brokerd`. The human
|
|
||||||
staged the doc copy into `ai/claude-code/plans/` and
|
|
||||||
requested this commit after the 7 implementation
|
|
||||||
commits landed.
|
|
||||||
|
|
||||||
## Response summary
|
|
||||||
|
|
||||||
The staged design doc for the `brokerd` -> (`datad` +
|
|
||||||
`brokerd`) split: context, target supervision topology,
|
|
||||||
load-bearing verified facts, per-stage file-level
|
|
||||||
changes with gates, risk register and an end-to-end
|
|
||||||
verification matrix. Produced via 3 parallel explore
|
|
||||||
agents + 1 architect agent + human Q&A before any code
|
|
||||||
was written; all 7 implementation commits in this
|
|
||||||
branch reference its stages in their provenance
|
|
||||||
entries.
|
|
||||||
|
|
||||||
## Files changed
|
|
||||||
|
|
||||||
- `ai/claude-code/plans/datad_service.md` — the design
|
|
||||||
plan doc (human-staged copy of the AI-authored plan)
|
|
||||||
|
|
||||||
## Human edits
|
|
||||||
|
|
||||||
None to the doc content — committed as generated. NB:
|
|
||||||
the implementation deviated from the plan-as-written in
|
|
||||||
4 places, see the raw file's deviation log.
|
|
||||||
|
|
@ -1,51 +0,0 @@
|
||||||
---
|
|
||||||
model: claude-fable-5[1m]
|
|
||||||
service: claude
|
|
||||||
timestamp: 2026-06-10T17:33:09Z
|
|
||||||
git_ref: datad_service
|
|
||||||
diff_cmd: git log -1 -p --follow -- ai/prompt-io/claude/20260610T173309Z_f15f8178_prompt_io.md
|
|
||||||
---
|
|
||||||
|
|
||||||
NOTE: diff-ref mode entry; the AI output here is the
|
|
||||||
committed doc itself (a design plan, not code) so no
|
|
||||||
verbatim copy is duplicated:
|
|
||||||
|
|
||||||
> `git log -1 -p --follow -- ai/claude-code/plans/datad_service.md`
|
|
||||||
|
|
||||||
## Implementation deviation log
|
|
||||||
|
|
||||||
Where the landed 7-commit series differs from the plan
|
|
||||||
as written (recorded for reviewer transparency; the
|
|
||||||
plan doc is committed unmodified):
|
|
||||||
|
|
||||||
1. Commit ordering: the `pytest` config-dir isolation
|
|
||||||
fix lands BEFORE the `tractor`-API drift port. The
|
|
||||||
plan's stage gates assumed a runnable+isolated test
|
|
||||||
baseline; per-commit gating exposed that without
|
|
||||||
isolation the paper-EMS test reads the user's real
|
|
||||||
(polluted) `account.kraken.paper.toml` and reddens.
|
|
||||||
2. The plan's "stage 0: full pytest green" gate
|
|
||||||
required first repairing pre-existing branch
|
|
||||||
breakage vs `tractor` git `main` (boot
|
|
||||||
`AttributeError`, stale discovery/exc/position
|
|
||||||
APIs) — that repair became its own commit
|
|
||||||
("Port service+tests to latest `tractor` APIs")
|
|
||||||
rather than plan-stage work.
|
|
||||||
3. Stage-4 fail-fast placement: the plan said
|
|
||||||
`broker_init()` raises on brokerd-ep-less backends;
|
|
||||||
implementation moved the raise to `spawn_brokerd()`
|
|
||||||
since `piker ledger` calls `broker_init()` directly
|
|
||||||
even for paper accounts on datad-only backends.
|
|
||||||
4. `brokers/_daemon.py` change grouping: the
|
|
||||||
import-cleanup hunks (`exceptiongroup`, `_FeedsBus`
|
|
||||||
type-only import) landed with the caps-sec slim
|
|
||||||
commit instead of the fixture-slim commit, keeping
|
|
||||||
each intermediate tree import-clean without
|
|
||||||
sub-hunk surgery.
|
|
||||||
|
|
||||||
Also of record: the plan's "Verification" matrix was
|
|
||||||
executed as written (per-suite gates each stage, the
|
|
||||||
headless datad-feed smoke, the kucoin fail-fast unit
|
|
||||||
check); the known pre-existing ~50% second-runtime-boot
|
|
||||||
test wedge was characterized and excluded as a
|
|
||||||
regression via revert-testing.
|
|
||||||
|
|
@ -1,60 +0,0 @@
|
||||||
---
|
|
||||||
model: claude-fable-5[1m]
|
|
||||||
service: claude
|
|
||||||
session: 32d15f9a-b2d3-4c26-bdc9-190219141a25
|
|
||||||
timestamp: 2026-06-10T21:35:49Z
|
|
||||||
git_ref: datad_service
|
|
||||||
diff_cmd: git log -1 -p --follow -- ai/prompt-io/claude/20260610T213549Z_f084e899_prompt_io.md
|
|
||||||
scope: code
|
|
||||||
substantive: true
|
|
||||||
raw_file: 20260610T213549Z_f084e899_prompt_io.raw.md
|
|
||||||
---
|
|
||||||
|
|
||||||
## Prompt
|
|
||||||
|
|
||||||
User bug report from live `ib` testing of the
|
|
||||||
(datad|brokerd)-split branch:
|
|
||||||
|
|
||||||
> ok doing some testing and noticing live orders do not
|
|
||||||
> work otb, pikerd show this on live submission,
|
|
||||||
> [pdb traceback: `RuntimeError("Can not order
|
|
||||||
> {symbol}, no live feed?")` at `ib/api.py:1152`
|
|
||||||
> `submit_limit()` with `self._contracts == {}` in the
|
|
||||||
> `brokerd.ib` actor + a `TrioTaskExited` teardown
|
|
||||||
> cascade]
|
|
||||||
|
|
||||||
Follow-up user direction fixing the design intent:
|
|
||||||
|
|
||||||
> i think the main thing here is that [brokerd] should
|
|
||||||
> be able to always submit orders without a live feed
|
|
||||||
> being up and registered in the same subactor right?
|
|
||||||
|
|
||||||
## Response summary
|
|
||||||
|
|
||||||
Root cause: pre-split the feed code populated the
|
|
||||||
actor-local `Client._contracts` cache (via
|
|
||||||
`get_mkt_info()` writing `mkt.bs_fqme` keys) in the
|
|
||||||
SAME process as order submission; post-split the
|
|
||||||
trading actor's client is never warmed. Fix: lazily
|
|
||||||
qualify + cache the contract per order request in
|
|
||||||
`handle_order_requests()` by running the same
|
|
||||||
`get_mkt_info(fqme, proxy=...)` ep the feed side uses,
|
|
||||||
plus per-order error relay (`BrokerdError`) so one bad
|
|
||||||
submission can't crash the whole trades dialog (the
|
|
||||||
`TrioTaskExited` storm was teardown cascade from the
|
|
||||||
original raise).
|
|
||||||
|
|
||||||
## Files changed
|
|
||||||
|
|
||||||
- `piker/brokers/ib/broker.py` — thread `proxies` into
|
|
||||||
`handle_order_requests()`; lazy contract qualify on
|
|
||||||
cache-miss; guard `submit_limit()` w/ `BrokerdError`
|
|
||||||
relay; uncomment the (anticipatory) `get_mkt_info`
|
|
||||||
import
|
|
||||||
- `piker/brokers/ib/api.py` — fix the non-f-string
|
|
||||||
raise msg + document the new qualification contract
|
|
||||||
|
|
||||||
## Human edits
|
|
||||||
|
|
||||||
None — committed as generated. Live `ib` order retest
|
|
||||||
performed by the human post-commit.
|
|
||||||
|
|
@ -1,57 +0,0 @@
|
||||||
---
|
|
||||||
model: claude-fable-5[1m]
|
|
||||||
service: claude
|
|
||||||
timestamp: 2026-06-10T21:35:49Z
|
|
||||||
git_ref: datad_service
|
|
||||||
diff_cmd: git log -1 -p --follow -- ai/prompt-io/claude/20260610T213549Z_f084e899_prompt_io.md
|
|
||||||
---
|
|
||||||
|
|
||||||
NOTE: diff-ref mode entry (code committed in the same
|
|
||||||
commit as this log); recorded from the live debug
|
|
||||||
session per the `/prompt-io` skill rules.
|
|
||||||
|
|
||||||
> `git log -1 -p --follow -- piker/brokers/ib/broker.py`
|
|
||||||
> `git log -1 -p --follow -- piker/brokers/ib/api.py`
|
|
||||||
|
|
||||||
Key diagnostic chain (from session):
|
|
||||||
|
|
||||||
- pdb showed `Client._contracts == {}` inside
|
|
||||||
`brokerd.ib`'s `submit_limit()`; the cache has
|
|
||||||
exactly TWO write sites: `Client.find_contracts()`
|
|
||||||
(api.py, keys `f'{sym}.{exch}.ib'`) and
|
|
||||||
`symbols.get_mkt_info()` (key `mkt.bs_fqme`, eg.
|
|
||||||
'nvda.nasdaq' — NO `.ib` suffix).
|
|
||||||
- `BrokerdOrder.symbol` arrives as the bs_fqme form
|
|
||||||
('nvda.nasdaq') so ONLY the `get_mkt_info()` write
|
|
||||||
site produces the key `submit_limit()` reads —
|
|
||||||
ie. pre-split it was the feed's in-proc
|
|
||||||
`get_mkt_info(sym, proxy=proxy)` call keeping orders
|
|
||||||
working, NOT `find_contracts()`.
|
|
||||||
- the existing TODO at `symbols.py:642-644` literally
|
|
||||||
predicted this: "this is going to be problematic
|
|
||||||
if/when we split out the datad vs. brokerd actors
|
|
||||||
since the mktmap lookup table will now be
|
|
||||||
inaccessible.."
|
|
||||||
- instance identity verified: `proxy._aio_ns` IS the
|
|
||||||
same `Client` obj as `_accounts2clients[account]`
|
|
||||||
(both sourced from the `load_aio_clients()` cache via
|
|
||||||
`open_client_proxies()`), so a brokerd-side
|
|
||||||
`get_mkt_info(fqme, proxy=proxies[account])` warms
|
|
||||||
exactly the dict `submit_limit()` reads. It also
|
|
||||||
populates `client._cons2mkts` which the
|
|
||||||
position-audit path (`broker.py` backup-table code)
|
|
||||||
needs in this actor anyway.
|
|
||||||
- the `TrioTaskExited` storm in the user's log
|
|
||||||
(`recv_trade_updates`, `open_aio_client_method_relay`
|
|
||||||
aio tasks) is teardown cascade: the raise crashed
|
|
||||||
`handle_order_requests` -> nursery teardown ripped
|
|
||||||
the trio sides of still-running aio relay tasks.
|
|
||||||
Hence the added per-order try/except ->
|
|
||||||
`BrokerdError` relay hardening so a single bad
|
|
||||||
submission degrades to an EMS error msg instead of
|
|
||||||
killing the backend's entire order-ctl dialog.
|
|
||||||
|
|
||||||
Verification: `tests/test_services.py` (5 passed) +
|
|
||||||
`tests/test_ems.py` (6 passed) regression-green; live
|
|
||||||
`ib` submission retest delegated to the human (needs a
|
|
||||||
running TWS/gw).
|
|
||||||
|
|
@ -1,34 +0,0 @@
|
||||||
# AI Prompt I/O Log — claude
|
|
||||||
|
|
||||||
This directory tracks prompt inputs and model
|
|
||||||
outputs for AI-assisted development using
|
|
||||||
`claude` (claude-code CLI).
|
|
||||||
|
|
||||||
## Policy
|
|
||||||
|
|
||||||
Prompt logging follows the
|
|
||||||
[NLNet generative AI policy][nlnet-ai].
|
|
||||||
All substantive AI contributions are logged
|
|
||||||
with:
|
|
||||||
- Model name and version
|
|
||||||
- Timestamps
|
|
||||||
- The prompts that produced the output
|
|
||||||
- Unedited model output (`.raw.md` files)
|
|
||||||
|
|
||||||
[nlnet-ai]: https://nlnet.nl/foundation/policies/generativeAI/
|
|
||||||
|
|
||||||
## Usage
|
|
||||||
|
|
||||||
Entries are created by the `/prompt-io` skill
|
|
||||||
or automatically via `/commit-msg` integration.
|
|
||||||
|
|
||||||
Each commit carrying AI-generated changes links
|
|
||||||
to its provenance entry via a `Prompt-IO:`
|
|
||||||
commit-msg trailer; entries use "diff-ref mode"
|
|
||||||
(pointers into `git log -p` instead of verbatim
|
|
||||||
code copies) to avoid duplicating committed
|
|
||||||
code.
|
|
||||||
|
|
||||||
Human contributors remain accountable for all
|
|
||||||
code decisions. AI-generated content is never
|
|
||||||
presented as human-authored work.
|
|
||||||
|
|
@ -1,64 +0,0 @@
|
||||||
---
|
|
||||||
model: gpt-6 (exact variant not exposed)
|
|
||||||
service: codex
|
|
||||||
timestamp: 2026-09-23T16:46:56.830897+00:00
|
|
||||||
git_ref: efa72980
|
|
||||||
session: 01a0a5b9-f872-7ca3-ab8a-d1cffc14e895
|
|
||||||
scope: code
|
|
||||||
substantive: true
|
|
||||||
raw_file: 20260923T164656Z_efa72980_prompt_io.raw.md
|
|
||||||
---
|
|
||||||
|
|
||||||
## Prompt
|
|
||||||
|
|
||||||
Investigate existing branch fixes before repairing IB day-skipping.
|
|
||||||
Create an isolated worktree on ib_connectivity_harden and add a test
|
|
||||||
that checks actual persisted data, leaving live gap repair for later.
|
|
||||||
|
|
||||||
## Response summary
|
|
||||||
|
|
||||||
Preserve IB request boundaries on HMDS no-data responses. Retain the
|
|
||||||
existing earliest-history classification and bounded reset handling.
|
|
||||||
Separate requested and published boundaries during backfill so the
|
|
||||||
first response after an empty interval does not lose its end bar.
|
|
||||||
Prove supplied bars survive both SHM publication and a fresh parquet
|
|
||||||
load while a simulated venue closure remains empty.
|
|
||||||
|
|
||||||
## Files changed
|
|
||||||
|
|
||||||
- `piker/brokers/ib/feed.py`: complete HMDS no-data requests unchanged.
|
|
||||||
- `piker/tsp/_history.py`: filter overlap against published history.
|
|
||||||
- `tests/test_ib_history.py`: six empty-response boundary cases.
|
|
||||||
- `tests/test_history_backfill.py`: two SHM/parquet integration cases.
|
|
||||||
|
|
||||||
## Human edits
|
|
||||||
|
|
||||||
The human identified the live gap, required prior-branch discovery,
|
|
||||||
selected ib_connectivity_harden as the base, and requested stronger
|
|
||||||
end-to-end evidence. They corrected overstated reproduction claims,
|
|
||||||
insisted on plain explanations, and pointed out that valid venue
|
|
||||||
closures must remain gaps. Existing data repair was explicitly deferred.
|
|
||||||
No direct human source edits have been observed; changes await review.
|
|
||||||
|
|
||||||
## Review follow-up
|
|
||||||
|
|
||||||
The human's tuicr review asked whether None was part of the backfill
|
|
||||||
protocol and requested plain explanations of the two dates and overlap
|
|
||||||
filter. Clarified the IB-internal empty-response sentinel and its
|
|
||||||
translation into NoData/DataUnavailable before reaching the backfiller.
|
|
||||||
Renamed last_start_dt to request_end_dt and published_start_dt to
|
|
||||||
oldest_shm_dt. Added a concrete empty-interval example at the filter and
|
|
||||||
explained that only successful SHM writes move oldest_shm_dt.
|
|
||||||
These changes clarify naming and documentation without changing the
|
|
||||||
previously tested request or publication behavior.
|
|
||||||
|
|
||||||
## History-filter review follow-up
|
|
||||||
|
|
||||||
The human flagged the unused append_until_dt argument and confusing
|
|
||||||
append/prepend terminology in diff_history(). The helper had one
|
|
||||||
caller, using only prepend_until_dt. Removed the helper and inlined
|
|
||||||
its inclusive lower-time cutoff beside the SHM duplicate filter.
|
|
||||||
The human's nearby TODO comment is resolved by this removal; the
|
|
||||||
replacement comment describes the retained timestamps directly.
|
|
||||||
The storage comparison remains >= backfill_until_dt; the SHM
|
|
||||||
comparison remains < oldest_shm_dt.
|
|
||||||
|
|
@ -1,87 +0,0 @@
|
||||||
---
|
|
||||||
model: gpt-6 (exact variant not exposed)
|
|
||||||
service: codex
|
|
||||||
timestamp: 2026-09-23T16:46:56.830897+00:00
|
|
||||||
git_ref: efa72980
|
|
||||||
diff_cmd: git diff efa72980 -- piker/brokers/ib/feed.py piker/tsp/_history.py tests/test_ib_history.py tests/test_history_backfill.py
|
|
||||||
---
|
|
||||||
|
|
||||||
## Prompt
|
|
||||||
|
|
||||||
The human requested the IB request-boundary repair in a new worktree
|
|
||||||
based on ib_connectivity_harden and a test exercising real persistence.
|
|
||||||
The existing MNQ parquet gap is outside this repair scope.
|
|
||||||
|
|
||||||
## Code output
|
|
||||||
|
|
||||||
> `git diff efa72980 -- piker/brokers/ib/feed.py`
|
|
||||||
|
|
||||||
> `git diff efa72980 -- piker/tsp/_history.py`
|
|
||||||
|
|
||||||
> `git diff efa72980 -- tests/test_ib_history.py`
|
|
||||||
|
|
||||||
> `git diff efa72980 -- tests/test_history_backfill.py`
|
|
||||||
|
|
||||||
get_bars() now completes HMDS no-data responses without changing the
|
|
||||||
requested date. open_history_client() retains its NoData and earliest
|
|
||||||
history classification. start_backfill() tracks the published boundary
|
|
||||||
separately from the request cursor after empty intervals.
|
|
||||||
|
|
||||||
## Unedited progress output
|
|
||||||
|
|
||||||
The new test failed on the original code as intended: the
|
|
||||||
temporary parquet contained only 4 bars instead of 2,004.
|
|
||||||
The day-skip caused 2,000 available bars to be omitted.
|
|
||||||
|
|
||||||
The test also includes a simulated venue closure that must remain
|
|
||||||
a gap. It checks that we fetch available bars without inventing
|
|
||||||
data for an empty interval.
|
|
||||||
|
|
||||||
The IB fix now preserves all 2,004 bars in parquet. The SHM
|
|
||||||
assertion exposed a second bug: after stepping over an empty
|
|
||||||
interval, the backfiller drops the last bar of the next response
|
|
||||||
as though it were already published.
|
|
||||||
|
|
||||||
I’ll separate the request date from the published-data boundary
|
|
||||||
so the same test can verify that both memory and disk retain
|
|
||||||
every available bar.
|
|
||||||
|
|
||||||
## Validation
|
|
||||||
|
|
||||||
Using the existing fsp_backfill_sync/py313 interpreter, with cwd in
|
|
||||||
ib_history_request_bounds and the Piker import checked against that cwd:
|
|
||||||
|
|
||||||
- tests/test_history_backfill.py: 11 passed in 1.92s.
|
|
||||||
- tests/test_ib_history.py: 17 passed in 0.36s.
|
|
||||||
- Ruff on all four changed Python files: all checks passed.
|
|
||||||
- git diff --check: passed.
|
|
||||||
|
|
||||||
The integration test uses simulated IB bars responses, real IB history
|
|
||||||
adapter and backfill logic, real SHM, and temporary NativeDB parquet.
|
|
||||||
It checks a fresh disk load. It does not test a live gateway, actor RPC,
|
|
||||||
or Qt rendering, and does not prove the origin of the user's live gap.
|
|
||||||
|
|
||||||
## Unedited review response
|
|
||||||
|
|
||||||
`None` is internal to the IB adapter. The backfiller receives
|
|
||||||
`NoData`, or `DataUnavailable` when the requested date has reached
|
|
||||||
the known beginning of history. I’ll make that distinction explicit.
|
|
||||||
|
|
||||||
The two dates answer different questions: “where should the next
|
|
||||||
history query end?” and “what is the oldest bar already in SHM?”
|
|
||||||
An empty reply changes only the first. I’ll rename them to
|
|
||||||
`request_end_dt` and `oldest_shm_dt` and explain the filtering with
|
|
||||||
a concrete example.
|
|
||||||
|
|
||||||
> `git diff efa72980 -- piker/brokers/ib/feed.py piker/tsp/_history.py`
|
|
||||||
|
|
||||||
## History-filter review follow-up
|
|
||||||
|
|
||||||
The human flagged the unused append_until_dt argument and confusing
|
|
||||||
append/prepend terminology in diff_history(). The helper had one
|
|
||||||
caller, using only prepend_until_dt. Removed the helper and inlined
|
|
||||||
its inclusive lower-time cutoff beside the SHM duplicate filter.
|
|
||||||
The human's nearby TODO comment is resolved by this removal; the
|
|
||||||
replacement comment describes the retained timestamps directly.
|
|
||||||
The storage comparison remains >= backfill_until_dt; the SHM
|
|
||||||
comparison remains < oldest_shm_dt.
|
|
||||||
|
|
@ -1,51 +0,0 @@
|
||||||
---
|
|
||||||
model: gpt-6 (exact variant not exposed)
|
|
||||||
service: codex
|
|
||||||
timestamp: 2026-09-24T16:57:57.342784+00:00
|
|
||||||
git_ref: 1da9b760
|
|
||||||
scope: code
|
|
||||||
substantive: true
|
|
||||||
raw_file: 20260924T165757Z_1da9b760_prompt_io.raw.md
|
|
||||||
---
|
|
||||||
|
|
||||||
## Prompt
|
|
||||||
|
|
||||||
Document and test the existing null-segment repair, including its
|
|
||||||
flat-bar fallback, while distinguishing packed timestamp gaps.
|
|
||||||
|
|
||||||
## Response summary
|
|
||||||
|
|
||||||
Documented SHM-only repair, provider placement assumptions, fallback
|
|
||||||
fields, notifications, and concurrency limits. Added boundary and
|
|
||||||
no-history coverage and fixed epoch-dependent comparison tolerance
|
|
||||||
which could synthesize a duplicate timestamp at the right margin.
|
|
||||||
|
|
||||||
## Files changed
|
|
||||||
|
|
||||||
- piker/tsp/_history.py: helper contract and absolute tolerance.
|
|
||||||
- tests/test_history_backfill.py: four additional parameter/case runs.
|
|
||||||
|
|
||||||
## Human edits
|
|
||||||
|
|
||||||
The human identified cancellation as a source of stored gaps,
|
|
||||||
correctly described packed indexes and insertion costs, pointed to
|
|
||||||
maybe_fill_null_segments(), and requested immediate documentation
|
|
||||||
and tests. No direct source edits observed during this change;
|
|
||||||
the patch remains uncommitted for review.
|
|
||||||
|
|
||||||
## Close-copy factoring follow-up
|
|
||||||
|
|
||||||
The human requested stronger internal types, a reusable close-copy
|
|
||||||
helper, searchable comments for copying close and synthesizing
|
|
||||||
stamps, and research into historical venue closure queries.
|
|
||||||
Extracted _copy_close_to_null_rows(), preserving non-OHLC fields and
|
|
||||||
rejecting occupied slots before mutation. Added explicit callback,
|
|
||||||
array, segment-index and date annotations. Preserved the existing
|
|
||||||
provenance TODO and suggested avoiding volume sentinels.
|
|
||||||
Four helper cases cover successful fill and unchanged rejected
|
|
||||||
inputs. Backfill suite: 19 passed in 2.01s; Ruff and diff check pass.
|
|
||||||
|
|
||||||
IB documents historical SCHEDULE requests separately from contract
|
|
||||||
hours. The installed ib_async exposes reqHistoricalScheduleAsync.
|
|
||||||
No live schedule request or venue behavior change was performed.
|
|
||||||
Source: https://interactivebrokers.github.io/tws-api/historical_bars.html
|
|
||||||
|
|
@ -1,48 +0,0 @@
|
||||||
---
|
|
||||||
model: gpt-6 (exact variant not exposed)
|
|
||||||
service: codex
|
|
||||||
timestamp: 2026-09-24T16:57:57.342784+00:00
|
|
||||||
git_ref: 1da9b760
|
|
||||||
diff_cmd: git diff 1da9b760 -- piker/tsp/_history.py tests/test_history_backfill.py
|
|
||||||
---
|
|
||||||
|
|
||||||
## Prompt
|
|
||||||
|
|
||||||
The human requested documentation and tests for
|
|
||||||
maybe_fill_null_segments(), identifying its flat OHLC fallback
|
|
||||||
and asking how packed SHM could accept later recovered bars.
|
|
||||||
|
|
||||||
## Code output
|
|
||||||
|
|
||||||
> `git diff 1da9b760 -- piker/tsp/_history.py`
|
|
||||||
|
|
||||||
> `git diff 1da9b760 -- tests/test_history_backfill.py`
|
|
||||||
|
|
||||||
Document the existing slot-based repair and its limitations.
|
|
||||||
Extend fallback coverage to NoData and DataUnavailable; verify
|
|
||||||
the observed packed MNQ gap remains unchanged. Reproduce the
|
|
||||||
relative-tolerance failure at a modern epoch, then disable relative
|
|
||||||
tolerance in the synthetic timestamp boundary check.
|
|
||||||
|
|
||||||
## Validation
|
|
||||||
|
|
||||||
Before the fix: modern-epoch boundary case failed, zero-epoch
|
|
||||||
case passed. The overfull synthetic run duplicated the right bar.
|
|
||||||
After the fix: 15 backfill tests passed in 1.99s.
|
|
||||||
|
|
||||||
## Close-copy factoring follow-up
|
|
||||||
|
|
||||||
The human requested stronger internal types, a reusable close-copy
|
|
||||||
helper, searchable comments for copying close and synthesizing
|
|
||||||
stamps, and research into historical venue closure queries.
|
|
||||||
Extracted _copy_close_to_null_rows(), preserving non-OHLC fields and
|
|
||||||
rejecting occupied slots before mutation. Added explicit callback,
|
|
||||||
array, segment-index and date annotations. Preserved the existing
|
|
||||||
provenance TODO and suggested avoiding volume sentinels.
|
|
||||||
Four helper cases cover successful fill and unchanged rejected
|
|
||||||
inputs. Backfill suite: 19 passed in 2.01s; Ruff and diff check pass.
|
|
||||||
|
|
||||||
IB documents historical SCHEDULE requests separately from contract
|
|
||||||
hours. The installed ib_async exposes reqHistoricalScheduleAsync.
|
|
||||||
No live schedule request or venue behavior change was performed.
|
|
||||||
Source: https://interactivebrokers.github.io/tws-api/historical_bars.html
|
|
||||||
|
|
@ -1,91 +0,0 @@
|
||||||
---
|
|
||||||
model: gpt-6 (exact variant not exposed)
|
|
||||||
service: codex
|
|
||||||
timestamp: 2026-09-24T18:54:56.893490+00:00
|
|
||||||
git_ref: 1da9b760
|
|
||||||
scope: code
|
|
||||||
substantive: true
|
|
||||||
raw_file: 20260924T185456Z_1da9b760_schedule_prompt_io.raw.md
|
|
||||||
---
|
|
||||||
|
|
||||||
## Prompt
|
|
||||||
|
|
||||||
The human requested: first freeze a commit-plan for the existing
|
|
||||||
close-copy/null-repair changes, then implement ib_async historical
|
|
||||||
schedule support with tests, use it for datad.ib history-buffer gap
|
|
||||||
spacing, and plan a common on-demand segment interface for CLI and
|
|
||||||
chart selection. Avoid reindexing in this first pass and record
|
|
||||||
fallback insertion only as a TODO.
|
|
||||||
|
|
||||||
## Response summary
|
|
||||||
|
|
||||||
Implemented historical schedule normalization, bounded startup
|
|
||||||
reservations, explicit actor-local synthetic ranges, and two read
|
|
||||||
RPC functions. Added deterministic adapter tests and real SHM/parquet
|
|
||||||
startup coverage. Wrote the segment-backfill interface plan.
|
|
||||||
|
|
||||||
This entry covers the new schedule work only. The earlier frozen
|
|
||||||
null-repair plan has its own provenance. The shared _history.py file
|
|
||||||
contains changes from both scopes until the earlier commit is made.
|
|
||||||
|
|
||||||
## Human edits
|
|
||||||
|
|
||||||
The human identified cancellation during backfill as the reproducible
|
|
||||||
trigger, explained the packed-index insertion tradeoff, requested
|
|
||||||
reuse of venue closure knowledge, and chose placeholders to avoid
|
|
||||||
expensive rewrites. They required explicit placeholder annotation,
|
|
||||||
proposed later graphics compression, and asked to defer insertion
|
|
||||||
fallbacks. They also requested the existing null-repair commit plan
|
|
||||||
before the new schedule work. These are human-directed architecture
|
|
||||||
and scope choices; no acceptance or task completion state is inferred.
|
|
||||||
|
|
||||||
## Validation and limits
|
|
||||||
|
|
||||||
55 targeted tests passed; Ruff and diff checks passed. No live IB
|
|
||||||
qualification was performed. Reservations are synthetic, not recovered
|
|
||||||
observations. Repair execution, persistence of unfinished requests,
|
|
||||||
chart compression, FSP handling, export filtering, and request-level
|
|
||||||
IB cancellation remain follow-up work documented in the plan.
|
|
||||||
|
|
||||||
## Files changed
|
|
||||||
|
|
||||||
- `piker/brokers/ib/venues.py`
|
|
||||||
- `piker/brokers/ib/api.py`
|
|
||||||
- `piker/brokers/ib/feed.py`
|
|
||||||
- `piker/tsp/_reserve.py`
|
|
||||||
- `piker/tsp/_history.py`
|
|
||||||
- `tests/providers/test_ib_schedule.py`
|
|
||||||
- `tests/providers/test_history_reservations.py`
|
|
||||||
- `docs/history_segment_backfill.rst`
|
|
||||||
|
|
||||||
## Pre-commit module review
|
|
||||||
|
|
||||||
The human requested placing historical schedule normalization in the
|
|
||||||
existing venues.py module, naming the allocation module _reserve.py,
|
|
||||||
and adding the missing project license headers. Applied those moves,
|
|
||||||
updated imports and test patch targets, and added the standard AGPL
|
|
||||||
headers to the new allocation module and both new test modules.
|
|
||||||
|
|
||||||
## Local tuicr review follow-up
|
|
||||||
|
|
||||||
The human supplied eleven comments requesting a configurable newest
|
|
||||||
gap count, caller and metadata references, explanations of guards
|
|
||||||
and provider-specific limits, multiline boolean style, capacity
|
|
||||||
fallback rationale, an explanation of numpy split boundaries, and
|
|
||||||
clearer test cases, fixture types, and module docstrings.
|
|
||||||
|
|
||||||
Added newest_gaps=32, with zero disabling queries and negative values
|
|
||||||
rejected. Documented tsdb_backfill ownership and the local/RPC range
|
|
||||||
accessors. Explained recoverable overflow and half-open synthetic
|
|
||||||
range grouping. Typed tmp_path as pathlib.Path and monkeypatch as
|
|
||||||
pytest.MonkeyPatch. Added four request-budget cases.
|
|
||||||
|
|
||||||
Verification: reservation tests 11 passed; schedule tests 7 passed;
|
|
||||||
Ruff and git diff --check passed. Changes remain unstaged for review.
|
|
||||||
No forge review or CI state was queried; this was local feedback.
|
|
||||||
|
|
||||||
## Provider-test layout review
|
|
||||||
|
|
||||||
The human requested grouping both new history/schedule test modules
|
|
||||||
under tests/providers. Moved both files without changing test logic;
|
|
||||||
they retain the shared tests/conftest.py fixtures.
|
|
||||||
|
|
@ -1,100 +0,0 @@
|
||||||
---
|
|
||||||
model: gpt-6 (exact variant not exposed)
|
|
||||||
service: codex
|
|
||||||
timestamp: 2026-09-24T18:54:56.893490+00:00
|
|
||||||
git_ref: 1da9b760
|
|
||||||
diff_cmd: git diff HEAD~1..HEAD
|
|
||||||
---
|
|
||||||
|
|
||||||
> `git diff HEAD~1..HEAD -- piker/brokers/ib/_schedule.py`
|
|
||||||
|
|
||||||
> `git diff HEAD~1..HEAD -- piker/brokers/ib/api.py`
|
|
||||||
|
|
||||||
> `git diff HEAD~1..HEAD -- piker/brokers/ib/feed.py`
|
|
||||||
|
|
||||||
> `git diff HEAD~1..HEAD -- piker/tsp/_reservations.py`
|
|
||||||
|
|
||||||
> `git diff HEAD~1..HEAD -- piker/tsp/_history.py`
|
|
||||||
|
|
||||||
> `git diff HEAD~1..HEAD -- tests/test_ib_schedule.py`
|
|
||||||
|
|
||||||
> `git diff HEAD~1..HEAD -- tests/test_history_reservations.py`
|
|
||||||
|
|
||||||
> `git diff HEAD~1..HEAD -- docs/history_segment_backfill.rst`
|
|
||||||
|
|
||||||
## Generated output
|
|
||||||
|
|
||||||
Added Client.history_schedule() over the installed ib_async API,
|
|
||||||
normalizing dated sessions to UTC with explicit response coverage.
|
|
||||||
Reject ambiguous DST times, reversed sessions, and overlapping
|
|
||||||
sessions. Match the existing extended-hours bars policy.
|
|
||||||
|
|
||||||
Added startup-only schedule reservations to tsdb_backfill before
|
|
||||||
prepending older stored history. Reserve open-session sample slots
|
|
||||||
only where the reply covers the full candidate gap. Return synthetic
|
|
||||||
UTC ranges separately from OHLCV; copy preceding close into flat
|
|
||||||
placeholders without writing those placeholders to parquet.
|
|
||||||
Unknown coverage and capacity failures retain packed time gaps.
|
|
||||||
|
|
||||||
Added datad.ib schedule and reservation metadata RPC functions and
|
|
||||||
a proposed shared CLI/chart segment request protocol. The repair
|
|
||||||
queue, durable unresolved intervals, chart compression, FSP handling,
|
|
||||||
and coordinated insertion remain deferred. No live data was repaired.
|
|
||||||
|
|
||||||
Verification is complete: 55 targeted tests pass, including the
|
|
||||||
new schedule and real SHM/parquet tests. I also added checks for
|
|
||||||
reversed session bounds and the 25-hour day at the autumn DST
|
|
||||||
change.
|
|
||||||
|
|
||||||
The segment API plan uses UTC ranges for both CLI and chart
|
|
||||||
requests, a provider-owned queue, durable unfinished intervals,
|
|
||||||
and replacement of reserved slots without reindexing. Live IB
|
|
||||||
schedule coverage still needs qualification.
|
|
||||||
|
|
||||||
## Limits
|
|
||||||
|
|
||||||
Only stored-history gaps are reserved before prepend; published rows
|
|
||||||
are never shifted. Newest 32 gaps, at most 31 days each, are queried
|
|
||||||
serially with a 10-second schedule timeout. This can delay history
|
|
||||||
publication. Underlying IB request cancellation is still a TODO.
|
|
||||||
Manual whole-SHM export requires synthetic-range exclusion before
|
|
||||||
it can preserve provider-only storage. The chart currently displays
|
|
||||||
flat placeholders, and synthetic-aware FSP treatment is deferred.
|
|
||||||
|
|
||||||
Tests: history backfill 19, IB history 17, method proxy 5, schedule 7,
|
|
||||||
reservations 7. Sandbox local-socket failures were rerun with approval.
|
|
||||||
Ruff and git diff --check passed. No live broker requests were made.
|
|
||||||
|
|
||||||
## Module-layout review response
|
|
||||||
|
|
||||||
Schedule normalization now lives in piker.brokers.ib.venues beside
|
|
||||||
the existing session helpers. The allocation module is tsp._reserve.
|
|
||||||
Added the project AGPL headers to the new module and test files.
|
|
||||||
Earlier diff pointers record the original generated paths; use
|
|
||||||
the following paths for the reviewed layout:
|
|
||||||
|
|
||||||
> `git diff HEAD~1..HEAD -- piker/brokers/ib/venues.py`
|
|
||||||
|
|
||||||
> `git diff HEAD~1..HEAD -- piker/tsp/_reserve.py`
|
|
||||||
|
|
||||||
## Local review response
|
|
||||||
|
|
||||||
The fixes pass all 18 tests in the two affected files.
|
|
||||||
The new limit tests cover zero, negative, one-gap, and two-gap
|
|
||||||
budgets, including newest-first selection.
|
|
||||||
|
|
||||||
Capacity overflow remains a warning: no partial reservation is
|
|
||||||
published for that gap, and the original observations remain
|
|
||||||
available. The comments now explain that fallback explicitly.
|
|
||||||
|
|
||||||
Diff pointers above include the review edits to tsp._reserve, both
|
|
||||||
test modules, and the interface document. This review did not query
|
|
||||||
a forge or publish replies.
|
|
||||||
|
|
||||||
## Provider-test paths
|
|
||||||
|
|
||||||
Moved both new test modules under tests/providers as requested.
|
|
||||||
|
|
||||||
> `git diff HEAD~1..HEAD -- tests/providers/test_ib_schedule.py`
|
|
||||||
|
|
||||||
> `git diff HEAD~1..HEAD -- tests/providers/test_history_reservations.py`
|
|
||||||
|
|
@ -1,45 +0,0 @@
|
||||||
---
|
|
||||||
model: gpt-6
|
|
||||||
service: codex
|
|
||||||
timestamp: 2026-09-24T23:24:05.525363+00:00
|
|
||||||
git_ref: 441358e2
|
|
||||||
scope: code
|
|
||||||
substantive: true
|
|
||||||
raw_file: 20260924T232405Z_441358e2_schedule_coverage_prompt_io.raw.md
|
|
||||||
---
|
|
||||||
|
|
||||||
## Prompt
|
|
||||||
|
|
||||||
User: "do it", approving the proposed bounded wider-window retry
|
|
||||||
for incomplete schedule coverage and its MNQ regression test.
|
|
||||||
Previous live qualification showed a one-day request began after
|
|
||||||
the gap's left edge; a wider query covered it and excluded the
|
|
||||||
one-hour venue closure.
|
|
||||||
|
|
||||||
## Response summary
|
|
||||||
|
|
||||||
Added bounded backwards widening under one timeout, preserving actual
|
|
||||||
coverage and fixed end bounds. Added captured-response allocation,
|
|
||||||
unknown-coverage, and retry-timeout tests, and updated API documentation.
|
|
||||||
39 selected tests and Ruff passed. Live data was not changed.
|
|
||||||
|
|
||||||
## Files changed
|
|
||||||
|
|
||||||
- piker/brokers/ib/api.py — bounded schedule coverage retries.
|
|
||||||
- tests/providers/test_ib_schedule.py — captured MNQ and retry regressions.
|
|
||||||
- docs/history_segment_backfill.rst — retry contract and limits.
|
|
||||||
|
|
||||||
## Human edits
|
|
||||||
|
|
||||||
The user established schedule-driven allocation, explicit unknown
|
|
||||||
coverage, and avoiding reindexing as the design constraints, supplied
|
|
||||||
the MNQ gap, and authorized this focused coverage fix. No direct human
|
|
||||||
source edits occurred during this patch; review is pending.
|
|
||||||
|
|
||||||
## Follow-up review
|
|
||||||
|
|
||||||
The user requested factoring and documenting the repeated 86400
|
|
||||||
conversion. Added piker.calc.SECONDS_PER_DAY, documented elapsed
|
|
||||||
versus calendar/session days once, and reused it in the IB query
|
|
||||||
and reservation span check. Named the client retry limit locally.
|
|
||||||
Schedule tests: 11 passed; Ruff passed after this adjustment.
|
|
||||||
|
|
@ -1,66 +0,0 @@
|
||||||
---
|
|
||||||
model: gpt-6
|
|
||||||
service: codex
|
|
||||||
timestamp: 2026-09-24T23:24:05.525363+00:00
|
|
||||||
git_ref: 441358e2
|
|
||||||
diff_cmd: git diff HEAD~1..HEAD
|
|
||||||
---
|
|
||||||
|
|
||||||
> `git diff HEAD~1..HEAD -- piker/brokers/ib/api.py`
|
|
||||||
|
|
||||||
> `git diff HEAD~1..HEAD -- tests/providers/test_ib_schedule.py`
|
|
||||||
|
|
||||||
> `git diff HEAD~1..HEAD -- docs/history_segment_backfill.rst`
|
|
||||||
|
|
||||||
Client.history_schedule() widens incomplete left coverage by doubling
|
|
||||||
numDays up to 31 while preserving endDateTime and actual returned
|
|
||||||
coverage. All attempts share an asyncio timeout. Missing right coverage
|
|
||||||
returns unchanged because extending the lookback cannot repair it.
|
|
||||||
Tests replay the captured MNQ schedules through the real client and
|
|
||||||
reservation allocator, check bounded incomplete coverage and retry
|
|
||||||
cancellation under one timeout. Documentation describes these limits.
|
|
||||||
|
|
||||||
## User-facing output
|
|
||||||
|
|
||||||
The captured MNQ case now passes through the real client method
|
|
||||||
and allocator: the request widens from one day to two, then
|
|
||||||
reserves 51,824 slots while leaving the venue closure compressed.
|
|
||||||
|
|
||||||
The tests also cover permanently incomplete replies and a timeout
|
|
||||||
during the retry. I’m running the related regression suites now.
|
|
||||||
|
|
||||||
All 39 selected tests passed, and Ruff passed. The reservation
|
|
||||||
suite needed a retry outside the sandbox because Trio’s local
|
|
||||||
wakeup socket was blocked.
|
|
||||||
|
|
||||||
The fix and regression tests are ready for review. I’m recording
|
|
||||||
the prompt provenance; live SHM and parquet remain unchanged.
|
|
||||||
|
|
||||||
## Validation commands and results
|
|
||||||
|
|
||||||
Worktree-local py313 in the existing Nix environment; import piker
|
|
||||||
resolved inside this worktree.
|
|
||||||
|
|
||||||
- py313/bin/python -m pytest -p no:xonsh -q -x --tb=short --no-header tests/providers/test_ib_schedule.py tests/providers/test_history_reservations.py
|
|
||||||
Sandbox: 11 passed then PermissionError creating Trio wakeup socket.
|
|
||||||
Escalated rerun: 22 passed in 1.02s; fake IB, disposable test SHM.
|
|
||||||
- py313/bin/python -m pytest -p no:xonsh -q -x --tb=short --no-header tests/test_ib_history.py
|
|
||||||
17 passed in 0.69s.
|
|
||||||
- ruff check piker/brokers/ib/api.py tests/providers/test_ib_schedule.py
|
|
||||||
All checks passed (existing deprecated configuration warning).
|
|
||||||
- git diff --check: passed.
|
|
||||||
|
|
||||||
No live service restart, live history repair, staging, or commit.
|
|
||||||
|
|
||||||
## Follow-up user review and generated changes
|
|
||||||
|
|
||||||
User: "feels like this same magic # from the prior commit should be
|
|
||||||
factored and commented once? `num_days: int = max(1, ceil(seconds /
|
|
||||||
86400))` other then that all looks decent"
|
|
||||||
|
|
||||||
> `git diff HEAD~1..HEAD -- piker/calc.py piker/tsp/_reserve.py`
|
|
||||||
|
|
||||||
Added SECONDS_PER_DAY for both consumers, with one comment explaining
|
|
||||||
fixed elapsed days versus calendar days and trading sessions. Named
|
|
||||||
the client's 31-day retry limit locally. After this change the
|
|
||||||
schedule suite passed 11 tests and Ruff passed all four Python files.
|
|
||||||
|
|
@ -1,41 +0,0 @@
|
||||||
---
|
|
||||||
model: gpt-5.6-sol
|
|
||||||
provider: openai
|
|
||||||
service: opencode
|
|
||||||
session: ses_0799212ebffe42arY96czXn89F
|
|
||||||
timestamp: 2026-07-21T21:50:55Z
|
|
||||||
git_ref: c6ec3d41
|
|
||||||
scope: docs
|
|
||||||
substantive: true
|
|
||||||
raw_file: 20260721T215055Z_c6ec3d41_prompt_io.raw.md
|
|
||||||
---
|
|
||||||
|
|
||||||
## Prompt
|
|
||||||
|
|
||||||
The user asked to boot a plan to continue work on the
|
|
||||||
time-series gap checker and bug fixing of the history
|
|
||||||
backfiller. The user authorized prompt-I/O capture for the
|
|
||||||
plan documentation after the repository audit.
|
|
||||||
|
|
||||||
## Response summary
|
|
||||||
|
|
||||||
Produced a bounded repair plan based on current source,
|
|
||||||
historical issue caches, existing profiling notes, and a
|
|
||||||
test-coverage audit. The plan prioritizes reproducible
|
|
||||||
correctness failures and storage durability before venue
|
|
||||||
classification, annotation performance, or manual repair
|
|
||||||
UX.
|
|
||||||
|
|
||||||
## Files changed
|
|
||||||
|
|
||||||
- `plans/opencode/gap-checker-backfiller-repair.md` -
|
|
||||||
phased implementation and verification plan
|
|
||||||
- `ai/prompt-io/opencode/20260721T215055Z_c6ec3d41_prompt_io.raw.md`
|
|
||||||
- unedited response record
|
|
||||||
- `ai/prompt-io/opencode/20260721T215055Z_c6ec3d41_prompt_io.md`
|
|
||||||
- provenance metadata and response summary
|
|
||||||
|
|
||||||
## Human edits
|
|
||||||
|
|
||||||
None - generated plan has not yet been edited by the
|
|
||||||
human.
|
|
||||||
|
|
@ -1,26 +0,0 @@
|
||||||
---
|
|
||||||
model: gpt-5.6-sol
|
|
||||||
provider: openai
|
|
||||||
service: opencode
|
|
||||||
timestamp: 2026-07-21T21:50:55Z
|
|
||||||
git_ref: c6ec3d41
|
|
||||||
diff_cmd: git diff HEAD~1..HEAD
|
|
||||||
---
|
|
||||||
|
|
||||||
> `git diff -- plans/opencode/gap-checker-backfiller-repair.md`
|
|
||||||
|
|
||||||
Generated a phased implementation plan for continuing
|
|
||||||
piker's gap checker and history backfiller repairs. The
|
|
||||||
plan separates data correctness from annotation
|
|
||||||
performance and orders work by dependency: deterministic
|
|
||||||
tests, canonical frame/gap normalization, structured
|
|
||||||
backfill completion, non-destructive atomic NativeDB
|
|
||||||
writes, exact null repair, provider/venue classification,
|
|
||||||
annotation vectorization, and crash qualification.
|
|
||||||
|
|
||||||
The plan is grounded in current code paths, local issue
|
|
||||||
caches `#62`, `#71`, and `#75`, and the existing 1,285-gap
|
|
||||||
profiling handoff. It records confirmed present defects,
|
|
||||||
design invariants, per-phase acceptance gates, test
|
|
||||||
commands, commit boundaries, and explicitly deferred UX
|
|
||||||
and provider work.
|
|
||||||
|
|
@ -1,46 +0,0 @@
|
||||||
---
|
|
||||||
model: gpt-5.6-sol
|
|
||||||
provider: openai
|
|
||||||
service: opencode
|
|
||||||
session: ses_0799212ebffe42arY96czXn89F
|
|
||||||
timestamp: 2026-07-22T00:13:57Z
|
|
||||||
git_ref: 16f3cd6f
|
|
||||||
scope: code
|
|
||||||
substantive: true
|
|
||||||
raw_file: 20260722T001357Z_16f3cd6f_prompt_io.raw.md
|
|
||||||
---
|
|
||||||
|
|
||||||
## Prompt
|
|
||||||
|
|
||||||
Continue the gap-checker and history-backfiller repair plan. Start
|
|
||||||
with deterministic analysis characterization, then fix canonical
|
|
||||||
sorting, dedupe-derived columns, positive gap detection, and null
|
|
||||||
grouping.
|
|
||||||
|
|
||||||
During review, the user challenged the replacement null-grouping
|
|
||||||
algorithm because its initial tests did not assert exact endpoints.
|
|
||||||
The user explicitly required retaining detailed, human-oriented
|
|
||||||
algorithm documentation such as the existing ASCII segment map.
|
|
||||||
|
|
||||||
## Response summary
|
|
||||||
|
|
||||||
Added deterministic analysis regressions and corrected normalization
|
|
||||||
so sorting and duplicate resolution precede derived-column
|
|
||||||
calculation. Unified null grouping for NumPy and Polars inputs,
|
|
||||||
documented its exact inclusive-boundary contract with an adapted
|
|
||||||
visual map, and verified all small null layouts against an independent
|
|
||||||
state-machine oracle. Retained sorted smart-dedupe output.
|
|
||||||
|
|
||||||
## Files changed
|
|
||||||
|
|
||||||
- `piker/tsp/_anal.py` - canonical gap and documented null grouping
|
|
||||||
- `piker/tsp/_dedupe_smart.py` - sorted duplicate resolution
|
|
||||||
- `tests/test_tsp_analysis.py` - exact and exhaustive regressions
|
|
||||||
- `ai/prompt-io/opencode/20260722T001357Z_16f3cd6f_prompt_io.raw.md`
|
|
||||||
- unedited response record
|
|
||||||
- `ai/prompt-io/opencode/20260722T001357Z_16f3cd6f_prompt_io.md`
|
|
||||||
- provenance metadata and response summary
|
|
||||||
|
|
||||||
## Human edits
|
|
||||||
|
|
||||||
None - generated changes have not yet been edited by the human.
|
|
||||||
|
|
@ -1,40 +0,0 @@
|
||||||
---
|
|
||||||
model: gpt-5.6-sol
|
|
||||||
provider: openai
|
|
||||||
service: opencode
|
|
||||||
timestamp: 2026-07-22T00:13:57Z
|
|
||||||
git_ref: 16f3cd6f
|
|
||||||
diff_cmd: git diff HEAD~1..HEAD
|
|
||||||
---
|
|
||||||
|
|
||||||
> `git diff HEAD~1..HEAD -- piker/tsp/_anal.py`
|
|
||||||
|
|
||||||
Generated canonical gap and null-segment normalization. Derived
|
|
||||||
datetime and delta columns are recomputed after final sorting and
|
|
||||||
duplicate resolution, and negative deltas no longer become ordinary
|
|
||||||
gaps.
|
|
||||||
|
|
||||||
NumPy and Polars null grouping share one implementation while
|
|
||||||
retaining the human-oriented visual map of contiguous zero runs. The
|
|
||||||
documented contract defines absolute inclusive endpoints, margin
|
|
||||||
expansion, frame boundary clamping, contiguous-index validation, and
|
|
||||||
requested-column selection.
|
|
||||||
|
|
||||||
> `git diff HEAD~1..HEAD -- piker/tsp/_dedupe_smart.py`
|
|
||||||
|
|
||||||
Adjusted smart dedupe to honor sorted output and resolve duplicate
|
|
||||||
timestamps without retaining stale order.
|
|
||||||
|
|
||||||
> `git diff HEAD~1..HEAD -- tests/test_tsp_analysis.py`
|
|
||||||
|
|
||||||
Added deterministic synthetic OHLCV coverage for sorted dedupe, fresh
|
|
||||||
derived columns, and positive gap detection. Null grouping is checked
|
|
||||||
against readable endpoint examples and an independent state-machine
|
|
||||||
oracle over every layout through six rows, three margin widths, and
|
|
||||||
both NumPy and Polars inputs.
|
|
||||||
|
|
||||||
Verification generated with the patch:
|
|
||||||
|
|
||||||
`timeout -k 5 30 python -m pytest -q tests/test_tsp_analysis.py`
|
|
||||||
|
|
||||||
Result: 11 passed.
|
|
||||||
|
|
@ -1,36 +0,0 @@
|
||||||
---
|
|
||||||
model: gpt-5.6-sol
|
|
||||||
provider: openai
|
|
||||||
service: opencode
|
|
||||||
session: ses_0799212ebffe42arY96czXn89F
|
|
||||||
timestamp: 2026-07-22T00:13:58Z
|
|
||||||
git_ref: 16f3cd6f
|
|
||||||
scope: code
|
|
||||||
substantive: true
|
|
||||||
raw_file: 20260722T001358Z_16f3cd6f_prompt_io.raw.md
|
|
||||||
---
|
|
||||||
|
|
||||||
## Prompt
|
|
||||||
|
|
||||||
Continue the structured backfill phase after the analysis regressions.
|
|
||||||
Characterize provider exhaustion and genuinely empty frames, then remove
|
|
||||||
completion paths that can hang after child-task exit.
|
|
||||||
|
|
||||||
## Response summary
|
|
||||||
|
|
||||||
Removed orphan-prone completion events from the structured nursery flow
|
|
||||||
and checked empty provider frames before indexing timestamp endpoints.
|
|
||||||
Added deterministic timeout-bounded regressions for both exit paths.
|
|
||||||
|
|
||||||
## Files changed
|
|
||||||
|
|
||||||
- `piker/tsp/_history.py` - structured completion and empty-frame fixes
|
|
||||||
- `tests/test_history_backfill.py` - deterministic exit regressions
|
|
||||||
- `ai/prompt-io/opencode/20260722T001358Z_16f3cd6f_prompt_io.raw.md`
|
|
||||||
- unedited response record
|
|
||||||
- `ai/prompt-io/opencode/20260722T001358Z_16f3cd6f_prompt_io.md`
|
|
||||||
- provenance metadata and response summary
|
|
||||||
|
|
||||||
## Human edits
|
|
||||||
|
|
||||||
None - generated changes have not yet been edited by the human.
|
|
||||||
|
|
@ -1,25 +0,0 @@
|
||||||
---
|
|
||||||
model: gpt-5.6-sol
|
|
||||||
provider: openai
|
|
||||||
service: opencode
|
|
||||||
timestamp: 2026-07-22T00:13:58Z
|
|
||||||
git_ref: 16f3cd6f
|
|
||||||
diff_cmd: git diff HEAD~1..HEAD
|
|
||||||
---
|
|
||||||
|
|
||||||
> `git diff HEAD~1..HEAD -- piker/tsp/_history.py`
|
|
||||||
|
|
||||||
Removed redundant completion events whose consumers waited only after
|
|
||||||
their owning nurseries had joined. Added empty provider-frame handling
|
|
||||||
before timestamp endpoint indexing so provider exhaustion exits cleanly.
|
|
||||||
|
|
||||||
> `git diff HEAD~1..HEAD -- tests/test_history_backfill.py`
|
|
||||||
|
|
||||||
Added bounded, actor-free regressions for `DataUnavailable` and empty
|
|
||||||
provider-frame completion.
|
|
||||||
|
|
||||||
Verification generated with the patch:
|
|
||||||
|
|
||||||
`timeout -k 5 60 python -m pytest -q tests/test_history_backfill.py`
|
|
||||||
|
|
||||||
Result: 2 passed.
|
|
||||||
|
|
@ -1,39 +0,0 @@
|
||||||
---
|
|
||||||
model: gpt-5.6-sol
|
|
||||||
provider: openai
|
|
||||||
service: opencode
|
|
||||||
session: ses_0799212ebffe42arY96czXn89F
|
|
||||||
timestamp: 2026-07-22T02:41:33Z
|
|
||||||
git_ref: 3a4f6737
|
|
||||||
scope: code
|
|
||||||
substantive: true
|
|
||||||
raw_file: 20260722T024133Z_3a4f6737_prompt_io.raw.md
|
|
||||||
---
|
|
||||||
|
|
||||||
## Prompt
|
|
||||||
|
|
||||||
The user challenged the analysis commit message's claim that only
|
|
||||||
positive sample-step gaps are detected and asked whether out-of-order
|
|
||||||
timestamps were simply ignored. After confirming the missing separate
|
|
||||||
reporting path, the user instructed the agent to implement the fix.
|
|
||||||
|
|
||||||
## Response summary
|
|
||||||
|
|
||||||
Separated missing-history gaps from timestamp-ordering errors without
|
|
||||||
changing the existing `dedupe()` return shape. Added a structured
|
|
||||||
detector and made normalization warn before sorting removes evidence of
|
|
||||||
duplicates or reversed timestamps.
|
|
||||||
|
|
||||||
## Files changed
|
|
||||||
|
|
||||||
- `piker/tsp/_anal.py` - ordering detector and dedupe reporting
|
|
||||||
- `piker/tsp/__init__.py` - public detector export
|
|
||||||
- `tests/test_tsp_analysis.py` - exact ordering-error regression
|
|
||||||
- `ai/prompt-io/opencode/20260722T024133Z_3a4f6737_prompt_io.raw.md`
|
|
||||||
- unedited response record
|
|
||||||
- `ai/prompt-io/opencode/20260722T024133Z_3a4f6737_prompt_io.md`
|
|
||||||
- provenance metadata and response summary
|
|
||||||
|
|
||||||
## Human edits
|
|
||||||
|
|
||||||
None - generated changes have not yet been edited by the human.
|
|
||||||
|
|
@ -1,30 +0,0 @@
|
||||||
---
|
|
||||||
model: gpt-5.6-sol
|
|
||||||
provider: openai
|
|
||||||
service: opencode
|
|
||||||
timestamp: 2026-07-22T02:41:33Z
|
|
||||||
git_ref: 3a4f6737
|
|
||||||
diff_cmd: git diff HEAD~1..HEAD
|
|
||||||
---
|
|
||||||
|
|
||||||
> `git diff HEAD~1..HEAD -- piker/tsp/_anal.py`
|
|
||||||
|
|
||||||
Added `detect_time_ordering_errors()` to return non-positive timestamp
|
|
||||||
steps as structured rows. `dedupe()` now warns with those rows before
|
|
||||||
sorting and duplicate removal erase the original ordering evidence.
|
|
||||||
|
|
||||||
> `git diff HEAD~1..HEAD -- piker/tsp/__init__.py`
|
|
||||||
|
|
||||||
Exported the ordering detector through the public TSP package.
|
|
||||||
|
|
||||||
> `git diff HEAD~1..HEAD -- tests/test_tsp_analysis.py`
|
|
||||||
|
|
||||||
Changed the negative-delta regression to verify exact predecessor,
|
|
||||||
timestamp, and delta values, confirm exclusion from missing-history
|
|
||||||
gaps, and require explicit normalization reporting.
|
|
||||||
|
|
||||||
Verification generated with the patch:
|
|
||||||
|
|
||||||
- `tests/test_tsp_analysis.py`: 11 passed
|
|
||||||
- Ruff: passed
|
|
||||||
- `git diff --check`: passed
|
|
||||||
|
|
@ -1,43 +0,0 @@
|
||||||
---
|
|
||||||
model: gpt-5.6-sol
|
|
||||||
provider: openai
|
|
||||||
service: opencode
|
|
||||||
session: ses_0799212ebffe42arY96czXn89F
|
|
||||||
timestamp: 2026-07-22T02:41:34Z
|
|
||||||
git_ref: 3a4f6737
|
|
||||||
scope: code
|
|
||||||
substantive: true
|
|
||||||
raw_file: 20260722T024134Z_3a4f6737_prompt_io.raw.md
|
|
||||||
---
|
|
||||||
|
|
||||||
## Prompt
|
|
||||||
|
|
||||||
The user asked why the in-progress `piker/storage` changes were omitted
|
|
||||||
from the earlier commit plan. After receiving an inventory of unfinished
|
|
||||||
Phase 3 requirements, the user instructed the agent to complete the work.
|
|
||||||
|
|
||||||
## Response summary
|
|
||||||
|
|
||||||
Finished non-destructive, atomic NativeDB updates and integrated them
|
|
||||||
with history startup and reverse backfill. Expanded tests through two
|
|
||||||
adversarial review rounds that caught and corrected cross-client lock
|
|
||||||
blocking, stale temp indexing, discontinuous persisted indexes, startup
|
|
||||||
snapshot races, missing latest-frame persistence, notification wedges,
|
|
||||||
and mutable/truncated SHM persistence.
|
|
||||||
|
|
||||||
## Files changed
|
|
||||||
|
|
||||||
- `piker/storage/__init__.py` - incremental storage contract
|
|
||||||
- `piker/storage/nativedb.py` - validated atomic merge persistence
|
|
||||||
- `piker/storage/marketstore/__init__.py` - non-duplicate update mode
|
|
||||||
- `piker/tsp/_history.py` - provider-delta and startup persistence
|
|
||||||
- `tests/test_storage_nativedb.py` - NativeDB durability regressions
|
|
||||||
- `tests/test_history_backfill.py` - persistence lifecycle regressions
|
|
||||||
- `ai/prompt-io/opencode/20260722T024134Z_3a4f6737_prompt_io.raw.md`
|
|
||||||
- unedited response record
|
|
||||||
- `ai/prompt-io/opencode/20260722T024134Z_3a4f6737_prompt_io.md`
|
|
||||||
- provenance metadata and response summary
|
|
||||||
|
|
||||||
## Human edits
|
|
||||||
|
|
||||||
None - generated changes have not yet been edited by the human.
|
|
||||||
|
|
@ -1,43 +0,0 @@
|
||||||
---
|
|
||||||
model: gpt-5.6-sol
|
|
||||||
provider: openai
|
|
||||||
service: opencode
|
|
||||||
timestamp: 2026-07-22T02:41:34Z
|
|
||||||
git_ref: 3a4f6737
|
|
||||||
diff_cmd: git diff HEAD~1..HEAD
|
|
||||||
---
|
|
||||||
|
|
||||||
> `git diff HEAD~1..HEAD -- piker/storage/__init__.py piker/storage/nativedb.py piker/storage/marketstore/__init__.py`
|
|
||||||
|
|
||||||
Added an incremental storage update contract. NativeDB now validates
|
|
||||||
canonical numeric OHLCV data, serializes writers both actor-locally and
|
|
||||||
through cancellable filesystem locks, merges with incoming-wins seam
|
|
||||||
resolution, regenerates contiguous indexes, reopens and validates a
|
|
||||||
same-directory temporary parquet, fsyncs it, atomically replaces the
|
|
||||||
target, fsyncs the directory, and publishes cache only after replacement.
|
|
||||||
|
|
||||||
NativeDB indexing ignores lock and crash-left temporary files.
|
|
||||||
MarketStore uses its non-duplicate write mode for incremental updates.
|
|
||||||
|
|
||||||
> `git diff HEAD~1..HEAD -- piker/tsp/_history.py`
|
|
||||||
|
|
||||||
Separated full provider deltas from capacity-truncated SHM pushes.
|
|
||||||
Backfill persists before SHM mutation, publishes bounded notifications,
|
|
||||||
and delays latest-frame publication until the concurrent pre-update
|
|
||||||
storage snapshot completes.
|
|
||||||
|
|
||||||
> `git diff HEAD~1..HEAD -- tests/test_storage_nativedb.py tests/test_history_backfill.py`
|
|
||||||
|
|
||||||
Added deterministic coverage for merge preservation, conflict policy,
|
|
||||||
atomic failures, reopened-candidate validation, cache consistency,
|
|
||||||
schema and timestamp invariants, contiguous index regeneration,
|
|
||||||
separate series, stale temp indexing, cancellable file-lock contention,
|
|
||||||
full provider-delta persistence, latest-frame ordering, and bounded
|
|
||||||
notification teardown.
|
|
||||||
|
|
||||||
Verification generated with the patch:
|
|
||||||
|
|
||||||
- deterministic analysis/backfill/storage set: 31 passed
|
|
||||||
- Ruff: passed
|
|
||||||
- `git diff --check`: passed
|
|
||||||
- final adversarial review: no findings
|
|
||||||
|
|
@ -1,40 +0,0 @@
|
||||||
---
|
|
||||||
model: gpt-5.6-sol
|
|
||||||
provider: openai
|
|
||||||
service: opencode
|
|
||||||
session: unavailable
|
|
||||||
timestamp: 2026-07-27T20:47:11Z
|
|
||||||
git_ref: 84a6d47b
|
|
||||||
scope: code
|
|
||||||
substantive: true
|
|
||||||
raw_file: 20260727T204711Z_84a6d47b_prompt_io.raw.md
|
|
||||||
---
|
|
||||||
|
|
||||||
## Prompt
|
|
||||||
|
|
||||||
The user identified that `backfiller_deep_fixes` introduced persistent
|
|
||||||
`.parquet.lock` files, questioned whether Parquet or Arrow already handled
|
|
||||||
writer coordination, and clarified that piker's `tractor` structured
|
|
||||||
concurrency design already provides single-writer ownership through the
|
|
||||||
persistent `datad` feed task. The user then asked to open the backfiller
|
|
||||||
worktree and resolve the redundant locking.
|
|
||||||
|
|
||||||
## Response summary
|
|
||||||
|
|
||||||
Removed redundant actor-local and filesystem locking while preserving
|
|
||||||
atomic crash-safe Parquet replacement. Replaced the synthetic contention
|
|
||||||
test with regressions for the actual cross-branch failure: writes create
|
|
||||||
no lock sidecars, and legacy lock artifacts are ignored during indexing.
|
|
||||||
|
|
||||||
## Files changed
|
|
||||||
|
|
||||||
- `piker/storage/nativedb.py` - rely on `datad` writer ownership
|
|
||||||
- `tests/test_storage_nativedb.py` - lock-sidecar regressions
|
|
||||||
- `ai/prompt-io/opencode/20260727T204711Z_84a6d47b_prompt_io.raw.md`
|
|
||||||
- unedited response record
|
|
||||||
- `ai/prompt-io/opencode/20260727T204711Z_84a6d47b_prompt_io.md`
|
|
||||||
- provenance metadata and response summary
|
|
||||||
|
|
||||||
## Human edits
|
|
||||||
|
|
||||||
None - generated changes have not been edited by the human.
|
|
||||||
|
|
@ -1,30 +0,0 @@
|
||||||
---
|
|
||||||
model: gpt-5.6-sol
|
|
||||||
provider: openai
|
|
||||||
service: opencode
|
|
||||||
timestamp: 2026-07-27T20:47:11Z
|
|
||||||
git_ref: 84a6d47b
|
|
||||||
diff_cmd: git diff HEAD~1..HEAD
|
|
||||||
---
|
|
||||||
|
|
||||||
> `git diff HEAD~1..HEAD -- piker/storage/nativedb.py`
|
|
||||||
|
|
||||||
Removed actor-local and filesystem writer locks from NativeDB. The
|
|
||||||
persistent `datad` feed task already owns writes for each series, while
|
|
||||||
its timeframe children write distinct files. NativeDB retains validated
|
|
||||||
temporary writes, file and directory synchronization, and atomic target
|
|
||||||
replacement for crash durability. The exact `.parquet` suffix filter is
|
|
||||||
retained so legacy lock sidecars are harmless.
|
|
||||||
|
|
||||||
> `git diff HEAD~1..HEAD -- tests/test_storage_nativedb.py`
|
|
||||||
|
|
||||||
Removed the synthetic cross-client lock-contention test. Added regression
|
|
||||||
coverage proving replacement and incremental writes create no lock
|
|
||||||
sidecars. Extended indexing coverage with the legacy
|
|
||||||
`.parquet.lock` artifact that caused upstream `flake_update` to crash.
|
|
||||||
|
|
||||||
Verification generated with the patch:
|
|
||||||
|
|
||||||
- NativeDB and history regression set: 20 passed
|
|
||||||
- Ruff: passed
|
|
||||||
- `git diff --check`: passed
|
|
||||||
|
|
@ -1,42 +0,0 @@
|
||||||
---
|
|
||||||
model: gpt-5.6-sol
|
|
||||||
provider: openai
|
|
||||||
service: opencode
|
|
||||||
session: ses_0799212ebffe42arY96czXn89F
|
|
||||||
timestamp: 2026-07-27T21:24:54Z
|
|
||||||
git_ref: 689df816
|
|
||||||
scope: code
|
|
||||||
substantive: true
|
|
||||||
raw_file: 20260727T212454Z_689df816_prompt_io.raw.md
|
|
||||||
---
|
|
||||||
|
|
||||||
## Prompt
|
|
||||||
|
|
||||||
The user reran the known IB FQME on `backfiller_deep_fixes` and reported
|
|
||||||
that `publish_latest_frame()` still failed because NativeDB required an
|
|
||||||
`index` absent from IB's provider frame. The user called out the missed
|
|
||||||
tabular schema boundary and requested an actual end-to-end test suite
|
|
||||||
rather than storage fakes.
|
|
||||||
|
|
||||||
## Response summary
|
|
||||||
|
|
||||||
Repaired the IB-provider-to-NativeDB boundary by canonicalizing durable
|
|
||||||
fields and mapping reloads into provider-specific SHM buffers. Added a
|
|
||||||
local actor/SHM/Parquet integration regression covering first startup and
|
|
||||||
restart, plus focused conversion and timestamp-coercion tests. Two review
|
|
||||||
rounds found and resolved restart hydration and fractional-time hazards.
|
|
||||||
|
|
||||||
## Files changed
|
|
||||||
|
|
||||||
- `piker/storage/nativedb.py` - canonical storage and SHM field mapping
|
|
||||||
- `piker/tsp/_anal.py` - name-based Polars-to-NumPy conversion
|
|
||||||
- `tests/test_history_backfill.py` - actor/SHM/Parquet integration test
|
|
||||||
- `tests/test_storage_nativedb.py` - schema conversion regressions
|
|
||||||
- `ai/prompt-io/opencode/20260727T212454Z_689df816_prompt_io.raw.md`
|
|
||||||
- unedited response record
|
|
||||||
- `ai/prompt-io/opencode/20260727T212454Z_689df816_prompt_io.md`
|
|
||||||
- provenance metadata and response summary
|
|
||||||
|
|
||||||
## Human edits
|
|
||||||
|
|
||||||
None - generated changes have not been edited by the human.
|
|
||||||
|
|
@ -1,34 +0,0 @@
|
||||||
---
|
|
||||||
model: gpt-5.6-sol
|
|
||||||
provider: openai
|
|
||||||
service: opencode
|
|
||||||
timestamp: 2026-07-27T21:24:54Z
|
|
||||||
git_ref: 689df816
|
|
||||||
diff_cmd: git diff HEAD~1..HEAD
|
|
||||||
---
|
|
||||||
|
|
||||||
> `git diff HEAD~1..HEAD -- piker/storage/nativedb.py piker/tsp/_anal.py`
|
|
||||||
|
|
||||||
Normalized index-less provider frames to NativeDB's canonical names,
|
|
||||||
order, and dtypes before merge and publication. Provider-only columns
|
|
||||||
are excluded from durable files, derived indexes are regenerated, and
|
|
||||||
fractional timestamps are rejected before integer conversion. Added a
|
|
||||||
canonical field map for hydrating provider-specific SHM buffers while
|
|
||||||
leaving provider-only fields at their defaults. Changed Polars-to-NumPy
|
|
||||||
conversion to select columns by field name rather than position.
|
|
||||||
|
|
||||||
> `git diff HEAD~1..HEAD -- tests/test_history_backfill.py tests/test_storage_nativedb.py`
|
|
||||||
|
|
||||||
Added deterministic coverage using IB's actual provider dtype and a real
|
|
||||||
local `tractor` root actor, IB-typed shared-memory buffers, NativeDB,
|
|
||||||
Parquet persistence, first-start publication, and restart hydration.
|
|
||||||
Added focused regressions for reordered/extra Polars columns and
|
|
||||||
fractional timestamp rejection.
|
|
||||||
|
|
||||||
Verification generated with the patch:
|
|
||||||
|
|
||||||
- NativeDB and history integration set: 23 passed
|
|
||||||
- Ruff: passed
|
|
||||||
- `git diff --check`: passed
|
|
||||||
- second adversarial review: no findings
|
|
||||||
- no live gateway or network tests run
|
|
||||||
|
|
@ -1,41 +0,0 @@
|
||||||
---
|
|
||||||
model: gpt-5.6-sol
|
|
||||||
provider: openai
|
|
||||||
service: opencode
|
|
||||||
session: ses_0799212ebffe42arY96czXn89F
|
|
||||||
timestamp: 2026-07-27T21:59:06Z
|
|
||||||
git_ref: 689df816
|
|
||||||
scope: code
|
|
||||||
substantive: true
|
|
||||||
raw_file: 20260727T215906Z_689df816_prompt_io.raw.md
|
|
||||||
---
|
|
||||||
|
|
||||||
## Prompt
|
|
||||||
|
|
||||||
After the actor/SHM/Parquet integration test exposed a stale SHM triplet,
|
|
||||||
the user requested leak-cleaner machinery in the pytest harness using
|
|
||||||
Tractor's existing ownership and lifetime patterns.
|
|
||||||
|
|
||||||
## Response summary
|
|
||||||
|
|
||||||
Added creator-scoped SHM leak tracking to pytest without scanning
|
|
||||||
`/dev/shm` or treating attachments as owned. The fixture cooperates with
|
|
||||||
normal Tractor teardown, verifies POSIX object identity before fallback
|
|
||||||
cleanup, restores process-local token state, and fails after cleaning a
|
|
||||||
leak. Regressions cover the exact pre-actor-registration allocation window,
|
|
||||||
positional creators, and external attachment safety.
|
|
||||||
|
|
||||||
## Files changed
|
|
||||||
|
|
||||||
- `tests/conftest.py` - creator-scoped SHM leak tracking fixture
|
|
||||||
- `tests/test_shm_cleanup.py` - ownership and failure-window regressions
|
|
||||||
- `.claude/skills/run-tests/test-harness-reference.md`
|
|
||||||
- document current-process SHM cleanup behavior
|
|
||||||
- `ai/prompt-io/opencode/20260727T215906Z_689df816_prompt_io.raw.md`
|
|
||||||
- unedited response record
|
|
||||||
- `ai/prompt-io/opencode/20260727T215906Z_689df816_prompt_io.md`
|
|
||||||
- provenance metadata and response summary
|
|
||||||
|
|
||||||
## Human edits
|
|
||||||
|
|
||||||
None - generated changes have not been edited by the human.
|
|
||||||
|
|
@ -1,25 +0,0 @@
|
||||||
---
|
|
||||||
model: gpt-5.6-sol
|
|
||||||
provider: openai
|
|
||||||
service: opencode
|
|
||||||
timestamp: 2026-07-27T21:59:06Z
|
|
||||||
git_ref: 689df816
|
|
||||||
diff_cmd: git diff HEAD~1..HEAD
|
|
||||||
---
|
|
||||||
|
|
||||||
> `git diff HEAD~1..HEAD -- tests/conftest.py tests/test_shm_cleanup.py .claude/skills/run-tests/test-harness-reference.md`
|
|
||||||
|
|
||||||
Added function-scoped SHM ownership tracking around Tractor's test-process
|
|
||||||
factory. The fixture records only successful creators, removes ownership
|
|
||||||
records during normal Tractor teardown, verifies POSIX object identity
|
|
||||||
before fallback unlink, restores the token-cache baseline, and fails after
|
|
||||||
cleaning any leak. Regressions reproduce allocation failure before actor
|
|
||||||
lifetime registration and prove external attachments remain untouched.
|
|
||||||
|
|
||||||
Verification generated with the patch:
|
|
||||||
|
|
||||||
- SHM cleanup regressions: 2 passed
|
|
||||||
- combined SHM, NativeDB, and history set: 25 passed
|
|
||||||
- Ruff: passed
|
|
||||||
- `git diff --check`: passed
|
|
||||||
- final adversarial review: no findings
|
|
||||||
|
|
@ -1,42 +0,0 @@
|
||||||
---
|
|
||||||
model: gpt-5.6-sol
|
|
||||||
provider: openai
|
|
||||||
service: opencode
|
|
||||||
session: 24fb9765-a550-4570-8350-f0fc9b7e17db
|
|
||||||
timestamp: 2026-07-28T00:32:47Z
|
|
||||||
git_ref: ed85721c
|
|
||||||
scope: code
|
|
||||||
substantive: true
|
|
||||||
raw_file: 20260728T003247Z_ed85721c_prompt_io.raw.md
|
|
||||||
---
|
|
||||||
|
|
||||||
## Prompt
|
|
||||||
|
|
||||||
Build a safe xonsh workflow for qualifying NativeDB and reverse-backfill
|
|
||||||
correctness against a known gappy IB chart, beginning with
|
|
||||||
`mnq.cme.20260918`. Cover baseline, fresh, restart, append, and expansion
|
|
||||||
testing through implementation, adversarial review, and verification.
|
|
||||||
|
|
||||||
## Response summary
|
|
||||||
|
|
||||||
Implemented a private disposable xonsh qualification workflow. Evidence is
|
|
||||||
tied to a fixed case, report checksum, snapshot checksum, worktree import,
|
|
||||||
and phase metadata. Added explicit path and symlink safety, source
|
|
||||||
replacement handling, xonsh child environments, and lifecycle preservation
|
|
||||||
comparisons.
|
|
||||||
|
|
||||||
## Files changed
|
|
||||||
|
|
||||||
- `snippets/nativedb_backfill_audit.xsh` - disposable qualification helpers
|
|
||||||
- `docs/manual_backfill_qualification.rst` - operator lifecycle runbook
|
|
||||||
- `tests/test_backfill_audit_snippet.py` - xonsh and comparison regressions
|
|
||||||
- `.claude/skills/run-tests/test-harness-reference.md`
|
|
||||||
- qualification test-harness guidance
|
|
||||||
- `ai/prompt-io/opencode/20260728T003247Z_ed85721c_prompt_io.raw.md`
|
|
||||||
- unedited response record
|
|
||||||
- `ai/prompt-io/opencode/20260728T003247Z_ed85721c_prompt_io.md`
|
|
||||||
- provenance metadata and response summary
|
|
||||||
|
|
||||||
## Human edits
|
|
||||||
|
|
||||||
None - generated changes have not been edited by the human.
|
|
||||||
|
|
@ -1,32 +0,0 @@
|
||||||
---
|
|
||||||
model: gpt-5.6-sol
|
|
||||||
provider: openai
|
|
||||||
service: opencode
|
|
||||||
timestamp: 2026-07-28T00:32:47Z
|
|
||||||
git_ref: ed85721c
|
|
||||||
diff_cmd: git diff HEAD~1..HEAD
|
|
||||||
---
|
|
||||||
|
|
||||||
> `git diff HEAD~1..HEAD -- snippets/nativedb_backfill_audit.xsh docs/manual_backfill_qualification.rst tests/test_backfill_audit_snippet.py .claude/skills/run-tests/test-harness-reference.md`
|
|
||||||
|
|
||||||
Added sourceable xonsh helpers that create a private disposable root, bind
|
|
||||||
child processes to the reviewed worktree, capture checksum-bound phase
|
|
||||||
evidence, safely seed and clear test history, and compare lifecycle
|
|
||||||
preservation.
|
|
||||||
|
|
||||||
The workflow rejects path traversal, pre-existing symlinks, output
|
|
||||||
collisions, mismatched case/report/snapshot evidence, and same-phase
|
|
||||||
comparison. The accompanying runbook covers known-bad replay, fresh
|
|
||||||
backfill, restart, append, gap triage, and expansion cases.
|
|
||||||
|
|
||||||
Verification generated with the patch:
|
|
||||||
|
|
||||||
- xonsh qualification helper regressions: passed
|
|
||||||
- xonsh sourceability: passed
|
|
||||||
- Ruff: passed
|
|
||||||
- `git diff --check`: passed
|
|
||||||
|
|
||||||
Adversarial review drove additional evidence binding, worktree executable,
|
|
||||||
environment propagation, source replacement, and descriptor-level safety
|
|
||||||
fixes. Residual concurrent mutation risk is stated explicitly in the private
|
|
||||||
disposable-root runbook.
|
|
||||||
|
|
@ -1,40 +0,0 @@
|
||||||
---
|
|
||||||
model: gpt-5.6-sol
|
|
||||||
provider: openai
|
|
||||||
service: opencode
|
|
||||||
session: 24fb9765-a550-4570-8350-f0fc9b7e17db
|
|
||||||
timestamp: 2026-07-28T00:50:34Z
|
|
||||||
git_ref: ed85721c
|
|
||||||
scope: code
|
|
||||||
substantive: true
|
|
||||||
raw_file: 20260728T005034Z_ed85721c_prompt_io.raw.md
|
|
||||||
---
|
|
||||||
|
|
||||||
## Prompt
|
|
||||||
|
|
||||||
Add reusable read-only tooling to qualify raw NativeDB Parquet against a
|
|
||||||
known gappy IB chart, beginning with `mnq.cme.20260918`. Preserve malformed
|
|
||||||
evidence and distinguish structural validity from unclassified positive
|
|
||||||
gaps without opening or mutating the storage runtime.
|
|
||||||
|
|
||||||
## Response summary
|
|
||||||
|
|
||||||
Implemented raw Parquet audit reporting with human and JSON output, bounded
|
|
||||||
cadence details, exact integer arithmetic, strict status, collision-safe
|
|
||||||
snapshot capture, and source path protections. Regressions cover malformed
|
|
||||||
schema and values, timestamp precision, gap arithmetic, corrupt bytes,
|
|
||||||
copy failures, symlinks, traversal, and read-only CLI behavior.
|
|
||||||
|
|
||||||
## Files changed
|
|
||||||
|
|
||||||
- `piker/storage/_audit.py` - raw Parquet audit and snapshot implementation
|
|
||||||
- `piker/storage/cli.py` - `piker store audit` command
|
|
||||||
- `tests/test_storage_audit.py` - audit and CLI regressions
|
|
||||||
- `ai/prompt-io/opencode/20260728T005034Z_ed85721c_prompt_io.raw.md`
|
|
||||||
- unedited response record
|
|
||||||
- `ai/prompt-io/opencode/20260728T005034Z_ed85721c_prompt_io.md`
|
|
||||||
- provenance metadata and response summary
|
|
||||||
|
|
||||||
## Human edits
|
|
||||||
|
|
||||||
None - generated changes have not been edited by the human.
|
|
||||||
|
|
@ -1,27 +0,0 @@
|
||||||
---
|
|
||||||
model: gpt-5.6-sol
|
|
||||||
provider: openai
|
|
||||||
service: opencode
|
|
||||||
timestamp: 2026-07-28T00:50:34Z
|
|
||||||
git_ref: ed85721c
|
|
||||||
diff_cmd: git diff HEAD~1..HEAD
|
|
||||||
---
|
|
||||||
|
|
||||||
> `git diff HEAD~1..HEAD -- piker/storage/_audit.py piker/storage/cli.py tests/test_storage_audit.py`
|
|
||||||
|
|
||||||
Added a raw, read-only NativeDB Parquet audit with human and JSON output,
|
|
||||||
bounded cadence evidence, exact integer arithmetic, pre-parse snapshot
|
|
||||||
capture, and strict qualification status.
|
|
||||||
|
|
||||||
The command rejects unsafe FQME paths, symlinked sources and outputs,
|
|
||||||
destination collisions, non-regular source nodes, malformed canonical
|
|
||||||
schema, non-finite values, invalid timestamps, noncanonical indexes, and
|
|
||||||
sub-period cadence. Copy-failure cleanup preserves source bytes while
|
|
||||||
completed corrupt snapshots remain available for diagnosis.
|
|
||||||
|
|
||||||
Verification generated with the patch:
|
|
||||||
|
|
||||||
- storage audit and CLI regressions: passed
|
|
||||||
- Ruff: passed
|
|
||||||
- `git diff --check`: passed
|
|
||||||
- final adversarial review: no findings
|
|
||||||
|
|
@ -1,39 +0,0 @@
|
||||||
---
|
|
||||||
model: gpt-5.6-sol
|
|
||||||
provider: openai
|
|
||||||
service: opencode
|
|
||||||
session: 24fb9765-a550-4570-8350-f0fc9b7e17db
|
|
||||||
timestamp: 2026-07-28T23:27:18Z
|
|
||||||
git_ref: ed85721c
|
|
||||||
scope: code
|
|
||||||
substantive: true
|
|
||||||
raw_file: 20260728T232718Z_ed85721c_prompt_io.raw.md
|
|
||||||
---
|
|
||||||
|
|
||||||
## Prompt
|
|
||||||
|
|
||||||
Diagnose the live `pikerd` failure where backfill publication raised
|
|
||||||
`ValueError: OHLCV timestamps must be finite and positive` while merging the
|
|
||||||
known gappy MNQ NativeDB baseline with fresh IB history. Determine whether to
|
|
||||||
commit the pending audit stack or iterate first.
|
|
||||||
|
|
||||||
## Response summary
|
|
||||||
|
|
||||||
Traced the primary failure through `publish_latest_frame()` to
|
|
||||||
`NativeStorageClient.update_ohlcv()`. The incoming frame was valid, but the
|
|
||||||
canonicalized persisted baseline retained its epoch-zero row and caused the
|
|
||||||
merged frame to fail validation. Added persisted-row repair before merge and
|
|
||||||
a regression using the observed legacy schema characteristics.
|
|
||||||
|
|
||||||
## Files changed
|
|
||||||
|
|
||||||
- `piker/storage/nativedb.py` - drop non-positive persisted rows during merge
|
|
||||||
- `tests/test_storage_nativedb.py` - live MNQ failure-shape regression
|
|
||||||
- `ai/prompt-io/opencode/20260728T232718Z_ed85721c_prompt_io.raw.md`
|
|
||||||
- unedited response record
|
|
||||||
- `ai/prompt-io/opencode/20260728T232718Z_ed85721c_prompt_io.md`
|
|
||||||
- provenance metadata and response summary
|
|
||||||
|
|
||||||
## Human edits
|
|
||||||
|
|
||||||
None - generated changes have not been edited by the human.
|
|
||||||
|
|
@ -1,27 +0,0 @@
|
||||||
---
|
|
||||||
model: gpt-5.6-sol
|
|
||||||
provider: openai
|
|
||||||
service: opencode
|
|
||||||
timestamp: 2026-07-28T23:27:18Z
|
|
||||||
git_ref: ed85721c
|
|
||||||
diff_cmd: git diff HEAD~1..HEAD
|
|
||||||
---
|
|
||||||
|
|
||||||
> `git diff HEAD~1..HEAD -- piker/storage/nativedb.py tests/test_storage_nativedb.py`
|
|
||||||
|
|
||||||
Fixed incremental NativeDB updates against legacy persisted history with
|
|
||||||
non-positive timestamps. Incoming provider frames remain strictly validated,
|
|
||||||
while merge repair discards invalid persisted rows before concat, dedupe,
|
|
||||||
ordering, canonical-index rewrite, and atomic publication.
|
|
||||||
|
|
||||||
The regression reproduces the observed MNQ baseline shape: one epoch-zero
|
|
||||||
row, noncanonical absolute indexes, extra derived columns, and a valid fresh
|
|
||||||
IB frame. It proves the invalid row is removed while all valid old and new
|
|
||||||
bars survive with canonical indexes.
|
|
||||||
|
|
||||||
Verification generated with the patch:
|
|
||||||
|
|
||||||
- NativeDB regressions: 18 passed
|
|
||||||
- history backfill regressions: 6 passed
|
|
||||||
- focused Ruff: passed
|
|
||||||
- `git diff --check`: passed
|
|
||||||
|
|
@ -1,40 +0,0 @@
|
||||||
---
|
|
||||||
model: gpt-5.6-sol
|
|
||||||
provider: openai
|
|
||||||
service: opencode
|
|
||||||
session: 24fb9765-a550-4570-8350-f0fc9b7e17db
|
|
||||||
timestamp: 2026-07-29T03:41:06Z
|
|
||||||
git_ref: 922a5df8
|
|
||||||
scope: code
|
|
||||||
substantive: true
|
|
||||||
raw_file: 20260729T034106Z_922a5df8_prompt_io.raw.md
|
|
||||||
---
|
|
||||||
|
|
||||||
## Prompt
|
|
||||||
|
|
||||||
Diagnose the live IB reverse-fill failure where
|
|
||||||
`reqHistoricalDataAsync()` raised API error 10314 because its end date,
|
|
||||||
time, or timezone was invalid.
|
|
||||||
|
|
||||||
## Response summary
|
|
||||||
|
|
||||||
Traced the failure to `Client.bars()`, where an unfinished EST conversion
|
|
||||||
left the original datetime for `ib_async` to serialize as
|
|
||||||
`YYYYMMDD HH:MM:SS UTC`. Converted aware boundaries explicitly to UTC and
|
|
||||||
passed IB's accepted `YYYYMMDD-HH:MM:SS` string while retaining a blank
|
|
||||||
latest-request boundary. Added exact request-argument regressions.
|
|
||||||
|
|
||||||
## Files changed
|
|
||||||
|
|
||||||
- `piker/brokers/ib/api.py` - explicit UTC-dash history boundary format
|
|
||||||
- `tests/test_ib_history.py` - IB request-format regressions
|
|
||||||
- `.claude/skills/run-tests/test-harness-reference.md`
|
|
||||||
- deterministic IB history test mapping
|
|
||||||
- `ai/prompt-io/opencode/20260729T034106Z_922a5df8_prompt_io.raw.md`
|
|
||||||
- unedited response record
|
|
||||||
- `ai/prompt-io/opencode/20260729T034106Z_922a5df8_prompt_io.md`
|
|
||||||
- provenance metadata and response summary
|
|
||||||
|
|
||||||
## Human edits
|
|
||||||
|
|
||||||
None - generated changes have not been edited by the human.
|
|
||||||
|
|
@ -1,25 +0,0 @@
|
||||||
---
|
|
||||||
model: gpt-5.6-sol
|
|
||||||
provider: openai
|
|
||||||
service: opencode
|
|
||||||
timestamp: 2026-07-29T03:41:06Z
|
|
||||||
git_ref: 922a5df8
|
|
||||||
diff_cmd: git diff HEAD~1..HEAD
|
|
||||||
---
|
|
||||||
|
|
||||||
> `git diff HEAD~1..HEAD -- piker/brokers/ib/api.py tests/test_ib_history.py .claude/skills/run-tests/test-harness-reference.md`
|
|
||||||
|
|
||||||
Fixed IB reverse-history requests to serialize aware end boundaries with
|
|
||||||
IB's explicit UTC-dash syntax, `YYYYMMDD-HH:MM:SS`. This bypasses
|
|
||||||
`ib_async`'s trailing-`UTC` representation, which the live Gateway rejected
|
|
||||||
with API error 10314, and removes the unfinished EST comparison breakpoint.
|
|
||||||
|
|
||||||
The regression captures the exact `reqHistoricalDataAsync()` argument for
|
|
||||||
latest, stdlib-aware, Pendulum UTC, and America/New_York inputs.
|
|
||||||
|
|
||||||
Verification generated with the patch:
|
|
||||||
|
|
||||||
- IB history request regressions: 4 passed
|
|
||||||
- focused Ruff: passed
|
|
||||||
- `git diff --check`: passed
|
|
||||||
- adversarial review: no findings
|
|
||||||
|
|
@ -1,40 +0,0 @@
|
||||||
---
|
|
||||||
model: gpt-5.6-sol
|
|
||||||
provider: openai
|
|
||||||
service: opencode
|
|
||||||
session: 24fb9765-a550-4570-8350-f0fc9b7e17db
|
|
||||||
timestamp: 2026-07-29T04:17:23Z
|
|
||||||
git_ref: ce33deb6
|
|
||||||
scope: code
|
|
||||||
substantive: true
|
|
||||||
raw_file: 20260729T041723Z_ce33deb6_prompt_io.raw.md
|
|
||||||
---
|
|
||||||
|
|
||||||
## Prompt
|
|
||||||
|
|
||||||
Diagnose why the chart still showed a zero first OHLCV row after the
|
|
||||||
epoch-zero timestamp repair, causing auto-y-ranging to include zero.
|
|
||||||
|
|
||||||
## Response summary
|
|
||||||
|
|
||||||
Audited live 60s and 1s NativeDB files and found one exact 60s first-slot
|
|
||||||
sentinel at epoch 60 with a completely zero OHLCV payload. Added narrowly
|
|
||||||
scoped read and merge repair, matching incoming validation, sentinel-only
|
|
||||||
startup handling, canonical reindexing, and explicit audit classification.
|
|
||||||
Preserved modern all-zero bars as unclassified evidence rather than assuming
|
|
||||||
all zero-priced instruments are corrupt.
|
|
||||||
|
|
||||||
## Files changed
|
|
||||||
|
|
||||||
- `piker/storage/nativedb.py` - first-slot sentinel repair and validation
|
|
||||||
- `piker/storage/_audit.py` - sentinel classification and zero-row evidence
|
|
||||||
- `tests/test_storage_nativedb.py` - hydration, merge, and boundary regressions
|
|
||||||
- `tests/test_storage_audit.py` - sentinel and modern-zero audit regressions
|
|
||||||
- `ai/prompt-io/opencode/20260729T041723Z_ce33deb6_prompt_io.raw.md`
|
|
||||||
- unedited response record
|
|
||||||
- `ai/prompt-io/opencode/20260729T041723Z_ce33deb6_prompt_io.md`
|
|
||||||
- provenance metadata and response summary
|
|
||||||
|
|
||||||
## Human edits
|
|
||||||
|
|
||||||
None - generated changes have not been edited by the human.
|
|
||||||
|
|
@ -1,31 +0,0 @@
|
||||||
---
|
|
||||||
model: gpt-5.6-sol
|
|
||||||
provider: openai
|
|
||||||
service: opencode
|
|
||||||
timestamp: 2026-07-29T04:17:23Z
|
|
||||||
git_ref: ce33deb6
|
|
||||||
diff_cmd: git diff HEAD~1..HEAD
|
|
||||||
---
|
|
||||||
|
|
||||||
> `git diff HEAD~1..HEAD -- piker/storage/nativedb.py piker/storage/_audit.py tests/test_storage_nativedb.py tests/test_storage_audit.py`
|
|
||||||
|
|
||||||
Fixed the remaining legacy first-slot sentinel observed in live MNQ 60s
|
|
||||||
history: `time=60` with zero open, high, low, close, and volume. NativeDB now
|
|
||||||
repairs the exact sentinel before read hydration and merge, reindexes the
|
|
||||||
filtered view, and persists the repaired frame on the next valid update.
|
|
||||||
|
|
||||||
The repair remains narrowly scoped. Modern all-zero OHLC bars stay valid for
|
|
||||||
spreads or synthetic instruments, null-containing malformed rows still fail
|
|
||||||
validation, new writes can not recreate the first-slot sentinel, and a file
|
|
||||||
containing only sentinels loads as no history so fresh backfill can proceed.
|
|
||||||
|
|
||||||
The raw audit reports total all-zero price rows, classifies first-slot
|
|
||||||
sentinels as structural violations, and leaves later all-zero rows visible as
|
|
||||||
unclassified warnings.
|
|
||||||
|
|
||||||
Verification generated with the patch:
|
|
||||||
|
|
||||||
- NativeDB, audit, history, xonsh, and IB regressions: 61 passed
|
|
||||||
- focused Ruff: passed
|
|
||||||
- `git diff --check`: passed
|
|
||||||
- final adversarial review: no findings
|
|
||||||
|
|
@ -1,34 +0,0 @@
|
||||||
---
|
|
||||||
model: openai/gpt-5.6-sol
|
|
||||||
service: opencode
|
|
||||||
session: 6409e3bb-1a9a-4ada-b6c4-c554115143e9
|
|
||||||
timestamp: 2026-07-29T18:54:06Z
|
|
||||||
git_ref: d359b1cb
|
|
||||||
scope: code
|
|
||||||
substantive: true
|
|
||||||
raw_file: 20260729T185406Z_d359b1cb_prompt_io.raw.md
|
|
||||||
---
|
|
||||||
|
|
||||||
## Prompt
|
|
||||||
|
|
||||||
The user reported that QQQ 60-second zeros appeared as a flat line after
|
|
||||||
reload, clarified that the outstanding concern was the prior `ldshm`
|
|
||||||
failure, and instructed the agent to implement the more focused fix.
|
|
||||||
|
|
||||||
## Response summary
|
|
||||||
|
|
||||||
Changed `ldshm` to classify invalid SHM snapshots before downstream
|
|
||||||
analysis. Cadence is inferred only from observed finite positive steps;
|
|
||||||
buffers containing non-finite or non-positive timestamps, or lacking a
|
|
||||||
positive cadence, are skipped rather than partially replacement-written
|
|
||||||
or reloaded. Added deterministic helper and command-level regressions.
|
|
||||||
|
|
||||||
## Files changed
|
|
||||||
|
|
||||||
- `piker/storage/cli.py` - inspect cadence and reject invalid SHM snapshots before mutation.
|
|
||||||
- `tests/test_ldshm.py` - cover QQQ-like zeros, corrupt timestamps, cadence selection, degenerate buffers, and mutation prevention.
|
|
||||||
- `.claude/skills/run-tests/test-harness-reference.md` - register the deterministic regression target.
|
|
||||||
|
|
||||||
## Human edits
|
|
||||||
|
|
||||||
None - generated output remains uncommitted.
|
|
||||||
|
|
@ -1,58 +0,0 @@
|
||||||
---
|
|
||||||
model: openai/gpt-5.6-sol
|
|
||||||
service: opencode
|
|
||||||
timestamp: 2026-07-29T18:54:06Z
|
|
||||||
git_ref: d359b1cb
|
|
||||||
diff_cmd: git diff HEAD~1..HEAD
|
|
||||||
---
|
|
||||||
|
|
||||||
The user clarified that the QQQ 60-second chart now displayed a flat
|
|
||||||
forward-filled segment, then asked whether the previously reported
|
|
||||||
`piker store ldshm` failure had been fixed. After being told it had not,
|
|
||||||
the user instructed: "ok but i read above you have a more focussed fix
|
|
||||||
todo now? so have at it!"
|
|
||||||
|
|
||||||
The implementation focused on the exact failure boundary. It does not
|
|
||||||
filter invalid rows into a partial replacement and does not mutate live
|
|
||||||
SHM. Instead, `ldshm` inspects one captured Polars snapshot, infers its
|
|
||||||
cadence from observed finite positive timestamp steps, and skips the
|
|
||||||
entire buffer before analysis, persistence, or reload when any timestamp
|
|
||||||
is non-finite or non-positive.
|
|
||||||
|
|
||||||
> `git diff HEAD~1..HEAD -- piker/storage/cli.py`
|
|
||||||
|
|
||||||
Generated `_shm_period_and_invalid_count()` and integrated its result in
|
|
||||||
`ldshm`. The helper selects the most frequent observed positive step,
|
|
||||||
using the smallest observed step as the deterministic tie-break. Short
|
|
||||||
or cadence-degenerate buffers are skipped. Invalid snapshots are logged
|
|
||||||
and cannot reach deduplication, NativeDB replacement, or SHM reload.
|
|
||||||
|
|
||||||
> `git diff HEAD~1..HEAD -- tests/test_ldshm.py`
|
|
||||||
|
|
||||||
Generated deterministic regressions for the QQQ-shaped interior zero
|
|
||||||
run, observed-step cadence selection, all invalid timestamp classes,
|
|
||||||
short and degenerate buffers, and command-level prevention of dedupe,
|
|
||||||
storage writes, and reload. The command-level test uses an immutable
|
|
||||||
Polars snapshot and failing spies, without live SHM mutation or timing.
|
|
||||||
|
|
||||||
> `git diff HEAD~1..HEAD -- .claude/skills/run-tests/test-harness-reference.md`
|
|
||||||
|
|
||||||
Added the deterministic `ldshm` regression target and its change-to-test
|
|
||||||
mapping to the repository test harness reference.
|
|
||||||
|
|
||||||
Verification output:
|
|
||||||
|
|
||||||
```text
|
|
||||||
.......................................................... [100%]
|
|
||||||
58 passed in 0.70s
|
|
||||||
```
|
|
||||||
|
|
||||||
`git diff --check` passed. Ruff was unavailable in the worktree's
|
|
||||||
existing environment. Three adversarial review passes found and drove
|
|
||||||
fixes for partial-replacement data loss, invented median timeframes,
|
|
||||||
mixed SHM observations, and pre-guard null-analysis failures. The final
|
|
||||||
review reported no remaining findings.
|
|
||||||
|
|
||||||
The patch intentionally does not claim that `ldshm` is safe to run as a
|
|
||||||
general concurrent repair writer. It handles invalid live snapshots by
|
|
||||||
refusing mutation rather than weakening NativeDB validation.
|
|
||||||
|
|
@ -1,37 +0,0 @@
|
||||||
---
|
|
||||||
model: openai/gpt-5.6-sol
|
|
||||||
service: opencode
|
|
||||||
session: 253ade33-07ba-45b5-8839-be35d17ec164
|
|
||||||
timestamp: 2026-07-29T20:40:46Z
|
|
||||||
git_ref: ad6c560f
|
|
||||||
scope: code
|
|
||||||
substantive: true
|
|
||||||
raw_file: 20260729T204046Z_ad6c560f_prompt_io.raw.md
|
|
||||||
---
|
|
||||||
|
|
||||||
## Prompt
|
|
||||||
|
|
||||||
The user reported that the QQQ 1-second chart still contained a gap after
|
|
||||||
the `ldshm` fix, asked the agent to inspect it, and authorized continued
|
|
||||||
implementation after a short pause and shutdown of the live workspace.
|
|
||||||
|
|
||||||
## Response summary
|
|
||||||
|
|
||||||
Diagnosed a 653-row active RT SHM null reservation caused by inclusive IB
|
|
||||||
reverse-query overlaps and startup repair orchestration. Prevented extra
|
|
||||||
physical SHM overlap rows, serialized and bounded IB history recovery,
|
|
||||||
made blank/error responses terminate, and constrained synthetic repair to
|
|
||||||
valid interior timestamp boundaries. Added deterministic concurrency and
|
|
||||||
data-layout regressions.
|
|
||||||
|
|
||||||
## Files changed
|
|
||||||
|
|
||||||
- `piker/brokers/ib/feed.py` - bound and serialize IB history reset/retry behavior.
|
|
||||||
- `piker/tsp/_history.py` - prevent SHM overlap skew and safely finalize null repair.
|
|
||||||
- `tests/test_ib_history.py` - cover blank, cancellation, reset, and pacing interleavings.
|
|
||||||
- `tests/test_history_backfill.py` - cover overlap and null-repair invariants.
|
|
||||||
- `.claude/skills/run-tests/test-harness-reference.md` - register deterministic coverage.
|
|
||||||
|
|
||||||
## Human edits
|
|
||||||
|
|
||||||
None - generated output remains uncommitted.
|
|
||||||
|
|
@ -1,72 +0,0 @@
|
||||||
---
|
|
||||||
model: openai/gpt-5.6-sol
|
|
||||||
service: opencode
|
|
||||||
timestamp: 2026-07-29T20:40:46Z
|
|
||||||
git_ref: ad6c560f
|
|
||||||
diff_cmd: git diff HEAD~1..HEAD
|
|
||||||
---
|
|
||||||
|
|
||||||
The user reported that QQQ still displayed a gap on the 1-second chart
|
|
||||||
after the focused `ldshm` fix and asked the agent to inspect it. The user
|
|
||||||
then paused and resumed implementation while shutting down the live piker
|
|
||||||
trading workspace.
|
|
||||||
|
|
||||||
Read-only live inspection found the active datad generation held 653
|
|
||||||
contiguous zero-time rows in QQQ's 1-second RT SHM, while durable Parquet
|
|
||||||
contained no zero rows. The run occupied absolute indexes 168148 through
|
|
||||||
168800 between timestamps 1785348398 and 1785349051. The stale actor
|
|
||||||
generation and active 60-second history were separately identified.
|
|
||||||
|
|
||||||
> `git diff HEAD~1..HEAD -- piker/tsp/_history.py`
|
|
||||||
|
|
||||||
Generated history changes that exclude each inclusive reverse-query
|
|
||||||
endpoint already present in SHM while retaining the full provider frame
|
|
||||||
for NativeDB merge. Startup null repair now runs after reverse backfill,
|
|
||||||
uses bounded UI notifications, handles provider no-data without an
|
|
||||||
interactive pause, fills only exact interior zero groups, and refuses
|
|
||||||
synthetic timestamps unless both valid boundaries establish the expected
|
|
||||||
cadence.
|
|
||||||
|
|
||||||
> `git diff HEAD~1..HEAD -- piker/brokers/ib/feed.py`
|
|
||||||
|
|
||||||
Generated IB history reset changes that make blank responses complete,
|
|
||||||
bound repeated cancellation and reset failures per request, serialize
|
|
||||||
timeout and pacing farm resets with one actor-global Trio lock, carry
|
|
||||||
explicit reset results, and coordinate pacing handoff with queued timeout
|
|
||||||
waiters.
|
|
||||||
|
|
||||||
> `git diff HEAD~1..HEAD -- tests/test_history_backfill.py`
|
|
||||||
|
|
||||||
Generated regressions for inclusive endpoint exclusion, full NativeDB
|
|
||||||
provider deltas, strict synthetic gap boundaries, empty-provider local
|
|
||||||
fallback, bounded notifications, and preservation of valid margin rows.
|
|
||||||
|
|
||||||
> `git diff HEAD~1..HEAD -- tests/test_ib_history.py`
|
|
||||||
|
|
||||||
Generated deterministic regressions for blank IB responses, cancellation
|
|
||||||
limits, stalled reset cancellation, pacing reset failure limits,
|
|
||||||
cross-request reset serialization, timeout-to-pacing handoff, and
|
|
||||||
successful handoff timeout restart.
|
|
||||||
|
|
||||||
> `git diff HEAD~1..HEAD -- .claude/skills/run-tests/test-harness-reference.md`
|
|
||||||
|
|
||||||
Registered the deterministic history-backfill target and expanded the IB
|
|
||||||
history mapping to include `feed.py`.
|
|
||||||
|
|
||||||
Verification output:
|
|
||||||
|
|
||||||
```text
|
|
||||||
........................................................................ [ 93%]
|
|
||||||
..... [100%]
|
|
||||||
77 passed in 2.12s
|
|
||||||
```
|
|
||||||
|
|
||||||
`git diff --check` passed. Multiple adversarial review rounds identified
|
|
||||||
and drove fixes for unbounded IB empty/cancellation/reset paths, stale
|
|
||||||
reset ownership, actor-global failure poisoning, concurrent farm resets,
|
|
||||||
double-counted pacing handoffs, physical endpoint overlap, and unsafe
|
|
||||||
synthetic timestamp boundaries. Final review reported no findings.
|
|
||||||
|
|
||||||
Live validation was intentionally deferred because the user shut down the
|
|
||||||
trading workspace. A fresh datad/chart startup is required to regenerate
|
|
||||||
SHM and verify that QQQ no longer contains the 653-row null run.
|
|
||||||
|
|
@ -1,41 +0,0 @@
|
||||||
---
|
|
||||||
model: openai/gpt-5.6-sol
|
|
||||||
service: opencode
|
|
||||||
session: ses_0799212ebffe42arY96czXn89F
|
|
||||||
timestamp: 2026-07-31T16:52:15Z
|
|
||||||
git_ref: 0846cbd4
|
|
||||||
scope: code
|
|
||||||
substantive: true
|
|
||||||
raw_file: 20260731T165215Z_0846cbd4_prompt_io.raw.md
|
|
||||||
---
|
|
||||||
|
|
||||||
## Prompt
|
|
||||||
|
|
||||||
Evaluate and update `piker store` so bare groups and endpoints immediately
|
|
||||||
show help; prepare for eventual optional fuzzy selection in `anal` and
|
|
||||||
`ldshm`; aggregate recent time-series diagnostics into CLI endpoints. The
|
|
||||||
user selected the safe-foundation scope.
|
|
||||||
|
|
||||||
## Response summary
|
|
||||||
|
|
||||||
Added immediate help UX, exact NativeDB and SHM identities, read-only
|
|
||||||
`series` and `shm` diagnostics, selector-ready exact SHM names, and robust
|
|
||||||
discovery/attachment. Fixed related `ldshm` option and no-match failures.
|
|
||||||
Deferred optional `fzf` and mutation-default changes.
|
|
||||||
|
|
||||||
## Files changed
|
|
||||||
|
|
||||||
- `piker/storage/cli.py` - help UX and read-only series/SHM diagnostics.
|
|
||||||
- `piker/storage/nativedb.py` - canonical compound series identities.
|
|
||||||
- `piker/tsp/_history.py` - exact SHM discovery and dtype resolution.
|
|
||||||
- `piker/data/_sharedmem.py` - attach-only existing-segment helper.
|
|
||||||
- `piker/tsp/__init__.py` - SHM identity helper exports.
|
|
||||||
- `tests/test_store_cli.py` - command UX and endpoint regressions.
|
|
||||||
- `tests/test_ldshm.py` - SHM diagnostics and operational regressions.
|
|
||||||
- `tests/test_storage_nativedb.py` - series parsing/index regressions.
|
|
||||||
- `docs/manual_backfill_qualification.rst` - triage command documentation.
|
|
||||||
- `.claude/skills/run-tests/test-harness-reference.md` - test mapping.
|
|
||||||
|
|
||||||
## Human edits
|
|
||||||
|
|
||||||
None - generated output remains uncommitted.
|
|
||||||
|
|
@ -1,93 +0,0 @@
|
||||||
---
|
|
||||||
model: openai/gpt-5.6-sol
|
|
||||||
service: opencode
|
|
||||||
timestamp: 2026-07-31T16:52:15Z
|
|
||||||
git_ref: 0846cbd4
|
|
||||||
diff_cmd: git diff HEAD~1..HEAD
|
|
||||||
---
|
|
||||||
|
|
||||||
The user requested an evaluation and possible update of `piker store`,
|
|
||||||
including immediate help output when a group or endpoint is invoked with
|
|
||||||
no arguments. The user also proposed eventual optional `fzf` selection for
|
|
||||||
`anal` and `ldshm`, and authorized aggregating the recent time-series and
|
|
||||||
SHM debugging work into CLI endpoints. After reviewing three scopes, the
|
|
||||||
user selected the recommended safe foundation rather than implementing
|
|
||||||
interactive fuzzy selection or changing mutation defaults immediately.
|
|
||||||
|
|
||||||
> `git diff HEAD~1..HEAD -- piker/storage/cli.py`
|
|
||||||
|
|
||||||
Generated no-argument help behavior for the `store` group and every
|
|
||||||
endpoint. Added read-only `series` and `shm` commands, JSON and Rich output,
|
|
||||||
exact `--shm-name` plumbing, SHM cadence/invalid/order/gap summaries, and
|
|
||||||
safe corrupt-epoch formatting. Fixed existing `ldshm` no-write/no-reload
|
|
||||||
control flow and no-match debugger behavior.
|
|
||||||
|
|
||||||
> `git diff HEAD~1..HEAD -- piker/storage/nativedb.py`
|
|
||||||
|
|
||||||
Generated canonical `NativeSeriesRef` parsing and deterministic listing.
|
|
||||||
Changed the internal index to preserve `(fqme, period)` identities while
|
|
||||||
keeping unique-FQME compatibility listing and cardinality. Hardened
|
|
||||||
malformed, sidecar, Unicode-period, symlink, and traversal handling.
|
|
||||||
|
|
||||||
> `git diff HEAD~1..HEAD -- piker/tsp/_history.py`
|
|
||||||
|
|
||||||
Generated exact `ShmBufferRef` parsing and stable discovery, replacing
|
|
||||||
unsafe substring globbing. Added dtype resolution that mirrors history
|
|
||||||
allocation, exact-ref attachment, disappearance handling, and optional
|
|
||||||
exact-name iteration.
|
|
||||||
|
|
||||||
> `git diff HEAD~1..HEAD -- piker/data/_sharedmem.py`
|
|
||||||
|
|
||||||
Generated attach-only SHM access which never creates a segment when a
|
|
||||||
discovered object disappears.
|
|
||||||
|
|
||||||
> `git diff HEAD~1..HEAD -- piker/tsp/__init__.py`
|
|
||||||
|
|
||||||
Exported selector-ready SHM identity, parsing, listing, and dtype helpers.
|
|
||||||
|
|
||||||
> `git diff HEAD~1..HEAD -- tests/test_store_cli.py`
|
|
||||||
|
|
||||||
Generated command-level help, exact durable-series JSON, read-only SHM JSON,
|
|
||||||
and unknown exact-name regressions.
|
|
||||||
|
|
||||||
> `git diff HEAD~1..HEAD -- tests/test_ldshm.py`
|
|
||||||
|
|
||||||
Generated SHM identity, summary, ordering, dtype, forged-name, disabled
|
|
||||||
mutation-flag, and gap-markup regressions.
|
|
||||||
|
|
||||||
> `git diff HEAD~1..HEAD -- tests/test_storage_nativedb.py`
|
|
||||||
|
|
||||||
Generated compound index, canonical parser, malformed filename, and dotted
|
|
||||||
FQME regressions.
|
|
||||||
|
|
||||||
> `git diff HEAD~1..HEAD -- docs/manual_backfill_qualification.rst`
|
|
||||||
|
|
||||||
Documented `store series` and `store shm` as separate read-only durable and
|
|
||||||
live-memory triage layers.
|
|
||||||
|
|
||||||
> `git diff HEAD~1..HEAD -- .claude/skills/run-tests/test-harness-reference.md`
|
|
||||||
|
|
||||||
Registered the deterministic storage CLI UX target and mapping.
|
|
||||||
|
|
||||||
Verification output:
|
|
||||||
|
|
||||||
```text
|
|
||||||
........................................................................ [ 71%]
|
|
||||||
............................. [100%]
|
|
||||||
101 passed in 1.72s
|
|
||||||
```
|
|
||||||
|
|
||||||
Real `piker store` and `piker store audit` invocations printed complete
|
|
||||||
help without `--help`, exiting with Click's normal no-args help status 2.
|
|
||||||
`git diff --check` passed. Ruff was unavailable in the existing worktree
|
|
||||||
environment.
|
|
||||||
|
|
||||||
Adversarial review drove fixes for provider dtype reconstruction, observed
|
|
||||||
versus expected cadence, exact SHM grammar, discovery races, Typer direct
|
|
||||||
defaults, malformed NativeDB periods, compound cardinality compatibility,
|
|
||||||
undefined `ldshm` reload payloads, no-match debugger hangs, ordering
|
|
||||||
corruption reporting, and dtype precedence. Final review found no issues.
|
|
||||||
|
|
||||||
Optional external `fzf` or built-in numbered selection remains deferred.
|
|
||||||
The exact series and SHM identities added here are the intended foundation
|
|
||||||
for that later patch.
|
|
||||||
|
|
@ -1,39 +0,0 @@
|
||||||
---
|
|
||||||
model: openai/gpt-5.6-sol
|
|
||||||
service: opencode
|
|
||||||
session: ses_0799212ebffe42arY96czXn89F
|
|
||||||
timestamp: 2026-08-01T01:53:21Z
|
|
||||||
git_ref: 74ea2152
|
|
||||||
scope: code
|
|
||||||
substantive: true
|
|
||||||
raw_file: 20260801T015321Z_74ea2152_prompt_io.raw.md
|
|
||||||
---
|
|
||||||
|
|
||||||
## Prompt
|
|
||||||
|
|
||||||
Make `piker store ls` default to all NativeDB series as a compact counterpart
|
|
||||||
to detailed `series`, with sensible no-flag output and a visible help
|
|
||||||
description. Consolidate SHM inspection and repair under canonical `shm`,
|
|
||||||
because attachment to volatile segments is implied, and make inspection the
|
|
||||||
safe default while retaining explicit persistence and reload controls.
|
|
||||||
|
|
||||||
## Response summary
|
|
||||||
|
|
||||||
Replaced backend-oriented `ls` with compact NativeDB discovery and made both
|
|
||||||
listing commands default to all series. Consolidated SHM behavior under a
|
|
||||||
read-only-by-default `shm` command, guarded explicit repair options, updated
|
|
||||||
executable references and docs, and added command regressions.
|
|
||||||
|
|
||||||
## Files changed
|
|
||||||
|
|
||||||
- `piker/storage/cli.py` - listing defaults and unified SHM command modes.
|
|
||||||
- `tests/test_store_cli.py` - listing, help, and SHM mode regressions.
|
|
||||||
- `tests/test_ldshm.py` - explicit persistence-path regressions.
|
|
||||||
- `docs/manual_backfill_qualification.rst` - revised safe CLI workflow.
|
|
||||||
- `snippets/claude_debug_helper.py` - explicit SHM persistence commands.
|
|
||||||
- `.claude/skills/run-tests/test-harness-reference.md` - safety and test map.
|
|
||||||
- `.claude/skills/piker-slang/examples.md` - canonical command example.
|
|
||||||
|
|
||||||
## Human edits
|
|
||||||
|
|
||||||
None - generated output incorporates the user's design refinements.
|
|
||||||
|
|
@ -1,78 +0,0 @@
|
||||||
---
|
|
||||||
model: openai/gpt-5.6-sol
|
|
||||||
service: opencode
|
|
||||||
timestamp: 2026-08-01T01:53:21Z
|
|
||||||
git_ref: 74ea2152
|
|
||||||
diff_cmd: git diff HEAD~1..HEAD
|
|
||||||
---
|
|
||||||
|
|
||||||
The user requested command-surface refinements after reviewing the new
|
|
||||||
storage CLI. Bare `ls` should default to all series and act as a compact
|
|
||||||
form of detailed `series`; commands should produce sensible default output
|
|
||||||
without requiring trigger flags; and `ls` needed a parent-help description.
|
|
||||||
The user first proposed folding read-only SHM inspection into `ldshm -r`,
|
|
||||||
then clarified that `shm` is the better canonical command because attaching
|
|
||||||
to volatile segments is implied by the resource name.
|
|
||||||
|
|
||||||
The resulting design uses `piker store ls` as a compact NativeDB FQME and
|
|
||||||
period view, while `piker store series` reports each exact sub-series with
|
|
||||||
size and path details. Both default to all records and accept an optional
|
|
||||||
FQME substring.
|
|
||||||
|
|
||||||
`piker store shm FQME` is read-only by default. Persistence and reload enter
|
|
||||||
the prior repair workflow only through explicit `--write-parquet` and
|
|
||||||
`--reload-parquet-to-shm` flags. Reload requires persistence, and read-only
|
|
||||||
formatting flags are rejected in repair mode instead of being ignored.
|
|
||||||
|
|
||||||
> `git diff HEAD~1..HEAD -- piker/storage/cli.py`
|
|
||||||
|
|
||||||
Generated shared NativeDB payload collection, compact and detailed output,
|
|
||||||
numeric period ordering, default-all semantics, the canonical `shm` command,
|
|
||||||
safe mode dispatch, explicit repair validation, and debugger propagation in
|
|
||||||
read-only mode.
|
|
||||||
|
|
||||||
> `git diff HEAD~1..HEAD -- tests/test_store_cli.py`
|
|
||||||
|
|
||||||
Generated regressions for no-flag listing, parent-help descriptions, numeric
|
|
||||||
period display, default read-only SHM JSON, reload dependencies, and repair
|
|
||||||
versus read-only option conflicts.
|
|
||||||
|
|
||||||
> `git diff HEAD~1..HEAD -- tests/test_ldshm.py`
|
|
||||||
|
|
||||||
Adjusted persistence regressions for the renamed command and explicit repair
|
|
||||||
entry while preserving the historical failure rationale.
|
|
||||||
|
|
||||||
> `git diff HEAD~1..HEAD -- docs/manual_backfill_qualification.rst`
|
|
||||||
|
|
||||||
Documented compact versus detailed NativeDB discovery, default read-only SHM
|
|
||||||
inspection, and the explicitly mutating repair invocation.
|
|
||||||
|
|
||||||
> `git diff HEAD~1..HEAD -- snippets/claude_debug_helper.py`
|
|
||||||
|
|
||||||
Updated executable debug helper commands to select explicit SHM persistence.
|
|
||||||
|
|
||||||
> `git diff HEAD~1..HEAD -- .claude/skills/run-tests/test-harness-reference.md`
|
|
||||||
|
|
||||||
Updated operational safety guidance and SHM regression mappings.
|
|
||||||
|
|
||||||
> `git diff HEAD~1..HEAD -- .claude/skills/piker-slang/examples.md`
|
|
||||||
|
|
||||||
Updated the command example to the canonical `shm` spelling.
|
|
||||||
|
|
||||||
Verification output:
|
|
||||||
|
|
||||||
```text
|
|
||||||
........................................................................ [ 69%]
|
|
||||||
................................ [100%]
|
|
||||||
104 passed in 5.77s
|
|
||||||
```
|
|
||||||
|
|
||||||
Real command-help rendering was checked for the parent group, `ls`, `series`,
|
|
||||||
and `shm`. Real read-only `ls` and `series --json` output was also exercised
|
|
||||||
against the configured NativeDB. `git diff --check` passed.
|
|
||||||
|
|
||||||
Adversarial review identified and drove fixes for lexicographic period
|
|
||||||
ordering, stale runbook safety wording, ignored mode-specific options, and an
|
|
||||||
explicit-default `--max-gaps 10` validation bypass. Final review found no
|
|
||||||
remaining issues. Real OS-backed SHM attachment and successful repair/reload
|
|
||||||
remain integration-level validation concerns.
|
|
||||||
|
|
@ -1,33 +0,0 @@
|
||||||
---
|
|
||||||
model: openai/gpt-5.6-sol
|
|
||||||
service: opencode
|
|
||||||
session: ses_0799212ebffe42arY96czXn89F
|
|
||||||
timestamp: 2026-08-05T19:50:01Z
|
|
||||||
git_ref: f0f3c806
|
|
||||||
scope: code
|
|
||||||
substantive: true
|
|
||||||
raw_file: 20260805T195001Z_f0f3c806_prompt_io.raw.md
|
|
||||||
---
|
|
||||||
|
|
||||||
## Prompt
|
|
||||||
|
|
||||||
Inspect the completed NVDA 60s SHM backfill before workspace restart, explain
|
|
||||||
the persistent zero region, and continue fixing the indexing logic so sparse
|
|
||||||
IB history does not leave physical null reservations.
|
|
||||||
|
|
||||||
## Response summary
|
|
||||||
|
|
||||||
Captured a 42,720-row live SHM null span whose valid margins shared the same
|
|
||||||
timestamp. Replaced wall-clock row reservation with actual provider-boundary
|
|
||||||
placement, fixed stale SHM first-index, endpoint insertion and field-map
|
|
||||||
semantics, published the final storage prepend, and added a full multi-frame
|
|
||||||
`tsdb_backfill()` regression using a real `ShmArray`.
|
|
||||||
|
|
||||||
## Files changed
|
|
||||||
|
|
||||||
- `piker/tsp/_history.py` - contiguous provider and NativeDB prepends.
|
|
||||||
- `tests/test_history_backfill.py` - sparse multi-frame SHM regression.
|
|
||||||
|
|
||||||
## Human edits
|
|
||||||
|
|
||||||
None - generated output incorporates the user's live observations.
|
|
||||||
|
|
@ -1,71 +0,0 @@
|
||||||
---
|
|
||||||
model: openai/gpt-5.6-sol
|
|
||||||
service: opencode
|
|
||||||
timestamp: 2026-08-05T19:50:01Z
|
|
||||||
git_ref: f0f3c806
|
|
||||||
diff_cmd: git diff HEAD~1..HEAD
|
|
||||||
---
|
|
||||||
|
|
||||||
The user reported that a completed NVDA 60s backfill left a large zero
|
|
||||||
region in live SHM and asked for immediate inspection before restarting the
|
|
||||||
workspace. The captured buffer contained 42,720 invalid rows at absolute
|
|
||||||
indexes `3065895..3108614`. The valid rows on both sides had the identical
|
|
||||||
timestamp `1781271360`, proving the region was an erroneous physical-index
|
|
||||||
reservation rather than a real temporal gap.
|
|
||||||
|
|
||||||
The user then asked the agent to continue fixing the backfill logic.
|
|
||||||
|
|
||||||
> `git diff HEAD~1..HEAD -- piker/tsp/_history.py`
|
|
||||||
|
|
||||||
Removed wall-clock-based SHM reservation from `tsdb_backfill()`. Reverse
|
|
||||||
provider frames now extend the visible `ShmArray._first` boundary on every
|
|
||||||
prepend, beginning exactly at the current first index after inclusive query
|
|
||||||
endpoints are removed. Added `_prepend_tsdb_history()` to trim storage rows
|
|
||||||
against the actual earliest provider timestamp and prepend them directly
|
|
||||||
adjacent only after reverse provider retrieval completes.
|
|
||||||
|
|
||||||
This preserves sparse venue history as ordinal samples rather than treating
|
|
||||||
every elapsed calendar period as a physical SHM row. It also prevents the
|
|
||||||
old `_first + 1` insertion point from overwriting the earliest published bar.
|
|
||||||
|
|
||||||
> `git diff HEAD~1..HEAD -- tests/test_history_backfill.py`
|
|
||||||
|
|
||||||
Added a full `tsdb_backfill()` regression modeled on the live NVDA failure.
|
|
||||||
It uses a real `ShmArray`, NativeDB's field map, two inclusive reverse-provider
|
|
||||||
frames, and overlapping stored history. Assertions prove the complete
|
|
||||||
timestamp sequence is contiguous and lossless, contains no non-positive
|
|
||||||
timestamps, sends complete provider frames to durable storage, and notifies
|
|
||||||
chart/FSP consumers after the final storage prepend.
|
|
||||||
|
|
||||||
Live evidence captured before workspace restart:
|
|
||||||
|
|
||||||
```text
|
|
||||||
rows 1575296 first_index 1568233 last_index 3143528
|
|
||||||
invalid_count 42720
|
|
||||||
left boundary
|
|
||||||
[(3065893, 1781271300, 204.03, 204.82)
|
|
||||||
(3065894, 1781271360, 204.81, 204.72)
|
|
||||||
(3065895, 0, 0. , 0. )]
|
|
||||||
right boundary
|
|
||||||
[(3108614, 0, 0. , 0. )
|
|
||||||
(3108615, 1781271360, 204.81, 205.44)
|
|
||||||
(3108616, 1781271420, 205.46, 205.48)]
|
|
||||||
```
|
|
||||||
|
|
||||||
Verification output:
|
|
||||||
|
|
||||||
```text
|
|
||||||
........................................................................ [ 68%]
|
|
||||||
................................. [100%]
|
|
||||||
105 passed in 4.80s
|
|
||||||
```
|
|
||||||
|
|
||||||
Adversarial review caught stale `_first` semantics and an off-by-one
|
|
||||||
insertion point in the initial patch. The first live restart then exposed an
|
|
||||||
undefined `time_key` local at the new `tsdb_backfill()` call site, which
|
|
||||||
crashed `datad.ib` and propagated to the chart as `RemoteActorError`. The
|
|
||||||
helper now derives the source timestamp field from its field map, and the
|
|
||||||
test enters the full enclosing lifecycle so the original `NameError` can not
|
|
||||||
recur unnoticed. A final review also drove explicit publication after stored
|
|
||||||
history is prepended and deterministic final-stage synchronization. Final
|
|
||||||
review found no remaining issues; another live restart remains pending.
|
|
||||||
|
|
@ -1,36 +0,0 @@
|
||||||
---
|
|
||||||
model: openai/gpt-5.6-sol
|
|
||||||
service: opencode
|
|
||||||
session: ses_0799212ebffe42arY96czXn89F
|
|
||||||
timestamp: 2026-08-11T20:58:26Z
|
|
||||||
git_ref: 06d5ea5d
|
|
||||||
scope: code
|
|
||||||
substantive: true
|
|
||||||
raw_file: 20260811T205826Z_06d5ea5d_prompt_io.raw.md
|
|
||||||
---
|
|
||||||
|
|
||||||
## Prompt
|
|
||||||
|
|
||||||
Diagnose and fix an IB live-order failure where lazy NVDA contract
|
|
||||||
qualification raised `tractor.trionics.Lagged` after the order reached
|
|
||||||
`brokerd.ib`. Determine whether recent history changes or cancelled symbol
|
|
||||||
searches caused the stale state.
|
|
||||||
|
|
||||||
## Response summary
|
|
||||||
|
|
||||||
Traced the failure to competing consumers of a one-slot Trio broadcast, not
|
|
||||||
the recent SHM change or stale asyncio client state. Replaced broadcast
|
|
||||||
fan-out with one response/status demultiplexer, retained request-ID
|
|
||||||
correlation, handled cancellation and teardown, documented the task/channel
|
|
||||||
ownership model, warned on diagnostic metadata eviction, and added
|
|
||||||
deterministic proxy regressions.
|
|
||||||
|
|
||||||
## Files changed
|
|
||||||
|
|
||||||
- `piker/brokers/ib/api.py` - sole-reader IB proxy response routing.
|
|
||||||
- `tests/test_ib_method_proxy.py` - status, concurrency, cancellation and EOF regressions.
|
|
||||||
- `.claude/skills/run-tests/test-harness-reference.md` - proxy test mapping.
|
|
||||||
|
|
||||||
## Human edits
|
|
||||||
|
|
||||||
None - generated output follows the user's live failure diagnosis.
|
|
||||||
|
|
@ -1,66 +0,0 @@
|
||||||
---
|
|
||||||
model: openai/gpt-5.6-sol
|
|
||||||
service: opencode
|
|
||||||
timestamp: 2026-08-11T20:58:26Z
|
|
||||||
git_ref: 06d5ea5d
|
|
||||||
diff_cmd: git diff HEAD~1..HEAD
|
|
||||||
---
|
|
||||||
|
|
||||||
The user reported that certain IB order entries might have broken and
|
|
||||||
provided a live failure for a first NVDA sell order. EMS delivered the
|
|
||||||
request to `brokerd.ib`, but lazy `get_mkt_info('nvda.nasdaq')` contract
|
|
||||||
qualification failed in `MethodProxy._run_method()` with:
|
|
||||||
|
|
||||||
```text
|
|
||||||
tractor.trionics._broadcast.Lagged:
|
|
||||||
Task `piker.brokers.ib.broker.handle_order_requests` overrun and
|
|
||||||
dropped `0` values
|
|
||||||
```
|
|
||||||
|
|
||||||
The user suspected a cancelled symbol-search task might have left stale
|
|
||||||
asyncio state. Cancellation can leave an orphaned method response, but the
|
|
||||||
observed failure was stale Trio broadcast state. `open_client_proxy()`
|
|
||||||
broadcast a one-slot `LinkedTaskChannel` to an always-running IB status relay
|
|
||||||
and an idle method receiver. Status traffic advanced only the relay, so the
|
|
||||||
first method call for an uncached symbol entered an already-lagged receiver.
|
|
||||||
Orders for eagerly cached contracts bypassed this path.
|
|
||||||
|
|
||||||
The immediately preceding sparse-history commit did not modify IB order,
|
|
||||||
symbol, or proxy code; it exposed no direct dependency on this failure.
|
|
||||||
|
|
||||||
> `git diff HEAD~1..HEAD -- piker/brokers/ib/api.py`
|
|
||||||
|
|
||||||
Generated a sole-reader message demultiplexer for each IB client proxy.
|
|
||||||
`relay_client_proxy_messages()` owns every asyncio-to-Trio receive, handles
|
|
||||||
status/error traffic, and dispatches `mid`-tagged responses into per-call
|
|
||||||
`MethodProxy` wait slots. Cancelled calls remove their pending slot, causing
|
|
||||||
late responses to be logged and dropped without skewing later calls. Clean
|
|
||||||
channel EOF is accepted during normal proxy teardown.
|
|
||||||
|
|
||||||
Follow-up review restored the masked reconnect TODO and per-message case
|
|
||||||
rationale, documented the one-reader/many-writer task hierarchy and separate
|
|
||||||
per-proxy channels, and added a warning with request method metadata when the
|
|
||||||
bounded unresolved-request diagnostic table evicts its oldest entry.
|
|
||||||
|
|
||||||
> `git diff HEAD~1..HEAD -- tests/test_ib_method_proxy.py`
|
|
||||||
|
|
||||||
Generated deterministic regressions for idle status traffic before first
|
|
||||||
qualification, two concurrent calls with reverse-order responses, cancelled
|
|
||||||
calls with late responses followed by a successful call, and graceful
|
|
||||||
channel EOF. Every synchronization wait is timeout-bounded.
|
|
||||||
|
|
||||||
> `git diff HEAD~1..HEAD -- .claude/skills/run-tests/test-harness-reference.md`
|
|
||||||
|
|
||||||
Registered the deterministic IB method-proxy target and source mapping.
|
|
||||||
|
|
||||||
Verification output:
|
|
||||||
|
|
||||||
```text
|
|
||||||
........................ [100%]
|
|
||||||
25 passed, 4 warnings in 3.44s
|
|
||||||
```
|
|
||||||
|
|
||||||
The warnings are existing Tractor/Trio deprecations from the EMS test.
|
|
||||||
Adversarial review drove a clean-EOF guard and bounded polling. Final review
|
|
||||||
found no actionable issues. Live IB order validation remains pending and
|
|
||||||
requires restarting `brokerd.ib`.
|
|
||||||
|
|
@ -1,34 +0,0 @@
|
||||||
---
|
|
||||||
model: openai/gpt-5.6-sol
|
|
||||||
service: opencode
|
|
||||||
session: unavailable
|
|
||||||
timestamp: 2026-08-13T20:24:02Z
|
|
||||||
git_ref: 12b0d66e
|
|
||||||
scope: code
|
|
||||||
substantive: true
|
|
||||||
raw_file: 20260813T202402Z_12b0d66e_prompt_io.raw.md
|
|
||||||
---
|
|
||||||
|
|
||||||
## Prompt
|
|
||||||
|
|
||||||
Continue the paired piker broadcast-consumer work after Tractor PR 490
|
|
||||||
stabilization. Stop cached feed callers from mutating or consuming shared
|
|
||||||
quote-stream roots, then review, verify and prepare a commit boundary.
|
|
||||||
|
|
||||||
## Response summary
|
|
||||||
|
|
||||||
Made provider root streams private to the cached `Feed`, gave every caller a
|
|
||||||
local subscribed view, corrected feed cache identity, and made multi-provider
|
|
||||||
relays own synchronized child subscriptions and independent send-channel
|
|
||||||
clones.
|
|
||||||
|
|
||||||
## Files changed
|
|
||||||
|
|
||||||
- `piker/data/feed.py` - caller-local feed views, complete cache keys, and
|
|
||||||
subscribed multi-provider relays with independent channel lifetimes.
|
|
||||||
- `tests/test_feed_broadcast.py` - real broadcast ownership, lifecycle and
|
|
||||||
cache-identity regressions, including sibling survival after provider EOC.
|
|
||||||
|
|
||||||
## Human edits
|
|
||||||
|
|
||||||
None - generated output follows the requested cached-feed ownership item.
|
|
||||||
|
|
@ -1,51 +0,0 @@
|
||||||
---
|
|
||||||
model: openai/gpt-5.6-sol
|
|
||||||
service: opencode
|
|
||||||
timestamp: 2026-08-13T20:24:02Z
|
|
||||||
git_ref: 12b0d66e
|
|
||||||
diff_cmd: git diff HEAD~1..HEAD
|
|
||||||
---
|
|
||||||
|
|
||||||
The user asked to continue the paired piker broadcast-consumer work after
|
|
||||||
Tractor PR 490 stabilization. The next item was to stop cached feed callers
|
|
||||||
from mutating or consuming shared `Feed`/`Flume` quote-stream roots, then
|
|
||||||
review, verify and prepare a commit boundary.
|
|
||||||
|
|
||||||
> `git diff HEAD~1..HEAD -- piker/data/feed.py`
|
|
||||||
|
|
||||||
Changed `maybe_open_feed()` so the cached `Feed` exclusively owns provider
|
|
||||||
root streams and every caller, including the first cache owner, receives a
|
|
||||||
caller-local `Feed`/`Flume` view backed by one child subscription per provider.
|
|
||||||
The cache identity now includes normalized full symbol shape and stream policy.
|
|
||||||
|
|
||||||
Changed `Feed.open_multi_stream()` so every provider relay subscribes instead
|
|
||||||
of directly consuming a shared stream, and synchronizes relay startup before
|
|
||||||
yielding the merged channel. Each relay owns a cloned send channel so one
|
|
||||||
provider EOC cannot terminate its siblings. The module-level relay returns its
|
|
||||||
live child receiver through `TaskStatus` for typed retention and introspection.
|
|
||||||
Documented `Feed.pause()`/`Feed.resume()` as provider-context-wide controls
|
|
||||||
shared by local cache users.
|
|
||||||
|
|
||||||
> `git diff HEAD~1..HEAD -- tests/test_feed_broadcast.py`
|
|
||||||
|
|
||||||
Added real `Feed`, `Flume`, and Tractor broadcast regressions covering both
|
|
||||||
cache-miss owner and cache-hit paths, one-provider-many-market mapping,
|
|
||||||
multi-provider relay fan-out and startup, duplex controls, lexical child
|
|
||||||
closure, sibling survival after provider EOC, cached-root immutability, and
|
|
||||||
full cache-key differentiation.
|
|
||||||
|
|
||||||
Verification output:
|
|
||||||
|
|
||||||
```text
|
|
||||||
... [100%]
|
|
||||||
3 passed in 0.02s
|
|
||||||
|
|
||||||
. [100%]
|
|
||||||
1 passed in 0.01s
|
|
||||||
```
|
|
||||||
|
|
||||||
The second result is the existing EMS broadcast ownership regression. Ruff,
|
|
||||||
E501, compilation and whitespace checks passed. Repeated adversarial review
|
|
||||||
found and resolved shared `Feed.streams`, incomplete cache-key, fake-lifecycle,
|
|
||||||
relay-startup and first-owner asymmetry issues; final review reported no
|
|
||||||
findings.
|
|
||||||
|
|
@ -1,32 +0,0 @@
|
||||||
---
|
|
||||||
model: openai/gpt-5.6-sol
|
|
||||||
service: opencode
|
|
||||||
session: unavailable
|
|
||||||
timestamp: 2026-08-13T23:07:44Z
|
|
||||||
git_ref: ddac3553
|
|
||||||
scope: code
|
|
||||||
substantive: true
|
|
||||||
raw_file: 20260813T230744Z_ddac3553_prompt_io.raw.md
|
|
||||||
---
|
|
||||||
|
|
||||||
## Prompt
|
|
||||||
|
|
||||||
Continue the piker broadcast-consumer audit after the Deribit ownership commit
|
|
||||||
and prepare the next smallest ownership fix.
|
|
||||||
|
|
||||||
## Response summary
|
|
||||||
|
|
||||||
Replaced IB's manual raw quote-channel registry with a symbol-keyed cached
|
|
||||||
context and gave every caller a lexical child from one retained
|
|
||||||
`LinkedTaskChannel` broadcaster root.
|
|
||||||
|
|
||||||
## Files changed
|
|
||||||
|
|
||||||
- `piker/brokers/ib/feed.py` - cached raw-channel lifetime and lexical child
|
|
||||||
ownership.
|
|
||||||
- `tests/test_ib_broadcast.py` - concurrent fan-out, cache-key and child
|
|
||||||
lifetime regression.
|
|
||||||
|
|
||||||
## Human edits
|
|
||||||
|
|
||||||
None - generated output follows the requested broadcast-consumer audit.
|
|
||||||
|
|
@ -1,39 +0,0 @@
|
||||||
---
|
|
||||||
model: openai/gpt-5.6-sol
|
|
||||||
service: opencode
|
|
||||||
timestamp: 2026-08-13T23:07:44Z
|
|
||||||
git_ref: ddac3553
|
|
||||||
diff_cmd: git diff HEAD~1..HEAD
|
|
||||||
---
|
|
||||||
|
|
||||||
The user asked to continue the piker broadcast-consumer audit after committing
|
|
||||||
the Deribit ownership fix. The next isolated item was IB's actor-local
|
|
||||||
`_quote_streams` cache, which retained a raw channel owned by its first caller
|
|
||||||
and constructed competing broadcaster roots for later callers.
|
|
||||||
|
|
||||||
> `git diff HEAD~1..HEAD -- piker/brokers/ib/feed.py`
|
|
||||||
|
|
||||||
Factored `_open_aio_quote_channel()` as the raw asyncio-channel context and
|
|
||||||
made `open_aio_quote_stream()` cache that context by symbol through
|
|
||||||
`maybe_open_context()`. Every caller now receives a lexical child from the
|
|
||||||
retained `LinkedTaskChannel` broadcaster, while the raw source remains alive
|
|
||||||
until its final cache user exits. Removed the manual `_quote_streams` registry
|
|
||||||
and teardown mutation.
|
|
||||||
|
|
||||||
> `git diff HEAD~1..HEAD -- tests/test_ib_broadcast.py`
|
|
||||||
|
|
||||||
Added a concurrent ownership regression using real Tractor broadcast state.
|
|
||||||
It proves miss and hit callers use the same symbol/context factory, receive the
|
|
||||||
same quote through distinct children, close independently, and leave the owner
|
|
||||||
live after the hit exits.
|
|
||||||
|
|
||||||
Verification output:
|
|
||||||
|
|
||||||
```text
|
|
||||||
. [100%]
|
|
||||||
1 passed in 0.01s
|
|
||||||
```
|
|
||||||
|
|
||||||
The new test passes through a source-isolated IB feed import. Test Ruff, E501,
|
|
||||||
Python compilation and whitespace checks pass. Normal package collection
|
|
||||||
remains blocked by the existing Qt environment mismatch.
|
|
||||||
|
|
@ -1,43 +0,0 @@
|
||||||
---
|
|
||||||
model: openai/gpt-5.6-sol
|
|
||||||
service: opencode
|
|
||||||
session: unavailable
|
|
||||||
timestamp: 2026-08-14T03:27:16Z
|
|
||||||
git_ref: 2aa66c2c
|
|
||||||
scope: code
|
|
||||||
substantive: true
|
|
||||||
raw_file: 20260814T032716Z_2aa66c2c_prompt_io.raw.md
|
|
||||||
---
|
|
||||||
|
|
||||||
## Prompt
|
|
||||||
|
|
||||||
Prototype chart-actor-owned gap overlays from the current backfiller tip,
|
|
||||||
with actor-local startup and UI toggling. Preserve remote annotation IPC and
|
|
||||||
make the gap-specific path strict and `msgspec`-typed.
|
|
||||||
|
|
||||||
## Response summary
|
|
||||||
|
|
||||||
Adds a chart-local controller and vectorized SHM detector, starts historical
|
|
||||||
gap overlays with display state, toggles the focused timeframe through the
|
|
||||||
existing input loop and exposes the same renderer through a strict Tractor
|
|
||||||
payload context. Keeps all legacy remote annotation commands and separates
|
|
||||||
local graphics from each remote context's teardown ownership.
|
|
||||||
|
|
||||||
## Files changed
|
|
||||||
|
|
||||||
- `piker/ui/_gaps.py` - typed messages, detection and local controller.
|
|
||||||
- `piker/ui/_remote_ctl.py` - IPC clients and chart-actor endpoints.
|
|
||||||
- `piker/ui/_display.py` - actor-local startup integration.
|
|
||||||
- `piker/ui/_interaction.py` - focused `Ctrl+G` toggle.
|
|
||||||
- `piker/ui/_widget.py` - controller reference.
|
|
||||||
- `piker/ui/_annotate.py` - safe FQME-aware repositioning.
|
|
||||||
- `piker/tsp/_annotate.py` - typed annotation-client reference.
|
|
||||||
- `piker/storage/cli.py` - renamed annotation client context.
|
|
||||||
- `tests/test_gap_overlays.py` - detector and schema regression.
|
|
||||||
- `tests/test_ldshm.py` - renamed annotation-client test seams.
|
|
||||||
- `plans/opencode/chart-local-gap-overlays.md` - architecture and landing plan.
|
|
||||||
|
|
||||||
## Human edits
|
|
||||||
|
|
||||||
None - the work remains an uncommitted disposable-worktree prototype for
|
|
||||||
human review and live chart qualification.
|
|
||||||
|
|
@ -1,72 +0,0 @@
|
||||||
---
|
|
||||||
model: openai/gpt-5.6-sol
|
|
||||||
service: opencode
|
|
||||||
timestamp: 2026-08-14T03:27:16Z
|
|
||||||
git_ref: 2aa66c2c
|
|
||||||
diff_cmd: git diff HEAD~1..HEAD
|
|
||||||
---
|
|
||||||
|
|
||||||
## Prompt
|
|
||||||
|
|
||||||
The user asked what work had been done so far, then authorized autonomous
|
|
||||||
work while AFK. The target was to scheme and prototype an actor-local gap
|
|
||||||
annotator from the `backfiller_deep_fixes` tip without mutating existing
|
|
||||||
branches. The user then clarified twice that the annotation IPC API must
|
|
||||||
remain available and should become more rigorous and `msgspec`-typed rather
|
|
||||||
than being removed.
|
|
||||||
|
|
||||||
## Generated code
|
|
||||||
|
|
||||||
> `git diff HEAD~1..HEAD -- piker/ui/_gaps.py`
|
|
||||||
|
|
||||||
Adds tagged gap request/reply structs, render-ready gap specs, vectorized
|
|
||||||
OHLCV timestamp-gap detection and chart-local overlay ownership.
|
|
||||||
|
|
||||||
> `git diff HEAD~1..HEAD -- piker/ui/_remote_ctl.py`
|
|
||||||
|
|
||||||
Adds strict Tractor gap IPC, `AnnotClient` stream mapping/locking and
|
|
||||||
owner-scoped endpoint cleanup while retaining the legacy annotation endpoint.
|
|
||||||
|
|
||||||
> `git diff HEAD~1..HEAD -- piker/ui/_display.py`
|
|
||||||
|
|
||||||
Starts the local 60-second overlay after display-state registration.
|
|
||||||
|
|
||||||
> `git diff HEAD~1..HEAD -- piker/ui/_interaction.py`
|
|
||||||
|
|
||||||
Routes `Ctrl+G` from the focused chart's existing async input loop directly
|
|
||||||
to the local controller.
|
|
||||||
|
|
||||||
> `git diff HEAD~1..HEAD -- piker/ui/_widget.py`
|
|
||||||
|
|
||||||
Stores the chart actor's local controller reference.
|
|
||||||
|
|
||||||
> `git diff HEAD~1..HEAD -- piker/ui/_annotate.py`
|
|
||||||
|
|
||||||
Keeps redraw repositioning isolated by FQME and synchronizes arrow geometry.
|
|
||||||
|
|
||||||
> `git diff HEAD~1..HEAD -- piker/tsp/_annotate.py piker/storage/cli.py`
|
|
||||||
|
|
||||||
Renames the remote annotation control type and context manager to the
|
|
||||||
client-oriented `AnnotClient` / `open_annot_client` interface.
|
|
||||||
|
|
||||||
> `git diff HEAD~1..HEAD -- tests/test_gap_overlays.py`
|
|
||||||
|
|
||||||
Adds deterministic detector, directional payload-schema, typed error and
|
|
||||||
msgpack roundtrip regressions.
|
|
||||||
|
|
||||||
> `git diff HEAD~1..HEAD -- tests/test_ldshm.py`
|
|
||||||
|
|
||||||
Updates storage CLI test seams for the renamed annotation client context.
|
|
||||||
|
|
||||||
## Generated architecture output
|
|
||||||
|
|
||||||
> `git diff HEAD~1..HEAD -- plans/opencode/chart-local-gap-overlays.md`
|
|
||||||
|
|
||||||
Documents ownership, startup, UI, strict IPC, refresh and branch landing
|
|
||||||
semantics.
|
|
||||||
|
|
||||||
## Verification output
|
|
||||||
|
|
||||||
Source resolution points to this worktree. The targeted regression passes,
|
|
||||||
Ruff reports no changed-file findings, `compileall` succeeds, `git diff
|
|
||||||
--check` succeeds and Tractor's strict gap payload decoder initializes.
|
|
||||||
|
|
@ -1,39 +0,0 @@
|
||||||
---
|
|
||||||
model: openai/gpt-5.6-sol
|
|
||||||
service: opencode
|
|
||||||
session: unavailable
|
|
||||||
timestamp: 2026-08-17T22:20:36Z
|
|
||||||
git_ref: f65df6a1
|
|
||||||
scope: code
|
|
||||||
substantive: true
|
|
||||||
raw_file: 20260817T222036Z_f65df6a1_prompt_io.raw.md
|
|
||||||
---
|
|
||||||
|
|
||||||
## Prompt
|
|
||||||
|
|
||||||
Add end-to-end coverage for chart-local gap annotations in the existing
|
|
||||||
gap-overlay worktree without disturbing concurrent manual backfiller testing
|
|
||||||
in the root checkout.
|
|
||||||
|
|
||||||
## Response summary
|
|
||||||
|
|
||||||
Adds real offscreen Qt and real Tractor actor integration coverage. The tests
|
|
||||||
drive startup registration, PyQtGraph insertion/removal, focused Ctrl-G input,
|
|
||||||
history-prepend repositioning, duplicate-FQME chart identity, typed endpoint
|
|
||||||
exchange and cancellation with stale-response recovery. Defects exposed by
|
|
||||||
those tests are fixed in manager cleanup, Qt geometry notification and PyQt6
|
|
||||||
event routing.
|
|
||||||
|
|
||||||
## Files changed
|
|
||||||
|
|
||||||
- `piker/ui/_gaps.py` - correct PyQtGraph ownership and bounds behavior.
|
|
||||||
- `piker/ui/_annotate.py` - notify Qt before reposition geometry changes.
|
|
||||||
- `piker/ui/_display.py` - expose deterministic startup registration.
|
|
||||||
- `piker/ui/_interaction.py` - focus toggles and PyQt6 event registration.
|
|
||||||
- `tests/test_gap_overlays.py` - real Qt and Tractor integration coverage.
|
|
||||||
- `plans/opencode/chart-local-gap-overlays.md` - update coverage boundaries.
|
|
||||||
- `plans/opencode/chart-local-gap-overlays.summary.md` - update test totals.
|
|
||||||
|
|
||||||
## Human edits
|
|
||||||
|
|
||||||
None - the changes remain uncommitted for human review.
|
|
||||||
|
|
@ -1,59 +0,0 @@
|
||||||
---
|
|
||||||
model: openai/gpt-5.6-sol
|
|
||||||
service: opencode
|
|
||||||
timestamp: 2026-08-17T22:20:36Z
|
|
||||||
git_ref: f65df6a1
|
|
||||||
diff_cmd: git diff f65df6a1..HEAD
|
|
||||||
---
|
|
||||||
|
|
||||||
## Prompt
|
|
||||||
|
|
||||||
The user asked whether the chart-local gap annotation feature had complete
|
|
||||||
end-to-end coverage, then requested implementation in the existing gap-overlay
|
|
||||||
worktree while manual NativeDB/backfiller qualification continued in the root
|
|
||||||
checkout. The user initially declined a tests-only Prompt-IO entry. Real Qt
|
|
||||||
and Tractor integration tests then exposed production lifecycle and PyQt6
|
|
||||||
event-routing defects, making the resulting patch substantive code work and
|
|
||||||
requiring this full provenance entry.
|
|
||||||
|
|
||||||
## Generated code
|
|
||||||
|
|
||||||
> `git diff HEAD~1..HEAD -- piker/ui/_gaps.py`
|
|
||||||
|
|
||||||
Removes manager-owned graphics through their `PlotItem`, retains owning-plot
|
|
||||||
registrations, and excludes pixel-sized gap arrows from automatic data bounds.
|
|
||||||
|
|
||||||
> `git diff HEAD~1..HEAD -- piker/ui/_annotate.py`
|
|
||||||
|
|
||||||
Moves Qt geometry-change notification ahead of rectangle and arrow mutation so
|
|
||||||
scene spatial indexing follows history-prepend repositioning.
|
|
||||||
|
|
||||||
> `git diff HEAD~1..HEAD -- piker/ui/_display.py`
|
|
||||||
|
|
||||||
Extracts `_register_gap_overlays()` as the deterministic chart startup seam
|
|
||||||
which registers display states and renders default historical gap layers.
|
|
||||||
|
|
||||||
> `git diff HEAD~1..HEAD -- piker/ui/_interaction.py`
|
|
||||||
|
|
||||||
Adds focused-chart toggle routing, updates the gap keyboard path to PyQt6 event
|
|
||||||
enums, and keeps unrelated cached chart states untouched.
|
|
||||||
|
|
||||||
> `git diff HEAD~1..HEAD -- tests/test_gap_overlays.py`
|
|
||||||
|
|
||||||
Adds offscreen real-Qt manager, startup, keyboard, cached-chart and reposition
|
|
||||||
coverage plus a real Tractor actor endpoint/client dialog with deterministic
|
|
||||||
cancellation and stale-response recovery.
|
|
||||||
|
|
||||||
## Generated architecture output
|
|
||||||
|
|
||||||
> `git diff HEAD~1..HEAD -- plans/opencode/chart-local-gap-overlays.md plans/opencode/chart-local-gap-overlays.summary.md`
|
|
||||||
|
|
||||||
Records the automated integration boundary and remaining broker-backed manual
|
|
||||||
qualification requirements.
|
|
||||||
|
|
||||||
## Verification output
|
|
||||||
|
|
||||||
The source-isolated non-actor and related suites pass 24 tests with one actor
|
|
||||||
case deselected. The isolated actor case passes with one upstream Tractor/Trio
|
|
||||||
deprecation warning. Ruff, compileall, import resolution and whitespace checks
|
|
||||||
pass. Independent review reports no remaining P1/P2 findings.
|
|
||||||
|
|
@ -1,43 +0,0 @@
|
||||||
---
|
|
||||||
model: gpt-5.6-sol
|
|
||||||
service: opencode
|
|
||||||
session: tractor-addr-unpacking
|
|
||||||
timestamp: 2026-08-20T16:35:19Z
|
|
||||||
git_ref: 5f631e19
|
|
||||||
scope: code
|
|
||||||
substantive: true
|
|
||||||
raw_file: 20260820T163519Z_5f631e19_prompt_io.raw.md
|
|
||||||
---
|
|
||||||
|
|
||||||
## Prompt
|
|
||||||
|
|
||||||
The human asked to implement the Piker compatibility work caused by
|
|
||||||
tractor's canonical tagged-address migration, run the previously
|
|
||||||
deferred tractor verification, and keep runtime-managed WireGuard
|
|
||||||
bindspaces in a later separate PR.
|
|
||||||
|
|
||||||
## Response summary
|
|
||||||
|
|
||||||
Made Piker normalize legacy and tagged registry inputs before semantic
|
|
||||||
comparison and process-global storage. Updated runtime address typing and
|
|
||||||
test assertions for canonical output, retained legacy caller input, and
|
|
||||||
removed a duplicate pytest log-level option that blocked downstream test
|
|
||||||
collection with current tractor.
|
|
||||||
|
|
||||||
## Files changed
|
|
||||||
|
|
||||||
- `piker/service/_registry.py` - canonical registry normalization and
|
|
||||||
shared address typing.
|
|
||||||
- `piker/service/_actor_runtime.py` - tagged-compatible runtime API
|
|
||||||
annotations.
|
|
||||||
- `tests/conftest.py` - canonical fixture comparison and shared pytest
|
|
||||||
option ownership.
|
|
||||||
- `tests/test_services.py` - registry regression and canonical runtime
|
|
||||||
assertions.
|
|
||||||
|
|
||||||
## Human edits
|
|
||||||
|
|
||||||
The human explicitly requested the downstream implementation and chose
|
|
||||||
to keep WireGuard bindspace lifecycle work in a separate future tractor
|
|
||||||
PR. The agent implemented and tested those directions; no direct manual
|
|
||||||
source edits were observed.
|
|
||||||
|
|
@ -1,25 +0,0 @@
|
||||||
---
|
|
||||||
model: gpt-5.6-sol
|
|
||||||
service: opencode
|
|
||||||
timestamp: 2026-08-20T16:35:19Z
|
|
||||||
git_ref: 5f631e19
|
|
||||||
diff_cmd: git diff HEAD~1..HEAD
|
|
||||||
---
|
|
||||||
|
|
||||||
# Raw output - tractor tagged-address compatibility
|
|
||||||
|
|
||||||
The human requested downstream Piker qualification after tractor PR
|
|
||||||
#505 switched transport address emission to canonical tagged tuples.
|
|
||||||
|
|
||||||
> `git diff HEAD~1..HEAD -- piker/service/_registry.py piker/service/_actor_runtime.py tests/conftest.py tests/test_services.py`
|
|
||||||
|
|
||||||
Normalized legacy and tagged registry addresses through tractor's
|
|
||||||
address codec before Piker stores or compares them. Extended service
|
|
||||||
runtime annotations to cover both tuple shapes, updated runtime tests to
|
|
||||||
compare canonical serialized values, and removed Piker's duplicate
|
|
||||||
registration of tractor's shared `--ll` pytest option.
|
|
||||||
|
|
||||||
Ruff passed. The focused normalization regression and Piker runtime boot
|
|
||||||
passed against the local tractor checkout at `5d92595f`. The wider
|
|
||||||
service module reached an unrelated pre-existing logger-name assertion
|
|
||||||
in `maybe_spawn_daemon()` after both address-focused tests passed.
|
|
||||||
|
|
@ -1,39 +0,0 @@
|
||||||
---
|
|
||||||
model: gpt-5.6-sol
|
|
||||||
service: opencode
|
|
||||||
session: tractor-addr-unpacking
|
|
||||||
timestamp: 2026-08-21T04:20:58Z
|
|
||||||
git_ref: b7aca4c0
|
|
||||||
scope: code
|
|
||||||
substantive: true
|
|
||||||
raw_file: 20260821T042058Z_b7aca4c0_prompt_io.raw.md
|
|
||||||
---
|
|
||||||
|
|
||||||
## Prompt
|
|
||||||
|
|
||||||
The human approved refining Piker's logger-name handling after the
|
|
||||||
wider service test exposed a stale parent-logger assertion during the
|
|
||||||
tractor tagged-address compatibility work.
|
|
||||||
|
|
||||||
## Response summary
|
|
||||||
|
|
||||||
Changed Piker's logger helpers to remove only an exact `piker.` prefix,
|
|
||||||
preserving complete requested submodule names. Removed daemon startup
|
|
||||||
assertions that incorrectly required parent logger names and added a
|
|
||||||
regression covering both logger factories and names that the old
|
|
||||||
character-set stripping silently corrupted.
|
|
||||||
|
|
||||||
## Files changed
|
|
||||||
|
|
||||||
- `piker/log.py` - exact relative logger-name derivation.
|
|
||||||
- `piker/service/_daemon.py` - remove stale parent-name assertion.
|
|
||||||
- `piker/data/_daemon.py` - remove stale parent-name assertion.
|
|
||||||
- `piker/brokers/_daemon.py` - remove stale parent-name assertion.
|
|
||||||
- `piker/clearing/_ems.py` - remove redundant logger assertion.
|
|
||||||
- `tests/test_logging.py` - package logger namespace regressions.
|
|
||||||
|
|
||||||
## Human edits
|
|
||||||
|
|
||||||
The human explicitly approved this follow-up after reviewing the
|
|
||||||
residual Piker service-suite gap. The agent implemented and verified
|
|
||||||
the refinement; no direct manual source edits were observed.
|
|
||||||
|
|
@ -1,24 +0,0 @@
|
||||||
---
|
|
||||||
model: gpt-5.6-sol
|
|
||||||
service: opencode
|
|
||||||
timestamp: 2026-08-21T04:20:58Z
|
|
||||||
git_ref: b7aca4c0
|
|
||||||
diff_cmd: git diff HEAD~1..HEAD
|
|
||||||
---
|
|
||||||
|
|
||||||
# Raw output - preserve Piker logger namespaces
|
|
||||||
|
|
||||||
The human approved a follow-up refinement after the full Piker service
|
|
||||||
module exposed a stale logger-name assertion while qualifying tractor
|
|
||||||
PR #505.
|
|
||||||
|
|
||||||
> `git diff HEAD~1..HEAD -- piker/log.py piker/service/_daemon.py piker/data/_daemon.py piker/brokers/_daemon.py piker/clearing/_ems.py tests/test_logging.py`
|
|
||||||
|
|
||||||
Replaced character-set `str.lstrip()` calls with exact package-prefix
|
|
||||||
removal, removed assertion-only parent-logger expectations from daemon
|
|
||||||
startup paths, and added regression coverage for both logger factories
|
|
||||||
and prefix-sensitive submodule names.
|
|
||||||
|
|
||||||
Ruff passed after removing newly unused logger assignments. All four
|
|
||||||
logger regressions and all six service integration tests passed against
|
|
||||||
the local tractor PR #505 checkout.
|
|
||||||
|
|
@ -1,46 +0,0 @@
|
||||||
---
|
|
||||||
model: openai/gpt-5.6-sol
|
|
||||||
service: opencode
|
|
||||||
session: unavailable
|
|
||||||
timestamp: 2026-08-26T22:26:00Z
|
|
||||||
git_ref: 5f631e19
|
|
||||||
scope: code
|
|
||||||
substantive: true
|
|
||||||
raw_file: 20260826T222600Z_5f631e19_prompt_io.raw.md
|
|
||||||
---
|
|
||||||
|
|
||||||
## Prompt
|
|
||||||
|
|
||||||
Diagnose and fix the hanging Binance USD-M perpetual feed for
|
|
||||||
`NVDAUSDT.USDTM.PERP.BINANCE` by comparing piker with Binance's current
|
|
||||||
API and changelog. Ensure unsupported or silent websocket feeds fail
|
|
||||||
loudly into `datad` and chart consumers instead of blocking startup.
|
|
||||||
|
|
||||||
## Response summary
|
|
||||||
|
|
||||||
Migrates Binance USD-M market data from the retired legacy websocket to
|
|
||||||
the split public and market routes, handles Binance's new aggregate-trade
|
|
||||||
schema, and introduces bounded connection and first-data startup. Direct
|
|
||||||
stream URLs avoid reconnect-time subscription-ACK races, while targeted
|
|
||||||
tests preserve routing and fail-loud behavior.
|
|
||||||
|
|
||||||
## Files changed
|
|
||||||
|
|
||||||
- `piker/brokers/binance/feed.py` - route, merge, normalize, and bound
|
|
||||||
Binance quote streams.
|
|
||||||
- `piker/brokers/binance/venues.py` - define current USD-M websocket
|
|
||||||
endpoints.
|
|
||||||
- `piker/data/_web_bs.py` - support bounded websocket context startup.
|
|
||||||
- `tests/test_binance.py` - cover routes, schemas, and startup failures.
|
|
||||||
- `ai/prompt-io/opencode/20260826T222600Z_5f631e19_prompt_io.raw.md` -
|
|
||||||
retain the generated-output record.
|
|
||||||
- `ai/prompt-io/opencode/20260826T222600Z_5f631e19_prompt_io.md` - record
|
|
||||||
structured provenance for the code changes.
|
|
||||||
|
|
||||||
## Human edits
|
|
||||||
|
|
||||||
The human explicitly required fail-loud behavior rather than merely
|
|
||||||
updating the endpoint, and supplied a direct CLI reproduction proving
|
|
||||||
both `datad.binance` and the chart caller blocked. Those requirements
|
|
||||||
drove the connection and first-data deadlines and their regressions. The
|
|
||||||
changes remain uncommitted for human review.
|
|
||||||
|
|
@ -1,57 +0,0 @@
|
||||||
---
|
|
||||||
model: openai/gpt-5.6-sol
|
|
||||||
service: opencode
|
|
||||||
timestamp: 2026-08-26T22:26:00Z
|
|
||||||
git_ref: 5f631e19
|
|
||||||
diff_cmd: git diff HEAD~1..HEAD
|
|
||||||
---
|
|
||||||
|
|
||||||
## Prompt
|
|
||||||
|
|
||||||
The user reported that selecting
|
|
||||||
`NVDAUSDT.USDTM.PERP.BINANCE` connected Binance's USD-M
|
|
||||||
websocket but never loaded the feed or downstream chart. They asked
|
|
||||||
OpenCode to compare Binance's API changelog with piker's FQME and feed
|
|
||||||
code, diagnose the failure, and solve it.
|
|
||||||
|
|
||||||
The user clarified that future provider failures must raise loudly
|
|
||||||
instead of silently blocking feed and chart startup. They also confirmed
|
|
||||||
the same hang from
|
|
||||||
`piker -l info --pdb chart nvdausdt.usdtm.perp.binance`.
|
|
||||||
|
|
||||||
## Generated code
|
|
||||||
|
|
||||||
> `git diff HEAD~1..HEAD -- piker/brokers/binance/feed.py`
|
|
||||||
|
|
||||||
Routes USD-M book-ticker and aggregate-trade data through Binance's new
|
|
||||||
`/public` and `/market` websocket paths, merges both sockets into the
|
|
||||||
existing quote stream, accepts the new `st` aggregate-trade field, and
|
|
||||||
requires both stream classes before declaring startup complete. It also
|
|
||||||
removes the misleading missing-provider warning, derives feed mode from
|
|
||||||
resolved pair metadata, and raises `DataUnavailable` for connection or
|
|
||||||
live-data startup timeouts.
|
|
||||||
|
|
||||||
> `git diff HEAD~1..HEAD -- piker/brokers/binance/venues.py`
|
|
||||||
|
|
||||||
Defines Binance's current public and market USD-M websocket endpoints
|
|
||||||
and makes the market endpoint the regular futures websocket API root.
|
|
||||||
|
|
||||||
> `git diff HEAD~1..HEAD -- piker/data/_web_bs.py`
|
|
||||||
|
|
||||||
Adds an optional connection deadline to `open_autorecon_ws()` so callers
|
|
||||||
can fail context entry when repeated websocket handshakes never become
|
|
||||||
connected.
|
|
||||||
|
|
||||||
> `git diff HEAD~1..HEAD -- tests/test_binance.py`
|
|
||||||
|
|
||||||
Adds regressions for exact Binance stream routing, silent live-data
|
|
||||||
startup, silent websocket connection startup, and the current USD-M
|
|
||||||
aggregate-trade payload schema.
|
|
||||||
|
|
||||||
## Verification output
|
|
||||||
|
|
||||||
`pytest -q tests/test_binance.py` passes all four tests. Compile checks,
|
|
||||||
focused Ruff fatal-error checks, and `git diff --check` pass. Live probes
|
|
||||||
through piker's merged stream received both required stream classes and
|
|
||||||
returned normalized trade quotes for `NVDAUSDT` USD-M and `BTCUSDT`
|
|
||||||
spot.
|
|
||||||
|
|
@ -1,35 +0,0 @@
|
||||||
---
|
|
||||||
model: openai/gpt-5.6-sol
|
|
||||||
service: opencode
|
|
||||||
session: pid-469264
|
|
||||||
timestamp: 2026-08-28T06:31:26Z
|
|
||||||
git_ref: 1606122b
|
|
||||||
scope: code
|
|
||||||
substantive: true
|
|
||||||
raw_file: 20260828T063126Z_1606122b_prompt_io.raw.md
|
|
||||||
---
|
|
||||||
|
|
||||||
## Prompt
|
|
||||||
|
|
||||||
Review the staged IB broker patch before committing it. After the review
|
|
||||||
identified a deterministic error-parser crash, continue with the available
|
|
||||||
next steps while leaving the typed EMS worktree and Ruff change untouched.
|
|
||||||
|
|
||||||
## Response summary
|
|
||||||
|
|
||||||
Repaired bracketed IB API error parsing by splitting on the matched error
|
|
||||||
prefix. Added a deterministic regression that routes bracketed and plain
|
|
||||||
strings through the real trade-event relay and verifies their EMS error
|
|
||||||
messages.
|
|
||||||
|
|
||||||
## Files changed
|
|
||||||
|
|
||||||
- `piker/brokers/ib/broker.py` - safely extracts bracketed API error codes.
|
|
||||||
- `tests/test_ib_broker.py` - covers bracketed and plain error relay inputs.
|
|
||||||
|
|
||||||
## Human edits
|
|
||||||
|
|
||||||
The human required review before the root commit, accepted the identified
|
|
||||||
parser defect as the next work item, and explicitly authorized continuation.
|
|
||||||
The pre-existing staged IB refactor is human-authored; the agent applied only
|
|
||||||
the parser repair and focused regression described above.
|
|
||||||
|
|
@ -1,30 +0,0 @@
|
||||||
---
|
|
||||||
model: openai/gpt-5.6-sol
|
|
||||||
service: opencode
|
|
||||||
timestamp: 2026-08-28T06:31:26Z
|
|
||||||
git_ref: fix_symcache_search
|
|
||||||
diff_cmd: git diff HEAD~1..HEAD
|
|
||||||
---
|
|
||||||
|
|
||||||
## Prompt
|
|
||||||
|
|
||||||
The human asked for a review of the root checkout's staged IB patch before
|
|
||||||
committing it. The review found that bracketed API error strings call
|
|
||||||
`str.partition()` without its required separator. The human then instructed
|
|
||||||
the agent to continue with available next steps, or ask only if unsure.
|
|
||||||
|
|
||||||
## Response
|
|
||||||
|
|
||||||
> `git diff HEAD~1..HEAD -- piker/brokers/ib/broker.py`
|
|
||||||
|
|
||||||
The generated parser repair uses the recognized `[code ` or `[Errno ` prefix
|
|
||||||
to extract the numeric code without changing the surrounding event relay.
|
|
||||||
|
|
||||||
> `git diff HEAD~1..HEAD -- tests/test_ib_broker.py`
|
|
||||||
|
|
||||||
The generated regression drives bracketed and plain strings through
|
|
||||||
`deliver_trade_events()` and verifies that one `BrokerdError` reaches EMS
|
|
||||||
with the expected reason and unknown request ID.
|
|
||||||
|
|
||||||
The typed EMS worktree and the unrelated Ruff policy change were left
|
|
||||||
untouched. No commit was created.
|
|
||||||
|
|
@ -1,37 +0,0 @@
|
||||||
---
|
|
||||||
model: openai/gpt-5.6-sol
|
|
||||||
service: opencode
|
|
||||||
session: 3e2eb8c9-6cfb-49a5-bf1a-1c0c2c2a4ddd
|
|
||||||
timestamp: 2026-08-28T20:03:52Z
|
|
||||||
git_ref: dd676857
|
|
||||||
scope: config
|
|
||||||
substantive: true
|
|
||||||
raw_file: 20260828T200352Z_dd676857_prompt_io.raw.md
|
|
||||||
---
|
|
||||||
|
|
||||||
## Prompt
|
|
||||||
|
|
||||||
Update the xonsh version used by the repository dev shell before stacking
|
|
||||||
the Binance work, while retaining the dotrc compatibility shim as an
|
|
||||||
immediate workaround for older shells.
|
|
||||||
|
|
||||||
## Response summary
|
|
||||||
|
|
||||||
Raises the xonsh dependency floor to `0.24.2` and refreshes its lock entry.
|
|
||||||
The new release supports Python 3.13, handles Git alias completion natively,
|
|
||||||
and works in the updated Nix development shell.
|
|
||||||
|
|
||||||
## Files changed
|
|
||||||
|
|
||||||
- `pyproject.toml` - require xonsh `0.24.2` or newer.
|
|
||||||
- `uv.lock` - lock xonsh `0.24.2` wheels and source metadata.
|
|
||||||
- `ai/prompt-io/opencode/20260828T200352Z_dd676857_prompt_io.raw.md` - retain
|
|
||||||
the triggering prompt, generated configuration summary, and checks.
|
|
||||||
- `ai/prompt-io/opencode/20260828T200352Z_dd676857_prompt_io.md` - record
|
|
||||||
structured provenance for the dependency update.
|
|
||||||
|
|
||||||
## Human edits
|
|
||||||
|
|
||||||
The user identified old xonsh behavior as the root cause, reported the
|
|
||||||
separate dotrc compatibility shim, and directed that the repository update
|
|
||||||
remain a distinct flake-layer change before the Binance stack continues.
|
|
||||||
|
|
@ -1,36 +0,0 @@
|
||||||
---
|
|
||||||
model: openai/gpt-5.6-sol
|
|
||||||
service: opencode
|
|
||||||
timestamp: 2026-08-28T20:03:52Z
|
|
||||||
git_ref: dd676857
|
|
||||||
diff_cmd: git diff HEAD~1..HEAD
|
|
||||||
---
|
|
||||||
|
|
||||||
## Prompt
|
|
||||||
|
|
||||||
The user traced broken multiline conflict-resolution pastes to the old
|
|
||||||
xonsh release installed by the repository dev shell. A separate dotrc
|
|
||||||
agent added a compatibility shim, and the user directed that xonsh still
|
|
||||||
be updated alongside the flake work before continuing the Binance stack.
|
|
||||||
|
|
||||||
## Generated configuration
|
|
||||||
|
|
||||||
> `git diff HEAD~1..HEAD -- pyproject.toml uv.lock`
|
|
||||||
|
|
||||||
Raises the repository's xonsh requirement from `0.22.2` to `0.24.2` and
|
|
||||||
refreshes only xonsh's locked distribution metadata. The selected release
|
|
||||||
supports Python 3.13, includes native Git-config alias completion, and fixes
|
|
||||||
Nix metadata checks.
|
|
||||||
|
|
||||||
## Verification output
|
|
||||||
|
|
||||||
- `uv lock --upgrade-package xonsh` resolves 111 packages and updates only
|
|
||||||
xonsh from `0.22.4` to `0.24.2`.
|
|
||||||
- Fresh `nix develop` replaces the worktree-local xonsh installation.
|
|
||||||
- `xonsh --version` reports `0.24.2`.
|
|
||||||
- A multi-command xonsh invocation executes Git commands successfully.
|
|
||||||
- `nix flake check --no-build` passes for `x86_64-linux`.
|
|
||||||
- `git diff --check` passes.
|
|
||||||
|
|
||||||
The existing Polars `1.36.0` lock remains unchanged despite uv reporting
|
|
||||||
that its distributions are yanked for an `unpivot` regression.
|
|
||||||
|
|
@ -1,46 +0,0 @@
|
||||||
---
|
|
||||||
model: openai/gpt-5.6-sol
|
|
||||||
service: opencode
|
|
||||||
session: c3a1b1ba-65c6-4c8e-bd12-e596031b0eb0
|
|
||||||
timestamp: 2026-08-29T17:55:42Z
|
|
||||||
git_ref: e09e306a
|
|
||||||
scope: code
|
|
||||||
substantive: true
|
|
||||||
raw_file: 20260829T175542Z_e09e306a_prompt_io.raw.md
|
|
||||||
---
|
|
||||||
|
|
||||||
## Prompt
|
|
||||||
|
|
||||||
Preserve the existing Binance USD-M commit above the reviewed flake/xonsh
|
|
||||||
stack, then add missing deterministic tests and correctness fixes as a
|
|
||||||
separate follow-up with a complete editor-gated commit plan.
|
|
||||||
|
|
||||||
## Response summary
|
|
||||||
|
|
||||||
Hardens Binance feed startup around its supported one-symbol contract,
|
|
||||||
direct websocket routes, current provider schemas, and first-quote
|
|
||||||
readiness. Adds deterministic Binance regressions and separates generic
|
|
||||||
reconnect behavior into its own tested commit boundary.
|
|
||||||
|
|
||||||
## Files changed
|
|
||||||
|
|
||||||
- `piker/brokers/binance/feed.py` - validate feed cardinality, route
|
|
||||||
USD-M stream classes, and declare readiness after either quote class.
|
|
||||||
- `tests/test_binance.py` - cover routes, first-quote startup, cardinality,
|
|
||||||
and documented book-ticker and aggregate-trade schemas.
|
|
||||||
- `ai/prompt-io/opencode/20260829T175542Z_e09e306a_prompt_io.raw.md` - retain
|
|
||||||
the triggering prompt, generated-code summary, and verification output.
|
|
||||||
- `ai/prompt-io/opencode/20260829T175542Z_e09e306a_prompt_io.md` - record
|
|
||||||
structured provenance for the follow-up.
|
|
||||||
|
|
||||||
## Human edits
|
|
||||||
|
|
||||||
The user rejected reconstructing already committed Binance work, required
|
|
||||||
preservation through cherry-pick, selected the flake/xonsh stack base, and
|
|
||||||
required the review fixes to remain a separate tested boundary. The user also
|
|
||||||
corrected the workflow to require an automatic `/commit-plan` at every stack
|
|
||||||
review gate. During Tuicr review, the user rejected trade-gated startup,
|
|
||||||
requested a feed-specific initialization error and async-generator TODO,
|
|
||||||
and required generic reconnect changes and tests to land independently.
|
|
||||||
The user then clarified that either L1 or trade data must unblock startup,
|
|
||||||
rather than preferring one stream class or waiting independently for both.
|
|
||||||
|
|
@ -1,43 +0,0 @@
|
||||||
---
|
|
||||||
model: openai/gpt-5.6-sol
|
|
||||||
service: opencode
|
|
||||||
timestamp: 2026-08-29T17:55:42Z
|
|
||||||
git_ref: e09e306a
|
|
||||||
diff_cmd: git diff HEAD~1..HEAD
|
|
||||||
---
|
|
||||||
|
|
||||||
## Prompt
|
|
||||||
|
|
||||||
The user asked to land recent branch work in small chronological patch sets,
|
|
||||||
including missing tests. For Binance, the user required preserving the
|
|
||||||
existing committed topic patch through cherry-pick and placing newly found
|
|
||||||
correctness repairs in a separate follow-up boundary above the flake/xonsh
|
|
||||||
stack. The user also required every yielded stack layer to include a complete
|
|
||||||
`/commit-plan` with patch-specific checks.
|
|
||||||
|
|
||||||
## Generated code
|
|
||||||
|
|
||||||
> `git diff HEAD~1..HEAD -- piker/brokers/binance/feed.py`
|
|
||||||
|
|
||||||
Restricts Binance feeds to the supported one-symbol contract, renders direct
|
|
||||||
quote routes, separates L1 liveness from first-trade readiness, and derives
|
|
||||||
the contextual trade timeout from Trio's enclosing effective deadline.
|
|
||||||
|
|
||||||
> `git diff HEAD~1..HEAD -- piker/data/_web_bs.py`
|
|
||||||
|
|
||||||
Lets structured cancellation bypass reconnect throttling and ensures handled
|
|
||||||
handshake rejections retry before accessing connection-local cancel state.
|
|
||||||
|
|
||||||
> `git diff HEAD~1..HEAD -- tests/test_binance.py`
|
|
||||||
|
|
||||||
Adds deterministic regressions for symbol cardinality, route selection,
|
|
||||||
silent startup, immediate rejection, cancellation cleanup, aggregate-trade
|
|
||||||
schema evolution, delayed trades, and finite/infinite/tiny deadlines.
|
|
||||||
|
|
||||||
## Verification output
|
|
||||||
|
|
||||||
- Targeted Binance tests: `10 passed in 0.26s`.
|
|
||||||
- Ruff passes for all three changed Python files.
|
|
||||||
- `git diff --check` passes.
|
|
||||||
- Independent final review reports no actionable findings.
|
|
||||||
- No live Binance network probe was run.
|
|
||||||
|
|
@ -1,42 +0,0 @@
|
||||||
---
|
|
||||||
model: openai/gpt-5.6-sol
|
|
||||||
service: opencode
|
|
||||||
session: c3a1b1ba-65c6-4c8e-bd12-e596031b0eb0
|
|
||||||
timestamp: 2026-09-01T02:42:35Z
|
|
||||||
git_ref: e09e306a
|
|
||||||
scope: code
|
|
||||||
substantive: true
|
|
||||||
raw_file: 20260901T024235Z_e09e306a_prompt_io.raw.md
|
|
||||||
---
|
|
||||||
|
|
||||||
## Prompt
|
|
||||||
|
|
||||||
Address the Tuicr review by separating generic websocket reconnect
|
|
||||||
changes from Binance-specific work and testing the real reconnect task
|
|
||||||
hierarchy rather than a reimplementation.
|
|
||||||
|
|
||||||
## Response summary
|
|
||||||
|
|
||||||
Preserves structured cancellation and safe first-handshake retries in
|
|
||||||
`_web_bs`. Adds three focused regressions in a dedicated websocket test
|
|
||||||
module and packages them as an independent commit boundary.
|
|
||||||
|
|
||||||
## Files changed
|
|
||||||
|
|
||||||
- `piker/data/_web_bs.py` - restart handled handshake failures and let
|
|
||||||
cancellation bypass the reconnect throttle.
|
|
||||||
- `tests/test_websocket.py` - exercise real reconnect startup, retry,
|
|
||||||
and cancellation task paths with controlled transport boundaries.
|
|
||||||
- `ai/prompt-io/opencode/20260901T024235Z_e09e306a_prompt_io.raw.md`
|
|
||||||
- retain the review prompt, generated-code summary, and verification.
|
|
||||||
- `ai/prompt-io/opencode/20260901T024235Z_e09e306a_prompt_io.md` - record
|
|
||||||
structured provenance for the generic reconnect boundary.
|
|
||||||
|
|
||||||
## Human edits
|
|
||||||
|
|
||||||
The user identified that these changes were generic rather than
|
|
||||||
Binance-specific, required a separate commit and test module, requested
|
|
||||||
coverage of the real reconnect hierarchy, and corrected code prose to
|
|
||||||
refer to `trio` consistently. A follow-up review required the production
|
|
||||||
code to explain why `trio.Cancelled` bypasses the broad reconnect-error
|
|
||||||
handler and cite the regression which audits that teardown invariant.
|
|
||||||
|
|
@ -1,33 +0,0 @@
|
||||||
---
|
|
||||||
model: openai/gpt-5.6-sol
|
|
||||||
service: opencode
|
|
||||||
timestamp: 2026-09-01T02:42:35Z
|
|
||||||
git_ref: e09e306a
|
|
||||||
diff_cmd: git diff HEAD~1..HEAD
|
|
||||||
---
|
|
||||||
|
|
||||||
## Prompt
|
|
||||||
|
|
||||||
The user requested that generic `_web_bs` reconnect fixes and tests be
|
|
||||||
separated from the Binance backend patch. The review also required tests
|
|
||||||
to exercise the real reconnect task hierarchy, cite the previous faulty
|
|
||||||
branches, and consistently name `trio` in code prose.
|
|
||||||
|
|
||||||
## Generated code
|
|
||||||
|
|
||||||
> `git diff HEAD~1..HEAD -- piker/data/_web_bs.py`
|
|
||||||
|
|
||||||
Ensures handled handshake failures immediately restart the reconnect
|
|
||||||
loop and lets structured cancellation bypass shielded error throttling.
|
|
||||||
|
|
||||||
> `git diff HEAD~1..HEAD -- tests/test_websocket.py`
|
|
||||||
|
|
||||||
Adds deterministic scheduler-real regressions for handshake startup
|
|
||||||
timeouts, immediate rejection retries, and cancellation propagation.
|
|
||||||
|
|
||||||
## Verification output
|
|
||||||
|
|
||||||
- Generic and Binance targeted tests: `9 passed in 0.06s`.
|
|
||||||
- Ruff passes for all four changed Python files.
|
|
||||||
- `git diff --check` passes.
|
|
||||||
- No live network probe was run.
|
|
||||||
Some files were not shown because too many files have changed in this diff Show More
Loading…
Reference in New Issue