Compare commits
77 Commits
main
...
wkt/land_i
| Author | SHA1 | Date |
|---|---|---|
|
|
5f75a37923 | |
|
|
350e2ccd4a | |
|
|
2347ad6198 | |
|
|
e4d9f43ffd | |
|
|
e3f2e9ce2a | |
|
|
e197e90005 | |
|
|
d41b68b80c | |
|
|
0d2c88ce3e | |
|
|
892898cfd0 | |
|
|
05e47d0e33 | |
|
|
ad61d6689a | |
|
|
c1b3e17069 | |
|
|
7cec91e287 | |
|
|
50b84d68f2 | |
|
|
a26d69c269 | |
|
|
8a4a1f67d8 | |
|
|
1b76dd64fd | |
|
|
9851ff886a | |
|
|
9cfe2591d9 | |
|
|
fc532d6acb | |
|
|
61cd084ef2 | |
|
|
13c036d16a | |
|
|
79d7b48fd3 | |
|
|
db3d41a022 | |
|
|
0142476966 | |
|
|
5b3370ddbe | |
|
|
4d3f77f9df | |
|
|
25e63211c6 | |
|
|
d27f953144 | |
|
|
4402d391f0 | |
|
|
29c0fb72b9 | |
|
|
3e3e0fdd46 | |
|
|
b349332e6b | |
|
|
5183d4094f | |
|
|
251987fc4b | |
|
|
63c763800a | |
|
|
7a06a620e2 | |
|
|
3053324f04 | |
|
|
cb5e81d443 | |
|
|
51795a4e07 | |
|
|
495c091546 | |
|
|
2a826fba17 | |
|
|
6575db6463 | |
|
|
3ec447a45a | |
|
|
11aa2eeb1e | |
|
|
9dde7a9867 | |
|
|
02bb193f87 | |
|
|
6f626a9c45 | |
|
|
72d0bdad5e | |
|
|
dc618229bd | |
|
|
635314c757 | |
|
|
d1571ddfc7 | |
|
|
7bef50b0a5 | |
|
|
e8e0730a50 | |
|
|
4133f07340 | |
|
|
507c75c669 | |
|
|
1fa45dfa98 | |
|
|
bde2457bdd | |
|
|
1ba9bdbac0 | |
|
|
d86f29d416 | |
|
|
69a6edd6f8 | |
|
|
bbe9deaa62 | |
|
|
70e8d60326 | |
|
|
2c007d3ada | |
|
|
1fde12d3fd | |
|
|
c347ab08ea | |
|
|
c5155ce814 | |
|
|
d7cefa71f3 | |
|
|
20bfbc2380 | |
|
|
d7f956a566 | |
|
|
d26286f6cc | |
|
|
22f166a5c6 | |
|
|
6c8709d309 | |
|
|
23632d2153 | |
|
|
d85e57334c | |
|
|
6c8d17da75 | |
|
|
1976f6a5de |
|
|
@ -47,7 +47,7 @@ that's why it's like 1000x faster ya know?"
|
|||
- Wrap all code symbols: `function()`,
|
||||
`ClassName`, `field_name`
|
||||
- File paths: `piker/ui/_remote_ctl.py`
|
||||
- Commands: `git status`, `piker store ldshm`
|
||||
- Commands: `git status`, `piker store shm`
|
||||
|
||||
**Explain like you're pair programming:**
|
||||
```
|
||||
|
|
|
|||
|
|
@ -0,0 +1,143 @@
|
|||
# Split `brokerd.<broker>` into trading-only `brokerd` + new `datad.<broker>`
|
||||
|
||||
## Context
|
||||
|
||||
Today a single `brokerd.<broker>` actor hosts BOTH concerns:
|
||||
|
||||
- **data feed service tasks**: the `_FeedsBus` + `open_feed_bus()` ep (`piker/data/feed.py:464`), per-symbol `allocate_persistent_feed()` tasks (shm writers via `sample_and_broadcast()`, history backfill via `piker.tsp`), plus backend eps `stream_quotes`, `open_history_client`, `open_symbol_search`, `get_mkt_info` from `piker/brokers/<backend>/feed.py`/`symbols.py`,
|
||||
- **live order-control tasks**: `open_trade_dialog()` from `piker/brokers/<backend>/broker.py`, driven by `emsd`.
|
||||
|
||||
The codebase already anticipates this split: `piker/data/validate.py:70-91` groups backend eps into `'datad'` vs `'brokerd'` kinds, `piker/brokers/ib/__init__.py:62-70` already declares `_brokerd_mods`/`_datad_mods`, and `piker/brokers/_daemon.py:62` carries the literal TODO *"rename the daemon to datad prolly once we split up broker vs. data tasks into separate actors?"*. This work executes that split.
|
||||
|
||||
**User-decided constraints:**
|
||||
|
||||
- `datad.<broker>` is a **sibling** of `brokerd.<broker>` under `pikerd`, spawned via the existing `Services` + `maybe_spawn_daemon()` machinery (`piker/service/_daemon.py:46`).
|
||||
- **Hard cutover, staged by layer** — no dual-mode runtime flag; every stage lands fully working.
|
||||
- Post-split `brokerd` is **trading-only and lazily spawned solely by emsd's** `open_brokerd_dialog` path; UI/CLI/feed code never spawns it. Chart-only and paper sessions run with **zero** brokerd processes.
|
||||
|
||||
## Target topology
|
||||
|
||||
```
|
||||
pikerd
|
||||
├── datad.ib feed bus, shm writers, tsp history, symbol search
|
||||
├── brokerd.ib open_trade_dialog only (EMS-spawned, lazy)
|
||||
├── emsd
|
||||
│ └── paperboi.ib (paper mode; opens its feed via datad)
|
||||
└── samplerd
|
||||
```
|
||||
|
||||
## Key verified facts (load-bearing)
|
||||
|
||||
- `feed.portals` has exactly ONE trading consumer: `piker/clearing/_ems.py:671` (`portal = feed.portals[brokermod]` → handed to `open_brokerd_dialog`). This is the single coupling forcing feed + trading into one actor.
|
||||
- `assert 'brokerd' in servicename` at `piker/data/feed.py:502` is the only actor-name assert in the tree.
|
||||
- `piker ledger` (`piker/accounting/cli.py:100-157`) is a hidden consumer: it calls `broker_init()` directly, spawns its own ad-hoc actor, enters `_setup_persistent_brokerd`, then calls `open_brokerd_dialog(brokermod, portal, ...)` **with an explicit portal** — the new signature must keep a `portal:` override param.
|
||||
- kraken's feed→broker coupling is mild: `kraken/broker.py:77-81` imports `NoBsWs`/`open_autorecon_ws` (really from `piker.data._web_bs`) and `stream_messages` (pure parser) from `feed.py`. In-process imports only — `enable_modules` gates RPC, not imports. No live shared state; each actor opens its own WS.
|
||||
- ib: default `client_id=6116` (`ib/api.py:1320`) with linear retry `clientId=client_id + i` (`:1403`). Two ib actors connecting concurrently will collide → needs a role-based id offset. Both daemons need `_spawn_kwargs={'infect_asyncio': True}` (copied by both init fns).
|
||||
- binance has no `symbols.py` (`get_mkt_info`/`open_symbol_search` live in `binance/feed.py`); kucoin/questrade/robinhood are flat single-file with NO `__enable_modules__`; deribit has no `broker.py`.
|
||||
- `tests/test_services.py:80,185` assert `brokerd` actor names incl. the paper-mode flow asserting `brokerd.kraken` spawns (`:229,:237`) — must invert post-split.
|
||||
- `_root_modules` (`piker/service/_actor_runtime.py:156`) must gain the new datad daemon mod so `pikerd_portal.run(spawn_datad, ...)` resolves.
|
||||
- `piker/brokers/core.py:175` `symbol_search` does `portal.run(search_w_brokerd)` where the target fn lives in `piker.brokers.core` → that mod must stay RPC-enabled in datad.
|
||||
|
||||
## Module decision
|
||||
|
||||
New file **`piker/data/_daemon.py`** hosts all datad machinery (`_setup_persistent_datad`, `datad_init`, `spawn_datad`, `maybe_spawn_datad`, `_datad_service_mods`). Mirrors the `piker.brokers._daemon` / `piker.data._sampling` (samplerd) per-subsystem convention and satisfies the existing TODO at `brokers/_daemon.py:49` ("move this def to the `.data` subpkg"). `piker.brokers._daemon` keeps brokerd-only code, slimmed. Do NOT over-DRY the two ~40-line init fns into a shared factory yet (samplerd precedent accepts the duplication).
|
||||
|
||||
---
|
||||
|
||||
## Stage 0 — prep: backend module grouping + ep introspection (no topology change)
|
||||
|
||||
1. `piker/data/validate.py`: add a `get_eps(mod, kind) -> dict[str, Callable]` helper returning the backend's defined eps for a daemon-kind from `_eps` (missing eps excluded). Used later by both init fns + fail-fast checks.
|
||||
2. Declare daemon-kind module groups in each split backend's `__init__.py`, keeping `__enable_modules__` as the (deduped) union so behavior is unchanged:
|
||||
- `kraken`: `_brokerd_mods = ['api', 'broker']`, `_datad_mods = ['api', 'feed', 'symbols']`
|
||||
- `binance`: `_brokerd_mods = ['api', 'broker']`, `_datad_mods = ['api', 'feed']`
|
||||
- `deribit`: `_brokerd_mods = []`, `_datad_mods = ['api', 'feed']`
|
||||
- `ib`: adjust existing groups so each includes `'api'`
|
||||
- flat backends (kucoin etc.): no attrs — init fns fall back to enabling just `modpath` (existing behavior).
|
||||
3. Hygiene: `kraken/broker.py:77-81` imports `NoBsWs`/`open_autorecon_ws` from `piker.data._web_bs` directly (keep the `stream_messages` import from `.feed`).
|
||||
|
||||
**Gate**: full pytest green, zero behavior change.
|
||||
|
||||
## Stage 1 — introduce `datad` daemon machinery (additive)
|
||||
|
||||
New `piker/data/_daemon.py`:
|
||||
|
||||
- `_datad_service_mods: list[str]` — datad-always-enabled mods (successor to the data side of `_data_mods` from `brokers/_daemon.py:52`): `['piker.brokers.core', 'piker.brokers.data', 'piker.data', 'piker.data.feed', 'piker.data._sampling', 'piker.data._daemon']`.
|
||||
- `_setup_persistent_datad(ctx, brokername, loglevel)` — `@tractor.context` fixture; logging boilerplate (as `_setup_persistent_brokerd:81-88`), then allocates the actor-global feed bus exactly as the brokerd fixture does today (`brokers/_daemon.py:105-121`): `assert not feed._bus`, open service nursery, `feed.get_feed_bus(brokername, service_nursery)`, `ctx.started()`, `sleep_forever()`.
|
||||
- `datad_init(brokername, ...)` — mirrors `broker_init()` (`brokers/_daemon.py:132`): actor name `f'datad.{brokername}'`, copies backend `_spawn_kwargs` (**critical for ib infect_asyncio**), builds `enable_modules` from `getattr(brokermod, '_datad_mods', getattr(brokermod, '__enable_modules__', []))`.
|
||||
- `spawn_datad(brokername, ...)` — mirrors `spawn_brokerd()` (`brokers/_daemon.py:202`): `Services.actor_n.start_actor(dname, enable_modules=_datad_service_mods + backend_mods, ...)` + `Services.start_service_task(dname, portal, _setup_persistent_datad, ...)`.
|
||||
- `maybe_spawn_datad(brokername, ...)` — wraps `maybe_spawn_daemon(service_name=f'datad.{brokername}', service_task_target=spawn_datad, ...)` exactly like `maybe_spawn_brokerd` (`brokers/_daemon.py:256`).
|
||||
|
||||
Supporting edits:
|
||||
|
||||
- `piker/service/_actor_runtime.py:156`: add `'piker.data._daemon'` to `_root_modules` (keep `'piker.brokers._daemon'`).
|
||||
- `piker/service/__init__.py`: re-export `spawn_datad`/`maybe_spawn_datad` next to the brokerd ones (`:56-57`).
|
||||
- `tests/test_services.py`: new `test_datad_spawn` — `open_test_pikerd` + `maybe_spawn_datad('kraken')` + `ensure_service('datad.kraken')`. (Do NOT route a feed through it yet — `open_feed_bus`'s assert still says brokerd.)
|
||||
|
||||
**Gate**: suite green + new spawn test; nothing routes through datad yet.
|
||||
|
||||
## Stage 2 — clearing layer: emsd self-spawns brokerd (decouple from `feed.portals`)
|
||||
|
||||
Sequenced BEFORE the feed cutover so live trading works at every boundary (otherwise `feed.portals` would hand emsd a datad portal).
|
||||
|
||||
`piker/clearing/_ems.py`:
|
||||
|
||||
1. `open_brokerd_dialog()` (`:336`) new signature: `(brokermod, exec_mode, fqme=None, portal: tractor.Portal|None = None, loglevel=None)`. Internals: keep trades-ep detection (`:400-417`); acquire the brokerd actor ONLY when a live ep will actually open, via a small inner `@acm _acquire_live_portal()` that yields the passed `portal` if given (the `piker ledger` path) else `async with maybe_spawn_brokerd(brokermod.name, loglevel=loglevel)` — **the single place brokerd boots post-split**. Move the eager `portal.open_context(trades_endpoint, ...)` construction (`:425`) inside that block; the paper short-circuit (`:432`) never touches it.
|
||||
2. `Router.maybe_open_brokerd_dialog()` (`:581`) — drop the `portal` param; `Router.open_trade_relays()` (`:640`) — delete `portal = feed.portals[brokermod]` (`:671`) and the pass-through (`:685`).
|
||||
3. `piker/accounting/cli.py:144`: switch to keyword form `open_brokerd_dialog(brokermod, exec_mode=..., portal=portal, loglevel=...)`.
|
||||
|
||||
Pre-cutover this is a pure refactor: emsd's `maybe_spawn_brokerd` just *finds* the already-running brokerd via the registry.
|
||||
|
||||
**Gate**: `tests/test_ems.py` + services suite green; manual paper order on kraken; `piker ledger` smoke.
|
||||
|
||||
## Stage 3 — feed layer hard cutover to datad
|
||||
|
||||
1. `piker/data/feed.py`: import `maybe_spawn_datad` (replacing `maybe_spawn_brokerd`, `:56-58`); `open_feed()` `:895` → `maybe_spawn_datad(...)` (rename `brokerd_ctxs` → `datad_ctxs`); `open_feed_bus()` `:502` → `assert 'datad' in servicename`; comment sweep.
|
||||
2. `piker/brokers/_daemon.py` `_setup_persistent_brokerd()`: slim to trading-only fixture — logging setup, `ctx.started()`, `sleep_forever()`. Drop the bus alloc, `assert not feed._bus`, the service nursery (backend `open_trade_dialog` ctxs own their task trees), the `eg.ExceptionGroup` handler, and the stale `_FeedsBus` TYPE_CHECKING import. (`piker ledger`'s ad-hoc actor enters this same slimmed fixture — exactly what it needs.)
|
||||
3. Repoint remaining data-flavored spawn sites `maybe_spawn_brokerd` → `maybe_spawn_datad`:
|
||||
- `piker/ui/_app.py:40,57` (symbol search; optionally rename `install_brokerd_search` → `install_datad_search`, def at `piker/data/feed.py:766`)
|
||||
- `piker/brokers/core.py:33,175` (`symbol_search`)
|
||||
- `piker/brokers/cli.py:38,190,320` (`brokercheck`, `record`)
|
||||
- `piker/ui/cli.py:30,72,121` (legacy kivy monitor/optschain) + best-effort `piker/ui/kivy/option_chain.py:498` `wait_for_actor('brokerd')` → `'datad'`
|
||||
4. `tests/test_services.py`: `:80,:185` `actor_name = 'datad'`; paper-mode test asserts `datad.kraken` + `paperboi.kraken` + `emsd` AND adds the headline negative check: `assert 'brokerd.kraken' not in services.service_tasks`.
|
||||
|
||||
**Gate**: full suite with inverted assertions; manual: chart on binance/kraken shows `datad.<broker>` in `piker services` and NO brokerd; paper order fills; symbol search works; history backfill sane.
|
||||
|
||||
## Stage 4 — caps-sec slimming, validation, ib client-id
|
||||
|
||||
1. `piker/brokers/_daemon.py`: delete `_data_mods` (`:52`); `spawn_brokerd()` `:236` → `enable_modules=tractor_kwargs.pop('enable_modules')`; `broker_init()` `:184` reads `getattr(brokermod, '_brokerd_mods', getattr(brokermod, '__enable_modules__', []))`. Resulting brokerd enable set has no `piker.data.*` at all.
|
||||
2. Fail-fast ep validation via Stage-0 `validate.get_eps()`: `broker_init` raises a clear error when `get_eps(mod, 'brokerd')` is empty (e.g. "kucoin is datad-only — use paper mode"); `datad_init` warns analogously.
|
||||
3. ib client-id mitigation in `load_aio_clients()` (`ib/api.py:~1316`): role-based offset when `client_id` is the 6116 default (e.g. `+16` when `'datad' in tractor.current_actor().name`), optionally configurable via `brokers.toml` keys.
|
||||
4. Docs/comment sweep: resolve `brokers/_daemon.py:62` TODO, `piker/cli/__init__.py:253` TODO, daemon list in `piker/service/__init__.py:21` docstring.
|
||||
|
||||
**Gate**: full suite; audit greps clean: `grep -rn maybe_spawn_brokerd piker/` hits only `clearing/_ems.py`, `service/__init__.py`, `brokers/_daemon.py`; no spawn-path `'brokerd'` literals left in `piker/data`, `piker/ui`, `piker/brokers/cli.py`.
|
||||
|
||||
---
|
||||
|
||||
## Risk register
|
||||
|
||||
| Risk | Sev | Mitigation |
|
||||
|---|---|---|
|
||||
| ib: `datad.ib` + `brokerd.ib` both connect to TWS/gw; `client_id=6116` collision burns `connect_timeout` retries | High (live ib only) | Stage 4 role-based id offset; both daemons inherit `infect_asyncio` via `_spawn_kwargs` copy in both init fns |
|
||||
| `piker ledger` ad-hoc actor path silently broken by signature change | Med | `portal:` override kept on `open_brokerd_dialog` (Stage 2); explicit smoke test |
|
||||
| datad-only backends (kucoin/deribit) → useless brokerd spawn | Low | already covered: `open_brokerd_dialog` forces `exec_mode='paper'` when no trades ep (`_ems.py:413-417`) BEFORE the lazy spawn; Stage 4 fail-fast |
|
||||
| brokerd in-process symbology/ledger needs (ib `broker.py` imports `.symbols`; kraken uses `stream_messages`) | None | verified pure in-process imports; `enable_modules` gates RPC only |
|
||||
| paper-mode test asserts brokerd spawns (`test_services.py:237`) | Low | Stage 3 deliberately inverts it |
|
||||
| doubled per-broker clients (HTTP/WS, symcache loads) | Low | each actor already opens its own WS today; symcache is disk-read-mostly |
|
||||
|
||||
## Verification (per stage gates above, plus end-to-end)
|
||||
|
||||
- `pytest tests/test_services.py tests/test_feeds.py tests/test_ems.py` at every stage (kraken/binance public endpoints, no creds needed).
|
||||
- Manual smoke matrix post-Stage-3: `pikerd -l info` + `piker chart btcusdt.spot.binance` → `piker services` shows `datad.binance`, no brokerd; submit paper order → `paperboi.binance` appears, still no brokerd; symbol search; `piker ledger <broker>.paper`.
|
||||
- If ib creds/gateway available: live ib chart + small live order to exercise dual-actor client-id path (Stage 4).
|
||||
|
||||
## Critical files
|
||||
|
||||
- **new** `piker/data/_daemon.py` — all datad machinery
|
||||
- `piker/brokers/_daemon.py` — slimmed brokerd fixture/init/spawn
|
||||
- `piker/data/feed.py` — spawn cutover + actor-name assert
|
||||
- `piker/clearing/_ems.py` — emsd lazy brokerd spawn (`open_brokerd_dialog`, `Router`)
|
||||
- `piker/service/_actor_runtime.py`, `piker/service/__init__.py` — root mods + re-exports
|
||||
- `piker/data/validate.py` — `get_eps()` helper
|
||||
- backend `__init__.py`s (kraken/binance/deribit/ib) — `_datad_mods`/`_brokerd_mods`
|
||||
- `piker/accounting/cli.py` — keyword-form dialog call
|
||||
- `tests/test_services.py` — inverted + new assertions
|
||||
|
|
@ -0,0 +1,233 @@
|
|||
# Bounded IB recovery candidate
|
||||
|
||||
Prepared 2026-10-22 on `wkt/land_ib_recovery`, in the main checkout
|
||||
`/home/goodboy/repos/piker`. Base:
|
||||
`dc618229bd8c9c90aa3160e26677b553a7d472c7`.
|
||||
Changes are unstaged and uncommitted. No acceptance is implied.
|
||||
|
||||
## Patch selection and provenance
|
||||
|
||||
Keep these existing landing commits:
|
||||
|
||||
| Commit | Reason |
|
||||
| --- | --- |
|
||||
| `4133f073` and its ancestors | Packaging, Nix, runtime API and pin work |
|
||||
| `e8e0730a` | Correlate MethodProxy requests and responses |
|
||||
| `7bef50b0` | One response reader, per-request waiters |
|
||||
| `d1571ddf` | Propagate daemon-handler exceptions |
|
||||
| `dc618229` | Isolate cached feed stream consumers |
|
||||
|
||||
Proposed separate implementation boundaries, in this order:
|
||||
|
||||
1. Back out the code and original test from
|
||||
`635314c757c74a1cd95c5cf3dc7dfd03be984b00`.
|
||||
This restores the direct-owner `_quote_streams` implementation.
|
||||
Retain the historical prompt records. Replace the removed
|
||||
mocked-cache regression with the real-bridge restart reproducer.
|
||||
This backout is not a claim that arbitrary concurrent direct
|
||||
callers have safe broadcast ownership.
|
||||
2. Backport **only** `piker/brokers/ib/feed.py` from
|
||||
`0846cbd4a4197e47415689cf5e9d146ba3174c32`.
|
||||
Retain its seven history-adapter regressions in
|
||||
`tests/test_ib_history.py`. This includes reset serialization,
|
||||
completion/result signalling and bounded cancellation/failure.
|
||||
It does not include the original SHM/storage orchestration patch.
|
||||
3. Backport **only** `piker/brokers/ib/feed.py` from
|
||||
`1da9b760a3bc6b96501110cab2b5d09846d42a1d`.
|
||||
Retain `test_empty_history_preserves_request_boundary` and its
|
||||
six cases. This preserves no-data request boundaries and wakes
|
||||
the waiter. It does not include the original history-layer
|
||||
endpoint/filtering changes. Add the real-bridge history recovery
|
||||
case after these adapter changes are present.
|
||||
4. Import the code and original regression from
|
||||
`12b0d66e854247e92cb5a8668be71d816c72d856`.
|
||||
Add a separate real-broadcaster/production-loop regression.
|
||||
5. Remove the single-null-sample debugger stop in `get_null_segs()`.
|
||||
This is a local UX cleanup retaining the existing `None` result.
|
||||
It does not backport the null-grouping/deduplication rewrite in
|
||||
`976db6c33c4b8ed07eee3047226417d3928d9cca`.
|
||||
6. Record cross-boundary regression results and live qualification
|
||||
evidence, distinguishing the known callback failure from passes.
|
||||
|
||||
All three imported source commits were authored by
|
||||
`goodboy <goodboy_foss@protonmail.com>`; preserve that authorship
|
||||
and the full source SHA when committing each partial backport.
|
||||
Use explicit partial-backport wording, not a whole-commit cherry
|
||||
claim. New real-runtime tests are this session's additional work.
|
||||
|
||||
The current aggregate working diff contains these boundaries in
|
||||
the same feed file. Staging that entire file would combine the
|
||||
first three boundaries. Reconstruct/stage the original per-source
|
||||
patches in the above order when preparing commits.
|
||||
|
||||
## Quote-cache history correction
|
||||
|
||||
`4af885d90f6e4bcfa963f2f7a1d40436d3c928f3` and landing commit
|
||||
`635314c757c74a1cd95c5cf3dc7dfd03be984b00` have the same stable
|
||||
patch ID: `bc54892c25e957be112b135bb9e383ec4dd123c5`.
|
||||
They are equivalent replays, not different implementations.
|
||||
|
||||
`wkt/ib_downstream_pin_eval@e929fdc2` lacks that quote-cache
|
||||
change and retains direct ownership. The original qualification
|
||||
did not validate the combination assembled for the landing.
|
||||
|
||||
After this candidate's edits, `piker/brokers/ib/feed.py` matches
|
||||
the source at `1da9b760` byte-for-byte. Compared with the evaluated
|
||||
tip, only the later schedule/reservation APIs are absent.
|
||||
The EMS code patch has the same stable patch ID as the original
|
||||
`12b0d66e` EMS code patch:
|
||||
`50fee047e94fd67c8d9feab66859fbe3ab25b64c`.
|
||||
|
||||
## Real-runtime evidence
|
||||
|
||||
`tests/test_ib_quote_restart.py` uses real TCP Tractor actors,
|
||||
an asyncio-infected child, the installed pinned Tractor bridge,
|
||||
production Piker quote setup, and real IB/eventkit ticker events.
|
||||
Only the external IB connection/input is replaced with replay data.
|
||||
|
||||
Before the quote-cache backout, cancellation after ticker delivery
|
||||
raised `RemoteActorError[TrioTaskExited]` through
|
||||
`maybe_open_context()` / `_Cache.run_ctx` teardown, matching the
|
||||
reported fatal path.
|
||||
|
||||
After the backout:
|
||||
|
||||
- Two quote generations finish; each subscription is cancelled,
|
||||
each asyncio producer is done and each ticker handler removed.
|
||||
- A second RPC repeats the sequence in the same actor successfully.
|
||||
- The strict callback-cleanliness case still fails: Tractor's
|
||||
`signal_trio_when_done()` raises `TrioTaskExited` into asyncio's
|
||||
exception handler even though the Trio consumer recovers.
|
||||
|
||||
The history integration case keeps `Client.bars()`, MethodProxy,
|
||||
both relays, `get_bars()`, `wait_on_data_reset()` and
|
||||
`data_reset_hack()` real. It replaces external contract discovery,
|
||||
IB history replies and VNC input. A real IB error event announces
|
||||
farm recovery. The replay rejects retries before recovery and
|
||||
proves one reset followed by completion for both 1s and 60s data.
|
||||
|
||||
The new EMS case runs the production dark-clearing loop over a
|
||||
real Tractor broadcaster: both EMS and a sibling receive a tick,
|
||||
EMS cancellation removes only its subscription, and the sibling
|
||||
continues with an unchanged strict lag policy.
|
||||
|
||||
## Verification
|
||||
|
||||
Unique selected cases: **38 passed, 1 failed**.
|
||||
The failure is intentionally visible, not skipped or xfailed.
|
||||
|
||||
Run from the checkout's working Nix environment:
|
||||
|
||||
```xsh
|
||||
timeout -k 5 45 py313/bin/python -m pytest -p no:xonsh -q -x --tb=short tests/test_ib_history.py tests/test_ems_broadcast.py tests/test_ib_method_proxy.py tests/test_feed_broadcast.py
|
||||
timeout -k 5 45 py313/bin/python -m pytest -p no:xonsh -q --tb=short tests/test_ib_quote_restart.py
|
||||
timeout -k 5 45 py313/bin/python -m pytest -p no:xonsh -q -x --tb=short tests/test_logging.py tests/test_startup.py
|
||||
timeout -k 5 45 py313/bin/python -m pytest -p no:xonsh -q -x --tb=short tests/test_services.py::test_runtime_boot
|
||||
git diff --check
|
||||
```
|
||||
|
||||
The adapter/proxy/feed/EMS selection initially passed 21 cases;
|
||||
the added real EMS case then passed with its imported case.
|
||||
The bridge file passed two cases and failed callback cleanliness.
|
||||
Logging/startup passed 13 cases; runtime boot passed one with the
|
||||
existing `Registrar.uid` deprecation warning.
|
||||
|
||||
The agent shell inherited another project's environment and lacked
|
||||
native libraries. Tests used the existing Piker `py313` via
|
||||
`/tmp/nix-shell.zfKbeB/opencode/piker_test_env.py`, which supplies
|
||||
libraries from the pinned flake's existing Qt dependency closure,
|
||||
excluding Nix Qt itself and glibc. No environment sync or shellHook
|
||||
ran. Piker resolved to this checkout; Tractor resolved to its
|
||||
installed package in `py313`.
|
||||
|
||||
Workflow-state preparation initially reported changed legacy
|
||||
files/modes after migration. On 2026-10-23 the human authorized
|
||||
reconciliation. Inspection found only two missing legacy guidance
|
||||
snapshots; the neutral copies matched their saved hashes and modes.
|
||||
Restoring those local snapshots resolved preparation without
|
||||
changing archived messages or the original migration receipt.
|
||||
|
||||
## Human live qualification, 2026-10-23
|
||||
|
||||
The human ran these journeys with the installed Tractor pin,
|
||||
the existing IB Gateway and pre-existing NativeDB. The agent did
|
||||
not operate the broker connection, VNC, orders or database.
|
||||
|
||||
- Initial QQQ startup/backfill recovered after pacing error 162
|
||||
with one reset attempt, then displayed live updates. The reset
|
||||
waiter appears to have timed out; the log did not establish an
|
||||
acknowledged farm reset or live quote-stream restart.
|
||||
- Sequential chart exits and reattachment preserved all four
|
||||
daemon PIDs; a read-only process/listener check confirmed this.
|
||||
- The overlapping two-chart journey used `pikerd@961108`,
|
||||
`brokerd.ib@964765`, `samplerd@965900` and `emsd@969015`.
|
||||
B started at 08:16:31; C at 08:17:33. B closed at 08:18:11,
|
||||
replacement B started at 08:18:24 and had FSPs up by 08:18:28,
|
||||
then C closed at 08:18:37. The human reported the exercise worked;
|
||||
survivor updates are a manual observation, while timestamps and
|
||||
unchanged daemon identities corroborate overlap and service reuse.
|
||||
- Startup in that journey hit the explicit single-null-sample
|
||||
breakpoint several times before the charts overlapped. The small
|
||||
debugger cleanup above was applied afterward; this live run does
|
||||
not itself verify that later edit.
|
||||
- Departing charts still report cancellation groups and an FSP
|
||||
receiver-close `ClosedResourceError`, followed by the Qt close
|
||||
handler's self-SIGINT. Peer-disconnected final result warnings
|
||||
remain contained in the persistent services.
|
||||
- Daemon SIGINT at 08:19:43 reproduced cached shared-client teardown:
|
||||
`load_clients_for_trio()` / `_Cache.run_ctx` promoted
|
||||
`TrioTaskExited` through `open_trade_dialog()` and entered PDB.
|
||||
After the human continued, all three children were reported
|
||||
exited with status 0 and the root terminated at 08:19:45.
|
||||
Cancellation-only RPC error reporting and late cancel-ack
|
||||
diagnostics also remain.
|
||||
|
||||
The human explicitly accepted deferring this daemon teardown fix
|
||||
from #94 and requested loud attribution for the bounded backports.
|
||||
This is an operational smoke result, not clean-shutdown coverage.
|
||||
|
||||
## Downstream reconciliation requirements
|
||||
|
||||
Before landing #94, the human wants an easy replay of the downstream
|
||||
histories intended to follow it. Rehearse against the agreed #94
|
||||
tree in disposable worktrees, then inspect range-diffs, final code
|
||||
and relevant regressions. This note does not authorize rewriting
|
||||
the original branches or changing acceptance markers.
|
||||
|
||||
- `0846cbd4` and `1da9b760` are partial backports. Their imported
|
||||
adapter hunks may already be present, but their history/SHM/storage
|
||||
changes and omitted integration tests remain necessary downstream.
|
||||
Never drop either whole source commit merely because `feed.py`
|
||||
matches or its adapter hunk becomes empty.
|
||||
- `12b0d66e` has source-equivalent EMS code and its imported test,
|
||||
plus an additional local real-broadcaster test. Preserve that extra
|
||||
regression when resolving test-file overlap.
|
||||
- The quote-cache backout intentionally restores direct ownership
|
||||
for this landing. Reintroducing original `4af885d9` downstream
|
||||
requires qualifying cached-owner teardown with the matching
|
||||
Tractor revision; the backout is not a permanent ownership design.
|
||||
- The local single-null breakpoint removal retains legacy behavior.
|
||||
Prefer the full downstream `976db6c3` algorithm when it is replayed;
|
||||
preserving this `return None` guard would lose that later fix.
|
||||
- Real-bridge tests and manual qualification notes are local work,
|
||||
not evidence that omitted portions of the source commits landed.
|
||||
|
||||
Commit messages must name full source SHAs, original authorship,
|
||||
imported paths/tests, omissions, local additions and replay guidance.
|
||||
Use explicit partial-backport language for adapter subsets; do not
|
||||
claim a full cherry-pick or reuse the source's broader fix claims.
|
||||
|
||||
## Remaining decision
|
||||
|
||||
This candidate removes the reproduced fatal quote-restart
|
||||
propagation and imports existing reset fixes. The human exercised
|
||||
live QQQ startup, backfill and concurrent chart consumer lifetimes.
|
||||
Completion-callback and teardown errors remain; the human accepts
|
||||
them as follow-ups outside the bounded #94 landing.
|
||||
The storage/SHM portions of the downstream fixes are not covered by
|
||||
the adapter backport or its integration tests.
|
||||
|
||||
`ddac3553` is Deribit-specific and omitted. Qt lifecycle commit
|
||||
`1a2d4893` stays a separate follow-up. Further Tractor teardown
|
||||
changes or storage imports require reassessing the bounded landing
|
||||
rather than silently expanding this candidate.
|
||||
|
|
@ -0,0 +1,71 @@
|
|||
# IB recovery qualification, 2026-11-06
|
||||
|
||||
This records human-operated qualification of the bounded #94
|
||||
landing on `wkt/land_ib_recovery`, starting from `495c0915` plus
|
||||
the narrow history catch described below. It supplements the
|
||||
historical evidence in `ai/ib-recovery-candidate.md`.
|
||||
|
||||
## Runtime and dependency source
|
||||
|
||||
The evaluated environment uses Python 3.13, Trio 0.34.0 and Tractor
|
||||
`8fe574c1e447358dafd8ebf5ada3b0714c688f78`. Tractor was an editable
|
||||
local checkout during evaluation. The human subsequently published
|
||||
that exact commit as GitHub's `goodboy/tractor:piker_pin`; a read-only
|
||||
remote query confirmed the published OID before restoring Piker's
|
||||
Git source and recording the same OID in `uv.lock`.
|
||||
|
||||
Offline lock validation resolves 126 packages successfully. No
|
||||
environment sync was performed. This validates the lock metadata;
|
||||
it does not claim a fresh installation from the published source.
|
||||
|
||||
## Manual observations
|
||||
|
||||
The human operated Gateway, chart windows and paper orders. The
|
||||
agent did not operate live services or the account.
|
||||
|
||||
- Same-feed overlap, close/reopen and paper order submit/cancel
|
||||
worked after the existing order-dialog guard was imported.
|
||||
- Different-feed overlap originally exposed a timed-out one-second
|
||||
MNQ March 2027 gap query. PDB showed `timedout=True` and
|
||||
`out is None`. Its uncaught `NoData` canceled sibling history and
|
||||
quote tasks even though `pikerd` remained alive.
|
||||
- After the narrow catch, the human reported the overlap journeys
|
||||
working, then confirmed cases 5 and 6 with Ctrl+Alt+F: overlapping
|
||||
live feeds survived the Gateway data reset; closing all charts
|
||||
and reopening with `pikerd` left running restored live updates.
|
||||
- Earlier Ctrl+Shift+R attempts left feeds live, but do not prove
|
||||
that the intended Gateway data reset was triggered.
|
||||
|
||||
These are manual observations, not automated timing or latency
|
||||
measurements. Ctrl+Alt+R was not qualified here; Piker's utility
|
||||
labels it a connection reset.
|
||||
|
||||
## Narrow history backport
|
||||
|
||||
Only the `NoData`/`DataUnavailable` catch around `get_hist()` in
|
||||
`maybe_fill_null_segments()` is copied from
|
||||
`0846cbd4a4197e47415689cf5e9d146ba3174c32`, authored by
|
||||
`goodboy <goodboy_foss@protonmail.com>`. The existing gap fallback
|
||||
remains in place; this does not import broader history repair.
|
||||
|
||||
The new regression uses real null detection and checks that a
|
||||
sibling in the same nursery still runs after either provider
|
||||
exception. Both cases failed before the catch. With it, the two
|
||||
focused history files passed: 17 tests. Earlier bounded-recovery
|
||||
and Tractor qualification results remain separate evidence, not
|
||||
fresh test runs of the final dependency commit tree.
|
||||
|
||||
## Remaining recovery limits
|
||||
|
||||
A full Gateway restart was triggered accidentally through its
|
||||
auto-restart settings dialog. The disconnect delivered the string
|
||||
`Peer closed connection.` to `deliver_trade_events()`. The string
|
||||
branch then reached `err['reqid']`, raising `TypeError`; its `code`
|
||||
local can also retain an earlier event's value. Full Gateway restart
|
||||
recovery is outside this bounded landing.
|
||||
|
||||
The earlier history-failure teardown also exposed a separate
|
||||
Tractor completion-callback assertion after `BrokenResourceError`.
|
||||
The history catch does not establish that this callback race is
|
||||
fixed. Broader history recovery and these disconnect paths remain
|
||||
downstream work.
|
||||
|
|
@ -0,0 +1,52 @@
|
|||
---
|
||||
created_at: 2026-10-07T16:54:53Z
|
||||
generated_by:
|
||||
harness: opencode
|
||||
provider: openai
|
||||
model: gpt-6.1-sol
|
||||
git_ref: bde2457b
|
||||
scope: code
|
||||
substantive: true
|
||||
raw_file: 20261007T165453Z_bde2457b_prompt_io.raw.md
|
||||
---
|
||||
|
||||
## Prompt
|
||||
|
||||
The user requested fixing all seven local PR94 review findings and
|
||||
exporting the review for human inspection in Tuicr. They asked that a
|
||||
Tractor pin adjustment be considered before adding Piker workarounds.
|
||||
|
||||
## Response summary
|
||||
|
||||
Correct the endpoint defaults, Click argument shape, daemon binding and
|
||||
registrar option handling, chart fallback, and stray debugger pause.
|
||||
Keep the absent datad implementation's test outside this landing
|
||||
branch. Add offline startup coverage and export original findings
|
||||
without modifying review states. The checked newer Tractor address
|
||||
branch still rejects mixed transports, so leave the dependency pin.
|
||||
|
||||
## Files changed
|
||||
|
||||
- `config/conf.toml`: TCP-only default endpoints.
|
||||
- `piker/cli/__init__.py`: repeatable and intact multiaddresses.
|
||||
- `piker/service/_actor_runtime.py`: validate daemon listeners.
|
||||
- `piker/service/_daemon.py`: initialize and consume registrar options.
|
||||
- `piker/ui/cli.py`: fallback to automatic chart binding.
|
||||
- `piker/brokers/binance/feed.py`: remove unconditional debugger entry.
|
||||
- `tests/test_services.py`: defer test requiring missing datad API.
|
||||
- `tests/test_startup.py`: CLI/config and service lifecycle regressions.
|
||||
|
||||
## Human edits
|
||||
|
||||
The human identified that datad tests arrived through the agent's
|
||||
mixed cherry-picks and explicitly requested deferring that test. They
|
||||
requested a chart default for existing configs, expanded the task to
|
||||
all review findings, and directed checking whether a newer Tractor pin
|
||||
could reduce the fix scope. No direct manual source edits were observed
|
||||
during these changes; acceptance remains with the human.
|
||||
|
||||
During the human review pass, the human confirmed removing the Binance
|
||||
pause and correcting CLI endpoint parsing. They requested an explicit
|
||||
template note about this pin's single-transport limitation and detailed
|
||||
comments distinguishing registrar discovery endpoints from daemon bind
|
||||
endpoints. Those explanations were added at the human's direction.
|
||||
|
|
@ -0,0 +1,61 @@
|
|||
---
|
||||
created_at: 2026-10-07T16:54:53Z
|
||||
generated_by:
|
||||
harness: opencode
|
||||
provider: openai
|
||||
model: gpt-6.1-sol
|
||||
git_ref: bde2457b
|
||||
diff_cmd: git diff HEAD~1..HEAD
|
||||
---
|
||||
|
||||
## Prompt
|
||||
|
||||
Fix all findings from the local PR94 review, prefer a Tractor pin
|
||||
change if it reduces work, and export the original review for Tuicr.
|
||||
|
||||
## Generated changes
|
||||
|
||||
> `git diff HEAD~1..HEAD -- config/conf.toml`
|
||||
|
||||
Use TCP-only defaults, retaining commented UDS alternatives. Both
|
||||
checked Tractor address branch tips still reject mixed transports.
|
||||
|
||||
> `git diff HEAD~1..HEAD -- piker/cli/__init__.py`
|
||||
|
||||
Make Click multiaddresses repeatable and normalize single strings.
|
||||
|
||||
> `git diff HEAD~1..HEAD -- piker/service/_actor_runtime.py`
|
||||
|
||||
Validate requested daemon bind addresses instead of remote registrars.
|
||||
Normalize accepted endpoints and allow TCP ephemeral port requests.
|
||||
|
||||
> `git diff HEAD~1..HEAD -- piker/service/_daemon.py`
|
||||
|
||||
Resolve explicit or inherited registrars independently of other daemon
|
||||
options and consume the explicit keyword before forwarding kwargs.
|
||||
|
||||
> `git diff HEAD~1..HEAD -- piker/ui/cli.py`
|
||||
|
||||
Default chart bind endpoints to None when network config is absent.
|
||||
|
||||
> `git diff HEAD~1..HEAD -- piker/brokers/binance/feed.py`
|
||||
|
||||
Remove the unconditional market-info debugger pause.
|
||||
|
||||
> `git diff HEAD~1..HEAD -- tests/test_services.py tests/test_startup.py`
|
||||
|
||||
Remove the prematurely cherry-picked datad test and add offline startup
|
||||
regressions, including a real registrar/child daemon lifecycle check.
|
||||
|
||||
## Verification
|
||||
|
||||
- Startup and logging tests: 13 passed.
|
||||
- Existing runtime boot and tagged registrar tests: 2 passed.
|
||||
- Scoped native Ruff and diff checks pass.
|
||||
- The first new actor test used a removed nursery API; it was corrected
|
||||
to the installed start_actor/Portal.run interface before passing.
|
||||
- Import initially failed without native libraries. Tests use the
|
||||
existing flake's GCC, zlib, Qt and PyQt paths without executing its
|
||||
environment-rebuilding shell hook.
|
||||
- Original seven findings exported locally as Tuicr review-add payloads
|
||||
bound to bde2457b; no session comments or remote PR were changed.
|
||||
|
|
@ -0,0 +1,45 @@
|
|||
---
|
||||
created_at: 2026-10-10T23:28:52Z
|
||||
generated_by:
|
||||
harness: opencode
|
||||
provider: openai
|
||||
model: openai/gpt-6.1-sol
|
||||
git_ref: 1fa45dfa
|
||||
scope: config
|
||||
substantive: true
|
||||
raw_file: 20261010T232852Z_1fa45dfa_prompt_io.raw.md
|
||||
---
|
||||
|
||||
## Prompt
|
||||
|
||||
The human rejected keeping the April Tractor pin as a landing baseline.
|
||||
They requested the reviewed `wkt/addr_unpacking` branch, removal of the
|
||||
new rollback adaptations, reuse of original Piker histories where
|
||||
possible, and a candidate prepared within their gateway relocation
|
||||
window. They asked specifically whether the runtime validation review
|
||||
hunk duplicated older endpoint history.
|
||||
|
||||
## Response summary
|
||||
|
||||
Select the reviewed address branch at an immutable locked revision and
|
||||
preserve all committed application/test compatibility code. Verify the
|
||||
minimal dependency boundary offline and preview existing consumer
|
||||
fixes without importing unrelated datad/backfill changes. Distinguish
|
||||
original-source test results from live teardown acceptance, which is
|
||||
not available in this preparation window.
|
||||
|
||||
## Files changed
|
||||
|
||||
- `pyproject.toml`: select Tractor `wkt/addr_unpacking`.
|
||||
- `uv.lock`: resolve reviewed `a966928d` and setproctitle 1.3.8 while
|
||||
preserving Polars 1.36.0 and the rest of the locked graph.
|
||||
|
||||
## Human edits
|
||||
|
||||
The human redirected the task from an older known-good runtime to a
|
||||
modern landing stack and insisted on reusing original history rather
|
||||
than building new compatibility adapters. They proposed stricter
|
||||
exception-group behavior as a hypothesis and set the live-test/time
|
||||
constraints. Those directions materially changed the dependency
|
||||
boundary. No direct manual edits to this final boundary were observed;
|
||||
runtime acceptance and landing remain human-owned.
|
||||
|
|
@ -0,0 +1,52 @@
|
|||
---
|
||||
created_at: 2026-10-10T23:28:52Z
|
||||
generated_by:
|
||||
harness: opencode
|
||||
provider: openai
|
||||
model: openai/gpt-6.1-sol
|
||||
git_ref: 1fa45dfa
|
||||
diff_cmd: git diff HEAD~1..HEAD
|
||||
---
|
||||
|
||||
## Prompt
|
||||
|
||||
Undo the April Tractor rollback and select the reviewed
|
||||
`wkt/addr_unpacking` branch. Prefer original downstream compatibility
|
||||
commits to new adaptations, and prepare an offline-verified candidate
|
||||
while live IB verification is deferred for gateway relocation.
|
||||
|
||||
## Generated changes
|
||||
|
||||
> `git diff HEAD~1..HEAD -- pyproject.toml uv.lock`
|
||||
|
||||
Change Tractor's source from `piker_pin` to `wkt/addr_unpacking` and
|
||||
lock it at `a966928dead4cbf369ad244834ce9dbe988ac8ca`. Restore application
|
||||
and test sources byte-for-byte to the committed PR94 tree. Keep the
|
||||
rejected rollback journal only in ignored local experiment reports.
|
||||
Preserve Polars 1.36.0 by retaining its exact existing lock blocks;
|
||||
setproctitle re-resolves from 1.3.7 to 1.3.8. `uv sync --locked` accepts
|
||||
the resulting graph. Actual imports confirm the reviewed Tractor pin
|
||||
and Trio 0.32.0, not the hypothesized Trio 0.33.
|
||||
|
||||
## Original-history previews
|
||||
|
||||
The existing MethodProxy stack is retained. Read-only merge previews
|
||||
show original `4af885d9` and `b691422b` can be applied without the datad
|
||||
cutover. Original-source regressions run through temporary in-memory
|
||||
overlays against the real pinned runtime: seven cases pass and one
|
||||
cache-hit case fails because `log.at_least_level('info')` looks up INFO
|
||||
in a custom-level mapping. A separate standard logging correction
|
||||
produces eight passing cases. Its patch and execution handoff are saved
|
||||
under ignored `.ai/code-review/reports/pr94_modern_stack/`; no consumer
|
||||
cherry-picks or source edits are performed by this boundary.
|
||||
|
||||
## Verification
|
||||
|
||||
- Startup and logging: 13 passed.
|
||||
- Runtime boot and tagged registrar normalization: 2 passed.
|
||||
- Locked sync and source whitespace checks pass.
|
||||
- Original application/test trees are unchanged from PR94 HEAD.
|
||||
- Consumer overlay with explicit logging correction: 8 passed.
|
||||
- Correction patch validates against its exact preview tree using a
|
||||
private temporary index; the real index is untouched.
|
||||
- Live IB startup, SIGINT and surviving-pikerd reattachment are pending.
|
||||
|
|
@ -0,0 +1,45 @@
|
|||
---
|
||||
created_at: 2026-11-06T18:56:28.621577+00:00
|
||||
generated_by:
|
||||
harness: codex
|
||||
provider: OpenAI
|
||||
git_ref: 495c0915
|
||||
scope: code
|
||||
substantive: true
|
||||
raw_file: 20261106T185628Z_495c0915_215e9b_prompt_io.raw.md
|
||||
---
|
||||
|
||||
## Prompt
|
||||
|
||||
User reported a different-feed overlap failure and asked whether
|
||||
a fix could be extracted without importing broader history work.
|
||||
PDB confirmed MNQ March 2027 one-second history timed out and
|
||||
returned None, producing NoData. After continuing PDB, sibling
|
||||
history and quote tasks terminated while pikerd remained alive.
|
||||
User requested continuing with the narrow backport.
|
||||
|
||||
## Response summary
|
||||
|
||||
Partial backport of the NoData/DataUnavailable catch from
|
||||
0846cbd4a4197e47415689cf5e9d146ba3174c32 into
|
||||
maybe_fill_null_segments(). The existing gap fallback remains in use.
|
||||
A new parametrized regression uses real null detection and verifies
|
||||
that a sibling in the same nursery survives the failed repair.
|
||||
|
||||
Both new cases failed before the catch. The focused history suites
|
||||
passed after it: 17 passed. This does not qualify live overlap or fix
|
||||
IB reset churn or the separate Tractor completion-callback assertion.
|
||||
|
||||
## Files changed
|
||||
|
||||
- `piker/tsp/_history.py`: existing downstream exception catch.
|
||||
- `tests/test_history_backfill.py`: sibling survival regression.
|
||||
|
||||
## Human edits
|
||||
|
||||
User supplied live reproduction, PDB locals and cancellation logs,
|
||||
and explicitly constrained scope to a small history extraction.
|
||||
The production catch is copied from existing downstream commit
|
||||
0846cbd4, not newly designed here. New regression is agent-authored;
|
||||
no direct human source edits observed during this patch. Review and
|
||||
live qualification remain with the user.
|
||||
|
|
@ -0,0 +1,22 @@
|
|||
---
|
||||
created_at: 2026-11-06T18:56:28.621577+00:00
|
||||
generated_by:
|
||||
harness: codex
|
||||
provider: OpenAI
|
||||
git_ref: 495c0915
|
||||
diff_cmd: git diff HEAD~1..HEAD
|
||||
---
|
||||
|
||||
> `git diff HEAD~1..HEAD -- piker/tsp/_history.py`
|
||||
|
||||
> `git diff HEAD~1..HEAD -- tests/test_history_backfill.py`
|
||||
|
||||
Partial backport of the NoData/DataUnavailable catch from
|
||||
0846cbd4a4197e47415689cf5e9d146ba3174c32 into
|
||||
maybe_fill_null_segments(). The existing gap fallback remains in use.
|
||||
A new parametrized regression uses real null detection and verifies
|
||||
that a sibling in the same nursery survives the failed repair.
|
||||
|
||||
Both new cases failed before the catch. The focused history suites
|
||||
passed after it: 17 passed. This does not qualify live overlap or fix
|
||||
IB reset churn or the separate Tractor completion-callback assertion.
|
||||
|
|
@ -0,0 +1,41 @@
|
|||
---
|
||||
created_at: 2026-11-06T20:17:46.622316+00:00
|
||||
generated_by:
|
||||
harness: codex
|
||||
provider: OpenAI
|
||||
git_ref: 495c0915
|
||||
scope: config
|
||||
substantive: true
|
||||
raw_file: 20261106T201746Z_495c0915_612dfc_prompt_io.raw.md
|
||||
---
|
||||
|
||||
## Prompt
|
||||
|
||||
The human requested a commit plan, then a fast-forward landing of
|
||||
the #94 branch and PR updates. They agreed the temporary local
|
||||
Tractor source must be replaced, published piker_pin themselves,
|
||||
and reported "pushed it". They also supplied the manual recovery
|
||||
results and full Gateway restart failure diagnostics.
|
||||
|
||||
## Response summary
|
||||
|
||||
Restore the Git Tractor source after local qualification and lock
|
||||
its published piker_pin at 8fe574c1e447358dafd8ebf5ada3b0714c688f78.
|
||||
Retain the evaluated Trio 0.34.0 lock and require Trio >=0.34.0.
|
||||
Record the human's overlap, data-reset and reopen observations,
|
||||
the narrow history extraction, and full-restart recovery limits.
|
||||
Offline uv lock validation passed without an environment sync.
|
||||
|
||||
## Files changed
|
||||
|
||||
- `pyproject.toml`: Trio minimum; restore existing Tractor Git source.
|
||||
- `uv.lock`: published Tractor OID and evaluated Trio 0.34.0.
|
||||
- `ai/ib-recovery-qualification-20261106.md`: scoped evidence.
|
||||
|
||||
## Human edits
|
||||
|
||||
The human selected the bounded landing scope, performed live tests,
|
||||
identified the more aggressive reset distinction, and published
|
||||
the tested Tractor ref. The agent applied the metadata edits and
|
||||
wrote the qualification record from those observations. No human
|
||||
acceptance or task-state transitions are inferred by this record.
|
||||
|
|
@ -0,0 +1,21 @@
|
|||
---
|
||||
created_at: 2026-11-06T20:17:46.622316+00:00
|
||||
generated_by:
|
||||
harness: codex
|
||||
provider: OpenAI
|
||||
git_ref: 495c0915
|
||||
diff_cmd: git diff HEAD~1..HEAD
|
||||
---
|
||||
|
||||
> `git diff HEAD~1..HEAD -- pyproject.toml`
|
||||
|
||||
> `git diff HEAD~1..HEAD -- uv.lock`
|
||||
|
||||
> `git diff HEAD~1..HEAD -- ai/ib-recovery-qualification-20261106.md`
|
||||
|
||||
Restore the Git Tractor source after local qualification and lock
|
||||
its published piker_pin at 8fe574c1e447358dafd8ebf5ada3b0714c688f78.
|
||||
Retain the evaluated Trio 0.34.0 lock and require Trio >=0.34.0.
|
||||
Record the human's overlap, data-reset and reopen observations,
|
||||
the narrow history extraction, and full-restart recovery limits.
|
||||
Offline uv lock validation passed without an environment sync.
|
||||
|
|
@ -0,0 +1,11 @@
|
|||
# AI Prompt I/O Log
|
||||
|
||||
This directory records substantive AI-assisted changes, prompts,
|
||||
original harness identity, and unedited outputs. Human contributors
|
||||
remain accountable for code decisions.
|
||||
|
||||
Records follow the [NLNet generative AI policy][policy]. Each
|
||||
structured entry links its immutable `.raw.md` partner. Code output
|
||||
uses Git diff references to avoid duplication.
|
||||
|
||||
[policy]: https://nlnet.nl/foundation/policies/generativeAI/
|
||||
|
|
@ -0,0 +1,59 @@
|
|||
---
|
||||
model: claude-fable-5[1m]
|
||||
service: claude
|
||||
session: 32d15f9a-b2d3-4c26-bdc9-190219141a25
|
||||
timestamp: 2026-06-10T17:08:59Z
|
||||
git_ref: datad_service
|
||||
diff_cmd: git log -1 -p --follow -- ai/prompt-io/claude/20260610T170859Z_75cefe10_prompt_io.md
|
||||
scope: code
|
||||
substantive: true
|
||||
raw_file: 20260610T170859Z_75cefe10_prompt_io.raw.md
|
||||
---
|
||||
|
||||
## Prompt
|
||||
|
||||
Session-initiating instruction (driving all 7 commits in
|
||||
this series):
|
||||
|
||||
> ok i want you to become the distributed runtime and
|
||||
> concurrency expert for this project - namely acquire
|
||||
> a deep understanding of tractor and how it's used.
|
||||
> then i want you to attempt to factor our current
|
||||
> brokerd service daemon into 2 daemons: a brokerd
|
||||
> which only hosts live/paper trading endpoint tasks
|
||||
> [..] a new `datad` subdaemon which in a separate
|
||||
> subactor serves all the data feed service tasks [..]
|
||||
> give me a mega detailed plan on how to approach this,
|
||||
> and a staged approach for the implementation.
|
||||
|
||||
Proximate driver for THIS commit: during the approved
|
||||
plan's stage-0 test gate the agent discovered test
|
||||
subactors were writing into the user's REAL
|
||||
`~/.config/piker/accounting/` files (a bogus paper fill
|
||||
landed in `account.kraken.paper.toml`) because the old
|
||||
test-dir override in `config.get_app_dir()` was
|
||||
commented out and could never work in spawned
|
||||
subactors anyway. Fix applied autonomously under the
|
||||
approved plan; no explicit per-fix user prompt.
|
||||
|
||||
## Response summary
|
||||
|
||||
Restore `pytest` config-dir isolation by resolving the
|
||||
per-test tmp dir lazily (at conf-path access time) from
|
||||
`tractor` runtime-vars, which propagate down the actor
|
||||
tree; route all conf-path derivation through
|
||||
`config.get_conf_dir()` so the override is effective in
|
||||
every (sub)actor.
|
||||
|
||||
## Files changed
|
||||
|
||||
- `piker/config.py` — add `_maybe_use_test_dir()`;
|
||||
hook in `get_conf_dir()`; route `get_conf_path()` +
|
||||
`load()` mkdir through it
|
||||
- `piker/accounting/_ledger.py` — derive ledger dir
|
||||
via `config.get_conf_dir()` (not the global)
|
||||
- `piker/accounting/_pos.py` — same for account files
|
||||
|
||||
## Human edits
|
||||
|
||||
None — committed as generated.
|
||||
|
|
@ -0,0 +1,46 @@
|
|||
---
|
||||
model: claude-fable-5[1m]
|
||||
service: claude
|
||||
timestamp: 2026-06-10T17:08:59Z
|
||||
git_ref: datad_service
|
||||
diff_cmd: git log -1 -p --follow -- ai/prompt-io/claude/20260610T170859Z_75cefe10_prompt_io.md
|
||||
---
|
||||
|
||||
NOTE: diff-ref mode entry (code committed in the same
|
||||
commit as this log); backfilled from the live dev
|
||||
session transcript per the `/prompt-io` skill rules.
|
||||
|
||||
> `git log -1 -p --follow -- piker/config.py`
|
||||
|
||||
Generated: `config._maybe_use_test_dir()` — lazily
|
||||
reads `piker_test_dir` from
|
||||
`tractor.runtime._state._runtime_vars['piker_vars']`
|
||||
(pre-loaded by `open_piker_runtime()` from the
|
||||
`tests.conftest._open_test_pikerd()` overrides) and
|
||||
calls `_override_config_dir()` when set. Hooked at the
|
||||
top of `get_conf_dir()`; `get_conf_path()` and
|
||||
`load()`'s dir-creation rerouted through
|
||||
`get_conf_dir()`.
|
||||
|
||||
> `git log -1 -p --follow -- piker/accounting/_ledger.py`
|
||||
> `git log -1 -p --follow -- piker/accounting/_pos.py`
|
||||
|
||||
Generated: ledger/account dir derivation switched from
|
||||
the `config._config_dir` module global to
|
||||
`config.get_conf_dir()` + `mkdir(parents=True,
|
||||
exist_ok=True)` for nested tmp-dir creation.
|
||||
|
||||
Key diagnostic reasoning (verbatim from session):
|
||||
|
||||
The old (commented-out) `get_app_dir()` override gated
|
||||
on `'pytest' in sys.modules` which can NEVER work in
|
||||
spawned subactors (fresh procs, no pytest import); as
|
||||
a result test `paperboi`/daemon actors were writing
|
||||
into the user's REAL `~/.config/piker/accounting/`
|
||||
files. Evidence: `account.kraken.paper.toml` gained a
|
||||
single 0.001 xbtusdt clear stamped 2026-06-09T18:47Z
|
||||
(a test fill), and `trades_kraken_paper.toml` (252B)
|
||||
contained only that fill. The resolution must be
|
||||
checked lazily at config-path access time (NOT import
|
||||
time) since sub-actors only receive runtime-vars once
|
||||
their `tractor` runtime has fully booted.
|
||||
|
|
@ -0,0 +1,53 @@
|
|||
---
|
||||
model: claude-fable-5[1m]
|
||||
service: claude
|
||||
session: 32d15f9a-b2d3-4c26-bdc9-190219141a25
|
||||
timestamp: 2026-06-10T17:10:22Z
|
||||
git_ref: datad_service
|
||||
diff_cmd: git log -1 -p --follow -- ai/prompt-io/claude/20260610T171022Z_4485f2b9_prompt_io.md
|
||||
scope: code
|
||||
substantive: true
|
||||
raw_file: 20260610T171022Z_4485f2b9_prompt_io.raw.md
|
||||
---
|
||||
|
||||
## Prompt
|
||||
|
||||
Same session-initiating `brokerd`-split instruction (see
|
||||
`20260610T170859Z_75cefe10_prompt_io.md`). Proximate
|
||||
driver: the approved plan's per-stage test gates could
|
||||
not run AT ALL — the branch base was broken vs.
|
||||
`tractor` git `main` (`AttributeError: 'tuple' object
|
||||
has no attribute 'unwrap'` at `pikerd` boot, stale
|
||||
`arbiter_sockaddr`/`.type`/`.size` API refs). The agent
|
||||
fixed forward autonomously to (re)establish the gate
|
||||
baseline, continuing the `repair_tests` branch lineage
|
||||
(verified already merged in ancestry via `git cherry`
|
||||
during a user-requested branch-overlap survey).
|
||||
|
||||
## Response summary
|
||||
|
||||
Port the service layer + test suites to current
|
||||
`tractor` APIs: addr-type normalization in
|
||||
`open_pikerd()`, `query_actor()`/`wait_for_actor()`
|
||||
kwarg renames, modern self-cancel absorption semantics
|
||||
in the cancel-method test harness, exc/position attr
|
||||
renames, a paper-EMS startup-budget bump and a syntax
|
||||
fix in `.deribit.api`.
|
||||
|
||||
## Files changed
|
||||
|
||||
- `piker/service/_actor_runtime.py` — `wrap_address()`
|
||||
normalize before `.unwrap()` in `open_pikerd()`
|
||||
- `piker/service/_registry.py` — `check_for_service()`
|
||||
-> `query_actor(regaddr=)` + 2-tuple yield +
|
||||
`open_registry(addrs=)`
|
||||
- `piker/brokers/deribit/api.py` — missing comma in
|
||||
`tractor.trionics` import tuple
|
||||
- `tests/test_services.py` — `registry_addr=` kwarg,
|
||||
raddr unwraps, cancel-semantics harness rewrite,
|
||||
`fail_after` 9 -> 19s
|
||||
- `tests/test_ems.py` — `.boxed_type`, `pp.cumsize`
|
||||
|
||||
## Human edits
|
||||
|
||||
None — committed as generated.
|
||||
|
|
@ -0,0 +1,56 @@
|
|||
---
|
||||
model: claude-fable-5[1m]
|
||||
service: claude
|
||||
timestamp: 2026-06-10T17:10:22Z
|
||||
git_ref: datad_service
|
||||
diff_cmd: git log -1 -p --follow -- ai/prompt-io/claude/20260610T171022Z_4485f2b9_prompt_io.md
|
||||
---
|
||||
|
||||
NOTE: diff-ref mode entry (code committed in the same
|
||||
commit as this log); backfilled from the live dev
|
||||
session transcript per the `/prompt-io` skill rules.
|
||||
|
||||
> `git log -1 -p --follow -- piker/service/_actor_runtime.py`
|
||||
|
||||
Generated: normalize each registry addr via
|
||||
`tractor.discovery._addr.wrap_address()` before
|
||||
`.unwrap()`-ing for the `accept_addrs` bind check —
|
||||
entries may be raw `tuple`s when passed in from (test)
|
||||
client code. Import-path precedent taken from
|
||||
`piker/cli/__init__.py:336`.
|
||||
|
||||
> `git log -1 -p --follow -- piker/service/_registry.py`
|
||||
|
||||
Generated: `check_for_service()` ported to
|
||||
`tractor.query_actor(name, regaddr=...)` (kwarg was
|
||||
`arbiter_sockaddr=`), unpacking the new
|
||||
`(sockaddr, portal)` yield, and passing the
|
||||
now-required `open_registry(addrs=Registry.addrs)`.
|
||||
|
||||
> `git log -1 -p --follow -- tests/test_services.py`
|
||||
> `git log -1 -p --follow -- tests/test_ems.py`
|
||||
> `git log -1 -p --follow -- piker/brokers/deribit/api.py`
|
||||
|
||||
Key diagnostic reasoning (verbatim from session):
|
||||
|
||||
- the "DID NOT RAISE ContextCancelled" failure: in this
|
||||
test the client actor IS pikerd (in-proc), and
|
||||
current `tractor` main absorbs a `ContextCancelled`
|
||||
whose canceller is your own actor — self-requested
|
||||
cancels now exit cleanly instead of raising; the
|
||||
'sigint' variant propagates a bare collapsed
|
||||
`KeyboardInterrupt` rather than a
|
||||
`BaseExceptionGroup`.
|
||||
- the hard-coded `trio.fail_after(9)` startup budget is
|
||||
marginal — full stack boot (pikerd -> emsd ->
|
||||
brokerd.kraken -> paperboi + live kraken symbology
|
||||
fetch) occasionally exceeds 9s -> bumped to 19s.
|
||||
- `RemoteActorError.type` -> `.boxed_type`;
|
||||
`Position.size` -> `.cumsize` (the paper engine
|
||||
populates `BrokerdPosition.size` from `pp.cumsize`).
|
||||
- overlap survey (user-requested): all of the
|
||||
`repair_tests` branch commits are already in this
|
||||
stack's ancestry; this commit finishes that branch's
|
||||
port mission (its f4c4f1e2 fixed `conftest.py`'s
|
||||
`arbiter_sockaddr` usage; this fixes the remaining
|
||||
`test_services.py` + `check_for_service()` sites).
|
||||
|
|
@ -0,0 +1,54 @@
|
|||
---
|
||||
model: claude-fable-5[1m]
|
||||
service: claude
|
||||
session: 32d15f9a-b2d3-4c26-bdc9-190219141a25
|
||||
timestamp: 2026-06-10T17:11:05Z
|
||||
git_ref: datad_service
|
||||
diff_cmd: git log -1 -p --follow -- ai/prompt-io/claude/20260610T171105Z_bc6e18d7_prompt_io.md
|
||||
scope: code
|
||||
substantive: true
|
||||
raw_file: 20260610T171105Z_bc6e18d7_prompt_io.raw.md
|
||||
---
|
||||
|
||||
## Prompt
|
||||
|
||||
Same session-initiating `brokerd`-split instruction (see
|
||||
`20260610T170859Z_75cefe10_prompt_io.md`); this is the
|
||||
approved plan's "stage 0" prep. Plan-shaping user
|
||||
decisions captured via in-session Q&A:
|
||||
|
||||
- `datad.<broker>` topology: sibling of
|
||||
`brokerd.<broker>` under `pikerd` (vs. child
|
||||
subactor).
|
||||
- migration: hard cutover, staged by layer (no
|
||||
dual-mode runtime flag).
|
||||
- post-split `brokerd` scope: trading-only +
|
||||
EMS-lazy-spawned (charts/CLI never touch it).
|
||||
|
||||
## Response summary
|
||||
|
||||
Declare per-daemon-kind backend submod groups
|
||||
(`_datad_mods`/`_brokerd_mods`) in the split-style
|
||||
backends, keyed to the pre-existing
|
||||
`piker.data.validate._eps` contract; add a
|
||||
`validate.get_eps()` introspection helper; ws-import
|
||||
hygiene in `.kraken.broker`. Zero behavior change
|
||||
(`__enable_modules__` unions unchanged).
|
||||
|
||||
## Files changed
|
||||
|
||||
- `piker/data/validate.py` — add `get_eps()`
|
||||
- `piker/brokers/kraken/__init__.py` — mod groups
|
||||
- `piker/brokers/binance/__init__.py` — mod groups
|
||||
(note: no `symbols.py`; search eps live in `.feed`)
|
||||
- `piker/brokers/deribit/__init__.py` — datad-only
|
||||
groups (no `broker.py` yet)
|
||||
- `piker/brokers/ib/__init__.py` — add `'api'` to the
|
||||
pre-existing `_datad_mods`
|
||||
- `piker/brokers/kraken/broker.py` — import
|
||||
`NoBsWs`/`open_autorecon_ws` from
|
||||
`piker.data._web_bs` directly
|
||||
|
||||
## Human edits
|
||||
|
||||
None — committed as generated.
|
||||
|
|
@ -0,0 +1,46 @@
|
|||
---
|
||||
model: claude-fable-5[1m]
|
||||
service: claude
|
||||
timestamp: 2026-06-10T17:11:05Z
|
||||
git_ref: datad_service
|
||||
diff_cmd: git log -1 -p --follow -- ai/prompt-io/claude/20260610T171105Z_bc6e18d7_prompt_io.md
|
||||
---
|
||||
|
||||
NOTE: diff-ref mode entry (code committed in the same
|
||||
commit as this log); backfilled from the live dev
|
||||
session transcript per the `/prompt-io` skill rules.
|
||||
|
||||
> `git log -1 -p --follow -- piker/data/validate.py`
|
||||
|
||||
Generated: `get_eps(mod, kind) -> dict[str, Callable]`
|
||||
returning the daemon-kind's ep funcs defined by a
|
||||
backend mod, keyed by ep name, missing eps excluded;
|
||||
sourced from the existing `_eps` grouping table
|
||||
(`'middleware' | 'datad' | 'brokerd'`).
|
||||
|
||||
> `git log -1 -p --follow -- piker/brokers/kraken/__init__.py`
|
||||
> `git log -1 -p --follow -- piker/brokers/binance/__init__.py`
|
||||
> `git log -1 -p --follow -- piker/brokers/deribit/__init__.py`
|
||||
> `git log -1 -p --follow -- piker/brokers/ib/__init__.py`
|
||||
> `git log -1 -p --follow -- piker/brokers/kraken/broker.py`
|
||||
|
||||
Key exploration findings driving the design (from the
|
||||
planning phase, 3 parallel explore agents + 1 architect
|
||||
agent):
|
||||
|
||||
- the codebase already anticipated this split:
|
||||
`validate._eps` groups eps into 'datad' vs 'brokerd'
|
||||
kinds and `ib/__init__.py` already declared
|
||||
`_brokerd_mods`/`_datad_mods`; `brokers/_daemon.py`
|
||||
carried the literal TODO "rename the daemon to datad
|
||||
prolly once we split up broker vs. data tasks into
|
||||
separate actors?".
|
||||
- the feared kraken feed<->broker "shared ws state"
|
||||
hazard is mild: `NoBsWs`/`open_autorecon_ws`
|
||||
originate in `piker.data._web_bs` (re-exported via
|
||||
`.kraken.feed`); only `stream_messages` is feed-local
|
||||
and it's a pure parser — in-process imports only,
|
||||
each actor opens its own ws conn.
|
||||
- `enable_modules` gates RPC entry, NOT python imports,
|
||||
so backend trading mods may keep importing feed-side
|
||||
helpers in-process post-split.
|
||||
|
|
@ -0,0 +1,47 @@
|
|||
---
|
||||
model: claude-fable-5[1m]
|
||||
service: claude
|
||||
session: 32d15f9a-b2d3-4c26-bdc9-190219141a25
|
||||
timestamp: 2026-06-10T17:11:42Z
|
||||
git_ref: datad_service
|
||||
diff_cmd: git log -1 -p --follow -- ai/prompt-io/claude/20260610T171142Z_119d2c04_prompt_io.md
|
||||
scope: code
|
||||
substantive: true
|
||||
raw_file: 20260610T171142Z_119d2c04_prompt_io.raw.md
|
||||
---
|
||||
|
||||
## Prompt
|
||||
|
||||
Same session-initiating `brokerd`-split instruction (see
|
||||
`20260610T170859Z_75cefe10_prompt_io.md`); this is the
|
||||
approved plan's "stage 1": introduce the `datad` daemon
|
||||
machinery additively (nothing routes through it yet).
|
||||
User-decided constraint applied: `datad.<broker>` is a
|
||||
SIBLING of `brokerd.<broker>` under `pikerd`, spawned
|
||||
via the existing `Services` + `maybe_spawn_daemon()`
|
||||
machinery.
|
||||
|
||||
## Response summary
|
||||
|
||||
New `piker/data/_daemon.py` hosting the
|
||||
`datad.<broker>` feed-only daemon-actor: lifetime
|
||||
fixture owning the actor-global `_FeedsBus`, init/spawn
|
||||
fns mirroring `.brokers._daemon` conventions and the
|
||||
`samplerd` sub-daemon precedent, plus root-mod
|
||||
registration, `piker.service` re-exports and a spawn
|
||||
test.
|
||||
|
||||
## Files changed
|
||||
|
||||
- `piker/data/_daemon.py` — NEW:
|
||||
`_setup_persistent_datad()`, `datad_init()`,
|
||||
`spawn_datad()`, `maybe_spawn_datad()`,
|
||||
`_datad_service_mods`
|
||||
- `piker/service/_actor_runtime.py` — add
|
||||
`piker.data._daemon` to `_root_modules`
|
||||
- `piker/service/__init__.py` — re-export spawn eps
|
||||
- `tests/test_services.py` — add `test_datad_spawn`
|
||||
|
||||
## Human edits
|
||||
|
||||
None — committed as generated.
|
||||
|
|
@ -0,0 +1,60 @@
|
|||
---
|
||||
model: claude-fable-5[1m]
|
||||
service: claude
|
||||
timestamp: 2026-06-10T17:11:42Z
|
||||
git_ref: datad_service
|
||||
diff_cmd: git log -1 -p --follow -- ai/prompt-io/claude/20260610T171142Z_119d2c04_prompt_io.md
|
||||
---
|
||||
|
||||
NOTE: diff-ref mode entry (code committed in the same
|
||||
commit as this log); backfilled from the live dev
|
||||
session transcript per the `/prompt-io` skill rules.
|
||||
|
||||
> `git log -1 -p --follow -- piker/data/_daemon.py`
|
||||
|
||||
Generated symbols + key design decisions:
|
||||
|
||||
- `_datad_service_mods: list[str]` — datad-always
|
||||
enabled mods, the data-side successor to the old
|
||||
`piker.brokers._daemon._data_mods` set; kept minimal
|
||||
per the caps-sec model.
|
||||
- `_setup_persistent_datad()` — `@tractor.context`
|
||||
lifetime fixture: console-log setup then allocates
|
||||
the actor-global feed bus via
|
||||
`feed.get_feed_bus(brokername, service_nursery)`
|
||||
exactly as the old brokerd fixture did, pinned open
|
||||
with `ctx.started()` + `sleep_forever()`.
|
||||
- `datad_init()` — actor name `f'datad.{brokername}'`;
|
||||
copies backend `_spawn_kwargs` (CRITICAL for `ib`'s
|
||||
`infect_asyncio=True`); builds `enable_modules` from
|
||||
`getattr(brokermod, '_datad_mods',
|
||||
getattr(brokermod, '__enable_modules__', []))` as
|
||||
the flat-backend fallback.
|
||||
- `spawn_datad()` — `Services.actor_n.start_actor()` +
|
||||
`Services.start_service_task()` exactly mirroring
|
||||
`spawn_brokerd()`; dedup-composes enable mods via
|
||||
`list(dict.fromkeys(...))`.
|
||||
- `maybe_spawn_datad()` — wraps `maybe_spawn_daemon(
|
||||
service_name=f'datad.{brokername}', ...)`.
|
||||
|
||||
> `git log -1 -p --follow -- piker/service/_actor_runtime.py`
|
||||
> `git log -1 -p --follow -- piker/service/__init__.py`
|
||||
> `git log -1 -p --follow -- tests/test_services.py`
|
||||
|
||||
Design rationale (verbatim from session):
|
||||
|
||||
- `_root_modules` must gain `piker.data._daemon` so
|
||||
`pikerd_portal.run(spawn_datad, ...)` resolves in
|
||||
the root.
|
||||
- the `Services`-based impl style deliberately mirrors
|
||||
`spawn_brokerd()` so the eventual `tractor.hilevel`
|
||||
`ServiceMngr` port (see the `service_mng_to_tractor`
|
||||
branch's d8c21d44 prep, surfaced by the
|
||||
user-requested branch-overlap survey) lands
|
||||
symmetrically on both spawn fns.
|
||||
- mod placement (`piker/data/_daemon.py` vs.
|
||||
generalizing `piker.brokers._daemon`) follows the
|
||||
per-subsystem daemon-mod convention
|
||||
(`.clearing._ems`, `.data._sampling`) and resolves
|
||||
the existing TODO at `brokers/_daemon.py:49` ("move
|
||||
this def to the `.data` subpkg").
|
||||
|
|
@ -0,0 +1,47 @@
|
|||
---
|
||||
model: claude-fable-5[1m]
|
||||
service: claude
|
||||
session: 32d15f9a-b2d3-4c26-bdc9-190219141a25
|
||||
timestamp: 2026-06-10T17:12:26Z
|
||||
git_ref: datad_service
|
||||
diff_cmd: git log -1 -p --follow -- ai/prompt-io/claude/20260610T171226Z_64181219_prompt_io.md
|
||||
scope: code
|
||||
substantive: true
|
||||
raw_file: 20260610T171226Z_64181219_prompt_io.raw.md
|
||||
---
|
||||
|
||||
## Prompt
|
||||
|
||||
Same session-initiating `brokerd`-split instruction (see
|
||||
`20260610T170859Z_75cefe10_prompt_io.md`); this is the
|
||||
approved plan's "stage 2": decouple the clearing layer
|
||||
from `feed.portals` BEFORE the feed cutover so live
|
||||
trading works at every stage boundary. User-decided
|
||||
constraint applied: post-split `brokerd` is
|
||||
trading-only and spawned LAZILY only by `emsd`'s
|
||||
`open_brokerd_dialog()` path.
|
||||
|
||||
## Response summary
|
||||
|
||||
Kill the single coupling forcing feed + trading eps
|
||||
into one actor (`Router.open_trade_relays()` pulling
|
||||
its trades portal from `feed.portals[brokermod]`):
|
||||
`open_brokerd_dialog()` now (maybe) spawns/finds
|
||||
`brokerd.<broker>` itself and ONLY when a live
|
||||
trades-ep will actually open; paper mode never touches
|
||||
it. Pre-cutover this is a pure refactor (registry
|
||||
lookup finds the same feed-spawned daemon).
|
||||
|
||||
## Files changed
|
||||
|
||||
- `piker/clearing/_ems.py` — `open_brokerd_dialog()`
|
||||
re-sig + inner `acquire_live_portal()`;
|
||||
`Router.maybe_open_brokerd_dialog()` drops `portal`
|
||||
param; `open_trade_relays()` drops the
|
||||
`feed.portals` lookup
|
||||
- `piker/accounting/cli.py` — keyword-form `portal=`
|
||||
override kept for the `piker ledger` ad-hoc actor
|
||||
|
||||
## Human edits
|
||||
|
||||
None — committed as generated.
|
||||
|
|
@ -0,0 +1,43 @@
|
|||
---
|
||||
model: claude-fable-5[1m]
|
||||
service: claude
|
||||
timestamp: 2026-06-10T17:12:26Z
|
||||
git_ref: datad_service
|
||||
diff_cmd: git log -1 -p --follow -- ai/prompt-io/claude/20260610T171226Z_64181219_prompt_io.md
|
||||
---
|
||||
|
||||
NOTE: diff-ref mode entry (code committed in the same
|
||||
commit as this log); backfilled from the live dev
|
||||
session transcript per the `/prompt-io` skill rules.
|
||||
|
||||
> `git log -1 -p --follow -- piker/clearing/_ems.py`
|
||||
|
||||
Generated: new `open_brokerd_dialog()` signature
|
||||
`(brokermod, exec_mode, fqme=None, portal=None,
|
||||
loglevel=None)` with an inner `@acm
|
||||
acquire_live_portal()` that yields the caller-provided
|
||||
`portal` override (the `piker ledger` path) else
|
||||
`maybe_spawn_brokerd(brokermod.name)` — designated THE
|
||||
one place a live, credentialed `brokerd.<broker>` gets
|
||||
booted post-split. The eager
|
||||
`portal.open_context(trades_endpoint, ...)`
|
||||
construction moved inside that block so the paper
|
||||
short-circuit never acquires a live portal.
|
||||
|
||||
> `git log -1 -p --follow -- piker/accounting/cli.py`
|
||||
|
||||
Key analysis (verbatim from session):
|
||||
|
||||
- `feed.portals` had exactly ONE trading consumer:
|
||||
`piker/clearing/_ems.py:671` (`portal =
|
||||
feed.portals[brokermod]`) — the single coupling
|
||||
forcing feed + trading into one actor.
|
||||
- `piker ledger` (`accounting/cli.py:100-157`) is a
|
||||
hidden consumer: it calls `broker_init()` directly,
|
||||
spawns its own ad-hoc actor and passes the portal
|
||||
positionally — the new signature must keep an
|
||||
explicit `portal:` override param for this path.
|
||||
- stage sequencing: clearing decouple lands BEFORE the
|
||||
feed cutover, otherwise `feed.portals` would hand
|
||||
emsd a `datad` portal and live `open_trade_dialog`
|
||||
RPC would fail.
|
||||
|
|
@ -0,0 +1,48 @@
|
|||
---
|
||||
model: claude-fable-5[1m]
|
||||
service: claude
|
||||
session: 32d15f9a-b2d3-4c26-bdc9-190219141a25
|
||||
timestamp: 2026-06-10T17:12:59Z
|
||||
git_ref: datad_service
|
||||
diff_cmd: git log -1 -p --follow -- ai/prompt-io/claude/20260610T171259Z_59d5d9a6_prompt_io.md
|
||||
scope: code
|
||||
substantive: true
|
||||
raw_file: 20260610T171259Z_59d5d9a6_prompt_io.raw.md
|
||||
---
|
||||
|
||||
## Prompt
|
||||
|
||||
Same session-initiating `brokerd`-split instruction (see
|
||||
`20260610T170859Z_75cefe10_prompt_io.md`); this is the
|
||||
approved plan's "stage 3": the feed-layer hard cutover
|
||||
(user-decided: no dual-mode runtime flag, each stage
|
||||
lands fully working).
|
||||
|
||||
## Response summary
|
||||
|
||||
The topology flip: all data-feed consumers route to
|
||||
`datad.<broker>`; the `brokerd` lifetime fixture slims
|
||||
to trading-only; UI/CLI data-flavoured spawn sites
|
||||
repointed; test expectations inverted incl. an explicit
|
||||
negative assert that paper sessions spawn NO
|
||||
`brokerd.<broker>`.
|
||||
|
||||
## Files changed
|
||||
|
||||
- `piker/data/feed.py` — `open_feed()` ->
|
||||
`maybe_spawn_datad()`; `open_feed_bus()` actor-name
|
||||
assert -> `'datad'`; `install_brokerd_search` ->
|
||||
`install_datad_search`; comment sweep
|
||||
- `piker/brokers/_daemon.py` —
|
||||
`_setup_persistent_brokerd()` slimmed (feed-bus
|
||||
alloc moved to the datad fixture)
|
||||
- `piker/ui/_app.py`, `piker/ui/cli.py`,
|
||||
`piker/brokers/core.py`, `piker/brokers/cli.py`,
|
||||
`piker/ui/kivy/option_chain.py` — spawn-site
|
||||
repoints
|
||||
- `tests/test_services.py` — datad assertions + the
|
||||
no-`brokerd.kraken` negative check
|
||||
|
||||
## Human edits
|
||||
|
||||
None — committed as generated.
|
||||
|
|
@ -0,0 +1,54 @@
|
|||
---
|
||||
model: claude-fable-5[1m]
|
||||
service: claude
|
||||
timestamp: 2026-06-10T17:12:59Z
|
||||
git_ref: datad_service
|
||||
diff_cmd: git log -1 -p --follow -- ai/prompt-io/claude/20260610T171259Z_59d5d9a6_prompt_io.md
|
||||
---
|
||||
|
||||
NOTE: diff-ref mode entry (code committed in the same
|
||||
commit as this log); backfilled from the live dev
|
||||
session transcript per the `/prompt-io` skill rules.
|
||||
|
||||
> `git log -1 -p --follow -- piker/data/feed.py`
|
||||
|
||||
Generated: `open_feed()` spawn cutover with the
|
||||
`maybe_spawn_datad` import done "relative-direct" from
|
||||
`._daemon` — NOT via `piker.service` — to dodge a
|
||||
partial-init cycle: when `piker.data.feed` loads as
|
||||
part of `piker.service.__init__` executing its own
|
||||
`piker.data._daemon` import, the service pkg is
|
||||
mid-init and its `maybe_spawn_datad` binding does not
|
||||
exist yet. The `open_feed_bus()` local-state sanity
|
||||
assert flips `'brokerd' in servicename` ->
|
||||
`'datad' in servicename` (the only actor-name assert
|
||||
in the tree, verified by grep).
|
||||
|
||||
> `git log -1 -p --follow -- piker/brokers/_daemon.py`
|
||||
|
||||
Generated: `_setup_persistent_brokerd()` slimmed to
|
||||
console-log setup + pinned-open ctx; drops the bus
|
||||
alloc, the `assert not feed._bus`, the service nursery
|
||||
(backend `open_trade_dialog()` ctxs own their task
|
||||
trees) and the `eg.ExceptionGroup` handler. The
|
||||
`piker ledger` ad-hoc actor enters this same slimmed
|
||||
fixture — exactly what it needs.
|
||||
|
||||
> `git log -1 -p --follow -- piker/ui/_app.py`
|
||||
> `git log -1 -p --follow -- piker/ui/cli.py`
|
||||
> `git log -1 -p --follow -- piker/brokers/core.py`
|
||||
> `git log -1 -p --follow -- piker/brokers/cli.py`
|
||||
> `git log -1 -p --follow -- piker/ui/kivy/option_chain.py`
|
||||
> `git log -1 -p --follow -- tests/test_services.py`
|
||||
|
||||
Verification (from session): per-suite gates green at
|
||||
this commit (services 5-passed incl. the new negative
|
||||
assert, feeds 3-passed); a headless live smoke
|
||||
(`maybe_open_pikerd` + `open_feed(['xbtusdt.kraken'])`
|
||||
on an alt registry port) confirmed quotes flowing via
|
||||
`datad.kraken` + `samplerd` with `check_for_service(
|
||||
'brokerd.kraken') is None`. Known pre-existing flake
|
||||
documented: `test_multi_fill_positions`' second
|
||||
in-proc runtime boot wedges ~50% (zombie subactor w/
|
||||
unread parent-IPC bytes); reproduced with the split
|
||||
fully reverted so NOT a regression of this work.
|
||||
|
|
@ -0,0 +1,43 @@
|
|||
---
|
||||
model: claude-fable-5[1m]
|
||||
service: claude
|
||||
session: 32d15f9a-b2d3-4c26-bdc9-190219141a25
|
||||
timestamp: 2026-06-10T17:13:44Z
|
||||
git_ref: datad_service
|
||||
diff_cmd: git log -1 -p --follow -- ai/prompt-io/claude/20260610T171344Z_eee19de0_prompt_io.md
|
||||
scope: code
|
||||
substantive: true
|
||||
raw_file: 20260610T171344Z_eee19de0_prompt_io.raw.md
|
||||
---
|
||||
|
||||
## Prompt
|
||||
|
||||
Same session-initiating `brokerd`-split instruction (see
|
||||
`20260610T170859Z_75cefe10_prompt_io.md`); this is the
|
||||
approved plan's final "stage 4": caps-sec slimming of
|
||||
the (live, credentialed) trading actor + the `ib`
|
||||
dual-daemon `client_id` collision mitigation flagged in
|
||||
the plan's risk register.
|
||||
|
||||
## Response summary
|
||||
|
||||
`brokerd` loses ALL `piker.data.*` (feed) RPC mods;
|
||||
spawn fails fast for datad-only backends with a "use
|
||||
paper-mode" error; `ib`'s default api-gw `client_id`
|
||||
gets a per-daemon-kind offset so `datad.ib` +
|
||||
`brokerd.ib` don't collide on connect.
|
||||
|
||||
## Files changed
|
||||
|
||||
- `piker/brokers/_daemon.py` — `_data_mods` -> minimal
|
||||
`_brokerd_service_mods`; `broker_init()` reads
|
||||
`_brokerd_mods` (fallback `__enable_modules__`);
|
||||
`spawn_brokerd()` fail-fast via `validate.get_eps()`
|
||||
- `piker/brokers/ib/api.py` — role-based `client_id`
|
||||
offset in `load_aio_clients()`
|
||||
- `piker/cli/__init__.py` — resolved "expose datad"
|
||||
TODO
|
||||
|
||||
## Human edits
|
||||
|
||||
None — committed as generated.
|
||||
|
|
@ -0,0 +1,41 @@
|
|||
---
|
||||
model: claude-fable-5[1m]
|
||||
service: claude
|
||||
timestamp: 2026-06-10T17:13:44Z
|
||||
git_ref: datad_service
|
||||
diff_cmd: git log -1 -p --follow -- ai/prompt-io/claude/20260610T171344Z_eee19de0_prompt_io.md
|
||||
---
|
||||
|
||||
NOTE: diff-ref mode entry (code committed in the same
|
||||
commit as this log); backfilled from the live dev
|
||||
session transcript per the `/prompt-io` skill rules.
|
||||
|
||||
> `git log -1 -p --follow -- piker/brokers/_daemon.py`
|
||||
|
||||
Generated: the fail-fast originally landed in
|
||||
`broker_init()` but was relocated to `spawn_brokerd()`
|
||||
mid-implementation after realizing `piker ledger`
|
||||
calls `broker_init()` directly even for paper accounts
|
||||
on datad-only backends (would have crashed the cli);
|
||||
the service-spawn path is the correct enforcement
|
||||
seam. Error text:
|
||||
|
||||
Backend 'kucoin' offers NO `brokerd` (live
|
||||
order-control) eps!? It is likely a datad-only
|
||||
provider, use paper-mode for clearing instead.
|
||||
|
||||
(verified live via a `trio.run()` unit check.)
|
||||
|
||||
> `git log -1 -p --follow -- piker/brokers/ib/api.py`
|
||||
|
||||
Generated: in `load_aio_clients()`, when `client_id`
|
||||
is the 6116 default: `datad`-named actors offset +16
|
||||
(disjoint from `brokerd`'s linear `client_id + i`
|
||||
retry range), other non-`brokerd` (ad-hoc test/cli)
|
||||
actors +32. Rationale from the plan's risk register:
|
||||
post-split BOTH per-broker daemons connect to the same
|
||||
TWS/gw endpoint; a shared default id collides and
|
||||
burns up to `connect_timeout * retries` (90s) in
|
||||
retry cycles.
|
||||
|
||||
> `git log -1 -p --follow -- piker/cli/__init__.py`
|
||||
|
|
@ -0,0 +1,66 @@
|
|||
---
|
||||
model: claude-fable-5[1m]
|
||||
service: claude
|
||||
session: 32d15f9a-b2d3-4c26-bdc9-190219141a25
|
||||
timestamp: 2026-06-10T17:33:09Z
|
||||
git_ref: datad_service
|
||||
diff_cmd: git log -1 -p --follow -- ai/prompt-io/claude/20260610T173309Z_f15f8178_prompt_io.md
|
||||
scope: docs
|
||||
substantive: true
|
||||
raw_file: 20260610T173309Z_f15f8178_prompt_io.raw.md
|
||||
---
|
||||
|
||||
## Prompt
|
||||
|
||||
The session-initiating instruction, verbatim (this doc
|
||||
IS its "mega detailed plan" deliverable):
|
||||
|
||||
> ok i want you to become the distributed runtime and
|
||||
> concurrency expert for this project - namely acquire
|
||||
> a deep understanding of tractor and how it's used.
|
||||
> then i want you to atttempt to factor our current
|
||||
> brokerd service daemon into 2 daemons:
|
||||
>
|
||||
> - a brokerd which only hosts live/paper trading
|
||||
> endoint tasks as defined namely within all
|
||||
> piker/brokers/<backend>/broker.py mods
|
||||
> - a new `datad` subdaemon which in a separate
|
||||
> subactor serves all the data feed service tasks
|
||||
> namely delivered by endpoints in
|
||||
> piker/brokers/<backend>/feed.py (or similar for
|
||||
> less mature backends) and as more rigorously
|
||||
> defined by the validation machinery in
|
||||
> piker.data.validate.
|
||||
>
|
||||
> give me a mega detailed plan on how to approach
|
||||
> this, and a staged approach for the implementation.
|
||||
|
||||
Plan-shaping user decisions (in-session Q&A): sibling
|
||||
topology under `pikerd`; hard cutover staged by layer;
|
||||
trading-only EMS-lazy-spawned `brokerd`. The human
|
||||
staged the doc copy into `ai/claude-code/plans/` and
|
||||
requested this commit after the 7 implementation
|
||||
commits landed.
|
||||
|
||||
## Response summary
|
||||
|
||||
The staged design doc for the `brokerd` -> (`datad` +
|
||||
`brokerd`) split: context, target supervision topology,
|
||||
load-bearing verified facts, per-stage file-level
|
||||
changes with gates, risk register and an end-to-end
|
||||
verification matrix. Produced via 3 parallel explore
|
||||
agents + 1 architect agent + human Q&A before any code
|
||||
was written; all 7 implementation commits in this
|
||||
branch reference its stages in their provenance
|
||||
entries.
|
||||
|
||||
## Files changed
|
||||
|
||||
- `ai/claude-code/plans/datad_service.md` — the design
|
||||
plan doc (human-staged copy of the AI-authored plan)
|
||||
|
||||
## Human edits
|
||||
|
||||
None to the doc content — committed as generated. NB:
|
||||
the implementation deviated from the plan-as-written in
|
||||
4 places, see the raw file's deviation log.
|
||||
|
|
@ -0,0 +1,51 @@
|
|||
---
|
||||
model: claude-fable-5[1m]
|
||||
service: claude
|
||||
timestamp: 2026-06-10T17:33:09Z
|
||||
git_ref: datad_service
|
||||
diff_cmd: git log -1 -p --follow -- ai/prompt-io/claude/20260610T173309Z_f15f8178_prompt_io.md
|
||||
---
|
||||
|
||||
NOTE: diff-ref mode entry; the AI output here is the
|
||||
committed doc itself (a design plan, not code) so no
|
||||
verbatim copy is duplicated:
|
||||
|
||||
> `git log -1 -p --follow -- ai/claude-code/plans/datad_service.md`
|
||||
|
||||
## Implementation deviation log
|
||||
|
||||
Where the landed 7-commit series differs from the plan
|
||||
as written (recorded for reviewer transparency; the
|
||||
plan doc is committed unmodified):
|
||||
|
||||
1. Commit ordering: the `pytest` config-dir isolation
|
||||
fix lands BEFORE the `tractor`-API drift port. The
|
||||
plan's stage gates assumed a runnable+isolated test
|
||||
baseline; per-commit gating exposed that without
|
||||
isolation the paper-EMS test reads the user's real
|
||||
(polluted) `account.kraken.paper.toml` and reddens.
|
||||
2. The plan's "stage 0: full pytest green" gate
|
||||
required first repairing pre-existing branch
|
||||
breakage vs `tractor` git `main` (boot
|
||||
`AttributeError`, stale discovery/exc/position
|
||||
APIs) — that repair became its own commit
|
||||
("Port service+tests to latest `tractor` APIs")
|
||||
rather than plan-stage work.
|
||||
3. Stage-4 fail-fast placement: the plan said
|
||||
`broker_init()` raises on brokerd-ep-less backends;
|
||||
implementation moved the raise to `spawn_brokerd()`
|
||||
since `piker ledger` calls `broker_init()` directly
|
||||
even for paper accounts on datad-only backends.
|
||||
4. `brokers/_daemon.py` change grouping: the
|
||||
import-cleanup hunks (`exceptiongroup`, `_FeedsBus`
|
||||
type-only import) landed with the caps-sec slim
|
||||
commit instead of the fixture-slim commit, keeping
|
||||
each intermediate tree import-clean without
|
||||
sub-hunk surgery.
|
||||
|
||||
Also of record: the plan's "Verification" matrix was
|
||||
executed as written (per-suite gates each stage, the
|
||||
headless datad-feed smoke, the kucoin fail-fast unit
|
||||
check); the known pre-existing ~50% second-runtime-boot
|
||||
test wedge was characterized and excluded as a
|
||||
regression via revert-testing.
|
||||
|
|
@ -0,0 +1,60 @@
|
|||
---
|
||||
model: claude-fable-5[1m]
|
||||
service: claude
|
||||
session: 32d15f9a-b2d3-4c26-bdc9-190219141a25
|
||||
timestamp: 2026-06-10T21:35:49Z
|
||||
git_ref: datad_service
|
||||
diff_cmd: git log -1 -p --follow -- ai/prompt-io/claude/20260610T213549Z_f084e899_prompt_io.md
|
||||
scope: code
|
||||
substantive: true
|
||||
raw_file: 20260610T213549Z_f084e899_prompt_io.raw.md
|
||||
---
|
||||
|
||||
## Prompt
|
||||
|
||||
User bug report from live `ib` testing of the
|
||||
(datad|brokerd)-split branch:
|
||||
|
||||
> ok doing some testing and noticing live orders do not
|
||||
> work otb, pikerd show this on live submission,
|
||||
> [pdb traceback: `RuntimeError("Can not order
|
||||
> {symbol}, no live feed?")` at `ib/api.py:1152`
|
||||
> `submit_limit()` with `self._contracts == {}` in the
|
||||
> `brokerd.ib` actor + a `TrioTaskExited` teardown
|
||||
> cascade]
|
||||
|
||||
Follow-up user direction fixing the design intent:
|
||||
|
||||
> i think the main thing here is that [brokerd] should
|
||||
> be able to always submit orders without a live feed
|
||||
> being up and registered in the same subactor right?
|
||||
|
||||
## Response summary
|
||||
|
||||
Root cause: pre-split the feed code populated the
|
||||
actor-local `Client._contracts` cache (via
|
||||
`get_mkt_info()` writing `mkt.bs_fqme` keys) in the
|
||||
SAME process as order submission; post-split the
|
||||
trading actor's client is never warmed. Fix: lazily
|
||||
qualify + cache the contract per order request in
|
||||
`handle_order_requests()` by running the same
|
||||
`get_mkt_info(fqme, proxy=...)` ep the feed side uses,
|
||||
plus per-order error relay (`BrokerdError`) so one bad
|
||||
submission can't crash the whole trades dialog (the
|
||||
`TrioTaskExited` storm was teardown cascade from the
|
||||
original raise).
|
||||
|
||||
## Files changed
|
||||
|
||||
- `piker/brokers/ib/broker.py` — thread `proxies` into
|
||||
`handle_order_requests()`; lazy contract qualify on
|
||||
cache-miss; guard `submit_limit()` w/ `BrokerdError`
|
||||
relay; uncomment the (anticipatory) `get_mkt_info`
|
||||
import
|
||||
- `piker/brokers/ib/api.py` — fix the non-f-string
|
||||
raise msg + document the new qualification contract
|
||||
|
||||
## Human edits
|
||||
|
||||
None — committed as generated. Live `ib` order retest
|
||||
performed by the human post-commit.
|
||||
|
|
@ -0,0 +1,57 @@
|
|||
---
|
||||
model: claude-fable-5[1m]
|
||||
service: claude
|
||||
timestamp: 2026-06-10T21:35:49Z
|
||||
git_ref: datad_service
|
||||
diff_cmd: git log -1 -p --follow -- ai/prompt-io/claude/20260610T213549Z_f084e899_prompt_io.md
|
||||
---
|
||||
|
||||
NOTE: diff-ref mode entry (code committed in the same
|
||||
commit as this log); recorded from the live debug
|
||||
session per the `/prompt-io` skill rules.
|
||||
|
||||
> `git log -1 -p --follow -- piker/brokers/ib/broker.py`
|
||||
> `git log -1 -p --follow -- piker/brokers/ib/api.py`
|
||||
|
||||
Key diagnostic chain (from session):
|
||||
|
||||
- pdb showed `Client._contracts == {}` inside
|
||||
`brokerd.ib`'s `submit_limit()`; the cache has
|
||||
exactly TWO write sites: `Client.find_contracts()`
|
||||
(api.py, keys `f'{sym}.{exch}.ib'`) and
|
||||
`symbols.get_mkt_info()` (key `mkt.bs_fqme`, eg.
|
||||
'nvda.nasdaq' — NO `.ib` suffix).
|
||||
- `BrokerdOrder.symbol` arrives as the bs_fqme form
|
||||
('nvda.nasdaq') so ONLY the `get_mkt_info()` write
|
||||
site produces the key `submit_limit()` reads —
|
||||
ie. pre-split it was the feed's in-proc
|
||||
`get_mkt_info(sym, proxy=proxy)` call keeping orders
|
||||
working, NOT `find_contracts()`.
|
||||
- the existing TODO at `symbols.py:642-644` literally
|
||||
predicted this: "this is going to be problematic
|
||||
if/when we split out the datad vs. brokerd actors
|
||||
since the mktmap lookup table will now be
|
||||
inaccessible.."
|
||||
- instance identity verified: `proxy._aio_ns` IS the
|
||||
same `Client` obj as `_accounts2clients[account]`
|
||||
(both sourced from the `load_aio_clients()` cache via
|
||||
`open_client_proxies()`), so a brokerd-side
|
||||
`get_mkt_info(fqme, proxy=proxies[account])` warms
|
||||
exactly the dict `submit_limit()` reads. It also
|
||||
populates `client._cons2mkts` which the
|
||||
position-audit path (`broker.py` backup-table code)
|
||||
needs in this actor anyway.
|
||||
- the `TrioTaskExited` storm in the user's log
|
||||
(`recv_trade_updates`, `open_aio_client_method_relay`
|
||||
aio tasks) is teardown cascade: the raise crashed
|
||||
`handle_order_requests` -> nursery teardown ripped
|
||||
the trio sides of still-running aio relay tasks.
|
||||
Hence the added per-order try/except ->
|
||||
`BrokerdError` relay hardening so a single bad
|
||||
submission degrades to an EMS error msg instead of
|
||||
killing the backend's entire order-ctl dialog.
|
||||
|
||||
Verification: `tests/test_services.py` (5 passed) +
|
||||
`tests/test_ems.py` (6 passed) regression-green; live
|
||||
`ib` submission retest delegated to the human (needs a
|
||||
running TWS/gw).
|
||||
|
|
@ -0,0 +1,34 @@
|
|||
# AI Prompt I/O Log — claude
|
||||
|
||||
This directory tracks prompt inputs and model
|
||||
outputs for AI-assisted development using
|
||||
`claude` (claude-code CLI).
|
||||
|
||||
## Policy
|
||||
|
||||
Prompt logging follows the
|
||||
[NLNet generative AI policy][nlnet-ai].
|
||||
All substantive AI contributions are logged
|
||||
with:
|
||||
- Model name and version
|
||||
- Timestamps
|
||||
- The prompts that produced the output
|
||||
- Unedited model output (`.raw.md` files)
|
||||
|
||||
[nlnet-ai]: https://nlnet.nl/foundation/policies/generativeAI/
|
||||
|
||||
## Usage
|
||||
|
||||
Entries are created by the `/prompt-io` skill
|
||||
or automatically via `/commit-msg` integration.
|
||||
|
||||
Each commit carrying AI-generated changes links
|
||||
to its provenance entry via a `Prompt-IO:`
|
||||
commit-msg trailer; entries use "diff-ref mode"
|
||||
(pointers into `git log -p` instead of verbatim
|
||||
code copies) to avoid duplicating committed
|
||||
code.
|
||||
|
||||
Human contributors remain accountable for all
|
||||
code decisions. AI-generated content is never
|
||||
presented as human-authored work.
|
||||
|
|
@ -0,0 +1,46 @@
|
|||
---
|
||||
model: gpt-5.6-sol
|
||||
provider: openai
|
||||
service: opencode
|
||||
session: ses_0799212ebffe42arY96czXn89F
|
||||
timestamp: 2026-07-22T00:13:57Z
|
||||
git_ref: 16f3cd6f
|
||||
scope: code
|
||||
substantive: true
|
||||
raw_file: 20260722T001357Z_16f3cd6f_prompt_io.raw.md
|
||||
---
|
||||
|
||||
## Prompt
|
||||
|
||||
Continue the gap-checker and history-backfiller repair plan. Start
|
||||
with deterministic analysis characterization, then fix canonical
|
||||
sorting, dedupe-derived columns, positive gap detection, and null
|
||||
grouping.
|
||||
|
||||
During review, the user challenged the replacement null-grouping
|
||||
algorithm because its initial tests did not assert exact endpoints.
|
||||
The user explicitly required retaining detailed, human-oriented
|
||||
algorithm documentation such as the existing ASCII segment map.
|
||||
|
||||
## Response summary
|
||||
|
||||
Added deterministic analysis regressions and corrected normalization
|
||||
so sorting and duplicate resolution precede derived-column
|
||||
calculation. Unified null grouping for NumPy and Polars inputs,
|
||||
documented its exact inclusive-boundary contract with an adapted
|
||||
visual map, and verified all small null layouts against an independent
|
||||
state-machine oracle. Retained sorted smart-dedupe output.
|
||||
|
||||
## Files changed
|
||||
|
||||
- `piker/tsp/_anal.py` - canonical gap and documented null grouping
|
||||
- `piker/tsp/_dedupe_smart.py` - sorted duplicate resolution
|
||||
- `tests/test_tsp_analysis.py` - exact and exhaustive regressions
|
||||
- `ai/prompt-io/opencode/20260722T001357Z_16f3cd6f_prompt_io.raw.md`
|
||||
- unedited response record
|
||||
- `ai/prompt-io/opencode/20260722T001357Z_16f3cd6f_prompt_io.md`
|
||||
- provenance metadata and response summary
|
||||
|
||||
## Human edits
|
||||
|
||||
None - generated changes have not yet been edited by the human.
|
||||
|
|
@ -0,0 +1,40 @@
|
|||
---
|
||||
model: gpt-5.6-sol
|
||||
provider: openai
|
||||
service: opencode
|
||||
timestamp: 2026-07-22T00:13:57Z
|
||||
git_ref: 16f3cd6f
|
||||
diff_cmd: git diff HEAD~1..HEAD
|
||||
---
|
||||
|
||||
> `git diff HEAD~1..HEAD -- piker/tsp/_anal.py`
|
||||
|
||||
Generated canonical gap and null-segment normalization. Derived
|
||||
datetime and delta columns are recomputed after final sorting and
|
||||
duplicate resolution, and negative deltas no longer become ordinary
|
||||
gaps.
|
||||
|
||||
NumPy and Polars null grouping share one implementation while
|
||||
retaining the human-oriented visual map of contiguous zero runs. The
|
||||
documented contract defines absolute inclusive endpoints, margin
|
||||
expansion, frame boundary clamping, contiguous-index validation, and
|
||||
requested-column selection.
|
||||
|
||||
> `git diff HEAD~1..HEAD -- piker/tsp/_dedupe_smart.py`
|
||||
|
||||
Adjusted smart dedupe to honor sorted output and resolve duplicate
|
||||
timestamps without retaining stale order.
|
||||
|
||||
> `git diff HEAD~1..HEAD -- tests/test_tsp_analysis.py`
|
||||
|
||||
Added deterministic synthetic OHLCV coverage for sorted dedupe, fresh
|
||||
derived columns, and positive gap detection. Null grouping is checked
|
||||
against readable endpoint examples and an independent state-machine
|
||||
oracle over every layout through six rows, three margin widths, and
|
||||
both NumPy and Polars inputs.
|
||||
|
||||
Verification generated with the patch:
|
||||
|
||||
`timeout -k 5 30 python -m pytest -q tests/test_tsp_analysis.py`
|
||||
|
||||
Result: 11 passed.
|
||||
|
|
@ -0,0 +1,36 @@
|
|||
---
|
||||
model: gpt-5.6-sol
|
||||
provider: openai
|
||||
service: opencode
|
||||
session: ses_0799212ebffe42arY96czXn89F
|
||||
timestamp: 2026-07-22T00:13:58Z
|
||||
git_ref: 16f3cd6f
|
||||
scope: code
|
||||
substantive: true
|
||||
raw_file: 20260722T001358Z_16f3cd6f_prompt_io.raw.md
|
||||
---
|
||||
|
||||
## Prompt
|
||||
|
||||
Continue the structured backfill phase after the analysis regressions.
|
||||
Characterize provider exhaustion and genuinely empty frames, then remove
|
||||
completion paths that can hang after child-task exit.
|
||||
|
||||
## Response summary
|
||||
|
||||
Removed orphan-prone completion events from the structured nursery flow
|
||||
and checked empty provider frames before indexing timestamp endpoints.
|
||||
Added deterministic timeout-bounded regressions for both exit paths.
|
||||
|
||||
## Files changed
|
||||
|
||||
- `piker/tsp/_history.py` - structured completion and empty-frame fixes
|
||||
- `tests/test_history_backfill.py` - deterministic exit regressions
|
||||
- `ai/prompt-io/opencode/20260722T001358Z_16f3cd6f_prompt_io.raw.md`
|
||||
- unedited response record
|
||||
- `ai/prompt-io/opencode/20260722T001358Z_16f3cd6f_prompt_io.md`
|
||||
- provenance metadata and response summary
|
||||
|
||||
## Human edits
|
||||
|
||||
None - generated changes have not yet been edited by the human.
|
||||
|
|
@ -0,0 +1,25 @@
|
|||
---
|
||||
model: gpt-5.6-sol
|
||||
provider: openai
|
||||
service: opencode
|
||||
timestamp: 2026-07-22T00:13:58Z
|
||||
git_ref: 16f3cd6f
|
||||
diff_cmd: git diff HEAD~1..HEAD
|
||||
---
|
||||
|
||||
> `git diff HEAD~1..HEAD -- piker/tsp/_history.py`
|
||||
|
||||
Removed redundant completion events whose consumers waited only after
|
||||
their owning nurseries had joined. Added empty provider-frame handling
|
||||
before timestamp endpoint indexing so provider exhaustion exits cleanly.
|
||||
|
||||
> `git diff HEAD~1..HEAD -- tests/test_history_backfill.py`
|
||||
|
||||
Added bounded, actor-free regressions for `DataUnavailable` and empty
|
||||
provider-frame completion.
|
||||
|
||||
Verification generated with the patch:
|
||||
|
||||
`timeout -k 5 60 python -m pytest -q tests/test_history_backfill.py`
|
||||
|
||||
Result: 2 passed.
|
||||
|
|
@ -0,0 +1,39 @@
|
|||
---
|
||||
model: gpt-5.6-sol
|
||||
provider: openai
|
||||
service: opencode
|
||||
session: ses_0799212ebffe42arY96czXn89F
|
||||
timestamp: 2026-07-22T02:41:33Z
|
||||
git_ref: 3a4f6737
|
||||
scope: code
|
||||
substantive: true
|
||||
raw_file: 20260722T024133Z_3a4f6737_prompt_io.raw.md
|
||||
---
|
||||
|
||||
## Prompt
|
||||
|
||||
The user challenged the analysis commit message's claim that only
|
||||
positive sample-step gaps are detected and asked whether out-of-order
|
||||
timestamps were simply ignored. After confirming the missing separate
|
||||
reporting path, the user instructed the agent to implement the fix.
|
||||
|
||||
## Response summary
|
||||
|
||||
Separated missing-history gaps from timestamp-ordering errors without
|
||||
changing the existing `dedupe()` return shape. Added a structured
|
||||
detector and made normalization warn before sorting removes evidence of
|
||||
duplicates or reversed timestamps.
|
||||
|
||||
## Files changed
|
||||
|
||||
- `piker/tsp/_anal.py` - ordering detector and dedupe reporting
|
||||
- `piker/tsp/__init__.py` - public detector export
|
||||
- `tests/test_tsp_analysis.py` - exact ordering-error regression
|
||||
- `ai/prompt-io/opencode/20260722T024133Z_3a4f6737_prompt_io.raw.md`
|
||||
- unedited response record
|
||||
- `ai/prompt-io/opencode/20260722T024133Z_3a4f6737_prompt_io.md`
|
||||
- provenance metadata and response summary
|
||||
|
||||
## Human edits
|
||||
|
||||
None - generated changes have not yet been edited by the human.
|
||||
|
|
@ -0,0 +1,30 @@
|
|||
---
|
||||
model: gpt-5.6-sol
|
||||
provider: openai
|
||||
service: opencode
|
||||
timestamp: 2026-07-22T02:41:33Z
|
||||
git_ref: 3a4f6737
|
||||
diff_cmd: git diff HEAD~1..HEAD
|
||||
---
|
||||
|
||||
> `git diff HEAD~1..HEAD -- piker/tsp/_anal.py`
|
||||
|
||||
Added `detect_time_ordering_errors()` to return non-positive timestamp
|
||||
steps as structured rows. `dedupe()` now warns with those rows before
|
||||
sorting and duplicate removal erase the original ordering evidence.
|
||||
|
||||
> `git diff HEAD~1..HEAD -- piker/tsp/__init__.py`
|
||||
|
||||
Exported the ordering detector through the public TSP package.
|
||||
|
||||
> `git diff HEAD~1..HEAD -- tests/test_tsp_analysis.py`
|
||||
|
||||
Changed the negative-delta regression to verify exact predecessor,
|
||||
timestamp, and delta values, confirm exclusion from missing-history
|
||||
gaps, and require explicit normalization reporting.
|
||||
|
||||
Verification generated with the patch:
|
||||
|
||||
- `tests/test_tsp_analysis.py`: 11 passed
|
||||
- Ruff: passed
|
||||
- `git diff --check`: passed
|
||||
|
|
@ -0,0 +1,43 @@
|
|||
---
|
||||
model: gpt-5.6-sol
|
||||
provider: openai
|
||||
service: opencode
|
||||
session: ses_0799212ebffe42arY96czXn89F
|
||||
timestamp: 2026-07-22T02:41:34Z
|
||||
git_ref: 3a4f6737
|
||||
scope: code
|
||||
substantive: true
|
||||
raw_file: 20260722T024134Z_3a4f6737_prompt_io.raw.md
|
||||
---
|
||||
|
||||
## Prompt
|
||||
|
||||
The user asked why the in-progress `piker/storage` changes were omitted
|
||||
from the earlier commit plan. After receiving an inventory of unfinished
|
||||
Phase 3 requirements, the user instructed the agent to complete the work.
|
||||
|
||||
## Response summary
|
||||
|
||||
Finished non-destructive, atomic NativeDB updates and integrated them
|
||||
with history startup and reverse backfill. Expanded tests through two
|
||||
adversarial review rounds that caught and corrected cross-client lock
|
||||
blocking, stale temp indexing, discontinuous persisted indexes, startup
|
||||
snapshot races, missing latest-frame persistence, notification wedges,
|
||||
and mutable/truncated SHM persistence.
|
||||
|
||||
## Files changed
|
||||
|
||||
- `piker/storage/__init__.py` - incremental storage contract
|
||||
- `piker/storage/nativedb.py` - validated atomic merge persistence
|
||||
- `piker/storage/marketstore/__init__.py` - non-duplicate update mode
|
||||
- `piker/tsp/_history.py` - provider-delta and startup persistence
|
||||
- `tests/test_storage_nativedb.py` - NativeDB durability regressions
|
||||
- `tests/test_history_backfill.py` - persistence lifecycle regressions
|
||||
- `ai/prompt-io/opencode/20260722T024134Z_3a4f6737_prompt_io.raw.md`
|
||||
- unedited response record
|
||||
- `ai/prompt-io/opencode/20260722T024134Z_3a4f6737_prompt_io.md`
|
||||
- provenance metadata and response summary
|
||||
|
||||
## Human edits
|
||||
|
||||
None - generated changes have not yet been edited by the human.
|
||||
|
|
@ -0,0 +1,43 @@
|
|||
---
|
||||
model: gpt-5.6-sol
|
||||
provider: openai
|
||||
service: opencode
|
||||
timestamp: 2026-07-22T02:41:34Z
|
||||
git_ref: 3a4f6737
|
||||
diff_cmd: git diff HEAD~1..HEAD
|
||||
---
|
||||
|
||||
> `git diff HEAD~1..HEAD -- piker/storage/__init__.py piker/storage/nativedb.py piker/storage/marketstore/__init__.py`
|
||||
|
||||
Added an incremental storage update contract. NativeDB now validates
|
||||
canonical numeric OHLCV data, serializes writers both actor-locally and
|
||||
through cancellable filesystem locks, merges with incoming-wins seam
|
||||
resolution, regenerates contiguous indexes, reopens and validates a
|
||||
same-directory temporary parquet, fsyncs it, atomically replaces the
|
||||
target, fsyncs the directory, and publishes cache only after replacement.
|
||||
|
||||
NativeDB indexing ignores lock and crash-left temporary files.
|
||||
MarketStore uses its non-duplicate write mode for incremental updates.
|
||||
|
||||
> `git diff HEAD~1..HEAD -- piker/tsp/_history.py`
|
||||
|
||||
Separated full provider deltas from capacity-truncated SHM pushes.
|
||||
Backfill persists before SHM mutation, publishes bounded notifications,
|
||||
and delays latest-frame publication until the concurrent pre-update
|
||||
storage snapshot completes.
|
||||
|
||||
> `git diff HEAD~1..HEAD -- tests/test_storage_nativedb.py tests/test_history_backfill.py`
|
||||
|
||||
Added deterministic coverage for merge preservation, conflict policy,
|
||||
atomic failures, reopened-candidate validation, cache consistency,
|
||||
schema and timestamp invariants, contiguous index regeneration,
|
||||
separate series, stale temp indexing, cancellable file-lock contention,
|
||||
full provider-delta persistence, latest-frame ordering, and bounded
|
||||
notification teardown.
|
||||
|
||||
Verification generated with the patch:
|
||||
|
||||
- deterministic analysis/backfill/storage set: 31 passed
|
||||
- Ruff: passed
|
||||
- `git diff --check`: passed
|
||||
- final adversarial review: no findings
|
||||
|
|
@ -0,0 +1,40 @@
|
|||
---
|
||||
model: gpt-5.6-sol
|
||||
provider: openai
|
||||
service: opencode
|
||||
session: unavailable
|
||||
timestamp: 2026-07-27T20:47:11Z
|
||||
git_ref: 84a6d47b
|
||||
scope: code
|
||||
substantive: true
|
||||
raw_file: 20260727T204711Z_84a6d47b_prompt_io.raw.md
|
||||
---
|
||||
|
||||
## Prompt
|
||||
|
||||
The user identified that `backfiller_deep_fixes` introduced persistent
|
||||
`.parquet.lock` files, questioned whether Parquet or Arrow already handled
|
||||
writer coordination, and clarified that piker's `tractor` structured
|
||||
concurrency design already provides single-writer ownership through the
|
||||
persistent `datad` feed task. The user then asked to open the backfiller
|
||||
worktree and resolve the redundant locking.
|
||||
|
||||
## Response summary
|
||||
|
||||
Removed redundant actor-local and filesystem locking while preserving
|
||||
atomic crash-safe Parquet replacement. Replaced the synthetic contention
|
||||
test with regressions for the actual cross-branch failure: writes create
|
||||
no lock sidecars, and legacy lock artifacts are ignored during indexing.
|
||||
|
||||
## Files changed
|
||||
|
||||
- `piker/storage/nativedb.py` - rely on `datad` writer ownership
|
||||
- `tests/test_storage_nativedb.py` - lock-sidecar regressions
|
||||
- `ai/prompt-io/opencode/20260727T204711Z_84a6d47b_prompt_io.raw.md`
|
||||
- unedited response record
|
||||
- `ai/prompt-io/opencode/20260727T204711Z_84a6d47b_prompt_io.md`
|
||||
- provenance metadata and response summary
|
||||
|
||||
## Human edits
|
||||
|
||||
None - generated changes have not been edited by the human.
|
||||
|
|
@ -0,0 +1,30 @@
|
|||
---
|
||||
model: gpt-5.6-sol
|
||||
provider: openai
|
||||
service: opencode
|
||||
timestamp: 2026-07-27T20:47:11Z
|
||||
git_ref: 84a6d47b
|
||||
diff_cmd: git diff HEAD~1..HEAD
|
||||
---
|
||||
|
||||
> `git diff HEAD~1..HEAD -- piker/storage/nativedb.py`
|
||||
|
||||
Removed actor-local and filesystem writer locks from NativeDB. The
|
||||
persistent `datad` feed task already owns writes for each series, while
|
||||
its timeframe children write distinct files. NativeDB retains validated
|
||||
temporary writes, file and directory synchronization, and atomic target
|
||||
replacement for crash durability. The exact `.parquet` suffix filter is
|
||||
retained so legacy lock sidecars are harmless.
|
||||
|
||||
> `git diff HEAD~1..HEAD -- tests/test_storage_nativedb.py`
|
||||
|
||||
Removed the synthetic cross-client lock-contention test. Added regression
|
||||
coverage proving replacement and incremental writes create no lock
|
||||
sidecars. Extended indexing coverage with the legacy
|
||||
`.parquet.lock` artifact that caused upstream `flake_update` to crash.
|
||||
|
||||
Verification generated with the patch:
|
||||
|
||||
- NativeDB and history regression set: 20 passed
|
||||
- Ruff: passed
|
||||
- `git diff --check`: passed
|
||||
|
|
@ -0,0 +1,42 @@
|
|||
---
|
||||
model: gpt-5.6-sol
|
||||
provider: openai
|
||||
service: opencode
|
||||
session: ses_0799212ebffe42arY96czXn89F
|
||||
timestamp: 2026-07-27T21:24:54Z
|
||||
git_ref: 689df816
|
||||
scope: code
|
||||
substantive: true
|
||||
raw_file: 20260727T212454Z_689df816_prompt_io.raw.md
|
||||
---
|
||||
|
||||
## Prompt
|
||||
|
||||
The user reran the known IB FQME on `backfiller_deep_fixes` and reported
|
||||
that `publish_latest_frame()` still failed because NativeDB required an
|
||||
`index` absent from IB's provider frame. The user called out the missed
|
||||
tabular schema boundary and requested an actual end-to-end test suite
|
||||
rather than storage fakes.
|
||||
|
||||
## Response summary
|
||||
|
||||
Repaired the IB-provider-to-NativeDB boundary by canonicalizing durable
|
||||
fields and mapping reloads into provider-specific SHM buffers. Added a
|
||||
local actor/SHM/Parquet integration regression covering first startup and
|
||||
restart, plus focused conversion and timestamp-coercion tests. Two review
|
||||
rounds found and resolved restart hydration and fractional-time hazards.
|
||||
|
||||
## Files changed
|
||||
|
||||
- `piker/storage/nativedb.py` - canonical storage and SHM field mapping
|
||||
- `piker/tsp/_anal.py` - name-based Polars-to-NumPy conversion
|
||||
- `tests/test_history_backfill.py` - actor/SHM/Parquet integration test
|
||||
- `tests/test_storage_nativedb.py` - schema conversion regressions
|
||||
- `ai/prompt-io/opencode/20260727T212454Z_689df816_prompt_io.raw.md`
|
||||
- unedited response record
|
||||
- `ai/prompt-io/opencode/20260727T212454Z_689df816_prompt_io.md`
|
||||
- provenance metadata and response summary
|
||||
|
||||
## Human edits
|
||||
|
||||
None - generated changes have not been edited by the human.
|
||||
|
|
@ -0,0 +1,34 @@
|
|||
---
|
||||
model: gpt-5.6-sol
|
||||
provider: openai
|
||||
service: opencode
|
||||
timestamp: 2026-07-27T21:24:54Z
|
||||
git_ref: 689df816
|
||||
diff_cmd: git diff HEAD~1..HEAD
|
||||
---
|
||||
|
||||
> `git diff HEAD~1..HEAD -- piker/storage/nativedb.py piker/tsp/_anal.py`
|
||||
|
||||
Normalized index-less provider frames to NativeDB's canonical names,
|
||||
order, and dtypes before merge and publication. Provider-only columns
|
||||
are excluded from durable files, derived indexes are regenerated, and
|
||||
fractional timestamps are rejected before integer conversion. Added a
|
||||
canonical field map for hydrating provider-specific SHM buffers while
|
||||
leaving provider-only fields at their defaults. Changed Polars-to-NumPy
|
||||
conversion to select columns by field name rather than position.
|
||||
|
||||
> `git diff HEAD~1..HEAD -- tests/test_history_backfill.py tests/test_storage_nativedb.py`
|
||||
|
||||
Added deterministic coverage using IB's actual provider dtype and a real
|
||||
local `tractor` root actor, IB-typed shared-memory buffers, NativeDB,
|
||||
Parquet persistence, first-start publication, and restart hydration.
|
||||
Added focused regressions for reordered/extra Polars columns and
|
||||
fractional timestamp rejection.
|
||||
|
||||
Verification generated with the patch:
|
||||
|
||||
- NativeDB and history integration set: 23 passed
|
||||
- Ruff: passed
|
||||
- `git diff --check`: passed
|
||||
- second adversarial review: no findings
|
||||
- no live gateway or network tests run
|
||||
|
|
@ -0,0 +1,41 @@
|
|||
---
|
||||
model: gpt-5.6-sol
|
||||
provider: openai
|
||||
service: opencode
|
||||
session: ses_0799212ebffe42arY96czXn89F
|
||||
timestamp: 2026-07-27T21:59:06Z
|
||||
git_ref: 689df816
|
||||
scope: code
|
||||
substantive: true
|
||||
raw_file: 20260727T215906Z_689df816_prompt_io.raw.md
|
||||
---
|
||||
|
||||
## Prompt
|
||||
|
||||
After the actor/SHM/Parquet integration test exposed a stale SHM triplet,
|
||||
the user requested leak-cleaner machinery in the pytest harness using
|
||||
Tractor's existing ownership and lifetime patterns.
|
||||
|
||||
## Response summary
|
||||
|
||||
Added creator-scoped SHM leak tracking to pytest without scanning
|
||||
`/dev/shm` or treating attachments as owned. The fixture cooperates with
|
||||
normal Tractor teardown, verifies POSIX object identity before fallback
|
||||
cleanup, restores process-local token state, and fails after cleaning a
|
||||
leak. Regressions cover the exact pre-actor-registration allocation window,
|
||||
positional creators, and external attachment safety.
|
||||
|
||||
## Files changed
|
||||
|
||||
- `tests/conftest.py` - creator-scoped SHM leak tracking fixture
|
||||
- `tests/test_shm_cleanup.py` - ownership and failure-window regressions
|
||||
- `.claude/skills/run-tests/test-harness-reference.md`
|
||||
- document current-process SHM cleanup behavior
|
||||
- `ai/prompt-io/opencode/20260727T215906Z_689df816_prompt_io.raw.md`
|
||||
- unedited response record
|
||||
- `ai/prompt-io/opencode/20260727T215906Z_689df816_prompt_io.md`
|
||||
- provenance metadata and response summary
|
||||
|
||||
## Human edits
|
||||
|
||||
None - generated changes have not been edited by the human.
|
||||
|
|
@ -0,0 +1,25 @@
|
|||
---
|
||||
model: gpt-5.6-sol
|
||||
provider: openai
|
||||
service: opencode
|
||||
timestamp: 2026-07-27T21:59:06Z
|
||||
git_ref: 689df816
|
||||
diff_cmd: git diff HEAD~1..HEAD
|
||||
---
|
||||
|
||||
> `git diff HEAD~1..HEAD -- tests/conftest.py tests/test_shm_cleanup.py .claude/skills/run-tests/test-harness-reference.md`
|
||||
|
||||
Added function-scoped SHM ownership tracking around Tractor's test-process
|
||||
factory. The fixture records only successful creators, removes ownership
|
||||
records during normal Tractor teardown, verifies POSIX object identity
|
||||
before fallback unlink, restores the token-cache baseline, and fails after
|
||||
cleaning any leak. Regressions reproduce allocation failure before actor
|
||||
lifetime registration and prove external attachments remain untouched.
|
||||
|
||||
Verification generated with the patch:
|
||||
|
||||
- SHM cleanup regressions: 2 passed
|
||||
- combined SHM, NativeDB, and history set: 25 passed
|
||||
- Ruff: passed
|
||||
- `git diff --check`: passed
|
||||
- final adversarial review: no findings
|
||||
|
|
@ -0,0 +1,42 @@
|
|||
---
|
||||
model: gpt-5.6-sol
|
||||
provider: openai
|
||||
service: opencode
|
||||
session: 24fb9765-a550-4570-8350-f0fc9b7e17db
|
||||
timestamp: 2026-07-28T00:32:47Z
|
||||
git_ref: ed85721c
|
||||
scope: code
|
||||
substantive: true
|
||||
raw_file: 20260728T003247Z_ed85721c_prompt_io.raw.md
|
||||
---
|
||||
|
||||
## Prompt
|
||||
|
||||
Build a safe xonsh workflow for qualifying NativeDB and reverse-backfill
|
||||
correctness against a known gappy IB chart, beginning with
|
||||
`mnq.cme.20260918`. Cover baseline, fresh, restart, append, and expansion
|
||||
testing through implementation, adversarial review, and verification.
|
||||
|
||||
## Response summary
|
||||
|
||||
Implemented a private disposable xonsh qualification workflow. Evidence is
|
||||
tied to a fixed case, report checksum, snapshot checksum, worktree import,
|
||||
and phase metadata. Added explicit path and symlink safety, source
|
||||
replacement handling, xonsh child environments, and lifecycle preservation
|
||||
comparisons.
|
||||
|
||||
## Files changed
|
||||
|
||||
- `snippets/nativedb_backfill_audit.xsh` - disposable qualification helpers
|
||||
- `docs/manual_backfill_qualification.rst` - operator lifecycle runbook
|
||||
- `tests/test_backfill_audit_snippet.py` - xonsh and comparison regressions
|
||||
- `.claude/skills/run-tests/test-harness-reference.md`
|
||||
- qualification test-harness guidance
|
||||
- `ai/prompt-io/opencode/20260728T003247Z_ed85721c_prompt_io.raw.md`
|
||||
- unedited response record
|
||||
- `ai/prompt-io/opencode/20260728T003247Z_ed85721c_prompt_io.md`
|
||||
- provenance metadata and response summary
|
||||
|
||||
## Human edits
|
||||
|
||||
None - generated changes have not been edited by the human.
|
||||
|
|
@ -0,0 +1,32 @@
|
|||
---
|
||||
model: gpt-5.6-sol
|
||||
provider: openai
|
||||
service: opencode
|
||||
timestamp: 2026-07-28T00:32:47Z
|
||||
git_ref: ed85721c
|
||||
diff_cmd: git diff HEAD~1..HEAD
|
||||
---
|
||||
|
||||
> `git diff HEAD~1..HEAD -- snippets/nativedb_backfill_audit.xsh docs/manual_backfill_qualification.rst tests/test_backfill_audit_snippet.py .claude/skills/run-tests/test-harness-reference.md`
|
||||
|
||||
Added sourceable xonsh helpers that create a private disposable root, bind
|
||||
child processes to the reviewed worktree, capture checksum-bound phase
|
||||
evidence, safely seed and clear test history, and compare lifecycle
|
||||
preservation.
|
||||
|
||||
The workflow rejects path traversal, pre-existing symlinks, output
|
||||
collisions, mismatched case/report/snapshot evidence, and same-phase
|
||||
comparison. The accompanying runbook covers known-bad replay, fresh
|
||||
backfill, restart, append, gap triage, and expansion cases.
|
||||
|
||||
Verification generated with the patch:
|
||||
|
||||
- xonsh qualification helper regressions: passed
|
||||
- xonsh sourceability: passed
|
||||
- Ruff: passed
|
||||
- `git diff --check`: passed
|
||||
|
||||
Adversarial review drove additional evidence binding, worktree executable,
|
||||
environment propagation, source replacement, and descriptor-level safety
|
||||
fixes. Residual concurrent mutation risk is stated explicitly in the private
|
||||
disposable-root runbook.
|
||||
|
|
@ -0,0 +1,40 @@
|
|||
---
|
||||
model: gpt-5.6-sol
|
||||
provider: openai
|
||||
service: opencode
|
||||
session: 24fb9765-a550-4570-8350-f0fc9b7e17db
|
||||
timestamp: 2026-07-28T00:50:34Z
|
||||
git_ref: ed85721c
|
||||
scope: code
|
||||
substantive: true
|
||||
raw_file: 20260728T005034Z_ed85721c_prompt_io.raw.md
|
||||
---
|
||||
|
||||
## Prompt
|
||||
|
||||
Add reusable read-only tooling to qualify raw NativeDB Parquet against a
|
||||
known gappy IB chart, beginning with `mnq.cme.20260918`. Preserve malformed
|
||||
evidence and distinguish structural validity from unclassified positive
|
||||
gaps without opening or mutating the storage runtime.
|
||||
|
||||
## Response summary
|
||||
|
||||
Implemented raw Parquet audit reporting with human and JSON output, bounded
|
||||
cadence details, exact integer arithmetic, strict status, collision-safe
|
||||
snapshot capture, and source path protections. Regressions cover malformed
|
||||
schema and values, timestamp precision, gap arithmetic, corrupt bytes,
|
||||
copy failures, symlinks, traversal, and read-only CLI behavior.
|
||||
|
||||
## Files changed
|
||||
|
||||
- `piker/storage/_audit.py` - raw Parquet audit and snapshot implementation
|
||||
- `piker/storage/cli.py` - `piker store audit` command
|
||||
- `tests/test_storage_audit.py` - audit and CLI regressions
|
||||
- `ai/prompt-io/opencode/20260728T005034Z_ed85721c_prompt_io.raw.md`
|
||||
- unedited response record
|
||||
- `ai/prompt-io/opencode/20260728T005034Z_ed85721c_prompt_io.md`
|
||||
- provenance metadata and response summary
|
||||
|
||||
## Human edits
|
||||
|
||||
None - generated changes have not been edited by the human.
|
||||
|
|
@ -0,0 +1,27 @@
|
|||
---
|
||||
model: gpt-5.6-sol
|
||||
provider: openai
|
||||
service: opencode
|
||||
timestamp: 2026-07-28T00:50:34Z
|
||||
git_ref: ed85721c
|
||||
diff_cmd: git diff HEAD~1..HEAD
|
||||
---
|
||||
|
||||
> `git diff HEAD~1..HEAD -- piker/storage/_audit.py piker/storage/cli.py tests/test_storage_audit.py`
|
||||
|
||||
Added a raw, read-only NativeDB Parquet audit with human and JSON output,
|
||||
bounded cadence evidence, exact integer arithmetic, pre-parse snapshot
|
||||
capture, and strict qualification status.
|
||||
|
||||
The command rejects unsafe FQME paths, symlinked sources and outputs,
|
||||
destination collisions, non-regular source nodes, malformed canonical
|
||||
schema, non-finite values, invalid timestamps, noncanonical indexes, and
|
||||
sub-period cadence. Copy-failure cleanup preserves source bytes while
|
||||
completed corrupt snapshots remain available for diagnosis.
|
||||
|
||||
Verification generated with the patch:
|
||||
|
||||
- storage audit and CLI regressions: passed
|
||||
- Ruff: passed
|
||||
- `git diff --check`: passed
|
||||
- final adversarial review: no findings
|
||||
|
|
@ -0,0 +1,39 @@
|
|||
---
|
||||
model: gpt-5.6-sol
|
||||
provider: openai
|
||||
service: opencode
|
||||
session: 24fb9765-a550-4570-8350-f0fc9b7e17db
|
||||
timestamp: 2026-07-28T23:27:18Z
|
||||
git_ref: ed85721c
|
||||
scope: code
|
||||
substantive: true
|
||||
raw_file: 20260728T232718Z_ed85721c_prompt_io.raw.md
|
||||
---
|
||||
|
||||
## Prompt
|
||||
|
||||
Diagnose the live `pikerd` failure where backfill publication raised
|
||||
`ValueError: OHLCV timestamps must be finite and positive` while merging the
|
||||
known gappy MNQ NativeDB baseline with fresh IB history. Determine whether to
|
||||
commit the pending audit stack or iterate first.
|
||||
|
||||
## Response summary
|
||||
|
||||
Traced the primary failure through `publish_latest_frame()` to
|
||||
`NativeStorageClient.update_ohlcv()`. The incoming frame was valid, but the
|
||||
canonicalized persisted baseline retained its epoch-zero row and caused the
|
||||
merged frame to fail validation. Added persisted-row repair before merge and
|
||||
a regression using the observed legacy schema characteristics.
|
||||
|
||||
## Files changed
|
||||
|
||||
- `piker/storage/nativedb.py` - drop non-positive persisted rows during merge
|
||||
- `tests/test_storage_nativedb.py` - live MNQ failure-shape regression
|
||||
- `ai/prompt-io/opencode/20260728T232718Z_ed85721c_prompt_io.raw.md`
|
||||
- unedited response record
|
||||
- `ai/prompt-io/opencode/20260728T232718Z_ed85721c_prompt_io.md`
|
||||
- provenance metadata and response summary
|
||||
|
||||
## Human edits
|
||||
|
||||
None - generated changes have not been edited by the human.
|
||||
|
|
@ -0,0 +1,27 @@
|
|||
---
|
||||
model: gpt-5.6-sol
|
||||
provider: openai
|
||||
service: opencode
|
||||
timestamp: 2026-07-28T23:27:18Z
|
||||
git_ref: ed85721c
|
||||
diff_cmd: git diff HEAD~1..HEAD
|
||||
---
|
||||
|
||||
> `git diff HEAD~1..HEAD -- piker/storage/nativedb.py tests/test_storage_nativedb.py`
|
||||
|
||||
Fixed incremental NativeDB updates against legacy persisted history with
|
||||
non-positive timestamps. Incoming provider frames remain strictly validated,
|
||||
while merge repair discards invalid persisted rows before concat, dedupe,
|
||||
ordering, canonical-index rewrite, and atomic publication.
|
||||
|
||||
The regression reproduces the observed MNQ baseline shape: one epoch-zero
|
||||
row, noncanonical absolute indexes, extra derived columns, and a valid fresh
|
||||
IB frame. It proves the invalid row is removed while all valid old and new
|
||||
bars survive with canonical indexes.
|
||||
|
||||
Verification generated with the patch:
|
||||
|
||||
- NativeDB regressions: 18 passed
|
||||
- history backfill regressions: 6 passed
|
||||
- focused Ruff: passed
|
||||
- `git diff --check`: passed
|
||||
|
|
@ -0,0 +1,40 @@
|
|||
---
|
||||
model: gpt-5.6-sol
|
||||
provider: openai
|
||||
service: opencode
|
||||
session: 24fb9765-a550-4570-8350-f0fc9b7e17db
|
||||
timestamp: 2026-07-29T03:41:06Z
|
||||
git_ref: 922a5df8
|
||||
scope: code
|
||||
substantive: true
|
||||
raw_file: 20260729T034106Z_922a5df8_prompt_io.raw.md
|
||||
---
|
||||
|
||||
## Prompt
|
||||
|
||||
Diagnose the live IB reverse-fill failure where
|
||||
`reqHistoricalDataAsync()` raised API error 10314 because its end date,
|
||||
time, or timezone was invalid.
|
||||
|
||||
## Response summary
|
||||
|
||||
Traced the failure to `Client.bars()`, where an unfinished EST conversion
|
||||
left the original datetime for `ib_async` to serialize as
|
||||
`YYYYMMDD HH:MM:SS UTC`. Converted aware boundaries explicitly to UTC and
|
||||
passed IB's accepted `YYYYMMDD-HH:MM:SS` string while retaining a blank
|
||||
latest-request boundary. Added exact request-argument regressions.
|
||||
|
||||
## Files changed
|
||||
|
||||
- `piker/brokers/ib/api.py` - explicit UTC-dash history boundary format
|
||||
- `tests/test_ib_history.py` - IB request-format regressions
|
||||
- `.claude/skills/run-tests/test-harness-reference.md`
|
||||
- deterministic IB history test mapping
|
||||
- `ai/prompt-io/opencode/20260729T034106Z_922a5df8_prompt_io.raw.md`
|
||||
- unedited response record
|
||||
- `ai/prompt-io/opencode/20260729T034106Z_922a5df8_prompt_io.md`
|
||||
- provenance metadata and response summary
|
||||
|
||||
## Human edits
|
||||
|
||||
None - generated changes have not been edited by the human.
|
||||
|
|
@ -0,0 +1,25 @@
|
|||
---
|
||||
model: gpt-5.6-sol
|
||||
provider: openai
|
||||
service: opencode
|
||||
timestamp: 2026-07-29T03:41:06Z
|
||||
git_ref: 922a5df8
|
||||
diff_cmd: git diff HEAD~1..HEAD
|
||||
---
|
||||
|
||||
> `git diff HEAD~1..HEAD -- piker/brokers/ib/api.py tests/test_ib_history.py .claude/skills/run-tests/test-harness-reference.md`
|
||||
|
||||
Fixed IB reverse-history requests to serialize aware end boundaries with
|
||||
IB's explicit UTC-dash syntax, `YYYYMMDD-HH:MM:SS`. This bypasses
|
||||
`ib_async`'s trailing-`UTC` representation, which the live Gateway rejected
|
||||
with API error 10314, and removes the unfinished EST comparison breakpoint.
|
||||
|
||||
The regression captures the exact `reqHistoricalDataAsync()` argument for
|
||||
latest, stdlib-aware, Pendulum UTC, and America/New_York inputs.
|
||||
|
||||
Verification generated with the patch:
|
||||
|
||||
- IB history request regressions: 4 passed
|
||||
- focused Ruff: passed
|
||||
- `git diff --check`: passed
|
||||
- adversarial review: no findings
|
||||
|
|
@ -0,0 +1,40 @@
|
|||
---
|
||||
model: gpt-5.6-sol
|
||||
provider: openai
|
||||
service: opencode
|
||||
session: 24fb9765-a550-4570-8350-f0fc9b7e17db
|
||||
timestamp: 2026-07-29T04:17:23Z
|
||||
git_ref: ce33deb6
|
||||
scope: code
|
||||
substantive: true
|
||||
raw_file: 20260729T041723Z_ce33deb6_prompt_io.raw.md
|
||||
---
|
||||
|
||||
## Prompt
|
||||
|
||||
Diagnose why the chart still showed a zero first OHLCV row after the
|
||||
epoch-zero timestamp repair, causing auto-y-ranging to include zero.
|
||||
|
||||
## Response summary
|
||||
|
||||
Audited live 60s and 1s NativeDB files and found one exact 60s first-slot
|
||||
sentinel at epoch 60 with a completely zero OHLCV payload. Added narrowly
|
||||
scoped read and merge repair, matching incoming validation, sentinel-only
|
||||
startup handling, canonical reindexing, and explicit audit classification.
|
||||
Preserved modern all-zero bars as unclassified evidence rather than assuming
|
||||
all zero-priced instruments are corrupt.
|
||||
|
||||
## Files changed
|
||||
|
||||
- `piker/storage/nativedb.py` - first-slot sentinel repair and validation
|
||||
- `piker/storage/_audit.py` - sentinel classification and zero-row evidence
|
||||
- `tests/test_storage_nativedb.py` - hydration, merge, and boundary regressions
|
||||
- `tests/test_storage_audit.py` - sentinel and modern-zero audit regressions
|
||||
- `ai/prompt-io/opencode/20260729T041723Z_ce33deb6_prompt_io.raw.md`
|
||||
- unedited response record
|
||||
- `ai/prompt-io/opencode/20260729T041723Z_ce33deb6_prompt_io.md`
|
||||
- provenance metadata and response summary
|
||||
|
||||
## Human edits
|
||||
|
||||
None - generated changes have not been edited by the human.
|
||||
|
|
@ -0,0 +1,31 @@
|
|||
---
|
||||
model: gpt-5.6-sol
|
||||
provider: openai
|
||||
service: opencode
|
||||
timestamp: 2026-07-29T04:17:23Z
|
||||
git_ref: ce33deb6
|
||||
diff_cmd: git diff HEAD~1..HEAD
|
||||
---
|
||||
|
||||
> `git diff HEAD~1..HEAD -- piker/storage/nativedb.py piker/storage/_audit.py tests/test_storage_nativedb.py tests/test_storage_audit.py`
|
||||
|
||||
Fixed the remaining legacy first-slot sentinel observed in live MNQ 60s
|
||||
history: `time=60` with zero open, high, low, close, and volume. NativeDB now
|
||||
repairs the exact sentinel before read hydration and merge, reindexes the
|
||||
filtered view, and persists the repaired frame on the next valid update.
|
||||
|
||||
The repair remains narrowly scoped. Modern all-zero OHLC bars stay valid for
|
||||
spreads or synthetic instruments, null-containing malformed rows still fail
|
||||
validation, new writes can not recreate the first-slot sentinel, and a file
|
||||
containing only sentinels loads as no history so fresh backfill can proceed.
|
||||
|
||||
The raw audit reports total all-zero price rows, classifies first-slot
|
||||
sentinels as structural violations, and leaves later all-zero rows visible as
|
||||
unclassified warnings.
|
||||
|
||||
Verification generated with the patch:
|
||||
|
||||
- NativeDB, audit, history, xonsh, and IB regressions: 61 passed
|
||||
- focused Ruff: passed
|
||||
- `git diff --check`: passed
|
||||
- final adversarial review: no findings
|
||||
|
|
@ -0,0 +1,34 @@
|
|||
---
|
||||
model: openai/gpt-5.6-sol
|
||||
service: opencode
|
||||
session: 6409e3bb-1a9a-4ada-b6c4-c554115143e9
|
||||
timestamp: 2026-07-29T18:54:06Z
|
||||
git_ref: d359b1cb
|
||||
scope: code
|
||||
substantive: true
|
||||
raw_file: 20260729T185406Z_d359b1cb_prompt_io.raw.md
|
||||
---
|
||||
|
||||
## Prompt
|
||||
|
||||
The user reported that QQQ 60-second zeros appeared as a flat line after
|
||||
reload, clarified that the outstanding concern was the prior `ldshm`
|
||||
failure, and instructed the agent to implement the more focused fix.
|
||||
|
||||
## Response summary
|
||||
|
||||
Changed `ldshm` to classify invalid SHM snapshots before downstream
|
||||
analysis. Cadence is inferred only from observed finite positive steps;
|
||||
buffers containing non-finite or non-positive timestamps, or lacking a
|
||||
positive cadence, are skipped rather than partially replacement-written
|
||||
or reloaded. Added deterministic helper and command-level regressions.
|
||||
|
||||
## Files changed
|
||||
|
||||
- `piker/storage/cli.py` - inspect cadence and reject invalid SHM snapshots before mutation.
|
||||
- `tests/test_ldshm.py` - cover QQQ-like zeros, corrupt timestamps, cadence selection, degenerate buffers, and mutation prevention.
|
||||
- `.claude/skills/run-tests/test-harness-reference.md` - register the deterministic regression target.
|
||||
|
||||
## Human edits
|
||||
|
||||
None - generated output remains uncommitted.
|
||||
|
|
@ -0,0 +1,58 @@
|
|||
---
|
||||
model: openai/gpt-5.6-sol
|
||||
service: opencode
|
||||
timestamp: 2026-07-29T18:54:06Z
|
||||
git_ref: d359b1cb
|
||||
diff_cmd: git diff HEAD~1..HEAD
|
||||
---
|
||||
|
||||
The user clarified that the QQQ 60-second chart now displayed a flat
|
||||
forward-filled segment, then asked whether the previously reported
|
||||
`piker store ldshm` failure had been fixed. After being told it had not,
|
||||
the user instructed: "ok but i read above you have a more focussed fix
|
||||
todo now? so have at it!"
|
||||
|
||||
The implementation focused on the exact failure boundary. It does not
|
||||
filter invalid rows into a partial replacement and does not mutate live
|
||||
SHM. Instead, `ldshm` inspects one captured Polars snapshot, infers its
|
||||
cadence from observed finite positive timestamp steps, and skips the
|
||||
entire buffer before analysis, persistence, or reload when any timestamp
|
||||
is non-finite or non-positive.
|
||||
|
||||
> `git diff HEAD~1..HEAD -- piker/storage/cli.py`
|
||||
|
||||
Generated `_shm_period_and_invalid_count()` and integrated its result in
|
||||
`ldshm`. The helper selects the most frequent observed positive step,
|
||||
using the smallest observed step as the deterministic tie-break. Short
|
||||
or cadence-degenerate buffers are skipped. Invalid snapshots are logged
|
||||
and cannot reach deduplication, NativeDB replacement, or SHM reload.
|
||||
|
||||
> `git diff HEAD~1..HEAD -- tests/test_ldshm.py`
|
||||
|
||||
Generated deterministic regressions for the QQQ-shaped interior zero
|
||||
run, observed-step cadence selection, all invalid timestamp classes,
|
||||
short and degenerate buffers, and command-level prevention of dedupe,
|
||||
storage writes, and reload. The command-level test uses an immutable
|
||||
Polars snapshot and failing spies, without live SHM mutation or timing.
|
||||
|
||||
> `git diff HEAD~1..HEAD -- .claude/skills/run-tests/test-harness-reference.md`
|
||||
|
||||
Added the deterministic `ldshm` regression target and its change-to-test
|
||||
mapping to the repository test harness reference.
|
||||
|
||||
Verification output:
|
||||
|
||||
```text
|
||||
.......................................................... [100%]
|
||||
58 passed in 0.70s
|
||||
```
|
||||
|
||||
`git diff --check` passed. Ruff was unavailable in the worktree's
|
||||
existing environment. Three adversarial review passes found and drove
|
||||
fixes for partial-replacement data loss, invented median timeframes,
|
||||
mixed SHM observations, and pre-guard null-analysis failures. The final
|
||||
review reported no remaining findings.
|
||||
|
||||
The patch intentionally does not claim that `ldshm` is safe to run as a
|
||||
general concurrent repair writer. It handles invalid live snapshots by
|
||||
refusing mutation rather than weakening NativeDB validation.
|
||||
|
|
@ -0,0 +1,37 @@
|
|||
---
|
||||
model: openai/gpt-5.6-sol
|
||||
service: opencode
|
||||
session: 253ade33-07ba-45b5-8839-be35d17ec164
|
||||
timestamp: 2026-07-29T20:40:46Z
|
||||
git_ref: ad6c560f
|
||||
scope: code
|
||||
substantive: true
|
||||
raw_file: 20260729T204046Z_ad6c560f_prompt_io.raw.md
|
||||
---
|
||||
|
||||
## Prompt
|
||||
|
||||
The user reported that the QQQ 1-second chart still contained a gap after
|
||||
the `ldshm` fix, asked the agent to inspect it, and authorized continued
|
||||
implementation after a short pause and shutdown of the live workspace.
|
||||
|
||||
## Response summary
|
||||
|
||||
Diagnosed a 653-row active RT SHM null reservation caused by inclusive IB
|
||||
reverse-query overlaps and startup repair orchestration. Prevented extra
|
||||
physical SHM overlap rows, serialized and bounded IB history recovery,
|
||||
made blank/error responses terminate, and constrained synthetic repair to
|
||||
valid interior timestamp boundaries. Added deterministic concurrency and
|
||||
data-layout regressions.
|
||||
|
||||
## Files changed
|
||||
|
||||
- `piker/brokers/ib/feed.py` - bound and serialize IB history reset/retry behavior.
|
||||
- `piker/tsp/_history.py` - prevent SHM overlap skew and safely finalize null repair.
|
||||
- `tests/test_ib_history.py` - cover blank, cancellation, reset, and pacing interleavings.
|
||||
- `tests/test_history_backfill.py` - cover overlap and null-repair invariants.
|
||||
- `.claude/skills/run-tests/test-harness-reference.md` - register deterministic coverage.
|
||||
|
||||
## Human edits
|
||||
|
||||
None - generated output remains uncommitted.
|
||||
|
|
@ -0,0 +1,72 @@
|
|||
---
|
||||
model: openai/gpt-5.6-sol
|
||||
service: opencode
|
||||
timestamp: 2026-07-29T20:40:46Z
|
||||
git_ref: ad6c560f
|
||||
diff_cmd: git diff HEAD~1..HEAD
|
||||
---
|
||||
|
||||
The user reported that QQQ still displayed a gap on the 1-second chart
|
||||
after the focused `ldshm` fix and asked the agent to inspect it. The user
|
||||
then paused and resumed implementation while shutting down the live piker
|
||||
trading workspace.
|
||||
|
||||
Read-only live inspection found the active datad generation held 653
|
||||
contiguous zero-time rows in QQQ's 1-second RT SHM, while durable Parquet
|
||||
contained no zero rows. The run occupied absolute indexes 168148 through
|
||||
168800 between timestamps 1785348398 and 1785349051. The stale actor
|
||||
generation and active 60-second history were separately identified.
|
||||
|
||||
> `git diff HEAD~1..HEAD -- piker/tsp/_history.py`
|
||||
|
||||
Generated history changes that exclude each inclusive reverse-query
|
||||
endpoint already present in SHM while retaining the full provider frame
|
||||
for NativeDB merge. Startup null repair now runs after reverse backfill,
|
||||
uses bounded UI notifications, handles provider no-data without an
|
||||
interactive pause, fills only exact interior zero groups, and refuses
|
||||
synthetic timestamps unless both valid boundaries establish the expected
|
||||
cadence.
|
||||
|
||||
> `git diff HEAD~1..HEAD -- piker/brokers/ib/feed.py`
|
||||
|
||||
Generated IB history reset changes that make blank responses complete,
|
||||
bound repeated cancellation and reset failures per request, serialize
|
||||
timeout and pacing farm resets with one actor-global Trio lock, carry
|
||||
explicit reset results, and coordinate pacing handoff with queued timeout
|
||||
waiters.
|
||||
|
||||
> `git diff HEAD~1..HEAD -- tests/test_history_backfill.py`
|
||||
|
||||
Generated regressions for inclusive endpoint exclusion, full NativeDB
|
||||
provider deltas, strict synthetic gap boundaries, empty-provider local
|
||||
fallback, bounded notifications, and preservation of valid margin rows.
|
||||
|
||||
> `git diff HEAD~1..HEAD -- tests/test_ib_history.py`
|
||||
|
||||
Generated deterministic regressions for blank IB responses, cancellation
|
||||
limits, stalled reset cancellation, pacing reset failure limits,
|
||||
cross-request reset serialization, timeout-to-pacing handoff, and
|
||||
successful handoff timeout restart.
|
||||
|
||||
> `git diff HEAD~1..HEAD -- .claude/skills/run-tests/test-harness-reference.md`
|
||||
|
||||
Registered the deterministic history-backfill target and expanded the IB
|
||||
history mapping to include `feed.py`.
|
||||
|
||||
Verification output:
|
||||
|
||||
```text
|
||||
........................................................................ [ 93%]
|
||||
..... [100%]
|
||||
77 passed in 2.12s
|
||||
```
|
||||
|
||||
`git diff --check` passed. Multiple adversarial review rounds identified
|
||||
and drove fixes for unbounded IB empty/cancellation/reset paths, stale
|
||||
reset ownership, actor-global failure poisoning, concurrent farm resets,
|
||||
double-counted pacing handoffs, physical endpoint overlap, and unsafe
|
||||
synthetic timestamp boundaries. Final review reported no findings.
|
||||
|
||||
Live validation was intentionally deferred because the user shut down the
|
||||
trading workspace. A fresh datad/chart startup is required to regenerate
|
||||
SHM and verify that QQQ no longer contains the 653-row null run.
|
||||
|
|
@ -0,0 +1,41 @@
|
|||
---
|
||||
model: openai/gpt-5.6-sol
|
||||
service: opencode
|
||||
session: ses_0799212ebffe42arY96czXn89F
|
||||
timestamp: 2026-07-31T16:52:15Z
|
||||
git_ref: 0846cbd4
|
||||
scope: code
|
||||
substantive: true
|
||||
raw_file: 20260731T165215Z_0846cbd4_prompt_io.raw.md
|
||||
---
|
||||
|
||||
## Prompt
|
||||
|
||||
Evaluate and update `piker store` so bare groups and endpoints immediately
|
||||
show help; prepare for eventual optional fuzzy selection in `anal` and
|
||||
`ldshm`; aggregate recent time-series diagnostics into CLI endpoints. The
|
||||
user selected the safe-foundation scope.
|
||||
|
||||
## Response summary
|
||||
|
||||
Added immediate help UX, exact NativeDB and SHM identities, read-only
|
||||
`series` and `shm` diagnostics, selector-ready exact SHM names, and robust
|
||||
discovery/attachment. Fixed related `ldshm` option and no-match failures.
|
||||
Deferred optional `fzf` and mutation-default changes.
|
||||
|
||||
## Files changed
|
||||
|
||||
- `piker/storage/cli.py` - help UX and read-only series/SHM diagnostics.
|
||||
- `piker/storage/nativedb.py` - canonical compound series identities.
|
||||
- `piker/tsp/_history.py` - exact SHM discovery and dtype resolution.
|
||||
- `piker/data/_sharedmem.py` - attach-only existing-segment helper.
|
||||
- `piker/tsp/__init__.py` - SHM identity helper exports.
|
||||
- `tests/test_store_cli.py` - command UX and endpoint regressions.
|
||||
- `tests/test_ldshm.py` - SHM diagnostics and operational regressions.
|
||||
- `tests/test_storage_nativedb.py` - series parsing/index regressions.
|
||||
- `docs/manual_backfill_qualification.rst` - triage command documentation.
|
||||
- `.claude/skills/run-tests/test-harness-reference.md` - test mapping.
|
||||
|
||||
## Human edits
|
||||
|
||||
None - generated output remains uncommitted.
|
||||
|
|
@ -0,0 +1,93 @@
|
|||
---
|
||||
model: openai/gpt-5.6-sol
|
||||
service: opencode
|
||||
timestamp: 2026-07-31T16:52:15Z
|
||||
git_ref: 0846cbd4
|
||||
diff_cmd: git diff HEAD~1..HEAD
|
||||
---
|
||||
|
||||
The user requested an evaluation and possible update of `piker store`,
|
||||
including immediate help output when a group or endpoint is invoked with
|
||||
no arguments. The user also proposed eventual optional `fzf` selection for
|
||||
`anal` and `ldshm`, and authorized aggregating the recent time-series and
|
||||
SHM debugging work into CLI endpoints. After reviewing three scopes, the
|
||||
user selected the recommended safe foundation rather than implementing
|
||||
interactive fuzzy selection or changing mutation defaults immediately.
|
||||
|
||||
> `git diff HEAD~1..HEAD -- piker/storage/cli.py`
|
||||
|
||||
Generated no-argument help behavior for the `store` group and every
|
||||
endpoint. Added read-only `series` and `shm` commands, JSON and Rich output,
|
||||
exact `--shm-name` plumbing, SHM cadence/invalid/order/gap summaries, and
|
||||
safe corrupt-epoch formatting. Fixed existing `ldshm` no-write/no-reload
|
||||
control flow and no-match debugger behavior.
|
||||
|
||||
> `git diff HEAD~1..HEAD -- piker/storage/nativedb.py`
|
||||
|
||||
Generated canonical `NativeSeriesRef` parsing and deterministic listing.
|
||||
Changed the internal index to preserve `(fqme, period)` identities while
|
||||
keeping unique-FQME compatibility listing and cardinality. Hardened
|
||||
malformed, sidecar, Unicode-period, symlink, and traversal handling.
|
||||
|
||||
> `git diff HEAD~1..HEAD -- piker/tsp/_history.py`
|
||||
|
||||
Generated exact `ShmBufferRef` parsing and stable discovery, replacing
|
||||
unsafe substring globbing. Added dtype resolution that mirrors history
|
||||
allocation, exact-ref attachment, disappearance handling, and optional
|
||||
exact-name iteration.
|
||||
|
||||
> `git diff HEAD~1..HEAD -- piker/data/_sharedmem.py`
|
||||
|
||||
Generated attach-only SHM access which never creates a segment when a
|
||||
discovered object disappears.
|
||||
|
||||
> `git diff HEAD~1..HEAD -- piker/tsp/__init__.py`
|
||||
|
||||
Exported selector-ready SHM identity, parsing, listing, and dtype helpers.
|
||||
|
||||
> `git diff HEAD~1..HEAD -- tests/test_store_cli.py`
|
||||
|
||||
Generated command-level help, exact durable-series JSON, read-only SHM JSON,
|
||||
and unknown exact-name regressions.
|
||||
|
||||
> `git diff HEAD~1..HEAD -- tests/test_ldshm.py`
|
||||
|
||||
Generated SHM identity, summary, ordering, dtype, forged-name, disabled
|
||||
mutation-flag, and gap-markup regressions.
|
||||
|
||||
> `git diff HEAD~1..HEAD -- tests/test_storage_nativedb.py`
|
||||
|
||||
Generated compound index, canonical parser, malformed filename, and dotted
|
||||
FQME regressions.
|
||||
|
||||
> `git diff HEAD~1..HEAD -- docs/manual_backfill_qualification.rst`
|
||||
|
||||
Documented `store series` and `store shm` as separate read-only durable and
|
||||
live-memory triage layers.
|
||||
|
||||
> `git diff HEAD~1..HEAD -- .claude/skills/run-tests/test-harness-reference.md`
|
||||
|
||||
Registered the deterministic storage CLI UX target and mapping.
|
||||
|
||||
Verification output:
|
||||
|
||||
```text
|
||||
........................................................................ [ 71%]
|
||||
............................. [100%]
|
||||
101 passed in 1.72s
|
||||
```
|
||||
|
||||
Real `piker store` and `piker store audit` invocations printed complete
|
||||
help without `--help`, exiting with Click's normal no-args help status 2.
|
||||
`git diff --check` passed. Ruff was unavailable in the existing worktree
|
||||
environment.
|
||||
|
||||
Adversarial review drove fixes for provider dtype reconstruction, observed
|
||||
versus expected cadence, exact SHM grammar, discovery races, Typer direct
|
||||
defaults, malformed NativeDB periods, compound cardinality compatibility,
|
||||
undefined `ldshm` reload payloads, no-match debugger hangs, ordering
|
||||
corruption reporting, and dtype precedence. Final review found no issues.
|
||||
|
||||
Optional external `fzf` or built-in numbered selection remains deferred.
|
||||
The exact series and SHM identities added here are the intended foundation
|
||||
for that later patch.
|
||||
|
|
@ -0,0 +1,39 @@
|
|||
---
|
||||
model: openai/gpt-5.6-sol
|
||||
service: opencode
|
||||
session: ses_0799212ebffe42arY96czXn89F
|
||||
timestamp: 2026-08-01T01:53:21Z
|
||||
git_ref: 74ea2152
|
||||
scope: code
|
||||
substantive: true
|
||||
raw_file: 20260801T015321Z_74ea2152_prompt_io.raw.md
|
||||
---
|
||||
|
||||
## Prompt
|
||||
|
||||
Make `piker store ls` default to all NativeDB series as a compact counterpart
|
||||
to detailed `series`, with sensible no-flag output and a visible help
|
||||
description. Consolidate SHM inspection and repair under canonical `shm`,
|
||||
because attachment to volatile segments is implied, and make inspection the
|
||||
safe default while retaining explicit persistence and reload controls.
|
||||
|
||||
## Response summary
|
||||
|
||||
Replaced backend-oriented `ls` with compact NativeDB discovery and made both
|
||||
listing commands default to all series. Consolidated SHM behavior under a
|
||||
read-only-by-default `shm` command, guarded explicit repair options, updated
|
||||
executable references and docs, and added command regressions.
|
||||
|
||||
## Files changed
|
||||
|
||||
- `piker/storage/cli.py` - listing defaults and unified SHM command modes.
|
||||
- `tests/test_store_cli.py` - listing, help, and SHM mode regressions.
|
||||
- `tests/test_ldshm.py` - explicit persistence-path regressions.
|
||||
- `docs/manual_backfill_qualification.rst` - revised safe CLI workflow.
|
||||
- `snippets/claude_debug_helper.py` - explicit SHM persistence commands.
|
||||
- `.claude/skills/run-tests/test-harness-reference.md` - safety and test map.
|
||||
- `.claude/skills/piker-slang/examples.md` - canonical command example.
|
||||
|
||||
## Human edits
|
||||
|
||||
None - generated output incorporates the user's design refinements.
|
||||
|
|
@ -0,0 +1,78 @@
|
|||
---
|
||||
model: openai/gpt-5.6-sol
|
||||
service: opencode
|
||||
timestamp: 2026-08-01T01:53:21Z
|
||||
git_ref: 74ea2152
|
||||
diff_cmd: git diff HEAD~1..HEAD
|
||||
---
|
||||
|
||||
The user requested command-surface refinements after reviewing the new
|
||||
storage CLI. Bare `ls` should default to all series and act as a compact
|
||||
form of detailed `series`; commands should produce sensible default output
|
||||
without requiring trigger flags; and `ls` needed a parent-help description.
|
||||
The user first proposed folding read-only SHM inspection into `ldshm -r`,
|
||||
then clarified that `shm` is the better canonical command because attaching
|
||||
to volatile segments is implied by the resource name.
|
||||
|
||||
The resulting design uses `piker store ls` as a compact NativeDB FQME and
|
||||
period view, while `piker store series` reports each exact sub-series with
|
||||
size and path details. Both default to all records and accept an optional
|
||||
FQME substring.
|
||||
|
||||
`piker store shm FQME` is read-only by default. Persistence and reload enter
|
||||
the prior repair workflow only through explicit `--write-parquet` and
|
||||
`--reload-parquet-to-shm` flags. Reload requires persistence, and read-only
|
||||
formatting flags are rejected in repair mode instead of being ignored.
|
||||
|
||||
> `git diff HEAD~1..HEAD -- piker/storage/cli.py`
|
||||
|
||||
Generated shared NativeDB payload collection, compact and detailed output,
|
||||
numeric period ordering, default-all semantics, the canonical `shm` command,
|
||||
safe mode dispatch, explicit repair validation, and debugger propagation in
|
||||
read-only mode.
|
||||
|
||||
> `git diff HEAD~1..HEAD -- tests/test_store_cli.py`
|
||||
|
||||
Generated regressions for no-flag listing, parent-help descriptions, numeric
|
||||
period display, default read-only SHM JSON, reload dependencies, and repair
|
||||
versus read-only option conflicts.
|
||||
|
||||
> `git diff HEAD~1..HEAD -- tests/test_ldshm.py`
|
||||
|
||||
Adjusted persistence regressions for the renamed command and explicit repair
|
||||
entry while preserving the historical failure rationale.
|
||||
|
||||
> `git diff HEAD~1..HEAD -- docs/manual_backfill_qualification.rst`
|
||||
|
||||
Documented compact versus detailed NativeDB discovery, default read-only SHM
|
||||
inspection, and the explicitly mutating repair invocation.
|
||||
|
||||
> `git diff HEAD~1..HEAD -- snippets/claude_debug_helper.py`
|
||||
|
||||
Updated executable debug helper commands to select explicit SHM persistence.
|
||||
|
||||
> `git diff HEAD~1..HEAD -- .claude/skills/run-tests/test-harness-reference.md`
|
||||
|
||||
Updated operational safety guidance and SHM regression mappings.
|
||||
|
||||
> `git diff HEAD~1..HEAD -- .claude/skills/piker-slang/examples.md`
|
||||
|
||||
Updated the command example to the canonical `shm` spelling.
|
||||
|
||||
Verification output:
|
||||
|
||||
```text
|
||||
........................................................................ [ 69%]
|
||||
................................ [100%]
|
||||
104 passed in 5.77s
|
||||
```
|
||||
|
||||
Real command-help rendering was checked for the parent group, `ls`, `series`,
|
||||
and `shm`. Real read-only `ls` and `series --json` output was also exercised
|
||||
against the configured NativeDB. `git diff --check` passed.
|
||||
|
||||
Adversarial review identified and drove fixes for lexicographic period
|
||||
ordering, stale runbook safety wording, ignored mode-specific options, and an
|
||||
explicit-default `--max-gaps 10` validation bypass. Final review found no
|
||||
remaining issues. Real OS-backed SHM attachment and successful repair/reload
|
||||
remain integration-level validation concerns.
|
||||
|
|
@ -0,0 +1,33 @@
|
|||
---
|
||||
model: openai/gpt-5.6-sol
|
||||
service: opencode
|
||||
session: ses_0799212ebffe42arY96czXn89F
|
||||
timestamp: 2026-08-05T19:50:01Z
|
||||
git_ref: f0f3c806
|
||||
scope: code
|
||||
substantive: true
|
||||
raw_file: 20260805T195001Z_f0f3c806_prompt_io.raw.md
|
||||
---
|
||||
|
||||
## Prompt
|
||||
|
||||
Inspect the completed NVDA 60s SHM backfill before workspace restart, explain
|
||||
the persistent zero region, and continue fixing the indexing logic so sparse
|
||||
IB history does not leave physical null reservations.
|
||||
|
||||
## Response summary
|
||||
|
||||
Captured a 42,720-row live SHM null span whose valid margins shared the same
|
||||
timestamp. Replaced wall-clock row reservation with actual provider-boundary
|
||||
placement, fixed stale SHM first-index, endpoint insertion and field-map
|
||||
semantics, published the final storage prepend, and added a full multi-frame
|
||||
`tsdb_backfill()` regression using a real `ShmArray`.
|
||||
|
||||
## Files changed
|
||||
|
||||
- `piker/tsp/_history.py` - contiguous provider and NativeDB prepends.
|
||||
- `tests/test_history_backfill.py` - sparse multi-frame SHM regression.
|
||||
|
||||
## Human edits
|
||||
|
||||
None - generated output incorporates the user's live observations.
|
||||
|
|
@ -0,0 +1,71 @@
|
|||
---
|
||||
model: openai/gpt-5.6-sol
|
||||
service: opencode
|
||||
timestamp: 2026-08-05T19:50:01Z
|
||||
git_ref: f0f3c806
|
||||
diff_cmd: git diff HEAD~1..HEAD
|
||||
---
|
||||
|
||||
The user reported that a completed NVDA 60s backfill left a large zero
|
||||
region in live SHM and asked for immediate inspection before restarting the
|
||||
workspace. The captured buffer contained 42,720 invalid rows at absolute
|
||||
indexes `3065895..3108614`. The valid rows on both sides had the identical
|
||||
timestamp `1781271360`, proving the region was an erroneous physical-index
|
||||
reservation rather than a real temporal gap.
|
||||
|
||||
The user then asked the agent to continue fixing the backfill logic.
|
||||
|
||||
> `git diff HEAD~1..HEAD -- piker/tsp/_history.py`
|
||||
|
||||
Removed wall-clock-based SHM reservation from `tsdb_backfill()`. Reverse
|
||||
provider frames now extend the visible `ShmArray._first` boundary on every
|
||||
prepend, beginning exactly at the current first index after inclusive query
|
||||
endpoints are removed. Added `_prepend_tsdb_history()` to trim storage rows
|
||||
against the actual earliest provider timestamp and prepend them directly
|
||||
adjacent only after reverse provider retrieval completes.
|
||||
|
||||
This preserves sparse venue history as ordinal samples rather than treating
|
||||
every elapsed calendar period as a physical SHM row. It also prevents the
|
||||
old `_first + 1` insertion point from overwriting the earliest published bar.
|
||||
|
||||
> `git diff HEAD~1..HEAD -- tests/test_history_backfill.py`
|
||||
|
||||
Added a full `tsdb_backfill()` regression modeled on the live NVDA failure.
|
||||
It uses a real `ShmArray`, NativeDB's field map, two inclusive reverse-provider
|
||||
frames, and overlapping stored history. Assertions prove the complete
|
||||
timestamp sequence is contiguous and lossless, contains no non-positive
|
||||
timestamps, sends complete provider frames to durable storage, and notifies
|
||||
chart/FSP consumers after the final storage prepend.
|
||||
|
||||
Live evidence captured before workspace restart:
|
||||
|
||||
```text
|
||||
rows 1575296 first_index 1568233 last_index 3143528
|
||||
invalid_count 42720
|
||||
left boundary
|
||||
[(3065893, 1781271300, 204.03, 204.82)
|
||||
(3065894, 1781271360, 204.81, 204.72)
|
||||
(3065895, 0, 0. , 0. )]
|
||||
right boundary
|
||||
[(3108614, 0, 0. , 0. )
|
||||
(3108615, 1781271360, 204.81, 205.44)
|
||||
(3108616, 1781271420, 205.46, 205.48)]
|
||||
```
|
||||
|
||||
Verification output:
|
||||
|
||||
```text
|
||||
........................................................................ [ 68%]
|
||||
................................. [100%]
|
||||
105 passed in 4.80s
|
||||
```
|
||||
|
||||
Adversarial review caught stale `_first` semantics and an off-by-one
|
||||
insertion point in the initial patch. The first live restart then exposed an
|
||||
undefined `time_key` local at the new `tsdb_backfill()` call site, which
|
||||
crashed `datad.ib` and propagated to the chart as `RemoteActorError`. The
|
||||
helper now derives the source timestamp field from its field map, and the
|
||||
test enters the full enclosing lifecycle so the original `NameError` can not
|
||||
recur unnoticed. A final review also drove explicit publication after stored
|
||||
history is prepended and deterministic final-stage synchronization. Final
|
||||
review found no remaining issues; another live restart remains pending.
|
||||
|
|
@ -0,0 +1,36 @@
|
|||
---
|
||||
model: openai/gpt-5.6-sol
|
||||
service: opencode
|
||||
session: ses_0799212ebffe42arY96czXn89F
|
||||
timestamp: 2026-08-11T20:58:26Z
|
||||
git_ref: 06d5ea5d
|
||||
scope: code
|
||||
substantive: true
|
||||
raw_file: 20260811T205826Z_06d5ea5d_prompt_io.raw.md
|
||||
---
|
||||
|
||||
## Prompt
|
||||
|
||||
Diagnose and fix an IB live-order failure where lazy NVDA contract
|
||||
qualification raised `tractor.trionics.Lagged` after the order reached
|
||||
`brokerd.ib`. Determine whether recent history changes or cancelled symbol
|
||||
searches caused the stale state.
|
||||
|
||||
## Response summary
|
||||
|
||||
Traced the failure to competing consumers of a one-slot Trio broadcast, not
|
||||
the recent SHM change or stale asyncio client state. Replaced broadcast
|
||||
fan-out with one response/status demultiplexer, retained request-ID
|
||||
correlation, handled cancellation and teardown, documented the task/channel
|
||||
ownership model, warned on diagnostic metadata eviction, and added
|
||||
deterministic proxy regressions.
|
||||
|
||||
## Files changed
|
||||
|
||||
- `piker/brokers/ib/api.py` - sole-reader IB proxy response routing.
|
||||
- `tests/test_ib_method_proxy.py` - status, concurrency, cancellation and EOF regressions.
|
||||
- `.claude/skills/run-tests/test-harness-reference.md` - proxy test mapping.
|
||||
|
||||
## Human edits
|
||||
|
||||
None - generated output follows the user's live failure diagnosis.
|
||||
|
|
@ -0,0 +1,66 @@
|
|||
---
|
||||
model: openai/gpt-5.6-sol
|
||||
service: opencode
|
||||
timestamp: 2026-08-11T20:58:26Z
|
||||
git_ref: 06d5ea5d
|
||||
diff_cmd: git diff HEAD~1..HEAD
|
||||
---
|
||||
|
||||
The user reported that certain IB order entries might have broken and
|
||||
provided a live failure for a first NVDA sell order. EMS delivered the
|
||||
request to `brokerd.ib`, but lazy `get_mkt_info('nvda.nasdaq')` contract
|
||||
qualification failed in `MethodProxy._run_method()` with:
|
||||
|
||||
```text
|
||||
tractor.trionics._broadcast.Lagged:
|
||||
Task `piker.brokers.ib.broker.handle_order_requests` overrun and
|
||||
dropped `0` values
|
||||
```
|
||||
|
||||
The user suspected a cancelled symbol-search task might have left stale
|
||||
asyncio state. Cancellation can leave an orphaned method response, but the
|
||||
observed failure was stale Trio broadcast state. `open_client_proxy()`
|
||||
broadcast a one-slot `LinkedTaskChannel` to an always-running IB status relay
|
||||
and an idle method receiver. Status traffic advanced only the relay, so the
|
||||
first method call for an uncached symbol entered an already-lagged receiver.
|
||||
Orders for eagerly cached contracts bypassed this path.
|
||||
|
||||
The immediately preceding sparse-history commit did not modify IB order,
|
||||
symbol, or proxy code; it exposed no direct dependency on this failure.
|
||||
|
||||
> `git diff HEAD~1..HEAD -- piker/brokers/ib/api.py`
|
||||
|
||||
Generated a sole-reader message demultiplexer for each IB client proxy.
|
||||
`relay_client_proxy_messages()` owns every asyncio-to-Trio receive, handles
|
||||
status/error traffic, and dispatches `mid`-tagged responses into per-call
|
||||
`MethodProxy` wait slots. Cancelled calls remove their pending slot, causing
|
||||
late responses to be logged and dropped without skewing later calls. Clean
|
||||
channel EOF is accepted during normal proxy teardown.
|
||||
|
||||
Follow-up review restored the masked reconnect TODO and per-message case
|
||||
rationale, documented the one-reader/many-writer task hierarchy and separate
|
||||
per-proxy channels, and added a warning with request method metadata when the
|
||||
bounded unresolved-request diagnostic table evicts its oldest entry.
|
||||
|
||||
> `git diff HEAD~1..HEAD -- tests/test_ib_method_proxy.py`
|
||||
|
||||
Generated deterministic regressions for idle status traffic before first
|
||||
qualification, two concurrent calls with reverse-order responses, cancelled
|
||||
calls with late responses followed by a successful call, and graceful
|
||||
channel EOF. Every synchronization wait is timeout-bounded.
|
||||
|
||||
> `git diff HEAD~1..HEAD -- .claude/skills/run-tests/test-harness-reference.md`
|
||||
|
||||
Registered the deterministic IB method-proxy target and source mapping.
|
||||
|
||||
Verification output:
|
||||
|
||||
```text
|
||||
........................ [100%]
|
||||
25 passed, 4 warnings in 3.44s
|
||||
```
|
||||
|
||||
The warnings are existing Tractor/Trio deprecations from the EMS test.
|
||||
Adversarial review drove a clean-EOF guard and bounded polling. Final review
|
||||
found no actionable issues. Live IB order validation remains pending and
|
||||
requires restarting `brokerd.ib`.
|
||||
|
|
@ -0,0 +1,34 @@
|
|||
---
|
||||
model: openai/gpt-5.6-sol
|
||||
service: opencode
|
||||
session: unavailable
|
||||
timestamp: 2026-08-13T20:24:02Z
|
||||
git_ref: 12b0d66e
|
||||
scope: code
|
||||
substantive: true
|
||||
raw_file: 20260813T202402Z_12b0d66e_prompt_io.raw.md
|
||||
---
|
||||
|
||||
## Prompt
|
||||
|
||||
Continue the paired piker broadcast-consumer work after Tractor PR 490
|
||||
stabilization. Stop cached feed callers from mutating or consuming shared
|
||||
quote-stream roots, then review, verify and prepare a commit boundary.
|
||||
|
||||
## Response summary
|
||||
|
||||
Made provider root streams private to the cached `Feed`, gave every caller a
|
||||
local subscribed view, corrected feed cache identity, and made multi-provider
|
||||
relays own synchronized child subscriptions and independent send-channel
|
||||
clones.
|
||||
|
||||
## Files changed
|
||||
|
||||
- `piker/data/feed.py` - caller-local feed views, complete cache keys, and
|
||||
subscribed multi-provider relays with independent channel lifetimes.
|
||||
- `tests/test_feed_broadcast.py` - real broadcast ownership, lifecycle and
|
||||
cache-identity regressions, including sibling survival after provider EOC.
|
||||
|
||||
## Human edits
|
||||
|
||||
None - generated output follows the requested cached-feed ownership item.
|
||||
|
|
@ -0,0 +1,51 @@
|
|||
---
|
||||
model: openai/gpt-5.6-sol
|
||||
service: opencode
|
||||
timestamp: 2026-08-13T20:24:02Z
|
||||
git_ref: 12b0d66e
|
||||
diff_cmd: git diff HEAD~1..HEAD
|
||||
---
|
||||
|
||||
The user asked to continue the paired piker broadcast-consumer work after
|
||||
Tractor PR 490 stabilization. The next item was to stop cached feed callers
|
||||
from mutating or consuming shared `Feed`/`Flume` quote-stream roots, then
|
||||
review, verify and prepare a commit boundary.
|
||||
|
||||
> `git diff HEAD~1..HEAD -- piker/data/feed.py`
|
||||
|
||||
Changed `maybe_open_feed()` so the cached `Feed` exclusively owns provider
|
||||
root streams and every caller, including the first cache owner, receives a
|
||||
caller-local `Feed`/`Flume` view backed by one child subscription per provider.
|
||||
The cache identity now includes normalized full symbol shape and stream policy.
|
||||
|
||||
Changed `Feed.open_multi_stream()` so every provider relay subscribes instead
|
||||
of directly consuming a shared stream, and synchronizes relay startup before
|
||||
yielding the merged channel. Each relay owns a cloned send channel so one
|
||||
provider EOC cannot terminate its siblings. The module-level relay returns its
|
||||
live child receiver through `TaskStatus` for typed retention and introspection.
|
||||
Documented `Feed.pause()`/`Feed.resume()` as provider-context-wide controls
|
||||
shared by local cache users.
|
||||
|
||||
> `git diff HEAD~1..HEAD -- tests/test_feed_broadcast.py`
|
||||
|
||||
Added real `Feed`, `Flume`, and Tractor broadcast regressions covering both
|
||||
cache-miss owner and cache-hit paths, one-provider-many-market mapping,
|
||||
multi-provider relay fan-out and startup, duplex controls, lexical child
|
||||
closure, sibling survival after provider EOC, cached-root immutability, and
|
||||
full cache-key differentiation.
|
||||
|
||||
Verification output:
|
||||
|
||||
```text
|
||||
... [100%]
|
||||
3 passed in 0.02s
|
||||
|
||||
. [100%]
|
||||
1 passed in 0.01s
|
||||
```
|
||||
|
||||
The second result is the existing EMS broadcast ownership regression. Ruff,
|
||||
E501, compilation and whitespace checks passed. Repeated adversarial review
|
||||
found and resolved shared `Feed.streams`, incomplete cache-key, fake-lifecycle,
|
||||
relay-startup and first-owner asymmetry issues; final review reported no
|
||||
findings.
|
||||
|
|
@ -0,0 +1,32 @@
|
|||
---
|
||||
model: openai/gpt-5.6-sol
|
||||
service: opencode
|
||||
session: unavailable
|
||||
timestamp: 2026-08-13T23:07:44Z
|
||||
git_ref: ddac3553
|
||||
scope: code
|
||||
substantive: true
|
||||
raw_file: 20260813T230744Z_ddac3553_prompt_io.raw.md
|
||||
---
|
||||
|
||||
## Prompt
|
||||
|
||||
Continue the piker broadcast-consumer audit after the Deribit ownership commit
|
||||
and prepare the next smallest ownership fix.
|
||||
|
||||
## Response summary
|
||||
|
||||
Replaced IB's manual raw quote-channel registry with a symbol-keyed cached
|
||||
context and gave every caller a lexical child from one retained
|
||||
`LinkedTaskChannel` broadcaster root.
|
||||
|
||||
## Files changed
|
||||
|
||||
- `piker/brokers/ib/feed.py` - cached raw-channel lifetime and lexical child
|
||||
ownership.
|
||||
- `tests/test_ib_broadcast.py` - concurrent fan-out, cache-key and child
|
||||
lifetime regression.
|
||||
|
||||
## Human edits
|
||||
|
||||
None - generated output follows the requested broadcast-consumer audit.
|
||||
|
|
@ -0,0 +1,39 @@
|
|||
---
|
||||
model: openai/gpt-5.6-sol
|
||||
service: opencode
|
||||
timestamp: 2026-08-13T23:07:44Z
|
||||
git_ref: ddac3553
|
||||
diff_cmd: git diff HEAD~1..HEAD
|
||||
---
|
||||
|
||||
The user asked to continue the piker broadcast-consumer audit after committing
|
||||
the Deribit ownership fix. The next isolated item was IB's actor-local
|
||||
`_quote_streams` cache, which retained a raw channel owned by its first caller
|
||||
and constructed competing broadcaster roots for later callers.
|
||||
|
||||
> `git diff HEAD~1..HEAD -- piker/brokers/ib/feed.py`
|
||||
|
||||
Factored `_open_aio_quote_channel()` as the raw asyncio-channel context and
|
||||
made `open_aio_quote_stream()` cache that context by symbol through
|
||||
`maybe_open_context()`. Every caller now receives a lexical child from the
|
||||
retained `LinkedTaskChannel` broadcaster, while the raw source remains alive
|
||||
until its final cache user exits. Removed the manual `_quote_streams` registry
|
||||
and teardown mutation.
|
||||
|
||||
> `git diff HEAD~1..HEAD -- tests/test_ib_broadcast.py`
|
||||
|
||||
Added a concurrent ownership regression using real Tractor broadcast state.
|
||||
It proves miss and hit callers use the same symbol/context factory, receive the
|
||||
same quote through distinct children, close independently, and leave the owner
|
||||
live after the hit exits.
|
||||
|
||||
Verification output:
|
||||
|
||||
```text
|
||||
. [100%]
|
||||
1 passed in 0.01s
|
||||
```
|
||||
|
||||
The new test passes through a source-isolated IB feed import. Test Ruff, E501,
|
||||
Python compilation and whitespace checks pass. Normal package collection
|
||||
remains blocked by the existing Qt environment mismatch.
|
||||
|
|
@ -0,0 +1,43 @@
|
|||
---
|
||||
model: gpt-5.6-sol
|
||||
service: opencode
|
||||
session: tractor-addr-unpacking
|
||||
timestamp: 2026-08-20T16:35:19Z
|
||||
git_ref: 5f631e19
|
||||
scope: code
|
||||
substantive: true
|
||||
raw_file: 20260820T163519Z_5f631e19_prompt_io.raw.md
|
||||
---
|
||||
|
||||
## Prompt
|
||||
|
||||
The human asked to implement the Piker compatibility work caused by
|
||||
tractor's canonical tagged-address migration, run the previously
|
||||
deferred tractor verification, and keep runtime-managed WireGuard
|
||||
bindspaces in a later separate PR.
|
||||
|
||||
## Response summary
|
||||
|
||||
Made Piker normalize legacy and tagged registry inputs before semantic
|
||||
comparison and process-global storage. Updated runtime address typing and
|
||||
test assertions for canonical output, retained legacy caller input, and
|
||||
removed a duplicate pytest log-level option that blocked downstream test
|
||||
collection with current tractor.
|
||||
|
||||
## Files changed
|
||||
|
||||
- `piker/service/_registry.py` - canonical registry normalization and
|
||||
shared address typing.
|
||||
- `piker/service/_actor_runtime.py` - tagged-compatible runtime API
|
||||
annotations.
|
||||
- `tests/conftest.py` - canonical fixture comparison and shared pytest
|
||||
option ownership.
|
||||
- `tests/test_services.py` - registry regression and canonical runtime
|
||||
assertions.
|
||||
|
||||
## Human edits
|
||||
|
||||
The human explicitly requested the downstream implementation and chose
|
||||
to keep WireGuard bindspace lifecycle work in a separate future tractor
|
||||
PR. The agent implemented and tested those directions; no direct manual
|
||||
source edits were observed.
|
||||
|
|
@ -0,0 +1,25 @@
|
|||
---
|
||||
model: gpt-5.6-sol
|
||||
service: opencode
|
||||
timestamp: 2026-08-20T16:35:19Z
|
||||
git_ref: 5f631e19
|
||||
diff_cmd: git diff HEAD~1..HEAD
|
||||
---
|
||||
|
||||
# Raw output - tractor tagged-address compatibility
|
||||
|
||||
The human requested downstream Piker qualification after tractor PR
|
||||
#505 switched transport address emission to canonical tagged tuples.
|
||||
|
||||
> `git diff HEAD~1..HEAD -- piker/service/_registry.py piker/service/_actor_runtime.py tests/conftest.py tests/test_services.py`
|
||||
|
||||
Normalized legacy and tagged registry addresses through tractor's
|
||||
address codec before Piker stores or compares them. Extended service
|
||||
runtime annotations to cover both tuple shapes, updated runtime tests to
|
||||
compare canonical serialized values, and removed Piker's duplicate
|
||||
registration of tractor's shared `--ll` pytest option.
|
||||
|
||||
Ruff passed. The focused normalization regression and Piker runtime boot
|
||||
passed against the local tractor checkout at `5d92595f`. The wider
|
||||
service module reached an unrelated pre-existing logger-name assertion
|
||||
in `maybe_spawn_daemon()` after both address-focused tests passed.
|
||||
|
|
@ -0,0 +1,39 @@
|
|||
---
|
||||
model: gpt-5.6-sol
|
||||
service: opencode
|
||||
session: tractor-addr-unpacking
|
||||
timestamp: 2026-08-21T04:20:58Z
|
||||
git_ref: b7aca4c0
|
||||
scope: code
|
||||
substantive: true
|
||||
raw_file: 20260821T042058Z_b7aca4c0_prompt_io.raw.md
|
||||
---
|
||||
|
||||
## Prompt
|
||||
|
||||
The human approved refining Piker's logger-name handling after the
|
||||
wider service test exposed a stale parent-logger assertion during the
|
||||
tractor tagged-address compatibility work.
|
||||
|
||||
## Response summary
|
||||
|
||||
Changed Piker's logger helpers to remove only an exact `piker.` prefix,
|
||||
preserving complete requested submodule names. Removed daemon startup
|
||||
assertions that incorrectly required parent logger names and added a
|
||||
regression covering both logger factories and names that the old
|
||||
character-set stripping silently corrupted.
|
||||
|
||||
## Files changed
|
||||
|
||||
- `piker/log.py` - exact relative logger-name derivation.
|
||||
- `piker/service/_daemon.py` - remove stale parent-name assertion.
|
||||
- `piker/data/_daemon.py` - remove stale parent-name assertion.
|
||||
- `piker/brokers/_daemon.py` - remove stale parent-name assertion.
|
||||
- `piker/clearing/_ems.py` - remove redundant logger assertion.
|
||||
- `tests/test_logging.py` - package logger namespace regressions.
|
||||
|
||||
## Human edits
|
||||
|
||||
The human explicitly approved this follow-up after reviewing the
|
||||
residual Piker service-suite gap. The agent implemented and verified
|
||||
the refinement; no direct manual source edits were observed.
|
||||
|
|
@ -0,0 +1,24 @@
|
|||
---
|
||||
model: gpt-5.6-sol
|
||||
service: opencode
|
||||
timestamp: 2026-08-21T04:20:58Z
|
||||
git_ref: b7aca4c0
|
||||
diff_cmd: git diff HEAD~1..HEAD
|
||||
---
|
||||
|
||||
# Raw output - preserve Piker logger namespaces
|
||||
|
||||
The human approved a follow-up refinement after the full Piker service
|
||||
module exposed a stale logger-name assertion while qualifying tractor
|
||||
PR #505.
|
||||
|
||||
> `git diff HEAD~1..HEAD -- piker/log.py piker/service/_daemon.py piker/data/_daemon.py piker/brokers/_daemon.py piker/clearing/_ems.py tests/test_logging.py`
|
||||
|
||||
Replaced character-set `str.lstrip()` calls with exact package-prefix
|
||||
removal, removed assertion-only parent-logger expectations from daemon
|
||||
startup paths, and added regression coverage for both logger factories
|
||||
and prefix-sensitive submodule names.
|
||||
|
||||
Ruff passed after removing newly unused logger assignments. All four
|
||||
logger regressions and all six service integration tests passed against
|
||||
the local tractor PR #505 checkout.
|
||||
|
|
@ -0,0 +1,46 @@
|
|||
---
|
||||
model: openai/gpt-5.6-sol
|
||||
service: opencode
|
||||
session: unavailable
|
||||
timestamp: 2026-08-26T22:26:00Z
|
||||
git_ref: 5f631e19
|
||||
scope: code
|
||||
substantive: true
|
||||
raw_file: 20260826T222600Z_5f631e19_prompt_io.raw.md
|
||||
---
|
||||
|
||||
## Prompt
|
||||
|
||||
Diagnose and fix the hanging Binance USD-M perpetual feed for
|
||||
`NVDAUSDT.USDTM.PERP.BINANCE` by comparing piker with Binance's current
|
||||
API and changelog. Ensure unsupported or silent websocket feeds fail
|
||||
loudly into `datad` and chart consumers instead of blocking startup.
|
||||
|
||||
## Response summary
|
||||
|
||||
Migrates Binance USD-M market data from the retired legacy websocket to
|
||||
the split public and market routes, handles Binance's new aggregate-trade
|
||||
schema, and introduces bounded connection and first-data startup. Direct
|
||||
stream URLs avoid reconnect-time subscription-ACK races, while targeted
|
||||
tests preserve routing and fail-loud behavior.
|
||||
|
||||
## Files changed
|
||||
|
||||
- `piker/brokers/binance/feed.py` - route, merge, normalize, and bound
|
||||
Binance quote streams.
|
||||
- `piker/brokers/binance/venues.py` - define current USD-M websocket
|
||||
endpoints.
|
||||
- `piker/data/_web_bs.py` - support bounded websocket context startup.
|
||||
- `tests/test_binance.py` - cover routes, schemas, and startup failures.
|
||||
- `ai/prompt-io/opencode/20260826T222600Z_5f631e19_prompt_io.raw.md` -
|
||||
retain the generated-output record.
|
||||
- `ai/prompt-io/opencode/20260826T222600Z_5f631e19_prompt_io.md` - record
|
||||
structured provenance for the code changes.
|
||||
|
||||
## Human edits
|
||||
|
||||
The human explicitly required fail-loud behavior rather than merely
|
||||
updating the endpoint, and supplied a direct CLI reproduction proving
|
||||
both `datad.binance` and the chart caller blocked. Those requirements
|
||||
drove the connection and first-data deadlines and their regressions. The
|
||||
changes remain uncommitted for human review.
|
||||
|
|
@ -0,0 +1,57 @@
|
|||
---
|
||||
model: openai/gpt-5.6-sol
|
||||
service: opencode
|
||||
timestamp: 2026-08-26T22:26:00Z
|
||||
git_ref: 5f631e19
|
||||
diff_cmd: git diff HEAD~1..HEAD
|
||||
---
|
||||
|
||||
## Prompt
|
||||
|
||||
The user reported that selecting
|
||||
`NVDAUSDT.USDTM.PERP.BINANCE` connected Binance's USD-M
|
||||
websocket but never loaded the feed or downstream chart. They asked
|
||||
OpenCode to compare Binance's API changelog with piker's FQME and feed
|
||||
code, diagnose the failure, and solve it.
|
||||
|
||||
The user clarified that future provider failures must raise loudly
|
||||
instead of silently blocking feed and chart startup. They also confirmed
|
||||
the same hang from
|
||||
`piker -l info --pdb chart nvdausdt.usdtm.perp.binance`.
|
||||
|
||||
## Generated code
|
||||
|
||||
> `git diff HEAD~1..HEAD -- piker/brokers/binance/feed.py`
|
||||
|
||||
Routes USD-M book-ticker and aggregate-trade data through Binance's new
|
||||
`/public` and `/market` websocket paths, merges both sockets into the
|
||||
existing quote stream, accepts the new `st` aggregate-trade field, and
|
||||
requires both stream classes before declaring startup complete. It also
|
||||
removes the misleading missing-provider warning, derives feed mode from
|
||||
resolved pair metadata, and raises `DataUnavailable` for connection or
|
||||
live-data startup timeouts.
|
||||
|
||||
> `git diff HEAD~1..HEAD -- piker/brokers/binance/venues.py`
|
||||
|
||||
Defines Binance's current public and market USD-M websocket endpoints
|
||||
and makes the market endpoint the regular futures websocket API root.
|
||||
|
||||
> `git diff HEAD~1..HEAD -- piker/data/_web_bs.py`
|
||||
|
||||
Adds an optional connection deadline to `open_autorecon_ws()` so callers
|
||||
can fail context entry when repeated websocket handshakes never become
|
||||
connected.
|
||||
|
||||
> `git diff HEAD~1..HEAD -- tests/test_binance.py`
|
||||
|
||||
Adds regressions for exact Binance stream routing, silent live-data
|
||||
startup, silent websocket connection startup, and the current USD-M
|
||||
aggregate-trade payload schema.
|
||||
|
||||
## Verification output
|
||||
|
||||
`pytest -q tests/test_binance.py` passes all four tests. Compile checks,
|
||||
focused Ruff fatal-error checks, and `git diff --check` pass. Live probes
|
||||
through piker's merged stream received both required stream classes and
|
||||
returned normalized trade quotes for `NVDAUSDT` USD-M and `BTCUSDT`
|
||||
spot.
|
||||
|
|
@ -0,0 +1,35 @@
|
|||
---
|
||||
model: openai/gpt-5.6-sol
|
||||
service: opencode
|
||||
session: pid-469264
|
||||
timestamp: 2026-08-28T06:31:26Z
|
||||
git_ref: 1606122b
|
||||
scope: code
|
||||
substantive: true
|
||||
raw_file: 20260828T063126Z_1606122b_prompt_io.raw.md
|
||||
---
|
||||
|
||||
## Prompt
|
||||
|
||||
Review the staged IB broker patch before committing it. After the review
|
||||
identified a deterministic error-parser crash, continue with the available
|
||||
next steps while leaving the typed EMS worktree and Ruff change untouched.
|
||||
|
||||
## Response summary
|
||||
|
||||
Repaired bracketed IB API error parsing by splitting on the matched error
|
||||
prefix. Added a deterministic regression that routes bracketed and plain
|
||||
strings through the real trade-event relay and verifies their EMS error
|
||||
messages.
|
||||
|
||||
## Files changed
|
||||
|
||||
- `piker/brokers/ib/broker.py` - safely extracts bracketed API error codes.
|
||||
- `tests/test_ib_broker.py` - covers bracketed and plain error relay inputs.
|
||||
|
||||
## Human edits
|
||||
|
||||
The human required review before the root commit, accepted the identified
|
||||
parser defect as the next work item, and explicitly authorized continuation.
|
||||
The pre-existing staged IB refactor is human-authored; the agent applied only
|
||||
the parser repair and focused regression described above.
|
||||
|
|
@ -0,0 +1,30 @@
|
|||
---
|
||||
model: openai/gpt-5.6-sol
|
||||
service: opencode
|
||||
timestamp: 2026-08-28T06:31:26Z
|
||||
git_ref: fix_symcache_search
|
||||
diff_cmd: git diff HEAD~1..HEAD
|
||||
---
|
||||
|
||||
## Prompt
|
||||
|
||||
The human asked for a review of the root checkout's staged IB patch before
|
||||
committing it. The review found that bracketed API error strings call
|
||||
`str.partition()` without its required separator. The human then instructed
|
||||
the agent to continue with available next steps, or ask only if unsure.
|
||||
|
||||
## Response
|
||||
|
||||
> `git diff HEAD~1..HEAD -- piker/brokers/ib/broker.py`
|
||||
|
||||
The generated parser repair uses the recognized `[code ` or `[Errno ` prefix
|
||||
to extract the numeric code without changing the surrounding event relay.
|
||||
|
||||
> `git diff HEAD~1..HEAD -- tests/test_ib_broker.py`
|
||||
|
||||
The generated regression drives bracketed and plain strings through
|
||||
`deliver_trade_events()` and verifies that one `BrokerdError` reaches EMS
|
||||
with the expected reason and unknown request ID.
|
||||
|
||||
The typed EMS worktree and the unrelated Ruff policy change were left
|
||||
untouched. No commit was created.
|
||||
|
|
@ -0,0 +1,37 @@
|
|||
---
|
||||
model: openai/gpt-5.6-sol
|
||||
service: opencode
|
||||
session: 3e2eb8c9-6cfb-49a5-bf1a-1c0c2c2a4ddd
|
||||
timestamp: 2026-08-28T20:03:52Z
|
||||
git_ref: dd676857
|
||||
scope: config
|
||||
substantive: true
|
||||
raw_file: 20260828T200352Z_dd676857_prompt_io.raw.md
|
||||
---
|
||||
|
||||
## Prompt
|
||||
|
||||
Update the xonsh version used by the repository dev shell before stacking
|
||||
the Binance work, while retaining the dotrc compatibility shim as an
|
||||
immediate workaround for older shells.
|
||||
|
||||
## Response summary
|
||||
|
||||
Raises the xonsh dependency floor to `0.24.2` and refreshes its lock entry.
|
||||
The new release supports Python 3.13, handles Git alias completion natively,
|
||||
and works in the updated Nix development shell.
|
||||
|
||||
## Files changed
|
||||
|
||||
- `pyproject.toml` - require xonsh `0.24.2` or newer.
|
||||
- `uv.lock` - lock xonsh `0.24.2` wheels and source metadata.
|
||||
- `ai/prompt-io/opencode/20260828T200352Z_dd676857_prompt_io.raw.md` - retain
|
||||
the triggering prompt, generated configuration summary, and checks.
|
||||
- `ai/prompt-io/opencode/20260828T200352Z_dd676857_prompt_io.md` - record
|
||||
structured provenance for the dependency update.
|
||||
|
||||
## Human edits
|
||||
|
||||
The user identified old xonsh behavior as the root cause, reported the
|
||||
separate dotrc compatibility shim, and directed that the repository update
|
||||
remain a distinct flake-layer change before the Binance stack continues.
|
||||
|
|
@ -0,0 +1,36 @@
|
|||
---
|
||||
model: openai/gpt-5.6-sol
|
||||
service: opencode
|
||||
timestamp: 2026-08-28T20:03:52Z
|
||||
git_ref: dd676857
|
||||
diff_cmd: git diff HEAD~1..HEAD
|
||||
---
|
||||
|
||||
## Prompt
|
||||
|
||||
The user traced broken multiline conflict-resolution pastes to the old
|
||||
xonsh release installed by the repository dev shell. A separate dotrc
|
||||
agent added a compatibility shim, and the user directed that xonsh still
|
||||
be updated alongside the flake work before continuing the Binance stack.
|
||||
|
||||
## Generated configuration
|
||||
|
||||
> `git diff HEAD~1..HEAD -- pyproject.toml uv.lock`
|
||||
|
||||
Raises the repository's xonsh requirement from `0.22.2` to `0.24.2` and
|
||||
refreshes only xonsh's locked distribution metadata. The selected release
|
||||
supports Python 3.13, includes native Git-config alias completion, and fixes
|
||||
Nix metadata checks.
|
||||
|
||||
## Verification output
|
||||
|
||||
- `uv lock --upgrade-package xonsh` resolves 111 packages and updates only
|
||||
xonsh from `0.22.4` to `0.24.2`.
|
||||
- Fresh `nix develop` replaces the worktree-local xonsh installation.
|
||||
- `xonsh --version` reports `0.24.2`.
|
||||
- A multi-command xonsh invocation executes Git commands successfully.
|
||||
- `nix flake check --no-build` passes for `x86_64-linux`.
|
||||
- `git diff --check` passes.
|
||||
|
||||
The existing Polars `1.36.0` lock remains unchanged despite uv reporting
|
||||
that its distributions are yanked for an `unpivot` regression.
|
||||
|
|
@ -0,0 +1,46 @@
|
|||
---
|
||||
model: openai/gpt-5.6-sol
|
||||
service: opencode
|
||||
session: c3a1b1ba-65c6-4c8e-bd12-e596031b0eb0
|
||||
timestamp: 2026-08-29T17:55:42Z
|
||||
git_ref: e09e306a
|
||||
scope: code
|
||||
substantive: true
|
||||
raw_file: 20260829T175542Z_e09e306a_prompt_io.raw.md
|
||||
---
|
||||
|
||||
## Prompt
|
||||
|
||||
Preserve the existing Binance USD-M commit above the reviewed flake/xonsh
|
||||
stack, then add missing deterministic tests and correctness fixes as a
|
||||
separate follow-up with a complete editor-gated commit plan.
|
||||
|
||||
## Response summary
|
||||
|
||||
Hardens Binance feed startup around its supported one-symbol contract,
|
||||
direct websocket routes, current provider schemas, and first-quote
|
||||
readiness. Adds deterministic Binance regressions and separates generic
|
||||
reconnect behavior into its own tested commit boundary.
|
||||
|
||||
## Files changed
|
||||
|
||||
- `piker/brokers/binance/feed.py` - validate feed cardinality, route
|
||||
USD-M stream classes, and declare readiness after either quote class.
|
||||
- `tests/test_binance.py` - cover routes, first-quote startup, cardinality,
|
||||
and documented book-ticker and aggregate-trade schemas.
|
||||
- `ai/prompt-io/opencode/20260829T175542Z_e09e306a_prompt_io.raw.md` - retain
|
||||
the triggering prompt, generated-code summary, and verification output.
|
||||
- `ai/prompt-io/opencode/20260829T175542Z_e09e306a_prompt_io.md` - record
|
||||
structured provenance for the follow-up.
|
||||
|
||||
## Human edits
|
||||
|
||||
The user rejected reconstructing already committed Binance work, required
|
||||
preservation through cherry-pick, selected the flake/xonsh stack base, and
|
||||
required the review fixes to remain a separate tested boundary. The user also
|
||||
corrected the workflow to require an automatic `/commit-plan` at every stack
|
||||
review gate. During Tuicr review, the user rejected trade-gated startup,
|
||||
requested a feed-specific initialization error and async-generator TODO,
|
||||
and required generic reconnect changes and tests to land independently.
|
||||
The user then clarified that either L1 or trade data must unblock startup,
|
||||
rather than preferring one stream class or waiting independently for both.
|
||||
|
|
@ -0,0 +1,43 @@
|
|||
---
|
||||
model: openai/gpt-5.6-sol
|
||||
service: opencode
|
||||
timestamp: 2026-08-29T17:55:42Z
|
||||
git_ref: e09e306a
|
||||
diff_cmd: git diff HEAD~1..HEAD
|
||||
---
|
||||
|
||||
## Prompt
|
||||
|
||||
The user asked to land recent branch work in small chronological patch sets,
|
||||
including missing tests. For Binance, the user required preserving the
|
||||
existing committed topic patch through cherry-pick and placing newly found
|
||||
correctness repairs in a separate follow-up boundary above the flake/xonsh
|
||||
stack. The user also required every yielded stack layer to include a complete
|
||||
`/commit-plan` with patch-specific checks.
|
||||
|
||||
## Generated code
|
||||
|
||||
> `git diff HEAD~1..HEAD -- piker/brokers/binance/feed.py`
|
||||
|
||||
Restricts Binance feeds to the supported one-symbol contract, renders direct
|
||||
quote routes, separates L1 liveness from first-trade readiness, and derives
|
||||
the contextual trade timeout from Trio's enclosing effective deadline.
|
||||
|
||||
> `git diff HEAD~1..HEAD -- piker/data/_web_bs.py`
|
||||
|
||||
Lets structured cancellation bypass reconnect throttling and ensures handled
|
||||
handshake rejections retry before accessing connection-local cancel state.
|
||||
|
||||
> `git diff HEAD~1..HEAD -- tests/test_binance.py`
|
||||
|
||||
Adds deterministic regressions for symbol cardinality, route selection,
|
||||
silent startup, immediate rejection, cancellation cleanup, aggregate-trade
|
||||
schema evolution, delayed trades, and finite/infinite/tiny deadlines.
|
||||
|
||||
## Verification output
|
||||
|
||||
- Targeted Binance tests: `10 passed in 0.26s`.
|
||||
- Ruff passes for all three changed Python files.
|
||||
- `git diff --check` passes.
|
||||
- Independent final review reports no actionable findings.
|
||||
- No live Binance network probe was run.
|
||||
|
|
@ -0,0 +1,42 @@
|
|||
---
|
||||
model: openai/gpt-5.6-sol
|
||||
service: opencode
|
||||
session: c3a1b1ba-65c6-4c8e-bd12-e596031b0eb0
|
||||
timestamp: 2026-09-01T02:42:35Z
|
||||
git_ref: e09e306a
|
||||
scope: code
|
||||
substantive: true
|
||||
raw_file: 20260901T024235Z_e09e306a_prompt_io.raw.md
|
||||
---
|
||||
|
||||
## Prompt
|
||||
|
||||
Address the Tuicr review by separating generic websocket reconnect
|
||||
changes from Binance-specific work and testing the real reconnect task
|
||||
hierarchy rather than a reimplementation.
|
||||
|
||||
## Response summary
|
||||
|
||||
Preserves structured cancellation and safe first-handshake retries in
|
||||
`_web_bs`. Adds three focused regressions in a dedicated websocket test
|
||||
module and packages them as an independent commit boundary.
|
||||
|
||||
## Files changed
|
||||
|
||||
- `piker/data/_web_bs.py` - restart handled handshake failures and let
|
||||
cancellation bypass the reconnect throttle.
|
||||
- `tests/test_websocket.py` - exercise real reconnect startup, retry,
|
||||
and cancellation task paths with controlled transport boundaries.
|
||||
- `ai/prompt-io/opencode/20260901T024235Z_e09e306a_prompt_io.raw.md`
|
||||
- retain the review prompt, generated-code summary, and verification.
|
||||
- `ai/prompt-io/opencode/20260901T024235Z_e09e306a_prompt_io.md` - record
|
||||
structured provenance for the generic reconnect boundary.
|
||||
|
||||
## Human edits
|
||||
|
||||
The user identified that these changes were generic rather than
|
||||
Binance-specific, required a separate commit and test module, requested
|
||||
coverage of the real reconnect hierarchy, and corrected code prose to
|
||||
refer to `trio` consistently. A follow-up review required the production
|
||||
code to explain why `trio.Cancelled` bypasses the broad reconnect-error
|
||||
handler and cite the regression which audits that teardown invariant.
|
||||
|
|
@ -0,0 +1,33 @@
|
|||
---
|
||||
model: openai/gpt-5.6-sol
|
||||
service: opencode
|
||||
timestamp: 2026-09-01T02:42:35Z
|
||||
git_ref: e09e306a
|
||||
diff_cmd: git diff HEAD~1..HEAD
|
||||
---
|
||||
|
||||
## Prompt
|
||||
|
||||
The user requested that generic `_web_bs` reconnect fixes and tests be
|
||||
separated from the Binance backend patch. The review also required tests
|
||||
to exercise the real reconnect task hierarchy, cite the previous faulty
|
||||
branches, and consistently name `trio` in code prose.
|
||||
|
||||
## Generated code
|
||||
|
||||
> `git diff HEAD~1..HEAD -- piker/data/_web_bs.py`
|
||||
|
||||
Ensures handled handshake failures immediately restart the reconnect
|
||||
loop and lets structured cancellation bypass shielded error throttling.
|
||||
|
||||
> `git diff HEAD~1..HEAD -- tests/test_websocket.py`
|
||||
|
||||
Adds deterministic scheduler-real regressions for handshake startup
|
||||
timeouts, immediate rejection retries, and cancellation propagation.
|
||||
|
||||
## Verification output
|
||||
|
||||
- Generic and Binance targeted tests: `9 passed in 0.06s`.
|
||||
- Ruff passes for all four changed Python files.
|
||||
- `git diff --check` passes.
|
||||
- No live network probe was run.
|
||||
|
|
@ -0,0 +1,41 @@
|
|||
---
|
||||
model: openai/gpt-5.6-sol
|
||||
service: opencode
|
||||
session: 4dc920cf-e82e-4c1e-83db-75a0a9efe82b
|
||||
timestamp: 2026-09-01T23:00:43Z
|
||||
git_ref: e91f41a6
|
||||
scope: config
|
||||
substantive: true
|
||||
raw_file: 20260901T230043Z_e91f41a6_prompt_io.raw.md
|
||||
---
|
||||
|
||||
## Prompt
|
||||
|
||||
Update `wkt/land_flake_update` to Tractor's restored `piker_pin` after
|
||||
reusing the existing runtime compatibility commits. Keep stale
|
||||
historical lock snapshots out of the final dependency state.
|
||||
|
||||
## Response summary
|
||||
|
||||
Updates piker's Python and `msgspec` requirements for current Tractor,
|
||||
then generates one fresh lock at Tractor `piker_pin@0d95a9e6`. The
|
||||
dependency commit completes the compatibility stack already
|
||||
cherry-picked onto the flake branch.
|
||||
|
||||
## Files changed
|
||||
|
||||
- `pyproject.toml` - require Python 3.13 and `msgspec>=0.20.0`.
|
||||
- `uv.lock` - resolve Tractor `piker_pin@0d95a9e6`, `msgspec 0.21.1`,
|
||||
and current transport dependencies.
|
||||
- `ai/prompt-io/opencode/20260901T230043Z_e91f41a6_prompt_io.raw.md`
|
||||
- preserve the prompt, generated-change summary, and resolution notes.
|
||||
- `ai/prompt-io/opencode/20260901T230043Z_e91f41a6_prompt_io.md` - record
|
||||
structured provenance for the dependency update.
|
||||
|
||||
## Human edits
|
||||
|
||||
The user identified that Tractor compatibility should land with the
|
||||
flake update rather than the Binance patch, supplied four known source
|
||||
commits, required a broader local-history audit for missing ports, and
|
||||
directed a fresh lock from the restored `piker_pin` instead of replaying
|
||||
historical pins.
|
||||
|
|
@ -0,0 +1,35 @@
|
|||
---
|
||||
model: openai/gpt-5.6-sol
|
||||
service: opencode
|
||||
timestamp: 2026-09-01T23:00:43Z
|
||||
git_ref: e91f41a6
|
||||
diff_cmd: git diff HEAD~1..HEAD
|
||||
---
|
||||
|
||||
## Prompt
|
||||
|
||||
The user requested that the flake-update branch absorb the Tractor
|
||||
runtime compatibility work before the Binance branch lands. They
|
||||
directed reuse of existing compatibility commits, exclusion of stale
|
||||
historical lock pins, and a fresh update to Tractor's restored
|
||||
`piker_pin`.
|
||||
|
||||
## Generated changes
|
||||
|
||||
> `git diff HEAD~1..HEAD -- pyproject.toml`
|
||||
|
||||
Narrows piker support to Python 3.13 and raises the direct `msgspec`
|
||||
requirement to the version family required by current Tractor.
|
||||
|
||||
> `git diff --cached -- uv.lock`
|
||||
|
||||
Relocks against Tractor `piker_pin@0d95a9e6`, `msgspec 0.21.1`, and the
|
||||
current transport dependency graph while retaining the existing Xonsh
|
||||
update.
|
||||
|
||||
## Resolution notes
|
||||
|
||||
- Historical lock hunks from reused commits were intentionally omitted.
|
||||
- Tractor runtime compatibility remains represented by the preceding
|
||||
individually preserved commits on `wkt/land_flake_update`.
|
||||
- Project checks are rendered in the commit plan for human execution.
|
||||
|
|
@ -0,0 +1,37 @@
|
|||
---
|
||||
model: openai/gpt-6-astra
|
||||
service: opencode
|
||||
session: unavailable
|
||||
timestamp: 2026-09-16T19:40:52Z
|
||||
git_ref: 749ca1f0
|
||||
scope: code
|
||||
substantive: true
|
||||
raw_file: 20260916T194052Z_749ca1f0_prompt_io.raw.md
|
||||
---
|
||||
|
||||
## Prompt
|
||||
|
||||
Fix wrong IB fill/commission pairing and empty ledger normalization;
|
||||
retain and investigate silent socket loss during network roaming.
|
||||
|
||||
## Response summary
|
||||
|
||||
Preserve execution identity from callbacks into accounting, prevent
|
||||
empty commission IDs from erasing ledger identity, and propagate actual
|
||||
socket disconnection as a transport exception. The existing data-farm
|
||||
reset mechanism is not a socket-session reconnect implementation.
|
||||
|
||||
## Files changed
|
||||
|
||||
- `piker/brokers/ib/broker.py`: fill correlation and socket termination.
|
||||
- `piker/brokers/ib/ledger.py`: authoritative execution ID.
|
||||
- `piker/brokers/ib/api.py`: disconnect event propagation.
|
||||
- `tests/test_ib_trade_events.py`: accounting event regressions.
|
||||
- `tests/test_ib_method_proxy.py`: socket-disconnect regression.
|
||||
|
||||
## Human edits
|
||||
|
||||
The human supplied live traceback and debugger values proving the
|
||||
execution mismatch, requested a regression rather than a KeyError
|
||||
guard, and required continued attention to roaming recovery. No human
|
||||
source edits to this patch have been observed; acceptance is pending.
|
||||
|
|
@ -0,0 +1,45 @@
|
|||
---
|
||||
model: openai/gpt-6-astra
|
||||
service: opencode
|
||||
timestamp: 2026-09-16T19:40:52Z
|
||||
git_ref: 749ca1f0
|
||||
diff_cmd: git diff HEAD -- piker/brokers/ib tests/test_ib_trade_events.py tests/test_ib_method_proxy.py
|
||||
---
|
||||
|
||||
## Human instructions and evidence
|
||||
|
||||
Fix silent IB socket loss after network roaming, reported as
|
||||
`[Errno 104] Connection reset by peer`. Also investigate a live
|
||||
MNQ accounting crash in `emit_pp_update()` with empty normalized
|
||||
ledger entries. The human supplied debugger evidence that a commission
|
||||
for execution 4261 was paired with execution 4262 on the same contract
|
||||
and account, and explicitly requested implementation and regression
|
||||
tests. They asked to retain the roaming task and continue working
|
||||
through follow-up messages.
|
||||
|
||||
## Generated changes
|
||||
|
||||
> `git diff HEAD -- piker/brokers/ib/broker.py`
|
||||
|
||||
Preserve the callback-supplied fill with its commission. Track positions
|
||||
by account and contract ID and pending commissioned fills by execution.
|
||||
Reject mismatched commission/execution IDs. Propagate socket closure
|
||||
instead of returning normally from the trade-event bridge.
|
||||
|
||||
> `git diff HEAD -- piker/brokers/ib/ledger.py`
|
||||
|
||||
Keep Execution.execId authoritative when flattening commission fields.
|
||||
|
||||
> `git diff HEAD -- piker/brokers/ib/api.py`
|
||||
|
||||
Relay the authoritative disconnected event as ConnectionError, retaining
|
||||
the most recent API error text, and raise it from the proxy reader.
|
||||
Automatic session reconnect/resynchronization remains unimplemented.
|
||||
|
||||
> `git diff HEAD -- tests/test_ib_trade_events.py tests/test_ib_method_proxy.py`
|
||||
|
||||
Cover overlapping executions, zero commissions, position-before/after
|
||||
commission ordering, empty commission IDs, and eventkit socket loss.
|
||||
Nine focused tests passed, including the actual eventkit callback
|
||||
bridge and its disconnect exception. Focused Ruff checks and
|
||||
`git diff --check` also passed.
|
||||
|
|
@ -0,0 +1,61 @@
|
|||
---
|
||||
model: openai/gpt-6-astra
|
||||
service: opencode
|
||||
scope: code
|
||||
substantive: true
|
||||
git_ref: 5eb4fc9d
|
||||
raw_file: 20260916_ib_connectivity_integration.raw.md
|
||||
---
|
||||
|
||||
## Prompt
|
||||
|
||||
Investigate IB Gateway loss, silent chart survival, infected-actor
|
||||
shutdown errors and leaked shared memory. Integrate pending tagged
|
||||
address and broadcast-consumer compatibility work. The human required
|
||||
isolated worktrees and preservation of the live setup, and selected
|
||||
tractor piker_pin rather than main as the runtime fix base.
|
||||
|
||||
## Response summary
|
||||
|
||||
Created isolated piker and tractor worktrees and initially integrated
|
||||
c07e3cbd and 4af885d9 without committing. That pending merge was later
|
||||
aborted. The current piker scope is the original IB fail-loud patch
|
||||
and its regression tests on 5eb4fc9d; neither upstream history is
|
||||
merged. The live interpreter installation was not changed.
|
||||
|
||||
Integration-era logging and Deribit compatibility fixes were archived
|
||||
outside this patch and are absent from the current worktree. The raw
|
||||
companion preserves that earlier response as historical output.
|
||||
|
||||
## Generated files
|
||||
|
||||
- piker/brokers/ib/api.py: propagate actual socket disconnection with
|
||||
the last API error through the method-proxy relay.
|
||||
- piker/brokers/ib/broker.py: raise on trade-event socket loss and
|
||||
propagate the disconnect event to the trade consumer.
|
||||
- tests/test_ib_method_proxy.py: exercise API error plus disconnect.
|
||||
- tests/test_ib_trade_events.py: preserve callback fill identity and
|
||||
raise when the callback bridge disconnects.
|
||||
|
||||
Existing IB fail-loud edits and their tests were copied from the
|
||||
earlier session, whose provenance remains in the fill commit.
|
||||
The commit plan includes these four files and both provenance logs.
|
||||
|
||||
## Verification
|
||||
|
||||
Historically, twenty selected piker regression cases passed on the
|
||||
integrated tree against isolated, patched tractor sources. Those
|
||||
results do not validate the current post-abort tree or installed
|
||||
pinned tractor. The new exact-tree commit-plan checks remain pending;
|
||||
no project tests were run during this planning pass. No live
|
||||
service/container tests or installed dependency changes were performed.
|
||||
Original live-worktree diff matched SHA256
|
||||
cbdfa439e656ef3a5effaa0f0e1a7706f1070de863fc0e85222d36294eed8524.
|
||||
|
||||
## Human contributions
|
||||
|
||||
The human identified the dependency branches, corrected the proposed
|
||||
tractor main base to piker_pin, and required a new worktree after the
|
||||
agent initially started an uncommitted merge in the live checkout.
|
||||
That merge was aborted and preservation was verified before isolated
|
||||
work continued. The human supplied all live incident evidence.
|
||||
|
|
@ -0,0 +1,25 @@
|
|||
---
|
||||
model: openai/gpt-6-astra
|
||||
service: opencode
|
||||
git_ref: 5eb4fc9d
|
||||
diff_cmd: git diff HEAD
|
||||
---
|
||||
|
||||
> git diff HEAD -- piker/data/feed.py piker/brokers/deribit/api.py piker/brokers/deribit/feed.py
|
||||
|
||||
Use the standard logging predicate for INFO and repair import-time
|
||||
Deribit message struct/annotation compatibility encountered while
|
||||
validating the merged broadcast-consumer baseline.
|
||||
|
||||
The isolated checks pass, including real shm cleanup:
|
||||
|
||||
- All 20 selected piker integration cases pass after the narrow
|
||||
compatibility fixes.
|
||||
- Tractor's EOF tests pass for TCP and UDS, including an idle
|
||||
owner and an unaffected peer.
|
||||
- All four late-cancellation variants unlink their test-owned
|
||||
data/first/last shm segments before parent teardown.
|
||||
|
||||
The original live-worktree diff still matches its saved hash.
|
||||
We're ready for a Gateway-container test of the full actor tree;
|
||||
automatic reconnect is still a separate, unfinished step.
|
||||
|
|
@ -0,0 +1,42 @@
|
|||
---
|
||||
model: openai/gpt-6-astra
|
||||
service: opencode
|
||||
scope: code
|
||||
substantive: true
|
||||
git_ref: efa72980
|
||||
raw_file: 20260924_ib_idle_timer.raw.md
|
||||
---
|
||||
|
||||
## Prompt
|
||||
|
||||
The human reported silent IB roaming and requested an immediate fix
|
||||
after investigation found the custom nanosecond timestamps incompatible
|
||||
with ib_async's datetime-based idle timeout arithmetic.
|
||||
|
||||
## Response summary
|
||||
|
||||
Override NonShittyWrapper._setTimer to accept integer nanosecond packet
|
||||
timestamps while delegating datetime/sentinel states upstream. Keep the
|
||||
packet timestamp unchanged on integer-path expiry, clear the timer
|
||||
before timeoutEvent emission, and preserve reentrant listener rearming.
|
||||
No default heartbeat, reconnect, or idle-timeout arming is introduced.
|
||||
|
||||
## Files changed
|
||||
|
||||
- piker/brokers/ib/api.py: compatible idle timer arithmetic/lifecycle.
|
||||
- tests/test_ib_wrapper_timeout.py: three deterministic offline tests.
|
||||
|
||||
## Verification
|
||||
|
||||
Nineteen tests passed: new wrapper-timeout regressions and existing IB
|
||||
history/method-proxy tests. Restoring the upstream method in memory
|
||||
reproduced datetime-minus-int TypeError. No IB service or live network
|
||||
was contacted; installed dependencies and original user work were
|
||||
unchanged. The isolated worktree index remains unchanged.
|
||||
|
||||
## Human contributions
|
||||
|
||||
The human supplied rfkill/suspend/roam evidence, requested a session
|
||||
health design, and prioritized fixing the discovered timestamp mismatch.
|
||||
The implementation deliberately preserves their nanosecond feed schema.
|
||||
Acceptance and live validation remain pending.
|
||||
|
|
@ -0,0 +1,17 @@
|
|||
---
|
||||
model: openai/gpt-6-astra
|
||||
service: opencode
|
||||
git_ref: efa72980
|
||||
diff_cmd: git diff HEAD -- piker/brokers/ib/api.py tests/test_ib_wrapper_timeout.py
|
||||
---
|
||||
|
||||
> git diff HEAD -- piker/brokers/ib/api.py tests/test_ib_wrapper_timeout.py
|
||||
|
||||
NonShittyWrapper._setTimer accepts upstream datetime state and piker's
|
||||
integer packet-nanosecond state. Expiry clears its handle and timeout
|
||||
before notifying listeners, without replacing packet timestamps.
|
||||
Tests cover traffic rescheduling, exactly-once expiry, pretraffic and
|
||||
sentinel handling, disabling, and callback rearming.
|
||||
|
||||
Verification: 19 passed, including existing IB history/proxy tests.
|
||||
Wall-clock packet silence remains distinct from session health.
|
||||
|
|
@ -0,0 +1,52 @@
|
|||
---
|
||||
created_at: 2026-10-21T22:44:21Z
|
||||
generated_by:
|
||||
harness: opencode
|
||||
provider: openai
|
||||
model: openai/gpt-6.1-sol
|
||||
git_ref: 507c75c6
|
||||
scope: config
|
||||
substantive: true
|
||||
raw_file: 20261021T224421Z_507c75c6_piker_pin_prompt_io.raw.md
|
||||
---
|
||||
|
||||
## Prompt
|
||||
|
||||
The human asked to sanity-check the current PR94 landing state against
|
||||
the qualified new Tractor pin before deduplicating branch history. After
|
||||
the initial offline checks passed, the human authorized changing the
|
||||
selector to piker_pin, resolving the lock, and verifying the installed
|
||||
dependency without the temporary local-source PYTHONPATH override.
|
||||
The human explicitly authorized network resolution and a scoped update
|
||||
of Tractor/runtime dependencies while preserving unrelated packages.
|
||||
|
||||
## Response summary
|
||||
|
||||
Updated the branch selector and lock to published piker_pin@b64c89f0.
|
||||
Installed the exact locked Tractor revision, verified its installed
|
||||
source metadata, and reran 14 offline logger/startup cases without a
|
||||
local Tractor source override. Preserved the rest of the environment.
|
||||
|
||||
## Files changed
|
||||
|
||||
- `pyproject.toml`: select piker_pin and bound Polars below 2.0.
|
||||
- `uv.lock`: lock b64c89f0 and Polars/runtime 1.44.2.
|
||||
- This provenance pair.
|
||||
|
||||
## Verification and limits
|
||||
|
||||
14 distinct offline cases passed against the installed Tractor update
|
||||
with the existing Polars 1.36 installation. The actor.uid deprecation
|
||||
warning remains. No live provider/order testing occurred.
|
||||
|
||||
The initial resolver selected Polars 2.0 and rejected retaining 1.36
|
||||
for its Windows target. The human then requested the latest pre-2.0
|
||||
release instead. PyPI metadata identified non-yanked stable 1.44.2;
|
||||
adding `<2.0.0` to the existing lower bound and relocking succeeded
|
||||
with both polars and polars-runtime-32 at 1.44.2, without platform edits.
|
||||
The raw partner retains the original, superseded 2.0 generation history.
|
||||
|
||||
The scoped install changed only Tractor. Polars 1.44.2 is locked but
|
||||
not installed or data-path qualified yet. The human's existing staged
|
||||
checkpoint is preserved; the pre-2.0 correction is in the worktree.
|
||||
No history cleanup, staging or commit was performed by this agent.
|
||||
|
|
@ -0,0 +1,45 @@
|
|||
---
|
||||
created_at: 2026-10-21T22:44:21Z
|
||||
generated_by:
|
||||
harness: opencode
|
||||
provider: openai
|
||||
model: openai/gpt-6.1-sol
|
||||
git_ref: 507c75c6
|
||||
diff_cmd: git diff HEAD~1..HEAD
|
||||
---
|
||||
|
||||
> `git diff HEAD~1..HEAD -- pyproject.toml uv.lock`
|
||||
|
||||
Changed the landing branch's Tractor selector from wkt/addr_unpacking
|
||||
to piker_pin and resolved the published branch to
|
||||
b64c89f05a9fc8ac2878551eb5eda36574ebef6a. An offline branch lookup lacked
|
||||
the cached name, so the human explicitly authorized network resolution
|
||||
and a scoped installed-runtime update preserving unrelated packages.
|
||||
|
||||
The resolver advanced Polars and polars-runtime-32 from 1.36.0 to 2.0.0.
|
||||
An attempt to retain both exact 1.36.0 versions was rejected for the
|
||||
resolver's Windows target. The valid lock with the visible Polars bump
|
||||
is retained for review; no platform restriction or source constraint was
|
||||
introduced to bypass the failure.
|
||||
|
||||
Installed only Tractor and its required dependencies with uv pip rather
|
||||
than syncing/pruning the whole environment. The installer reported one
|
||||
package replacement: Tractor a966928d to b64c89f0. Qt distribution versions
|
||||
remained PyQt6 6.10.1, PyQt6-Qt6 6.10.1 and PyQt6-sip 13.10.3.
|
||||
|
||||
Removed PYTHONPATH from verification subprocesses. Piker resolved inside
|
||||
land_flake_update and Tractor resolved from that checkout's py313
|
||||
site-packages. Installed VCS metadata matched the exact locked revision.
|
||||
The temporary LD_LIBRARY_PATH used the binding's bundled Qt libraries
|
||||
and matching native dependencies; no global environment setup changed.
|
||||
|
||||
Verification output:
|
||||
- Logger/non-actor startup: 12 passed, 1 deselected in 0.05s.
|
||||
- test_runtime_boot: 1 passed, 1 warning in 0.04s.
|
||||
- test_daemon_bind_distinct_from_registrar: 1 passed in 0.77s.
|
||||
- Existing warning: actor.uid deprecation; no failure.
|
||||
- git diff --check passed.
|
||||
|
||||
This verifies the installed Tractor pin, not a full sync to all new lock
|
||||
versions. Polars was not upgraded in the environment by the scoped install.
|
||||
No live broker feed, order, branch rewrite, staging or commit was run.
|
||||
|
|
@ -0,0 +1,67 @@
|
|||
---
|
||||
created_at: 2026-10-22T21:22:48Z
|
||||
generated_by:
|
||||
harness: opencode
|
||||
provider: openai
|
||||
model: openai/gpt-6-astra
|
||||
git_ref: dc618229bd8c9c90aa3160e26677b553a7d472c7
|
||||
scope: code
|
||||
substantive: true
|
||||
raw_file: 20261022T212248Z_dc618229_ib_recovery_prompt_io.raw.md
|
||||
---
|
||||
|
||||
## Prompt
|
||||
|
||||
The human approved trying the bounded candidate: back out the IB
|
||||
quote-context cache, import adapter-only reset/boundary fixes and
|
||||
the missing EMS subscription fix, preserve source authorship and
|
||||
commit boundaries, and exercise the real Tractor bridge. They
|
||||
required an explicit keep/backout audit and clarification that
|
||||
the evaluated downstream lineage lacked the broadcast fix.
|
||||
|
||||
## Response summary
|
||||
|
||||
Verified identical stable patch IDs for original 4af885d9 and
|
||||
landing replay 635314c7. Created `wkt/land_ib_recovery` at dc618229
|
||||
in the main checkout. Applied source-equivalent adapter backports
|
||||
from 0846cbd4 and 1da9b760 and the EMS code from 12b0d66e, with
|
||||
source attribution recorded in `ai/ib-recovery-candidate.md`.
|
||||
|
||||
The real-runtime quote test first reproduced the reported fatal
|
||||
TrioTaskExited propagation with caching enabled. The direct-owner
|
||||
backout restores restart and subsequent RPC survival, but a strict
|
||||
callback-cleanliness test remains red. Real bridge history pacing
|
||||
recovery and real broadcaster dark clearing pass. No live broker,
|
||||
VNC, orders or TSDB operations were performed by the agent.
|
||||
|
||||
On 2026-10-23 the human supplied successful live startup/backfill,
|
||||
sequential reattachment and overlapping chart-close/reopen results,
|
||||
plus the still-failing daemon SIGINT teardown. The human accepted
|
||||
deferring teardown from #94, requested loud partial-backport
|
||||
attribution and required downstream rebase rehearsals before landing.
|
||||
The agent removed only the explicit single-null debugger stop,
|
||||
retaining its legacy result. Downstream `976db6c3` instead rewrites
|
||||
null grouping; the local cleanup does not claim that algorithmic fix.
|
||||
|
||||
## Files changed
|
||||
|
||||
- `piker/brokers/ib/feed.py`: source-equivalent backout/backports.
|
||||
- `piker/clearing/_ems.py`: original EMS subscription patch.
|
||||
- `tests/test_ib_broadcast.py`: removed with the reverted behavior.
|
||||
- `tests/test_ib_history.py`: original adapter regressions only.
|
||||
- `tests/test_ems_broadcast.py`: imported and real-broadcaster tests.
|
||||
- `tests/test_ib_quote_restart.py`: real Tractor boundary tests.
|
||||
- `piker/tsp/_anal.py`: local single-null breakpoint removal only.
|
||||
- `ai/ib-recovery-candidate.md`: provenance and qualification limits.
|
||||
|
||||
## Human edits
|
||||
|
||||
The human required original fixes and authorship, rejected broad
|
||||
speculative integration work, required real concurrency boundaries
|
||||
in tests, and corrected the interpretation of live qualification.
|
||||
They performed the live tests, chose to defer known teardown defects
|
||||
from #94, permitted the minimal debugger cleanup, and requested
|
||||
explicit downstream discrepancies and rebase compatibility checks.
|
||||
The agent applied those directed edits; no direct manual source edits
|
||||
are known in this candidate. The raw partner retains the originally
|
||||
captured output; this structured record includes later contributions.
|
||||
|
|
@ -0,0 +1,69 @@
|
|||
---
|
||||
created_at: 2026-10-22T21:22:48Z
|
||||
generated_by:
|
||||
harness: opencode
|
||||
provider: openai
|
||||
model: openai/gpt-6-astra
|
||||
git_ref: dc618229bd8c9c90aa3160e26677b553a7d472c7
|
||||
diff_cmd: git diff dc618229 -- piker/brokers/ib/feed.py piker/clearing/_ems.py tests
|
||||
---
|
||||
|
||||
## Human direction
|
||||
|
||||
> yeah i guess this is our best approach strat for sure.
|
||||
|
||||
> so you're saying these 2 are not the same patch?
|
||||
> please tell me no lol
|
||||
|
||||
> so detail which commits to drop vs. keep please in detail so i can
|
||||
> audit your choices this time..
|
||||
|
||||
> let's try it and consider bailing and re-trying if this leads us into
|
||||
> a further (and deeper) sinkhole, bc as it stands right now this whole
|
||||
> endevour has turned into a complete shitfest XD
|
||||
|
||||
> you didn't respond to my ask about the wkt/ib_downstream_pin_eval stuff and it missing the bcast rx commit either
|
||||
|
||||
## Unedited diagnostic response
|
||||
|
||||
The real-bridge test reproduced the exact fatal path:
|
||||
`maybe_open_context()` teardown raises `TrioTaskExited` back into
|
||||
the quote consumer.
|
||||
|
||||
With only the quote-cache backout, both quote generations complete
|
||||
and their asyncio tasks finish. Tractor still emits
|
||||
`TrioTaskExited` callback errors, so the strict test remains red.
|
||||
That narrows the result: the backout removes the fatal propagation
|
||||
in this reproducer, but doesn’t make bridge teardown clean.
|
||||
|
||||
## Code output pointers
|
||||
|
||||
> `git diff dc618229 -- piker/brokers/ib/feed.py`
|
||||
|
||||
Back out 635314c7's quote owner and import the original adapter-only
|
||||
deltas from 0846cbd4 and 1da9b760.
|
||||
|
||||
> `git diff dc618229 -- piker/clearing/_ems.py`
|
||||
|
||||
Import 12b0d66e's dedicated dark-clearing quote subscription.
|
||||
|
||||
> `git diff dc618229 -- tests/test_ib_broadcast.py tests/test_ib_quote_restart.py tests/test_ib_history.py tests/test_ems_broadcast.py`
|
||||
|
||||
Replace the reverted mock-cache test with a real-runtime restart
|
||||
reproducer; retain original history/EMS tests and add real bridge
|
||||
history recovery and real broadcaster EMS coverage. Newly added
|
||||
test files remain untracked until the human stages them, so the
|
||||
diff pointers include them only after staging or committing.
|
||||
|
||||
## Verification output
|
||||
|
||||
```text
|
||||
21 passed in 0.53s
|
||||
1 failed, 2 passed in 5.53s
|
||||
13 passed in 1.41s
|
||||
1 passed, 1 warning in 0.04s
|
||||
2 passed in 0.03s
|
||||
```
|
||||
|
||||
The last run includes the original EMS case again and one additional
|
||||
real-broadcaster case: 38 unique passes and one strict callback failure.
|
||||
|
|
@ -0,0 +1,77 @@
|
|||
---
|
||||
created_at: 2026-10-28T16:08:46Z
|
||||
generated_by:
|
||||
harness: opencode
|
||||
provider: openai
|
||||
model: openai/gpt-6.1-sol
|
||||
git_ref: 7242dcb9
|
||||
scope: code
|
||||
substantive: true
|
||||
raw_file: 20261028T160846Z_7242dcb9_websocket_close.raw.md
|
||||
---
|
||||
|
||||
## Prompt
|
||||
|
||||
The human reported repeated live Binance perp crashes after correct
|
||||
worktree startup: the real trio-websocket reader attempted a Pong
|
||||
while wsproto was LOCAL_CLOSING. They asked whether an existing
|
||||
downstream websocket fix should be reused, then requested:
|
||||
|
||||
> ok add the fix now, follow by a /commit-plan and then i'll test
|
||||
> before saving it ya?
|
||||
|
||||
They approved a real websocket reader/close regression covering
|
||||
recovery and intentional shutdown, and separately authorized
|
||||
reconciling this worktree's legacy workflow bookkeeping while
|
||||
preserving existing archives.
|
||||
|
||||
## Response summary
|
||||
|
||||
The older `3d83b61f3f7b982e84a4230dba5777f1407c78f1` wrapper
|
||||
suppressed escaped communication failures. The landing replay
|
||||
`97eb8347` of `d50ab6124cd0e1646c82fa4a85d7c13bfff82611`
|
||||
added propagation; it did not restart a reader-context failure.
|
||||
Existing `LocalProtocolError` membership in NoBsWs.recon_errors
|
||||
only protected the receive relay, not reader-nursery context exit.
|
||||
|
||||
Classify these errors around the complete websocket context and
|
||||
retry instead of restoring outer suppression. Place collapse_eg
|
||||
outside that context, invalidate stale connection readiness and
|
||||
socket references, and leave the retry wait cancellable. Preserve
|
||||
unexpected failures and mixed exception groups for propagation.
|
||||
After review, use RuntimeError rather than ValueError for the
|
||||
test-only unexpected-failure sentinel. Its injected exception has
|
||||
no underlying cause. Production recovery logs retain the original
|
||||
traceback and now name the URL; unexpected exceptions use bare
|
||||
raise to preserve identity, existing causes and notes.
|
||||
|
||||
The deterministic seam controls protocol-state ordering in the real
|
||||
client Ping handler. It does not replace the websocket transport,
|
||||
reader, context manager, reconnect task or wsproto exception.
|
||||
|
||||
## Files changed
|
||||
|
||||
- `piker/data/_web_bs.py`: reader-context failure recovery.
|
||||
- `tests/test_websocket.py`: loopback recovery and two shutdown
|
||||
variants; retain unexpected-failure propagation control.
|
||||
- This provenance pair: live report, lineage and test scope.
|
||||
|
||||
## Verification
|
||||
|
||||
Baseline uses new tests against original HEAD websocket code in an
|
||||
isolated exact-tree checkout. The recovery case fails with the
|
||||
reported Pong-in-LOCAL_CLOSING LocalProtocolError.
|
||||
|
||||
Fixed code passes 13 cases across tests/test_websocket.py and
|
||||
tests/test_binance.py. Exact-boundary verification and native-library
|
||||
configuration are retained in the ignored commit-plan package.
|
||||
Live testing remains human-owned and pending.
|
||||
|
||||
## Human edits
|
||||
|
||||
No direct manual source edits observed in this follow-up. The human
|
||||
supplied live failures, corrected the earlier daemon/worktree setup,
|
||||
set the narrow landing scope, requested source-fix reuse, and will
|
||||
test before reviewing and committing. Existing commits are retained.
|
||||
The human requested the RuntimeError sentinel and preservation of
|
||||
underlying error context. The raw partner retains the initial output.
|
||||
|
|
@ -0,0 +1,36 @@
|
|||
---
|
||||
created_at: 2026-10-28T16:08:46Z
|
||||
generated_by:
|
||||
harness: opencode
|
||||
provider: openai
|
||||
model: openai/gpt-6.1-sol
|
||||
git_ref: 7242dcb9
|
||||
diff_cmd: git diff HEAD~1..HEAD
|
||||
---
|
||||
|
||||
Implemented a focused websocket-context reconnect fix and real
|
||||
loopback reader/close regressions in the Binance landing worktree.
|
||||
|
||||
> `git diff HEAD~1..HEAD -- piker/data/_web_bs.py`
|
||||
|
||||
Move exception-group collapse outside the websocket context so
|
||||
reader-nursery exit errors are classified by the reconnect loop.
|
||||
Retry NoBsWs.recon_errors there, clear the stale connected event
|
||||
and socket reference, and honor cancellation through unshielded
|
||||
retry sleep. Retain unexpected-error propagation at the outer API.
|
||||
|
||||
> `git diff HEAD~1..HEAD -- tests/test_websocket.py`
|
||||
|
||||
Exercise real TCP, handshake, Ping frames, reader dispatch and
|
||||
wsproto Pong validation. A narrow handler gate advances the real
|
||||
protocol state to LOCAL_CLOSING without sending the Close frame,
|
||||
preventing a peer reply from racing the controlled state. Require
|
||||
fresh application data from the replacement connection. Separate
|
||||
variants cancel at the reader gate and at the actual retry wait.
|
||||
Update the existing unexpected-failure control to use ValueError
|
||||
because LocalProtocolError is now intentionally recoverable.
|
||||
|
||||
The isolated original-code/new-test baseline fails with the exact
|
||||
Pong-in-LOCAL_CLOSING error. The fixed isolated tree passes all 13
|
||||
selected Binance/websocket cases. No live provider, broker orders
|
||||
or production daemon operations were performed by the agent.
|
||||
Some files were not shown because too many files have changed in this diff Show More
Loading…
Reference in New Issue