diff --git a/ai/prompt-io/opencode/20260818T064340Z_09ddcf50_prompt_io.md b/ai/prompt-io/opencode/20260818T064340Z_09ddcf50_prompt_io.md new file mode 100644 index 00000000..25797a90 --- /dev/null +++ b/ai/prompt-io/opencode/20260818T064340Z_09ddcf50_prompt_io.md @@ -0,0 +1,38 @@ +--- +model: openai/gpt-5.6-sol +service: opencode +session: unavailable +timestamp: 2026-08-18T06:43:40Z +git_ref: 09ddcf50 +scope: docs +substantive: true +raw_file: 20260818T064340Z_09ddcf50_prompt_io.raw.md +--- + +## Prompt + +Audit all development branches and worktrees, consolidate duplicate and +rebased histories into the smallest subsystem-scoped PRs, preserve useful +chronology, identify stale refs, omit obsolete AI-skill changes, and recover +any useful abandoned work before proposing an optimal route into `main`. + +## Response summary + +Adds a repository-wide landing plan grounded in exact branch ancestry, +patch-equivalence, worktree state, and object recovery. The plan separates +independent fixes from the runtime/datad/backfill dependency chain, reduces +the DPI branch swarm to one five-commit series, records deferred provider +work, and proposes staged cleanup only after replacement PRs land. + +## Files changed + +- `plans/opencode/repo-land-plan.md` - record the audited landing sequence, + exact patch sources, blockers, worktree constraints, and cleanup groups. +- `ai/prompt-io/opencode/20260818T064340Z_09ddcf50_prompt_io.raw.md` - retain + the unedited prompt and generated-output summary. +- `ai/prompt-io/opencode/20260818T064340Z_09ddcf50_prompt_io.md` - record + structured provenance for the generated documentation. + +## Human edits + +None - the changes remain uncommitted for human review. diff --git a/ai/prompt-io/opencode/20260818T064340Z_09ddcf50_prompt_io.raw.md b/ai/prompt-io/opencode/20260818T064340Z_09ddcf50_prompt_io.raw.md new file mode 100644 index 00000000..db67f926 --- /dev/null +++ b/ai/prompt-io/opencode/20260818T064340Z_09ddcf50_prompt_io.raw.md @@ -0,0 +1,45 @@ +--- +model: openai/gpt-5.6-sol +service: opencode +timestamp: 2026-08-18T06:43:40Z +git_ref: 09ddcf50 +diff_cmd: git diff HEAD~1..HEAD +--- + +## Prompt + +The user asked OpenCode to act as repository supervisor and audit every +development branch and worktree for an optimal route into `main`. The audit +needed to identify duplicate and rebased histories, especially the DPI/font +branches, reconstruct subsystem-scoped patch sets, preserve chronology, +propose stale-ref removal, exclude obsolete AI-skill changes, and recover any +useful abandoned commits. + +The user later checked that the long-running audit was still active. OpenCode +confirmed progress and completed the worktree, branch, patch-ID, reflog, +stash, and unreachable-object sweep before writing the plan. + +## Generated documentation + +> `git diff HEAD~1..HEAD -- plans/opencode/repo-land-plan.md` + +Creates a repository-wide landing plan based on `main` and `gitea/main` at +`1abd47609b161c4b9fd460274dc9123755211436`. It records the audit scope, +integration rules, dependency-ordered PR lanes, exact source commits, branch +cleanup groups, worktree ownership constraints, deferred histories, and the +abandoned-object recovery result. + +The plan reduces aggregate development histories into independent fixes, a +five-commit DPI series, graphics/gap foundations, runtime and service ports, +the datad/brokerd split, post-datad concurrency repairs, NativeDB/backfill +work, chart-local overlays, provider follow-ups, and build/tooling changes. +It explicitly excludes generated skill deployment, prompt records from +product patches, editable dependency pins, and plan-only commits. + +## Audit output + +The read-only audit covers seven worktrees, 168 recent unmerged refs, eight +stash entries, patch-equivalent rebases, and 483 unreachable commits. It finds +no recovery-worthy application patch that exists only in a stash or +unreachable object. Current forge status remains a separate refresh step +because this pass uses local and cached repository state. diff --git a/plans/opencode/repo-land-plan.md b/plans/opencode/repo-land-plan.md new file mode 100644 index 00000000..c314f483 --- /dev/null +++ b/plans/opencode/repo-land-plan.md @@ -0,0 +1,529 @@ +# Repository Landing Plan + +## Audit Snapshot + +This plan is based on the local repository state audited on 2026-08-18. +No refs, worktrees, indexes, stashes, or remote state were changed. + +The integration baseline is: + +```text +main 1abd47609b161c4b9fd460274dc9123755211436 +gitea/main 1abd47609b161c4b9fd460274dc9123755211436 +``` + +`dev/main` and `srht/main` both stop at +`a00e9c0e648dbba29e6dc4e4cc51267b24aa478c`, 241 commits behind the +integration baseline. They must not be used as PR bases. + +The audit covered: + +- all seven registered worktrees; +- 168 local or cached remote-tracking refs with post-2025 unmerged tips; +- patch-equivalent and rebased histories; +- eight stash entries; +- reflogs and 483 unreachable commits dated in 2026; +- old merged topic branches and abandoned provider branches. + +No recovery-worthy application patch exists only in a stash or unreachable +object. Useful abandoned work remains reachable through a retained branch. + +Current forge state was not queried. PR 92 and Tractor PRs 475/490 are +mentioned by local plan artifacts, but their current head, base, and status +must be refreshed before branch preparation. + +## Governing Rules + +1. Create clean extraction branches from the then-current `main`; do not + merge any aggregate development branch wholesale. +2. Preserve source commit order inside each lane, but consolidate fixup, + formatting, and test-repair commits where noted. +3. Remove `ai/prompt-io/**`, generated provider links, local skill deployment, + plan-only commits, editable dependency pins, and local lockfile paths from + product PRs. +4. Keep source and backup refs until the extracted PR is accepted and its + resulting tree is checked against the intended source tree. +5. Refresh forge metadata and dependency SHAs immediately before preparing + each branch. Cached remote-tracking refs are not proof of current PR state. +6. Land independent small fixes while the service/datad chain is blocked. + +## Landing Sequence + +### Phase 0: Refresh External State + +Before changing history: + +1. Refresh `main` and identify current open/closed PRs for every retained + Gitea branch. +2. Verify whether documented piker PR 92 still points at + `wkt/fix_broadcast_consumers` / `4af885d9`. +3. Verify Tractor PR 475 and PR 490, record their accepted OIDs, and determine + the first released or stable Tractor revision containing the required + service and address APIs. +4. Select the PyQtGraph source policy: official 0.14, upstream, or the Pikers + fork. Do not let an incidental lockfile decide this. +5. Snapshot branch-to-PR mappings before deleting or rewriting any topic ref. + +### Phase 1: Independent Low-Risk Patches + +These do not depend on the service/datad stack and should land first. + +#### 1A. Accounting Synopsis + +Extract `45d2b53bf912d006f486c644f2add2e313ec6fcc` as a one-commit documentation +PR touching only `piker/accounting/README.rst`. + +#### 1B. Symcache Correctness + +Create one two-commit PR in this order: + +```text +a3ed73a40d42db4bbc3032a296413d33b12eab99 strip TOML-incompatible None +5f392ff0bb0281422d4fdb8e4ebd26e70b04a54a persist actor-local cache +``` + +Add focused serialization and repeated-open tests. The first patch comes from +an old base; the second is a direct child of the audited `main`. + +#### 1C. Small IB Correctness Fixes + +Use one small IB PR, preserving this order: + +```text +5df511f7d2de7712156a3556e1d67986706408c0 handle stringified errno +de82df727d2e828c5759e32aa758f7f1f7d498e1 expiry and venue helpers +79be47635f5e4eef81374e63f774e62a1cec5931 unknown venue calendars +170c95da2815bc0b1fd48e4332f7ff4b4ccaeb99 ledger exchange fallback +``` + +The first and fourth patches have compatible `main` preimages. Keep the venue +pair together and add unknown-calendar and exchange-fallback tests. + +#### 1D. Qt Graceful SIGINT + +Extract only `8dee35cfa7fbbf9eb614dd709ef86632649a15b4^!`; never use its inherited +67-commit branch range. Require focused Qt/KBI shutdown coverage. + +#### 1E. DPI and Font Auto-Calculation + +Replace the DPI branch swarm with one five-commit PR sourced from the clean +range `1abd4760..768b6f31`: + +1. Consolidate `cdb0e841`, `574fb80d`, `3b0c27ec`, `5c7f00a3`, and `e600d61e` + into the font-size calculation and test commit. +2. Consolidate `5b1c80a8` and `ccc5a745` into Qt screen diagnostics. +3. Keep `3f77180b` as the physical-PPI helper. +4. Consolidate `88f66bae`, `17e7232d`, `2e42d7e1`, and `ce8ad59f` into global + font zoom and axis relayout. +5. Keep `768b6f31` as the macOS axis gate. + +Validate headless Qt, repeated Qt6 zoom, and macOS axis behavior. Evaluate +`ee353099` separately for its environment-variable finding; do not silently +fold it into the feature. + +### Phase 2: Graphics and Gap Foundation + +The current `gap_annotator` branch mixes DPI, dependency, data hygiene, and +annotation work. Replace it rather than merging its 24 commits. + +#### 2A. PyQtGraph Policy + +After the Phase 0 decision, make one dependency-only PR and regenerate the +lock on current `main`. Candidate source intents are: + +```text +34281459518d838b85f3eaa9a9eae7d6f30c2ca0 official pyqtgraph 0.14 +edecdb3ede62104bf7f881330786dd545d07e799 upstream development revision +d98c1706 later official-pin variant +``` + +These are alternatives, not a stack. + +#### 2B. Gap Annotation Core + +Consolidate `8c08ddd3` and `073176a4` into one batch-rendering/client-protocol +commit, while incorporating the later ownership, indexing, cleanup, routing, +and tests from Phase 6. Remove forced profiling and the editable Tractor pin +`f9979956`. + +Keep unrelated canonical-line changes outside this PR: + +```text +b87710e9 null-bar tolerance +e30957b6 cyclic Flume.feed removal +31859e9d crypto-future symbol handling +392ea616 obsolete poetry.lock removal +``` + +Review each as a small independent data/dependency patch rather than making +it a hidden prerequisite. + +### Phase 3: Runtime and Service Foundation + +This is the first blocking chain. Land in the following order. + +#### 3A. Service Manager to Tractor + +Use only the clean cached Gitea range: + +```text +1abd47609b161c4b9fd460274dc9123755211436 + ..8a02f6ea39c11ebc463a5b0a33dbfd5d6492a26e +``` + +Replace the `hilevel_serman` development pin with the verified Tractor OID. +Settle self-service cancellation semantics before accepting test changes. + +#### 3B. Multi-Address Runtime + +Extract only: + +```text +604e5fcf9c73642804115074d80ff688da39325a use tractor.Address +5466acb7647b762db3688ec424e1fff618ca062a bind-address and registry endpoints +``` + +Add tuple/wrapped-address, multi-bind, and registry-election tests. Do not +merge `multiaddrs` or the 54-commit inherited branch range. + +#### 3C. Current Runtime/Test Repair + +Land `4485f2b9^..bc6e18d7` on 3A and 3B. Remove prompt provenance and avoid the +private `tractor.discovery._addr.wrap_address()` API if a public equivalent +exists. Do not make broader accepted cancellation outcomes hide regressions. + +### Phase 4: Datad and Brokerd Split + +Split the original aggregate `datad_service` line into reviewable PRs. + +#### 4A. Daemon Contracts and Datad Foundation + +```text +119d2c049556f240e6b457d08c5c6d564ece011e +6418121923f33f17ac7283c23ffe984af1f8f98f +``` + +#### 4B. EMS and Feed Ownership Split + +```text +59d5d9a66d5b71d49cb87d514d0b2436190d330a +eee19de0901a8f7367885f6c8e8b8af18626776c +f15f8178a3fd9af223673666db194e6f9234ca7e +``` + +Replace live-Kraken-dependent startup tests rather than increasing timeouts. +Verify capabilities, actor IDs, client-ID offsets, and teardown. + +#### 4C. Datad Logging and IB Contract Qualification + +Keep datad logging with datad: + +```text +ba32f286b9651d6f65a4a17d9e9992a8a5b267c0 +``` + +Use a separate IB follow-up in this order: + +```text +456c6a55677febd0e61bf394a1acc835c1941223 +b0766764f03b921b1fe05bbe8586db698c8d2c03 +``` + +The contract qualification patches repair caches that no longer warm in the +same process after the datad split. + +Omit `f084e899` and `b73300c8`: they are design/provenance and AI-skill-only +commits, not runtime dependencies. + +### Phase 5: Post-Datad Concurrency Fixes + +#### 5A. IB MethodProxy Correlation + +Combine/rework these as one focused request-correlation series: + +```text +0404e4230ef00cbde21cfa2f73006b4dd1211fcd request ID correlation +2aa66c2cf6c76b8e270af9367d27ca16f63e7024 response ownership/routing +``` + +Preserve the final concurrency, cancellation, and EOF tests. Strip skill and +prompt files. + +#### 5B. Broadcast Consumer Ownership + +Preserve source chronology but submit small adapter-scoped PRs where forge +review permits: + +```text +12b0d66e854247e92cb5a8668be71d816c72d856 EMS dark quotes +b691422b2678e1e7b3d4b7e1d70c25d6c644c6d3 generic cached feeds +ddac3553a3bd4e2c29a7d2188dffb2c10ee065da Deribit +4af885d90f6e4bcfa963f2f7a1d40436d3c928f3 IB +``` + +Each source commit has a dedicated test. If local plan data is current, these +are represented by piker PR 92; verify before replacing that PR. + +### Phase 6: Backfill, Storage, and Chart Gaps + +#### 6A. Backfill and NativeDB Correctness + +Rebuild the source sequence without plan/provenance commits: + +```text +976db6c3 normalize gaps after sorted dedupe +931f22d2 drop orphaned backfill events +c60a5a45 report non-positive timestamp steps +b000db94 atomic non-destructive NativeDB backfills +689df816 remove redundant writer locks +19c6d98e normalize provider OHLCV +ed85721c clean test-owned SHM leaks +1f309170 repair legacy timestamps +d7d537d5 raw OHLCV audit +922a5df8 qualification workflow +ce33deb6 IB UTC history syntax +d359b1cb repair first-slot sentinels +ad6c560f guard invalid SHM snapshots +0846cbd4 repair IB startup gaps +74ea2152 expose storage state +f0f3c806 simplify store discovery +06d5ea5d repair sparse IB prepends +``` + +Split this into NativeDB/storage core and IB-history adapter PRs if the +intermediate tree remains testable. Omit `99fb9d2e`, `6913e80d`, and +`84a6d47b`; they are plans/profiling notes. Strip `ai/prompt-io/**`, skill +files, and `plans/**` from product commits. + +Known blockers are deterministic NativeDB/SHM tests, live-provider +qualification, exact-null repair, venue classification, and explicit history +revision publication. + +#### 6B. Chart-Local Gap Overlays + +Land this final visual ownership/test sequence: + +```text +f65df6a138cb8dc2119feb421c122a2c8ec2b89e +78982d8280870be6e2f7aa183e86f06265b63773 +711666f9c831f9ae46b3a615c7f4c39beb1de140 +b738c9f4f7a67da0da64b5acc5adcb3f1a10388b +335190a4e3ff5712626288ac5e81693b9646c140 +``` + +Omit `09ddcf50`, which is a pytest-qt plan only. Preserve the real actor test. +History-revision-triggered overlay refresh remains follow-up work and should +not block the ownership correction. + +### Phase 7: Independent Provider and UI Follow-Ups + +These can proceed in parallel once their direct prerequisites settle. + +#### 7A. Kraken Account and Token Repair + +Use two focused PRs: + +```text +ebc5bbd4 + 923b4de2 account aliases/config loading +5387538b + 709269fc + Kraken hunks +from 8299c658 stale token/session refresh +``` + +Add account-default, expiry, reconnect, and stale-token tests. + +#### 7B. EMS Edit-State Repair + +Extract `8cefc1bd`, `e1cd3fd9`, and the EMS portion of `9f7c38a3`. Add +rapid-edit/cancel tests. Reimplement the nested order-mode lifetime patch from +`e05c2589` only after this PR, with explicit cancellation and context-exit +coverage. + +#### 7C. General Broker Error Schema + +Preserve this order: + +```text +7f8a198c98528feb8c0981245a64288b84b946fd +84c742a885e8ca374b98fea5b84858bd2c3c38c4 +bf43036fe10513f5532cacf24d4f381944f9aec8 +``` + +Adapt IPC exception registration to the accepted Tractor API and remove +unrelated IB hunks. + +#### 7D. Annotation Refresh Positioning + +Rebase `555c2fc27e4dca0215454611b169232758955b8f` and add zoom, range, and pan +tests. This is unique but should follow the DPI/gap graphics lanes. + +### Phase 8: Build, Tooling, and Documentation + +#### 8A. Nix Dev-Shell Refresh + +Apply in order and regenerate the final lock after dependency decisions: + +```text +f6dc218a8e6028489a07e1c1fb4971d9f2fbda58 +3da8900f681a15fd4ea9f4728ff788633b54275f +ceb806c1365497242ab025681f41f9c3d37078ab +58ffe487759ca6816f7224488b99966f4acaf6a5 +``` + +#### 8B. Run-Tests Harness + +Create one reproducible repository-tooling PR from the functional files in +`c83983b0` and `e766c158`. Keep the Piker `py313`, Nix, and +`pytest -p no:xonsh` reference in-repo. Do not rely on ignored external +symlinks in a fresh clone. + +#### 8C. Optional Dependencies and Lint Policy + +Reapply the VisiData intent from `b55e7e1d` to current `pyproject.toml` and +regenerate the lock; do not cherry-pick its stale lockfile. + +Treat `c86e69b2` and `cd23021b` as a Ruff-policy proposal. Run a full-tree +violation audit before accepting it. Only then consider the dependent lint +skill in `bb2ea50f`. + +#### 8D. NGI Proposal Documents + +Use `ccb6f191`, `7060faa5`, and `b05292b0` as a documentation-only series. +Omit `ca100290` unless its generated summary is intentionally reviewed and +accepted. + +## Deferred but Preserved Work + +### Deribit Backend + +Preserve `5e371f1d..b209512e` as the canonical 13-commit restoration source. +It is high-value but 249 commits behind, lacks focused tests, contains +whitespace errors, and may target a drifted API. Validate and reconstruct it; +never merge the old branch directly. + +Keep the max-pain range `b209512e..b9321dbb` behind the backend restoration. +Defer OI/native-storage commits `8f1e082c` and `f756f0fd` until both Deribit +and NativeDB APIs settle. + +### Symbol Search + +Do not land `fixing_piker_search` / `376a900b`. It pauses almost every backend +and mixes registry initialization. Salvage only the `brokermod.name` and +nursery-argument correction into a new tested patch after MethodProxy work. + +### Other Deferred Work + +- Recreate private testing helpers from `4f2fd73b`; do not cherry-pick its + 73-commit aggregate ancestry. +- Evaluate legacy Binance HTTPX commits `01a43763` and `0c61b0d9` against the + current API before deciding whether to drop them. +- Preserve broad macOS branches until their unique geometry, SHM, config, + script, dependency, and documentation changes receive a separate audit. +- Drop decimal EMS `TOSQUASH` remnants unless a current regression proves + they are still required. + +## Worktree Disposition + +All worktrees are read-only until their source lanes land. + +| Worktree branch | Head | State | Disposition | +|---|---|---|---| +| `chart_local_gapper` | `09ddcf50` | dirty root worktree, one plan commit above chart source | retain; never reset; omit plan commit | +| `wkt/backfiller_deep_fixes` | `f0f3c806` | clean | retain as backfill source | +| `wkt/fix_broadcast_consumers` | `4af885d9` | untracked PR-message files | retain through PR 92 verification | +| `wkt/flake_update` | `ceb806c1` | untracked skill config | retain as Nix source | +| `wkt/run_tests_skill` | `c83983b0` | untracked skill config | retain as harness source | +| `wkt/chart_local_gap_overlays` | `335190a4` | clean | retain as chart source | +| `moar_ai_skillz` | `4880f0d3` | clean | reject generated deployment state; remove after archival | + +The root worktree also has user-owned `.gitignore` changes and untracked +`.modden/`, `todo/`, and `wkts/`. No cleanup operation may touch them. + +## Stale Branch Cleanup + +Deletion is a final post-landing operation, not part of extraction. Preserve a +mapping from each removed ref to its replacement merge commit. + +### Already Merged or Fully Superseded + +Propose removing local and matching remote topic refs for: + +```text +brokers_refinery* +hist_backfill_fixes* backups +CONT2_hist_fixes +cont_hist_fixes +ib_venue_closures* +stop_is_oec +stop_is_eoc +providers_sync +port_to_latest_tractor +to_asyncio_api_update +tractor_struct_and_godw_mod +tpt_closed_and_finally_footguns* +fix_tractor_logging_POSTREBASEBACKUP +wayland_nix_py313 +nixround2* +bump_polars and dev/bump_polars +pdbp_bump +ems_no_last_required +decimal_prices_thru_ems* backups +binance_api_3.1* +alt_tpts_for_perf* +kucoin_and_binance_BACKUP +``` + +### Remove After Replacement PRs Land + +```text +dpi-font-auto-calc* +dpi_scaling_round2 +dpi_font_autocalc_fomo_bits +axes_w_zoom_refinery +gap_annotator_BACKUP +dev/gap_annotator +service_mng_to_tractor_PRE_MAIN_REBASE +local aggregate service_mng_to_tractor +repair_tests +multiaddrs +datad_post_mortem duplicate +dev/wkt/fix_broadcast_consumers duplicate +backfiller_deep_fixes / ib_methproxy_refinery aliases +chart_local_gapper aliases +flake_update aliases +worktree-ngi0_submission +py_codestyling aggregate +moar_ai_skillz local/dev/gitea refs +``` + +Do not blanket-delete `macos_fixes_*`, `macos_*with_readme`, +`macos_dnks_vibed_fixes`, Deribit sources, or `nested_order_mode_block` until +their unrelated unique commits are preserved or explicitly rejected. + +## Abandoned Object Sweep + +No additional recovery lane is required. + +- Unreachable datad and backfiller commits are patch-equivalent to retained + canonical lines. +- Temporary chart commit-plan trees match final chart commits. +- Unreachable multi-address and order-mode experiments are superseded or + contain debugging pauses/cancellation suppression. +- Seven stashes contain local dependencies, lock churn, skill links, or + `.gitignore` work. +- Stash `4500bb12` contains a unique profiler predicate change, but it makes + the default environment mode ineffective and should not be recovered. + +Keep stashes untouched until their owners explicitly dispose of them. + +## Completion Criteria + +The repository is ready for branch cleanup only when: + +1. every selected lane has a clean `main`-based branch and verified forge + head/base OIDs; +2. extracted product trees have been compared against their source intent; +3. dependency pins are reproducible and contain no editable local paths; +4. lane-specific tests and the full supported suite pass; +5. accepted PR merge commits are recorded against all source refs; +6. no dirty worktree or stash is touched by cleanup; +7. deferred branches retain explicit owners and reasons for preservation.