530 lines
18 KiB
Markdown
530 lines
18 KiB
Markdown
# Repository Landing Plan
|
|
|
|
## Audit Snapshot
|
|
|
|
This plan is based on the local repository state audited on 2026-08-18.
|
|
No refs, worktrees, indexes, stashes, or remote state were changed.
|
|
|
|
The integration baseline is:
|
|
|
|
```text
|
|
main 1abd47609b161c4b9fd460274dc9123755211436
|
|
gitea/main 1abd47609b161c4b9fd460274dc9123755211436
|
|
```
|
|
|
|
`dev/main` and `srht/main` both stop at
|
|
`a00e9c0e648dbba29e6dc4e4cc51267b24aa478c`, 241 commits behind the
|
|
integration baseline. They must not be used as PR bases.
|
|
|
|
The audit covered:
|
|
|
|
- all seven registered worktrees;
|
|
- 168 local or cached remote-tracking refs with post-2025 unmerged tips;
|
|
- patch-equivalent and rebased histories;
|
|
- eight stash entries;
|
|
- reflogs and 483 unreachable commits dated in 2026;
|
|
- old merged topic branches and abandoned provider branches.
|
|
|
|
No recovery-worthy application patch exists only in a stash or unreachable
|
|
object. Useful abandoned work remains reachable through a retained branch.
|
|
|
|
Current forge state was not queried. PR 92 and Tractor PRs 475/490 are
|
|
mentioned by local plan artifacts, but their current head, base, and status
|
|
must be refreshed before branch preparation.
|
|
|
|
## Governing Rules
|
|
|
|
1. Create clean extraction branches from the then-current `main`; do not
|
|
merge any aggregate development branch wholesale.
|
|
2. Preserve source commit order inside each lane, but consolidate fixup,
|
|
formatting, and test-repair commits where noted.
|
|
3. Remove `ai/prompt-io/**`, generated provider links, local skill deployment,
|
|
plan-only commits, editable dependency pins, and local lockfile paths from
|
|
product PRs.
|
|
4. Keep source and backup refs until the extracted PR is accepted and its
|
|
resulting tree is checked against the intended source tree.
|
|
5. Refresh forge metadata and dependency SHAs immediately before preparing
|
|
each branch. Cached remote-tracking refs are not proof of current PR state.
|
|
6. Land independent small fixes while the service/datad chain is blocked.
|
|
|
|
## Landing Sequence
|
|
|
|
### Phase 0: Refresh External State
|
|
|
|
Before changing history:
|
|
|
|
1. Refresh `main` and identify current open/closed PRs for every retained
|
|
Gitea branch.
|
|
2. Verify whether documented piker PR 92 still points at
|
|
`wkt/fix_broadcast_consumers` / `4af885d9`.
|
|
3. Verify Tractor PR 475 and PR 490, record their accepted OIDs, and determine
|
|
the first released or stable Tractor revision containing the required
|
|
service and address APIs.
|
|
4. Select the PyQtGraph source policy: official 0.14, upstream, or the Pikers
|
|
fork. Do not let an incidental lockfile decide this.
|
|
5. Snapshot branch-to-PR mappings before deleting or rewriting any topic ref.
|
|
|
|
### Phase 1: Independent Low-Risk Patches
|
|
|
|
These do not depend on the service/datad stack and should land first.
|
|
|
|
#### 1A. Accounting Synopsis
|
|
|
|
Extract `45d2b53bf912d006f486c644f2add2e313ec6fcc` as a one-commit documentation
|
|
PR touching only `piker/accounting/README.rst`.
|
|
|
|
#### 1B. Symcache Correctness
|
|
|
|
Create one two-commit PR in this order:
|
|
|
|
```text
|
|
a3ed73a40d42db4bbc3032a296413d33b12eab99 strip TOML-incompatible None
|
|
5f392ff0bb0281422d4fdb8e4ebd26e70b04a54a persist actor-local cache
|
|
```
|
|
|
|
Add focused serialization and repeated-open tests. The first patch comes from
|
|
an old base; the second is a direct child of the audited `main`.
|
|
|
|
#### 1C. Small IB Correctness Fixes
|
|
|
|
Use one small IB PR, preserving this order:
|
|
|
|
```text
|
|
5df511f7d2de7712156a3556e1d67986706408c0 handle stringified errno
|
|
de82df727d2e828c5759e32aa758f7f1f7d498e1 expiry and venue helpers
|
|
79be47635f5e4eef81374e63f774e62a1cec5931 unknown venue calendars
|
|
170c95da2815bc0b1fd48e4332f7ff4b4ccaeb99 ledger exchange fallback
|
|
```
|
|
|
|
The first and fourth patches have compatible `main` preimages. Keep the venue
|
|
pair together and add unknown-calendar and exchange-fallback tests.
|
|
|
|
#### 1D. Qt Graceful SIGINT
|
|
|
|
Extract only `8dee35cfa7fbbf9eb614dd709ef86632649a15b4^!`; never use its inherited
|
|
67-commit branch range. Require focused Qt/KBI shutdown coverage.
|
|
|
|
#### 1E. DPI and Font Auto-Calculation
|
|
|
|
Replace the DPI branch swarm with one five-commit PR sourced from the clean
|
|
range `1abd4760..768b6f31`:
|
|
|
|
1. Consolidate `cdb0e841`, `574fb80d`, `3b0c27ec`, `5c7f00a3`, and `e600d61e`
|
|
into the font-size calculation and test commit.
|
|
2. Consolidate `5b1c80a8` and `ccc5a745` into Qt screen diagnostics.
|
|
3. Keep `3f77180b` as the physical-PPI helper.
|
|
4. Consolidate `88f66bae`, `17e7232d`, `2e42d7e1`, and `ce8ad59f` into global
|
|
font zoom and axis relayout.
|
|
5. Keep `768b6f31` as the macOS axis gate.
|
|
|
|
Validate headless Qt, repeated Qt6 zoom, and macOS axis behavior. Evaluate
|
|
`ee353099` separately for its environment-variable finding; do not silently
|
|
fold it into the feature.
|
|
|
|
### Phase 2: Graphics and Gap Foundation
|
|
|
|
The current `gap_annotator` branch mixes DPI, dependency, data hygiene, and
|
|
annotation work. Replace it rather than merging its 24 commits.
|
|
|
|
#### 2A. PyQtGraph Policy
|
|
|
|
After the Phase 0 decision, make one dependency-only PR and regenerate the
|
|
lock on current `main`. Candidate source intents are:
|
|
|
|
```text
|
|
34281459518d838b85f3eaa9a9eae7d6f30c2ca0 official pyqtgraph 0.14
|
|
edecdb3ede62104bf7f881330786dd545d07e799 upstream development revision
|
|
d98c1706 later official-pin variant
|
|
```
|
|
|
|
These are alternatives, not a stack.
|
|
|
|
#### 2B. Gap Annotation Core
|
|
|
|
Consolidate `8c08ddd3` and `073176a4` into one batch-rendering/client-protocol
|
|
commit, while incorporating the later ownership, indexing, cleanup, routing,
|
|
and tests from Phase 6. Remove forced profiling and the editable Tractor pin
|
|
`f9979956`.
|
|
|
|
Keep unrelated canonical-line changes outside this PR:
|
|
|
|
```text
|
|
b87710e9 null-bar tolerance
|
|
e30957b6 cyclic Flume.feed removal
|
|
31859e9d crypto-future symbol handling
|
|
392ea616 obsolete poetry.lock removal
|
|
```
|
|
|
|
Review each as a small independent data/dependency patch rather than making
|
|
it a hidden prerequisite.
|
|
|
|
### Phase 3: Runtime and Service Foundation
|
|
|
|
This is the first blocking chain. Land in the following order.
|
|
|
|
#### 3A. Service Manager to Tractor
|
|
|
|
Use only the clean cached Gitea range:
|
|
|
|
```text
|
|
1abd47609b161c4b9fd460274dc9123755211436
|
|
..8a02f6ea39c11ebc463a5b0a33dbfd5d6492a26e
|
|
```
|
|
|
|
Replace the `hilevel_serman` development pin with the verified Tractor OID.
|
|
Settle self-service cancellation semantics before accepting test changes.
|
|
|
|
#### 3B. Multi-Address Runtime
|
|
|
|
Extract only:
|
|
|
|
```text
|
|
604e5fcf9c73642804115074d80ff688da39325a use tractor.Address
|
|
5466acb7647b762db3688ec424e1fff618ca062a bind-address and registry endpoints
|
|
```
|
|
|
|
Add tuple/wrapped-address, multi-bind, and registry-election tests. Do not
|
|
merge `multiaddrs` or the 54-commit inherited branch range.
|
|
|
|
#### 3C. Current Runtime/Test Repair
|
|
|
|
Land `4485f2b9^..bc6e18d7` on 3A and 3B. Remove prompt provenance and avoid the
|
|
private `tractor.discovery._addr.wrap_address()` API if a public equivalent
|
|
exists. Do not make broader accepted cancellation outcomes hide regressions.
|
|
|
|
### Phase 4: Datad and Brokerd Split
|
|
|
|
Split the original aggregate `datad_service` line into reviewable PRs.
|
|
|
|
#### 4A. Daemon Contracts and Datad Foundation
|
|
|
|
```text
|
|
119d2c049556f240e6b457d08c5c6d564ece011e
|
|
6418121923f33f17ac7283c23ffe984af1f8f98f
|
|
```
|
|
|
|
#### 4B. EMS and Feed Ownership Split
|
|
|
|
```text
|
|
59d5d9a66d5b71d49cb87d514d0b2436190d330a
|
|
eee19de0901a8f7367885f6c8e8b8af18626776c
|
|
f15f8178a3fd9af223673666db194e6f9234ca7e
|
|
```
|
|
|
|
Replace live-Kraken-dependent startup tests rather than increasing timeouts.
|
|
Verify capabilities, actor IDs, client-ID offsets, and teardown.
|
|
|
|
#### 4C. Datad Logging and IB Contract Qualification
|
|
|
|
Keep datad logging with datad:
|
|
|
|
```text
|
|
ba32f286b9651d6f65a4a17d9e9992a8a5b267c0
|
|
```
|
|
|
|
Use a separate IB follow-up in this order:
|
|
|
|
```text
|
|
456c6a55677febd0e61bf394a1acc835c1941223
|
|
b0766764f03b921b1fe05bbe8586db698c8d2c03
|
|
```
|
|
|
|
The contract qualification patches repair caches that no longer warm in the
|
|
same process after the datad split.
|
|
|
|
Omit `f084e899` and `b73300c8`: they are design/provenance and AI-skill-only
|
|
commits, not runtime dependencies.
|
|
|
|
### Phase 5: Post-Datad Concurrency Fixes
|
|
|
|
#### 5A. IB MethodProxy Correlation
|
|
|
|
Combine/rework these as one focused request-correlation series:
|
|
|
|
```text
|
|
0404e4230ef00cbde21cfa2f73006b4dd1211fcd request ID correlation
|
|
2aa66c2cf6c76b8e270af9367d27ca16f63e7024 response ownership/routing
|
|
```
|
|
|
|
Preserve the final concurrency, cancellation, and EOF tests. Strip skill and
|
|
prompt files.
|
|
|
|
#### 5B. Broadcast Consumer Ownership
|
|
|
|
Preserve source chronology but submit small adapter-scoped PRs where forge
|
|
review permits:
|
|
|
|
```text
|
|
12b0d66e854247e92cb5a8668be71d816c72d856 EMS dark quotes
|
|
b691422b2678e1e7b3d4b7e1d70c25d6c644c6d3 generic cached feeds
|
|
ddac3553a3bd4e2c29a7d2188dffb2c10ee065da Deribit
|
|
4af885d90f6e4bcfa963f2f7a1d40436d3c928f3 IB
|
|
```
|
|
|
|
Each source commit has a dedicated test. If local plan data is current, these
|
|
are represented by piker PR 92; verify before replacing that PR.
|
|
|
|
### Phase 6: Backfill, Storage, and Chart Gaps
|
|
|
|
#### 6A. Backfill and NativeDB Correctness
|
|
|
|
Rebuild the source sequence without plan/provenance commits:
|
|
|
|
```text
|
|
976db6c3 normalize gaps after sorted dedupe
|
|
931f22d2 drop orphaned backfill events
|
|
c60a5a45 report non-positive timestamp steps
|
|
b000db94 atomic non-destructive NativeDB backfills
|
|
689df816 remove redundant writer locks
|
|
19c6d98e normalize provider OHLCV
|
|
ed85721c clean test-owned SHM leaks
|
|
1f309170 repair legacy timestamps
|
|
d7d537d5 raw OHLCV audit
|
|
922a5df8 qualification workflow
|
|
ce33deb6 IB UTC history syntax
|
|
d359b1cb repair first-slot sentinels
|
|
ad6c560f guard invalid SHM snapshots
|
|
0846cbd4 repair IB startup gaps
|
|
74ea2152 expose storage state
|
|
f0f3c806 simplify store discovery
|
|
06d5ea5d repair sparse IB prepends
|
|
```
|
|
|
|
Split this into NativeDB/storage core and IB-history adapter PRs if the
|
|
intermediate tree remains testable. Omit `99fb9d2e`, `6913e80d`, and
|
|
`84a6d47b`; they are plans/profiling notes. Strip `ai/prompt-io/**`, skill
|
|
files, and `plans/**` from product commits.
|
|
|
|
Known blockers are deterministic NativeDB/SHM tests, live-provider
|
|
qualification, exact-null repair, venue classification, and explicit history
|
|
revision publication.
|
|
|
|
#### 6B. Chart-Local Gap Overlays
|
|
|
|
Land this final visual ownership/test sequence:
|
|
|
|
```text
|
|
f65df6a138cb8dc2119feb421c122a2c8ec2b89e
|
|
78982d8280870be6e2f7aa183e86f06265b63773
|
|
711666f9c831f9ae46b3a615c7f4c39beb1de140
|
|
b738c9f4f7a67da0da64b5acc5adcb3f1a10388b
|
|
335190a4e3ff5712626288ac5e81693b9646c140
|
|
```
|
|
|
|
Omit `09ddcf50`, which is a pytest-qt plan only. Preserve the real actor test.
|
|
History-revision-triggered overlay refresh remains follow-up work and should
|
|
not block the ownership correction.
|
|
|
|
### Phase 7: Independent Provider and UI Follow-Ups
|
|
|
|
These can proceed in parallel once their direct prerequisites settle.
|
|
|
|
#### 7A. Kraken Account and Token Repair
|
|
|
|
Use two focused PRs:
|
|
|
|
```text
|
|
ebc5bbd4 + 923b4de2 account aliases/config loading
|
|
5387538b + 709269fc + Kraken hunks
|
|
from 8299c658 stale token/session refresh
|
|
```
|
|
|
|
Add account-default, expiry, reconnect, and stale-token tests.
|
|
|
|
#### 7B. EMS Edit-State Repair
|
|
|
|
Extract `8cefc1bd`, `e1cd3fd9`, and the EMS portion of `9f7c38a3`. Add
|
|
rapid-edit/cancel tests. Reimplement the nested order-mode lifetime patch from
|
|
`e05c2589` only after this PR, with explicit cancellation and context-exit
|
|
coverage.
|
|
|
|
#### 7C. General Broker Error Schema
|
|
|
|
Preserve this order:
|
|
|
|
```text
|
|
7f8a198c98528feb8c0981245a64288b84b946fd
|
|
84c742a885e8ca374b98fea5b84858bd2c3c38c4
|
|
bf43036fe10513f5532cacf24d4f381944f9aec8
|
|
```
|
|
|
|
Adapt IPC exception registration to the accepted Tractor API and remove
|
|
unrelated IB hunks.
|
|
|
|
#### 7D. Annotation Refresh Positioning
|
|
|
|
Rebase `555c2fc27e4dca0215454611b169232758955b8f` and add zoom, range, and pan
|
|
tests. This is unique but should follow the DPI/gap graphics lanes.
|
|
|
|
### Phase 8: Build, Tooling, and Documentation
|
|
|
|
#### 8A. Nix Dev-Shell Refresh
|
|
|
|
Apply in order and regenerate the final lock after dependency decisions:
|
|
|
|
```text
|
|
f6dc218a8e6028489a07e1c1fb4971d9f2fbda58
|
|
3da8900f681a15fd4ea9f4728ff788633b54275f
|
|
ceb806c1365497242ab025681f41f9c3d37078ab
|
|
58ffe487759ca6816f7224488b99966f4acaf6a5
|
|
```
|
|
|
|
#### 8B. Run-Tests Harness
|
|
|
|
Create one reproducible repository-tooling PR from the functional files in
|
|
`c83983b0` and `e766c158`. Keep the Piker `py313`, Nix, and
|
|
`pytest -p no:xonsh` reference in-repo. Do not rely on ignored external
|
|
symlinks in a fresh clone.
|
|
|
|
#### 8C. Optional Dependencies and Lint Policy
|
|
|
|
Reapply the VisiData intent from `b55e7e1d` to current `pyproject.toml` and
|
|
regenerate the lock; do not cherry-pick its stale lockfile.
|
|
|
|
Treat `c86e69b2` and `cd23021b` as a Ruff-policy proposal. Run a full-tree
|
|
violation audit before accepting it. Only then consider the dependent lint
|
|
skill in `bb2ea50f`.
|
|
|
|
#### 8D. NGI Proposal Documents
|
|
|
|
Use `ccb6f191`, `7060faa5`, and `b05292b0` as a documentation-only series.
|
|
Omit `ca100290` unless its generated summary is intentionally reviewed and
|
|
accepted.
|
|
|
|
## Deferred but Preserved Work
|
|
|
|
### Deribit Backend
|
|
|
|
Preserve `5e371f1d..b209512e` as the canonical 13-commit restoration source.
|
|
It is high-value but 249 commits behind, lacks focused tests, contains
|
|
whitespace errors, and may target a drifted API. Validate and reconstruct it;
|
|
never merge the old branch directly.
|
|
|
|
Keep the max-pain range `b209512e..b9321dbb` behind the backend restoration.
|
|
Defer OI/native-storage commits `8f1e082c` and `f756f0fd` until both Deribit
|
|
and NativeDB APIs settle.
|
|
|
|
### Symbol Search
|
|
|
|
Do not land `fixing_piker_search` / `376a900b`. It pauses almost every backend
|
|
and mixes registry initialization. Salvage only the `brokermod.name` and
|
|
nursery-argument correction into a new tested patch after MethodProxy work.
|
|
|
|
### Other Deferred Work
|
|
|
|
- Recreate private testing helpers from `4f2fd73b`; do not cherry-pick its
|
|
73-commit aggregate ancestry.
|
|
- Evaluate legacy Binance HTTPX commits `01a43763` and `0c61b0d9` against the
|
|
current API before deciding whether to drop them.
|
|
- Preserve broad macOS branches until their unique geometry, SHM, config,
|
|
script, dependency, and documentation changes receive a separate audit.
|
|
- Drop decimal EMS `TOSQUASH` remnants unless a current regression proves
|
|
they are still required.
|
|
|
|
## Worktree Disposition
|
|
|
|
All worktrees are read-only until their source lanes land.
|
|
|
|
| Worktree branch | Head | State | Disposition |
|
|
|---|---|---|---|
|
|
| `chart_local_gapper` | `09ddcf50` | dirty root worktree, one plan commit above chart source | retain; never reset; omit plan commit |
|
|
| `wkt/backfiller_deep_fixes` | `f0f3c806` | clean | retain as backfill source |
|
|
| `wkt/fix_broadcast_consumers` | `4af885d9` | untracked PR-message files | retain through PR 92 verification |
|
|
| `wkt/flake_update` | `ceb806c1` | untracked skill config | retain as Nix source |
|
|
| `wkt/run_tests_skill` | `c83983b0` | untracked skill config | retain as harness source |
|
|
| `wkt/chart_local_gap_overlays` | `335190a4` | clean | retain as chart source |
|
|
| `moar_ai_skillz` | `4880f0d3` | clean | reject generated deployment state; remove after archival |
|
|
|
|
The root worktree also has user-owned `.gitignore` changes and untracked
|
|
`.modden/`, `todo/`, and `wkts/`. No cleanup operation may touch them.
|
|
|
|
## Stale Branch Cleanup
|
|
|
|
Deletion is a final post-landing operation, not part of extraction. Preserve a
|
|
mapping from each removed ref to its replacement merge commit.
|
|
|
|
### Already Merged or Fully Superseded
|
|
|
|
Propose removing local and matching remote topic refs for:
|
|
|
|
```text
|
|
brokers_refinery*
|
|
hist_backfill_fixes* backups
|
|
CONT2_hist_fixes
|
|
cont_hist_fixes
|
|
ib_venue_closures*
|
|
stop_is_oec
|
|
stop_is_eoc
|
|
providers_sync
|
|
port_to_latest_tractor
|
|
to_asyncio_api_update
|
|
tractor_struct_and_godw_mod
|
|
tpt_closed_and_finally_footguns*
|
|
fix_tractor_logging_POSTREBASEBACKUP
|
|
wayland_nix_py313
|
|
nixround2*
|
|
bump_polars and dev/bump_polars
|
|
pdbp_bump
|
|
ems_no_last_required
|
|
decimal_prices_thru_ems* backups
|
|
binance_api_3.1*
|
|
alt_tpts_for_perf*
|
|
kucoin_and_binance_BACKUP
|
|
```
|
|
|
|
### Remove After Replacement PRs Land
|
|
|
|
```text
|
|
dpi-font-auto-calc*
|
|
dpi_scaling_round2
|
|
dpi_font_autocalc_fomo_bits
|
|
axes_w_zoom_refinery
|
|
gap_annotator_BACKUP
|
|
dev/gap_annotator
|
|
service_mng_to_tractor_PRE_MAIN_REBASE
|
|
local aggregate service_mng_to_tractor
|
|
repair_tests
|
|
multiaddrs
|
|
datad_post_mortem duplicate
|
|
dev/wkt/fix_broadcast_consumers duplicate
|
|
backfiller_deep_fixes / ib_methproxy_refinery aliases
|
|
chart_local_gapper aliases
|
|
flake_update aliases
|
|
worktree-ngi0_submission
|
|
py_codestyling aggregate
|
|
moar_ai_skillz local/dev/gitea refs
|
|
```
|
|
|
|
Do not blanket-delete `macos_fixes_*`, `macos_*with_readme`,
|
|
`macos_dnks_vibed_fixes`, Deribit sources, or `nested_order_mode_block` until
|
|
their unrelated unique commits are preserved or explicitly rejected.
|
|
|
|
## Abandoned Object Sweep
|
|
|
|
No additional recovery lane is required.
|
|
|
|
- Unreachable datad and backfiller commits are patch-equivalent to retained
|
|
canonical lines.
|
|
- Temporary chart commit-plan trees match final chart commits.
|
|
- Unreachable multi-address and order-mode experiments are superseded or
|
|
contain debugging pauses/cancellation suppression.
|
|
- Seven stashes contain local dependencies, lock churn, skill links, or
|
|
`.gitignore` work.
|
|
- Stash `4500bb12` contains a unique profiler predicate change, but it makes
|
|
the default environment mode ineffective and should not be recovered.
|
|
|
|
Keep stashes untouched until their owners explicitly dispose of them.
|
|
|
|
## Completion Criteria
|
|
|
|
The repository is ready for branch cleanup only when:
|
|
|
|
1. every selected lane has a clean `main`-based branch and verified forge
|
|
head/base OIDs;
|
|
2. extracted product trees have been compared against their source intent;
|
|
3. dependency pins are reproducible and contain no editable local paths;
|
|
4. lane-specific tests and the full supported suite pass;
|
|
5. accepted PR merge commits are recorded against all source refs;
|
|
6. no dirty worktree or stash is touched by cleanup;
|
|
7. deferred branches retain explicit owners and reasons for preservation.
|