diff --git a/.agents/skills/taken/SKILL.md b/.agents/skills/taken/SKILL.md new file mode 120000 index 00000000..ac12754e --- /dev/null +++ b/.agents/skills/taken/SKILL.md @@ -0,0 +1 @@ +../../../../../../taken/wkts/federated_project_stores/.agents/skills/taken/SKILL.md \ No newline at end of file diff --git a/.agents/skills/taken/agents/openai.yaml b/.agents/skills/taken/agents/openai.yaml new file mode 120000 index 00000000..87522fbe --- /dev/null +++ b/.agents/skills/taken/agents/openai.yaml @@ -0,0 +1 @@ +../../../../../../../taken/wkts/federated_project_stores/.agents/skills/taken/agents/openai.yaml \ No newline at end of file diff --git a/.agents/skills/taken/references/agents.md b/.agents/skills/taken/references/agents.md new file mode 120000 index 00000000..e3257375 --- /dev/null +++ b/.agents/skills/taken/references/agents.md @@ -0,0 +1 @@ +../../../../../../../taken/wkts/federated_project_stores/.agents/skills/taken/references/agents.md \ No newline at end of file diff --git a/.agents/skills/taken/references/writes.md b/.agents/skills/taken/references/writes.md new file mode 120000 index 00000000..d5d973cf --- /dev/null +++ b/.agents/skills/taken/references/writes.md @@ -0,0 +1 @@ +../../../../../../../taken/wkts/federated_project_stores/.agents/skills/taken/references/writes.md \ No newline at end of file diff --git a/.ai/workflow-state.json b/.ai/workflow-state.json new file mode 100644 index 00000000..6ed6c082 --- /dev/null +++ b/.ai/workflow-state.json @@ -0,0 +1 @@ +{"version": 1, "backend": "legacy"} diff --git a/.gitignore b/.gitignore index 83fb5194..724908d2 100644 --- a/.gitignore +++ b/.gitignore @@ -306,3 +306,19 @@ claude_wkts # BEGIN ai.skillz: direct:symlink:opencode:command:taken-export /.opencode/commands/taken-export.md # END ai.skillz: direct:symlink:opencode:command:taken-export + +# Local worktrees and taken runtime artifacts +/.ai/state/ +.*.taken.lock + +# Version the federated project store despite local excludes +!/.taken/ +!/.taken/current.org +!/.taken/sources.toml +!/.taken/README.md + +# BEGIN ai.skillz: workflow-state +/.ai/state/ +/.ai/commit-msg/conf.toml +/.ai/pr-msg/conf.toml +# END ai.skillz: workflow-state diff --git a/.taken/README.md b/.taken/README.md new file mode 100644 index 00000000..78b4ecae --- /dev/null +++ b/.taken/README.md @@ -0,0 +1,22 @@ +# piker project store + +`current.org` contains the intact piker scope moved from the +LNS working corpus. Task/checkbox states, carry metadata and prose +remain human-owned. `sources.toml` selects this branch's store. + +Read the local `taken` skill through `.agents/skills/taken/SKILL.md`. +It requires a complete-file revision guard for each authorized save. +Use an explicitly selected source; do not write a global LNS copy. +This migration preserves a historical project scope, not a claim that +its old task descriptions or branch paths reflect current reality. + +The global storespace is registered in the LNS migration worktree's +`taken/sources.toml`. Modden, Tractor and Piker include snapshots of their pre-existing +local stores. The LNS consolidation ledger pins those inputs. Before +landing, reconcile any newer edits there and retarget active agents; +do not overwrite another agent's source or infer acceptance. + +Before removing this migration worktree after landing, update the +global registration and the cross-repository taken skill link to the +landed locations. The skill link intentionally points to the Taken +migration branch until that code and documentation land. diff --git a/.taken/current.org b/.taken/current.org new file mode 100644 index 00000000..04ee4a12 --- /dev/null +++ b/.taken/current.org @@ -0,0 +1,290 @@ +# vim: et sw=2 ts=2 +#+TITLE: piker project store +#+TODO: TODO(t) WIP(w!) WAIT(a@) | DONE(d!) NOPE(n@) + +# Checkbox states: [ ] open, [-] partial, [x] complete + +* piker +:PROPERTIES: +:CATEGORY: piker +:END: +** TODO prs +:PROPERTIES: +:CREATED: [2026-07-15] +:CARRIED: 29 +:END: +- [ ] move the below into *** todos with pr/branch links + * [ ] DPI-aware font scaling + manual kb bindings. + * [ ] gap_annotator for time gaps. + * [ ] moar `claude`-y skills, + * [ ] /lint with ruff + * [ ] /run-tests +*** WIP qualify known-FQME history backfill +:PROPERTIES: +:CREATED: [2026-07-24] +:CARRIED: 21 +:REPO: /home/goodboy/repos/piker +:END: +- [x] preserve the dirty `flake_update` main checkout before + switching branches +- [x] check out `backfiller_deep_fixes` in the main checkout and + verify every process imports it +- [ ] complete the below qualification plan + * [ ] select one known-problem provider, FQME, timeframe, date range, + and original symptom + * [ ] preserve and checksum any known-bad parquet + * [ ] bootstrap `pikerd` with disposable config and NativeDB storage + * [ ] record baseline timestamps, rows, uniqueness, nulls, + ordering, + indexes, gaps, and readability + * [ ] reproduce fresh-backfill, restart, and append behavior for + only + the selected case + * [ ] convert each reproduced failure into a failing deterministic + test before fixing it + * [ ] rerun the exact manual lifecycle after every production + correction + * [ ] add deterministic subprocess coverage for process death and + cross-process locking + * [ ] exercise Ctrl-C, SIGTERM, SIGKILL, provider interruption, and + restart recovery + * [ ] qualify MarketStore and actual sampler backpressure separately + * [ ] expand to other timeframes and providers only after the + selected FQME passes +**** TODO fixed-FQME-case backfill qualify +:PROPERTIES: +:CREATED: [2026-07-29] +:CARRIED: 17 +:REPO: piker/ +:END: +- [ ] follow through using both new files, + * ./docs/manual_backfill_qualification.rst + * ./snippets/nativedb_backfill_audit.xsh +- [ ] choose and record one known-problem FQME, timeframe, UTC range, + and original chart symptom +- [ ] source `snippets/nativedb_backfill_audit.xsh` and initialize a + private disposable root with only the required provider config +- [ ] run repository and environment preflight at commit `0846cbd4` +- [ ] seed and audit the known-bad baseline, then replay repair and run + `bfq_compare()` without losing original timestamps +- [ ] clear only the disposable Parquet and qualify a fresh backfill +- [ ] restart against fresh storage and compare `fresh` to `restart` +- [ ] leave one subsequent run active long enough to append, then compare + `restart` to `append` with `require_newer=True` +- [ ] restart QQQ 1s on fresh datad SHM and verify the former 653-row + zero-time reservation does not recur +- [ ] preserve chart logs, audit JSON, Parquet snapshots, and exact UTC + gap endpoints for review +**** TODO review ^qualify^ evidence and close follow-ups +:PROPERTIES: +:CREATED: [2026-07-29] +:CARRIED: 17 +:REPO: /piker +:END: +- [ ] verify every qualification process imports commit `0846cbd4` +- [ ] review baseline, repair, fresh, restart, and append reports for + structural, timestamp, endpoint, and gap regressions +- [ ] distinguish durable Parquet gaps from SHM hydration or chart-only + gaps using the runbook triage table +- [ ] convert each reproduced failure into a deterministic regression + before changing production code +- [ ] prepare the final qualification summary and any required patch, + PR, or worktree-lifecycle follow-ups +*** TODO cross-harness AI skill integration +:PROPERTIES: +:CREATED: [2026-07-21] +:CARRIED: 24 +:REPO: /home/goodboy/repos/piker +:ISSUE: https://www.pikers.dev/pikers/piker/issues/79 +:END: +- [ ] review and land the portable piker skill and OpenCode docs patch +- [ ] reconcile the active global `commit-msg` override with the + repo-local shared skill and style guide +- [ ] verify skill and command discovery after restarting Claude Code + and OpenCode +- [ ] fold remaining skill-content review into the canonical trackers: + * [ ] https://www.pikers.dev/pikers/piker/issues/69 + * [ ] https://www.pikers.dev/pikers/piker/issues/79 +*** TODO vectorize `gap_annotator` client spec building +:PROPERTIES: +:CREATED: [2026-07-21] +:CARRIED: 24 +:REPO: /home/goodboy/repos/piker +:END: +- [ ] replace per-row Polars slicing and filtering in + `piker/tsp/_annotate.py:139` with extracted columns and batched + lookups +- [ ] preserve missing-prior-row, epoch-skip, direction, label, and + batch-payload behavior +- [ ] benchmark the 1285-gap case against the handoff in + `notes_to_self/claudy/gapsannotator_sesh.md` +- [ ] add focused correctness coverage for generated rectangle, arrow, + and text specs +*** TODO integrate the piker `ruff` lint skill +:PROPERTIES: +:CREATED: [2026-07-21] +:CARRIED: 24 +:REPO: /home/goodboy/repos/piker +:END: +- [ ] review the clean but unmerged `claudy_writes_ruff_conf` branch +- [ ] validate its `ruff.toml` policy and staged-file checker against + the current tree +- [ ] port the `/lint` skill to shared Agent Skills frontmatter with + thin harness-specific commands +- [ ] rebase and prepare the accepted lint/config changes for landing +*** TODO reconcile stale Claude plans and worktrees +:PROPERTIES: +:CREATED: [2026-07-21] +:CARRIED: 24 +:REPO: /home/goodboy/repos/piker +:END: +- [ ] reconcile `ai/claude-code/plans/shm_port.md` with the implemented + `tractor.ipc._shm` shim +- [ ] classify `datad_service.md` as an architecture record rather than + an active implementation plan +- [ ] inspect generated state before retiring the merged + `wkt/flake_update` and `worktree-ngi0_submission` worktrees +- [ ] preserve the unmerged lint worktree until its branch is reviewed +** TODO chart-local gap overlay follow-ups +:PROPERTIES: +:CREATED: [2026-08-14] +:CARRIED: 7 +:REPO: /home/goodboy/repos/piker +:END: +- [ ] trigger `GapOverlayMngr.refresh()` from an explicit history + revision/publication event after repair or prepend +- [ ] expose gap-overlay UX through UI config + * [ ] startup visibility and enabled timeframes + * [ ] default `Ctrl+G` key binding +- [ ] live-qualify chart-local overlay behavior + * [ ] actor startup and focused 1s/60s toggling + * [ ] multi-symbol and duplicate-FQME cached charts + * [ ] secondary-`Viz` placement and backfill repositioning + * [ ] remote disconnect and owner-scoped graphics cleanup +- [ ] add focused Qt/actor regressions for the qualified lifecycle + +** TODO factor typed chart annotation IPC dialogs +:PROPERTIES: +:CREATED: [2026-08-14] +:CARRIED: 7 +:REPO: /home/goodboy/repos/piker +:END: +- [ ] exercise the strict `SetGapOverlay`/`GapOverlay` + dialog under a live Tractor context + * [ ] wrong-direction and unrelated payload rejection + * [ ] timeout, cancellation, stale reply, and disconnect recovery +- [ ] factor keyed send/receive locking, fresh request IDs, and stale + response correlation into a generic IPC dialog helper after a + second typed consumer establishes the interface +- [ ] migrate legacy `remote_annotate()` dict commands to tagged + request/response message types +- [ ] decide whether to fold `remote_gap_overlays()` into the general + annotation endpoint after the legacy protocol is typed +** proj_mgmt + :PROPERTIES: + :CATEGORY: project + :END: +*** TODO ngi0 aug01 funding submit! + DEADLINE: <2026-08-01 Sat> + :PROPERTIES: + :CREATED: [2026-06-01] +:CARRIED: 33 + :END: +- [ ] wip pr: https://github.com/goodboy/tractor/pull/431 + https://nlnet.nl/propose/ + https://nlnet.nl/foundation/policies/generativeAI/ + https://nlnet.nl/commonsfund/ +*** TODO MEETING(s) + :PROPERTIES: + :CREATED: [2026-05-21] +:CARRIED: 33 + :END: +** ui + :PROPERTIES: + :CATEGORY: piker.ui + :END: +*** TODO dpi_auto_calc + :PROPERTIES: + :CREATED: [2026-07-15] +:CARRIED: 29 + :END: +- [ ] PUT PRs here +** brokers + :PROPERTIES: + :CATEGORY: piker.brokers + :END: +*** TODO IB: follow-up for search results bs! + :PROPERTIES: + :CREATED: [2026-05-21] +:CARRIED: 33 + :END: + * https://www.pikers.dev/pikers/piker/issues/85 + - [X] fix the `si.expiry..ib` search result? + - [ ] introspect why `Client.search_symbols()` is returning the + empty `.` case! + - presumably by something weird in the server-side endpoint? + - [ ] figure out better tuning of client side fuzzy-filter versus + server-side versus UI final display pass! +*** TODO ish for port to `uv`, + :PROPERTIES: + :CREATED: [2026-05-21] +:CARRIED: 33 + :END: + * https://github.com/ib-api-reloaded/ib_async/issues/204 + - [ ] mention to track in #68 +*** TODO profiling: try newer libs, + :PROPERTIES: + :CREATED: [2026-05-21] +:CARRIED: 33 + :END: + - [ ] https://github.com/pikers/piker/issues/337 +** perf/tooling + :PROPERTIES: + :CATEGORY: piker.profile + :END: +- [ ] link existing GH issues for trying new + libs! + +** FSP backfill: branch work +*** TODO review and commit FSP backfill synchronization +:PROPERTIES: +:ID: piker-fsp-backfill-sync-review-and-commit +:CREATED: [2026-09-03] +:CARRIED: 0 +:REPO: /home/goodboy/repos/piker/wkts/fsp_backfill_sync +:END: +- [ ] review committed token-targeted history graphics change + (`fadab3d2`) +- [ ] review and commit the FSP stream-contract boundary + * yo yo yo + * `piker/data/__init__.py` + * `piker/data/ticktools.py` + * `piker/fsp/_api.py` + * `piker/fsp/_momo.py` + * staged portion of `piker/fsp/_volume.py` +- [ ] review and commit the synchronization and live-volume boundary + * explicit `Flume` RT/history cadence + * immutable history snapshots and absolute SHM-bound synchronization + * relevant backfill filtering and non-duplicating destination steps + * nonnegative sampled and derived volume + * M4 graphics-cache invalidation and unchanged-update suppression + * FSP synchronization regressions and Prompt-IO provenance +- [ ] review and commit the `piker-fsp-expert` skill boundary +- [ ] run boundary checks and the final test suite +- [ ] inspect each staged diff and commit message before committing +*** TODO add cumulative M4 downsampling for volume curves +:PROPERTIES: +:ID: piker-cumulative-m4-volume-downsampling +:CREATED: [2026-09-03] +:CARRIED: 0 +:REPO: /home/goodboy/repos/piker/wkts/fsp_backfill_sync +:END: +- [ ] define cumulative bucket semantics for step-style data + * sum all source values represented by each pixel bucket + * preserve bucket time/index placement and zero-volume gaps +- [ ] add a sum-reducer variant alongside price min/max M4 +- [ ] route unit and dollar-volume curves through cumulative downsampling +- [ ] keep price, OHLC, and non-cumulative signal curves on min/max M4 +- [ ] preserve incremental appends, view changes, and cache invalidation +- [ ] test sum conservation across zoom levels and partial edge buckets +- [ ] profile the cumulative path against the existing M4 implementation diff --git a/.taken/sources.toml b/.taken/sources.toml new file mode 100644 index 00000000..5a36e390 --- /dev/null +++ b/.taken/sources.toml @@ -0,0 +1,8 @@ +version = 1 + +[[sources]] +id = "piker" +path = "current.org" +repo = ".." +enabled = true +agenda = true diff --git a/ai/prompt-io/20261003T001837Z_9115d24_project_stores.md b/ai/prompt-io/20261003T001837Z_9115d24_project_stores.md new file mode 100644 index 00000000..51aef9f8 --- /dev/null +++ b/ai/prompt-io/20261003T001837Z_9115d24_project_stores.md @@ -0,0 +1,34 @@ +--- +created_at: 2026-10-03T00:18:37.208688+00:00 +generated_by: + harness: codex + provider: openai + model: gpt-6 +git_ref: 9115d24 +scope: data +substantive: true +raw_file: 20261003T001837Z_9115d24_project_stores.raw.md +--- + +## Prompt + +Split the LNS working corpus into per-repository federated stores on +open-wkt branches and connect them to one global storespace. Preserve +human task states. Add a local agent workflow, update taken-export, +and design rollover, supervised concurrency, activity and ranked views. + +## Response summary + +Create the repository's migration branch and preserve its imported +scope, with revision-aware local source configuration and a shared +local taken skill. Keep original checkout stores and indexes intact. +Existing live branch stores remain independently registered. The LNS +manifest and migration ledger describe the authoritative split paths. + +## Human edits + +The human chose the per-repo split and worktree delivery, authorized +local auto-mode, and required supervisor-based locking: the supervisor +owns write authority and can revoke delegated human/subagent edit +grants through cancellation. The agent applied these directions. +No task/checkbox acceptance changes or Git commits were authorized. diff --git a/ai/prompt-io/20261003T001837Z_9115d24_project_stores.raw.md b/ai/prompt-io/20261003T001837Z_9115d24_project_stores.raw.md new file mode 100644 index 00000000..f72ea8e2 --- /dev/null +++ b/ai/prompt-io/20261003T001837Z_9115d24_project_stores.raw.md @@ -0,0 +1,36 @@ +--- +created_at: 2026-10-03T00:18:37.208688+00:00 +generated_by: + harness: codex + provider: openai + model: gpt-6 +git_ref: 9115d24 +scope: data +substantive: true +raw_file: 20261003T001837Z_9115d24_project_stores.raw.md +--- + +## Prompt excerpts + +“break it up into appropriate per repo federated store files” +“intro the sub-store updates each on /open-wkt branches” +“a concise entrypoint to view all sub-project repo's federated stores” +“supervisor based locking” +“an edit capability can only be acquired from the supervisor” +“a human editor can be cancelled from their write access” + +## Generated output (diff reference) + +> `git diff HEAD~1..HEAD -- .taken/current.org` + +Create the repository's migration branch and preserve its imported +scope, with revision-aware local source configuration and a shared +local taken skill. Keep original checkout stores and indexes intact. +Existing live branch stores remain independently registered. The LNS +manifest and migration ledger describe the authoritative split paths. + +## Explanation + +The global launch now works: one Neovim window shows all 13 sources, +with normal highlighting and the new help menu. The full AI export +reports complete coverage with no diagnostics.