pikers
/
piker
4
1
Fork 0
Code Issues 20 Pull Requests 13 Packages Projects Releases Wiki Activity
piker/.taken/current.org

16 KiB
Raw Blame History

piker project store

piker

TODO prs

  • move the below into * todos with pr/branch links

    • DPI-aware font scaling + manual kb bindings.
    • gap_annotator for time gaps.
    • moar `claude`-y skills,
    • /lint with ruff
    • /run-tests

WIP qualify known-FQME history backfill

  • [x] preserve the dirty `flake_update` main checkout before switching branches
  • [x] check out `backfiller_deep_fixes` in the main checkout and verify every process imports it

  • complete the below qualification plan

    • select one known-problem provider, FQME, timeframe, date range, and original symptom
    • preserve and checksum any known-bad parquet
    • bootstrap `pikerd` with disposable config and NativeDB storage
    • record baseline timestamps, rows, uniqueness, nulls, ordering,

    indexes, gaps, and readability

    • reproduce fresh-backfill, restart, and append behavior for only

    the selected case

    • convert each reproduced failure into a failing deterministic test before fixing it
    • rerun the exact manual lifecycle after every production correction
    • add deterministic subprocess coverage for process death and cross-process locking
    • exercise Ctrl-C, SIGTERM, SIGKILL, provider interruption, and restart recovery
    • qualify MarketStore and actual sampler backpressure separately
    • expand to other timeframes and providers only after the selected FQME passes
TODO fixed-FQME-case backfill qualify

  • follow through using both new files,

    • ./docs/manual_backfill_qualification.rst
    • ./snippets/nativedb_backfill_audit.xsh
  • choose and record one known-problem FQME, timeframe, UTC range, and original chart symptom
  • source `snippets/nativedb_backfill_audit.xsh` and initialize a private disposable root with only the required provider config
  • run repository and environment preflight at commit `0846cbd4`
  • seed and audit the known-bad baseline, then replay repair and run `bfq_compare()` without losing original timestamps
  • clear only the disposable Parquet and qualify a fresh backfill
  • restart against fresh storage and compare `fresh` to `restart`
  • leave one subsequent run active long enough to append, then compare `restart` to `append` with `require_newer=True`
  • restart QQQ 1s on fresh datad SHM and verify the former 653-row zero-time reservation does not recur
  • preserve chart logs, audit JSON, Parquet snapshots, and exact UTC gap endpoints for review
TODO review ^qualify^ evidence and close follow-ups
  • verify every qualification process imports commit `0846cbd4`
  • review baseline, repair, fresh, restart, and append reports for structural, timestamp, endpoint, and gap regressions
  • distinguish durable Parquet gaps from SHM hydration or chart-only gaps using the runbook triage table
  • convert each reproduced failure into a deterministic regression before changing production code
  • prepare the final qualification summary and any required patch, PR, or worktree-lifecycle follow-ups

TODO cross-harness AI skill integration

TODO vectorize `gap_annotator` client spec building

  • replace per-row Polars slicing and filtering in `piker/tsp/_annotate.py:139` with extracted columns and batched lookups
  • preserve missing-prior-row, epoch-skip, direction, label, and batch-payload behavior
  • benchmark the 1285-gap case against the handoff in `notes_to_self/claudy/gapsannotator_sesh.md`
  • add focused correctness coverage for generated rectangle, arrow, and text specs

TODO integrate the piker `ruff` lint skill

  • review the clean but unmerged `claudy_writes_ruff_conf` branch
  • validate its `ruff.toml` policy and staged-file checker against the current tree
  • port the `/lint` skill to shared Agent Skills frontmatter with thin harness-specific commands
  • rebase and prepare the accepted lint/config changes for landing

TODO reconcile stale Claude plans and worktrees

  • reconcile `ai/claude-code/plans/shm_port.md` with the implemented `tractor.ipc._shm` shim
  • classify `datad_service.md` as an architecture record rather than an active implementation plan
  • inspect generated state before retiring the merged `wkt/flake_update` and `worktree-ngi0_submission` worktrees
  • preserve the unmerged lint worktree until its branch is reviewed

TODO chart-local gap overlay follow-ups

  • trigger `GapOverlayMngr.refresh()` from an explicit history revision/publication event after repair or prepend

  • expose gap-overlay UX through UI config

    • startup visibility and enabled timeframes
    • default `Ctrl+G` key binding

  • live-qualify chart-local overlay behavior

    • actor startup and focused 1s/60s toggling
    • multi-symbol and duplicate-FQME cached charts
    • secondary-`Viz` placement and backfill repositioning
    • remote disconnect and owner-scoped graphics cleanup
  • add focused Qt/actor regressions for the qualified lifecycle

TODO factor typed chart annotation IPC dialogs

  • exercise the strict `SetGapOverlay`/`GapOverlay` dialog under a live Tractor context

    • wrong-direction and unrelated payload rejection
    • timeout, cancellation, stale reply, and disconnect recovery
  • factor keyed send/receive locking, fresh request IDs, and stale response correlation into a generic IPC dialog helper after a second typed consumer establishes the interface
  • migrate legacy `remote_annotate()` dict commands to tagged request/response message types
  • decide whether to fold `remote_gap_overlays()` into the general annotation endpoint after the legacy protocol is typed

proj_mgmt

TODO ngi0 aug01 funding submit!

DEADLINE: <2026-08-01 Sat>

TODO MEETING(s)

ui

TODO dpi_auto_calc

  • PUT PRs here

brokers

TODO IB: follow-up for search results bs!

  • https://www.pikers.dev/pikers/piker/issues/85
  • fix the `si.expiry..ib` search result?

  • introspect why `Client.search_symbols()` is returning the empty `.<venue>` case!

    • presumably by something weird in the server-side endpoint?
  • figure out better tuning of client side fuzzy-filter versus server-side versus UI final display pass!

TODO ish for port to `uv`,

TODO profiling: try newer libs,

perf/tooling

  • link existing GH issues for trying new libs!

FSP backfill: branch work

TODO review and commit FSP backfill synchronization

  • review committed token-targeted history graphics change (`fadab3d2`)

  • review and commit the FSP stream-contract boundary

    • yo yo yo
    • `piker/data/__init__.py`
    • `piker/data/ticktools.py`
    • `piker/fsp/_api.py`
    • `piker/fsp/_momo.py`
    • staged portion of `piker/fsp/_volume.py`

  • review and commit the synchronization and live-volume boundary

    • explicit `Flume` RT/history cadence
    • immutable history snapshots and absolute SHM-bound synchronization
    • relevant backfill filtering and non-duplicating destination steps
    • nonnegative sampled and derived volume
    • M4 graphics-cache invalidation and unchanged-update suppression
    • FSP synchronization regressions and Prompt-IO provenance
  • review and commit the `piker-fsp-expert` skill boundary
  • run boundary checks and the final test suite
  • inspect each staged diff and commit message before committing

TODO add cumulative M4 downsampling for volume curves

  • define cumulative bucket semantics for step-style data

    • sum all source values represented by each pixel bucket
    • preserve bucket time/index placement and zero-volume gaps
  • add a sum-reducer variant alongside price min/max M4
  • route unit and dollar-volume curves through cumulative downsampling
  • keep price, OHLC, and non-cumulative signal curves on min/max M4
  • preserve incremental appends, view changes, and cache invalidation
  • test sum conservation across zoom levels and partial edge buckets
  • profile the cumulative path against the existing M4 implementation

TODO review and save the Trend Calculus prototype

Research and prototype guide

  • Regenerate and review six Trend Calculus behavior boundaries on trend_calc_on_jev after workflow commit f148fbcc on moar_ai_skillz lands; retain source/test pairs and inspect every message before saving.
  • Run each boundary check from the pinned package and the final safe regression sequence; the working-tree offline suite passed 253 tests.
  • Qualify one selected live source through startup, quiet bars, skipped wakeups, history repair, replay and owner cancellation.
  • Coordinate inherited stream/backfill work with the existing FSP synchronization task.

TODO qualify Jev regime judgments against real responses

Research and prototype guide

  • Once TYPESAFE_API_KEY is available and a live request is authorized, send one small synthetic multi-timeframe request through the HTTPX/Trio client.
  • Verify the documented request/response schema, model identity, distributions, usage, errors and measured end-to-end latency.
  • Evaluate useful regime labels and confidence on held-out trend snapshots, including flat, reversing, aligned and conflicting timeframes.
  • Measure sustained pacing, coalescing, source freshness and cost before enabling realtime judgments.
  • Consider an SDK AnyIO port or Trio backend only after the direct HTTP prototype demonstrates useful classifications.

TODO connect finalized trends to the Jev provider-service

Research and prototype guide

  • Obtain the separate system-1 provider-service repository or interface and identify its input, output, lifecycle and ownership contracts.
  • Adapt serve_regimes and RegimeClassifier to the service nursery and typed publication interface without mixing model labels into deterministic trend state.
  • Configure market, periods, model, credentials, request pacing, evidence retention and freshness limits explicitly.
  • Verify reconnect, revision/gap invalidation, backpressure, stale-response rejection and shutdown with the actual service.

TODO recover Pine source and check Trend Calculus parity

Research and prototype guide

  • Recover the human's Pine implementation and record its provenance/version alongside the pinned Lua and Scala research.
  • Compare startup seeding, extrema ties, FHLS splitting, opposing pivots and recursive reversal orders.
  • Add independent fixtures for meaningful semantic differences; distinguish extremum time from confirmation time and mutable-bar previews.

TODO qualify multitimeframe pivots and regime annotations

Research and prototype guide

  • Inspect actual 1/5/15-second layers and hover information on a selected live market; compare confirmed pivots and regime boundaries to finalized source evidence.
  • Verify chart-axis placement, gaps, revisions, hidden charts, duplicate cached feeds and owner-scoped cleanup.
  • Qualify model rectangles at response receipt and expiry, including stale classifications and source-segment changes.
  • Measure quiet-bar display delay before deciding whether publication needs an explicit output notification.
  • Coordinate shared history-overlay lifecycle with the existing chart-local gap overlay follow-ups; route any annotation IPC factoring to the existing typed-dialog task.

TODO qualify a native subsecond source for trend regimes

Research and prototype guide

  • Define the required high-frequency periods and producer contract; the numeric/MTF APIs accept fractional periods but the current native clock uses whole seconds.
  • Extend or select the source/sampler time grid before enabling subsecond chart/service periods; retain strictly increasing coordinates and complete aligned buckets.
  • Measure compute, SHM publication, service latency and chart refresh under the selected cadence, including gaps, replay and cancellation.