Commit Graph

6 Commits (6ca6ab2a102f2ddbdf669c17b8877f3fdce1e1bd)

Author SHA1 Message Date
Gud Boi 74ea2152f9 Expose storage state through `piker store`
Make bare storage groups and eps render help, so the CLI is
discoverable without memorizing `Typer` option conventions. Add
read-only `series` and `shm` eps for durable `NativeDB` series and
live history buffers, with `Rich`-table or `JSON` output.

Deats,
- preserve exact `(fqme, period)` identities for `NativeDB`
  series;
- parse exact `(generation, fqme, kind)` names for SHM buffers;
- reconstruct provider dtypes when attaching to live buffers;
- tolerate buffers disappearing between discovery and attach.

Also,
- report expected and observed cadence separately, plus invalid
  rows, ordering defects and ranked gaps;
- add exact `ldshm --shm-name` selection and keep disabled write
  and reload paths from reading undefined state or entering `pdb`;
- document the triage layers and cover help, identity parsing,
  read-only diagnostics, dtype selection and `ldshm` flow.

Prompt-IO: ai/prompt-io/opencode/20260731T165215Z_0846cbd4_prompt_io.md

(this patch was generated in some part by `opencode` using `gpt-5.6-sol` (`openai`))
2026-07-31 17:26:11 -04:00
Gud Boi d359b1cbfb Repair first-slot OHLC sentinels in `NativeDB`
Filter known zero-initialized epoch slots before persisted history
hydrates SHM or merges with a fresh provider frame.

Deats,
- reindex the repaired read view for same-session chart publication
- treat sentinel-only files as empty history for fresh backfill
- reject new first-slot sentinels but preserve modern zero-price bars
- classify raw sentinel evidence separately in `store audit`

Prompt-IO: ai/prompt-io/opencode/20260729T041723Z_ce33deb6_prompt_io.md

(this patch was generated in some part by `opencode` using `gpt-5.6-sol` (`openai`))
2026-07-29 13:30:51 -04:00
Gud Boi 1f30917047 Repair legacy timestamps in `NativeDB` merges
Keep incoming provider frames strict while dropping non-positive rows
from persisted history before merge validation.

This lets known-bad MNQ history repair itself on the next valid IB
frame without losing valid old bars or publishing noncanonical indexes.

Prompt-IO: ai/prompt-io/opencode/20260728T232718Z_ed85721c_prompt_io.md

(this patch was generated in some part by `opencode` using `gpt-5.6-sol` (`openai`))
2026-07-28 19:37:15 -04:00
Gud Boi 19c6d98e6d Normalize provider OHLCV for `NativeDB`
IB history frames omit the derived `index` and append `count`.
Normalize provider data to durable names, order, and dtypes before
merge, and reject fractional epoch truncation.

Also,
- map canonical storage fields into provider-specific SHM on reload
- select Polars columns by name instead of position
- exercise first-start and restart through actor, SHM, and Parquet

Prompt-IO: ai/prompt-io/opencode/20260727T212454Z_689df816_prompt_io.md

(this patch was generated in some part by `opencode` using `gpt-5.6-sol` (`openai`))
2026-07-27 18:08:33 -04:00
Gud Boi 689df816d6 Drop redundant `NativeDB` writer locks
`datad`'s persistent feed task already owns each series writer.
The actor-local and file locks duplicated this SC contract and left
`.parquet.lock` sidecars that upstream indexed as data.

- preserve validated temp writes, `fsync()`, and atomic replacement
- keep exact-suffix indexing for legacy lock and temp sidecars
- prove full and incremental writes create no lock artifacts

Prompt-IO: ai/prompt-io/opencode/20260727T204711Z_84a6d47b_prompt_io.md

(this patch was generated in some part by `opencode` using `gpt-5.6-sol` (`openai`))
2026-07-27 16:58:00 -04:00
Gud Boi b000db94e0 Make `NativeDB` backfills atomic and non-destructive
- merge provider deltas without dropping unaffected history
- validate reopened temp parquet before atomic replacement
- serialize writers with actor-local and cancellable file locks
- publish cache only after disk replacement
- persist full startup/backfill frames before SHM publication
- bound sampler notifications and avoid MarketStore duplicates

Prompt-IO: ai/prompt-io/opencode/20260722T024134Z_3a4f6737_prompt_io.md

(this patch was generated in some part by `opencode` using `gpt-5.6-sol` (`openai`))
2026-07-24 18:33:06 -04:00