Commit Graph

9 Commits (bcca507462af3ada3adfcbde6c4e8dd5d9f58909)

Author SHA1 Message Date
Gud Boi 430a7591a9 Factor and bound synthetic SHM null repair
Extract `_copy_close_to_null_rows()` with explicit mutation
bounds, searchable close-copy comments, and preserved activity
fields. Type and document `maybe_fill_null_segments()` including
its distinction from packed timestamp-gap repair.

Disable relative epoch tolerance when validating synthetic
stamps: an overfull reservation must not duplicate the right
boundary. Cover modern epochs, typed empty replies, packed gaps,
and unchanged rejected inputs.

Prompt-IO: ai/prompt-io/codex/20260924T165757Z_1da9b760_prompt_io.md

(this patch was generated in some part by `codex` using
`gpt-6` (`openai`))
2026-11-13 17:29:30 -05:00
Gud Boi f106caf59c Preserve IB history request bounds on no data
Complete HMDS no-data replies without silently retrying a day
older. Let `open_history_client()` classify the original request
and leave frame stepping to `start_backfill()`.

Distinguish `request_end_dt` from `oldest_shm_dt` so the first
frame after an empty interval retains its unwritten end bar.
Inline the storage time cutoff and drop the unused two-direction
`diff_history()` helper. Explain both dates and filtering rules.

Add request-boundary tests and real SHM/NativeDB regressions:
all 2,004 supplied bars survive, while the closure stays empty.

Prompt-IO: ai/prompt-io/codex/20260923T164656Z_efa72980_prompt_io.md

(this patch was generated in some part by `codex` using
`gpt-6` (`openai`))
2026-11-13 17:29:28 -05:00
Gud Boi e3f2e9ce2a Fix sparse IB history prepends in `ShmArray`
IB returns trading samples, not one row per wall-clock period.
Converting elapsed time to a physical prepend offset left 42,720
zero rows in NVDA's 60s buffer even though both valid margins
shared the same timestamp.

Use the actual earliest provider sample as the NativeDB boundary.
After reverse retrieval, prepend older stored rows directly beside
it.

Deats,
- start reverse insertion at the current first index;
- update `ShmArray._first` after every provider prepend;
- trim provider/NativeDB overlap by its timestamp;
- publish the final storage prepend to chart and FSP consumers;
- derive the source `time` field from `ohlc_key_map`;
- cover the full lifecycle with a real multi-frame `ShmArray`.

Prompt-IO: ai/prompt-io/opencode/20260805T195001Z_f0f3c806_prompt_io.md

(this patch was generated in some part by `opencode` using `gpt-5.6-sol` (`openai`))
2026-11-13 15:47:52 -05:00
Gud Boi 0d2c88ce3e Repair IB startup gaps in `ShmArray`
Keep reverse history and stored SHM layout aligned so sparse IB
frames do not expose permanent zero-time reservations.

Deats,
- omit inclusive query endpoints from SHM but retain full storage
- serialize and bound farm resets, blanks, and cancellations
- run the final null sweep after reverse retrieval completes
- require valid timestamp boundaries for synthetic gap rows
- cover the live QQQ 653-row failure and reset interleavings

Prompt-IO: ai/prompt-io/opencode/20260729T204046Z_ad6c560f_prompt_io.md

(this patch was generated in some part by `opencode` using `gpt-5.6-sol` (`openai`))
2026-11-11 18:35:47 -05:00
Gud Boi 8a4a1f67d8 Defer SHM null repair without valid time bounds
Require positive, increasing timestamp bounds before querying a SHM
gap or applying synthetic fallback. Defer unbounded edge runs
without entering the debugger or rewriting their rows, and continue
checking later segments.

Cover leading, trailing and entirely null frames with provider,
debugger and notification calls forbidden; assert the complete
frame stays unchanged.

(this patch was generated in some part by `Codex` using
`GPT-6` (`OpenAI`))
2026-11-11 11:18:11 -05:00
Gud Boi 1b76dd64fd Normalize provider OHLCV for `NativeDB`
IB history frames omit the derived `index` and append `count`.
Normalize provider data to durable names, order, and dtypes before
merge, and reject fractional epoch truncation.

Also,
- map canonical storage fields into provider-specific SHM on reload
- select Polars columns by name instead of position
- exercise first-start and restart through actor, SHM, and Parquet

Prompt-IO: ai/prompt-io/opencode/20260727T212454Z_689df816_prompt_io.md

(this patch was generated in some part by `opencode` using `gpt-5.6-sol` (`openai`))
2026-11-10 16:02:36 -05:00
Gud Boi 9cfe2591d9 Make `NativeDB` backfills atomic and non-destructive
- merge provider deltas without dropping unaffected history
- validate reopened temp parquet before atomic replacement
- serialize writers with actor-local and cancellable file locks
- publish cache only after disk replacement
- persist full startup/backfill frames before SHM publication
- bound sampler notifications and avoid MarketStore duplicates

Prompt-IO: ai/prompt-io/opencode/20260722T024134Z_3a4f6737_prompt_io.md

(this patch was generated in some part by `opencode` using `gpt-5.6-sol` (`openai`))
2026-11-10 16:00:32 -05:00
Gud Boi 51795a4e07 Contain provider failures during SHM gap repair
Partially backport only the `get_hist()` exception catch from
0846cbd4a4, authored by goodboy
<goodboy_foss@protonmail.com>. Keep the existing gap fallback.

A timed-out IB query raises `NoData` from null-segment repair.
Letting it escape cancels sibling history and live quote tasks
while `pikerd` can remain alive. Catch `NoData` and
`DataUnavailable` locally and stop querying this repair pass.

Add a regression using real null detection and a waiting sibling
in the same nursery. Both provider failures must return without
canceling the sibling. The two cases failed before this catch;
the focused history files passed 17 tests with it.

Partial-backport-of: 0846cbd4a4
Prompt-IO: ai/prompt-io/20261106T185628Z_495c0915_215e9b_prompt_io.md

(this patch was generated in some part by `codex` (`openai`))
2026-11-06 15:20:58 -05:00
Gud Boi 2a826fba17 `.tsp._history`: drop orphaned backfill events
- let structured nursery joins own child-task completion
- stop empty provider frames before timestamp endpoint indexing
- cover provider exhaustion and empty frames with bounded tests

Prompt-IO: ai/prompt-io/opencode/20260722T001358Z_16f3cd6f_prompt_io.md

(this patch was generated in some part by `opencode` using `gpt-5.6-sol` (`openai`))

(cherry picked from commit 931f22d235)
2026-11-06 13:41:07 -05:00