piker/ai/prompt-io/codex/20260924T185456Z_1da9b760_s...

92 lines
3.7 KiB
Markdown
Raw Normal View History

Use IB schedules to reserve missing SHM history Interrupted backfill can leave stored observations separated by missing trading time. Query dated IB sessions before prepending stored history and reserve synthetic samples only during covered open sessions. Keep venue closures compressed and preserve the indexes of already published rows. Deats, - Add `Client.history_schedule()` with UTC coverage, extended hours, bounded duration, timeout, and rejection of ambiguous DST times. - Keep schedule normalization in `venues` and allocation in `.tsp._reserve`, with the project AGPL headers. - Expose schedule and synthetic-range queries through `datad.ib`. - Bound reservations by available SHM capacity and inspect only the newest 32 gaps by default via `newest_gaps`. Zero disables queries; leave unknown and oversized ranges unresolved. - Record synthetic ranges separately from OHLCV. Automatic storage writes retain provider observations; placeholders copy close. - Group the new tests under `tests/providers`. - Cover request formatting, DST, failure and capacity cases, plus real SHM/NativeDB startup placement with a fake provider. - Document a common CLI/chart segment-repair interface and defer coordinated insertion, repair execution, and reader integration. Validation: 55 targeted tests passed in the working tree; live IB schedule coverage remains unqualified. After the module moves, 33 affected tests passed again; the local review follow-up passed 18 affected cases, including four new request-budget cases. The commit runner repeats lint and isolated regression checks against this exact boundary. Prompt-IO: ai/prompt-io/codex/20260924T185456Z_1da9b760_schedule_prompt_io.md (this patch was generated in some part by `codex` using `gpt-6` (`openai`))
2026-09-24 23:09:22 +00:00
---
model: gpt-6 (exact variant not exposed)
service: codex
timestamp: 2026-09-24T18:54:56.893490+00:00
git_ref: 1da9b760
scope: code
substantive: true
raw_file: 20260924T185456Z_1da9b760_schedule_prompt_io.raw.md
---
## Prompt
The human requested: first freeze a commit-plan for the existing
close-copy/null-repair changes, then implement ib_async historical
schedule support with tests, use it for datad.ib history-buffer gap
spacing, and plan a common on-demand segment interface for CLI and
chart selection. Avoid reindexing in this first pass and record
fallback insertion only as a TODO.
## Response summary
Implemented historical schedule normalization, bounded startup
reservations, explicit actor-local synthetic ranges, and two read
RPC functions. Added deterministic adapter tests and real SHM/parquet
startup coverage. Wrote the segment-backfill interface plan.
This entry covers the new schedule work only. The earlier frozen
null-repair plan has its own provenance. The shared _history.py file
contains changes from both scopes until the earlier commit is made.
## Human edits
The human identified cancellation during backfill as the reproducible
trigger, explained the packed-index insertion tradeoff, requested
reuse of venue closure knowledge, and chose placeholders to avoid
expensive rewrites. They required explicit placeholder annotation,
proposed later graphics compression, and asked to defer insertion
fallbacks. They also requested the existing null-repair commit plan
before the new schedule work. These are human-directed architecture
and scope choices; no acceptance or task completion state is inferred.
## Validation and limits
55 targeted tests passed; Ruff and diff checks passed. No live IB
qualification was performed. Reservations are synthetic, not recovered
observations. Repair execution, persistence of unfinished requests,
chart compression, FSP handling, export filtering, and request-level
IB cancellation remain follow-up work documented in the plan.
## Files changed
- `piker/brokers/ib/venues.py`
- `piker/brokers/ib/api.py`
- `piker/brokers/ib/feed.py`
- `piker/tsp/_reserve.py`
- `piker/tsp/_history.py`
- `tests/providers/test_ib_schedule.py`
- `tests/providers/test_history_reservations.py`
- `docs/history_segment_backfill.rst`
## Pre-commit module review
The human requested placing historical schedule normalization in the
existing venues.py module, naming the allocation module _reserve.py,
and adding the missing project license headers. Applied those moves,
updated imports and test patch targets, and added the standard AGPL
headers to the new allocation module and both new test modules.
## Local tuicr review follow-up
The human supplied eleven comments requesting a configurable newest
gap count, caller and metadata references, explanations of guards
and provider-specific limits, multiline boolean style, capacity
fallback rationale, an explanation of numpy split boundaries, and
clearer test cases, fixture types, and module docstrings.
Added newest_gaps=32, with zero disabling queries and negative values
rejected. Documented tsdb_backfill ownership and the local/RPC range
accessors. Explained recoverable overflow and half-open synthetic
range grouping. Typed tmp_path as pathlib.Path and monkeypatch as
pytest.MonkeyPatch. Added four request-budget cases.
Verification: reservation tests 11 passed; schedule tests 7 passed;
Ruff and git diff --check passed. Changes remain unstaged for review.
No forge review or CI state was queried; this was local feedback.
## Provider-test layout review
The human requested grouping both new history/schedule test modules
under tests/providers. Moved both files without changing test logic;
they retain the shared tests/conftest.py fixtures.