646 lines
30 KiB
ReStructuredText
646 lines
30 KiB
ReStructuredText
|
|
Trend Calculus: sources and Piker semantics
|
||
|
|
==========================================
|
||
|
|
|
||
|
|
This implementation follows Andrew J. Morgan's FHLS Trend Calculus.
|
||
|
|
It classifies observed price structure; it does not establish
|
||
|
|
predictive accuracy or a profitable trading rule. Research was
|
||
|
|
collected on 2026-10-04/05. The source inventory below records what
|
||
|
|
was found and distinguishes missing material from working code.
|
||
|
|
|
||
|
|
Algorithm
|
||
|
|
---------
|
||
|
|
|
||
|
|
Partition an ordered scalar series into disjoint windows containing
|
||
|
|
``n >= 2`` observations. Summarize each window by its high and low,
|
||
|
|
retaining each extremum's original coordinate. First and second in
|
||
|
|
FHLS mean the earlier and later of these two extrema. They do not
|
||
|
|
mean the opening and closing observation.
|
||
|
|
|
||
|
|
For consecutive windows ``previous`` and ``current``, define::
|
||
|
|
|
||
|
|
trend = sign(
|
||
|
|
sign(current.high - previous.high)
|
||
|
|
+ sign(current.low - previous.low)
|
||
|
|
)
|
||
|
|
|
||
|
|
The implementation uses ordinal comparisons to avoid overflowing
|
||
|
|
the differences. When the result is zero, introduce a synthetic
|
||
|
|
window joining the previous window's second extremum to the current
|
||
|
|
window's first extremum. Compare previous to synthetic, then
|
||
|
|
synthetic to current. Carry the prior direction through unresolved
|
||
|
|
zero comparisons. A complex window can confirm two opposite
|
||
|
|
reversals at the same observation; retaining only its final trend
|
||
|
|
would lose an event.
|
||
|
|
|
||
|
|
A confirmed down-to-up change selects the preceding summary's low;
|
||
|
|
an up-to-down change selects its high. Apply the same calculation to
|
||
|
|
the resulting ordered reversal points for the next reversal order.
|
||
|
|
The order counts successive reductions of the pivot stream. It is
|
||
|
|
independent of a wall-clock timeframe such as one or five seconds.
|
||
|
|
|
||
|
|
The live recurrence retains one partial window, one completed
|
||
|
|
window and the prior direction per order. Its numeric state is
|
||
|
|
``O(orders)`` regardless of history length or window size. Historical
|
||
|
|
scanning calls the same Numba recurrence. Returning a full historical
|
||
|
|
result necessarily allocates arrays proportional to history length
|
||
|
|
and order count.
|
||
|
|
|
||
|
|
A completed window emits at most two reversals. Because each configured
|
||
|
|
window contains at least two observations, two incoming reversals can
|
||
|
|
complete at most one next-order window during that source update.
|
||
|
|
Induction bounds every order to two events per source observation;
|
||
|
|
the fixed event slots therefore do not truncate a larger cascade.
|
||
|
|
|
||
|
|
Deliberate semantic choices
|
||
|
|
--------------------------
|
||
|
|
|
||
|
|
``piker.fsp._trend.TrendCalculator`` accepts scalar observations,
|
||
|
|
finite strictly increasing timestamps, and strictly increasing
|
||
|
|
nonnegative source indices. Coordinates need not be contiguous.
|
||
|
|
An OHLC candle alone cannot reveal whether its high or low happened
|
||
|
|
first; feeding fabricated high/low order would change the algorithm.
|
||
|
|
The chart FSP therefore uses close prices.
|
||
|
|
|
||
|
|
Production startup is neutral. The first completed window establishes
|
||
|
|
the baseline without inventing a pivot. ``ready`` becomes true after
|
||
|
|
two completed windows at that order, including when the direction
|
||
|
|
remains zero on flat data. Thus zero is not a calibrated "sideways"
|
||
|
|
regime probability. Readiness and direction are separate information.
|
||
|
|
Higher orders warm up only when enough lower-order pivots exist.
|
||
|
|
|
||
|
|
Equal highs and lows in ordinary FHLS windows retain their earliest
|
||
|
|
observation, matching the original Lua implementation. Synthetic
|
||
|
|
bridges have a narrower, distinct rule: when their endpoint values
|
||
|
|
are equal, high belongs to the second endpoint and low to the first.
|
||
|
|
That distinction affects pivot coordinates even when trends agree.
|
||
|
|
|
||
|
|
An optional ``lua_seed=True`` reproduces the reference's initial
|
||
|
|
comparison with the first window's last observation. It is a seed
|
||
|
|
policy for reference investigation, not a promise to reproduce all
|
||
|
|
Lua command-line startup quirks. Normal FSPs use neutral startup.
|
||
|
|
|
||
|
|
NaN clears all recurrence orders and requires fresh warmup. Infinity
|
||
|
|
and ambiguous coordinate ordering are rejected before state changes.
|
||
|
|
The scalar calculator does not guess session boundaries from elapsed
|
||
|
|
time: its caller decides whether a gap is intentional or a reset.
|
||
|
|
Prepending or revising source history requires replay from the new
|
||
|
|
origin; changing a prefix can affect every subsequent reversal.
|
||
|
|
|
||
|
|
Causality and output
|
||
|
|
--------------------
|
||
|
|
|
||
|
|
``TrendCalculator.push()`` returns all newly detected reversals.
|
||
|
|
Each record carries both coordinates:
|
||
|
|
|
||
|
|
* ``index``, ``time``, ``price`` locate the original pivot;
|
||
|
|
* ``detected_index``, ``detected_time`` locate the observation that
|
||
|
|
first confirmed it;
|
||
|
|
* ``direction`` is the new trend: ``+1`` for a bottom, ``-1`` for a
|
||
|
|
top; ``order`` starts at one.
|
||
|
|
|
||
|
|
A higher-order pivot retains its original price/time while detection
|
||
|
|
time propagates from the input observation that completed its
|
||
|
|
higher-order window. Strategies must use detection time, not the
|
||
|
|
earlier pivot time, when deciding what information was available.
|
||
|
|
|
||
|
|
``TrendCalculator.extend()`` returns per-observation ``trends`` and
|
||
|
|
``ready`` arrays. ``pulses`` stores only the last detected direction
|
||
|
|
per order and row. ``pivot_prices`` and ``pivot_times`` retain both
|
||
|
|
possible sides: axis two is top then bottom, with NaN for no event.
|
||
|
|
Use complete push events when the event stream itself is required.
|
||
|
|
|
||
|
|
The ``trend_calculus`` FSP exposes four orders through ``trend``,
|
||
|
|
``trend_2``, ``trend_3`` and ``trend_4``. Its ``ready`` field counts
|
||
|
|
ready orders. Each order also exposes ``top_N``, ``top_time_N``,
|
||
|
|
``bottom_N`` and ``bottom_time_N``. Pivot prices are published at
|
||
|
|
their detection row, with the paired field retaining pivot time.
|
||
|
|
Its ``window`` argument changes the first-order close window only:
|
||
|
|
the four sizes are ``(window, 2, 2, 2)``, defaulting to all twos.
|
||
|
|
The numerical and timeframe APIs accept explicit per-order tuples.
|
||
|
|
|
||
|
|
Piker's source SHM timestamps name candle starts. Historical rows
|
||
|
|
preceding the last row are finalized; the last row is provisional.
|
||
|
|
Quotes and source sampler steps preview that row from a copy of
|
||
|
|
committed recurrence state. These wakeups inspect authoritative source
|
||
|
|
SHM; they are not additional scalar observations. Each newly finalized
|
||
|
|
row advances committed state once, including when several steps arrive
|
||
|
|
before a wakeup. Sparse pivot fields clear when the destination steps;
|
||
|
|
source revisions require a cascade replay.
|
||
|
|
|
||
|
|
For realtime regime consumers, use finalized closes with an explicit
|
||
|
|
interval-end clock. ``piker.fsp._regime.MultiTimeframeTrend`` forms
|
||
|
|
epoch-aligned buckets from complete base bars, discards partial
|
||
|
|
startup buckets and resets recurrence after missing base bars. Its
|
||
|
|
snapshots distinguish timeframe period, recursive order, source
|
||
|
|
revision, sequence, gap segment and availability time. An old model
|
||
|
|
response must not overwrite newer source evidence.
|
||
|
|
|
||
|
|
The scalar calculator and current SHM ``trend_calculus`` FSP count
|
||
|
|
observations without inferring resets from elapsed timestamp gaps;
|
||
|
|
NaN explicitly resets their recurrence. ``MultiTimeframeTrend`` instead
|
||
|
|
requires contiguous base bars and starts a new segment after a missing
|
||
|
|
interval. Consequently, the simple one-second FSP and the one-second
|
||
|
|
timeframe tracker can differ across session gaps. That difference is
|
||
|
|
an ingestion policy, not a different FHLS recurrence.
|
||
|
|
|
||
|
|
Using the numerical and timeframe APIs
|
||
|
|
--------------------------------------
|
||
|
|
|
||
|
|
The numerical API has no provider, GUI or credential requirement.
|
||
|
|
This complete example confirms both sides of a complex window:
|
||
|
|
|
||
|
|
.. code-block:: python
|
||
|
|
|
||
|
|
from piker.fsp._trend import TrendCalculator
|
||
|
|
|
||
|
|
calculator = TrendCalculator((2, 2, 2))
|
||
|
|
prices: list[float] = [1., 2., 4., 3., 2., 5.]
|
||
|
|
row: int
|
||
|
|
price: float
|
||
|
|
for row, price in enumerate(prices):
|
||
|
|
update = calculator.push(
|
||
|
|
1_700_000_000. + row,
|
||
|
|
price,
|
||
|
|
index=100 + row,
|
||
|
|
)
|
||
|
|
|
||
|
|
print(update.trends)
|
||
|
|
print(update.reversals[
|
||
|
|
['index', 'direction', 'detected_index']
|
||
|
|
])
|
||
|
|
|
||
|
|
The last update has a top at source index 102 and a bottom at 104,
|
||
|
|
both detected at 105. A consumer only learns these events when it
|
||
|
|
receives that update. For batch analysis, pass aligned one-dimensional
|
||
|
|
``values``, ``times`` and optional signed integer ``indices`` arrays
|
||
|
|
to ``calculator.extend()``. Use a fresh calculator for an independent
|
||
|
|
historical replay; ``extend()`` advances an existing instance.
|
||
|
|
|
||
|
|
For wall-clock timeframes, submit each completed base bar once, using
|
||
|
|
its opening timestamp. The tracker converts this to availability at
|
||
|
|
the interval end:
|
||
|
|
|
||
|
|
.. code-block:: python
|
||
|
|
|
||
|
|
from piker.fsp._regime import MultiTimeframeTrend
|
||
|
|
|
||
|
|
tracker = MultiTimeframeTrend(
|
||
|
|
'synthetic.test',
|
||
|
|
base_period=1.,
|
||
|
|
periods=(1., 5.),
|
||
|
|
windows=(2, 2, 2),
|
||
|
|
)
|
||
|
|
prices: list[float] = [100., 101., 99., 103., 102.] * 4
|
||
|
|
row: int
|
||
|
|
price: float
|
||
|
|
for row, price in enumerate(prices):
|
||
|
|
snapshot = tracker.push(
|
||
|
|
1_700_000_000. + row,
|
||
|
|
price,
|
||
|
|
index=100 + row,
|
||
|
|
)
|
||
|
|
|
||
|
|
state: dict = snapshot.to_state()
|
||
|
|
|
||
|
|
These periods describe independent close series; ``windows`` still
|
||
|
|
counts observations and reversal points. Subsecond periods must be
|
||
|
|
whole microseconds, and every configured period must be a whole
|
||
|
|
multiple of ``base_period``. Startup midway through a larger bucket
|
||
|
|
does not produce a shortened first observation. On history repair,
|
||
|
|
call ``tracker.reset()`` and replay the corrected completed bars.
|
||
|
|
|
||
|
|
Source inventory
|
||
|
|
----------------
|
||
|
|
|
||
|
|
Project research and earlier implementation work:
|
||
|
|
|
||
|
|
* `Piker #79`_ is the older trend-detection issue; `Piker #103`_ is
|
||
|
|
the consolidated research entrypoint. It links Morgan's paper,
|
||
|
|
Lua, Scala, a working mathematical manuscript, and a public
|
||
|
|
TradingView/Pine implementation.
|
||
|
|
* `Piker #187`_ explicitly proposes Trend Calculus in the native FSP
|
||
|
|
set and links the planned Python implementation.
|
||
|
|
* `Piker #325`_ connects trend extrema computation with faster chart
|
||
|
|
y-range reductions. Its moving min/max proposal concerns
|
||
|
|
performance; it is not an alternative definition of FHLS.
|
||
|
|
* `Python implementation issue #1`_ requests offline/online support
|
||
|
|
and recurrence updates across timeframes. The ``setup`` branch
|
||
|
|
inspected at ``30afcfbd8cbb7b84d984fa2e183c6ae392da3479`` has a
|
||
|
|
placeholder ``tc`` module and Ray moving-average experiments.
|
||
|
|
Those experiments do not implement FHLS reversal detection.
|
||
|
|
|
||
|
|
Primary algorithm sources:
|
||
|
|
|
||
|
|
* `Morgan's method document`_, ``HowToStudyTrends_v1.03.pdf``:
|
||
|
|
the original conceptual account. Inspected PDF SHA-256::
|
||
|
|
|
||
|
|
2dc506a0449c13c01d2d798f120411df84e5667b611d92204ba0122e2aac7fd5
|
||
|
|
|
||
|
|
* `Original Lua program`_, ByteSumoLtd/TrendCalculus-lua commit
|
||
|
|
``7f54f9a101283cc188dbcd9282292fc3f73de8e9``,
|
||
|
|
``make/trendcalculus.lua``. Its header credits Andrew J. Morgan /
|
||
|
|
ByteSumo Limited and declares GPL version 3 or later. Inspected
|
||
|
|
source SHA-256::
|
||
|
|
|
||
|
|
c099fac68c945768c71f7c09804a21777295bbbf2e7dd61db59d9cdfd78f49c0
|
||
|
|
|
||
|
|
* `Spark TrendCalculus2`_, commit
|
||
|
|
``3fb6ef23e985d9d6e0414d3b682afbeef51c52e8``. The source declares
|
||
|
|
Apache-2.0. It uses earliest equal high and latest equal low, and
|
||
|
|
initializes from a zero dummy point or an excluded first point.
|
||
|
|
Those choices can change reversals and recursive warmup relative
|
||
|
|
to Lua and Piker. Inspected source SHA-256::
|
||
|
|
|
||
|
|
3d0723836d230c89d99537f1075537c3bbf809cebefd12caa6a1b517e7bc98b5
|
||
|
|
|
||
|
|
* `Working mathematical manuscript`_, ``main.tex``: describes
|
||
|
|
earliest equal low and latest equal high, a fixed initial positive
|
||
|
|
trend and an initial synthetic reversal. These are different from
|
||
|
|
both the Scala code's tie choices and native neutral startup.
|
||
|
|
Its displayed reversal-sign expression also disagrees with the
|
||
|
|
following low/high selection branches. It is a working draft,
|
||
|
|
not a unique executable specification; the executable references
|
||
|
|
and independently checked cases determine the conventions here.
|
||
|
|
Inspected source SHA-256::
|
||
|
|
|
||
|
|
e7b6fdff06fbde195a81281022cce300135d88e7484a97d1b02d621c5bfc4eab
|
||
|
|
|
||
|
|
* `Spark examples`_ provide historical visualization context.
|
||
|
|
The `public Pine indicator`_ linked from Piker #103 describes two
|
||
|
|
lagged Donchian channels and their midpoints for visual analysis.
|
||
|
|
That description does not establish recursive FHLS parity. No
|
||
|
|
complete Pine source or the user's separate Pine implementation
|
||
|
|
was recovered in this search, so no Pine parity claim is made.
|
||
|
|
|
||
|
|
Local research locations inspected, relative to the user's repos
|
||
|
|
directory:
|
||
|
|
|
||
|
|
* ``lns/todo/piker/quant/trend.toml`` links the Python ``setup``
|
||
|
|
branch, authors, Piker #187 and the Ray SMA example.
|
||
|
|
* ``lns/the_legacy_tab_nightmare/trading.toml`` links the working
|
||
|
|
manuscript, Spark comparisons/PRs, examples, and a former local
|
||
|
|
``TrendCalculus/notes_redesign_amorgan.pdf``. The latter file was
|
||
|
|
not present at its recorded path.
|
||
|
|
* ``data/bookmarks/bookmarks.html`` preserves the older ByteSumo
|
||
|
|
Bitbucket project URL. Duplicate bookmarks are not independent
|
||
|
|
algorithm evidence.
|
||
|
|
|
||
|
|
A separate filename inventory across ``/home/goodboy/repos`` included
|
||
|
|
hidden and ignored files while excluding Git internals, worktrees,
|
||
|
|
virtual environments, dependency directories, caches and build outputs.
|
||
|
|
It found no ``*.pine`` or ``*.pinescript`` files, TrendCalculus-named
|
||
|
|
source files, or copy of ``notes_redesign_amorgan.pdf`` beyond this
|
||
|
|
change's new documentation and fixture.
|
||
|
|
|
||
|
|
No earlier native Trend Calculus implementation was found in the
|
||
|
|
searched Piker source, tests, docs, snippets or available research
|
||
|
|
notes. This is a bounded search result, not proof that no private
|
||
|
|
copy or unindexed history exists. User-owned research/task states
|
||
|
|
were left untouched.
|
||
|
|
|
||
|
|
Verification and reproducibility
|
||
|
|
-------------------------------
|
||
|
|
|
||
|
|
``tests/test_trend_reference.py`` uses a separate batch oracle that
|
||
|
|
materializes complete FHLS windows, inserts synthetic windows and
|
||
|
|
then reduces the expanded sequence. It never calls the production
|
||
|
|
kernel or copies its mutable recurrence state. Coverage includes:
|
||
|
|
|
||
|
|
* every eight-point sequence over three prices, for three window
|
||
|
|
tuples: 19,683 histories with ties, flat startup and inside/outside
|
||
|
|
windows; direction, readiness, pulses and both pivot sides agree;
|
||
|
|
* another 2,187 exhaustive six-point histories using the optional Lua
|
||
|
|
seed, while keeping readiness tied to two actual complete windows;
|
||
|
|
* 48 seeded 512-point random walks across both startup policies,
|
||
|
|
five recursive orders and mixed window sizes, checking every event's
|
||
|
|
source identity and detection coordinates on irregular timestamps
|
||
|
|
and source indices;
|
||
|
|
* actual unmodified Lua output for windows 2, 3, 4 and 7, covering
|
||
|
|
62 reversals, including original pivot locations and earliest
|
||
|
|
detection prefixes.
|
||
|
|
|
||
|
|
The original-Lua fixture is
|
||
|
|
``tests/_inputs/trend_calculus_lua.json``. It records source/runtime
|
||
|
|
provenance, the complete synthetic inputs, event schema and SHA-256
|
||
|
|
hashes of each input and full output CSV. To reproduce an output,
|
||
|
|
write the fixture values with header ``ticker,date,last`` and rows
|
||
|
|
``x,<zero-based-row>,<value>``, using LF line endings. Run the pinned
|
||
|
|
program on Lua 5.2.4 with ``-n<window> -r -H -p1``. For detection
|
||
|
|
times, rerun each complete-window prefix and assign newly emitted
|
||
|
|
events to that prefix's last input row. Prefix outputs must preserve
|
||
|
|
all earlier emitted rows. No Lua executable or network is needed
|
||
|
|
for these Python reference tests.
|
||
|
|
|
||
|
|
The pinned Lua CLI has observable quirks:
|
||
|
|
|
||
|
|
* Do not pass ``-h`` with its built-in column mapping. That path
|
||
|
|
consumes the first real observation as a second header, shifting
|
||
|
|
every window boundary. ``-H`` controls only output headers.
|
||
|
|
* A flat first complete window can fail while reading a nonexistent
|
||
|
|
prior trend during bridge processing.
|
||
|
|
* The simple-window reversal branch suppresses initial output until
|
||
|
|
``offset > 2*n + 1``; split-window branches use other guards.
|
||
|
|
|
||
|
|
The fixture begins with increasing complete windows to establish a
|
||
|
|
common baseline before reversals. It establishes exact parity for
|
||
|
|
those inputs, not universal startup equivalence between programs.
|
||
|
|
The native oracle separately covers the flat and complex startup
|
||
|
|
cases excluded by that reference limitation.
|
||
|
|
|
||
|
|
Integration verification is separate from mathematical parity:
|
||
|
|
|
||
|
|
* ``tests/test_trend.py`` exercises committed/preview handoff,
|
||
|
|
quote-free sampler wakeups, indexed publication and cancellation.
|
||
|
|
* ``tests/test_trend_sampling.py`` exercises the actual sampler with
|
||
|
|
a controlled clock. Repeated or backward rounded epochs must not
|
||
|
|
append duplicate timestamps, including after realtime bootstrap.
|
||
|
|
* ``tests/test_trend_shm.py`` runs actual Tractor shared memory with a
|
||
|
|
separate read-only attachment, nonzero origin, quiet sampling and
|
||
|
|
an equal-bound historical repair. A separate child actor then
|
||
|
|
attaches read-only using the transported token and checks every
|
||
|
|
repaired field. This verifies cross-process visibility after replay,
|
||
|
|
not atomicity during concurrent writes. It needs actor runtime/socket
|
||
|
|
access, but no broker or model service.
|
||
|
|
* ``tests/test_trend_regimes.py`` checks complete aligned buckets,
|
||
|
|
fractional clocks, missing intervals and revision replay;
|
||
|
|
``tests/test_regime_feed.py`` checks native source admission,
|
||
|
|
subscription ordering, quote receiver ownership and lifecycle.
|
||
|
|
* ``tests/test_jev.py`` uses mocked transport and controlled clocks
|
||
|
|
for response validation, bounded projection, pacing, retry, source
|
||
|
|
invalidation and cancellation while publication is blocked.
|
||
|
|
* ``tests/test_trend_graphics.py`` and
|
||
|
|
``tests/test_trend_mtf_graphics.py`` exercise Qt geometry, causal
|
||
|
|
timestamp mapping, revision invalidation, model expiry and teardown.
|
||
|
|
|
||
|
|
Performance measurement
|
||
|
|
-----------------------
|
||
|
|
|
||
|
|
The repository example measures compilation/warmup, batch computation,
|
||
|
|
individual committed observations and copied-state previews separately.
|
||
|
|
Run it in the existing Piker Python environment:
|
||
|
|
|
||
|
|
.. code-block:: xsh
|
||
|
|
|
||
|
|
py313/bin/python examples/benchmark_trend.py --rows 1000000 --live-rows 20000
|
||
|
|
|
||
|
|
One x86-64 run on 2026-10-05 used Python 3.13.14, NumPy 2.3.5 and
|
||
|
|
Numba 0.62.1 with four reversal orders:
|
||
|
|
|
||
|
|
=========================== ============================
|
||
|
|
Measurement Observed result
|
||
|
|
=========================== ============================
|
||
|
|
One million-row batch 0.116473 seconds
|
||
|
|
Batch throughput 8.586 million rows/second
|
||
|
|
Scalar commit, median/p99 6.963 / 10.670 microseconds
|
||
|
|
Copy and preview, median/p99 12.223 / 18.835 microseconds
|
||
|
|
Initial kernel warmup 3.004 seconds
|
||
|
|
Retained NumPy state 936 bytes
|
||
|
|
=========================== ============================
|
||
|
|
|
||
|
|
These are reproducible measurements from one machine and workload,
|
||
|
|
not latency guarantees. Retained-state bytes exclude Python objects,
|
||
|
|
compiler caches and returned history arrays. Timings exclude source
|
||
|
|
SHM copies, actor IPC, chart rendering and provider inference. The
|
||
|
|
native sampler currently timestamps periods at or below one second
|
||
|
|
with integer seconds; subsecond end-to-end operation requires a
|
||
|
|
producer with an appropriate event clock, despite the numerical and
|
||
|
|
timeframe APIs supporting fractional-second observations.
|
||
|
|
|
||
|
|
TypeSafe/Jev integration
|
||
|
|
-----------------------
|
||
|
|
|
||
|
|
The user supplied the `TypeSafe model documentation`_,
|
||
|
|
`Python SDK documentation`_ and `agent skill`_. The adapter uses
|
||
|
|
Piker's existing HTTPX/Trio stack against the documented `System One
|
||
|
|
HTTP API`_; installing the separate Python SDK is unnecessary. The
|
||
|
|
default model is pinned to ``jev-1.13.0``. Importing the adapter does
|
||
|
|
not read credentials or open a connection.
|
||
|
|
|
||
|
|
``build_regime_request(snapshot)`` produces one Choice question for
|
||
|
|
each ready timeframe, plus an independent alignment question when
|
||
|
|
multiple timeframes are ready. Candidate regimes are ``uptrend``,
|
||
|
|
``downtrend``, ``range``, ``transition`` and ``unknown``. Questions
|
||
|
|
explicitly identify state paths because question identifiers do not
|
||
|
|
reach the model. ``JevClassification`` retains every candidate's
|
||
|
|
probability, the selected label, the service's separate confidence
|
||
|
|
value, model/prompt versions, source provenance and receipt time.
|
||
|
|
These semantic judgments do not change the numerical recurrence.
|
||
|
|
Model input uses the most recent 16 closes and 16 reversals per
|
||
|
|
timeframe by default. ``evidence_limit`` configures that projection
|
||
|
|
independently of longer chart history; source snapshots stay unchanged.
|
||
|
|
|
||
|
|
Inspect a complete synthetic request, then exercise HTTP parsing with
|
||
|
|
explicitly simulated responses:
|
||
|
|
|
||
|
|
.. code-block:: xsh
|
||
|
|
|
||
|
|
py313/bin/python examples/trend_regimes.py
|
||
|
|
py313/bin/python examples/trend_regimes.py --mock
|
||
|
|
|
||
|
|
The synthetic example includes quarter-second evidence. Its ``--live``
|
||
|
|
mode sends exactly one synthetic request and reads ``TYPESAFE_API_KEY``
|
||
|
|
from the caller's environment. It consumes provider credit and makes no
|
||
|
|
broker connection. A real provider request has not been verified without
|
||
|
|
an API key; mocked results test the integration, not model quality.
|
||
|
|
|
||
|
|
The native live bridge subscribes to the sampler before copying history,
|
||
|
|
excludes the mutable final source row, and emits only the newest snapshot
|
||
|
|
after each captured history range. It catches up skipped sample steps
|
||
|
|
and invalidates old judgments before replaying a prepend, truncation,
|
||
|
|
backfill or lost sampler notice. ``run_regime_feed()`` supplies these
|
||
|
|
snapshots to a synchronous callback without requiring a model client.
|
||
|
|
Use ``serve_regimes()`` to compose that reader and a Jev worker:
|
||
|
|
|
||
|
|
.. code-block:: python
|
||
|
|
|
||
|
|
from collections.abc import Awaitable, Callable
|
||
|
|
|
||
|
|
import httpx
|
||
|
|
|
||
|
|
from piker.fsp._jev import JevClassification, JevClient
|
||
|
|
from piker.fsp._regime_feed import serve_regimes
|
||
|
|
from piker.service import open_piker_runtime
|
||
|
|
|
||
|
|
async def run_live(
|
||
|
|
fqme: str,
|
||
|
|
api_key: str,
|
||
|
|
publish: Callable[[JevClassification], Awaitable[None]],
|
||
|
|
) -> None:
|
||
|
|
async with (
|
||
|
|
open_piker_runtime('trend-regimes'),
|
||
|
|
httpx.AsyncClient() as http,
|
||
|
|
):
|
||
|
|
await serve_regimes(
|
||
|
|
fqme,
|
||
|
|
JevClient(http, api_key),
|
||
|
|
publish,
|
||
|
|
periods=(1., 5., 15.),
|
||
|
|
min_interval=1.,
|
||
|
|
max_age=5.,
|
||
|
|
max_source_lag=2.,
|
||
|
|
max_source_age=5.,
|
||
|
|
)
|
||
|
|
|
||
|
|
Calling this function opens the configured broker feed and sends live
|
||
|
|
inference requests. Supply an existing Piker market identifier and a
|
||
|
|
cancellation-cooperative async consumer. The example periods assume
|
||
|
|
a one-second source; omit ``periods`` to derive 1x/5x/15x periods from
|
||
|
|
the actual source. Native subsecond source periods are rejected until
|
||
|
|
the sampler has a compatible clock. A separate correctly timestamped
|
||
|
|
producer can use ``consume_regime_source()`` directly.
|
||
|
|
|
||
|
|
The worker permits one in-flight request and retains only the newest
|
||
|
|
pending snapshot. ``min_interval`` limits request start frequency;
|
||
|
|
``max_age`` limits local submission age, ``max_source_lag`` limits
|
||
|
|
event-time lag, and ``max_source_age`` filters stale wall-clock source
|
||
|
|
data before submission. Setting the last option to ``None`` admits
|
||
|
|
historical replay. Source revision or gap-segment changes revoke old
|
||
|
|
results. A blocked publication is also cancelled when its evidence
|
||
|
|
expires or becomes stale. A downstream consumer must still check
|
||
|
|
provenance on already-delivered messages, which cannot be revoked.
|
||
|
|
|
||
|
|
Current source SHM has no transactional revision word. Copy-time bound
|
||
|
|
checks detect appends and origin changes; an in-place correction with
|
||
|
|
unchanged bounds becomes identifiable through its backfill notice.
|
||
|
|
Replay converges to that corrected source. A publication without a
|
||
|
|
local task checkpoint is not an atomic cross-process memory transaction.
|
||
|
|
|
||
|
|
Malformed/authentication responses fail explicitly. Transient failures
|
||
|
|
use bounded retries and freshness checks; the adapter bounds HTTP body
|
||
|
|
sizes and timeouts. The owning nursery closes the feed, sampler and
|
||
|
|
in-flight worker on cancellation. Neither the bridge nor the classifier
|
||
|
|
submits trades. Regime rubrics and probability usefulness still need
|
||
|
|
evaluation on recorded market data and real provider responses.
|
||
|
|
|
||
|
|
Chart overlays
|
||
|
|
--------------
|
||
|
|
|
||
|
|
Preview synthetic multiple-timeframe annotations without a broker,
|
||
|
|
model request or credentials:
|
||
|
|
|
||
|
|
.. code-block:: xsh
|
||
|
|
|
||
|
|
py313/bin/python examples/trend_graphics.py --output /tmp/trend.png
|
||
|
|
|
||
|
|
The offscreen example accepts ``--periods 1 5 15`` and ``--order 1``
|
||
|
|
and isolates its Qt/configuration state from the running application.
|
||
|
|
|
||
|
|
Enable the native order-one reversal and direction overlay in the
|
||
|
|
existing Piker configuration's ``[ui]`` section::
|
||
|
|
|
||
|
|
[ui]
|
||
|
|
trend_calculus = true
|
||
|
|
|
||
|
|
This starts one Trend Calculus writer for the chart's realtime source
|
||
|
|
and displays confirmed pivot arrows with translucent direction
|
||
|
|
rectangles. It makes no Jev request and does not read API credentials.
|
||
|
|
Hover text distinguishes the pivot's bar from the later time when
|
||
|
|
the reversal became observable. The mutable last FSP row is excluded
|
||
|
|
from confirmed markers and direction runs.
|
||
|
|
|
||
|
|
Within the existing chart owner's nursery,
|
||
|
|
``await FspAdmin.open_trend_overlay(order=2)`` attaches another
|
||
|
|
recursive order while reusing that admin's writer. Optional ``chart``
|
||
|
|
and ``chart_period_s`` arguments attach the source evidence to another
|
||
|
|
price chart. Changing the chart period does not recompute the source
|
||
|
|
at that period: its actual realtime sampling period stays explicit.
|
||
|
|
|
||
|
|
For independent wall-clock timeframes on the native realtime chart,
|
||
|
|
configure::
|
||
|
|
|
||
|
|
[ui]
|
||
|
|
trend_timeframes = [1, 5, 15]
|
||
|
|
|
||
|
|
This option takes precedence over ``trend_calculus = true``. It uses
|
||
|
|
one ``MultiTimeframeTrend`` reader and one ``SnapshotTrendOverlay``
|
||
|
|
with separate marker/rectangle layers for each period. It opens no Jev
|
||
|
|
client. ``await FspAdmin.open_multi_timeframe_trends(...)`` exposes the
|
||
|
|
same integration programmatically and accepts a recursive ``order``.
|
||
|
|
All periods must be multiples of the actual realtime source period.
|
||
|
|
|
||
|
|
Snapshots preserve interval-end pivot and detection times. Markers
|
||
|
|
attach to the base bar that contributed each aggregated close. A
|
||
|
|
bootstrap snapshot contains recent reversals and the latest direction,
|
||
|
|
so its deterministic rectangle history starts with that observed
|
||
|
|
direction. It does not reconstruct earlier direction runs from one
|
||
|
|
snapshot. Skipped complete buckets leave unobserved intervals unpainted.
|
||
|
|
Source revisions and missing-base-bar segments revoke prior annotations.
|
||
|
|
|
||
|
|
An existing caller-owned client can share this exact evidence with
|
||
|
|
the model worker inside the chart's nursery:
|
||
|
|
|
||
|
|
.. code-block:: python
|
||
|
|
|
||
|
|
import trio
|
||
|
|
|
||
|
|
from piker.fsp._jev import JevClient, RegimeClassifier
|
||
|
|
from piker.ui._fsp import FspAdmin
|
||
|
|
from piker.ui._trend_mtf import SnapshotTrendOverlay
|
||
|
|
|
||
|
|
async def attach_classifier(
|
||
|
|
admin: FspAdmin,
|
||
|
|
client: JevClient,
|
||
|
|
nursery: trio.Nursery,
|
||
|
|
) -> tuple[RegimeClassifier, SnapshotTrendOverlay]:
|
||
|
|
worker: RegimeClassifier = RegimeClassifier(client)
|
||
|
|
overlay = await admin.open_multi_timeframe_trends(
|
||
|
|
periods=(1., 5., 15.),
|
||
|
|
on_snapshot=worker.submit,
|
||
|
|
on_invalidate=worker.invalidate,
|
||
|
|
max_source_age=5.,
|
||
|
|
classification_ttl=5.,
|
||
|
|
)
|
||
|
|
nursery.start_soon(worker.run, overlay.publish_classification)
|
||
|
|
return worker, overlay
|
||
|
|
|
||
|
|
Graphics retain their longer deterministic history while the client
|
||
|
|
projects bounded model evidence. The source-age gate applies to the
|
||
|
|
optional model callback, so historical chart bootstrap still works.
|
||
|
|
The adapter checks market, revision, segment, sequence and source lag
|
||
|
|
before accepting a model response. Classified runs start at receipt
|
||
|
|
and expire after the configured TTL; they cannot paint future history
|
||
|
|
before the displayed event clock reaches receipt. Cancelling the chart
|
||
|
|
owner removes all layers and stops its sampler reader.
|
||
|
|
|
||
|
|
For computations that already publish separate timeframe SHMs, create
|
||
|
|
one ``piker.ui._trend.TrendOverlay`` per destination SHM with its own
|
||
|
|
``source_period_s``, target ``chart_period_s`` and ``order``. Mapping
|
||
|
|
uses timestamps and preserves gaps rather than snapping missing
|
||
|
|
observations onto the next candle. Geometry counts are bounded and
|
||
|
|
items are reused instead of adding individual Qt items on every tick.
|
||
|
|
|
||
|
|
Call ``update(force=True)`` after a matching destination or chart
|
||
|
|
history revision, and ``remove()`` when its owner exits. Normal
|
||
|
|
``FspAdmin`` integration routes destination-specific revisions and
|
||
|
|
teardown automatically; direct attachments must retain this ownership.
|
||
|
|
Hidden widgets preserve revision invalidations until their next
|
||
|
|
visible update. Repainting follows quotes and the existing one-second
|
||
|
|
sampler wakeup. If that wakeup precedes asynchronous FSP publication,
|
||
|
|
a quote-free finalization can appear on the next wakeup, roughly one
|
||
|
|
second later; the integration does not emit IPC for every output row.
|
||
|
|
|
||
|
|
For model annotations, ``classification_interval()`` converts a
|
||
|
|
matching ``JevClassification`` into a ``RegimeInterval``. Supply the
|
||
|
|
chart's current market, source revision, timeframe and display end.
|
||
|
|
The interval starts no earlier than the model response's receipt.
|
||
|
|
``overlay.set_regimes(intervals)`` displays an owner-maintained,
|
||
|
|
bounded collection of these intervals. An empty collection clears
|
||
|
|
model rectangles; ``None`` restores deterministic direction runs.
|
||
|
|
The caller remains responsible for response ordering and expiry.
|
||
|
|
|
||
|
|
.. _Piker #79: https://github.com/pikers/piker/issues/79
|
||
|
|
.. _Piker #103: https://github.com/pikers/piker/issues/103
|
||
|
|
.. _Piker #187: https://github.com/pikers/piker/issues/187
|
||
|
|
.. _Piker #325: https://github.com/pikers/piker/issues/325
|
||
|
|
.. _Python implementation issue #1: https://github.com/bytesumo/TrendCalculus/issues/1
|
||
|
|
.. _Morgan's method document: https://github.com/bytesumo/TrendCalculus/blob/3d5c598cd3764a40633979c4c6f4188191dd81ef/HowToStudyTrends_v1.03.pdf
|
||
|
|
.. _Original Lua program: https://github.com/ByteSumoLtd/TrendCalculus-lua/blob/7f54f9a101283cc188dbcd9282292fc3f73de8e9/make/trendcalculus.lua
|
||
|
|
.. _Spark TrendCalculus2: https://github.com/lamastex/spark-trend-calculus/blob/3fb6ef23e985d9d6e0414d3b682afbeef51c52e8/src/main/scala/org/lamastex/spark/trendcalculus/TrendCalculus2.scala
|
||
|
|
.. _Working mathematical manuscript: https://github.com/lamastex/working-manuscript-trend-calculus/blob/master/main.tex
|
||
|
|
.. _Spark examples: https://lamastex.github.io/spark-trend-calculus-examples/
|
||
|
|
.. _public Pine indicator: https://www.tradingview.com/script/ohSdmhS0-TrendCalculus/
|
||
|
|
.. _TypeSafe model documentation: https://docs.typesafe.ai/models
|
||
|
|
.. _Python SDK documentation: https://docs.typesafe.ai/sdk/python
|
||
|
|
.. _agent skill: https://docs.typesafe.ai/agent-skill
|
||
|
|
.. _System One HTTP API: https://docs.typesafe.ai/api
|