# Financial Signal Processing

## Model the Clock First

Market inputs can be quote events, trades, book changes, fixed-time bars, or
revised historical bars. Name both event time and processing time. For
irregular events, either use time-aware recurrences or define the resampling
and interpolation policy explicitly.

Never let backfilled information leak into a value presented as historically
tradable. Distinguish corrected analysis history from the signal values that a
live strategy could have observed.

## Useful Technique Families

### Online Moments and Robust Statistics

Use Welford/Chan recurrences for stable online moments, compensated summation
for long accumulations, and mergeable summaries when partitioning actors.
Rolling median, quantiles, MAD, and winsorized measures are useful under heavy
tails, but exact rolling order statistics need more state than moments.

### Time-Aware Exponential Filters

For irregular event intervals, derive decay from elapsed time:

```text
alpha(dt) = 1 - exp(-dt / tau)
y_t = y_prev + alpha(dt) * (x_t - y_prev)
```

Record initialization, session reset, gap, and backfill behavior. A fixed
sample alpha is valid only after an explicit regularization policy.

### State-Space Models

Kalman and robust state-space filters fit latent price, spread, volatility,
and lead/lag estimates. Prefer square-root or Joseph-form covariance updates
when conditioning is poor. State revisions after backfill require replay from
a checkpoint or a bounded smoother; silently prepending observations without
replaying state is incorrect.

### Point Processes and Microstructure

Trade and quote arrivals are events, not merely sampled amplitudes. Consider
intensity, duration, signed order flow, imbalance, spread, queue change, and
self-excitation models. Validate against provider-specific aggregation and
duplicate/out-of-order behavior before interpreting a statistic.

### Volatility and Covariance

Realized variance, bipower variation, EW covariance, and range estimators can
be updated online. Asynchronous cross-market covariance needs synchronization
or estimators designed for non-synchronous observations; naive row alignment
creates lead/lag and Epps-effect artifacts.

### Spectral and Multiscale Methods

FFT methods assume a regular grid and a window. Declare detrending, tapering,
overlap, normalization, and latency. For tick data, resample deliberately or
use irregular-time methods. Wavelets and multiresolution filters can separate
horizons, but boundary handling and causal delay must be visible to strategy
code.

### Change and Anomaly Detection

CUSUM, Page-Hinkley, sequential likelihood ratios, and robust z-scores are
cheap online tools. Calibrate false alarms under dependence and regime shifts;
do not treat IID thresholds as market guarantees.

## Numerical Implementation Order

1. Establish a scalar reference with explicit state and semantics.
2. Vectorize historical bootstrap with NumPy.
3. Keep realtime updates as O(1) recurrences where possible.
4. Use Numba for measured numeric kernels with stable dtypes and no Python
   object traffic.
5. Use Polars or Arrow for columnar historical transforms and interchange,
   not automatically for each tick.
6. Partition by independent market/operator state before adding threads or a
   new distributed runtime.

## Existing Stack

- NumPy 2.x: canonical dense and structured-array kernels.
- Numba: compiled CPU loops and recurrences that do not vectorize cleanly.
- Polars: parallel/lazy columnar history preparation and validation.
- PyArrow: columnar interchange, storage, and ingest boundaries.
- Tractor/Trio: structured distributed execution and lifecycle.
- PyQtGraph: visible-range rendering, not a compute scheduler.

## Algorithm Acceptance Criteria

Every new FSP should define:

- input fields, clock, ordering, duplicate, gap, and revision policy;
- output dtype, units, identity, and parameterization;
- warm-up length and initialization bias;
- causal latency and strategy-visible publication point;
- historical/realtime equivalence tolerance;
- reset and session-boundary behavior;
- complexity, state size, and representative throughput;
- replay behavior when historical source data changes.