Commit Graph

2094 Commits (7d628c4059bafd5bd931496b303005efaf0ca19f)

Author SHA1 Message Date
Tyler Goodlet 7d628c4059 Prototype out writing `1Sec` OHLCV data 2022-04-30 11:38:24 -04:00
Tyler Goodlet 6dfe59cce6 Better doc string 2022-04-30 11:38:24 -04:00
Tyler Goodlet a465a11782 Persist backing `/data/` filesystem across container runs 2022-04-30 11:38:24 -04:00
Tyler Goodlet 807685d27e Get basic OHLCV writes working with `anyio` client 2022-04-30 11:38:24 -04:00
Tyler Goodlet aea42ccbd9 Pass in daemon name to `start_ahab()` 2022-04-30 11:38:24 -04:00
Tyler Goodlet 8fb9308e21 Map the grpc port and add graceful container teardown
Not sure how I missed mapping the 5995 grpc port 🤦; done now.
Also adds graceful teardown using SIGINT with included container
logging relayed to the piker console B).
2022-04-30 11:38:24 -04:00
Tyler Goodlet bbae8ad426 Revive `ms-shell` sub-cmd 2022-04-30 11:38:24 -04:00
Tyler Goodlet 893ac7a986 Add WIP backfiller from data feed helper 2022-04-30 11:38:24 -04:00
Tyler Goodlet 91856ddda8 Better handle nested erros from docker client 2022-04-30 11:38:24 -04:00
Tyler Goodlet 25e2e13bd7 Add back in legacy write loop for reference 2022-04-30 11:38:24 -04:00
Tyler Goodlet 2427c96336 Add back in OHLCV dtype template and client side ws streamer 2022-04-30 11:38:24 -04:00
Tyler Goodlet 8732b2bd5e Drop ununsed `Services` ref 2022-04-30 11:38:24 -04:00
Tyler Goodlet ff9208c15b Py3.9+ type updates 2022-04-30 11:38:24 -04:00
Tyler Goodlet a2547a548f Add `--tsdb` flag to start `marketstore` with `pikerd` 2022-04-30 11:38:24 -04:00
Tyler Goodlet 3873b8619e De-escalate sudo perms in `pikerd` once docker spawns 2022-04-30 11:38:24 -04:00
Tyler Goodlet 326d05ac82 Handle the non-root perms case specifically too 2022-04-30 11:38:24 -04:00
Tyler Goodlet dcab99e3d2 Add explicit no-docker error and supervisor start task-func 2022-04-30 11:38:24 -04:00
Tyler Goodlet 5a5df21f94 Type annot updates 2022-04-30 11:38:24 -04:00
Tyler Goodlet d7c1286e5d Drop old client instantiate line 2022-04-30 11:38:24 -04:00
Tyler Goodlet 8acaa28df0 Drop import, it's got madness with and SIGINT? 2022-04-30 11:38:24 -04:00
Tyler Goodlet bb45100168 Add a super simple `marketstore` container supervisor 2022-04-30 11:38:24 -04:00
Tyler Goodlet d9ded54e10 Extract non-sudo user for config dir path 2022-04-30 11:38:24 -04:00
Tyler Goodlet 1bde86a7b2 Basic module-script for spawning `marketstore`, needs correct bind mount usage 2022-04-30 11:38:24 -04:00
Guillermo Rodriguez 5031892dcf Still WIP, switch to using new marketstore client, missing streaming from marketstore 2022-04-30 11:38:24 -04:00
Guillermo Rodriguez afe6f0b42b Simplify and optimize tick format, similar to techtonicdb's 2022-04-30 11:38:24 -04:00
Guillermo Rodriguez 5d539b7c49 Add multi ingestor support and update to new feed API 2022-04-30 11:38:24 -04:00
Tyler Goodlet e2ce341f93 Only update y mxmn from L1 when last index in view 2022-04-30 11:38:24 -04:00
goodboy 84399e8131
Merge pull request #289 from pikers/big_data_lines
"Big data" lines
2022-04-30 11:37:50 -04:00
Tyler Goodlet 5921d18d66 Only update y-range from L1 mxmn when last index in view
We still have to always keep track of the last max and min
though.
2022-04-30 11:36:23 -04:00
Tyler Goodlet cdc882657a Drop old `pyqtgraph` downsample code 2022-04-30 11:36:23 -04:00
Tyler Goodlet 62d08eaf85 Tweak log-scaler for more detail 2022-04-30 11:36:23 -04:00
Tyler Goodlet f2f00dcc52 Drop `._ic` debugging prints 2022-04-30 11:36:23 -04:00
Tyler Goodlet ee831baeb3 Display loop mega-cleanup
The most important changes include:
- iterating the new `Flow` type and updating graphics
- adding detailed profiling
- increasing the min uppx before graphics updates are throttled
- including the L1 spread in y-range calcs so that you never have the
  bid/ask go "out of view"..
- pass around `Flow`s instead of shms
- drop all the old prototyped downsampling code
2022-04-30 11:36:23 -04:00
Tyler Goodlet 7c615a403b Allow passing a `plotItem` to `.draw_curve()`
If manually managing an overlay you'll likely call `.overlay_plotitem()`
and then a plotting method so we need to accept a plot item input so
that the chart's pi doesn't get assigned incorrectly in the `Flow` entry
(though it is by default if no input is provided).

More,
- add a `Flow.graphics` field and set it to the `pg.GraphicsObject`.
- make `Flow.maxmin()` return `None` in the "can't calculate" cases.
2022-04-30 11:36:23 -04:00
Tyler Goodlet b8374dbe9a Fsp UI initialization updates
- set shm refs on `Flow` entries.
- don't run a graphics cycle on 'update' msgs from the engine
  if the containing chart is hidden.
- drop `volume` from flows map and disable auto-yranging
  once $vlm comes up.
2022-04-30 11:36:23 -04:00
Tyler Goodlet 454cd7920d Disconnect signals in `ChartView.disable_auto_yrange()`
Allows for removing resize callbacks for a flow/overlay that you wish to
remove from view (eg. unit volume after dollar volume is up) and thus
less general interaction callback overhead for any plot you don't wish
to show or resize.

Further,
- drop the `autoscale_linked_plots` block for now since with
  multi-view-box overlays each register their own vb resize slots
- pull the graphics object from the chart's `Flow` map inside
  `.maybe_downsample_graphics()`
2022-04-30 11:36:23 -04:00
Tyler Goodlet ca283660de Fix bug where if `yrange` was passed the mxmin callback was still used.. 2022-04-30 11:36:23 -04:00
Tyler Goodlet d4eddbdb25 Guard against zero px width 2022-04-30 11:36:23 -04:00
Tyler Goodlet eec329a221 Add `Flow` type with a real chitty mxmn cacheing method
This new type wraps a shm data flow and will eventually include things
like incremental path-graphics updates and serialization + bg downsampling
techniques. The main immediate motivation was to get a cached y-range max/min
calc going since profiling revealed the `numpy` equivalents were
actually quite slow as the data set grows large. Likely we can use all
this to drive a streaming mx/mn routine that's always launched as part
of each on-host flow.

This is our official foray into use of `msgspec.Struct` B) and I have to
say, pretty impressed; we'll likely completely ditch `pydantic` from
here on out.
2022-04-30 11:36:23 -04:00
Tyler Goodlet a1de097a43 Loop for first graphic with xvec 2022-04-30 11:36:23 -04:00
Tyler Goodlet b5f2558cec Only `.maybe_downsample_graphics()` on manual changes
We don't need update graphics on every x-range change since that's what
the display loop does. Instead, only on manual changes do we make manual
calls into `.update_graphics_from_array()` and be sure to iterate all
linked subplots and all their embedded graphics.
2022-04-30 11:36:23 -04:00
Tyler Goodlet 1a95712680 Don't return early on ds line render to avoid breaking profiling
The pg profiler seems to have trouble with early `return`s in function
calls (likely muckery with the GC/`.__delete__()`) so let's just try
to avoid it for now until we either fix it (probably by implementing as
a ctx mngr) or use diff one.
2022-04-30 11:36:23 -04:00
Tyler Goodlet b20e9e58ee Use HL tracer by default, seems to be faster? 2022-04-30 11:36:23 -04:00
Tyler Goodlet 4bc2bbda69 Allow passing "ms slower then" value on cli to `--profile` 2022-04-30 11:36:23 -04:00
Tyler Goodlet b524929cb6 Only bail up pan updates if uppx > 16 2022-04-30 11:36:23 -04:00
Tyler Goodlet f95d22bfd3 Delegate `BarItems.x_uppx()` to internal ds curve 2022-04-30 11:36:23 -04:00
Tyler Goodlet 91de281b7e Downsample curves even less frequently 2022-04-30 11:36:23 -04:00
Tyler Goodlet 2284e61eda Only pass vr for bars, allow source vb in autorange 2022-04-30 11:36:23 -04:00
Tyler Goodlet 082b02776c Drop the unit-volume chart once $vlm is fully drawn 2022-04-30 11:36:23 -04:00
Tyler Goodlet 27e3d0ef80 Ensure we update the volume array, not graphics
Ugh, turns out the wacky `ChartView.maxmin` callback stuff we did (for
determining y-range sizings) currently requires that the volume array
has a "bars in view" result.. so let's make that keep working without
rendering the graphics for the curve (since we're disabling them once
$vlm comes up).
2022-04-30 11:36:23 -04:00