85e2602d2e
Drop `ms-shell`, add `piker storesh` cmd
2022-04-13 08:11:57 -04:00
edcac1e768
Add diffing logic to `tsdb_history_update()`
...
Add some basic `numpy` epoch slice logic to generate append and prepend
arrays to write to the db.
Mooar cool things,
- add a `Storage.delete_ts()` method to wipe a column series from the db
easily.
- don't attempt to read in any OHLC series by default on client load
- add some `pyqtgraph` profiling and drop manual latency measures
- if no db series for the fqsn exists write the entire shm array
2022-04-13 08:11:57 -04:00
73b8719984
Drop `pandas` to `numpy` converter
2022-04-13 08:11:57 -04:00
6b17370711
Disable re-connect for now in ib script
2022-04-13 08:11:57 -04:00
cd3dbc9275
Ensure bfqsn is lower cased for feed api consumers
...
Also, Start tinkering with `tractor.trionics.ipython_embed()`
In effort to get back to a usable REPL around the mkts client
this adds usage of the new `tractor` integration api as well as logic
for skipping backfilling if existing tsdb arrays are found.
2022-04-13 08:11:57 -04:00
87fab6c5b1
Try downsampling mkts data
2022-04-13 08:11:57 -04:00
2d719f88d2
Drop old type annot
2022-04-13 08:11:57 -04:00
d3fb9f228d
Comment each special key combo
2022-04-13 08:11:57 -04:00
58f6aa4308
Load any symbol-matching shm array if no `marketstored` found
2022-04-13 08:11:57 -04:00
1eb62e7678
Get ib key hack script to work with reconnect
2022-04-13 08:11:57 -04:00
965ea120e9
Move ib data reset script into a new `scripts/` dir
2022-04-13 08:11:57 -04:00
fbdcbc9304
Use new `tractor.query_actor()` for service checking
2022-04-13 08:11:57 -04:00
7f87d09bf0
Return all timeframe arrays if `timeframe` not passed as input
2022-04-13 08:11:57 -04:00
f7a0f117a0
Make `pikerd` work again without `--tsdb` flag
2022-04-13 08:11:57 -04:00
ae986e9fcf
Add a service checker predicate
2022-04-13 08:11:57 -04:00
cad6508b28
Allow kill-child-proc-with-root-perms to fail silently in `tractor` reaping
2022-04-13 08:11:57 -04:00
46ad8557cd
Proxy `marketstore` container log level to our own
2022-04-13 08:11:57 -04:00
85658a165c
Prototype a high level `Storage` api
...
Starts a wrapper around the `marketstore` client to do basic ohlcv query
and retrieval and prototypes out write methods for ohlc and tick.
Try to connect to `marketstore` automatically (which will fail if not
started currently) but we will eventually first do a service query.
Further:
- get `pikerd` working with and without `--tsdb` flag.
- support spawning `brokerd` with no real-time quotes.
- bring back in "fqsn" support that was originally not
in this history before commits factoring.
2022-04-13 08:11:57 -04:00
387cb51ca2
Doc str formatting
2022-04-13 08:11:57 -04:00
3c6f412a5b
Add latency measures around diffs/writes to mkts
2022-04-13 08:11:57 -04:00
852f9e3e64
Prototype out writing `1Sec` OHLCV data
2022-04-13 08:11:57 -04:00
0af809849b
Better doc string
2022-04-13 08:11:57 -04:00
5871aba04c
Persist backing `/data/` filesystem across container runs
2022-04-13 08:11:57 -04:00
0904b48a23
Get basic OHLCV writes working with `anyio` client
2022-04-13 08:11:57 -04:00
c2b5a3d225
Pass in daemon name to `start_ahab()`
2022-04-13 08:11:57 -04:00
4c5ea60035
Map the grpc port and add graceful container teardown
...
Not sure how I missed mapping the 5995 grpc port 🤦 ; done now.
Also adds graceful teardown using SIGINT with included container
logging relayed to the piker console B).
2022-04-13 08:11:57 -04:00
abf7b14717
Revive `ms-shell` sub-cmd
2022-04-13 08:11:57 -04:00
a6613161c3
Add WIP backfiller from data feed helper
2022-04-13 08:11:57 -04:00
6d38f3d0cb
Better handle nested erros from docker client
2022-04-13 08:11:57 -04:00
0877d09bbf
Add back in legacy write loop for reference
2022-04-13 08:11:57 -04:00
98ec2b71ae
Add back in OHLCV dtype template and client side ws streamer
2022-04-13 08:11:57 -04:00
00cd77a0ad
Drop ununsed `Services` ref
2022-04-13 08:11:57 -04:00
cbf1dca0b1
Py3.9+ type updates
2022-04-13 08:11:57 -04:00
7cdf7f5d13
Add `--tsdb` flag to start `marketstore` with `pikerd`
2022-04-13 08:11:57 -04:00
4130a1d2d6
De-escalate sudo perms in `pikerd` once docker spawns
2022-04-13 08:11:57 -04:00
17aa59b1f9
Handle the non-root perms case specifically too
2022-04-13 08:11:57 -04:00
c27ce637c8
Add explicit no-docker error and supervisor start task-func
2022-04-13 08:11:56 -04:00
86b9c9a8f9
Type annot updates
2022-04-13 08:11:56 -04:00
54741ad9c6
Drop old client instantiate line
2022-04-13 08:11:56 -04:00
4670e66348
Drop import, it's got madness with and SIGINT?
2022-04-13 08:11:56 -04:00
a3620a93ca
Add a super simple `marketstore` container supervisor
2022-04-13 08:11:56 -04:00
ddd92bb2ba
Extract non-sudo user for config dir path
2022-04-13 08:11:56 -04:00
12a1074f0e
Basic module-script for spawning `marketstore`, needs correct bind mount usage
2022-04-13 08:11:56 -04:00
eb41ecb4db
Still WIP, switch to using new marketstore client, missing streaming from marketstore
2022-04-13 08:11:56 -04:00
bec76bfe59
Simplify and optimize tick format, similar to techtonicdb's
2022-04-13 08:11:56 -04:00
ea1d7e4598
Add multi ingestor support and update to new feed API
2022-04-13 08:11:56 -04:00
681ea18c41
Only update y mxmn from L1 when last index in view
2022-04-13 08:11:56 -04:00
b02a43466f
Only update y-range from L1 mxmn when last index in view
...
We still have to always keep track of the last max and min
though.
2022-04-13 08:11:34 -04:00
229d269ae4
Drop old `pyqtgraph` downsample code
2022-04-13 08:11:34 -04:00
6f4216e102
Tweak log-scaler for more detail
2022-04-13 08:11:34 -04:00
Tyler Goodlet
Guillermo Rodriguez