Commit Graph

2502 Commits (dpbackup)

Author SHA1 Message Date
Tyler Goodlet 6f172479eb Drop task-per-method `trio`-`asyncio` proxying
Use method proxies through the remaining endpoints and drop the old
spawn-a-task-per-method-call style helpers from module.
2022-06-04 20:44:03 -04:00
Tyler Goodlet a96f1dec3a Proxy heaven, choose one "preferred data client"
In order to expose more `asyncio` powered `Client` methods to endpoint
task-code this adds a more extensive and layered set of `MethodProxy`
loading routines, in dependency order these are:
- `load_clients_for_trio()` a `tractor.to_asyncio.open_channel_from()`
  entry-point factory for loading all scanned clients on the `asyncio` side
  and delivering them over the inter-task channel to a `trio`-side task.
- `get_preferred_data_client()` a simple client instance loading routine
  which reads from the users `brokers.toml -> `prefer_data_account:
  list[str]` which must list account names, in priority order, that are
  acceptable to be used as the main "data connection client" such that
  only one of the detected clients is used for data (whereas the rest
  are used only for order entry).
- `open_client_proxies()` which delivers the detected `Client` set
  wrapped each in a `MethodProxy`.
- `open_data_client()` which directly delivers the preferred data client
  as a proxy for `trio` tasks.
- update `open_client_method_proxy()` and `open_client_proxy` to require
  an input `Client` instance.

Further impl details:
- add `MethodProxy._aio_ns` to ref the original `asyncio` side proxied instance
- add `Client.trades()` to pull executions from the last day/session
- load proxies inside `trades_dialogue` and use the new `.trades()`
  method to try and pull a fill ledger for eventual correct pp price
  calcs (pertains to #307)..
2022-06-04 20:44:03 -04:00
goodboy 86caf5f6a3
Merge pull request #322 from pikers/dockerize_ib_gw
Dockerize `ib-gw` 🏄🏼
2022-06-04 20:42:32 -04:00
Tyler Goodlet 72b4273ddc Link to container readme 2022-06-04 20:40:36 -04:00
Tyler Goodlet 4281936ff4 Add readme for `ib-gw` container usage 2022-06-04 20:18:29 -04:00
Guillermo Rodriguez 4ddf04f68b
Merge pull request #328 from pikers/windows_tiling_fix
fix windows snap problem by removing maximum window size
2022-06-04 20:58:37 -03:00
dinkus 339fcda727 fix windows snap problem by removing maximum window size 2022-06-04 17:53:27 -04:00
Tyler Goodlet 4b7d7d688e Bind to port 3003 2022-06-03 10:22:50 -04:00
Tyler Goodlet 7ae7b2f864 Lol, bind vnc server to localhost only 2022-06-03 10:21:56 -04:00
Tyler Goodlet fa9f8c78c3 Only bind IBC command server to localhost 2022-06-03 10:21:37 -04:00
Tyler Goodlet 3bbbc21d2b Comment unneeded port map for now 2022-06-03 10:21:29 -04:00
Tyler Goodlet b03603a6b4 Drop password auth from vnc server
Currently we're held back by an `asyncvnc` issue,
https://github.com/barneygale/asyncvnc/issues/1 but even still, given
we're running the container to be only accessible by localhost i'm not
sure we need this for the moment (or at all) anyway.
2022-05-24 09:29:35 -04:00
Tyler Goodlet 81b77df544 Flip tz to NY, add note about .env file 2022-05-24 09:29:35 -04:00
Tyler Goodlet a79a99fc71 Add working, template docker setup for `ib-gw`
Based on the now defunct project @
https://github.com/waytrade/ib-gateway-docker

Adds a `docker-compose.yml` and necessary gateway and `IBC` config
files to make it possible to spin up a local gateway on localhost:4002
and connect to it without issue using `ib_insync`.

Next up, we'll want to,
- automated the equivalent docker-compose steps using our
  `.data._ahab` supervisor system
- probably simplify and roll our own container (likely alpine or nixos
  based) which drops uneeded deps (`socat`, vnc) and adds `xdotool`.
- allow for API socket mapping to just be pulled direct from
  a user's `brokers.toml` and we'll just pass that direct to `IBC`'s
  config.
2022-05-21 14:22:11 -04:00
goodboy 9f47515f59
Merge pull request #320 from pikers/drop_pandas
Drop `pandas`
2022-05-15 13:57:06 -04:00
Tyler Goodlet 09f2f32d5b Drop `pandas` timestamp for qt 2022-05-15 13:49:54 -04:00
Tyler Goodlet e718120cc7 Drop `pandas` as dep 2022-05-15 13:49:54 -04:00
Tyler Goodlet fb5df5ab5e Drop `pandas` usage throughout brokers cli 2022-05-15 13:49:50 -04:00
Tyler Goodlet 6e2e2fc03f Use `pendulum` for timestamp parsing 2022-05-15 13:45:44 -04:00
Tyler Goodlet a3b2ba9ae9 Use `numpy.datetime64` for x-axis tick strings 2022-05-15 13:45:37 -04:00
goodboy 7083c5a0bd
Merge pull request #319 from pikers/no_ib_pps
No ib pps? Account names should still load.
2022-05-13 16:26:46 -04:00
Tyler Goodlet de55565f60 We already collect account values/names in `load_io_clients()` 2022-05-12 14:21:31 -04:00
Tyler Goodlet d0530c4e26 Deliver accounts from query instead of just pps with `ib` 2022-05-12 13:22:51 -04:00
goodboy 21b16b4a9e
Merge pull request #318 from pikers/trimeter_dep
Add `trimeter` dep.. that we forgot
2022-05-11 16:28:29 -04:00
Tyler Goodlet ed85079d0f Add `trimeter` dep.. that we forgot 2022-05-11 13:19:47 -04:00
goodboy 5b540a53e9
Merge pull request #317 from pikers/l1_precision_fix
Convert `binance` tick/lot step sizes to `float`
2022-05-11 11:17:43 -04:00
Tyler Goodlet fb91e27651 Well that was easy, convert tick/lot step sizes to `float` 2022-05-11 10:41:03 -04:00
goodboy 482fc1da10
Merge pull request #308 from pikers/storage_layer
Storage layer: initial `marketstore` tsdb support with async OHLCV history loading.
2022-05-11 10:18:33 -04:00
Tyler Goodlet b3f9c4f93d Only assert if input array actually has a size 2022-05-10 17:59:24 -04:00
Tyler Goodlet 09431aad85 Add support for no `._first.value` update shm prepends 2022-05-10 17:59:16 -04:00
Tyler Goodlet 8219307bf5 Double up shm buffer size 2022-05-10 17:59:08 -04:00
Tyler Goodlet b910eceb3b Add `ShmArray.ustruct()`: return an unstructured array copy
We return a copy (since since a view doesn't seem to work..) of the
(field filtered) shm array contents which is the same index-length as
the source data.

Further, fence off the resource tracker disable-hack into a helper
routine.
2022-05-10 17:58:57 -04:00
Tyler Goodlet 1657f51edc Manually fetch missing out-of-order history frames
It seems once in a while a frame can get missed or dropped (at least
with binance?) so in those cases, when the request erlangs is already at
max, we just manually request the missing frame and presume things will
work out XD

Further, discard out of order frames that are "from the future" that
somehow end up in the async queue once in a while? Not sure why this
happens but it seems thus far just discarding them is nbd.
2022-05-10 17:25:20 -04:00
Tyler Goodlet b1246446c2 Raise error on 'fatal' and 'error' log levels 2022-05-10 17:25:20 -04:00
Tyler Goodlet 083a3296e7 Better formatted startup logging output 2022-05-10 14:55:52 -04:00
Tyler Goodlet 769e803695 Write `mkts.yml` from template if one dne 2022-05-10 14:55:52 -04:00
Tyler Goodlet e196e9d1a0 Factor `marketstore` container specifics into `piker.data.marketstore` 2022-05-10 14:55:52 -04:00
Tyler Goodlet 9ddfae44d2 Parametrize and deliver (relevant) mkts config in `start_ahab()` 2022-05-10 14:55:52 -04:00
Tyler Goodlet 277ca29018 Always write missing history frames to tsdb (again) 2022-05-10 14:55:52 -04:00
Tyler Goodlet 26fddae3c0 Fix earliest frame-end not-yet-pushed check
Bleh/🤦, the ``end_dt`` in scope is not the "earliest" frame's
`end_dt` in the async response queue.. Parse the queue's latest epoch
and use **that** to compare to the last last pushed datetime index..

Add more detailed logging to help debug any (un)expected datetime index
gaps.
2022-05-10 14:55:52 -04:00
Tyler Goodlet 4b6ecbfc79 Bring binance requests down to 3/sec; seems faster? 2022-05-10 14:55:52 -04:00
Tyler Goodlet 30ddf63ec0 Handle gaps greater then a frame within a frame 2022-05-09 11:15:14 -04:00
Tyler Goodlet 8e08fb7b23 Add comment about un-reffed vars meant for use in shell 2022-05-09 11:15:14 -04:00
Tyler Goodlet fb9b6990ae Drop unneeded/commented cancel-by-msg code; roots perms wasn't the problem 2022-05-09 11:15:14 -04:00
Tyler Goodlet 1676bceee1 Don't offset the start index by a step 2022-05-09 11:15:14 -04:00
Tyler Goodlet c9a621fc2a Fix less-then-frame off by one slice, add db write toggle and disable 2022-05-09 11:15:14 -04:00
Tyler Goodlet 0324404b03 Include epoch timestamp in quote label for now 2022-05-09 11:15:14 -04:00
Tyler Goodlet 61e9db3229 Handle ``iter_dts()`` already exhausted edge case 2022-05-09 11:15:14 -04:00
Tyler Goodlet 4a6f01747c Label "humanized" sample period in window title-bar" 2022-05-09 11:15:14 -04:00
Tyler Goodlet e4a900168d Add timeframe key to seconds map 2022-05-09 11:15:14 -04:00