Commit Graph

167 Commits (78c8a843e4fd477d2d9498d92dfe9b27603108c4)

Author SHA1 Message Date
Tyler Goodlet f872fbecf8 Hook IB up to shared memory system
Adjust the `data.open_feed()` api to take a shm token so the
broker-daemon can attach a previously created (by the parent actor) mem
buf and push real-time tick data. There's still some sloppiness here in
terms of ensuring only one mem buf per symbol (can be seen in
`stream_quotes()`) which should really managed at the data api level.
Add a bar incrementing stream-task which delivers increment msgs to any
consumers.
2020-10-02 12:13:50 -04:00
Tyler Goodlet 17491ba819 Disconnect stdlib's resource_tracker, fix .push()
Logic in `SharedArray.push()` was totally wrong.
Remove all the `multiprocessing.resource_tracker` crap such that we
aren't loading an extra process at every layer and we don't get tons of
errors when cleaning on in an SC way.
2020-10-02 12:13:50 -04:00
Tyler Goodlet 712e36b9d5 First draft of a shared numpy array sub-system
This adds a shared memory "incrementing array" sub-sys interface
for single writer, multi-reader style data passing. The main motivation
is to avoid multiple copies of the same `numpy` array across actors
(plus now we can start being fancy like ray).

There still seems to be some odd issues with the "resource tracker"
complaining at teardown (likely partially to do with SIGINT stuff) so
some further digging in the stdlib code is likely coming.

Pertains to #107 and #98
2020-10-02 12:13:50 -04:00
Tyler Goodlet 1a1e768126 Port to new data apis 2020-10-02 12:13:28 -04:00
Tyler Goodlet b7c924046a Begin to use `@tractor.msg.pub` throughout streaming API
Since the new FSP system will require time aligned data amongst actors,
it makes sense to share broker data feeds as much as possible on a local
system. There doesn't seem to be downside to this approach either since
if not fanning-out in our code, the broker (server) has to do it anyway
(and who knows how junk their implementation is) though with more
clients, sockets etc. in memory on our end. It also preps the code for
introducing a more "serious" pub-sub systems like zeromq/nanomessage.
2020-09-29 17:06:28 -04:00
goodboy 67a75c3080
Merge pull request #114 from pikers/facepalm
Add marketstore cli stuff; not sure how this got missed?!
2020-09-26 11:29:36 -04:00
Tyler Goodlet 53eb564f06 Add marketstore cli stuff; not sure how this got missed?! 2020-09-25 16:08:15 -04:00
Tyler Goodlet 05d2985f5f Clarify some odd spots 2020-09-02 11:32:54 -04:00
Tyler Goodlet 316137fdf2 Begin to wrap marketstore as a data feed
Wrap the sync client in an async interface in anticipation of an actual
async client. This starts support for the `open_fee()`/`stream_quotes()`
api though the tick normalization isn't correct yet.
2020-09-02 00:40:35 -04:00
Tyler Goodlet 702c63f607 Define "packetizer" in specific broker mod
Allows for formatting published quotes using a broker specific
formatting callback.
2020-09-02 00:36:19 -04:00
Tyler Goodlet 3c4699abef Pass broker name 2020-09-01 18:31:52 -04:00
Tyler Goodlet 78784a4bf3 Port to new data apis 2020-09-01 18:30:55 -04:00
Tyler Goodlet 57a8db8cba Start enforcing a common stream setup api
Add routines for brokerd spawning and quote stream creation.
2020-09-01 18:18:48 -04:00
Tyler Goodlet f6f6d98a95 Allow passing in tbk keys to query 2020-09-01 18:14:11 -04:00
Tyler Goodlet bc9af977a4 Update quote cache on each loop 2020-09-01 13:25:41 -04:00
Tyler Goodlet 436e4d2df4 Add tbk tick streaming with trio-websocket 2020-09-01 13:25:41 -04:00
Tyler Goodlet a6c692fb8b Add support for TICK ingest to marketstore 2020-09-01 13:25:09 -04:00