Commit Graph

95 Commits (e54d928405dbbbb8b18b75b988cc59e68c30f795)

Author SHA1 Message Date
jaredgoldman e54d928405 Reformat fake fill in paper engine,
Ensure tests pass, refactor test wrapper
2023-02-25 18:59:40 -05:00
jaredgoldman c99381216d Ensure actual pp is sent to ems
ensure not to write pp header on startup

Comment out pytest settings
Add comments explaining delete_testing_dir fixture
use nonlocal instead of global for test state

Add unpacking get_fqsn method
Format test_paper
Add comments explaining sync/async book.send calls
2023-02-25 18:59:40 -05:00
algorandpa 7bd8019876 Add back cleanup fixture 2023-02-25 18:59:39 -05:00
algorandpa 8122e6c86f Disable cleanup to see if CI passes 2023-02-25 18:59:39 -05:00
algorandpa 7e87dc52eb Scope fixture to session 2023-02-25 18:59:39 -05:00
algorandpa 2c366d7349 Fix type 2023-02-25 18:59:39 -05:00
algorandpa 9acbfacd4c only clean up if _testing file exists 2023-02-25 18:59:39 -05:00
algorandpa 316ead577d Remove scoping 2023-02-25 18:59:39 -05:00
algorandpa 4b6d3fe138 Scope cleanup fixture to module 2023-02-25 18:59:39 -05:00
algorandpa 0dec2b9c89 Enable backpressure during data-feed layer startup to avoid overruns 2023-02-25 18:59:39 -05:00
algorandpa e5cefeb44b Format to prep for PR 2023-02-25 18:59:39 -05:00
algorandpa 7142a6a7ca Add hacky cleanup solution for _testng data 2023-02-25 18:59:39 -05:00
algorandpa dff8abd6ad Minor reformatting 2023-02-25 18:59:39 -05:00
algorandpa 95b9dacb7a Break test into steps 2023-02-25 18:59:39 -05:00
algorandpa df868cec35 Assert that trades persist in ems after teardown and startup 2023-02-25 18:59:39 -05:00
algorandpa 84cd1e0059 initial commit on copy 2023-02-25 18:59:39 -05:00
algorandpa 86b4386522 minor changes, prepare for rebase of overlays branch 2023-02-25 18:59:39 -05:00
Guillermo Rodriguez 47bf45f30e
Merge pull request #464 from pikers/elasticsearch_integration
Elasticsearch integration
2023-02-24 16:38:37 -03:00
Esmeralda Gallardo b96e2c314a
Minor style changes and removed unnecesary comments 2023-02-24 15:11:15 -03:00
Esmeralda Gallardo f96d6a04b6
Fixed UnboundLocalError on _ahab. Added test for marketstore's initialization 2023-02-22 13:28:07 -03:00
Guillermo Rodriguez acc6249d88
Remove unnesesary arguments to some pikerd functions, fix container init error
by switching from log reading to quering es health endpoint, fix install on ci
and add more logging.
2023-02-21 20:45:10 -03:00
Tyler Goodlet 0b678c97f4 Pass `loglevel: str` cli value through to service tests 2023-02-21 12:02:26 -05:00
Esmeralda Gallardo 4122c482ba
Added new tests for elasticsearch's and marketstore's initialization and stop 2023-02-21 13:34:29 -03:00
Esmeralda Gallardo b5cdf14036
Modified elasticsearch file name to 'elastic' to avoid name errors. Applied changes suggested in the pr. 2023-02-21 13:34:29 -03:00
Tyler Goodlet b3400f0d9c Add `loglevel: str` fixture, passthrough to `open_test_pikerd()` 2023-02-21 10:54:18 -05:00
Tyler Goodlet cd3e9b1b2a Move quest fixtures to test mod, clean out old travis fixture 2023-02-21 10:54:18 -05:00
Tyler Goodlet 69074f4fa5 Bump up service tree spawn timeout a couple secs 2023-01-26 17:59:25 -05:00
Tyler Goodlet fe4fb37b58 Add service tree tests for data-feeds and the EMS 2023-01-24 15:15:27 -05:00
Tyler Goodlet 7cfd431a2b Yield `Services` in `open_test_pikerd()` fixture 2023-01-24 15:15:27 -05:00
Tyler Goodlet 37e0ec7b7d Assert fixture caller is `pikerd` 2023-01-13 13:21:49 -05:00
Tyler Goodlet 6a1bb13feb Add base `pikerd` service tree custom check test 2023-01-13 13:21:49 -05:00
Tyler Goodlet 593db0ed0d Only run `kraken` feed tests in CI, use `open_test_pikerd()` 2023-01-10 11:09:19 -05:00
Tyler Goodlet 06622105cd Add a `open_test_pikerd()` acm fixture for easy booting of the service stack 2023-01-10 11:09:19 -05:00
Tyler Goodlet f232d6d4ee Add `ci_env` detector fixture 2023-01-10 11:09:19 -05:00
Tyler Goodlet b7e1443618 Use ETH on kraken to ensure enough quotes 2023-01-10 11:09:19 -05:00
Tyler Goodlet 5d021ffb85 Bump up timeout on multi-feed test for CI 2023-01-10 11:09:19 -05:00
Tyler Goodlet e348968113 Add multi-broker streaming test using both `binance` and `kraken` 2023-01-10 11:09:19 -05:00
Tyler Goodlet 36868bb86e Add `kraken` test, ensure single broker-provider for now 2023-01-10 11:09:19 -05:00
Tyler Goodlet bb6452b969 Further feed syncing fixes wrt to `Flumes`
Sync per-symbol sampler loop start to subscription registers such that
the loop can't start until the consumer's stream subscription is added;
the task-sync uses a `trio.Event`. This patch also drops a ton of
commented cruft.

Further adjustments needed to get parity with prior functionality:
- pass init msg 'symbol_info' field to the `Symbol.broker_info: dict`.
- ensure the `_FeedsBus._subscriptions` table uses the broker specific
  (without brokername suffix) as keys for lookup so that the sampler
  loop doesn't have to append in the brokername as a suffix.
- ensure the `open_feed_bus()` flumes-table-msg returned sent by
  `tractor.Context.started()` uses the `.to_msg()` form of all flume
  structs.
- ensure `maybe_open_feed()` uses `tractor.MsgStream.subscribe()` on all
  `Flume.stream`s on cache hits using the
  `tractor.trionics.gather_contexts()` helper.
2023-01-10 11:09:19 -05:00
Tyler Goodlet 32b36aa042 Expect init startup quotes from each symbol 2023-01-10 11:09:19 -05:00
Tyler Goodlet 18dc8b08e4 First draft aggregate feedz support
Orient shm-flow-arrays around the new idea of a `Flume` which provides
access, mgmt and basic measure of real-time data flow sets (see water
flow management semantics).

- We discard the previous idea of a "init message" which contained all
  the shm attachment info and instead send a startup message full of
  `Flume.to_msg()`s which are symmetrically loaded on the caller actor
  side.

- Create data-flows "entries" for every passed in fqsn such that the consumer gets back
  streams and shm for each, now all wrapped in `Flume` types. For now we
  allocate `brokermod.stream_quotes()` tasks 1-to-1 for each fqsn
  (instead of expecting each backend to do multi-plexing, though we
  might want that eventually) as well a `_FeedsBus._subscriber` entry
  for each. The pause/resume management loop is adjusted to match.
  Previously `Feed`s were  allocated 1-to-1 with each fqsn.

- Make `Feed` a `Struct` subtype instead of a `@dataclass` and move all
  flow specific attrs to the new `Flume`:
  - move `.index_stream()`, `.get_ds_info()` to `Flume`.
  - drop `.receive()`: each fqsn entry will now require knowledge of
    separate streams by feed users.
  - add multi-fqsn tables: `.flumes`, `.streams` which point to the
    appropriate per-symbol entries.

- Async load all `Flume`s from all contexts and all quote streams using
  `tractor.trionics.gather_contexts()` on the client `open_feed()` side.

- Update feeds test to include streaming 2 symbols on the same (binance)
  backend.
2023-01-10 11:09:18 -05:00
Tyler Goodlet c7d5db5f90 Start data feed layer test suite
Initial test that starts a `binance` feed and reads the quote messages
alongside shm buffers for 1s and 1m OHLC; just prints to console for
now.

Template out parametrization for multi-symbol quote-multiplexed feeds
which coming soon B)
2023-01-10 11:09:18 -05:00
Tyler Goodlet 1bf1965a8b Drop `tractor.log` level override fixture 2023-01-10 11:09:18 -05:00
Tyler Goodlet 63e9af002d Drop `tractor.testing` import in qt tests 2022-10-28 16:09:55 -04:00
Tyler Goodlet 2312b6aeb2 Fix conftest config mod import 2021-09-11 18:15:42 -04:00
Tyler Goodlet 088ea8c3ec Skip CLI tests due to questrade dependency for now 2021-05-22 15:12:12 -04:00
Tyler Goodlet 0228db7811 Skip questrade tests for now
They need to be run with a local private API key and haven't been ported
to latest data apis anyway. It's also a broker most peeps aren't going
to be using any time soon.
2021-05-20 16:11:49 -04:00
Tyler Goodlet 44eb9aa9ff Create a blank config if none exists 2019-03-18 00:04:34 -04:00
Tyler Goodlet 88bb5b9f05 Handle file not found 2019-03-17 23:22:00 -04:00
Tyler Goodlet b99c5d2abe Use confdir if provided even when in CI 2019-03-17 23:04:04 -04:00