Add note about appending parquet files on write
parent
f8ab3bde35
commit
c1546eb043
|
@ -296,7 +296,11 @@ class NativeStorageClient:
|
||||||
else:
|
else:
|
||||||
df = ohlcv
|
df = ohlcv
|
||||||
|
|
||||||
# TODO: use a proper profiler
|
# TODO: in terms of managing the ultra long term data
|
||||||
|
# - use a proper profiler to measure all this IO and
|
||||||
|
# roundtripping!
|
||||||
|
# - try out ``fastparquet``'s append writing:
|
||||||
|
# https://fastparquet.readthedocs.io/en/latest/api.html#fastparquet.write
|
||||||
start = time.time()
|
start = time.time()
|
||||||
df.write_parquet(path)
|
df.write_parquet(path)
|
||||||
delay: float = round(
|
delay: float = round(
|
||||||
|
|
Loading…
Reference in New Issue