wehub-resource-sync
957 lines
73 KiB
Plaintext
957 lines
73 KiB
Plaintext
{
|
||
"cells": [
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "d75febdb",
|
||
"metadata": {
|
||
"papermill": {
|
||
"duration": 0.001339,
|
||
"end_time": "2026-06-13T02:34:30.800665+00:00",
|
||
"exception": false,
|
||
"start_time": "2026-06-13T02:34:30.799326+00:00",
|
||
"status": "completed"
|
||
}
|
||
},
|
||
"source": [
|
||
"# Incremental Updates: Keeping Market Data Fresh\n",
|
||
"\n",
|
||
"**Docker image**: `ml4t`\n",
|
||
"\n",
|
||
"**Purpose**: Walk through the update lifecycle — initial load, daily delta,\n",
|
||
"gap detection, and a small health dashboard — so the same flow that runs\n",
|
||
"in `data/download_all.py --update` is visible end-to-end.\n",
|
||
"\n",
|
||
"**Learning objectives**:\n",
|
||
"1. Compare full-refresh and incremental update costs.\n",
|
||
"2. Drive a daily delta with `DataManager.update(..., fill_gaps=False)` and\n",
|
||
" understand why the `fill_gaps=True` default is unsafe for OHLCV data.\n",
|
||
"3. Verify completeness with `GapDetector(exclude_weekends=True)`.\n",
|
||
"4. Render a 3-panel health dashboard (volume, freshness, issues).\n",
|
||
"\n",
|
||
"**Book reference**: §2.4 (storing data) — operational counterpart to the\n",
|
||
"storage benchmarks in notebooks 20 and 21.\n",
|
||
"\n",
|
||
"**Prerequisites**: ml4t-data installed; live network access for Yahoo Finance.\n",
|
||
"\n",
|
||
"**Why incremental updates**: a full refresh of 500 symbols × 10 years pulls\n",
|
||
"~1.26M rows; an incremental run pulls roughly the trading days since the\n",
|
||
"last fetch. The gap is two orders of magnitude in network and disk I/O."
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "a6d7ef1e",
|
||
"metadata": {
|
||
"papermill": {
|
||
"duration": 0.000984,
|
||
"end_time": "2026-06-13T02:34:30.803608+00:00",
|
||
"exception": false,
|
||
"start_time": "2026-06-13T02:34:30.802624+00:00",
|
||
"status": "completed"
|
||
}
|
||
},
|
||
"source": [
|
||
"## Setup"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 1,
|
||
"id": "58b5c9a4",
|
||
"metadata": {
|
||
"execution": {
|
||
"iopub.execute_input": "2026-06-13T02:34:30.806393Z",
|
||
"iopub.status.busy": "2026-06-13T02:34:30.806285Z",
|
||
"iopub.status.idle": "2026-06-13T02:34:33.691392Z",
|
||
"shell.execute_reply": "2026-06-13T02:34:33.690954Z"
|
||
},
|
||
"lines_to_next_cell": 2,
|
||
"papermill": {
|
||
"duration": 2.887432,
|
||
"end_time": "2026-06-13T02:34:33.691941+00:00",
|
||
"exception": false,
|
||
"start_time": "2026-06-13T02:34:30.804509+00:00",
|
||
"status": "completed"
|
||
}
|
||
},
|
||
"outputs": [
|
||
{
|
||
"name": "stderr",
|
||
"output_type": "stream",
|
||
"text": [
|
||
".venv/lib/python3.14/site-packages/kaleido/scopes/plotly.py:32: DeprecationWarning: \n",
|
||
"Use of plotly.io.kaleido.scope.default_format is deprecated and support will be removed after September 2025.\n",
|
||
"Please use plotly.io.defaults.default_format instead.\n",
|
||
"\n",
|
||
" self.default_format = \"png\"\n",
|
||
".venv/lib/python3.14/site-packages/kaleido/scopes/plotly.py:33: DeprecationWarning: \n",
|
||
"Use of plotly.io.kaleido.scope.default_width is deprecated and support will be removed after September 2025.\n",
|
||
"Please use plotly.io.defaults.default_width instead.\n",
|
||
"\n",
|
||
" self.default_width = 700\n",
|
||
".venv/lib/python3.14/site-packages/kaleido/scopes/plotly.py:34: DeprecationWarning: \n",
|
||
"Use of plotly.io.kaleido.scope.default_height is deprecated and support will be removed after September 2025.\n",
|
||
"Please use plotly.io.defaults.default_height instead.\n",
|
||
"\n",
|
||
" self.default_height = 500\n",
|
||
".venv/lib/python3.14/site-packages/kaleido/scopes/plotly.py:35: DeprecationWarning: \n",
|
||
"Use of plotly.io.kaleido.scope.default_scale is deprecated and support will be removed after September 2025.\n",
|
||
"Please use plotly.io.defaults.default_scale instead.\n",
|
||
"\n",
|
||
" self.default_scale = 1\n",
|
||
".venv/lib/python3.14/site-packages/plotly/io/_kaleido.py:133: DeprecationWarning: \n",
|
||
"Use of plotly.io.kaleido.scope.mathjax is deprecated and support will be removed after September 2025.\n",
|
||
"Please use plotly.io.defaults.mathjax instead.\n",
|
||
"\n",
|
||
" setattr(self._scope, name, value)\n",
|
||
".venv/lib/python3.14/site-packages/plotly/io/_kaleido.py:141: DeprecationWarning: \n",
|
||
"Use of plotly.io.kaleido.scope.plotlyjs is deprecated and support will be removed after September 2025.\n",
|
||
"Please use plotly.io.defaults.plotlyjs instead.\n",
|
||
"\n",
|
||
" scope.plotlyjs = os.path.join(package_dir, \"plotly.min.js\")\n",
|
||
".venv/lib/python3.14/site-packages/plotly/io/_kaleido.py:147: DeprecationWarning: \n",
|
||
"Use of plotly.io.kaleido.scope.mathjax is deprecated and support will be removed after September 2025.\n",
|
||
"Please use plotly.io.defaults.mathjax instead.\n",
|
||
"\n",
|
||
" scope.mathjax = (\n"
|
||
]
|
||
},
|
||
{
|
||
"name": "stdout",
|
||
"output_type": "stream",
|
||
"text": [
|
||
"Demo storage: 02_financial_data_universe/output/incremental_updates\n"
|
||
]
|
||
}
|
||
],
|
||
"source": [
|
||
"\"\"\"Incremental Updates — Keeping market data fresh with daily delta fetches.\"\"\"\n",
|
||
"\n",
|
||
"import logging\n",
|
||
"import shutil\n",
|
||
"from datetime import datetime\n",
|
||
"\n",
|
||
"import matplotlib.pyplot as plt\n",
|
||
"import polars as pl\n",
|
||
"import structlog\n",
|
||
"\n",
|
||
"# Quiet ml4t-data's structured logger so the notebook focuses on demo output.\n",
|
||
"structlog.configure(\n",
|
||
" wrapper_class=structlog.make_filtering_bound_logger(logging.WARNING),\n",
|
||
")\n",
|
||
"\n",
|
||
"from ml4t.data import DataManager\n",
|
||
"from ml4t.data.storage import HiveStorage\n",
|
||
"from ml4t.data.storage.backend import StorageConfig\n",
|
||
"from ml4t.data.update_manager import GapDetector\n",
|
||
"from ml4t.data.validation import OHLCVValidator\n",
|
||
"\n",
|
||
"from utils.paths import get_output_dir\n",
|
||
"from utils.style import COLORS\n",
|
||
"\n",
|
||
"# Storage for this notebook's demos. Wipe any prior-run artifacts so the\n",
|
||
"# initial-load + update sequence is reproducible.\n",
|
||
"DEMO_DIR = get_output_dir(2, \"incremental_updates\")\n",
|
||
"if DEMO_DIR.exists():\n",
|
||
" shutil.rmtree(DEMO_DIR)\n",
|
||
"DEMO_DIR.mkdir(parents=True, exist_ok=True)\n",
|
||
"\n",
|
||
"print(f\"Demo storage: {DEMO_DIR}\")"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 2,
|
||
"id": "c74fa438",
|
||
"metadata": {
|
||
"execution": {
|
||
"iopub.execute_input": "2026-06-13T02:34:33.696140Z",
|
||
"iopub.status.busy": "2026-06-13T02:34:33.696046Z",
|
||
"iopub.status.idle": "2026-06-13T02:34:33.697709Z",
|
||
"shell.execute_reply": "2026-06-13T02:34:33.697369Z"
|
||
},
|
||
"papermill": {
|
||
"duration": 0.004322,
|
||
"end_time": "2026-06-13T02:34:33.698137+00:00",
|
||
"exception": false,
|
||
"start_time": "2026-06-13T02:34:33.693815+00:00",
|
||
"status": "completed"
|
||
},
|
||
"tags": [
|
||
"parameters"
|
||
]
|
||
},
|
||
"outputs": [],
|
||
"source": [
|
||
"# Production defaults — Papermill injects overrides for CI"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "f2508be6",
|
||
"metadata": {
|
||
"papermill": {
|
||
"duration": 0.001684,
|
||
"end_time": "2026-06-13T02:34:33.701572+00:00",
|
||
"exception": false,
|
||
"start_time": "2026-06-13T02:34:33.699888+00:00",
|
||
"status": "completed"
|
||
}
|
||
},
|
||
"source": [
|
||
"---\n",
|
||
"\n",
|
||
"## 1. Full Refresh vs Incremental Update\n",
|
||
"\n",
|
||
"Let's demonstrate the difference with a concrete example."
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "112dd303",
|
||
"metadata": {
|
||
"papermill": {
|
||
"duration": 0.001402,
|
||
"end_time": "2026-06-13T02:34:33.704649+00:00",
|
||
"exception": false,
|
||
"start_time": "2026-06-13T02:34:33.703247+00:00",
|
||
"status": "completed"
|
||
}
|
||
},
|
||
"source": [
|
||
"### Step 1: Initial Load (Full History)"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 3,
|
||
"id": "718e33e0",
|
||
"metadata": {
|
||
"execution": {
|
||
"iopub.execute_input": "2026-06-13T02:34:33.707093Z",
|
||
"iopub.status.busy": "2026-06-13T02:34:33.707021Z",
|
||
"iopub.status.idle": "2026-06-13T02:34:34.207175Z",
|
||
"shell.execute_reply": "2026-06-13T02:34:34.206645Z"
|
||
},
|
||
"papermill": {
|
||
"duration": 0.501841,
|
||
"end_time": "2026-06-13T02:34:34.207449+00:00",
|
||
"exception": false,
|
||
"start_time": "2026-06-13T02:34:33.705608+00:00",
|
||
"status": "completed"
|
||
}
|
||
},
|
||
"outputs": [
|
||
{
|
||
"name": "stdout",
|
||
"output_type": "stream",
|
||
"text": [
|
||
"=== Initial Load (Full History) ===\n"
|
||
]
|
||
},
|
||
{
|
||
"name": "stdout",
|
||
"output_type": "stream",
|
||
"text": [
|
||
" AAPL: stored (502 rows)\n",
|
||
" MSFT: stored (502 rows)\n"
|
||
]
|
||
},
|
||
{
|
||
"name": "stdout",
|
||
"output_type": "stream",
|
||
"text": [
|
||
" GOOGL: stored (502 rows)\n"
|
||
]
|
||
}
|
||
],
|
||
"source": [
|
||
"# Initialize storage and DataManager\n",
|
||
"config = StorageConfig(base_path=DEMO_DIR / \"updates_demo\", compression=\"zstd\")\n",
|
||
"storage = HiveStorage(config=config)\n",
|
||
"dm = DataManager(storage=storage)\n",
|
||
"\n",
|
||
"symbols = [\"AAPL\", \"MSFT\", \"GOOGL\"]\n",
|
||
"\n",
|
||
"print(\"=== Initial Load (Full History) ===\")\n",
|
||
"for symbol in symbols:\n",
|
||
" dm.load(symbol, \"2023-01-01\", \"2024-12-31\", provider=\"yahoo\")\n",
|
||
" meta = dm.get_metadata(symbol)\n",
|
||
" print(f\" {symbol}: stored ({meta['row_count']} rows)\")"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "7302cebe",
|
||
"metadata": {
|
||
"papermill": {
|
||
"duration": 0.001019,
|
||
"end_time": "2026-06-13T02:34:34.209620+00:00",
|
||
"exception": false,
|
||
"start_time": "2026-06-13T02:34:34.208601+00:00",
|
||
"status": "completed"
|
||
}
|
||
},
|
||
"source": [
|
||
"### Step 2: Incremental Update (Only New Data)\n",
|
||
"\n",
|
||
"`DataManager.update()` reads the last timestamp in storage, refetches a small\n",
|
||
"overlap (`lookback_days=7`) plus everything since, and merges. We pass\n",
|
||
"`fill_gaps=False` deliberately — the library default `fill_gaps=True` runs a\n",
|
||
"calendar-unaware gap detector that treats every weekend and US holiday as a\n",
|
||
"missing trading day and forward-fills it, silently inflating each symbol's\n",
|
||
"row count by hundreds of phantom bars. For OHLCV data, the right behavior is\n",
|
||
"to leave non-trading days absent and rely on a calendar-aware completeness\n",
|
||
"check downstream."
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 4,
|
||
"id": "51847602",
|
||
"metadata": {
|
||
"execution": {
|
||
"iopub.execute_input": "2026-06-13T02:34:34.212302Z",
|
||
"iopub.status.busy": "2026-06-13T02:34:34.212190Z",
|
||
"iopub.status.idle": "2026-06-13T02:34:34.646493Z",
|
||
"shell.execute_reply": "2026-06-13T02:34:34.646013Z"
|
||
},
|
||
"papermill": {
|
||
"duration": 0.436157,
|
||
"end_time": "2026-06-13T02:34:34.646776+00:00",
|
||
"exception": false,
|
||
"start_time": "2026-06-13T02:34:34.210619+00:00",
|
||
"status": "completed"
|
||
}
|
||
},
|
||
"outputs": [
|
||
{
|
||
"name": "stdout",
|
||
"output_type": "stream",
|
||
"text": [
|
||
"=== Incremental Update ===\n",
|
||
" AAPL: updated (864 rows)\n"
|
||
]
|
||
},
|
||
{
|
||
"name": "stdout",
|
||
"output_type": "stream",
|
||
"text": [
|
||
" MSFT: updated (864 rows)\n",
|
||
" GOOGL: updated (864 rows)\n"
|
||
]
|
||
}
|
||
],
|
||
"source": [
|
||
"print(\"=== Incremental Update ===\")\n",
|
||
"for symbol in symbols:\n",
|
||
" dm.update(symbol, lookback_days=7, provider=\"yahoo\", fill_gaps=False)\n",
|
||
" meta = dm.get_metadata(symbol)\n",
|
||
" print(f\" {symbol}: updated ({meta['row_count']} rows)\")"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "becf8020",
|
||
"metadata": {
|
||
"papermill": {
|
||
"duration": 0.001051,
|
||
"end_time": "2026-06-13T02:34:34.649006+00:00",
|
||
"exception": false,
|
||
"start_time": "2026-06-13T02:34:34.647955+00:00",
|
||
"status": "completed"
|
||
}
|
||
},
|
||
"source": [
|
||
"Each symbol started with 502 rows (2023-01 through 2024-12). The update\n",
|
||
"fetches the 7-day overlap plus all sessions through today and merges; the\n",
|
||
"resulting row count equals the original 502 plus exactly the new trading\n",
|
||
"sessions — no synthetic weekend/holiday rows."
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "405ebd2f",
|
||
"metadata": {
|
||
"papermill": {
|
||
"duration": 0.000965,
|
||
"end_time": "2026-06-13T02:34:34.650988+00:00",
|
||
"exception": false,
|
||
"start_time": "2026-06-13T02:34:34.650023+00:00",
|
||
"status": "completed"
|
||
}
|
||
},
|
||
"source": [
|
||
"---\n",
|
||
"\n",
|
||
"## 2. Update Strategies\n",
|
||
"\n",
|
||
"ml4t-data supports four strategies for different scenarios:"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 5,
|
||
"id": "838761e4",
|
||
"metadata": {
|
||
"execution": {
|
||
"iopub.execute_input": "2026-06-13T02:34:34.653707Z",
|
||
"iopub.status.busy": "2026-06-13T02:34:34.653578Z",
|
||
"iopub.status.idle": "2026-06-13T02:34:34.658034Z",
|
||
"shell.execute_reply": "2026-06-13T02:34:34.657593Z"
|
||
},
|
||
"papermill": {
|
||
"duration": 0.006275,
|
||
"end_time": "2026-06-13T02:34:34.658285+00:00",
|
||
"exception": false,
|
||
"start_time": "2026-06-13T02:34:34.652010+00:00",
|
||
"status": "completed"
|
||
}
|
||
},
|
||
"outputs": [
|
||
{
|
||
"data": {
|
||
"text/html": [
|
||
"<div><style>\n",
|
||
".dataframe > thead > tr,\n",
|
||
".dataframe > tbody > tr {\n",
|
||
" text-align: right;\n",
|
||
" white-space: pre-wrap;\n",
|
||
"}\n",
|
||
"</style>\n",
|
||
"<small>shape: (4, 3)</small><table border=\"1\" class=\"dataframe\"><thead><tr><th>strategy</th><th>behavior</th><th>use_case</th></tr><tr><td>str</td><td>str</td><td>str</td></tr></thead><tbody><tr><td>"INCREMENTAL"</td><td>"Fetch data after last stored t…</td><td>"Daily updates (default, fastes…</td></tr><tr><td>"APPEND_ONLY"</td><td>"Add new rows; never modify exi…</td><td>"Audit-safe archives"</td></tr><tr><td>"FULL_REFRESH"</td><td>"Re-download and replace all da…</td><td>"Recovery after corruption"</td></tr><tr><td>"BACKFILL"</td><td>"Fetch missing periods inside e…</td><td>"Patch holes in historical data"</td></tr></tbody></table></div>"
|
||
],
|
||
"text/plain": [
|
||
"shape: (4, 3)\n",
|
||
"┌──────────────┬─────────────────────────────────┬─────────────────────────────────┐\n",
|
||
"│ strategy ┆ behavior ┆ use_case │\n",
|
||
"│ --- ┆ --- ┆ --- │\n",
|
||
"│ str ┆ str ┆ str │\n",
|
||
"╞══════════════╪═════════════════════════════════╪═════════════════════════════════╡\n",
|
||
"│ INCREMENTAL ┆ Fetch data after last stored t… ┆ Daily updates (default, fastes… │\n",
|
||
"│ APPEND_ONLY ┆ Add new rows; never modify exi… ┆ Audit-safe archives │\n",
|
||
"│ FULL_REFRESH ┆ Re-download and replace all da… ┆ Recovery after corruption │\n",
|
||
"│ BACKFILL ┆ Fetch missing periods inside e… ┆ Patch holes in historical data │\n",
|
||
"└──────────────┴─────────────────────────────────┴─────────────────────────────────┘"
|
||
]
|
||
},
|
||
"execution_count": 5,
|
||
"metadata": {},
|
||
"output_type": "execute_result"
|
||
}
|
||
],
|
||
"source": [
|
||
"pl.DataFrame(\n",
|
||
" {\n",
|
||
" \"strategy\": [\"INCREMENTAL\", \"APPEND_ONLY\", \"FULL_REFRESH\", \"BACKFILL\"],\n",
|
||
" \"behavior\": [\n",
|
||
" \"Fetch data after last stored timestamp\",\n",
|
||
" \"Add new rows; never modify existing\",\n",
|
||
" \"Re-download and replace all data\",\n",
|
||
" \"Fetch missing periods inside existing range\",\n",
|
||
" ],\n",
|
||
" \"use_case\": [\n",
|
||
" \"Daily updates (default, fastest)\",\n",
|
||
" \"Audit-safe archives\",\n",
|
||
" \"Recovery after corruption\",\n",
|
||
" \"Patch holes in historical data\",\n",
|
||
" ],\n",
|
||
" }\n",
|
||
")"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "b35ea6de",
|
||
"metadata": {
|
||
"papermill": {
|
||
"duration": 0.001062,
|
||
"end_time": "2026-06-13T02:34:34.660471+00:00",
|
||
"exception": false,
|
||
"start_time": "2026-06-13T02:34:34.659409+00:00",
|
||
"status": "completed"
|
||
}
|
||
},
|
||
"source": [
|
||
"The `IncrementalUpdater` class provides low-level control over these strategies.\n",
|
||
"For most use cases, `DataManager.update()` (which uses `INCREMENTAL`) is sufficient."
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "bc22f2bb",
|
||
"metadata": {
|
||
"papermill": {
|
||
"duration": 0.001013,
|
||
"end_time": "2026-06-13T02:34:34.662545+00:00",
|
||
"exception": false,
|
||
"start_time": "2026-06-13T02:34:34.661532+00:00",
|
||
"status": "completed"
|
||
}
|
||
},
|
||
"source": [
|
||
"---\n",
|
||
"\n",
|
||
"## 3. Gap Detection\n",
|
||
"\n",
|
||
"Before running a backtest, verify that your data is complete.\n",
|
||
"Gaps can occur from failed downloads, provider outages, or holiday handling."
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 6,
|
||
"id": "d3fde1a9",
|
||
"metadata": {
|
||
"execution": {
|
||
"iopub.execute_input": "2026-06-13T02:34:34.665227Z",
|
||
"iopub.status.busy": "2026-06-13T02:34:34.665158Z",
|
||
"iopub.status.idle": "2026-06-13T02:34:34.683454Z",
|
||
"shell.execute_reply": "2026-06-13T02:34:34.682956Z"
|
||
},
|
||
"papermill": {
|
||
"duration": 0.020141,
|
||
"end_time": "2026-06-13T02:34:34.683732+00:00",
|
||
"exception": false,
|
||
"start_time": "2026-06-13T02:34:34.663591+00:00",
|
||
"status": "completed"
|
||
}
|
||
},
|
||
"outputs": [
|
||
{
|
||
"name": "stdout",
|
||
"output_type": "stream",
|
||
"text": [
|
||
"=== Gap Detection ===\n",
|
||
"AAPL: 35 gap(s) found\n",
|
||
" 2023-01-16 -> 2023-01-16 (1 days)\n",
|
||
" 2023-02-20 -> 2023-02-20 (1 days)\n",
|
||
" 2023-04-07 -> 2023-04-07 (1 days)\n",
|
||
" 2023-05-29 -> 2023-05-29 (1 days)\n",
|
||
" 2023-06-19 -> 2023-06-19 (1 days)\n",
|
||
" ... and 30 more\n",
|
||
"MSFT: 35 gap(s) found\n",
|
||
" 2023-01-16 -> 2023-01-16 (1 days)\n",
|
||
" 2023-02-20 -> 2023-02-20 (1 days)\n",
|
||
" 2023-04-07 -> 2023-04-07 (1 days)\n",
|
||
" 2023-05-29 -> 2023-05-29 (1 days)\n",
|
||
" 2023-06-19 -> 2023-06-19 (1 days)\n",
|
||
" ... and 30 more\n",
|
||
"GOOGL: 35 gap(s) found\n",
|
||
" 2023-01-16 -> 2023-01-16 (1 days)\n",
|
||
" 2023-02-20 -> 2023-02-20 (1 days)\n",
|
||
" 2023-04-07 -> 2023-04-07 (1 days)\n",
|
||
" 2023-05-29 -> 2023-05-29 (1 days)\n",
|
||
" 2023-06-19 -> 2023-06-19 (1 days)\n",
|
||
" ... and 30 more\n"
|
||
]
|
||
}
|
||
],
|
||
"source": [
|
||
"gap_detector = GapDetector(exclude_weekends=True)\n",
|
||
"\n",
|
||
"print(\"=== Gap Detection ===\")\n",
|
||
"for symbol in symbols:\n",
|
||
" key = f\"equities/daily/{symbol}\"\n",
|
||
" df = storage.read(key).collect()\n",
|
||
" gaps = gap_detector.detect_gaps(df, frequency=\"daily\")\n",
|
||
" if gaps:\n",
|
||
" print(f\"{symbol}: {len(gaps)} gap(s) found\")\n",
|
||
" for gap in gaps[:5]:\n",
|
||
" print(f\" {gap['start'].date()} -> {gap['end'].date()} ({gap['size_days']} days)\")\n",
|
||
" if len(gaps) > 5:\n",
|
||
" print(f\" ... and {len(gaps) - 5} more\")\n",
|
||
" else:\n",
|
||
" print(f\"{symbol}: complete (no gaps)\")"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "ac7bd869",
|
||
"metadata": {
|
||
"papermill": {
|
||
"duration": 0.001109,
|
||
"end_time": "2026-06-13T02:34:34.686135+00:00",
|
||
"exception": false,
|
||
"start_time": "2026-06-13T02:34:34.685026+00:00",
|
||
"status": "completed"
|
||
}
|
||
},
|
||
"source": [
|
||
"`exclude_weekends=True` filters Saturdays and Sundays. US market holidays\n",
|
||
"(MLK Day, Good Friday, Thanksgiving, etc.) still register as one-day gaps\n",
|
||
"because the detector has no exchange calendar — fine for routine update\n",
|
||
"health checks, but pair with a calendar-aware completeness check before\n",
|
||
"trusting the result for backtest panels."
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "8e056249",
|
||
"metadata": {
|
||
"lines_to_next_cell": 2,
|
||
"papermill": {
|
||
"duration": 0.001032,
|
||
"end_time": "2026-06-13T02:34:34.688260+00:00",
|
||
"exception": false,
|
||
"start_time": "2026-06-13T02:34:34.687228+00:00",
|
||
"status": "completed"
|
||
}
|
||
},
|
||
"source": [
|
||
"---\n",
|
||
"\n",
|
||
"## 4. Data Health Dashboard\n",
|
||
"\n",
|
||
"In production, monitor data health across your entire universe."
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 7,
|
||
"id": "a4427812",
|
||
"metadata": {
|
||
"execution": {
|
||
"iopub.execute_input": "2026-06-13T02:34:34.691152Z",
|
||
"iopub.status.busy": "2026-06-13T02:34:34.691014Z",
|
||
"iopub.status.idle": "2026-06-13T02:34:34.694342Z",
|
||
"shell.execute_reply": "2026-06-13T02:34:34.693956Z"
|
||
},
|
||
"papermill": {
|
||
"duration": 0.005257,
|
||
"end_time": "2026-06-13T02:34:34.694563+00:00",
|
||
"exception": false,
|
||
"start_time": "2026-06-13T02:34:34.689306+00:00",
|
||
"status": "completed"
|
||
}
|
||
},
|
||
"outputs": [],
|
||
"source": [
|
||
"def data_health_report(storage: HiveStorage, symbols: list[str]) -> pl.DataFrame:\n",
|
||
" \"\"\"Per-symbol freshness, gap count, and validation issue count.\"\"\"\n",
|
||
" validator = OHLCVValidator(max_return_threshold=0.5)\n",
|
||
" detector = GapDetector(exclude_weekends=True)\n",
|
||
" now = datetime.now()\n",
|
||
" rows = []\n",
|
||
" for symbol in symbols:\n",
|
||
" df = storage.read(f\"equities/daily/{symbol}\").collect()\n",
|
||
" last_date = df[\"timestamp\"].max().replace(tzinfo=None)\n",
|
||
" days_stale = (now - last_date).days\n",
|
||
" gaps = detector.detect_gaps(df, frequency=\"daily\")\n",
|
||
" result = validator.validate(df)\n",
|
||
" rows.append(\n",
|
||
" {\n",
|
||
" \"symbol\": symbol,\n",
|
||
" \"status\": \"stale\" if days_stale > 5 else \"fresh\",\n",
|
||
" \"rows\": len(df),\n",
|
||
" \"last_date\": last_date.date(),\n",
|
||
" \"days_stale\": days_stale,\n",
|
||
" \"gaps\": len(gaps),\n",
|
||
" \"issues\": result.error_count if not result.passed else 0,\n",
|
||
" }\n",
|
||
" )\n",
|
||
" return pl.DataFrame(rows)"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 8,
|
||
"id": "46de7e24",
|
||
"metadata": {
|
||
"execution": {
|
||
"iopub.execute_input": "2026-06-13T02:34:34.697499Z",
|
||
"iopub.status.busy": "2026-06-13T02:34:34.697388Z",
|
||
"iopub.status.idle": "2026-06-13T02:34:34.767458Z",
|
||
"shell.execute_reply": "2026-06-13T02:34:34.766938Z"
|
||
},
|
||
"papermill": {
|
||
"duration": 0.072037,
|
||
"end_time": "2026-06-13T02:34:34.767718+00:00",
|
||
"exception": false,
|
||
"start_time": "2026-06-13T02:34:34.695681+00:00",
|
||
"status": "completed"
|
||
}
|
||
},
|
||
"outputs": [
|
||
{
|
||
"data": {
|
||
"text/html": [
|
||
"<div><style>\n",
|
||
".dataframe > thead > tr,\n",
|
||
".dataframe > tbody > tr {\n",
|
||
" text-align: right;\n",
|
||
" white-space: pre-wrap;\n",
|
||
"}\n",
|
||
"</style>\n",
|
||
"<small>shape: (3, 7)</small><table border=\"1\" class=\"dataframe\"><thead><tr><th>symbol</th><th>status</th><th>rows</th><th>last_date</th><th>days_stale</th><th>gaps</th><th>issues</th></tr><tr><td>str</td><td>str</td><td>i64</td><td>date</td><td>i64</td><td>i64</td><td>i64</td></tr></thead><tbody><tr><td>"AAPL"</td><td>"fresh"</td><td>864</td><td>2026-06-12</td><td>0</td><td>35</td><td>1</td></tr><tr><td>"MSFT"</td><td>"fresh"</td><td>864</td><td>2026-06-12</td><td>0</td><td>35</td><td>1</td></tr><tr><td>"GOOGL"</td><td>"fresh"</td><td>864</td><td>2026-06-12</td><td>0</td><td>35</td><td>1</td></tr></tbody></table></div>"
|
||
],
|
||
"text/plain": [
|
||
"shape: (3, 7)\n",
|
||
"┌────────┬────────┬──────┬────────────┬────────────┬──────┬────────┐\n",
|
||
"│ symbol ┆ status ┆ rows ┆ last_date ┆ days_stale ┆ gaps ┆ issues │\n",
|
||
"│ --- ┆ --- ┆ --- ┆ --- ┆ --- ┆ --- ┆ --- │\n",
|
||
"│ str ┆ str ┆ i64 ┆ date ┆ i64 ┆ i64 ┆ i64 │\n",
|
||
"╞════════╪════════╪══════╪════════════╪════════════╪══════╪════════╡\n",
|
||
"│ AAPL ┆ fresh ┆ 864 ┆ 2026-06-12 ┆ 0 ┆ 35 ┆ 1 │\n",
|
||
"│ MSFT ┆ fresh ┆ 864 ┆ 2026-06-12 ┆ 0 ┆ 35 ┆ 1 │\n",
|
||
"│ GOOGL ┆ fresh ┆ 864 ┆ 2026-06-12 ┆ 0 ┆ 35 ┆ 1 │\n",
|
||
"└────────┴────────┴──────┴────────────┴────────────┴──────┴────────┘"
|
||
]
|
||
},
|
||
"execution_count": 8,
|
||
"metadata": {},
|
||
"output_type": "execute_result"
|
||
}
|
||
],
|
||
"source": [
|
||
"report = data_health_report(storage, symbols)\n",
|
||
"report"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 9,
|
||
"id": "caaa47f6",
|
||
"metadata": {
|
||
"execution": {
|
||
"iopub.execute_input": "2026-06-13T02:34:34.770909Z",
|
||
"iopub.status.busy": "2026-06-13T02:34:34.770806Z",
|
||
"iopub.status.idle": "2026-06-13T02:34:34.916494Z",
|
||
"shell.execute_reply": "2026-06-13T02:34:34.915803Z"
|
||
},
|
||
"papermill": {
|
||
"duration": 0.147883,
|
||
"end_time": "2026-06-13T02:34:34.916946+00:00",
|
||
"exception": false,
|
||
"start_time": "2026-06-13T02:34:34.769063+00:00",
|
||
"status": "completed"
|
||
}
|
||
},
|
||
"outputs": [
|
||
{
|
||
"data": {
|
||
"image/png": "iVBORw0KGgoAAAANSUhEUgAABXkAAAGRCAYAAAAwzbBQAAAAOnRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjEwLjksIGh0dHBzOi8vbWF0cGxvdGxpYi5vcmcvJkbTWQAAAAlwSFlzAAAPYQAAD2EBqD+naQAAclVJREFUeJzt3Xd0FFUfxvFnS3pPIEDovXeQIog0AREQrCgiiqBgL9j1VUEUBUWkSFOaFBUUEBRQpIgoHaRL7xBCQiBts+X9A1mNJCHUySTfzzmcw+60506yubO/vXvH4nCkeQQAAAAAAAAAMCWr0QEAAAAAAAAAAJePIi8AAAAAAAAAmBhFXgAAAAAAAAAwMYq8AAAAAAAAAGBiFHkBAABygXGTZ6pRm25q1KabGrd9QLfc0VsPP/mGPvviK8WdSrisfU756nut27j1qmXs3P0ZDRszNdNljdp009ezF161Y523buNWNWrTTbv3HZQkpac7NW7yTO3cvT/DekePxapRm2769ff1l3yM/oNHe899k3bd1ebOx/Toc+9o0ow5OpuUfFXacTlZx02eqbZ3PXbVj3+1tL3rMY2bPNPoGAAAAJBkNzoAAAAAzgkOCtTH774oSTqblKwdu/bp2+9/1uwfftHH776oSuVLX9L+pnz9ve7s2Fp1ala5FnENke50avyUb1WkUEFVKFvyqu23ZPEYvf58b3k8HiWeOavN23dp6jfzNXv+Lxo+6FUVKVzwqh0LAAAAuNoYyQsAAJBL2GxWVatcTtUql1PDejX04L0dNfmzgYqKDNcbA4fL5XIbHTHPCvD3U7XK5VS9Snnd2KC2Hn3wLk0a+a4cjnQNGDLG6HiGSXM4jI4AAACAHGAkLwAAQC4WEhykx3veq+de/1Cr1v2pRvVrSpJGjp+uFas26OixWAUHB6p29cp6qvd9iooMl3RuaoXTiWc1fsq3Gj/lW0nSiA9eVZ2aVTT1m/n6aelKHTh0TL6+PqpSsayefvR+FS9a+KpmX/bbWn0x9Tvt2XdIwcGBateqiR7rcZfs9nOXoPsOHNH4KbO0actOnT5zVjGFCqpju+a6+/ZbZLVmPhah5e2PSJIGDBnjLb7Omvixd3lqWpre/2S8flryuwID/dWhzc3q2a1zlvvLTnTBKD10f2d9+OkX2nfgiEqViFFKaqpGjJuh1ev/1PHYU4oMD1OjG2qq70N3Kygo0LvtnB+XaNrM+TpyLFYB/n4qXbKY+j3RQ2VKFbvkrBu37NRHIydq3/4jKlm8iJ5//EHVrFbRu9zlcuuLqd/q+wXLdCrhtIoVKaQHu3ZSmxaNvev8ufUvTZoxR9t27lFSUqqKFy2k++9qrzYtbvSuM2/hMg0YMkbjPnlbI8ZP05btu/XgvR318P2dtf7P7fpoxCQdOHRUpUsW1XN9u1/y+QQAAMC1Q5EXAAAgl6tTs7JsNpu2bN/lLfKeSkjUg/d2VIGoCCWcPqNpM+friZcG6svR78tqter9N5/REy8OVPOm9dWxbXNJUukSMZKkEydP6Y6OrVUkuoCSklP07bzFevTZd/TVF4MV/K9CZaY8Hjldrotm/mnp7/rf+yN0+60t9NhDd+vwkeMa9cVXcrs9eqr3fZKk2LhTKlGsiG5p0VhBAf7aufuAxk2eqTSHQw/e2zHT/Q4f9KqeeGmgetzXSTfeUFuSFBUZ7p23eMT46Wp+Y30NfP0prdmwRZ9/+a1KlyyqVs0aXjRzZm6oU02StGX7LpUqEaPUVIfcbrce7XG3wsNCdCI2ThOmzdFr736qoQNfkiSt/3O7Phj2hXp1v0PVK5dTUnKK/ty264L5fXOSNTXNobcHjVL3ezsoKjJc02bO13Ovf6ivPh/sLeiPnfSNpnw9Tz27dVblCmW05NfVemvQSFks0i3NzxV6j504qRpVKqhz+5by9fXRpi07NWDIGFksFu865/3v/RHqclsr9by/i4KDAxUbF6/nXvtQVSqW0buvP6WTp+L11qBRSk1jlC8AAEBuQZEXAAAgl/Pz9VV4WLBOxSd6n3v9+d7e/7tcblWrXE6d7n9KG7fsVO3qlVSxXCnZbFZFF4hUtcrlMuzvmce6Zdi2fp3qan9PXy37ba1ubd002yzTZv2gabN+yHYdj8ejEeOmqV3LJur35EPnnqxbXT6+Pho8fIIevLeDwkJDVL92NdWvXc27TY1qFZWalqY5P/ySZZG3csUykqRiRQpd0C5JqlWtkp569H5J0g11q+v3NZu0dMWayy7yRheIlCSdij8tSYoID9WLTz3kXe50uVSkcLQee+4dHTtxUoWjC2jr9t0qV7p4hjY0bVT3srKmpTn0aI+7vKNy69asotu7Pa0Z3/6ovj3v1enEs5rx7QI9dF8nPXTf7ZKkhvVq6MTJUxo/ZZa3gNv65kbefXo8HtWqXkknTp7SnB+WXFDkvavTLbqnc1vv4+HjpsnX10dD+r8gf38/Seemt3hr0KhLPJsAAAC4VijyAgAAmIDHk/HxytUb9fmX32nv/kNKSk7xPn/w0FHVrl4p231t3rZLYyZ+ox279inxzNl/tj187KI52ra8UXff3uaC5x9+8k3v/w8cOqpjJ+LUslnDDKN+69WsIocjXbv3HVKdGpWV5nBo0vS5WrB4hY7Hxsnp/Gddp8slu8120Tz/1aButQyPS5UoquOxcZe8n/M88lzw3A8//apps37QocPHlJKa5n3+wKFjKhxdQOXLltSI8dM19LMpanZjPVWrVE4+Phdeduc0a7Mb63n/HxjgrxvqVNPWHXskSXv2H1JqWppaNG2QYZuWzRpowOAxik9IVER4qBLPJGnc5JlavnKtYk/Gy+U+N79zwQIRFxyv8Q21MjzeumO3bqhTzVvglaRmjesJAAAAuQdFXgAAgFwuzeHQ6cSziowIlXSu6Nbvfx+pWeO66n7PbYoID5PFIj3y9FtKc6Rnu69jJ07q6VffV5UKZfXS0w+rQGS4fHzsev6NwRfdVpIiwsNUuUKZbNdJSDxXOH7u9Q8zXX7i70LmyPHTNeeHperZrbMqlCulkOBALVu5VhOmzpbDkS57wKUXeYODgjI89vGxy5GDdmUl9mS8JCkyIkyStGTFar3z4WfqcltL9XnoboWGBOnkqQS9/PZQ73FuqFNNrz3fS19/t1BffbdAAQF+atuyiZ545F4F+PtfUtbAAH/5+/lmeC4iPFS79h6UJMXFJfydLzTDOpHh5/ImnjmriPBQDRg8Wpu379JD992u0iWLKigwQLO+/1nLV669oM3n23pe3KnTKle6RIbn/P39FBjgLwAAAOQOFHkBAAByuXUbt8nlcqla5fKSpKUr1igiLEQDXntSFotFknT0+Mkc7ev3NZuUlurQB28/6y04Ol0uJZ5Jump5w0LOFS9ffrqnKpQrecHymMIFJUmLl63SXZ1aq9vdt3mX/bZqw1XLcTWsWvunJHmnhli8bJWqVir7zzQUktZt2nbBdu1b36T2rW9SfEKilqxYrU9Gf6mgAH/17XnvJR0/OSVVqWmODIXe+IREFfh7Pt6oqHDvc2GhId51TiWcm14iNCRYaQ6HVqxar+cf76Eut7X0ruNxL8r0mOd/p86LigxTfEJihudSU9OUnJJ6SW0BAADAtXPptxkGAADAdXPmbJJGjJ+uYjGFvPPXpjnSZbfbMhTjFixeccG2PvYLR4ampTlksVpks/4zSvbnpX/IlYObqeVUiWJFVLBAhI4ej1XlCmUu+He+GJnmcMjHx8e7ncvl1qIlv2e7bx+7/e9tL390bk6diI3TF1O/U92aVVSyeMzfx82YWZIWLv4ty31EhIeqc/uWqlWtovYeOHxZOZauWOP9f3JKqlat26wqf89NXKZkMfn7+ennZasybLN42R8qUaywIsJDlZ7ulNvtke+/poxISk7R8t/X5+j4lSuU0ap1m5X6r6kplv62JpstAAAAcL0xkhcAACCXcLnc2rxtlyQpOTlF23ft1bff/6zUNIc+fvdF2WznPp+/oU41zfj2R308arKaNKytP7f+lWmRt2TxGK1YtUEN69VQQIC/ShQrorq1qsrtdmvAkDHq0LaZ9u4/rKnfzFdIcOBVa4fVatVTve7X2x+OUlJyihrVrym73a4jx05o2W9rNfD1p+Tv76f6tatp5txFKhZTSKEhQZo59yelp2dfvPXxsSumcEEtXvaHypYqJl9fnwumErgcKalp2rxtlzwej84mJWvT1p36bt5iBQb467V/3eTuhjrVNHj4RE2YOltVKpXVytUbtGbDlgz7GjtpphLPnFWdGpUVFhainbv2a/2m7erz8D2XnMvPz1ejJ3ytlNRUFYiM0NSZ8+R0Or3zIoeFBuuezm00Ydp3stmsqlyhjJb8ulq/rdqod155XJIUHBSoyhXK6PMvv1NQYIAsVosmz/hewUEBGeZzzso9ndtq5tyf9MKbQ3Rvl3Y6eSpek6bPld9/ppEAAACAcSjyAgAA5BJnk5LV65m3ZLFYFBQYoGIxhdSmxY26q9Mtivr76/nSuRtjPd7zXn09e6Hm/LBE1SqX0+B3XtDdD7+QYX9P9OqqwcMn6Pk3hig1LU0jPnhVdWpW0evPP6rxU2Zp6W9rVL5MCb37+pN6feDwq9qWVjc3VFBQgCZOm63vFyyT1WZR0cLRurFBbdn/HlH6/OMPatCwzzV4xAT5+frq1tZN1axxPb3/yfhs9/3iUw/r07FT9dTL78uRnq5ZEz++4rz7Dx5Rr2fektVqUVBgoEqViFHXO9qpy22tFBz0TwH89ltb6vDRWH313QKlpTt0Q+3qevvlvnrk6be861SuUEbTv/1BPy39XcnJqSpcKEo9H+isezpfeMO6i/H389Wb/R7VRyMnad/BIypZLEZDBvRTgah/bpjWq/udstls+vb7nzU+YZaKxRTSWy/1UeubG3nXefvlvho07HO98+FohYUG686OrZWalqZv5mQ+ZcO/RReI1Ef9X9BHoybr1QGfqFTxGP3vpT566a2PLrk9AAAAuDYsDkfahbcMBgAAAAAAAACYAnPyAgAAAAAAAICJUeQFAAAAAAAAABOjyAsAAAAAAAAAJkaRFwAAAAAAAABMjCIvAAAAAAAAAJgYRV4AAAAAAAAAMDGKvAAAAAAAAABgYhR5AQAAAAAAAMDEKPICAAAAAAAAgIlR5AUAAAAAAAAAE6PIa1KN2nTT0t/WGB0DAIB8oW+/Afp41GSjYwAAAABApuxGB7iW+g8erfmLlkuSbDabQkOCVK50CbVu3kjtWzeV1ZrzGve8hcs09LMpWjRrzBVluv/Rl1WjSgW99PTDFyz74adf9d7QcZrz5acKDwu5ouMAAHA15Ma+VDpXdF2/afsFzy+fP1F2m+2K9w8AwNWSm/vS8mVK6tk+D1zxvgAAxsvTRV5Jalivhl5/vrdcbrfi409r5ZpNGjpqsn5ZvkofvP3cdX8j2KHNzRo/ZZaefqyb/P18Myybt3CZmjSsQ4EXAJCr5La+9LxO7ZqrV/c7MjyXWZb0dKd8fPL8JQ8AIBfLrX0pACDvyPPveHx9fBQVGS5Jii4QqYrlS6ta5XJ68qX3NH/hMnVs11ySNG3mfH2/cJmOHI1VaEiQmjSsrccf6arAAH+t27hVA4ac+6S0UZtukqSe3TrrkQfu0A8//aqvvlugA4eOyt/fT3VrVtEzfbopMjws0zxtW96okZ9P15JfV6ltyybe548cO6F1m7ZpSP8XJEmz5v6kqTPn63hsnGIKF1SPrrerXasmme5z3catevzFgVo4c7RCgoMkSTt379eDfV/TrIkfq0jhgt5PfP/3Yh99OvZLHY89pcb1a+rNfo/p5+V/aNzkWUpKSlbblk309KPdZLOd+zTZ4UjXZxO+1qIlK3X2bLLKlCqmx3veozo1q1zhTwYAYBa5rS89z8/P15vr3zp3f0Yd2jTTwcPHtWzlGt18Y3298cKj2rh5h0Z9PkPb/tqr8NAQNbuxnvo8fLcC/P0lSTPnLtL0WT/qROwpBQUFqFa1ihr4xtPe/Xo8Hg0fN01zflgiHx+7OrdvoUce+KfI3KhNN73yTE+tWLVBf6z9UwWjIvRU7/vUtFFd7zq79x3U8LHTtHHzDvn7+6lB3ep6+tFu3g94Fy9fpfFTZunQkePy9/NThbIl9cHbzyrA/9w5HD5uuvbuPyy73abSJYvq7ZcfV5FCBS7xJwoAuN5ya1/6b9n1g9n1T5mNCH7prY8VHByoN154VNLF31cePX5SQ0ZM1KYtO5Se7lKRQgX0RK+uanxDrSs67wCQn+T5Im9m6tWqqvJlSmjJijXeztRiseq5Pt0VU7igDh87oQ8/naAR46ap35MPqXqVCnrmsW4aO2mmZoz/UJIUEHDuDaHT5VLvB+9UiWJFFJ+QqE9Gf6kBg8foowH9Mj12eFiImjaso+8XLMtQ5J23cLmiC0SqQd3qWrJitT7+bLKeeayb6teuphV/rNe7Q8YoukCk6ta6/OJqalqavp69QO+88oSSU1L1yjtD9fI7QxUcFKiP+vfT4aMn9OqAT1SjSgW1urmhJGnIiInae+Cw+r/yuApERWjpijV69rUPNWX0eypetPBlZwEAmJuRfWlOTP1mvh6+v7N6dussSTp05Liefe0D9X7wLr32fG/FJyRqyMiJGjJ8ol5/4VFt27lHH4+crDdffEzVq1RQ4pmz2rh5R4Z9zl+0XF3vaKfxw97Wn1v/0oAhY1SjSgXdULe6d53xU77V44/cqyce6apv5izUW4NGadakoQoLDdaZs0l68sX31KFtMz39aDelORwaOX66Xn/3Uw3/4FWdjIvXm++N0OOP3KubG9dTUkqqNm7eLo/n3Dl66e2h6tjuZr3zyuNyOp3asmO3LJbLPkUAAIPlpr40u34wu/4ppy72vnLw8AlyOp0aOfgNBfj7ae/+w962AQByJl8WeSWpZPEY7dp7wPv43i5tvf8vUrigHu1xlz4Y9rn6PfmQfHzsCg4KlMViuWDEUIc2zbz/L1okWs/1fUAPP/mmklNSFZhFp9Sh7c167vUPdeTYCcUUjpbH49H8n5arXesmslqtmvrNfLVvfZPu6NBaklSiWBFt3rZLU2fOu6Iir9PpUr8nH1KxmEKSpOZNbtCPP6/QvBkjFBjgr9Ili6pOjcpau3GrWt3cUMdOnNS8hcv07ZRPVDAqQpJ0/13t9fuaTfp+wVL1efiey84CADA/I/tSSZr1/U+a++MS7+Pbb22hpx69X5JUt1YV3Xfnrd5lAz8eq1taNPZmLF60sJ7r0119+w1Qv6ce0vETcfL399ONDWorKDBARQoVUMVypTIcr1zp4urZrYt3+2/mLNKaDVsyFHlvvaWpbmneWJL02EN366vvFmrrjt1qVL+mvpmzSBXKlczQf772XC916va0Dhw6qpSUVLlcLt18Y33v6NxypYtLkk4nntXZpGTd2KC2tx8vVaJolucGAGAORvel52XXD8adSsiyf8qJnLyvPB4bp+ZN6nv3W7RIdI73DwA4J98WeT0ejyz6Z/jLqnWbNWnGHO0/eFTJySlyulxyONKVmpomf3+/LPez/a+9Gjd5lnbtOaAzZ5Pkdp/7OPP4iTiVLpn5m68b6lRTdIFIfb9gmXo/eKdWr9+i4yfidNst5zrmfQeOqNOtzTNsU6NqBX313YIrarO/n5/3jaEkRUaEqUihAhk6/ciIMMWfTpQk7d57UC63W/c8/EKG/TjSnQoLDb6iLAAA8zOyL5WkNs0b68GunbyPz09ZJEmVy5fJsO6uPQe0a+9BLVz827/yS263R0ePxap+nWoqXKiA7uzxnBrWq6GG9WqoWeN6GXKXLVMiwz4LRIbrVEJihufKlf5nnQB/fwUFBij+73X+2nNAazduVYtOPS9oy+GjJ3RDneqqV6uquj32shrUraEGdaqredMbFBoSpLDQYLVv3VTPvvqB6teppvq1q6rlTQ1U4O83ywAAczK6Lz0vu36wXJmSWfZPOZGT95V3d7pFH3w6QX+s/VP1a1c7V/D9T78LAMhevi3y7jt4REUKF5QkHT0Wq35vDlHn21rqsR53KzQkSBu37NTAj8Yq3emUvzLvTFNSU/XMq4PUoG4NvfVSH4WHhep4bJyeeXWQ0p3OLI9ttVp1a+um+uGn5XrkgS6at3CZ6tSsfNmfVlr+vhvrv78u48zk+HZ7xsn8LZbMnrPI8/cFQUpKmmxWq74Y3v+CO77m5NNgAEDeZmRfKklBQYFZTh303zfCySlpuv3WFrqr0y0XrFs4uoB8fOyaMGKA1m3cplXr/tTYSTM1bvIsff7pO97i8X9vimOxWOT5z3dVM+1XPef71VQ1aVBHfXte+E2YAlHhstmsGvb+y9q09S+tWvunvp6zUKMnfK1xw95STOFovf7Co7rr9jb6fc0m/bT0D42Z+I0+ee9lVatcLtvzBADIvYzuS88LCgzIth/Mrn+yWqySMvaHTtc/x83J+8qO7ZqrQd0aWrFqg1at+1OTZszRU73vz7TfBgBkznrxVfKeNRu2aPfeg2repL6kc596uj1uPdX7PlWrXE4lihXRybj4DNvY7Xa53e4Mz+0/eFSnE8+q78P3qFb1SipVIkbxCadzlOG2NjfpeOwpLVmxRkt/W6MObW72LitVIkZ/bvkrw/qbtuzM8muZEX/frCXuVIL3ub92H8h03UtRoVzJc3d/TUhU8aKFM/zL7EY3AID8Izf0pZeiYrlS2rv/8AX9WfGiheXjc+4zb7vNphvqVNMTj3TV5M8G6ujxk1q7YetVzbBn/yEVKVzwggznb/5msVhUs2oF9ep+hyaOeFd2H5uWrliTYR8P3ttRY4f+T2VKFdPCX37L6nAAgFwut/Wl2fWD2fVP4WEhOhmX4N2Py+XWnn2HvI9z+r6yUHSUutzWUu+/+Yy63nGrZv/wyyW3AQDyszw/kteRnn5uDiG3W/Hxp7VyzSZNnjFXNzaorXatmkqSisUUktPp0tezF6pJwzratGWnvp33c4b9FClUQMkpqVq9frPKlykpfz9fFSoYJR8fu76es1Cd27fUnn0H9cXU73KUK6ZwtOrWrKL3h46Xr49dNzep5112/13t9fq7n6pC2ZKqX6eafv19nZauWKNh77+c6b6KxRRSoYJRGjd5ph576G4dOHRUU2fOv7wT9i8lihVRmxaN9c6Ho/VU7/tUoVxJxSec0ZoNW1SudHHd2KD2FR8DAJD75da+9FI8cPdteuSZtzR4+ER1bHez96Yuq9Zt1gtPPKhff1+vI8dOqFb1SgoJDtLKVRvk8bhVoliRq5bhjo6tNeeHJXrzvRHqdld7hYYE69CR4/ppyUq98mwvbf9rj1av36IGdasrIjxUW7bvVsLpMypVoqiOHDuh7+b/oqYN66hAVIQOHDqqg4ePqd2/buIKAMi9cntfml0/uGX7riz7J+ncPPjDRk/Vij/Wq2iRQpo+6wedOZvs3XdO3ld+PGqyGtWvqRLFCuvMmSSt27hVpYrHXNlJB4B8Js8XeX9fs0m3dX1CNptNocFBKlemhJ7t84Bubd3U+1WR8mVL6qlH79eUr77XqC++Uq1qFdXnoXv0zoefefdTo2oFdW7fUm8MHK7TiWfVs1tnPfLAHXr9+d767Iuv9fV3C1WhXCk90es+vfi/j3KUrUPbZlrz/hZ16dBKfr6+3uebNa6nZx97QFNnztfHn01WTOGCeu353qpTM/Obrtntdr39yuP68NMv1O2xV1SlQhk92uMuvTZg2BWcuXNef763vpg6W8PGTFVs3CmFh4aoauVyFHgBIB/JzX1pTpUrU0IjP3xNoyd8rT7P95fH41HRIoXUslkDSVJIcKCWrFitcZNnyZGeruIxhfX2y4+rTKliVy1DwagIjf74TY0YP13PvDpIjnSnCkcXUMN6NWS1WhQUGKANf27XV98uUFJyigoXitKTve5To/o1dSr+tPYfPKIfFi3X6TNnFRUZrjs6tNbt7VtctXwAgGsnt/el2fWD+w4czrJ/ks7d9G3XngN658PRstmsurdLW9X9z3vXi72vdLvdGjx8gmJPxisoMEAN6tXQM4/df6WnHQDyFYvDkea5+GoAAAAAAAAAgNwoX87JCwAAAAAAAAB5BUVeAAAAAAAAADAxirwAAAAAAAAAYGIUeQEAAAAAAADAxCjyAgAAAAAAAICJUeQFAAAAAAAAABPLk0Vej8cjp9Mpj8djdBQAAEyJvhQAgCtDXwoAuJ7yZJHX5XKpafsecrlcRkcBAMCU6EuRF720c6Fe2rnQ6BiAKR3fMEbHN4wxOoap0JcCAK6nPFnkBQAAAAAAAID8wm50AAAAAOB6iPYNMjoCYFp2v3CjIwAAgGxQ5AUAAEC+8HypG42OAJhWVOW7jY4AAACywXQNAAAAAAAAAGBiFHkBAACQL6S6nEp1OY2OAZiS2+WQ2+UwOgYAAMgCRV4AAADkC//bvVj/273Y6BiAKcX+OUGxf04wOgYAAMgCRV4AAAAAAAAAMDGKvAAAAAAAAABgYhR5AQAAAAAAAMDEKPICAAAAAAAAgIlR5AUAAAAAAAAAE6PICwAAAAAAAAAmZjc6AAAAAHA9vFDqRqMjAKYVVeluoyMAAIBs5Okib4FSdY2OAADIBRIObTQ6AoBcoKBvkNERANOy+4cbHcG0kjp0kt1idAoAwLVw+BG/HK9bocusa5iE6RoAAAAAAAAAwNQo8gIAACBfGH1wtUYfXG10DMCU4nfNVfyuuUbHAAAAWcjT0zUAAAAA5+1JiTc6AmBajrNHjY4AAACywUheAAAAAAAAADAxirwAAAAAAAAAYGIUeQEAAAAAAADAxCjyAgAAAAAAAICJUeQFAAAAAAAAABOzGx0AAAAAuB7aF6hgdATAtEJiGhodAQAAZIMiLwAAAPKFmyJLGR0BMK3A6BpGRwAAANlgugYAAAAAAAAAMDGKvAAAAMgXlp3ap2Wn9hkdAzCl5BOblHxik9ExAABAFpiuAQAAAPnCvJM7JTFtA3A5zhz5XRLTNgAAkFsxkhcAAAAAAAAATIwiLwAAAAAAAACYGEVeAAAAAAAAADAxirwAAAAAAAAAYGIUeQEAAAAAAADAxOxGBwAAAACuhzIBEUZHAEzLN7iI0REAAEA2KPICAAAgX3i0eH2jIwCmFVGug9ERAABANpiuAQAAAAAAAABMjCIvAAAA8oVYR5JiHUlGxwBMyZmaIGdqgtExAABAFijyAgAAIF8YvG+FBu9bYXQMwJTitn+luO1fGR0DAABkgSIvAAAAAAAAAJgYRV4AAAAAAAAAMDGKvAAAAAAAAABgYhR5AQAAAAAAAMDEKPICAAAAAAAAgIlR5AUAAAAAAAAAE7MbHQAAAAC4Ht4u28LoCIBpFazew+gIAAAgGxR5AQAAkC/427j0BS6X1eZrdAQAAJANpmsAAAAAAAAAABOjyAsAAIB8Yci+FRqyb4XRMQBTitv2leK2fWV0DAAAkAW+swYAAK67/SkJ+mT/Sq0+fVhuueVnteuewtV1f5GaCrbzlWBcGyccSUZHAEzLmZZgdAQAAJANirwAAOC6SXe79NjWOfoj4ZBOpGcsuO3cvUQjDvyhV8rcpK5FahiUEAAAAADMhyIvAAC4Ltwej+7f9I1+O31ADrfrguUOj0vHHGf11q7Fcnnc6hZT6/qHBAAAAAATyrbI63Q6NXH6HC38ZaVsVqvsPnYViS6gng90UYWyJbXvwGF9Onaq9h88KkkqWTxGT/a6T6VKxHj38evv6/X5l7OUlJwip9OlG+pU1xOP3KugoEBJksfj0TdzFum7+YvldLrk7++r8NAQdb+no+rWqqKjx2LVve9rWjRrzDU8DQAAXBv0pf/45thmrTp9KNMC77/FO1M1aO+v6lCwksJ8/K9TOgBAbkVfCgDAxWVb5B0wZIxSUtI0duhbCg0JkiStWrdZBw4eVUR4qPq+8K6efqyb2rRoLEla+MtvevzFdzVp5LuKigzXytUbNWjY5xr8zvOqWK6UnC6XPvlsil54c4hGDn5dFotFYyZ+ozUbturjAf0UXTBKkrR1x25t/2uf6taqco2bDwDAtUVf+o+RB1cp2Z2eo3XjHMn6/PA6PVuq8TVOBQDI7ehLAQC4OGtWCw4ePqalK9bq1ed6eTtSSbqhTjW1urmhZs39SbVrVPJ2pJJ0S/PGqlWtombO/UmSNGHabPXo2lEVy5WSJNltNj3V+34dPnZCazduVXJKqqZ+M1+vPdfL25FKUpWKZdXltpZXu60AAFxX9KX/OJKaqJPpyTle3ym3Zh7fcg0TAQDMgL4UAICcybLIu2PXPhWLKaSw0OAsl1erUv6C56tVKa8du/b+s07ljOv4+NhVqVxpbf9rr/buPywfH3uGr9EAAJBX0Jf+IzY9WekXmabhvy42rQNwqe4pXE33FK5mdAzAlEJLNFdoiebX/bj0pQAA5EyOb7x26Mhxvdr/E6U5HKpe+cJO9HJZLBbv/1PTHOr9zFtKdzpVIDJCnw56Jcf7+WbOIs2cu0jSufmUAADIbfJzX+pntcn6r5w5canrAxdTJ5QCDnC5AiKvXr91JfJzXwoAQHayHMlbsVwpHTpyXIlnkiRJxWIKadKogep+T0edOZusiuVKafPWvy7YbvPWv1Th76/BVCxXSpu3ZVwnPd2p7bv2qmK5UipdsqgcjnTtP3hEkuTv56tJowaq3xM9dDrxzCU15M6OrTVt7AeaNvYDTfnsvUvaFgCAa4G+9B+lAyLkb/W5pG3KBERc1QwAAPOhLwUAIGeyLPIWL1pYTRvV0cCPx+rM2STv8ympaZKkzre11LpN27Rg8W/eZQt/+U3rN23XHbe1kiQ9eG9HTZg6Wzt375ckOV0uDRvzpYoUKqh6taoqMMBf93Zpq/c+HqcTJ09dcAwAAMyMvvQffla7WkaVyfH6kfYAvVi66TVMhPzoh9id+iF2p9ExAFM6e2SVzh5Zdd2PS18KAEDOWByOtCy/Q5Ke7tSEabP109KVslltCgkJUnhYiLrddZuqVymvPfsO6dOxU3Xg0FFZLBaVKFZET/W+T6VKFPXuY9lva/X5l98qJTVVTqdL9WtX0xO9uio4KFDSua+wfPXdAs35cYmcTpfCw0IU4O+nezq3VaP6NXX0WKzu6PGcCkSFe/dZqGCUxg59K8tGOZ1ONW3fQ9u2/HnlZwgAYHoJhzYadmyz96XL502Q3Z7j2Z2ydTTtjNqunaSjadmPirLJovphxTSnzv1X5bjAeS/tXChJGlThFoOTAOZzfMMYSVKhWr2v+7HN3pf+aI2XnRmIACBPOvyIX47XrdBl1jVMcpEir1lR5AUA/JuRRV6zuhZFXklae/qwHt78nY46Mi/0+lisqhxcUN/Vuk/B9pxfMAE5QZEXuHxGFnnNiiIvAOR9uanIe/XetQEAAFxE3bCi+r5uN723Z5l+SzigJJdDTrdbfja7wux+6lq4hvqUuEF+Vi5RAAAAACCneAcFAACuq+L+YRpZpYOSXA5tOXtCKS6nonwDVDUoOsPdzQEAAAAAOUORFwAAGCLI5qsbwooZHQMAAAAATM9qdAAAAAAAAAAAwOVjJC8AAADyherBhYyOAJiWf1hpoyMAAIBsUOQFAABAvtAtpqbREQDTCivd2ugIAAAgG0zXAAAAAAAAAAAmRpEXAAAA+cK+lATtS0kwOgZgSo6kY3IkHTM6BgAAyAJFXgAAAOQLow6u0qiDq4yOAZhS/F9zFP/XHKNjAACALFDkBQAAAAAAAAATo8gLAAAAAAAAACZGkRcAAAAAAAAATIwiLwAAAAAAAACYGEVeAAAAAAAAADAxu9EBAAAAgOvBJovREQDTslgYHwQAQG5GkRcAAAD5wsAKrY2OAJhWdM1HjI4AAACywcexAAAAAAAAAGBiFHkBAAAAAAAAwMQo8gIAACBfeGf3L3pn9y9GxwBMKXbzJMVunmR0DAAAkAXm5AUAAEC+kORKNzoCYFpuZ6rREQAAQDYYyQsAAAAAAAAAJkaRFwAAAAAAAABMjCIvAAAAAAAAAJgYRV4AAAAAAAAAMDGKvAAAAAAAAABgYnajAwAAAADXw8NF6xgdATCt8DLtjI4AAACyQZEXAAAA+ULFoAJGRwBMyy+0uNERAABANpiuAQAAAAAAAABMjCIvAAAA8oVvjm3RN8e2GB0DMKXEA0uVeGCp0TEAAEAWKPICAAAgX1ideFirEw8bHQMwpZRTO5RyaofRMQAAQBYo8gIAAAAAAACAiVHkBQAAAAAAAAATo8gLAAAAAAAAACZGkRcAAAAAAAAATIwiLwAAAAAAAACYmN3oANfSyX1rZbfn6SYCAAAghxqFFTc6AmBagQWqGB3BtILmzuZ9KQDkURWMDvAv9DQAAADIF24vVNnoCIBphRRrYnQEAACQDaZrAAAAAAAAAAATo8gLAACAfGHL2RPacvaE0TEAU0o7vU9pp/cZHQMAAGSBIi8AAADyhUlHNmjSkQ1GxwBMKWHvQiXsXWh0DAAAkAWKvAAAAAAAAABgYhR5AQAAAAAAAMDEKPICAAAAAAAAgIlR5AUAAAAAAAAAE6PICwAAAAAAAAAmZjc6AAAAAHA9hNn9jI4AmJbNJ8joCAAAIBsUeQEAAJAvvFqmmdERANMqUPV+oyMAAIBsMF0DAAAAAAAAAJgYRV4AAAAAAAAAMDGKvAAAAMgXXtq5UC/tXGh0DMCUjm8Yo+MbxhgdAwAAZIEiLwAAAAAAAACYGEVeAAAAAAAAADAxirwAAAAAAAAAYGIUeQEAAAAAAADAxCjyAgAAAAAAAICJUeQFAAAAAAAAABOzGx3gWipQqq7REQAAuUDCoY1GRwCQCzxVoqHREQDTiqzQxegIppXUoZPsFqNTAACuhcOP+OV43QpdZl3DJHm8yAsAAACcV9Q/1OgIgGn5BBYwOgIAAMgG0zUAAAAAAAAAgIlR5AUAAEC+MOHwek04vN7oGIApJexZoIQ9C4yOAQAAssB0DQAAAMgXtiXFGh0BMK20xP1GRwAAANlgJC8AAAAAAAAAmBhFXgAAAAAAAAAwMYq8AAAAAAAAAGBiFHkBAAAAAAAAwMQo8gIAAAAAAACAidmNDgAAAABcD60iyxgdATCtoEJ1jI4AAACyQZEXAAAA+ULrAuWMjgCYVnCRekZHAAAA2WC6BgAAAAAAAAAwMYq8AAAAyBf+SDikPxIOGR0DMKWUuG1KidtmdAwAAJAFpmsAAABAvjDrxFZJUoPwYgYnAcwn8eBySVJAVGWDkwAAgMwwkhcAAAAAAAAATIwiLwAAAAAAAACYGEVeAAAAAAAAADAxirwAAAAAAAAAYGIUeQEAAAAAAADAxOxGBwAAAACuhxi/EKMjAKblExBldAQAAJANirwAAADIF54u2cjoCIBpRVa8w+gIAAAgG0zXAAAAAAAAAAAmRpEXAAAA+cJZZ5rOOtOMjgGYkjs9Re70FKNjAACALFDkBQAAQL7Qf89S9d+z1OgYgCnFbpms2C2TjY4BAACyQJEXAAAAAAAAAEyMIi8AAAAAAAAAmBhFXgAAAAAAAAAwMYq8AAAAAAAAAGBiFHkBAAAAAAAAwMQo8gIAAAAAAACAidmNDgAAAABcD6+UvsnoCIBpFahyn9ERAABANijyAgAAIF8I9/E3OgJgWjbfYKMjAACAbFDkBQAAhth45pgWxP6lRFeaivmHqkuhqor2DTI6FgAAAACYDkVeAABwXS2M3aV39vyik45knXKmSDp3k4CRB1apQlCUhlVqrxj/UGNDIk8aceAPSdLjJRoYnAQwn1M7v5MkRVa43dAcAAAgcxR5AQDAdTPh8Hq9v3eZTqWnZHjeLemY46yOOc7qtnVT9E2trioTGGFMSORZB1JPGx0BMK305BNGRwAAANmwGh0AAADkD5vPHtcHe5dfUOD9r0Npibp/09dyedzXKRkAAAAAmNtVG8nbufszSkt1aM7UYbLbz+127YateuKlgbr79jZ6sldXDRszVWvWb5HNZpPT5VTHtjer6x236uixWN350HMqW6q4d38tbmqgxcvOfaUuOSVVsXHxKlmsiCSpTs0qeuaxblcrOgAAhssP/eh7e5bpZHpyjtY97jirhSd3qV3BCtc4FQAgr8gPfSkAAFm5qtM1FIqO0vKV69S86Q2SpLkLlqhyhdKSpBnfLtDJuHhN+myg7Dab0hwOHT7yz1d+AgMCNGnUwAz769G1kyRp3catGvrZlAuWAwCQl+TlfjTRmarNZ4/neP2zLoeGHfidIi8A4JLk5b4UAIDsXNXpGtrfcpO+X7hUknQ2KVmbt+1Sg3o1JEknTp5SRHio7DabJMnP11dlShW7mocHAMDU8nI/eiD1tNJcrkvaJv4i0zoAAPBfebkvBQAgO1d1JG+NqhU0a+5Pio2L1/KV69TipgayWc/VkTu1a65nX/tAazduU82qFVSvdlW1aNpANtu55ckpKere51Xvvj58+3kVio7K8bG/mbNIM+cukiR5PJ6r2CoAAK4PI/tR6dr2pZezP3pzAMClyst9KQAA2bmqRV5JatvqRs1buEzLflurt1/uqwWLV0iSypQqpm8mfKSNW3boz61/adzkmfrx5181pH8/SZl/NeZS3Nmxte7s2FqS5HQ61bR9jytuCwAA15tR/ah0bfvSov5h8rXaLmmbYJvvVTs+IEmdClYyOgJgWiFFbzQ6Qo7l1b4UAIDsXPUib7tWTdXj8ddVomhhFS9aOMMyHx+76tWqqnq1qqpj25t1W9cndDrx7NWOAACAaeXVfjTSJ0BlAiN0zJGzvIFWu3oXq3eNUyG/aRxRwugIgGkFFqxqdIQcy6t9KQAA2bnqRd6CURHq89DdKlk8JsPz6//cruIxhVQgKkKStP2vvQoNCVZIcKCSk5lzDwAAKW/3o6+Uvknd/5ypeGfqRdeN8g1S50JVrkMqAEBek5f7UgAAsnLVi7ySdFubZhc8d/zESQ39bLIcjnT52O0KCPDXB289K6v1qt77DQAA08ur/WiD8OJ6MKa2vjiyTqedaVmuV8g3SOOq3n7J0zsAF7M4bo8kqUVUGYOTAOaTdHy9JCmoUG2Dk+RMXu1LAQDIisXhSMtzs8Gfn/to25Y/jY4CAMgFEg5tNDqC6ZzvS5fPmyC7/ep+Jvz5obUacXCVTqWnKMnl8D4f5ROowr5BGl65g6qGRF/VYwKS9NLOhZKkQRVuMTgJYD7HN4yRJBWq1dvgJOZxvi/90Rovu8XoNACAa+HwI345XrdCl1nXMMk1GskLAACQlYeL1VX3mNr64eROfR+7Q0mudBX2DdZDxeqoajDFXQAAAAC4VBR5AQDAdWe3WtUhupI6RFcyOgoAAAAAmB6TDwEAAAAAAACAiVHkBQAAAAAAAAATo8gLAAAAAAAAACbGnLwAAADIFyoERhkdATAt35BiRkcAAADZoMgLAACAfKFnsbpGRwBMK6LsrUZHAAAA2WC6BgAAAAAAAAAwMYq8AAAAyBeOp53V8bSzRscATMmZckrOlFNGxwAAAFmgyAsAAIB84aP9v+mj/b8ZHQMwpbgd3yhuxzdGxwAAAFmgyAsAAAAAAAAAJkaRFwAAAAAAAABMjCIvAAAAAAAAAJgYRV4AAAAAAAAAMDGKvAAAAAAAAABgYhR5AQAAAAAAAMDE7EYHAAAAAK6Hd8u1MjoCYFrRNXoaHQEAAGSDIi8AAADyBbuVL7EBl8titRkdAQAAZIMrXQAAAAAAAAAwMYq8AAAAyBc+2LtcH+xdbnQMwJRObp2uk1unGx0DAABkgekaAAAAkC/EpacYHQEwLZcj0egIAAAgG4zkBQAAAAAAAAATo8gLAAAAAAAAACZGkRcAAAAAAAAATIwiLwAAAAAAAACYGEVeAAAAAAAAADAxu9EBAAAAgOvh/iI1jI4AmFZYyZZGRwAAANmgyAsAAIB8oUZIYaMjAKblH1HW6AgAACAbTNcAAAAAAAAAACZGkRcAAAD5wtwT2zX3xHajYwCmdObwSp05vNLoGAAAIAsUeQEAAJAv/JpwQL8mHDA6BmBKybF/Kjn2T6NjAACALFDkBQAAAAAAAAATo8gLAAAAAAAAACZGkRcAAAAAAAAATIwiLwAAAAAAAACYmN3oANfSyX1rZbfn6SYCAAAAAHKxoLmzeV8KAHlUBaMD/As9DQAAAPKF2iGFjY4AmJZ/RDmjIwAAgGxQ5AUAAEC+cG+RGkZHAEwrrGQLoyMAAIBsMCcvAAAAAAAAAJgYRV4AAADkC7uTT2l38imjYwCm5DhzRI4zR4yOAQAAskCRFwAAAPnCmENrNObQGqNjAKYUv/t7xe/+3ugYAAAgCxR5AQAAAAAAAMDEKPICAAAAAAAAgIlR5AUAAAAAAAAAE6PICwAAAAAAAAAmRpEXAAAAAAAAAEzMbnQAAAAA4HrwtdiMjgCYlsXqY3QEAACQDYq8AAAAyBf6l29pdATAtKJrPGR0BAAAkA2mawAAAAAAAAAAE6PICwAAAAAAAAAmRpEXAAAA+cIbf/2sN/762egYgCmd2PSFTmz6wugYAAAgC8zJCwAAgHzB4XEZHQEwLY873egIAAAgG4zkBQAAAAAAAAATo8gLAAAAAAAAACZGkRcAAAAAAAAATCxPz8lboFRdoyMAAHKBhEMbjY4AAADyqaQOnWS3GJ0CAHAtHH7EL8frVugy6xomYSQvAAAAAAAAAJhanh7JCwAAAJzXu1g9oyMAphVR9jajIwAAgGxQ5AUAAEC+UDYw0ugIgGn5hsQYHQEAAGSD6RoAAAAAAAAAwMQo8gIAACBfmH50k6Yf3WR0DMCUTu9frNP7FxsdAwAAZIEiLwAAAPKF9WeOaf2ZY0bHAEwpNX6XUuN3GR0DAABkgSIvAAAAAAAAAJgYRV4AAAAAAAAAMDGKvAAAAAAAAABgYhR5AQAAAAAAAMDEKPICAAAAAAAAgInZjQ4AAAAAXA9NwksYHQEwrcCC1Y2OAAAAskGRFwAAAPlCh+hKRkcATCukaCOjIwAAgGwwXQMAAAAAAAAAmBhFXgAAAOQLm84c06Yzx4yOAZhSavxupcbvNjoGAADIAtM1AAAAIF/48ugmSVKNkMIGJwHM5/T+nyVJ/hFlDU4CAAAyw0heAAAAAAAAADAxirwAAAAAAAAAYGIUeQEAAAAAAADAxCjyAgAAAAAAAICJUeQFAAAAAAAAABOzGx0AAAAAuB6ifAKMjgCYls031OgIAAAgGxR5AQAAkC+8WLqp0REA0ypQ5V6jIwAAgGwwXQMAAAAAAAAAmBhFXgAAAOQLTrdbTrfb6BiAKXncLnncLqNjAACALFDkBQAAQL7w2q6f9Nqun4yOAZjSiU3jdWLTeKNjAACALFDkBQAAAAAAAAATo8gLAAAAAAAAACZGkRcAAAAAAAAATIwiLwAAAAAAAACYGEVeAAAAAAAAADAxirwAAAAAAAAAYGJ2owMAAID8ye3x6EhaolJcTkX6BCjKN9DoSMjjnivZ2OgIgGlFVbzT6AgAACAbFHkBAMB1dTo9VaMPrtbXxzcrxe2U2+ORzWJRId9gPVmyoToUrCSrxWJ0TORBhfyCjY4AmJY9INLoCAAAIBsUeQEAwHWzJzle92ycrkOpiXLJk2HZcUeSntk+X1OObNSXNe6Sr9VmUEoAAAAAMBfm5AUAANdFfHqK7t44XftTT19Q4D0vyZWulQkH9ciWb69zOuQH4w+t1fhDa42OAZhS/O75it893+gYAAAgC1e9yJuUnKIWnXrq3Y/GXrDs8NETatz2AX3+ZcY3bvMWLlOrzr3Uvc+ruq/XS3r2tQ907MRJSVL/waM1fdaPVzsmAAC5Vl7tSz/d/7sOpZ6+6HoOj0trTh/RzqST1yEV8pOdyXHamRxndAzAlBxnDslx5pDRMXIsr/alAABk5aoXeX9e+rsqli+tpStWKzklNcOy7xcsVd2aVTRv4TJ5PBlH8NStWUWTRg3U1LGDVKJYYX3y2ZdXOxoAAKaQF/tSl8et2bHb5c7h+ifTkzV434prmgkAkHflxb4UAIDsXPUi79wFS/XA3bepVrVK+mnp797nXS635i1apuf6dldgQIDWbNiS5T4a1K2hA4eOXu1oAACYQl7sS/ckxyvFlX5J22w6c+wapQEA5HV5sS8FACA7V/XGa3v3H9bx2Dg1qFtDLpdLk2Z8r45tb5Yk/bF2k6ILRKp0yaLq0LaZ5v64VPVrV7tgHy6XW4uX/aGK5Utd0rG/mbNIM+cukqQLPo0FAMAs8mpfmuxOl8uT03G857jpzwEAlyGv9qUAAGTnqhZ55/64RO1aNpHNZlWjG2pp0LDPte/AYZUqUVRzf1yq29o0kyS1adFYYyfNVOKZJIWGBEmS1m7cqu59XpUkVShXSk/1uu+Sjn1nx9a6s2NrSZLT6VTT9j2uXsMAALhO8mpfGm73l91iu6Rt7FbuDwsAuHR5tS8FACA7V63I63Q69ePPK2Sz27RwyUpJUmqqQ3N+PPc1md9WbdDWHbs1cfqcv9d3acHiFbqr0y2Szs19NOitZ69WHAAATCcv96Ul/MMU7uOv2PSkHK1vldQmqty1DQUg1zrrdOj72B3am3JK/lYfNYkoqXqhMbJYLEZHQy6Xl/tSAACyc9WKvMtXrlNMkYIa98nb3uf2HTisvv3eVURYiG5qXFf9X33Cu2zl6o0a9cVX3s4UAID8Li/3pRaLRQ8XraO3di1Wmsd10fUL+gapT4kG1yEZ8hM+OMj9zjrT1G/nAq1MOKiTjmQ5/v57EXHQX5E+gXquVGPdVfjCr9bj2gsuUt/oCDmSl/tSAACyc9W+Bzl3wVK1aXFjhudKlSiqggUiNP+n5bqlReMMy+rXqaaTcfHa8dfei+57/JRZ6nj/k95/Py/742rFBgAg18jrfekDMbVUJThaNmU/Ei/U5qfuMbUV7Rt0nZIhv2gRVUYtosoYHQNZSEhPVbu1k/Tt8a06knbGW+CVpHhnqnannNJrf/2kgbuXGpgy/woqVFtBhWobHeOi8npfCgBAViwOR1qemw3+/NxH27b8aXQUAEAukHBoo9ERTOd8X7p83gTZ7VdvCv+zTofu3fSVdiadVIIz9YLlBX0C1T2mll4s3ZSvZQP5TMd1X+qP0wd1sTcn4XZ/fVr5NrUpcPkjszt3f0a+Pnb5+fpKkipVKK1Xn+112fs7b97CZVr229ocf93/69kLlZKaqu73dJQkzflxiSbPmCuPx6O6Nauo35M9cvQ3+KEn3tCTvbqqTs0qV5T/v379fb1+/X2dXn6m51Xdb35xvi/90RovO10aAORJhx/xy/G6FbrMuoZJrvKN1wAAALITbPfV3Nr3a/XpQ/pg3wrtT0mQW27ZLFa1iCyjJ0s0VFH/UKNjIo/6Lf6AJKlxRAmDk+C/diad1K7kuIsWeCUpwZmqD/cuv6IiryT1f/VJVShbMsvlLpdbNtu1uwFkappD02f9oCmj35MkHTl2QmMnfqMJIwYoMiJML771kb6b/4v3Jl5GaNKwtsZNnqmDh48pyjdOkhRYsKpheQAAQNYo8gIAgOvKYrHohvDi+qbWvUZHQT4zO3a7JIq8udFH+37TyfTkHK9/NO2M9iSfUpnAyKuWYd7CZZq/aLlCQ4J14PBRvfx0T9lsVo0cP0NJySlyud168N6OanlTA8UnJOqtQSN18lSCLBaLKpUrpddfeFSSlJySqjffG67d+w7J18dHA157UkWLRF9wvF+Wr1KNqhUU4O8vSVq8fJWaNKyjqMhwSVLn9i01cfqcTIu8m7bs1ODhE+RyuVW5Qmm5XP9MbTFt5nwtWrJSTqdLdrtNz/bprupVymvx8lWaPX+xPnnvZUnnith39nhWHw14UXa7Tf0Hj1ZKSprcHrdualRXj/a4S5LU8qYGmvPDEt1ZL1ESRV4AAHIrirwAAAAADLUj+eQlrZ/gTNWOpJNXVOR9Y+Cn3ukaHu7WWZK0ZcduTRwxQCWLx+jM2SQ9/uJAfdT/BRWIilDC6TPq8fjrql6lvH5e9oeKFC7oLZieTjzr3e+2nXs0adS7iikcrZHjp2vyV3P18tMXTnewbtM2ValY1vv4+Ik4FS5UwPu4SKGCOn4i7oLt0tOdemPgcL32fG/dUKea/lj7p+YtWu5d3rZlE3W941ZJ0uZtu9R/8GjNGP+hmjWup0/HTtX+g0dUsniMlv++VkVjCql0yaL6eNQk3digth68t+MF7alWpbyGjf5Sd9YrfuknGQAAXDcUeQEAAAAYyuO5tNuEuD0euXI0uUPW/jtdw7yFy1S9cnmVLB4jSfpz6186cvSEnnv9wwzbHTh0VNUqldOMWT9q2OgvVat6JTWsV8O7vFrlcoopHP33/8vr69kLMz1+7MlTGbbLqf0Hj8hms+mGOtUkSQ3qVs8wUnjn7v2aMG22EhPPymaz6sCho0pNc8jfz1d33NZKM+f+pOf6dtfMOT95RwnXqlZJw8dNU0pKqmrXqKz6tf8ZrRsVEaYTJ09JosgLAEBuRpEXAAAAgKFi/EO1NSk2x+uH2f1V0j/squcIDPD3/t/j8ah0yWIaO/R/ma47ceS7Wr1+s5asWK0xk77RxBHvSpJ3dLAkWa1WudyuTLf38/OVw5HufVwoOkqHj5zwPj56PFaFoqMuKX96ulOvvDNUwz94VVUqllVSUrJademt9PR0+fv5qmO75rqv94tq16qJDh09rqYN60qSmje9QdWrlNeqdZv1zZyFmvHtj/poQD9JksORnqFNAAAgd7p2dxIAAAAAgBx4tmRjRdj9L77i36J8A1QtuNA1TCRVr1JBR4+f0Kp1m73P7dy9X+npTh05dkKBAf5q1ayhnu/bXQcPHVNKauol7b9c6RI6cOio93HzJjfo19/XKe5Ugjwej76d97NaNWt4wXYli8fI5XJp7YatkqRV6zbr8NFzxWGHw6F0p1OFo89N+/D17EUZtg0NCVLThnX18ttDdfutLbw3ljt4+JgiI8J0a+umerxnV23etsu7zb4DR1S+DPNYAwCQ2zGSFwAAAICh6oXGqJBfsOKdFy+UBtl81KtYPVkslmuaKTQkSEPeeUGfjp2mT8dMldPlVKGCURr01rNat3Gbps36QTarVS6XW0/06qrgoMBL2n/zpjdo4EdjvDc4K1okWo88cIcefe4dSVLtGpXVuX2LC7bz8bGr/6tPnLvxmtutyhXKeIuwQUGBevTBu9TzqTcVFhqiVjdfWCTudGtzzV+0XJ3aNfc+t3j5Ki34eYXsPnZ53B69+NRD3mW/r9mo5k3rS9p6Se0DAADXl8XhSLuyyaxyIafTqabte2jblj+NjgIAyAUSDm00OoLpnO9Ll8+bILudz4SRN4w48Ick6fESDQxOgszsS0lQ5/Vf6nDamSzXCbT56OaI0vqiWudrXuS9Hp5/40P17NYlww3YrrUvv56nfQeP6LXnel103YTTZ/TESwP1xaf9dWbv95KkyAq3X+OEecf5vvRHa7zs5v91BQBk4vAjfjlet0KXWdcwCSN5AQAAkE9Q3M3dSgWEa3bt+/XIlu90ODVRsenJ3mW+FpsK+AaqQ8FKertcizxR4JWkZ/t0174DR67b8e7r9ZIsFos+frdfjtY/fPS4XnzyIfn42CnuAgCQy1HkBQAAAJArlAgI18J6PfRXUpzGHlqjfSnx8rPa1SyylLoWqaEgW966AVixmEIqFnNt5xb+t6ljB13S+lUrlbtGSQAAwNVGkRcAAAD5QkL6uflew31yfoMvGKN8UJQ+qNjG6Bj4F5fjrCTJ5htscBIAAJAZq9EBAAAAgOvhvb3L9N7eZUbHAEzp5NapOrl1qtExAABAFijyAgAAAAAAAICJUeQFAAAAAAAAABOjyAsAAAAgX3n6lffV7bFX1L3Pq3rsuXe0Y9e+HG/b9q7HdPRY7FXLMn3Wj4o7leB9PG7yTH08avJV2/9/zVu4TC+99fElbzdt0T6Nm7M702W//r5effsNuNJoAABcEqdHmuT210OuUPV0hepRV4jedAVpl8dmdDRDcOM1AAAAAPnKgNeeVEhwkCRpyYrVGjB4jCZ/NtCQLDO++1F1alZWVGT4JW/rcrllszFuBwCQP33oCVSKx6Jh1jMKsXgkSWs9dh30WFXO4jI43fVHkRcAAABAvnK+wCtJSUkpsliyXnf5yrUaMX6G7HabGtarkWHZsDFTteHPbXI6XQoKDNDLz/RUyeIx+vLreTp4+JhefqanJOnM2STd9dDzmjF+sMJCg73bj5/yrU7GxeuNgZ/Kz9dXr7/wqCQp7lSCXnhziA4fPa6oiHC9+/pTCgsN1ryFyzR/0XKFhgTrwOGjevnpnrLZrBo5foaSklPkcrv14L0d1fKmBopPSNRbg0bq5KkEWSwWVSpXyrv/5JRUvfnecO3ed0i+Pj4a8NqTKlokWpI05evvNX/hclmsFpUrXUL9nuyh4KDADO12Op36eNRkrVq3WSHBQapVreLl/zAAALgMhzxWrfD4aqr1tLfAK0l1LU7JIu3xWPWJO0hpkhyyqIXFoW7WVEnSRLe/9npsOiuL4mRVUbnUz5qsMItHKz0++tztL6sklyx6yJqiGy3pxjTyElHkBQAAAHBdvbRzYZbLnirRUEX9QyVJEw6v17akzKdGaBVZRq0LlJMk/ZFwSLNObNWgCrfkOMPbH3ymdRu3SpKGDOiX6TqnEk5rwJCx+mzIGypdsqi+m79YpxPPepc/cPdteqr3fZKkRUtW6uNRkzV04Evq2O5m3fNwPz3+yL0KCQ7S9wuXqWmjuhkKvJLUs1tnfb9wqfq/+qQqlC0pSVr22xpt2bFbE4b3V1hoiN4YOFzfzV+sB+/tKEnasmO3Jo4YoJLFY3TmbJIef3GgPur/ggpERSjh9Bn1ePx1Va9SXj8v+0NFChfUJ++9LEkZcm/buUeTRr2rmMLRGjl+uiZ/NVcvP91TK1dv1PcLlmns0P8pJDhI7w8dr5HjZ+jFpx7KkPu7+b9o/6GjmjpmkCTpmVcH5fi8AwBwNezy2BQjl0L/VeD9t8Jy60PrGflapDSP9JQ7RHU86ary9wjfzbJrjDVRkRaPPnEHaLwnQM9ZkvWF21/PWpNVxeKS2yMlK5tPgnMZirwAAADIF94o08zoCMhF/vfiY5KkeYuWaeT46fook0Lvlm27VK50cZUuWVSS1KHNzfpo5CTv8lXrNuubOQuVnJwqt8etxDNJks6NFG7etL6+X7BU93Zpp2+//1n9X30ix9ka1q2hsNAQSVK1yuW0e99B77LqlcurZPEYSdKfW//SkaMn9NzrH2bY/sCho6pWqZxmzPpRw0Z/qVrVK2UYhVytcjnFFI7++//l9fXsc0X31es2q1WzBt6Rzp1va6nXBgyTJAUVrCl3UrIkac2GLWrXqol8fM69nbytTTPNXbAkx+0DAOBqO+Kx6m13kNJkUVWLU49YUjTME6Bdbrus8ihWVu322LxF3gaWdEX+XSBub3HoLfe5vq+2xakR7kDdZHGorsVpqmkfKPICAAAgXwi2+xkdAX/L6YjbHkVr52i9BuHF1CC82GVlad/6Jn0w7AudTjyjFX9s0PRZP0iS7r69zQUjb/89rcOxEyc1ZMREff7pOyoWU0i79hxQnxf+ufnYXZ3a6MW3PlKpEkUVHhaiiuVK5TiTr6+P9/9Wq1Uul9v7ODDA3/t/j8ej0iWLaezQ/2W6n4kj39Xq9Zu1ZMVqjZn0jSaOeFeS5Ofrm3H/7szfwFr+1WCLzS6LNfO3j9lNdwEAwLVQzuLSEY9NZzwWhVg8irG4Ndp2Rgvcvlrh8dHnClCoPBptTZTNIr3lCpIjm1G555f0saZon8eqDR4ffeAOVEuLQ/dY065Po64Qs/QDAAAAyDfOnE1SbFy89/HS39YoLDRYoSHBurV1U00aNVCTRg3UbW2aqVrl8tq196D2HTgiSfp+wVKlpzslSWeTUmS321QgMlwej0ffzFmU4TilSsQopnBBvT90vO7s2DrLPEGBATr79wjZS1W9SgUdPX5Cq9Zt9j63c/d+pac7deTYCQUG+KtVs4Z6vm93HTx0TCmpqdnur36davp52R9K+jvPd/MWq0Hd6heuV7uqFvz8m5xOp9LTnfp+4bLLyg8AwOUqZnGrkSVdQ9yBOuv5p3ib+ne59ozHooJyy2aRDnqsWiufDNv/4fFR/N/b/eDxVR3Luf79gMeqUha3bremqYMlTds85hkfa56kAAAAwBX4ZP9KSdLTJRsZnARGOpuUotcGDFOawyGrxarwsBANfueFDKNWz4sID9Vrz/XSy+8MlY/drob1anhH95YrXVytb26o+3q/rLDQYN3UuO4F23dq11xDRk5Ui6Y3ZJnn7k636P2h4+Xv98+N13IqNCRIQ955QZ+OnaZPx0yV0+VUoYJRGvTWs1q3cZumzfpBtr9HAj/Rq+sFN1D7r0b1a2r3voPq9czbGW68JkkpJ7cpLSXd2649+w6pa6+XvDde275r7yVlBwDgSr1oSdJU+esJd4hskoLlUbjcuteaKh9Jg9xBWujyVYzcqqWMN0+rLqcGuoN08l83XpOkz90BOiib7PLIX9LT1sv7INYIFocjLfMZik3M6XSqafse2rblT6OjAABygYRDG42OYDrn+9Ll8ybIbuczYeQN52/2dSk35wKuxODhExUZEaqH7+9sdJQrdnzDGElSoVq9DU5iHuf70h+t8bIzpQUA5BoT3f5KkkV9rSlXvK/Dj+R8OrAKXWZd8fGyw3QNAAAAAHAVxcbF656e/bRj1z7d07mt0XEAAEA+wNAcAAAAALiKCkZFaMb4D42OAQAAMvGgNfs56s2KkbwAAAAAAAAAYGIUeQEAAAAAAADAxCjyAgAAAAAAAICJMScvAAAA8oUu0VWMjgCYVmjxpkZHAAAA2aDICwAAgHyhQXgxoyMAphUQVdnoCACAXOTwI37XbN9Fx6Vds33nZUzXAAAAACDfOJ14Rt37vOr9d/fDL6hJu+46nXg2R9s/9MQbWrdx61XLM2/hMu07cCTD45fe+viq7f+/1m3cqu59Xr3k7bLLtXvfQXXu/swVJgMAIOdedQXrO/eFheberhAt9/hkus0Ct6/edAVJknZ4bOrvDsp0vRSP1MoVcdEMZz0Wfb8+4/jZ8Ut9te2wMeVWRvICAAAgX1h0cpckqXWBcgYngZHCQkM0adRA7+Mvv56n9X9uV1hosCF55i1apuDgQJUqEXPJ2zpdLtlttmuQ6kKpCXvlTI2/LscCAOBi2lrTNM3tr9v1z6jfHR6bTsmqRkq/6PYVLS69YUm6ogxnZdG8DT66rbbT+1zPZo4r2ueVoMgLAACAfOGnU3skUeTNDY5vGJPlssgKXeQTWECSlLBngdIS92e6XlChOgouUk+SlBK3TYkHl6tQrd6XnGXugqXq8/DdWS7ftGWnBg+fIJfLrcoVSsvlcnmXTZs5X4uWrJTT6ZLdbtOzfbqrepXyWrx8lWbPX6xP3ntZkuRyuXVnj2f10YAXVbpkUe/2c374Rdt37tUnn32p8ZNn6bGHzuVITknVm+8N1+59h+Tr46MBrz2pokWitW7jVg0ePlFVK5fTjr/26sGunVShbEkN/Wyy4hMS5Uh3qlO75rqr0y1KTXNowODR2r3voOw2uyIjQjPk+fDTL7Rpy19yuVx6o9+jqlyhjCTph59+1ZffzJMkFSoYqZee7qnoApFKS9wvZ9ppb/Zxk2dqweLfFBQYoEb1a17yeQcA4Eo0VrqGKVB7PDaVsZzrm3/0+KqRJV0vuYOVLIscsqiWJV2PW1JktWTcfoPHrlHuAI22nZEkfe/21dcef/nLo6aWjEXige5AHfLYlC6poDx6wZqkSItHQ92BSkmX3vjGX1aLR2/fkab35vjplupO1S3tUmKKNGGZr44nWuXxSPf7/azO7VtKkjp3f0btWjbR6vWbFXfqtDq0baaH7rv9is4JRV4AAAAA+dKmLTt15mySbmxQO9Pl6elOvTFwuF57vrduqFNNf6z9U/MWLfcub9uyibrecaskafO2Xeo/eLRmjP9QzRrX06djp2r/wSMqWTxGy39fq6IxhTIUeCWpY7vm+nHxCt3Tua2aNT5XsJ63cJm27dyjSaPeVUzhaI0cP12Tv5qrl5/uKUnad/CIXniyh157rpdcLrceefp/+t+LfVSqRIxSU9P0yDNvqWqlsjpx8pTOJiVr2tgPJCnDdBT7Dx7Rq889on5PPqRZ3/+s0RO+1tCBL2n3voMaPm6avhjeX9EFIjVh6my99/E4ffzuixlyr/hjvRYvW6UJw/srMDBAb38w6gp/EgAAXBq7RWplcehHj6/6WlLk8Ei/eHz1ifWMoi1uBVgkl0d60x2kpfJRc0vWo3v3eqya6AnQZ9ZERVk8Gu/2z7C8ryVF4VaPJGma20+TPAF6xpKsZ6zJ6m0PVf87UzPd7+RffVU43KOn2qQqMUXqP3W2ypcpqWqVzw04OJuUrLFD31LC6TO6s8dzan/LTYouEHn55+SytwQAAACAy5DTEbfhZdrkaL2AqMqXdWOwuQuWql2rJllOebD/4BHZbDbdUKeaJKlB3eoqWiTau3zn7v2aMG22EhPPymaz6sCho0pNc8jfz1d33NZKM+f+pOf6dtfMOT/pzo6tc5yrWuVyiikc/ff/y+vr2Qu9y2IKF1SdGufaeuDQUe3Zf0hvvjfcuzw5OVV7DxxWrWqVtO/AEX346ReqXb2yGt3wz2jbojGFVLXSuTeY1SuX09S/R+6u27hNDevV8L7B7NKhlT6f+q1cLneGfGs2bFHLZg0UFBQoSbr91hbauGVnjtsHAMDV0M6SpufcIerlSdFyj49KyK1CcmuMJ0Cb3XZ5JCXIqlJyq3k2Uzhs8PjoBku6oiznCrkdLGma5gnwLl/s8dVPbl85ZJFDUqg8Ocq35bBNbzc8VwAODZBublJPq9dv9hZ5b2neWJIUHhaimCLROnosliJvVk7uWyu7PU83EQAAAMBlSE5J1eJlf2j8sHe8z61at1nDx06VJLVo2kBNGmY+wlc6N8r3lXeGavgHr6pKxbJKSkpWqy69lZ6eLn8/X3Vs11z39X5R7Vo10aGjx9W0Yd0cZ/Pz9fX+32q1yuX+Z4qIwIB/Rhd5PB6FhgRnmGP436aOHaS1G7Zo9fotGjF+miaOHPj3/v+5IY3VZpXL7c50e4sl06czWS+HK+ZTQXNn874UQN4zq8s123XQ/Hk5Wq+KpOLPvKX1d9yqRd//rE7NG2nOqQSd2XNAn7/4mPx8ffXJ6Clyuz0K6vOA/BYuk/23tQp661kFbNwq62dTFDRqoHy/WyD7rn0KeuHRc8ePjZO6Pa2g+fO0cfMOzR48WmOG/k+R4WFavnKtxk6aqaBRAxV4LFbWvq+pQpcvvZkCVg5QTKO2qtC4nmxTH1OZNkNVpHBBSdK8UZMz5Pf9V39ss1ozTAl1OYy53RsAAAAAGOinpb+rXJkSGW54dkOdapo0aqAmjRqoHvd1UsniMXK5XFq7Yaukc0Xgw0dPSJIcDofSnU4Vjj43f/DXsxdl2H9oSJCaNqyrl98eqttvbSGbLfO3XkGBATqblHxZbShRvIiCAgP0/YKl3ucOHj6m04lndSI2ThaL1LRRXT3Z6z55PNKJ2Lhs91enZmX9vmaTYuPO3WDt23k/q16tqhdkr1+7mhYvW6Wk5BR5PB59N/+Xy8oPAMCV6tCmmSZOn6OtO3erVbOGOnMmSVERYfLz9VXcqQQtXrbqovuoW6uKfl+7SXGnEiRJ385b7F2WeDZJgQH+CgsJUXq6U9/N/2dZYGCA0hwOpac7/7tLSVL92lU1+4dzfWR8QqKWrljj/XbQtcDHiQAAAADynbk/LlWndjdnu46Pj139X33i3I3X3G5VrlBG5cuUkCQFBQXq0QfvUs+n3lRYaIha3dzwgu073dpc8xctV6d2zbM8Rqd2LfTp2C81Y9aP3huv5ZTdZtPgd57X0M+maPq3P8rtcissLERvv9xXu/cd0qjPZ8jjkVwul9q2vFHlypTQuo1bs9xf2VLF9cQjXfXsa+fm8S1UMFIvP9PzgvUa31BLW3fsVo/HX+fGawCQT1XoMsvoCJKkls0aauhnU9SyWUMFBvjr7s5t9Gr/Ybqv10sqEBWh+rWrXnQfZUsVV89uXfTY8/0VEOCvm2+s513WqF4NLfh5he7p+YLCQkNUv3ZVxZ4892FoWGiw2rVsom6PvaLAAH99Mbx/hv0+17e7Pvj0C93/6MuSR3qwa0fvdEnXgsXhSMvZRBIm4nQ61bR9Dy2fN4GvxQAAcBnoS5EXTTi8XpLUo2jWX8EHrqYvv56nfQeP6LXnehkd5Yol7FkgKefzJIO+FABwfdHTAAAAIF+guIvr6b5eL8lisejjd/sZHeWqoLgLAEDuRpEXAAAAAK6yqWMHGR0BAADkI9x4DQAAAPnC4dREHU5NNDoGYErpySeVnnzS6BgAACALFHkBAACQLww78LuGHfjd6BiAKZ3aOUunduaOm+wAAIALUeQFAAAAAAAAABOjyAsAAAAAAAAAJkaRFwAAAAAAAABMjCIvAAAAAAAAAJgYRV4AAAAAAAAAMDG70QGuBY/HI0lyOp0GJwEA5FU2m00Wi8XoGNcMfSnyIrfLLYnfa+ByOK/B64e+FACAK/PvvtTicKR5DM5z1aWmpqp5p0eMjgEAyMOWz5sguz1PflYqib4UAHDt0ZcCAHBl/t2X5ske1dfXVyWKFdaUz97L058M5wbdHntFUz57z+gYeR7n+frgPF8feeU822w2oyNcU76+vvpl9rjrMsoqr/xOnEd7cjfak7vRntztarcnP/SlefV9aV773T6PdpkL7TIX2nVt/LsvzZNFXqvVKqvVKh8fH6Oj5HkWiyVPf/qeW3Cerw/O8/XBeTYHq9Uqf3//63KsvPY7QXtyN9qTu9Ge3C2vteday8vvS/Pq7wLtMhfaZS6069rjxmsAAAAAAAAAYGJ5tsh7R4fWRkfIFzjP1wfn+frgPF8fnGf8V177naA9uRvtyd1oT+6W19pzPeTVc0a7zIV2mQvtMpfc1K48eeM1AAAAAAAAAMgv8uxIXgAAAAAAAADIDyjyAgAAAAAAAICJUeQFAAAAAAAAABOzGx3gWjh4+Jje+XC0TieeUXBQgF5//lGVKVXM6Fimk+Zw6M2BI7T3wGH5+foqIjxU/Z7soeJFC+tUwmm988FnOnz0hHx9fPTCkz1Uu3olScp2GbL2/YKlevejsXr/f8+oWeN6nONrwOFI17AxU/XH2k3y9fVR+TIl9NZLfbP9m8Hfk0v326oNGj3xa3ncHrlcbt13161q3/omfqeRqbz0Gvto5CQt/32djh0/qYkj31WFsiWNjnRFsrsOMKunX3lfcfGnZbVYFBjgr2f7dlfFcqWMjnVF/nv9YGaduz8jXx+7/Hx9JUnd7+moVjc3NDjV5cvqusOMTiee0ZMvved9nJrm0JGjJzRvxkiFhQYbmCx3y0t93L/llddqVv222X9uWbXLzD+3y61N5HbZtatvvwE6djxOwUEBkqR2rZuqa5d2BifOuayuucz++sqqXbnl9ZUnb7z2xIsD1a5VE7W/5SYtXr5KU76aq88/7W90LNNJczi0dsNWNapfUxaLRV/PXqhffl2lkR++rgFDxqhwdJQeeeAObd2xWy+/M1SzJn4su92e7TJk7uixWL35/khJHnW7+zY1a1yPc3wNDP1silwul57r210Wi0VxpxIUFRme7d8M/p5cGo/HozZ3PqaRH76mcmVK6OixWN37yIua/9VIfTxqMr/TuEBeeo2t/3O7ihYuqEef769B/3s2TxR5s7oOMKszZ5MUEhwkSVqyYrXGT/5Wkz8baHCqy5fZ9YOZde7+TJ547ZyX1XVHXvDl1/O0/s/tGvzO80ZHydXyUh/3b3nltZpVv232n1tW7TLzz+1yaxO5XXbt6ttvgO7p3Na0fXtW11xmf31l1a7c8vrKc9M1nEo4rW1/7VGbljdKkpo3qa/jsad08PAxg5OZj5+vrxrfUEsWi0WSVK1yOR09flKStHjZH+rcvqUkqUrFsioQGaF1m7ZfdBku5Ha7NfDjcXr+8e7y8fmnI+IcX10pqamau2CJHutxl/d3OioyPNu/Gfw9uTwWi0VnkpIlSUnJKQoNDZavjw+/07hAXnuN1a5eSdEFo4yOcdVkdx1gVucvyiUpKSlFfzfNlLK6fkDukNV1R14xd8FSdWjbzOgYuVpe6+Pyosz67bzwc8tr1yPS5dcmcru8eK11XmbXXHnh9ZXbryXz3BXhidhTKhAZLrvNJulcsaFQwSgdj40z9dcLc4OvvlugmxrV0enEM3K6XBkuVIsUKqDjsXHZLkPmps38QTWqVlCl8qW9z3GOr77DR04oNCRYE6fP0er1W+Tn66OeD3RRSHBQln8zgpMD+XtyiSwWi/q/+oReeWeoAvz9lHg2Se+98YySU1L4ncYF6LPN5fx1gNm9/cFnWrdxqyRpyIB+Bqe5fJldP+QF73z4mTwej6pULKu+D9+jiPBQoyNdlqyuO+rXrmZ0tCu2actOnTmbpBsb1DY6Sq6W1/u4vPJa/S9+buaQk9qEGf33Wmvk+BkaM/EblS5RVH0evkdFi0QbmO7S/feaK6+8vrK6lswNr688N5IX18aEabN16Mhx9XnoHqOj5Cm79x3UkhWr9dB9nYyOkue5XC4dO35SpUoU1RfD++vZvt31xsDhcrlcRkfLU5wulyZM+07vvfmMvp38iT59/xW98+EouVxuo6MBuAJ56Trgfy8+ptlfDlPvHndq5PjpRse5LHn1+mHU4Nc15bP3NHHEAIWHhqj/4NFGR7psWV13nIo/bXS0KzZ3wVK1a9XE+yYd+U9eeq3mJ3nl55aXrkn+7b/t+l+/Ppox/kNN+ew91axWUS+8OdjghJcuL1xzZSazduWW11eeK/JGF4zUyVMJcv5duPF4PDoeG6dCeezrCtfTl1/P09IVa/TRgH7y9/dTWGiIbFab4k4leNc5evykChWMynYZLrTxzx06ejxWdz38gjp3f0Zbtu3WoKGf6+elf3COr7JC0QVktVrUpsW5r4ZULFdKMYUL6ujxk1n+zeDvyaX7a/d+nYxL8N7soErFsoouEKldew7wO40L8Bozh/9eB+QV7VvfpLUbt+p04hmjo1yyrK4fZs39yehoV6RwdAFJkt1u1z2d22jj5h0GJ7p8WV137Np70OBkVyY5JVWLl/2h225hqoaLyct9XF56rf4XP7fc7VJqE2aS2bVWoehzbbBYLLqr0y06cjTWlNcs0j/XXNEF8tbr69/Xkrnl9ZXniryR4WGqWK6UFvy8QpL0y6+rFV0g0lRDv3OTaTPna9GSlfrkvZczzD3S4qYb9O28nyVJW3fsVmxcvOrUqHTRZcioS4dW+n7aCH07aai+nTRUVSuX1UvPPKwuHVpxjq+y8LAQ1atVVX+s3SRJOnLshI4ci1WNqhWy/JvB35NLV6hglOJOJWjfgcOSzt2d+NCREypRvAi/07gAr7HcL6vrADM6czZJsXHx3sdLf1ujsNBghYYEG5jq8mR3/WBWKampOnM2yft40ZKVht+85Epkdd1RqkSMwcmuzE9Lf1e5MiVM347rIa/2cXnttfpf/Nxyr8upTZhBZu1yulwZvvnxy/JViowIVVhoiFExL0lW11wR4aGmfn1l1S5fX59c8/qyOBxpHkOOfA3tP3hEA4aM0enEswoKDNBrz/dWudLFjY5lOidi49Sp29MqWiRagQH+kiQfHx+NH/a2TsWf1tsfjNKRY7Hysdv1/OMPqm6tKpKU7TJk79930OQcX32Hj57QwI/GKiHxjKwWqx6+/3Y1b3pDtn8z+Hty6Rb+8psmTp8jq8Uqt8et7vd0VJsWjfmdRqby0mvs/U/G67dVG3Tq1GmFhgYrMMBf30z4yOhYly276wAzOnr8pF4bMExpDoesFqvCw0L0ZO/7TPcmNzNmvwO3dK6PfqX/J3K73fJ4PCpaOFrP9nlARQoXNDraZcvqusPMej3ztjq1u1m3tWEkb07kpT7uvLz0Ws2q3zb7zy2zdn3y3sum/rldbm0it8uqXcM/eEV9X3hXjvR0WS1WhYUF6+ne96u8Sa5ZsrvmMvPrK6t2BQUG5JrXV54s8gIAAAAAAABAfpHnpmsAAAAAAAAAgPyEIi8AAAAAAAAAmBhFXgAAAAAAAAAwMYq8AAAAAAAAAGBiFHkBAAAAAAAAwMQo8gIAAAAAAACAiVHkBQAAAAAAAAATo8gLAAAASBr0yecaMW660TGumtZdemvdxq1GxwAAAMB1YDc6AICro2+/Adq8bZfsNpvsdrvKli6up3rfp8oVyhgdDQCQj53vn3zsdlksFhUqGKUGdavrgXs6KCI89LpmSUpO0cjx07Vs5TolJSUrJDhINapWUP9Xn5AkvfT0w9c1z3njJs/UX7sPaNBbz2Z4vm+/AbqpUT3d26XtNc9wPY8FAMi7Nm7eoQnTZmvL9l3yeKTC0QXUpkVj3dO5rXx8rl0Jin4MoMgL5Cl9H75X93Zpq/R0p0ZP/FqvvPOJvpvyidGxAAD53Pn+yePxaN+BI/r8y2/10BNv6PNP31FkRNh1y/HJ6Ck6GZegiSMGKDIiTCdi4/TrHxuu2/EBAMjLfv19vf73/gj16n6n/vdiH4WHhWjfgSOa/NVcnTyVoCKFChgdEcjTKPICeZCPj123tmqqL7+ep/iERIUEB2rMpJlauPg3pTkcqluzip5//EFFhIdq8oy52rl7v3cUU4/HX5fdbtO4T96WJL3yzieqXqW87rvzVi1YvELjJs/SqfjTCgwMUOf2LfTw/Z2NbCoAwEQsFotKlyyq/73URw/2eU1TZ87XE490VXJKqt56f6Q2b9slR3q6ypcpoef6dlf5siUVn5Co27s9rWnjBimmcLQkKc3hUIeuT+jjd19U+TIl9cGnX+jX39fJ6XSpUMEovfZ8L1WpWPaC42/Ztlvd7r7NW1iOLhilLre19C7vP3i0goMC9WyfB3T0WKy6PPis3uz3mMZPmaXTiWd1U+O6euWZnrLbz11Cb/9rr4aPnaadu/fLZrOqVbOGev7xByVJO/7aq2FjpuqvPQcUGhKkB+7uoE63Nr/sczdv4TLN+PZHNapfU9/N/0UB/n564J7bdEeH1pIkt9utsZNnavb8X2SzWvVg104Ztt+xa58+GjlJe/cfls1mVf3aVfX84w8qLDREw0Z/qY2bd2jztl0aM/Fr1axWUR+/+6KSU1I1cvwM/fr7Ojkc6WpQr4aef7y7goMCL7sdAIC8yePx6ONRk9Tt7tsyjKYtVSJGb7zwqCRp2849+njUZO3df1gFosL10H2365bmjSVl7IMl6czZJN1yx6OaNfFjFSlcUP0Hj5bdZlNySqp+W7VBBaLC9dJTD6tOzSpZ9mNAfsOcvEAelJrm0NwFSxQeFqKQkCBNnD5XK/5Yr88+ekMzJ34si8WitwaNlCTVqVlF6zZtkyQlnklSbFy8Dh4+rqTkFHk8Hq3btE11a1VRSmqq+g8eo1ef66WfvxunqWPeV8N6NYxsJgDApOw2m25qXFfrN22XJHncbt3SvLFmTvpI82aMUIVyJfX6wE/l8XgUER6qGxvU0vxFy73bL12xRgUiI1S1UjnN/2m5du05oK+/GKJFs8bovTefVlREeKbHrV61vD7/8lt9N3+xdu05II/Hc9GsK1dv1MSR72rq2EFas36LFiz+TZJ04uQpPfHiQDVvWl9zp32qbycNVcubGkiS4k4l6KlXBqnzbS31w1ejNOitZzVu8kytXr/5is7bnn2HZLFY9P204er/6hMa+fkMrf/z3Dmct2i55i9crpEfvq6vvxii7Tv3KDklxbut1WJR34fv0fwZI/Tl6PcVezJeI8fPkCQ99ej9qlmtovo+fK8Wzx7vfWP87pAxSjxzVpM/G6iZkz6Sy+XUkBETr6gNAIC86eDhYzpyLFatb26U6fIzZ5P07GsfqPXNjfTDVyPV78mH9P7Q8dq4ZWeOj/HTst/VuX0LLZw5Rm1bNlH/IWMkZd2PAfkNRV4gDxn1xQy17tJbLTr11MJfVuq9N56W3WbTjz//qoe63q7C0QUUGOCvpx69X6vWbVZsXLwqlS+ttDSH9u4/rHWbtqpWtYqqXqW8Nm7eoZ2790uSKpQtKUmy223ad+Cwdx7DzEZJAQCQEwWjIpR45qwkKSgoUK1ubqgAf3/5+frqkQfu0IFDxxQbFy9J6tD2Zv3w06/eouz8RcvV/pabJMk7qmffgSPyeDwqUayICkVHZXrM5/p0V+f2LTVv4XL1fOp/an/P45o2c362OR++v7OCAgNUMCpCDevV0Pa/9kqSFvy8QpXKl9YdHVrLz9dX/v5+qlW9kiTph59/Va3qFdWqWUPZbFaVLVVc7W+5SQt/WXlF58zf30+PPNBFPj52Va9SXm2a36gffjpX/F6weIXu6nSLSpWIkb+/n/r2vFdu9z9F7PJlS6pmtYqy2+2KjAjTvXe0837Im5n4hEQtWbFaLzzRQyHBQQrw91ev7nfqp6W/y+VyX1E7AAB5T/zpM5KkggUiMl3+26oNCg8L1V2dbpHdbledGpV1S/PG+uFfH+JeTOP6tVSnZhXZbFa1v+UmHTt+UqcTz1yV/EBewHQNQB7S56F7dG+Xtjpx8pRe/N9H2rX3oGpVr6QTJ0+pSOF/5j8qGBUhXx8fnYg9pYJREapZraLWbtyq/QePqG7NKnKkO7V241ZFRYSrTo3KslgsCvD314dvP6dpM3/QiHHTVbZ0cfXufqfq1qpiYIsBAGYVGxev0JBgSee+gfLpmC/12+qNSjxzVlbLuXEIp0+fUXSBSDWoW13pTqfWb9qmYkULa/2m7Xqz32OSpLatmujkqQR9MOwLHY+NU9NGdfRkr/sUHhZywTF9fX1035236r47b5XDka7Fy//Qux+NVZlSxdWgbvVMc0ZF/jNnsL+/n84mJUuSjp04qWJFC2W6zdHjJ7Vy9Ua17tLb+5zb7VbNahUzXd9us8vpcl7wvNPpkt1u8z4uEBXhnSpCkgoXivKOhj4Zl6DC0f/09ZERYfL18fE+Pnj4mD4dM1Xbdu5RcmqqPG5Phn1f2IZYud0e3fFgxpvBWS1WxcUnKLpAZJbbAgDyn/DQc3167Ml4FYu5sH88EXvqgjl5Y4oU1IY/d+T4GP+exz/A30+SlJycqrDQC/t8ID+iyAvkQdEFIvXKMz3V54UBanZjPUUXiNTRYydVtVI5See+RupIT1d0wXNv0OrWrKJ1G7dq34EjurNja6WnO/XuR2MVGRGqhvVqevdbv3Y11a9dTU6nUzPn/qSX3v5YC2eOltXKlwIAADnndLm0bOVaNa5fS5I0beZ8bf9rn0YPeUPRBaO88/CdH4dqtVrVvvVNmrdouUoUK6IGdat73+jZbTb16NpJPbp20qn403rzvREaP2WWd27crPj6+qhtyyaaPutH7d57MMsib1YKRxfQH+v+zHRZoYKRata4nne++4vuq1CUFvwSm+E5t9uto8diM7whPhkXL6fT6S30Hj8R5x0xVSAqXMdOnPSueyrhtBzp6d7HHwz7QiWKFdYb/QYpJDhIS39bowGDx3iXny+s/9OGKFmtFs2d+qn8/34jDQBAVkoUK6IihQrqpyW/q8d9nS5YHl0wUkePn8zw3NFjJ70fGgb4+ystzeFdFncq4ZKO/99+DMiPeBUAeVTF8qVVu0ZlTZw2R21b3qiJ0+fo+Ik4Jaek6pPRX6p+7WoqGHXujWHdmpX1x7o/dTY5WSWLx6hs6eI6cfKU1m/arnq1qkqSTsWf1pIVq5WUnCKbzaagwADZbFmPAAIAIDP7DhxR/w9HKykpRV3vaCdJSkpOka+vj0JCgpSckqrPvvjqgu1ua9NMS1as1twfl+i2Ns28z6/ZsEU7d++X0+WSv7+ffH19suyfxk+ZpU1bdio1zSGXy63lK9dq74HDqla53CW3o02Lxtq2Y49mff+zHI50paamacPf8+O2bdlEazds1S/LV8npdMrpdGrn7v3aumN3pvtqVL+mTsUn6uvZC5XmcCg1zaEvps6WxWpR7RqVveulpqbp8y+/U3q6U1u279KCxb+pTfMbJUm33NxIM+cu0v6DR5Sa5tCoz7+S1WrxbpuUnKLAgAAFBQbo+Ik4ffn1vAwZIiNCdfjoce/jqMhw3dSongaPmKiEv7+CG3cqQUtWrL7kcwUAyPssFoue69tdk7+aq69nL/ROo3Dg0FG9+9FY1ahaUfEJiZo5d5GcLpc2/LldC3/5Te1aNZEkVSxfSn+s3aSTcfFKSk7R+CnfXtLx/9uPAfkRI3mBPKxH10564sWBmj7+Q6WkpqnXs2/J4UhXnZpV9NZLfbzrlS9bUjarTXX+fiNpsVhUu3olbdi8XaVLFpV0bkTRV98t0LtDxsrtcatE0SIa+PpTjOIFAFzUyM+na+ykb2SxWFSwQIQa1aupz4e/o8jwc6Nxu3Zpp/+9P1Lt73lcYWEh6t39Ts36/ucM+yhaJFqVy5fRnv2H1LhBLe/zp+JPa/DwiToRGyc/P1/Vr11VPbt1zjSHzWrTh59O0JHjJ2S1WBVTpKBeffYR1aha4ZLbFF0wSsPef0XDx07VqM9nyMduV+vmDVWreiVFF4jUxwNf1Mjx0zVo2Odyuz0qVSJGvbrfkem+wkJD9Ml7L2n4uGkaP2WWrBarKpYvpaHvvqTAAH/vemVKFZPL5dJtXZ+Qv5+vHu1xl3fapNvaNNORY7Hq8/wAWW0WPXhvJy35NcC77dOP3q9Bn3yumXMXqXixwmrb4kbt3X/Yu/yezm3Vf/AYte7SWzWqVtCQ/i/o9Rd6a9zkmXr4yTd1+swZRYaHqVWzhrr5xvqXfL4AAHlfk4a19dGAfvpi6ncaM/EbSVKh6Ci1bXmjCkSG66MB/TT0syka9flXKhAVoX5P9vBOZdS2xY1at3Gb7n3kRYWHhahnty76aenvOT52Zv0YkN9YHI60i99WGAAAADDYgCFjFBoSrKd632d0lOtu3sJlmvHtj5o0aqDRUQAAAJALMQQPAAAAud6hI8f1y6+r1Ll9C6OjAAAAALkO0zUAAAAgV3v/k/Fa9MtKPXB3BxUvWtjoOAAAAECuw3QNAAAAAAAAAGBiTNcAAAAAAAAAACZGkRcAAAAAAAAATIwiLwAAAAAAAACY2P8By0O6438ae6AAAAAASUVORK5CYII=",
|
||
"text/plain": [
|
||
"<Figure size 1400x400 with 3 Axes>"
|
||
]
|
||
},
|
||
"metadata": {},
|
||
"output_type": "display_data"
|
||
}
|
||
],
|
||
"source": [
|
||
"fig, axes = plt.subplots(1, 3, figsize=(14, 4), constrained_layout=True)\n",
|
||
"\n",
|
||
"axes[0].barh(report[\"symbol\"].to_list(), report[\"rows\"].to_list(), color=COLORS[\"blue\"])\n",
|
||
"axes[0].set_xlabel(\"Rows\")\n",
|
||
"axes[0].set_title(\"Data Volume\")\n",
|
||
"\n",
|
||
"stale_days = report[\"days_stale\"].to_list()\n",
|
||
"freshness_color = [\n",
|
||
" COLORS[\"positive\"] if d <= 3 else COLORS[\"amber\"] if d <= 7 else COLORS[\"negative\"]\n",
|
||
" for d in stale_days\n",
|
||
"]\n",
|
||
"# Plot bars; zero-width bars (days_stale == 0) get a small marker so the\n",
|
||
"# panel is not blank when every symbol is fresh.\n",
|
||
"axes[1].barh(report[\"symbol\"].to_list(), stale_days, color=freshness_color)\n",
|
||
"zero_mask = [i for i, d in enumerate(stale_days) if d == 0]\n",
|
||
"if zero_mask:\n",
|
||
" axes[1].scatter(\n",
|
||
" [0] * len(zero_mask),\n",
|
||
" [report[\"symbol\"].to_list()[i] for i in zero_mask],\n",
|
||
" color=[freshness_color[i] for i in zero_mask],\n",
|
||
" s=80,\n",
|
||
" marker=\"o\",\n",
|
||
" zorder=3,\n",
|
||
" label=\"Fresh (0 days)\",\n",
|
||
" )\n",
|
||
"axes[1].axvline(3, color=COLORS[\"positive\"], linestyle=\"--\", alpha=0.6, label=\"3-day threshold\")\n",
|
||
"axes[1].axvline(7, color=COLORS[\"amber\"], linestyle=\"--\", alpha=0.6, label=\"7-day threshold\")\n",
|
||
"axes[1].set_xlim(left=-0.5)\n",
|
||
"axes[1].set_xlabel(\"Days Since Update\")\n",
|
||
"axes[1].set_title(\"Data Freshness\")\n",
|
||
"axes[1].legend(fontsize=8, loc=\"lower right\")\n",
|
||
"\n",
|
||
"gaps = report[\"gaps\"].to_list()\n",
|
||
"issues = report[\"issues\"].to_list()\n",
|
||
"axes[2].barh(report[\"symbol\"].to_list(), gaps, color=COLORS[\"negative\"], label=\"Gaps\")\n",
|
||
"axes[2].barh(\n",
|
||
" report[\"symbol\"].to_list(), issues, left=gaps, color=COLORS[\"amber\"], label=\"Validation\"\n",
|
||
")\n",
|
||
"axes[2].set_xlabel(\"Count\")\n",
|
||
"axes[2].set_title(\"Data Issues\")\n",
|
||
"axes[2].legend(fontsize=8, loc=\"lower right\")\n",
|
||
"\n",
|
||
"fig.suptitle(\"Data Health Dashboard\")\n",
|
||
"plt.show()"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "ac84f7a5",
|
||
"metadata": {
|
||
"papermill": {
|
||
"duration": 0.001681,
|
||
"end_time": "2026-06-13T02:34:34.920089+00:00",
|
||
"exception": false,
|
||
"start_time": "2026-06-13T02:34:34.918408+00:00",
|
||
"status": "completed"
|
||
}
|
||
},
|
||
"source": [
|
||
"---\n",
|
||
"\n",
|
||
"## 5. The Book's Update Workflow\n",
|
||
"\n",
|
||
"The book's `data/download_all.py` script implements exactly this pattern\n",
|
||
"for all asset classes. Run it with `--update` to extend datasets to the present:\n",
|
||
"\n",
|
||
"```bash\n",
|
||
"# Initial download (run once)\n",
|
||
"python data/download_all.py\n",
|
||
"\n",
|
||
"# Update to present (run daily/weekly)\n",
|
||
"python data/download_all.py --update\n",
|
||
"```\n",
|
||
"\n",
|
||
"Under the hood, `download_all.py` uses:\n",
|
||
"- `ETFDataManager.from_config(\"data/etfs/config.yaml\")` → `manager.update()`\n",
|
||
"- `CryptoDataManager.from_config(\"data/crypto/config.yaml\")` → `manager.update()`\n",
|
||
"- `MacroDataManager.from_config(\"data/macro/config.yaml\")` → `manager.download_treasury_yields()`\n",
|
||
"- `FuturesDataManager.from_config(\"data/futures/config.yaml\")` → `manager.download_all()`\n",
|
||
"\n",
|
||
"Each manager reads its YAML config for symbols, date ranges, and provider settings,\n",
|
||
"then updates only what's new.\n",
|
||
"\n",
|
||
"### Config-Driven Downloads\n",
|
||
"\n",
|
||
"The ETF config at `data/etfs/config.yaml` defines:\n",
|
||
"```yaml\n",
|
||
"etfs:\n",
|
||
" provider: yahoo\n",
|
||
" start: '2006-01-01'\n",
|
||
" end: '2025-12-31'\n",
|
||
" frequency: daily\n",
|
||
" tickers:\n",
|
||
" us_equity_broad:\n",
|
||
" symbols: [SPY, QQQ, IWM, ...]\n",
|
||
" us_sectors:\n",
|
||
" symbols: [XLB, XLC, XLE, ...]\n",
|
||
" # ... 9 categories, 100 ETFs total\n",
|
||
"```\n",
|
||
"\n",
|
||
"When you run `--update` in 2026+, it extends data beyond the configured end date\n",
|
||
"to the present — no config changes needed."
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "9185dd8a",
|
||
"metadata": {
|
||
"papermill": {
|
||
"duration": 0.001176,
|
||
"end_time": "2026-06-13T02:34:34.922504+00:00",
|
||
"exception": false,
|
||
"start_time": "2026-06-13T02:34:34.921328+00:00",
|
||
"status": "completed"
|
||
}
|
||
},
|
||
"source": [
|
||
"---\n",
|
||
"\n",
|
||
"## Summary\n",
|
||
"\n",
|
||
"### Key Patterns\n",
|
||
"\n",
|
||
"| Pattern | Command | When |\n",
|
||
"|---------|---------|------|\n",
|
||
"| Initial load | `dm.load(symbol, start, end)` | First time |\n",
|
||
"| Daily update | `dm.update(symbol, lookback_days=7)` | Every trading day |\n",
|
||
"| Gap check | `updater.detect_gaps(df, \"daily\")` | Before backtesting |\n",
|
||
"| Full refresh | `UpdateStrategy.FULL_REFRESH` | After data corruption |\n",
|
||
"| Batch update | `download_all.py --update` | Cron job |\n",
|
||
"\n",
|
||
"### Production Checklist\n",
|
||
"\n",
|
||
"1. **Initial download**: `python data/download_all.py` (run once, ~10 min)\n",
|
||
"2. **Schedule updates**: Add `download_all.py --update` to cron (daily at 6 PM)\n",
|
||
"3. **Monitor health**: Check freshness, gaps, and validation before backtests\n",
|
||
"4. **Validate data**: Use `OHLCVValidator` on every load (see `13_data_quality_framework`)\n",
|
||
"\n",
|
||
"### Key Takeaways\n",
|
||
"\n",
|
||
"- **`lookback_days` is the operating lever**, not the symbol set. Set it once\n",
|
||
" to cover provider revision windows (Yahoo retro-adjusts ~5 trading days; CRSP\n",
|
||
" ~30) and the strategy generalizes across instruments.\n",
|
||
"- **Gap detection runs against stored data, not stream data.** A daily cron\n",
|
||
" that finishes with `detect_gaps()` catches missed trading days before any\n",
|
||
" backtest reads stale partitions.\n",
|
||
"- **Full refresh is the recovery path.** Use `UpdateStrategy.FULL_REFRESH`\n",
|
||
" when validation flags a regression; never patch a corrupted parquet in place.\n",
|
||
"- **Configurable end dates eliminate config drift.** Symbol-list YAMLs use\n",
|
||
" today-as-default; `--update` extends data beyond the file's `end:` field\n",
|
||
" without needing edits each calendar year.\n",
|
||
"- **Cron + idempotent CLI is the production interface.** The same\n",
|
||
" `download_all.py --update` works in a notebook, a CI run, and a 6 PM cron.\n",
|
||
"\n",
|
||
"Next: `20_storage_benchmark_file` compares file-format throughput for the\n",
|
||
"parquet write path implicit in every update; `21_storage_benchmark_database`\n",
|
||
"extends the same comparison to database engines.\n",
|
||
"\n",
|
||
"### Cross-References\n",
|
||
"\n",
|
||
"- **Data quality**: `13_data_quality_framework` — validation and anomaly detection.\n",
|
||
"- **DataManager basics**: `18_data_management` — fetch, batch, Universe, storage.\n",
|
||
"- **Storage benchmarks**: `20_storage_benchmark_file` (file formats) and\n",
|
||
" `21_storage_benchmark_database` (database engines).\n",
|
||
"- **Download scripts**: `data/download_all.py` — the book's orchestrator.\n",
|
||
"- **ml4t-data docs**: [ml4trading.io/docs/data/user-guide/incremental-updates/](https://ml4trading.io/docs/data/user-guide/incremental-updates/)"
|
||
]
|
||
}
|
||
],
|
||
"metadata": {
|
||
"kernelspec": {
|
||
"display_name": "Python 3 (ipykernel)",
|
||
"language": "python",
|
||
"name": "python3"
|
||
},
|
||
"language_info": {
|
||
"codemirror_mode": {
|
||
"name": "ipython",
|
||
"version": 3
|
||
},
|
||
"file_extension": ".py",
|
||
"mimetype": "text/x-python",
|
||
"name": "python",
|
||
"nbconvert_exporter": "python",
|
||
"pygments_lexer": "ipython3",
|
||
"version": "3.14.3"
|
||
},
|
||
"papermill": {
|
||
"default_parameters": {},
|
||
"duration": 5.777638,
|
||
"end_time": "2026-06-13T02:34:35.938814+00:00",
|
||
"environment_variables": {},
|
||
"exception": null,
|
||
"input_path": "02_financial_data_universe/19_incremental_updates.ipynb",
|
||
"output_path": "02_financial_data_universe/19_incremental_updates.ipynb",
|
||
"parameters": {},
|
||
"start_time": "2026-06-13T02:34:30.161176+00:00",
|
||
"version": "2.7.0"
|
||
},
|
||
"jupytext": {
|
||
"cell_metadata_filter": "tags,-all"
|
||
},
|
||
"ml4t_provenance": {
|
||
"source_py_blob": "efccccee2c22ee49cad07144ca4dd390ddec8908",
|
||
"executed_at": "2026-06-13T02:34:36.157814+00:00",
|
||
"executor": "cpu-local",
|
||
"production": true,
|
||
"parameters": {},
|
||
"notes": "executed-state finalization batch (2026-06-13 run)"
|
||
}
|
||
},
|
||
"nbformat": 4,
|
||
"nbformat_minor": 5
|
||
}
|