wehub-resource-sync
119 lines
6.1 KiB
Markdown
119 lines
6.1 KiB
Markdown
# Equity Data
|
||
|
||
Equity datasets organised by **type**: market data (OHLCV, options,
|
||
microstructure), fundamentals (SEC filings + XBRL), positioning (13F,
|
||
Form 4), and firm characteristics (standalone packaged dataset).
|
||
|
||
## Directory
|
||
|
||
```
|
||
equities/
|
||
├── loader.py # All equities loaders
|
||
├── README.md # (this file)
|
||
│
|
||
├── market/ # Market data (OHLCV, options, microstructure)
|
||
│ ├── us_equities/ # NASDAQ Data Link Wiki Prices (1962-2018)
|
||
│ ├── sp500/ # AlgoSeek daily bars + options slices
|
||
│ ├── nasdaq100/ # AlgoSeek minute bars
|
||
│ ├── microstructure/ # TAQ / MBO / ITCH / IEX tick data
|
||
│ ├── sp500.csv # Index reference file
|
||
│ └── us_equities_profile.json
|
||
│
|
||
├── fundamentals/ # SEC regulatory filings (text + XBRL)
|
||
│ ├── 10k/sp100/ # Per-ticker 10-K text (S&P 100)
|
||
│ ├── 10q/sp500/mda.parquet # 10-Q MD&A aggregate (S&P 500)
|
||
│ ├── 8k/sp100/ # 8-K event filings (S&P 100)
|
||
│ ├── xbrl/ # XBRL financial facts (CIK × quarter × concept)
|
||
│ ├── filings_download.py # Unified 10-K / 10-Q / 8-K downloader
|
||
│ ├── xbrl_download.py
|
||
│ └── README.md
|
||
│
|
||
├── positioning/ # SEC regulatory filings (positions / insider activity)
|
||
│ ├── 13f/ # 13F institutional holdings (per-cik + bulk)
|
||
│ ├── form4/ # Form 4 insider transactions (raw XML)
|
||
│ ├── 13f_download.py
|
||
│ ├── form4_download.py
|
||
│ └── README.md
|
||
│
|
||
└── firm_characteristics/ # Standalone packaged dataset (Chen-Pelger-Zhu)
|
||
├── firm_characteristics_all.parquet
|
||
├── download.py
|
||
└── README.md
|
||
```
|
||
|
||
## Loaders (all in `equities/loader.py`)
|
||
|
||
### Market
|
||
|
||
| Loader | Dataset |
|
||
| ---------------------------------- | ---------------------------------------------------------- |
|
||
| `load_sp500_index()` | S&P 500 index OHLCV (bundled) |
|
||
| `load_us_equities()` | 3,199 US stocks (1962-2018, frozen) — NASDAQ DL |
|
||
| `load_sp500_daily_bars()` | S&P 500 daily OHLCV — AlgoSeek |
|
||
| `load_sp500_options()` | Raw options chains (legacy) — AlgoSeek |
|
||
| `load_sp500_options_eda()` | Options EDA slice (8 symbols, 2019-2020) — AlgoSeek slim |
|
||
| `load_sp500_options_straddles_raw()` | ATM-band raw chains, lifecycle-preserving (2017-2021) |
|
||
| `load_sp500_options_surface()` | Daily IV surface summary |
|
||
| `load_sp500_options_straddles()` | Daily ATM straddles |
|
||
| `load_nasdaq100_bars()` | NASDAQ-100 minute bars (resampling, quotes, microstructure)|
|
||
| `load_nasdaq100_taq()` | TAQ tick data (AAPL, 2020-03-13 / 2020-03-16) |
|
||
| `load_mbo_data()` | MBO order book data — Databento |
|
||
| `load_nasdaq_itch()` | NASDAQ ITCH messages |
|
||
| `load_iex_hist()` | IEX DEEP/TOPS data (free) |
|
||
|
||
### Fundamentals
|
||
|
||
| Loader | Dataset |
|
||
| ---------------------------------- | ---------------------------------------------------------- |
|
||
| `load_sp500_10q_mda()` | S&P 500 10-Q MD&A text (2017-2021) |
|
||
| `load_sec_filings(form_type, universe)` | 10-K / 10-Q / 8-K aggregate text |
|
||
| `resolve_sec_filings_dir()` | Per-ticker filings directory (Ch22 RAG) |
|
||
| `load_sec_xbrl_fundamentals()` | XBRL financial facts (CIK × quarter × concept) |
|
||
|
||
### Positioning
|
||
|
||
| Loader | Dataset |
|
||
| ---------------------------------- | ---------------------------------------------------------- |
|
||
| `load_institutional_holdings_13f()` | 13F holdings (per-cik, 10 curated managers) |
|
||
| `load_13f_bulk_holdings(quarter)` | 13F full universe (~3M rows per quarter) |
|
||
| `load_13f_stock_features()` | Stock-level features (breadth, concentration, value) |
|
||
| `load_13f_edges()` | Institution → stock edge list (graph construction) |
|
||
|
||
### Firm Characteristics
|
||
|
||
| Loader | Dataset |
|
||
| ---------------------------------- | ---------------------------------------------------------- |
|
||
| `load_firm_characteristics()` | Chen-Pelger-Zhu panel (~180 features; returns + accounting)|
|
||
|
||
## Quick Start
|
||
|
||
```bash
|
||
# Market
|
||
uv run python data/equities/market/us_equities/download.py
|
||
|
||
# Fundamentals
|
||
uv run python data/equities/fundamentals/filings_download.py --form 10-Q --universe sp500
|
||
uv run python data/equities/fundamentals/xbrl_download.py
|
||
|
||
# Positioning
|
||
uv run python data/equities/positioning/13f_download.py
|
||
uv run python data/equities/positioning/form4_download.py --ticker TSLA --count 20
|
||
|
||
# Firm characteristics
|
||
uv run python data/equities/firm_characteristics/download.py
|
||
```
|
||
|
||
## Book Usage
|
||
|
||
- **Ch2** — market datasets appear in `02_financial_data_universe/*_eda.py`.
|
||
- **Ch3** — microstructure (TAQ / MBO / ITCH / IEX).
|
||
- **Ch4** — fundamentals (XBRL, SEC filings), positioning (13F, Form 4).
|
||
- **Ch8** — fundamentals + macro cross-sectional features.
|
||
- **Ch10** — filing-text signals from 10-Q MD&A; asset embeddings from 13F.
|
||
- **Ch11-14** — firm characteristics as the standard ML asset-pricing benchmark.
|
||
- **Ch16-20** — NASDAQ-100 + S&P 500 options case studies.
|
||
- **Ch22-23** — RAG over SP100 10-Ks; institutional ownership graph.
|
||
|
||
See the sub-READMEs in each type directory for license, disk footprint,
|
||
download commands, and on-disk layout details.
|