wehub-resource-sync
1509 lines
58 KiB
Plaintext
1509 lines
58 KiB
Plaintext
{
|
||
"cells": [
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "adaf08d3",
|
||
"metadata": {
|
||
"tags": []
|
||
},
|
||
"source": [
|
||
"# Text Data Extraction: Structuring Unstructured Financial Documents\n",
|
||
"\n",
|
||
"**Chapter 4: Fundamental and Alternative Data**\n",
|
||
"**Docker image**: `ml4t`\n",
|
||
"**Section Reference**: See Section 4.5 for text dataset engineering concepts\n",
|
||
"\n",
|
||
"## Purpose\n",
|
||
"\n",
|
||
"Corporate filings contain valuable information locked in unstructured text. This notebook\n",
|
||
"demonstrates how to extract and structure high-value text blocks (MD&A, Risk Factors) from\n",
|
||
"10-K and 10-Q filings, creating clean datasets ready for NLP analysis in later chapters.\n",
|
||
"\n",
|
||
"## Learning Objectives\n",
|
||
"\n",
|
||
"After completing this notebook, you will be able to:\n",
|
||
"- Navigate SEC EDGAR filing structure\n",
|
||
"- Extract specific sections from 10-K/10-Q filings (MD&A, Risk Factors)\n",
|
||
"- Clean and normalize extracted text\n",
|
||
"- Create point-in-time correct text datasets\n",
|
||
"- Prepare data for downstream NLP analysis\n",
|
||
"\n",
|
||
"## Cross-References\n",
|
||
"\n",
|
||
"- **Upstream**: SEC EDGAR filings via `02_sec_filing_explorer.py`\n",
|
||
"- **Downstream**: Chapter 10 `08_text_feature_evaluation.py` (NLP feature engineering)\n",
|
||
"- **Related**: `02_sec_filing_explorer.py` (filing access)\n",
|
||
"\n",
|
||
"## Key Concepts\n",
|
||
"\n",
|
||
"- **10-K**: Annual report with comprehensive company information\n",
|
||
"- **10-Q**: Quarterly report with interim financial information\n",
|
||
"- **MD&A**: Management's Discussion and Analysis of Financial Condition\n",
|
||
"- **Risk Factors**: Required disclosure of material risks\n",
|
||
"- **Item Numbers**: Standard sections in SEC filings (Item 7 = MD&A)"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 1,
|
||
"id": "959cc2b5",
|
||
"metadata": {
|
||
"execution": {
|
||
"iopub.execute_input": "2026-05-16T23:42:29.853419Z",
|
||
"iopub.status.busy": "2026-05-16T23:42:29.853286Z",
|
||
"iopub.status.idle": "2026-05-16T23:42:30.362862Z",
|
||
"shell.execute_reply": "2026-05-16T23:42:30.362342Z"
|
||
},
|
||
"tags": []
|
||
},
|
||
"outputs": [],
|
||
"source": [
|
||
"\"\"\"Text Data Extraction — extract and structure high-value text blocks from SEC filings for NLP analysis.\"\"\"\n",
|
||
"\n",
|
||
"import warnings\n",
|
||
"\n",
|
||
"warnings.filterwarnings(\"ignore\")\n",
|
||
"\n",
|
||
"import os\n",
|
||
"import re\n",
|
||
"from datetime import datetime\n",
|
||
"from pathlib import Path\n",
|
||
"\n",
|
||
"import numpy as np\n",
|
||
"import polars as pl\n",
|
||
"from bs4 import BeautifulSoup\n",
|
||
"from edgar import Company, set_identity"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 2,
|
||
"id": "7b2a7030",
|
||
"metadata": {
|
||
"execution": {
|
||
"iopub.execute_input": "2026-05-16T23:42:30.367215Z",
|
||
"iopub.status.busy": "2026-05-16T23:42:30.367058Z",
|
||
"iopub.status.idle": "2026-05-16T23:42:30.369174Z",
|
||
"shell.execute_reply": "2026-05-16T23:42:30.368881Z"
|
||
},
|
||
"tags": [
|
||
"parameters"
|
||
]
|
||
},
|
||
"outputs": [],
|
||
"source": [
|
||
"# Production defaults — Papermill injects overrides for CI\n",
|
||
"EDGAR_TICKER = \"AAPL\"\n",
|
||
"EDGAR_FORM = \"10-K\""
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "0daaf10f",
|
||
"metadata": {
|
||
"tags": []
|
||
},
|
||
"source": [
|
||
"## 1. SEC Filing Structure\n",
|
||
"\n",
|
||
"10-K filings follow a standard structure:\n",
|
||
"\n",
|
||
"| Item | Section | Content |\n",
|
||
"|------|---------|---------|\n",
|
||
"| Item 1 | Business | Company description and operations |\n",
|
||
"| Item 1A | Risk Factors | Material risks to the business |\n",
|
||
"| Item 7 | MD&A | Management's analysis of financial condition |\n",
|
||
"| Item 7A | Quantitative Disclosures | Market risk exposures |\n",
|
||
"| Item 8 | Financial Statements | Audited financials |\n",
|
||
"\n",
|
||
"The most valuable sections for NLP analysis:\n",
|
||
"- **Item 1A (Risk Factors)**: Sentiment, risk themes, changes over time\n",
|
||
"- **Item 7 (MD&A)**: Management's perspective on performance"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 3,
|
||
"id": "b317fde4",
|
||
"metadata": {
|
||
"execution": {
|
||
"iopub.execute_input": "2026-05-16T23:42:30.377937Z",
|
||
"iopub.status.busy": "2026-05-16T23:42:30.377815Z",
|
||
"iopub.status.idle": "2026-05-16T23:42:30.381933Z",
|
||
"shell.execute_reply": "2026-05-16T23:42:30.381575Z"
|
||
},
|
||
"lines_to_next_cell": 2,
|
||
"tags": []
|
||
},
|
||
"outputs": [],
|
||
"source": [
|
||
"# Define section patterns for extraction\n",
|
||
"# Note: 10-K and 10-Q have different item numbers for the same content\n",
|
||
"# 10-K: MD&A = Item 7, Risk Factors = Item 1A\n",
|
||
"# 10-Q: MD&A = Item 2 (Part I), Risk Factors = Item 1A (Part II, if updated)\n",
|
||
"\n",
|
||
"SECTION_PATTERNS_10K = {\n",
|
||
" \"risk_factors\": {\n",
|
||
" # Line-anchored patterns to avoid Table of Contents matches\n",
|
||
" \"start_patterns\": [\n",
|
||
" r\"^\\s*ITEM\\s*1A\\.?\\s*[-–—]?\\s*RISK\\s*FACTORS\",\n",
|
||
" r\"^\\s*Item\\s*1A\\.?\\s*[-–—]?\\s*Risk\\s*Factors\",\n",
|
||
" ],\n",
|
||
" \"end_patterns\": [\n",
|
||
" r\"^\\s*ITEM\\s*1B\\b\",\n",
|
||
" r\"^\\s*ITEM\\s*2\\b\",\n",
|
||
" r\"^\\s*Item\\s*1B\\b\",\n",
|
||
" r\"^\\s*Item\\s*2\\b\",\n",
|
||
" ],\n",
|
||
" \"item_number\": \"1A\",\n",
|
||
" },\n",
|
||
" \"mda\": {\n",
|
||
" # Handle both straight and curly apostrophes: ' (U+0027) and ' (U+2019)\n",
|
||
" \"start_patterns\": [\n",
|
||
" r\"^\\s*ITEM\\s*7\\.?\\s*[-–—]?\\s*MANAGEMENT(?:'|\\u2019)?S?\\s*DISCUSSION\",\n",
|
||
" r\"^\\s*Item\\s*7\\.?\\s*[-–—]?\\s*Management(?:'|\\u2019)?s?\\s*Discussion\",\n",
|
||
" ],\n",
|
||
" \"end_patterns\": [\n",
|
||
" r\"^\\s*ITEM\\s*7A\\b\",\n",
|
||
" r\"^\\s*ITEM\\s*8\\b\",\n",
|
||
" r\"^\\s*Item\\s*7A\\b\",\n",
|
||
" r\"^\\s*Item\\s*8\\b\",\n",
|
||
" ],\n",
|
||
" \"item_number\": \"7\",\n",
|
||
" },\n",
|
||
" \"business\": {\n",
|
||
" \"start_patterns\": [\n",
|
||
" r\"^\\s*ITEM\\s*1\\.?\\s*[-–—]?\\s*BUSINESS\",\n",
|
||
" r\"^\\s*Item\\s*1\\.?\\s*[-–—]?\\s*Business\",\n",
|
||
" ],\n",
|
||
" \"end_patterns\": [\n",
|
||
" r\"^\\s*ITEM\\s*1A\\b\",\n",
|
||
" r\"^\\s*ITEM\\s*2\\b\",\n",
|
||
" r\"^\\s*Item\\s*1A\\b\",\n",
|
||
" r\"^\\s*Item\\s*2\\b\",\n",
|
||
" ],\n",
|
||
" \"item_number\": \"1\",\n",
|
||
" },\n",
|
||
"}\n",
|
||
"\n",
|
||
"# 10-Q patterns differ from 10-K\n",
|
||
"SECTION_PATTERNS_10Q = {\n",
|
||
" \"risk_factors\": {\n",
|
||
" # In 10-Q, risk factor updates appear in Part II, Item 1A\n",
|
||
" # Line-anchored patterns with MULTILINE support (handled in extract_section)\n",
|
||
" \"start_patterns\": [\n",
|
||
" r\"^\\s*ITEM\\s*1A\\.?\\s*[-–—]?\\s*RISK\\s*FACTORS\",\n",
|
||
" r\"^\\s*Item\\s*1A\\.?\\s*[-–—]?\\s*Risk\\s*Factors\",\n",
|
||
" ],\n",
|
||
" \"end_patterns\": [\n",
|
||
" r\"^\\s*ITEM\\s*1B\\b\",\n",
|
||
" r\"^\\s*ITEM\\s*2\\b\",\n",
|
||
" r\"^\\s*Item\\s*2\\b\",\n",
|
||
" ],\n",
|
||
" \"item_number\": \"1A\",\n",
|
||
" # For 10-Q risk factors, prefer matches after PART II\n",
|
||
" \"require_after\": r\"PART\\s*II\",\n",
|
||
" },\n",
|
||
" \"mda\": {\n",
|
||
" # In 10-Q, MD&A is Part I, Item 2 (NOT Item 7)\n",
|
||
" # Handle both straight and curly apostrophes\n",
|
||
" \"start_patterns\": [\n",
|
||
" r\"^\\s*ITEM\\s*2\\.?\\s*[-–—]?\\s*MANAGEMENT(?:'|\\u2019)?S?\\s*DISCUSSION\",\n",
|
||
" r\"^\\s*Item\\s*2\\.?\\s*[-–—]?\\s*Management(?:'|\\u2019)?s?\\s*Discussion\",\n",
|
||
" ],\n",
|
||
" \"end_patterns\": [\n",
|
||
" r\"^\\s*ITEM\\s*3\\b\",\n",
|
||
" r\"^\\s*Item\\s*3\\b\",\n",
|
||
" ],\n",
|
||
" \"item_number\": \"2\",\n",
|
||
" },\n",
|
||
"}\n",
|
||
"\n",
|
||
"# Default to 10-K patterns (backward compatible)\n",
|
||
"SECTION_PATTERNS = SECTION_PATTERNS_10K\n",
|
||
"\n",
|
||
"\n",
|
||
"def get_patterns_for_form(form_type: str) -> dict:\n",
|
||
" \"\"\"Get appropriate section patterns based on filing type.\"\"\"\n",
|
||
" form_type_upper = form_type.upper() if form_type else \"10-K\"\n",
|
||
" if \"10-Q\" in form_type_upper:\n",
|
||
" return SECTION_PATTERNS_10Q\n",
|
||
" return SECTION_PATTERNS_10K"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "c6b9fe4a",
|
||
"metadata": {
|
||
"lines_to_next_cell": 2,
|
||
"tags": []
|
||
},
|
||
"source": [
|
||
"## 2. Text Cleaning Functions\n",
|
||
"\n",
|
||
"Raw SEC filings contain HTML, special characters, and formatting that must be cleaned."
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 4,
|
||
"id": "7da03d4e",
|
||
"metadata": {
|
||
"execution": {
|
||
"iopub.execute_input": "2026-05-16T23:42:30.388405Z",
|
||
"iopub.status.busy": "2026-05-16T23:42:30.388225Z",
|
||
"iopub.status.idle": "2026-05-16T23:42:30.391214Z",
|
||
"shell.execute_reply": "2026-05-16T23:42:30.390834Z"
|
||
},
|
||
"tags": []
|
||
},
|
||
"outputs": [],
|
||
"source": [
|
||
"def clean_html(html_text: str, drop_tables: bool = False) -> str:\n",
|
||
" \"\"\"Remove HTML tags and convert to plain text while preserving paragraph boundaries.\"\"\"\n",
|
||
" if not html_text:\n",
|
||
" return \"\"\n",
|
||
"\n",
|
||
" soup = BeautifulSoup(html_text, \"html.parser\")\n",
|
||
"\n",
|
||
" # Remove script and style elements\n",
|
||
" for script in soup([\"script\", \"style\"]):\n",
|
||
" script.decompose()\n",
|
||
"\n",
|
||
" # Optionally remove tables (embedded financial tables in narrative sections)\n",
|
||
" if drop_tables:\n",
|
||
" for table in soup.find_all(\"table\"):\n",
|
||
" table.decompose()\n",
|
||
"\n",
|
||
" # Preserve paragraph boundaries by using newline separator\n",
|
||
" text = soup.get_text(separator=\"\\n\")\n",
|
||
"\n",
|
||
" return text"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 5,
|
||
"id": "cc7ac3e0",
|
||
"metadata": {
|
||
"execution": {
|
||
"iopub.execute_input": "2026-05-16T23:42:30.394871Z",
|
||
"iopub.status.busy": "2026-05-16T23:42:30.394712Z",
|
||
"iopub.status.idle": "2026-05-16T23:42:30.397931Z",
|
||
"shell.execute_reply": "2026-05-16T23:42:30.397610Z"
|
||
},
|
||
"tags": []
|
||
},
|
||
"outputs": [],
|
||
"source": [
|
||
"def clean_text(text: str) -> str:\n",
|
||
" \"\"\"\n",
|
||
" Clean extracted text while preserving paragraph breaks.\n",
|
||
"\n",
|
||
" Removes common filing artifacts and normalizes whitespace without\n",
|
||
" flattening everything into a single line (preserves paragraph structure\n",
|
||
" for downstream chunking and change detection).\n",
|
||
" \"\"\"\n",
|
||
" if not text:\n",
|
||
" return \"\"\n",
|
||
"\n",
|
||
" # Normalize line endings\n",
|
||
" text = text.replace(\"\\r\\n\", \"\\n\").replace(\"\\r\", \"\\n\")\n",
|
||
"\n",
|
||
" # Remove common filing artifacts (line-anchored for safety)\n",
|
||
" text = re.sub(r\"(?im)^\\s*table of contents\\s*$\", \"\", text)\n",
|
||
" text = re.sub(r\"(?im)^\\s*page\\s+\\d+\\s*$\", \"\", text)\n",
|
||
" text = re.sub(r\"(?im)^\\s*\\d+\\s*of\\s*\\d+\\s*$\", \"\", text)\n",
|
||
"\n",
|
||
" # Remove URLs\n",
|
||
" text = re.sub(r\"https?://\\S+\", \"\", text)\n",
|
||
"\n",
|
||
" # Remove excessive special characters but keep newlines\n",
|
||
" text = re.sub(r\"[_=\\-]{3,}\", \" \", text)\n",
|
||
" text = re.sub(r\"[•●◦▪]\", \" \", text)\n",
|
||
"\n",
|
||
" # Normalize spaces within each line, but preserve line breaks\n",
|
||
" text = \"\\n\".join(re.sub(r\"[ \\t]+\", \" \", line).strip() for line in text.split(\"\\n\"))\n",
|
||
"\n",
|
||
" # Collapse excessive blank lines to paragraph breaks\n",
|
||
" text = re.sub(r\"\\n{3,}\", \"\\n\\n\", text).strip()\n",
|
||
"\n",
|
||
" return text"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 6,
|
||
"id": "d33bc058",
|
||
"metadata": {
|
||
"execution": {
|
||
"iopub.execute_input": "2026-05-16T23:42:30.401595Z",
|
||
"iopub.status.busy": "2026-05-16T23:42:30.401460Z",
|
||
"iopub.status.idle": "2026-05-16T23:42:30.407030Z",
|
||
"shell.execute_reply": "2026-05-16T23:42:30.406614Z"
|
||
},
|
||
"tags": []
|
||
},
|
||
"outputs": [
|
||
{
|
||
"name": "stdout",
|
||
"output_type": "stream",
|
||
"text": [
|
||
"Sample cleaning:\n",
|
||
"Original HTML length: 228\n",
|
||
"Cleaned text: ITEM 1A. RISK FACTORS\n",
|
||
"\n",
|
||
"The following risks could\n",
|
||
"materially\n",
|
||
"affect our business:\n",
|
||
"\n",
|
||
"Competition may increase\n",
|
||
"\n",
|
||
"Economic conditions may deteriorate\n",
|
||
"\n",
|
||
"Page 15 of 120\n"
|
||
]
|
||
}
|
||
],
|
||
"source": [
|
||
"def normalize_for_analysis(text: str) -> str:\n",
|
||
" \"\"\"\n",
|
||
" Normalize text for NLP analysis.\n",
|
||
"\n",
|
||
" Converts to lowercase, removes numbers (optional),\n",
|
||
" and standardizes formatting.\n",
|
||
" \"\"\"\n",
|
||
" if not text:\n",
|
||
" return \"\"\n",
|
||
"\n",
|
||
" # Convert to lowercase\n",
|
||
" text = text.lower()\n",
|
||
"\n",
|
||
" # Standardize common abbreviations\n",
|
||
" text = re.sub(r\"\\bu\\.s\\.?\\b\", \"united states\", text)\n",
|
||
" text = re.sub(r\"\\be\\.g\\.?\\b\", \"for example\", text)\n",
|
||
" text = re.sub(r\"\\bi\\.e\\.?\\b\", \"that is\", text)\n",
|
||
"\n",
|
||
" # Remove possessive 's\n",
|
||
" text = re.sub(r\"'s\\b\", \"\", text)\n",
|
||
"\n",
|
||
" # Remove numbers (optional - depends on use case)\n",
|
||
" # text = re.sub(r\"\\d+\", \"\", text)\n",
|
||
"\n",
|
||
" return text\n",
|
||
"\n",
|
||
"\n",
|
||
"# Test cleaning\n",
|
||
"sample_html = \"\"\"\n",
|
||
"<p>ITEM 1A. RISK FACTORS</p>\n",
|
||
"<p>The following risks could <b>materially</b> affect our business:</p>\n",
|
||
"<p>• Competition may increase</p>\n",
|
||
"<p>• Economic conditions may deteriorate</p>\n",
|
||
"<p>Page 15 of 120</p>\n",
|
||
"<p>Table of Contents</p>\n",
|
||
"\"\"\"\n",
|
||
"\n",
|
||
"cleaned = clean_text(clean_html(sample_html))\n",
|
||
"print(\"Sample cleaning:\")\n",
|
||
"print(f\"Original HTML length: {len(sample_html)}\")\n",
|
||
"print(f\"Cleaned text: {cleaned}\")"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "b1c3e33b",
|
||
"metadata": {
|
||
"lines_to_next_cell": 2,
|
||
"tags": []
|
||
},
|
||
"source": [
|
||
"## 3. Section Extraction"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 7,
|
||
"id": "69aba95d",
|
||
"metadata": {
|
||
"execution": {
|
||
"iopub.execute_input": "2026-05-16T23:42:30.417832Z",
|
||
"iopub.status.busy": "2026-05-16T23:42:30.417637Z",
|
||
"iopub.status.idle": "2026-05-16T23:42:30.428151Z",
|
||
"shell.execute_reply": "2026-05-16T23:42:30.427688Z"
|
||
},
|
||
"tags": []
|
||
},
|
||
"outputs": [
|
||
{
|
||
"name": "stdout",
|
||
"output_type": "stream",
|
||
"text": [
|
||
"BUSINESS (728 chars):\n",
|
||
"Our company is a leading provider of technology solutions for enterprise\n",
|
||
"customers worldwide. We design, manufacture, and support a portfolio of\n",
|
||
"hardware, software, and cloud services that help organi...\n",
|
||
"\n",
|
||
"RISK_FACTORS (541 chars):\n",
|
||
"An investment in our securities involves a high degree of risk.\n",
|
||
"You should carefully consider the following risk factors:\n",
|
||
"\n",
|
||
"COMPETITION RISKS\n",
|
||
"We face significant competition from larger companies with ...\n",
|
||
"\n",
|
||
"MDA (424 chars):\n",
|
||
"AND ANALYSIS OF FINANCIAL CONDITION\n",
|
||
"\n",
|
||
"Overview\n",
|
||
"The past year has been transformative for our company.\n",
|
||
"Revenue increased 25% driven by strong product adoption.\n",
|
||
"\n",
|
||
"Results of Operations\n",
|
||
"Our gross margin im...\n",
|
||
"\n"
|
||
]
|
||
}
|
||
],
|
||
"source": [
|
||
"def extract_section(text: str, section_name: str, patterns: dict = None) -> str | None:\n",
|
||
" \"\"\"\n",
|
||
" Extract a specific section from a 10-K/10-Q filing using anchored item boundaries.\n",
|
||
"\n",
|
||
" Robust to TOC duplicates by finding all start_pattern matches and\n",
|
||
" picking the candidate (start, end) span with the most content\n",
|
||
" between it and the next end_pattern. Inline references inside\n",
|
||
" narrative text are filtered out by the same span-length test —\n",
|
||
" they don't have a matching end-pattern Item N+1 immediately after.\n",
|
||
"\n",
|
||
" Parameters\n",
|
||
" ----------\n",
|
||
" text : str\n",
|
||
" Full filing text\n",
|
||
" section_name : str\n",
|
||
" Name of section to extract (e.g., 'risk_factors', 'mda')\n",
|
||
" patterns : dict\n",
|
||
" Custom patterns, or use SECTION_PATTERNS\n",
|
||
"\n",
|
||
" Returns\n",
|
||
" -------\n",
|
||
" str or None\n",
|
||
" Extracted section text, or None if no candidate has substantive content.\n",
|
||
" \"\"\"\n",
|
||
" if not text:\n",
|
||
" return None\n",
|
||
"\n",
|
||
" if patterns is None:\n",
|
||
" patterns = SECTION_PATTERNS\n",
|
||
"\n",
|
||
" if section_name not in patterns:\n",
|
||
" raise ValueError(f\"Unknown section: {section_name}\")\n",
|
||
"\n",
|
||
" config = patterns[section_name]\n",
|
||
" flags = re.IGNORECASE | re.MULTILINE | re.DOTALL\n",
|
||
"\n",
|
||
" # Some patterns may require matching after a specific marker\n",
|
||
" # (e.g., PART II for 10-Q risk factors).\n",
|
||
" offset = 0\n",
|
||
" search_text = text\n",
|
||
" require_after = config.get(\"require_after\")\n",
|
||
" if require_after:\n",
|
||
" after_match = re.search(require_after, search_text, flags)\n",
|
||
" if after_match:\n",
|
||
" offset = after_match.start()\n",
|
||
" search_text = search_text[after_match.start() :]\n",
|
||
"\n",
|
||
" # Collect every candidate start position from every start pattern.\n",
|
||
" starts: list[int] = []\n",
|
||
" for pattern in config[\"start_patterns\"]:\n",
|
||
" starts.extend(offset + m.end() for m in re.finditer(pattern, search_text, flags))\n",
|
||
" if not starts:\n",
|
||
" return None\n",
|
||
" starts = sorted(set(starts))\n",
|
||
"\n",
|
||
" # For each candidate start, find the next end_pattern match and\n",
|
||
" # measure the span. The real section beats every TOC entry on\n",
|
||
" # span length because TOC entries are followed by the next TOC\n",
|
||
" # entry within a few dozen characters.\n",
|
||
" best_span: tuple[int, int] | None = None\n",
|
||
" for start_pos in starts:\n",
|
||
" end_pos = len(text)\n",
|
||
" for end_pattern in config[\"end_patterns\"]:\n",
|
||
" end_match = re.search(end_pattern, text[start_pos:], flags)\n",
|
||
" if end_match:\n",
|
||
" end_pos = start_pos + end_match.start()\n",
|
||
" break\n",
|
||
" span = end_pos - start_pos\n",
|
||
" if best_span is None or span > (best_span[1] - best_span[0]):\n",
|
||
" best_span = (start_pos, end_pos)\n",
|
||
"\n",
|
||
" if best_span is None or (best_span[1] - best_span[0]) < 200:\n",
|
||
" return None\n",
|
||
"\n",
|
||
" return clean_text(text[best_span[0] : best_span[1]])\n",
|
||
"\n",
|
||
"\n",
|
||
"# Create sample 10-K text for demonstration\n",
|
||
"sample_10k = \"\"\"\n",
|
||
"FORM 10-K\n",
|
||
"ANNUAL REPORT\n",
|
||
"\n",
|
||
"Table of Contents\n",
|
||
"\n",
|
||
"PART I\n",
|
||
"\n",
|
||
"ITEM 1. BUSINESS\n",
|
||
"\n",
|
||
"Our company is a leading provider of technology solutions for enterprise\n",
|
||
"customers worldwide. We design, manufacture, and support a portfolio of\n",
|
||
"hardware, software, and cloud services that help organizations modernize\n",
|
||
"their data infrastructure.\n",
|
||
"\n",
|
||
"We operate across three reportable segments — Cloud Platform, Enterprise\n",
|
||
"Software, and Professional Services — and sell directly through a global\n",
|
||
"sales force as well as a network of channel partners. Our customers span\n",
|
||
"financial services, healthcare, public sector, and manufacturing.\n",
|
||
"\n",
|
||
"Recent strategic initiatives include expanding our hyperscale data center\n",
|
||
"footprint, deepening integrations with leading public cloud providers,\n",
|
||
"and broadening our generative-AI product portfolio.\n",
|
||
"\n",
|
||
"ITEM 1A. RISK FACTORS\n",
|
||
"\n",
|
||
"An investment in our securities involves a high degree of risk.\n",
|
||
"You should carefully consider the following risk factors:\n",
|
||
"\n",
|
||
"COMPETITION RISKS\n",
|
||
"We face significant competition from larger companies with more resources.\n",
|
||
"New entrants may disrupt our market position.\n",
|
||
"\n",
|
||
"ECONOMIC RISKS\n",
|
||
"Economic downturns could reduce customer spending on our products.\n",
|
||
"Currency fluctuations may impact our international revenue.\n",
|
||
"\n",
|
||
"REGULATORY RISKS\n",
|
||
"Changes in regulations could increase our compliance costs.\n",
|
||
"Data privacy laws continue to evolve across jurisdictions.\n",
|
||
"\n",
|
||
"ITEM 1B. UNRESOLVED STAFF COMMENTS\n",
|
||
"\n",
|
||
"None.\n",
|
||
"\n",
|
||
"ITEM 2. PROPERTIES\n",
|
||
"\n",
|
||
"We lease our headquarters in San Francisco, California.\n",
|
||
"\n",
|
||
"PART II\n",
|
||
"\n",
|
||
"ITEM 7. MANAGEMENT'S DISCUSSION AND ANALYSIS OF FINANCIAL CONDITION\n",
|
||
"\n",
|
||
"Overview\n",
|
||
"The past year has been transformative for our company.\n",
|
||
"Revenue increased 25% driven by strong product adoption.\n",
|
||
"\n",
|
||
"Results of Operations\n",
|
||
"Our gross margin improved to 72% from 68% in the prior year.\n",
|
||
"Operating expenses remained well controlled.\n",
|
||
"\n",
|
||
"Liquidity and Capital Resources\n",
|
||
"We ended the year with $500 million in cash.\n",
|
||
"We believe current resources are sufficient for operations.\n",
|
||
"\n",
|
||
"ITEM 7A. QUANTITATIVE AND QUALITATIVE DISCLOSURES ABOUT MARKET RISK\n",
|
||
"\n",
|
||
"We are exposed to interest rate and foreign currency risks.\n",
|
||
"\n",
|
||
"ITEM 8. FINANCIAL STATEMENTS\n",
|
||
"\"\"\"\n",
|
||
"\n",
|
||
"for section in [\"business\", \"risk_factors\", \"mda\"]:\n",
|
||
" extracted = extract_section(sample_10k, section)\n",
|
||
" if extracted:\n",
|
||
" preview = extracted[:200] + \"...\" if len(extracted) > 200 else extracted\n",
|
||
" print(f\"{section.upper()} ({len(extracted)} chars):\")\n",
|
||
" print(preview)\n",
|
||
" print()"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "438d5831",
|
||
"metadata": {
|
||
"lines_to_next_cell": 2,
|
||
"tags": []
|
||
},
|
||
"source": [
|
||
"## 4. Building a Text Dataset"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 8,
|
||
"id": "0b5c7317",
|
||
"metadata": {
|
||
"execution": {
|
||
"iopub.execute_input": "2026-05-16T23:42:30.438564Z",
|
||
"iopub.status.busy": "2026-05-16T23:42:30.438490Z",
|
||
"iopub.status.idle": "2026-05-16T23:42:30.445368Z",
|
||
"shell.execute_reply": "2026-05-16T23:42:30.444964Z"
|
||
},
|
||
"tags": []
|
||
},
|
||
"outputs": [
|
||
{
|
||
"data": {
|
||
"text/html": [
|
||
"<div><style>\n",
|
||
".dataframe > thead > tr,\n",
|
||
".dataframe > tbody > tr {\n",
|
||
" text-align: right;\n",
|
||
" white-space: pre-wrap;\n",
|
||
"}\n",
|
||
"</style>\n",
|
||
"<small>shape: (4, 11)</small><table border=\"1\" class=\"dataframe\"><thead><tr><th>cik</th><th>company_name</th><th>accession_no</th><th>filing_type</th><th>filing_date</th><th>accepted_at</th><th>period_end</th><th>section</th><th>text</th><th>text_length</th><th>word_count</th></tr><tr><td>str</td><td>str</td><td>str</td><td>str</td><td>datetime[μs]</td><td>datetime[μs]</td><td>datetime[μs]</td><td>str</td><td>str</td><td>i64</td><td>i64</td></tr></thead><tbody><tr><td>"0000320193"</td><td>"Apple Inc."</td><td>"0000320193-24-000081"</td><td>"10-K"</td><td>2024-10-31 00:00:00</td><td>2024-10-31 16:35:12</td><td>2024-09-28 00:00:00</td><td>"risk_factors"</td><td>"An investment in our securitie…</td><td>541</td><td>74</td></tr><tr><td>"0000320193"</td><td>"Apple Inc."</td><td>"0000320193-24-000081"</td><td>"10-K"</td><td>2024-10-31 00:00:00</td><td>2024-10-31 16:35:12</td><td>2024-09-28 00:00:00</td><td>"mda"</td><td>"AND ANALYSIS OF FINANCIAL COND…</td><td>424</td><td>64</td></tr><tr><td>"0000789019"</td><td>"Microsoft Corporation"</td><td>"0000789019-24-000042"</td><td>"10-K"</td><td>2024-07-30 00:00:00</td><td>2024-07-30 16:05:45</td><td>2024-06-30 00:00:00</td><td>"risk_factors"</td><td>"An investment in our securitie…</td><td>541</td><td>74</td></tr><tr><td>"0000789019"</td><td>"Microsoft Corporation"</td><td>"0000789019-24-000042"</td><td>"10-K"</td><td>2024-07-30 00:00:00</td><td>2024-07-30 16:05:45</td><td>2024-06-30 00:00:00</td><td>"mda"</td><td>"AND ANALYSIS OF FINANCIAL COND…</td><td>424</td><td>64</td></tr></tbody></table></div>"
|
||
],
|
||
"text/plain": [
|
||
"shape: (4, 11)\n",
|
||
"┌───────────┬───────────┬───────────┬───────────┬───┬───────────┬───────────┬───────────┬──────────┐\n",
|
||
"│ cik ┆ company_n ┆ accession ┆ filing_ty ┆ … ┆ section ┆ text ┆ text_leng ┆ word_cou │\n",
|
||
"│ --- ┆ ame ┆ _no ┆ pe ┆ ┆ --- ┆ --- ┆ th ┆ nt │\n",
|
||
"│ str ┆ --- ┆ --- ┆ --- ┆ ┆ str ┆ str ┆ --- ┆ --- │\n",
|
||
"│ ┆ str ┆ str ┆ str ┆ ┆ ┆ ┆ i64 ┆ i64 │\n",
|
||
"╞═══════════╪═══════════╪═══════════╪═══════════╪═══╪═══════════╪═══════════╪═══════════╪══════════╡\n",
|
||
"│ 000032019 ┆ Apple ┆ 000032019 ┆ 10-K ┆ … ┆ risk_fact ┆ An invest ┆ 541 ┆ 74 │\n",
|
||
"│ 3 ┆ Inc. ┆ 3-24-0000 ┆ ┆ ┆ ors ┆ ment in ┆ ┆ │\n",
|
||
"│ ┆ ┆ 81 ┆ ┆ ┆ ┆ our secur ┆ ┆ │\n",
|
||
"│ ┆ ┆ ┆ ┆ ┆ ┆ itie… ┆ ┆ │\n",
|
||
"│ 000032019 ┆ Apple ┆ 000032019 ┆ 10-K ┆ … ┆ mda ┆ AND ┆ 424 ┆ 64 │\n",
|
||
"│ 3 ┆ Inc. ┆ 3-24-0000 ┆ ┆ ┆ ┆ ANALYSIS ┆ ┆ │\n",
|
||
"│ ┆ ┆ 81 ┆ ┆ ┆ ┆ OF ┆ ┆ │\n",
|
||
"│ ┆ ┆ ┆ ┆ ┆ ┆ FINANCIAL ┆ ┆ │\n",
|
||
"│ ┆ ┆ ┆ ┆ ┆ ┆ COND… ┆ ┆ │\n",
|
||
"│ 000078901 ┆ Microsoft ┆ 000078901 ┆ 10-K ┆ … ┆ risk_fact ┆ An invest ┆ 541 ┆ 74 │\n",
|
||
"│ 9 ┆ Corporati ┆ 9-24-0000 ┆ ┆ ┆ ors ┆ ment in ┆ ┆ │\n",
|
||
"│ ┆ on ┆ 42 ┆ ┆ ┆ ┆ our secur ┆ ┆ │\n",
|
||
"│ ┆ ┆ ┆ ┆ ┆ ┆ itie… ┆ ┆ │\n",
|
||
"│ 000078901 ┆ Microsoft ┆ 000078901 ┆ 10-K ┆ … ┆ mda ┆ AND ┆ 424 ┆ 64 │\n",
|
||
"│ 9 ┆ Corporati ┆ 9-24-0000 ┆ ┆ ┆ ┆ ANALYSIS ┆ ┆ │\n",
|
||
"│ ┆ on ┆ 42 ┆ ┆ ┆ ┆ OF ┆ ┆ │\n",
|
||
"│ ┆ ┆ ┆ ┆ ┆ ┆ FINANCIAL ┆ ┆ │\n",
|
||
"│ ┆ ┆ ┆ ┆ ┆ ┆ COND… ┆ ┆ │\n",
|
||
"└───────────┴───────────┴───────────┴───────────┴───┴───────────┴───────────┴───────────┴──────────┘"
|
||
]
|
||
},
|
||
"execution_count": 8,
|
||
"metadata": {},
|
||
"output_type": "execute_result"
|
||
}
|
||
],
|
||
"source": [
|
||
"_DEFAULT_SECTIONS = (\"risk_factors\", \"mda\")\n",
|
||
"\n",
|
||
"\n",
|
||
"def create_text_dataset(\n",
|
||
" filings: list[dict], sections: list[str] = _DEFAULT_SECTIONS\n",
|
||
") -> pl.DataFrame:\n",
|
||
" \"\"\"\n",
|
||
" Create a structured dataset from multiple filings.\n",
|
||
"\n",
|
||
" Parameters\n",
|
||
" ----------\n",
|
||
" filings : list of dict\n",
|
||
" Each dict should contain:\n",
|
||
" - cik: Company CIK\n",
|
||
" - company_name: Company name\n",
|
||
" - accession_no: EDGAR accession number (stable filing identifier)\n",
|
||
" - filing_date: Filing date\n",
|
||
" - accepted_at: SEC acceptance timestamp (for intraday PIT)\n",
|
||
" - period_end: Reporting period end date\n",
|
||
" - filing_type: 10-K, 10-Q, etc.\n",
|
||
" - text: Raw filing text\n",
|
||
" sections : list of str\n",
|
||
" Sections to extract\n",
|
||
"\n",
|
||
" Returns\n",
|
||
" -------\n",
|
||
" pl.DataFrame\n",
|
||
" Structured text dataset with PIT-correct schema\n",
|
||
"\n",
|
||
" Note\n",
|
||
" ----\n",
|
||
" Automatically uses correct section patterns based on filing_type (10-K vs 10-Q).\n",
|
||
" In 10-Q, MD&A is Item 2 (not Item 7 as in 10-K).\n",
|
||
" \"\"\"\n",
|
||
" records = []\n",
|
||
"\n",
|
||
" for filing in filings:\n",
|
||
" filing_type = filing.get(\"filing_type\", \"10-K\")\n",
|
||
" base_record = {\n",
|
||
" \"cik\": filing.get(\"cik\"),\n",
|
||
" \"company_name\": filing.get(\"company_name\"),\n",
|
||
" # Stable filing identifier (prefer over cik + filing_date)\n",
|
||
" \"accession_no\": filing.get(\"accession_no\"),\n",
|
||
" \"filing_type\": filing_type,\n",
|
||
" # PIT timestamps\n",
|
||
" \"filing_date\": filing.get(\"filing_date\"),\n",
|
||
" \"accepted_at\": filing.get(\"accepted_at\"), # Intraday PIT\n",
|
||
" \"period_end\": filing.get(\"period_end\"),\n",
|
||
" }\n",
|
||
"\n",
|
||
" text = filing.get(\"text\", \"\")\n",
|
||
"\n",
|
||
" # Use form-type-aware patterns\n",
|
||
" patterns = get_patterns_for_form(filing_type)\n",
|
||
"\n",
|
||
" for section in sections:\n",
|
||
" # Skip sections not defined for this form type\n",
|
||
" if section not in patterns:\n",
|
||
" continue\n",
|
||
"\n",
|
||
" extracted = extract_section(text, section, patterns)\n",
|
||
"\n",
|
||
" record = {\n",
|
||
" **base_record,\n",
|
||
" \"section\": section,\n",
|
||
" \"text\": extracted or \"\",\n",
|
||
" \"text_length\": len(extracted) if extracted else 0,\n",
|
||
" \"word_count\": len(extracted.split()) if extracted else 0,\n",
|
||
" }\n",
|
||
" records.append(record)\n",
|
||
"\n",
|
||
" return pl.DataFrame(records)\n",
|
||
"\n",
|
||
"\n",
|
||
"# Create sample filings with PIT-correct schema\n",
|
||
"sample_filings = [\n",
|
||
" {\n",
|
||
" \"cik\": \"0000320193\",\n",
|
||
" \"company_name\": \"Apple Inc.\",\n",
|
||
" \"accession_no\": \"0000320193-24-000081\",\n",
|
||
" \"filing_date\": datetime(2024, 10, 31),\n",
|
||
" \"accepted_at\": datetime(2024, 10, 31, 16, 35, 12), # SEC acceptance timestamp\n",
|
||
" \"period_end\": datetime(2024, 9, 28),\n",
|
||
" \"filing_type\": \"10-K\",\n",
|
||
" \"text\": sample_10k,\n",
|
||
" },\n",
|
||
" {\n",
|
||
" \"cik\": \"0000789019\",\n",
|
||
" \"company_name\": \"Microsoft Corporation\",\n",
|
||
" \"accession_no\": \"0000789019-24-000042\",\n",
|
||
" \"filing_date\": datetime(2024, 7, 30),\n",
|
||
" \"accepted_at\": datetime(2024, 7, 30, 16, 5, 45),\n",
|
||
" \"period_end\": datetime(2024, 6, 30),\n",
|
||
" \"filing_type\": \"10-K\",\n",
|
||
" \"text\": sample_10k, # Using same sample for demo\n",
|
||
" },\n",
|
||
"]\n",
|
||
"\n",
|
||
"text_dataset = create_text_dataset(sample_filings)\n",
|
||
"text_dataset"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "e4039d22",
|
||
"metadata": {
|
||
"lines_to_next_cell": 2,
|
||
"tags": []
|
||
},
|
||
"source": [
|
||
"## 5. Text Statistics and Quality Checks"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 9,
|
||
"id": "947f2307",
|
||
"metadata": {
|
||
"execution": {
|
||
"iopub.execute_input": "2026-05-16T23:42:30.452191Z",
|
||
"iopub.status.busy": "2026-05-16T23:42:30.452111Z",
|
||
"iopub.status.idle": "2026-05-16T23:42:30.454589Z",
|
||
"shell.execute_reply": "2026-05-16T23:42:30.454283Z"
|
||
},
|
||
"lines_to_next_cell": 2,
|
||
"tags": []
|
||
},
|
||
"outputs": [],
|
||
"source": [
|
||
"def calculate_text_statistics(text: str) -> dict:\n",
|
||
" \"\"\"Calculate statistics about extracted text.\"\"\"\n",
|
||
" if not text:\n",
|
||
" return {\n",
|
||
" \"word_count\": 0,\n",
|
||
" \"sentence_count\": 0,\n",
|
||
" \"avg_word_length\": 0,\n",
|
||
" \"avg_sentence_length\": 0,\n",
|
||
" \"unique_words\": 0,\n",
|
||
" \"lexical_diversity\": 0,\n",
|
||
" }\n",
|
||
"\n",
|
||
" words = text.lower().split()\n",
|
||
" sentences = re.split(r\"[.!?]+\", text)\n",
|
||
" sentences = [s.strip() for s in sentences if s.strip()]\n",
|
||
"\n",
|
||
" word_count = len(words)\n",
|
||
" unique_words = len(set(words))\n",
|
||
"\n",
|
||
" return {\n",
|
||
" \"word_count\": word_count,\n",
|
||
" \"sentence_count\": len(sentences),\n",
|
||
" \"avg_word_length\": np.mean([len(w) for w in words]) if words else 0,\n",
|
||
" \"avg_sentence_length\": word_count / len(sentences) if sentences else 0,\n",
|
||
" \"unique_words\": unique_words,\n",
|
||
" \"lexical_diversity\": unique_words / word_count if word_count > 0 else 0,\n",
|
||
" }"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "e495ed4c",
|
||
"metadata": {
|
||
"lines_to_next_cell": 2,
|
||
"tags": []
|
||
},
|
||
"source": [
|
||
"### Quality Check Extraction\n",
|
||
"Flag potential issues: missing sections, truncated extractions, or boundary contamination."
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 10,
|
||
"id": "a7534900",
|
||
"metadata": {
|
||
"execution": {
|
||
"iopub.execute_input": "2026-05-16T23:42:30.460452Z",
|
||
"iopub.status.busy": "2026-05-16T23:42:30.460386Z",
|
||
"iopub.status.idle": "2026-05-16T23:42:30.467099Z",
|
||
"shell.execute_reply": "2026-05-16T23:42:30.466816Z"
|
||
},
|
||
"tags": []
|
||
},
|
||
"outputs": [
|
||
{
|
||
"data": {
|
||
"text/html": [
|
||
"<div><style>\n",
|
||
".dataframe > thead > tr,\n",
|
||
".dataframe > tbody > tr {\n",
|
||
" text-align: right;\n",
|
||
" white-space: pre-wrap;\n",
|
||
"}\n",
|
||
"</style>\n",
|
||
"<small>shape: (4, 4)</small><table border=\"1\" class=\"dataframe\"><thead><tr><th>company_name</th><th>section</th><th>word_count</th><th>quality_flag</th></tr><tr><td>str</td><td>str</td><td>i64</td><td>str</td></tr></thead><tbody><tr><td>"Apple Inc."</td><td>"risk_factors"</td><td>74</td><td>"too_short"</td></tr><tr><td>"Apple Inc."</td><td>"mda"</td><td>64</td><td>"too_short"</td></tr><tr><td>"Microsoft Corporation"</td><td>"risk_factors"</td><td>74</td><td>"too_short"</td></tr><tr><td>"Microsoft Corporation"</td><td>"mda"</td><td>64</td><td>"too_short"</td></tr></tbody></table></div>"
|
||
],
|
||
"text/plain": [
|
||
"shape: (4, 4)\n",
|
||
"┌───────────────────────┬──────────────┬────────────┬──────────────┐\n",
|
||
"│ company_name ┆ section ┆ word_count ┆ quality_flag │\n",
|
||
"│ --- ┆ --- ┆ --- ┆ --- │\n",
|
||
"│ str ┆ str ┆ i64 ┆ str │\n",
|
||
"╞═══════════════════════╪══════════════╪════════════╪══════════════╡\n",
|
||
"│ Apple Inc. ┆ risk_factors ┆ 74 ┆ too_short │\n",
|
||
"│ Apple Inc. ┆ mda ┆ 64 ┆ too_short │\n",
|
||
"│ Microsoft Corporation ┆ risk_factors ┆ 74 ┆ too_short │\n",
|
||
"│ Microsoft Corporation ┆ mda ┆ 64 ┆ too_short │\n",
|
||
"└───────────────────────┴──────────────┴────────────┴──────────────┘"
|
||
]
|
||
},
|
||
"execution_count": 10,
|
||
"metadata": {},
|
||
"output_type": "execute_result"
|
||
}
|
||
],
|
||
"source": [
|
||
"def quality_check_extraction(df: pl.DataFrame, min_words: int = 100) -> pl.DataFrame:\n",
|
||
" \"\"\"\n",
|
||
" Check quality of extracted text.\n",
|
||
"\n",
|
||
" Flags potential issues:\n",
|
||
" - Missing (extraction returned nothing)\n",
|
||
" - Too short (extraction may have failed partially)\n",
|
||
" - Too long (may have captured extra sections)\n",
|
||
" - Boundary leak (captured content from next section)\n",
|
||
" \"\"\"\n",
|
||
" # Add quality flags - NOTE: order matters! Check missing first since 0 < min_words\n",
|
||
" result = df.with_columns(\n",
|
||
" [\n",
|
||
" pl.when(pl.col(\"word_count\") == 0)\n",
|
||
" .then(pl.lit(\"missing\"))\n",
|
||
" .when(pl.col(\"word_count\") < min_words)\n",
|
||
" .then(pl.lit(\"too_short\"))\n",
|
||
" .when(pl.col(\"word_count\") > 50000)\n",
|
||
" .then(pl.lit(\"suspiciously_long\"))\n",
|
||
" .otherwise(pl.lit(\"ok\"))\n",
|
||
" .alias(\"quality_flag\")\n",
|
||
" ]\n",
|
||
" )\n",
|
||
"\n",
|
||
" # Check for boundary contamination (e.g., \"Item 8\" appearing in MD&A)\n",
|
||
" # This indicates the end boundary wasn't properly detected\n",
|
||
" result = result.with_columns(\n",
|
||
" pl.when((pl.col(\"section\") == \"mda\") & pl.col(\"text\").str.contains(r\"(?i)\\bITEM\\s*8\\b\"))\n",
|
||
" .then(pl.lit(True))\n",
|
||
" .when(\n",
|
||
" (pl.col(\"section\") == \"risk_factors\")\n",
|
||
" & pl.col(\"text\").str.contains(r\"(?i)\\bITEM\\s*2\\b.*\\bPROPERTIES\\b\")\n",
|
||
" )\n",
|
||
" .then(pl.lit(True))\n",
|
||
" .otherwise(pl.lit(False))\n",
|
||
" .alias(\"possible_boundary_leak\")\n",
|
||
" )\n",
|
||
"\n",
|
||
" return result\n",
|
||
"\n",
|
||
"\n",
|
||
"text_dataset_checked = quality_check_extraction(text_dataset)\n",
|
||
"text_dataset_checked.select([\"company_name\", \"section\", \"word_count\", \"quality_flag\"])"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 11,
|
||
"id": "b0e04cb7",
|
||
"metadata": {
|
||
"execution": {
|
||
"iopub.execute_input": "2026-05-16T23:42:30.470857Z",
|
||
"iopub.status.busy": "2026-05-16T23:42:30.470784Z",
|
||
"iopub.status.idle": "2026-05-16T23:42:30.473273Z",
|
||
"shell.execute_reply": "2026-05-16T23:42:30.472983Z"
|
||
},
|
||
"tags": []
|
||
},
|
||
"outputs": [
|
||
{
|
||
"name": "stdout",
|
||
"output_type": "stream",
|
||
"text": [
|
||
"Text Statistics (Risk Factors section):\n",
|
||
" word_count: 74\n",
|
||
" sentence_count: 7\n",
|
||
" avg_word_length: 6.28\n",
|
||
" avg_sentence_length: 10.57\n",
|
||
" unique_words: 63\n",
|
||
" lexical_diversity: 0.85\n"
|
||
]
|
||
}
|
||
],
|
||
"source": [
|
||
"sample_text = text_dataset.filter(\n",
|
||
" (pl.col(\"section\") == \"risk_factors\") & (pl.col(\"word_count\") > 0)\n",
|
||
")[\"text\"][0]\n",
|
||
"\n",
|
||
"stats = calculate_text_statistics(sample_text)\n",
|
||
"print(\"Text Statistics (Risk Factors section):\")\n",
|
||
"for k, v in stats.items():\n",
|
||
" print(f\" {k}: {v:.2f}\" if isinstance(v, float) else f\" {k}: {v}\")"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "978f5052",
|
||
"metadata": {
|
||
"lines_to_next_cell": 2,
|
||
"tags": []
|
||
},
|
||
"source": [
|
||
"## 6. Change Analysis Over Time\n",
|
||
"\n",
|
||
"Track how MD&A and Risk Factors change between filings."
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 12,
|
||
"id": "2bdd5191",
|
||
"metadata": {
|
||
"execution": {
|
||
"iopub.execute_input": "2026-05-16T23:42:30.479497Z",
|
||
"iopub.status.busy": "2026-05-16T23:42:30.479428Z",
|
||
"iopub.status.idle": "2026-05-16T23:42:30.481688Z",
|
||
"shell.execute_reply": "2026-05-16T23:42:30.481364Z"
|
||
},
|
||
"tags": []
|
||
},
|
||
"outputs": [],
|
||
"source": [
|
||
"def calculate_text_similarity(text1: str, text2: str) -> dict:\n",
|
||
" \"\"\"\n",
|
||
" Calculate similarity between two text documents.\n",
|
||
"\n",
|
||
" Uses simple word overlap metrics (Jaccard similarity).\n",
|
||
" For production, consider using TF-IDF or embeddings.\n",
|
||
" \"\"\"\n",
|
||
" if not text1 or not text2:\n",
|
||
" return {\"jaccard\": 0, \"word_overlap_pct\": 0}\n",
|
||
"\n",
|
||
" words1 = set(text1.lower().split())\n",
|
||
" words2 = set(text2.lower().split())\n",
|
||
"\n",
|
||
" intersection = words1 & words2\n",
|
||
" union = words1 | words2\n",
|
||
"\n",
|
||
" jaccard = len(intersection) / len(union) if union else 0\n",
|
||
" overlap_pct = len(intersection) / len(words1) if words1 else 0\n",
|
||
"\n",
|
||
" return {\n",
|
||
" \"jaccard_similarity\": jaccard,\n",
|
||
" \"word_overlap_pct\": overlap_pct,\n",
|
||
" \"new_words\": len(words2 - words1),\n",
|
||
" \"removed_words\": len(words1 - words2),\n",
|
||
" }"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 13,
|
||
"id": "9af7bb68",
|
||
"metadata": {
|
||
"execution": {
|
||
"iopub.execute_input": "2026-05-16T23:42:30.485135Z",
|
||
"iopub.status.busy": "2026-05-16T23:42:30.485073Z",
|
||
"iopub.status.idle": "2026-05-16T23:42:30.487421Z",
|
||
"shell.execute_reply": "2026-05-16T23:42:30.487100Z"
|
||
},
|
||
"tags": []
|
||
},
|
||
"outputs": [],
|
||
"source": [
|
||
"def extract_new_risk_factors(old_text: str, new_text: str) -> list[str]:\n",
|
||
" \"\"\"\n",
|
||
" Identify potentially new risk factors by finding new paragraphs.\n",
|
||
"\n",
|
||
" Simple heuristic: paragraphs in new filing not present in old.\n",
|
||
" \"\"\"\n",
|
||
" if not old_text or not new_text:\n",
|
||
" return []\n",
|
||
"\n",
|
||
" # Split into paragraphs\n",
|
||
" old_paragraphs = set(p.strip().lower() for p in old_text.split(\"\\n\\n\") if len(p.strip()) > 50)\n",
|
||
" new_paragraphs = [p.strip() for p in new_text.split(\"\\n\\n\") if len(p.strip()) > 50]\n",
|
||
"\n",
|
||
" new_risks = []\n",
|
||
" for para in new_paragraphs:\n",
|
||
" # Check if paragraph is substantially new (low similarity to all old paragraphs)\n",
|
||
" para_lower = para.lower()\n",
|
||
" if para_lower not in old_paragraphs:\n",
|
||
" # Simple check - could be improved with fuzzy matching\n",
|
||
" new_risks.append(para[:200] + \"...\" if len(para) > 200 else para)\n",
|
||
"\n",
|
||
" return new_risks[:5] # Return top 5 new items"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 14,
|
||
"id": "428d5bf0",
|
||
"metadata": {
|
||
"execution": {
|
||
"iopub.execute_input": "2026-05-16T23:42:30.490894Z",
|
||
"iopub.status.busy": "2026-05-16T23:42:30.490784Z",
|
||
"iopub.status.idle": "2026-05-16T23:42:30.493035Z",
|
||
"shell.execute_reply": "2026-05-16T23:42:30.492801Z"
|
||
},
|
||
"tags": []
|
||
},
|
||
"outputs": [
|
||
{
|
||
"name": "stdout",
|
||
"output_type": "stream",
|
||
"text": [
|
||
"Similarity Metrics:\n",
|
||
" jaccard_similarity: 0.310\n",
|
||
" word_overlap_pct: 0.929\n",
|
||
" new_words: 28\n",
|
||
" removed_words: 1\n",
|
||
"\n",
|
||
"New Risk Factors Identified: 3\n",
|
||
" 1. COMPETITION RISKS\n",
|
||
"We face significant competition from larger companies with more resources....\n",
|
||
" 2. AI TECHNOLOGY RISKS\n",
|
||
"Rapid advances in artificial intelligence may disrupt our business model.\n",
|
||
"We may...\n",
|
||
" 3. CYBERSECURITY RISKS\n",
|
||
"We face increasing threats from sophisticated cyber attacks....\n"
|
||
]
|
||
}
|
||
],
|
||
"source": [
|
||
"# Simulate two versions of risk factors\n",
|
||
"old_risks = \"\"\"\n",
|
||
"COMPETITION RISKS\n",
|
||
"We face significant competition from larger companies.\n",
|
||
"\n",
|
||
"ECONOMIC RISKS\n",
|
||
"Economic downturns could reduce customer spending.\n",
|
||
"\"\"\"\n",
|
||
"\n",
|
||
"new_risks = \"\"\"\n",
|
||
"COMPETITION RISKS\n",
|
||
"We face significant competition from larger companies with more resources.\n",
|
||
"\n",
|
||
"ECONOMIC RISKS\n",
|
||
"Economic downturns could reduce customer spending.\n",
|
||
"\n",
|
||
"AI TECHNOLOGY RISKS\n",
|
||
"Rapid advances in artificial intelligence may disrupt our business model.\n",
|
||
"We may need to make significant investments to remain competitive.\n",
|
||
"\n",
|
||
"CYBERSECURITY RISKS\n",
|
||
"We face increasing threats from sophisticated cyber attacks.\n",
|
||
"\"\"\"\n",
|
||
"\n",
|
||
"similarity = calculate_text_similarity(old_risks, new_risks)\n",
|
||
"print(\"Similarity Metrics:\")\n",
|
||
"for k, v in similarity.items():\n",
|
||
" print(f\" {k}: {v:.3f}\" if isinstance(v, float) else f\" {k}: {v}\")\n",
|
||
"\n",
|
||
"new_items = extract_new_risk_factors(old_risks, new_risks)\n",
|
||
"print(f\"\\nNew Risk Factors Identified: {len(new_items)}\")\n",
|
||
"for i, item in enumerate(new_items, 1):\n",
|
||
" print(f\" {i}. {item[:100]}...\")"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "a6cc7776",
|
||
"metadata": {
|
||
"lines_to_next_cell": 2,
|
||
"tags": []
|
||
},
|
||
"source": [
|
||
"## 7. Saving the Dataset"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 15,
|
||
"id": "d783f693",
|
||
"metadata": {
|
||
"execution": {
|
||
"iopub.execute_input": "2026-05-16T23:42:30.499705Z",
|
||
"iopub.status.busy": "2026-05-16T23:42:30.499643Z",
|
||
"iopub.status.idle": "2026-05-16T23:42:30.501537Z",
|
||
"shell.execute_reply": "2026-05-16T23:42:30.501300Z"
|
||
},
|
||
"tags": []
|
||
},
|
||
"outputs": [],
|
||
"source": [
|
||
"def save_text_dataset(df: pl.DataFrame, output_path: str, format: str = \"parquet\") -> None:\n",
|
||
" \"\"\"\n",
|
||
" Save text dataset to file.\n",
|
||
"\n",
|
||
" Parameters\n",
|
||
" ----------\n",
|
||
" df : pl.DataFrame\n",
|
||
" Text dataset\n",
|
||
" output_path : str\n",
|
||
" Output file path\n",
|
||
" format : str\n",
|
||
" 'parquet' or 'csv'\n",
|
||
" \"\"\"\n",
|
||
" path = Path(output_path)\n",
|
||
" path.parent.mkdir(parents=True, exist_ok=True)\n",
|
||
"\n",
|
||
" if format == \"parquet\":\n",
|
||
" df.write_parquet(path)\n",
|
||
" elif format == \"csv\":\n",
|
||
" df.write_csv(path)\n",
|
||
" else:\n",
|
||
" raise ValueError(f\"Unknown format: {format}\")\n",
|
||
"\n",
|
||
" print(f\"Saved {len(df)} records to {path}\")"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 16,
|
||
"id": "6046edd1",
|
||
"metadata": {
|
||
"execution": {
|
||
"iopub.execute_input": "2026-05-16T23:42:30.508209Z",
|
||
"iopub.status.busy": "2026-05-16T23:42:30.508074Z",
|
||
"iopub.status.idle": "2026-05-16T23:42:30.509742Z",
|
||
"shell.execute_reply": "2026-05-16T23:42:30.509472Z"
|
||
},
|
||
"tags": []
|
||
},
|
||
"outputs": [
|
||
{
|
||
"name": "stdout",
|
||
"output_type": "stream",
|
||
"text": [
|
||
"\n",
|
||
"To save dataset:\n",
|
||
" save_text_dataset(text_dataset, 'data/sec_text_data.parquet')\n"
|
||
]
|
||
}
|
||
],
|
||
"source": [
|
||
"# Example save (commented out to avoid file creation in demo)\n",
|
||
"# save_text_dataset(text_dataset, \"/tmp/sec_text_data.parquet\")\n",
|
||
"print(\"\\nTo save dataset:\")\n",
|
||
"print(\" save_text_dataset(text_dataset, 'data/sec_text_data.parquet')\")"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "94c660be",
|
||
"metadata": {
|
||
"tags": []
|
||
},
|
||
"source": [
|
||
"## 8. End-to-End Run Against a Live EDGAR Filing\n",
|
||
"\n",
|
||
"The previous sections developed the extraction pipeline against an\n",
|
||
"inline sample 10-K. The same helpers run unchanged against a real\n",
|
||
"filing: `edgartools` fetches the latest AAPL 10-K from EDGAR, our\n",
|
||
"`extract_section` + `clean_text` + `calculate_text_statistics`\n",
|
||
"helpers process it, and the resulting per-section summary mirrors\n",
|
||
"what the synthetic example produced earlier in the notebook.\n",
|
||
"\n",
|
||
"The SEC requires a real User-Agent identity for every request and\n",
|
||
"blocks placeholder addresses. Set `EDGAR_IDENTITY` in your environment\n",
|
||
"to `\"Your Name your.email@domain.com\"` before running this section:\n",
|
||
"\n",
|
||
"```bash\n",
|
||
"export EDGAR_IDENTITY=\"Jane Doe jane@example.org\"\n",
|
||
"```\n",
|
||
"\n",
|
||
"The fetch is read-only, rate-limited by edgartools, and typically\n",
|
||
"completes in 1–3 seconds."
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 17,
|
||
"id": "a51a61c0",
|
||
"metadata": {
|
||
"execution": {
|
||
"iopub.execute_input": "2026-05-16T23:42:30.521249Z",
|
||
"iopub.status.busy": "2026-05-16T23:42:30.521167Z",
|
||
"iopub.status.idle": "2026-05-16T23:42:31.209555Z",
|
||
"shell.execute_reply": "2026-05-16T23:42:31.208498Z"
|
||
},
|
||
"tags": []
|
||
},
|
||
"outputs": [
|
||
{
|
||
"name": "stdout",
|
||
"output_type": "stream",
|
||
"text": [
|
||
"AAPL 10-K: accession 0000320193-25-000079, filed 2025-10-31\n"
|
||
]
|
||
}
|
||
],
|
||
"source": [
|
||
"edgar_identity = os.environ.get(\"EDGAR_IDENTITY\")\n",
|
||
"if not edgar_identity:\n",
|
||
" raise RuntimeError(\n",
|
||
" \"EDGAR_IDENTITY environment variable is not set. The SEC requires a \"\n",
|
||
" \"real User-Agent (name + email) for every EDGAR request and blocks \"\n",
|
||
" \"placeholder addresses. Set it before running this section, e.g. \"\n",
|
||
" '`export EDGAR_IDENTITY=\"Jane Doe jane@example.org\"`.'\n",
|
||
" )\n",
|
||
"set_identity(edgar_identity)\n",
|
||
"\n",
|
||
"company = Company(EDGAR_TICKER)\n",
|
||
"latest_filing = company.get_filings(form=EDGAR_FORM).latest()\n",
|
||
"print(\n",
|
||
" f\"{EDGAR_TICKER} {EDGAR_FORM}: accession {latest_filing.accession_no}, filed {latest_filing.filing_date}\"\n",
|
||
")"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "55712373",
|
||
"metadata": {
|
||
"tags": []
|
||
},
|
||
"source": [
|
||
"Fetch the filing text and run the section extractor with the\n",
|
||
"form-aware patterns. The two highest-value sections (Risk Factors\n",
|
||
"and MD&A) come back as plain strings — ready for `clean_text`,\n",
|
||
"tokenisation, or downstream NLP feature engineering."
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "code",
|
||
"execution_count": 18,
|
||
"id": "67ce88e7",
|
||
"metadata": {
|
||
"execution": {
|
||
"iopub.execute_input": "2026-05-16T23:42:31.232014Z",
|
||
"iopub.status.busy": "2026-05-16T23:42:31.231910Z",
|
||
"iopub.status.idle": "2026-05-16T23:42:32.061206Z",
|
||
"shell.execute_reply": "2026-05-16T23:42:32.060845Z"
|
||
},
|
||
"tags": []
|
||
},
|
||
"outputs": [
|
||
{
|
||
"data": {
|
||
"text/html": [
|
||
"<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\"></pre>\n"
|
||
],
|
||
"text/plain": []
|
||
},
|
||
"metadata": {},
|
||
"output_type": "display_data"
|
||
},
|
||
{
|
||
"data": {
|
||
"text/html": [
|
||
"<div><style>\n",
|
||
".dataframe > thead > tr,\n",
|
||
".dataframe > tbody > tr {\n",
|
||
" text-align: right;\n",
|
||
" white-space: pre-wrap;\n",
|
||
"}\n",
|
||
"</style>\n",
|
||
"<small>shape: (3, 4)</small><table border=\"1\" class=\"dataframe\"><thead><tr><th>section</th><th>extracted</th><th>char_count</th><th>word_count</th></tr><tr><td>str</td><td>bool</td><td>i64</td><td>i64</td></tr></thead><tbody><tr><td>"business"</td><td>true</td><td>16043</td><td>2332</td></tr><tr><td>"risk_factors"</td><td>true</td><td>68195</td><td>9836</td></tr><tr><td>"mda"</td><td>true</td><td>18569</td><td>2762</td></tr></tbody></table></div>"
|
||
],
|
||
"text/plain": [
|
||
"shape: (3, 4)\n",
|
||
"┌──────────────┬───────────┬────────────┬────────────┐\n",
|
||
"│ section ┆ extracted ┆ char_count ┆ word_count │\n",
|
||
"│ --- ┆ --- ┆ --- ┆ --- │\n",
|
||
"│ str ┆ bool ┆ i64 ┆ i64 │\n",
|
||
"╞══════════════╪═══════════╪════════════╪════════════╡\n",
|
||
"│ business ┆ true ┆ 16043 ┆ 2332 │\n",
|
||
"│ risk_factors ┆ true ┆ 68195 ┆ 9836 │\n",
|
||
"│ mda ┆ true ┆ 18569 ┆ 2762 │\n",
|
||
"└──────────────┴───────────┴────────────┴────────────┘"
|
||
]
|
||
},
|
||
"execution_count": 18,
|
||
"metadata": {},
|
||
"output_type": "execute_result"
|
||
}
|
||
],
|
||
"source": [
|
||
"real_text = latest_filing.text()\n",
|
||
"form_patterns = get_patterns_for_form(EDGAR_FORM)\n",
|
||
"\n",
|
||
"real_risk = extract_section(real_text, \"risk_factors\", form_patterns)\n",
|
||
"real_mda = extract_section(real_text, \"mda\", form_patterns)\n",
|
||
"real_business = extract_section(real_text, \"business\", form_patterns)\n",
|
||
"\n",
|
||
"real_stats = pl.DataFrame(\n",
|
||
" [\n",
|
||
" {\n",
|
||
" \"section\": name,\n",
|
||
" \"extracted\": text is not None,\n",
|
||
" \"char_count\": len(text) if text else 0,\n",
|
||
" \"word_count\": calculate_text_statistics(text)[\"word_count\"] if text else 0,\n",
|
||
" }\n",
|
||
" for name, text in [\n",
|
||
" (\"business\", real_business),\n",
|
||
" (\"risk_factors\", real_risk),\n",
|
||
" (\"mda\", real_mda),\n",
|
||
" ]\n",
|
||
" ]\n",
|
||
")\n",
|
||
"real_stats"
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "47ad6360",
|
||
"metadata": {
|
||
"tags": []
|
||
},
|
||
"source": [
|
||
"A successful run shows non-zero word counts on each row — the same\n",
|
||
"pipeline that produced the synthetic-text statistics in Section 5\n",
|
||
"works against a current SEC filing without code changes. From here,\n",
|
||
"the cleaned strings feed directly into the NLP feature pipeline in\n",
|
||
"Chapter 10."
|
||
]
|
||
},
|
||
{
|
||
"cell_type": "markdown",
|
||
"id": "64d4ba3e",
|
||
"metadata": {
|
||
"tags": []
|
||
},
|
||
"source": [
|
||
"## 9. Key Takeaways\n",
|
||
"\n",
|
||
"### Text Extraction Best Practices\n",
|
||
"\n",
|
||
"1. **Use line-anchored regex patterns for section boundaries**\n",
|
||
" - Item numbers are standardized (Item 1A, Item 7)\n",
|
||
" - But formatting varies - need multiple patterns with `^` anchoring\n",
|
||
" - Skip Table of Contents matches by searching after TOC marker\n",
|
||
"\n",
|
||
"2. **Clean thoroughly but preserve paragraph structure**\n",
|
||
" - Remove HTML tags and filing artifacts\n",
|
||
" - Normalize whitespace *within* lines, but keep paragraph breaks\n",
|
||
" - Paragraph structure enables change detection over time\n",
|
||
"\n",
|
||
"3. **Validate extraction quality**\n",
|
||
" - Check text length (too short = extraction failed)\n",
|
||
" - Check for boundary leaks (e.g., \"Item 8\" in MD&A)\n",
|
||
" - Track quality metrics for iterative rule refinement\n",
|
||
"\n",
|
||
"4. **Track point-in-time with stable identifiers**\n",
|
||
" - Use **accession number** as primary key (not cik + filing_date)\n",
|
||
" - Use **accepted_at** for intraday PIT correctness\n",
|
||
" - Keep both filing_date and period_end for different query patterns\n",
|
||
"\n",
|
||
"5. **Store efficiently with audit trail**\n",
|
||
" - Parquet format for large datasets\n",
|
||
" - Include metadata (CIK, accession, timestamps, section)\n",
|
||
" - Consider storing both raw and cleaned text for reproducibility\n",
|
||
"\n",
|
||
"### Most Valuable Sections\n",
|
||
"\n",
|
||
"| Section | Alpha Use Case |\n",
|
||
"|---------|----------------|\n",
|
||
"| **Risk Factors (1A)** | Sentiment analysis, risk themes, change detection |\n",
|
||
"| **MD&A (Item 7)** | Management sentiment, outlook changes |\n",
|
||
"| **Business (Item 1)** | Competitive position, strategy changes |\n",
|
||
"\n",
|
||
"### Forward Reference\n",
|
||
"\n",
|
||
"This structured text dataset is the input for NLP techniques covered in later chapters:\n",
|
||
"- Sentiment scoring (positive/negative language)\n",
|
||
"- Topic modeling (identify themes)\n",
|
||
"- Named entity recognition (extract companies, products)\n",
|
||
"- Text embeddings (semantic similarity)"
|
||
]
|
||
}
|
||
],
|
||
"metadata": {
|
||
"kernelspec": {
|
||
"display_name": "Python 3 (ipykernel)",
|
||
"language": "python",
|
||
"name": "python3"
|
||
},
|
||
"language_info": {
|
||
"codemirror_mode": {
|
||
"name": "ipython",
|
||
"version": 3
|
||
},
|
||
"file_extension": ".py",
|
||
"mimetype": "text/x-python",
|
||
"name": "python",
|
||
"nbconvert_exporter": "python",
|
||
"pygments_lexer": "ipython3",
|
||
"version": "3.14.3"
|
||
},
|
||
"papermill": {
|
||
"default_parameters": {},
|
||
"duration": 3.284106,
|
||
"end_time": "2026-05-16T23:42:32.483343+00:00",
|
||
"environment_variables": {},
|
||
"exception": null,
|
||
"input_path": "04_fundamental_alternative_data/14_text_data_extraction.ipynb",
|
||
"output_path": "04_fundamental_alternative_data/14_text_data_extraction.ipynb",
|
||
"parameters": {},
|
||
"start_time": "2026-05-16T23:42:29.199237+00:00",
|
||
"version": "2.7.0"
|
||
}
|
||
},
|
||
"nbformat": 4,
|
||
"nbformat_minor": 5
|
||
}
|