ITEM 1A. RISK FACTORS
\n", "The following risks could materially affect our business:
\n", "• Competition may increase
\n", "• Economic conditions may deteriorate
\n", "Page 15 of 120
\n", "Table of Contents
\n", "\"\"\"\n", "\n", "cleaned = clean_text(clean_html(sample_html))\n", "print(\"Sample cleaning:\")\n", "print(f\"Original HTML length: {len(sample_html)}\")\n", "print(f\"Cleaned text: {cleaned}\")" ] }, { "cell_type": "markdown", "id": "b1c3e33b", "metadata": { "lines_to_next_cell": 2, "tags": [] }, "source": [ "## 3. Section Extraction" ] }, { "cell_type": "code", "execution_count": 7, "id": "69aba95d", "metadata": { "execution": { "iopub.execute_input": "2026-05-16T23:42:30.417832Z", "iopub.status.busy": "2026-05-16T23:42:30.417637Z", "iopub.status.idle": "2026-05-16T23:42:30.428151Z", "shell.execute_reply": "2026-05-16T23:42:30.427688Z" }, "tags": [] }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "BUSINESS (728 chars):\n", "Our company is a leading provider of technology solutions for enterprise\n", "customers worldwide. We design, manufacture, and support a portfolio of\n", "hardware, software, and cloud services that help organi...\n", "\n", "RISK_FACTORS (541 chars):\n", "An investment in our securities involves a high degree of risk.\n", "You should carefully consider the following risk factors:\n", "\n", "COMPETITION RISKS\n", "We face significant competition from larger companies with ...\n", "\n", "MDA (424 chars):\n", "AND ANALYSIS OF FINANCIAL CONDITION\n", "\n", "Overview\n", "The past year has been transformative for our company.\n", "Revenue increased 25% driven by strong product adoption.\n", "\n", "Results of Operations\n", "Our gross margin im...\n", "\n" ] } ], "source": [ "def extract_section(text: str, section_name: str, patterns: dict = None) -> str | None:\n", " \"\"\"\n", " Extract a specific section from a 10-K/10-Q filing using anchored item boundaries.\n", "\n", " Robust to TOC duplicates by finding all start_pattern matches and\n", " picking the candidate (start, end) span with the most content\n", " between it and the next end_pattern. Inline references inside\n", " narrative text are filtered out by the same span-length test —\n", " they don't have a matching end-pattern Item N+1 immediately after.\n", "\n", " Parameters\n", " ----------\n", " text : str\n", " Full filing text\n", " section_name : str\n", " Name of section to extract (e.g., 'risk_factors', 'mda')\n", " patterns : dict\n", " Custom patterns, or use SECTION_PATTERNS\n", "\n", " Returns\n", " -------\n", " str or None\n", " Extracted section text, or None if no candidate has substantive content.\n", " \"\"\"\n", " if not text:\n", " return None\n", "\n", " if patterns is None:\n", " patterns = SECTION_PATTERNS\n", "\n", " if section_name not in patterns:\n", " raise ValueError(f\"Unknown section: {section_name}\")\n", "\n", " config = patterns[section_name]\n", " flags = re.IGNORECASE | re.MULTILINE | re.DOTALL\n", "\n", " # Some patterns may require matching after a specific marker\n", " # (e.g., PART II for 10-Q risk factors).\n", " offset = 0\n", " search_text = text\n", " require_after = config.get(\"require_after\")\n", " if require_after:\n", " after_match = re.search(require_after, search_text, flags)\n", " if after_match:\n", " offset = after_match.start()\n", " search_text = search_text[after_match.start() :]\n", "\n", " # Collect every candidate start position from every start pattern.\n", " starts: list[int] = []\n", " for pattern in config[\"start_patterns\"]:\n", " starts.extend(offset + m.end() for m in re.finditer(pattern, search_text, flags))\n", " if not starts:\n", " return None\n", " starts = sorted(set(starts))\n", "\n", " # For each candidate start, find the next end_pattern match and\n", " # measure the span. The real section beats every TOC entry on\n", " # span length because TOC entries are followed by the next TOC\n", " # entry within a few dozen characters.\n", " best_span: tuple[int, int] | None = None\n", " for start_pos in starts:\n", " end_pos = len(text)\n", " for end_pattern in config[\"end_patterns\"]:\n", " end_match = re.search(end_pattern, text[start_pos:], flags)\n", " if end_match:\n", " end_pos = start_pos + end_match.start()\n", " break\n", " span = end_pos - start_pos\n", " if best_span is None or span > (best_span[1] - best_span[0]):\n", " best_span = (start_pos, end_pos)\n", "\n", " if best_span is None or (best_span[1] - best_span[0]) < 200:\n", " return None\n", "\n", " return clean_text(text[best_span[0] : best_span[1]])\n", "\n", "\n", "# Create sample 10-K text for demonstration\n", "sample_10k = \"\"\"\n", "FORM 10-K\n", "ANNUAL REPORT\n", "\n", "Table of Contents\n", "\n", "PART I\n", "\n", "ITEM 1. BUSINESS\n", "\n", "Our company is a leading provider of technology solutions for enterprise\n", "customers worldwide. We design, manufacture, and support a portfolio of\n", "hardware, software, and cloud services that help organizations modernize\n", "their data infrastructure.\n", "\n", "We operate across three reportable segments — Cloud Platform, Enterprise\n", "Software, and Professional Services — and sell directly through a global\n", "sales force as well as a network of channel partners. Our customers span\n", "financial services, healthcare, public sector, and manufacturing.\n", "\n", "Recent strategic initiatives include expanding our hyperscale data center\n", "footprint, deepening integrations with leading public cloud providers,\n", "and broadening our generative-AI product portfolio.\n", "\n", "ITEM 1A. RISK FACTORS\n", "\n", "An investment in our securities involves a high degree of risk.\n", "You should carefully consider the following risk factors:\n", "\n", "COMPETITION RISKS\n", "We face significant competition from larger companies with more resources.\n", "New entrants may disrupt our market position.\n", "\n", "ECONOMIC RISKS\n", "Economic downturns could reduce customer spending on our products.\n", "Currency fluctuations may impact our international revenue.\n", "\n", "REGULATORY RISKS\n", "Changes in regulations could increase our compliance costs.\n", "Data privacy laws continue to evolve across jurisdictions.\n", "\n", "ITEM 1B. UNRESOLVED STAFF COMMENTS\n", "\n", "None.\n", "\n", "ITEM 2. PROPERTIES\n", "\n", "We lease our headquarters in San Francisco, California.\n", "\n", "PART II\n", "\n", "ITEM 7. MANAGEMENT'S DISCUSSION AND ANALYSIS OF FINANCIAL CONDITION\n", "\n", "Overview\n", "The past year has been transformative for our company.\n", "Revenue increased 25% driven by strong product adoption.\n", "\n", "Results of Operations\n", "Our gross margin improved to 72% from 68% in the prior year.\n", "Operating expenses remained well controlled.\n", "\n", "Liquidity and Capital Resources\n", "We ended the year with $500 million in cash.\n", "We believe current resources are sufficient for operations.\n", "\n", "ITEM 7A. QUANTITATIVE AND QUALITATIVE DISCLOSURES ABOUT MARKET RISK\n", "\n", "We are exposed to interest rate and foreign currency risks.\n", "\n", "ITEM 8. FINANCIAL STATEMENTS\n", "\"\"\"\n", "\n", "for section in [\"business\", \"risk_factors\", \"mda\"]:\n", " extracted = extract_section(sample_10k, section)\n", " if extracted:\n", " preview = extracted[:200] + \"...\" if len(extracted) > 200 else extracted\n", " print(f\"{section.upper()} ({len(extracted)} chars):\")\n", " print(preview)\n", " print()" ] }, { "cell_type": "markdown", "id": "438d5831", "metadata": { "lines_to_next_cell": 2, "tags": [] }, "source": [ "## 4. Building a Text Dataset" ] }, { "cell_type": "code", "execution_count": 8, "id": "0b5c7317", "metadata": { "execution": { "iopub.execute_input": "2026-05-16T23:42:30.438564Z", "iopub.status.busy": "2026-05-16T23:42:30.438490Z", "iopub.status.idle": "2026-05-16T23:42:30.445368Z", "shell.execute_reply": "2026-05-16T23:42:30.444964Z" }, "tags": [] }, "outputs": [ { "data": { "text/html": [ "| cik | company_name | accession_no | filing_type | filing_date | accepted_at | period_end | section | text | text_length | word_count |
|---|---|---|---|---|---|---|---|---|---|---|
| str | str | str | str | datetime[μs] | datetime[μs] | datetime[μs] | str | str | i64 | i64 |
| "0000320193" | "Apple Inc." | "0000320193-24-000081" | "10-K" | 2024-10-31 00:00:00 | 2024-10-31 16:35:12 | 2024-09-28 00:00:00 | "risk_factors" | "An investment in our securitie… | 541 | 74 |
| "0000320193" | "Apple Inc." | "0000320193-24-000081" | "10-K" | 2024-10-31 00:00:00 | 2024-10-31 16:35:12 | 2024-09-28 00:00:00 | "mda" | "AND ANALYSIS OF FINANCIAL COND… | 424 | 64 |
| "0000789019" | "Microsoft Corporation" | "0000789019-24-000042" | "10-K" | 2024-07-30 00:00:00 | 2024-07-30 16:05:45 | 2024-06-30 00:00:00 | "risk_factors" | "An investment in our securitie… | 541 | 74 |
| "0000789019" | "Microsoft Corporation" | "0000789019-24-000042" | "10-K" | 2024-07-30 00:00:00 | 2024-07-30 16:05:45 | 2024-06-30 00:00:00 | "mda" | "AND ANALYSIS OF FINANCIAL COND… | 424 | 64 |
| company_name | section | word_count | quality_flag |
|---|---|---|---|
| str | str | i64 | str |
| "Apple Inc." | "risk_factors" | 74 | "too_short" |
| "Apple Inc." | "mda" | 64 | "too_short" |
| "Microsoft Corporation" | "risk_factors" | 74 | "too_short" |
| "Microsoft Corporation" | "mda" | 64 | "too_short" |
| section | extracted | char_count | word_count |
|---|---|---|---|
| str | bool | i64 | i64 |
| "business" | true | 16043 | 2332 |
| "risk_factors" | true | 68195 | 9836 |
| "mda" | true | 18569 | 2762 |