chore: import upstream snapshot with attribution

2026-07-13 12:46:28 +08:00
commit 2cab53bc94
2985 changed files with 1288070 additions and 0 deletions
@@ -0,0 +1,898 @@
+# API Reference - Programmatic Usage
+
+**Version:** 3.7.0
+**Last Updated:** 2026-06-11
+**Status:** ✅ Verified against v3.7.0 (every import and signature in this document was checked by importing it)
+
+---
+
+## Overview
+
+Skill Seekers can be used programmatically for integration into other tools, automation scripts, and CI/CD pipelines. This guide covers the Python APIs available for developers who want to embed Skill Seekers functionality into their own applications.
+
+> **Stability note — read this first**
+>
+> The **stable, supported interface of the PyPI package is the `skill-seekers` CLI** (and the MCP server). The Python API documented here is real and importable — it is the same code the CLI runs — but it tracks the implementation: module paths, signatures, and config-dict keys may change between minor releases. **Semver guarantees do not extend to these internals.** If you import these modules, pin an exact version (`skill-seekers==3.7.0`) and re-verify on upgrade.
+
+**Use Cases:**
+- Automated documentation skill generation in CI/CD
+- Batch processing multiple documentation sources
+- Custom skill generation workflows
+- Integration with internal tooling
+- Automated skill updates on documentation changes
+
+Each example below is marked **[offline]** (no network, no AI), **[network]** (fetches remote content), or **[AI]** (calls an LLM API or spawns a local agent).
+
+---
+
+## Installation
+
+### Basic Installation
+
+```bash
+pip install skill-seekers
+```
+
+### With Platform Dependencies
+
+```bash
+# Google Gemini support
+pip install skill-seekers[gemini]
+
+# OpenAI ChatGPT support
+pip install skill-seekers[openai]
+
+# All LLM platform support
+pip install skill-seekers[all-llms]
+
+# Everything (all source types + platforms, except video-full)
+pip install skill-seekers[all]
+```
+
+### Development Installation
+
+```bash
+git clone https://github.com/yusufkaraaslan/Skill_Seekers.git
+cd Skill_Seekers
+pip install -e ".[all-llms]"
+```
+
+---
+
+## Core APIs
+
+### 1. Skill Conversion API (`get_converter`)
+
+The primary programmatic entry point mirrors the `skill-seekers create` command: a factory returns a `SkillConverter` for any of the 18 source types, and `run()` executes the full extract → build pipeline.
+
+```python
+from skill_seekers.cli.skill_converter import get_converter, CONVERTER_REGISTRY
+
+# get_converter(source_type: str, config: dict[str, Any]) -> SkillConverter
+# SkillConverter.run() -> int   (0 = success, non-zero = failure)
+
+print(sorted(CONVERTER_REGISTRY))
+# ['asciidoc', 'chat', 'config', 'confluence', 'epub', 'github', 'html',
+#  'jupyter', 'local', 'manpage', 'notion', 'openapi', 'pdf', 'pptx',
+#  'rss', 'video', 'web', 'word']
+```
+
+#### Basic Usage — web documentation **[network]**
+
+```python
+from skill_seekers.cli.skill_converter import get_converter
+
+config = {
+    "name": "django",
+    "description": "Use when working with Django web framework",
+    "base_url": "https://docs.djangoproject.com/en/5.0/",
+    "selectors": {"main_content": "article", "title": "h1", "code_blocks": "pre code"},
+    "url_patterns": {"include": ["/en/5.0/"], "exclude": []},
+    "max_pages": 50,
+    "rate_limit": 0.5,
+    "output_dir": "output/django",
+}
+
+converter = get_converter("web", config)
+exit_code = converter.run()          # scrapes, then builds output/django/SKILL.md
+print("ok" if exit_code == 0 else "failed")
+```
+
+#### Template method contract
+
+`run()` is a template method on the `SkillConverter` base class:
+
+1. `extract()` — source-specific extraction (scrape, parse, clone, …)
+2. `build_skill()` — categorize content and write `SKILL.md` + `references/`
+
+`run()` **returns an exit code instead of raising**: exceptions from `extract()`/`build_skill()` are logged and converted to return value `1`. Check the return value, not a `try/except`.
+
+```python
+converter = get_converter("pdf", {"name": "manual", "pdf_path": "manual.pdf"})
+
+# Reuse existing on-disk extracted data (skip extraction, rebuild only):
+converter.skip_scrape = True   # run() checks this attribute
+converter.run()
+```
+
+#### Factory errors **[offline]**
+
+- `ValueError` — unknown source type (message lists supported types)
+- `RuntimeError` — the source type's optional dependency is not installed (message includes the `pip install` hint)
+
+#### Unified config through the factory **[offline construction]**
+
+The `"config"` source type wraps the multi-source `UnifiedScraper` (section 4) behind the same factory. It takes the **factory-shaped dict** — only `config_path` is required:
+
+```python
+from skill_seekers.cli.skill_converter import get_converter
+
+scraper = get_converter("config", {
+    "config_path": "configs/unified/react-unified.json",
+    "output_dir": "output/react-complete",   # optional override
+    "merge_mode": "rule-based",              # optional: 'rule-based' | 'claude-enhanced'
+    "dry_run": True,                         # optional: preview sources, write nothing
+})
+scraper.run()
+```
+
+---
+
+### 2. Source Detection API
+
+`SourceDetector` is what `skill-seekers create` uses to auto-detect the source type from a raw input string. It returns a `SourceInfo` dataclass.
+
+#### Basic Usage **[offline]**
+
+```python
+from skill_seekers.cli.source_detector import SourceDetector
+
+detector = SourceDetector()
+
+# detect(source: str) -> SourceInfo
+info = detector.detect("https://docs.djangoproject.com/")
+print(info.type)            # 'web'
+print(info.parsed)          # {'url': 'https://docs.djangoproject.com/'}
+print(info.suggested_name)  # 'djangoproject'
+print(info.raw_input)       # original input string
+
+detector.detect("fastapi/fastapi").type   # 'github'  -> parsed: {'repo': 'fastapi/fastapi'}
+detector.detect("./manual.pdf").type      # 'pdf'     -> parsed: {'file_path': './manual.pdf'}
+detector.detect("./my-project").type      # 'local'   -> parsed: {'directory': '/abs/path/my-project'}
+detector.detect("configs/react.json").type  # 'config' -> parsed: {'config_path': 'configs/react.json'}
+```
+
+`SourceInfo` fields: `type`, `parsed` (dict, shape depends on `type`), `suggested_name`, `raw_input`.
+
+Note: local-directory detection requires the path to exist on disk — a non-existent `./name` falls through to other detectors (e.g. `owner/repo` GitHub shorthand).
+
+#### Detect-then-convert pipeline **[network for web/github]**
+
+```python
+from skill_seekers.cli.source_detector import SourceDetector
+from skill_seekers.cli.skill_converter import get_converter
+
+info = SourceDetector().detect("./manual.pdf")
+
+config = {
+    "name": info.suggested_name,
+    "pdf_path": info.parsed["file_path"],
+    "output_dir": f"output/{info.suggested_name}",
+}
+get_converter(info.type, config).run()
+```
+
+(The CLI's `create_command.py:_build_config()` is the canonical mapping from `SourceInfo.parsed` to each converter's config keys.)
+
+---
+
+### 3. Direct Converter Construction
+
+Every converter class can be constructed directly with a config dict (the factory does nothing more than registry lookup + optional-dependency check). The config keys below are read by each converter's `__init__` and are verified against v3.7.0.
+
+#### PDF — `PDFToSkillConverter` **[offline — local file processing]**
+
+```python
+from skill_seekers.cli.pdf_scraper import PDFToSkillConverter
+
+converter = PDFToSkillConverter({
+    "name": "product-manual",                  # required
+    "pdf_path": "manual.pdf",                  # path to the PDF
+    "description": "Product manual reference", # optional
+    "output_dir": "output/product-manual",     # optional (default: output/<name>)
+    "extract_options": {                       # optional
+        "chunk_size": 10,         # pages per chunk
+        "min_quality": 5.0,       # quality threshold for extracted text
+        "extract_images": True,
+        "min_image_size": 100,
+    },
+    "categories": {},                          # optional keyword mapping
+})
+converter.run()
+```
+
+#### Web — `DocToSkillConverter` **[network]**
+
+```python
+from skill_seekers.cli.doc_scraper import DocToSkillConverter
+
+converter = DocToSkillConverter({
+    "name": "react",                                  # required
+    "base_url": "https://react.dev/",                 # required
+    "selectors": {"main_content": "article", "title": "h1", "code_blocks": "pre code"},
+    "url_patterns": {"include": ["/learn", "/reference"], "exclude": ["/blog"]},
+    "categories": {},          # optional; smart categorization fills the gap
+    "rate_limit": 0.5,         # seconds between requests
+    "max_pages": 200,          # -1 = unlimited
+    "start_urls": [],          # optional explicit seed URLs
+    "llms_txt_url": None,      # optional llms.txt source
+    "browser": False,          # Playwright rendering for JS-heavy sites
+    "workers": 1,              # parallel scrape workers
+    "async_mode": False,       # asyncio scraping (faster on large sites)
+    "doc_version": "",         # stamped into SKILL.md metadata
+    "output_dir": "output/react",
+})
+converter.run()
+```
+
+Constructor also accepts `dry_run=True` / `resume=True` keyword arguments (or the same keys in the config dict).
+
+#### GitHub — `GitHubScraper` **[network — GitHub API; set `GITHUB_TOKEN` for higher rate limits]**
+
+```python
+from skill_seekers.cli.github_scraper import GitHubScraper
+
+converter = GitHubScraper({
+    "repo": "fastapi/fastapi",        # required, owner/repo
+    "name": "fastapi",                # optional (default: repo short name)
+    "local_repo_path": None,          # optional local clone => unlimited analysis, no API limits
+    "include_code": True,
+    "include_issues": True,
+    "max_issues": 100,
+    "max_comments": 0,
+    "issue_labels": [],               # filter issues by label
+    "issue_state": "all",             # 'open' | 'closed' | 'all'
+    "include_changelog": True,
+    "include_releases": True,
+    "output_dir": "output/fastapi",
+})
+converter.run()
+```
+
+The remaining 15 converters follow the same pattern; see `CONVERTER_REGISTRY` in `src/skill_seekers/cli/skill_converter.py` for the module/class of each, and each class's `__init__` for its config keys (e.g. `word` reads `docx_path`, `local` reads `directory` + the C3.x `detect_patterns`/`extract_test_examples`/… toggles).
+
+---
+
+### 4. Unified Multi-Source Scraping API
+
+`UnifiedScraper` combines multiple sources (any of the 18 supported types) into a single merged skill. It is itself a `SkillConverter` (registered as source type `"config"`).
+
+#### Construction forms
+
+```python
+from skill_seekers.cli.unified_scraper import UnifiedScraper
+
+# 1. Path to a unified config JSON file
+scraper = UnifiedScraper("configs/unified/react-unified.json")
+
+# 2. Already-loaded unified config dict (name + description required)
+scraper = UnifiedScraper({"name": "react-complete", "description": "...", "sources": [...]})
+
+# 3. Factory-shaped dict (what get_converter("config", ...) passes through)
+scraper = UnifiedScraper({"config_path": "configs/unified/react-unified.json"})
+
+# Keyword overrides (win over the config file's values)
+scraper = UnifiedScraper(
+    "configs/unified/react-unified.json",
+    merge_mode="rule-based",        # or 'claude-enhanced' (AI merge)
+    output_dir="output/react-complete",
+    dry_run=False,
+)
+```
+
+#### Running **[network — scrapes each source; AI if merge_mode='claude-enhanced']**
+
+```python
+scraper = UnifiedScraper("configs/unified/react-unified.json")
+scraper.run()    # scrape all sources -> merge -> detect conflicts -> build skill
+```
+
+#### Dry run preview **[offline]**
+
+```python
+UnifiedScraper("configs/unified/react-unified.json", dry_run=True).run()
+# Logs the sources that WOULD be scraped and the output directory; writes nothing.
+```
+
+#### Conflict detection
+
+Conflict detection is a **method on the instance**, not a module-level function. It is called automatically by `run()` after merging; you can also drive the phases manually:
+
+```python
+scraper = UnifiedScraper("configs/unified/react-unified.json")
+scraper.scrape_all_sources()              # [network]
+merged = scraper.merge_sources()
+conflicts = scraper.detect_conflicts()    # -> list of conflict records
+scraper.build_skill(merged)
+```
+
+---
+
+### 5. Skill Packaging API
+
+Package skills for different platforms using the adaptor architecture (Strategy + Factory).
+
+#### Basic Packaging **[offline]**
+
+```python
+from pathlib import Path
+from skill_seekers.cli.adaptors import get_adaptor, ADAPTORS
+
+# get_adaptor(platform: str, config: dict = None) -> SkillAdaptor
+print(sorted(ADAPTORS))
+# ['atlas', 'chroma', 'claude', 'deepseek', 'faiss', 'fireworks', 'gemini',
+#  'haystack', 'ibm-bob', 'kimi', 'langchain', 'llama-index', 'markdown',
+#  'minimax', 'openai', 'opencode', 'openrouter', 'pinecone', 'qdrant',
+#  'qwen', 'together', 'weaviate']
+
+adaptor = get_adaptor("claude")
+
+# package(skill_dir: Path, output_path: Path, ...) -> Path
+package_path = adaptor.package(Path("output/react"), Path("output"))
+print(package_path)   # output/react.zip
+```
+
+`get_adaptor` raises `ValueError` for an unknown platform, and `ImportError` if the platform's optional dependency is missing (with an install hint).
+
+#### Packaging with chunking (RAG/vector targets) **[offline]**
+
+```python
+package_path = adaptor.package(
+    Path("output/react"),
+    Path("output"),
+    enable_chunking=True,        # split content into token-bounded chunks
+    chunk_max_tokens=512,
+    preserve_code_blocks=True,   # never split inside a code fence
+    chunk_overlap_tokens=50,
+)
+```
+
+#### Multi-Platform Packaging **[offline]**
+
+```python
+from pathlib import Path
+from skill_seekers.cli.adaptors import get_adaptor
+
+for platform in ["claude", "gemini", "openai", "markdown"]:
+    adaptor = get_adaptor(platform)
+    pkg = adaptor.package(Path("output/react"), Path("output"))
+    print(f"{platform}: {pkg}")
+```
+
+#### Formatting and capability checks **[offline]**
+
+```python
+from pathlib import Path
+from skill_seekers.cli.adaptors import get_adaptor
+from skill_seekers.cli.adaptors.base import SkillAdaptor, SkillMetadata
+
+adaptor = get_adaptor("claude")
+
+adaptor.PLATFORM               # 'claude'
+adaptor.supports_upload()      # True
+adaptor.supports_enhancement() # True
+adaptor.get_env_var_name()     # 'ANTHROPIC_API_KEY'
+
+# format_skill_md(skill_dir: Path, metadata: SkillMetadata) -> str
+meta = SkillMetadata(name="my-skill", description="When to use this skill")
+text = adaptor.format_skill_md(Path("output/my-skill"), meta)
+```
+
+`SkillMetadata` fields: `name`, `description`, `version` (default `"1.0.0"`), `doc_version`, `author`, `tags`.
+
+#### Shared Embedding Methods
+
+The base `SkillAdaptor` class provides two shared embedding helpers inherited by all vector database adaptors (chroma, weaviate, pinecone, qdrant, faiss):
+
+- `_generate_openai_embeddings(texts, model)` — generate embeddings via the OpenAI API. **[network]**
+- `_generate_st_embeddings(texts, model)` — generate embeddings using a local sentence-transformers model. **[offline]**
+
+These are underscore-prefixed (internal) but shared deliberately, so vector adaptors do not re-implement embedding logic.
+
+---
+
+### 6. Skill Upload API
+
+Upload packaged skills to LLM platforms via their APIs. Signature on the base class:
+
+```python
+# upload(package_path: Path, api_key: str, **kwargs) -> dict[str, Any]
+```
+
+The returned dict's keys are **platform-specific** — inspect the concrete adaptor's `upload()` (e.g. `src/skill_seekers/cli/adaptors/claude.py`) for the exact shape. Check `adaptor.supports_upload()` first: adaptors that don't support upload (e.g. `markdown`) return a result dict with `"success": False` and an explanatory `"message"` instead of uploading.
+
+#### Claude AI Upload **[network — Anthropic API]**
+
+```python
+import os
+from pathlib import Path
+from skill_seekers.cli.adaptors import get_adaptor
+
+adaptor = get_adaptor("claude")
+result = adaptor.upload(
+    Path("output/react.zip"),
+    api_key=os.environ["ANTHROPIC_API_KEY"],
+)
+```
+
+#### Google Gemini Upload **[network — requires `pip install skill-seekers[gemini]`]**
+
+```python
+adaptor = get_adaptor("gemini")
+result = adaptor.upload(Path("output/react.tar.gz"), api_key=os.environ["GOOGLE_API_KEY"])
+```
+
+#### OpenAI Upload **[network — requires `pip install skill-seekers[openai]`]**
+
+```python
+adaptor = get_adaptor("openai")
+result = adaptor.upload(Path("output/react-openai.zip"), api_key=os.environ["OPENAI_API_KEY"])
+```
+
+Use `adaptor.get_env_var_name()` to discover which environment variable a platform conventionally reads, and `adaptor.validate_api_key(key)` for a cheap format check before uploading.
+
+---
+
+### 7. AI Enhancement API
+
+Enhance skills with AI-powered improvements. All API-mode enhancement routes
+through the shared `AgentClient` (`skill_seekers.cli.agent_client`), which
+centralizes provider selection (Anthropic/Gemini/OpenAI/Moonshot), model and
+base-URL overrides, the truncation gate, timeout policy, and atomic
+backup-then-save of SKILL.md.
+
+#### API Mode Enhancement (per-platform adaptor) **[AI — provider API call]**
+
+```python
+import os
+from pathlib import Path
+from skill_seekers.cli.adaptors import get_adaptor
+
+adaptor = get_adaptor('claude')  # also: gemini, openai, and OpenAI-compatible targets
+
+# Enhance SKILL.md via the platform's API (returns True on success).
+# The original is backed up to SKILL.md.backup and the save is atomic.
+ok = adaptor.enhance(
+    Path('output/react/'),
+    os.getenv('ANTHROPIC_API_KEY'),
+)
+```
+
+#### Direct AgentClient usage **[AI]**
+
+```python
+from skill_seekers.cli.agent_client import AgentClient
+
+client = AgentClient(mode='api')          # or mode='local' (spawns a local agent)
+reply = client.call('Summarize this skill...', timeout=600)
+```
+
+`AgentClient(mode='auto'|'api'|'local', agent=None, api_key=None, provider=None, base_url=None, model=None)`; `call(prompt, max_tokens=4096, timeout=None, output_file=None, cwd=None, system=None, temperature=None) -> str | None`. Also: `is_available()`, `get_model()`, `detect_api_key()`.
+
+#### LOCAL Mode Enhancement (local coding agent, free) **[AI — spawns local agent]**
+
+```python
+from skill_seekers.cli.enhance_skill_local import LocalSkillEnhancer
+
+enhancer = LocalSkillEnhancer(
+    'output/react/',
+    agent='claude',        # claude, codex, copilot, opencode, kimi, custom
+)
+enhancer.run(background=True)   # or headless=True (default), daemon=True
+```
+
+Monitor background runs from the CLI:
+
+```bash
+skill-seekers enhance-status output/react/ --watch
+```
+
+> LOCAL mode sets `SKILL_SEEKER_ENHANCE_ACTIVE=1` in the spawned agent's
+> environment and refuses to start when it is already set, preventing
+> recursive agent spawns.
+
+---
+
+### 8. Execution Context
+
+`ExecutionContext` is the centralized, pydantic-validated settings singleton the CLI builds from argparse + config files. Converters and enhancement read from it; programmatic callers can initialize and override it.
+
+```python
+from skill_seekers.cli.execution_context import ExecutionContext
+
+# Classmethods:
+#   initialize(args=None, config_path=None, source_info=None) -> ExecutionContext
+#   get() -> ExecutionContext          (active override, else base singleton)
+#   is_initialized() -> bool
+#   reset() -> None                    (mainly for tests)
+
+ExecutionContext.is_initialized()   # False until initialize() is called
+ctx = ExecutionContext.initialize() # defaults when args is None
+
+ctx.enhancement.level    # 2
+ctx.scraping.max_pages   # -1 (unlimited)
+ctx.output.output_dir    # None
+ctx.analysis.depth       # 'surface'
+```
+
+#### Temporary overrides (context manager) **[offline]**
+
+`override(**kwargs)` is a context manager; double-underscore keys address nested settings groups (`source`, `enhancement`, `output`, `scraping`, `analysis`). Overrides are **context-local** (stored in a `contextvars.ContextVar`), so concurrent asyncio tasks each see only their own override, and nested overrides stack and unwind cleanly:
+
+```python
+ctx = ExecutionContext.get()
+
+with ctx.override(enhancement__level=3, scraping__max_pages=100):
+    active = ExecutionContext.get()
+    assert active.enhancement.level == 3      # inside: overridden
+
+assert ExecutionContext.get().enhancement.level == 2  # outside: restored
+```
+
+Caveat: contextvars flow into asyncio tasks automatically but into worker threads only via `contextvars.copy_context().run(...)` — a bare `threading.Thread` sees the base singleton, not your override.
+
+---
+
+### 9. Services Layer (`skill_seekers.services`)
+
+Domain logic shared by the CLI and the MCP server. Importable **without** the `[mcp]` extra. Import from the submodules:
+
+```python
+from skill_seekers.services.marketplace_manager import MarketplaceManager
+from skill_seekers.services.source_manager import SourceManager
+from skill_seekers.services.config_publisher import ConfigPublisher, detect_category
+from skill_seekers.services.git_repo import GitConfigRepo
+```
+
+#### Marketplace registry CRUD **[offline — local registry file]**
+
+```python
+mm = MarketplaceManager()        # or MarketplaceManager(config_dir="~/.skill-seekers")
+mm.list_marketplaces()           # -> list[dict]; also: add/get/update/remove_marketplace
+```
+
+#### Config source registry CRUD **[offline]**
+
+```python
+sm = SourceManager()
+sm.list_sources()                # also: add/get/update/remove_source
+```
+
+#### Config category detection **[offline]**
+
+```python
+detect_category({"name": "react", "description": "React frontend UI library docs"})
+# 'web-frameworks'   (keyword scoring over CATEGORY_KEYWORDS)
+```
+
+#### Git-backed config repositories **[network — clones/pulls]**
+
+```python
+repo = GitConfigRepo()                       # or GitConfigRepo(cache_dir=...)
+repo.validate_git_url("https://github.com/owner/configs.git")   # offline check
+path = repo.clone_or_pull("https://github.com/owner/configs.git")  # [network]
+configs = repo.find_configs(path)
+```
+
+`ConfigPublisher` (`ConfigPublisher(cache_dir=None)`) pushes configs to registered config-source repos; `MarketplacePublisher` publishes packaged skills to plugin-marketplace repos. Both perform git pushes **[network]**.
+
+---
+
+## Configuration Objects
+
+The full config-file schema (single-source and unified) is documented in **[CONFIG_FORMAT.md](CONFIG_FORMAT.md)** — that is the authoritative reference. Summary:
+
+### Web (single-source) config keys
+
+These are the keys `DocToSkillConverter` reads (same dict whether loaded from a `configs/*.json` file or built in code):
+
+| Field | Type | Default | Description |
+|-------|------|---------|-------------|
+| `name` | string | *required* | Skill name (alphanumeric + hyphens) |
+| `base_url` | string | *required* | Documentation website URL |
+| `description` | string | generated | When to use this skill |
+| `selectors` | object | `{}` | CSS selectors (`main_content`, `title`, `code_blocks`) |
+| `url_patterns` | object | `{}` | `include` / `exclude` URL substring lists |
+| `categories` | object | `{}` | Category keywords mapping |
+| `rate_limit` | float | `0.5` | Delay between requests (seconds) |
+| `max_pages` | int | `-1` | Maximum pages to scrape (-1 = unlimited) |
+| `start_urls` | array | `[]` | Explicit seed URLs |
+| `llms_txt_url` | string | `null` | URL to llms.txt file |
+| `async_mode` | bool | `false` | Asyncio scraping (faster on large sites) |
+| `browser` | bool | `false` | Playwright rendering for JS-heavy sites |
+| `workers` | int | `1` | Parallel scrape workers |
+| `output_dir` | string | `output/<name>` | Where the skill is written |
+
+### Unified Config Schema (Multi-Source)
+
+Supports all 18 source types: `documentation`, `github`, `pdf`, `local`, `word`, `video`, `epub`, `jupyter`, `html`, `openapi`, `asciidoc`, `pptx`, `rss`, `manpage`, `confluence`, `notion`, `chat`, `config`.
+
+```json
+{
+  "name": "framework-unified",
+  "description": "Complete framework documentation",
+  "merge_mode": "rule-based",
+  "sources": [
+    {
+      "type": "documentation",
+      "base_url": "https://docs.example.com/",
+      "selectors": { "main_content": "article" }
+    },
+    {
+      "type": "github",
+      "repo": "org/repo",
+      "include_code": true
+    },
+    {
+      "type": "pdf",
+      "path": "manual.pdf"
+    },
+    {
+      "type": "openapi",
+      "path": "specs/openapi.yaml"
+    },
+    {
+      "type": "video",
+      "url": "https://www.youtube.com/watch?v=example"
+    },
+    {
+      "type": "jupyter",
+      "path": "notebooks/examples.ipynb"
+    },
+    {
+      "type": "confluence",
+      "base_url": "https://company.atlassian.net/wiki",
+      "space_key": "DOCS"
+    }
+  ]
+}
+```
+
+Configs are validated on load by `skill_seekers.cli.config_validator.validate_config(config_path)`, which the CLI and `UnifiedScraper` call for you.
+
+---
+
+## Error Handling
+
+The Python API signals failure three different ways — match on the layer you call:
+
+```python
+from pathlib import Path
+from skill_seekers.cli.skill_converter import get_converter
+from skill_seekers.cli.adaptors import get_adaptor
+
+# 1. Factory-time errors RAISE:
+try:
+    converter = get_converter("web", config)
+except ValueError as e:      # unknown source type
+    print(e)
+except RuntimeError as e:    # missing optional dependency (includes pip install hint)
+    print(e)
+
+try:
+    adaptor = get_adaptor("chroma")
+except ValueError as e:      # unknown platform
+    print(e)
+except ImportError as e:     # optional dependency not installed
+    print(e)
+
+# 2. Conversion errors are RETURN CODES (run() catches and logs exceptions):
+if converter.run() != 0:
+    raise SystemExit("skill build failed — see log output")
+
+# 3. Adaptor operations either RAISE (network/API errors during real uploads)
+#    or report failure in the returned dict — gate on capability and check
+#    result["success"]:
+if adaptor.supports_upload():
+    result = adaptor.upload(Path("output/react.zip"), api_key=key)
+    if not result.get("success"):
+        print(result.get("message"))
+```
+
+There is no `skill_seekers.exceptions` module — standard exceptions (`ValueError`, `RuntimeError`, `ImportError`, `FileNotFoundError`) are used throughout.
+
+---
+
+## Testing Your Integration
+
+Use `dry_run` and small `max_pages` limits to keep tests fast and offline-friendly:
+
+```python
+from skill_seekers.cli.skill_converter import get_converter
+from skill_seekers.cli.source_detector import SourceDetector
+
+
+def test_source_detection():            # [offline]
+    info = SourceDetector().detect("https://docs.example.com/")
+    assert info.type == "web"
+    assert info.parsed["url"] == "https://docs.example.com/"
+
+
+def test_unified_dry_run(tmp_path):     # [offline] — previews without scraping
+    import json
+    cfg = tmp_path / "unified.json"
+    cfg.write_text(json.dumps({
+        "name": "test",
+        "description": "Test skill",   # name + description are required
+        "sources": [{"type": "github", "repo": "owner/repo"}],
+    }))
+    scraper = get_converter("config", {"config_path": str(cfg), "dry_run": True})
+    assert scraper.run() == 0
+
+
+def test_packaging(tmp_path):           # [offline]
+    from pathlib import Path
+    from skill_seekers.cli.adaptors import get_adaptor
+
+    skill = tmp_path / "skill"
+    skill.mkdir()
+    (skill / "SKILL.md").write_text("---\nname: t\ndescription: d\n---\n# T\n")
+    pkg = get_adaptor("markdown").package(skill, tmp_path)
+    assert pkg.exists()
+```
+
+---
+
+## Performance Notes
+
+- **Async scraping**: set `"async_mode": True` in a web config for 2–3x faster scraping on large sites; `"workers": N` parallelizes the thread-based scraper.
+- **Rebuild without re-scraping**: set `converter.skip_scrape = True` before `run()` to rebuild `SKILL.md` from existing on-disk extracted data (`output/<name>_data/`).
+- **Resume**: web configs support checkpointing — pass `resume=True` to `DocToSkillConverter` (or `"resume": True` in the config) to continue an interrupted scrape.
+- **Batch processing**: converters are independent; run several `get_converter(...).run()` calls in a `ThreadPoolExecutor`. Don't share one `ExecutionContext.override()` across plain threads (see section 8 caveat).
+
+---
+
+## CI/CD Integration Examples
+
+For pipelines, prefer the CLI — it is the stable interface:
+
+### GitHub Actions
+
+```yaml
+name: Generate Skills
+
+on:
+  schedule:
+    - cron: '0 0 * * *'  # Daily at midnight
+  workflow_dispatch:
+
+jobs:
+  generate-skills:
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v3
+
+      - uses: actions/setup-python@v4
+        with:
+          python-version: '3.11'
+
+      - name: Install Skill Seekers
+        run: pip install skill-seekers[all-llms]
+
+      - name: Generate Skills
+        env:
+          ANTHROPIC_API_KEY: ${{ secrets.ANTHROPIC_API_KEY }}
+          GOOGLE_API_KEY: ${{ secrets.GOOGLE_API_KEY }}
+        run: |
+          skill-seekers install --config react --target claude
+          skill-seekers install --config vue --target gemini
+
+      - name: Archive Skills
+        uses: actions/upload-artifact@v3
+        with:
+          name: skills
+          path: output/**/*.zip
+```
+
+### GitLab CI
+
+```yaml
+generate_skills:
+  image: python:3.11
+  script:
+    - pip install skill-seekers[all-llms]
+    - skill-seekers install --config react --target claude
+    - skill-seekers install --config vue --target gemini --no-upload
+  artifacts:
+    paths:
+      - output/
+  only:
+    - schedules
+```
+
+---
+
+## Best Practices
+
+### 1. **Prefer the CLI for automation; pin the version for Python imports**
+```bash
+pip install skill-seekers==3.7.0   # internals can shift between minors
+```
+
+### 2. **Use the factory, not hardcoded classes**
+```python
+# Good: registry-driven
+converter = get_converter(info.type, config)
+adaptor = get_adaptor(target_platform)
+
+# Brittle: hardcoded imports break when modules move
+```
+
+### 3. **Check run() return codes**
+```python
+if get_converter("web", config).run() != 0:
+    raise SystemExit(1)   # run() logs the exception; it does not re-raise
+```
+
+### 4. **Cache scraped data, rebuild cheaply**
+```python
+converter = get_converter("web", config)
+converter.run()                 # first run: scrape + build (slow)
+
+converter = get_converter("web", config)
+converter.skip_scrape = True
+converter.run()                 # rebuild from output/<name>_data/ (fast)
+```
+
+### 5. **Probe adaptor capabilities before calling**
+```python
+adaptor = get_adaptor(platform)
+if adaptor.supports_upload():
+    adaptor.upload(pkg, api_key=os.environ[adaptor.get_env_var_name()])
+```
+
+### 6. **Use dry runs in tests**
+```python
+get_converter("config", {"config_path": cfg, "dry_run": True}).run()
+```
+
+---
+
+## API Reference Summary
+
+| API | Import | Use Case |
+|-----|--------|----------|
+| **Skill conversion factory** | `skill_seekers.cli.skill_converter.get_converter` | Any of the 18 source types → skill |
+| **Converter registry** | `skill_seekers.cli.skill_converter.CONVERTER_REGISTRY` | Source type → (module, class) lookup |
+| **Source detection** | `skill_seekers.cli.source_detector.SourceDetector` | Auto-detect type from raw input |
+| **Web docs** | `skill_seekers.cli.doc_scraper.DocToSkillConverter` | Documentation websites |
+| **GitHub repos** | `skill_seekers.cli.github_scraper.GitHubScraper` | Code + docs + community analysis |
+| **PDF** | `skill_seekers.cli.pdf_scraper.PDFToSkillConverter` | PDF documents |
+| **Local codebase** | `skill_seekers.cli.codebase_scraper.CodebaseAnalyzer` | Local directories (C3.x pipeline) |
+| **Multi-source** | `skill_seekers.cli.unified_scraper.UnifiedScraper` | Merge 18 source types + conflict detection |
+| **Packaging / upload / enhance** | `skill_seekers.cli.adaptors.get_adaptor` | 22 platform targets |
+| **AI enhancement** | `skill_seekers.cli.agent_client.AgentClient` | API or local-agent LLM calls |
+| **Local-agent enhancement** | `skill_seekers.cli.enhance_skill_local.LocalSkillEnhancer` | Free enhancement via coding agents |
+| **Settings singleton** | `skill_seekers.cli.execution_context.ExecutionContext` | Initialize / get / override settings |
+| **Marketplace registry** | `skill_seekers.services.marketplace_manager.MarketplaceManager` | Marketplace CRUD |
+| **Config sources** | `skill_seekers.services.source_manager.SourceManager` | Config source registry CRUD |
+| **Config publishing** | `skill_seekers.services.config_publisher` | Push configs; `detect_category()` |
+| **Git config repos** | `skill_seekers.services.git_repo.GitConfigRepo` | Clone/pull + config discovery |
+
+The other 14 converter classes (word, epub, video, jupyter, html, openapi, asciidoc, pptx, rss, manpage, confluence, notion, chat) are listed in `CONVERTER_REGISTRY`.
+
+---
+
+## Additional Resources
+
+- **[Main Documentation](../../README.md)** - Complete user guide
+- **[CLI Reference](CLI_REFERENCE.md)** - The stable command-line interface
+- **[Config Format](CONFIG_FORMAT.md)** - Authoritative config schema
+- **[MCP Setup](../guides/MCP_SETUP.md)** - MCP server integration
+- **[Multi-LLM Support](../integrations/MULTI_LLM_SUPPORT.md)** - Platform comparison
+- **[CHANGELOG](../../CHANGELOG.md)** - Version history and API changes
+
+---
+
+**Version:** 3.7.0
+**Last Updated:** 2026-06-11