kage/docs/content/reference/cli.md at e6afa91e09c42331d5bb52dfcf4d47b660706653

Files

T

Duc-Tam Nguyen e6afa91e09 Add the clone engine, CLI, tests, CI, and docs

kage renders every page in headless Chrome, snapshots the final
DOM, strips all JavaScript, and localises CSS, images, and fonts
so a site can be browsed offline as a plain folder of files.

The engine is split into small packages:

  urlx      deterministic URL to local-path mapping and scope rules
  sanitize  remove scripts, on* handlers, and javascript: URLs
  asset     rewrite HTML and CSS references, download assets
  browser   headless Chrome pool over the DevTools protocol
  robots    robots.txt matcher
  clone     the orchestrator: a polite resumable breadth-first crawl

The cli package wires a cobra and fang command surface with two
commands, clone and serve. Every pure package has table tests; the
browser and clone packages add Chrome-driven end-to-end tests that
skip when no browser is present or under -short.

CI runs gofmt, vet, build, race tests, golangci-lint, govulncheck,
and a tidy check on Linux and macOS. A goreleaser config fans one
tag out to archives, deb/rpm/apk, a Chromium-bundled GHCR image,
and the package managers. A tago docs site builds to Pages and
Cloudflare.

2026-06-14 18:22:25 +07:00

2.8 KiB

Raw Blame History

title, description, weight

title	description	weight
CLI reference	Every kage command and flag.	10

kage [command] [flags]

Two commands: clone fetches a site into an offline folder, serve previews one. Run kage <command> --help for the canonical, up-to-date list.

kage clone

kage clone <url> [flags]

Renders each page in headless Chrome, strips all JavaScript, localises CSS, images, and fonts, and writes a browsable mirror to <out>/<host>/.

Output

Flag	Default	Meaning
`-o, --out`	`kage-out`	Output root; the mirror lands in `<out>/<host>/`
`--reserved`	`_kage`	Reserved directory name for assets and crawl state
`-f, --force`	`false`	Delete any existing mirror for the host before crawling
`--no-resume`	`false`	Do not read or write resume state

Scope

Flag	Default	Meaning
`-p, --max-pages`	`0`	Stop after N pages (0 = unlimited)
`-d, --max-depth`	`0`	Link-follow depth cap (0 = unlimited)
`--scope-prefix`		Only crawl pages whose path starts with this prefix
`--subdomains`	`false`	Treat subdomains of the seed host as in scope
`--exclude`		Path prefixes to skip (repeatable)
`--traversal`	`bfs`	Frontier order: `bfs` or `dfs`

Politeness

Flag	Default	Meaning
`--no-robots`	`false`	Ignore `robots.txt`
`--no-sitemap`	`false`	Do not seed URLs from `sitemap.xml`
`--user-agent`	Chrome UA	User-Agent for asset and robots fetches

Rendering

Flag	Default	Meaning
`--scroll`	`false`	Auto-scroll each page to trigger lazy loading
`--settle`	`1.5s`	Network-idle quiet period before snapshotting the DOM
`--render-timeout`	`30s`	Hard cap per page render
`--headful`	`false`	Run Chrome with a visible window (debugging)
`--chrome`		Path to the Chrome/Chromium binary
`--control-url`		Attach to an existing Chrome DevTools endpoint
`--keep-noscript`	`false`	Unwrap `<noscript>` content instead of dropping it

Concurrency and limits

Flag	Default	Meaning
`--workers`	`4`	Concurrent page render workers
`--asset-workers`	`8`	Concurrent asset download workers
`--browser-pages`	`4`	Chrome page-pool size
`--max-asset-mb`	`25`	Skip assets larger than N MB
`--timeout`	`30s`	Per-request timeout
`-q, --quiet`	`false`	Suppress per-page progress lines

kage serve

kage serve [dir] [flags]

Runs a local static file server over a cloned folder. With no dir, serves the current directory.

Flag	Default	Meaning
`-a, --addr`	`127.0.0.1:8800`	Address to listen on

2.8 KiB Raw Blame History