Duc-Tam Nguyen
e6afa91e09
kage renders every page in headless Chrome, snapshots the final DOM, strips all JavaScript, and localises CSS, images, and fonts so a site can be browsed offline as a plain folder of files. The engine is split into small packages: urlx deterministic URL to local-path mapping and scope rules sanitize remove scripts, on* handlers, and javascript: URLs asset rewrite HTML and CSS references, download assets browser headless Chrome pool over the DevTools protocol robots robots.txt matcher clone the orchestrator: a polite resumable breadth-first crawl The cli package wires a cobra and fang command surface with two commands, clone and serve. Every pure package has table tests; the browser and clone packages add Chrome-driven end-to-end tests that skip when no browser is present or under -short. CI runs gofmt, vet, build, race tests, golangci-lint, govulncheck, and a tidy check on Linux and macOS. A goreleaser config fans one tag out to archives, deb/rpm/apk, a Chromium-bundled GHCR image, and the package managers. A tago docs site builds to Pages and Cloudflare.
49 lines
1.4 KiB
Markdown
49 lines
1.4 KiB
Markdown
---
|
|
title: "Resuming a run"
|
|
description: "Pick up an interrupted clone where it left off, and start fresh when you want to."
|
|
weight: 30
|
|
---
|
|
|
|
Cloning a large site can take a while, and runs get interrupted: you press
|
|
Ctrl-C, your laptop sleeps, the network drops. kage is built to pick up where it
|
|
left off.
|
|
|
|
## How resume works
|
|
|
|
As it writes each page, kage records it in a small state file inside the mirror,
|
|
at `<host>/_kage/state.json`. When a run ends, for any reason, that file holds
|
|
the set of pages already written. Resume is **on by default**: the next time you
|
|
run the same clone, kage loads the state and skips every page it already wrote,
|
|
re-crawling only what is left.
|
|
|
|
```bash
|
|
kage clone example.com
|
|
# ... press Ctrl-C partway through ...
|
|
# interrupted; resume state saved (rerun to continue)
|
|
|
|
kage clone example.com
|
|
# resume: 137 pages already done
|
|
```
|
|
|
|
Ctrl-C is a clean stop: kage cancels in-flight renders, flushes the state file,
|
|
and exits. You will not lose the pages already written.
|
|
|
|
## Start fresh
|
|
|
|
To ignore any previous run and rebuild the mirror from scratch, delete the
|
|
existing host folder first with `--force`:
|
|
|
|
```bash
|
|
kage clone example.com --force
|
|
```
|
|
|
|
This removes `kage-out/example.com/` before crawling, so nothing from a prior run
|
|
carries over.
|
|
|
|
To run without reading or writing any resume state at all, for a strictly
|
|
one-shot clone, use `--no-resume`:
|
|
|
|
```bash
|
|
kage clone example.com --no-resume
|
|
```
|