From 26dabd03bf2860d6982ca42f0c411d7ff591c466 Mon Sep 17 00:00:00 2001 From: Duc-Tam Nguyen Date: Sun, 14 Jun 2026 20:17:31 +0700 Subject: [PATCH] Document kage pack and kage open Add a packing guide, the pack/open command reference, README usage, and an Unreleased changelog section covering the zim package and the two commands. --- CHANGELOG.md | 18 ++++ README.md | 23 +++++ docs/content/guides/packing-a-mirror.md | 123 ++++++++++++++++++++++++ docs/content/reference/cli.md | 41 +++++++- 4 files changed, 203 insertions(+), 2 deletions(-) create mode 100644 docs/content/guides/packing-a-mirror.md diff --git a/CHANGELOG.md b/CHANGELOG.md index b8c941f..49bf475 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -6,6 +6,24 @@ All notable changes to kage are recorded here. The format follows ## [Unreleased] +### Added + +- `kage pack ` packs a cloned folder into one distributable file. + `--format zim` (the default) writes an open ZIM archive, the same single-file + format Kiwix uses; `--format binary` appends that archive to a copy of kage to + produce a self-contained executable that serves the site offline when run. + Flags cover the output path, metadata (`--title`, `--description`, + `--language`, `--date`), a `--base` binary for cross-platform viewers, and + `--no-compress`. +- `kage open ` serves a packed ZIM over a local HTTP server and opens + your browser, the read side of `kage pack --format zim`. +- A pure-Go `zim` package that writes and reads the ZIM format: a fixed header, + MIME and pointer lists, zstd-compressed (or stored) clusters, redirects, and a + trailing MD5. It reads xz clusters so archives from other tooling open, and + writes zstd or stored only. Packing is deterministic: the same mirror produces + a byte-identical archive, with the UUID derived from the content rather than + randomised. + ## [0.1.0] - 2026-06-14 The first release. kage clones a live website into a self-contained folder you diff --git a/README.md b/README.md index 545c2e5..bec4664 100644 --- a/README.md +++ b/README.md @@ -54,6 +54,8 @@ the `KAGE_CHROME` environment variable. The container image bundles Chromium. ```bash kage clone [flags] kage serve [dir] [flags] +kage pack [flags] +kage open [flags] ``` ### Clone @@ -111,6 +113,27 @@ kage serve kage-out/example.com # open http://127.0.0.1:8800 ``` +### Pack + +`kage pack` collapses a mirror into one distributable file. The default is an +open ZIM archive (the format Kiwix uses); `--format binary` produces a +self-contained executable that serves the site offline when run. + +```bash +# A ZIM archive, browsable with kage open or any ZIM reader +kage pack kage-out/example.com +kage open example.com.zim + +# A single executable that is the site +kage pack kage-out/example.com --format binary +./example +``` + +Packing is deterministic: the same mirror produces a byte-identical archive. The +ZIM holds the whole mirror with text zstd-compressed and media stored as-is, so +it is one tidy file to move, checksum, or hand to someone. The binary carries a +full kage, so the recipient needs nothing installed. + ## How it works ``` diff --git a/docs/content/guides/packing-a-mirror.md b/docs/content/guides/packing-a-mirror.md new file mode 100644 index 0000000..865fbbf --- /dev/null +++ b/docs/content/guides/packing-a-mirror.md @@ -0,0 +1,123 @@ +--- +title: "Packing a mirror" +description: "Turn a cloned folder into one ZIM file or a self-contained offline viewer with kage pack." +weight: 30 +--- + +A clone is a folder of files, which is easy to browse but awkward to move around: +copying thousands of small files is slow, and handing someone a directory is less +tidy than handing them one file. `kage pack` collapses a mirror into a single +distributable artifact, either an open ZIM archive or a self-contained executable +that serves the site offline when run. + +## Pack to a ZIM file + +ZIM is the open, single-file offline-archive format Kiwix uses. `kage pack` +writes one from a cloned host directory: + +```bash +kage pack kage-out/example.com +``` + +``` +packed example.com.zim + size 4.2 MiB + open kage open example.com.zim +``` + +The whole mirror, pages and assets, lives in that one file. Text is zstd +compressed; already-compressed media (images, fonts, video) is stored as-is. +Packing the same mirror twice produces a byte-identical file, so a ZIM is safe to +checksum, diff, and cache. + +If you cloned with the default output directory, you can pass a bare host name and +kage finds the mirror for you: + +```bash +kage clone example.com +kage pack example.com +``` + +### Read it back + +`kage open` is the read side: it serves a ZIM over a local HTTP server and opens +your browser, the same way `kage serve` does for a folder. + +```bash +kage open example.com.zim +``` + +``` +kage open example.com.zim + open http://127.0.0.1:8800 + press Ctrl-C to stop +``` + +Any other ZIM reader (Kiwix among them) can also open the file. kage writes a +structurally valid archive with the standard metadata; full-text search indexes +are not written, so browsing works everywhere but in-reader search is limited. + +## Pack to a self-contained binary + +`--format binary` appends the ZIM to a copy of kage, producing one executable +that *is* the site. Run it and it serves the mirror on a free local port and +opens your browser; it ignores its arguments, because the binary is the site, not +the kage CLI. + +```bash +kage pack kage-out/example.com --format binary +``` + +``` +packed example + size 21.9 MiB + run ./example to view the site offline +``` + +```bash +./example +``` + +``` +serving offline site at http://127.0.0.1:52431 (Ctrl-C to stop) +``` + +The binary carries a full kage, so it is tens of megabytes regardless of site +size; the trade is that the recipient needs nothing installed, not even kage. + +### Build a viewer for another platform + +The appended archive is platform-independent; only the base executable carries +the architecture. Point `--base` at a kage binary built for another OS (download +one from a kage release) to produce a viewer for that platform from your own +machine: + +```bash +# From macOS, build a Windows viewer +kage pack kage-out/example.com --format binary --base kage-windows-amd64.exe +# -> example.exe +``` + +### macOS note + +A binary you built or downloaded may be quarantined by Gatekeeper on first run. +kage prints the exact command to clear it: + +```bash +xattr -d com.apple.quarantine ./example +``` + +## Metadata and options + +```bash +kage pack kage-out/example.com \ + --title "Example, offline" \ + --description "A snapshot taken for archival" \ + --language eng \ + --date 2026-06-14 +``` + +`--title` defaults to the main page's ``, then the host name. `--date` +defaults to today; pass a fixed value for a fully reproducible file. `--no-compress` +stores every cluster raw, which packs fastest and lets a reader without zstd open +the result. `-o/--out` overrides the output path for either format. diff --git a/docs/content/reference/cli.md b/docs/content/reference/cli.md index 8786739..0ab795a 100644 --- a/docs/content/reference/cli.md +++ b/docs/content/reference/cli.md @@ -8,8 +8,9 @@ weight: 10 kage [command] [flags] ``` -Two commands: `clone` fetches a site into an offline folder, `serve` previews -one. Run `kage <command> --help` for the canonical, up-to-date list. +Four commands: `clone` fetches a site into an offline folder, `serve` previews +one, `pack` collapses a mirror into a single file, and `open` serves a packed +file. Run `kage <command> --help` for the canonical, up-to-date list. ## kage clone @@ -84,3 +85,39 @@ current directory. | Flag | Default | Meaning | |------|---------|---------| | `-a, --addr` | `127.0.0.1:8800` | Address to listen on | + +## kage pack + +``` +kage pack <mirror-dir> [flags] +``` + +Packs a cloned mirror into one distributable file: an open ZIM archive, or a +self-contained executable that serves the site offline when run. A bare host name +is resolved against the default output directory, so `kage pack example.com` +works right after `kage clone example.com`. + +| Flag | Default | Meaning | +|------|---------|---------| +| `--format` | `zim` | Output format: `zim` or `binary` | +| `-o, --out` | per format | Output path; `<host>.zim` for zim, `<host>` (or `<host>.exe`) for binary | +| `--base` | this kage | Base kage binary to append to (`--format binary`); point at another platform's binary to build a viewer for it | +| `--no-compress` | `false` | Store every cluster raw, no zstd | +| `--title` | main page `<title>` | Archive title | +| `--description` | | Archive description | +| `--language` | `eng` | Archive language code | +| `--date` | today | Archive date (`YYYY-MM-DD`); pass a fixed value for a reproducible file | + +## kage open + +``` +kage open <file.zim> [flags] +``` + +Serves a packed ZIM over a local HTTP server for offline reading, the read side +of `kage pack --format zim`. + +| Flag | Default | Meaning | +|------|---------|---------| +| `-a, --addr` | `127.0.0.1:8800` | Address to listen on | +| `--open` | `true` | Open the default browser (`--open=false` to skip) |