Add license and readme

Start the kage repo: a tool that clones a website into a
self-contained folder you can browse offline, with the
JavaScript stripped out.
This commit is contained in:
Duc-Tam Nguyen
2026-06-14 18:21:50 +07:00
commit f7c5a9a1c6
5 changed files with 178 additions and 0 deletions
+11
View File
@@ -0,0 +1,11 @@
/bin/
/dist/
/kage-out/
*.out
.DS_Store
# Built docs site (tago output) and its cache
/docs/public/
/docs/public-pages/
/docs/public-cf/
**/.tago-cache.db
+3
View File
@@ -0,0 +1,3 @@
[submodule "docs/themes/tago-doks"]
path = docs/themes/tago-doks
url = https://github.com/tamnd/tago-doks
+21
View File
@@ -0,0 +1,21 @@
MIT License
Copyright (c) 2026 Duc-Tam Nguyen
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
+142
View File
@@ -0,0 +1,142 @@
# kage
**kage** (影, "shadow") clones a website into a self-contained folder you can
browse offline, with all the JavaScript stripped out. It renders every page in
headless Chrome, snapshots the final rendered DOM, removes every script and
event handler, and downloads the CSS, images, and fonts and rewrites them to
local paths. The result looks like the live site but runs no code: a plain
folder of `.html` files you can open straight from disk.
```bash
kage clone example.com
kage serve kage-out/example.com
```
## Why
Saving a page with "Save As" gives you a copy that still phones home, still runs
analytics, and often renders blank because the markup is built by JavaScript at
runtime. kage takes the opposite approach:
- **Render first, save second.** Each page goes through real headless Chrome, so
a page whose content is assembled by JavaScript is captured the way a human
would have seen it, not as an empty shell.
- **Strip every script.** Once the DOM is captured, kage removes all `<script>`
tags, every `on*` event handler, and any `javascript:` URL. The saved page is
inert: no tracking, no network calls, no surprises.
- **Keep the layout.** Stylesheets, images, fonts, and media are downloaded and
rewritten to relative local paths, so the offline copy looks like the original.
- **Stay browsable.** In-scope links are rewritten to point at the other saved
pages, so you can click around the mirror exactly as you would the live site.
## Install
```bash
# Go
go install github.com/tamnd/kage/cmd/kage@latest
# Homebrew (once the tap is published)
brew install tamnd/tap/kage
# Container (Chromium bundled)
docker run -v "$PWD/out:/out" ghcr.io/tamnd/kage clone example.com
```
Prebuilt archives, `.deb`/`.rpm`/`.apk` packages, and a multi-arch image are
attached to each [release](https://github.com/tamnd/kage/releases).
kage drives a real browser, so it needs Chrome or Chromium available. It finds a
system install automatically; point it at a specific binary with `--chrome` or
the `KAGE_CHROME` environment variable. The container image bundles Chromium.
## Usage
```bash
kage clone <url> [flags]
kage serve [dir] [flags]
```
### Clone
```bash
# Clone a whole site into kage-out/<host>/
kage clone https://example.com
# Limit the crawl
kage clone example.com --max-pages 200 --max-depth 3
# Only a section of the site
kage clone example.com --scope-prefix /docs
# Include subdomains, and trigger lazy-loaded images by scrolling
kage clone example.com --subdomains --scroll
# Resume an interrupted run (on by default; Ctrl-C saves state)
kage clone example.com
```
Common flags:
| Flag | Default | Meaning |
|------|---------|---------|
| `-o, --out` | `kage-out` | Output root; the mirror lands in `<out>/<host>/` |
| `-p, --max-pages` | `0` | Stop after N pages (0 = unlimited) |
| `-d, --max-depth` | `0` | Link-follow depth cap (0 = unlimited) |
| `--scope-prefix` | | Only crawl pages whose path starts with this prefix |
| `--subdomains` | `false` | Treat subdomains of the seed host as in scope |
| `--exclude` | | Path prefixes to skip (repeatable) |
| `--scroll` | `false` | Auto-scroll each page to trigger lazy loading |
| `--workers` | `4` | Concurrent page render workers |
| `--no-robots` | `false` | Ignore `robots.txt` (be polite) |
| `-f, --force` | `false` | Delete any existing mirror for the host first |
| `--chrome` | | Path to the Chrome/Chromium binary |
Run `kage clone --help` for the full list.
### Serve
`kage serve` runs a local static file server over a cloned folder so links and
assets resolve the way they would on a real host:
```bash
kage serve kage-out/example.com
# open http://127.0.0.1:8800
```
## How it works
```
seed URL ─▶ headless Chrome ─▶ final DOM ─▶ strip JS ─▶ localise assets ─▶ disk
(render) (snapshot) (sanitize) (rewrite links)
```
A clone is a polite breadth-first crawl. Pages are rendered by a pool of Chrome
tabs; assets are fetched over plain HTTP by a separate worker pool. Every URL
maps deterministically to a local path, so links can be rewritten before the
asset they point at has even finished downloading. The crawl honours
`robots.txt` and seeds itself from `sitemap.xml` by default. Output layout:
```
kage-out/example.com/
├── index.html # the home page, scripts stripped
├── about/index.html # /about
├── _kage/ # reserved: assets and crawl state
│ ├── example.com/site.css # localised stylesheet (url() rewritten)
│ ├── example.com/logo.png
│ └── state.json # visited set, for --resume
└── ...
```
## Building from source
```bash
git clone https://github.com/tamnd/kage
cd kage
make build # -> bin/kage
make test # full suite, including Chrome-driven end-to-end tests
make test-short # skip the tests that launch a browser
```
## License
MIT. See [LICENSE](LICENSE).
Vendored Submodule
+1
Submodule docs/themes/tago-doks added at d506e0b273