Add the clone engine, CLI, tests, CI, and docs

kage renders every page in headless Chrome, snapshots the final
DOM, strips all JavaScript, and localises CSS, images, and fonts
so a site can be browsed offline as a plain folder of files.

The engine is split into small packages:

  urlx      deterministic URL to local-path mapping and scope rules
  sanitize  remove scripts, on* handlers, and javascript: URLs
  asset     rewrite HTML and CSS references, download assets
  browser   headless Chrome pool over the DevTools protocol
  robots    robots.txt matcher
  clone     the orchestrator: a polite resumable breadth-first crawl

The cli package wires a cobra and fang command surface with two
commands, clone and serve. Every pure package has table tests; the
browser and clone packages add Chrome-driven end-to-end tests that
skip when no browser is present or under -short.

CI runs gofmt, vet, build, race tests, golangci-lint, govulncheck,
and a tidy check on Linux and macOS. A goreleaser config fans one
tag out to archives, deb/rpm/apk, a Chromium-bundled GHCR image,
and the package managers. A tago docs site builds to Pages and
Cloudflare.
This commit is contained in:
Duc-Tam Nguyen
2026-06-14 18:22:25 +07:00
parent f7c5a9a1c6
commit e6afa91e09
48 changed files with 4697 additions and 0 deletions
+45
View File
@@ -0,0 +1,45 @@
// Package cli wires kage's command surface: the cobra tree, the global flags,
// and the fang-rendered help and errors. The actual work lives in the clone,
// browser, sanitize, asset, and urlx packages; this layer only parses flags and
// prints progress.
package cli
import (
"context"
"fmt"
"github.com/charmbracelet/fang"
"github.com/spf13/cobra"
)
// Execute builds the root command and runs it through fang. main passes the
// signal-aware context so Ctrl-C cancels the in-flight clone and flushes resume
// state. It returns the process exit code.
func Execute(ctx context.Context) int {
root := newRoot()
opts := []fang.Option{
fang.WithVersion(Version),
}
if err := fang.Execute(ctx, root, opts...); err != nil {
return 1
}
return 0
}
// newRoot assembles the command tree.
func newRoot() *cobra.Command {
root := &cobra.Command{
Use: "kage",
Short: "Clone any website for offline viewing, with the JavaScript stripped out",
Long: "kage (影, \"shadow\") renders each page in headless Chrome, snapshots the\n" +
"final DOM, removes every script and event handler, and localises the CSS,\n" +
"images, and fonts so the saved copy looks like the live site but runs no\n" +
"code. The result is a plain folder you can open straight from disk.",
Version: fmt.Sprintf("%s (commit %s, built %s)", Version, Commit, Date),
SilenceUsage: true,
SilenceErrors: true,
}
root.AddCommand(newCloneCmd())
root.AddCommand(newServeCmd())
return root
}