An extensionless link is queued as a page, so the page worker navigated to
it in headless Chrome. When such a link served a binary, a zip or a CSV,
Chrome saved the file to the user's Downloads folder, a surprise side effect
of a clone (issue #32).
Deny Chrome-initiated downloads browser-wide, since kage fetches every asset
through its own downloader and never needs the browser to write a file. Then
watch the main document's response, and when it is not HTML, return a typed
ErrNotHTML so the page worker reroutes the URL to the asset downloader, where
the existing size and media policy decides whether to localise it or leave it
on the live web.
Verified against the two URLs from the issue, a zip and a CSV: both land
under the mirror's reserved tree and nothing is written to Downloads.
kage launched Chrome with --no-sandbox unconditionally, which turns off the
browser's main security boundary for every run, including ordinary desktop
use where the sandbox works fine. Since kage renders pages from the open web,
a renderer exploit could then reach the host. Reported in #10.
Keep the sandbox on by default and drop it only where it genuinely cannot
initialize: inside a container, or when running as root (Chrome refuses to
start a sandbox as root). Containers are detected from IN_DOCKER or the
/.dockerenv marker, and there kage also sets --disable-dev-shm-usage because
the default 64 MB /dev/shm is too small for the renderer on large pages.
Whenever the sandbox is dropped kage says so on stderr, so it is never silent.
Thanks to Dimitrios Prasakis for the report and to the commenter on Hacker
News who suggested the IN_DOCKER opt-in.
kage renders every page in headless Chrome, snapshots the final
DOM, strips all JavaScript, and localises CSS, images, and fonts
so a site can be browsed offline as a plain folder of files.
The engine is split into small packages:
urlx deterministic URL to local-path mapping and scope rules
sanitize remove scripts, on* handlers, and javascript: URLs
asset rewrite HTML and CSS references, download assets
browser headless Chrome pool over the DevTools protocol
robots robots.txt matcher
clone the orchestrator: a polite resumable breadth-first crawl
The cli package wires a cobra and fang command surface with two
commands, clone and serve. Every pure package has table tests; the
browser and clone packages add Chrome-driven end-to-end tests that
skip when no browser is present or under -short.
CI runs gofmt, vet, build, race tests, golangci-lint, govulncheck,
and a tidy check on Linux and macOS. A goreleaser config fans one
tag out to archives, deb/rpm/apk, a Chromium-bundled GHCR image,
and the package managers. A tago docs site builds to Pages and
Cloudflare.
Duc-Tam Nguyen