Merge pull request #40 from tamnd/release-0.3.4

Cut the v0.3.4 release notes
2026-06-17 16:11:35 +07:00 · 2026-06-17 16:09:05 +07:00 · 2026-06-17 16:03:05 +07:00 · 2026-06-17 15:51:19 +07:00 · 2026-06-17 08:27:21 +05:30 · 2026-06-16 23:26:52 +08:00
12 changed files with 556 additions and 11 deletions
@@ -6,6 +6,50 @@ All notable changes to kage are recorded here. The format follows

 ## [Unreleased]

+## [0.3.4] - 2026-06-17
+
+### Fixed
+
+- `kage serve` now stops on Ctrl-C instead of ignoring it.
+  The preview server was started with a blocking call that never watched for an interrupt, so the only way to stop it was to kill the process.
+  kage now shuts the server down gracefully on an interrupt or a `SIGTERM`, with a short timeout before it forces the listener closed, so a preview exits cleanly. Thanks to Xirui Wang (#35) and Kaidi Zhao (#38).
+- A page whose JavaScript builds a deeply nested object graph no longer fails to clone.
+  Chrome's DevTools Protocol returns "Object reference chain is too long" while loading such a page, but the HTML has already loaded and the error is only about Chrome's internal object tracking, not the document.
+  kage now recognises that specific error and finishes rendering the page instead of dropping it (reported in #36). Thanks to Gautam Kumar (#39).
+
+## [0.3.3] - 2026-06-16
+
+### Fixed
+
+- Chrome no longer downloads a file to your Downloads folder when a crawl follows a link that turns out to be a binary (reported in #32).
+  An extensionless link is queued as a page, so the page worker navigated to it in Chrome, and a link that served a zip or a CSV made Chrome save the file to `~/Downloads`, a surprise side effect of a clone.
+  kage now denies Chrome-initiated downloads browser-wide, since every asset is fetched through kage's own downloader, and detects a navigation whose response is not HTML and reroutes that URL to the asset downloader, where the size and media policy decides whether to localise it or leave it on the live web.
+
+## [0.3.2] - 2026-06-16
+
+### Fixed
+
+- Saved pages now declare their character encoding, so text no longer mojibakes in a reader.
+  kage writes every page as UTF-8, but a source that set its charset only in the HTTP `Content-Type` header, with no `<meta charset>` in the markup, lost that signal once the page became a standalone file.
+  A reader serving the bytes without a charset then fell back to its locale encoding and turned every curly quote, dash, and non-breaking space into mojibake (reported in #16 and #29).
+  kage now inserts a `<meta charset="utf-8">` at the top of `<head>` when the page does not already declare one, so the page is self-describing in any reader.
+
+## [0.3.1] - 2026-06-15
+
+### Fixed
+
+- A served mirror whose entry point is a nested page no longer loses its CSS and
+  images when opened at the root. kage saves each page's asset links as
+  mirror-relative paths (`../_kage/...`) computed for that page's own location,
+  but the viewer answered `/` with the main page's bytes in place, so the browser
+  resolved those relative URLs against `/` and missed every one. A
+  `developer.apple.com/documentation` mirror, whose main page is
+  `developer.apple.com/documentation/index.html`, landed at `/` completely
+  unstyled. kage now redirects `/` to the main page's canonical content path, the
+  way the archive's `W/mainPage` redirect already does, so the browser resolves
+  the page's relative assets correctly. Kiwix was unaffected because it follows
+  that redirect itself.
+
 ## [0.3.0] - 2026-06-15

 ### Added
@@ -180,7 +224,11 @@ can browse offline, with every script stripped out.
  a multi-arch container image on GHCR (Chromium bundled), checksums, SBOMs, and
  a cosign signature, all cut from one version tag by GoReleaser.

-[Unreleased]: https://github.com/tamnd/kage/compare/v0.3.0...HEAD
+[Unreleased]: https://github.com/tamnd/kage/compare/v0.3.4...HEAD
+[0.3.4]: https://github.com/tamnd/kage/compare/v0.3.3...v0.3.4
+[0.3.3]: https://github.com/tamnd/kage/compare/v0.3.2...v0.3.3
+[0.3.2]: https://github.com/tamnd/kage/compare/v0.3.1...v0.3.2
+[0.3.1]: https://github.com/tamnd/kage/compare/v0.3.0...v0.3.1
 [0.3.0]: https://github.com/tamnd/kage/compare/v0.2.1...v0.3.0
 [0.2.1]: https://github.com/tamnd/kage/compare/v0.2.0...v0.2.1
 [0.2.0]: https://github.com/tamnd/kage/compare/v0.1.2...v0.2.0
@@ -67,6 +67,21 @@ type RenderResult struct {
 	Title    string
 }

+// ErrNotHTML reports that a URL kage tried to render as a page is not HTML: the
+// server returned some other content type (a zip, a CSV, a PDF, a bare image).
+// Such a URL reaches the page worker when its link carried no file extension to
+// classify it by. The caller reroutes it to the asset downloader, where the
+// asset policy decides whether to localise or leave it remote, instead of saving
+// an empty or broken page or letting Chrome download it (issue #32).
+type ErrNotHTML struct {
+	URL         string
+	ContentType string
+}
+
+func (e *ErrNotHTML) Error() string {
+	return fmt.Sprintf("not HTML (%s): %s", e.ContentType, e.URL)
+}
+
 // Render navigates to rawURL, lets it settle, and returns the final rendered
 // HTML. It acquires a page slot from the pool and releases it when done.
 func (p *Pool) Render(ctx context.Context, rawURL string) (RenderResult, error) {
@@ -90,11 +105,33 @@ func (p *Pool) Render(ctx context.Context, rawURL string) (RenderResult, error)

 	page = page.Context(ctx).Timeout(p.opts.RenderTimeout)

-	if err := page.Navigate(rawURL); err != nil {
-		return RenderResult{}, fmt.Errorf("navigate %s: %w", rawURL, err)
+	// Watch the main document's response so a navigation that turns out to be a
+	// non-HTML resource (a zip, a CSV, a bare image) is caught and handed back for
+	// the asset downloader, rather than rendered as a broken page or, with downloads
+	// denied, left as an aborted navigation (issue #32). The content type arrives in
+	// the response headers whether Chrome renders the body or aborts it as a denied
+	// download, so this catches both.
+	mainContentType := watchMainDocument(page)
+
+	navErr := page.Navigate(rawURL)
+	// A denied download aborts the navigation, so inspect the captured content type
+	// before treating a navigation error as a failure. waitFor gives the response
+	// event a brief moment to be processed; for an HTML page it returns at once.
+	if ct := waitFor(ctx, mainContentType, 2*time.Second); ct != "" && !isHTML(ct) {
+		return RenderResult{}, &ErrNotHTML{URL: rawURL, ContentType: ct}
+	}
+	if navErr != nil {
+		return RenderResult{}, fmt.Errorf("navigate %s: %w", rawURL, navErr)
 	}
 	if err := page.WaitLoad(); err != nil {
-		return RenderResult{}, fmt.Errorf("wait load %s: %w", rawURL, err)
+		// Chrome's DevTools Protocol may return "Object reference chain is too
+		// long" when a page's JavaScript builds deeply nested object graphs.
+		// The page has still loaded its HTML — the error is only about Chrome's
+		// internal object tracking, not about the document. Log the warning and
+		// continue rendering rather than failing the entire page (issue #36).
+		if !isObjRefChainError(err) {
+			return RenderResult{}, fmt.Errorf("wait load %s: %w", rawURL, err)
+		}
 	}
 	settle(page, p.opts.Settle)
 	if p.opts.Scroll {
@@ -169,6 +206,21 @@ func (p *Pool) getBrowser() (*rod.Browser, error) {
 	if err := b.Connect(); err != nil {
 		return nil, fmt.Errorf("connect Chrome: %w", err)
 	}
+
+	// kage never wants Chrome to write a file to disk. Every asset is fetched
+	// through kage's own downloader, which applies the size and media policy, so a
+	// Chrome-initiated download is only ever an accident: navigating an <a> link
+	// that turns out to be a binary (a zip, an installer, a CSV) makes Chrome save
+	// it to the user's Downloads folder, a surprise side effect of a crawl
+	// (issue #32). Denying downloads browser-wide stops that. The navigation is
+	// aborted instead, and Render's non-HTML detection reroutes the URL through the
+	// asset downloader, where the asset policy decides its fate. This is
+	// best-effort: if the call is unsupported, the non-HTML detection still keeps
+	// the binary out of the saved mirror.
+	_ = proto.BrowserSetDownloadBehavior{
+		Behavior: proto.BrowserSetDownloadBehaviorBehaviorDeny,
+	}.Call(b)
+
 	p.browser = b
 	return b, nil
 }
@@ -321,6 +373,83 @@ func envBool(name string) (val, ok bool) {
 	}
 }

+// watchMainDocument subscribes to network responses and returns an accessor for
+// the main document's content type. The first Document-type response is the main
+// frame's navigation; later Document responses are sub-frames (iframes), whose
+// type kage does not police, so only the first is kept. The accessor is safe to
+// call from another goroutine. Any setup error leaves the accessor returning "",
+// which the caller reads as "unknown, render normally".
+func watchMainDocument(page *rod.Page) func() string {
+	var (
+		mu sync.Mutex
+		ct string
+	)
+	if err := (proto.NetworkEnable{}).Call(page); err != nil {
+		return func() string { return "" }
+	}
+	wait := page.EachEvent(func(e *proto.NetworkResponseReceived) {
+		if e.Type != proto.NetworkResourceTypeDocument || e.Response == nil {
+			return
+		}
+		mu.Lock()
+		if ct == "" {
+			ct = e.Response.MIMEType
+		}
+		mu.Unlock()
+	})
+	// EachEvent's wait blocks until the page context ends, draining events as they
+	// arrive; run it for the page's lifetime. The deferred page.Close in Render
+	// cancels the context and unblocks it.
+	go wait()
+	return func() string {
+		mu.Lock()
+		defer mu.Unlock()
+		return ct
+	}
+}
+
+// waitFor polls get until it returns a non-empty value, the deadline passes, or
+// the context is cancelled, then returns whatever it last saw. It exists because
+// the network response is processed on another goroutine, so the value may not be
+// set the instant Navigate returns; an HTML page sets it within a few
+// milliseconds, while a never-arriving response simply waits out the deadline.
+func waitFor(ctx context.Context, get func() string, deadline time.Duration) string {
+	const step = 20 * time.Millisecond
+	for waited := time.Duration(0); waited < deadline; waited += step {
+		if v := get(); v != "" {
+			return v
+		}
+		select {
+		case <-ctx.Done():
+			return get()
+		case <-time.After(step):
+		}
+	}
+	return get()
+}
+
+// isHTML reports whether a document content type is one kage renders and saves as
+// a page. HTML and XHTML qualify; an empty type is treated as HTML so an unlabelled
+// response still renders. Anything else (a zip, a CSV, a PDF, a bare image or
+// JSON) is an asset that reached the page worker because its link carried no file
+// extension to classify it by.
+func isHTML(contentType string) bool {
+	mt := strings.ToLower(strings.TrimSpace(contentType))
+	if i := strings.IndexByte(mt, ';'); i >= 0 {
+		mt = strings.TrimSpace(mt[:i])
+	}
+	return mt == "" || mt == "text/html" || mt == "application/xhtml+xml"
+}
+
+// isObjRefChainError reports whether err is the Chrome DevTools Protocol error
+// "Object reference chain is too long" (code -32000). This surfaces when a
+// page's JavaScript builds deeply nested object graphs. The page has still
+// loaded — Chrome's internal state tracking hit a limit, not the document
+// itself (issue #36).
+func isObjRefChainError(err error) bool {
+	return err != nil && strings.Contains(err.Error(), "Object reference chain is too long")
+}
+
 // settle waits for the network to go quiet for d, recovering from any rod
 // panic and capping the wait so a chatty page can never hang the worker.
 func settle(page *rod.Page, d time.Duration) {
@@ -2,6 +2,7 @@ package browser

 import (
 	"context"
+	"errors"
 	"net/http"
 	"net/http/httptest"
 	"os"
@@ -115,3 +116,86 @@ func TestRenderCapturesFinalDOM(t *testing.T) {
 		t.Errorf("render did not capture the JS-built DOM:\n%s", res.HTML)
 	}
 }
+
+func TestIsHTML(t *testing.T) {
+	cases := []struct {
+		ct   string
+		want bool
+	}{
+		{"text/html", true},
+		{"text/html; charset=utf-8", true},
+		{"TEXT/HTML", true},
+		{"  text/html ", true},
+		{"application/xhtml+xml", true},
+		{"", true}, // unknown: render rather than misclassify
+		{"application/zip", false},
+		{"text/csv", false},
+		{"application/pdf", false},
+		{"image/png", false},
+		{"application/json", false},
+		{"application/octet-stream", false},
+	}
+	for _, c := range cases {
+		if got := isHTML(c.ct); got != c.want {
+			t.Errorf("isHTML(%q) = %v, want %v", c.ct, got, c.want)
+		}
+	}
+}
+
+func TestRenderRoutesNonHTML(t *testing.T) {
+	if testing.Short() {
+		t.Skip("render test drives Chrome; skipped under -short")
+	}
+	if _, ok := LookChrome(); !ok {
+		t.Skip("no Chrome/Chromium found; skipping render test")
+	}
+
+	srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
+		switch r.URL.Path {
+		case "/page":
+			w.Header().Set("Content-Type", "text/html; charset=utf-8")
+			_, _ = w.Write([]byte(`<!doctype html><html><body><p>a real page</p></body></html>`))
+		case "/file.zip", "/download":
+			// A binary served with no useful extension on the path, the shape that
+			// makes Chrome download to ~/Downloads when navigated to (issue #32).
+			w.Header().Set("Content-Type", "application/zip")
+			_, _ = w.Write([]byte("PK\x03\x04 not really a zip"))
+		case "/data":
+			w.Header().Set("Content-Type", "text/csv")
+			_, _ = w.Write([]byte("a,b\n1,2\n"))
+		default:
+			http.NotFound(w, r)
+		}
+	}))
+	defer srv.Close()
+
+	p := New(Options{Headless: true, Workers: 1, Settle: 300 * time.Millisecond, RenderTimeout: 20 * time.Second})
+	defer func() { _ = p.Close() }()
+
+	ctx, cancel := context.WithTimeout(context.Background(), 60*time.Second)
+	defer cancel()
+
+	// A real HTML page renders as before.
+	if res, err := p.Render(ctx, srv.URL+"/page"); err != nil {
+		t.Errorf("render HTML page: %v", err)
+	} else if !strings.Contains(res.HTML, "a real page") {
+		t.Errorf("HTML page did not render:\n%s", res.HTML)
+	}
+
+	// Non-HTML navigation targets come back as *ErrNotHTML so the caller can route
+	// them to the asset downloader instead of saving a broken page or downloading.
+	for _, tc := range []struct{ path, wantCT string }{
+		{"/download", "application/zip"},
+		{"/data", "text/csv"},
+	} {
+		_, err := p.Render(ctx, srv.URL+tc.path)
+		var notHTML *ErrNotHTML
+		if !errors.As(err, &notHTML) {
+			t.Errorf("Render(%s) error = %v, want *ErrNotHTML", tc.path, err)
+			continue
+		}
+		if !strings.Contains(notHTML.ContentType, tc.wantCT) {
+			t.Errorf("Render(%s) content type = %q, want %q", tc.path, notHTML.ContentType, tc.wantCT)
+		}
+	}
+}
@@ -1,11 +1,13 @@
 package cli

 import (
+	"context"
 	"fmt"
 	"net"
 	"net/http"
 	"os"
 	"path/filepath"
+	"time"

 	"github.com/spf13/cobra"
 )
@@ -24,14 +26,14 @@ func newServeCmd() *cobra.Command {
 			if len(args) == 1 {
 				dir = args[0]
 			}
-			return runServe(dir, addr)
+			return runServe(cmd.Context(), dir, addr)
 		},
 	}
 	cmd.Flags().StringVarP(&addr, "addr", "a", "127.0.0.1:8800", "address to listen on")
 	return cmd
 }

-func runServe(dir, addr string) error {
+func runServe(ctx context.Context, dir, addr string) error {
 	info, err := os.Stat(dir)
 	if err != nil {
 		return fmt.Errorf("cannot serve %q: %w", dir, err)
@@ -50,5 +52,24 @@ func runServe(dir, addr string) error {
 	fmt.Fprintln(os.Stderr, styleTitle.Render("kage serve")+" "+styleDim.Render(abs))
 	fmt.Fprintln(os.Stderr, "  open "+styleAccent.Render("http://"+ln.Addr().String()))
 	fmt.Fprintln(os.Stderr, styleDim.Render("  press Ctrl-C to stop"))
-	return srv.Serve(ln)
+
+	srvErr := make(chan error, 1)
+	go func() { srvErr <- srv.Serve(ln) }()
+
+	select {
+	case err := <-srvErr:
+		if err != nil && err != http.ErrServerClosed {
+			return err
+		}
+	case <-ctx.Done():
+		shutdownCtx, cancel := context.WithTimeout(context.Background(), 3*time.Second)
+		defer cancel()
+		if err := srv.Shutdown(shutdownCtx); err != nil {
+			_ = srv.Close()
+		}
+		if err := <-srvErr; err != nil && err != http.ErrServerClosed {
+			return err
+		}
+	}
+	return nil
 }
@@ -255,6 +255,22 @@ func (c *Cloner) processPage(ctx context.Context, j pageItem) {

 	res, err := c.pool.Render(ctx, j.u.String())
 	if err != nil {
+		var notHTML *browser.ErrNotHTML
+		if errors.As(err, &notHTML) {
+			// The URL is not a page but a file (a zip, a CSV, a bare image) that
+			// reached the page worker through an extensionless link. Hand it to the
+			// asset downloader, where the size and media policy decides whether to
+			// localise it or leave it remote, rather than saving a broken page or
+			// letting Chrome download it to the user's Downloads folder (issue #32).
+			c.front.markVisited(key)
+			if c.wantAsset(j.u) {
+				c.enqueueAsset(ctx, j.u, "")
+				c.logf("not a page, fetching as asset (%s): %s", notHTML.ContentType, j.u.String())
+			} else {
+				c.logf("not a page, left on the live web (%s): %s", notHTML.ContentType, j.u.String())
+			}
+			return
+		}
 		c.failPage(j.u.String(), fmt.Errorf("render: %w", err))
 		return
 	}
@@ -261,6 +261,79 @@ func TestCloneRefreshReRenders(t *testing.T) {
 	}
 }

+// TestCloneRoutesNonHTMLToAsset guards issue #32: an extensionless link that
+// turns out to be a file (a zip) is classified as a page up front, but once the
+// page worker sees it is not HTML it must be handed to the asset downloader, not
+// saved as a broken page nor downloaded by Chrome to ~/Downloads.
+func TestCloneRoutesNonHTMLToAsset(t *testing.T) {
+	if testing.Short() {
+		t.Skip("clone test drives Chrome; skipped under -short")
+	}
+	if _, ok := browser.LookChrome(); !ok {
+		t.Skip("no Chrome/Chromium found; skipping clone test")
+	}
+
+	mux := http.NewServeMux()
+	mux.HandleFunc("/", func(w http.ResponseWriter, r *http.Request) {
+		if r.URL.Path != "/" {
+			http.NotFound(w, r)
+			return
+		}
+		w.Header().Set("Content-Type", "text/html; charset=utf-8")
+		// The link has no extension, so it is queued as a page; the server then
+		// answers it with a zip.
+		_, _ = w.Write([]byte(`<!doctype html><html><body>
+<h1>Home</h1><a href="/download">grab the bundle</a></body></html>`))
+	})
+	mux.HandleFunc("/download", func(w http.ResponseWriter, r *http.Request) {
+		w.Header().Set("Content-Type", "application/zip")
+		_, _ = w.Write([]byte("PK\x03\x04 pretend bundle"))
+	})
+	srv := httptest.NewServer(mux)
+	defer srv.Close()
+
+	seed, err := urlx.ParseSeed(srv.URL)
+	if err != nil {
+		t.Fatalf("parse seed: %v", err)
+	}
+
+	out := t.TempDir()
+	cfg := DefaultConfig()
+	cfg.OutDir = out
+	cfg.Settle = 300 * time.Millisecond
+	cfg.RenderTimeout = 20 * time.Second
+	cfg.Timeout = 10 * time.Second
+
+	ctx, cancel := context.WithTimeout(context.Background(), 90*time.Second)
+	defer cancel()
+
+	res, err := New(seed, cfg, t.Logf).Run(ctx)
+	if err != nil {
+		t.Fatalf("run: %v", err)
+	}
+
+	root := res.OutDir
+	// The home page is a real page and is written.
+	if !fileExists(filepath.Join(root, "index.html")) {
+		t.Error("home page was not written")
+	}
+	// The zip is NOT saved as a page: no download/index.html exists.
+	if fileExists(filepath.Join(root, "download", "index.html")) {
+		t.Error("non-HTML target was saved as a page")
+	}
+	// The zip is fetched as an asset under the reserved tree instead.
+	if res.Assets < 1 {
+		t.Errorf("expected the zip to be fetched as an asset, assets=%d", res.Assets)
+	}
+	assetDir := filepath.Join(root, cfg.Reserved)
+	if !anyFileUnder(t, assetDir, "download") {
+		t.Error("the zip was not downloaded into the reserved asset tree")
+	}
+	if res.PageErrors != 0 {
+		t.Errorf("a rerouted non-HTML target must not count as a page error, got %d", res.PageErrors)
+	}
+}
+
 func readFile(t *testing.T, path string) string {
 	t.Helper()
 	b, err := os.ReadFile(path)
@@ -7,6 +7,7 @@ import (
 	"context"
 	"os"
 	"os/signal"
+	"syscall"

 	"github.com/tamnd/kage/cli"
 	"github.com/tamnd/kage/viewer"
@@ -18,7 +19,13 @@ func main() {
 	// thread; in the default build this is a harmless no-op.
 	viewer.LockMainThread()

-	ctx, stop := signal.NotifyContext(context.Background(), os.Interrupt)
+	ctx, stop := signal.NotifyContext(context.Background(), os.Interrupt, syscall.SIGTERM)
 	defer stop()
+
+	go func() {
+		<-ctx.Done()
+		stop()
+	}()
+
 	os.Exit(cli.Execute(ctx))
 }
@@ -6,6 +6,31 @@ weight: 40

 The authoritative, commit-level history lives in [`CHANGELOG.md`](https://github.com/tamnd/kage/blob/main/CHANGELOG.md) and on the [releases page](https://github.com/tamnd/kage/releases). This page summarises each version.

+## v0.3.4
+
+Two community fixes: a clean stop for `kage serve`, and pages with heavy JavaScript that used to be dropped.
+
+- **`kage serve` stops on Ctrl-C.** The preview server was started with a blocking call that never watched for an interrupt, so stopping it meant killing the process. kage now shuts the server down gracefully on an interrupt or a `SIGTERM`, with a short timeout before forcing the listener closed. Thanks to Xirui Wang ([#35](https://github.com/tamnd/kage/pull/35)) and Kaidi Zhao ([#38](https://github.com/tamnd/kage/pull/38)).
+- **Pages with deeply nested JavaScript still clone.** Chrome's DevTools Protocol returns "Object reference chain is too long" while loading a page whose script builds a deeply nested object graph, but the page's HTML has already loaded and the error is only about Chrome's internal object tracking. kage now recognises that error and finishes rendering instead of dropping the page ([#36](https://github.com/tamnd/kage/issues/36)). Thanks to Gautam Kumar ([#39](https://github.com/tamnd/kage/pull/39)).
+
+## v0.3.3
+
+A fix for Chrome saving a file to your Downloads folder mid-crawl.
+
+- **A crawl never writes to your Downloads folder.** A link with no file extension is queued as a page, so the page worker opened it in Chrome. When such a link served a binary, a zip or a CSV, Chrome saved the file to `~/Downloads`, a surprise side effect of running a clone ([#32](https://github.com/tamnd/kage/issues/32)). kage now denies Chrome-initiated downloads outright, since every asset is fetched through kage's own downloader and the browser never needs to write a file. As a second layer, kage detects a navigation whose response is not HTML and reroutes that URL to the asset downloader, where the size and media policy decides whether to localise it or leave it on the live web, instead of saving a broken page.
+
+## v0.3.2
+
+A fix for garbled text on pages that did not carry a charset of their own.
+
+- **Saved pages declare UTF-8.** kage writes every page as UTF-8, but a site that set its charset only in the HTTP `Content-Type` header, with no `<meta charset>` in the markup, lost that signal once the page became a standalone file. A reader serving the bytes without a charset fell back to its locale encoding and turned every curly quote, dash, and non-breaking space into mojibake. kage now inserts a `<meta charset="utf-8">` at the top of `<head>` when the page does not already declare one, so the page renders correctly in any reader.
+
+## v0.3.1
+
+A fix for broken styling when a packed mirror's home page is a nested page.
+
+- **`/` redirects to the main page instead of serving it in place.** A page's saved asset links are mirror-relative (`../_kage/...`), computed for that page's own location. The viewer was answering `/` with the main page's bytes directly, so the browser resolved those links against `/` and 404ed the page's CSS and images. A `developer.apple.com/documentation` mirror opened at `/` came up completely unstyled. kage now redirects `/` to the main page's canonical path, the way the archive's `W/mainPage` redirect does, so relative assets resolve correctly. Kiwix already followed that redirect, so it was never affected.
+
 ## v0.3.0

 Leaner mirrors, and a way to publish one as a dataset. A clone now keeps the assets that make a site readable offline and leaves the bulk downloads on the live web, and a packed archive converts to a columnar table that drops straight into dataset tooling.
@@ -326,6 +326,24 @@ func TestHandler(t *testing.T) {
 	if code, body := get("/"); code != 200 || !bytes.Contains([]byte(body), []byte("Example Home")) {
 		t.Errorf("GET / = %d %.30q", code, body)
 	}
+
+	// "/" must redirect to the main page's canonical content path, not serve its
+	// bytes in place, so the page's mirror-relative asset URLs resolve against the
+	// right base instead of 404ing.
+	noFollow := &http.Client{CheckRedirect: func(*http.Request, []*http.Request) error {
+		return http.ErrUseLastResponse
+	}}
+	resp, err := noFollow.Get(srv.URL + "/")
+	if err != nil {
+		t.Fatal(err)
+	}
+	_ = resp.Body.Close()
+	if resp.StatusCode != http.StatusFound {
+		t.Errorf("GET / status = %d, want 302", resp.StatusCode)
+	}
+	if loc := resp.Header.Get("Location"); loc != "/index.html" {
+		t.Errorf("GET / Location = %q, want %q", loc, "/index.html")
+	}
 	if code, _ := get("/about/index.html"); code != 200 {
 		t.Errorf("GET /about/index.html = %d", code)
 	}
@@ -8,14 +8,24 @@ import (
 	"github.com/tamnd/kage/zim"
 )

-// Handler serves a ZIM archive over HTTP. "/" maps to the archive's main page;
-// "/a/b.png" maps to the C/a/b.png content entry. Because the saved HTML's links
-// are mirror-relative paths, and those are exactly the C urls, a click in a
+// Handler serves a ZIM archive over HTTP. "/" redirects to the archive's main
+// page; "/a/b.png" maps to the C/a/b.png content entry. Because the saved HTML's
+// links are mirror-relative paths, and those are exactly the C urls, a click in a
 // served page hits the right entry with no rewriting. A miss is a plain 404.
 func Handler(r *zim.Reader) http.Handler {
 	return http.HandlerFunc(func(w http.ResponseWriter, req *http.Request) {
 		p := strings.TrimPrefix(req.URL.Path, "/")
 		if p == "" {
+			// The main page's saved HTML carries mirror-relative asset URLs
+			// (../_kage/...) computed for its own nested location, so serving its
+			// bytes at "/" would resolve them against the wrong base and 404 the
+			// page's CSS and images. Redirect to the page's canonical content path
+			// instead, the way the archive's W/mainPage redirect does, so the
+			// browser resolves those relative URLs correctly.
+			if ns, url, ok := r.MainPageRef(); ok && ns == zim.NamespaceContent {
+				http.Redirect(w, req, "/"+url, http.StatusFound)
+				return
+			}
 			blob, err := r.MainPage()
 			if err != nil {
 				http.NotFound(w, req)
@@ -41,6 +41,7 @@ type Report struct {
 	MetaRefreshRemoved  int
 	DeadLinksRemoved    int
 	CondCommentsRemoved int
+	CharsetAdded        bool
 }

 // jsURLAttrs are attributes whose value may be a javascript: URL.
@@ -70,6 +71,7 @@ func Strip(doc []byte, opts Options) ([]byte, Report, error) {
 func CleanTree(root *html.Node, opts Options) Report {
 	var rep Report
 	clean(root, opts, &rep)
+	rep.CharsetAdded = ensureCharset(root)
 	if opts.Banner != "" {
 		insertBanner(root, opts.Banner)
 	}
@@ -227,6 +229,68 @@ func unwrapNoscript(parent, ns *html.Node) {
 	parent.RemoveChild(ns)
 }

+// ensureCharset guarantees the document declares UTF-8, inserting a
+// <meta charset="utf-8"> at the top of <head> when none is present, and reports
+// whether it added one. kage renders every saved page as UTF-8, but a source
+// that set its charset only in the HTTP Content-Type header, with no <meta>
+// charset in the markup, loses that signal once the page is a standalone file.
+// A reader then serving the bytes without a charset falls back to its locale
+// encoding and mojibakes every multibyte character (curly quotes, dashes, a
+// non-breaking space). Declaring the charset in the markup makes the page
+// self-describing in any reader, kage's own viewer and Kiwix alike.
+func ensureCharset(root *html.Node) bool {
+	head := findElement(root, atom.Head)
+	if head == nil {
+		return false
+	}
+	if hasCharsetMeta(head) {
+		return false
+	}
+	meta := &html.Node{
+		Type:     html.ElementNode,
+		Data:     "meta",
+		DataAtom: atom.Meta,
+		Attr:     []html.Attribute{{Key: "charset", Val: "utf-8"}},
+	}
+	// The declaration must precede any content for a reader to honour it, so it
+	// goes first in <head>.
+	head.InsertBefore(meta, head.FirstChild)
+	return true
+}
+
+// hasCharsetMeta reports whether head already declares a character encoding,
+// either as <meta charset="..."> or the older <meta http-equiv="Content-Type"
+// content="...; charset=...">.
+func hasCharsetMeta(head *html.Node) bool {
+	for c := head.FirstChild; c != nil; c = c.NextSibling {
+		if c.Type != html.ElementNode || c.DataAtom != atom.Meta {
+			continue
+		}
+		if attr(c, "charset") != "" {
+			return true
+		}
+		if strings.EqualFold(attr(c, "http-equiv"), "content-type") &&
+			strings.Contains(strings.ToLower(attr(c, "content")), "charset=") {
+			return true
+		}
+	}
+	return false
+}
+
+// findElement returns the first element node of the given atom in document
+// order, or nil if none exists.
+func findElement(n *html.Node, a atom.Atom) *html.Node {
+	if n.Type == html.ElementNode && n.DataAtom == a {
+		return n
+	}
+	for c := n.FirstChild; c != nil; c = c.NextSibling {
+		if found := findElement(c, a); found != nil {
+			return found
+		}
+	}
+	return nil
+}
+
 // insertBanner prepends an HTML comment to the document.
 func insertBanner(root *html.Node, text string) {
 	c := &html.Node{Type: html.CommentNode, Data: " " + text + " "}
@@ -166,3 +166,53 @@ func TestKeepMetaRefreshPlain(t *testing.T) {
 		t.Error("JS-target meta refresh must be removed regardless")
 	}
 }
+
+func TestCharsetAddedWhenMissing(t *testing.T) {
+	// A page whose source declared its charset only in the HTTP header has no
+	// <meta charset>. The saved file must gain one so a reader does not fall back
+	// to its locale encoding and mojibake the UTF-8 text.
+	in := `<html><head><title>Quotes</title></head><body><p>` +
+		"“curly” — café</p></body></html>"
+	out, rep, err := Strip([]byte(in), Options{})
+	if err != nil {
+		t.Fatal(err)
+	}
+	if !rep.CharsetAdded {
+		t.Error("CharsetAdded = false, want true")
+	}
+	s := string(out)
+	if !strings.Contains(strings.ToLower(s), `<meta charset="utf-8"/>`) {
+		t.Errorf("expected an injected meta charset:\n%s", s)
+	}
+	// It must sit at the very start of <head>, before any content.
+	headIdx := strings.Index(s, "<head>")
+	metaIdx := strings.Index(strings.ToLower(s), "<meta charset")
+	titleIdx := strings.Index(s, "<title>")
+	if headIdx >= metaIdx || metaIdx >= titleIdx {
+		t.Errorf("meta charset must come first in head (head=%d meta=%d title=%d)", headIdx, metaIdx, titleIdx)
+	}
+	// The original bytes are preserved as UTF-8.
+	if !strings.Contains(s, "café") {
+		t.Error("UTF-8 content should be preserved")
+	}
+}
+
+func TestCharsetNotDuplicated(t *testing.T) {
+	// A page that already declares a charset, in either form, is left alone.
+	cases := []string{
+		`<html><head><meta charset="utf-8"><title>x</title></head><body></body></html>`,
+		`<html><head><meta http-equiv="Content-Type" content="text/html; charset=ISO-8859-1"><title>x</title></head><body></body></html>`,
+	}
+	for _, in := range cases {
+		out, rep, err := Strip([]byte(in), Options{})
+		if err != nil {
+			t.Fatal(err)
+		}
+		if rep.CharsetAdded {
+			t.Errorf("CharsetAdded = true for a page that already declares one:\n%s", in)
+		}
+		if n := strings.Count(strings.ToLower(string(out)), "charset"); n != 1 {
+			t.Errorf("charset count = %d, want 1:\n%s", n, out)
+		}
+	}
+}
Author	SHA1	Message	Date
Tam Nguyen Duc	8bb1e344d2	Merge pull request #40 from tamnd/release-0.3.4 Cut the v0.3.4 release notes	2026-06-17 16:11:35 +07:00
Duc-Tam Nguyen	122a80210c	Cut the v0.3.4 release notes	2026-06-17 16:09:05 +07:00
Tam Nguyen Duc	0a8ab2e238	Merge pull request #38 from zkd-11/fix-serve-signal fix: serve Ctrl-C stop	2026-06-17 16:03:05 +07:00
Tam Nguyen Duc	994e94f3fb	Merge pull request #39 from GautamKumarOffical/fix/handle-object-reference-chain-error fix: continue rendering when Chrome hits 'Object reference chain is too long'	2026-06-17 15:51:19 +07:00
Gautam Kumar	eb683ddd2c	fix: continue rendering when Chrome hits 'Object reference chain is too long' When a page's JavaScript builds deeply nested object graphs, Chrome's DevTools Protocol returns error -32000 'Object reference chain is too long' during WaitLoad. The page has still loaded its HTML — the error is about Chrome's internal object tracking, not the document itself. This change detects this specific error and proceeds with rendering instead of failing the entire page, so sites with complex JS still get cloned successfully (issue #36). Signed-off-by: Gautam Kumar <gautamkumarofficial@users.noreply.github.com>	2026-06-17 08:27:21 +05:30
Kaden	c85745b230	fix: serve Ctrl-C stop	2026-06-16 23:26:52 +08:00
Tam Nguyen Duc	602f4dfa40	Merge pull request #35 from Xirui/main	2026-06-16 11:54:36 +07:00
Xirui	e80d38a8bc	fix: kage serve Ctrl-C handling	2026-06-16 15:38:47 +12:00
Tam Nguyen Duc	8178b767fd	Merge pull request #34 from tamnd/release-0.3.3 Cut the v0.3.3 release notes	2026-06-16 10:16:52 +07:00
Duc-Tam Nguyen	a68d2b12e0	Cut the v0.3.3 release notes	2026-06-16 10:14:19 +07:00
Tam Nguyen Duc	5cbb7f83ea	Merge pull request #33 from tamnd/fix-download-deny Stop Chrome downloading files a crawl links to	2026-06-16 10:13:34 +07:00
Duc-Tam Nguyen	bf850c6b93	Stop Chrome downloading files a crawl links to An extensionless link is queued as a page, so the page worker navigated to it in headless Chrome. When such a link served a binary, a zip or a CSV, Chrome saved the file to the user's Downloads folder, a surprise side effect of a clone (issue #32). Deny Chrome-initiated downloads browser-wide, since kage fetches every asset through its own downloader and never needs the browser to write a file. Then watch the main document's response, and when it is not HTML, return a typed ErrNotHTML so the page worker reroutes the URL to the asset downloader, where the existing size and media policy decides whether to localise it or leave it on the live web. Verified against the two URLs from the issue, a zip and a CSV: both land under the mirror's reserved tree and nothing is written to Downloads.	2026-06-16 09:41:43 +07:00
Tam Nguyen Duc	83822c1629	Merge pull request #31 from tamnd/release-0.3.2 Cut the v0.3.2 release notes	2026-06-16 00:21:03 +07:00
Duc-Tam Nguyen	3ec5c8fdaf	Cut the v0.3.2 release notes Date the Unreleased charset fix as 0.3.2, update the compare links, and add a v0.3.2 summary to the docs release-notes page.	2026-06-16 00:18:35 +07:00
Tam Nguyen Duc	7709bccfcd	Merge pull request #30 from tamnd/fix-charset-utf8 Add a meta charset to saved pages so text does not mojibake	2026-06-16 00:17:52 +07:00
Duc-Tam Nguyen	16c9fcbcfe	Add a meta charset to saved pages so text does not mojibake A page that declared its charset only in the HTTP Content-Type header, with no meta charset in the markup, lost that signal once kage saved it as a file. A reader serving the bytes without a charset then fell back to its locale encoding and garbled every curly quote, dash, and nbsp. kage writes UTF-8, so it now inserts a meta charset utf-8 at the top of head when the page does not already declare one. Verified on the blog post from the report: the saved HTML now carries the meta and renders the quotes correctly.	2026-06-16 00:15:27 +07:00
Tam Nguyen Duc	2d8a64ca6f	Merge pull request #28 from tamnd/release-0.3.1 Cut the v0.3.1 release notes	2026-06-15 23:14:41 +07:00
Duc-Tam Nguyen	a96233cf73	Cut the v0.3.1 release notes Move the Unreleased fix into a dated 0.3.1 section, update the compare links, and add a v0.3.1 summary to the docs release-notes page. This patch redirects / to the main page so a mirror whose entry point is a nested page keeps its CSS and images when served.	2026-06-15 23:12:08 +07:00
Tam Nguyen Duc	2064ef918a	Merge pull request #27 from tamnd/fix-mainpage-redirect Redirect / to the main page instead of serving it in place	2026-06-15 22:49:08 +07:00
Duc-Tam Nguyen	9eabd93098	Redirect / to the main page instead of serving it in place The served main page broke its own CSS and images when the entry point was a nested page. kage saves each page's asset links as mirror-relative paths (../_kage/...) computed for that page's own location, but the handler served the main page's bytes directly at /, so the browser resolved those relative URLs against / and 404ed every one of them. A developer.apple.com/documentation mirror landed at / with no styles. Redirect / to the main page's canonical content path, the way the archive's W/mainPage redirect already does, so the browser navigates to the page's real URL and resolves its relative assets correctly. Kiwix was unaffected because it follows that redirect itself.	2026-06-15 22:46:42 +07:00
Tam Nguyen Duc	2ccf7bc5db	Merge pull request #26 from tamnd/release-0.3.0 Cut the v0.3.0 release notes	2026-06-15 21:32:23 +07:00