chore: import upstream snapshot with attribution
Backend Tests / Tests (other) (push) Failing after 1s
Backend Tests / Static Checks (push) Failing after 2s
Backend Tests / Tests (internal) (push) Failing after 2s
Backend Tests / Tests (server) (push) Failing after 1s
Backend Tests / Tests (store) (push) Failing after 1s
Build Canary Image / build-frontend (push) Failing after 1s
Frontend Tests / Lint (push) Failing after 1s
Build Canary Image / build-push (linux/amd64) (push) Has been skipped
Build Canary Image / build-push (linux/arm64) (push) Has been skipped
Build Canary Image / merge (push) Has been skipped
Frontend Tests / Build (push) Failing after 1s
Release Please / release-please (push) Failing after 0s
Proto Linter / Lint Protos (push) Failing after 2s

This commit is contained in:
wehub-resource-sync
2026-07-13 12:02:24 +08:00
commit 4cddfcf2f3
1040 changed files with 200957 additions and 0 deletions
+13
View File
@@ -0,0 +1,13 @@
web/node_modules
web/dist
.git
.github
build/
tmp/
memos
*.md
.gitignore
.golangci.yaml
.dockerignore
docs/
.DS_Store
+1
View File
@@ -0,0 +1 @@
github: usememos
+162
View File
@@ -0,0 +1,162 @@
name: Bug Report
description: Report reproducible behavior that is broken in the current version
type: Bug
body:
- type: markdown
attributes:
value: |
Thanks for reporting a bug. Please help maintainers reproduce it on current code.
Before submitting:
- Search [existing issues](https://github.com/usememos/memos/issues) to avoid duplicates.
- Upgrade and retest with the [latest release](https://github.com/usememos/memos/releases/latest), current `main`, or the [demo site](https://demo.usememos.com/).
- Use [Discussions](https://github.com/usememos/memos/discussions) for questions, setup help, or configuration troubleshooting.
Maintainers normally fix bugs in current code and future releases, not older released versions.
- type: checkboxes
id: pre-check
attributes:
label: Pre-submission Checklist
options:
- label: I have searched existing issues and confirmed this has not already been reported
required: true
- label: I have reproduced this bug on the latest Memos release, current `main`, or the demo site, or this report is specifically about upgrading to the latest release
required: true
- label: This is a bug report, not a question or configuration request
required: true
- type: dropdown
id: area
attributes:
label: Affected Area
description: Choose the main area where the bug appears.
multiple: true
options:
- Web UI
- API
- Authentication / SSO
- Database / migration
- Resources / attachments
- Search / filters
- Markdown / editor
- Deployment / startup
- Other
validations:
required: true
- type: dropdown
id: reproduced-on
attributes:
label: Reproduced On
description: Where did you confirm the issue still happens?
options:
- Latest release
- Current main branch
- Demo site
- Upgrade to latest release
- I cannot test the latest version
validations:
required: true
- type: input
id: version
attributes:
label: Memos Version Tested
description: Report the exact version where you reproduced the bug. Use the version shown in **Settings > System**, the container image tag, the release binary version, or `memos --version`. Please do not write `latest` only.
placeholder: "v0.29.x, ghcr.io/usememos/memos:v0.29.x, main@abcdef, or demo site"
validations:
required: true
- type: input
id: previous-version
attributes:
label: Previous Version
description: If this happened after an upgrade, enter the version you upgraded from. Leave blank if not applicable.
placeholder: "v0.28.x, unknown, or not applicable"
- type: textarea
id: environment
attributes:
label: Environment Details
description: Include the deployment, database, client, and network details that apply.
placeholder: |
- Deployment: Docker / binary / source / demo site
- Database: SQLite / PostgreSQL / MySQL
- Browser: Firefox 126 on macOS 15
- Server OS: Ubuntu 24.04
- Reverse proxy: nginx / Caddy / Traefik / none
- Related settings: SSO / SMTP / storage / webhooks / none
validations:
required: true
- type: dropdown
id: frequency
attributes:
label: Reproduction Frequency
description: How reliably can you reproduce this?
options:
- Always
- Often
- Sometimes
- Once
- Unknown
validations:
required: true
- type: textarea
id: reproduction-steps
attributes:
label: Steps to Reproduce
description: Provide minimal, numbered steps that start from a clean page/session when possible.
placeholder: |
1. Open ...
2. Click ...
3. Enter ...
4. Wait for ...
5. See ...
validations:
required: true
- type: textarea
id: current-behavior
attributes:
label: Current Behavior
description: What happened? Include exact error messages, status codes, or visible UI state if available.
placeholder: The application ...
validations:
required: true
- type: textarea
id: expected-behavior
attributes:
label: Expected Behavior
description: What did you expect to happen instead?
placeholder: I expected...
validations:
required: true
- type: textarea
id: regression
attributes:
label: Regression Information
description: If this worked before, describe when it last worked and what changed before it broke.
placeholder: |
This last worked in ...
It started after ...
- type: textarea
id: logs
attributes:
label: Logs
description: Paste relevant server logs, browser console output, or network errors. Remove secrets before submitting.
render: shell
placeholder: |
Paste logs here.
- type: textarea
id: additional-context
attributes:
label: Additional Context
description: Attach screenshots, screen recordings, sample data, exports, or links to related issues.
placeholder: Drag and drop files here or add any extra context.
+8
View File
@@ -0,0 +1,8 @@
blank_issues_enabled: false
contact_links:
- name: Questions & Support
url: https://github.com/usememos/memos/discussions
about: Ask questions or get help in GitHub Discussions — please don't open issues for questions
- name: Documentation
url: https://www.usememos.com/docs
about: Check the documentation before opening an issue
+109
View File
@@ -0,0 +1,109 @@
name: Feature Request
description: Propose a product improvement with a clear use case
type: Feature
body:
- type: markdown
attributes:
value: |
Thanks for suggesting an improvement. Please describe the problem and workflow first so maintainers can evaluate the request.
Before submitting:
- Search [existing feature requests](https://github.com/usememos/memos/issues?q=label%3Aenhancement) to avoid duplicates.
- Use [Discussions](https://github.com/usememos/memos/discussions) for broad ideas, setup help, or questions.
- Keep one feature request focused on one user problem.
- type: checkboxes
id: pre-check
attributes:
label: Pre-submission Checklist
options:
- label: I have searched existing issues and confirmed this has not already been requested
required: true
- label: This is a feature request, not a bug report or support question
required: true
- type: dropdown
id: area
attributes:
label: Area
description: Choose the main area this request affects.
multiple: true
options:
- Web UI
- Mobile / responsive experience
- API
- Authentication / SSO
- Administration
- Memo editor
- Search / filters
- Tags / organization
- Resources / attachments
- Import / export
- Deployment / operations
- Other
validations:
required: true
- type: textarea
id: problem
attributes:
label: Problem or Use Case
description: What problem does this solve? Describe the user need, not just the proposed UI or API.
placeholder: |
I often need to ...
Today, this is difficult because ...
validations:
required: true
- type: textarea
id: users
attributes:
label: Who Would Use This?
description: Describe the user, role, team, or workflow that would benefit.
placeholder: |
This helps ...
They currently ...
validations:
required: true
- type: textarea
id: solution
attributes:
label: Proposed Solution
description: Describe the behavior you would like Memos to support. Include UI, API, or configuration details if relevant.
placeholder: |
Memos should ...
validations:
required: true
- type: textarea
id: alternatives
attributes:
label: Alternatives Considered
description: Existing workarounds, related tools, or simpler alternatives you considered.
placeholder: |
I tried ...
It does not work well because ...
- type: textarea
id: compatibility
attributes:
label: Compatibility and Migration
description: Note any expected impact on existing users, APIs, storage, permissions, or deployments.
placeholder: |
This should not affect existing memos because ...
Existing API clients may need ...
- type: textarea
id: examples
attributes:
label: Examples
description: Link to screenshots, mockups, API sketches, related apps, or existing issues that show the desired behavior.
placeholder: Drag and drop files here or add links/examples.
- type: textarea
id: additional-context
attributes:
label: Additional Context
description: Add any extra context that does not fit above.
placeholder: Any other details.
+91
View File
@@ -0,0 +1,91 @@
name: Backend Tests
on:
push:
branches: [main]
pull_request:
branches: [main]
paths:
- "go.mod"
- "go.sum"
- "**.go"
concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: true
env:
GO_VERSION: "1.26.2"
jobs:
static-checks:
name: Static Checks
runs-on: ubuntu-latest
steps:
- name: Checkout code
uses: actions/checkout@v6
- name: Setup Go
uses: actions/setup-go@v6
with:
go-version: ${{ env.GO_VERSION }}
cache: true
cache-dependency-path: go.sum
- name: Verify go.mod is tidy
run: |
go mod tidy -go=${{ env.GO_VERSION }}
git diff --exit-code
- name: Run golangci-lint
uses: golangci/golangci-lint-action@v9
with:
version: v2.11.3
args: --timeout=3m
tests:
name: Tests (${{ matrix.test-group }})
runs-on: ubuntu-latest
strategy:
fail-fast: false
matrix:
test-group: [store, server, internal, other]
steps:
- name: Checkout code
uses: actions/checkout@v6
- name: Setup Go
uses: actions/setup-go@v6
with:
go-version: ${{ env.GO_VERSION }}
cache: true
cache-dependency-path: go.sum
- name: Run tests
run: |
case "${{ matrix.test-group }}" in
store)
# Run store tests for all drivers (sqlite, mysql, postgres)
go test -v -coverprofile=coverage.out -covermode=atomic ./store/...
;;
server)
go test -v -race -coverprofile=coverage.out -covermode=atomic ./server/...
;;
internal)
go test -v -race -coverprofile=coverage.out -covermode=atomic ./internal/...
;;
other)
go test -v -race -coverprofile=coverage.out -covermode=atomic \
./cmd/... ./internal/... ./proto/...
;;
esac
env:
DRIVER: ${{ matrix.test-group == 'store' && '' || 'sqlite' }}
- name: Upload coverage
if: github.event_name == 'push' && github.ref == 'refs/heads/main'
uses: codecov/codecov-action@v5
with:
files: ./coverage.out
flags: ${{ matrix.test-group }}
fail_ci_if_error: false
+180
View File
@@ -0,0 +1,180 @@
name: Build Canary Image
on:
push:
branches: [main]
concurrency:
group: ${{ github.workflow }}-${{ github.repository }}
cancel-in-progress: true
permissions:
contents: read
jobs:
build-frontend:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v6
- uses: pnpm/action-setup@v4.2.0
with:
version: 11.0.1
- uses: actions/setup-node@v6
with:
node-version: "24"
cache: pnpm
cache-dependency-path: "web/pnpm-lock.yaml"
- name: Get pnpm store directory
id: pnpm-cache
shell: bash
run: echo "STORE_PATH=$(pnpm store path)" >> $GITHUB_OUTPUT
- name: Setup pnpm cache
uses: actions/cache@v5
with:
path: ${{ steps.pnpm-cache.outputs.STORE_PATH }}
key: ${{ runner.os }}-pnpm-store-${{ hashFiles('web/pnpm-lock.yaml') }}
restore-keys: ${{ runner.os }}-pnpm-store-
- run: pnpm install --frozen-lockfile
working-directory: web
- name: Run frontend build
run: pnpm release
working-directory: web
- name: Upload frontend artifacts
uses: actions/upload-artifact@v6
with:
name: frontend-dist
path: server/router/frontend/dist
retention-days: 1
build-push:
needs: build-frontend
runs-on: ubuntu-latest
permissions:
contents: read
packages: write
strategy:
fail-fast: false
matrix:
platform:
- linux/amd64
- linux/arm64
steps:
- uses: actions/checkout@v6
with:
fetch-depth: 0
- name: Resolve version from latest tag
id: version
run: |
version="$(git describe --tags --abbrev=0 2>/dev/null || echo dev)"
echo "version=${version#v}" >> "$GITHUB_OUTPUT"
- name: Download frontend artifacts
uses: actions/download-artifact@v7
with:
name: frontend-dist
path: server/router/frontend/dist
- name: Set up QEMU
uses: docker/setup-qemu-action@v3
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3
- name: Login to Docker Hub
uses: docker/login-action@v3
with:
username: ${{ secrets.DOCKER_HUB_USERNAME }}
password: ${{ secrets.DOCKER_HUB_TOKEN }}
- name: Login to GitHub Container Registry
uses: docker/login-action@v3
with:
registry: ghcr.io
username: ${{ github.actor }}
password: ${{ github.token }}
- name: Build and push by digest
id: build
uses: docker/build-push-action@v6
with:
context: .
file: ./scripts/Dockerfile
platforms: ${{ matrix.platform }}
build-args: |
VERSION=${{ steps.version.outputs.version }}
COMMIT=${{ github.sha }}
cache-from: type=gha,scope=build-${{ matrix.platform }}
cache-to: type=gha,mode=max,scope=build-${{ matrix.platform }}
outputs: type=image,name=neosmemo/memos,push-by-digest=true,name-canonical=true,push=true
- name: Export digest
run: |
mkdir -p /tmp/digests
digest="${{ steps.build.outputs.digest }}"
touch "/tmp/digests/${digest#sha256:}"
- name: Upload digest
uses: actions/upload-artifact@v6
with:
name: digests-${{ strategy.job-index }}
path: /tmp/digests/*
if-no-files-found: error
retention-days: 1
merge:
needs: build-push
runs-on: ubuntu-latest
permissions:
contents: read
packages: write
steps:
- name: Download digests
uses: actions/download-artifact@v7
with:
pattern: digests-*
merge-multiple: true
path: /tmp/digests
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3
- name: Docker meta
id: meta
uses: docker/metadata-action@v5
with:
images: |
neosmemo/memos
ghcr.io/usememos/memos
flavor: |
latest=false
tags: |
type=raw,value=canary
- name: Login to Docker Hub
uses: docker/login-action@v3
with:
username: ${{ secrets.DOCKER_HUB_USERNAME }}
password: ${{ secrets.DOCKER_HUB_TOKEN }}
- name: Login to GitHub Container Registry
uses: docker/login-action@v3
with:
registry: ghcr.io
username: ${{ github.actor }}
password: ${{ github.token }}
- name: Create manifest list and push
working-directory: /tmp/digests
run: |
docker buildx imagetools create $(jq -cr '.tags | map("-t " + .) | join(" ")' <<< "$DOCKER_METADATA_OUTPUT_JSON") \
$(printf 'neosmemo/memos@sha256:%s ' *)
env:
DOCKER_METADATA_OUTPUT_JSON: ${{ steps.meta.outputs.json }}
- name: Inspect images
run: |
docker buildx imagetools inspect neosmemo/memos:canary
docker buildx imagetools inspect ghcr.io/usememos/memos:canary
+17
View File
@@ -0,0 +1,17 @@
name: Demo Deploy
on:
workflow_dispatch:
jobs:
deploy-demo:
runs-on: ubuntu-latest
steps:
- name: Trigger Render Deploy
run: |
curl -X POST "${{ secrets.RENDER_DEPLOY_HOOK }}" \
-H "Content-Type: application/json" \
-d '{"trigger": "github_action"}'
- name: Deployment Status
run: echo "Demo deployment triggered successfully on Render"
+76
View File
@@ -0,0 +1,76 @@
name: Frontend Tests
on:
push:
branches: [main]
pull_request:
branches: [main]
paths:
- "web/**"
concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: true
env:
NODE_VERSION: "24"
PNPM_VERSION: "11.0.1"
jobs:
lint:
name: Lint
runs-on: ubuntu-latest
steps:
- name: Checkout code
uses: actions/checkout@v6
- name: Setup pnpm
uses: pnpm/action-setup@v4.2.0
with:
version: ${{ env.PNPM_VERSION }}
- name: Setup Node.js
uses: actions/setup-node@v6
with:
node-version: ${{ env.NODE_VERSION }}
cache: pnpm
cache-dependency-path: web/pnpm-lock.yaml
- name: Install dependencies
working-directory: web
run: pnpm install --frozen-lockfile
- name: Run lint
working-directory: web
run: pnpm lint
- name: Run unit tests
working-directory: web
run: pnpm test
build:
name: Build
runs-on: ubuntu-latest
steps:
- name: Checkout code
uses: actions/checkout@v6
- name: Setup pnpm
uses: pnpm/action-setup@v4.2.0
with:
version: ${{ env.PNPM_VERSION }}
- name: Setup Node.js
uses: actions/setup-node@v6
with:
node-version: ${{ env.NODE_VERSION }}
cache: pnpm
cache-dependency-path: web/pnpm-lock.yaml
- name: Install dependencies
working-directory: web
run: pnpm install --frozen-lockfile
- name: Build frontend
working-directory: web
run: pnpm build
+40
View File
@@ -0,0 +1,40 @@
name: Proto Linter
on:
push:
branches: [main]
pull_request:
branches: [main]
paths:
- "proto/**"
concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: true
jobs:
lint:
name: Lint Protos
runs-on: ubuntu-latest
steps:
- name: Checkout code
uses: actions/checkout@v6
with:
fetch-depth: 0
- name: Setup buf
uses: bufbuild/buf-setup-action@v1
with:
github_token: ${{ github.token }}
- name: Run buf lint
uses: bufbuild/buf-lint-action@v1
with:
input: proto
- name: Check buf format
run: |
if [[ $(buf format -d) ]]; then
echo "❌ Proto files are not formatted. Run 'buf format -w' to fix."
exit 1
fi
+37
View File
@@ -0,0 +1,37 @@
name: Release Please
on:
push:
branches:
- main
workflow_dispatch:
permissions:
contents: write
issues: write
pull-requests: write
concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: true
jobs:
release-please:
runs-on: ubuntu-latest
steps:
- name: Check release token
env:
RELEASE_PLEASE_TOKEN: ${{ secrets.RELEASE_PLEASE_TOKEN }}
run: |
if [ -z "$RELEASE_PLEASE_TOKEN" ]; then
echo "RELEASE_PLEASE_TOKEN must be set to a fine-grained PAT so release-please tags can trigger release.yml." >&2
exit 1
fi
- name: Run release-please
uses: googleapis/release-please-action@v4
with:
# Use a fine-grained PAT so generated tags trigger release.yml.
token: ${{ secrets.RELEASE_PLEASE_TOKEN }}
config-file: release-please-config.json
manifest-file: .release-please-manifest.json
+384
View File
@@ -0,0 +1,384 @@
name: Release
on:
push:
tags:
- "v*.*.*"
workflow_dispatch:
concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: true
permissions:
contents: read
env:
GO_VERSION: "1.26.2"
NODE_VERSION: "24"
PNPM_VERSION: "11.0.1"
ARTIFACT_RETENTION_DAYS: 60
ARTIFACT_PREFIX: memos
jobs:
prepare:
name: Extract Version
runs-on: ubuntu-latest
outputs:
version: ${{ steps.version.outputs.version }}
tag: ${{ steps.version.outputs.tag }}
major_minor: ${{ steps.version.outputs.major_minor }}
is_prerelease: ${{ steps.version.outputs.is_prerelease }}
steps:
- name: Extract version
id: version
env:
REF_NAME: ${{ github.ref_name }}
EVENT_NAME: ${{ github.event_name }}
run: |
if [ "$EVENT_NAME" = "workflow_dispatch" ]; then
echo "tag=" >> "$GITHUB_OUTPUT"
echo "version=manual-${GITHUB_SHA::7}" >> "$GITHUB_OUTPUT"
echo "major_minor=" >> "$GITHUB_OUTPUT"
echo "is_prerelease=false" >> "$GITHUB_OUTPUT"
exit 0
fi
if [[ ! "$REF_NAME" =~ ^v([0-9]+\.[0-9]+\.[0-9]+)(-rc\.[0-9]+)?$ ]]; then
echo "Unsupported release tag format: $REF_NAME" >&2
exit 1
fi
version="${BASH_REMATCH[1]}${BASH_REMATCH[2]}"
major_minor="${BASH_REMATCH[1]%.*}"
is_prerelease=false
if [ -n "${BASH_REMATCH[2]}" ]; then
is_prerelease=true
fi
echo "tag=${REF_NAME}" >> "$GITHUB_OUTPUT"
echo "version=${version}" >> "$GITHUB_OUTPUT"
echo "major_minor=${major_minor}" >> "$GITHUB_OUTPUT"
echo "is_prerelease=${is_prerelease}" >> "$GITHUB_OUTPUT"
build-frontend:
name: Build Frontend
needs: prepare
runs-on: ubuntu-latest
steps:
- name: Checkout code
uses: actions/checkout@v6
- name: Setup pnpm
uses: pnpm/action-setup@v4.2.0
with:
version: ${{ env.PNPM_VERSION }}
- name: Setup Node.js
uses: actions/setup-node@v6
with:
node-version: ${{ env.NODE_VERSION }}
cache: pnpm
cache-dependency-path: web/pnpm-lock.yaml
- name: Get pnpm store directory
id: pnpm-cache
shell: bash
run: echo "STORE_PATH=$(pnpm store path)" >> "$GITHUB_OUTPUT"
- name: Setup pnpm cache
uses: actions/cache@v4
with:
path: ${{ steps.pnpm-cache.outputs.STORE_PATH }}
key: ${{ runner.os }}-pnpm-store-${{ hashFiles('web/pnpm-lock.yaml') }}
restore-keys: ${{ runner.os }}-pnpm-store-
- name: Install dependencies
working-directory: web
run: pnpm install --frozen-lockfile
- name: Build frontend release assets
working-directory: web
run: pnpm release
- name: Upload frontend artifacts
uses: actions/upload-artifact@v4
with:
name: frontend-dist
path: server/router/frontend/dist
retention-days: 1
build-binaries:
name: Build ${{ matrix.goos }}-${{ matrix.goarch }}${{ matrix.goarm && format('v{0}', matrix.goarm) || '' }}
needs: [prepare, build-frontend]
runs-on: ubuntu-latest
strategy:
fail-fast: false
matrix:
include:
- goos: linux
goarch: amd64
- goos: linux
goarch: arm64
- goos: linux
goarch: arm
goarm: "7"
- goos: darwin
goarch: amd64
- goos: darwin
goarch: arm64
- goos: windows
goarch: amd64
steps:
- name: Checkout code
uses: actions/checkout@v6
- name: Setup Go
uses: actions/setup-go@v6
with:
go-version: ${{ env.GO_VERSION }}
cache: true
- name: Download frontend artifacts
uses: actions/download-artifact@v4
with:
name: frontend-dist
path: server/router/frontend/dist
- name: Build binary
env:
GOOS: ${{ matrix.goos }}
GOARCH: ${{ matrix.goarch }}
GOARM: ${{ matrix.goarm }}
CGO_ENABLED: "0"
run: |
output_name="memos"
if [ "$GOOS" = "windows" ]; then
output_name="memos.exe"
fi
mkdir -p build
go build \
-trimpath \
-ldflags="-s -w -X github.com/usememos/memos/internal/version.Version=${{ needs.prepare.outputs.version }} -X github.com/usememos/memos/internal/version.Commit=${{ github.sha }} -extldflags '-static'" \
-tags netgo,osusergo \
-o "build/${output_name}" \
./cmd/memos
- name: Package binary
env:
VERSION: ${{ needs.prepare.outputs.version }}
GOOS: ${{ matrix.goos }}
GOARCH: ${{ matrix.goarch }}
GOARM: ${{ matrix.goarm }}
run: |
cd build
package_name="${ARTIFACT_PREFIX}_${VERSION}_${GOOS}_${GOARCH}"
if [ -n "$GOARM" ]; then
package_name="${package_name}v${GOARM}"
fi
if [ "$GOOS" = "windows" ]; then
artifact_name="${package_name}.zip"
zip -q "${artifact_name}" memos.exe
else
artifact_name="${package_name}.tar.gz"
tar czf "${artifact_name}" memos
fi
echo "artifact_name=${artifact_name}" >> "$GITHUB_ENV"
- name: Upload binary artifact
uses: actions/upload-artifact@v4
with:
name: ${{ env.artifact_name }}
path: build/${{ env.artifact_name }}
retention-days: ${{ env.ARTIFACT_RETENTION_DAYS }}
checksums:
name: Generate Checksums
needs: [prepare, build-binaries]
runs-on: ubuntu-latest
steps:
- name: Download binary artifacts
uses: actions/download-artifact@v4
with:
path: artifacts
pattern: ${{ env.ARTIFACT_PREFIX }}_*
merge-multiple: true
- name: Generate checksums
working-directory: artifacts
run: sha256sum * > checksums.txt
- name: Upload checksum artifact
uses: actions/upload-artifact@v4
with:
name: checksums
path: artifacts/checksums.txt
retention-days: ${{ env.ARTIFACT_RETENTION_DAYS }}
release:
name: Publish GitHub Release
needs: [prepare, build-binaries, checksums]
if: github.event_name != 'workflow_dispatch'
runs-on: ubuntu-latest
permissions:
contents: write
steps:
- name: Download binary artifacts
uses: actions/download-artifact@v4
with:
path: artifacts
pattern: ${{ env.ARTIFACT_PREFIX }}_*
merge-multiple: true
- name: Download checksum artifact
uses: actions/download-artifact@v4
with:
name: checksums
path: artifacts
- name: Publish release assets
uses: softprops/action-gh-release@v2
with:
tag_name: ${{ needs.prepare.outputs.tag }}
name: ${{ needs.prepare.outputs.tag }}
prerelease: ${{ needs.prepare.outputs.is_prerelease == 'true' }}
files: artifacts/*
build-push:
name: Build Image ${{ matrix.platform }}
needs: [prepare, build-frontend]
if: github.event_name != 'workflow_dispatch'
runs-on: ubuntu-latest
permissions:
contents: read
packages: write
strategy:
fail-fast: false
matrix:
platform:
- linux/amd64
- linux/arm/v7
- linux/arm64
steps:
- name: Checkout code
uses: actions/checkout@v6
- name: Download frontend artifacts
uses: actions/download-artifact@v4
with:
name: frontend-dist
path: server/router/frontend/dist
- name: Set up QEMU
uses: docker/setup-qemu-action@v3
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3
- name: Login to Docker Hub
uses: docker/login-action@v3
with:
username: ${{ secrets.DOCKER_HUB_USERNAME }}
password: ${{ secrets.DOCKER_HUB_TOKEN }}
- name: Login to GitHub Container Registry
uses: docker/login-action@v3
with:
registry: ghcr.io
username: ${{ github.actor }}
password: ${{ github.token }}
- name: Build and push by digest
id: build
uses: docker/build-push-action@v6
with:
context: .
file: ./scripts/Dockerfile
platforms: ${{ matrix.platform }}
build-args: |
VERSION=${{ needs.prepare.outputs.version }}
COMMIT=${{ github.sha }}
cache-from: type=gha,scope=release-${{ matrix.platform }}
cache-to: type=gha,mode=max,scope=release-${{ matrix.platform }}
outputs: type=image,name=neosmemo/memos,push-by-digest=true,name-canonical=true,push=true
- name: Export digest
run: |
mkdir -p /tmp/digests
digest="${{ steps.build.outputs.digest }}"
touch "/tmp/digests/${digest#sha256:}"
- name: Upload digest
uses: actions/upload-artifact@v4
with:
name: digests-${{ strategy.job-index }}
path: /tmp/digests/*
if-no-files-found: error
retention-days: 1
merge-images:
name: Publish Release Image Tags
needs: [prepare, build-push]
if: github.event_name != 'workflow_dispatch'
runs-on: ubuntu-latest
permissions:
contents: read
packages: write
steps:
- name: Download digests
uses: actions/download-artifact@v4
with:
pattern: digests-*
merge-multiple: true
path: /tmp/digests
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3
- name: Login to Docker Hub
uses: docker/login-action@v3
with:
username: ${{ secrets.DOCKER_HUB_USERNAME }}
password: ${{ secrets.DOCKER_HUB_TOKEN }}
- name: Login to GitHub Container Registry
uses: docker/login-action@v3
with:
registry: ghcr.io
username: ${{ github.actor }}
password: ${{ github.token }}
- name: Create manifest list and push
working-directory: /tmp/digests
run: |
version="${{ needs.prepare.outputs.version }}"
if [ "${{ needs.prepare.outputs.is_prerelease }}" = "true" ]; then
docker buildx imagetools create \
-t "neosmemo/memos:${version}" \
-t "ghcr.io/usememos/memos:${version}" \
$(printf 'neosmemo/memos@sha256:%s ' *)
exit 0
fi
docker buildx imagetools create \
-t "neosmemo/memos:${version}" \
-t "neosmemo/memos:${{ needs.prepare.outputs.major_minor }}" \
-t "neosmemo/memos:stable" \
-t "ghcr.io/usememos/memos:${version}" \
-t "ghcr.io/usememos/memos:${{ needs.prepare.outputs.major_minor }}" \
-t "ghcr.io/usememos/memos:stable" \
$(printf 'neosmemo/memos@sha256:%s ' *)
- name: Inspect images
run: |
docker buildx imagetools inspect neosmemo/memos:${{ needs.prepare.outputs.version }}
if [ "${{ needs.prepare.outputs.is_prerelease }}" = "true" ]; then
exit 0
fi
docker buildx imagetools inspect neosmemo/memos:stable
+24
View File
@@ -0,0 +1,24 @@
name: Close Stale
on:
schedule:
- cron: "0 */8 * * *" # Every 8 hours
jobs:
close-stale:
name: Close Stale Issues and PRs
runs-on: ubuntu-latest
permissions:
issues: write
pull-requests: write
steps:
- name: Mark and close stale issues and PRs
uses: actions/stale@v10.1.1
with:
# Issues: mark stale after 14 days of inactivity, close after 3 more days
days-before-issue-stale: 14
days-before-issue-close: 3
# Pull requests: mark stale after 14 days of inactivity, close after 3 more days
days-before-pr-stale: 14
days-before-pr-close: 3
+33
View File
@@ -0,0 +1,33 @@
# temp folder
tmp
# Frontend asset
web/dist
# Build artifacts
build/
bin/
memos
.DS_Store
# Jetbrains
.idea
# Docker Compose Environment File
.env
dist
# VSCode settings
.vscode
# Git worktrees
.worktrees/
# Local pnpm store (project-scoped, created when --config.store-dir is set
# without an existing store; contains a symlink back to the workspace).
.pnpm-store/
# Frontend test coverage output (Vitest + @vitest/coverage-v8).
web/coverage/
+101
View File
@@ -0,0 +1,101 @@
version: "2"
linters:
enable:
- revive
- govet
- staticcheck
- misspell
- gocritic
- sqlclosecheck
- rowserrcheck
- nilerr
- godot
- forbidigo
- mirror
- bodyclose
disable:
- errcheck
settings:
exhaustive:
explicit-exhaustive-switch: false
staticcheck:
checks:
- all
- -ST1000
- -ST1003
- -ST1021
- -QF1003
revive:
# Default to run all linters so that new rules in the future could automatically be added to the static check.
enable-all-rules: true
rules:
# The following rules are too strict and make coding harder. We do not enable them for now.
- name: file-header
disabled: true
- name: line-length-limit
disabled: true
- name: function-length
disabled: true
- name: max-public-structs
disabled: true
- name: function-result-limit
disabled: true
- name: banned-characters
disabled: true
- name: argument-limit
disabled: true
- name: cognitive-complexity
disabled: true
- name: cyclomatic
disabled: true
- name: confusing-results
disabled: true
- name: add-constant
disabled: true
- name: flag-parameter
disabled: true
- name: nested-structs
disabled: true
- name: import-shadowing
disabled: true
- name: early-return
disabled: true
- name: use-any
disabled: true
- name: exported
disabled: true
- name: unhandled-error
disabled: true
- name: if-return
disabled: true
- name: max-control-nesting
disabled: true
- name: redefines-builtin-id
disabled: true
- name: package-comments
disabled: true
gocritic:
disabled-checks:
- ifElseChain
govet:
settings:
printf: # The name of the analyzer, run `go tool vet help` to see the list of all analyzers
funcs: # Run `go tool vet help printf` to see the full configuration of `printf`.
- common.Errorf
enable-all: true
disable:
- fieldalignment
- shadow
forbidigo:
forbid:
- pattern: 'fmt\.Errorf(# Please use errors\.Wrap\|Wrapf\|Errorf instead)?'
- pattern: 'ioutil\.ReadDir(# Please use os\.ReadDir)?'
formatters:
enable:
- goimports
settings:
goimports:
local-prefixes:
- github.com/usememos/memos
+3
View File
@@ -0,0 +1,3 @@
{
".": "0.29.1"
}
+125
View File
@@ -0,0 +1,125 @@
# AGENTS.md
Repository instructions for AI coding agents. Keep this file short, concrete, and tied to commands that actually work in this
repo. If a fact here conflicts with source files or CI config, trust the source file and update this guide.
## Project Snapshot
Memos is a self-hosted note-taking app.
- Backend: Go 1.26.2, Echo v5, Connect RPC, gRPC-Gateway, Protocol Buffers.
- Frontend: React 19, TypeScript 6, Vite 8, Tailwind CSS v4, React Query v5.
- Storage: SQLite, MySQL, PostgreSQL.
- Generated API outputs: `proto/gen/` for Go/OpenAPI, `web/src/types/proto/` for TypeScript.
## Working Rules
- Read relevant code before editing; prefer local patterns over new abstractions.
- Keep diffs scoped. Do not do repo-wide cleanup, dependency churn, or generated-file rewrites unless the task requires it.
- Do not hand-edit generated proto outputs. Change `.proto` files, then run `buf generate`.
- Add migrations for all database drivers when schema changes, and update each driver's `LATEST.sql`.
- Add public API endpoints to `server/router/api/v1/acl_config.go`.
- Ask before adding heavy dependencies, changing auth/token behavior, or altering Docker/release workflows.
## Commands
Run from the repository root unless a command starts with `cd`.
```bash
# Backend
go run ./cmd/memos --port 8081 # Start backend dev server
go test ./... # Run all Go tests
go test -v ./store/... # Store tests, including DB drivers via TestContainers
go test -v -race ./server/... # Server tests with race detector
go test -v -race ./internal/... # Internal package tests with race detector
go test -v -run TestFoo ./pkg/... # Run matching Go tests
go mod tidy -go=1.26.2 # Match CI tidy check
golangci-lint run # Go lint, config: .golangci.yaml
golangci-lint run --fix # Auto-fix lint, including goimports
# Frontend
cd web && pnpm install # Install dependencies
cd web && pnpm dev # Dev server on :3001, proxying API to :8081
cd web && pnpm lint # Type check + Biome lint
cd web && pnpm test # Vitest unit tests
cd web && pnpm build # Production build
cd web && pnpm release # Build SPA into server/router/frontend/dist
# Protocol Buffers
cd proto && buf generate # Regenerate Go + TypeScript + OpenAPI
cd proto && buf lint # Lint proto files
cd proto && buf format -w # Format proto files
```
## Code Map
| Path | Purpose |
| --- | --- |
| `cmd/memos/main.go` | Cobra/Viper CLI setup and server startup |
| `server/server.go` | Echo HTTP server and background runner wiring |
| `server/auth/` | JWT access tokens, refresh tokens, PAT handling |
| `server/router/api/v1/` | Connect/gRPC-Gateway services, ACL config, SSE hub |
| `server/router/frontend/` | Static SPA serving |
| `server/router/fileserver/` | Native HTTP file serving, thumbnails, range requests |
| `server/runner/` | Background memo processing and S3 presign refresh |
| `store/` | Store facade, cache, migrations, driver interface |
| `store/db/{sqlite,mysql,postgres}/` | Database-specific drivers and SQL |
| `proto/api/v1/` | Public API service definitions |
| `proto/store/` | Internal storage proto messages |
| `internal/` | App-private packages: scheduler, cron, email, CEL filter, markdown, idp, S3 |
| `web/src/connect.ts` | Connect RPC clients, auth interceptor, access-token refresh |
| `web/src/auth-state.ts` | Token storage and BroadcastChannel cross-tab sync |
| `web/src/hooks/` | React Query hooks for server state |
| `web/src/contexts/` | React context for client/UI state |
| `web/src/components/` | Radix/Tailwind UI components and feature components |
| `web/src/themes/` | CSS themes using OKLch color tokens |
## Change Routing
| Change | Update | Verify |
| --- | --- | --- |
| Go service or router behavior | Service code under `server/`, tests near package | `go test -v -race ./server/...` |
| Store or migration behavior | `store/`, all three DB driver migrations, `LATEST.sql` | `go test -v ./store/...` |
| Internal package logic | Relevant `internal/` package tests | `go test -v -race ./internal/...` |
| Frontend behavior | Components/hooks/contexts under `web/src/` | `cd web && pnpm lint && pnpm test` |
| Frontend production output | Vite config or release-sensitive UI | `cd web && pnpm build` or `pnpm release` |
| Proto API | `.proto` source plus generated outputs | `cd proto && buf generate && buf lint` |
| Public unauthenticated route | `server/router/api/v1/acl_config.go` | Targeted server test or manual route check |
## Go Conventions
- Wrap errors with `errors.Wrap(err, "context")` from `github.com/pkg/errors`; do not use `fmt.Errorf`.
- Return service errors with `status.Errorf(codes.X, "message")`.
- Keep imports grouped as stdlib, third-party, then `github.com/usememos/memos`; goimports is run by golangci-lint.
- Add doc comments for exported identifiers; godot enforces exported comment punctuation.
- Avoid package-level mutable state unless the surrounding package already uses that pattern.
## Frontend Conventions
- Use `@/` for absolute imports.
- Follow Biome formatting: 2-space indent, double quotes, semicolons, 140-character line width.
- Put server data in React Query hooks under `web/src/hooks/`; keep UI-only state in contexts or component state.
- Use Tailwind CSS v4 utilities, `cn()` for class merging, and CVA for variants.
- Reuse Radix primitives and existing components before adding new UI primitives.
- Keep generated proto TypeScript under `web/src/types/proto/` out of manual edits and Biome rewrites.
## Database And Proto Rules
- Schema changes require SQLite, MySQL, and PostgreSQL migrations plus `LATEST.sql` updates.
- Fresh-install SQL and incremental migrations must stay equivalent.
- Proto field changes must preserve compatibility unless the task explicitly allows a breaking API change.
- Regenerate after proto edits and include both Go/OpenAPI and TypeScript generated outputs.
## Verification Policy
- Run the narrowest relevant checks while iterating.
- Before finishing, run the checks that match the changed surface from "Change Routing".
- For docs-only changes, `git diff --check` is sufficient unless the docs include runnable examples that should be tested.
- If a required check cannot run locally, report the reason and the exact command that remains.
## CI Reference
- Backend CI: Go 1.26.2, `go mod tidy -go=1.26.2`, golangci-lint v2.11.3, test groups `store`, `server`, `internal`, `other`.
- Frontend CI: Node 24, pnpm 11.0.1, `pnpm lint`, `pnpm test`, `pnpm build`.
- Proto CI: `buf lint` and `buf format` check.
- Docker: `scripts/Dockerfile`, Alpine 3.21 runtime, non-root user, port 5230, multi-arch amd64/arm64/arm/v7.
+183
View File
@@ -0,0 +1,183 @@
# Changelog
## [0.29.1](https://github.com/usememos/memos/compare/v0.29.0...v0.29.1) (2026-06-04)
### Bug Fixes
* **markdown:** keep task item content in one grid column ([7c3bff4](https://github.com/usememos/memos/commit/7c3bff4e98223fb99a5d49f093e3cd57dd140ba4))
* support &lt;meta name=description&gt; in link previews ([#6000](https://github.com/usememos/memos/issues/6000)) ([e8d32e8](https://github.com/usememos/memos/commit/e8d32e87d1d6e4927250ad5794ba5965e0260153))
* **web:** render video attachment posters on mobile ([0e2a9a9](https://github.com/usememos/memos/commit/0e2a9a9c0ce0e2da63535210c553b3ae2f79b239))
## [0.29.0](https://github.com/usememos/memos/compare/v0.28.0...v0.29.0) (2026-05-27)
### Features
* **about:** add about page with bird sprites ([411ba7b](https://github.com/usememos/memos/commit/411ba7b34c0c23ad217a7897835097887a28a036))
* **activity-calendar:** aggregate by ViewContext.timeBasis ([8daef1d](https://github.com/usememos/memos/commit/8daef1dc89544512fff78fd64d8216b2babc2e42))
* add &lt;Placeholder&gt; component with ASCII bird states ([#5949](https://github.com/usememos/memos/issues/5949)) ([8c16ffa](https://github.com/usememos/memos/commit/8c16ffa1f1410f9413cb57ebe251b4b604f9aeff))
* add configurable `--log-level` flag ([#5934](https://github.com/usememos/memos/issues/5934)) ([f1e2a06](https://github.com/usememos/memos/commit/f1e2a06b46c7f45ba6562c308d2703fdfb4067b0))
* add dedicated shortcuts page ([#5942](https://github.com/usememos/memos/issues/5942)) ([1df6479](https://github.com/usememos/memos/commit/1df6479443c334732b862d60a0f76e61f00ebb89))
* add link metadata endpoints ([9c5c604](https://github.com/usememos/memos/commit/9c5c604944428d3f682f35c9d05a87c1d4c3152f))
* **frontend:** add pixel bird tilemaps ([cf55f11](https://github.com/usememos/memos/commit/cf55f1107273c928e7c2d6bc26df5f3efea26458))
* **memo:** add task list quick actions ([#5983](https://github.com/usememos/memos/issues/5983)) ([648b3bd](https://github.com/usememos/memos/commit/648b3bd812e5bf2b1abb9f8b3529efefe4734e7c))
* **memo:** create memos on the selected calendar date ([#5925](https://github.com/usememos/memos/issues/5925)) ([ef55013](https://github.com/usememos/memos/commit/ef55013418d68e2d6a24a6d02a0365c1ffff03d7))
* **notification:** add smtp email settings ([cd4f28a](https://github.com/usememos/memos/commit/cd4f28ae1058a125e245ea0a6eae16aea71b8f9d))
* **placeholder:** add woodpecker tilemap ([638e4f3](https://github.com/usememos/memos/commit/638e4f398e90c556f70af150a79538312c8fc760))
* render link metadata cards ([0bc5669](https://github.com/usememos/memos/commit/0bc56694b0ca347ab1eb083f62997a22007b763d))
* **stats:** admin instance resource statistics ([ea0625d](https://github.com/usememos/memos/commit/ea0625da45a419e08158ab4102051f3fe5e59b87))
* **stats:** support filtered all-user stats ([88ac3ec](https://github.com/usememos/memos/commit/88ac3ec31ee3e808db82663c04a31cf730d1221e))
* **transcription:** explicit STT settings with provider, model, prompt ([#5926](https://github.com/usememos/memos/issues/5926)) ([238f27d](https://github.com/usememos/memos/commit/238f27dea149492a78a8994470b0aac55fece78d))
### Bug Fixes
* avoid update event on memo create attachments ([#5961](https://github.com/usememos/memos/issues/5961)) ([3c3382a](https://github.com/usememos/memos/commit/3c3382a3c652bed3b58058931ff3cbf1d875b9a8))
* delete user cleanup ([#5981](https://github.com/usememos/memos/issues/5981)) ([e53b7d9](https://github.com/usememos/memos/commit/e53b7d96e70965529ba3b04ea720ea6966f77b60))
* **editor:** wrap selected text when pasting URL ([e0bb3a2](https://github.com/usememos/memos/commit/e0bb3a2e684e932309df2f9fb0ff774ce8e6b692))
* **fileserver:** preserve HDR image metadata in thumbnails ([c724232](https://github.com/usememos/memos/commit/c7242324a18962e453e7f2a0309a7c07b358bdbe))
* **frontend:** correct static cache headers ([084f40b](https://github.com/usememos/memos/commit/084f40bc9e5922ff2d41c08dce837b96696437aa))
* **frontend:** use correct url path for memos in sitemap.xml ([#5921](https://github.com/usememos/memos/issues/5921)) ([603781f](https://github.com/usememos/memos/commit/603781f792b2603fbda93146988a64a93c92d760))
* **httpgetter:** prevent DNS rebinding in link metadata fetch ([078488c](https://github.com/usememos/memos/commit/078488ca818626fbcab69bdfb4b93b58dca6b6eb))
* **markdown:** align list items with checkboxes ([e008b1a](https://github.com/usememos/memos/commit/e008b1a23c77945eff87707eb15578d4d6d74e4c))
* **memo:** enforce parent visibility for comments ([4a1e401](https://github.com/usememos/memos/commit/4a1e401bd99c7850ac48e9693b25c9ebeecfb042))
* **s3presign:** preserve motion media payload ([7f1f53f](https://github.com/usememos/memos/commit/7f1f53ffc417962f6930d1e6e3783eacd911003d))
* **security:** enforce attachment ownership on memo updates ([35bf761](https://github.com/usememos/memos/commit/35bf761b8c2c3f155bdc06e9b373b6076116a725))
* **sse:** stream initial response and refresh tokens ([21303e8](https://github.com/usememos/memos/commit/21303e879d2c7a857306206d2cbdae83f61983e6))
* **user:** omit internal settings from list responses ([#5917](https://github.com/usememos/memos/issues/5917)) ([1df3fe7](https://github.com/usememos/memos/commit/1df3fe79559ccf94b6c71e9ffb58e870ed43820d))
* **web:** sync avatar changes immediately after profile update ([#5903](https://github.com/usememos/memos/issues/5903)) ([328396a](https://github.com/usememos/memos/commit/328396a97f2219592032e8418c383e0ab7edaca6))
### Performance Improvements
* lazy load heavy first-screen dependencies ([#5947](https://github.com/usememos/memos/issues/5947)) ([a6024ee](https://github.com/usememos/memos/commit/a6024eebf129c4b8fb65ee0cba09c6657274bfde))
## [0.28.0](https://github.com/usememos/memos/compare/v0.27.1...v0.28.0) (2026-04-27)
### Features
* **auth:** add SSO user identity linkage ([#5883](https://github.com/usememos/memos/issues/5883)) ([d688914](https://github.com/usememos/memos/commit/d688914b2864791eeadbf21c882608632875f17c))
* **memos:** choose created or updated time for memos ([#5894](https://github.com/usememos/memos/issues/5894)) ([c268551](https://github.com/usememos/memos/commit/c268551a16929a2cbea6891951feff91926bba59))
* redesign account and SSO management ([#5886](https://github.com/usememos/memos/issues/5886)) ([ee17998](https://github.com/usememos/memos/commit/ee1799851e88674a6920c7a56d93428fcf95e662))
### Bug Fixes
* **auth:** harden authorization and username validation ([#5890](https://github.com/usememos/memos/issues/5890)) ([0fb83a7](https://github.com/usememos/memos/commit/0fb83a745dd5057ade45a3caad2c444af2239113))
* disable modal prop on DropdownMenu to prevent scroll disappearing ([#5861](https://github.com/usememos/memos/issues/5861)) ([d98f665](https://github.com/usememos/memos/commit/d98f6659190b8d1a8252e64549d9120d85e05d33))
* fix legacy username auth flows ([#5885](https://github.com/usememos/memos/issues/5885)) ([30c0611](https://github.com/usememos/memos/commit/30c0611a82f9254952a74650095105254f2940e4))
* **markdown:** split mixed task and bullet lists ([e2c6084](https://github.com/usememos/memos/commit/e2c60845eaff9a78b8d8eb3ccc9a067ef5690976))
* reduce list memo query overhead ([#5880](https://github.com/usememos/memos/issues/5880)) ([5063804](https://github.com/usememos/memos/commit/50638040f618b02b0c6d010e1d41554c75067517))
* **web:** preserve task checkbox state ([#5867](https://github.com/usememos/memos/issues/5867)) ([b5863d7](https://github.com/usememos/memos/commit/b5863d76be3cfbf3e0f8237d8e762122b5a0a679))
## [0.27.1](https://github.com/usememos/memos/compare/v0.27.0...v0.27.1) (2026-04-19)
### Bug Fixes
* mixed-case user resource names ([#5853](https://github.com/usememos/memos/issues/5853)) ([01be01f](https://github.com/usememos/memos/commit/01be01f4b7676af41bdd1758b1e9b096aa922546))
* **release:** inject build version into artifacts ([f8a304b](https://github.com/usememos/memos/commit/f8a304bae33086320b39095d631288156eec4249))
* user resource names can be uuidv4 from idp sub claim ([#5856](https://github.com/usememos/memos/issues/5856)) ([bbded58](https://github.com/usememos/memos/commit/bbded584ce85a856d863485768e08b53adec7244))
## [0.27.0](https://github.com/usememos/memos/compare/v0.26.2...v0.27.0) (2026-04-18)
### Features
* add --allow-private-webhooks flag to bypass SSRF protection ([#5694](https://github.com/usememos/memos/issues/5694)) ([cd5816c](https://github.com/usememos/memos/commit/cd5816c428931f56561f7d48b649a33858812539))
* add blur_content attribute to tag metadata settings ([#5767](https://github.com/usememos/memos/issues/5767)) ([45b2153](https://github.com/usememos/memos/commit/45b21530d9ebbc730e77bcbcf407f4c295b52e93))
* add Gemini transcription provider ([#5830](https://github.com/usememos/memos/issues/5830)) ([d87539a](https://github.com/usememos/memos/commit/d87539a1e1261590ab355220e33fa0ae00bd9431))
* add GitHub release installer and release workflow ([1ed542c](https://github.com/usememos/memos/commit/1ed542c21db421e9aa0cc47417a92c6bd103e257))
* add live refresh via Server-Sent Events (SSE) with visual indicator ([#5638](https://github.com/usememos/memos/issues/5638)) ([ea0892a](https://github.com/usememos/memos/commit/ea0892a8b26d8806a093e3a942562dfc22794f68))
* add MCP server with PAT authentication ([47d9414](https://github.com/usememos/memos/commit/47d9414702dc18966af385352b960bfe451511b7))
* add outline navigation to memo detail sidebar ([#5771](https://github.com/usememos/memos/issues/5771)) ([6b30579](https://github.com/usememos/memos/commit/6b3057990396f9cb6f21706c2e99dddebc35ffd5))
* **ai:** add BYOK audio transcription ([#5832](https://github.com/usememos/memos/issues/5832)) ([101704c](https://github.com/usememos/memos/commit/101704c8eac17c7f34508d8db6c53bc972061cdb))
* **ai:** add instance AI providers and transcription ([#5829](https://github.com/usememos/memos/issues/5829)) ([83ed32f](https://github.com/usememos/memos/commit/83ed32f1195841d2d6c057c3e00086a3147879a2))
* **attachments:** add Live Photo and Motion Photo support ([#5810](https://github.com/usememos/memos/issues/5810)) ([4b4e719](https://github.com/usememos/memos/commit/4b4e719470184e49cd62084b1aa53c9a777a9fec))
* **cli:** add version subcommand ([#5731](https://github.com/usememos/memos/issues/5731)) ([0ba4c0f](https://github.com/usememos/memos/commit/0ba4c0f397c532a5a314f35ddeb3ce3633e84d42))
* **editor:** add voice note recording to the memo composer ([#5801](https://github.com/usememos/memos/issues/5801)) ([c0d5854](https://github.com/usememos/memos/commit/c0d5854f678f357cf4054d448e0311f2ee90f8ac))
* extract title from first H1 heading into memo property ([#5726](https://github.com/usememos/memos/issues/5726)) ([1e688b2](https://github.com/usememos/memos/commit/1e688b2a5d78c2972f4b0078a7b033dae5d1cbf3))
* **i18n:** update sse connect label in Chinese ([#5732](https://github.com/usememos/memos/issues/5732)) ([89c6902](https://github.com/usememos/memos/commit/89c69028953fa33dbb6e0e257d764636c0fc2d09))
* **instance:** add canonical tag metadata setting ([#5736](https://github.com/usememos/memos/issues/5736)) ([65d14fb](https://github.com/usememos/memos/commit/65d14fbb63053fc1fbf1f63881c81adbc9d01671))
* **instance:** add notification transport setting ([#5737](https://github.com/usememos/memos/issues/5737)) ([a249d06](https://github.com/usememos/memos/commit/a249d06e2e0f33e0e3fea34471a891352dacc466))
* **mcp:** enhance MCP server with full capabilities and new tools ([#5720](https://github.com/usememos/memos/issues/5720)) ([b8e9ee2](https://github.com/usememos/memos/commit/b8e9ee2b26a59e1b19aab6db3ca39656fc18785a))
* **mcp:** harden tool exposure and side effects ([#5850](https://github.com/usememos/memos/issues/5850)) ([583c3d2](https://github.com/usememos/memos/commit/583c3d24f4d785faa5e034c1b88ed90eda119baa))
* **mcp:** refactor MCP server to standard protocol structure ([803d488](https://github.com/usememos/memos/commit/803d488a5f8f55477cd3ad4cc4cf0fac98901dd3))
* **memo-editor:** add compact live waveform recorder panel ([#5817](https://github.com/usememos/memos/issues/5817)) ([e51985a](https://github.com/usememos/memos/commit/e51985a29ffeeecd318b2fb793358ed45bb0eff4))
* **memo-preview:** support comment metadata in previews ([#5768](https://github.com/usememos/memos/issues/5768)) ([e176b28](https://github.com/usememos/memos/commit/e176b28c801cc3bb3718208a96a7d64c36620d0f))
* **memo:** add image sharing in detail view ([38fc22b](https://github.com/usememos/memos/commit/38fc22b7541b8a9ddcd848cf77054fcf844eb87f))
* **memo:** add share links for private memos ([#5742](https://github.com/usememos/memos/issues/5742)) ([3f3133d](https://github.com/usememos/memos/commit/3f3133d6e2f404061e147f9dd2424680dc0303a3))
* **mentions:** add memo mention parsing, notifications, and rendering ([#5811](https://github.com/usememos/memos/issues/5811)) ([24fc8ab](https://github.com/usememos/memos/commit/24fc8ab8ca68d092e7b12a5d51c48ca9420f72b5))
* replace auto-increment ID with UID for identity provider resource names ([#5687](https://github.com/usememos/memos/issues/5687)) ([92d937b](https://github.com/usememos/memos/commit/92d937b1aa87365152edc005a3b736e6bf9b9e45))
* show inline comment preview in list view ([3a5d3c8](https://github.com/usememos/memos/commit/3a5d3c8ff92ae5c24559dc4574300d6475820a87))
* **store:** change default storage type to local filesystem ([78efa68](https://github.com/usememos/memos/commit/78efa6802e2fd96c981e86c5b658fb4b5c05091f))
* treat tag setting keys as anchored regex patterns ([#5759](https://github.com/usememos/memos/issues/5759)) ([9e04049](https://github.com/usememos/memos/commit/9e04049632e63f1cf53535773f170c1aa5af7168))
* **ui:** allow navigating between images with arrows in preview dialog ([#5669](https://github.com/usememos/memos/issues/5669)) ([104d2ec](https://github.com/usememos/memos/commit/104d2ec0a6973e983c46724165fdc0916515ad16))
* **user:** add per-user tag metadata settings ([#5735](https://github.com/usememos/memos/issues/5735)) ([330291d](https://github.com/usememos/memos/commit/330291d4d9fa95649e940001898929b4f6f0367e))
* **web:** add demo mode banner ([#5836](https://github.com/usememos/memos/issues/5836)) ([35504cc](https://github.com/usememos/memos/commit/35504cc8bd6f9921291fb565633d549388a3f948))
* **webhook:** dispatch webhook on memo comment creation ([7c1defb](https://github.com/usememos/memos/commit/7c1defba01fbc91cc81e1a0841cecb6738056db7))
### Bug Fixes
* access token refresh on web app ([#5681](https://github.com/usememos/memos/issues/5681)) ([3010f10](https://github.com/usememos/memos/commit/3010f10eafb49af5aadd71a94b3ef7a6ec71f617))
* add unix socket file permission setting (755 -&gt; 660) ([#5849](https://github.com/usememos/memos/issues/5849)) ([0fc1dab](https://github.com/usememos/memos/commit/0fc1dab28b33f7fdbe9a21cd3bd3affe75dd7f19))
* **api:** appease image size lint ([ff6389a](https://github.com/usememos/memos/commit/ff6389a5ef73772fcb5c132bf1cb169fb4744c00))
* **api:** improve SSE hub design and fix double-broadcast on comments ([c53677f](https://github.com/usememos/memos/commit/c53677fcba202a8eed35c8ecdfaacea5152b8691))
* **api:** make credentials write-only and restrict sensitive settings to admins ([9d3a74b](https://github.com/usememos/memos/commit/9d3a74bcccf934aa2fa95ee240b56cc3b3a25776))
* **api:** reduce memory pressure in backend paths ([c456637](https://github.com/usememos/memos/commit/c45663761d148e94d3e1b30f4810922e7c2571ab))
* **api:** remove public activity service ([#5734](https://github.com/usememos/memos/issues/5734)) ([04f239a](https://github.com/usememos/memos/commit/04f239a2fc74f7090c779a6a7b8a08238fd31b88))
* **api:** restrict user email exposure to self and admins ([#5784](https://github.com/usememos/memos/issues/5784)) ([a24d420](https://github.com/usememos/memos/commit/a24d4209222814e4a8bedfc0392e91462bc855ad))
* **api:** switch user resource names to usernames ([#5779](https://github.com/usememos/memos/issues/5779)) ([acddef1](https://github.com/usememos/memos/commit/acddef1f3dcca44806bcbb85e9dc5cae7daa5285))
* **api:** tolerate missing related users in memo conversions ([#5809](https://github.com/usememos/memos/issues/5809)) ([25feef3](https://github.com/usememos/memos/commit/25feef3aadd34bfd474d7e3b685815a6509bc4c6))
* **auth:** recover session via refresh cookie when localStorage is empty ([#5748](https://github.com/usememos/memos/issues/5748)) ([551ee1d](https://github.com/usememos/memos/commit/551ee1d81f398abdcaf73a0c2b782c8379b0e3f2))
* backend tests action ([065e817](https://github.com/usememos/memos/commit/065e817470b1d3b6b00222525791b85d825c1492))
* clear content search filter when selecting shortcut ([#5499](https://github.com/usememos/memos/issues/5499)) ([2c3f9e3](https://github.com/usememos/memos/commit/2c3f9e3bfbe09e5d0028d1525c1df83c03f547c9))
* correct typos in comments, error messages, and identifiers ([#5704](https://github.com/usememos/memos/issues/5704)) ([8f43e80](https://github.com/usememos/memos/commit/8f43e8075b62706f99da4005ab04b165b35215f6))
* detect legacy installations with empty schema version ([9628d3d](https://github.com/usememos/memos/commit/9628d3de21289e887ffeb6cc4b6710913a0de44e))
* **editor:** show newly linked memos when editing a memo with attachments ([026ea92](https://github.com/usememos/memos/commit/026ea92f7b4005c30ca84340c22018740659ab51))
* ensure comment divs span full width in MemoDetail ([ce44164](https://github.com/usememos/memos/commit/ce441644af34ffff156f1793c40011c2b82e8d5f))
* **fileserver:** render SVG attachment previews ([40fd700](https://github.com/usememos/memos/commit/40fd700fb8be27a8d7a300a2636cf4f366572176))
* **filter:** enforce CEL syntax semantics ([0e89407](https://github.com/usememos/memos/commit/0e89407ee91deda87e0df7464a89d26d1e9a88b3))
* **frontend:** restore sitemap and robots routes ([fee7fcd](https://github.com/usememos/memos/commit/fee7fcd6608b9d07da1f146c33c9bb8f898f708f))
* handle chunk load errors after redeployment with auto-reload ([#5703](https://github.com/usememos/memos/issues/5703)) ([bdd3554](https://github.com/usememos/memos/commit/bdd3554b897246bac5c1e6fd801495c24b6a656e))
* harden memo content iframe and HTML sanitization ([7e21b72](https://github.com/usememos/memos/commit/7e21b728b346e80023c93e8986130f83e70584f9))
* hide transcribe button without AI provider ([ab53329](https://github.com/usememos/memos/commit/ab5332901fd6626f0a2ae6b99671af9663f68078))
* improve image preview dialog and live photo trigger ([aafcc21](https://github.com/usememos/memos/commit/aafcc21ae6f96a313fb372365dd1af8a1a69d1ba))
* improve installer compatibility and docs ([f90d9a4](https://github.com/usememos/memos/commit/f90d9a49a78d0764a6c5cf8f6720e69ea1164b25))
* improve KaTeX and Mermaid error handling and overflow ([6b37fcc](https://github.com/usememos/memos/commit/6b37fcc01b5dcded6a2ec254fbaf895f7812d072))
* include plain URLs and tags in memo snippet generation ([#5688](https://github.com/usememos/memos/issues/5688)) ([3d4f793](https://github.com/usememos/memos/commit/3d4f793f97b567bda823a60b7fe10e09e14499ce))
* **lint:** correct goimports struct literal alignment after removing write-only credential fields ([#5794](https://github.com/usememos/memos/issues/5794)) ([9610ed8](https://github.com/usememos/memos/commit/9610ed8fc809b3e7aa4b93e4909634957fa94cf7))
* **map:** align dark mode map styling ([7ac9989](https://github.com/usememos/memos/commit/7ac9989d43f3967b466f340da3841290d86b20f1))
* **map:** refine Leaflet controls and memo map styling ([894b3eb](https://github.com/usememos/memos/commit/894b3eb045c7d2c2acce847c34a85295c909dcf7))
* **markdown:** support height/width attributes on img elements ([737acbb](https://github.com/usememos/memos/commit/737acbba2f5e3dd911da3a77d2d5d1cde1bf8ba1))
* **memo-editor:** scope Cmd+Enter save to the active editor ([#5745](https://github.com/usememos/memos/issues/5745)) ([05810e7](https://github.com/usememos/memos/commit/05810e7882cb03d448d9d8fa84bc6ca54eafbed0))
* **mysql:** handle CreateMemo custom timestamps with FROM_UNIXTIME ([#5673](https://github.com/usememos/memos/issues/5673)) ([09d73e8](https://github.com/usememos/memos/commit/09d73e8b6e16c43a61767817909a472916c83daa))
* normalize attachment MIME types before validation ([c3e7e2c](https://github.com/usememos/memos/commit/c3e7e2c316d01bf857931c38303efc87307f2a19))
* preserve draft content when tab is suspended or editor remounts ([9ca7122](https://github.com/usememos/memos/commit/9ca71229a6c0dfad44b76ed06fced46c5bef49c7))
* prevent local attachment uploads from overwriting files ([4add9b0](https://github.com/usememos/memos/commit/4add9b04ad5d0b7608c985281f5975dc39f9a572))
* prevent stale comment drafts from being restored ([e520b63](https://github.com/usememos/memos/commit/e520b637fd8d0f6331674003abe72f9dfbae8231))
* remove duplicate Japanese locale keys ([efeb28c](https://github.com/usememos/memos/commit/efeb28c872a2bd6edd28fa3b68bd0b15af923d50))
* render audio attachments as inline players ([#5699](https://github.com/usememos/memos/issues/5699)) ([2ccb98a](https://github.com/usememos/memos/commit/2ccb98a6cbc40544e81eeeca2a2cbbaf8348f45e))
* restrict archived memo access to creator only ([#5707](https://github.com/usememos/memos/issues/5707)) ([f4154d0](https://github.com/usememos/memos/commit/f4154d090be3896c8e0dd83440f24953bbeb308c))
* **routing:** redirect unauthenticated users to /explore when public visibility is allowed ([98859eb](https://github.com/usememos/memos/commit/98859eb5e5065a7c12ada983b7c38c5af5035bb6))
* **server:** close SSE clients during shutdown ([a5ddd5a](https://github.com/usememos/memos/commit/a5ddd5adafef282f00b23e00e145119d4316f24f))
* sync html lang attribute with active locale ([#5753](https://github.com/usememos/memos/issues/5753)) ([be00abe](https://github.com/usememos/memos/commit/be00abe852479820b7c574e15e276d70961ab7bd))
* tag parsing truncates emojis with variation selectors ([#5496](https://github.com/usememos/memos/issues/5496)) ([3ea6ea3](https://github.com/usememos/memos/commit/3ea6ea3108f58dc34f39fa6f21d761bd2e97732b))
* **tags:** allow blur-only tag metadata ([#5800](https://github.com/usememos/memos/issues/5800)) ([1921b57](https://github.com/usememos/memos/commit/1921b57662c2129d179930f71e5c42caf9070a19))
* toggle focus mode do not reset editor height ([#5504](https://github.com/usememos/memos/issues/5504)) ([0729779](https://github.com/usememos/memos/commit/0729779e0427c68bdbc5b6b711a5683f66ba9bbc))
* **ui:** show comment editor above the comment list ([#5662](https://github.com/usememos/memos/issues/5662)) ([6b0736b](https://github.com/usememos/memos/commit/6b0736b29325ff7698b7395dc761f2ca41e6521c))
* **ui:** unify metadata component styles across MemoView and MemoEditor ([664b8c5](https://github.com/usememos/memos/commit/664b8c56290026ba2beb7b75db24c684824b8670))
* unify live photo previews around LIVE badge playback ([6b0487d](https://github.com/usememos/memos/commit/6b0487dcd857ba72189ca0624cb4985961f08f25))
* **webhooks:** trigger memo updates for attachment and relation changes ([#5795](https://github.com/usememos/memos/issues/5795)) ([acbc914](https://github.com/usememos/memos/commit/acbc914dea8d4e533028b03929193a4af1ac32b4))
* **web:** prevent MemoContent prop leaks ([22519b5](https://github.com/usememos/memos/commit/22519b57a0a971ab927a03015c994c63b8c3f1da))
* **web:** refine attachment media layout ([a0d83e1](https://github.com/usememos/memos/commit/a0d83e1a9e9d0ba50a0103e594adb011728faf8c))
* **web:** refresh memo detail cache after editor save ([333c9df](https://github.com/usememos/memos/commit/333c9df233240a843a9e62d17c6ff464690636f8))
* **web:** use BroadcastChannel to sync token refreshes across tabs ([bbdc998](https://github.com/usememos/memos/commit/bbdc998646e8093223a448378a2c29690f0d8612))
### Performance Improvements
* batch load memo relations when listing memos ([#5692](https://github.com/usememos/memos/issues/5692)) ([1e82714](https://github.com/usememos/memos/commit/1e82714a52c72455a6ba02605cc641d7ad4fbf1a))
+2
View File
@@ -0,0 +1,2 @@
# These owners will be the default owners for everything in the repo.
* @usememos/moderators
+21
View File
@@ -0,0 +1,21 @@
MIT License
Copyright (c) 2025 Memos
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
+88
View File
@@ -0,0 +1,88 @@
> ✨ Featured Sponsor: [CodeRabbit](https://coderabbit.link/usememos) — Cut code review time & bugs in half, instantly.
# Memos
<img align="right" height="96px" src="https://raw.githubusercontent.com/usememos/.github/refs/heads/main/assets/logo-rounded.png" alt="Memos" />
Open-source, self-hosted note-taking tool built for quick capture. Markdown-native, lightweight, and fully yours.
[![Home](https://img.shields.io/badge/🏠-usememos.com-blue?style=flat-square)](https://usememos.com)
[![Live Demo](https://img.shields.io/badge/✨-Try%20Demo-orange?style=flat-square)](https://demo.usememos.com/)
[![Docs](https://img.shields.io/badge/📚-Documentation-green?style=flat-square)](https://usememos.com/docs)
[![Discord](https://img.shields.io/badge/💬-Discord-5865f2?style=flat-square&logo=discord&logoColor=white)](https://discord.gg/tfPJa4UmAv)
[![Docker Pulls](https://img.shields.io/docker/pulls/neosmemo/memos?style=flat-square&logo=docker)](https://hub.docker.com/r/neosmemo/memos)
<img src="https://raw.githubusercontent.com/usememos/.github/refs/heads/main/assets/demo.png" alt="Memos Demo Screenshot" height="512" />
## Features
- **Instant Capture** — Timeline-first UI. Open, write, done — no folders to navigate.
- **Total Data Ownership** — Self-hosted on your infrastructure. Notes stored in Markdown, always portable. Zero telemetry.
- **Radical Simplicity** — Single Go binary, ~20MB Docker image. One command to deploy with SQLite, MySQL, or PostgreSQL.
- **Open & Extensible** — MIT-licensed with full REST and gRPC APIs for integration.
## Quick Start
### Docker (Recommended)
```bash
docker run -d \
--name memos \
-p 5230:5230 \
-v ~/.memos:/var/opt/memos \
neosmemo/memos:stable
```
Open `http://localhost:5230` and start writing!
### Native Binary
```bash
curl -fsSL https://raw.githubusercontent.com/usememos/memos/main/scripts/install.sh | sh
```
### Try the Live Demo
Don't want to install yet? Try our [live demo](https://demo.usememos.com/) first!
### Other Installation Methods
- **Docker Compose** - Recommended for production deployments
- **Pre-built Binaries** - Available for Linux, macOS, and Windows
- **Kubernetes** - Helm charts and manifests available
- **Build from Source** - For development and customization
See our [installation guide](https://usememos.com/docs/deploy) for detailed instructions.
## Contributing
Contributions are welcome — bug reports, feature suggestions, pull requests, documentation, and translations.
- [Report bugs](https://github.com/usememos/memos/issues/new?template=bug_report.md)
- [Suggest features](https://github.com/usememos/memos/issues/new?template=feature_request.md)
- [Submit pull requests](https://github.com/usememos/memos/pulls)
- [Improve documentation](https://github.com/usememos/dotcom)
- [Help with translations](https://github.com/usememos/memos/tree/main/web/src/locales)
## Sponsors
* [**CodeRabbit** - Cut code review time & bugs in half, instantly](https://coderabbit.link/usememos)
* [**SSD Nodes** - Affordable VPS hosting for self-hosters](https://ssdnodes.com/?utm_source=memos&utm_medium=sponsor)
* [**InstaPods** - Get your app online in seconds](https://instapods.com/?utm_source=memos&utm_medium=sponsor) • [Deploy Memos in 30 Seconds](https://instapods.com/apps/memos/?utm_source=memos&utm_medium=sponsor)
Love Memos? [Sponsor us on GitHub](https://github.com/sponsors/usememos) to help keep the project growing!
## Star History
[![Star History Chart](https://api.star-history.com/svg?repos=usememos/memos&type=Date)](https://star-history.com/#usememos/memos&Date)
## License
Memos is open-source software licensed under the [MIT License](LICENSE). See our [Privacy Policy](https://usememos.com/privacy) for details on data handling.
---
**[Website](https://usememos.com)** • **[Documentation](https://usememos.com/docs)** • **[Demo](https://demo.usememos.com/)** • **[Discord](https://discord.gg/tfPJa4UmAv)** • **[X/Twitter](https://x.com/usememos)**
<a href="https://vercel.com/oss">
<img alt="Vercel OSS Program" src="https://vercel.com/oss/program-badge.svg" />
</a>
+7
View File
@@ -0,0 +1,7 @@
# WeHub 来源说明
- 原始项目:`usememos/memos`
- 原始仓库:https://github.com/usememos/memos
- 导入方式:上游默认分支的最新快照
- 原作者、版权和许可证信息以原始仓库及本仓库 LICENSE 为准
- 本文件仅用于记录来源,不代表 WeHub 是原项目作者
+41
View File
@@ -0,0 +1,41 @@
# Security Policy
## Supported Versions
Memos is currently a `0.x` project. Security fixes are only provided for the latest release. Older releases are not supported for security updates, and fixes are not backported.
If you run Memos in production, keep your instance updated to the latest release.
## Reporting a Vulnerability
Please report security issues privately by email: `dev@usememos.com`
Do not open public GitHub issues, discussions, or pull requests for suspected vulnerabilities.
Please include:
- A clear description of the issue
- Steps to reproduce
- Affected version or commit
- Deployment details that matter to reproduction
- Your assessment of impact
We will review reports as time permits and fix valid issues in regular releases.
## Disclosure and CVEs
Memos is self-hosted software and is still in the `0.x` stage. At this stage, we do not run a formal disclosure program, publish separate security advisories for every issue, or request CVE IDs.
Security fixes may be shipped directly in normal releases or noted briefly in release notes and changelogs.
## Self-Hosted Deployment Notes
The security posture of a Memos instance depends heavily on how it is deployed and operated. In particular:
- Keep Memos updated
- Put it behind a properly configured reverse proxy when exposed to the internet
- Require authentication for any non-public deployment
- Use TLS in production
- Limit access to trusted users and administrators
Reports that depend entirely on intentionally unsafe deployment choices, unsupported local patches, or administrator actions may be treated as deployment issues rather than product vulnerabilities.
+42
View File
@@ -0,0 +1,42 @@
# Identity Provider Bootstrap
Memos automatically reconciles OAuth2 identity providers from JSON files before the server starts. This is useful when a deployment platform provides configuration as mounted secret files.
By default, Memos scans `/etc/secrets` for files named `memos-idp-*.json`:
```bash
memos
```
For example, `/etc/secrets/memos-idp-primary.json` can contain:
```json
{
"uid": "primary-sso",
"name": "Company SSO",
"type": "OAUTH2",
"identifierFilter": "",
"config": {
"oauth2Config": {
"clientId": "client-id",
"clientSecret": "client-secret",
"authUrl": "https://idp.example.com/oauth/authorize",
"tokenUrl": "https://idp.example.com/oauth/token",
"userInfoUrl": "https://idp.example.com/oauth/userinfo",
"scopes": ["profile", "email"],
"fieldMapping": {
"identifier": "sub",
"displayName": "name",
"email": "email",
"avatarUrl": "picture"
}
}
}
}
```
Each file contains exactly one `memos.store.IdentityProvider` encoded as protobuf JSON. The database-generated numeric `id` must be omitted; `uid` is the provider's stable identifier.
After database migrations and demo seeding, Memos reads all matching files in filename order and validates every provider before writing anything. Each provider is then created or updated by its stable `uid`; providers omitted from the files are left unchanged. Reapplying the files is safe and updates credentials on restart, which supports secret rotation. Duplicate provider UIDs across files are rejected.
A missing directory or a directory without matching files is a normal no-op. If a matching file is unreadable, oversized, or invalid, Memos does not start. Files with other names are ignored so the directory can safely contain unrelated secrets. Keep files containing `clientSecret` outside source control and provide them through the deployment platform's secret-management facility.
@@ -0,0 +1,35 @@
## Background & Context
The MemoDetail page (`web/src/pages/MemoDetail.tsx`) renders a memo's full content with a right sidebar (`MemoDetailSidebar`) on desktop. The sidebar currently shows share controls, timestamps, property badges (links, to-do, code), and tags. Memo content is rendered via `react-markdown` with custom heading components (h1h6) in `MemoContent/markdown/Heading.tsx`. The page already has hash-based scroll-to-element infrastructure for comments (lines 3943 of MemoDetail.tsx).
## Issue Statement
When viewing a memo with structured headings, there is no outline or table of contents in the MemoDetail sidebar, so readers cannot see the document structure at a glance or jump to specific sections. Heading elements rendered by the `Heading` component do not have `id` attributes, preventing any anchor-based navigation to them.
## Current State
- **`web/src/pages/MemoDetail.tsx`** (93 lines) — MemoDetail page. Hash-based `scrollIntoView` logic exists at lines 3943 but only targets comment elements. The sidebar is rendered at lines 8286 as `<MemoDetailSidebar>`.
- **`web/src/components/MemoDetailSidebar/MemoDetailSidebar.tsx`** (109 lines) — Desktop sidebar. Contains sections: Share (lines 5865), Created-at (lines 6776), Properties (lines 7889), Tags (lines 91102). Uses a reusable `SidebarSection` helper (lines 2432). No outline section exists.
- **`web/src/components/MemoDetailSidebar/MemoDetailSidebarDrawer.tsx`** (36 lines) — Mobile drawer wrapper for the sidebar.
- **`web/src/components/MemoContent/markdown/Heading.tsx`** (30 lines) — Renders h1h6 with Tailwind classes. Does not generate `id` attributes on heading elements.
- **`web/src/components/MemoContent/index.tsx`** (157 lines) — ReactMarkdown renderer. Maps h1h6 to the `Heading` component (lines 72101). No rehype-slug or equivalent plugin is installed.
- **`web/src/utils/markdown-manipulation.ts`** (143 lines) — Contains `fromMarkdown()` MDAST parsing with GFM extensions. Used for task extraction; no heading extraction exists.
## Non-Goals
- Redesigning the overall MemoDetail layout or sidebar structure beyond adding the outline section.
- Supporting outline for the mobile drawer — the outline section should appear in the desktop sidebar only for now.
- Adding active heading highlighting on scroll (scroll-spy behavior).
- Supporting outline in the memo list/timeline view (compact mode).
- Changing the existing heading visual styles in the content area.
## Open Questions
- Which heading levels to include in the outline? (default: h1h4, matching user request)
- How to generate slug IDs — install `rehype-slug` or custom implementation? (default: lightweight custom slug generation in the Heading component to avoid a new dependency)
- Should the outline section be hidden when no headings exist? (default: yes, hide entirely like Properties/Tags sections)
- Should duplicate heading text produce unique slugs (e.g. `my-heading`, `my-heading-1`)? (default: yes, append numeric suffix for duplicates)
## Scope
**M** — Touches 34 files (Heading component, MemoDetailSidebar, possibly a new outline component, and a slug utility). Existing patterns (SidebarSection, hash scrolling) apply directly. No new subsystem or novel approach required.
@@ -0,0 +1,47 @@
## Task List
T1: Add heading extraction utility [S] — T2: Add slug IDs to Heading component [S] — T3: Create MemoOutline sidebar component [M] — T4: Integrate outline into MemoDetailSidebar [S]
### T1: Add heading extraction utility [S]
**Objective**: Provide a function to extract h1h4 headings from markdown content with slugified IDs, reusing the existing MDAST parsing pattern from `markdown-manipulation.ts`.
**Files**: `web/src/utils/markdown-manipulation.ts`
**Implementation**: Add `HeadingItem` interface (text, level, slug) and `extractHeadings(markdown: string): HeadingItem[]` function. Use existing `fromMarkdown()` + `visit()` pattern. Visit `"heading"` nodes with depth 14, extract text from children, generate slug via `slugify()` helper (lowercase, replace non-alphanumeric with hyphens, deduplicate). Export both.
**Validation**: `cd web && pnpm lint` — no new errors
### T2: Add slug IDs to Heading component [S]
**Objective**: Generate deterministic `id` attributes on h1h6 elements so outline links can scroll to them via `#hash`.
**Files**: `web/src/components/MemoContent/markdown/Heading.tsx`
**Implementation**: In `Heading` (~line 13), extract text from `children` using a `getTextContent(children)` helper that recursively extracts string content from React children. Generate slug with the same `slugify` logic. Apply `id={slug}` to the rendered `<Component>`.
**Validation**: `cd web && pnpm lint` — no new errors
### T3: Create MemoOutline sidebar component [M]
**Objective**: Create a modern, Claude/Linear-style outline component that renders h1h4 headings as anchor links with indentation by level.
**Size**: M (new component file, modern styling)
**Files**:
- Create: `web/src/components/MemoDetailSidebar/MemoOutline.tsx`
**Implementation**:
1. Props: `{ headings: HeadingItem[] }` from `markdown-manipulation.ts`
2. Render a `<nav>` with vertical list of `<a href="#slug">` links
3. Styling per level: h1 no indent, h2 `pl-3`, h3 `pl-6`, h4 `pl-9`. Text size: h1 `text-[13px] font-medium`, h2h4 `text-[13px] font-normal`. Color: `text-muted-foreground` with `hover:text-foreground` transition. Left border accent line (2px) along the nav. Smooth scroll on click via `scrollIntoView`.
4. Each link: `block py-1 truncate transition-colors` with level-based indentation
**Boundaries**: No scroll-spy / active state tracking. No mobile drawer integration.
**Dependencies**: T1
**Expected Outcome**: Component renders a clean, modern outline navigation.
**Validation**: `cd web && pnpm lint` — no new errors
### T4: Integrate outline into MemoDetailSidebar [S]
**Objective**: Add the outline section as the first section in `MemoDetailSidebar`, shown only when headings exist.
**Files**: `web/src/components/MemoDetailSidebar/MemoDetailSidebar.tsx`
**Implementation**: Import `extractHeadings` and `MemoOutline`. In `MemoDetailSidebar` (~line 48), compute `headings = useMemo(() => extractHeadings(memo.content), [memo.content])`. Before the Share section (~line 58), add conditional: `{headings.length > 0 && <SidebarSection label="Outline"><MemoOutline headings={headings} /></SidebarSection>}`.
**Validation**: `cd web && pnpm lint && pnpm build` — no errors
## Out-of-Scope Tasks
- Scroll-spy / active heading highlighting in the outline
- Mobile drawer outline support
- Outline in memo list view (compact mode)
- Changing existing heading visual styles in content area
@@ -0,0 +1,98 @@
## Background & Context
Memos has a content-blur feature: when a memo's tag list contains the literal string `NSFW`
(case-insensitive), the memo body is rendered with a `blur-lg` CSS class and a click-to-reveal
overlay is shown. This was simplified in v0.26.x from a previous admin-configurable system
(which had an on/off toggle and a custom-tag-list) down to a single hardcoded tag name.
In the same release cycle, an `InstanceTagsSetting` system was introduced that lets admins attach
metadata (currently only `background_color`) to tag name patterns via a regex-keyed map. This
system has its own proto definitions, store layer, API service handlers, frontend context, utility
library, and settings UI — all independent of the blur feature.
A sponsor raised (orgs/usememos/discussions/5708) that the hardcoded tag name is inconvenient:
users who organised their content under a different tag (e.g. a non-English word, a project-
specific label, or simply a term they prefer) must re-tag all existing memos just to use the blur
feature. Community comments echo the same concern and additionally ask for the ability to disable
the blur globally.
## Issue Statement
The memo content-blur trigger is evaluated exclusively against the hardcoded string `"NSFW"`
(case-insensitive) in `MemoView.tsx`, with no connection to the `InstanceTagsSetting` system,
making it impossible for an administrator to designate any other tag name — or set of tag name
patterns — as a blur trigger, and preventing users from re-using existing tag taxonomies to
activate content blurring.
## Current State
**Blur detection — frontend**
| File | Lines | Behaviour |
|------|-------|-----------|
| `web/src/components/MemoView/MemoView.tsx` | 2730 | `const nsfw = memoData.tags?.some((tag) => tag.toUpperCase() === "NSFW") ?? false;` — single hardcoded string comparison |
| `web/src/components/MemoView/MemoViewContext.tsx` | 1619 | Context shape exposes `nsfw: boolean`, `showNSFWContent: boolean`, `toggleNsfwVisibility` |
| `web/src/components/MemoView/components/MemoBody.tsx` | 1123, 37, 53 | Applies `blur-lg transition-all duration-200` when `nsfw && !showNSFWContent`; renders `NsfwOverlay` button using i18n key `memo.click-to-show-nsfw-content` |
| `web/src/components/MemoPreview/MemoPreview.tsx` | 2427 | Stub context value: `nsfw: false`, `showNSFWContent: false` — blur never active in preview |
**Localisation strings that contain the "NSFW" term**
| File | Keys |
|------|------|
| `web/src/locales/en.json` (and ~30 other locale files) | `memo.click-to-hide-nsfw-content`, `memo.click-to-show-nsfw-content`, `settings.enable-blur-nsfw-content` |
Note: most non-English translations already use "sensitive content" rather than "NSFW" in these
keys; English is the outlier.
**Tag metadata system — proto**
| File | Lines | Content |
|------|-------|---------|
| `proto/api/v1/instance_service.proto` | 168181 | `message TagMetadata { google.type.Color background_color = 1; }` nested inside `InstanceSetting`; `TagsSetting` is a `map<string, TagMetadata>` |
| `proto/store/instance_setting.proto` | 113124 | `message InstanceTagMetadata { google.type.Color background_color = 1; }` inside `InstanceTagsSetting` |
**Tag metadata system — backend**
| File | Lines | Content |
|------|-------|---------|
| `store/instance_setting.go` | 166192 | `GetInstanceTagsSetting()` retrieves and caches the tags map |
| `server/router/api/v1/instance_service.go` | 300328 | `convertInstanceTagsSettingFromStore()` / `convertInstanceTagsSettingToStore()` convert between store and API representations, field-by-field |
| `server/router/api/v1/instance_service.go` | 387409 | `validateInstanceTagsSetting()` validates each key as a regex pattern and the color value |
**Tag metadata system — frontend**
| File | Lines | Content |
|------|-------|---------|
| `web/src/lib/tag.ts` | 2843 | `findTagMetadata(tag, tagsSetting)` — exact-match then regex-match lookup returning `TagMetadata \| undefined` |
| `web/src/components/MemoContent/Tag.tsx` | 2338 | Calls `findTagMetadata()` to apply `background_color` to inline tag chips |
| `web/src/components/Settings/TagsSection.tsx` | 36206 | Admin settings UI for managing the tag→metadata map; currently shows only a colour picker per tag |
| `web/src/contexts/InstanceContext.tsx` | 8399 | `tagsSetting` selector and fetch during app initialisation |
## Non-Goals
- Redesigning or replacing the `InstanceTagsSetting` proto or store structure beyond adding one field.
- Providing a per-user (as opposed to per-instance) blur preference.
- Changing how background-color metadata is stored, validated, or rendered.
- Adding a global on/off toggle for blurring (separate from per-tag configuration).
- Modifying the blur visual effect (CSS class, animation, overlay button layout).
- Migrating or auto-converting any existing memos that were tagged with `NSFW`.
- Changing non-English locale strings that already use neutral terminology.
## Open Questions
1. Should the `blur_content` field in tag metadata be configurable per-tag only by admins, or also by individual users via user-level tag settings? (default: admin-only, matching the existing `InstanceTagsSetting` access model)
2. When a memo has multiple tags and more than one of them has `blur_content = true`, should the blur activate if _any_ matching tag has the flag set, or only if _all_ matching tags do? (default: any — OR semantics, consistent with the current single-tag check)
3. Should there be a migration that automatically sets `blur_content = true` for any existing `InstanceTagsSetting` entry whose key is `"NSFW"` (case-insensitive)? (default: no automatic migration; admins reconfigure manually)
4. What should the English-locale i18n key strings say, given that "NSFW" is to be avoided? (default: "Click to show sensitive content" / "Click to hide sensitive content")
## Scope
**M** — the change adds one `bool` field to two existing proto messages, threads it through two
existing conversion functions in the backend, replaces one hardcoded string comparison in
`MemoView.tsx` with a call to the already-present `findTagMetadata()` utility, adds a checkbox to
the existing `TagsSection.tsx` settings UI, and renames three i18n keys. All required patterns
(field addition, conversion, `findTagMetadata` lookup, settings UI checkbox) already exist in the
codebase.
@@ -0,0 +1,63 @@
## Execution Log
### T1: Add blur_content field to proto messages
**Status**: Completed
**Files Changed**: `proto/api/v1/instance_service.proto`, `proto/store/instance_setting.proto`
**Validation**: `buf lint` — PASS
**Path Corrections**: None
**Deviations**: None
### T2: Regenerate proto code
**Status**: Completed
**Files Changed**: `proto/gen/` (Go + OpenAPI), `web/src/types/proto/` (TypeScript)
**Validation**: `grep blur_content` in generated files — PASS (field present in Go, TS, OpenAPI)
**Path Corrections**: None
**Deviations**: None
### T3: Thread blur_content through backend conversions
**Status**: Completed
**Files Changed**: `server/router/api/v1/instance_service.go`
**Validation**: `go build ./...` — PASS
**Path Corrections**: None
**Deviations**: None
### T4: Replace hardcoded NSFW check with tag metadata lookup
**Status**: Completed
**Files Changed**: `web/src/components/MemoView/MemoView.tsx`, `web/src/components/MemoView/MemoViewContext.tsx`, `web/src/components/MemoView/components/MemoBody.tsx`, `web/src/components/MemoPreview/MemoPreview.tsx`
**Validation**: `pnpm lint` — PASS
**Path Corrections**: i18n key update (T6) was pulled forward to unblock TypeScript type checking, since the i18n key type is statically checked.
**Deviations**: None
### T5: Add blur checkbox to TagsSection settings UI
**Status**: Completed
**Files Changed**: `web/src/components/Settings/TagsSection.tsx`
**Validation**: `pnpm lint` — PASS
**Path Corrections**: None
**Deviations**: None
### T6: Update English i18n keys
**Status**: Completed
**Files Changed**: `web/src/locales/en.json`
**Validation**: `grep -c "nsfw\|NSFW" en.json` — returns 0, PASS
**Path Corrections**: Executed during T4/T5 to unblock type checking. Added `setting.tags.blur-content` key (not in original plan but required by T5's new checkbox column).
**Deviations**: None
## Completion Declaration
**All tasks completed successfully.**
Summary of changes:
- Added `bool blur_content = 2` to both API and store proto TagMetadata messages
- Regenerated Go, TypeScript, and OpenAPI code
- Threaded `blur_content` through `convertInstanceTagsSettingFromStore()` and `convertInstanceTagsSettingToStore()`
- Replaced hardcoded `tag.toUpperCase() === "NSFW"` with `findTagMetadata(tag, tagsSetting)?.blurContent` lookup
- Renamed context fields: `nsfw``blurred`, `showNSFWContent``showBlurredContent`, `toggleNsfwVisibility``toggleBlurVisibility`
- Renamed `NsfwOverlay``BlurOverlay` component
- Expanded TagsSection local state to track `{ color, blur }` per tag and added a "Blur content" checkbox column
- Updated English i18n: renamed NSFW keys to "sensitive content", removed unused key, added `blur-content` setting key
@@ -0,0 +1,89 @@
## Task List
T1: Add blur_content field to proto messages [S] — T2: Regenerate proto code [S] — T3: Thread blur_content through backend conversions [S] — T4: Replace hardcoded NSFW check with tag metadata lookup [M] — T5: Add blur checkbox to TagsSection settings UI [S] — T6: Update English i18n keys [S]
### T1: Add blur_content field to proto messages [S]
**Objective**: Add a `bool blur_content` field to both the API and store proto TagMetadata messages.
**Files**: `proto/api/v1/instance_service.proto`, `proto/store/instance_setting.proto`
**Implementation**:
- In `proto/api/v1/instance_service.proto` (~line 171), add `bool blur_content = 2;` to `message TagMetadata` after `background_color`
- In `proto/store/instance_setting.proto` (~line 115), add `bool blur_content = 2;` to `message InstanceTagMetadata` after `background_color`
**Validation**: `cd proto && buf lint` — no errors
### T2: Regenerate proto code [S]
**Objective**: Regenerate Go + TypeScript + OpenAPI from updated proto definitions.
**Files**: `proto/gen/` (generated), `web/src/types/proto/` (generated)
**Implementation**: Run `cd proto && buf generate`
**Dependencies**: T1
**Validation**: `grep -r "blur_content\|blurContent" proto/gen/ web/src/types/proto/ | head -10` — shows new field in generated Go and TS files
### T3: Thread blur_content through backend conversion functions [S]
**Objective**: Pass `blur_content` through the store↔API conversion functions so the field round-trips correctly.
**Files**: `server/router/api/v1/instance_service.go`
**Implementation**:
- In `convertInstanceTagsSettingFromStore()` (~line 306): add `BlurContent: metadata.GetBlurContent()` to the `InstanceSetting_TagMetadata` struct literal
- In `convertInstanceTagsSettingToStore()` (~line 321): add `BlurContent: metadata.GetBlurContent()` to the `InstanceTagMetadata` struct literal
**Dependencies**: T2
**Validation**: `cd /Users/steven/Projects/usememos/memos && go build ./...` — compiles without errors
### T4: Replace hardcoded NSFW check with tag metadata lookup [M]
**Objective**: Replace the hardcoded `tag.toUpperCase() === "NSFW"` check with a lookup against `InstanceTagsSetting` via the existing `findTagMetadata()` utility, so any tag with `blur_content: true` triggers the blur.
**Size**: M (3 files, moderate logic)
**Files**:
- Modify: `web/src/components/MemoView/MemoView.tsx`
- Modify: `web/src/components/MemoView/MemoViewContext.tsx`
- Modify: `web/src/components/MemoView/components/MemoBody.tsx`
- Modify: `web/src/components/MemoPreview/MemoPreview.tsx`
**Implementation**:
1. In `MemoView.tsx`:
- Import `useInstance` from `@/contexts/InstanceContext` and `findTagMetadata` from `@/lib/tag`
- Replace `const nsfw = memoData.tags?.some((tag) => tag.toUpperCase() === "NSFW") ?? false;` with a check that iterates `memoData.tags` and uses `findTagMetadata(tag, tagsSetting)?.blurContent` — OR semantics (any match triggers blur)
- Rename state/variables: `showNSFWContent``showBlurredContent`, `nsfw``blurred`, `toggleNsfwVisibility``toggleBlurVisibility`
2. In `MemoViewContext.tsx`:
- Rename interface fields: `nsfw``blurred`, `showNSFWContent``showBlurredContent`, `toggleNsfwVisibility``toggleBlurVisibility`
3. In `MemoBody.tsx`:
- Update destructured context fields to use new names (`blurred`, `showBlurredContent`, `toggleBlurVisibility`)
- Rename `NsfwOverlay` component to `BlurOverlay`
- Change i18n key from `memo.click-to-show-nsfw-content` to `memo.click-to-show-sensitive-content`
4. In `MemoPreview.tsx`:
- Update stub context to use new field names (`blurred`, `showBlurredContent`, `toggleBlurVisibility`)
**Boundaries**: Do NOT change blur CSS classes, animation, or overlay layout
**Dependencies**: T2
**Validation**: `cd web && pnpm lint` — no type or lint errors
### T5: Add blur checkbox to TagsSection settings UI [S]
**Objective**: Add a "Blur content" checkbox column to the tag settings table so admins can toggle `blur_content` per tag pattern.
**Files**: `web/src/components/Settings/TagsSection.tsx`
**Implementation**:
- Expand `localTags` state from `Record<string, string>` (hex only) to `Record<string, { color: string; blur: boolean }>` to track both fields
- Update `useEffect` sync, `originalHexMap` comparison, `handleColorChange`, `handleRemoveTag`, `handleAddTag` to work with the new shape
- Add a new `[newTagBlur, setNewTagBlur]` state for the add-tag row (default `false`)
- In `handleSave`, pass `blurContent` when creating `InstanceSetting_TagMetadata`
- Add a new column to `SettingTable` between "Background color" and "Actions": header `t("setting.tags.blur-content")`, renders a checkbox bound to `localTags[row.name].blur`
- Add the i18n key `setting.tags.blur-content` to `en.json` with value `"Blur content"`
**Dependencies**: T2
**Validation**: `cd web && pnpm lint` — no type or lint errors
### T6: Update English i18n keys [S]
**Objective**: Rename NSFW-specific i18n keys in `en.json` to use neutral "sensitive content" terminology.
**Files**: `web/src/locales/en.json`
**Implementation**:
- Change key `memo.click-to-show-nsfw-content``memo.click-to-show-sensitive-content` with value `"Click to show sensitive content"`
- Change key `memo.click-to-hide-nsfw-content``memo.click-to-hide-sensitive-content` with value `"Click to hide sensitive content"` (dead key but renamed for consistency)
- The key `settings.enable-blur-nsfw-content` is unused in code — remove it
**Dependencies**: T4 (key rename must match code references)
**Validation**: `grep -c "nsfw" web/src/locales/en.json` — returns `0`
## Out-of-Scope Tasks
- Updating non-English locale files (per non-goals: "Changing non-English locale strings that already use neutral terminology")
- Adding automatic migration for existing NSFW tag entries
- Per-user blur preferences
- Global on/off toggle for blurring
- Modifying blur visual effect (CSS, animation, overlay layout)
@@ -0,0 +1,41 @@
## Background & Context
User resources in Memos v1 are exposed through Connect/gRPC-Gateway handlers in `server/router/api/v1`, proto resource definitions in `proto/api/v1`, frontend profile flows in `web/src`, and MCP JSON helpers in `server/router/mcp`. The store schema already persists both an internal integer `id` and a unique `username` for each user. The GitHub issue reports that public user resource names such as `users/2` are still emitted across responses and nested user-scoped resources. Existing code already mixes identifier forms: `GetUser` accepts either `users/{id}` or `users/{username}`, the fileserver avatar route accepts either identifier, and the frontend profile page already enters the API through `users/{username}` before reusing the returned `user.name`.
## Issue Statement
Across the v1 API surface, canonical user resource names are currently constructed from `store.User.ID` rather than `store.User.Username`, and many handlers parse those emitted names back into integers for authorization and lookup. As a result, top-level user resources and nested user-scoped references in settings, stats, shortcuts, webhooks, notifications, memo creators, reactions, and MCP payloads expose sequential database IDs and couple downstream callers to integer-based user tokens in server-emitted names.
## Current State
- `store/user.go:26-42` defines `store.User` with both `ID int32` and `Username string`; `store/migration/sqlite/LATEST.sql:10-21` declares `username TEXT NOT NULL UNIQUE`.
- `server/router/api/v1/user_service.go:72-102` handles `GetUser` by extracting `users/{id_or_username}` and resolving either a numeric ID or a username; `server/router/api/v1/user_service.go:914-937` still serializes `User.name` as `users/{id}` and derives avatar URLs from that name.
- `server/router/api/v1/resource_name.go:67-89` has two different parsing paths: `ExtractUserIDFromName` only accepts numeric user tokens, while `extractUserIdentifierFromName` accepts either token and is currently only used by `GetUser`.
- `server/router/api/v1/user_service.go:335-369`, `server/router/api/v1/user_service.go:372-460`, `server/router/api/v1/user_service.go:463-517`, `server/router/api/v1/user_service.go:536-676`, `server/router/api/v1/user_service.go:679-911`, and `server/router/api/v1/user_service.go:1400-1488` parse numeric user segments for settings, personal access tokens, webhooks, and notifications, and emit names such as `users/%d/settings/...`, `users/%d/webhooks/...`, and `users/%d/notifications/%d`.
- `server/router/api/v1/shortcut_service.go:20-43` parses `users/{user}/shortcuts/{shortcut}` by converting the `user` segment to `int32`, and constructs shortcut names as `users/%d/shortcuts/%s`.
- `server/router/api/v1/user_service_stats.go:63-65`, `server/router/api/v1/user_service_stats.go:113`, `server/router/api/v1/user_service_stats.go:132-145`, `server/router/api/v1/user_service_stats.go:214-223` emit `users/%d/stats` and `users/%d/memos/%d`, and resolve stats requests through numeric `ExtractUserIDFromName`.
- `server/router/api/v1/memo_service_converter.go:26-37` serializes `Memo.creator` as `users/{id}`; `server/router/api/v1/reaction_service.go:154-164` serializes `Reaction.creator` as `users/{id}`; `server/router/api/v1/memo_service.go:636-643` and `server/router/api/v1/memo_service.go:815-845` parse `memo.Creator` through the numeric helper for inbox and webhook flows.
- `server/router/mcp/tools_memo.go:75-86`, `server/router/mcp/tools_attachment.go:29-37`, and `server/router/mcp/tools_reaction.go:64-71` plus `server/router/mcp/tools_reaction.go:133-138` serialize creator fields as `users/{id}` in MCP tool output.
- `server/router/fileserver/fileserver.go:153-181` and `server/router/fileserver/fileserver.go:533-539` currently resolve avatar requests by either numeric ID or username.
- `proto/api/v1/user_service.proto:22-29` and `proto/api/v1/user_service.proto:247-256` document `GetUser` accepting both `users/{id}` and `users/{username}`. The same proto file defines the `User` resource at `proto/api/v1/user_service.proto:161-178` and nested user resource formats at `proto/api/v1/user_service.proto:307-317` and `proto/api/v1/user_service.proto:361-373`; example text still uses numeric user tokens such as `users/123/settings/GENERAL`.
- `web/src/pages/UserProfile.tsx:74-86` requests `users/{username}` from the route param, and `web/src/layouts/MainLayout.tsx:37-48` stores the returned canonical `user.name` for later stats requests.
## Non-Goals
- Replacing internal `user.id` primary keys, foreign keys, or existing store schemas.
- Introducing a new opaque UUID-based public user identifier.
- Changing user discovery, public profile visibility, or authorization rules beyond how user resource names are parsed and emitted.
- Adding username history, redirect, or alias preservation for old usernames after a rename.
- Redesigning unrelated resource naming schemes such as memo, attachment, share, or identity-provider identifiers.
## Open Questions
- Which public surfaces are in scope for username-based canonical output? (default: all server-emitted v1 API and MCP payload fields that currently contain `users/{...}` resource names)
- Should legacy numeric inputs continue to resolve on user-scoped endpoints beyond `GetUser`? (default: no, accept only username-based user resource names)
- If a username changes, must previously emitted `users/{old-username}` names continue to resolve? (default: no additional alias or redirect layer; only the current username remains valid)
- Should notification, webhook, shortcut, and personal-access-token child identifiers keep their existing child token formats while only the parent user token changes? (default: yes)
- Does the issue include avatar URLs and other derived file paths that are built from `User.name`? (default: yes, because avatar URLs are emitted from the same canonical user name field)
## Scope
**L** — Current behavior spans `server/router/api/v1`, `server/router/mcp`, `server/router/fileserver`, `proto/api/v1`, frontend consumers in `web/src`, and the request parsers that turn user resource names back into internal IDs. Changing both emitted and accepted user resource names across those surfaces is a broad API contract change rather than a single local edit.
@@ -0,0 +1,63 @@
## References
- [AIP-122: Resource names](https://google.aip.dev/122)
- [AIP-123: Resource types](https://google.aip.dev/123)
- [AIP-148: Standard fields](https://google.aip.dev/148)
- [AIP-180: Backwards compatibility](https://google.aip.dev/180)
- [Insecure Direct Object Reference Prevention Cheat Sheet](https://cheatsheetseries.owasp.org/cheatsheets/Insecure_Direct_Object_Reference_Prevention_Cheat_Sheet.html)
- [REST API endpoints for users - GitHub Docs](https://docs.github.com/en/enterprise-server%403.19/rest/users/users)
- [Users API - GitLab Docs](https://docs.gitlab.com/api/users/)
- [API Usage - Gitea Documentation](https://docs.gitea.com/next/development/api-usage)
## Industry Baseline
`AIP-122: Resource names` and `AIP-148: Standard fields` treat `name` as the canonical identifier that clients store and reuse, and expect request `name` and `parent` fields to accept the same resource-name vocabulary across a service. `AIP-122` also allows aliases for lookup, but requires responses to emit the canonical resource name.
`REST API endpoints for users - GitHub Docs` and `API Usage - Gitea Documentation` use username-based public user paths and nested user-scoped routes, while keeping numeric or system-assigned identifiers as separate data or alternate endpoints when a durable internal identifier is required.
`Users API - GitLab Docs` shows a mixed-input compatibility pattern on some endpoints with `id_or_username`, which keeps older callers working while allowing username-oriented public routes.
`Insecure Direct Object Reference Prevention Cheat Sheet` treats enumerable numeric identifiers as a defense-in-depth concern, but not a substitute for authorization. Replacing `users/{id}` with `users/{username}` changes discoverability characteristics, but permission checks still have to enforce access from internal user IDs.
`AIP-180: Backwards compatibility` treats changes to resource-name format and server-generated field construction as breaking. Any design that changes emitted `User.name` values inside `v1` has to preserve as much request compatibility as possible and document the remaining response-format risk explicitly.
## Research Summary
Memos already has most of the prerequisites for username-based canonical names. The schema stores a unique username, `GetUser` already resolves either ID or username, the fileserver avatar route already uses an `identifier` abstraction, and the frontend profile page already starts from `users/{username}`. No database migration is required to identify users by username at the API boundary.
The current coupling problem is concentrated in two places. First, response builders serialize `users/{id}` in many modules, including memo conversion, stats, settings, shortcuts, notifications, webhooks, and MCP JSON helpers. Second, many request handlers assume they can parse a numeric ID back out of those names for authorization and storage lookups.
Research points to a common pattern of canonical public resource names plus server-side resolution to internal IDs. In Memos, switching the canonical token from numeric ID to username can reuse the existing unique username column and existing username lookups, but `AIP-123: Resource types` and `AIP-180: Backwards compatibility` still make clear that changing accepted and emitted resource-name formats inside `v1` is a breaking API contract change. That makes this design a deliberate contract replacement rather than a compatibility layer.
## Design Goals
- All server-emitted v1 and MCP response fields that serialize user resource names under `users/{...}` use the current username token instead of the numeric database ID.
- User-scoped request fields that reference `users/{...}` accept username-based resource names only.
- Authorization, ownership checks, inbox/webhook dispatch, and other internal workflows continue to operate on `store.User.ID` after resolving the public resource name.
- List and batch endpoints avoid introducing per-item user lookups when serializing username-based names.
- No database schema, foreign-key, or storage-key redesign is required.
## Non-Goals
- Replacing internal `user.id` primary keys, foreign keys, or existing store schemas.
- Introducing a new opaque UUID-based public user identifier.
- Changing user discovery, public profile visibility, or authorization rules beyond how user resource names are parsed and emitted.
- Adding username history, redirect, or alias preservation for old usernames after a rename.
- Redesigning unrelated resource naming schemes such as memo, attachment, share, or identity-provider identifiers.
- Adding a new API version as part of this issue.
## Proposed Design
Introduce a single canonical user-name builder in the v1 API layer that serializes `users/{username}` from resolved user data, and route every public user-name emitter through it. This includes `convertUserFromStore`, memo and reaction creator fields, user stats, settings, shortcuts, webhooks, notifications, personal-access-token names, webhook payloads, avatar URLs derived from `User.name`, and the MCP JSON helpers. This satisfies the first design goal and aligns the public resource shape with `AIP-122: Resource names`.
Introduce a shared user-token resolver in `server/router/api/v1` that extracts the `users/{token}` segment, validates it as a username-form resource token, resolves the corresponding `store.User`, and then passes the resolved internal ID into permission checks and storage lookups. This replaces numeric-only parsing in helpers such as `ExtractUserIDFromName`, `ExtractUserIDAndSettingKeyFromName`, shortcut and webhook parsers, personal-access-token deletion, and notification parsing. The fileserver's current `getUserByIdentifier` behavior shows both lookup styles exist today, but the API-layer contract for this issue becomes username-only rather than dual-mode.
Keep child resource tokens unchanged and only change the user segment. For names such as `users/{user}/settings/{setting}`, `users/{user}/webhooks/{webhook}`, `users/{user}/notifications/{notification}`, `users/{user}/shortcuts/{shortcut}`, and `users/{user}/personalAccessTokens/{token}`, the parent `user` token is resolved from the username, while the child token keeps its existing format and storage mapping. This is narrower than redesigning child identifiers and keeps the issue bounded to the user-resource segment.
Use response-side user resolution strategies that match endpoint shape. Single-resource handlers can resolve one user directly and serialize the username immediately. List and batch handlers such as memo conversion, stats aggregation, notifications, and MCP list output should collect distinct user IDs first and resolve usernames once per response, reusing the store's existing user lookup path and cache where available. This keeps username-based output from turning into hidden N+1 query behavior and satisfies the performance goal without changing persistence.
Replace the public user-resource contract rather than extending it. Server-emitted `name`, `parent`, `creator`, and `sender` fields become username-based canonical output, and handlers that currently accept `users/{id}` are updated to require `users/{username}`. `AIP-180: Backwards compatibility` indicates that changing both the construction and accepted format of an existing resource name is a breaking change for clients that persist, compare, or generate old `name` values. The design therefore requires updated proto comments, API examples, handler tests, and release notes to make the new canonical form and the removed numeric form explicit.
Do not add a username alias table in this issue. If a username changes, newly serialized resource names use the current username, and previously emitted username-based names stop resolving unless they match the current username. This keeps the scope aligned with existing `UpdateUser` behavior and avoids introducing a new subsystem for historical username resolution. The alternative of adding permanent old-username aliases was rejected because it expands the problem from canonical serialization into identity-history management.
Do not solve this by adding a second public identifier field and leaving `User.name` numeric. `AIP-122: Resource names` treats `name` as the canonical resource identifier, and the GitHub issue is specifically about the public names currently emitted under `users/{id}`. Adding a second field would preserve the exposed sequential identifier in the canonical slot and fail the primary design goal. Likewise, introducing a new opaque UUID-based public identifier was rejected because the repository already has a unique username field and the issue is scoped to replacing numeric user resource names with that existing identifier.
@@ -0,0 +1,62 @@
## Execution Log
### T1: Add username-only user resource helpers
**Status**: Completed
**Files Changed**:
- `server/router/api/v1/user_resource_name.go`
- `server/router/api/v1/resource_name.go`
- `server/router/api/v1/user_service.go`
- `server/router/api/v1/test/user_resource_name_test.go`
**Validation**: `go test -v ./server/router/api/v1/test -run 'TestUserResourceName'` — PASS
**Path Corrections**: Tightened username-token validation so numeric-only `users/1` fails at the resource-name layer instead of falling through to `NotFound`.
**Deviations**: None
### T2: Migrate user-scoped API handlers
**Status**: Completed
**Files Changed**:
- `server/router/api/v1/user_service.go`
- `server/router/api/v1/shortcut_service.go`
- `server/router/api/v1/user_service_stats.go`
- `server/router/api/v1/test/shortcut_service_test.go`
- `server/router/api/v1/test/user_service_stats_test.go`
- `server/router/api/v1/test/user_notification_test.go`
- `server/router/api/v1/test/user_service_registration_test.go`
**Validation**: `go test -v ./server/router/api/v1/test -run 'Test(ListShortcuts|GetShortcut|CreateShortcut|UpdateShortcut|DeleteShortcut|ShortcutFiltering|ShortcutCRUDComplete|GetUserStats_TagCount|ListUserNotifications|UserRegistration)'` — PASS
**Path Corrections**: Updated test fixtures to use valid username-form resource names (`users/testuser`, `users/test-user`) and corrected one stale registration-name expectation during the later broader suite rerun.
**Deviations**: None
### T3: Migrate memo, reaction, MCP, and avatar user references
**Status**: Completed
**Files Changed**:
- `server/router/api/v1/memo_service_converter.go`
- `server/router/api/v1/memo_service.go`
- `server/router/api/v1/reaction_service.go`
- `server/router/mcp/tools_memo.go`
- `server/router/mcp/tools_attachment.go`
- `server/router/mcp/tools_reaction.go`
- `server/router/fileserver/fileserver.go`
- `server/router/api/v1/test/memo_service_test.go`
- `server/router/api/v1/test/reaction_service_test.go`
**Validation**: `go test ./server/router/api/v1/... ./server/router/mcp/... ./server/router/fileserver/...` — PASS
**Path Corrections**: Removed an unused fileserver import after the first package build failed; kept MCP tool helper signatures stable for undeclared callers and switched tool call sites to username-aware wrappers.
**Deviations**: None
### T4: Update contract docs and regression tests
**Status**: Completed
**Files Changed**:
- `proto/api/v1/user_service.proto`
- `proto/api/v1/shortcut_service.proto`
- `web/src/layouts/MainLayout.tsx`
- `web/src/components/MemoExplorer/ShortcutsSection.tsx`
- `server/router/fileserver/README.md`
**Validation**: `go test -v ./server/router/api/v1/test/...` — PASS
**Path Corrections**: None
**Deviations**: None
## Completion Declaration
All tasks completed successfully
@@ -0,0 +1,106 @@
## Task List
Task Index
T1: Add username-only user resource helpers [L] — T2: Migrate user-scoped API handlers [L] — T3: Migrate memo, reaction, MCP, and avatar user references [L] — T4: Update contract docs and regression tests [L]
### T1: Add username-only user resource helpers [L]
**Objective**: Establish one v1 API mechanism for serializing `users/{username}` and resolving username-based user resource names back to internal user records, including root `GetUser` handling.
**Size**: L (multiple files, shared identifier logic used across handlers)
**Files**:
- Create: `server/router/api/v1/user_resource_name.go`
- Modify: `server/router/api/v1/resource_name.go`
- Modify: `server/router/api/v1/user_service.go`
- Test: `server/router/api/v1/test/user_resource_name_test.go`
**Implementation**:
1. In `server/router/api/v1/user_resource_name.go`: add the shared helper surface for canonical user-name construction, extracting the `users/{token}` segment, validating the username-form token, and resolving the corresponding `store.User`.
2. In `server/router/api/v1/resource_name.go`: replace `ExtractUserIDFromName()`s numeric-only behavior with username-oriented resolution helpers or thin wrappers that delegate to the new shared module.
3. In `server/router/api/v1/user_service.go`: update `GetUser()` (~lines 72-102) and `convertUserFromStore()` (~lines 914-937) to use username-only resource names and reject legacy numeric `users/{id}` requests.
4. In `server/router/api/v1/test/user_resource_name_test.go`: add direct coverage for `GetUser users/{username}` success, canonical `User.name == users/{username}`, and rejection of `users/{id}`.
**Boundaries**: Do not migrate nested user-scoped handlers, memo/reaction emitters, MCP output, or fileserver behavior in this task.
**Dependencies**: None
**Expected Outcome**: Shared username-only helper logic exists, root user resources serialize as `users/{username}`, and root numeric user-name requests fail.
**Validation**: `go test -v ./server/router/api/v1/test -run 'TestUserResourceName'` — expected output includes `PASS` and `ok`
### T2: Migrate user-scoped API handlers [L]
**Objective**: Convert user-scoped v1 handlers and nested resource emitters to require `users/{username}` while continuing to authorize and store by resolved internal user ID.
**Size**: L (multiple handlers in one large service plus shortcut and stats code)
**Files**:
- Modify: `server/router/api/v1/user_service.go`
- Modify: `server/router/api/v1/shortcut_service.go`
- Modify: `server/router/api/v1/user_service_stats.go`
- Test: `server/router/api/v1/test/shortcut_service_test.go`
- Test: `server/router/api/v1/test/user_service_stats_test.go`
- Test: `server/router/api/v1/test/user_notification_test.go`
- Test: `server/router/api/v1/test/user_service_registration_test.go`
**Implementation**:
1. In `server/router/api/v1/user_service.go`: update settings, PAT, webhook, and notification parsing/emission paths (~lines 335-911 and ~1400-1488) to resolve `users/{username}` and emit username-based parent/child resource names.
2. In `server/router/api/v1/shortcut_service.go`: update shortcut name parsing and construction (~lines 20-43) plus handler entry points to use username parents and nested names.
3. In `server/router/api/v1/user_service_stats.go`: update stats request parsing and `UserStats.name` / `PinnedMemos` serialization (~lines 63-65, 113, 132-145, 214-223) to use usernames.
4. In the listed tests: replace numeric user-name inputs with username-based parents, assert username-based emitted names, and add numeric-request rejection coverage for representative user-scoped endpoints.
**Boundaries**: Do not change memo/reaction creator fields, MCP JSON output, or fileserver avatar routing in this task.
**Dependencies**: T1
**Expected Outcome**: User settings, notifications, shortcuts, stats, PATs, and webhooks all accept only `users/{username}` and emit only username-based user resource names.
**Validation**: `go test -v ./server/router/api/v1/test -run 'Test(ListShortcuts|GetShortcut|CreateShortcut|UpdateShortcut|DeleteShortcut|ShortcutFiltering|ShortcutCRUDComplete|GetUserStats_TagCount|ListUserNotifications|UserRegistration)'` — expected output includes `PASS` and `ok`
### T3: Migrate memo, reaction, MCP, and avatar user references [L]
**Objective**: Remove numeric user resource names from memo/reaction-related API responses, dependent webhook/inbox flows, MCP JSON output, and avatar URLs/routing.
**Size**: L (cross-package serialization and lookup changes, including response-side user resolution)
**Files**:
- Modify: `server/router/api/v1/memo_service_converter.go`
- Modify: `server/router/api/v1/memo_service.go`
- Modify: `server/router/api/v1/reaction_service.go`
- Modify: `server/router/mcp/tools_memo.go`
- Modify: `server/router/mcp/tools_attachment.go`
- Modify: `server/router/mcp/tools_reaction.go`
- Modify: `server/router/fileserver/fileserver.go`
- Test: `server/router/api/v1/test/memo_service_test.go`
- Test: `server/router/api/v1/test/reaction_service_test.go`
**Implementation**:
1. In `server/router/api/v1/memo_service_converter.go`: update `convertMemoFromStore()` (~lines 16-73) to serialize `Memo.creator` from resolved usernames rather than numeric IDs, using response-side batching or shared lookup helpers so list responses do not regress into hidden per-item lookups.
2. In `server/router/api/v1/reaction_service.go`: update `convertReactionFromStore()` (~lines 154-164) to emit username-based creators.
3. In `server/router/api/v1/memo_service.go`: update memo comment, webhook dispatch, and webhook payload helpers (~lines 636-643 and 815-845) to resolve username-based memo creators before using internal IDs.
4. In `server/router/mcp/tools_memo.go`, `server/router/mcp/tools_attachment.go`, and `server/router/mcp/tools_reaction.go`: replace `users/%d` creator serialization with username-based values.
5. In `server/router/fileserver/fileserver.go`: change avatar lookup to accept username identifiers only and ensure avatar URLs derived from `User.name` continue to resolve under `users/{username}`.
6. In the listed tests: update creator assertions to `users/{username}` and add representative rejection coverage where numeric user names previously flowed through memo/reaction-related paths.
**Boundaries**: Do not update proto comments, README examples, or frontend comments in this task.
**Dependencies**: T1
**Expected Outcome**: Memo/reaction creators, webhook payload creators, MCP creator fields, and avatar-derived user paths no longer expose numeric user IDs.
**Validation**: `go test ./server/router/api/v1/... ./server/router/mcp/... ./server/router/fileserver/...` — expected output includes `ok` for all touched packages
### T4: Update contract docs and regression tests [L]
**Objective**: Align public contract comments/examples and the final regression suite with the username-only user resource-name contract.
**Size**: L (multiple contract/documentation files plus end-to-end regression coverage)
**Files**:
- Modify: `proto/api/v1/user_service.proto`
- Modify: `proto/api/v1/shortcut_service.proto`
- Modify: `web/src/layouts/MainLayout.tsx`
- Modify: `web/src/components/MemoExplorer/ShortcutsSection.tsx`
- Modify: `server/router/fileserver/README.md`
- Modify: `server/router/api/v1/test/user_resource_name_test.go`
- Modify: `server/router/api/v1/test/shortcut_service_test.go`
- Modify: `server/router/api/v1/test/user_service_stats_test.go`
- Modify: `server/router/api/v1/test/user_notification_test.go`
- Modify: `server/router/api/v1/test/memo_service_test.go`
- Modify: `server/router/api/v1/test/reaction_service_test.go`
- Modify: `server/router/api/v1/test/user_service_registration_test.go`
**Implementation**:
1. In `proto/api/v1/user_service.proto` and `proto/api/v1/shortcut_service.proto`: rewrite resource-name comments and examples so they document username-only user resource names and remove `users/{id}` examples.
2. In `web/src/layouts/MainLayout.tsx` and `web/src/components/MemoExplorer/ShortcutsSection.tsx`: update inline comments/examples that still describe numeric user resource names.
3. In `server/router/fileserver/README.md`: replace numeric avatar examples with username-based examples.
4. In the listed test files: finish any remaining request/response assertions so the suite consistently encodes the username-only contract and explicitly rejects numeric user resource names where that contract is externally visible.
**Boundaries**: Do not add schema migrations, generated proto output refreshes, or username-history behavior.
**Dependencies**: T2, T3
**Expected Outcome**: Source comments, examples, and regression tests all describe and enforce a username-only `users/{username}` public contract.
**Validation**: `go test -v ./server/router/api/v1/test/...` — expected output includes `PASS` and `ok`
## Out-of-Scope Tasks
- Database schema or migration changes for the `user` table or foreign keys.
- Username history, alias, redirect, or backward-compatibility layers.
- A new opaque public user identifier or a new API version.
- Opportunistic refactors outside the files listed above.
- Generated code refreshes (`buf generate`) unless a later approved plan revision explicitly requires schema changes.
@@ -0,0 +1,47 @@
## Background & Context
Memos is a self-hosted note-taking product whose main write path is the React memo composer in `web/src/components/MemoEditor`. Memo content is stored as Markdown text, attachments are uploaded through the v1 attachment API, and the server already has dedicated file-serving behavior for media playback. The most recent relevant change in this area was commit `63a17d89`, which refactored audio attachment rendering into reusable playback components. That change improved how audio files are displayed after upload; it did not add a microphone-driven input path inside the compose flow.
## Issue Statement
Memo creation currently starts from typed text plus file upload and metadata pickers, while audio support in the product begins only after an audio file already exists as an attachment. Users who want to capture memo content by speaking must leave the compose flow to record elsewhere, then upload or manually transcribe the result, because the editor has no direct path from microphone input to memo text or an in-progress audio attachment.
## Current State
- `web/src/components/MemoEditor/index.tsx:26-154` assembles the compose flow from `EditorContent`, `EditorMetadata`, and `EditorToolbar`, and persists drafts through `memoService.save`.
- `web/src/components/MemoEditor/Editor/index.tsx:27-214` implements the editor surface as a `<textarea>` with slash commands and tag suggestions. It has no microphone entrypoint, recording lifecycle, or transcript state.
- `web/src/components/MemoEditor/components/EditorToolbar.tsx:10-54` renders the bottom toolbar with `InsertMenu`, visibility, cancel, and save actions. There is no first-class voice action in the primary control row.
- `web/src/components/MemoEditor/Toolbar/InsertMenu.tsx:40-189` exposes upload, link-memo, location, and focus-mode actions, and uses a hidden `<input type="file">` for attachments. It does not expose microphone capture or dictation.
- `web/src/components/MemoEditor/components/EditorContent.tsx:12-54` handles drag-and-drop and paste for binary files only, and `web/src/components/MemoEditor/hooks/useFileUpload.ts:4-33` handles file-picker selection only.
- `web/src/components/MemoEditor/state/types.ts:8-30`, `web/src/components/MemoEditor/state/actions.ts:6-78`, and `web/src/components/MemoEditor/state/reducer.ts:4-130` track memo text, metadata, local files, and loading flags. There is no state for microphone permission, recording mode, partial transcript, cleanup review, or a pending audio blob.
- `web/src/components/MemoEditor/hooks/useAutoSave.ts:4-8` saves only the current `content` string to local storage. There is no draft persistence model for an in-progress voice session.
- `web/src/components/MemoEditor/services/validationService.ts:9-30` allows save when the draft has text, saved attachments, or local files, and `web/src/components/MemoEditor/services/uploadService.ts:8-26` uploads local files to `AttachmentService`. This means the existing save path can already persist an audio blob if one is present as a `LocalFile`.
- `web/src/components/MemoEditor/types/attachment.ts:4-28` classifies editor-side files only as `image`, `video`, or `document`, so an unsaved audio recording would currently fall into the generic document path in the editor draft surface.
- `web/src/utils/attachment.ts:15-38` recognizes `audio/*`, `web/src/components/MemoMetadata/Attachment/AttachmentListView.tsx:98-130` groups persisted attachments into visual/audio/docs sections, and `web/src/components/MemoMetadata/Attachment/AudioAttachmentItem.tsx:48-173` renders the dedicated audio playback card added by the last commit.
- `server/server.go:71-74` and `server/router/fileserver/fileserver.go:120-149,187-214` already treat video/audio attachments as native HTTP media streams once an attachment exists.
- `proto/api/v1/attachment_service.proto:48-90` and `server/router/api/v1/attachment_service.go:64-167` define binary attachment upload and metadata only. There is no transcription request/response shape, language hint, transcript cleanup option, or voice-session metadata in the API.
- `proto/api/v1/memo_service.proto:176-245` defines memo content as a single Markdown string plus optional attachments and relations. There is no separate speech transcript field or audio-note abstraction in the memo resource.
- `proto/api/v1/instance_service.proto:56-90` and `server/router/api/v1/instance_service.go:36-139` expose instance settings for `GENERAL`, `STORAGE`, `MEMO_RELATED`, `TAGS`, and `NOTIFICATION` only. There is no speech-provider or transcription-retention configuration surface.
- No existing implementation found for `getUserMedia`, `MediaRecorder`, browser speech recognition, or server-side transcription anywhere under `web/src`, `server`, `proto`, `plugin`, or `store`.
## Non-Goals
- Redesigning the current persisted audio attachment playback UI introduced in commit `63a17d89`.
- Building a full duplex spoken assistant or chatbot response loop inside Memos.
- Replacing the Markdown textarea editor with a different editor architecture.
- Shipping native desktop or mobile OS integrations such as global system-wide hotkeys.
- Redesigning attachment storage backends or the general file upload pipeline beyond voice-related usage.
- Adding broad AI rewrite/edit commands unrelated to capturing spoken memo text into the current draft.
## Open Questions
- Which client surfaces are in scope for the first rollout? (default: the existing React memo composer in the web app, including touch-friendly mobile-browser behavior)
- Is the first release a conversational voice mode or a dictation workflow? (default: dictation-first voice capture that inserts text into the current memo draft rather than opening a separate assistant session)
- Should Memos retain the raw recording after transcription? (default: no by default; keeping the recording is an explicit user choice that stores it as a normal attachment)
- Where does transcription execute? (default: behind a server-owned API so behavior, provider choice, and privacy copy are instance-controlled rather than browser-vendor specific)
- How much transcript cleanup is in scope? (default: punctuation plus limited filler/self-correction cleanup, with a review step before insertion)
- Does this issue include spoken edit commands such as “rewrite this shorter”? (default: no, only spoken text capture and insertion or replacement)
## Scope
**L** — The current gap spans the memo composer UI, editor state model, local file preview behavior, attachment save path, public API surface, and instance settings. There is no existing microphone or transcription implementation to extend, and a complete voice-input workflow would introduce both a new client interaction model and a new server contract rather than a single local edit.
@@ -0,0 +1,58 @@
## References
- [OpenAI Help: ChatGPT Release Notes](https://help.openai.com/en/articles/6825453-chatgpt-release-notes%3F.ejs)
- [Anthropic Support: Using voice mode on Claude mobile apps](https://support.anthropic.com/en/articles/11101966-using-voice-mode-on-claude-mobile-apps)
- [Typeless](https://www.typeless.com/)
- [Typeless FAQ](https://www.typeless.com/help/faqs)
- [DeltaCircuit/react-media-recorder README](https://github.com/DeltaCircuit/react-media-recorder/blob/master/README.md)
## Industry Baseline
ChatGPT, Claude, and Typeless all treat voice capture as a first-class entrypoint near the main compose surface rather than as a secondary attachment action. The common pattern is immediate access to the microphone, visible recording state, and explicit stop/discard control.
Those products also keep the capture loop short. The user starts recording, sees a clear recording state, and either keeps or cancels the result. Even when the product supports richer voice features, the initial interaction cost stays low.
The `react-media-recorder` reference reflects the common browser implementation pattern behind that interaction: explicit recorder states, start/stop commands, generated blob URLs, and preview playback of the recorded media. That maps well to the current Memos editor because the editor already knows how to persist local files through the attachment upload path.
## Research Summary
The current Memos composer already has the downstream path needed for recorded audio files: local draft files can be attached in the editor, `uploadService` can persist them through `AttachmentService`, and persisted audio attachments already have dedicated playback UI. What is missing is the upstream capture step inside the composer.
Given the revised scope, the right fit is not dictation or voice chat. The immediate problem is only that users cannot create an audio file from the memo composer itself. That means the smallest useful design is a browser voice recorder that produces a normal draft attachment.
Because the scope is now frontend-only, the design should not introduce new server contracts, new instance settings, or any transcription workflow. The recorded clip should flow through the existing `LocalFile -> uploadService -> attachment` path exactly like other draft files.
## Design Goals
- A user can start recording from the memo composer in one explicit action without opening the attachment menu.
- Recording state is visible and explicit: idle, requesting permission, recording, recorded, unsupported, or error.
- A completed recording can be kept as a draft audio file or discarded before memo save.
- Kept recordings reuse the existing local-file and attachment-save flow with no backend changes.
- The draft attachment surface renders recorded audio as playable audio, not as a generic document row.
- Save is blocked while recording is actively in progress, but succeeds once a recording has been stopped and kept.
## Non-Goals
- Transcribing audio to text.
- Adding any proto, store, server, or instance-settings changes.
- Building a spoken assistant or voice-chat mode.
- Redesigning persisted audio playback beyond the minimum draft preview needed for local recordings.
- Adding background recording, global hotkeys, or native device integrations.
## Proposed Design
Add a `Voice note` entry to `web/src/components/MemoEditor/Toolbar/InsertMenu.tsx` rather than keeping a separate always-visible mic button in `web/src/components/MemoEditor/components/EditorToolbar.tsx`. This keeps the recorder close to the existing file and metadata insertion actions, reduces toolbar clutter, and still gives the user a direct path to create an audio attachment from the composer.
Introduce a `VoiceRecorderPanel` inside the memo editor layout, rendered between the editor body and the bottom metadata/toolbar area in `web/src/components/MemoEditor/index.tsx`. The panel is responsible for showing recorder state and actions, but it does not alter memo content. Its job is only to create or discard a draft audio file.
Add a dedicated `voiceRecorder` slice to `web/src/components/MemoEditor/state/types.ts` and `web/src/components/MemoEditor/state/reducer.ts`. The slice should hold support state, permission state, recorder status, elapsed time, pending error, and the most recent recorded draft clip before the user decides to keep or discard it. Keeping this state separate from `content`, `metadata`, and `localFiles` prevents recorder lifecycle state from leaking into unrelated editor behaviors.
Implement the browser media lifecycle in a dedicated hook such as `useVoiceRecorder`, following the `MediaRecorder` state pattern shown in `react-media-recorder`. The hook owns capability detection, `getUserMedia` requests, recorder start/stop, elapsed-time updates, blob assembly, and cleanup of tracks and blob URLs. The editor UI consumes only the hook output and dispatches reducer actions from it.
When the user stops a recording, convert the captured blob into a `File` and then into the existing `LocalFile` shape already used by file upload, paste, and drag-and-drop flows. The user can then either keep that `LocalFile`, which appends it to `state.localFiles`, or discard it, which revokes the blob URL and clears the recorder state. This keeps the design aligned with the existing upload path and avoids introducing a parallel attachment model.
Extend `web/src/components/MemoEditor/types/attachment.ts` so local `audio/*` files are classified as audio rather than falling back to `document`. Then update `web/src/components/MemoMetadata/Attachment/AttachmentListEditor.tsx` so draft audio files render with playable audio controls and normal remove behavior. This reuses the recent product investment in better audio presentation without requiring persisted attachments before preview is possible.
Update `web/src/components/MemoEditor/services/validationService.ts` so save remains allowed for normal local files and kept audio drafts, but not while `voiceRecorder.status` is actively `recording` or `requesting_permission`. That avoids saving a memo in the middle of a live recording session while preserving the existing rule that a memo may be saved with attachments and no text.
Do not introduce transcription, transcript review, speech-provider configuration, or server upload-before-save behavior in this design. Those alternatives were intentionally rejected because they expand the problem from “create an audio file quickly” into a larger speech-input subsystem. The current narrowed issue only requires fast recording and clean integration with the existing attachment flow.
@@ -0,0 +1,40 @@
## Execution Log
### T1: Add recorder state and browser capture hook
**Status**: Completed
**Files Changed**:
- `web/src/components/MemoEditor/hooks/useVoiceRecorder.ts`
- `web/src/components/MemoEditor/hooks/index.ts`
- `web/src/components/MemoEditor/state/types.ts`
- `web/src/components/MemoEditor/state/actions.ts`
- `web/src/components/MemoEditor/state/reducer.ts`
- `web/src/components/MemoEditor/services/memoService.ts`
**Validation**: `cd web && pnpm lint` — PASS
**Path Corrections**: Added `web/src/components/MemoEditor/services/memoService.ts` after plan update because `memoService.fromMemo()` also constructs `EditorState`.
**Deviations**: None after the approved plan correction.
Implemented a dedicated `voiceRecorder` editor state slice, reducer/actions for recorder lifecycle updates, a browser `MediaRecorder` hook that produces a `LocalFile` preview, and the matching `fromMemo()` defaults needed to keep the editor state shape valid for existing memo edit flows.
### T2: Add composer recorder UI and draft audio handling
**Status**: Completed
**Files Changed**:
- `web/src/components/MemoEditor/components/VoiceRecorderPanel.tsx`
- `web/src/components/MemoEditor/components/EditorToolbar.tsx`
- `web/src/components/MemoEditor/components/index.ts`
- `web/src/components/MemoEditor/index.tsx`
- `web/src/components/MemoEditor/types/components.ts`
- `web/src/components/MemoEditor/types/attachment.ts`
- `web/src/components/MemoMetadata/Attachment/AttachmentListEditor.tsx`
- `web/src/components/MemoEditor/services/validationService.ts`
- `web/src/locales/en.json`
**Validation**: `cd web && pnpm lint` — PASS
**Path Corrections**: None
**Deviations**: None
Added a `Voice note` action to the editor tool dropdown, wired the memo editor to start recording and render an inline recorder/review panel, let users keep a completed clip as a normal draft `LocalFile`, rendered local audio drafts with playable controls in the attachment editor, and blocked save only while permission is pending or recording is live.
## Completion Declaration
**Execution completed successfully** — the frontend memo composer now has a tool-dropdown voice recorder entrypoint that creates draft audio files through the existing attachment flow, with no backend or transcription changes.
@@ -0,0 +1,63 @@
## Task List
Task Index
T1: Add recorder state and browser capture hook [M] — T2: Add composer recorder UI and draft audio handling [L]
### T1: Add recorder state and browser capture hook [M]
**Objective**: Introduce the frontend-only state and hook needed to record audio in the browser and convert the finished clip into the existing `LocalFile` draft format.
**Size**: M (2-3 files, moderate logic)
**Files**:
- Create: `web/src/components/MemoEditor/hooks/useVoiceRecorder.ts`
- Modify: `web/src/components/MemoEditor/state/types.ts`
- Modify: `web/src/components/MemoEditor/state/actions.ts`
- Modify: `web/src/components/MemoEditor/state/reducer.ts`
- Modify: `web/src/components/MemoEditor/hooks/index.ts`
- Modify: `web/src/components/MemoEditor/services/memoService.ts`
**Implementation**:
1. In `web/src/components/MemoEditor/state/types.ts`, add a `voiceRecorder` state slice for recorder support, permission, status, elapsed seconds, pending error, and the latest temporary recording preview.
2. In `web/src/components/MemoEditor/state/actions.ts`, add actions for support/permission updates, recorder status changes, timer updates, temporary recording storage, and recorder reset.
3. In `web/src/components/MemoEditor/state/reducer.ts`, implement the new voice-recorder actions without changing existing content, metadata, or save behavior.
4. In new `web/src/components/MemoEditor/hooks/useVoiceRecorder.ts`, implement browser capability detection, `getUserMedia`, `MediaRecorder` setup, start/stop lifecycle, blob collection, cleanup, and conversion of the stopped recording into a `File` plus preview URL compatible with `LocalFile`.
5. In `web/src/components/MemoEditor/services/memoService.ts`, update `fromMemo()` so loaded memo state includes the new `voiceRecorder` defaults required by `EditorState`.
6. In `web/src/components/MemoEditor/hooks/index.ts`, export the new hook for editor integration.
**Boundaries**: This task must not add any toolbar/panel UI, attachment rendering updates, or transcription/network behavior.
**Dependencies**: None
**Expected Outcome**: The memo editor has a recorder state model and a reusable browser recording hook that can produce a draft audio file.
**Validation**: `cd web && pnpm lint` — expected output: TypeScript and Biome checks pass.
### T2: Add composer recorder UI and draft audio handling [L]
**Objective**: Add a voice-recorder entry inside the composer tool dropdown and make kept recordings behave like draft audio attachments in the existing save flow.
**Size**: L (multiple files, coordinated UI/state integration)
**Files**:
- Create: `web/src/components/MemoEditor/components/VoiceRecorderPanel.tsx`
- Modify: `web/src/components/MemoEditor/index.tsx`
- Modify: `web/src/components/MemoEditor/components/EditorToolbar.tsx`
- Modify: `web/src/components/MemoEditor/components/index.ts`
- Modify: `web/src/components/MemoEditor/types/components.ts`
- Modify: `web/src/components/MemoEditor/types/attachment.ts`
- Modify: `web/src/components/MemoMetadata/Attachment/AttachmentListEditor.tsx`
- Modify: `web/src/components/MemoEditor/services/validationService.ts`
- Modify: `web/src/locales/en.json`
**Implementation**:
1. In `web/src/components/MemoEditor/Toolbar/InsertMenu.tsx` and `web/src/components/MemoEditor/components/EditorToolbar.tsx`, add a `Voice note` action to the existing compose tool dropdown instead of a separate toolbar button.
2. In new `web/src/components/MemoEditor/components/VoiceRecorderPanel.tsx`, render the recorder states `unsupported`, `idle`, `requesting_permission`, `recording`, `recorded`, and `error`, with explicit `Start`, `Stop`, `Keep`, and `Discard` actions.
3. In `web/src/components/MemoEditor/index.tsx`, render the recorder panel between editor content and the metadata/toolbar group, wire it to the editor context, and on `Keep` append the produced `LocalFile` to `state.localFiles`.
4. In `web/src/components/MemoEditor/types/attachment.ts`, classify local `audio/*` files as audio instead of generic documents.
5. In `web/src/components/MemoMetadata/Attachment/AttachmentListEditor.tsx`, render local draft audio items with playable audio controls while preserving existing remove behavior and existing attachment reordering rules.
6. In `web/src/components/MemoEditor/services/validationService.ts`, block save while a recording is actively running or permission is still being requested, but continue to allow save for kept draft audio files.
7. In `web/src/components/MemoEditor/components/index.ts`, `web/src/components/MemoEditor/types/components.ts`, and `web/src/locales/en.json`, add the exports, prop types, and English labels needed for the recorder UI.
**Boundaries**: This task must not add transcription, backend/API calls, settings UI, or redesign persisted audio playback beyond local draft preview.
**Dependencies**: T1
**Expected Outcome**: A user can choose `Voice note` from the memo composer tool dropdown, record audio in the browser, keep or discard the clip, preview a kept clip as a draft audio attachment, and save it through the existing attachment upload path.
**Validation**: `cd web && pnpm lint` — expected output: TypeScript and Biome checks pass with the new recorder workflow.
## Out-of-Scope Tasks
- Any transcription or speech-to-text behavior.
- Any proto, store, server, or instance-settings changes.
- Any speech provider configuration.
- Assistant-style voice conversations or spoken edit commands.
- Full locale backfill beyond the required English copy for this feature.
@@ -0,0 +1,49 @@
## Background & Context
Memos stores memo bodies as markdown, rebuilds derived memo metadata into `MemoPayload`, exposes user notifications through the inbox model, and renders memo content in the React client with custom markdown plugins. The requested `@someone` feature spans both top-level memos and memo comments: users need to type `@`, pick a valid person, render the mention inline, and notify the mentioned user. The current product already has adjacent primitives for this work: a backend markdown extension for `#tag`, an inbox-backed notification center, a generic editor suggestion popup, public user profiles under username-based routes, and a memo update path that already rebuilds payloads on create and edit.
External product behavior is consistent on the core interaction but different on scope. Notion supports real-time `@` suggestions inside pages, comments, and discussions, stores mention notifications in an inbox, and suppresses notification if the mentioned user cannot access the content. Confluence supports autocomplete mentions for people and teams, sends a notification on the first mention, and does not keep notifying on repeated mentions in the same page. Coda supports `@` mentions inside comment threads, treats mentions and thread participation as notification triggers, and allows broader comment-subscription settings beyond explicit mentions. These patterns suggest that the common baseline for Memos is inline autocomplete, access-aware notification, deduplication, and a clear separation between mention notifications and broader thread-subscription features.
## Issue Statement
Memos does not currently recognize `@username` tokens as structured content in memo bodies or comment bodies, does not expose any non-admin user-search endpoint that the editor can use to suggest mentionable users, does not persist or diff mention metadata during memo create or update flows, and does not have an inbox or API notification type for mentions. As a result, `@someone` currently behaves as plain text and cannot drive inline rendering, target validation, or notification delivery.
## Current State
- `server/router/api/v1/memo_service.go:32-159` creates memos by copying raw `request.Memo.Content` into `store.Memo`, enforcing length limits, and calling `memopayload.RebuildMemoPayload`; `server/router/api/v1/memo_service.go:404-510` rebuilds payload only when `content` changes during memo updates.
- `server/router/api/v1/memo_service.go:590-681` creates memo comments by internally creating another memo and only generates inbox notifications for non-private comments to the parent memo creator via `InboxMessage_MEMO_COMMENT`.
- `server/router/api/v1/memo_update_helpers.go:27-77` only dispatches webhook and SSE side effects after memo updates; there is no mention-diff side-effect hook.
- `internal/markdown/markdown.go:20-24` defines extracted markdown metadata as `Tags` plus `Property`; `internal/markdown/markdown.go:68-89` only wires the custom tag extension; `internal/markdown/markdown.go:324-386` extracts tags and properties but no mention metadata.
- `internal/markdown/extensions/tag.go:13-23` and the related tag parser/AST types are the only custom inline markdown extension path today.
- `proto/store/memo.proto:7-29` limits `MemoPayload` to `property`, `location`, and `tags`; there is no repeated mention field or structured mention metadata.
- `proto/store/inbox.proto:7-24` defines only `InboxMessage_MEMO_COMMENT`; `proto/api/v1/user_service.proto:592-679` defines only `UserNotification_MEMO_COMMENT`.
- `server/router/api/v1/user_service.go:1272-1312` lists notifications by filtering inbox rows to `InboxMessage_MEMO_COMMENT` only; `server/router/api/v1/user_service.go:1433-1524` converts only that message type into API notifications.
- `web/src/pages/Inboxes.tsx:19-114` and `web/src/components/Inbox/MemoCommentMessage.tsx` only render memo comment notifications; other notification types would currently be dropped.
- `server/router/api/v1/user_service.go:32-70` exposes `ListUsers` only to admins, and `store/user.go:59-74` plus `store/db/sqlite/user.go:88-175` support exact-match user filtering but no general search, ranking, or pagination for mention autocomplete.
- `server/router/api/v1/acl_config.go:20-27` whitelists `/memos.api.v1.UserService/SearchUsers`, but `proto/api/v1/user_service.proto:16-120` does not define a `SearchUsers` RPC and there is no server implementation.
- `web/src/components/MemoEditor/Editor/index.tsx:189-214`, `web/src/components/MemoEditor/Editor/useSuggestions.ts:28-158`, and `web/src/components/MemoEditor/Editor/TagSuggestions.tsx:10-49` provide a reusable textarea suggestion popup, but it is only instantiated for `#tag`.
- `web/src/components/MemoContent/index.tsx:53-136`, `web/src/utils/remark-plugins/remark-tag.ts:24-112`, and `web/src/components/MemoContent/Tag.tsx` parse and render `#tag` as a structured inline element; there is no `remarkMention` equivalent.
- `web/src/hooks/useUserQueries.ts:176-245` has `useListUsers()` for admin listing and `useUsersByNames()` for fetching known usernames one by one, but nothing that returns ranked candidates for an in-editor `@` query.
- `web/src/router/index.tsx:65-72` already routes public user profiles at `u/:username`, so inline mention rendering can target username-based profile URLs without inventing a new frontend route.
## Non-Goals
- Adding group mentions, team mentions, page mentions, or date mentions.
- Building a general “watch this memo/thread” subscription system beyond explicit mentions.
- Adding email, push, Slack, or webhook delivery for mentions in this issue.
- Redesigning memo visibility, access control, or per-user sharing semantics.
- Making old mentions follow username changes automatically.
- Redesigning the editor away from the current textarea-based implementation.
## Open Questions
- Which content surfaces are in scope for `@mention`? (default: top-level memos and memo comments, because both already share the same memo content pipeline)
- What mention token syntax should be recognized? (default: `@username` only, using canonical usernames rather than display names)
- Should edits trigger mention notifications after the initial create? (default: yes, but only for newly added mention targets compared with the memos previous mention set)
- What happens if someone types `@username` in content the target cannot access? (default: render the token as a mention in the authors view, but do not send a notification unless the target can already access the memo/comment under existing visibility rules)
- Should mentioning yourself create an inbox item? (default: no, because self-mentions do not require attention routing)
- Should the mention candidate API be public like `GetUser`, or authenticated like the editor? (default: authenticated only, because ranked user search is a broader directory-enumeration surface than fetching a known public profile)
## Scope
**L** — The work crosses markdown parsing, memo payload extraction, memo create/update side effects, inbox and notification protos, user search APIs, three SQL drivers, React editor autocomplete, markdown rendering, and inbox UI. The repository already contains adjacent pieces for tags and comment notifications, but `@mention` requires stitching several existing subsystems together rather than extending a single isolated module.
@@ -0,0 +1,69 @@
## References
- [Comments, mentions & reactions - Notion Help Center](https://www.notion.com/help/comments-mentions-and-reminders)
- [Notification settings - Notion Help Center](https://www.notion.com/help/notification-settings)
- [Mention a person or team - Confluence Cloud](https://support.atlassian.com/confluence-cloud/docs/mention-a-person-or-team/)
- [Comment on Coda docs - Coda Help](https://help.coda.io/hc/en-us/articles/39555917053069-Comment-on-Coda-docs)
- [Customize notifications from comments - Coda Help](https://help.coda.io/hc/en-us/articles/39555901119117-Customize-notifications-from-comments)
## Industry Baseline
`Comments, mentions & reactions - Notion Help Center` shows the most common editor-side behavior: typing `@` triggers real-time search, mentions can live inline in page bodies and comments, clicking an inbox item takes the user back to the exact context, and no notification is sent when the target cannot access the page. `Notification settings - Notion Help Center` also separates in-product inbox behavior from secondary delivery like desktop or email.
`Mention a person or team - Confluence Cloud` adds two useful guardrails for a collaborative editor: autocomplete suggestions appear directly from `@`, and notifications are intentionally deduplicated so people are notified on the first mention rather than on every repeated mention in the same page.
`Comment on Coda docs` and `Customize notifications from comments` show a narrower scope for mentions inside comments, but reinforce two patterns that matter for Memos: explicit `@` mentions are a distinct notification trigger from generic participation, and products often keep mention notifications separate from broader thread-subscription or owner-subscription rules.
Across these products, the default implementation is not “parse arbitrary display text and hope it matches a user.” The stable interaction is: search among valid workspace members, insert a canonical mention token, render it differently from plain text, and only notify when access and deduplication rules say the event is meaningful.
## Research Summary
Memos already has the right extension points to adopt that baseline without a storage redesign. The backend has a custom inline markdown extension pipeline for `#tag`, memo create and update both rebuild `MemoPayload`, and the inbox model already represents user-facing attention items. The frontend editor already has a trigger-character suggestion popup, the markdown renderer already recognizes custom inline nodes, and public user profiles are already routed by username.
The biggest mismatch is user discovery. The current `ListUsers` path is admin-only and exact-match oriented, while mention autocomplete needs a normal authenticated user search API that can return ranked candidates by username and display name. The second mismatch is notification shape: the inbox and API layers only understand memo-comment notifications today, so a mention feature cannot be expressed as a first-class notification without extending the inbox proto and inbox UI.
Research also suggests that Memos should stay narrower than Notion or Confluence. There is no existing concept of teams, group mentions, page mentions, or per-page ACLs. The codebase already treats usernames as the public user token and memo visibility as a coarse `PUBLIC/PROTECTED/PRIVATE` rule. The best fit is therefore person mentions only, keyed by canonical username, with notification rules that are access-aware and deduplicated across repeated edits.
## Design Goals
- Typing `@` in the memo editor or comment editor shows ranked, authenticated user candidates and inserts a canonical `@username` token on selection.
- The backend extracts mention targets from memo/comment content during create, update, and payload rebuild, and produces the same mention set for equivalent content across all supported databases.
- Mention notifications are created only for newly added targets, at most once per target per memo revision, and never for self-mentions or inaccessible private content.
- Memo content renders resolved mentions as interactive inline entities and degrades unresolved tokens to plain text.
- The inbox API and inbox UI expose mention notifications as a first-class type distinct from comment notifications.
- The design does not require a relational schema migration; it only extends existing proto-backed JSON payloads and server/frontend code paths.
## Non-Goals
- Adding group mentions, team mentions, page mentions, or date mentions.
- Building a generic watch/subscription system for memo activity.
- Sending mention notifications through email, push, Slack, or webhooks.
- Making mention references survive username changes automatically.
- Replacing the textarea editor with a richer block editor.
- Redesigning memo visibility or introducing user-level memo sharing.
## Proposed Design
Support only canonical `@username` mentions in this issue. The parser should recognize the same username token vocabulary that the API already accepts for public user names, instead of trying to match display names or arbitrary free text. This keeps mention authoring aligned with existing user resource naming and avoids ambiguous matches when multiple users share similar display names. Mention suggestions may show both display name and username, but the inserted source text remains `@username`.
Add a backend markdown mention extension parallel to the existing tag extension. Introduce `internal/markdown/ast.MentionNode`, `internal/markdown/parser.NewMentionParser()`, and `internal/markdown/extensions.MentionExtension`, then wire it into `internal/markdown/markdown.go` next to `TagExtension`. The mention parser should require a word boundary before `@` so email addresses and URLs do not become mentions, and it should normalize the captured token to lowercase before lookup because usernames are canonicalized that way in the API layer.
Extend `storepb.MemoPayload` with a repeated mention metadata field, for example `repeated Mention mentions`, where each item stores at least `username` and resolved `user_id`. The raw markdown remains the source of truth for author-visible text, but the payload becomes the normalized server-side mention set for diffing and notification decisions. This reuses the existing memo payload rebuild path and avoids reparsing memo bodies in multiple side-effect handlers. No SQL migration is required because memo payloads are already stored as proto-backed JSON blobs in each database driver.
Teach `memopayload.RebuildMemoPayload` to resolve mention metadata while rebuilding tags and properties. The extraction step should walk the markdown AST once, collect raw `@username` tokens, resolve them to active users via the store, deduplicate by `user_id`, and populate `memo.Payload.Mentions`. Unresolved usernames should not fail memo creation; they should simply be omitted from normalized mention metadata so the feature remains tolerant of free-typed text. This mirrors how the frontend can degrade unresolved tokens back to plain text.
Add a dedicated mention side-effect helper around memo create and update flows. On create, after the memo is persisted and the final payload is available, compute the normalized mentioned user set from `memo.Payload.Mentions` and create inbox items for allowed targets. On update, diff the previous and new normalized mention sets and only notify targets that were newly added in the latest saved revision. This follows the Confluence-style deduplication pattern and prevents repeated notifications when a memo is edited without changing its mention set. If a mention is removed and later re-added, it counts as newly added again and may generate a fresh inbox item.
Apply access and duplication rules before writing inbox rows. Self-mentions are ignored. For top-level memos, notify only when the target can already read the memo under current visibility rules. For comments, notify the mentioned user when they can read the comment context and are not already covered by the existing memo-comment notification to the parent memo owner for that same event. This keeps mention notifications meaningful and avoids sending an owner both a comment notification and a mention notification for the same comment creation unless future product requirements explicitly want both. For `PRIVATE` memos and `PRIVATE` comments, mentions remain author-visible text but do not generate inbox notifications for other users.
Extend inbox storage and API notifications with a dedicated mention type instead of overloading the existing comment type. Add `MEMO_MENTION` to `proto/store/inbox.proto` with a payload that can represent both top-level memos and comments, such as `memo_id` plus optional `related_memo_id`. Mirror that in `proto/api/v1/user_service.proto` with `UserNotification_MEMO_MENTION` and `MemoMentionPayload`. Reuse the current notification conversion pattern in `server/router/api/v1/user_service.go`: resolve memo names from stored IDs, return a first-class mention payload, and let the inbox page render a separate mention card component. This keeps the notification center composable as new activity types appear.
Add an authenticated user-search endpoint specifically for mention autocomplete. The repository already has a stale public-method placeholder for `SearchUsers`, but no proto or handler. Define `SearchUsers` in `proto/api/v1/user_service.proto`, remove it from the public ACL list, and implement it in `server/router/api/v1/user_service.go` as an authenticated RPC that accepts a short query string plus page size. Extend `store.FindUser` with search-oriented fields and implement driver-specific case-insensitive matching in SQLite, MySQL, and PostgreSQL over `username` and `nickname`, ordered by exact username match, username prefix, nickname prefix, then a stable fallback. This produces a usable editor candidate list without reusing the admin-only `ListUsers` contract.
Implement frontend mention suggestions by reusing the existing generic textarea suggestion system. Add a `MentionSuggestions` component beside `TagSuggestions`, hook it into `web/src/components/MemoEditor/Editor/index.tsx`, and back it with a debounced `useSearchUsers(query)` hook. The popup should render avatar, display name, and `@username`, while selection inserts `@username ` exactly. Because `useSuggestions` currently operates on local item arrays, it can stay generic if the mention hook owns the remote query and passes the current ranked results down as `items`.
Implement frontend mention rendering with a dedicated markdown plugin and component instead of trying to infer mentions from links or plain spans. Add `remarkMention` beside `remarkTag`, a `Mention` inline component beside `Tag`, and a mention type guard in `web/src/types/markdown.ts`. The renderer should link resolved mentions to `/u/:username`, show display name or username with avatar-based affordance when lookup data is available, and render unresolved mention text non-interactively. To avoid N-per-mention network fetches, `MemoContent` should collect mentioned usernames from content and hydrate them through the existing `useUsersByNames()` hook once per memo render tree.
Render mention notifications as their own inbox card. Reuse the existing `MemoCommentMessage` pattern, but resolve the source memo/comment and optional related memo from the `MemoMentionPayload`. The card should show who mentioned the user, in what memo or comment, a short snippet, and navigate to the relevant memo detail on click. `web/src/pages/Inboxes.tsx` should switch on both `MEMO_COMMENT` and `MEMO_MENTION` so the inbox can grow by type without silently discarding new notifications.
Do not solve username drift in this issue. If a user later changes username, existing raw markdown still contains the old `@username` text, and rebuilt payload metadata will stop resolving unless the old token still matches a live username. This is acceptable for the current scope because username-history and alias resolution are already out of scope elsewhere in the codebase. The alternative of storing opaque mention IDs in source markdown or adding a username-alias subsystem was rejected because it turns a contained collaboration feature into a broader identity migration project.
@@ -0,0 +1,37 @@
## Execution Log
### T1: Add backend mention parsing and payload extraction
**Status**: Completed
**Files Changed**: `internal/markdown/ast/mention.go`, `internal/markdown/parser/mention.go`, `internal/markdown/extensions/mention.go`, `internal/markdown/markdown.go`, `internal/markdown/renderer/markdown_renderer.go`, `server/runner/memopayload/runner.go`, `server/router/api/v1/memo_service.go`, `server/router/api/v1/v1.go`, `server/router/api/v1/test/test_helper.go`, `internal/markdown/markdown_test.go`
**Validation**: `go test ./internal/markdown` — PASS
**Path Corrections**: `RebuildMemoPayload` needed `context + store` so mention resolution could happen during payload rebuild.
**Deviations**: None
### T2: Add mention notifications and user search APIs
**Status**: Completed
**Files Changed**: `proto/store/memo.proto`, `proto/store/inbox.proto`, `proto/api/v1/user_service.proto`, `server/router/api/v1/user_service.go`, `server/router/api/v1/connect_services.go`, `server/router/api/v1/acl_config.go`, `server/router/api/v1/acl_config_test.go`, `server/router/api/v1/memo_mention_helpers.go`, `store/user.go`, `store/db/sqlite/user.go`, `store/db/postgres/user.go`, `store/db/mysql/user.go`, `server/router/api/v1/test/user_notification_test.go`, `server/router/api/v1/test/user_search_test.go`
**Validation**: `go test ./server/router/api/v1/...` — PASS
**Path Corrections**: Unknown legacy inbox message types are filtered server-side to keep unread counts aligned with rendered cards.
**Deviations**: None
### T3: Add frontend mention autocomplete, rendering, and inbox UI
**Status**: Completed
**Files Changed**: `web/src/components/MemoEditor/Editor/MentionSuggestions.tsx`, `web/src/components/MemoEditor/Editor/index.tsx`, `web/src/components/MemoEditor/Editor/useSuggestions.ts`, `web/src/hooks/useUserQueries.ts`, `web/src/utils/remark-plugins/remark-mention.ts`, `web/src/components/MemoContent/MentionContext.tsx`, `web/src/components/MemoContent/Mention.tsx`, `web/src/components/MemoContent/index.tsx`, `web/src/components/MemoContent/ConditionalComponent.tsx`, `web/src/types/markdown.ts`, `web/src/components/Inbox/MemoMentionMessage.tsx`, `web/src/pages/Inboxes.tsx`
**Validation**: `pnpm lint && pnpm build` — PASS
**Path Corrections**: Editor autocomplete reused the existing generic suggestion hook by exposing the live query rather than duplicating keyboard navigation logic.
**Deviations**: None
### T4: Regenerate code and validate the feature
**Status**: Completed
**Files Changed**: `proto/gen/**`, `web/src/types/proto/**`
**Validation**: `buf generate` — PASS; `go test ./internal/markdown ./server/router/api/v1/...` — PASS; `pnpm lint` — PASS; `pnpm build` — PASS
**Path Corrections**: None
**Deviations**: None
## Completion Declaration
All tasks completed successfully
+104
View File
@@ -0,0 +1,104 @@
## Task List
### Task Index
T1: Add backend mention parsing and payload extraction [M] — T2: Add mention notifications and user search APIs [L] — T3: Add frontend mention autocomplete, rendering, and inbox UI [L] — T4: Regenerate code and validate the feature [M]
### T1: Add backend mention parsing and payload extraction [M]
**Objective**: Parse `@username` tokens into structured mention metadata during memo payload rebuilds.
**Size**: M
**Files**:
- Create: `internal/markdown/ast/mention.go`
- Create: `internal/markdown/parser/mention.go`
- Create: `internal/markdown/extensions/mention.go`
- Modify: `internal/markdown/markdown.go`
- Modify: `internal/markdown/renderer/markdown_renderer.go`
- Modify: `server/runner/memopayload/runner.go`
- Modify: `server/router/api/v1/memo_service.go`
- Test: `internal/markdown/markdown_test.go`
**Implementation**:
1. Add mention AST/parser/extension parallel to the existing tag implementation.
2. Extend extracted markdown data and `MemoPayload` rebuild to collect normalized mentions and resolve them to users.
3. Update memo create/update and background payload rebuild paths to use the new mention-aware payload builder.
**Boundaries**: Do not add a relational schema migration.
**Dependencies**: None
**Expected Outcome**: Memo payloads carry normalized mention metadata rebuilt from markdown content.
**Validation**: `go test ./internal/markdown` — expected `ok`
### T2: Add mention notifications and user search APIs [L]
**Objective**: Expose mention-aware APIs and create inbox items for newly added mentions.
**Size**: L
**Files**:
- Modify: `proto/store/memo.proto`
- Modify: `proto/store/inbox.proto`
- Modify: `proto/api/v1/user_service.proto`
- Modify: `server/router/api/v1/user_service.go`
- Modify: `server/router/api/v1/connect_services.go`
- Modify: `server/router/api/v1/acl_config.go`
- Modify: `server/router/api/v1/acl_config_test.go`
- Create: `server/router/api/v1/memo_mention_helpers.go`
- Modify: `store/user.go`
- Modify: `store/db/sqlite/user.go`
- Modify: `store/db/postgres/user.go`
- Modify: `store/db/mysql/user.go`
- Test: `server/router/api/v1/test/user_notification_test.go`
- Test: `server/router/api/v1/test/user_search_test.go`
**Implementation**:
1. Extend proto contracts with `MemoPayload.mentions`, `InboxMessage.MEMO_MENTION`, `UserNotification.MEMO_MENTION`, and `SearchUsers`.
2. Implement authenticated user search over username and nickname.
3. Add mention notification side effects for memo create/update/comment flows with diffing and duplicate suppression.
4. Convert inbox rows into either comment or mention notifications and filter unknown legacy types.
**Boundaries**: Do not add email/push/webhook mention delivery.
**Dependencies**: T1
**Expected Outcome**: Mentioned users receive inbox notifications and the editor has an API to fetch mention candidates.
**Validation**: `go test ./server/router/api/v1/...` — expected `ok`
### T3: Add frontend mention autocomplete, rendering, and inbox UI [L]
**Objective**: Let users insert mentions from the editor and render/read them in the UI.
**Size**: L
**Files**:
- Create: `web/src/components/MemoEditor/Editor/MentionSuggestions.tsx`
- Modify: `web/src/components/MemoEditor/Editor/index.tsx`
- Modify: `web/src/components/MemoEditor/Editor/useSuggestions.ts`
- Modify: `web/src/hooks/useUserQueries.ts`
- Create: `web/src/utils/remark-plugins/remark-mention.ts`
- Create: `web/src/components/MemoContent/MentionContext.tsx`
- Create: `web/src/components/MemoContent/Mention.tsx`
- Modify: `web/src/components/MemoContent/index.tsx`
- Modify: `web/src/components/MemoContent/ConditionalComponent.tsx`
- Modify: `web/src/types/markdown.ts`
- Create: `web/src/components/Inbox/MemoMentionMessage.tsx`
- Modify: `web/src/pages/Inboxes.tsx`
**Implementation**:
1. Add `@` autocomplete backed by `SearchUsers`.
2. Add markdown mention parsing/rendering and hydrate mentioned users once per memo render.
3. Add a dedicated inbox card for memo mention notifications.
**Boundaries**: Do not redesign the textarea editor.
**Dependencies**: T2
**Expected Outcome**: Users can insert, see, and open mentions from memo content and inbox notifications.
**Validation**: `pnpm lint && pnpm build` — expected success
### T4: Regenerate code and validate the feature [M]
**Objective**: Regenerate generated code and verify backend/frontend behavior.
**Size**: M
**Files**:
- Modify: `proto/gen/**`
- Modify: `web/src/types/proto/**`
**Implementation**:
1. Run `buf generate` after proto changes.
2. Re-run focused Go tests and frontend lint/build.
**Boundaries**: Do not broaden into unrelated CI cleanup.
**Dependencies**: T1, T2, T3
**Expected Outcome**: Generated code matches the new APIs and validations pass.
**Validation**: `buf generate`, `go test ./internal/markdown ./server/router/api/v1/...`, `pnpm lint`, `pnpm build`
## Out-of-Scope Tasks
- Group/team mentions
- Username alias migration
- Email or push delivery for mentions
- Watch/subscription semantics beyond explicit mentions
@@ -0,0 +1,70 @@
## Background & Context
SSO sign-in in memos currently treats the IdP-provided identifier as the local username. The identifier value comes from the OAuth2 UserInfo claim named in `FieldMapping.identifier`, while local usernames are validated by `validateUsername` against `base.UIDMatcher`. Real IdPs frequently emit identifiers such as email addresses, opaque subject IDs, or provider-specific account IDs that are valid authentication subjects but are not valid memos usernames.
The existing issue artifacts under `docs/plans/2026-04-21-sso-user-identity-linkage/` already scope a persistent linkage between SSO identities and local users. A broader review of upstream open source schemas now shows that similar systems converge on separating external identity from the local user row, but do not converge on one universal table name or one exact column set. That difference matters because the implementation problem is narrower than "copy one upstream schema exactly" and broader than "pick any new table name locally."
## Issue Statement
The SSO sign-in path in `server/router/api/v1/auth_service.go` resolves and creates users from `userInfo.Identifier` through `User.Username`, and no provider-scoped external identity record exists to resolve a local user independently of that username; as a result, provider-issued identifiers that are valid authentication subjects but invalid memos usernames fail the sign-in path, and the future persistence model still requires an explicit schema decision among several verified upstream identity-link patterns.
## Current State
**Sign-in path**`server/router/api/v1/auth_service.go:124-173`
- Lines 124-132 apply `identifier_filter` to `userInfo.Identifier`.
- Lines 135-137 call `GetUser(FindUser{Username: &userInfo.Identifier})`.
- Lines 151-159 create a new `store.User` with `Username: userInfo.Identifier`.
- No persistent linkage record is read or written during SSO sign-in.
**Username validation**`server/router/api/v1/user_resource_name.go:33-38`
- `validateUsername` rejects empty strings, fully numeric usernames, and values that fail `base.UIDMatcher`.
- `base.UIDMatcher` is defined in `internal/base/resource_name.go:5-6` as `^[a-zA-Z0-9]([a-zA-Z0-9-]{0,34}[a-zA-Z0-9])?$`.
**User model**`store/user.go:26-77`
- `store.User` contains `Username`, `Email`, `Nickname`, `AvatarURL`, and other local account fields.
- `store.FindUser` supports lookup by `ID`, `Username`, `Email`, and related filters.
- No external identity field exists on the user model.
**Current database schema**`store/migration/sqlite/LATEST.sql:9-79`
- `user` table (`lines 10-22`) stores `username` as a unique column and has no external identity column.
- `idp` table (`lines 72-79`) stores `uid` as the stable identifier for an IdP instance.
- The latest checked-in migration version is `0.27` under all three backends (`store/migration/sqlite/0.27`, `store/migration/postgres/0.27`, `store/migration/mysql/0.27`).
**IdP user info mapping**`internal/idp/idp.go:3-8`, `internal/idp/oauth2/oauth2.go:105-129`
- `IdentityProviderUserInfo` carries `Identifier`, `DisplayName`, `Email`, and `AvatarURL`.
- `Identifier` is loaded from the configured claim and is required to be non-empty.
- `DisplayName` falls back to `Identifier` when not mapped.
**No existing linkage persistence**
- No `identity`, `identities`, `user_identity`, `external_login_user`, or similar structure exists anywhere under `store/`.
## Non-Goals
- Changing `UIDMatcher` or `validateUsername`.
- Changing how `FieldMapping` maps OAuth2 claims into `IdentityProviderUserInfo`.
- Changing how password-based local sign-in works.
- Changing `identifier_filter` behavior.
- Supporting IdP types other than `OAUTH2` in this issue.
- Providing UI or API surfaces for linking or unlinking external identities.
- Migrating or renaming existing usernames already stored in `user`.
- Automatically linking pre-existing users whose current `User.Username` happens to match an IdP identifier.
- Adding SCIM, directory sync, or per-group / multi-tenant IdP scoping.
- Storing provider access tokens or profile payloads unless a live memos code path requires them.
## Open Questions
1. When a new SSO user's `userInfo.Identifier` does not yield a valid username, what value is used as the initial `User.Username`? (default: derive from `DisplayName`, then `Email`, then `Identifier`, normalizing to a valid username and retrying with a short suffix on collision)
2. Should an existing local user be linkable to an SSO identity after registration? (default: no — out of scope for this issue)
3. Should one local user be linkable to multiple external identities across different IdP instances? (default: yes — allow multiple rows per `user_id`, one per provider-scoped external identifier)
4. What schema vocabulary should represent the provider-scoped external identity record? (default: use table `user_identity` to match current memos table naming, with `provider` and `extern_uid` as the stored linkage fields)
5. Should the linkage schema store only lookup fields or also provider metadata such as tokens and raw profile data? (default: lookup fields only for this issue)
6. Should the linkage table be added across SQLite, PostgreSQL, and MySQL? (default: yes — mirror the existing migration strategy across all supported backends)
## Scope
**L** — the work still spans a new persistence structure across three database backends, store-layer types and driver implementations, sign-in path changes, username derivation behavior, and now an explicit design choice among several verified upstream schema patterns rather than a single assumed naming scheme.
@@ -0,0 +1,173 @@
## References
1. **GitLab `db/init_structure.sql` — `identities` table** (verified)
https://gitlab.com/gitlab-org/gitlab/-/raw/master/db/init_structure.sql?ref_type=heads
GitLab stores external identities in a separate `identities` table with fields including `extern_uid`, `provider`, `user_id`, `created_at`, and `updated_at`, plus provider-specific extensions such as `saml_provider_id`.
2. **Gitea `models/user/external_login_user.go` — `ExternalLoginUser`** (verified)
https://raw.githubusercontent.com/go-gitea/gitea/main/models/user/external_login_user.go
Gitea persists external account links separately from the user row. The core linkage fields are `ExternalID`, `LoginSourceID`, and `UserID`, with optional provider metadata and tokens.
3. **Discourse `app/models/user_associated_account.rb` — `user_associated_accounts`** (verified)
https://raw.githubusercontent.com/discourse/discourse/main/app/models/user_associated_account.rb
Discourse uses a dedicated association table with `provider_name`, `provider_uid`, `user_id`, timestamps, and JSONB metadata. It enforces uniqueness on both `(provider_name, provider_uid)` and `(provider_name, user_id)`.
4. **Keycloak `FederatedIdentityEntity.java` — `FEDERATED_IDENTITY`** (verified)
https://raw.githubusercontent.com/keycloak/keycloak/main/model/jpa/src/main/java/org/keycloak/models/jpa/entities/FederatedIdentityEntity.java
Keycloak models brokered identities separately with `USER_ID`, `IDENTITY_PROVIDER`, `FEDERATED_USER_ID`, `FEDERATED_USERNAME`, `REALM_ID`, and `TOKEN`, using provider-scoped lookup queries rather than local username lookup.
5. **Immich `user.table.ts` and initial migration — `users.oauthId`** (verified)
https://raw.githubusercontent.com/immich-app/immich/main/server/src/schema/tables/user.table.ts
https://raw.githubusercontent.com/immich-app/immich/main/server/src/schema/migrations/1744910873969-InitialMigration.ts
Immich stores a single `oauthId` column directly on the `users` table instead of a separate linkage table.
6. **Mattermost `server/public/model/user.go` — `AuthData` and `AuthService` on `User`** (verified)
https://raw.githubusercontent.com/mattermost/mattermost/master/server/public/model/user.go
Mattermost stores external-auth linkage directly on the user model through `AuthData` and `AuthService`.
## Industry Baseline
There is no single universal table name for SSO identity linkage across open source systems. The consistent pattern is structural rather than nominal: systems that support multiple external providers or explicit account linking separate the external identity from the local username and from the core user row.
Three recurring schema families appear in the references:
- **Provider-scoped identity-link table**. GitLab `identities` and Gitea `ExternalLoginUser` both persist a provider discriminator, an external identifier, and a local `user_id` in a separate structure. This is the most direct fit when one local user may be linked to multiple IdPs.
- **Association table with richer metadata**. Discourse `user_associated_accounts` uses the same provider-plus-external-id pattern, but also stores provider payloads and enforces one account per provider per user.
- **User-row coupling**. Immich and Mattermost place the external identifier on the user row itself. This minimizes schema surface area, but it ties the user record to one external binding shape and reduces flexibility when multiple providers or multiple linked identities are in scope.
Keycloak's `FEDERATED_IDENTITY` falls in the separate-table family as well, but introduces an additional realm scope and stores broker-specific metadata because Keycloak is itself an identity broker rather than an application with a local username model.
Known trade-offs:
- Separate tables add one extra lookup during sign-in compared with `GetUser(Username)`, but they avoid coupling authentication subjects to user-facing usernames.
- Richer association tables support future linking, unlinking, and metadata sync, but they increase write paths and retention of provider data.
- User-row columns are simpler for one-provider or one-link systems, but they do not fit a product that already supports multiple configured IdP instances.
## Research Summary
- GitLab, Gitea, Discourse, and Keycloak all separate the external identity record from the local username.
- The recurring core fields are a **provider scope**, an **external subject/UID**, and a **local user ID**. Exact names vary: `provider`, `login_source_id`, `provider_name`, `identity_provider`, `extern_uid`, `external_id`, `provider_uid`, `federated_user_id`.
- Table naming is not standardized across projects. `identities` is common and recognizable, but `external_login_user`, `user_associated_accounts`, and `FEDERATED_IDENTITY` are also established patterns.
- Single-column user-row designs such as Immich `oauthId` and Mattermost `AuthData` / `AuthService` are simpler, but they align with products that bind one external identity shape directly to the user row rather than a product that already has multiple `idp` records.
- For memos, the verified references support a provider-scoped link table and do not support continuing to use `User.Username` as the external identity lookup key. They also suggest that `provider` and `extern_uid` are clearer linkage-field names than repo-specific names like `idp_id`, even if the table name itself follows memos naming conventions.
## Design Goals
1. **G1 — Username-independent lookup**: SSO sign-in resolves an existing user without requiring `userInfo.Identifier` to pass `validateUsername`. Verifiable: a sign-in flow with `Identifier = "jane@example.com"` or an opaque subject string reaches user lookup through the linkage table, not through `FindUser{Username: ...}`.
2. **G2 — Provider-scoped uniqueness**: Two configured IdP instances may issue the same external identifier string without colliding. Verifiable: uniqueness is enforced on the pair `(provider, extern_uid)`, not on `extern_uid` alone.
3. **G3 — No new external-auth column on `user`**: The local `user` table remains a local account record rather than the storage site for external identity subjects. Verifiable: migration files do not add `oauth_id`, `auth_data`, `provider_uid`, or similar columns to `user`.
4. **G4 — Repo-aligned table naming with reference-aligned linkage fields**: The new linkage schema follows memos table naming conventions while using linkage field names that map directly to upstream patterns. Verifiable: the design uses table `user_identity` together with fields `provider` and `extern_uid`.
5. **G5 — Minimal persistence surface for this issue**: The linkage record stores lookup fields required by sign-in and excludes provider payloads that no current memos code path reads. Verifiable: the initial schema contains no token, JSON payload, or raw-profile columns.
## Non-Goals
All non-goals from `definition.md` apply. Additionally:
- Matching GitLab, Gitea, Discourse, or Keycloak field-for-field beyond the linkage concepts required by memos.
- Adding SCIM-, SAML-, or realm-specific columns such as Keycloak `REALM_ID` or GitLab `saml_provider_id`.
- Enforcing a one-provider-per-user rule like Discourse's `(provider_name, user_id)` uniqueness.
- Storing provider tokens, raw payloads, or synchronization metadata in the initial linkage table.
## Proposed Design
### 1. Use a separate `user_identity` table
Adopt the separate-table family used by GitLab `identities`, Gitea `ExternalLoginUser`, Discourse `user_associated_accounts`, and Keycloak `FEDERATED_IDENTITY`, while keeping the stored fields limited to what memos currently needs for sign-in (G1, G2, G3, G5). The table name should follow memos' existing singular / relationship-style naming pattern rather than copying GitLab's pluralized table name directly (G4).
Proposed logical schema:
```sql
CREATE TABLE user_identity (
id INTEGER PRIMARY KEY AUTOINCREMENT,
user_id INTEGER NOT NULL,
provider TEXT NOT NULL, -- stores idp.uid
extern_uid TEXT NOT NULL,
created_ts BIGINT NOT NULL DEFAULT (strftime('%s', 'now')),
updated_ts BIGINT NOT NULL DEFAULT (strftime('%s', 'now')),
UNIQUE (provider, extern_uid)
);
CREATE INDEX user_identity_user_id_idx ON user_identity(user_id);
```
Naming decisions:
- **`user_identity`** follows the repo's current table naming pattern. Existing memos tables are singular or singular relationship tables such as `user`, `memo`, `memo_relation`, `memo_share`, and `idp`, so `user_identity` is more consistent locally than GitLab's plural `identities` (G4).
- **`provider`** is the stored `idp.uid`. This maps directly to GitLab `provider`, Keycloak `IDENTITY_PROVIDER`, Discourse `provider_name`, and Gitea's effective provider scope via `LoginSourceID`.
- **`extern_uid`** uses GitLab's term for the provider-issued subject and avoids coupling the schema to a specific protocol word such as `sub`, `email`, or `external_id`.
Rejected alternatives:
- **`identities`**: recognizable from GitLab, but inconsistent with the repo's current schema naming pattern.
- **`user_identity` with `idp_id`**: workable on naming, but less reference-aligned on the linkage fields than `provider` / `extern_uid`.
- **Add `oauth_id`-style column(s) to `user`**: simpler, but inconsistent with memos supporting multiple `idp` rows and weaker on G2 and G3, matching the Immich and Mattermost trade-off rather than the multi-provider references.
No foreign key is proposed from `provider` to `idp.uid` in the initial design. The current memos migrations do not rely heavily on cross-backend foreign-key behavior, and the linkage record must remain stable even if IdP rows are reworked at the application layer.
### 2. Add a dedicated store model and driver methods
Add a new store module for the linkage record instead of extending `store.User` (G3):
- `store/user_identity.go`
- `store/db/sqlite/user_identity.go`
- `store/db/postgres/user_identity.go`
- `store/db/mysql/user_identity.go`
The store type should mirror the lookup schema:
- `UserIdentity`: `ID`, `UserID`, `Provider`, `ExternUID`, `CreatedTs`, `UpdatedTs`
- `FindUserIdentity`: `Provider`, `ExternUID`, `UserID`
Only create and read operations are required in this issue. Update and delete paths are deferred because no current sign-in flow needs them (G5). The create path must surface provider/external-UID uniqueness conflicts so the sign-in flow can reconcile concurrent first-login races instead of leaving an unlinked local user behind.
### 3. Route SSO lookup through the linkage record
Change the `auth_service.SignIn` SSO branch so that, after `identifier_filter` passes, the flow is:
1. Resolve the current IdP instance from `idp.uid`.
2. Query `user_identity` by `(provider = idp.uid, extern_uid = userInfo.Identifier)`.
3. On hit, load the local user by `user_id`.
4. On miss, apply the existing registration gate, derive a valid local username from display-oriented fields, and execute local user creation plus `user_identity` insertion in a single transaction.
5. If the `user_identity` insert loses a race on the unique `(provider, extern_uid)` key, discard the provisional linkage result, re-read `user_identity`, and continue sign-in using the winning row's `user_id`.
This directly addresses the current runtime coupling in `server/router/api/v1/auth_service.go:135-169` and satisfies G1 and G2. `User.Username` becomes a local account attribute again instead of the SSO subject store, and the miss path becomes idempotent under concurrent first sign-ins for the same external identity.
### 4. Keep username derivation separate from external identity persistence
The initial local username for new SSO-created users should be derived from user-facing profile data rather than copied from `extern_uid`:
- first choice: `DisplayName`
- fallback: `Email`
- fallback: `Identifier`
Normalization and collision handling remain local username concerns, not identity-link concerns. This keeps the `user_identity` table focused on lookup semantics (G1, G5) and avoids implying that external identifiers must resemble usernames.
### 5. Versioned migrations across all three backends
Add a new migration version after `0.27` for SQLite, PostgreSQL, and MySQL. Update `LATEST.sql` for each backend to include the `user_identity` table and its `user_id` index.
The migrations should preserve the same logical fields across backends:
- `id`
- `user_id`
- `provider`
- `extern_uid`
- `created_ts`
- `updated_ts`
- unique key on `(provider, extern_uid)`
- secondary index on `user_id`
The implementation derived from these migrations should treat `(provider, extern_uid)` as both the lookup key and the concurrency guard for first login. A uniqueness conflict on that pair is a recoverable race outcome, not an unrecoverable error path.
### 6. Document the pattern families explicitly in the issue docs
Keep the issue docs explicit that memos is selecting one member of a broader family of established designs:
- GitLab / Gitea / Keycloak / Discourse: separate identity record
- Immich / Mattermost: user-row external-auth fields
That distinction matters because the design choice is not "copy GitLab exactly"; it is "use the multi-provider separate-identity family, keep memos-compatible table naming, and adopt upstream-aligned linkage fields where they improve recognizability" (G4).
@@ -0,0 +1,105 @@
## Execution Log
### T1: Add `user_identity` migrations + LATEST.sql updates
**Status**: Completed
**Files Changed**:
- Created: `store/migration/sqlite/0.28/00__user_identity.sql`
- Created: `store/migration/postgres/0.28/00__user_identity.sql`
- Created: `store/migration/mysql/0.28/00__user_identity.sql`
- Modified: `store/migration/sqlite/LATEST.sql`
- Modified: `store/migration/postgres/LATEST.sql`
- Modified: `store/migration/mysql/LATEST.sql`
**Validation**:
- `rg 'CREATE TABLE \`?user_identity\`?' store/migration` — PASS (hits in all 6 expected files).
- `rg 'UNIQUE \(\`?provider\`?, \`?extern_uid\`?\)' store/migration` — PASS (6 hits).
- `go build ./...` — PASS.
**Path Corrections**: None.
**Deviations**: None.
### T2: Add `store.UserIdentity` model, `Store` methods, and driver interface
**Status**: Completed
**Files Changed**:
- Created: `store/user_identity.go`
- Modified: `store/driver.go`
**Validation**:
- Interface-only build is expected to fail until T3T5; deferred compile check to T5.
- `rg 'CreateUserIdentity|ListUserIdentities' store/driver.go store/user_identity.go` — PASS (method declarations present in both files).
**Path Corrections**: None.
**Deviations**: None.
### T3: Implement SQLite driver for `user_identity`
**Status**: Completed
**Files Changed**:
- Created: `store/db/sqlite/user_identity.go`
**Validation**:
- `go build ./store/db/sqlite/...` — PASS.
**Path Corrections**: None.
**Deviations**: None.
### T4: Implement Postgres driver for `user_identity`
**Status**: Completed
**Files Changed**:
- Created: `store/db/postgres/user_identity.go`
**Validation**:
- `go build ./store/db/postgres/...` — PASS.
**Path Corrections**: None.
**Deviations**: None.
### T5: Implement MySQL driver for `user_identity`
**Status**: Completed
**Files Changed**:
- Created: `store/db/mysql/user_identity.go`
**Validation**:
- `go build ./...` — PASS (whole repo compiles; all drivers satisfy the `Driver` interface).
**Path Corrections**: None.
**Deviations**: None.
### T6: Add store-layer tests for `user_identity`
**Status**: Completed
**Files Changed**:
- Created: `store/test/user_identity_test.go`
**Validation**:
- `DRIVER=sqlite go test ./store/test/ -run TestUserIdentity -count=1 -v` — PASS:
- `TestUserIdentityCreateAndGet` — PASS
- `TestUserIdentityListByUserID` — PASS
- `TestUserIdentityUniqueConflict` — PASS
- `TestUserIdentitySameExternUIDDifferentProviders` — PASS
**Path Corrections**: None.
**Deviations**: None.
### T7: Add SSO username derivation helper
**Status**: Completed
**Files Changed**:
- Created: `server/router/api/v1/sso_username.go`
**Validation**:
- `go build ./server/router/api/v1/...` — PASS.
- `go vet ./server/router/api/v1/...` — PASS.
**Path Corrections**: None.
**Deviations**: None.
### T8: Route SSO sign-in through `user_identity` linkage
**Status**: Completed
**Files Changed**:
- Modified: `server/router/api/v1/auth_service.go`
- `SignIn` SSO branch now delegates user resolution to a new `resolveSSOUser` method.
- `resolveSSOUser` does: `user_identity` lookup → hit path (load user by linked `user_id`); miss path (registration gate → `deriveSSOUsername` → create user → create linkage → race recovery on unique(provider, extern_uid)).
- Added `isUserIdentityUniqueViolation` helper (string match on the three backends' unique-constraint error strings, matching the pattern in `memo_service.go:103105`).
**Validation**:
- `go build ./...` — PASS.
- `go vet ./...` — PASS.
- `DRIVER=sqlite go test ./store/test/ -run TestUserIdentity -count=1` — PASS (regression check).
**Path Corrections**:
- The plan pseudocode referenced `identityProvider.UID`; the actual protobuf type `storepb.IdentityProvider` exposes the field as `Uid`. Used `identityProvider.Uid` in the implementation. No semantic deviation.
**Deviations**: None.
## Completion Declaration
**All tasks completed successfully.**
@@ -0,0 +1,328 @@
## Task List
**Task Index**
> T1: Add `user_identity` migrations + LATEST.sql updates for all three backends [M] — T2: Add `store.UserIdentity` model and `Store` methods + driver interface [M] — T3: Implement SQLite driver for `user_identity` [M] — T4: Implement Postgres driver for `user_identity` [M] — T5: Implement MySQL driver for `user_identity` [M] — T6: Add store-layer tests for `user_identity` [M] — T7: Add SSO username derivation helper [M] — T8: Route SSO sign-in through `user_identity` linkage [L]
### T1: Add `user_identity` migrations + LATEST.sql updates [M]
**Objective**: Create the `user_identity` persistence structure across SQLite, Postgres, and MySQL, and reflect it in `LATEST.sql` for fresh installs (G1, G2, G3, G4, G5; design §1, §5).
**Size**: M (3 new migration files, 3 LATEST.sql edits; straightforward DDL).
**Files**:
- Create: `store/migration/sqlite/0.28/00__user_identity.sql`
- Create: `store/migration/postgres/0.28/00__user_identity.sql`
- Create: `store/migration/mysql/0.28/00__user_identity.sql`
- Modify: `store/migration/sqlite/LATEST.sql`
- Modify: `store/migration/postgres/LATEST.sql`
- Modify: `store/migration/mysql/LATEST.sql`
**Implementation**:
1. `store/migration/sqlite/0.28/00__user_identity.sql`:
```sql
CREATE TABLE user_identity (
id INTEGER PRIMARY KEY AUTOINCREMENT,
user_id INTEGER NOT NULL,
provider TEXT NOT NULL,
extern_uid TEXT NOT NULL,
created_ts BIGINT NOT NULL DEFAULT (strftime('%s', 'now')),
updated_ts BIGINT NOT NULL DEFAULT (strftime('%s', 'now')),
UNIQUE (provider, extern_uid)
);
CREATE INDEX idx_user_identity_user_id ON user_identity(user_id);
```
2. `store/migration/postgres/0.28/00__user_identity.sql`: same logical schema with Postgres types — `id SERIAL PRIMARY KEY`, `user_id INTEGER NOT NULL`, `provider TEXT NOT NULL`, `extern_uid TEXT NOT NULL`, `created_ts BIGINT NOT NULL DEFAULT EXTRACT(EPOCH FROM NOW())`, `updated_ts BIGINT NOT NULL DEFAULT EXTRACT(EPOCH FROM NOW())`, `UNIQUE(provider, extern_uid)`, plus `CREATE INDEX idx_user_identity_user_id ON user_identity(user_id);`. Include a 2-line header comment describing the table purpose (pattern-match `04__memo_share.sql`).
3. `store/migration/mysql/0.28/00__user_identity.sql`: same logical schema with MySQL syntax — backticked identifiers, `INT NOT NULL AUTO_INCREMENT PRIMARY KEY`, `VARCHAR(256)` for `provider`, `VARCHAR(256)` for `extern_uid` (so unique key fits within index limits), `BIGINT NOT NULL DEFAULT (UNIX_TIMESTAMP())` for timestamps, `UNIQUE(provider, extern_uid)`, plus `CREATE INDEX idx_user_identity_user_id ON user_identity(user_id);`.
4. Append a `-- user_identity` section to each `LATEST.sql` mirroring the corresponding migration file (schema only, same indentation style used by neighboring tables in that file).
**Boundaries**: Must NOT alter the `user` or `idp` tables; must NOT add FK from `user_identity.provider` to `idp.uid`; must NOT add columns beyond `id`, `user_id`, `provider`, `extern_uid`, `created_ts`, `updated_ts`.
**Dependencies**: None.
**Expected Outcome**: New migration files exist; `LATEST.sql` for each backend contains a `user_identity` table block and its `user_id` index.
**Validation**:
- `rg -n "CREATE TABLE user_identity" store/migration` — expects one hit per backend in both the 0.28 migration and `LATEST.sql` (6 hits total).
- `rg -n "UNIQUE ?\\(provider, extern_uid\\)" store/migration` — expects 6 hits total.
- `go build ./...` — expects PASS (no code changes affect the build; confirms no stray syntax issues).
---
### T2: Add `store.UserIdentity` model, `Store` methods, and driver interface [M]
**Objective**: Provide a Go-level abstraction for the `user_identity` record with create/read operations wired through `store.Driver` (design §2, G3, G5).
**Size**: M (one new store file, one interface edit; simple CRUD-shaped code).
**Files**:
- Create: `store/user_identity.go`
- Modify: `store/driver.go`
**Implementation**:
1. `store/user_identity.go`:
- Types:
```go
type UserIdentity struct {
ID int32
UserID int32
Provider string
ExternUID string
CreatedTs int64
UpdatedTs int64
}
type FindUserIdentity struct {
ID *int32
UserID *int32
Provider *string
ExternUID *string
}
```
- Store methods (thin passthroughs to driver):
```go
func (s *Store) CreateUserIdentity(ctx context.Context, create *UserIdentity) (*UserIdentity, error)
func (s *Store) ListUserIdentities(ctx context.Context, find *FindUserIdentity) ([]*UserIdentity, error)
func (s *Store) GetUserIdentity(ctx context.Context, find *FindUserIdentity) (*UserIdentity, error) // returns (nil, nil) on no match
```
- No update/delete methods in this issue (design §2: create/read only).
2. `store/driver.go`: extend the `Driver` interface with:
```go
// UserIdentity model related methods.
CreateUserIdentity(ctx context.Context, create *UserIdentity) (*UserIdentity, error)
ListUserIdentities(ctx context.Context, find *FindUserIdentity) ([]*UserIdentity, error)
```
`GetUserIdentity` in `store` can be implemented locally by calling `ListUserIdentities` with `Limit`-free semantics and returning the first row, matching the `GetMemoShare`/`GetIdentityProvider` pattern (no new driver method required for "get").
**Boundaries**: Must NOT add fields to `store.User` or `store.UpdateUser`; must NOT add update/delete methods.
**Dependencies**: None (T3T5 will satisfy the new interface methods).
**Expected Outcome**: `store.UserIdentity`, `FindUserIdentity`, and three `Store` methods exist; `Driver` interface declares the two new methods.
**Validation**:
- `go build ./store/...` — expects FAIL until T3T5 implement the interface on each driver. Record as expected; final pass comes at end of T5.
- `rg -n "CreateUserIdentity|ListUserIdentities" store/driver.go store/user_identity.go` — expects method declarations in both files.
---
### T3: Implement SQLite driver for `user_identity` [M]
**Objective**: Implement `CreateUserIdentity` and `ListUserIdentities` for SQLite so the interface declared in T2 is satisfied (design §2).
**Size**: M (one new driver file; mirrors existing `memo_share.go` patterns).
**Files**:
- Create: `store/db/sqlite/user_identity.go`
**Implementation**:
1. `CreateUserIdentity`:
- Insert columns `user_id`, `provider`, `extern_uid` using `?` placeholders.
- Use `RETURNING id, created_ts, updated_ts` to populate generated fields, same pattern as `store/db/sqlite/memo_share.go:24`.
- Return the passed-in `create` struct with generated fields populated, or the error from `QueryRowContext(...).Scan(...)` (unique-constraint violation surfaces to caller unchanged).
2. `ListUserIdentities`:
- `where := []string{"1 = 1"}`; append clauses for `find.ID`, `find.UserID`, `find.Provider`, `find.ExternUID` when non-nil.
- `SELECT id, user_id, provider, extern_uid, created_ts, updated_ts FROM user_identity WHERE ... ORDER BY id ASC`.
- Scan rows into `[]*store.UserIdentity`; return `[]*store.UserIdentity{}` on no rows (not nil).
**Boundaries**: Must NOT introduce transaction helpers, upsert semantics, or extra scan columns.
**Dependencies**: T2.
**Expected Outcome**: SQLite driver compiles and returns populated rows.
**Validation**:
- `go build ./store/db/sqlite/...` — expects PASS.
---
### T4: Implement Postgres driver for `user_identity` [M]
**Objective**: Mirror T3 for Postgres using `$N` placeholders and `SERIAL` semantics (design §2).
**Size**: M (one new driver file; mirrors `store/db/postgres/memo_share.go`).
**Files**:
- Create: `store/db/postgres/user_identity.go`
**Implementation**:
- Same shape as T3, but:
- Use `placeholder(n)` / `placeholders(n)` helpers from `store/db/postgres/common.go`.
- Insert stmt `INSERT INTO user_identity (user_id, provider, extern_uid) VALUES (...) RETURNING id, created_ts, updated_ts`.
- List query identical SQL shape to SQLite (no backticks in Postgres; match `memo_share.go` style).
**Boundaries**: Same as T3.
**Dependencies**: T2.
**Expected Outcome**: Postgres driver compiles.
**Validation**:
- `go build ./store/db/postgres/...` — expects PASS.
---
### T5: Implement MySQL driver for `user_identity` [M]
**Objective**: Mirror T3/T4 for MySQL, using `LastInsertId()` + re-read pattern (MySQL's driver does not support `RETURNING`; design §2).
**Size**: M (one new driver file; mirrors `store/db/mysql/memo_share.go`).
**Files**:
- Create: `store/db/mysql/user_identity.go`
**Implementation**:
- `CreateUserIdentity`:
- `INSERT INTO user_identity (user_id, provider, extern_uid) VALUES (?, ?, ?)` via `ExecContext`.
- Get `LastInsertId()`, re-fetch via `GetUserIdentity(... ID: &id)` helper (internal unexported `listUserIdentitiesByID` or reuse `ListUserIdentities` with `FindUserIdentity{ID: &id}` + take first result).
- Mirror `memo_share.go` error-handling style (return `errors.Errorf("failed to create user identity")` when re-fetch returns nil, like memo_share does).
- `ListUserIdentities`:
- Same shape as T3, using backticked column names (`` `user_id` ``, `` `provider` ``, `` `extern_uid` ``) and `?` placeholders, matching the MySQL idiom used in `memo_share.go`.
**Boundaries**: Same as T3.
**Dependencies**: T2.
**Expected Outcome**: MySQL driver compiles; full repo builds.
**Validation**:
- `go build ./...` — expects PASS (entire repo compiles with all drivers satisfying the `Driver` interface introduced in T2).
---
### T6: Add store-layer tests for `user_identity` [M]
**Objective**: Exercise create + read paths plus the `(provider, extern_uid)` uniqueness guard across the active driver (G2).
**Size**: M (one new test file; patterns match existing store tests).
**Files**:
- Create: `store/test/user_identity_test.go`
**Implementation**:
1. `TestUserIdentityCreateAndGet`:
- Create host user via `createTestingHostUser`.
- `CreateUserIdentity` with `UserID=user.ID`, `Provider="idp-uid-1"`, `ExternUID="jane@example.com"`.
- `GetUserIdentity` by `(Provider, ExternUID)` — assert match on `UserID`, `Provider`, `ExternUID`, non-zero `ID`, non-zero `CreatedTs`.
2. `TestUserIdentityListByUserID`:
- Create two identities under the same `UserID` with two different `Provider` values.
- `ListUserIdentities` by `UserID` — assert length 2.
3. `TestUserIdentityUniqueConflict`:
- Insert one row with `(Provider="idp-A", ExternUID="sub-1")`.
- Insert a second row with identical `(Provider, ExternUID)` for a different `UserID`.
- Assert the second `CreateUserIdentity` returns a non-nil error (detection via `err != nil`; do not assert message since error strings differ per backend).
4. `TestUserIdentitySameExternUIDDifferentProviders`:
- Insert `(Provider="idp-A", ExternUID="sub-1")` and `(Provider="idp-B", ExternUID="sub-1")` under the same or different users.
- Assert both inserts succeed (G2: uniqueness is scoped to the pair, not `extern_uid` alone).
**Boundaries**: Must NOT test SSO sign-in or auth service behavior; must NOT test migration contents beyond what `NewTestingStore` already executes.
**Dependencies**: T1T5.
**Expected Outcome**: All four tests pass against SQLite.
**Validation**:
- `go test ./store/test/ -run TestUserIdentity -count=1` — expects all 4 tests PASS.
---
### T7: Add SSO username derivation helper [M]
**Objective**: Produce a valid `User.Username` for new SSO-created users from profile fields, independent of `extern_uid` (design §4).
**Size**: M (one new file with helper + small unit test; self-contained logic).
**Files**:
- Create: `server/router/api/v1/sso_username.go`
**Implementation**:
1. `deriveSSOUsername(ctx context.Context, stores *store.Store, userInfo *idp.IdentityProviderUserInfo) (string, error)`:
- Build ordered candidate list: `[userInfo.DisplayName, userInfo.Email, userInfo.Identifier]`, skipping empty values.
- For each candidate:
1. `base := normalizeToUsername(candidate)`
2. If `validateUsername(base) == nil`:
- If no existing user with `Username=base` (via `stores.GetUser(&FindUser{Username: &base})`), return `base`.
- Else: try up to N=8 suffix retries `base + "-" + randomSuffix(6)`, where the trimmed base ensures total length ≤ 36. If a candidate passes `validateUsername` and is unique, return it.
3. If all candidates are exhausted: fall back to a purely random username `"user-" + randomSuffix(10)` validated via `validateUsername`; retry up to 5 times before returning an error.
4. `normalizeToUsername(s string) string`:
- ASCII-fold / lowercase.
- Replace every character not in `[a-zA-Z0-9]` with `-`.
- Collapse consecutive `-` into one `-`.
- Trim leading/trailing `-`.
- Truncate to 36 chars, then re-trim trailing `-` so the string still ends in alphanumeric.
- Return `""` if the result is empty or fully numeric (so the caller falls through to the next candidate).
5. Use `internal/util.RandomString` for the random suffix (already imported by `auth_service.go`).
**Boundaries**: Must NOT modify `validateUsername` or `base.UIDMatcher`; must NOT write to `user_identity` or `user` directly; must NOT call `CreateUser`.
**Dependencies**: None.
**Expected Outcome**: New file `server/router/api/v1/sso_username.go` containing the exported-for-package helper `deriveSSOUsername` and internal `normalizeToUsername`.
**Validation**:
- `go build ./server/router/api/v1/...` — expects PASS.
- `go vet ./server/router/api/v1/...` — expects PASS.
---
### T8: Route SSO sign-in through `user_identity` linkage [L]
**Objective**: Replace the `FindUser{Username: &userInfo.Identifier}` lookup and `Username: userInfo.Identifier` user creation with `user_identity`-backed lookup and derived-username user creation, satisfying G1 and G2 end-to-end (design §3).
**Size**: L (non-trivial branching logic: lookup, miss path, registration gate, race recovery).
**Files**:
- Modify: `server/router/api/v1/auth_service.go`
**Implementation** (in `SignIn`, SSO branch, replacing current lines ~124173):
1. After `identifier_filter` check succeeds (existing `lines 124-133` unchanged), resolve the linkage:
```go
provider := identityProvider.Uid
externUID := userInfo.Identifier
existingIdentity, err := s.Store.GetUserIdentity(ctx, &store.FindUserIdentity{
Provider: &provider,
ExternUID: &externUID,
})
// error handling → codes.Internal
```
2. **Hit path**: if `existingIdentity != nil`, load `s.Store.GetUser(ctx, &store.FindUser{ID: &existingIdentity.UserID})`; set `existingUser`; skip creation.
3. **Miss path**: gate on `instanceGeneralSetting.DisallowUserRegistration` (reuse existing flow at current lines 143149), then:
1. `username, err := deriveSSOUsername(ctx, s.Store, userInfo)` — from T7. `codes.Internal` on error.
2. Generate random password + bcrypt hash (unchanged from current lines 160168).
3. `user, err := s.Store.CreateUser(ctx, &store.User{Username: username, Role: store.RoleUser, Nickname: userInfo.DisplayName, Email: userInfo.Email, AvatarURL: userInfo.AvatarURL, PasswordHash: string(passwordHash)})`.
4. `_, err := s.Store.CreateUserIdentity(ctx, &store.UserIdentity{UserID: user.ID, Provider: provider, ExternUID: externUID})`.
5. **Race recovery**: if `CreateUserIdentity` returns an error whose message matches one of the known unique-constraint markers (`strings.Contains(err.Error(), "UNIQUE constraint failed")`, `"duplicate key"`, `"Duplicate entry")` — reusing the same pattern as `server/router/api/v1/memo_service.go:103105`):
- `_ = s.Store.DeleteUser(ctx, &store.DeleteUser{ID: user.ID})` (best-effort cleanup of the provisional local user).
- Re-read the winning `user_identity` via `s.Store.GetUserIdentity(ctx, &FindUserIdentity{Provider: &provider, ExternUID: &externUID})`; if still nil, return `codes.Internal` (should not happen under correct semantics).
- Load its user via `s.Store.GetUser(ctx, &FindUser{ID: &winner.UserID})`; set `existingUser`.
6. On any other `CreateUserIdentity` error: best-effort `DeleteUser` cleanup, then return `codes.Internal`.
7. On full success: set `existingUser = user`.
4. Leave the remainder of `SignIn` (row-status check, `doSignIn`, response construction) untouched.
**Boundaries**: Must NOT touch the password-credentials branch; must NOT modify `identifier_filter` logic; must NOT touch `doSignIn`, `SignOut`, or `RefreshToken`; must NOT add new fields to `SignInRequest`/`SignInResponse`.
**Dependencies**: T2, T3, T6 minimum for SQLite confidence; T7 for the derivation helper.
**Expected Outcome**:
- Sign-in with an IdP-issued identifier that fails `base.UIDMatcher` (e.g., `jane@example.com`) succeeds: a `user_identity` row is created, and the local `User.Username` is a derived valid username.
- Repeat sign-in for the same `(provider, extern_uid)` pair loads the same user by linkage, not by username.
- Two IdPs emitting the same `extern_uid` can each link to their own local users without colliding (G2).
**Validation**:
- `go build ./...` — expects PASS.
- `go vet ./...` — expects PASS.
- `go test ./store/test/ -run TestUserIdentity -count=1` — expects PASS (T6 regression check; ensures no store-layer drift).
## Out-of-Scope Tasks
The following are explicitly deferred per `definition.md` / `design.md` and will NOT be attempted during this execution:
- UI or API surfaces for linking/unlinking external identities.
- Update or delete paths for `user_identity` rows.
- Backfill / migration of existing users whose current `Username` matches an IdP identifier.
- Non-OAUTH2 IdP types.
- Protobuf or API changes to `SignInRequest`/`SignInResponse`.
- Adding foreign keys between `user_identity.provider` and `idp.uid`.
- Running PostgreSQL or MySQL integration tests locally (validation commands only cover SQLite, which is the default `DRIVER` in `store/test/store.go`).
File diff suppressed because it is too large Load Diff
@@ -0,0 +1,738 @@
# Calendar-Date Memo Prefill — Implementation Plan
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
**Goal:** When the user picks a date in the activity calendar, the home memo editor pre-fills its `createTime`/`updateTime` to that date and exposes the existing `TimestampPopover` so the user can adjust before saving. Empty calendar dates also become clickable.
**Architecture:** Frontend-only. A new pure helper derives a `Date` from the `displayTime` filter; `PagedMemoList` reads `MemoFilterContext` and passes the derived `Date` into `MemoEditor` via a new `defaultCreateTime` prop; `MemoEditor` seeds its reducer state from the prop, renders the popover when the prop is set in create mode, and re-syncs on prop change. `CalendarCell` drops the `count > 0` gate on click handling.
**Tech Stack:** React 18 + TypeScript, Vite 7, Vitest + jsdom + @testing-library/react, Tailwind v4, react-router. State via React Context + `useReducer`.
**Spec:** `docs/superpowers/specs/2026-05-02-calendar-date-prefill-design.md`
---
## File map
**Created**
- `web/src/components/MemoEditor/utils/deriveDefaultCreateTime.ts` — pure helper `(filters, now?) => Date | undefined`.
- `web/tests/derive-default-create-time.test.ts` — Vitest unit tests for the helper.
- `web/tests/calendar-cell-empty-clickable.test.tsx` — RTL test that count=0 in-month cells are clickable.
**Modified**
- `web/src/components/ActivityCalendar/CalendarCell.tsx` — drop `day.count > 0` gate from click/interactivity/tooltip.
- `web/src/components/MemoEditor/types/components.ts` — add `defaultCreateTime?: Date` to `MemoEditorProps`.
- `web/src/components/MemoEditor/hooks/useMemoInit.ts` — accept optional `defaultCreateTime`; in create mode, dispatch `SET_TIMESTAMPS` to seed `{ createTime, updateTime }`.
- `web/src/components/MemoEditor/index.tsx` — accept the new prop, pass it to `useMemoInit`, add a `useEffect` that re-syncs timestamps when the prop changes in create mode, render `TimestampPopover` when `(!memo && state.timestamps.createTime)`.
- `web/src/components/PagedMemoList/PagedMemoList.tsx` — read `useMemoFilterContext`, derive `defaultCreateTime`, pass to `<MemoEditor>` at line ~155.
---
## Task 1: Pure helper `deriveDefaultCreateTimeFromFilters`
**Files:**
- Create: `web/src/components/MemoEditor/utils/deriveDefaultCreateTime.ts`
- Test: `web/tests/derive-default-create-time.test.ts`
The helper takes the `filters` array from `MemoFilterContext` plus an injectable `now`, finds any `displayTime` filter (value format `YYYY-MM-DD`, local-date — produced by `useDateFilterNavigation` which forwards the `dayjs().format("YYYY-MM-DD")` string from `CalendarDayCell.date`), and returns a `Date` of `selected_date + now's hh:mm:ss`. Returns `undefined` if no `displayTime` filter or the value is malformed.
- [ ] **Step 1: Write the failing tests**
Create `web/tests/derive-default-create-time.test.ts`:
```ts
import { describe, expect, it } from "vitest";
import { deriveDefaultCreateTimeFromFilters } from "@/components/MemoEditor/utils/deriveDefaultCreateTime";
import type { MemoFilter } from "@/contexts/MemoFilterContext";
describe("deriveDefaultCreateTimeFromFilters", () => {
const now = new Date(2026, 4, 2, 14, 32, 10); // 2026-05-02 14:32:10 local
it("returns undefined when no filters are set", () => {
expect(deriveDefaultCreateTimeFromFilters([], now)).toBeUndefined();
});
it("returns undefined when no displayTime filter is present", () => {
const filters: MemoFilter[] = [
{ factor: "tagSearch", value: "work" },
{ factor: "pinned", value: "true" },
];
expect(deriveDefaultCreateTimeFromFilters(filters, now)).toBeUndefined();
});
it("merges the displayTime date with the current local hh:mm:ss", () => {
const filters: MemoFilter[] = [{ factor: "displayTime", value: "2025-05-01" }];
const result = deriveDefaultCreateTimeFromFilters(filters, now);
expect(result).toBeDefined();
expect(result!.getFullYear()).toBe(2025);
expect(result!.getMonth()).toBe(4); // May (0-indexed)
expect(result!.getDate()).toBe(1);
expect(result!.getHours()).toBe(14);
expect(result!.getMinutes()).toBe(32);
expect(result!.getSeconds()).toBe(10);
});
it("ignores extra non-displayTime filters", () => {
const filters: MemoFilter[] = [
{ factor: "tagSearch", value: "work" },
{ factor: "displayTime", value: "2025-05-01" },
{ factor: "pinned", value: "true" },
];
const result = deriveDefaultCreateTimeFromFilters(filters, now);
expect(result?.getDate()).toBe(1);
});
it("returns undefined for a malformed YYYY-MM-DD value", () => {
const cases: MemoFilter[][] = [
[{ factor: "displayTime", value: "not-a-date" }],
[{ factor: "displayTime", value: "2025-13-40" }],
[{ factor: "displayTime", value: "" }],
[{ factor: "displayTime", value: "2025-5-1" }], // single-digit month/day
];
for (const filters of cases) {
expect(deriveDefaultCreateTimeFromFilters(filters, now)).toBeUndefined();
}
});
it("uses real `new Date()` when `now` is omitted", () => {
const filters: MemoFilter[] = [{ factor: "displayTime", value: "2025-05-01" }];
const before = new Date();
const result = deriveDefaultCreateTimeFromFilters(filters);
const after = new Date();
expect(result).toBeDefined();
// Time-of-day should fall between before and after (within 1s tolerance).
const resultTimeOnly = result!.getHours() * 3600 + result!.getMinutes() * 60 + result!.getSeconds();
const beforeTimeOnly = before.getHours() * 3600 + before.getMinutes() * 60 + before.getSeconds();
const afterTimeOnly = after.getHours() * 3600 + after.getMinutes() * 60 + after.getSeconds();
// Handle midnight rollover by allowing any value if before > after.
if (beforeTimeOnly <= afterTimeOnly) {
expect(resultTimeOnly).toBeGreaterThanOrEqual(beforeTimeOnly);
expect(resultTimeOnly).toBeLessThanOrEqual(afterTimeOnly);
}
});
});
```
- [ ] **Step 2: Run tests to verify they fail**
Run: `cd web && pnpm test derive-default-create-time`
Expected: FAIL — module `@/components/MemoEditor/utils/deriveDefaultCreateTime` does not exist.
- [ ] **Step 3: Implement the helper**
Create `web/src/components/MemoEditor/utils/deriveDefaultCreateTime.ts`:
```ts
import type { MemoFilter } from "@/contexts/MemoFilterContext";
const DATE_RE = /^(\d{4})-(\d{2})-(\d{2})$/;
/**
* Derive a default `createTime` for a new memo from the active memo filters.
* If a `displayTime:YYYY-MM-DD` filter is present, returns that local date
* combined with `now`'s wall-clock hh:mm:ss. Returns undefined otherwise or
* when the filter value is malformed.
*/
export function deriveDefaultCreateTimeFromFilters(
filters: MemoFilter[],
now: Date = new Date(),
): Date | undefined {
const dateFilter = filters.find((f) => f.factor === "displayTime");
if (!dateFilter) return undefined;
const match = DATE_RE.exec(dateFilter.value);
if (!match) return undefined;
const year = Number(match[1]);
const month = Number(match[2]);
const day = Number(match[3]);
// Construct a local-time Date and verify the components round-trip
// (catches things like 2025-13-40 that JS would silently roll forward).
const candidate = new Date(year, month - 1, day, now.getHours(), now.getMinutes(), now.getSeconds());
if (
candidate.getFullYear() !== year ||
candidate.getMonth() !== month - 1 ||
candidate.getDate() !== day
) {
return undefined;
}
return candidate;
}
```
- [ ] **Step 4: Run tests to verify they pass**
Run: `cd web && pnpm test derive-default-create-time`
Expected: PASS — all 6 cases.
- [ ] **Step 5: Run lint**
Run: `cd web && pnpm lint`
Expected: PASS (TypeScript + Biome happy).
- [ ] **Step 6: Commit**
```bash
git add web/src/components/MemoEditor/utils/deriveDefaultCreateTime.ts web/tests/derive-default-create-time.test.ts
git commit -m "feat(memo-editor): add deriveDefaultCreateTimeFromFilters helper"
```
---
## Task 2: Make empty calendar cells clickable
**Files:**
- Modify: `web/src/components/ActivityCalendar/CalendarCell.tsx`
- Test: `web/tests/calendar-cell-empty-clickable.test.tsx`
`CalendarCell` currently gates `handleClick`, `isInteractive`, and `shouldShowTooltip` on `day.count > 0`. Drop those gates so any in-month cell is clickable when `onClick` is provided. Out-of-month cells (early-returned at line ~38) stay unclickable.
- [ ] **Step 1: Write the failing test**
Create `web/tests/calendar-cell-empty-clickable.test.tsx`:
```tsx
import { fireEvent, render, screen } from "@testing-library/react";
import { describe, expect, it, vi } from "vitest";
import { CalendarCell } from "@/components/ActivityCalendar/CalendarCell";
import type { CalendarDayCell } from "@/components/ActivityCalendar/types";
const makeDay = (overrides: Partial<CalendarDayCell> = {}): CalendarDayCell => ({
date: "2025-05-01",
label: "1",
count: 0,
isCurrentMonth: true,
isToday: false,
isSelected: false,
...overrides,
});
describe("CalendarCell empty-day clickability", () => {
it("fires onClick for an in-month day with count=0", () => {
const onClick = vi.fn();
render(<CalendarCell day={makeDay()} maxCount={5} tooltipText="May 1, 2025" onClick={onClick} />);
const button = screen.getByRole("button", { name: /May 1, 2025/ });
fireEvent.click(button);
expect(onClick).toHaveBeenCalledWith("2025-05-01");
});
it("renders an empty in-month day as interactive (tabIndex 0, not aria-disabled)", () => {
render(<CalendarCell day={makeDay()} maxCount={5} tooltipText="May 1, 2025" onClick={() => {}} />);
const button = screen.getByRole("button", { name: /May 1, 2025/ });
expect(button).toHaveAttribute("tabindex", "0");
expect(button).toHaveAttribute("aria-disabled", "false");
});
it("still renders a populated in-month day as interactive", () => {
const onClick = vi.fn();
render(<CalendarCell day={makeDay({ count: 3 })} maxCount={5} tooltipText="May 1, 2025" onClick={onClick} />);
fireEvent.click(screen.getByRole("button", { name: /May 1, 2025/ }));
expect(onClick).toHaveBeenCalledWith("2025-05-01");
});
it("does not render out-of-month days as interactive (no role=button)", () => {
render(
<CalendarCell
day={makeDay({ isCurrentMonth: false })}
maxCount={5}
tooltipText="May 1, 2025"
onClick={() => {}}
/>,
);
expect(screen.queryByRole("button")).toBeNull();
});
});
```
- [ ] **Step 2: Run test to verify it fails**
Run: `cd web && pnpm test calendar-cell-empty-clickable`
Expected: FAIL — first two tests fail because the empty cell currently has `tabindex="-1"`, `aria-disabled="true"`, and `onClick` is not invoked.
- [ ] **Step 3: Edit `CalendarCell.tsx` to drop the count gate**
Modify `web/src/components/ActivityCalendar/CalendarCell.tsx`. Three edits:
(a) Replace `handleClick`:
```tsx
const handleClick = () => {
if (onClick) {
onClick(day.date);
}
};
```
(b) Replace `isInteractive`:
```tsx
const isInteractive = Boolean(onClick);
```
(c) Replace `shouldShowTooltip`:
```tsx
const shouldShowTooltip = tooltipText && !disableTooltip;
```
Leave the out-of-month early return (`if (!day.isCurrentMonth) { ... }`) untouched — out-of-month cells remain non-buttons.
- [ ] **Step 4: Run test to verify it passes**
Run: `cd web && pnpm test calendar-cell-empty-clickable`
Expected: PASS — all 4 cases.
- [ ] **Step 5: Run the full test suite to catch regressions**
Run: `cd web && pnpm test`
Expected: PASS. The existing `activity-calendar-tooltip.test.ts` covers `getTooltipText` (a separate utility) and shouldn't be affected.
- [ ] **Step 6: Run lint**
Run: `cd web && pnpm lint`
Expected: PASS.
- [ ] **Step 7: Commit**
```bash
git add web/src/components/ActivityCalendar/CalendarCell.tsx web/tests/calendar-cell-empty-clickable.test.tsx
git commit -m "feat(activity-calendar): allow clicking empty in-month dates"
```
---
## Task 3: Add `defaultCreateTime` prop to `MemoEditorProps`
**Files:**
- Modify: `web/src/components/MemoEditor/types/components.ts`
Type-only change. No tests at this step — TypeScript compiler is the gate. Subsequent tasks consume the prop.
- [ ] **Step 1: Add the prop**
Modify `web/src/components/MemoEditor/types/components.ts`. Replace the `MemoEditorProps` interface (lines 616) with:
```ts
export interface MemoEditorProps {
className?: string;
cacheKey?: string;
placeholder?: string;
/** Existing memo to edit. When provided, the editor initializes from it without fetching. */
memo?: Memo;
parentMemoName?: string;
autoFocus?: boolean;
/**
* Default `createTime` for a *new* memo (create mode only). When set, the
* editor seeds both `createTime` and `updateTime` to this value and renders
* the timestamp popover so the user can adjust before saving. Tracked live:
* if the prop changes after mount, the editor's timestamps re-sync. Ignored
* in edit mode (when `memo` is set).
*/
defaultCreateTime?: Date;
onConfirm?: (memoName: string) => void;
onCancel?: () => void;
}
```
- [ ] **Step 2: Verify compilation**
Run: `cd web && pnpm lint`
Expected: PASS (no consumer changes yet, prop is optional).
- [ ] **Step 3: Commit**
```bash
git add web/src/components/MemoEditor/types/components.ts
git commit -m "feat(memo-editor): add defaultCreateTime prop type"
```
---
## Task 4: Seed editor timestamps in `useMemoInit` (create mode)
**Files:**
- Modify: `web/src/components/MemoEditor/hooks/useMemoInit.ts`
`useMemoInit` currently handles edit mode (`if (memo)`) by calling `memoService.fromMemo(memo)` and `actions.initMemo(...)`. In create mode it only restores cached content and sets default visibility — it never touches timestamps. Extend the create branch so that when `defaultCreateTime` is set, it dispatches `SET_TIMESTAMPS` with `{ createTime: defaultCreateTime, updateTime: defaultCreateTime }`. This handles the *initial mount* case.
- [ ] **Step 1: Update `UseMemoInitOptions` and `useMemoInit`**
Modify `web/src/components/MemoEditor/hooks/useMemoInit.ts`. Replace the entire file with:
```ts
import { useEffect, useRef, useState } from "react";
import type { Memo, Visibility } from "@/types/proto/api/v1/memo_service_pb";
import type { EditorRefActions } from "../Editor";
import { cacheService, memoService } from "../services";
import { useEditorContext } from "../state";
interface UseMemoInitOptions {
editorRef: React.RefObject<EditorRefActions | null>;
memo?: Memo;
cacheKey?: string;
username: string;
autoFocus?: boolean;
defaultVisibility?: Visibility;
defaultCreateTime?: Date;
}
export const useMemoInit = ({
editorRef,
memo,
cacheKey,
username,
autoFocus,
defaultVisibility,
defaultCreateTime,
}: UseMemoInitOptions) => {
const { actions, dispatch } = useEditorContext();
const initializedRef = useRef(false);
const [isInitialized, setIsInitialized] = useState(false);
useEffect(() => {
if (initializedRef.current) return;
initializedRef.current = true;
const key = cacheService.key(username, cacheKey);
if (memo) {
const initialState = memoService.fromMemo(memo);
cacheService.clear(key);
dispatch(actions.initMemo(initialState));
} else {
const cachedContent = cacheService.load(key);
if (cachedContent) {
dispatch(actions.updateContent(cachedContent));
}
if (defaultVisibility !== undefined) {
dispatch(actions.setMetadata({ visibility: defaultVisibility }));
}
if (defaultCreateTime) {
dispatch(actions.setTimestamps({ createTime: defaultCreateTime, updateTime: defaultCreateTime }));
}
}
if (autoFocus) {
setTimeout(() => editorRef.current?.focus(), 100);
}
setIsInitialized(true);
}, [memo, cacheKey, username, autoFocus, defaultVisibility, defaultCreateTime, actions, dispatch, editorRef]);
return { isInitialized };
};
```
Notes:
- The `defaultCreateTime` dependency is added to the effect's deps to satisfy the linter, but `initializedRef` ensures the body runs only once. Live re-sync after mount is handled by a separate effect in Task 5.
- Edit mode is unchanged — `defaultCreateTime` is intentionally ignored when `memo` is set.
- [ ] **Step 2: Verify compilation**
Run: `cd web && pnpm lint`
Expected: PASS.
- [ ] **Step 3: Commit**
```bash
git add web/src/components/MemoEditor/hooks/useMemoInit.ts
git commit -m "feat(memo-editor): seed timestamps from defaultCreateTime on init"
```
---
## Task 5: Wire `defaultCreateTime` through `MemoEditor` and render popover
**Files:**
- Modify: `web/src/components/MemoEditor/index.tsx`
Three changes:
1. Destructure `defaultCreateTime` from props.
2. Pass it into `useMemoInit`.
3. Add a `useEffect` that dispatches `setTimestamps` whenever `defaultCreateTime` changes after mount (live re-sync per design Q3-A). Skip when `memo` is set.
4. Update the popover render condition so it shows in create mode too when timestamps are seeded.
- [ ] **Step 1: Destructure the prop and pass it to `useMemoInit`**
In `web/src/components/MemoEditor/index.tsx`, update the `MemoEditorImpl` destructuring (around line 42):
```tsx
const MemoEditorImpl: React.FC<MemoEditorProps> = ({
className,
cacheKey,
memo,
parentMemoName,
autoFocus,
placeholder,
defaultCreateTime,
onConfirm,
onCancel,
}) => {
```
And update the `useMemoInit` call (around line 71):
```tsx
const { isInitialized } = useMemoInit({
editorRef,
memo,
cacheKey,
username: currentUser?.name ?? "",
autoFocus,
defaultVisibility,
defaultCreateTime,
});
```
- [ ] **Step 2: Add the live re-sync effect**
In the same file, add a new `useEffect` after `useMemoInit` (and after `useAutoSave`) that re-syncs timestamps when `defaultCreateTime` changes in create mode. Place it just before the existing `useEffect` that fetches AI settings (around line 80):
```tsx
// Live-sync the draft's createTime/updateTime to the calendar-derived prop.
// Only applies in create mode; edit mode owns its own timestamps. Runs after
// initial mount (the seed value is set in useMemoInit), and again whenever
// the prop changes — e.g., when the user picks a different calendar date
// while the editor is open.
useEffect(() => {
if (memo) return;
if (!isInitialized) return;
dispatch(
actions.setTimestamps({
createTime: defaultCreateTime,
updateTime: defaultCreateTime,
}),
);
}, [defaultCreateTime, memo, isInitialized, actions, dispatch]);
```
Notes:
- We pass `undefined` through when the prop becomes undefined (filter cleared) — this resets timestamps to undefined so the editor falls back to "server-stamped now" on save, exactly the pre-feature behavior.
- The `isInitialized` guard avoids racing with `useMemoInit`'s one-shot seed.
- [ ] **Step 3: Update the popover render condition**
In the same file, find the existing block (around line 294):
```tsx
{memoName && (
<div className="w-full -mb-1">
<TimestampPopover />
</div>
)}
```
Replace with:
```tsx
{(memoName || (!memo && state.timestamps.createTime)) && (
<div className="w-full -mb-1">
<TimestampPopover />
</div>
)}
```
Now the popover renders in edit mode (unchanged) AND in create mode whenever a default timestamp has been seeded.
- [ ] **Step 4: Verify compilation**
Run: `cd web && pnpm lint`
Expected: PASS.
- [ ] **Step 5: Run all tests**
Run: `cd web && pnpm test`
Expected: PASS (no editor-specific tests added; existing tests continue to pass).
- [ ] **Step 6: Commit**
```bash
git add web/src/components/MemoEditor/index.tsx
git commit -m "feat(memo-editor): live-sync timestamps and reveal popover from defaultCreateTime"
```
---
## Task 6: Wire calendar selection through `PagedMemoList`
**Files:**
- Modify: `web/src/components/PagedMemoList/PagedMemoList.tsx`
The home memo editor is rendered at line ~155 of `PagedMemoList.tsx`. Read the current `MemoFilterContext` filters, derive the `defaultCreateTime`, and pass it to `<MemoEditor>`. Wrap in `useMemo` so the reference stays stable when filters don't change (avoids re-firing the live-sync effect).
- [ ] **Step 1: Add the imports and derivation**
Modify `web/src/components/PagedMemoList/PagedMemoList.tsx`. Add to the existing imports near the top:
```tsx
import { useMemo } from "react";
import { useMemoFilterContext } from "@/contexts/MemoFilterContext";
import { deriveDefaultCreateTimeFromFilters } from "@/components/MemoEditor/utils/deriveDefaultCreateTime";
```
If `useMemo` is already imported from `react` in this file, merge into the existing import rather than duplicating.
Inside the component body (above the `children` JSX, near the top of the function), add:
```tsx
const { filters } = useMemoFilterContext();
const defaultCreateTime = useMemo(
() => deriveDefaultCreateTimeFromFilters(filters),
[filters],
);
```
- [ ] **Step 2: Pass the prop to `<MemoEditor>`**
Replace the existing line ~155:
```tsx
{showMemoEditor ? <MemoEditor className="mb-2" cacheKey="home-memo-editor" placeholder={t("editor.any-thoughts")} /> : null}
```
with:
```tsx
{showMemoEditor ? (
<MemoEditor
className="mb-2"
cacheKey="home-memo-editor"
placeholder={t("editor.any-thoughts")}
defaultCreateTime={defaultCreateTime}
/>
) : null}
```
- [ ] **Step 3: Verify compilation**
Run: `cd web && pnpm lint`
Expected: PASS.
- [ ] **Step 4: Run all tests**
Run: `cd web && pnpm test`
Expected: PASS.
- [ ] **Step 5: Commit**
```bash
git add web/src/components/PagedMemoList/PagedMemoList.tsx
git commit -m "feat(home): pass calendar-selected date as default createTime to memo editor"
```
---
## Task 7: Manual smoke test
No code changes. Per `CLAUDE.md`: "For UI or frontend changes, start the dev server and use the feature in a browser before reporting the task as complete." Walk through the user-visible flows.
- [ ] **Step 1: Start the backend and frontend**
In one terminal:
```bash
go run ./cmd/memos --port 8081
```
In another:
```bash
cd web && pnpm dev
```
Open `http://localhost:3001`.
- [ ] **Step 2: Smoke — populated past date**
1. Sign in. Confirm the activity calendar is visible (statistics view).
2. Click a *past* date that already has memos.
3. Confirm the URL gains `?filter=displayTime:YYYY-MM-DD`.
4. Confirm the home memo editor shows the timestamp popover above the textarea, populated with the selected date.
5. Type a memo and click Save.
6. Clear the date filter (chip X). Reapply by clicking the same date.
7. Confirm the new memo appears under that date.
- [ ] **Step 3: Smoke — empty past date**
1. Pick a past date with **zero memos** in the calendar (lightest cell).
2. Confirm it is now clickable, the URL filter applies, and the empty-state shows.
3. Type and save a memo.
4. Confirm the memo appears for that date and the calendar cell tints up.
- [ ] **Step 4: Smoke — future date**
1. Click a future date in the current month.
2. Confirm the popover shows that date.
3. Save a memo. Confirm it appears under that date.
- [ ] **Step 5: Smoke — clear filter mid-draft**
1. Pick May 1 (or any non-today date). Type some content, do **not** save.
2. Click the filter chip X to clear the date filter.
3. Confirm the popover disappears and the draft content is preserved (autoSave behavior).
4. Save and confirm the memo gets a server-stamped "now" timestamp (i.e., appears under today).
- [ ] **Step 6: Smoke — change filter mid-draft**
1. Pick May 1. Type content.
2. Without saving, click May 3.
3. Confirm the popover updates to May 3.
4. Save. Confirm the memo appears under May 3.
- [ ] **Step 7: Smoke — comment editor unaffected**
1. Open any memo's detail view (or open the comments thread).
2. Confirm the reply editor does **not** show a timestamp popover.
3. Confirm the date filter has no visible effect on the reply editor.
- [ ] **Step 8: Smoke — edit mode unaffected**
1. Edit an existing memo (pencil icon).
2. Confirm the existing timestamp popover still works exactly as before, regardless of any active calendar filter.
- [ ] **Step 9: Smoke — empty-date click on Explore page**
1. Navigate to the Explore page (which also renders the calendar).
2. Click an empty date.
3. Confirm the URL filter applies and the empty-state shows. (No editor on Explore — that's correct.)
- [ ] **Step 10: Record results**
Note any unexpected behavior (especially: selection-ring contrast on the lowest-intensity background, mentioned as a flag in the spec). If the ring is too subtle, file a follow-up — *not* part of this plan.
---
## Self-review
**Spec coverage check:**
| Spec requirement | Task |
|---|---|
| Empty calendar dates clickable | Task 2 |
| Editor shows TimestampPopover in create mode when filter active | Task 5 (popover condition) |
| `createTime` = selected date + current local hh:mm:ss | Task 1 (helper) + Task 4 (seed) |
| `updateTime` mirrored to same value | Task 4 (seed) + Task 5 (live sync) |
| Live-derived: filter change re-syncs timestamps | Task 5 (live-sync `useEffect`) |
| Filter cleared → undefined → server-stamped "now" | Task 5 (passes `undefined` through) + Task 6 (helper returns undefined) |
| Future dates allowed (no clamp) | Task 1 (no clamp in helper); confirmed in Task 7 step 4 |
| Comment editor unaffected | Task 6 wires only `PagedMemoList`; confirmed in Task 7 step 7 |
| Edit mode unaffected | Task 4 + 5 explicitly guard on `memo`; confirmed in Task 7 step 8 |
| Empty-date click on Explore/Archived/Profile | Task 2 (calendar-side change); confirmed in Task 7 step 9 |
| DST/timezone uses local time | Task 1 (`new Date(y, m-1, d, h, mi, s)`) |
| Helper unit tests | Task 1 |
| `CalendarCell` empty-cell test | Task 2 |
| Manual smoke | Task 7 |
All spec requirements have a task. No gaps.
**Placeholder scan:** No "TBD"/"TODO" steps. Every code step shows the actual code.
**Type/name consistency check:**
- `MemoFilter` and `useMemoFilterContext` exist at `@/contexts/MemoFilterContext` (verified during exploration).
- `editorActions.setTimestamps` exists in `state/actions.ts:75` and accepts `Partial<EditorState["timestamps"]>` (verified). Calls in Tasks 4 and 5 match.
- `state.timestamps.createTime` is `Date | undefined` (verified `state/types.ts:27-29`). The popover render condition uses it as a truthy guard — `Date` instances are truthy, `undefined` is falsy.
- `useMemoFilterContext` (alias used in PagedMemoList) is exported from `MemoFilterContext.tsx:151` (verified).
- `deriveDefaultCreateTimeFromFilters` signature is identical between Task 1 (definition) and Task 6 (consumer).
- `defaultCreateTime: Date | undefined` flows consistently through `MemoEditorProps` (Task 3) → `MemoEditorImpl` destructuring (Task 5) → `useMemoInit` options (Task 4) → reducer payloads.
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
@@ -0,0 +1,774 @@
# First Screen Lazy Heavy Dependencies Implementation Plan
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
**Goal:** Keep Mermaid, Leaflet, React Leaflet, marker cluster, and feature CSS out of the auth/signup initial screen while preserving diagram, math, and map behavior when those features are used.
**Architecture:** Move runtime Leaflet objects behind plain coordinate data at parent boundaries, then lazy-load map implementations from small wrapper components. Replace Mermaid's static import with effect-time dynamic import, and move KaTeX/Leaflet CSS from the app entry into feature modules.
**Tech Stack:** React 19, TypeScript 6, Vite 8/Rolldown, React Router 7, React Query 5, Mermaid, KaTeX, Leaflet, React Leaflet, pnpm.
---
## File Structure
- Create `web/src/components/map/types.ts`
- Defines a lightweight `MapPoint` interface used by parents that must not import Leaflet.
- Create `web/src/components/map/LazyLocationPicker.tsx`
- Provides a `React.lazy` boundary and map-sized fallback for `LocationPicker`.
- Modify `web/src/main.tsx`
- Removes global Leaflet and KaTeX CSS imports.
- Modify `web/src/router/index.tsx`
- Lazy-loads the Home route so memo/editor/markdown modules are not part of the auth/signup entry graph.
- Modify `web/vite.config.mts`
- Keeps optional heavy dependency split groups from becoming entry preloads.
- Modify `web/src/components/map/LocationPicker.tsx`
- Imports Leaflet CSS inside the lazy implementation path.
- Accepts and emits plain `MapPoint` values at the public component boundary.
- Keeps `LatLng` runtime use internal to the map implementation.
- Modify `web/src/components/map/index.ts`
- Stops exporting `LocationPicker` and map utility helpers from the barrel so non-map imports do not pull Leaflet into parent chunks.
- Keeps exporting `useReverseGeocoding` because it has no Leaflet dependency.
- Modify `web/src/components/MemoEditor/types/insert-menu.ts`
- Replaces `LatLng` in editor state with `MapPoint`.
- Modify `web/src/components/MemoEditor/hooks/useLocation.ts`
- Removes runtime Leaflet import and stores plain coordinates.
- Modify `web/src/components/MemoEditor/Toolbar/InsertMenu.tsx`
- Removes runtime Leaflet import.
- Imports `useReverseGeocoding` directly from its file.
- Constructs plain coordinates from geolocation.
- Modify `web/src/components/MemoMetadata/Location/LocationDialog.tsx`
- Uses `LazyLocationPicker` and `MapPoint`.
- Only mounts the lazy picker while the dialog is open.
- Modify `web/src/components/MemoMetadata/Location/LocationDisplayView.tsx`
- Removes runtime Leaflet import and uses `LazyLocationPicker` inside the popover.
- Modify `web/src/pages/UserProfile.tsx`
- Lazy-loads `UserMemoMap` only when the map tab is active.
- Modify `web/src/components/MemoContent/MemoMarkdownRenderer.tsx`
- Imports KaTeX CSS from the markdown-rendering path.
- Modify `web/src/components/MemoContent/MermaidBlock.tsx`
- Dynamically imports Mermaid only when rendering a Mermaid block.
- Modify `web/src/components/UserMemoMap/UserMemoMap.tsx`
- Keeps marker cluster CSS in the lazy map implementation path.
- Verify with `pnpm lint`, `pnpm build`, and a production preview/browser network check.
## Task 1: Remove Leaflet From Editor Location State
**Files:**
- Create: `web/src/components/map/types.ts`
- Modify: `web/src/components/MemoEditor/types/insert-menu.ts`
- Modify: `web/src/components/MemoEditor/hooks/useLocation.ts`
- Modify: `web/src/components/MemoEditor/Toolbar/InsertMenu.tsx`
- [x] **Step 1: Add a Leaflet-free map coordinate type**
Create `web/src/components/map/types.ts`:
```ts
export interface MapPoint {
lat: number;
lng: number;
}
```
- [x] **Step 2: Replace editor location state type**
In `web/src/components/MemoEditor/types/insert-menu.ts`, replace the entire file with:
```ts
import type { MapPoint } from "@/components/map/types";
export interface LocationState {
placeholder: string;
position?: MapPoint;
latInput: string;
lngInput: string;
}
```
- [x] **Step 3: Remove Leaflet runtime usage from `useLocation`**
In `web/src/components/MemoEditor/hooks/useLocation.ts`, remove `import { LatLng } from "leaflet";`, add a type import, and use plain coordinate objects:
```ts
import { create } from "@bufbuild/protobuf";
import { useCallback, useMemo, useRef, useState } from "react";
import type { MapPoint } from "@/components/map/types";
import { Location, LocationSchema } from "@/types/proto/api/v1/memo_service_pb";
import { LocationState } from "../types/insert-menu";
export const useLocation = (initialLocation?: Location) => {
const [locationInitialized, setLocationInitialized] = useState(false);
const locationInitializedRef = useRef(locationInitialized);
locationInitializedRef.current = locationInitialized;
const [state, setState] = useState<LocationState>({
placeholder: initialLocation?.placeholder || "",
position: initialLocation ? { lat: initialLocation.latitude, lng: initialLocation.longitude } : undefined,
latInput: initialLocation ? String(initialLocation.latitude) : "",
lngInput: initialLocation ? String(initialLocation.longitude) : "",
});
const stateRef = useRef(state);
stateRef.current = state;
const updatePosition = useCallback((position?: MapPoint) => {
setState((prev) => ({
...prev,
position,
latInput: position ? String(position.lat) : "",
lngInput: position ? String(position.lng) : "",
}));
}, []);
const handlePositionChange = useCallback(
(position: MapPoint) => {
if (!locationInitializedRef.current) setLocationInitialized(true);
updatePosition(position);
},
[updatePosition],
);
const updateCoordinate = useCallback((type: "lat" | "lng", value: string) => {
const num = parseFloat(value);
const isValid = type === "lat" ? !isNaN(num) && num >= -90 && num <= 90 : !isNaN(num) && num >= -180 && num <= 180;
setState((prev) => {
const next = { ...prev, [type === "lat" ? "latInput" : "lngInput"]: value };
if (isValid && prev.position) {
const newPos = type === "lat" ? { lat: num, lng: prev.position.lng } : { lat: prev.position.lat, lng: num };
return { ...next, position: newPos, latInput: String(newPos.lat), lngInput: String(newPos.lng) };
}
return next;
});
}, []);
const setPlaceholder = useCallback((placeholder: string) => {
setState((prev) => ({ ...prev, placeholder }));
}, []);
const reset = useCallback(() => {
setState({
placeholder: "",
position: undefined,
latInput: "",
lngInput: "",
});
setLocationInitialized(false);
}, []);
const getLocation = useCallback((): Location | undefined => {
const { position, placeholder } = stateRef.current;
if (!position || !placeholder.trim()) {
return undefined;
}
return create(LocationSchema, {
latitude: position.lat,
longitude: position.lng,
placeholder,
});
}, []);
return useMemo(
() => ({ state, locationInitialized, handlePositionChange, updateCoordinate, setPlaceholder, reset, getLocation }),
[state, locationInitialized, handlePositionChange, updateCoordinate, setPlaceholder, reset, getLocation],
);
};
```
- [x] **Step 4: Remove Leaflet import from `InsertMenu`**
In `web/src/components/MemoEditor/Toolbar/InsertMenu.tsx`:
Remove:
```ts
import { LatLng } from "leaflet";
```
Replace:
```ts
import { useReverseGeocoding } from "@/components/map";
```
with:
```ts
import { useReverseGeocoding } from "@/components/map/useReverseGeocoding";
```
Replace the geolocation success handler body:
```ts
handleLocationPositionChange(new LatLng(position.coords.latitude, position.coords.longitude));
```
with:
```ts
handleLocationPositionChange({ lat: position.coords.latitude, lng: position.coords.longitude });
```
- [x] **Step 5: Run focused type check**
Run:
```bash
cd web && pnpm lint
```
Expected: TypeScript may still fail because map component props have not been converted yet. If the only failures are `LatLng`/`MapPoint` mismatches in map/location components, continue to Task 2. If unrelated failures appear, stop and inspect before editing further.
## Task 2: Lazy-Load Location Picker And Keep Leaflet Inside Map Modules
**Files:**
- Create: `web/src/components/map/LazyLocationPicker.tsx`
- Modify: `web/src/components/map/LocationPicker.tsx`
- Modify: `web/src/components/map/index.ts`
- Modify: `web/src/components/MemoMetadata/Location/LocationDialog.tsx`
- Modify: `web/src/components/MemoMetadata/Location/LocationDisplayView.tsx`
- [x] **Step 1: Create lazy location picker wrapper**
Create `web/src/components/map/LazyLocationPicker.tsx`:
```tsx
import { lazy, Suspense } from "react";
import { cn } from "@/lib/utils";
import type { MapPoint } from "./types";
interface LazyLocationPickerProps {
readonly?: boolean;
latlng?: MapPoint;
onChange?: (position: MapPoint) => void;
className?: string;
}
const LocationPicker = lazy(() => import("./LocationPicker"));
export const LazyLocationPicker = ({ className, ...props }: LazyLocationPickerProps) => {
return (
<Suspense
fallback={
<div
className={cn(
"memo-location-map relative isolate h-72 w-full overflow-hidden rounded-xl border border-border bg-muted/30 shadow-sm",
className,
)}
/>
}
>
<LocationPicker className={className} {...props} />
</Suspense>
);
};
```
- [x] **Step 2: Convert `LocationPicker` public props to `MapPoint`**
In `web/src/components/map/LocationPicker.tsx`:
Add CSS and type imports near the top:
```ts
import "leaflet/dist/leaflet.css";
import type { MapPoint } from "./types";
```
Keep Leaflet runtime import:
```ts
import L, { LatLng } from "leaflet";
```
Add helper functions after imports:
```ts
const toLatLng = (point: MapPoint): LatLng => new LatLng(point.lat, point.lng);
const fromLatLng = (latlng: LatLng): MapPoint => ({ lat: latlng.lat, lng: latlng.lng });
```
Update public-facing props:
```ts
interface LocationMarkerProps {
position: LatLng | undefined;
onChange: (position: MapPoint) => void;
readonly?: boolean;
}
```
In `LocationMarker`, replace:
```ts
onChange(e.latlng);
```
with:
```ts
onChange(fromLatLng(e.latlng));
```
Update `MapControlsProps`:
```ts
interface MapControlsProps {
position: MapPoint | undefined;
}
```
Update `LocationPickerProps`:
```ts
interface LocationPickerProps {
readonly?: boolean;
latlng?: MapPoint;
onChange?: (position: MapPoint) => void;
className?: string;
}
```
Replace:
```ts
const DEFAULT_CENTER_LAT_LNG = new LatLng(48.8584, 2.2945);
```
with:
```ts
const DEFAULT_CENTER: MapPoint = { lat: 48.8584, lng: 2.2945 };
```
Inside `LocationPicker`, replace:
```ts
const position = latlng || DEFAULT_CENTER_LAT_LNG;
```
with:
```ts
const position = latlng || DEFAULT_CENTER;
const mapCenter = toLatLng(position);
const markerPosition = latlng ? toLatLng(latlng) : mapCenter;
```
Replace the `MapContainer` props and marker call:
```tsx
<MapContainer
className="h-full w-full !bg-muted"
center={mapCenter}
zoom={13}
scrollWheelZoom={false}
zoomControl={false}
attributionControl={false}
>
<ThemedTileLayer />
<LocationMarker position={markerPosition} readonly={readOnly} onChange={onChange} />
<MapControls position={latlng} />
<MapCleanup />
</MapContainer>
```
- [x] **Step 3: Keep the map barrel Leaflet-free**
Replace `web/src/components/map/index.ts` with:
```ts
export { useReverseGeocoding } from "./useReverseGeocoding";
```
Do not export `LocationPicker`, `LazyLocationPicker`, `map-utils`, or Leaflet helpers from this barrel. Import map UI directly from `@/components/map/LazyLocationPicker` or the implementation file.
- [x] **Step 4: Use lazy picker in `LocationDialog`**
In `web/src/components/MemoMetadata/Location/LocationDialog.tsx`:
Remove:
```ts
import type { LatLng } from "leaflet";
import { LocationPicker } from "@/components/map";
```
Add:
```ts
import { LazyLocationPicker } from "@/components/map/LazyLocationPicker";
import type { MapPoint } from "@/components/map/types";
```
Change the prop type:
```ts
onPositionChange: (position: MapPoint) => void;
```
Replace:
```tsx
<LocationPicker className="h-full" latlng={position} onChange={onPositionChange} />
```
with:
```tsx
{open && <LazyLocationPicker className="h-full" latlng={position} onChange={onPositionChange} />}
```
- [x] **Step 5: Use lazy picker in `LocationDisplayView`**
In `web/src/components/MemoMetadata/Location/LocationDisplayView.tsx`:
Remove:
```ts
import { LatLng } from "leaflet";
import { LocationPicker } from "@/components/map";
```
Add:
```ts
import { LazyLocationPicker } from "@/components/map/LazyLocationPicker";
```
Replace:
```tsx
<LocationPicker latlng={new LatLng(location.latitude, location.longitude)} readonly={true} />
```
with:
```tsx
{popoverOpen && <LazyLocationPicker latlng={{ lat: location.latitude, lng: location.longitude }} readonly={true} />}
```
- [x] **Step 6: Run lint**
Run:
```bash
cd web && pnpm lint
```
Expected: PASS for the files changed so far, or only failures unrelated to this work. Fix any `MapPoint`/`LatLng` type errors before continuing.
## Task 3: Lazy-Load Profile Map
**Files:**
- Modify: `web/src/pages/UserProfile.tsx`
- Modify: `web/src/components/UserMemoMap/UserMemoMap.tsx`
- [x] **Step 1: Move `UserMemoMap` behind `React.lazy`**
In `web/src/pages/UserProfile.tsx`, replace the React import section:
```ts
import copy from "copy-to-clipboard";
import { ExternalLinkIcon, LayoutListIcon, type LucideIcon, MapIcon } from "lucide-react";
import { toast } from "react-hot-toast";
```
with:
```ts
import copy from "copy-to-clipboard";
import { lazy, Suspense } from "react";
import { ExternalLinkIcon, LayoutListIcon, type LucideIcon, MapIcon } from "lucide-react";
import { toast } from "react-hot-toast";
```
Remove:
```ts
import UserMemoMap from "@/components/UserMemoMap";
```
Add after the `type TabView = "memos" | "map";` line:
```ts
const UserMemoMap = lazy(() => import("@/components/UserMemoMap"));
```
Replace:
```tsx
<div className="">
<UserMemoMap creator={user.name} className="h-[60dvh] sm:h-[500px] rounded-xl" />
</div>
```
with:
```tsx
<div className="">
<Suspense fallback={<div className="h-[60dvh] sm:h-[500px] rounded-xl border border-border bg-muted/30" />}>
<UserMemoMap creator={user.name} className="h-[60dvh] sm:h-[500px] rounded-xl" />
</Suspense>
</div>
```
- [x] **Step 2: Keep Leaflet CSS in the lazy profile map implementation**
In `web/src/components/UserMemoMap/UserMemoMap.tsx`, add the Leaflet CSS import above marker cluster CSS:
```ts
import "leaflet/dist/leaflet.css";
import "leaflet.markercluster/dist/MarkerCluster.css";
```
The file should continue to import Leaflet, React Leaflet, marker clustering, and `map-utils` directly because this component is now loaded only from a lazy boundary.
- [x] **Step 3: Run lint**
Run:
```bash
cd web && pnpm lint
```
Expected: PASS, or only unrelated pre-existing failures.
## Task 4: Move KaTeX CSS And Dynamically Import Mermaid
**Files:**
- Modify: `web/src/main.tsx`
- Modify: `web/src/components/MemoContent/MemoMarkdownRenderer.tsx`
- Modify: `web/src/components/MemoContent/MermaidBlock.tsx`
- [x] **Step 1: Remove feature CSS from app entry**
In `web/src/main.tsx`, remove:
```ts
import "leaflet/dist/leaflet.css";
import "katex/dist/katex.min.css";
```
- [x] **Step 2: Load KaTeX CSS from markdown renderer**
In `web/src/components/MemoContent/MemoMarkdownRenderer.tsx`, add this import with the existing dependency imports:
```ts
import "katex/dist/katex.min.css";
```
- [x] **Step 3: Remove static Mermaid import**
In `web/src/components/MemoContent/MermaidBlock.tsx`, remove:
```ts
import mermaid from "mermaid";
```
- [x] **Step 4: Replace Mermaid initialization/render effects with dynamic import**
In `web/src/components/MemoContent/MermaidBlock.tsx`, remove the existing Mermaid initialization effect:
```ts
useEffect(() => {
mermaid.initialize({
startOnLoad: false,
theme: toMermaidTheme(currentTheme),
securityLevel: "strict",
fontFamily: "inherit",
suppressErrorRendering: true,
});
}, [currentTheme]);
```
Replace the existing render effect with:
```ts
useEffect(() => {
if (!codeContent) return;
let cancelled = false;
const renderDiagram = async () => {
try {
const { default: mermaid } = await import("mermaid");
if (cancelled) return;
mermaid.initialize({
startOnLoad: false,
theme: toMermaidTheme(currentTheme),
securityLevel: "strict",
fontFamily: "inherit",
suppressErrorRendering: true,
});
const id = `mermaid-${Math.random().toString(36).substring(7)}`;
const { svg: renderedSvg } = await mermaid.render(id, codeContent);
if (cancelled) return;
setSvg(renderedSvg);
setError("");
} catch (err) {
if (cancelled) return;
console.error("Failed to render mermaid diagram:", err);
setSvg("");
setError(formatErrorMessage(err));
}
};
renderDiagram();
return () => {
cancelled = true;
};
}, [codeContent, currentTheme]);
```
- [x] **Step 5: Run lint**
Run:
```bash
cd web && pnpm lint
```
Expected: PASS, or only unrelated pre-existing failures.
## Task 4.5: Remove Remaining Entry Preloads Found During Verification
**Files:**
- Modify: `web/src/router/index.tsx`
- Modify: `web/vite.config.mts`
- [x] **Step 1: Lazy-load the Home route**
`web/src/router/index.tsx` now uses `lazyWithReload(() => import("@/pages/Home"))` instead of a static Home import. This prevents Home, `PagedMemoList`, `MemoEditor`, `MemoContent`, KaTeX, and Mermaid code from entering the auth/signup app entry graph.
- [x] **Step 2: Tighten optional vendor split groups**
`web/vite.config.mts` no longer defines a manual `mermaid-vendor` group, because Rolldown emitted the preload helper from that group and forced an entry preload. The Leaflet group now matches only the `leaflet` package, not `react-leaflet`, so React does not get bundled into a Leaflet-named entry preload.
- [x] **Step 3: Re-run lint and build**
Run:
```bash
cd web && pnpm lint && pnpm build
```
Expected: PASS.
## Task 5: Build And Verify Initial Network Behavior
**Files:**
- No source edits expected unless verification finds a regression.
- [x] **Step 1: Build production frontend**
Run:
```bash
cd web && pnpm build
```
Expected: PASS. Build output should still contain separate Mermaid and Leaflet chunks, but they should not be required by the auth/signup initial route.
- [x] **Step 2: Inspect build output for heavy chunks**
Run:
```bash
cd web && find dist/assets -maxdepth 1 -type f \( -name '*mermaid*' -o -name '*leaflet*' -o -name '*katex*' \) -print | sort
```
Expected: Mermaid and Leaflet assets may exist as lazy chunks. Their existence is fine; the goal is that auth/signup does not request them initially.
- [x] **Step 3: Start production preview**
Run:
```bash
cd web && pnpm exec vite preview --host 127.0.0.1 --port 4173
```
Expected: Preview server starts on `http://127.0.0.1:4173/`. Keep this session running until browser verification is complete.
- [x] **Step 4: Verify `/auth/signup` network with browser tooling**
Open:
```text
http://127.0.0.1:4173/auth/signup
```
Expected initial document and asset requests do not include filenames containing:
```text
mermaid
leaflet
```
If KaTeX CSS still appears on `/auth/signup`, inspect the chunk initiator and remove any remaining static import path that reaches `MemoMarkdownRenderer` from auth/signup.
- [ ] **Step 5: Smoke test feature lazy loading**
Use the running app or a local backend/dev setup to verify:
```text
1. A memo containing a Mermaid code block renders the diagram and requests the Mermaid chunk only when the memo content appears.
2. A memo containing inline or block math displays KaTeX styling when memo content appears.
3. Opening the location picker loads Leaflet assets and the picker remains interactive.
4. Opening a memo location popover loads Leaflet assets and shows the pinned map.
5. Opening `/u/:username?view=map` loads the profile map and marker cluster behavior still works.
```
Expected: Features behave as before after their lazy chunks load.
Result in this session: live feature smoke was not completed because the production preview has no authenticated backend session or seeded memo data. Static verification confirmed the relevant feature chunks still exist and load paths are behind lazy imports.
- [x] **Step 6: Stop preview server**
Stop the preview command from Step 3 with `Ctrl-C`.
## Task 6: Commit Implementation
**Files:**
- All source files modified by Tasks 1-4.
- [x] **Step 1: Review final diff**
Run:
```bash
git diff -- web/src/main.tsx web/src/components/map web/src/components/MemoEditor web/src/components/MemoMetadata/Location web/src/pages/UserProfile.tsx web/src/components/MemoContent web/src/components/UserMemoMap/UserMemoMap.tsx
```
Expected: Diff is limited to lazy-loading heavy optional dependencies and plain coordinate type changes.
- [x] **Step 2: Run final verification**
Run:
```bash
cd web && pnpm lint && pnpm build
```
Expected: PASS.
- [x] **Step 3: Commit**
Run:
```bash
git add web/src/main.tsx web/src/components/map web/src/components/MemoEditor web/src/components/MemoMetadata/Location web/src/pages/UserProfile.tsx web/src/components/MemoContent web/src/components/UserMemoMap/UserMemoMap.tsx
git commit -m "perf: lazy load heavy first-screen dependencies"
```
Expected: Commit succeeds with only the intended source changes.
## Self-Review
- Spec coverage: The plan removes global Leaflet/KaTeX CSS, dynamically imports Mermaid, lazy-loads map UI, keeps Leaflet types out of parent boundaries, and verifies auth/signup network behavior.
- Placeholder scan: No placeholder markers, unresolved decisions, or vague generic handling steps remain.
- Type consistency: `MapPoint` is the shared parent-facing coordinate type; `LatLng` remains internal to `LocationPicker` and map implementation files only.
@@ -0,0 +1,776 @@
# Placeholder Component Implementation Plan
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
**Goal:** Replace `Empty.tsx` with a reusable `<Placeholder>` component covering empty / loading / noResults / notFound states, each rendering a hand-curated ASCII bird from a pool-shaped data file with subtle CSS-only animation.
**Architecture:** Single component (`Placeholder/index.tsx`) reads from a co-located `ascii-pool.ts` data file via a `pickPiece(variant)` picker. Motion is CSS keyframes in `Placeholder.css`, gated by `prefers-reduced-motion`. Default messages live in a `messages.ts` seam ready for future i18n. Integration is narrow: only `Inboxes.tsx` is rewired in this PR; `Empty.tsx` is deleted.
**Tech Stack:** React 19 · TypeScript · Tailwind v4 (via `@tailwindcss/vite`) · Vitest + `@testing-library/react` + jsdom · Biome for lint/format · `cn` helper from `@/lib/utils` for class composition.
---
## File Structure
**Create:**
- `web/src/components/Placeholder/index.tsx` — public component (default export)
- `web/src/components/Placeholder/Placeholder.css` — keyframes + motion classes
- `web/src/components/Placeholder/ascii-pool.ts` — types, `ASCII_POOL` array, `pickPiece()`
- `web/src/components/Placeholder/messages.ts``DEFAULT_MESSAGES` map
- `web/src/components/Placeholder/CREDITS.md` — Joan Stark attribution
- `web/tests/placeholder-pool.test.ts` — picker + pool integrity tests
- `web/tests/placeholder-component.test.tsx` — component render tests
**Modify:**
- `web/src/pages/Inboxes.tsx` — replace `<Empty />` with `<Placeholder variant="empty" message={…} />`
**Delete:**
- `web/src/components/Empty.tsx`
---
## Task 0: Commit this plan
**Files:**
- Add: `docs/superpowers/plans/2026-05-12-placeholder-component.md`
- [ ] **Step 1: Commit the plan document on its own**
```bash
git add docs/superpowers/plans/2026-05-12-placeholder-component.md
git commit -m "docs: add Placeholder component implementation plan"
```
This keeps the planning artifact separate from feature commits.
---
## Task 1: Scaffold pool types and picker (TDD)
**Files:**
- Create: `web/src/components/Placeholder/ascii-pool.ts`
- Test: `web/tests/placeholder-pool.test.ts`
- [ ] **Step 1: Write the failing tests for `pickPiece` and pool shape**
Create `web/tests/placeholder-pool.test.ts`:
```ts
import { describe, expect, it } from "vitest";
import { ASCII_POOL, pickPiece, type PlaceholderVariant } from "@/components/Placeholder/ascii-pool";
const VARIANTS: PlaceholderVariant[] = ["empty", "loading", "noResults", "notFound"];
describe("ASCII_POOL integrity", () => {
it("contains at least one piece per variant", () => {
for (const variant of VARIANTS) {
const matches = ASCII_POOL.filter((p) => p.variant === variant);
expect(matches.length, `variant=${variant}`).toBeGreaterThanOrEqual(1);
}
});
it("uses unique ids", () => {
const ids = ASCII_POOL.map((p) => p.id);
expect(new Set(ids).size).toBe(ids.length);
});
it("preserves the jgs credit on every piece", () => {
for (const piece of ASCII_POOL) {
expect(piece.credit, `piece=${piece.id}`).toMatch(/jgs/);
}
});
it("uses a known motion style on every piece", () => {
for (const piece of ASCII_POOL) {
expect(["bob", "flutter", "none"]).toContain(piece.motion);
}
});
});
describe("pickPiece", () => {
it("returns a piece matching the requested variant", () => {
for (const variant of VARIANTS) {
const piece = pickPiece(variant);
expect(piece.variant).toBe(variant);
}
});
it("returns a non-empty ascii string", () => {
const piece = pickPiece("empty");
expect(piece.ascii.length).toBeGreaterThan(0);
});
});
```
- [ ] **Step 2: Run the tests and verify they fail**
```bash
cd web && pnpm test placeholder-pool
```
Expected: FAIL — module `@/components/Placeholder/ascii-pool` not found.
- [ ] **Step 3: Implement `ascii-pool.ts` with types, an empty pool, and the picker**
Create `web/src/components/Placeholder/ascii-pool.ts`:
```ts
export type PlaceholderVariant = "empty" | "loading" | "noResults" | "notFound";
export type MotionStyle = "bob" | "flutter" | "none";
export interface AsciiPiece {
/** Stable identifier — used as React key and for debugging. */
id: string;
/** Which placeholder state this piece is shown for. */
variant: PlaceholderVariant;
/** ASCII art preserved verbatim — must keep every space. */
ascii: string;
/** Attribution shown beneath the bird, e.g. "jgs · 4/97". */
credit: string;
/** Motion hint applied to the <pre>. */
motion: MotionStyle;
}
export const ASCII_POOL: AsciiPiece[] = [];
export function pickPiece(variant: PlaceholderVariant): AsciiPiece {
const matches = ASCII_POOL.filter((p) => p.variant === variant);
if (matches.length === 0) {
throw new Error(`No ASCII piece registered for variant "${variant}"`);
}
return matches[Math.floor(Math.random() * matches.length)];
}
```
- [ ] **Step 4: Run the tests and verify they still fail (pool is empty)**
```bash
cd web && pnpm test placeholder-pool
```
Expected: FAIL — "contains at least one piece per variant" expectations not met (because `ASCII_POOL` is `[]`).
This is the expected red — pool gets seeded in the next task.
- [ ] **Step 5: Commit**
```bash
git add web/src/components/Placeholder/ascii-pool.ts web/tests/placeholder-pool.test.ts
git commit -m "feat(placeholder): scaffold ASCII pool types and picker"
```
---
## Task 2: Seed the four ASCII pieces
**Files:**
- Modify: `web/src/components/Placeholder/ascii-pool.ts`
- [ ] **Step 1: Replace the empty `ASCII_POOL` array with the four seed entries**
In `web/src/components/Placeholder/ascii-pool.ts`, replace `export const ASCII_POOL: AsciiPiece[] = [];` with the four entries below. Preserve every space and newline in the `ascii` strings exactly — they are template literals with escaped backslashes/backticks per JS rules.
```ts
export const ASCII_POOL: AsciiPiece[] = [
{
id: "jgs-crested-parrot",
variant: "empty",
credit: "jgs · 4/97",
motion: "bob",
ascii: ` .---.
/ 6_6
\\_ (__\\
// \\\\
(( ))
=====""===""=====
|||
|`,
},
{
id: "jgs-hummingbird-sm",
variant: "loading",
credit: "jgs · 7/98",
motion: "flutter",
ascii: ` , _
{ \\/\`o;====-
.----'-/\`-/
\`'-..-| /
/\\/\\
\`--\``,
},
{
id: "jgs-wide-eyed-owl",
variant: "noResults",
credit: "jgs · 2/01",
motion: "bob",
ascii: ` __ __
\\ \`-'"'-\` /
/ \\_ _/ \\
| d\\_/b |
.'\\ V /'.
/ '-...-' \\
| / \\ |
\\/\\ /\\/
==(||)---(||)==`,
},
{
id: "jgs-bird-flown-away",
variant: "notFound",
credit: "jgs · 7/96",
motion: "flutter",
ascii: ` ___
_,-' ______
.' .-' ____7
/ / ___7
_| / ___7
>(')\\ | ___7
\\\\/ \\_______
' _======>
\`'----\\\\\``,
},
];
```
- [ ] **Step 2: Run the pool tests and verify they pass**
```bash
cd web && pnpm test placeholder-pool
```
Expected: PASS for all six assertions.
- [ ] **Step 3: Commit**
```bash
git add web/src/components/Placeholder/ascii-pool.ts
git commit -m "feat(placeholder): seed pool with four jgs ASCII bird pieces"
```
---
## Task 3: Default messages
**Files:**
- Create: `web/src/components/Placeholder/messages.ts`
- [ ] **Step 1: Add a tiny test for the messages map**
Append to `web/tests/placeholder-pool.test.ts`:
```ts
import { DEFAULT_MESSAGES } from "@/components/Placeholder/messages";
describe("DEFAULT_MESSAGES", () => {
it("provides a non-empty message for every variant", () => {
for (const variant of VARIANTS) {
expect(DEFAULT_MESSAGES[variant], `variant=${variant}`).toBeTruthy();
expect(DEFAULT_MESSAGES[variant].trim().length).toBeGreaterThan(0);
}
});
});
```
- [ ] **Step 2: Run the test and verify it fails**
```bash
cd web && pnpm test placeholder-pool
```
Expected: FAIL — module `@/components/Placeholder/messages` not found.
- [ ] **Step 3: Create `messages.ts`**
```ts
import type { PlaceholderVariant } from "./ascii-pool";
/**
* Default copy shown beneath the ASCII art when no `message` prop is supplied.
*
* Future i18n: swap these strings for `t("placeholder.<variant>")` lookups via
* `react-i18next` without touching the component.
*/
export const DEFAULT_MESSAGES: Record<PlaceholderVariant, string> = {
empty: "No memos yet",
loading: "Loading…",
noResults: "Nothing matches that search",
notFound: "This page flew the coop",
};
```
- [ ] **Step 4: Run the test and verify it passes**
```bash
cd web && pnpm test placeholder-pool
```
Expected: PASS — all variants have a non-empty default message.
- [ ] **Step 5: Commit**
```bash
git add web/src/components/Placeholder/messages.ts web/tests/placeholder-pool.test.ts
git commit -m "feat(placeholder): add DEFAULT_MESSAGES map"
```
---
## Task 4: Animation keyframes
**Files:**
- Create: `web/src/components/Placeholder/Placeholder.css`
- [ ] **Step 1: Create the stylesheet**
```css
/*
* Animations for <Placeholder>.
*
* All keyframes are wrapped in a prefers-reduced-motion guard so users who
* opt out of motion see a static bird and an instantly-visible message.
*/
@media (prefers-reduced-motion: no-preference) {
.placeholder-motion-bob {
animation: placeholder-bob 3.4s ease-in-out infinite;
}
.placeholder-motion-flutter {
animation: placeholder-flutter 0.7s ease-in-out infinite;
}
.placeholder-fade-in {
animation: placeholder-fade 1s ease-out 0.3s both;
opacity: 0;
}
}
@keyframes placeholder-bob {
0%, 100% { transform: translateY(0); }
50% { transform: translateY(-4px); }
}
@keyframes placeholder-flutter {
0%, 100% { transform: translate(0, 0); }
50% { transform: translate(2px, -1px); }
}
@keyframes placeholder-fade {
to { opacity: 1; }
}
```
- [ ] **Step 2: Commit**
```bash
git add web/src/components/Placeholder/Placeholder.css
git commit -m "feat(placeholder): add motion keyframes with reduced-motion guard"
```
---
## Task 5: Placeholder component (TDD)
**Files:**
- Create: `web/src/components/Placeholder/index.tsx`
- Test: `web/tests/placeholder-component.test.tsx`
- [ ] **Step 1: Write the failing component tests**
Create `web/tests/placeholder-component.test.tsx`:
```tsx
import { render, screen } from "@testing-library/react";
import { describe, expect, it } from "vitest";
import Placeholder from "@/components/Placeholder";
import { DEFAULT_MESSAGES } from "@/components/Placeholder/messages";
describe("<Placeholder>", () => {
it("renders the default message for variant=empty", () => {
render(<Placeholder variant="empty" />);
expect(screen.getByText(DEFAULT_MESSAGES.empty)).toBeInTheDocument();
});
it("renders the default message for variant=loading", () => {
render(<Placeholder variant="loading" />);
expect(screen.getByText(DEFAULT_MESSAGES.loading)).toBeInTheDocument();
});
it("renders the default message for variant=noResults", () => {
render(<Placeholder variant="noResults" />);
expect(screen.getByText(DEFAULT_MESSAGES.noResults)).toBeInTheDocument();
});
it("renders the default message for variant=notFound", () => {
render(<Placeholder variant="notFound" />);
expect(screen.getByText(DEFAULT_MESSAGES.notFound)).toBeInTheDocument();
});
it("overrides the default message when `message` prop is passed", () => {
render(<Placeholder variant="empty" message="Custom copy goes here" />);
expect(screen.getByText("Custom copy goes here")).toBeInTheDocument();
expect(screen.queryByText(DEFAULT_MESSAGES.empty)).not.toBeInTheDocument();
});
it("renders the ASCII art inside a <pre> with aria-hidden", () => {
const { container } = render(<Placeholder variant="empty" />);
const pre = container.querySelector("pre");
expect(pre).not.toBeNull();
expect(pre).toHaveAttribute("aria-hidden", "true");
expect(pre!.textContent!.length).toBeGreaterThan(0);
});
it("renders a jgs credit string", () => {
render(<Placeholder variant="empty" />);
expect(screen.getByText(/jgs/)).toBeInTheDocument();
});
it('applies role="status" and aria-live="polite" ONLY when variant=loading', () => {
const { rerender, container } = render(<Placeholder variant="empty" />);
expect(container.querySelector('[role="status"]')).toBeNull();
rerender(<Placeholder variant="loading" />);
const live = container.querySelector('[role="status"]');
expect(live).not.toBeNull();
expect(live).toHaveAttribute("aria-live", "polite");
});
it("renders children below the message when provided", () => {
render(
<Placeholder variant="notFound">
<button type="button">Go home</button>
</Placeholder>,
);
expect(screen.getByRole("button", { name: "Go home" })).toBeInTheDocument();
});
it("merges a custom className onto the outer wrapper", () => {
const { container } = render(<Placeholder variant="empty" className="custom-test-class" />);
expect(container.firstChild).toHaveClass("custom-test-class");
});
});
```
- [ ] **Step 2: Run the tests and verify they fail**
```bash
cd web && pnpm test placeholder-component
```
Expected: FAIL — module `@/components/Placeholder` not found.
- [ ] **Step 3: Implement `index.tsx`**
Create `web/src/components/Placeholder/index.tsx`:
```tsx
import { useMemo, type ReactNode } from "react";
import { cn } from "@/lib/utils";
import { pickPiece, type MotionStyle, type PlaceholderVariant } from "./ascii-pool";
import { DEFAULT_MESSAGES } from "./messages";
import "./Placeholder.css";
interface PlaceholderProps {
variant: PlaceholderVariant;
message?: string;
children?: ReactNode;
className?: string;
}
const MOTION_CLASS: Record<MotionStyle, string> = {
bob: "placeholder-motion-bob",
flutter: "placeholder-motion-flutter",
none: "",
};
const Placeholder = ({ variant, message, children, className }: PlaceholderProps) => {
// Stable for the lifetime of this mount; re-rolls only if `variant` changes
// (which is rare in practice — most callers pass a constant).
const piece = useMemo(() => pickPiece(variant), [variant]);
const resolvedMessage = message ?? DEFAULT_MESSAGES[variant];
const isLoading = variant === "loading";
return (
<div
role={isLoading ? "status" : undefined}
aria-live={isLoading ? "polite" : undefined}
className={cn("flex flex-col items-center justify-center max-w-md mx-auto px-4 py-8", className)}
>
<pre
aria-hidden="true"
className={cn(
"font-mono text-xs sm:text-sm leading-tight text-muted-foreground whitespace-pre m-0",
MOTION_CLASS[piece.motion],
)}
>
{piece.ascii}
</pre>
<p className="mt-3 font-mono text-sm text-muted-foreground placeholder-fade-in">
{resolvedMessage}
</p>
<p className="mt-1 font-mono text-[10px] text-muted-foreground/60 placeholder-fade-in">
{piece.credit}
</p>
{children && <div className="mt-4">{children}</div>}
</div>
);
};
export default Placeholder;
```
- [ ] **Step 4: Run the tests and verify they pass**
```bash
cd web && pnpm test placeholder-component
```
Expected: PASS — all ten assertions green.
- [ ] **Step 5: Commit**
```bash
git add web/src/components/Placeholder/index.tsx web/tests/placeholder-component.test.tsx
git commit -m "feat(placeholder): implement <Placeholder> with variant-driven ASCII pool"
```
---
## Task 6: Attribution credits
**Files:**
- Create: `web/src/components/Placeholder/CREDITS.md`
- [ ] **Step 1: Create `CREDITS.md`**
```markdown
# ASCII Art Credits
The ASCII bird illustrations rendered by `<Placeholder>` are from **Joan Stark's**
classic ASCII art collection. Each piece is signed with her `jgs` tag and the
month/year it was published.
- Source archive: https://github.com/oldcompcz/jgs (Joan Stark's ASCII Art Gallery)
- Original site (preserved via WebArchive): https://web.archive.org/web/20091028013825/http://www.geocities.com/SoHo/7373/
- Wikipedia: https://en.wikipedia.org/wiki/Joan_Stark
Joan Stark distributed her art freely on Usenet and the early web. We retain
the `jgs` signature visible beneath each piece in the UI so attribution travels
with the art wherever it is shown.
If you add new ASCII pieces to `ascii-pool.ts`:
- Prefer well-attributed art from established collections.
- Keep the original artist signature in the `credit` field (e.g. `"jgs · 4/97"`).
- If using a different artist, link the source in this file.
```
- [ ] **Step 2: Commit**
```bash
git add web/src/components/Placeholder/CREDITS.md
git commit -m "docs(placeholder): credit Joan Stark for ASCII bird art"
```
---
## Task 7: Wire into `Inboxes.tsx` and delete `Empty.tsx`
**Files:**
- Modify: `web/src/pages/Inboxes.tsx`
- Delete: `web/src/components/Empty.tsx`
- [ ] **Step 1: Read the current empty-state block in `Inboxes.tsx`**
Open `web/src/pages/Inboxes.tsx`. The relevant block is at lines 99105:
```tsx
{notifications.length === 0 ? (
<div className="w-full py-16 flex flex-col justify-center items-center">
<Empty />
<p className="mt-4 text-sm text-muted-foreground">
{filter === "unread" ? t("inbox.no-unread") : filter === "archived" ? t("inbox.no-archived") : t("message.no-data")}
</p>
</div>
) : (
```
The outer `<div className="w-full py-16 flex flex-col justify-center items-center">` and the inner `<p>` both become redundant — `<Placeholder>` handles its own centering and message.
- [ ] **Step 2: Swap the import**
In `web/src/pages/Inboxes.tsx`, replace the import line:
```tsx
import Empty from "@/components/Empty";
```
with:
```tsx
import Placeholder from "@/components/Placeholder";
```
- [ ] **Step 3: Replace the empty-state JSX**
Replace lines 99105 (the existing empty-state block above) with:
```tsx
{notifications.length === 0 ? (
<Placeholder
variant="empty"
message={
filter === "unread"
? t("inbox.no-unread")
: filter === "archived"
? t("inbox.no-archived")
: t("message.no-data")
}
/>
) : (
```
(Only the truthy branch of the ternary changes; leave the `: (` start of the falsy branch and everything below it untouched.)
- [ ] **Step 4: Delete `Empty.tsx`**
```bash
git rm web/src/components/Empty.tsx
```
- [ ] **Step 5: Verify nothing else imports `Empty`**
```bash
cd /Users/steven/Projects/usememos/memos && grep -rn 'from "@/components/Empty"\|from "./Empty"\|from "../Empty"' web/src 2>/dev/null
```
Expected: no output. If anything matches, update that file to use `<Placeholder variant="empty" />` before continuing.
- [ ] **Step 6: Commit**
```bash
git add web/src/pages/Inboxes.tsx web/src/components/Empty.tsx
git commit -m "feat(inboxes): use <Placeholder variant=empty> in place of <Empty>"
```
---
## Task 8: Verification — lint, types, tests, build, dev preview
**Files:** none modified (verification only)
- [ ] **Step 1: Run the lint + typecheck**
```bash
cd web && pnpm lint
```
Expected: exits 0. If Biome flags anything (formatting, sort-order, unused imports), run `pnpm lint:fix` and re-run `pnpm lint`. Commit the fix separately if any changes are required:
```bash
git add -p web/src web/tests
git commit -m "chore(placeholder): biome auto-fixes"
```
- [ ] **Step 2: Run the full test suite**
```bash
cd web && pnpm test
```
Expected: all suites green, including `placeholder-pool` (8 assertions) and `placeholder-component` (10 assertions). Existing tests must still pass.
- [ ] **Step 3: Run the production build**
```bash
cd web && pnpm build
```
Expected: exits 0. Confirms TypeScript still compiles end-to-end and the new CSS import is picked up by Vite.
- [ ] **Step 4: Manual visual check — start the dev server**
```bash
cd web && pnpm dev
```
In a browser, navigate to the running URL (Vite prints it). Sign in with any test account, open the **Inbox** page, and confirm:
1. When inbox is empty, the crested-parrot ASCII bird is visible.
2. It bobs gently every ~3.4 seconds.
3. The message text (one of "no unread", "no archived", or "no data") appears below with a soft fade-in.
4. The small `jgs · 4/97` credit is visible below the message.
5. No console errors or warnings.
Stop the dev server with `Ctrl+C`.
- [ ] **Step 5: (Optional) Reduced-motion check**
Open browser DevTools → command menu → "Emulate CSS prefers-reduced-motion: reduce". Reload the inbox empty state. The bird should be **static** and the message should appear instantly (no fade).
- [ ] **Step 6: No-op commit point**
If steps 14 all passed and no further changes were needed, there is nothing to commit. Proceed to the next task.
---
## Task 9: Document the work in the PR body
**Files:** none modified
- [ ] **Step 1: Draft a PR description**
When opening the PR, use a body like:
```markdown
## Summary
- Adds a new `<Placeholder variant="empty | loading | noResults | notFound">` component that renders a hand-curated ASCII bird from a pool-shaped data file, with subtle CSS-only motion that respects `prefers-reduced-motion`.
- Replaces the single-purpose `Empty.tsx` (used in `Inboxes.tsx`) with `<Placeholder variant="empty">`.
- ASCII art is from Joan Stark's (jgs) classic collection — attribution is preserved on every piece and in a co-located `CREDITS.md`.
## Out of scope (follow-up opportunities)
- Wire `<Placeholder variant="noResults">` into the memo search results page.
- Wire `<Placeholder variant="notFound">` into the router 404 catch-all.
- Wire `<Placeholder variant="loading">` into Suspense fallbacks.
- Seed additional ASCII pieces per variant — the pool architecture supports it; just append entries to `ASCII_POOL`.
## Test plan
- [ ] `pnpm lint` clean
- [ ] `pnpm test` green (incl. new `placeholder-pool` and `placeholder-component` suites)
- [ ] `pnpm build` succeeds
- [ ] Inbox empty state shows the ASCII parrot, bobs, and renders the filter-specific message
- [ ] `prefers-reduced-motion: reduce` produces a static bird and an instantly-visible message
```
- [ ] **Step 2: Open the PR** (skip if user prefers to do this themselves)
This step is left as a manual handoff — do not push or open the PR unless the user has explicitly authorized it.
---
## Self-Review Notes
This plan covers the spec's nine sections as follows:
| Spec section | Implemented by |
|---|---|
| Public Component | Task 5 |
| ASCII Pool | Tasks 1, 2 |
| Default Messages | Task 3 |
| Animation | Task 4 (CSS) + Task 5 (component wires classes) |
| Accessibility | Task 5 (test assertions + impl) |
| File Layout | Tasks 16 (all five files created) |
| Integration | Task 7 (Inboxes rewire + Empty delete) |
| Credits | Task 6 |
| Testing | Tasks 1, 3, 5 (pool tests + component tests) |
No spec section is unimplemented. No "TBD" / "TODO" / vague-handwave language is used in any step. Types, method signatures, and class names referenced across tasks match:
- `PlaceholderVariant`, `MotionStyle`, `AsciiPiece`, `ASCII_POOL`, `pickPiece` — consistent across Tasks 1, 2, 3, 5
- `DEFAULT_MESSAGES` — defined in Task 3, consumed in Task 5
- `placeholder-motion-bob`, `placeholder-motion-flutter`, `placeholder-fade-in` — defined in Task 4 CSS, consumed in Task 5 component
- `cn` from `@/lib/utils` — matches existing codebase convention (verified in pre-plan exploration)
- `Placeholder` is a default export — matches the convention used by `Empty.tsx` and most other `web/src/components/*.tsx` files
@@ -0,0 +1,673 @@
# Remove react-use Implementation Plan
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
**Goal:** Remove `react-use` from the frontend while preserving current hook behavior and eliminating the transitive `js-cookie@2.2.1` dependency.
**Architecture:** Replace simple one-off `react-use` helpers with native React hooks inside the consuming components. Add two focused local hooks in `web/src/hooks/` for reused debounce behavior and typed localStorage state. Regenerate the pnpm lockfile from `web/package.json` instead of manually editing lockfile entries.
**Tech Stack:** React 19, TypeScript 6, Vite 8, Vitest 4, Testing Library, pnpm 11.
---
## File Structure
- Create `web/src/hooks/useDebouncedEffect.ts`
- Shared debounce hook for effect-style callbacks.
- Create `web/src/hooks/useLocalStorage.ts`
- Typed localStorage state hook for persisted UI preferences.
- Modify `web/src/hooks/index.ts`
- Export the two new hooks for existing `@/hooks` barrel import style.
- Create `web/tests/hooks.test.tsx`
- Unit tests for the two new local hooks.
- Modify `web/src/components/MemoEditor/Toolbar/InsertMenu.tsx`
- Replace `react-use` debounce import with local `useDebouncedEffect`.
- Modify `web/src/components/MemoEditor/hooks/useLinkMemo.ts`
- Replace deep `react-use` debounce import with local `useDebouncedEffect`.
- Modify `web/src/components/MemoExplorer/TagsSection.tsx`
- Replace deep `react-use` localStorage import with local `useLocalStorage`.
- Modify `web/src/components/TagTree.tsx`
- Replace `useToggle` with native `useState`.
- Modify `web/src/components/MobileHeader.tsx`
- Replace `useWindowScroll` with native `useState` and `useEffect`.
- Modify `web/src/layouts/RootLayout.tsx`
- Replace `usePrevious` with native `useRef` and `useEffect`.
- Modify `web/package.json`
- Remove the direct `react-use` dependency.
- Modify `web/pnpm-lock.yaml`
- Regenerate through pnpm after `react-use` is removed.
---
### Task 1: Add Failing Hook Tests
**Files:**
- Create: `web/tests/hooks.test.tsx`
- [ ] **Step 1: Write tests for local hook behavior**
Create `web/tests/hooks.test.tsx`:
```tsx
import { act, renderHook } from "@testing-library/react";
import { afterEach, beforeEach, describe, expect, it, vi } from "vitest";
import { useDebouncedEffect, useLocalStorage } from "@/hooks";
describe("useLocalStorage", () => {
beforeEach(() => {
window.localStorage.clear();
});
afterEach(() => {
window.localStorage.clear();
});
it("uses the default value when storage is empty", () => {
const { result } = renderHook(() => useLocalStorage("hook-test-empty", false));
expect(result.current[0]).toBe(false);
});
it("reads and writes JSON values", () => {
window.localStorage.setItem("hook-test-existing", JSON.stringify(true));
const { result } = renderHook(() => useLocalStorage("hook-test-existing", false));
expect(result.current[0]).toBe(true);
act(() => {
result.current[1](false);
});
expect(result.current[0]).toBe(false);
expect(window.localStorage.getItem("hook-test-existing")).toBe("false");
});
it("supports updater functions", () => {
const { result } = renderHook(() => useLocalStorage("hook-test-updater", false));
act(() => {
result.current[1]((current) => !current);
});
expect(result.current[0]).toBe(true);
expect(window.localStorage.getItem("hook-test-updater")).toBe("true");
});
it("falls back to the default value for malformed storage", () => {
window.localStorage.setItem("hook-test-malformed", "{bad json");
const { result } = renderHook(() => useLocalStorage("hook-test-malformed", true));
expect(result.current[0]).toBe(true);
});
});
describe("useDebouncedEffect", () => {
beforeEach(() => {
vi.useFakeTimers();
});
afterEach(() => {
vi.runOnlyPendingTimers();
vi.useRealTimers();
});
it("runs the latest callback after the delay", () => {
const calls: string[] = [];
const { rerender } = renderHook(
({ value }) => {
useDebouncedEffect(
() => {
calls.push(value);
},
100,
[value],
);
},
{ initialProps: { value: "first" } },
);
act(() => {
vi.advanceTimersByTime(99);
});
expect(calls).toEqual([]);
rerender({ value: "second" });
act(() => {
vi.advanceTimersByTime(100);
});
expect(calls).toEqual(["second"]);
});
it("clears the pending timeout on unmount", () => {
const calls: string[] = [];
const { unmount } = renderHook(() => {
useDebouncedEffect(
() => {
calls.push("called");
},
100,
[],
);
});
unmount();
act(() => {
vi.advanceTimersByTime(100);
});
expect(calls).toEqual([]);
});
});
```
- [ ] **Step 2: Run tests and verify they fail because hooks do not exist**
Run:
```bash
cd web
pnpm test tests/hooks.test.tsx
```
Expected: FAIL with missing exports for `useDebouncedEffect` and `useLocalStorage` from `@/hooks`.
- [ ] **Step 3: Commit failing tests**
```bash
git add web/tests/hooks.test.tsx
git commit -m "test: cover local frontend hooks"
```
---
### Task 2: Implement Local Shared Hooks
**Files:**
- Create: `web/src/hooks/useDebouncedEffect.ts`
- Create: `web/src/hooks/useLocalStorage.ts`
- Modify: `web/src/hooks/index.ts`
- Test: `web/tests/hooks.test.tsx`
- [ ] **Step 1: Add `useDebouncedEffect`**
Create `web/src/hooks/useDebouncedEffect.ts`:
```ts
import { type DependencyList, useEffect } from "react";
export const useDebouncedEffect = (effect: () => void | Promise<void>, delay: number, deps: DependencyList): void => {
useEffect(() => {
const timeout = window.setTimeout(() => {
void effect();
}, delay);
return () => {
window.clearTimeout(timeout);
};
}, [delay, ...deps]);
};
```
- [ ] **Step 2: Add `useLocalStorage`**
Create `web/src/hooks/useLocalStorage.ts`:
```ts
import { useCallback, useEffect, useState } from "react";
type SetLocalStorageValue<T> = T | ((currentValue: T) => T);
const readLocalStorageValue = <T>(key: string, defaultValue: T): T => {
if (typeof window === "undefined") {
return defaultValue;
}
try {
const storedValue = window.localStorage.getItem(key);
return storedValue === null ? defaultValue : (JSON.parse(storedValue) as T);
} catch {
return defaultValue;
}
};
export const useLocalStorage = <T>(key: string, defaultValue: T): [T, (value: SetLocalStorageValue<T>) => void] => {
const [storedValue, setStoredValue] = useState<T>(() => readLocalStorageValue(key, defaultValue));
useEffect(() => {
setStoredValue(readLocalStorageValue(key, defaultValue));
}, [key, defaultValue]);
const setValue = useCallback(
(value: SetLocalStorageValue<T>) => {
setStoredValue((currentValue) => {
const nextValue = typeof value === "function" ? (value as (currentValue: T) => T)(currentValue) : value;
if (typeof window !== "undefined") {
try {
window.localStorage.setItem(key, JSON.stringify(nextValue));
} catch {
// Keep React state updated even if persistence is unavailable.
}
}
return nextValue;
});
},
[key],
);
return [storedValue, setValue];
};
```
- [ ] **Step 3: Export hooks from the barrel**
Modify `web/src/hooks/index.ts` to include:
```ts
export * from "./useAsyncEffect";
export * from "./useCurrentUser";
export * from "./useDateFilterNavigation";
export * from "./useDebouncedEffect";
export * from "./useFilteredMemoStats";
export * from "./useLoading";
export * from "./useLocalStorage";
export * from "./useMediaQuery";
export * from "./useMemoFilters";
export * from "./useMemoSorting";
export * from "./useNavigateTo";
export * from "./useUserLocale";
export * from "./useUserTheme";
```
- [ ] **Step 4: Run hook tests and verify they pass**
Run:
```bash
cd web
pnpm test tests/hooks.test.tsx
```
Expected: PASS for all `useLocalStorage` and `useDebouncedEffect` tests.
- [ ] **Step 5: Commit shared hooks**
```bash
git add web/src/hooks/useDebouncedEffect.ts web/src/hooks/useLocalStorage.ts web/src/hooks/index.ts web/tests/hooks.test.tsx
git commit -m "feat: add local frontend hooks"
```
---
### Task 3: Replace Simple react-use Helpers With React Hooks
**Files:**
- Modify: `web/src/components/TagTree.tsx`
- Modify: `web/src/components/MobileHeader.tsx`
- Modify: `web/src/layouts/RootLayout.tsx`
- [ ] **Step 1: Replace `useToggle` in `TagTree`**
In `web/src/components/TagTree.tsx`, change the imports:
```ts
import { ChevronRightIcon, HashIcon } from "lucide-react";
import { useCallback, useEffect, useState } from "react";
import { type MemoFilter, useMemoFilterContext } from "@/contexts/MemoFilterContext";
```
Replace the toggle state in `TagItemContainer` with:
```ts
const [showSubTags, setShowSubTags] = useState(false);
useEffect(() => {
setShowSubTags(expandSubTags);
}, [expandSubTags]);
```
Replace `handleToggleBtnClick` with:
```ts
const handleToggleBtnClick = useCallback((event: React.MouseEvent) => {
event.stopPropagation();
setShowSubTags((current) => !current);
}, []);
```
- [ ] **Step 2: Replace `useWindowScroll` in `MobileHeader`**
In `web/src/components/MobileHeader.tsx`, replace the first import with React hooks:
```ts
import { useEffect, useState } from "react";
import useMediaQuery from "@/hooks/useMediaQuery";
import { cn } from "@/lib/utils";
import NavigationDrawer from "./NavigationDrawer";
```
Inside `MobileHeader`, replace `const { y: offsetTop } = useWindowScroll();` with:
```ts
const [offsetTop, setOffsetTop] = useState(() => {
if (typeof window === "undefined") return 0;
return window.scrollY;
});
useEffect(() => {
const handleScroll = () => {
setOffsetTop(window.scrollY);
};
window.addEventListener("scroll", handleScroll, { passive: true });
handleScroll();
return () => {
window.removeEventListener("scroll", handleScroll);
};
}, []);
```
- [ ] **Step 3: Replace `usePrevious` in `RootLayout`**
In `web/src/layouts/RootLayout.tsx`, change the React import and remove the `react-use` import:
```ts
import { useEffect, useRef } from "react";
import { Outlet, useLocation, useSearchParams } from "react-router-dom";
```
Replace:
```ts
const prevPathname = usePrevious(pathname);
```
with:
```ts
const prevPathnameRef = useRef<string | undefined>(undefined);
```
Replace the route filter clearing effect with:
```ts
useEffect(() => {
const prevPathname = prevPathnameRef.current;
// When the route changes and there is no filter in the search params, remove all filters.
if (prevPathname !== undefined && prevPathname !== pathname && !searchParams.has("filter")) {
removeFilter(() => true);
}
prevPathnameRef.current = pathname;
}, [pathname, searchParams, removeFilter]);
```
- [ ] **Step 4: Run TypeScript/lint check**
Run:
```bash
cd web
pnpm lint
```
Expected: PASS.
- [ ] **Step 5: Commit simple helper replacements**
```bash
git add web/src/components/TagTree.tsx web/src/components/MobileHeader.tsx web/src/layouts/RootLayout.tsx
git commit -m "refactor: replace simple react-use helpers"
```
---
### Task 4: Replace Debounce And LocalStorage Call Sites
**Files:**
- Modify: `web/src/components/MemoEditor/Toolbar/InsertMenu.tsx`
- Modify: `web/src/components/MemoEditor/hooks/useLinkMemo.ts`
- Modify: `web/src/components/MemoExplorer/TagsSection.tsx`
- Test: `web/tests/hooks.test.tsx`
- [ ] **Step 1: Update `InsertMenu` debounce import and call**
In `web/src/components/MemoEditor/Toolbar/InsertMenu.tsx`, remove:
```ts
import { useDebounce } from "react-use";
```
Add:
```ts
import { useDebouncedEffect } from "@/hooks";
```
Replace:
```ts
useDebounce(
() => {
setDebouncedPosition(locationState.position);
},
1000,
[locationState.position],
);
```
with:
```ts
useDebouncedEffect(
() => {
setDebouncedPosition(locationState.position);
},
1000,
[locationState.position],
);
```
- [ ] **Step 2: Update `useLinkMemo` debounce import and call**
In `web/src/components/MemoEditor/hooks/useLinkMemo.ts`, remove:
```ts
import useDebounce from "react-use/lib/useDebounce";
```
Add:
```ts
import { useDebouncedEffect } from "@/hooks";
```
Replace:
```ts
useDebounce(
```
with:
```ts
useDebouncedEffect(
```
Keep the existing async callback body, `300` delay, and `[isOpen, searchText]` dependency list unchanged.
- [ ] **Step 3: Update `TagsSection` localStorage import**
In `web/src/components/MemoExplorer/TagsSection.tsx`, replace:
```ts
import useLocalStorage from "react-use/lib/useLocalStorage";
```
with:
```ts
import { useLocalStorage } from "@/hooks";
```
Keep both call sites unchanged:
```ts
const [treeMode, setTreeMode] = useLocalStorage<boolean>("tag-view-as-tree", false);
const [treeAutoExpand, setTreeAutoExpand] = useLocalStorage<boolean>("tag-tree-auto-expand", false);
```
- [ ] **Step 4: Verify no source imports from `react-use` remain**
Run:
```bash
rg -n 'react-use' web/src web/package.json
```
Expected: only `web/package.json` still reports `react-use` before the dependency removal task.
- [ ] **Step 5: Run targeted hook tests and lint**
Run:
```bash
cd web
pnpm test tests/hooks.test.tsx
pnpm lint
```
Expected: both commands PASS.
- [ ] **Step 6: Commit call-site replacements**
```bash
git add web/src/components/MemoEditor/Toolbar/InsertMenu.tsx web/src/components/MemoEditor/hooks/useLinkMemo.ts web/src/components/MemoExplorer/TagsSection.tsx web/tests/hooks.test.tsx
git commit -m "refactor: use local debounce and storage hooks"
```
---
### Task 5: Remove react-use Dependency And Regenerate Lockfile
**Files:**
- Modify: `web/package.json`
- Modify: `web/pnpm-lock.yaml`
- [ ] **Step 1: Remove `react-use` from `web/package.json`**
In `web/package.json`, remove this dependency line:
```json
"react-use": "^17.6.0",
```
Do not add `js-cookie` as a direct dependency.
- [ ] **Step 2: Regenerate the lockfile**
Run:
```bash
cd web
pnpm install --lockfile-only
```
Expected: command succeeds and updates `web/pnpm-lock.yaml`.
- [ ] **Step 3: Verify dependency graph removal**
Run:
```bash
cd web
pnpm why react-use js-cookie
```
Expected: output does not list installed versions for `react-use` or `js-cookie`.
- [ ] **Step 4: Verify repository text references**
Run:
```bash
rg -n '"react-use"|react-use/lib|react-use@17\.6\.0|^\s+react-use:|js-cookie' web/package.json web/pnpm-lock.yaml web/src
```
Expected: no matches.
- [ ] **Step 5: Commit dependency removal**
```bash
git add web/package.json web/pnpm-lock.yaml
git commit -m "chore: remove react-use dependency"
```
---
### Task 6: Final Verification
**Files:**
- Verify: all files changed by Tasks 1-5
- [ ] **Step 1: Run frontend tests for the new hook coverage**
Run:
```bash
cd web
pnpm test tests/hooks.test.tsx
```
Expected: PASS.
- [ ] **Step 2: Run frontend lint**
Run:
```bash
cd web
pnpm lint
```
Expected: PASS.
- [ ] **Step 3: Run frontend build**
Run:
```bash
cd web
pnpm build
```
Expected: PASS.
- [ ] **Step 4: Confirm no removed dependency remains**
Run:
```bash
cd web
pnpm why react-use js-cookie
rg -n '"react-use"|react-use/lib|react-use@17\.6\.0|^\s+react-use:|js-cookie' src package.json pnpm-lock.yaml
```
Expected: no `react-use` or `js-cookie` dependency remains. The `rg` command returns no matches.
- [ ] **Step 5: Inspect final diff**
Run:
```bash
git diff --stat HEAD~5..HEAD
git status --short
```
Expected: diff contains only hook tests, local hooks, six call-site rewrites, and dependency files. Working tree is clean after all task commits.
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
@@ -0,0 +1,887 @@
# CEL Filter Surface Expansion — Implementation Plan
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
**Goal:** Let users write three more CEL constructs in the `filter` field — scalar `startsWith()`/`endsWith()` (case-insensitive), `matches(regex)`, and `all()` over tags (non-empty) — each compiled to SQL across SQLite/MySQL/Postgres.
**Architecture:** The `internal/filter` engine parses CEL with `cel-go`, walks the AST into a dialect-agnostic IR (`ir.go`), and renders dialect SQL (`render.go`). cel-go never evaluates — every feature must become a SQL `WHERE` fragment. We add IR nodes + parser recognition + per-dialect rendering, and register a Go-backed `REGEXP` function for SQLite (which has no built-in one).
**Tech Stack:** Go, `github.com/google/cel-go v0.28.0`, `modernc.org/sqlite` (pure-Go), `github.com/stretchr/testify/require`.
**Spec:** `docs/superpowers/specs/2026-06-15-cel-filter-surface-expansion-design.md`
---
## File structure
| File | Change | Responsibility |
|------|--------|----------------|
| `internal/filter/ir.go` | Modify | Replace `ContainsCondition` with `TextMatchCondition`; add `RegexCondition`; add `ComprehensionAll` kind |
| `internal/filter/parser.go` | Modify | Recognize top-level `contains`/`startsWith`/`endsWith`/`matches`; accept `all()` comprehension |
| `internal/filter/render.go` | Modify | Render text-match (LIKE), regex, and `all()` per-element subqueries; shared `foldedLike`/`likePattern`/`escapeLikeLiteral` helpers |
| `internal/filter/schema.go` | Modify | Add `cel.ValidateRegexLiterals()` validator; enable text matching on attachment `mime_type` |
| `internal/filter/engine_test.go` | Modify | Compile-level accept/reject unit tests |
| `store/db/sqlite/functions.go` | Modify | Register a Go-backed `regexp(pattern, value)` scalar function with a compiled-pattern cache |
| `store/db/sqlite/sqlite.go` | Modify | Call `ensureRegexpRegistered()` in `NewDB` |
| `store/test/memo_filter_test.go` | Modify | Behavioral tests for the new memo filters |
| `store/test/attachment_filter_test.go` | Modify | Behavioral tests for `filename`/`mime_type` |
| `internal/filter/README.md` | Modify | Document new syntax + regex cross-dialect caveat |
**Key design choices locked in:**
- The existing `Field.SupportsContains` flag is **reused** as the gate for *all* text-matching ops (`contains`/`startsWith`/`endsWith`/`matches`) — no rename, lower risk. We just enable it on `mime_type`.
- New scalar `startsWith`/`endsWith`/`contains` are **case-insensitive** (reuse the existing `memos_unicode_lower` / `ILIKE` machinery). `matches()` and `==` are case-sensitive.
- `all()` over a memo with zero tags does **not** match (non-empty guard).
- LIKE patterns escape `%` `_` `\`; only SQLite needs an explicit `ESCAPE '\'` clause (Postgres/MySQL default the escape char to backslash, and patterns are passed as bound parameters so no SQL-literal backslash hazard).
**Suggested task order** lands the two cheap features (text-match refactor, scalar prefix/suffix, regex) before the heavy `all()` work, giving a natural stop point. `all()` (Task 5) is the largest piece and could be deferred to a follow-up if needed.
---
### Task 1: Refactor `ContainsCondition` → `TextMatchCondition` (+ LIKE escaping)
Foundation refactor. No new user-facing behavior except that LIKE metacharacters in `contains()` values are now treated literally. Existing tests must stay green.
**Files:**
- Modify: `internal/filter/ir.go` (replace `ContainsCondition`)
- Modify: `internal/filter/parser.go` (`buildContainsCondition` → shared builder)
- Modify: `internal/filter/render.go` (`renderContainsCondition``renderTextMatch` + helpers)
- [ ] **Step 1: Add a failing escaping test** in `internal/filter/engine_test.go`
```go
func TestCompileContainsEscapesLikeWildcards(t *testing.T) {
t.Parallel()
engine, err := NewEngine(NewSchema())
require.NoError(t, err)
stmt, err := engine.CompileToStatement(context.Background(), `content.contains("50%_off")`, RenderOptions{Dialect: DialectSQLite})
require.NoError(t, err)
// The % and _ in the value must be escaped so they are matched literally,
// and SQLite needs an explicit ESCAPE clause.
require.Contains(t, stmt.SQL, `ESCAPE '\'`)
require.Equal(t, []any{`%50\%\_off%`}, stmt.Args)
}
```
- [ ] **Step 2: Run it to verify it fails**
Run: `go test ./internal/filter/ -run TestCompileContainsEscapesLikeWildcards -v`
Expected: FAIL (current renderer emits `%50%_off%` with no `ESCAPE`).
- [ ] **Step 3: Replace `ContainsCondition` in `internal/filter/ir.go`**
Delete the `ContainsCondition` struct + its `isCondition()` (lines ~76-82) and add:
```go
// TextMatchMode enumerates LIKE-based string match modes.
type TextMatchMode string
const (
TextMatchContains TextMatchMode = "contains"
TextMatchPrefix TextMatchMode = "prefix"
TextMatchSuffix TextMatchMode = "suffix"
)
// TextMatchCondition models a case-insensitive LIKE match on a scalar string field
// (content.contains/startsWith/endsWith).
type TextMatchCondition struct {
Field string
Mode TextMatchMode
Value string
}
func (*TextMatchCondition) isCondition() {}
```
- [ ] **Step 4: Update the parser in `internal/filter/parser.go`**
In `buildCallCondition`, replace the `case "contains":` line with:
```go
case "contains":
return buildTextMatchCondition(call, schema, TextMatchContains)
```
Delete `buildContainsCondition` (lines ~196-227) and add:
```go
func buildTextMatchCondition(call *exprv1.Expr_Call, schema Schema, mode TextMatchMode) (Condition, error) {
if call.Target == nil {
return nil, errors.New("text match requires a target")
}
targetName, err := getIdentName(call.Target)
if err != nil {
return nil, err
}
field, ok := schema.Field(targetName)
if !ok {
return nil, errors.Errorf("unknown identifier %q", targetName)
}
if !field.SupportsContains {
return nil, errors.Errorf("identifier %q does not support text matching", targetName)
}
if len(call.Args) != 1 {
return nil, errors.New("text match expects exactly one argument")
}
value, err := getConstValue(call.Args[0])
if err != nil {
return nil, errors.Wrap(err, "text match only supports literal arguments")
}
str, ok := value.(string)
if !ok {
return nil, errors.New("text match argument must be a string")
}
return &TextMatchCondition{Field: targetName, Mode: mode, Value: str}, nil
}
```
- [ ] **Step 5: Update the renderer in `internal/filter/render.go`**
In `renderCondition`, replace `case *ContainsCondition:` / `return r.renderContainsCondition(c)` with:
```go
case *TextMatchCondition:
return r.renderTextMatch(c)
```
Delete `renderContainsCondition` (lines ~449-469) and add:
```go
func (r *renderer) renderTextMatch(cond *TextMatchCondition) (renderResult, error) {
field, ok := r.schema.Field(cond.Field)
if !ok {
return renderResult{}, errors.Errorf("unknown field %q", cond.Field)
}
column := field.columnExpr(r.dialect)
pattern := likePattern(cond.Mode, cond.Value)
return renderResult{sql: r.foldedLike(column, pattern)}, nil
}
// foldedLike renders a case-insensitive LIKE comparison of colExpr against a
// (already metacharacter-escaped) pattern, using each dialect's case-folding.
func (r *renderer) foldedLike(colExpr, pattern string) string {
switch r.dialect {
case DialectSQLite:
// memos_unicode_lower gives Unicode-aware folding; ESCAPE '\' is required
// because SQLite has no default LIKE escape character.
return fmt.Sprintf(`memos_unicode_lower(%s) LIKE memos_unicode_lower(%s) ESCAPE '\'`, colExpr, r.addArg(pattern))
case DialectPostgres:
// ILIKE is case-insensitive; backslash is the default escape character.
return fmt.Sprintf("%s ILIKE %s", colExpr, r.addArg(pattern))
default: // MySQL: default collation is case-insensitive; backslash is the default escape.
return fmt.Sprintf("%s LIKE %s", colExpr, r.addArg(pattern))
}
}
// likePattern escapes LIKE metacharacters in value and wraps it for the mode.
func likePattern(mode TextMatchMode, value string) string {
escaped := escapeLikeLiteral(value)
switch mode {
case TextMatchPrefix:
return escaped + "%"
case TextMatchSuffix:
return "%" + escaped
default:
return "%" + escaped + "%"
}
}
// escapeLikeLiteral escapes the LIKE metacharacters \, %, and _ so user input
// is matched literally. Backslash is the escape character on all three dialects.
func escapeLikeLiteral(s string) string {
return strings.NewReplacer(`\`, `\\`, `%`, `\%`, `_`, `\_`).Replace(s)
}
```
- [ ] **Step 6: Run the new test + existing suites to verify green**
Run: `go test ./internal/filter/ -v`
Expected: PASS (including `TestCompileContainsEscapesLikeWildcards`).
Run: `go test ./store/test/ -run TestMemoFilterContent -v`
Expected: PASS (existing `contains` behavioral tests, including special-characters/unicode, still pass).
- [ ] **Step 7: Commit**
```bash
git add internal/filter/ir.go internal/filter/parser.go internal/filter/render.go internal/filter/engine_test.go
git commit -m "refactor(filter): unify string matching into TextMatchCondition with LIKE escaping
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>"
```
---
### Task 2: Scalar `startsWith()` / `endsWith()`
Wire the new prefix/suffix modes through the parser and enable text matching on attachment `mime_type`. Rendering already exists from Task 1.
**Files:**
- Modify: `internal/filter/parser.go` (add `startsWith`/`endsWith` cases)
- Modify: `internal/filter/schema.go` (enable `SupportsContains` on `mime_type`)
- Modify: `store/test/memo_filter_test.go`, `store/test/attachment_filter_test.go` (behavioral tests)
- Modify: `internal/filter/engine_test.go` (reject on unsupported field)
- [ ] **Step 1: Add failing behavioral tests** in `store/test/memo_filter_test.go`
```go
func TestMemoFilterContentStartsWith(t *testing.T) {
t.Parallel()
tc := NewMemoFilterTestContext(t)
defer tc.Close()
tc.CreateMemo(NewMemoBuilder("memo-todo", tc.User.ID).Content("TODO: buy milk"))
tc.CreateMemo(NewMemoBuilder("memo-done", tc.User.ID).Content("Done with milk"))
// Prefix match, case-insensitive (consistent with contains()).
memos := tc.ListWithFilter(`content.startsWith("todo")`)
require.Len(t, memos, 1)
require.Equal(t, "memo-todo", memos[0].UID)
memos = tc.ListWithFilter(`content.startsWith("nope")`)
require.Len(t, memos, 0)
}
func TestMemoFilterContentEndsWith(t *testing.T) {
t.Parallel()
tc := NewMemoFilterTestContext(t)
defer tc.Close()
tc.CreateMemo(NewMemoBuilder("memo-md", tc.User.ID).Content("notes.md"))
tc.CreateMemo(NewMemoBuilder("memo-txt", tc.User.ID).Content("notes.txt"))
memos := tc.ListWithFilter(`content.endsWith(".md")`)
require.Len(t, memos, 1)
require.Equal(t, "memo-md", memos[0].UID)
}
```
And in `store/test/attachment_filter_test.go`:
```go
func TestAttachmentFilterFilenameStartsWith(t *testing.T) {
t.Parallel()
tc := NewAttachmentFilterTestContextWithUser(t)
defer tc.Close()
tc.CreateAttachment(NewAttachmentBuilder(tc.CreatorID).Filename("invoice-2026.pdf").MimeType("application/pdf"))
tc.CreateAttachment(NewAttachmentBuilder(tc.CreatorID).Filename("photo.png").MimeType("image/png"))
got := tc.ListWithFilter(`filename.startsWith("invoice")`)
require.Len(t, got, 1)
require.Equal(t, "invoice-2026.pdf", got[0].Filename)
// mime_type prefix matching (newly enabled).
got = tc.ListWithFilter(`mime_type.startsWith("image/")`)
require.Len(t, got, 1)
require.Equal(t, "photo.png", got[0].Filename)
}
```
- [ ] **Step 2: Run to verify they fail**
Run: `go test ./store/test/ -run 'TestMemoFilterContentStartsWith|TestMemoFilterContentEndsWith|TestAttachmentFilterFilenameStartsWith' -v`
Expected: FAIL — `startsWith` hits `buildCallCondition`'s default branch ("unsupported call expression"), and `mime_type` is not yet text-matchable.
- [ ] **Step 3: Add parser cases** in `internal/filter/parser.go` `buildCallCondition`
Immediately after the `case "contains":` line, add:
```go
case "startsWith":
return buildTextMatchCondition(call, schema, TextMatchPrefix)
case "endsWith":
return buildTextMatchCondition(call, schema, TextMatchSuffix)
```
- [ ] **Step 4: Enable text matching on `mime_type`** in `internal/filter/schema.go`
In `NewAttachmentSchema`, add `SupportsContains: true` to the `mime_type` field entry:
```go
"mime_type": {
Name: "mime_type",
Kind: FieldKindScalar,
Type: FieldTypeString,
Column: Column{Table: "attachment", Name: "type"},
SupportsContains: true,
Expressions: map[DialectName]string{},
},
```
- [ ] **Step 5: Add a compile-reject unit test** in `internal/filter/engine_test.go`
```go
func TestCompileRejectsStartsWithOnUnsupportedField(t *testing.T) {
t.Parallel()
engine, err := NewEngine(NewSchema())
require.NoError(t, err)
_, err = engine.Compile(context.Background(), `visibility.startsWith("P")`)
require.Error(t, err)
require.Contains(t, err.Error(), "does not support text matching")
}
```
- [ ] **Step 6: Run tests to verify green**
Run: `go test ./internal/filter/ ./store/test/ -run 'StartsWith|EndsWith|TextMatch' -v`
Expected: PASS.
- [ ] **Step 7: Commit**
```bash
git add internal/filter/parser.go internal/filter/schema.go internal/filter/engine_test.go store/test/memo_filter_test.go store/test/attachment_filter_test.go
git commit -m "feat(filter): support startsWith()/endsWith() on scalar string fields
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>"
```
---
### Task 3: Register a SQLite `REGEXP` function
`modernc.org/sqlite` has no built-in `REGEXP`. SQLite desugars `X REGEXP Y` to `regexp(Y, X)`, so register a 2-arg `regexp(pattern, value)` scalar function backed by Go's `regexp`, mirroring `ensureUnicodeLowerRegistered`.
**Files:**
- Modify: `store/db/sqlite/functions.go`
- Modify: `store/db/sqlite/sqlite.go`
- Test: `store/db/sqlite/functions_test.go` (create)
- [ ] **Step 1: Write a failing test** — create `store/db/sqlite/functions_test.go`
```go
package sqlite
import (
"testing"
"github.com/stretchr/testify/require"
)
func TestRegexpFunctionMatches(t *testing.T) {
require.NoError(t, ensureRegexpRegistered())
re, err := compileRegexp(`^v\d+$`)
require.NoError(t, err)
require.True(t, re.MatchString("v12"))
require.False(t, re.MatchString("version"))
// Caching returns the same compiled instance.
re2, err := compileRegexp(`^v\d+$`)
require.NoError(t, err)
require.Same(t, re, re2)
_, err = compileRegexp(`(`)
require.Error(t, err)
}
```
- [ ] **Step 2: Run to verify it fails**
Run: `go test ./store/db/sqlite/ -run TestRegexpFunctionMatches -v`
Expected: FAIL — `ensureRegexpRegistered`/`compileRegexp` undefined.
- [ ] **Step 3: Implement in `store/db/sqlite/functions.go`**
Add `"errors"` and `"regexp"` to the imports, then append:
```go
var (
registerRegexpOnce sync.Once
registerRegexpErr error
// regexpCache memoizes compiled patterns; keys are pattern strings.
regexpCache sync.Map
)
// ensureRegexpRegistered registers a Go-backed `regexp(pattern, value)` scalar
// function so SQLite's `value REGEXP pattern` operator works (modernc.org/sqlite
// has no built-in implementation). Patterns use Go's RE2 syntax. Registered once
// globally; safe to call multiple times.
func ensureRegexpRegistered() error {
registerRegexpOnce.Do(func() {
registerRegexpErr = msqlite.RegisterScalarFunction("regexp", 2, func(_ *msqlite.FunctionContext, args []driver.Value) (driver.Value, error) {
if len(args) != 2 || args[0] == nil || args[1] == nil {
return int64(0), nil
}
pattern, ok := args[0].(string)
if !ok {
return nil, errors.New("regexp pattern must be a string")
}
var value string
switch v := args[1].(type) {
case string:
value = v
case []byte:
value = string(v)
default:
return int64(0), nil
}
re, err := compileRegexp(pattern)
if err != nil {
return nil, err
}
if re.MatchString(value) {
return int64(1), nil
}
return int64(0), nil
})
})
return registerRegexpErr
}
// compileRegexp compiles and caches a RE2 pattern.
func compileRegexp(pattern string) (*regexp.Regexp, error) {
if cached, ok := regexpCache.Load(pattern); ok {
return cached.(*regexp.Regexp), nil
}
re, err := regexp.Compile(pattern)
if err != nil {
return nil, err
}
regexpCache.Store(pattern, re)
return re, nil
}
```
- [ ] **Step 4: Wire into `NewDB`** in `store/db/sqlite/sqlite.go`
Right after the `ensureUnicodeLowerRegistered()` block, add:
```go
if err := ensureRegexpRegistered(); err != nil {
return nil, errors.Wrap(err, "failed to register sqlite regexp function")
}
```
- [ ] **Step 5: Run to verify green**
Run: `go test ./store/db/sqlite/ -run TestRegexpFunctionMatches -v`
Expected: PASS.
- [ ] **Step 6: Commit**
```bash
git add store/db/sqlite/functions.go store/db/sqlite/sqlite.go store/db/sqlite/functions_test.go
git commit -m "feat(sqlite): register Go-backed REGEXP function
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>"
```
---
### Task 4: `matches(regex)` on string fields
Add the IR node, parser recognition, per-dialect rendering, and the compile-time regex validator.
**Files:**
- Modify: `internal/filter/ir.go` (add `RegexCondition`)
- Modify: `internal/filter/parser.go` (`matches` case + builder)
- Modify: `internal/filter/render.go` (`renderRegex`)
- Modify: `internal/filter/schema.go` (add `cel.ValidateRegexLiterals()` to both schemas)
- Modify: `internal/filter/engine_test.go`, `store/test/memo_filter_test.go`
- [ ] **Step 1: Add failing tests** — compile-level in `internal/filter/engine_test.go`:
```go
func TestCompileRejectsMalformedRegex(t *testing.T) {
t.Parallel()
engine, err := NewEngine(NewSchema())
require.NoError(t, err)
_, err = engine.Compile(context.Background(), `content.matches("(")`)
require.Error(t, err)
}
func TestCompileMatchesRendersRegexOperator(t *testing.T) {
t.Parallel()
engine, err := NewEngine(NewSchema())
require.NoError(t, err)
stmt, err := engine.CompileToStatement(context.Background(), `content.matches("v[0-9]+")`, RenderOptions{Dialect: DialectPostgres})
require.NoError(t, err)
require.Contains(t, stmt.SQL, "~")
require.Equal(t, []any{"v[0-9]+"}, stmt.Args)
}
```
And behavioral in `store/test/memo_filter_test.go` (runs against SQLite by default, exercising the registered `REGEXP` function):
```go
func TestMemoFilterContentMatches(t *testing.T) {
t.Parallel()
tc := NewMemoFilterTestContext(t)
defer tc.Close()
tc.CreateMemo(NewMemoBuilder("memo-v1", tc.User.ID).Content("release v12 shipped"))
tc.CreateMemo(NewMemoBuilder("memo-plain", tc.User.ID).Content("no version here"))
memos := tc.ListWithFilter(`content.matches("v[0-9]+")`)
require.Len(t, memos, 1)
require.Equal(t, "memo-v1", memos[0].UID)
memos = tc.ListWithFilter(`content.matches("^xyz")`)
require.Len(t, memos, 0)
}
```
- [ ] **Step 2: Run to verify they fail**
Run: `go test ./internal/filter/ -run 'Malformed|MatchesRenders' -v && go test ./store/test/ -run TestMemoFilterContentMatches -v`
Expected: FAIL — `matches` is unhandled and no regex validator is configured.
- [ ] **Step 3: Add the IR node** in `internal/filter/ir.go`
```go
// RegexCondition models field.matches("pattern") on a string field.
type RegexCondition struct {
Field string
Pattern string
}
func (*RegexCondition) isCondition() {}
```
- [ ] **Step 4: Add parser support** in `internal/filter/parser.go`
In `buildCallCondition`, after the `case "endsWith":` block, add:
```go
case "matches":
return buildMatchesCondition(call, schema)
```
Then add the builder:
```go
func buildMatchesCondition(call *exprv1.Expr_Call, schema Schema) (Condition, error) {
if call.Target == nil {
return nil, errors.New("matches requires a target")
}
targetName, err := getIdentName(call.Target)
if err != nil {
return nil, err
}
field, ok := schema.Field(targetName)
if !ok {
return nil, errors.Errorf("unknown identifier %q", targetName)
}
if !field.SupportsContains {
return nil, errors.Errorf("identifier %q does not support matches()", targetName)
}
if len(call.Args) != 1 {
return nil, errors.New("matches expects exactly one argument")
}
value, err := getConstValue(call.Args[0])
if err != nil {
return nil, errors.Wrap(err, "matches only supports literal arguments")
}
pattern, ok := value.(string)
if !ok {
return nil, errors.New("matches argument must be a string")
}
return &RegexCondition{Field: targetName, Pattern: pattern}, nil
}
```
- [ ] **Step 5: Add the renderer** in `internal/filter/render.go`
In `renderCondition`, after the `case *TextMatchCondition:` arm, add:
```go
case *RegexCondition:
return r.renderRegex(c)
```
Then add:
```go
func (r *renderer) renderRegex(cond *RegexCondition) (renderResult, error) {
field, ok := r.schema.Field(cond.Field)
if !ok {
return renderResult{}, errors.Errorf("unknown field %q", cond.Field)
}
column := field.columnExpr(r.dialect)
switch r.dialect {
case DialectPostgres:
// POSIX regex match operator.
return renderResult{sql: fmt.Sprintf("%s ~ %s", column, r.addArg(cond.Pattern))}, nil
case DialectMySQL, DialectSQLite:
// MySQL has a native REGEXP operator; SQLite uses the registered regexp() function.
return renderResult{sql: fmt.Sprintf("%s REGEXP %s", column, r.addArg(cond.Pattern))}, nil
default:
return renderResult{}, errors.Errorf("unsupported dialect %s", r.dialect)
}
}
```
- [ ] **Step 6: Add the regex validator** in `internal/filter/schema.go`
Add the `cel` import line already present. In **both** `NewSchema` and `NewAttachmentSchema`, append the validator to the `envOptions` slice (e.g. after `nowFunction`):
```go
cel.ASTValidators(cel.ValidateRegexLiterals()),
```
- [ ] **Step 7: Run tests to verify green**
Run: `go test ./internal/filter/ -run 'Malformed|MatchesRenders' -v && go test ./store/test/ -run TestMemoFilterContentMatches -v`
Expected: PASS.
- [ ] **Step 8: Commit**
```bash
git add internal/filter/ir.go internal/filter/parser.go internal/filter/render.go internal/filter/schema.go internal/filter/engine_test.go store/test/memo_filter_test.go
git commit -m "feat(filter): support matches() regex on string fields
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>"
```
---
### Task 5: `all()` comprehension on tags (non-empty)
The heaviest task. `exists()` matches against the *serialized* JSON array and cannot express "every element matches", so `all()` needs real per-element iteration via `json_each` / `jsonb_array_elements_text` / `JSON_TABLE`, plus a non-empty guard.
**Files:**
- Modify: `internal/filter/ir.go` (add `ComprehensionAll`)
- Modify: `internal/filter/parser.go` (accept `all()` in `detectComprehensionKind`)
- Modify: `internal/filter/render.go` (`renderTagAll` + element predicate SQL; branch in `renderListComprehension`)
- Modify: `store/test/memo_filter_test.go`
- [ ] **Step 1: Add failing behavioral tests** in `store/test/memo_filter_test.go`
```go
func TestMemoFilterTagsAll(t *testing.T) {
t.Parallel()
tc := NewMemoFilterTestContext(t)
defer tc.Close()
tc.CreateMemo(NewMemoBuilder("memo-all-work", tc.User.ID).Content("all work").Tags("work/a", "work/b"))
tc.CreateMemo(NewMemoBuilder("memo-mixed", tc.User.ID).Content("mixed").Tags("work/a", "home"))
tc.CreateMemo(NewMemoBuilder("memo-untagged", tc.User.ID).Content("untagged"))
// Every tag starts with "work/": only the all-work memo qualifies.
memos := tc.ListWithFilter(`tags.all(t, t.startsWith("work/"))`)
require.Len(t, memos, 1)
require.Equal(t, "memo-all-work", memos[0].UID)
// Untagged memos must NOT match (non-empty guard, decision B).
require.NotContains(t, uids(memos), "memo-untagged")
}
func TestMemoFilterTagsAllEquals(t *testing.T) {
t.Parallel()
tc := NewMemoFilterTestContext(t)
defer tc.Close()
tc.CreateMemo(NewMemoBuilder("memo-only-x", tc.User.ID).Content("only x").Tags("x", "x"))
tc.CreateMemo(NewMemoBuilder("memo-x-and-y", tc.User.ID).Content("x and y").Tags("x", "y"))
memos := tc.ListWithFilter(`tags.all(t, t == "x")`)
require.Len(t, memos, 1)
require.Equal(t, "memo-only-x", memos[0].UID)
}
```
Add this helper near the top of `store/test/memo_filter_test.go` (after the imports) if not already present:
```go
func uids(memos []*store.Memo) []string {
out := make([]string, 0, len(memos))
for _, m := range memos {
out = append(out, m.UID)
}
return out
}
```
- [ ] **Step 2: Run to verify they fail**
Run: `go test ./store/test/ -run 'TestMemoFilterTagsAll' -v`
Expected: FAIL — `detectComprehensionKind` returns "all() comprehension is not supported".
- [ ] **Step 3: Add the IR kind** in `internal/filter/ir.go`
In the `ComprehensionKind` const block, add `ComprehensionAll`:
```go
const (
ComprehensionExists ComprehensionKind = "exists"
ComprehensionAll ComprehensionKind = "all"
)
```
- [ ] **Step 4: Accept `all()` in the parser** in `internal/filter/parser.go`
In `detectComprehensionKind`, replace the `all()` rejection block:
```go
// all() starts with true and uses AND (&&) - not supported
if accuInit.GetBoolValue() {
if step := comp.LoopStep.GetCallExpr(); step != nil && step.Function == "_&&_" {
return "", errors.New("all() comprehension is not supported; use exists() instead")
}
}
```
with:
```go
// all() starts with true and uses AND (&&) in the loop step.
if accuInit.GetBoolValue() {
if step := comp.LoopStep.GetCallExpr(); step != nil && step.Function == "_&&_" {
return ComprehensionAll, nil
}
}
```
- [ ] **Step 5: Branch and render in `internal/filter/render.go`**
At the top of `renderListComprehension`, right after the `field.Kind != FieldKindJSONList` guard, add:
```go
if cond.Kind == ComprehensionAll {
return r.renderTagAll(field, cond.Predicate)
}
```
Then add the new render path + element-predicate helper:
```go
// renderTagAll renders tags.all(t, <pred>): the array is non-empty AND no element
// fails the predicate. Element predicates use plain CEL semantics (case-insensitive
// for startsWith/endsWith/contains, case-sensitive for ==), evaluated per element.
func (r *renderer) renderTagAll(field Field, pred PredicateExpr) (renderResult, error) {
arrayExpr := jsonArrayExpr(r.dialect, field)
elemCond, err := r.elementPredicateSQL(pred)
if err != nil {
return renderResult{}, err
}
switch r.dialect {
case DialectSQLite:
nonEmpty := fmt.Sprintf("%s IS NOT NULL AND %s != '[]'", arrayExpr, arrayExpr)
sub := fmt.Sprintf("NOT EXISTS (SELECT 1 FROM json_each(%s) WHERE NOT (%s))", arrayExpr, elemCond)
return renderResult{sql: fmt.Sprintf("(%s AND %s)", nonEmpty, sub)}, nil
case DialectMySQL:
nonEmpty := fmt.Sprintf("%s IS NOT NULL AND JSON_LENGTH(%s) > 0", arrayExpr, arrayExpr)
sub := fmt.Sprintf("NOT EXISTS (SELECT 1 FROM JSON_TABLE(%s, '$[*]' COLUMNS (value VARCHAR(512) PATH '$')) AS elem WHERE NOT (%s))", arrayExpr, elemCond)
return renderResult{sql: fmt.Sprintf("(%s AND %s)", nonEmpty, sub)}, nil
case DialectPostgres:
nonEmpty := fmt.Sprintf("%s IS NOT NULL AND jsonb_array_length(%s) > 0", arrayExpr, arrayExpr)
sub := fmt.Sprintf("NOT EXISTS (SELECT 1 FROM jsonb_array_elements_text(%s) AS elem(value) WHERE NOT (%s))", arrayExpr, elemCond)
return renderResult{sql: fmt.Sprintf("(%s AND %s)", nonEmpty, sub)}, nil
default:
return renderResult{}, errors.Errorf("unsupported dialect %s", r.dialect)
}
}
// elementPredicateSQL builds the per-element SQL condition for an all() predicate.
// The iterated element is exposed as the unqualified column `value` on all dialects
// (json_each.value / JSON_TABLE column / elem(value)).
func (r *renderer) elementPredicateSQL(pred PredicateExpr) (string, error) {
switch p := pred.(type) {
case *EqualsPredicate:
return fmt.Sprintf("value = %s", r.addArg(p.Value)), nil
case *StartsWithPredicate:
return r.foldedLike("value", likePattern(TextMatchPrefix, p.Prefix)), nil
case *EndsWithPredicate:
return r.foldedLike("value", likePattern(TextMatchSuffix, p.Suffix)), nil
case *ContainsPredicate:
return r.foldedLike("value", likePattern(TextMatchContains, p.Substring)), nil
default:
return "", errors.Errorf("unsupported predicate %T in all()", pred)
}
}
```
> Note: `foldedLike`, `likePattern`, and `escapeLikeLiteral` were added in Task 1; reuse them as-is.
- [ ] **Step 6: Run tests to verify green**
Run: `go test ./store/test/ -run 'TestMemoFilterTagsAll' -v`
Expected: PASS.
Run: `go test ./store/test/ -run 'TestMemoFilterTagsExists' -v`
Expected: PASS (exists() rendering untouched).
- [ ] **Step 7: Commit**
```bash
git add internal/filter/ir.go internal/filter/parser.go internal/filter/render.go store/test/memo_filter_test.go
git commit -m "feat(filter): support all() comprehension over tags
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>"
```
---
### Task 6: Docs + full verification
**Files:**
- Modify: `internal/filter/README.md`
- [ ] **Step 1: Document the new syntax** — append to the "SQL Generation Notes" section of `internal/filter/README.md`:
```markdown
- **String Matching** — `content.contains(x)`, `content.startsWith(x)`, and
`content.endsWith(x)` render as case-insensitive `LIKE`/`ILIKE` with LIKE
metacharacters (`%`, `_`, `\`) escaped. Available on scalar string fields whose
schema sets `SupportsContains` (memo `content`; attachment `filename`,
`mime_type`).
- **Regex** — `field.matches("pattern")` renders to `~` (Postgres) or `REGEXP`
(MySQL/SQLite). SQLite uses a Go-backed `regexp` function registered in
`store/db/sqlite/functions.go`. Patterns are validated at compile time against
Go's RE2 via `cel.ValidateRegexLiterals()`. **Caveat:** regex *syntax* differs
per engine (Go RE2 on SQLite, POSIX ERE on Postgres, ICU on MySQL 8.0+), so
engine-specific patterns may not be portable.
- **Tag `all()`** — `tags.all(t, <pred>)` matches only non-empty tag sets where
every element satisfies the predicate, via per-element iteration
(`json_each` / `jsonb_array_elements_text` / `JSON_TABLE`).
```
- [ ] **Step 2: Run the full engine + store suite (SQLite)**
Run: `go test ./internal/filter/... ./store/...`
Expected: PASS.
- [ ] **Step 3: Vet and lint**
Run: `go vet ./internal/filter/... ./store/db/sqlite/...`
Expected: no output.
Run: `golangci-lint run internal/filter/... store/db/sqlite/...` (if available; skip if the binary is absent).
Expected: no findings.
- [ ] **Step 4: Cross-dialect verification (if Docker/CI DSNs available)**
Run MySQL and Postgres suites to confirm the `all()` subqueries and regex operators render correctly:
```bash
DRIVER=mysql go test ./store/test/ -run 'TagsAll|Matches|StartsWith|EndsWith'
DRIVER=postgres go test ./store/test/ -run 'TagsAll|Matches|StartsWith|EndsWith'
```
Expected: PASS. (These require the project's standard test DB setup; if unavailable locally, rely on CI which runs all three drivers.)
- [ ] **Step 5: Commit**
```bash
git add internal/filter/README.md
git commit -m "docs(filter): document string matching, regex, and tag all() support
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>"
```
---
## Self-review notes
- **Spec coverage:** ① scalar `startsWith`/`endsWith` → Task 2; ② `all()` non-empty → Task 5; ④ `matches()` + SQLite REGEXP fn + `ValidateRegexLiterals` → Tasks 3-4; the LIKE-escaping fix → Task 1; docs/caveat → Task 6. `lowerAscii`/`upperAscii` correctly omitted (dropped in spec). Hardening/native-AST migration correctly deferred to the follow-up spec.
- **Type consistency:** `TextMatchCondition`/`TextMatchMode`/`likePattern`/`foldedLike`/`escapeLikeLiteral` (Task 1) are reused by Tasks 2 and 5; `RegexCondition`/`renderRegex` (Task 4) and `ensureRegexpRegistered`/`compileRegexp` (Task 3) names match across their call sites; `ComprehensionAll` (Task 5) matches its parser and render references.
- **Element reference:** the unqualified `value` column is produced by `json_each` (SQLite), the `JSON_TABLE(... COLUMNS (value ...))` (MySQL), and `elem(value)` (Postgres), so `elementPredicateSQL` is dialect-agnostic.
```
@@ -0,0 +1,124 @@
# ActivityCalendar: Honor `timeBasis` (Create vs Update)
## Problem
The ActivityCalendar in `web/src/components/ActivityCalendar/` always aggregates by memo creation time, regardless of how the surrounding memo list is sorted.
The application already supports a global "time basis" toggle (`web/src/contexts/ViewContext.tsx:3`):
```ts
export type MemoTimeBasis = "create_time" | "update_time";
```
The toggle is persisted in `localStorage` and drives memo list ordering across the app. When a user switches the list to `update_time`, the heatmap below it continues to show creation counts — the two views literally disagree about what "today" means.
This is the user-visible bug we are fixing.
## Non-goals
The following were considered and explicitly excluded:
- **Tracking every individual edit event.** This would require resurrecting the `activity` table that was deliberately dropped in migration `0.27/03__drop_activity.sql`. The cost (write-path instrumentation, storage growth, privacy review) is not justified by this UI bug.
- **Tracking archive / restore / delete events.** Housekeeping actions, not contributions; would also leak private behavior on public Profile/Explore pages.
- **Adding comments or reactions to the heatmap count.** A reasonable separate feature, but a different decision (event-type expansion). Out of scope for this spec — one ticket, one problem.
- **Renaming `ActivityCalendar` to `ContributionCalendar`.** Out of scope.
## Design
### Semantics
The heatmap aggregates one timestamp per memo:
- When `timeBasis === "create_time"`: use `memo.created_ts` (current behavior).
- When `timeBasis === "update_time"`: use `memo.updated_ts`.
Each memo contributes exactly one cell of color, on the day of its chosen timestamp. This matches the list view's semantics exactly: in `update_time` mode, a memo edited on 5/1 and again on 5/2 appears once at 5/2 in the list, and the heatmap will show +1 on 5/2 and nothing on 5/1. The "lossiness" is identical to the lossiness already accepted by the list view — so by definition, the two are consistent.
### Backend
`UserStats` (proto/api/v1/user_service.proto) gains one field:
```proto
// The latest update timestamps of the user's memos.
repeated google.protobuf.Timestamp memo_updated_timestamps = 8;
```
The implementation mirrors `memo_created_timestamps` in `server/router/api/v1/user_service_stats.go`: in the same loop that appends `memo.CreatedTs` (line 115), also append `memo.UpdatedTs` to a parallel slice. The same `FindMemo` filters apply automatically: `RowStatus: NORMAL` (archived excluded), `ExcludeComments: true`, and the viewer-based visibility filter. Both `GetUserStats` and `ListAllUserStats` paths must be updated symmetrically.
No DB migration. No new tables. No new write paths.
### Frontend
`web/src/hooks/useFilteredMemoStats.ts` reads `useView().timeBasis` and switches its data source:
- `create_time``userStats.memoCreatedTimestamps` (today's behavior, untouched)
- `update_time``userStats.memoUpdatedTimestamps`
The `explore` context branch (which derives stats from the in-memory memo list rather than `userStats`) applies the same switch using `memo.createTime` vs `memo.updateTime` from the cached memos.
The `MonthCalendar` / `YearCalendar` components themselves require no changes — they receive an opaque `Record<date, count>` and render it. The change is confined to the data-source layer.
### Tooltip / labeling
A small but necessary clarification for the user: the cell tooltip should reflect which basis is active, e.g.
- `create_time` mode: "3 memos on May 2"
- `update_time` mode: "3 memos updated on May 2"
This belongs in `ActivityCalendar/utils.ts:getTooltipText`, which already takes a `t` translator. Add a `timeBasis` argument and pick the right i18n key.
## Components
| Unit | Responsibility | Depends on |
|---|---|---|
| `UserStats` proto | Carry both timestamp arrays | — |
| `GetUserStats` server impl | Populate both arrays from `memo` table | store |
| `useFilteredMemoStats` | Pick the correct array based on `timeBasis`; aggregate by day | `useView`, `useUserStats`, `useMemos` |
| `getTooltipText` | Render basis-aware tooltip | i18n |
| `MonthCalendar` / `YearCalendar` | Unchanged — render `Record<date, count>` | — |
## Data flow
```
ViewContext.timeBasis ──┐
useFilteredMemoStats ── pick array ── countBy(day) ── Record<date,count> ── MonthCalendar
userStats ───────────┤ (memo_created_timestamps OR memo_updated_timestamps)
memos cache ─────────┘ (createTime OR updateTime — explore context only)
```
## Error handling
No new failure modes.
`protobuf-es` generates `repeated` fields as non-optional `T[]`, so an older server that doesn't populate the new field deserializes it as `[]` (never `undefined`). Naïvely treating empty as "no data" would be wrong, because a user with zero memos also gets `[]`. Detection uses **length divergence**: since `memo.updated_ts` is initialized to `created_ts` at row creation, the two arrays are the same length whenever there are any memos. So:
- `created.length === 0 && updated.length === 0` — user has no memos, render empty.
- `created.length > 0 && updated.length === created.length` — new server, normal path.
- `created.length > 0 && updated.length === 0` — old server, fall back to `memoCreatedTimestamps` regardless of `timeBasis`, with a one-line `console.warn`.
## Testing
- Unit test `useFilteredMemoStats`: given a fixed `userStats`, switching `timeBasis` returns aggregations matching the expected source array.
- Unit test the new `getTooltipText` branch.
- Manual verification: in dev, toggle the global time basis and confirm:
- Heatmap recomputes
- A memo edited yesterday but created last week shows up "yesterday" in update mode and "last week" in create mode
- Tooltip text reflects the basis
## Migration / compatibility
- Proto field is additive (tag 8 is unused; tag 2 is `reserved`).
- Old clients ignore the new field.
- New clients tolerate old servers via the fallback above.
- No DB migration.
- No data backfill — `updated_ts` already exists on every memo row.
## Out-of-scope follow-ups (not part of this work)
These came up during brainstorming and are tracked here only so they aren't lost:
1. Adding comment / reaction event types to the heatmap count.
2. A "Memo History / Versions" feature (per-edit snapshots, diffs, optional commit messages). If pursued, the heatmap would become a downstream consumer of that history, and the field added here may be revisited.
@@ -0,0 +1,165 @@
# Create memo on selected calendar date — design
**Date:** 2026-05-02
**Scope:** Frontend-only.
## Problem
Clicking a date in the activity calendar filters the memo list to that date but does nothing for the inline editor. To create a memo dated for the selected day, the user must (1) create with today's timestamp, then (2) open the timestamp popover on the saved memo and edit `createTime`. This is a two-step retro-fill that defeats the calendar selection. Empty dates are also not clickable today, so the user cannot start the first memo for an empty day from the calendar at all.
## Goal
When a user picks a calendar date and immediately writes in the home editor, the resulting memo is created on that date — single-step.
## Non-goals
- Backend changes. The API already accepts custom `createTime` and `updateTime`.
- Changes to the comment/reply editor or the edit-mode editor.
- Changes outside the home page's editor render site (Explore/Archived/Profile pages have no editor).
- Reworking the timestamp popover UI itself.
- Empty-state copy changes on non-Home pages when an empty date is selected.
## User-visible behavior
1. **Empty calendar dates are clickable.** Clicking a date with zero memos sets the `displayTime` filter the same way a populated date does. Tooltip and selection ring still work.
2. **When the home editor renders with an active `displayTime` filter:**
- The `TimestampPopover` (already used in edit mode) appears in create mode, pre-populated with the selected date.
- The draft's `createTime` is set to **selected local date + current local hh:mm:ss** (e.g., picking May 1 at 14:32 → `2025-05-01 14:32`).
- The draft's `updateTime` is set to the same value, to avoid the saved memo immediately reading "updated today" relative to a back-dated `createTime`.
- The user can adjust either field via the popover before saving.
3. **When no `displayTime` filter is active**, the editor is identical to today: no popover in create mode, no override, server stamps with "now".
4. **Live derivation.** If the filter changes while a draft is in progress, the editor's prefilled timestamps re-sync to the new date. The popover stays visible so the change is observable. (Manual popover edits before the next filter change are overwritten — chosen tradeoff.)
5. **Future dates are allowed** (e.g., May 15 when today is May 2). Backend already accepts future timestamps.
6. **Other contexts** (Explore/Archived/Profile) gain empty-date clickability for navigation consistency, but have no editor and so no prefill behavior.
## Architecture
Four touch points, all in `web/src/`:
| File | Change |
|------|--------|
| `components/ActivityCalendar/CalendarCell.tsx` | Drop `day.count > 0` gate so empty in-month cells are clickable. |
| `components/MemoEditor/index.tsx` | Accept `defaultCreateTime?: Date` prop; render `TimestampPopover` in create mode when set; sync state on prop change. |
| `components/MemoEditor/utils/deriveDefaultCreateTime.ts` (new) | Pure helper: `(filters, now?) => Date \| undefined` derived from any `displayTime` filter. |
| `components/PagedMemoList/PagedMemoList.tsx` | At the home-editor render site (line 155), read `MemoFilterContext`, compute `defaultCreateTime`, pass as prop. |
### Data flow
```
CalendarCell click
→ useDateFilterNavigation
→ URL ?filter=displayTime:YYYY-MM-DD
→ MemoFilterContext re-renders
→ PagedMemoList recomputes defaultCreateTime via deriveDefaultCreateTimeFromFilters(filters)
→ <MemoEditor defaultCreateTime={...}> re-renders
→ editor reducer syncs state.timestamps (create + update) and renders TimestampPopover
→ save → memoService.ts:111 sends createTime/updateTime to API
```
## Component contracts
### `MemoEditor` — new prop
```ts
interface MemoEditorProps {
// ...existing props
/**
* When set in create mode (no `memo` prop), seeds the draft's
* createTime/updateTime and reveals the TimestampPopover so the
* user can adjust. Tracked live: changes after mount re-sync state.
* Ignored in edit mode (when `memo` is set).
*/
defaultCreateTime?: Date;
}
```
Internal behavior:
- On `INIT_MEMO` for create mode, if `defaultCreateTime` is set, payload `timestamps` is `{ createTime: defaultCreateTime, updateTime: defaultCreateTime }`.
- A `useEffect` keyed on `[defaultCreateTime?.getTime(), memo]` dispatches `SET_TIMESTAMPS` whenever the prop changes in create mode.
- Popover render condition becomes `memoName || (!memo && state.timestamps.createTime)`.
### `deriveDefaultCreateTimeFromFilters` — pure helper
```ts
// web/src/components/MemoEditor/utils/deriveDefaultCreateTime.ts
export function deriveDefaultCreateTimeFromFilters(
filters: MemoFilter[],
now: Date = new Date(),
): Date | undefined {
const dateFilter = filters.find((f) => f.factor === "displayTime");
if (!dateFilter) return undefined;
const [y, m, d] = dateFilter.value.split("-").map(Number);
if (!y || !m || !d) return undefined;
return new Date(y, m - 1, d, now.getHours(), now.getMinutes(), now.getSeconds());
}
```
Notes:
- Defensive parse — returns `undefined` for malformed values rather than throwing.
- `now` is injectable for deterministic tests.
- Multiple `displayTime` filters are not produced by current UI; `find` ignores extras safely.
### `PagedMemoList.tsx` — call-site change
```tsx
const { filters } = useMemoFilterContext();
const defaultCreateTime = useMemo(
() => deriveDefaultCreateTimeFromFilters(filters),
[filters],
);
// ...
{showMemoEditor ? (
<MemoEditor
className="mb-2"
cacheKey="home-memo-editor"
placeholder={t("editor.any-thoughts")}
defaultCreateTime={defaultCreateTime}
/>
) : null}
```
`useMemo` keyed on `filters` keeps the reference stable when the filter doesn't change, avoiding unnecessary editor re-syncs. `now` is captured once per filter change — matches "the local time when you picked the date".
### `CalendarCell.tsx` — empty-cell clickability
- `handleClick`: drop the `day.count > 0` check; just call `onClick(day.date)` if `onClick` is provided.
- `isInteractive`: `Boolean(onClick)`.
- `tabIndex` / `aria-disabled` / hover-cursor classes follow the new `isInteractive`.
- `shouldShowTooltip`: drop the `day.count > 0` gate; tooltip text already conveys the count.
- Out-of-month cells (existing early return) stay unclickable.
- The `selected` ring already works on count=0 cells. Visual contrast on the lowest-intensity background may need a small ring-weight bump in light theme; eyeball during implementation.
## Edge cases
- **No filter / filter cleared:** `defaultCreateTime` becomes `undefined`; editor falls back to current behavior.
- **User edits draft, then re-picks date:** live-derived; editor's `createTime` updates, popover reflects new value.
- **User manually edits via popover, then changes filter:** prop sync overwrites manual edit. Acceptable per design choice; popover keeps the change observable.
- **Draft cache (`cacheKey="home-memo-editor"`):** caches `content`, not `timestamps`. Reload restores text but `createTime` is freshly derived from current filter — consistent.
- **Future dates:** allowed. No clamp.
- **DST / timezone:** date arithmetic uses local time (`new Date(y, m-1, d, h, mi, s)`), matching `useDateFilterNavigation`'s local-date convention. Server receives an absolute `Timestamp`.
- **Comment editor (`MemoCommentSection`):** doesn't pass `defaultCreateTime` → no behavior change.
- **Edit mode (`memo` prop set):** prop is ignored; existing edit-mode popover is unchanged.
- **Empty-date click on Explore/Archived/Profile:** filters to empty date → "no memos" empty state. Acceptable.
## Testing
- **Unit (Vitest)** for `deriveDefaultCreateTimeFromFilters`:
- no `displayTime` filter → `undefined`
- valid `displayTime:2025-05-01` + injected `now=14:32:10``2025-05-01 14:32:10` local
- malformed value (`"not-a-date"`, `"2025-13-40"`) → `undefined`
- extra non-`displayTime` filters present → still works
- **Component (React Testing Library) for `CalendarCell`:** count=0 in-month cell is clickable, has correct `tabIndex`/`aria-disabled`, fires `onClick` with date.
- **Component for `MemoEditor`:** with `defaultCreateTime` prop, popover renders in create mode and `state.timestamps.createTime` matches; without prop, no popover; changing the prop re-syncs state.
- **Manual smoke (per CLAUDE.md UI-changes rule):** `pnpm dev`, click a non-today date (with and without existing memos), type a memo, save, confirm it appears under that date. Clear the filter chip; confirm a new memo posts to today.
## Risks
- The `useEffect` re-sync overwriting an in-progress popover edit is a *chosen* behavior. If users later complain, a "manual override sticky" flag is the natural follow-up. Not pre-built.
- Selection-ring contrast on the lowest-intensity background may need a small visual tweak; flagged for implementation.
## Out of scope (explicit)
- Sticky manual-override semantics for the popover.
- New empty-state copy on Explore/Archived/Profile when filtering to a date with zero memos.
- Any backend/API change.
- Any change to the comment editor, edit mode, or non-Home editor sites.
@@ -0,0 +1,511 @@
# STT and Audio-LLM Split — Design Spec
**Date:** 2026-05-02
**Status:** Draft, pending user review
## 1. Goal
Refactor `internal/ai/` to split **speech-to-text (STT)** and **audio-multimodal-LLM (Audio-LLM)** into two separate Go interfaces, aligning with mainstream OSS conventions (Vercel AI SDK, LiteLLM, the Go AI ecosystem). Update the public API handler to dispatch to the right interface based on provider type. Make two small comment improvements to `proto/store/instance_setting.proto` and the generated bindings; **no proto field changes**.
## 2. Non-Goals
The following are intentionally **out of scope** for this design:
- **`enabled` boolean field on `TranscriptionConfig`** (improvement #1 from the brainstorming) — keep using `provider_id == ""` as the disabled signal.
- **Direction C (audio → structured note pipeline)** — auto-summarization / tag extraction. Independent future feature.
- **Multi-provider STT with default-model selector** (Dify-style) — Memos has a single transcription config; that stays.
- **Per-model credential overrides**, **load balancing**, **capability YAML schemas** — Dify-style enterprise complexity.
- **Streaming transcription, retry policy, OpenAI Translations endpoint** — YAGNI.
- **`gpt-4o-audio-preview` user-facing support** — the `audiollm/openai` package will be implementable after this refactor, but UI support is a follow-up.
- **TTS** (text-to-speech) — different concern, not affected.
## 3. Background
The current `internal/ai/` has one `Transcriber` interface with two implementations: `openAITranscriber` (calls `/audio/transcriptions`, a real STT endpoint) and `geminiTranscriber` (calls `generateContent`, a multimodal-LLM endpoint dressed up to act like STT). This conflation has caused real symptoms:
- `TranscribeResponse.Language` and `Duration` are silently empty for Gemini (Gemini multimodal doesn't return them).
- The `Prompt` field has different semantics across providers — Whisper treats it as a soft hint that may be ignored; Gemini treats it as a literal instruction.
- Gemini's multimodal failure modes (safety filter, token-truncation, refusals) are flattened to a single "did not include text" error.
- Gemini-specific code (WebM transcoding, `maxGeminiInlineAudioSize`, `genai` SDK) lives in the same package as the OpenAI Whisper integration.
The brainstorming session (this conversation, 2026-05-02) ran two rounds of OSS research to validate the corrective direction.
## 4. Research Findings Summary
Detailed findings in conversation history; abridged here for design accountability.
### 4.1 SDK-Layer Research
| Source | Key Decision |
|---|---|
| **Vercel AI SDK** (`vercel/ai`) | `TranscriptionModelV3` is implemented **only** by providers with a dedicated STT endpoint (OpenAI Whisper/gpt-4o-transcribe, Deepgram, ElevenLabs, AssemblyAI). **Google provider deliberately does not implement it** — Gemini audio rides through `generateText` with `FilePart`. Two completely separate code paths. No "source" discriminator. Provider id is `vendor.modality` (`openai.transcription`); model is a free string. |
| **LiteLLM** (`BerriAI/litellm`) | `litellm.transcription()` only routes to providers with `/audio/transcriptions`-style endpoints. **Gemini is absent** from the transcription router (`litellm/llms/gemini/` has no `audio_transcription/` subdirectory). Multimodal audio rides through `litellm.completion()` with `{"type":"input_audio"}` content parts. Response is `text + usage`, no provider discriminator. |
| **Go AI SDKs** (`cloudwego/eino`, `tmc/langchaingo`, `sashabaranov/go-openai`) | One package per provider; provider identity = import path; **no provider enum**. `Model` is opaque string. OpenAI-compatible endpoints handled via `BaseURL` config field, never via separate package. go-openai's `audio.go` is structurally separate from `chat.go`. |
**Convergent finding:** All three ecosystems split STT and multimodal-audio into separate interfaces. None expose a "this came from a multimodal LLM" discriminator. None encode wire-format into the provider type enum.
### 4.2 Application-Layer Research
| Source | STT-Storage Design |
|---|---|
| **Open WebUI** | STT is a **flat singleton config block** (`audio.stt.*` namespace), completely separate from chat providers (`openai.*` namespace). `STT_ENGINE` enum dispatches; per-engine credentials side-by-side in one config. |
| **LobeChat** | STT is a **separate global user setting** (`UserTTSConfig`). But credentials silently piggyback on the `openai` chat provider's `keyVaults` — author has marked the helper `@deprecated`. |
| **Dify** | `ProviderEntity.supported_model_types` declares capabilities; STT is the `SPEECH2TEXT` enum value. STT info lives in a **separate "system model" config row** (`tenant_default_models(model_type='speech2text', provider_name, model_name)`) that **references** an existing provider. |
**Convergent finding:** Zero apps add STT-specific fields onto the AI provider entity. All three keep providers capability-agnostic and put STT config in a separate place.
### 4.3 Proto Schema Assessment
The current `proto/store/instance_setting.proto` `InstanceAISetting` + `TranscriptionConfig` is **already aligned with the mainstream pattern**:
-`AIProviderConfig` carries no STT-specific field (capability-agnostic)
-`TranscriptionConfig` is a separate pointer (`provider_id` references a provider)
-`AIProviderType` is vendor-level (`OPENAI`, `GEMINI`) — no wire-format suffix
-`model` is a free string
- ✅ Comments already document Whisper vs Gemini prompt semantics (though could be clearer)
The proto schema requires **no field changes**. Only two comment improvements (§7 below).
## 5. Current State (Files Touched)
```
internal/ai/
ai.go # ProviderType, ProviderConfig, errors
client.go # NewTranscriber factory, transcriberOptions, normalizeEndpoint, requireAPIKey
transcription.go # Transcriber interface, TranscribeRequest, TranscribeResponse
openai.go # openAITranscriber → /audio/transcriptions
openai_test.go
gemini.go # geminiTranscriber → generateContent (multimodal)
gemini_test.go
models.go # DefaultOpenAITranscriptionModel, DefaultGeminiTranscriptionModel
resolver.go # FindProvider
errors.go # ErrProviderNotFound, ErrCapabilityUnsupported
audio/
webm.go # IsWebMContentType, WebMOpusToWAV (used by Gemini path)
webm_test.go
server/router/api/v1/
ai_service.go # Transcribe handler (lines 42123)
proto/store/
instance_setting.proto # InstanceAISetting, AIProviderConfig, AIProviderType, TranscriptionConfig
web/src/components/Settings/
AISection.tsx # Provider list UI, TranscriptionForm
```
The handler at `server/router/api/v1/ai_service.go:42` is the **single integration point** between proto config and the `internal/ai/` SDK. It already discards Language/Duration (returns `{Text}` only), so the response narrowing is already in place.
## 6. Target Design
### 6.1 Package Structure
```
internal/ai/
ai.go # ProviderType (unchanged: OPENAI, GEMINI), ProviderConfig (unchanged)
resolver.go # FindProvider (unchanged)
errors.go # add ErrSTTNotSupported, ErrAudioLLMNotSupported
audio/
webm.go # unchanged — moves with audiollm/gemini consumer
webm_test.go
stt/
stt.go # Transcriber interface, Request, Response, Segment
factory.go # NewTranscriber(cfg ai.ProviderConfig, opts...) (Transcriber, error)
options.go # TranscriberOption, WithHTTPClient
openai/
openai.go # openAITranscriber → POST /audio/transcriptions
openai_test.go
audiollm/
audiollm.go # Model interface, Request, Response, FinishReason
factory.go # NewModel(cfg ai.ProviderConfig, opts...) (Model, error)
options.go # ModelOption, WithHTTPClient
gemini/
gemini.go # geminiModel → POST :generateContent (multimodal audio)
gemini_test.go
# openai/ — NOT created in this refactor; reserved for future gpt-4o-audio support
```
**Rationale (Go-ecosystem convention, per §4.1):** one package per provider; provider identity is import path; capability is implied by which umbrella package (`stt` vs `audiollm`) you import from. The runtime dispatch (factory) is the only place that translates `ProviderConfig.Type` enum → concrete implementation.
### 6.2 Interfaces
#### `internal/ai/stt/stt.go`
```go
package stt
import (
"context"
"io"
)
// Transcriber transcribes audio into text using a provider's dedicated STT endpoint
// (e.g. OpenAI /audio/transcriptions). Implementations are deterministic STT —
// they are NOT for multimodal LLMs that happen to accept audio input. For
// multimodal audio understanding, see internal/ai/audiollm.
type Transcriber interface {
Transcribe(ctx context.Context, req Request) (*Response, error)
}
type Request struct {
Audio io.Reader
Size int64
Filename string
ContentType string // IANA media type, e.g. "audio/wav"
Model string // provider-specific model id (e.g. "whisper-1", "gpt-4o-transcribe")
Prompt string // soft spelling/vocabulary hint (Whisper "prompt" parameter)
Language string // ISO 639-1, optional
}
type Response struct {
Text string
Language string // empty if provider did not return it (best-effort)
Segments []Segment // empty unless provider returned timestamps
}
type Segment struct {
Text string
Start float64
End float64
Speaker string // empty unless using a diarization-capable model (e.g. gpt-4o-transcribe-diarize)
}
```
#### `internal/ai/audiollm/audiollm.go`
```go
package audiollm
import (
"context"
"io"
)
// Model invokes a multimodal LLM with audio input. Implementations call
// chat-completions or generate-content style APIs that happen to accept audio.
// They are NOT deterministic STT — outputs may be refused, truncated, or
// rephrased per the LLM's behavior. For pure transcription, prefer
// internal/ai/stt where available.
type Model interface {
GenerateFromAudio(ctx context.Context, req Request) (*Response, error)
}
type Request struct {
Audio io.Reader
Size int64
ContentType string
Model string
Instructions string // literal instruction the model is expected to follow
Temperature *float32 // optional; nil leaves provider default
}
type Response struct {
Text string
FinishReason FinishReason
}
type FinishReason string
const (
FinishStop FinishReason = "stop" // model finished normally
FinishLength FinishReason = "length" // truncated by max-tokens
FinishSafety FinishReason = "safety" // safety filter blocked output
FinishOther FinishReason = "other" // anything else (incl. unknown)
)
```
#### Factory dispatch
```go
// internal/ai/stt/factory.go
package stt
import (
"github.com/pkg/errors"
"github.com/usememos/memos/internal/ai"
"github.com/usememos/memos/internal/ai/stt/openai"
)
func NewTranscriber(cfg ai.ProviderConfig, opts ...TranscriberOption) (Transcriber, error) {
switch cfg.Type {
case ai.ProviderOpenAI:
return openai.New(cfg, applyOptions(opts...))
case ai.ProviderGemini:
return nil, errors.Wrapf(ai.ErrSTTNotSupported,
"Gemini does not provide a dedicated STT endpoint; use audiollm.NewModel instead")
default:
return nil, errors.Wrapf(ai.ErrCapabilityUnsupported, "provider type %q", cfg.Type)
}
}
```
```go
// internal/ai/audiollm/factory.go
package audiollm
import (
"github.com/pkg/errors"
"github.com/usememos/memos/internal/ai"
"github.com/usememos/memos/internal/ai/audiollm/gemini"
)
func NewModel(cfg ai.ProviderConfig, opts ...ModelOption) (Model, error) {
switch cfg.Type {
case ai.ProviderGemini:
return gemini.New(cfg, applyOptions(opts...))
case ai.ProviderOpenAI:
// NOTE: gpt-4o-audio-preview support belongs here but is out of scope;
// see §2 (Non-Goals).
return nil, errors.Wrapf(ai.ErrAudioLLMNotSupported,
"OpenAI multimodal audio (gpt-4o-audio) is not yet implemented in this codebase")
default:
return nil, errors.Wrapf(ai.ErrCapabilityUnsupported, "provider type %q", cfg.Type)
}
}
```
### 6.3 Backend Handler Dispatch
The handler at `server/router/api/v1/ai_service.go:Transcribe` dispatches based on `provider.Type`:
```go
func (s *APIV1Service) Transcribe(ctx context.Context, request *v1pb.TranscribeRequest) (*v1pb.TranscribeResponse, error) {
// ... existing config loading, provider resolution, audio reading ...
switch provider.Type {
case ai.ProviderOpenAI:
text, err := s.transcribeViaSTT(ctx, provider, transcriptionCfg, audio, contentType)
if err != nil {
return nil, status.Errorf(codes.Internal, "failed to transcribe: %v", err)
}
return &v1pb.TranscribeResponse{Text: text}, nil
case ai.ProviderGemini:
text, err := s.transcribeViaAudioLLM(ctx, provider, transcriptionCfg, audio, contentType)
if err != nil {
return nil, status.Errorf(codes.Internal, "failed to transcribe: %v", err)
}
return &v1pb.TranscribeResponse{Text: text}, nil
default:
return nil, status.Errorf(codes.FailedPrecondition,
"provider type %q is not supported for transcription", provider.Type)
}
}
func (s *APIV1Service) transcribeViaSTT(ctx context.Context, provider ai.ProviderConfig,
cfg *storepb.TranscriptionConfig,
audio io.Reader, contentType string) (string, error) {
t, err := stt.NewTranscriber(provider)
if err != nil { return "", err }
resp, err := t.Transcribe(ctx, stt.Request{
Audio: audio,
Filename: "audio",
ContentType: contentType,
Model: resolveModel(provider, cfg.Model),
Prompt: cfg.Prompt, // Whisper: soft hint, may be ignored
Language: cfg.Language,
})
if err != nil { return "", err }
return resp.Text, nil
}
func (s *APIV1Service) transcribeViaAudioLLM(ctx context.Context, provider ai.ProviderConfig,
cfg *storepb.TranscriptionConfig,
audio io.Reader, contentType string) (string, error) {
m, err := audiollm.NewModel(provider)
if err != nil { return "", err }
resp, err := m.GenerateFromAudio(ctx, audiollm.Request{
Audio: audio,
ContentType: contentType,
Model: resolveModel(provider, cfg.Model),
Instructions: buildTranscriptionInstructions(cfg.Prompt, cfg.Language),
})
if err != nil { return "", err }
if resp.FinishReason != audiollm.FinishStop {
return "", errors.Errorf("transcription incomplete (finish reason: %s)", resp.FinishReason)
}
return resp.Text, nil
}
```
`buildTranscriptionInstructions` lives next to the handler and centralizes the literal instruction sent to multimodal LLMs:
```go
func buildTranscriptionInstructions(prompt, language string) string {
parts := []string{
"Transcribe the audio accurately. Return only the transcript text. " +
"Do not summarize, explain, or add content that is not spoken.",
}
if language != "" {
parts = append(parts, fmt.Sprintf("The input language is %s.", language))
}
if prompt != "" {
parts = append(parts, "Context and spelling hints:\n"+prompt)
}
return strings.Join(parts, "\n\n")
}
```
`resolveModel` returns `cfg.Model` if non-empty, else the per-provider default from `ai/models.go` (unchanged from today).
### 6.4 Implementation Notes Per Package
#### `internal/ai/stt/openai/openai.go`
- Identical wire behavior to current `internal/ai/openai.go::openAITranscriber.Transcribe`.
- Uses `github.com/openai/openai-go/v3` SDK (already a dep).
- Defaults `endpoint` to `https://api.openai.com/v1`. Trims trailing slash. Validates URL.
- Honors `cfg.Endpoint` to support OpenAI-compatible providers (Groq Whisper, faster-whisper self-hosted, Azure Whisper deployments). The user simply adds another `AIProviderConfig` row with `Type=OPENAI` and a different `Endpoint`.
- Supports any model the underlying endpoint accepts: `whisper-1`, `gpt-4o-transcribe`, `gpt-4o-mini-transcribe`, `gpt-4o-transcribe-diarize`, etc. The model string is opaque.
- Returns `Response.Language` and `Response.Segments` populated when the API returns them; otherwise empty.
#### `internal/ai/audiollm/gemini/gemini.go`
- Identical wire behavior to current `internal/ai/gemini.go::geminiTranscriber.Transcribe`, EXCEPT:
- Reads `Instructions` from the caller (not hardcoded inside the package).
- Maps `genai.FinishReason` to `audiollm.FinishReason` (`STOP→FinishStop`, `MAX_TOKENS→FinishLength`, `SAFETY→FinishSafety`, anything else → `FinishOther`).
- Returns `Response{Text, FinishReason}` instead of swallowing the finish reason into a generic error.
- Continues to use `internal/ai/audio.WebMOpusToWAV` for WebM transcoding (Gemini doesn't accept WebM).
- Continues to enforce `maxGeminiInlineAudioSize` (14 MiB) — File API support is out of scope.
- Uses `google.golang.org/genai` SDK (already a dep).
#### `internal/ai/errors.go`
Add:
```go
var ErrSTTNotSupported = errors.New("provider does not support speech-to-text capability")
var ErrAudioLLMNotSupported = errors.New("provider does not support multimodal audio capability")
```
Keep existing `ErrProviderNotFound` and `ErrCapabilityUnsupported`.
### 6.5 Proto Schema Changes
**Two comment-only updates. No field additions, no field renames, no breaking changes.**
#### Improvement #2 — `TranscriptionConfig.model` comment
Replace lines 179181 of `proto/store/instance_setting.proto`:
```proto
// model is the provider-specific model identifier.
// Empty string falls back to the engine default
// (whisper-1 for OPENAI providers, gemini-2.5-flash for GEMINI providers).
string model = 2;
```
with:
```proto
// model is the provider-specific model identifier.
// Empty string falls back to the engine default.
// OPENAI examples:
// - whisper-1 (legacy, lower cost)
// - gpt-4o-transcribe, gpt-4o-mini-transcribe (higher quality)
// - gpt-4o-transcribe-diarize (includes speaker labels)
// GEMINI examples:
// - gemini-2.5-flash (default, multimodal call)
// - gemini-2.5-pro
string model = 2;
```
**Rationale:** OpenAI's `/audio/transcriptions` endpoint now supports the `gpt-4o-transcribe` family in addition to `whisper-1`. The current comment is misleading — it implies Whisper is the only OpenAI option.
#### Improvement #3 — `TranscriptionConfig.prompt` comment
Replace lines 188191:
```proto
// prompt is a default spelling/vocabulary hint passed to the provider.
// Used as the OpenAI Whisper "prompt" parameter and folded into the Gemini
// generation prompt as a "Context and spelling hints" block.
string prompt = 4;
```
with:
```proto
// prompt is a default spelling/vocabulary hint passed to the provider.
// Used as the OpenAI Whisper "prompt" parameter (a soft hint that the model
// may ignore) and folded into the Gemini generation prompt as a "Context and
// spelling hints" block (which the LLM will treat more literally).
string prompt = 4;
```
**Rationale:** Same field, two semantically different behaviors. Surfacing this in the schema documentation (which propagates to generated Go and TypeScript via JSDoc) makes the cross-provider variability explicit for any caller reading the bindings cold.
After editing the proto, regenerate via `cd proto && buf format -w && buf generate`. The two regenerated files are:
- `proto/gen/store/instance_setting.pb.go`
- `web/src/types/proto/store/instance_setting_pb.ts`
#### Why NOT add an `enabled` field (improvement #1)
Out of scope per §2. Doing it would add a new field that the frontend, backend, and migration logic all need to handle, for the sole benefit of letting users "disable but keep the config." The current `provider_id == ""` semantics work; the cost of the change exceeds the benefit at this moment.
### 6.6 Frontend Impact
Minimal. `web/src/components/Settings/AISection.tsx` already:
- Switches the model placeholder per provider (`placeholderForProvider` at line 371, using `setting.ai.transcription-model-placeholder-gemini` / `-openai`).
- Disables the form when `providerId == ""`.
- Validates that the referenced provider exists.
Recommended adjustments (in scope):
1. **Update i18n model placeholder strings** in `web/src/locales/en.json` to reflect the new model examples:
- `setting.ai.transcription-model-placeholder-openai`: include `gpt-4o-transcribe` family alongside `whisper-1`.
- `setting.ai.transcription-model-placeholder-gemini`: confirm `gemini-2.5-flash` is the listed example.
2. **Update the prompt help text** (`setting.ai.transcription-prompt-help`) to note the cross-provider semantic difference, mirroring the new proto comment in user-facing language.
No structural component changes. No new fields. No state-shape changes.
### 6.7 What Stays Identical
- Database storage (`InstanceSetting` rows, `AISetting` blob) — proto field tags unchanged.
- API surface (`TranscribeRequest`, `TranscribeResponse` messages) — unchanged.
- gRPC/Connect endpoint paths — unchanged.
- Frontend state shape (`LocalTranscription`) — unchanged.
- All existing tests semantically unchanged (will be ported to new package paths).
## 7. Migration Path
The refactor is internal to the Go server. End-to-end behavior is preserved. Migration is staged so each stage is independently buildable, testable, and revertable.
| Stage | What | Compiles? | Tests pass? |
|---|---|---|---|
| A | Add `internal/ai/stt/` and `internal/ai/audiollm/` with new interfaces and (empty) factories. Add new errors. | ✅ | ✅ (no callers yet) |
| B | Implement `internal/ai/stt/openai/` — port behavior from current `openai.go::openAITranscriber`. Port tests to `stt/openai/openai_test.go`. | ✅ | ✅ |
| C | Implement `internal/ai/audiollm/gemini/` — port behavior from current `gemini.go::geminiTranscriber`, but: lift instructions out into the caller, return `FinishReason` instead of swallowing it. Port tests. | ✅ | ✅ |
| D | Refactor `server/router/api/v1/ai_service.go::Transcribe` to dispatch via the new factories. Add `transcribeViaSTT` and `transcribeViaAudioLLM`. Add `buildTranscriptionInstructions`. | ✅ | ✅ |
| E | Delete old files: `internal/ai/transcription.go`, `client.go`, `openai.go`, `openai_test.go`, `gemini.go`, `gemini_test.go`. | ✅ | ✅ |
| F | Update proto comments (#2 and #3), run `buf format -w && buf generate`. | ✅ | ✅ |
| G | Update `web/src/locales/en.json` strings for model placeholders and prompt help. | ✅ | ✅ |
Each stage is one commit. Reverting any single stage leaves the system in a working state.
## 8. Anti-Patterns Avoided (and Why)
| Anti-pattern | Where it would have come from | Why we're avoiding it |
|---|---|---|
| `ProviderType` enum with wire-format suffix (`OPENAI_TRANSCRIPTIONS`, `OPENAI_CHAT_AUDIO`) | Earlier brainstorming draft | Vercel/LiteLLM/Go ecosystem all use vendor-level identity; capability is implied by which interface you call. |
| `Response.Source` enum (`NativeSTT`, `MultimodalLLM`) | Earlier brainstorming draft | None of the three SDKs surveyed has this. It re-introduces the "pretend STT" smell at a different layer. |
| Adapter wrapping `audiollm.Model` as `stt.Transcriber` | Earlier brainstorming draft | Adapter would re-create the conflation we're trying to remove. Application-layer dispatch is honest. |
| Adding `transcription_*` fields to `AIProviderConfig` | Naive instinct | Three of three OSS apps surveyed (Open WebUI, LobeChat, Dify) do **not** do this. Pollutes the provider entity; repeats with every new capability. |
| Silently reusing chat provider credentials for STT (LobeChat's deprecated pattern) | LobeChat-style shortcut | LobeChat's own author marked the helper `@deprecated`. Memos's existing `provider_id` reference is more flexible (user can configure a different OpenAI-compatible endpoint, e.g. Groq, just for STT). |
| Per-model credential overrides, capability YAML, load balancing | Dify | Enterprise complexity that doesn't fit Memos's scope. |
| Auto-fallback from STT failure to multimodal-LLM transcription | Plausible "smart" idea | LiteLLM doesn't do this; failure modes and cost differ enough that fallback would surprise users. Explicit dispatch by provider type is what LiteLLM ships. |
## 9. Open Decisions
All resolved during brainstorming. None remain open. For the record:
1. **Direction A (split STT/Audio-LLM into separate interfaces) over Direction B (capability-flag system) over Direction C (audio-to-structured-note pipeline).** Resolved: A. Rationale: most honest abstraction, matches mainstream SDKs, leaves the door open to C as a future addition without rework.
2. **Provider type naming: vendor-level (`openai`/`gemini`) over wire-format-encoded.** Resolved: vendor-level. Rationale: matches Vercel/LiteLLM/Go convention; new OpenAI transcription model snapshots require zero schema or code changes.
3. **`TranscriptionConfig.Duration` field decision.** Not present in current proto; not added. Audio duration belongs to resource metadata (computed from the file at upload time), not to the transcription response.
4. **Multimodal failure-mode surface.** Resolved: expose `FinishReason` from `audiollm.Model` to the application layer; the Transcribe handler converts non-`Stop` reasons into informative errors.
## 10. Implementation Plan Pointer
Once this spec is approved, the implementation plan will be created at `docs/superpowers/plans/2026-05-02-stt-audiollm-split.md` covering Stages AG from §7 above as discrete, bite-sized tasks with TDD steps and per-stage commits.
@@ -0,0 +1,155 @@
# Transcription (STT) settings — design
**Date:** 2026-05-02
**Scope:** Backend + frontend. Schema-additive (no migration required).
## Problem
Memos has one AI feature today: audio transcription (speech-to-text). The current design has three concrete problems:
1. **Model is hard-coded per provider type.** `internal/ai/models.go` pins OpenAI to `gpt-4o-transcribe` and Gemini to `gemini-2.5-flash`. Users who want `whisper-1` (cheaper, often more accurate for non-English) or third-party Whisper-compatible endpoints (Groq's `whisper-large-v3-turbo`, self-hosted whisper.cpp / Speaches via OpenAI-compatible URL) cannot configure them at all.
2. **No explicit transcription configuration.** `InstanceAISetting.providers` is a generic credentials list. The frontend (`MemoEditor/index.tsx:65`) implicitly picks "the first provider with an API key whose type is in TRANSCRIPTION_PROVIDER_TYPES." Users cannot:
- Choose which provider runs transcription when they have multiple.
- Set a default language (Whisper API supports it but it is never sent).
- Set a `prompt` hint to bias spelling of proper nouns / jargon (a documented Whisper feature, surfaced by every other STT product).
3. **Gemini fails for browser-recorded audio.** `internal/ai/gemini.go:23` does not list `audio/webm` in `geminiSupportedContentTypes`, but `MediaRecorder` in browsers defaults to `audio/webm`. So selecting a Gemini provider for in-editor recording produces a content-type error every time.
## Goal
Let the operator configure transcription explicitly: which provider, which model, default language, and a spelling-hint prompt. Make the OpenAI provider work as a universal "OpenAI-compatible" engine so Groq / self-hosted Whisper / Speaches are reachable through endpoint override.
## Non-goals
- Adding STT engines beyond OpenAI and Gemini (Azure, Deepgram, AWS Transcribe — out of scope; the schema admits them later via `AIProviderType` enum).
- Other AI features (summarization, embeddings, tag suggestion). The schema is shaped so they fit later, but none are designed here.
- Per-call provider override at recording time. Research across all surveyed products (OpenWebUI, LibreChat, Whisper Memos, Superwhisper, etc.) confirms STT engine is a global preference, not an action-time choice. We follow the same pattern.
- Server-side audio transcoding (e.g., webm → wav for Gemini). See "Gemini webm" below for the chosen mitigation.
- Multi-user or per-user override of admin defaults. Memos' STT setting is instance-scoped, like every other instance setting.
## Naming
Field and message names follow cross-platform STT conventions, not Memos-internal shorthand:
| Concept | Chosen name | Rationale |
|---|---|---|
| Config message | `TranscriptionConfig` | AssemblyAI uses this exact identifier; matches OpenAI's `CreateTranscription*` verb family and Memos' existing `Transcribe` RPC. The `STT` acronym is not used as a type name in any major STT API. |
| Provider reference | `provider_id` (string) | Plain protobuf convention for a string-ID reference (`field_id`, `user_id` style). `engine` was rejected as an OpenWebUI-only term; typed message refs are not needed since providers are addressed by string ID. |
| Model | `model` | Unanimous across OpenAI, Google v2, Deepgram, OpenWebUI, LibreChat. Not `model_id`. |
| Default language | `language` | Bare `language` is the modern convention (OpenAI, Whisper family, Deepgram, Wyoming). `language_code` is the older Google/AWS form; we accept ISO 639-1 short codes the same way OpenAI does. |
| Spelling hint | `prompt` | OpenAI's public API field name and AssemblyAI's. Whisper's internal name is `initial_prompt`, but `prompt` is what users of `audio.transcriptions.create` recognize. |
A note on the message name collision: `proto/api/v1/ai_service.proto` already declares a `TranscriptionConfig` for **per-call** prompt/language overrides. The new store-level `TranscriptionConfig` lives in package `memos.store`, so the two compile cleanly. Memos already uses parallel `api.v1.X` / `store.X` message pairs (e.g. `User`, `Memo`); this matches that pattern.
## Architecture
### Schema (additive)
`proto/store/instance_setting.proto`:
```proto
message InstanceAISetting {
repeated AIProviderConfig providers = 1; // unchanged — credential pool
TranscriptionConfig transcription = 2; // NEW — feature config
}
message TranscriptionConfig {
// References an entry in providers[].id. Empty string = transcription disabled.
string provider_id = 1;
// Free text. Empty string = engine default (whisper-1 for OPENAI, gemini-2.5-flash for GEMINI).
string model = 2;
// ISO 639-1 short code. Empty string = auto-detect.
string language = 3;
// Up to ~200 tokens. Used as the OpenAI Whisper `prompt` parameter and as
// a "Context and spelling hints:" block in the Gemini prompt.
string prompt = 4;
}
```
`proto/api/v1/ai_service.proto`:
- `TranscribeRequest.provider_id` becomes optional. When omitted, the server resolves the provider from `InstanceAISetting.transcription.provider_id`.
- `TranscribeRequest.config` (per-call `TranscriptionConfig` with `prompt` / `language`) is kept for advanced overrides but its fields, when empty, fall back to the persisted defaults from `InstanceAISetting.transcription`.
### Backend changes
1. **`internal/ai/models.go`** — `DefaultTranscriptionModel` already exists; reuse it as the fallback when `TranscriptionConfig.model` is empty. No new code, just used from a new call site.
2. **`server/router/api/v1/ai_service.go`**:
- Read `InstanceAISetting.transcription` at the start of `Transcribe`.
- Resolve `provider_id` from request → fall back to `transcription.provider_id`. If both empty, return `FailedPrecondition` with a clear "transcription not configured" message.
- Resolve `model` similarly: request override → `transcription.model` → engine default via `DefaultTranscriptionModel`.
- Merge `language` and `prompt`: per-call overrides win; otherwise fall through to persisted defaults.
3. **`internal/ai/gemini.go`** — out of scope to fix the webm content-type list here. See mitigation below.
### Frontend changes
`web/src/components/Settings/AISection.tsx` is restructured into two settings groups inside the existing `SettingSection`:
1. **AI Integrations** (renamed from "Providers" — current behavior): list of credential entries (id, title, type, endpoint, api key). No functional changes; the rename communicates that this section is just credentials.
2. **Transcription** (new): three-segment form
- **Provider** — Select dropdown listing entries from group 1 by `title`. First option is "None — transcription disabled". Disabled with a hint "Add an AI integration first ↑" when group 1 is empty.
- **Model** — text input. Placeholder updates dynamically based on the selected provider's type (`whisper-1` for OPENAI, `gemini-2.5-flash` for GEMINI). Help text below: "Free text. Use the provider's model identifier — e.g., whisper-1, gpt-4o-transcribe, whisper-large-v3-turbo."
- **Default language** — text input, ISO 639-1 placeholder, empty = auto.
- **Prompt hints** — textarea, ~200 token soft limit, help text "Improves spelling of proper nouns and jargon. Whisper limit is ~224 tokens."
`web/src/components/MemoEditor/index.tsx:65` changes:
- Replace the "first provider with apiKey in TRANSCRIPTION_PROVIDER_TYPES" lookup with this enable rule: transcribe button shows iff `aiSetting.transcription.providerId` is non-empty AND the referenced provider exists in `aiSetting.providers` AND that provider has `apiKeySet === true`.
- The editor no longer needs to know the provider object itself for the call — see service change below.
`web/src/components/MemoEditor/services/transcriptionService.ts` is simplified: it stops accepting a `provider` argument and simply omits `provider_id` from the request. The server resolves the provider, model, language, and prompt from `InstanceAISetting.transcription`. (No override path is exposed at the editor layer; advanced callers can still pass `provider_id` directly via the proto if needed in the future.)
### How "OpenAI-compatible" backends work
To use Groq, Speaches, or self-hosted whisper.cpp:
1. In **AI Integrations**, add a provider with type `OPENAI`, set `endpoint` to e.g. `https://api.groq.com/openai/v1` or `http://speaches:8000/v1`, set the API key, give it a recognizable title ("Groq", "Self-hosted Whisper").
2. In **Transcription**, select that provider and set `model` to the backend's model identifier (`whisper-large-v3-turbo`, `Systran/faster-distil-whisper-large-v3`, etc.).
This is the universal escape hatch confirmed across OpenWebUI, LibreChat, and Whisper Obsidian plugin: don't enumerate every backend — let the OpenAI engine be a transport, not a brand.
## Gemini webm mitigation
The Gemini `audio/webm` failure is a real user-blocking bug but separate from the settings redesign. Three options were considered:
- **(a) Server-side transcode** with ffmpeg. Adds a heavy runtime dep; rejected as YAGNI.
- **(b) Switch MediaRecorder format** when STT engine is Gemini. Browser support for `audio/mp4` and `audio/wav` in `MediaRecorder` is patchy across Firefox / Safari / Chrome; rejected as fragile.
- **(c) Inline hint + accept the limitation.** Selected. The Transcription section shows a small warning under the model field when the chosen provider type is `GEMINI`: "Gemini does not accept browser-recorded `audio/webm`. For in-editor recording, use an OpenAI-compatible provider."
Server-side transcoding can be revisited later as a self-contained change if Gemini demand grows.
## Validation
Server validation (`server/router/api/v1/ai_service.go`):
- `transcription.provider_id`, when set, must reference an existing entry in `providers[]`. On `UpdateInstanceSetting` for the AI key, reject with `InvalidArgument` if it doesn't.
- `transcription.model` length cap: 256 chars (covers `Systran/faster-distil-whisper-large-v3`-style names with margin).
- `transcription.language` length cap: 32 chars (existing constant `maxTranscriptionLanguageLength`).
- `transcription.prompt` length cap: 4096 chars (existing constant `maxTranscriptionPromptLength`).
Frontend validation in `AISection.tsx`:
- "Save" disabled if `transcription.providerId` is set but the referenced provider was just deleted from the integrations list (in the same unsaved edit).
- Inline warning shown (but Save still allowed) if the referenced provider exists but has `apiKeySet === false` — surfacing the broken state so the operator can fix it without blocking unrelated edits to other settings.
## Backwards compatibility
The schema change is purely additive. Existing instances with `providers` configured but no `transcription` field default to `provider_id = ""`, which means transcription is disabled until the operator visits the new Transcription section and selects a provider.
This is a small UX regression for instances that were relying on the implicit "first provider wins" behavior — they now must make a one-click selection. Acceptable trade-off because:
- It makes the choice explicit (the implicit pick was the source of confusion when users had multiple providers).
- A one-time migration that auto-fills `transcription.provider_id` with the first STT-capable provider is feasible but adds complexity for a one-line user action. Skip the migration; document the change in the release notes.
## Testing
- `internal/ai/transcription_test.go` (existing) covers the transcribe RPC. Add cases for: empty `provider_id` falls back to setting; empty `model` falls back to `DefaultTranscriptionModel`; per-call overrides win over settings.
- `server/router/api/v1/test/ai_service_test.go` (existing) covers the API service. Add cases for the validation rules above (unknown provider_id, oversized model/language/prompt).
- Frontend: manual verification via the dev server (`pnpm dev` in `web/`) — load Settings, add a provider, configure transcription, verify the home editor's record button enables/disables based on `provider_id`. No new component tests required (existing AISection has none).
## Out of scope, explicitly
- Multiple transcription configurations / per-tag or per-user routing.
- Per-call provider override exposed in the editor UI.
- Test-transcription button in settings (worth doing later; deferred to keep this scope tight).
- Glossary / vocabulary list as a separate field — folded into `prompt` for now (Joplin/Superwhisper split this; we can add later if users ask).
- TTS settings. Memos has none today and none planned.
@@ -0,0 +1,113 @@
# First Screen Lazy Heavy Dependencies Design
## Context
The auth and signup pages currently fetch JavaScript and CSS assets for features that are not used on the first screen, including Mermaid, KaTeX, Leaflet, and React Leaflet. The current routing already uses lazy route components, so the remaining problem is eager imports from shared app entry points and feature modules.
The goal is to reduce first screen load time, especially for `/auth` and `/auth/signup`, without changing memo rendering, map behavior, or authenticated workflows.
## Goals
- Prevent Mermaid and Leaflet vendor chunks from loading on auth/signup before they are needed.
- Prevent Leaflet and KaTeX CSS from loading globally at app startup.
- Preserve current behavior when users view Mermaid diagrams, math content, memo location previews, profile maps, or location pickers.
- Keep fallbacks small and consistent with existing async rendering patterns.
- Verify the production build and confirm auth/signup network requests no longer include Mermaid or Leaflet chunks during initial render.
## Non-Goals
- Rewriting the markdown rendering pipeline.
- Removing support for Mermaid, KaTeX, Leaflet, or React Leaflet.
- Optimizing every authenticated route in this change.
- Changing server behavior, route guards, or authentication flow.
## Recommended Approach
Use feature-level lazy loading for heavy optional features. Keep the app shell and auth routes free of diagram, math styling, and map dependencies. Load those dependencies from the feature boundary where the user actually needs them.
This approach has the best balance of impact and risk because it removes known eager imports while preserving existing route structure and feature internals.
## Architecture
### App Entry
`web/src/main.tsx` should no longer import:
- `leaflet/dist/leaflet.css`
- `katex/dist/katex.min.css`
These styles are feature-specific and should be loaded from the map and markdown rendering paths.
### Mermaid
`web/src/components/MemoContent/MermaidBlock.tsx` should replace the static `import mermaid from "mermaid"` with an async `import("mermaid")` inside the render effect.
The component should keep its current behavior:
- initialize Mermaid with the resolved app theme;
- render when code content or theme changes;
- show the existing error fallback when rendering fails.
The Mermaid chunk should only be requested when a memo actually renders a Mermaid code block.
### KaTeX
KaTeX CSS should load from the memo markdown rendering path instead of the app entry. Since `rehype-katex` is only useful when memo markdown is rendered, loading the stylesheet near `MemoMarkdownRenderer` keeps auth/signup free of KaTeX CSS while preserving math output styling.
This change does not need content-level math detection. Loading KaTeX CSS with memo markdown is simpler and still removes it from the first auth/signup screen.
### Leaflet Maps
Leaflet-dependent UI should be moved behind lazy component boundaries:
- `UserMemoMap` should be lazy-loaded by the user profile route or by a small wrapper component.
- `LocationPicker` should be lazy-loaded where location UI is opened or displayed.
The underlying map implementations can continue using Leaflet, React Leaflet, marker clustering, and their current helpers. The key boundary is that parent components must not statically import the map implementation if that parent can be pulled into non-map first-screen chunks.
Leaflet CSS and marker cluster CSS should load inside the lazy map implementation path, not from `main.tsx`.
### Type Imports
Any imports from `leaflet` that are used only as TypeScript types should use `import type`. Runtime construction such as `new LatLng(...)` should be avoided in parent components that are meant to stay Leaflet-free; pass plain latitude/longitude data into lazy map wrappers and construct Leaflet objects inside the lazy implementation.
## Data Flow
Auth/signup initial render:
1. App entry initializes theme, locale, providers, auth, and instance data.
2. Router loads only the auth/signup route component and shared app shell dependencies.
3. Mermaid, Leaflet, React Leaflet, marker cluster, and feature CSS are not requested.
Memo markdown render:
1. Memo content renders with the existing markdown renderer.
2. KaTeX CSS loads with the markdown rendering path.
3. If a code block language is `mermaid`, `MermaidBlock` dynamically imports Mermaid and renders the diagram.
Map render:
1. A map feature mounts through a lazy boundary.
2. The lazy implementation imports Leaflet, React Leaflet, and required map CSS.
3. Existing map interactions and display behavior continue inside the loaded implementation.
## Error Handling
- Mermaid import or render failures should use the existing Mermaid error UI with the original code content visible.
- Lazy map boundaries should use minimal fallbacks sized like the eventual map container to avoid layout shift.
- Chunk load failures should continue to use the existing router chunk reload behavior where applicable.
## Testing
- Run `pnpm build` in `web`.
- Run `pnpm lint` in `web`.
- Inspect production build output to ensure Mermaid and Leaflet remain split chunks.
- Use a production preview or equivalent browser check for `/auth/signup` and confirm initial network requests do not include Mermaid or Leaflet JavaScript chunks.
- Smoke test memo content with Mermaid and math.
- Smoke test location picker, location popover, and user profile map.
## Risks
- Loading CSS from lazy paths can cause a small style delay the first time a map or math content appears. Use map-sized fallbacks and keep CSS imports in the feature implementation to minimize visible shifts.
- Moving Leaflet runtime types out of parent components may require small prop shape changes.
- Dynamic Mermaid import needs effect cancellation to avoid setting state after unmount.
@@ -0,0 +1,282 @@
# Placeholder Component Design
## Context
The web frontend currently renders empty states with a single `Empty.tsx` component that displays a lucide `BirdIcon`. It is used in exactly one place (`web/src/pages/Inboxes.tsx`) and offers no support for other state varieties — loading, no-search-results, or 404 pages — each of which is currently handled inconsistently or not at all.
The goal is to replace `Empty.tsx` with a single reusable `<Placeholder>` component that renders a hand-curated ASCII bird illustration plus a short muted message, supports four distinct variants, and ships with the architectural seam needed to grow a randomized pool of ASCII pieces per variant in future work.
The visual register is "cozy and minimal": muted colors, monospace throughout, gentle motion that respects `prefers-reduced-motion`. The ASCII art itself is drawn from Joan Stark's (jgs) classic ASCII bird collection (https://github.com/oldcompcz/jgs), preserving the `jgs` signature and date as a visible credit beneath each piece.
## Goals
- Provide a single `<Placeholder variant="…">` component covering empty, loading, no-results, and 404 states.
- Render real, recognizable ASCII bird art (not freehand sketches), preserving Joan Stark's attribution.
- Apply subtle CSS-only animation (bob for perched birds, flutter for in-flight birds, fade-in on the message).
- Respect `prefers-reduced-motion` — no animation when the user opts out.
- Provide a pool-shaped data file so future PRs can drop in additional ASCII pieces per variant without component changes.
- Replace the existing `Empty.tsx` in `Inboxes.tsx` as the proof of integration.
## Non-Goals
- Adding additional ASCII pieces beyond the four initial picks (one per variant). The pool architecture supports growth, but seeding more pieces is a follow-up.
- Wiring the component into search results, the 404 route, or Suspense fallbacks. Each is a candidate; each is a separate PR.
- Translating the default messages. The file structure leaves a seam for `i18next`, but the initial PR ships plain strings.
- Adding a JS animation library (Framer Motion, react-spring, etc.). Three keyframe animations do not justify a new dependency.
- Visual regression testing infrastructure.
## Recommended Approach
Build a single `<Placeholder>` component with a `variant` prop that selects a randomized ASCII piece from a co-located pool data file. Animation is CSS-only, with motion presets keyed off each piece's `motion` field. Accessibility uses `aria-hidden` on the decorative ASCII and a semantic `<p>` for the message.
This approach has the best balance of present-day simplicity and future extensibility: the initial pool contains one piece per variant, so the component is deterministic today, but the picker function and pool shape impose no constraints on how many pieces are added later.
## Architecture
### Public Component
**Path:** `web/src/components/Placeholder/index.tsx`
```tsx
import { useMemo } from "react";
import clsx from "clsx";
import { pickPiece, MotionStyle, PlaceholderVariant } from "./ascii-pool";
import { DEFAULT_MESSAGES } from "./messages";
import "./Placeholder.css";
interface PlaceholderProps {
variant: PlaceholderVariant;
message?: string;
children?: React.ReactNode;
className?: string;
}
const MOTION_CLASS: Record<MotionStyle, string> = {
bob: "placeholder-motion-bob",
flutter: "placeholder-motion-flutter",
none: "",
};
export function Placeholder({ variant, message, children, className }: PlaceholderProps) {
const piece = useMemo(() => pickPiece(variant), [variant]);
const resolvedMessage = message ?? DEFAULT_MESSAGES[variant];
const isLoading = variant === "loading";
return (
<div
role={isLoading ? "status" : undefined}
aria-live={isLoading ? "polite" : undefined}
className={clsx("flex flex-col items-center justify-center max-w-md mx-auto px-4 py-8", className)}
>
<pre
aria-hidden="true"
className={clsx(
"font-mono text-xs sm:text-sm leading-tight text-muted-foreground whitespace-pre",
MOTION_CLASS[piece.motion],
)}
>
{piece.ascii}
</pre>
<p className="mt-3 font-mono text-sm text-muted-foreground placeholder-fade-in">
{resolvedMessage}
</p>
<p className="mt-1 font-mono text-[10px] text-muted-foreground/60">{piece.credit}</p>
{children && <div className="mt-4">{children}</div>}
</div>
);
}
```
The `useMemo` keyed on `variant` ensures the piece is stable for the mount but re-rolls if the variant prop changes (rare in practice). Re-mounting re-rolls naturally.
### ASCII Pool
**Path:** `web/src/components/Placeholder/ascii-pool.ts`
```ts
export type PlaceholderVariant = "empty" | "loading" | "noResults" | "notFound";
export type MotionStyle = "bob" | "flutter" | "none";
export interface AsciiPiece {
id: string;
variant: PlaceholderVariant;
ascii: string;
credit: string;
motion: MotionStyle;
}
export const ASCII_POOL: AsciiPiece[] = [
{
id: "jgs-crested-parrot",
variant: "empty",
credit: "jgs · 4/97",
motion: "bob",
ascii: ` .---.
/ 6_6
\\_ (__\\
// \\\\
(( ))
=====""===""=====
|||
|`,
},
{
id: "jgs-hummingbird-sm",
variant: "loading",
credit: "jgs · 7/98",
motion: "flutter",
ascii: ` , _
{ \\/\`o;====-
.----'-/\`-/
\`'-..-| /
/\\/\\
\`--\``,
},
{
id: "jgs-wide-eyed-owl",
variant: "noResults",
credit: "jgs · 2/01",
motion: "bob",
ascii: ` __ __
\\ \`-'"'-\` /
/ \\_ _/ \\
| d\\_/b |
.'\\ V /'.
/ '-...-' \\
| / \\ |
\\/\\ /\\/
==(||)---(||)==`,
},
{
id: "jgs-bird-flown-away",
variant: "notFound",
credit: "jgs · 7/96",
motion: "flutter",
ascii: ` ___
_,-' ______
.' .-' ____7
/ / ___7
_| / ___7
>(')\\ | ___7
\\\\/ \\_______
' _======>
\`'----\\\\\``,
},
];
export function pickPiece(variant: PlaceholderVariant): AsciiPiece {
const matches = ASCII_POOL.filter(p => p.variant === variant);
return matches[Math.floor(Math.random() * matches.length)];
}
```
> Each `ascii` value is a template literal; backslashes and backticks are escaped per JS rules. The art shown here is verbatim from the brainstorm sign-off; implementation must preserve every space.
The `ascii` field of each entry holds the verbatim ASCII string. The four initial pieces are the ones validated during brainstorming:
- **empty** — Joan Stark's "early-bird" parrot with crest, perched on a branch
- **loading** — Joan Stark's compact hummingbird (mid-flight, fluttering)
- **noResults** — Joan Stark's wide-eyed two-feathered owl
- **notFound** — Joan Stark's flying-away bird with motion trails
Each piece's `ascii` is committed as a template literal preserving exact whitespace; the file is the source of truth and is small enough (~510 lines of art per piece) to keep inline rather than splitting into per-piece files.
### Default Messages
**Path:** `web/src/components/Placeholder/messages.ts`
```ts
export const DEFAULT_MESSAGES: Record<PlaceholderVariant, string> = {
empty: "No memos yet",
loading: "Loading…",
noResults: "Nothing matches that search",
notFound: "This page flew the coop",
};
```
Plain strings for the initial PR. A future i18n pass swaps these for `t("placeholder.empty")` etc. without touching the component.
### Animation
CSS keyframes live in a small co-located stylesheet (`Placeholder.css`) imported by `index.tsx`, scoped via a `.placeholder-…` class prefix to avoid leakage. The codebase uses Tailwind v4 plus plain CSS imports elsewhere; this matches the existing pattern.
```css
@media (prefers-reduced-motion: no-preference) {
.placeholder-motion-bob { animation: placeholder-bob 3.4s ease-in-out infinite; }
.placeholder-motion-flutter { animation: placeholder-flutter 0.7s ease-in-out infinite; }
.placeholder-fade-in { animation: placeholder-fade 1s ease-out 0.3s both; opacity: 0; }
}
@keyframes placeholder-bob {
0%, 100% { transform: translateY(0); }
50% { transform: translateY(-4px); }
}
@keyframes placeholder-flutter {
0%, 100% { transform: translate(0, 0); }
50% { transform: translate(2px, -1px); }
}
@keyframes placeholder-fade {
to { opacity: 1; }
}
```
The reduced-motion guard wraps all three rules. When the user prefers reduced motion, the bird is static and the message appears without fading.
### Accessibility
- ASCII `<pre>` carries `aria-hidden="true"`; screen readers do not announce it character-by-character.
- The message is a semantic `<p>`. It is the accessible name of the placeholder for assistive tech.
- For `variant="loading"` only, the wrapper has `role="status"` and `aria-live="polite"` so the loading message is announced when the placeholder appears.
- The credit line (`jgs · 4/97`) is visible-but-small below the message. It is not aria-hidden because it is intentionally part of the visual presentation.
- The component itself is not focusable. Anything passed as `children` (e.g. a "Go home" button on 404) participates normally in tab order.
- Color contrast relies on the existing `text-muted-foreground` token, which already meets WCAG AA in the project's theme.
### File Layout
```
web/src/components/Placeholder/
index.tsx # the <Placeholder> component (public export)
Placeholder.css # keyframes + .placeholder-motion-* classes
ascii-pool.ts # AsciiPiece type, ASCII_POOL array, pickPiece()
messages.ts # DEFAULT_MESSAGES map (i18n-ready seam)
CREDITS.md # Joan Stark attribution + link to oldcompcz/jgs
```
### Integration
- **Delete:** `web/src/components/Empty.tsx`.
- **Modify:** `web/src/pages/Inboxes.tsx` — replace the `<Empty />` import and usage with `<Placeholder variant="empty" />`.
Other potential call sites (search results page, router 404 catch-all, Suspense fallbacks) are explicitly out of scope for this PR. They are noted in the PR description as follow-up opportunities.
### Credits
**Path:** `web/src/components/Placeholder/CREDITS.md`
A short Markdown file pointing to https://github.com/oldcompcz/jgs and acknowledging Joan Stark's work. This survives even if the per-piece `credit` field is ever cleaned up by a refactor.
## Testing
A small Vitest suite covers:
- Each variant renders without throwing.
- The chosen `<pre>` content contains text from the matching pool entry.
- `aria-hidden="true"` on the `<pre>`.
- `role="status"` only present when `variant="loading"`.
- A custom `message` prop overrides the default.
- The credit text is present in the DOM.
No visual regression testing is added.
## Open Questions
None. All design decisions were validated during the brainstorming session — animation register (cozy), bird selections (jgs collection), naming (`Placeholder`, no "bird" in the name), text treatment (plain muted monospace), and integration scope (replace `Empty.tsx`, do not wire other sites yet).
## References
- Joan Stark's ASCII Art Gallery (jgs collection): https://github.com/oldcompcz/jgs
- Existing component being replaced: `web/src/components/Empty.tsx`
- Existing call site: `web/src/pages/Inboxes.tsx`
- Visual brainstorm artifacts: `.superpowers/brainstorm/1991-1778593581/content/` (mascot-approach, animation-vibe, bird-shapes-v2, jgs-bird-set, text-treatment-c)
@@ -0,0 +1,82 @@
# Remove react-use Design
## Context
The frontend has a direct dependency on `react-use@17.6.0`. Current source usage is limited to six imports:
- `usePrevious` in `web/src/layouts/RootLayout.tsx`
- `useLocalStorage` in `web/src/components/MemoExplorer/TagsSection.tsx`
- `useWindowScroll` in `web/src/components/MobileHeader.tsx`
- `useToggle` in `web/src/components/TagTree.tsx`
- `useDebounce` in `web/src/components/MemoEditor/Toolbar/InsertMenu.tsx`
- `useDebounce` in `web/src/components/MemoEditor/hooks/useLinkMemo.ts`
`react-use` pulls in `js-cookie@2.2.1` transitively. A direct `js-cookie@3.0.7` dependency does not remove that vulnerable transitive copy, so the better remediation is to remove `react-use` rather than adding another copy of `js-cookie`.
## Goals
- Remove `react-use` from `web/package.json` and `web/pnpm-lock.yaml`.
- Remove all source imports from `react-use` and `react-use/lib/*`.
- Prefer built-in React hooks directly where the replacement is simple.
- Add local hooks only where reuse or browser-side-effect cleanup makes the code clearer and safer.
- Confirm `react-use` and `js-cookie` are no longer present in the frontend dependency graph.
## Non-Goals
- Do not introduce another general-purpose React hooks library.
- Do not refactor unrelated frontend state management.
- Do not change user-visible behavior of tag preferences, tag tree expansion, scroll shadow behavior, route filter clearing, memo link search, or reverse geocoding debounce.
## Approach
Use native React hooks for one-off, simple behavior:
- Replace `useToggle` in `TagTree` with `useState(false)` plus local callbacks.
- Replace `usePrevious` in `RootLayout` with local `useRef` and `useEffect` while preserving the existing previous-path comparison.
- Replace `useWindowScroll` in `MobileHeader` with local `useState` and `useEffect` for the scroll listener.
Create focused local hooks where repeated behavior or cleanup consistency matters:
- Add `useDebouncedEffect` under `web/src/hooks/` for the two debounce call sites. It schedules a callback after the configured delay and clears the timeout when dependencies change or the component unmounts.
- Add a typed `useLocalStorage` under `web/src/hooks/` for tag display settings. It reads the stored value on initialization, falls back to the provided default, writes updates to `localStorage`, and tolerates unavailable or malformed storage by using the default.
Update imports to use `@/hooks/...` for local hooks. Keep hook APIs small and shaped around current usage rather than cloning the full `react-use` API.
## Data Flow And Behavior
Tag view settings continue to persist in `localStorage` under the same keys:
- `tag-view-as-tree`
- `tag-tree-auto-expand`
Debounced effects continue to defer:
- reverse-geocoding position updates in the memo editor insert menu by 1000 ms
- memo link search requests by 300 ms
The route filter clearing logic continues to run only when navigation changes route and the URL has no `filter` parameter.
The mobile header continues to show its shadow when the window has scrolled below the top.
## Error Handling
The local storage hook catches read and write failures. On read failure or malformed JSON, it returns the default value. On write failure, it keeps React state updated so the current UI interaction still works, while avoiding a thrown render or event-handler error.
Debounced effects do not swallow errors inside the user callback. Existing call sites already handle their own asynchronous errors where needed.
## Testing And Verification
Run frontend validation after implementation:
- `pnpm install --lockfile-only` from `web/` to regenerate `pnpm-lock.yaml`
- `pnpm why react-use js-cookie` from `web/` and confirm neither dependency remains
- `pnpm lint` from `web/`
- `pnpm build` from `web/`
Manual checks should cover:
- toggling tag tree mode and auto-expand persists across reload
- opening tag tree nodes still works
- mobile header shadow appears after scrolling
- memo link search still debounces and updates results
- location reverse geocoding still waits for position changes before lookup
@@ -0,0 +1,241 @@
# OpenAPI-Driven MCP Support Design
## Context
Memos previously had an MCP server, but it was removed in commit `2a4638b3` and should not be used as the baseline for this work. GitHub issue #6022 reported that some old MCP tools returned a bare JSON array in `result.structuredContent`, which strict MCP clients reject because `structuredContent` must be an object.
The new MCP support should start from zero and use the generated OpenAPI document at `proto/gen/openapi.yaml` as the source of truth. The OpenAPI file already describes the public REST gateway operations generated from protobuf definitions, including operation IDs, descriptions, parameters, request schemas, and response schemas.
## Goals
- Add MCP support back through a new implementation that is mechanically tied to `proto/gen/openapi.yaml`.
- Expose a standard MCP Streamable HTTP endpoint at the `/mcp` path.
- Register tools only; do not add MCP resources or prompts in the first version.
- Expose a curated memo-focused toolset derived from OpenAPI operations, not every API endpoint.
- Execute MCP tool calls through the existing API contract instead of duplicating store or service logic.
- Return object-shaped `structuredContent` for every tool result.
- Keep authentication and authorization behavior aligned with the existing API.
## Non-Goals
- Reviving or adapting the removed MCP package design.
- Adding custom MCP route aliases, readonly endpoints, or toolset filtering headers.
- Exposing every operation in `proto/gen/openapi.yaml` as an MCP tool.
- Adding MCP tools for admin settings, users, identity providers, webhooks, personal access tokens, authentication, share-link management, AI transcription, or bulk deletion.
- Adding `list_tags` or `search_memos` unless matching proto/API operations are added and OpenAPI is regenerated.
- Hand-editing generated OpenAPI or generated protobuf outputs.
## Recommended Approach
Build a new `server/router/mcp` package that parses `proto/gen/openapi.yaml` at startup, selects a curated allowlist of operation IDs, and converts those selected OpenAPI operations into MCP tools. Tool calls map MCP arguments into the selected operation's path parameters, query parameters, and JSON body, then execute the corresponding `/api/v1/...` HTTP request through the existing Echo/gRPC-Gateway route in-process.
This approach keeps OpenAPI as the authoritative contract while avoiding the usability and safety problems of exposing a large API-mirrored tool surface.
## Architecture
### MCP Service
Create a new MCP service package under `server/router/mcp`. `server.NewServer` creates the existing `APIV1Service`, registers the normal file, RSS, and API routes, then registers a new MCP service against the same Echo server.
The service exposes one MCP endpoint path:
```text
/mcp
```
The implementation must support standard Streamable HTTP client messages on `POST /mcp`. It may also support `GET /mcp` and `DELETE /mcp` if the chosen MCP transport implementation requires those methods for standards-compliant streaming or session cleanup.
The first version advertises only the MCP tools capability. It does not advertise prompts or resources.
### OpenAPI Operation Registry
At startup, the MCP service loads `proto/gen/openapi.yaml` and builds an operation registry keyed by `operationId`. Each parsed operation stores:
- operation ID
- HTTP method
- OpenAPI route template
- description
- path parameters
- query parameters
- JSON request body schema
- HTTP 200 JSON response schema
The parser should fail fast during service construction if any curated operation ID is missing or cannot be converted into a valid MCP tool schema.
### Tool Names
Tool names are derived from OpenAPI `operationId` values and normalized for MCP clients. The exact naming convention should be deterministic and tested. A practical convention is lower snake case without the `Service` suffix in the subject:
```text
MemoService_ListMemos -> memo_list_memos
AttachmentService_GetAttachment -> attachment_get_attachment
```
The OpenAPI `operationId` remains stored in tool metadata so tests and future diagnostics can prove which OpenAPI operation produced each MCP tool.
### Tool Schemas
Each MCP tool input schema is an object built from:
- OpenAPI path parameters
- OpenAPI query parameters
- JSON request body fields, when present
Required path parameters stay required. Required request bodies stay required. Optional query parameters stay optional. The schema should preserve OpenAPI descriptions and primitive types where possible.
Each MCP tool output schema is the OpenAPI HTTP 200 JSON response schema. For empty 200 responses with no JSON schema, the MCP output schema is:
```json
{ "type": "object", "properties": { "ok": { "type": "boolean" } } }
```
## Tool Scope
The first version exposes these curated OpenAPI operations:
- `MemoService_ListMemos`
- `MemoService_CreateMemo`
- `MemoService_GetMemo`
- `MemoService_UpdateMemo`
- `MemoService_DeleteMemo`
- `MemoService_ListMemoComments`
- `MemoService_CreateMemoComment`
- `MemoService_ListMemoAttachments`
- `MemoService_SetMemoAttachments`
- `MemoService_ListMemoReactions`
- `MemoService_UpsertMemoReaction`
- `MemoService_DeleteMemoReaction`
- `MemoService_ListMemoRelations`
- `MemoService_SetMemoRelations`
- `AttachmentService_ListAttachments`
- `AttachmentService_GetAttachment`
- `AttachmentService_DeleteAttachment`
Excluded in the first version:
- auth sign-in, sign-out, and refresh
- user management
- personal access token management
- identity provider management
- webhooks
- instance settings
- share-link management
- AI transcription
- bulk delete operations
- any operation not present in generated OpenAPI
## Data Flow
### Startup
1. `server.NewServer` creates the existing `APIV1Service`.
2. The new `MCPService` loads `proto/gen/openapi.yaml`.
3. The OpenAPI parser builds the operation registry.
4. The curated operation allowlist selects supported operations.
5. Each selected operation is registered as an MCP tool with input schema, output schema, description, annotations, and operation metadata.
### Tool Call
1. The client sends a standard MCP `tools/call` request to `/mcp`.
2. The MCP server validates the tool name and arguments against the OpenAPI-derived input schema.
3. The adapter substitutes path parameters into the OpenAPI route template.
4. The adapter encodes query parameters into the query string.
5. The adapter marshals request body arguments as JSON when the operation has a request body.
6. The adapter forwards the caller's `Authorization` header and executes the matching `/api/v1/...` request through the existing Echo handler in-process.
7. The API response JSON is decoded into `map[string]any`.
8. The MCP result returns a compact JSON text fallback plus object-shaped `structuredContent`.
## Result Shape
Every MCP result must use object-shaped `structuredContent`.
Rules:
- If the API response is a JSON object, return it unchanged.
- If the API response is empty, return `{ "ok": true }`.
- If an unexpected raw JSON array appears, wrap it as `{ "result": [...] }`.
- If an unexpected scalar appears, wrap it as `{ "result": value }`.
This directly addresses issue #6022 by preventing collection tools from returning bare arrays.
## Authentication And Origin Safety
The MCP endpoint accepts the same bearer credentials as the API:
```text
Authorization: Bearer <PAT-or-access-token>
```
The MCP adapter forwards the bearer header to the in-process API request. Public API operations can work without authentication when the API allows them. Mutating operations require authentication because the API already enforces that behavior.
For browser-origin safety, `/mcp` rejects cross-origin browser requests unless the `Origin` header is same-origin or matches the configured instance URL. Requests without an `Origin` header are allowed because desktop MCP clients commonly omit it.
## Errors
The MCP server should convert failures into MCP tool errors:
- Invalid MCP arguments: concise validation message.
- API `401` or `403`: preserve the API message where available.
- API `404`: report that the resource was not found.
- Other API errors: include the HTTP status code and decoded API message.
- Internal OpenAPI or adapter errors: log server-side details and return a concise tool error.
Adapter errors should not bypass MCP result formatting unless the underlying MCP framework requires protocol-level errors for invalid protocol messages.
## Tool Annotations
Tool annotations are derived from HTTP methods:
- `GET`: read-only, non-destructive, idempotent.
- `POST`: mutating unless the operation is explicitly known to be read-only.
- `PATCH`: mutating, non-idempotent by default.
- `DELETE`: destructive and idempotent by default.
These annotations are hints for clients and do not replace API authorization.
## Testing
### OpenAPI Parsing Tests
Tests should verify:
- every curated operation ID exists in `proto/gen/openapi.yaml`
- every selected tool has an object input schema
- every selected tool has an object output schema
- selected operations do not include admin, auth, webhook, identity provider, personal access token, instance setting, share-link, AI transcription, or bulk-delete operations
- tool names are deterministic and unique
### MCP Protocol Tests
Using an Echo test server and JSON-RPC requests, tests should verify:
- `initialize` succeeds
- `tools/list` returns only curated OpenAPI-derived tools
- tool definitions include input and output schemas
- no prompts or resources capabilities are advertised
- collection tool calls return object-shaped `structuredContent`, never a bare array
### Adapter Tests
Representative adapter tests should cover:
- `GET /api/v1/memos?pageSize=...`
- `POST /api/v1/memos`
- `GET /api/v1/memos/{memo}`
- `PATCH /api/v1/memos/{memo}`
- `DELETE /api/v1/memos/{memo}`
- `GET /api/v1/memos/{memo}/comments`
- `POST /api/v1/memos/{memo}/reactions`
Before finishing implementation, run:
```bash
go test ./server/router/mcp/... ./server/router/api/v1/... ./server/...
```
## Implementation Notes
- Do not hand-edit `proto/gen/openapi.yaml`.
- If a needed MCP tool is not represented by OpenAPI, add or adjust the proto/API surface first, run `cd proto && buf generate`, then derive the MCP tool from the regenerated OpenAPI.
- Prefer existing API auth and gateway behavior over duplicating authorization logic in the MCP adapter.
- Keep the first version intentionally small. Additional OpenAPI-derived tools can be added by extending the curated allowlist and tests.
@@ -0,0 +1,168 @@
# Markdown WYSIWYG Editor — Design
**Date:** 2026-06-11
**Status:** Approved pending user review
## Goal
Replace the plain `<textarea>` memo editor with a modern WYSIWYG editor that renders
markdown syntax live as the user types (Linear / Claude.ai composer style) while keeping
markdown — not HTML — as the only storage and API format. A toggle lets users drop back
to the raw textarea editor at any time.
## Background
The current editor (`web/src/components/MemoEditor/Editor/index.tsx`) is a textarea with
hand-rolled features layered on top: tag suggestions (`#`), slash commands (`/`),
Ctrl+B/I markdown shortcuts, list auto-continuation, URL-paste-to-link, IME composition
plumbing, auto-grow height, and a toolbar that inserts raw markdown strings through an
imperative, character-offset-based `EditorRefActions` API.
Live markdown editing is among the most-requested memos features
(usememos/memos#2216, #3495, #766, #4259).
### Prior art (researched 2026-06)
- **Linear** builds its editor directly on **ProseMirror** (+ y-prosemirror for collab).
- **ChatGPT's composer** is raw **ProseMirror**.
- **Claude.ai's composer** is **Tiptap** (headless framework on ProseMirror).
All three references are ProseMirror-family WYSIWYG editors with markdown input rules.
### Library options considered
| Option | Verdict |
|---|---|
| **Tiptap v3 + `@tiptap/markdown`** | **Chosen.** Claude.ai's stack. Best ecosystem; StarterKit covers the core set with input rules; Suggestion utility replaces hand-rolled tag/slash popups; official bidirectional markdown (early release — gated by a spike). First-class React bindings; best-in-class IME handling via ProseMirror. ~100 KB gzipped. |
| Milkdown | Markdown-first ProseMirror + remark; strongest fidelity by design, but thin UX building blocks, small community, low bus factor. |
| Raw ProseMirror + prosemirror-markdown | Linear's literal approach; maximum control, most engineering effort for the least product difference. |
| Lexical | Non-ProseMirror engine; markdown is a conversion target, historically weaker IME. |
| CodeMirror 6 live-preview (Obsidian style) | Perfect byte fidelity but a different UX than the named references; largely DIY. |
## Decisions (user-confirmed)
1. **Raw mode stays**: a per-editor toggle switches between WYSIWYG and the current
textarea editor. (Revised from an earlier "full replacement" decision.)
2. **Fidelity contract**: lossless for supported syntax — semantic round-trip for
everything the editor models; style normalization (e.g. `*``-` bullets) is
acceptable. Unknown/exotic syntax is preserved verbatim, never dropped.
3. **v1 scope — Linear-style core set** rendered live: bold/italic/strike/inline-code,
headings, ordered/unordered/task lists, links, blockquotes, code blocks, and memos
`#tags`. Tables, `$…$`/`$$…$$` math, and inline HTML remain literal text in the
editor (they still render richly in the memo view after save). Mermaid lives in
` ```mermaid ` fences, so it is just a code block while editing.
## Architecture
### Editor abstraction
`EditorContent` hosts one of two implementations behind a shared **`EditorController`**
interface:
- `focus()`, `getMarkdown()`, `setMarkdown()`, `insertMarkdown()`, `isEmpty()`
- Formatting **intents**: `toggleBold()`, `toggleItalic()`, `toggleTaskList()`, etc.
The **Tiptap editor** implements intents as ProseMirror commands. The **textarea editor**
implements them exactly as today (wrap selection in `**`). The toolbar dispatches intents
and never knows which editor is mounted. The textarea's existing string-surgery API
(`EditorRefActions`) becomes internal to the textarea implementation rather than the
public contract.
### Mode toggle
- A small icon button in the editor toolbar switches modes mid-edit.
- Switching is a markdown string handoff: WYSIWYG → raw serializes (the same path as
save, so no new fidelity risk); raw → WYSIWYG parses.
- The preference persists in localStorage (per device). No backend/proto changes; can
graduate to a server-side user setting later.
### Tiptap editor composition
Built on `@tiptap/react` `useEditor`. Extensions:
- **StarterKit** (configured): headings, bold/italic/strike/code, lists, blockquote,
code block — all with live input rules (`**bold**` converts as you type).
- **TaskList / TaskItem**, **Link** (includes URL-paste-over-selection → link),
**Placeholder** (i18n'd), **`@tiptap/markdown`**.
- **`Tag`** (custom): inline node for `#tag`, rendered as a styled token, serialized
back to `#tag` verbatim. `#` triggers a Suggestion-plugin popup backed by the
existing tag store.
- **`SlashCommand`** (custom): Suggestion-plugin popup on `/`, replacing
`SlashCommands.tsx` in WYSIWYG mode.
- **`PreservedBlock`** (custom): the fidelity workhorse — unmodeled constructs
(tables, math, inline HTML, unrecognized syntax) are captured at parse time into
nodes carrying their raw source, displayed as literal text (subtle mono styling),
and re-emitted byte-for-byte on serialize.
Hand-rolled textarea features that become native or config in WYSIWYG mode:
Ctrl+B/I keymaps (StarterKit), list auto-continuation (ProseMirror lists),
auto-grow height (contenteditable), IME composition (ProseMirror — better CJK
behavior than the textarea plumbing).
### Data flow
Unchanged at the boundaries. Memo markdown → `setMarkdown()` on load. On update,
serialized markdown (debounced) flows into the existing state reducer as
`state.content`, so auto-save, the localStorage draft cache (still a plain markdown
string), tag extraction, and the save path are untouched. The backend only ever sees
markdown. File paste/drag wires Tiptap `handlePaste`/`handleDrop` into the existing
`uploadService`.
## Fidelity contract
1. **Supported constructs** round-trip semantically; list-marker style may normalize;
content never changes meaning.
2. **Unmodeled constructs** round-trip byte-for-byte via `PreservedBlock`.
3. **A round-trip corpus test enforces this permanently** (see Testing). It is written
first, as a spike, and is the go/no-go gate on `@tiptap/markdown` (early release).
If the gate fails and custom Marked tokenizers cannot fix it, the fallback is
swapping only the parse/serialize layer to remark (already a memos dependency)
while keeping the Tiptap editor — editor and serialization are deliberately
decoupled for this reason.
## Error handling
- **Load guard (tripwire):** on opening an existing memo, parse → re-serialize →
compare semantically. If the round-trip would lose content, log it and show a
notice — "this memo contains syntax the editor can't safely edit" — with a
one-click switch to raw mode for that memo. Expected never to fire.
- **Draft safety:** the localStorage draft cache stays a markdown string on the same
debounce as today; drafts are interchangeable between both editor modes.
- **Save path:** reuses already-serialized `state.content`; save never triggers a
fresh parse, so it gains no new failure mode.
## Testing
- **Round-trip corpus test (the spike, written first):** fixture markdown files
(GFM, tables, math, mermaid, inline HTML, nested lists, CJK, emoji) → parse →
serialize → assert semantic equality for supported syntax, byte equality for
preserved blocks. Runs in the existing vitest/jsdom setup.
- **Extension unit tests:** `Tag` (parse/serialize/suggestion trigger),
`PreservedBlock` (verbatim round-trip), link-paste.
- **Component tests** (@testing-library): toolbar intents produce expected markdown
in both modes; slash/tag popups open and insert correctly; mode toggle hands
content across without loss.
- **Manual QA:** CJK IME composition, mobile Safari/Chrome soft keyboards, focus
mode, paste-image upload.
## Rollout
Feature branch, PR series:
1. **Spike:** corpus test + `@tiptap/markdown` verdict (throwaway if the gate fails).
2. **Core editor:** Tiptap behind `EditorController`, StarterKit set, markdown in/out,
wired into the state layer.
3. **Memos features:** `Tag` + suggestions, `SlashCommand`, `PreservedBlock`,
file paste/drop, toolbar intent conversion.
4. **Toggle + refactor:** mode toggle UI and persistence; textarea editor refactored
to implement `EditorController` (nothing deleted — it powers raw mode).
Dependency cost: ~100 KB gzipped (`@tiptap/core`, `@tiptap/react`, extensions,
`@tiptap/markdown` + MarkedJS). WYSIWYG is the default mode for all users.
## Out of scope (v1)
- Interactive table editing, live KaTeX/mermaid rendering while editing.
- Collaborative editing (Yjs).
- Server-side persistence of the mode preference.
- Mobile apps (web/PWA only — native apps are separate codebases).
@@ -0,0 +1,257 @@
# Design: Expand the CEL filter surface
- **Date:** 2026-06-15
- **Status:** Approved (design); ready for implementation planning
- **Area:** `internal/filter` (memo & attachment filter engine)
- **Follow-up spec:** CEL engine hardening + native-AST migration (separate, sequenced after this)
## Summary
memos lets API clients pass a CEL expression in the `filter` field of list
requests. The `internal/filter` engine uses `cel-go` purely as a **parse +
type-check frontend**, then walks the AST and translates it into a SQL `WHERE`
fragment for the active dialect (SQLite / MySQL / Postgres). cel-go never
evaluates anything.
This spec adds three new CEL constructs that users can write, each with a SQL
translation across all three dialects:
1. `startsWith()` / `endsWith()` on scalar string fields (case-insensitive).
2. `all()` comprehension on tag lists (matches only non-empty tag sets).
3. `matches(regex)` on string fields.
## Goals
- Expose the three constructs above through the existing
parse → IR → render pipeline.
- Keep parity across SQLite, MySQL, and Postgres, with golden tests for each.
- Preserve the engine's invariant: only schema-declared fields and explicitly
supported operations are accepted; everything else is rejected with a clear
error.
## Non-goals
- **Value-producing CEL features with no SQL form** are explicitly out of scope:
optional types (`?.`, `optional.of`), `map()` / `filter()` transforms, the
math extension, string-manipulation extensions (`replace`, `split`,
`substring`, `format`), and two-variable comprehensions. There is nothing to
push into a `WHERE` clause for these.
- **`lowerAscii()` / `upperAscii()`** — dropped. `contains()` is already
case-insensitive on all dialects, and the new `startsWith`/`endsWith` are
case-insensitive too (see decisions), so explicit case-folding adds little.
Revisit only if users ask.
- **Parser hardening and the native-AST proto migration** are a separate
follow-up spec. The one exception that rides along here is
`cel.ValidateRegexLiterals()`, which feature ③ requires for safety.
## Background: how the engine works today
Pipeline (see `internal/filter/README.md`):
1. **Parse**`env.Compile(filter)` parses and type-checks against the
memo/attachment environment declared in `schema.go`; the AST is converted via
`cel.AstToParsedExpr()`.
2. **Normalize**`parser.go` walks the CEL `Expr` and builds a
dialect-agnostic IR (`ir.go`): logical ops, comparisons, `IN`, `contains()`,
and `exists()` comprehensions over tag lists.
3. **Render**`render.go` walks the IR and emits dialect-specific SQL plus
placeholder args.
Two existing facts that shaped this design:
- **`contains()` is already case-insensitive** on all three dialects
(`render.go` `renderContainsCondition`): SQLite uses the custom
`memos_unicode_lower` function, Postgres uses `ILIKE`, MySQL relies on its
default case-insensitive collation.
- **Custom SQLite scalar functions are already registered**
(`store/db/sqlite/functions.go`, `ensureUnicodeLowerRegistered` via
`modernc.org/sqlite`'s `RegisterScalarFunction`, invoked from
`store/db/sqlite/sqlite.go`). The new `REGEXP` function follows this exact
pattern.
cel-go version: `v0.28.0` (latest is `v0.28.1`, a patch with nothing relevant to
memos). No version bump is required for this work.
## Resolved decisions
| # | Decision | Choice |
|---|----------|--------|
| A | Case-sensitivity of new scalar `startsWith`/`endsWith` | **Case-insensitive**, consistent with existing `contains()`. `==` stays case-sensitive (exact match). |
| B | `all()` over a memo with zero tags | **Require non-empty**: an untagged memo does NOT match an `all()` filter. (Diverges from strict CEL vacuous-truth, but matches search-box intuition.) |
| C | Keep `lowerAscii()` / `upperAscii()`? | **Drop** from this spec. |
## Detailed design
### ① `startsWith()` / `endsWith()` on scalar string fields
**Surface.** Allow `field.startsWith("x")` and `field.endsWith("x")` as
top-level boolean calls for scalar string fields. Today these functions are only
recognized *inside* tag comprehensions (`parser.go` `extractPredicate`).
Applicable fields: memo `content`; attachment `filename`, `mime_type`.
`creator` is intentionally **excluded**: it is an identity field with `==`/`!=`
semantics whose column is wrapped as `'users/' || username`, so prefix/suffix
matching there would match against the `users/` prefix and surprise users.
**Schema.** Generalize the per-field text-matching capability. Today `Field` has
`SupportsContains bool`. Replace/extend with a capability that also covers
prefix/suffix matching (e.g. a `SupportsTextMatch bool`, or reuse
`SupportsContains` to gate all three LIKE-based ops). Fields that already set
`SupportsContains: true` gain prefix/suffix support.
**Parser.** In `buildCallCondition`, recognize `startsWith` / `endsWith` calls
whose target is a scalar string field and whose single argument is a string
literal. Reject non-literal arguments and fields without the capability.
**IR.** Generalize `ContainsCondition` into a single node:
```go
type TextMatchMode string
const (
TextMatchContains TextMatchMode = "contains"
TextMatchPrefix TextMatchMode = "prefix"
TextMatchSuffix TextMatchMode = "suffix"
)
type TextMatchCondition struct {
Field string
Mode TextMatchMode
Value string
}
```
`contains()` migrates to `TextMatchCondition{Mode: TextMatchContains}`.
**Render.** Build a `LIKE` pattern from the (escaped) literal:
- prefix → `value%`
- suffix → `%value`
- contains → `%value%`
Reuse the existing case-insensitive rendering already used by `contains()`:
SQLite `memos_unicode_lower(col) LIKE memos_unicode_lower(?)`, Postgres
`col ILIKE $n`, MySQL `col LIKE ?`.
**LIKE-escaping fix.** The current `contains()` renderer interpolates the raw
value into the pattern without escaping `%`, `_`, or `\`. This means a search
for `50%` behaves as a wildcard. The new shared path will escape these
metacharacters (and emit `ESCAPE '\'` where required by the dialect). This
closes a small latent wildcard-injection inconsistency and applies uniformly to
contains/prefix/suffix.
### ② `all()` comprehension on tag lists
**Surface.** Allow `tags.all(t, <pred>)` where `<pred>` is one of the predicates
already supported for `exists()`: `t == "x"`, `t.startsWith("x")`,
`t.endsWith("x")`, `t.contains("x")`.
**Parser.** `detectComprehensionKind` currently accepts only `exists()` and
explicitly rejects `all()`. Add a `ComprehensionAll` kind (accumulator inits to
`true`, loop step uses `_&&_`). Reuse the existing predicate extraction.
**IR.** Add `ComprehensionAll` to the `ComprehensionKind` enum; the existing
`ListComprehensionCondition` already carries `Kind`.
**Render — proper per-element semantics.** The existing `exists()`
implementation matches the *serialized* JSON array text with `LIKE`, which works
for "at least one element matches a substring" but **cannot** express "every
element matches." `all()` therefore needs real per-element iteration. Decision B
(require non-empty) means: array is non-empty **AND** no element fails the
predicate.
- **SQLite:**
```sql
(<array> IS NOT NULL AND <array> != '[]'
AND NOT EXISTS (SELECT 1 FROM json_each(<array>)
WHERE NOT (<predicate on json_each.value>)))
```
- **Postgres:**
```sql
(<array> IS NOT NULL AND jsonb_array_length(<array>) > 0
AND NOT EXISTS (SELECT 1 FROM jsonb_array_elements_text(<array>) AS e(value)
WHERE NOT (<predicate on e.value>)))
```
- **MySQL:**
```sql
(<array> IS NOT NULL AND JSON_LENGTH(<array>) > 0
AND NOT EXISTS (SELECT 1 FROM JSON_TABLE(<array>, '$[*]'
COLUMNS (value VARCHAR(512) PATH '$')) AS j
WHERE NOT (<predicate on j.value>)))
```
The per-element predicate reuses LIKE/`=` against the element `value`
(case-insensitive for `startsWith`/`endsWith`/`contains`, consistent with ①).
Hierarchical-tag prefix behavior should match the existing `exists()` rendering
(a prefix matches the exact tag or a `tag/...` child).
> Note: this introduces correlated subqueries against the same `memo.payload`
> column the outer query already reads; confirm the generated SQL composes with
> the surrounding `WHERE` and placeholder offsets in `helpers.AppendConditions`.
### ④ `matches(regex)` on string fields
**Surface.** Allow `field.matches("pattern")` for the same free-text fields as ①
(`content`, `filename`, `mime_type`; `creator` excluded), literal pattern only.
**Env / validation.** Add `cel.ValidateRegexLiterals()` to the env options in
`schema.go` so malformed patterns fail at compile time with a clear message
(validated against Go's RE2).
**Parser / IR.** Recognize `matches` calls; add:
```go
type RegexCondition struct {
Field string
Pattern string
}
```
Reject non-literal patterns and fields without text-match capability.
**Render.**
- **Postgres:** `col ~ $n`
- **MySQL:** `col REGEXP ?`
- **SQLite:** `col REGEXP ?`. SQLite desugars `X REGEXP Y` to the function call
`regexp(Y, X)`, so register a 2-arg scalar function named `regexp(pattern,
value)` returning 1/0, backed by Go's `regexp` package, following the
`ensureUnicodeLowerRegistered` pattern in `store/db/sqlite/functions.go`.
Compile patterns lazily with a small cache (or rely on RE2 compile per call;
decide during implementation based on measured cost).
**Documented caveats** (engine differences are inherent, not bugs):
- Regex *syntax* differs per engine: SQLite uses Go RE2; Postgres uses POSIX
ERE; MySQL 8.0+ uses ICU. Portable patterns work everywhere; engine-specific
constructs may not. Document this in `internal/filter/README.md`.
- ReDoS risk is low: RE2 (SQLite path) is linear-time; Postgres/MySQL POSIX
engines do not catastrophically backtrack. `ValidateRegexLiterals()` rejects
patterns that don't compile under RE2 as a first-line guard.
## Testing strategy
For each feature, add golden tests in
`store/db/{sqlite,mysql,postgres}/memo_filter_test.go` (and the attachment
filter tests where applicable):
- **Happy path:** assert the exact SQL fragment and args per dialect.
- **Error paths:** non-literal argument, unsupported field, malformed regex,
unsupported predicate inside `all()`.
- **`all()` empty-set:** confirm an untagged memo does not match (decision B).
- **LIKE escaping:** confirm `%`, `_`, `\` in `contains`/`startsWith`/`endsWith`
values are treated literally.
Run `go test ./...` (engine unit tests plus all three dialect suites). The
`contains()``TextMatchCondition` refactor must keep existing golden outputs
unchanged except for the intentional escaping fix.
## Rollout / sequencing
This is the first of two specs. The second (already agreed) covers engine
hardening: tightened parser limits (`ParserExpressionSizeLimit`,
`ParserRecursionLimit`, `ParserErrorRecoveryLimit`), the
`ValidateComprehensionNestingLimit` / `ValidateHomogeneousAggregateLiterals`
validators, and migrating `parser.go` off the deprecated
`genproto/.../expr/v1alpha1` proto to the native `common/ast` API. Building this
surface-expansion spec first is acceptable; the hardening migration is a pure
refactor that the golden tests written here will help protect.
+154
View File
@@ -0,0 +1,154 @@
module github.com/usememos/memos
go 1.26.2
require (
connectrpc.com/connect v1.19.2
github.com/at-wat/ebml-go v0.18.0
github.com/aws/aws-sdk-go-v2 v1.41.6
github.com/aws/aws-sdk-go-v2/config v1.32.16
github.com/aws/aws-sdk-go-v2/credentials v1.19.15
github.com/aws/aws-sdk-go-v2/service/s3 v1.100.0
github.com/go-sql-driver/mysql v1.9.3
github.com/google/cel-go v0.28.1
github.com/google/jsonschema-go v0.4.3
github.com/google/uuid v1.6.0
github.com/gorilla/feeds v1.2.0
github.com/grpc-ecosystem/grpc-gateway/v2 v2.29.0
github.com/joho/godotenv v1.5.1
github.com/labstack/echo/v5 v5.1.0
github.com/lib/pq v1.12.3
github.com/lithammer/shortuuid/v4 v4.2.0
github.com/moby/moby/api v1.54.2
github.com/modelcontextprotocol/go-sdk v1.6.1
github.com/openai/openai-go/v3 v3.32.0
github.com/pion/opus v0.0.0-20260430223319-81a9c5dc5013
github.com/pkg/errors v0.9.1
github.com/spf13/cobra v1.10.2
github.com/spf13/viper v1.21.0
github.com/stretchr/testify v1.11.1
github.com/testcontainers/testcontainers-go v0.42.0
github.com/testcontainers/testcontainers-go/modules/mysql v0.42.0
github.com/testcontainers/testcontainers-go/modules/postgres v0.42.0
github.com/yuin/goldmark v1.8.2
golang.org/x/crypto v0.50.0
golang.org/x/mod v0.35.0
golang.org/x/net v0.53.0
golang.org/x/oauth2 v0.36.0
golang.org/x/sync v0.20.0
google.golang.org/genai v1.54.0
google.golang.org/genproto v0.0.0-20260427160629-7cedc36a6bc4
google.golang.org/genproto/googleapis/api v0.0.0-20260427160629-7cedc36a6bc4
google.golang.org/grpc v1.80.0
modernc.org/sqlite v1.50.0
)
require (
cel.dev/expr v0.25.1 // indirect
cloud.google.com/go v0.123.0 // indirect
cloud.google.com/go/auth v0.20.0 // indirect
cloud.google.com/go/compute/metadata v0.9.0 // indirect
dario.cat/mergo v1.0.2 // indirect
filippo.io/edwards25519 v1.2.0 // indirect
github.com/Azure/go-ansiterm v0.0.0-20250102033503-faa5f7b0171c // indirect
github.com/Microsoft/go-winio v0.6.2 // indirect
github.com/antlr4-go/antlr/v4 v4.13.1 // indirect
github.com/aws/aws-sdk-go-v2/service/signin v1.0.10 // indirect
github.com/cenkalti/backoff/v4 v4.3.0 // indirect
github.com/cespare/xxhash/v2 v2.3.0 // indirect
github.com/containerd/errdefs v1.0.0 // indirect
github.com/containerd/errdefs/pkg v0.3.0 // indirect
github.com/containerd/log v0.1.0 // indirect
github.com/containerd/platforms v0.2.1 // indirect
github.com/cpuguy83/dockercfg v0.3.2 // indirect
github.com/distribution/reference v0.6.0 // indirect
github.com/docker/go-connections v0.7.0 // indirect
github.com/docker/go-units v0.5.0 // indirect
github.com/dustin/go-humanize v1.0.1 // indirect
github.com/ebitengine/purego v0.10.0 // indirect
github.com/felixge/httpsnoop v1.0.4 // indirect
github.com/fsnotify/fsnotify v1.9.0 // indirect
github.com/go-logr/logr v1.4.3 // indirect
github.com/go-logr/stdr v1.2.2 // indirect
github.com/go-ole/go-ole v1.3.0 // indirect
github.com/go-viper/mapstructure/v2 v2.5.0 // indirect
github.com/google/go-cmp v0.7.0 // indirect
github.com/google/s2a-go v0.1.9 // indirect
github.com/googleapis/enterprise-certificate-proxy v0.3.15 // indirect
github.com/googleapis/gax-go/v2 v2.22.0 // indirect
github.com/gorilla/websocket v1.5.3 // indirect
github.com/inconshreveable/mousetrap v1.1.0 // indirect
github.com/klauspost/compress v1.18.5 // indirect
github.com/lufia/plan9stats v0.0.0-20260330125221-c963978e514e // indirect
github.com/magiconair/properties v1.8.10 // indirect
github.com/moby/docker-image-spec v1.3.1 // indirect
github.com/moby/go-archive v0.2.0 // indirect
github.com/moby/moby/client v0.4.1 // indirect
github.com/moby/patternmatcher v0.6.1 // indirect
github.com/moby/sys/sequential v0.6.0 // indirect
github.com/moby/sys/user v0.4.0 // indirect
github.com/moby/sys/userns v0.1.0 // indirect
github.com/moby/term v0.5.2 // indirect
github.com/ncruces/go-strftime v1.0.0 // indirect
github.com/opencontainers/go-digest v1.0.0 // indirect
github.com/opencontainers/image-spec v1.1.1 // indirect
github.com/pelletier/go-toml/v2 v2.3.0 // indirect
github.com/power-devops/perfstat v0.0.0-20240221224432-82ca36839d55 // indirect
github.com/remyoudompheng/bigfft v0.0.0-20230129092748-24d4a6f8daec // indirect
github.com/sagikazarmark/locafero v0.12.0 // indirect
github.com/segmentio/asm v1.1.3 // indirect
github.com/segmentio/encoding v0.5.4 // indirect
github.com/shirou/gopsutil/v4 v4.26.3 // indirect
github.com/sirupsen/logrus v1.9.4 // indirect
github.com/spf13/afero v1.15.0 // indirect
github.com/spf13/cast v1.10.0 // indirect
github.com/spf13/pflag v1.0.10 // indirect
github.com/subosito/gotenv v1.6.0 // indirect
github.com/tidwall/gjson v1.18.0 // indirect
github.com/tidwall/match v1.2.0 // indirect
github.com/tidwall/pretty v1.2.1 // indirect
github.com/tidwall/sjson v1.2.5 // indirect
github.com/tklauser/go-sysconf v0.3.16 // indirect
github.com/tklauser/numcpus v0.11.0 // indirect
github.com/yosida95/uritemplate/v3 v3.0.2 // indirect
github.com/yusufpapurcu/wmi v1.2.4 // indirect
go.opentelemetry.io/auto/sdk v1.2.1 // indirect
go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.68.0 // indirect
go.opentelemetry.io/otel v1.43.0 // indirect
go.opentelemetry.io/otel/metric v1.43.0 // indirect
go.opentelemetry.io/otel/trace v1.43.0 // indirect
go.yaml.in/yaml/v3 v3.0.4 // indirect
golang.org/x/exp v0.0.0-20260410095643-746e56fc9e2f // indirect
golang.org/x/image v0.39.0 // indirect
google.golang.org/api v0.276.0 // indirect
google.golang.org/genproto/googleapis/rpc v0.0.0-20260427160629-7cedc36a6bc4 // indirect
modernc.org/libc v1.72.1 // indirect
modernc.org/mathutil v1.7.1 // indirect
modernc.org/memory v1.11.0 // indirect
)
require (
github.com/aws/aws-sdk-go-v2/aws/protocol/eventstream v1.7.9 // indirect
github.com/aws/aws-sdk-go-v2/feature/ec2/imds v1.18.22 // indirect
github.com/aws/aws-sdk-go-v2/internal/configsources v1.4.22 // indirect
github.com/aws/aws-sdk-go-v2/internal/endpoints/v2 v2.7.22 // indirect
github.com/aws/aws-sdk-go-v2/internal/v4a v1.4.23 // indirect
github.com/aws/aws-sdk-go-v2/service/internal/accept-encoding v1.13.8 // indirect
github.com/aws/aws-sdk-go-v2/service/internal/checksum v1.9.14 // indirect
github.com/aws/aws-sdk-go-v2/service/internal/presigned-url v1.13.22 // indirect
github.com/aws/aws-sdk-go-v2/service/internal/s3shared v1.19.22 // indirect
github.com/aws/aws-sdk-go-v2/service/sso v1.30.16 // indirect
github.com/aws/aws-sdk-go-v2/service/ssooidc v1.35.20 // indirect
github.com/aws/aws-sdk-go-v2/service/sts v1.42.0 // indirect
github.com/aws/smithy-go v1.25.1 // indirect
github.com/davecgh/go-spew v1.1.2-0.20180830191138-d8f796af33cc // indirect
github.com/disintegration/imaging v1.6.2
github.com/golang-jwt/jwt/v5 v5.3.1
github.com/mattn/go-isatty v0.0.22 // indirect
github.com/pmezard/go-difflib v1.0.1-0.20181226105442-5d4384ee4fb2 // indirect
golang.org/x/sys v0.43.0 // indirect
golang.org/x/text v0.36.0
golang.org/x/time v0.15.0 // indirect
google.golang.org/protobuf v1.36.11
gopkg.in/yaml.v3 v3.0.1
)
+365
View File
@@ -0,0 +1,365 @@
cel.dev/expr v0.25.1 h1:1KrZg61W6TWSxuNZ37Xy49ps13NUovb66QLprthtwi4=
cel.dev/expr v0.25.1/go.mod h1:hrXvqGP6G6gyx8UAHSHJ5RGk//1Oj5nXQ2NI02Nrsg4=
cloud.google.com/go v0.123.0 h1:2NAUJwPR47q+E35uaJeYoNhuNEM9kM8SjgRgdeOJUSE=
cloud.google.com/go v0.123.0/go.mod h1:xBoMV08QcqUGuPW65Qfm1o9Y4zKZBpGS+7bImXLTAZU=
cloud.google.com/go/auth v0.20.0 h1:kXTssoVb4azsVDoUiF8KvxAqrsQcQtB53DcSgta74CA=
cloud.google.com/go/auth v0.20.0/go.mod h1:942/yi/itH1SsmpyrbnTMDgGfdy2BUqIKyd0cyYLc5Q=
cloud.google.com/go/compute/metadata v0.9.0 h1:pDUj4QMoPejqq20dK0Pg2N4yG9zIkYGdBtwLoEkH9Zs=
cloud.google.com/go/compute/metadata v0.9.0/go.mod h1:E0bWwX5wTnLPedCKqk3pJmVgCBSM6qQI1yTBdEb3C10=
connectrpc.com/connect v1.19.2 h1:McQ83FGdzL+t60peksi0gXC7MQ/iLKgLduAnThbM0mo=
connectrpc.com/connect v1.19.2/go.mod h1:tN20fjdGlewnSFeZxLKb0xwIZ6ozc3OQs2hTXy4du9w=
dario.cat/mergo v1.0.2 h1:85+piFYR1tMbRrLcDwR18y4UKJ3aH1Tbzi24VRW1TK8=
dario.cat/mergo v1.0.2/go.mod h1:E/hbnu0NxMFBjpMIE34DRGLWqDy0g5FuKDhCb31ngxA=
filippo.io/edwards25519 v1.2.0 h1:crnVqOiS4jqYleHd9vaKZ+HKtHfllngJIiOpNpoJsjo=
filippo.io/edwards25519 v1.2.0/go.mod h1:xzAOLCNug/yB62zG1bQ8uziwrIqIuxhctzJT18Q77mc=
github.com/AdaLogics/go-fuzz-headers v0.0.0-20240806141605-e8a1dd7889d6 h1:He8afgbRMd7mFxO99hRNu+6tazq8nFF9lIwo9JFroBk=
github.com/AdaLogics/go-fuzz-headers v0.0.0-20240806141605-e8a1dd7889d6/go.mod h1:8o94RPi1/7XTJvwPpRSzSUedZrtlirdB3r9Z20bi2f8=
github.com/Azure/go-ansiterm v0.0.0-20250102033503-faa5f7b0171c h1:udKWzYgxTojEKWjV8V+WSxDXJ4NFATAsZjh8iIbsQIg=
github.com/Azure/go-ansiterm v0.0.0-20250102033503-faa5f7b0171c/go.mod h1:xomTg63KZ2rFqZQzSB4Vz2SUXa1BpHTVz9L5PTmPC4E=
github.com/Microsoft/go-winio v0.6.2 h1:F2VQgta7ecxGYO8k3ZZz3RS8fVIXVxONVUPlNERoyfY=
github.com/Microsoft/go-winio v0.6.2/go.mod h1:yd8OoFMLzJbo9gZq8j5qaps8bJ9aShtEA8Ipt1oGCvU=
github.com/antlr4-go/antlr/v4 v4.13.1 h1:SqQKkuVZ+zWkMMNkjy5FZe5mr5WURWnlpmOuzYWrPrQ=
github.com/antlr4-go/antlr/v4 v4.13.1/go.mod h1:GKmUxMtwp6ZgGwZSva4eWPC5mS6vUAmOABFgjdkM7Nw=
github.com/at-wat/ebml-go v0.18.0 h1:SNkpBFR4jCQV1rI4Bm1tSuIYnusxe2qQ4GHJia9eQg4=
github.com/at-wat/ebml-go v0.18.0/go.mod h1:w1cJs7zmGsb5nnSvhWGKLCxvfu4FVx5ERvYDIalj1ww=
github.com/aws/aws-sdk-go-v2 v1.41.6 h1:1AX0AthnBQzMx1vbmir3Y4WsnJgiydmnJjiLu+LvXOg=
github.com/aws/aws-sdk-go-v2 v1.41.6/go.mod h1:dy0UzBIfwSeot4grGvY1AqFWN5zgziMmWGzysDnHFcQ=
github.com/aws/aws-sdk-go-v2/aws/protocol/eventstream v1.7.9 h1:adBsCIIpLbLmYnkQU+nAChU5yhVTvu5PerROm+/Kq2A=
github.com/aws/aws-sdk-go-v2/aws/protocol/eventstream v1.7.9/go.mod h1:uOYhgfgThm/ZyAuJGNQ5YgNyOlYfqnGpTHXvk3cpykg=
github.com/aws/aws-sdk-go-v2/config v1.32.16 h1:Q0iQ7quUgJP0F/SCRTieScnaMdXr9h/2+wze1u3cNeM=
github.com/aws/aws-sdk-go-v2/config v1.32.16/go.mod h1:duCCnJEFqpt2RC6no1iK6q+8HpwOAkiUua0pY507dQc=
github.com/aws/aws-sdk-go-v2/credentials v1.19.15 h1:fyvgWTszojq8hEnMi8PPBTvZdTtEVmAVyo+NFLHBhH4=
github.com/aws/aws-sdk-go-v2/credentials v1.19.15/go.mod h1:gJiYyMOjNg8OEdRWOf3CrFQxM2a98qmrtjx1zuiQfB8=
github.com/aws/aws-sdk-go-v2/feature/ec2/imds v1.18.22 h1:IOGsJ1xVWhsi+ZO7/NW8OuZZBtMJLZbk4P5HDjJO0jQ=
github.com/aws/aws-sdk-go-v2/feature/ec2/imds v1.18.22/go.mod h1:b+hYdbU+jGKfXE8kKM6g1+h+L/Go3vMvzlxBsiuGsxg=
github.com/aws/aws-sdk-go-v2/internal/configsources v1.4.22 h1:GmLa5Kw1ESqtFpXsx5MmC84QWa/ZrLZvlJGa2y+4kcQ=
github.com/aws/aws-sdk-go-v2/internal/configsources v1.4.22/go.mod h1:6sW9iWm9DK9YRpRGga/qzrzNLgKpT2cIxb7Vo2eNOp0=
github.com/aws/aws-sdk-go-v2/internal/endpoints/v2 v2.7.22 h1:dY4kWZiSaXIzxnKlj17nHnBcXXBfac6UlsAx2qL6XrU=
github.com/aws/aws-sdk-go-v2/internal/endpoints/v2 v2.7.22/go.mod h1:KIpEUx0JuRZLO7U6cbV204cWAEco2iC3l061IxlwLtI=
github.com/aws/aws-sdk-go-v2/internal/v4a v1.4.23 h1:FPXsW9+gMuIeKmz7j6ENWcWtBGTe1kH8r9thNt5Uxx4=
github.com/aws/aws-sdk-go-v2/internal/v4a v1.4.23/go.mod h1:7J8iGMdRKk6lw2C+cMIphgAnT8uTwBwNOsGkyOCm80U=
github.com/aws/aws-sdk-go-v2/service/internal/accept-encoding v1.13.8 h1:HtOTYcbVcGABLOVuPYaIihj6IlkqubBwFj10K5fxRek=
github.com/aws/aws-sdk-go-v2/service/internal/accept-encoding v1.13.8/go.mod h1:VsK9abqQeGlzPgUr+isNWzPlK2vKe9INMLWnY65f5Xs=
github.com/aws/aws-sdk-go-v2/service/internal/checksum v1.9.14 h1:xnvDEnw+pnj5mctWiYuFbigrEzSm35x7k4KS/ZkCANg=
github.com/aws/aws-sdk-go-v2/service/internal/checksum v1.9.14/go.mod h1:yS5rNogD8e0Wu9+l3MUwr6eENBzEeGejvINpN5PAYfY=
github.com/aws/aws-sdk-go-v2/service/internal/presigned-url v1.13.22 h1:PUmZeJU6Y1Lbvt9WFuJ0ugUK2xn6hIWUBBbKuOWF30s=
github.com/aws/aws-sdk-go-v2/service/internal/presigned-url v1.13.22/go.mod h1:nO6egFBoAaoXze24a2C0NjQCvdpk8OueRoYimvEB9jo=
github.com/aws/aws-sdk-go-v2/service/internal/s3shared v1.19.22 h1:SE+aQ4DEqG53RRCAIHlCf//B2ycxGH7jFkpnAh/kKPM=
github.com/aws/aws-sdk-go-v2/service/internal/s3shared v1.19.22/go.mod h1:ES3ynECd7fYeJIL6+oax+uIEljmfps0S70BaQzbMd/o=
github.com/aws/aws-sdk-go-v2/service/s3 v1.100.0 h1:7G26Sae6PMKn4kMcU5JzNfrm1YrKwyOhowXPYR2WiWY=
github.com/aws/aws-sdk-go-v2/service/s3 v1.100.0/go.mod h1:Fw9aqhJicIVee1VytBBjH+l+5ov6/PhbtIK/u3rt/ls=
github.com/aws/aws-sdk-go-v2/service/signin v1.0.10 h1:a1Fq/KXn75wSzoJaPQTgZO0wHGqE9mjFnylnqEPTchA=
github.com/aws/aws-sdk-go-v2/service/signin v1.0.10/go.mod h1:p6+MXNxW7IA6dMgHfTAzljuwSKD0NCm/4lbS4t6+7vI=
github.com/aws/aws-sdk-go-v2/service/sso v1.30.16 h1:x6bKbmDhsgSZwv6q19wY/u3rLk/3FGjJWyqKcIRufpE=
github.com/aws/aws-sdk-go-v2/service/sso v1.30.16/go.mod h1:CudnEVKRtLn0+3uMV0yEXZ+YZOKnAtUJ5DmDhilVnIw=
github.com/aws/aws-sdk-go-v2/service/ssooidc v1.35.20 h1:oK/njaL8GtyEihkWMD4k3VgHCT64RQKkZwh0DG5j8ak=
github.com/aws/aws-sdk-go-v2/service/ssooidc v1.35.20/go.mod h1:JHs8/y1f3zY7U5WcuzoJ/yAYGYtNIVPKLIbp61euvmg=
github.com/aws/aws-sdk-go-v2/service/sts v1.42.0 h1:ks8KBcZPh3PYISr5dAiXCM5/Thcuxk8l+PG4+A0exds=
github.com/aws/aws-sdk-go-v2/service/sts v1.42.0/go.mod h1:pFw33T0WLvXU3rw1WBkpMlkgIn54eCB5FYLhjDc9Foo=
github.com/aws/smithy-go v1.25.1 h1:J8ERsGSU7d+aCmdQur5Txg6bVoYelvQJgtZehD12GkI=
github.com/aws/smithy-go v1.25.1/go.mod h1:YE2RhdIuDbA5E5bTdciG9KrW3+TiEONeUWCqxX9i1Fc=
github.com/cenkalti/backoff/v4 v4.3.0 h1:MyRJ/UdXutAwSAT+s3wNd7MfTIcy71VQueUuFK343L8=
github.com/cenkalti/backoff/v4 v4.3.0/go.mod h1:Y3VNntkOUPxTVeUxJ/G5vcM//AlwfmyYozVcomhLiZE=
github.com/cespare/xxhash/v2 v2.3.0 h1:UL815xU9SqsFlibzuggzjXhog7bL6oX9BbNZnL2UFvs=
github.com/cespare/xxhash/v2 v2.3.0/go.mod h1:VGX0DQ3Q6kWi7AoAeZDth3/j3BFtOZR5XLFGgcrjCOs=
github.com/containerd/errdefs v1.0.0 h1:tg5yIfIlQIrxYtu9ajqY42W3lpS19XqdxRQeEwYG8PI=
github.com/containerd/errdefs v1.0.0/go.mod h1:+YBYIdtsnF4Iw6nWZhJcqGSg/dwvV7tyJ/kCkyJ2k+M=
github.com/containerd/errdefs/pkg v0.3.0 h1:9IKJ06FvyNlexW690DXuQNx2KA2cUJXx151Xdx3ZPPE=
github.com/containerd/errdefs/pkg v0.3.0/go.mod h1:NJw6s9HwNuRhnjJhM7pylWwMyAkmCQvQ4GpJHEqRLVk=
github.com/containerd/log v0.1.0 h1:TCJt7ioM2cr/tfR8GPbGf9/VRAX8D2B4PjzCpfX540I=
github.com/containerd/log v0.1.0/go.mod h1:VRRf09a7mHDIRezVKTRCrOq78v577GXq3bSa3EhrzVo=
github.com/containerd/platforms v0.2.1 h1:zvwtM3rz2YHPQsF2CHYM8+KtB5dvhISiXh5ZpSBQv6A=
github.com/containerd/platforms v0.2.1/go.mod h1:XHCb+2/hzowdiut9rkudds9bE5yJ7npe7dG/wG+uFPw=
github.com/cpuguy83/dockercfg v0.3.2 h1:DlJTyZGBDlXqUZ2Dk2Q3xHs/FtnooJJVaad2S9GKorA=
github.com/cpuguy83/dockercfg v0.3.2/go.mod h1:sugsbF4//dDlL/i+S+rtpIWp+5h0BHJHfjj5/jFyUJc=
github.com/cpuguy83/go-md2man/v2 v2.0.6/go.mod h1:oOW0eioCTA6cOiMLiUPZOpcVxMig6NIQQ7OS05n1F4g=
github.com/creack/pty v1.1.24 h1:bJrF4RRfyJnbTJqzRLHzcGaZK1NeM5kTC9jGgovnR1s=
github.com/creack/pty v1.1.24/go.mod h1:08sCNb52WyoAwi2QDyzUCTgcvVFhUzewun7wtTfvcwE=
github.com/davecgh/go-spew v1.1.2-0.20180830191138-d8f796af33cc h1:U9qPSI2PIWSS1VwoXQT9A3Wy9MM3WgvqSxFWenqJduM=
github.com/davecgh/go-spew v1.1.2-0.20180830191138-d8f796af33cc/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
github.com/disintegration/imaging v1.6.2 h1:w1LecBlG2Lnp8B3jk5zSuNqd7b4DXhcjwek1ei82L+c=
github.com/disintegration/imaging v1.6.2/go.mod h1:44/5580QXChDfwIclfc/PCwrr44amcmDAg8hxG0Ewe4=
github.com/distribution/reference v0.6.0 h1:0IXCQ5g4/QMHHkarYzh5l+u8T3t73zM5QvfrDyIgxBk=
github.com/distribution/reference v0.6.0/go.mod h1:BbU0aIcezP1/5jX/8MP0YiH4SdvB5Y4f/wlDRiLyi3E=
github.com/docker/go-connections v0.7.0 h1:6SsRfJddP22WMrCkj19x9WKjEDTB+ahsdiGYf0mN39c=
github.com/docker/go-connections v0.7.0/go.mod h1:no1qkHdjq7kLMGUXYAduOhYPSJxxvgWBh7ogVvptn3Q=
github.com/docker/go-units v0.5.0 h1:69rxXcBk27SvSaaxTtLh/8llcHD8vYHT7WSdRZ/jvr4=
github.com/docker/go-units v0.5.0/go.mod h1:fgPhTUdO+D/Jk86RDLlptpiXQzgHJF7gydDDbaIK4Dk=
github.com/dustin/go-humanize v1.0.1 h1:GzkhY7T5VNhEkwH0PVJgjz+fX1rhBrR7pRT3mDkpeCY=
github.com/dustin/go-humanize v1.0.1/go.mod h1:Mu1zIs6XwVuF/gI1OepvI0qD18qycQx+mFykh5fBlto=
github.com/ebitengine/purego v0.10.0 h1:QIw4xfpWT6GWTzaW5XEKy3HXoqrJGx1ijYHzTF0/ISU=
github.com/ebitengine/purego v0.10.0/go.mod h1:iIjxzd6CiRiOG0UyXP+V1+jWqUXVjPKLAI0mRfJZTmQ=
github.com/felixge/httpsnoop v1.0.4 h1:NFTV2Zj1bL4mc9sqWACXbQFVBBg2W3GPvqp8/ESS2Wg=
github.com/felixge/httpsnoop v1.0.4/go.mod h1:m8KPJKqk1gH5J9DgRY2ASl2lWCfGKXixSwevea8zH2U=
github.com/frankban/quicktest v1.14.6 h1:7Xjx+VpznH+oBnejlPUj8oUpdxnVs4f8XU8WnHkI4W8=
github.com/frankban/quicktest v1.14.6/go.mod h1:4ptaffx2x8+WTWXmUCuVU6aPUX1/Mz7zb5vbUoiM6w0=
github.com/fsnotify/fsnotify v1.9.0 h1:2Ml+OJNzbYCTzsxtv8vKSFD9PbJjmhYF14k/jKC7S9k=
github.com/fsnotify/fsnotify v1.9.0/go.mod h1:8jBTzvmWwFyi3Pb8djgCCO5IBqzKJ/Jwo8TRcHyHii0=
github.com/go-logr/logr v1.2.2/go.mod h1:jdQByPbusPIv2/zmleS9BjJVeZ6kBagPoEUsqbVz/1A=
github.com/go-logr/logr v1.4.3 h1:CjnDlHq8ikf6E492q6eKboGOC0T8CDaOvkHCIg8idEI=
github.com/go-logr/logr v1.4.3/go.mod h1:9T104GzyrTigFIr8wt5mBrctHMim0Nb2HLGrmQ40KvY=
github.com/go-logr/stdr v1.2.2 h1:hSWxHoqTgW2S2qGc0LTAI563KZ5YKYRhT3MFKZMbjag=
github.com/go-logr/stdr v1.2.2/go.mod h1:mMo/vtBO5dYbehREoey6XUKy/eSumjCCveDpRre4VKE=
github.com/go-ole/go-ole v1.2.6/go.mod h1:pprOEPIfldk/42T2oK7lQ4v4JSDwmV0As9GaiUsvbm0=
github.com/go-ole/go-ole v1.3.0 h1:Dt6ye7+vXGIKZ7Xtk4s6/xVdGDQynvom7xCFEdWr6uE=
github.com/go-ole/go-ole v1.3.0/go.mod h1:5LS6F96DhAwUc7C+1HLexzMXY1xGRSryjyPPKW6zv78=
github.com/go-sql-driver/mysql v1.9.3 h1:U/N249h2WzJ3Ukj8SowVFjdtZKfu9vlLZxjPXV1aweo=
github.com/go-sql-driver/mysql v1.9.3/go.mod h1:qn46aNg1333BRMNU69Lq93t8du/dwxI64Gl8i5p1WMU=
github.com/go-viper/mapstructure/v2 v2.5.0 h1:vM5IJoUAy3d7zRSVtIwQgBj7BiWtMPfmPEgAXnvj1Ro=
github.com/go-viper/mapstructure/v2 v2.5.0/go.mod h1:oJDH3BJKyqBA2TXFhDsKDGDTlndYOZ6rGS0BRZIxGhM=
github.com/golang-jwt/jwt/v5 v5.3.1 h1:kYf81DTWFe7t+1VvL7eS+jKFVWaUnK9cB1qbwn63YCY=
github.com/golang-jwt/jwt/v5 v5.3.1/go.mod h1:fxCRLWMO43lRc8nhHWY6LGqRcf+1gQWArsqaEUEa5bE=
github.com/golang/protobuf v1.5.4 h1:i7eJL8qZTpSEXOPTxNKhASYpMn+8e5Q6AdndVa1dWek=
github.com/golang/protobuf v1.5.4/go.mod h1:lnTiLA8Wa4RWRcIUkrtSVa5nRhsEGBg48fD6rSs7xps=
github.com/google/cel-go v0.28.1 h1:YWIwi77J4xIsYUwAF/iIuS6haffzIHS8yWI8glSbLWM=
github.com/google/cel-go v0.28.1/go.mod h1:X0bD6iVNR8pkROSOoHVdgTkzmRcosof7WQqCD6wcMc8=
github.com/google/go-cmp v0.7.0 h1:wk8382ETsv4JYUZwIsn6YpYiWiBsYLSJiTsyBybVuN8=
github.com/google/go-cmp v0.7.0/go.mod h1:pXiqmnSA92OHEEa9HXL2W4E7lf9JzCmGVUdgjX3N/iU=
github.com/google/jsonschema-go v0.4.3 h1:/DBOLZTfDow7pe2GmaJNhltueGTtDKICi8V8p+DQPd0=
github.com/google/jsonschema-go v0.4.3/go.mod h1:r5quNTdLOYEz95Ru18zA0ydNbBuYoo9tgaYcxEYhJVE=
github.com/google/pprof v0.0.0-20250317173921-a4b03ec1a45e h1:ijClszYn+mADRFY17kjQEVQ1XRhq2/JR1M3sGqeJoxs=
github.com/google/pprof v0.0.0-20250317173921-a4b03ec1a45e/go.mod h1:boTsfXsheKC2y+lKOCMpSfarhxDeIzfZG1jqGcPl3cA=
github.com/google/s2a-go v0.1.9 h1:LGD7gtMgezd8a/Xak7mEWL0PjoTQFvpRudN895yqKW0=
github.com/google/s2a-go v0.1.9/go.mod h1:YA0Ei2ZQL3acow2O62kdp9UlnvMmU7kA6Eutn0dXayM=
github.com/google/uuid v1.6.0 h1:NIvaJDMOsjHA8n1jAhLSgzrAzy1Hgr+hNrb57e+94F0=
github.com/google/uuid v1.6.0/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo=
github.com/googleapis/enterprise-certificate-proxy v0.3.15 h1:xolVQTEXusUcAA5UgtyRLjelpFFHWlPQ4XfWGc7MBas=
github.com/googleapis/enterprise-certificate-proxy v0.3.15/go.mod h1:vqVt9yG9480NtzREnTlmGSBmFrA+bzb0yl0TxoBQXOg=
github.com/googleapis/gax-go/v2 v2.22.0 h1:PjIWBpgGIVKGoCXuiCoP64altEJCj3/Ei+kSU5vlZD4=
github.com/googleapis/gax-go/v2 v2.22.0/go.mod h1:irWBbALSr0Sk3qlqb9SyJ1h68WjgeFuiOzI4Rqw5+aY=
github.com/gorilla/feeds v1.2.0 h1:O6pBiXJ5JHhPvqy53NsjKOThq+dNFm8+DFrxBEdzSCc=
github.com/gorilla/feeds v1.2.0/go.mod h1:WMib8uJP3BbY+X8Szd1rA5Pzhdfh+HCCAYT2z7Fza6Y=
github.com/gorilla/websocket v1.5.3 h1:saDtZ6Pbx/0u+bgYQ3q96pZgCzfhKXGPqt7kZ72aNNg=
github.com/gorilla/websocket v1.5.3/go.mod h1:YR8l580nyteQvAITg2hZ9XVh4b55+EU/adAjf1fMHhE=
github.com/grpc-ecosystem/grpc-gateway/v2 v2.29.0 h1:5VipnvEpbqr2gA2VbM+nYVbkIF28c5ZQfqCBQ5g2xfk=
github.com/grpc-ecosystem/grpc-gateway/v2 v2.29.0/go.mod h1:Hyl3n6Twe1hvtd9XUXDec4pTvgMSEixRuQKPTMH2bNs=
github.com/hashicorp/golang-lru/v2 v2.0.7 h1:a+bsQ5rvGLjzHuww6tVxozPZFVghXaHOwFs4luLUK2k=
github.com/hashicorp/golang-lru/v2 v2.0.7/go.mod h1:QeFd9opnmA6QUJc5vARoKUSoFhyfM2/ZepoAG6RGpeM=
github.com/inconshreveable/mousetrap v1.1.0 h1:wN+x4NVGpMsO7ErUn/mUI3vEoE6Jt13X2s0bqwp9tc8=
github.com/inconshreveable/mousetrap v1.1.0/go.mod h1:vpF70FUmC8bwa3OWnCshd2FqLfsEA9PFc4w1p2J65bw=
github.com/jackc/pgpassfile v1.0.0 h1:/6Hmqy13Ss2zCq62VdNG8tM1wchn8zjSGOBJ6icpsIM=
github.com/jackc/pgpassfile v1.0.0/go.mod h1:CEx0iS5ambNFdcRtxPj5JhEz+xB6uRky5eyVu/W2HEg=
github.com/jackc/pgservicefile v0.0.0-20221227161230-091c0ba34f0a h1:bbPeKD0xmW/Y25WS6cokEszi5g+S0QxI/d45PkRi7Nk=
github.com/jackc/pgservicefile v0.0.0-20221227161230-091c0ba34f0a/go.mod h1:5TJZWKEWniPve33vlWYSoGYefn3gLQRzjfDlhSJ9ZKM=
github.com/jackc/pgx/v5 v5.5.4 h1:Xp2aQS8uXButQdnCMWNmvx6UysWQQC+u1EoizjguY+8=
github.com/jackc/pgx/v5 v5.5.4/go.mod h1:ez9gk+OAat140fv9ErkZDYFWmXLfV+++K0uAOiwgm1A=
github.com/jackc/puddle/v2 v2.2.1 h1:RhxXJtFG022u4ibrCSMSiu5aOq1i77R3OHKNJj77OAk=
github.com/jackc/puddle/v2 v2.2.1/go.mod h1:vriiEXHvEE654aYKXXjOvZM39qJ0q+azkZFrfEOc3H4=
github.com/joho/godotenv v1.5.1 h1:7eLL/+HRGLY0ldzfGMeQkb7vMd0as4CfYvUVzLqw0N0=
github.com/joho/godotenv v1.5.1/go.mod h1:f4LDr5Voq0i2e/R5DDNOoa2zzDfwtkZa6DnEwAbqwq4=
github.com/klauspost/compress v1.18.5 h1:/h1gH5Ce+VWNLSWqPzOVn6XBO+vJbCNGvjoaGBFW2IE=
github.com/klauspost/compress v1.18.5/go.mod h1:cwPg85FWrGar70rWktvGQj8/hthj3wpl0PGDogxkrSQ=
github.com/kr/pretty v0.3.1 h1:flRD4NNwYAUpkphVc1HcthR4KEIFJ65n8Mw5qdRn3LE=
github.com/kr/pretty v0.3.1/go.mod h1:hoEshYVHaxMs3cyo3Yncou5ZscifuDolrwPKZanG3xk=
github.com/kr/text v0.2.0 h1:5Nx0Ya0ZqY2ygV366QzturHI13Jq95ApcVaJBhpS+AY=
github.com/kr/text v0.2.0/go.mod h1:eLer722TekiGuMkidMxC/pM04lWEeraHUUmBw8l2grE=
github.com/labstack/echo/v5 v5.1.0 h1:MvIRydoN+p9cx/zq8Lff6YXqUW2ZaEsOMISzEGSMrBI=
github.com/labstack/echo/v5 v5.1.0/go.mod h1:SyvlSdObGjRXeQfCCXW/sybkZdOOQZBmpKF0bvALaeo=
github.com/lib/pq v1.12.3 h1:tTWxr2YLKwIvK90ZXEw8GP7UFHtcbTtty8zsI+YjrfQ=
github.com/lib/pq v1.12.3/go.mod h1:/p+8NSbOcwzAEI7wiMXFlgydTwcgTr3OSKMsD2BitpA=
github.com/lithammer/shortuuid/v4 v4.2.0 h1:LMFOzVB3996a7b8aBuEXxqOBflbfPQAiVzkIcHO0h8c=
github.com/lithammer/shortuuid/v4 v4.2.0/go.mod h1:D5noHZ2oFw/YaKCfGy0YxyE7M0wMbezmMjPdhyEFe6Y=
github.com/lufia/plan9stats v0.0.0-20260330125221-c963978e514e h1:Q6MvJtQK/iRcRtzAscm/zF23XxJlbECiGPyRicsX+Ak=
github.com/lufia/plan9stats v0.0.0-20260330125221-c963978e514e/go.mod h1:autxFIvghDt3jPTLoqZ9OZ7s9qTGNAWmYCjVFWPX/zg=
github.com/magiconair/properties v1.8.10 h1:s31yESBquKXCV9a/ScB3ESkOjUYYv+X0rg8SYxI99mE=
github.com/magiconair/properties v1.8.10/go.mod h1:Dhd985XPs7jluiymwWYZ0G4Z61jb3vdS329zhj2hYo0=
github.com/mattn/go-isatty v0.0.22 h1:j8l17JJ9i6VGPUFUYoTUKPSgKe/83EYU2zBC7YNKMw4=
github.com/mattn/go-isatty v0.0.22/go.mod h1:ZXfXG4SQHsB/w3ZeOYbR0PrPwLy+n6xiMrJlRFqopa4=
github.com/mdelapenya/tlscert v0.2.0 h1:7H81W6Z/4weDvZBNOfQte5GpIMo0lGYEeWbkGp5LJHI=
github.com/mdelapenya/tlscert v0.2.0/go.mod h1:O4njj3ELLnJjGdkN7M/vIVCpZ+Cf0L6muqOG4tLSl8o=
github.com/moby/docker-image-spec v1.3.1 h1:jMKff3w6PgbfSa69GfNg+zN/XLhfXJGnEx3Nl2EsFP0=
github.com/moby/docker-image-spec v1.3.1/go.mod h1:eKmb5VW8vQEh/BAr2yvVNvuiJuY6UIocYsFu/DxxRpo=
github.com/moby/go-archive v0.2.0 h1:zg5QDUM2mi0JIM9fdQZWC7U8+2ZfixfTYoHL7rWUcP8=
github.com/moby/go-archive v0.2.0/go.mod h1:mNeivT14o8xU+5q1YnNrkQVpK+dnNe/K6fHqnTg4qPU=
github.com/moby/moby/api v1.54.2 h1:wiat9QAhnDQjA7wk1kh/TqHz2I1uUA7M7t9SAl/JNXg=
github.com/moby/moby/api v1.54.2/go.mod h1:+RQ6wluLwtYaTd1WnPLykIDPekkuyD/ROWQClE83pzs=
github.com/moby/moby/client v0.4.1 h1:DMQgisVoMkmMs7fp3ROSdiBnoAu8+vo3GggFl06M/wY=
github.com/moby/moby/client v0.4.1/go.mod h1:z52C9O2POPOsnxZAy//WtKcQ32P+jT/NGeXu/7nfjGQ=
github.com/moby/patternmatcher v0.6.1 h1:qlhtafmr6kgMIJjKJMDmMWq7WLkKIo23hsrpR3x084U=
github.com/moby/patternmatcher v0.6.1/go.mod h1:hDPoyOpDY7OrrMDLaYoY3hf52gNCR/YOUYxkhApJIxc=
github.com/moby/sys/sequential v0.6.0 h1:qrx7XFUd/5DxtqcoH1h438hF5TmOvzC/lspjy7zgvCU=
github.com/moby/sys/sequential v0.6.0/go.mod h1:uyv8EUTrca5PnDsdMGXhZe6CCe8U/UiTWd+lL+7b/Ko=
github.com/moby/sys/user v0.4.0 h1:jhcMKit7SA80hivmFJcbB1vqmw//wU61Zdui2eQXuMs=
github.com/moby/sys/user v0.4.0/go.mod h1:bG+tYYYJgaMtRKgEmuueC0hJEAZWwtIbZTB+85uoHjs=
github.com/moby/sys/userns v0.1.0 h1:tVLXkFOxVu9A64/yh59slHVv9ahO9UIev4JZusOLG/g=
github.com/moby/sys/userns v0.1.0/go.mod h1:IHUYgu/kao6N8YZlp9Cf444ySSvCmDlmzUcYfDHOl28=
github.com/moby/term v0.5.2 h1:6qk3FJAFDs6i/q3W/pQ97SX192qKfZgGjCQqfCJkgzQ=
github.com/moby/term v0.5.2/go.mod h1:d3djjFCrjnB+fl8NJux+EJzu0msscUP+f8it8hPkFLc=
github.com/modelcontextprotocol/go-sdk v1.6.1 h1:0zOSupjKUxPKSocPT1Wtago+mUHU2/uZ4xSOY0FGReU=
github.com/modelcontextprotocol/go-sdk v1.6.1/go.mod h1:kzm3kzFL1/+AziGOE0nUs3gvPoNxMCvkxokMkuFapXQ=
github.com/ncruces/go-strftime v1.0.0 h1:HMFp8mLCTPp341M/ZnA4qaf7ZlsbTc+miZjCLOFAw7w=
github.com/ncruces/go-strftime v1.0.0/go.mod h1:Fwc5htZGVVkseilnfgOVb9mKy6w1naJmn9CehxcKcls=
github.com/openai/openai-go/v3 v3.32.0 h1:aHp/3wkX1W6jB8zTtf9xV0aK0qPFSVDqS7AHmlJ4hXs=
github.com/openai/openai-go/v3 v3.32.0/go.mod h1:cdufnVK14cWcT9qA1rRtrXx4FTRsgbDPW7Ia7SS5cZo=
github.com/opencontainers/go-digest v1.0.0 h1:apOUWs51W5PlhuyGyz9FCeeBIOUDA/6nW8Oi/yOhh5U=
github.com/opencontainers/go-digest v1.0.0/go.mod h1:0JzlMkj0TRzQZfJkVvzbP0HBR3IKzErnv2BNG4W4MAM=
github.com/opencontainers/image-spec v1.1.1 h1:y0fUlFfIZhPF1W537XOLg0/fcx6zcHCJwooC2xJA040=
github.com/opencontainers/image-spec v1.1.1/go.mod h1:qpqAh3Dmcf36wStyyWU+kCeDgrGnAve2nCC8+7h8Q0M=
github.com/pelletier/go-toml/v2 v2.3.0 h1:k59bC/lIZREW0/iVaQR8nDHxVq8OVlIzYCOJf421CaM=
github.com/pelletier/go-toml/v2 v2.3.0/go.mod h1:2gIqNv+qfxSVS7cM2xJQKtLSTLUE9V8t9Stt+h56mCY=
github.com/pion/opus v0.0.0-20260430223319-81a9c5dc5013 h1:HDxWSNNH8R5G+y1xGM8AVsSu95rAmoOnVSdPTzoAtoI=
github.com/pion/opus v0.0.0-20260430223319-81a9c5dc5013/go.mod h1:t5Xog2n682JnawoykACE6nKVmupFvmJvkpM7x6bTv6g=
github.com/pkg/errors v0.9.1 h1:FEBLx1zS214owpjy7qsBeixbURkuhQAwrK5UwLGTwt4=
github.com/pkg/errors v0.9.1/go.mod h1:bwawxfHBFNV+L2hUp1rHADufV3IMtnDRdf1r5NINEl0=
github.com/pmezard/go-difflib v1.0.1-0.20181226105442-5d4384ee4fb2 h1:Jamvg5psRIccs7FGNTlIRMkT8wgtp5eCXdBlqhYGL6U=
github.com/pmezard/go-difflib v1.0.1-0.20181226105442-5d4384ee4fb2/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=
github.com/power-devops/perfstat v0.0.0-20240221224432-82ca36839d55 h1:o4JXh1EVt9k/+g42oCprj/FisM4qX9L3sZB3upGN2ZU=
github.com/power-devops/perfstat v0.0.0-20240221224432-82ca36839d55/go.mod h1:OmDBASR4679mdNQnz2pUhc2G8CO2JrUAVFDRBDP/hJE=
github.com/remyoudompheng/bigfft v0.0.0-20230129092748-24d4a6f8daec h1:W09IVJc94icq4NjY3clb7Lk8O1qJ8BdBEF8z0ibU0rE=
github.com/remyoudompheng/bigfft v0.0.0-20230129092748-24d4a6f8daec/go.mod h1:qqbHyh8v60DhA7CoWK5oRCqLrMHRGoxYCSS9EjAz6Eo=
github.com/rogpeppe/go-internal v1.14.1 h1:UQB4HGPB6osV0SQTLymcB4TgvyWu6ZyliaW0tI/otEQ=
github.com/rogpeppe/go-internal v1.14.1/go.mod h1:MaRKkUm5W0goXpeCfT7UZI6fk/L7L7so1lCWt35ZSgc=
github.com/russross/blackfriday/v2 v2.1.0/go.mod h1:+Rmxgy9KzJVeS9/2gXHxylqXiyQDYRxCVz55jmeOWTM=
github.com/sagikazarmark/locafero v0.12.0 h1:/NQhBAkUb4+fH1jivKHWusDYFjMOOKU88eegjfxfHb4=
github.com/sagikazarmark/locafero v0.12.0/go.mod h1:sZh36u/YSZ918v0Io+U9ogLYQJ9tLLBmM4eneO6WwsI=
github.com/segmentio/asm v1.1.3 h1:WM03sfUOENvvKexOLp+pCqgb/WDjsi7EK8gIsICtzhc=
github.com/segmentio/asm v1.1.3/go.mod h1:Ld3L4ZXGNcSLRg4JBsZ3//1+f/TjYl0Mzen/DQy1EJg=
github.com/segmentio/encoding v0.5.4 h1:OW1VRern8Nw6ITAtwSZ7Idrl3MXCFwXHPgqESYfvNt0=
github.com/segmentio/encoding v0.5.4/go.mod h1:HS1ZKa3kSN32ZHVZ7ZLPLXWvOVIiZtyJnO1gPH1sKt0=
github.com/shirou/gopsutil/v4 v4.26.3 h1:2ESdQt90yU3oXF/CdOlRCJxrP+Am1aBYubTMTfxJ1qc=
github.com/shirou/gopsutil/v4 v4.26.3/go.mod h1:LZ6ewCSkBqUpvSOf+LsTGnRinC6iaNUNMGBtDkJBaLQ=
github.com/sirupsen/logrus v1.9.4 h1:TsZE7l11zFCLZnZ+teH4Umoq5BhEIfIzfRDZ1Uzql2w=
github.com/sirupsen/logrus v1.9.4/go.mod h1:ftWc9WdOfJ0a92nsE2jF5u5ZwH8Bv2zdeOC42RjbV2g=
github.com/spf13/afero v1.15.0 h1:b/YBCLWAJdFWJTN9cLhiXXcD7mzKn9Dm86dNnfyQw1I=
github.com/spf13/afero v1.15.0/go.mod h1:NC2ByUVxtQs4b3sIUphxK0NioZnmxgyCrfzeuq8lxMg=
github.com/spf13/cast v1.10.0 h1:h2x0u2shc1QuLHfxi+cTJvs30+ZAHOGRic8uyGTDWxY=
github.com/spf13/cast v1.10.0/go.mod h1:jNfB8QC9IA6ZuY2ZjDp0KtFO2LZZlg4S/7bzP6qqeHo=
github.com/spf13/cobra v1.10.2 h1:DMTTonx5m65Ic0GOoRY2c16WCbHxOOw6xxezuLaBpcU=
github.com/spf13/cobra v1.10.2/go.mod h1:7C1pvHqHw5A4vrJfjNwvOdzYu0Gml16OCs2GRiTUUS4=
github.com/spf13/pflag v1.0.9/go.mod h1:McXfInJRrz4CZXVZOBLb0bTZqETkiAhM9Iw0y3An2Bg=
github.com/spf13/pflag v1.0.10 h1:4EBh2KAYBwaONj6b2Ye1GiHfwjqyROoF4RwYO+vPwFk=
github.com/spf13/pflag v1.0.10/go.mod h1:McXfInJRrz4CZXVZOBLb0bTZqETkiAhM9Iw0y3An2Bg=
github.com/spf13/viper v1.21.0 h1:x5S+0EU27Lbphp4UKm1C+1oQO+rKx36vfCoaVebLFSU=
github.com/spf13/viper v1.21.0/go.mod h1:P0lhsswPGWD/1lZJ9ny3fYnVqxiegrlNrEmgLjbTCAY=
github.com/stretchr/objx v0.5.3 h1:jmXUvGomnU1o3W/V5h2VEradbpJDwGrzugQQvL0POH4=
github.com/stretchr/objx v0.5.3/go.mod h1:rDQraq+vQZU7Fde9LOZLr8Tax6zZvy4kuNKF+QYS+U0=
github.com/stretchr/testify v1.11.1 h1:7s2iGBzp5EwR7/aIZr8ao5+dra3wiQyKjjFuvgVKu7U=
github.com/stretchr/testify v1.11.1/go.mod h1:wZwfW3scLgRK+23gO65QZefKpKQRnfz6sD981Nm4B6U=
github.com/subosito/gotenv v1.6.0 h1:9NlTDc1FTs4qu0DDq7AEtTPNw6SVm7uBMsUCUjABIf8=
github.com/subosito/gotenv v1.6.0/go.mod h1:Dk4QP5c2W3ibzajGcXpNraDfq2IrhjMIvMSWPKKo0FU=
github.com/testcontainers/testcontainers-go v0.42.0 h1:He3IhTzTZOygSXLJPMX7n44XtK+qhjat1nI9cneBbUY=
github.com/testcontainers/testcontainers-go v0.42.0/go.mod h1:vZjdY1YmUA1qEForxOIOazfsrdyORJAbhi0bp8plN30=
github.com/testcontainers/testcontainers-go/modules/mysql v0.42.0 h1:Yhv1k7vDpyzZePntg5R5Oj4ZMCyWpAfpJeRu1ROsgiU=
github.com/testcontainers/testcontainers-go/modules/mysql v0.42.0/go.mod h1:Z7SCTuiZlghAdRjkv3Ir0iXJKC2T2avbtxLR0DRe+ng=
github.com/testcontainers/testcontainers-go/modules/postgres v0.42.0 h1:GCbb1ndrF7OTDiIvxXyItaDab4qkzTFJ48LKFdM7EIo=
github.com/testcontainers/testcontainers-go/modules/postgres v0.42.0/go.mod h1:IRPBaI8jXdrNfD0e4Zm7Fbcgaz5shKxOQv4axiL09xs=
github.com/tidwall/gjson v1.14.2/go.mod h1:/wbyibRr2FHMks5tjHJ5F8dMZh3AcwJEMf5vlfC0lxk=
github.com/tidwall/gjson v1.18.0 h1:FIDeeyB800efLX89e5a8Y0BNH+LOngJyGrIWxG2FKQY=
github.com/tidwall/gjson v1.18.0/go.mod h1:/wbyibRr2FHMks5tjHJ5F8dMZh3AcwJEMf5vlfC0lxk=
github.com/tidwall/match v1.1.1/go.mod h1:eRSPERbgtNPcGhD8UCthc6PmLEQXEWd3PRB5JTxsfmM=
github.com/tidwall/match v1.2.0 h1:0pt8FlkOwjN2fPt4bIl4BoNxb98gGHN2ObFEDkrfZnM=
github.com/tidwall/match v1.2.0/go.mod h1:eRSPERbgtNPcGhD8UCthc6PmLEQXEWd3PRB5JTxsfmM=
github.com/tidwall/pretty v1.2.0/go.mod h1:ITEVvHYasfjBbM0u2Pg8T2nJnzm8xPwvNhhsoaGGjNU=
github.com/tidwall/pretty v1.2.1 h1:qjsOFOWWQl+N3RsoF5/ssm1pHmJJwhjlSbZ51I6wMl4=
github.com/tidwall/pretty v1.2.1/go.mod h1:ITEVvHYasfjBbM0u2Pg8T2nJnzm8xPwvNhhsoaGGjNU=
github.com/tidwall/sjson v1.2.5 h1:kLy8mja+1c9jlljvWTlSazM7cKDRfJuR/bOJhcY5NcY=
github.com/tidwall/sjson v1.2.5/go.mod h1:Fvgq9kS/6ociJEDnK0Fk1cpYF4FIW6ZF7LAe+6jwd28=
github.com/tklauser/go-sysconf v0.3.16 h1:frioLaCQSsF5Cy1jgRBrzr6t502KIIwQ0MArYICU0nA=
github.com/tklauser/go-sysconf v0.3.16/go.mod h1:/qNL9xxDhc7tx3HSRsLWNnuzbVfh3e7gh/BmM179nYI=
github.com/tklauser/numcpus v0.11.0 h1:nSTwhKH5e1dMNsCdVBukSZrURJRoHbSEQjdEbY+9RXw=
github.com/tklauser/numcpus v0.11.0/go.mod h1:z+LwcLq54uWZTX0u/bGobaV34u6V7KNlTZejzM6/3MQ=
github.com/yosida95/uritemplate/v3 v3.0.2 h1:Ed3Oyj9yrmi9087+NczuL5BwkIc4wvTb5zIM+UJPGz4=
github.com/yosida95/uritemplate/v3 v3.0.2/go.mod h1:ILOh0sOhIJR3+L/8afwt/kE++YT040gmv5BQTMR2HP4=
github.com/yuin/goldmark v1.8.2 h1:kEGpgqJXdgbkhcOgBxkC0X0PmoPG1ZyoZ117rDVp4zE=
github.com/yuin/goldmark v1.8.2/go.mod h1:ip/1k0VRfGynBgxOz0yCqHrbZXhcjxyuS66Brc7iBKg=
github.com/yusufpapurcu/wmi v1.2.4 h1:zFUKzehAFReQwLys1b/iSMl+JQGSCSjtVqQn9bBrPo0=
github.com/yusufpapurcu/wmi v1.2.4/go.mod h1:SBZ9tNy3G9/m5Oi98Zks0QjeHVDvuK0qfxQmPyzfmi0=
go.opentelemetry.io/auto/sdk v1.2.1 h1:jXsnJ4Lmnqd11kwkBV2LgLoFMZKizbCi5fNZ/ipaZ64=
go.opentelemetry.io/auto/sdk v1.2.1/go.mod h1:KRTj+aOaElaLi+wW1kO/DZRXwkF4C5xPbEe3ZiIhN7Y=
go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.68.0 h1:CqXxU8VOmDefoh0+ztfGaymYbhdB/tT3zs79QaZTNGY=
go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.68.0/go.mod h1:BuhAPThV8PBHBvg8ZzZ/Ok3idOdhWIodywz2xEcRbJo=
go.opentelemetry.io/otel v1.43.0 h1:mYIM03dnh5zfN7HautFE4ieIig9amkNANT+xcVxAj9I=
go.opentelemetry.io/otel v1.43.0/go.mod h1:JuG+u74mvjvcm8vj8pI5XiHy1zDeoCS2LB1spIq7Ay0=
go.opentelemetry.io/otel/metric v1.43.0 h1:d7638QeInOnuwOONPp4JAOGfbCEpYb+K6DVWvdxGzgM=
go.opentelemetry.io/otel/metric v1.43.0/go.mod h1:RDnPtIxvqlgO8GRW18W6Z/4P462ldprJtfxHxyKd2PY=
go.opentelemetry.io/otel/sdk v1.43.0 h1:pi5mE86i5rTeLXqoF/hhiBtUNcrAGHLKQdhg4h4V9Dg=
go.opentelemetry.io/otel/sdk v1.43.0/go.mod h1:P+IkVU3iWukmiit/Yf9AWvpyRDlUeBaRg6Y+C58QHzg=
go.opentelemetry.io/otel/sdk/metric v1.43.0 h1:S88dyqXjJkuBNLeMcVPRFXpRw2fuwdvfCGLEo89fDkw=
go.opentelemetry.io/otel/sdk/metric v1.43.0/go.mod h1:C/RJtwSEJ5hzTiUz5pXF1kILHStzb9zFlIEe85bhj6A=
go.opentelemetry.io/otel/trace v1.43.0 h1:BkNrHpup+4k4w+ZZ86CZoHHEkohws8AY+WTX09nk+3A=
go.opentelemetry.io/otel/trace v1.43.0/go.mod h1:/QJhyVBUUswCphDVxq+8mld+AvhXZLhe+8WVFxiFff0=
go.yaml.in/yaml/v3 v3.0.4 h1:tfq32ie2Jv2UxXFdLJdh3jXuOzWiL1fo0bu/FbuKpbc=
go.yaml.in/yaml/v3 v3.0.4/go.mod h1:DhzuOOF2ATzADvBadXxruRBLzYTpT36CKvDb3+aBEFg=
golang.org/x/crypto v0.50.0 h1:zO47/JPrL6vsNkINmLoo/PH1gcxpls50DNogFvB5ZGI=
golang.org/x/crypto v0.50.0/go.mod h1:3muZ7vA7PBCE6xgPX7nkzzjiUq87kRItoJQM1Yo8S+Q=
golang.org/x/exp v0.0.0-20260410095643-746e56fc9e2f h1:W3F4c+6OLc6H2lb//N1q4WpJkhzJCK5J6kUi1NTVXfM=
golang.org/x/exp v0.0.0-20260410095643-746e56fc9e2f/go.mod h1:J1xhfL/vlindoeF/aINzNzt2Bket5bjo9sdOYzOsU80=
golang.org/x/image v0.0.0-20191009234506-e7c1f5e7dbb8/go.mod h1:FeLwcggjj3mMvU+oOTbSwawSJRM1uh48EjtB4UJZlP0=
golang.org/x/image v0.39.0 h1:skVYidAEVKgn8lZ602XO75asgXBgLj9G/FE3RbuPFww=
golang.org/x/image v0.39.0/go.mod h1:sIbmppfU+xFLPIG0FoVUTvyBMmgng1/XAMhQ2ft0hpA=
golang.org/x/mod v0.35.0 h1:Ww1D637e6Pg+Zb2KrWfHQUnH2dQRLBQyAtpr/haaJeM=
golang.org/x/mod v0.35.0/go.mod h1:+GwiRhIInF8wPm+4AoT6L0FA1QWAad3OMdTRx4tFYlU=
golang.org/x/net v0.53.0 h1:d+qAbo5L0orcWAr0a9JweQpjXF19LMXJE8Ey7hwOdUA=
golang.org/x/net v0.53.0/go.mod h1:JvMuJH7rrdiCfbeHoo3fCQU24Lf5JJwT9W3sJFulfgs=
golang.org/x/oauth2 v0.36.0 h1:peZ/1z27fi9hUOFCAZaHyrpWG5lwe0RJEEEeH0ThlIs=
golang.org/x/oauth2 v0.36.0/go.mod h1:YDBUJMTkDnJS+A4BP4eZBjCqtokkg1hODuPjwiGPO7Q=
golang.org/x/sync v0.20.0 h1:e0PTpb7pjO8GAtTs2dQ6jYa5BWYlMuX047Dco/pItO4=
golang.org/x/sync v0.20.0/go.mod h1:9xrNwdLfx4jkKbNva9FpL6vEN7evnE43NNNJQ2LF3+0=
golang.org/x/sys v0.0.0-20190916202348-b4ddaad3f8a3/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20201204225414-ed752295db88/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20210616094352-59db8d763f22/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.1.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.43.0 h1:Rlag2XtaFTxp19wS8MXlJwTvoh8ArU6ezoyFsMyCTNI=
golang.org/x/sys v0.43.0/go.mod h1:4GL1E5IUh+htKOUEOaiffhrAeqysfVGipDYzABqnCmw=
golang.org/x/term v0.42.0 h1:UiKe+zDFmJobeJ5ggPwOshJIVt6/Ft0rcfrXZDLWAWY=
golang.org/x/term v0.42.0/go.mod h1:Dq/D+snpsbazcBG5+F9Q1n2rXV8Ma+71xEjTRufARgY=
golang.org/x/text v0.3.0/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ=
golang.org/x/text v0.36.0 h1:JfKh3XmcRPqZPKevfXVpI1wXPTqbkE5f7JA92a55Yxg=
golang.org/x/text v0.36.0/go.mod h1:NIdBknypM8iqVmPiuco0Dh6P5Jcdk8lJL0CUebqK164=
golang.org/x/time v0.15.0 h1:bbrp8t3bGUeFOx08pvsMYRTCVSMk89u4tKbNOZbp88U=
golang.org/x/time v0.15.0/go.mod h1:Y4YMaQmXwGQZoFaVFk4YpCt4FLQMYKZe9oeV/f4MSno=
golang.org/x/tools v0.44.0 h1:UP4ajHPIcuMjT1GqzDWRlalUEoY+uzoZKnhOjbIPD2c=
golang.org/x/tools v0.44.0/go.mod h1:KA0AfVErSdxRZIsOVipbv3rQhVXTnlU6UhKxHd1seDI=
gonum.org/v1/gonum v0.17.0 h1:VbpOemQlsSMrYmn7T2OUvQ4dqxQXU+ouZFQsZOx50z4=
gonum.org/v1/gonum v0.17.0/go.mod h1:El3tOrEuMpv2UdMrbNlKEh9vd86bmQ6vqIcDwxEOc1E=
google.golang.org/api v0.276.0 h1:nVArUtfLEihtW+b0DdcqRGK1xoEm2+ltAihyztq7MKY=
google.golang.org/api v0.276.0/go.mod h1:Fnag/EWUPIcJXuIkP1pjoTgS5vdxlk3eeemL7Do6bvw=
google.golang.org/genai v1.54.0 h1:ZQCa70WMTJDI11FdqWCzGvZ5PanpcpfoO6jl/lrSnGU=
google.golang.org/genai v1.54.0/go.mod h1:A3kkl0nyBjyFlNjgxIwKq70julKbIxpSxqKO5gw/gmk=
google.golang.org/genproto v0.0.0-20260427160629-7cedc36a6bc4 h1:2iMJZntwvmfgtse+s744JY7v7PgEdSBuFYXucvpOHNM=
google.golang.org/genproto v0.0.0-20260427160629-7cedc36a6bc4/go.mod h1:v14kaaboYyXQ1Gsu489Q+Hg/oN4B33mWtuOhF1HCeXA=
google.golang.org/genproto/googleapis/api v0.0.0-20260427160629-7cedc36a6bc4 h1:yOzSCGPx+cp5VO7IxvZ9SBFF7j1tZVcNtlHR2iYKtVo=
google.golang.org/genproto/googleapis/api v0.0.0-20260427160629-7cedc36a6bc4/go.mod h1:Q9HWtNeE7tM9npdIsEvqXj1QJIvVoeAV3rtXtS715Cw=
google.golang.org/genproto/googleapis/rpc v0.0.0-20260427160629-7cedc36a6bc4 h1:tEkOQcXgF6dH1G+MVKZrfpYvozGrzb91k6ha7jireSM=
google.golang.org/genproto/googleapis/rpc v0.0.0-20260427160629-7cedc36a6bc4/go.mod h1:4Hqkh8ycfw05ld/3BWL7rJOSfebL2Q+DVDeRgYgxUU8=
google.golang.org/grpc v1.80.0 h1:Xr6m2WmWZLETvUNvIUmeD5OAagMw3FiKmMlTdViWsHM=
google.golang.org/grpc v1.80.0/go.mod h1:ho/dLnxwi3EDJA4Zghp7k2Ec1+c2jqup0bFkw07bwF4=
google.golang.org/protobuf v1.36.11 h1:fV6ZwhNocDyBLK0dj+fg8ektcVegBBuEolpbTQyBNVE=
google.golang.org/protobuf v1.36.11/go.mod h1:HTf+CrKn2C3g5S8VImy6tdcUvCska2kB7j23XfzDpco=
gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
gopkg.in/check.v1 v1.0.0-20201130134442-10cb98267c6c h1:Hei/4ADfdWqJk1ZMxUNpqntNwaWcugrBjAiHlqqRiVk=
gopkg.in/check.v1 v1.0.0-20201130134442-10cb98267c6c/go.mod h1:JHkPIbrfpd72SG/EVd6muEfDQjcINNoR0C8j2r3qZ4Q=
gopkg.in/yaml.v3 v3.0.1 h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA=
gopkg.in/yaml.v3 v3.0.1/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=
gotest.tools/v3 v3.5.2 h1:7koQfIKdy+I8UTetycgUqXWSDwpgv193Ka+qRsmBY8Q=
gotest.tools/v3 v3.5.2/go.mod h1:LtdLGcnqToBH83WByAAi/wiwSFCArdFIUV/xxN4pcjA=
modernc.org/cc/v4 v4.28.1 h1:XpLbkYVQ24E8tX5u8+yWGvaxerxkR/S4zqxI8ZoSBuc=
modernc.org/cc/v4 v4.28.1/go.mod h1:OnovgIhbbMXMu1aISnJ0wvVD1KnW+cAUJkIrAWh+kVI=
modernc.org/ccgo/v4 v4.33.0 h1:dspBCm75jsj8Y/ufwAMVfe375L2iYdMyQ2QG/v3hL54=
modernc.org/ccgo/v4 v4.33.0/go.mod h1:+RhXBoRYzRwaH21mV/aj6XvQRDtfjcZfAlPMsQo8CR0=
modernc.org/fileutil v1.4.0 h1:j6ZzNTftVS054gi281TyLjHPp6CPHr2KCxEXjEbD6SM=
modernc.org/fileutil v1.4.0/go.mod h1:EqdKFDxiByqxLk8ozOxObDSfcVOv/54xDs/DUHdvCUU=
modernc.org/gc/v2 v2.6.5 h1:nyqdV8q46KvTpZlsw66kWqwXRHdjIlJOhG6kxiV/9xI=
modernc.org/gc/v2 v2.6.5/go.mod h1:YgIahr1ypgfe7chRuJi2gD7DBQiKSLMPgBQe9oIiito=
modernc.org/gc/v3 v3.1.2 h1:ZtDCnhonXSZexk/AYsegNRV1lJGgaNZJuKjJSWKyEqo=
modernc.org/gc/v3 v3.1.2/go.mod h1:HFK/6AGESC7Ex+EZJhJ2Gni6cTaYpSMmU/cT9RmlfYY=
modernc.org/goabi0 v0.2.0 h1:HvEowk7LxcPd0eq6mVOAEMai46V+i7Jrj13t4AzuNks=
modernc.org/goabi0 v0.2.0/go.mod h1:CEFRnnJhKvWT1c1JTI3Avm+tgOWbkOu5oPA8eH8LnMI=
modernc.org/libc v1.72.1 h1:db1xwJ6u1kE3KHTFTTbe2GCrczHPKzlURP0aDC4NGD0=
modernc.org/libc v1.72.1/go.mod h1:HRMiC/PhPGLIPM7GzAFCbI+oSgE3dhZ8FWftmRrHVlY=
modernc.org/mathutil v1.7.1 h1:GCZVGXdaN8gTqB1Mf/usp1Y/hSqgI2vAGGP4jZMCxOU=
modernc.org/mathutil v1.7.1/go.mod h1:4p5IwJITfppl0G4sUEDtCr4DthTaT47/N3aT6MhfgJg=
modernc.org/memory v1.11.0 h1:o4QC8aMQzmcwCK3t3Ux/ZHmwFPzE6hf2Y5LbkRs+hbI=
modernc.org/memory v1.11.0/go.mod h1:/JP4VbVC+K5sU2wZi9bHoq2MAkCnrt2r98UGeSK7Mjw=
modernc.org/opt v0.2.0 h1:tGyef5ApycA7FSEOMraay9SaTk5zmbx7Tu+cJs4QKZg=
modernc.org/opt v0.2.0/go.mod h1:03fq9lsNfvkYSfxrfUhZCWPk1lm4cq4N+Bh//bEtgns=
modernc.org/sortutil v1.2.1 h1:+xyoGf15mM3NMlPDnFqrteY07klSFxLElE2PVuWIJ7w=
modernc.org/sortutil v1.2.1/go.mod h1:7ZI3a3REbai7gzCLcotuw9AC4VZVpYMjDzETGsSMqJE=
modernc.org/sqlite v1.50.0 h1:eMowQSWLK0MeiQTdmz3lqoF5dqclujdlIKeJA11+7oM=
modernc.org/sqlite v1.50.0/go.mod h1:m0w8xhwYUVY3H6pSDwc3gkJ/irZT/0YEXwBlhaxQEew=
modernc.org/strutil v1.2.1 h1:UneZBkQA+DX2Rp35KcM69cSsNES9ly8mQWD71HKlOA0=
modernc.org/strutil v1.2.1/go.mod h1:EHkiggD70koQxjVdSBM3JKM7k6L0FbGE5eymy9i3B9A=
modernc.org/token v1.1.0 h1:Xl7Ap9dKaEs5kLoOQeQmPWevfnk/DM5qcLcYlA8ys6Y=
modernc.org/token v1.1.0/go.mod h1:UGzOrNV1mAFSEB63lOFHIpNRUVMvYTc6yu1SMY/XTDM=
pgregory.net/rapid v1.2.0 h1:keKAYRcjm+e1F0oAuU5F5+YPAWcyxNNRK2wud503Gnk=
pgregory.net/rapid v1.2.0/go.mod h1:PY5XlDGj0+V1FCq0o192FdRhpKHGTRIWBgqjDBTrq04=
+20
View File
@@ -0,0 +1,20 @@
package ai
// ProviderType identifies an AI provider implementation.
type ProviderType string
const (
// ProviderOpenAI is OpenAI's hosted API.
ProviderOpenAI ProviderType = "OPENAI"
// ProviderGemini is Google's Gemini API.
ProviderGemini ProviderType = "GEMINI"
)
// ProviderConfig configures a callable AI provider connection.
type ProviderConfig struct {
ID string
Title string
Type ProviderType
Endpoint string
APIKey string
}
+158
View File
@@ -0,0 +1,158 @@
// Package audio provides audio container/codec helpers for AI providers.
//
// The motivating use case is Gemini transcription: Gemini's audio inputs
// require WAV/MP3/AIFF/AAC/OGG/FLAC, but browser MediaRecorder defaults to
// WebM/Opus. This package converts WebM/Opus into 16-bit PCM WAV using
// pure-Go decoders — no ffmpeg or other system dependency.
package audio
import (
"bytes"
"encoding/binary"
"io"
"strings"
"github.com/at-wat/ebml-go"
"github.com/at-wat/ebml-go/webm"
"github.com/pion/opus"
"github.com/pkg/errors"
)
const (
opusOutputSampleRate = 48000
// maxOpusPacketSamples is Opus's spec maximum: 120 ms at 48 kHz.
maxOpusPacketSamples = 5760
// opusCodecID is the WebM TrackEntry CodecID for an Opus audio track.
opusCodecID = "A_OPUS"
// opusHeadMinLength is the minimum size of the OpusHead identification
// header stored in TrackEntry.CodecPrivate.
opusHeadMinLength = 19
)
// WebMOpusToWAV decodes a WebM/Opus file into 16-bit PCM WAV bytes.
//
// The output is mono or stereo at 48 kHz (Opus's native decode rate),
// regardless of the original encoder's hint. Pre-skip samples declared in
// the OpusHead are discarded to avoid the encoder's startup padding.
//
// The function reads the entire WebM document into memory; callers should
// enforce their own size limits before invoking it.
func WebMOpusToWAV(input []byte) ([]byte, error) {
var doc struct {
Header webm.EBMLHeader `ebml:"EBML"`
Segment webm.Segment `ebml:"Segment"`
}
if err := ebml.Unmarshal(bytes.NewReader(input), &doc); err != nil && !errors.Is(err, io.EOF) {
return nil, errors.Wrap(err, "parse webm")
}
track := findOpusTrack(doc.Segment.Tracks.TrackEntry)
if track == nil {
return nil, errors.New("webm has no Opus audio track")
}
if len(track.CodecPrivate) < opusHeadMinLength {
return nil, errors.Errorf("invalid OpusHead: expected at least %d bytes, got %d", opusHeadMinLength, len(track.CodecPrivate))
}
channels := int(track.Audio.Channels)
if channels < 1 || channels > 2 {
return nil, errors.Errorf("unsupported Opus channel count: %d", channels)
}
preSkip := int(binary.LittleEndian.Uint16(track.CodecPrivate[10:12]))
decoder := opus.NewDecoder()
if err := decoder.Init(opusOutputSampleRate, channels); err != nil {
return nil, errors.Wrap(err, "init opus decoder")
}
pcm := make([]int16, 0, 1<<16)
frame := make([]int16, maxOpusPacketSamples*channels)
decodeBlock := func(block ebml.Block) error {
if block.TrackNumber != track.TrackNumber {
return nil
}
for _, packet := range block.Data {
if len(packet) == 0 {
continue
}
n, err := decoder.DecodeToInt16(packet, frame)
if err != nil {
return errors.Wrap(err, "decode opus packet")
}
pcm = append(pcm, frame[:n*channels]...)
}
return nil
}
for _, cluster := range doc.Segment.Cluster {
for _, sb := range cluster.SimpleBlock {
if err := decodeBlock(sb); err != nil {
return nil, err
}
}
for _, bg := range cluster.BlockGroup {
if err := decodeBlock(bg.Block); err != nil {
return nil, err
}
}
}
skip := min(preSkip*channels, len(pcm))
pcm = pcm[skip:]
return encodeWAV(pcm, opusOutputSampleRate, channels), nil
}
// IsWebMContentType reports whether the MIME type is WebM audio.
// Both "audio/webm" and "audio/webm; codecs=opus" return true.
func IsWebMContentType(contentType string) bool {
contentType = strings.TrimSpace(contentType)
if contentType == "" {
return false
}
if i := strings.IndexByte(contentType, ';'); i >= 0 {
contentType = contentType[:i]
}
return strings.EqualFold(strings.TrimSpace(contentType), "audio/webm")
}
func findOpusTrack(entries []webm.TrackEntry) *webm.TrackEntry {
for i := range entries {
entry := &entries[i]
if entry.CodecID == opusCodecID && entry.Audio != nil {
return entry
}
}
return nil
}
// encodeWAV writes a standard RIFF/WAVE container around 16-bit PCM samples.
// Reference layout: http://soundfile.sapp.org/doc/WaveFormat/
func encodeWAV(samples []int16, sampleRate, channels int) []byte {
const bitsPerSample = 16
const bytesPerSample = bitsPerSample / 8
blockAlign := channels * bytesPerSample
byteRate := sampleRate * blockAlign
dataSize := len(samples) * bytesPerSample
buf := bytes.NewBuffer(make([]byte, 0, 44+dataSize))
buf.WriteString("RIFF")
_ = binary.Write(buf, binary.LittleEndian, uint32(36+dataSize))
buf.WriteString("WAVE")
buf.WriteString("fmt ")
_ = binary.Write(buf, binary.LittleEndian, uint32(16))
_ = binary.Write(buf, binary.LittleEndian, uint16(1)) // PCM
_ = binary.Write(buf, binary.LittleEndian, uint16(channels))
_ = binary.Write(buf, binary.LittleEndian, uint32(sampleRate))
_ = binary.Write(buf, binary.LittleEndian, uint32(byteRate))
_ = binary.Write(buf, binary.LittleEndian, uint16(blockAlign))
_ = binary.Write(buf, binary.LittleEndian, uint16(bitsPerSample))
buf.WriteString("data")
_ = binary.Write(buf, binary.LittleEndian, uint32(dataSize))
_ = binary.Write(buf, binary.LittleEndian, samples)
return buf.Bytes()
}
+48
View File
@@ -0,0 +1,48 @@
package audio
import (
"testing"
"github.com/stretchr/testify/require"
)
func TestIsWebMContentType(t *testing.T) {
cases := []struct {
in string
want bool
}{
{"audio/webm", true},
{"audio/webm;codecs=opus", true},
{"audio/webm; codecs=opus", true},
{"AUDIO/WEBM", true},
{" audio/webm ", true},
{"audio/wav", false},
{"audio/mp4", false},
{"video/webm", false},
{"", false},
{"webm", false},
}
for _, tc := range cases {
t.Run(tc.in, func(t *testing.T) {
require.Equal(t, tc.want, IsWebMContentType(tc.in))
})
}
}
func TestWebMOpusToWAV_RejectsInvalidInput(t *testing.T) {
t.Run("empty", func(t *testing.T) {
_, err := WebMOpusToWAV(nil)
require.Error(t, err)
})
t.Run("not webm", func(t *testing.T) {
_, err := WebMOpusToWAV([]byte("hello world this is not webm"))
require.Error(t, err)
})
t.Run("truncated webm header bytes", func(t *testing.T) {
// Valid EBML magic but no Segment.
_, err := WebMOpusToWAV([]byte{0x1A, 0x45, 0xDF, 0xA3})
require.Error(t, err)
})
}
+41
View File
@@ -0,0 +1,41 @@
// Package audiollm defines the multimodal-audio capability for AI providers.
// Implementations call chat-completions or generate-content style APIs that
// accept audio as input. For deterministic transcription, prefer internal/ai/stt
// where a dedicated STT endpoint exists.
package audiollm
import (
"context"
"io"
)
// Model invokes a multimodal LLM with audio input.
type Model interface {
GenerateFromAudio(ctx context.Context, req Request) (*Response, error)
}
// Request is the input to a multimodal-audio call.
type Request struct {
Audio io.Reader
Size int64
ContentType string
Model string
Instructions string // literal instruction the model is expected to follow
Temperature *float32 // optional; nil leaves the provider default in place
}
// Response is the output of a multimodal-audio call.
type Response struct {
Text string
FinishReason FinishReason
}
// FinishReason describes why the model stopped generating.
type FinishReason string
const (
FinishStop FinishReason = "stop" // model finished normally
FinishLength FinishReason = "length" // truncated by max-tokens
FinishSafety FinishReason = "safety" // safety filter blocked output
FinishOther FinishReason = "other" // anything else, including unknown
)
+202
View File
@@ -0,0 +1,202 @@
// Package gemini implements audiollm.Model against the Gemini generateContent
// endpoint. Used by Memos transcription when the user picks a Gemini provider:
// the handler issues a transcription instruction via audiollm.Request.Instructions.
package gemini
import (
"context"
"io"
"mime"
"net/url"
"strings"
"github.com/pkg/errors"
"google.golang.org/genai"
"github.com/usememos/memos/internal/ai"
"github.com/usememos/memos/internal/ai/audio"
"github.com/usememos/memos/internal/ai/audiollm"
)
const (
defaultEndpoint = "https://generativelanguage.googleapis.com/v1beta"
defaultAPIVersion = "v1beta"
maxInlineSize = 14 * 1024 * 1024
providerName = "Gemini"
)
var supportedContentTypes = map[string]string{
"audio/wav": "audio/wav",
"audio/x-wav": "audio/wav",
"audio/mp3": "audio/mp3",
"audio/mpeg": "audio/mp3",
"audio/aiff": "audio/aiff",
"audio/aac": "audio/aac",
"audio/ogg": "audio/ogg",
"audio/flac": "audio/flac",
"audio/x-flac": "audio/flac",
}
// Model implements audiollm.Model for Gemini generateContent.
type Model struct {
client *genai.Client
}
// New constructs a Model from a provider config.
func New(cfg ai.ProviderConfig, options audiollm.Options) (*Model, error) {
endpoint, err := normalizeEndpoint(cfg.Endpoint)
if err != nil {
return nil, err
}
if cfg.APIKey == "" {
return nil, errors.Errorf("%s API key is required", providerName)
}
baseURL, apiVersion, err := splitEndpoint(endpoint)
if err != nil {
return nil, err
}
httpOptions := genai.HTTPOptions{BaseURL: baseURL, APIVersion: apiVersion}
if options.HTTPClient != nil && options.HTTPClient.Timeout > 0 {
timeout := options.HTTPClient.Timeout
httpOptions.Timeout = &timeout
}
client, err := genai.NewClient(context.Background(), &genai.ClientConfig{
APIKey: cfg.APIKey,
Backend: genai.BackendGeminiAPI,
HTTPClient: options.HTTPClient,
HTTPOptions: httpOptions,
})
if err != nil {
return nil, errors.Wrap(err, "failed to create Gemini client")
}
return &Model{client: client}, nil
}
// GenerateFromAudio calls Gemini generateContent with the audio attached.
func (m *Model) GenerateFromAudio(ctx context.Context, req audiollm.Request) (*audiollm.Response, error) {
if strings.TrimSpace(req.Model) == "" {
return nil, errors.New("model is required")
}
if req.Audio == nil {
return nil, errors.New("audio is required")
}
if strings.TrimSpace(req.Instructions) == "" {
return nil, errors.New("instructions are required")
}
audioBytes, err := io.ReadAll(req.Audio)
if err != nil {
return nil, errors.Wrap(err, "failed to read audio")
}
if len(audioBytes) == 0 {
return nil, errors.New("audio is required")
}
contentType := req.ContentType
if audio.IsWebMContentType(contentType) {
wav, err := audio.WebMOpusToWAV(audioBytes)
if err != nil {
return nil, errors.Wrap(err, "failed to transcode webm audio for Gemini")
}
audioBytes = wav
contentType = "audio/wav"
}
if len(audioBytes) > maxInlineSize {
return nil, errors.Errorf("audio is too large for Gemini inline request; maximum size is %d bytes", maxInlineSize)
}
contentType, err = normalizeContentType(contentType)
if err != nil {
return nil, err
}
cfg := &genai.GenerateContentConfig{}
if req.Temperature != nil {
t := *req.Temperature
cfg.Temperature = &t
}
resp, err := m.client.Models.GenerateContent(ctx, normalizeModelName(req.Model), []*genai.Content{
genai.NewContentFromParts([]*genai.Part{
genai.NewPartFromBytes(audioBytes, contentType),
genai.NewPartFromText(req.Instructions),
}, genai.RoleUser),
}, cfg)
if err != nil {
return nil, errors.Wrap(err, "failed to send Gemini request")
}
return &audiollm.Response{
Text: strings.TrimSpace(resp.Text()),
FinishReason: mapFinishReason(resp),
}, nil
}
func mapFinishReason(resp *genai.GenerateContentResponse) audiollm.FinishReason {
if resp == nil || len(resp.Candidates) == 0 {
return audiollm.FinishOther
}
switch resp.Candidates[0].FinishReason {
case genai.FinishReasonStop:
return audiollm.FinishStop
case genai.FinishReasonMaxTokens:
return audiollm.FinishLength
case genai.FinishReasonSafety,
genai.FinishReasonRecitation,
genai.FinishReasonProhibitedContent,
genai.FinishReasonSPII,
genai.FinishReasonBlocklist,
genai.FinishReasonImageSafety,
genai.FinishReasonImageProhibitedContent,
genai.FinishReasonImageRecitation:
return audiollm.FinishSafety
default:
return audiollm.FinishOther
}
}
func normalizeEndpoint(endpoint string) (string, error) {
endpoint = strings.TrimSpace(endpoint)
if endpoint == "" {
endpoint = defaultEndpoint
}
if _, err := url.ParseRequestURI(endpoint); err != nil {
return "", errors.Wrapf(err, "invalid %s endpoint", providerName)
}
return strings.TrimRight(endpoint, "/"), nil
}
func splitEndpoint(endpoint string) (string, string, error) {
parsed, err := url.Parse(endpoint)
if err != nil {
return "", "", errors.Wrap(err, "invalid Gemini endpoint")
}
path := strings.TrimRight(parsed.Path, "/")
apiVersion := defaultAPIVersion
for _, supported := range []string{"v1alpha", "v1beta", "v1"} {
if path == "/"+supported || strings.HasSuffix(path, "/"+supported) {
apiVersion = supported
parsed.Path = strings.TrimSuffix(path, "/"+supported)
break
}
}
return strings.TrimRight(parsed.String(), "/"), apiVersion, nil
}
func normalizeContentType(contentType string) (string, error) {
mediaType, _, err := mime.ParseMediaType(strings.TrimSpace(contentType))
if err != nil {
return "", errors.Wrap(err, "invalid audio content type")
}
mediaType = strings.ToLower(mediaType)
normalized, ok := supportedContentTypes[mediaType]
if !ok {
return "", errors.Errorf("audio content type %q is not supported by Gemini", mediaType)
}
return normalized, nil
}
func normalizeModelName(model string) string {
return strings.TrimPrefix(strings.TrimSpace(model), "models/")
}
+106
View File
@@ -0,0 +1,106 @@
package gemini_test
import (
"context"
"encoding/base64"
"encoding/json"
"net/http"
"net/http/httptest"
"strings"
"testing"
"time"
"github.com/stretchr/testify/require"
"github.com/usememos/memos/internal/ai"
"github.com/usememos/memos/internal/ai/audiollm"
audiollmgemini "github.com/usememos/memos/internal/ai/audiollm/gemini"
)
func TestGenerateFromAudio(t *testing.T) {
t.Parallel()
server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
require.Equal(t, http.MethodPost, r.Method)
require.Equal(t, "/v1beta/models/gemini-2.5-flash:generateContent", r.URL.Path)
require.Equal(t, "test-key", r.Header.Get("x-goog-api-key"))
require.Equal(t, "application/json", r.Header.Get("Content-Type"))
var request struct {
Contents []struct {
Parts []struct {
Text string `json:"text"`
InlineData *struct {
MIMEType string `json:"mimeType"`
Data string `json:"data"`
} `json:"inlineData"`
} `json:"parts"`
} `json:"contents"`
GenerationConfig map[string]json.Number `json:"generationConfig"`
}
require.NoError(t, json.NewDecoder(r.Body).Decode(&request))
require.Len(t, request.Contents, 1)
require.Len(t, request.Contents[0].Parts, 2)
require.NotNil(t, request.Contents[0].Parts[0].InlineData)
require.Equal(t, "audio/mp3", request.Contents[0].Parts[0].InlineData.MIMEType)
audio, err := base64.StdEncoding.DecodeString(request.Contents[0].Parts[0].InlineData.Data)
require.NoError(t, err)
require.Equal(t, "audio bytes", string(audio))
require.Equal(t, "transcribe please", request.Contents[0].Parts[1].Text)
require.Equal(t, json.Number("0"), request.GenerationConfig["temperature"])
w.Header().Set("Content-Type", "application/json")
require.NoError(t, json.NewEncoder(w).Encode(map[string]any{
"candidates": []map[string]any{
{
"finishReason": "STOP",
"content": map[string]any{
"parts": []map[string]string{{"text": "hello from gemini"}},
},
},
},
}))
}))
defer server.Close()
model, err := audiollmgemini.New(ai.ProviderConfig{
Type: ai.ProviderGemini,
Endpoint: server.URL + "/v1beta",
APIKey: "test-key",
}, audiollm.ApplyOptions(nil))
require.NoError(t, err)
temp := float32(0)
ctx, cancel := context.WithTimeout(context.Background(), 5*time.Second)
defer cancel()
resp, err := model.GenerateFromAudio(ctx, audiollm.Request{
Model: "models/gemini-2.5-flash",
ContentType: "audio/mpeg",
Audio: strings.NewReader("audio bytes"),
Instructions: "transcribe please",
Temperature: &temp,
})
require.NoError(t, err)
require.Equal(t, "hello from gemini", resp.Text)
require.Equal(t, audiollm.FinishStop, resp.FinishReason)
}
func TestGenerateFromAudioRejectsUnsupportedContentType(t *testing.T) {
t.Parallel()
model, err := audiollmgemini.New(ai.ProviderConfig{
Type: ai.ProviderGemini,
Endpoint: "https://example.com/v1beta",
APIKey: "test-key",
}, audiollm.ApplyOptions(nil))
require.NoError(t, err)
_, err = model.GenerateFromAudio(context.Background(), audiollm.Request{
Model: "gemini-2.5-flash",
ContentType: "video/mp4",
Audio: strings.NewReader("video bytes"),
Instructions: "transcribe please",
})
require.Error(t, err)
require.Contains(t, err.Error(), "not supported by Gemini")
}
+34
View File
@@ -0,0 +1,34 @@
package audiollm
import (
"net/http"
"time"
)
const defaultHTTPTimeout = 2 * time.Minute
// Options is the resolved option set passed to provider implementations.
type Options struct {
HTTPClient *http.Client
}
// ModelOption customizes a Model.
type ModelOption func(*Options)
// WithHTTPClient overrides the HTTP client used by the model.
func WithHTTPClient(client *http.Client) ModelOption {
return func(o *Options) {
if client != nil {
o.HTTPClient = client
}
}
}
// ApplyOptions resolves a ModelOption slice into Options with defaults.
func ApplyOptions(opts []ModelOption) Options {
resolved := Options{HTTPClient: &http.Client{Timeout: defaultHTTPTimeout}}
for _, apply := range opts {
apply(&resolved)
}
return resolved
}
+17
View File
@@ -0,0 +1,17 @@
package ai
import "github.com/pkg/errors"
var (
// ErrProviderNotFound indicates that a requested provider ID does not exist.
ErrProviderNotFound = errors.New("AI provider not found")
// ErrCapabilityUnsupported indicates that the provider does not support the requested capability.
ErrCapabilityUnsupported = errors.New("AI provider capability unsupported")
// ErrSTTNotSupported indicates that the provider does not have a dedicated
// speech-to-text endpoint. Use the audiollm package for multimodal audio
// understanding when this is returned.
ErrSTTNotSupported = errors.New("provider does not support speech-to-text capability")
// ErrAudioLLMNotSupported indicates that the provider does not have a
// multimodal-audio LLM available in this codebase.
ErrAudioLLMNotSupported = errors.New("provider does not support multimodal audio capability")
)
+22
View File
@@ -0,0 +1,22 @@
package ai
import "github.com/pkg/errors"
const (
// DefaultOpenAITranscriptionModel is the built-in OpenAI transcription model.
DefaultOpenAITranscriptionModel = "whisper-1"
// DefaultGeminiTranscriptionModel is the built-in Gemini transcription model.
DefaultGeminiTranscriptionModel = "gemini-2.5-flash"
)
// DefaultTranscriptionModel returns the built-in transcription model for a provider.
func DefaultTranscriptionModel(providerType ProviderType) (string, error) {
switch providerType {
case ProviderOpenAI:
return DefaultOpenAITranscriptionModel, nil
case ProviderGemini:
return DefaultGeminiTranscriptionModel, nil
default:
return "", errors.Wrapf(ErrCapabilityUnsupported, "provider type %q", providerType)
}
}
+16
View File
@@ -0,0 +1,16 @@
package ai
import "github.com/pkg/errors"
// FindProvider returns the provider with the given ID.
func FindProvider(providers []ProviderConfig, providerID string) (*ProviderConfig, error) {
if providerID == "" {
return nil, errors.Wrap(ErrProviderNotFound, "provider ID is required")
}
for _, provider := range providers {
if provider.ID == providerID {
return &provider, nil
}
}
return nil, errors.Wrapf(ErrProviderNotFound, "provider ID %q", providerID)
}
+116
View File
@@ -0,0 +1,116 @@
// Package openai implements stt.Transcriber against the OpenAI
// /audio/transcriptions endpoint (and any compatible third-party endpoint
// such as Groq Whisper, faster-whisper self-hosted, or Azure Whisper).
package openai
import (
"context"
"mime"
"net/url"
"strings"
openaisdk "github.com/openai/openai-go/v3"
openaioption "github.com/openai/openai-go/v3/option"
"github.com/pkg/errors"
"github.com/usememos/memos/internal/ai"
"github.com/usememos/memos/internal/ai/stt"
)
const defaultEndpoint = "https://api.openai.com/v1"
// Transcriber implements stt.Transcriber for OpenAI-compatible STT endpoints.
type Transcriber struct {
client openaisdk.Client
}
// New constructs a Transcriber from a provider config.
func New(cfg ai.ProviderConfig, options stt.Options) (*Transcriber, error) {
endpoint, err := normalizeEndpoint(cfg.Endpoint)
if err != nil {
return nil, err
}
if cfg.APIKey == "" {
return nil, errors.New("OpenAI API key is required")
}
return &Transcriber{
client: openaisdk.NewClient(
openaioption.WithAPIKey(cfg.APIKey),
openaioption.WithBaseURL(endpoint),
openaioption.WithHTTPClient(options.HTTPClient),
),
}, nil
}
// Transcribe sends the audio to /audio/transcriptions.
func (t *Transcriber) Transcribe(ctx context.Context, req stt.Request) (*stt.Response, error) {
if strings.TrimSpace(req.Model) == "" {
return nil, errors.New("model is required")
}
if req.Audio == nil {
return nil, errors.New("audio is required")
}
filename, contentType, err := normalizeAudioMetadata(req)
if err != nil {
return nil, err
}
params := openaisdk.AudioTranscriptionNewParams{
File: openaisdk.File(req.Audio, filename, contentType),
Model: openaisdk.AudioModel(req.Model),
ResponseFormat: openaisdk.AudioResponseFormatJSON,
}
if req.Prompt != "" {
params.Prompt = openaisdk.String(req.Prompt)
}
if req.Language != "" {
params.Language = openaisdk.String(req.Language)
}
resp, err := t.client.Audio.Transcriptions.New(ctx, params)
if err != nil {
return nil, errors.Wrap(err, "failed to send OpenAI transcription request")
}
return &stt.Response{
Text: resp.Text,
Language: resp.Language,
}, nil
}
func normalizeEndpoint(endpoint string) (string, error) {
endpoint = strings.TrimSpace(endpoint)
if endpoint == "" {
endpoint = defaultEndpoint
}
if _, err := url.ParseRequestURI(endpoint); err != nil {
return "", errors.Wrap(err, "invalid OpenAI endpoint")
}
return strings.TrimRight(endpoint, "/"), nil
}
func normalizeAudioMetadata(req stt.Request) (string, string, error) {
filename := strings.TrimSpace(req.Filename)
if filename == "" {
filename = "audio"
}
contentType := strings.TrimSpace(req.ContentType)
if contentType == "" {
contentType = "application/octet-stream"
} else {
mediaType, _, err := mime.ParseMediaType(contentType)
if err != nil {
return "", "", errors.Wrap(err, "invalid audio content type")
}
contentType = mediaType
}
return sanitizeFilename(filename), contentType, nil
}
func sanitizeFilename(filename string) string {
filename = strings.NewReplacer("\r", "_", "\n", "_").Replace(filename)
if strings.TrimSpace(filename) == "" {
return "audio"
}
return filename
}
+68
View File
@@ -0,0 +1,68 @@
package openai_test
import (
"context"
"encoding/json"
"net/http"
"net/http/httptest"
"strings"
"testing"
"time"
"github.com/stretchr/testify/require"
"github.com/usememos/memos/internal/ai"
"github.com/usememos/memos/internal/ai/stt"
sttopenai "github.com/usememos/memos/internal/ai/stt/openai"
)
func TestTranscribe(t *testing.T) {
t.Parallel()
server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
require.Equal(t, http.MethodPost, r.Method)
require.Equal(t, "/audio/transcriptions", r.URL.Path)
require.Equal(t, "Bearer test-key", r.Header.Get("Authorization"))
require.NoError(t, r.ParseMultipartForm(10<<20))
require.Equal(t, "gpt-4o-transcribe", r.FormValue("model"))
require.Equal(t, "json", r.FormValue("response_format"))
require.Equal(t, "domain words", r.FormValue("prompt"))
require.Equal(t, "en", r.FormValue("language"))
file, header, err := r.FormFile("file")
require.NoError(t, err)
defer file.Close()
require.Equal(t, "voice.wav", header.Filename)
require.Equal(t, "audio/wav", header.Header.Get("Content-Type"))
w.Header().Set("Content-Type", "application/json")
require.NoError(t, json.NewEncoder(w).Encode(map[string]any{
"text": "hello world",
"language": "en",
"duration": 1.5,
}))
}))
defer server.Close()
transcriber, err := sttopenai.New(ai.ProviderConfig{
Type: ai.ProviderOpenAI,
Endpoint: server.URL,
APIKey: "test-key",
}, stt.ApplyOptions(nil))
require.NoError(t, err)
ctx, cancel := context.WithTimeout(context.Background(), 5*time.Second)
defer cancel()
response, err := transcriber.Transcribe(ctx, stt.Request{
Model: "gpt-4o-transcribe",
Filename: "voice.wav",
ContentType: "audio/wav",
Audio: strings.NewReader("RIFF"),
Prompt: "domain words",
Language: "en",
})
require.NoError(t, err)
require.Equal(t, "hello world", response.Text)
require.Equal(t, "en", response.Language)
// Note: Duration intentionally omitted from stt.Response — not exposed in the new contract.
}
+34
View File
@@ -0,0 +1,34 @@
package stt
import (
"net/http"
"time"
)
const defaultHTTPTimeout = 2 * time.Minute
// Options is the resolved option set passed to provider implementations.
type Options struct {
HTTPClient *http.Client
}
// TranscriberOption customizes a Transcriber.
type TranscriberOption func(*Options)
// WithHTTPClient overrides the HTTP client used by the transcriber.
func WithHTTPClient(client *http.Client) TranscriberOption {
return func(o *Options) {
if client != nil {
o.HTTPClient = client
}
}
}
// ApplyOptions resolves a TranscriberOption slice into Options with defaults.
func ApplyOptions(opts []TranscriberOption) Options {
resolved := Options{HTTPClient: &http.Client{Timeout: defaultHTTPTimeout}}
for _, apply := range opts {
apply(&resolved)
}
return resolved
}
+41
View File
@@ -0,0 +1,41 @@
// Package stt defines the speech-to-text capability for AI providers.
// Implementations call dedicated STT endpoints (e.g. OpenAI /audio/transcriptions)
// and return deterministic transcription output. For multimodal LLMs that
// happen to accept audio input, see internal/ai/audiollm.
package stt
import (
"context"
"io"
)
// Transcriber transcribes audio to text using a provider's dedicated STT endpoint.
type Transcriber interface {
Transcribe(ctx context.Context, req Request) (*Response, error)
}
// Request is the input to a transcription call.
type Request struct {
Audio io.Reader
Size int64
Filename string
ContentType string // IANA media type, e.g. "audio/wav"
Model string // provider-specific model id (e.g. "whisper-1", "gpt-4o-transcribe")
Prompt string // soft spelling/vocabulary hint (Whisper "prompt" parameter)
Language string // ISO 639-1, optional
}
// Response is the output of a transcription call.
type Response struct {
Text string
Language string // empty if provider did not return it
Segments []Segment // empty unless provider returned timestamps
}
// Segment is a timestamped portion of the transcript.
type Segment struct {
Text string
Start float64
End float64
Speaker string // empty unless using a diarization-capable model
}
+7
View File
@@ -0,0 +1,7 @@
package base
import "regexp"
var (
UIDMatcher = regexp.MustCompile("^[a-zA-Z0-9]([a-zA-Z0-9-]{0,34}[a-zA-Z0-9])?$")
)
+37
View File
@@ -0,0 +1,37 @@
package base
import (
"testing"
)
func TestUIDMatcher(t *testing.T) {
tests := []struct {
input string
expected bool
}{
{"", false},
{"-abc123", false},
{"012345678901234567890123456789", true},
{"1abc-123", true},
{"A123B456C789", true},
{"a", true},
{"ab", true},
{"a*b&c", false},
{"a--b", true},
{"a-1b-2c", true},
{"a1234567890123456789012345678901", true},
{"abc123", true},
{"abc123-", false},
{"123e4567-e89b-12d3-a456-426614174000", true}, // UUID v4 from IDP
{"a123456789012345678901234567890123456", false}, // 37 characters (too long)
}
for _, test := range tests {
t.Run(test.input, func(*testing.T) {
result := UIDMatcher.MatchString(test.input)
if result != test.expected {
t.Errorf("For input '%s', expected %v but got %v", test.input, test.expected, result)
}
})
}
}
+507
View File
@@ -0,0 +1,507 @@
# Email Plugin
SMTP email sending functionality for self-hosted Memos instances.
## Overview
This plugin provides a simple, reliable email sending interface following industry-standard SMTP protocols. It's designed for self-hosted environments where instance administrators configure their own email service, similar to platforms like GitHub, GitLab, and Discourse.
## Features
- Standard SMTP protocol support
- TLS/STARTTLS and SSL/TLS encryption
- HTML and plain text emails
- Multiple recipients (To, Cc, Bcc)
- Synchronous and asynchronous sending
- Detailed error reporting with context
- Works with all major email providers
- Reply-To header support
- RFC 5322 compliant message formatting
## Quick Start
### 1. Configure SMTP Settings
```go
import "github.com/usememos/memos/internal/email"
config := &email.Config{
SMTPHost: "smtp.gmail.com",
SMTPPort: 587,
SMTPUsername: "your-email@gmail.com",
SMTPPassword: "your-app-password",
FromEmail: "noreply@yourdomain.com",
FromName: "Memos",
UseTLS: true,
}
```
### 2. Create and Send Email
```go
message := &email.Message{
To: []string{"user@example.com"},
Subject: "Welcome to Memos!",
Body: "Thanks for signing up.",
IsHTML: false,
}
// Synchronous send (waits for result)
err := email.Send(config, message)
if err != nil {
log.Printf("Failed to send email: %v", err)
}
// Asynchronous send (returns immediately)
email.SendAsync(config, message)
```
## Provider Configuration
### Gmail
Requires an [App Password](https://support.google.com/accounts/answer/185833) (2FA must be enabled):
```go
config := &email.Config{
SMTPHost: "smtp.gmail.com",
SMTPPort: 587,
SMTPUsername: "your-email@gmail.com",
SMTPPassword: "your-16-char-app-password",
FromEmail: "your-email@gmail.com",
FromName: "Memos",
UseTLS: true,
}
```
**Alternative (SSL):**
```go
config := &email.Config{
SMTPHost: "smtp.gmail.com",
SMTPPort: 465,
SMTPUsername: "your-email@gmail.com",
SMTPPassword: "your-16-char-app-password",
FromEmail: "your-email@gmail.com",
FromName: "Memos",
UseSSL: true,
}
```
### SendGrid
```go
config := &email.Config{
SMTPHost: "smtp.sendgrid.net",
SMTPPort: 587,
SMTPUsername: "apikey",
SMTPPassword: "your-sendgrid-api-key",
FromEmail: "noreply@yourdomain.com",
FromName: "Memos",
UseTLS: true,
}
```
### AWS SES
```go
config := &email.Config{
SMTPHost: "email-smtp.us-east-1.amazonaws.com",
SMTPPort: 587,
SMTPUsername: "your-smtp-username",
SMTPPassword: "your-smtp-password",
FromEmail: "verified@yourdomain.com",
FromName: "Memos",
UseTLS: true,
}
```
**Note:** Replace `us-east-1` with your AWS region. Email must be verified in SES.
### Mailgun
```go
config := &email.Config{
SMTPHost: "smtp.mailgun.org",
SMTPPort: 587,
SMTPUsername: "postmaster@yourdomain.com",
SMTPPassword: "your-mailgun-smtp-password",
FromEmail: "noreply@yourdomain.com",
FromName: "Memos",
UseTLS: true,
}
```
### Self-Hosted SMTP (Postfix, Exim, etc.)
```go
config := &email.Config{
SMTPHost: "mail.yourdomain.com",
SMTPPort: 587,
SMTPUsername: "username",
SMTPPassword: "password",
FromEmail: "noreply@yourdomain.com",
FromName: "Memos",
UseTLS: true,
}
```
## HTML Emails
```go
message := &email.Message{
To: []string{"user@example.com"},
Subject: "Welcome to Memos!",
Body: `
<!DOCTYPE html>
<html>
<head>
<meta charset="UTF-8">
</head>
<body style="font-family: Arial, sans-serif;">
<h1 style="color: #333;">Welcome to Memos!</h1>
<p>We're excited to have you on board.</p>
<a href="https://yourdomain.com" style="background-color: #4CAF50; color: white; padding: 10px 20px; text-decoration: none; border-radius: 5px;">Get Started</a>
</body>
</html>
`,
IsHTML: true,
}
email.Send(config, message)
```
## Multiple Recipients
```go
message := &email.Message{
To: []string{"user1@example.com", "user2@example.com"},
Cc: []string{"manager@example.com"},
Bcc: []string{"admin@example.com"},
Subject: "Team Update",
Body: "Important team announcement...",
ReplyTo: "support@yourdomain.com",
}
email.Send(config, message)
```
## Testing
### Run Tests
```bash
# All tests
go test ./internal/email/... -v
# With coverage
go test ./internal/email/... -v -cover
# With race detector
go test ./internal/email/... -race
```
### Manual Testing
Create a simple test program:
```go
package main
import (
"log"
"github.com/usememos/memos/internal/email"
)
func main() {
config := &email.Config{
SMTPHost: "smtp.gmail.com",
SMTPPort: 587,
SMTPUsername: "your-email@gmail.com",
SMTPPassword: "your-app-password",
FromEmail: "your-email@gmail.com",
FromName: "Test",
UseTLS: true,
}
message := &email.Message{
To: []string{"recipient@example.com"},
Subject: "Test Email",
Body: "This is a test email from Memos email plugin.",
}
if err := email.Send(config, message); err != nil {
log.Fatalf("Failed to send email: %v", err)
}
log.Println("Email sent successfully!")
}
```
## Security Best Practices
### 1. Use TLS/SSL Encryption
Always enable encryption in production:
```go
// STARTTLS (port 587) - Recommended
config.UseTLS = true
// SSL/TLS (port 465)
config.UseSSL = true
```
### 2. Secure Credential Storage
Never hardcode credentials. Use environment variables:
```go
import "os"
config := &email.Config{
SMTPHost: os.Getenv("SMTP_HOST"),
SMTPPort: 587,
SMTPUsername: os.Getenv("SMTP_USERNAME"),
SMTPPassword: os.Getenv("SMTP_PASSWORD"),
FromEmail: os.Getenv("SMTP_FROM_EMAIL"),
UseTLS: true,
}
```
### 3. Use App-Specific Passwords
For Gmail and similar services, use app passwords instead of your main account password.
### 4. Validate and Sanitize Input
Always validate email addresses and sanitize content:
```go
// Validate before sending
if err := message.Validate(); err != nil {
return err
}
```
### 5. Implement Rate Limiting
Prevent abuse by limiting email sending:
```go
// Example using golang.org/x/time/rate
limiter := rate.NewLimiter(rate.Every(time.Second), 10) // 10 emails per second
if !limiter.Allow() {
return errors.New("rate limit exceeded")
}
```
### 6. Monitor and Log
Log email sending activity for security monitoring:
```go
if err := email.Send(config, message); err != nil {
slog.Error("Email send failed",
slog.String("recipient", message.To[0]),
slog.Any("error", err))
}
```
## Common Ports
| Port | Protocol | Security | Use Case |
|------|----------|----------|----------|
| **587** | SMTP + STARTTLS | Encrypted | **Recommended** for most providers |
| **465** | SMTP over SSL/TLS | Encrypted | Alternative secure option |
| **25** | SMTP | Unencrypted | Legacy, often blocked by ISPs |
| **2525** | SMTP + STARTTLS | Encrypted | Alternative when 587 is blocked |
**Port 587 (STARTTLS)** is the recommended standard for modern SMTP:
```go
config := &email.Config{
SMTPPort: 587,
UseTLS: true,
}
```
**Port 465 (SSL/TLS)** is the alternative:
```go
config := &email.Config{
SMTPPort: 465,
UseSSL: true,
}
```
## Error Handling
The package provides detailed, contextual errors:
```go
err := email.Send(config, message)
if err != nil {
// Error messages include context:
switch {
case strings.Contains(err.Error(), "invalid email configuration"):
// Configuration error (missing host, invalid port, etc.)
log.Printf("Configuration error: %v", err)
case strings.Contains(err.Error(), "invalid email message"):
// Message validation error (missing recipients, subject, body)
log.Printf("Message error: %v", err)
case strings.Contains(err.Error(), "authentication failed"):
// SMTP authentication failed (wrong credentials)
log.Printf("Auth error: %v", err)
case strings.Contains(err.Error(), "failed to connect"):
// Network/connection error
log.Printf("Connection error: %v", err)
case strings.Contains(err.Error(), "recipient rejected"):
// SMTP server rejected recipient
log.Printf("Recipient error: %v", err)
default:
log.Printf("Unknown error: %v", err)
}
}
```
### Common Error Messages
```
❌ "invalid email configuration: SMTP host is required"
→ Fix: Set config.SMTPHost
❌ "invalid email configuration: SMTP port must be between 1 and 65535"
→ Fix: Set valid config.SMTPPort (usually 587 or 465)
❌ "invalid email configuration: from email is required"
→ Fix: Set config.FromEmail
❌ "invalid email message: at least one recipient is required"
→ Fix: Set message.To with at least one email address
❌ "invalid email message: subject is required"
→ Fix: Set message.Subject
❌ "invalid email message: body is required"
→ Fix: Set message.Body
❌ "SMTP authentication failed"
→ Fix: Check credentials (username/password)
❌ "failed to connect to SMTP server"
→ Fix: Verify host/port, check firewall, ensure TLS/SSL settings match server
```
### Async Error Handling
For async sending, errors are logged automatically:
```go
email.SendAsync(config, message)
// Errors logged as:
// [WARN] Failed to send email asynchronously recipients=user@example.com error=...
```
## Dependencies
### Required
- **Go 1.25+**
- Standard library: `net/smtp`, `crypto/tls`
- `github.com/pkg/errors` - Error wrapping with context
### No External SMTP Libraries
This plugin uses Go's standard `net/smtp` library for maximum compatibility and minimal dependencies.
## API Reference
### Types
#### `Config`
```go
type Config struct {
SMTPHost string // SMTP server hostname
SMTPPort int // SMTP server port
SMTPUsername string // SMTP auth username
SMTPPassword string // SMTP auth password
FromEmail string // From email address
FromName string // From display name (optional)
UseTLS bool // Enable STARTTLS (port 587)
UseSSL bool // Enable SSL/TLS (port 465)
}
```
#### `Message`
```go
type Message struct {
To []string // Recipients
Cc []string // CC recipients (optional)
Bcc []string // BCC recipients (optional)
Subject string // Email subject
Body string // Email body (plain text or HTML)
IsHTML bool // true for HTML, false for plain text
ReplyTo string // Reply-To address (optional)
}
```
### Functions
#### `Send(config *Config, message *Message) error`
Sends an email synchronously. Blocks until email is sent or error occurs.
#### `SendAsync(config *Config, message *Message)`
Sends an email asynchronously in a goroutine. Returns immediately. Errors are logged.
#### `NewClient(config *Config) *Client`
Creates a new SMTP client for advanced usage.
#### `Client.Send(message *Message) error`
Sends email using the client's configuration.
## Architecture
```
internal/email/
├── config.go # SMTP configuration types
├── message.go # Email message types and formatting
├── client.go # SMTP client implementation
├── email.go # High-level Send/SendAsync API
├── doc.go # Package documentation
└── *_test.go # Unit tests
```
## License
Part of the Memos project. See main repository for license details.
## Contributing
This package follows the Memos contribution guidelines. Please ensure:
1. All code is tested (TDD approach)
2. Tests pass: `go test ./internal/email/... -v`
3. Code is formatted: `go fmt ./internal/email/...`
4. No linting errors: `golangci-lint run ./internal/email/...`
## Support
For issues and questions:
- Memos GitHub Issues: https://github.com/usememos/memos/issues
- Memos Documentation: https://usememos.com/docs
## Roadmap
Future enhancements may include:
- Email template system
- Attachment support
- Inline image embedding
- Email queuing system
- Delivery status tracking
- Bounce handling
+173
View File
@@ -0,0 +1,173 @@
package email
import (
"crypto/tls"
"net"
"net/smtp"
"time"
"github.com/pkg/errors"
)
const smtpOperationTimeout = 15 * time.Second
// Client represents an SMTP email client.
type Client struct {
config *Config
}
// NewClient creates a new email client with the given configuration.
func NewClient(config *Config) *Client {
return &Client{
config: config,
}
}
// validateConfig validates the client configuration.
func (c *Client) validateConfig() error {
if c.config == nil {
return errors.New("email configuration is required")
}
return c.config.Validate()
}
// createAuth creates an SMTP auth mechanism if credentials are provided.
func (c *Client) createAuth() smtp.Auth {
if c.config.SMTPUsername == "" && c.config.SMTPPassword == "" {
return nil
}
return smtp.PlainAuth("", c.config.SMTPUsername, c.config.SMTPPassword, c.config.SMTPHost)
}
// createTLSConfig creates a TLS configuration for secure connections.
func (c *Client) createTLSConfig() *tls.Config {
return &tls.Config{
ServerName: c.config.SMTPHost,
MinVersion: tls.VersionTLS12,
}
}
// Send sends an email message via SMTP.
func (c *Client) Send(message *Message) error {
// Validate configuration
if err := c.validateConfig(); err != nil {
return errors.Wrap(err, "invalid email configuration")
}
// Validate message
if message == nil {
return errors.New("message is required")
}
if err := message.Validate(); err != nil {
return errors.Wrap(err, "invalid email message")
}
// Format the message
body := message.Format(c.config.FromEmail, c.config.FromName)
// Get all recipients
recipients := message.GetAllRecipients()
// Create auth
auth := c.createAuth()
// Send based on encryption type
if c.config.UseSSL {
return c.sendWithSSL(auth, recipients, body)
}
return c.sendWithTLS(auth, recipients, body)
}
// sendWithTLS sends email using STARTTLS (port 587).
func (c *Client) sendWithTLS(auth smtp.Auth, recipients []string, body string) error {
serverAddr := c.config.GetServerAddress()
dialer := &net.Dialer{Timeout: smtpOperationTimeout}
conn, err := dialer.Dial("tcp", serverAddr)
if err != nil {
return errors.Wrapf(err, "failed to connect to SMTP server: %s", serverAddr)
}
defer conn.Close()
if err := conn.SetDeadline(time.Now().Add(smtpOperationTimeout)); err != nil {
return errors.Wrap(err, "failed to set SMTP connection deadline")
}
client, err := smtp.NewClient(conn, c.config.SMTPHost)
if err != nil {
return errors.Wrap(err, "failed to create SMTP client")
}
defer client.Quit()
if c.config.UseTLS {
if ok, _ := client.Extension("STARTTLS"); !ok {
return errors.New("SMTP server does not support STARTTLS")
}
if err := client.StartTLS(c.createTLSConfig()); err != nil {
return errors.Wrap(err, "failed to start SMTP STARTTLS")
}
}
return c.sendWithClient(client, auth, recipients, body)
}
// sendWithSSL sends email using SSL/TLS (port 465).
func (c *Client) sendWithSSL(auth smtp.Auth, recipients []string, body string) error {
serverAddr := c.config.GetServerAddress()
// Create TLS connection
tlsConfig := c.createTLSConfig()
dialer := &net.Dialer{Timeout: smtpOperationTimeout}
conn, err := tls.DialWithDialer(dialer, "tcp", serverAddr, tlsConfig)
if err != nil {
return errors.Wrapf(err, "failed to connect to SMTP server with SSL: %s", serverAddr)
}
defer conn.Close()
if err := conn.SetDeadline(time.Now().Add(smtpOperationTimeout)); err != nil {
return errors.Wrap(err, "failed to set SMTP connection deadline")
}
// Create SMTP client
client, err := smtp.NewClient(conn, c.config.SMTPHost)
if err != nil {
return errors.Wrap(err, "failed to create SMTP client")
}
defer client.Quit()
return c.sendWithClient(client, auth, recipients, body)
}
func (c *Client) sendWithClient(client *smtp.Client, auth smtp.Auth, recipients []string, body string) error {
if auth != nil {
if err := client.Auth(auth); err != nil {
return errors.Wrap(err, "SMTP authentication failed")
}
}
// Set sender
if err := client.Mail(c.config.FromEmail); err != nil {
return errors.Wrap(err, "failed to set sender")
}
// Set recipients
for _, recipient := range recipients {
if err := client.Rcpt(recipient); err != nil {
return errors.Wrapf(err, "failed to set recipient: %s", recipient)
}
}
// Send message body
writer, err := client.Data()
if err != nil {
return errors.Wrap(err, "failed to send DATA command")
}
if _, err := writer.Write([]byte(body)); err != nil {
return errors.Wrap(err, "failed to write message body")
}
if err := writer.Close(); err != nil {
return errors.Wrap(err, "failed to close message writer")
}
return nil
}
+121
View File
@@ -0,0 +1,121 @@
package email
import (
"testing"
"github.com/stretchr/testify/assert"
)
func TestNewClient(t *testing.T) {
config := &Config{
SMTPHost: "smtp.example.com",
SMTPPort: 587,
SMTPUsername: "user@example.com",
SMTPPassword: "password",
FromEmail: "noreply@example.com",
FromName: "Test App",
UseTLS: true,
}
client := NewClient(config)
assert.NotNil(t, client)
assert.Equal(t, config, client.config)
}
func TestClientValidateConfig(t *testing.T) {
tests := []struct {
name string
config *Config
wantErr bool
}{
{
name: "valid config",
config: &Config{
SMTPHost: "smtp.example.com",
SMTPPort: 587,
FromEmail: "test@example.com",
},
wantErr: false,
},
{
name: "nil config",
config: nil,
wantErr: true,
},
{
name: "invalid config",
config: &Config{
SMTPHost: "",
SMTPPort: 587,
},
wantErr: true,
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
client := NewClient(tt.config)
err := client.validateConfig()
if tt.wantErr {
assert.Error(t, err)
} else {
assert.NoError(t, err)
}
})
}
}
func TestClientSendValidation(t *testing.T) {
config := &Config{
SMTPHost: "smtp.example.com",
SMTPPort: 587,
FromEmail: "test@example.com",
}
client := NewClient(config)
tests := []struct {
name string
message *Message
wantErr bool
}{
{
name: "valid message",
message: &Message{
To: []string{"recipient@example.com"},
Subject: "Test",
Body: "Test body",
},
wantErr: false, // Will fail on actual send, but passes validation
},
{
name: "nil message",
message: nil,
wantErr: true,
},
{
name: "invalid message",
message: &Message{
To: []string{},
Subject: "Test",
Body: "Test",
},
wantErr: true,
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
err := client.Send(tt.message)
// We expect validation errors for invalid messages
// For valid messages, we'll get connection errors (which is expected in tests)
if tt.wantErr {
assert.Error(t, err)
// Should fail validation before attempting connection
assert.NotContains(t, err.Error(), "dial")
}
// Note: We don't assert NoError for valid messages because
// we don't have a real SMTP server in tests
})
}
}
+47
View File
@@ -0,0 +1,47 @@
package email
import (
"fmt"
"github.com/pkg/errors"
)
// Config represents the SMTP configuration for email sending.
// These settings should be provided by the self-hosted instance administrator.
type Config struct {
// SMTPHost is the SMTP server hostname (e.g., "smtp.gmail.com")
SMTPHost string
// SMTPPort is the SMTP server port (common: 587 for TLS, 465 for SSL, 25 for unencrypted)
SMTPPort int
// SMTPUsername is the SMTP authentication username (usually the email address)
SMTPUsername string
// SMTPPassword is the SMTP authentication password or app-specific password
SMTPPassword string
// FromEmail is the email address that will appear in the "From" field
FromEmail string
// FromName is the display name that will appear in the "From" field
FromName string
// UseTLS enables STARTTLS encryption (recommended for port 587)
UseTLS bool
// UseSSL enables SSL/TLS encryption (for port 465)
UseSSL bool
}
// Validate checks if the configuration is valid.
func (c *Config) Validate() error {
if c.SMTPHost == "" {
return errors.New("SMTP host is required")
}
if c.SMTPPort <= 0 || c.SMTPPort > 65535 {
return errors.New("SMTP port must be between 1 and 65535")
}
if c.FromEmail == "" {
return errors.New("from email is required")
}
return nil
}
// GetServerAddress returns the SMTP server address in the format "host:port".
func (c *Config) GetServerAddress() string {
return fmt.Sprintf("%s:%d", c.SMTPHost, c.SMTPPort)
}
+80
View File
@@ -0,0 +1,80 @@
package email
import (
"testing"
"github.com/stretchr/testify/assert"
)
func TestConfigValidation(t *testing.T) {
tests := []struct {
name string
config *Config
wantErr bool
}{
{
name: "valid config",
config: &Config{
SMTPHost: "smtp.gmail.com",
SMTPPort: 587,
SMTPUsername: "user@example.com",
SMTPPassword: "password",
FromEmail: "noreply@example.com",
FromName: "Memos",
},
wantErr: false,
},
{
name: "missing host",
config: &Config{
SMTPPort: 587,
SMTPUsername: "user@example.com",
SMTPPassword: "password",
FromEmail: "noreply@example.com",
},
wantErr: true,
},
{
name: "invalid port",
config: &Config{
SMTPHost: "smtp.gmail.com",
SMTPPort: 0,
SMTPUsername: "user@example.com",
SMTPPassword: "password",
FromEmail: "noreply@example.com",
},
wantErr: true,
},
{
name: "missing from email",
config: &Config{
SMTPHost: "smtp.gmail.com",
SMTPPort: 587,
SMTPUsername: "user@example.com",
SMTPPassword: "password",
},
wantErr: true,
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
err := tt.config.Validate()
if tt.wantErr {
assert.Error(t, err)
} else {
assert.NoError(t, err)
}
})
}
}
func TestConfigGetServerAddress(t *testing.T) {
config := &Config{
SMTPHost: "smtp.gmail.com",
SMTPPort: 587,
}
expected := "smtp.gmail.com:587"
assert.Equal(t, expected, config.GetServerAddress())
}
+98
View File
@@ -0,0 +1,98 @@
// Package email provides SMTP email sending functionality for self-hosted Memos instances.
//
// This package is designed for self-hosted environments where instance administrators
// configure their own SMTP servers. It follows industry-standard patterns used by
// platforms like GitHub, GitLab, and Discourse.
//
// # Configuration
//
// The package requires SMTP server configuration provided by the instance administrator:
//
// config := &email.Config{
// SMTPHost: "smtp.gmail.com",
// SMTPPort: 587,
// SMTPUsername: "your-email@gmail.com",
// SMTPPassword: "your-app-password",
// FromEmail: "noreply@yourdomain.com",
// FromName: "Memos Notifications",
// UseTLS: true,
// }
//
// # Common SMTP Settings
//
// Gmail (requires App Password):
// - Host: smtp.gmail.com
// - Port: 587 (TLS) or 465 (SSL)
// - Username: your-email@gmail.com
// - UseTLS: true (for port 587) or UseSSL: true (for port 465)
//
// SendGrid:
// - Host: smtp.sendgrid.net
// - Port: 587
// - Username: apikey
// - Password: your-sendgrid-api-key
// - UseTLS: true
//
// AWS SES:
// - Host: email-smtp.[region].amazonaws.com
// - Port: 587
// - Username: your-smtp-username
// - Password: your-smtp-password
// - UseTLS: true
//
// Mailgun:
// - Host: smtp.mailgun.org
// - Port: 587
// - Username: your-mailgun-smtp-username
// - Password: your-mailgun-smtp-password
// - UseTLS: true
//
// # Sending Email
//
// Synchronous (waits for completion):
//
// message := &email.Message{
// To: []string{"user@example.com"},
// Subject: "Welcome to Memos",
// Body: "Thank you for joining!",
// IsHTML: false,
// }
//
// err := email.Send(config, message)
// if err != nil {
// // Handle error
// }
//
// Asynchronous (returns immediately):
//
// email.SendAsync(config, message)
// // Errors are logged but not returned
//
// # HTML Email
//
// message := &email.Message{
// To: []string{"user@example.com"},
// Subject: "Welcome!",
// Body: "<html><body><h1>Welcome to Memos!</h1></body></html>",
// IsHTML: true,
// }
//
// # Security Considerations
//
// - Always use TLS (port 587) or SSL (port 465) for production
// - Store SMTP credentials securely (environment variables or secrets management)
// - Use app-specific passwords for services like Gmail
// - Validate and sanitize email content to prevent injection attacks
// - Rate limit email sending to prevent abuse
//
// # Error Handling
//
// The package returns descriptive errors for common issues:
// - Configuration validation errors (missing host, invalid port, etc.)
// - Message validation errors (missing recipients, subject, or body)
// - Connection errors (cannot reach SMTP server)
// - Authentication errors (invalid credentials)
// - SMTP protocol errors (recipient rejected, etc.)
//
// All errors are wrapped with context using github.com/pkg/errors for better debugging.
package email
+61
View File
@@ -0,0 +1,61 @@
package email
import (
"log/slog"
"github.com/pkg/errors"
)
type asyncEmailRequest struct {
config *Config
message *Message
}
var asyncEmailQueue = make(chan asyncEmailRequest, 128)
func init() {
for range 2 {
go func() {
for request := range asyncEmailQueue {
if err := Send(request.config, request.message); err != nil {
recipients := ""
if request.message != nil && len(request.message.To) > 0 {
recipients = request.message.To[0]
if len(request.message.To) > 1 {
recipients += " and others"
}
}
slog.Warn("Failed to send email asynchronously",
slog.String("recipients", recipients),
slog.Any("error", err))
}
}
}()
}
}
// Send sends an email synchronously.
// Returns an error if the email fails to send.
func Send(config *Config, message *Message) error {
if config == nil {
return errors.New("email configuration is required")
}
if message == nil {
return errors.New("email message is required")
}
client := NewClient(config)
return client.Send(message)
}
// SendAsync sends an email asynchronously.
// It enqueues the message for bounded asynchronous sending and does not wait for the response.
// Any errors are logged but not returned.
func SendAsync(config *Config, message *Message) {
select {
case asyncEmailQueue <- asyncEmailRequest{config: config, message: message}:
default:
slog.Warn("Dropped email because the async queue is full")
}
}
+127
View File
@@ -0,0 +1,127 @@
package email
import (
"testing"
"time"
"github.com/stretchr/testify/assert"
"golang.org/x/sync/errgroup"
)
func TestSend(t *testing.T) {
config := &Config{
SMTPHost: "smtp.example.com",
SMTPPort: 587,
FromEmail: "test@example.com",
}
message := &Message{
To: []string{"recipient@example.com"},
Subject: "Test",
Body: "Test body",
}
// This will fail to connect (no real server), but should validate inputs
err := Send(config, message)
// We expect an error because there's no real SMTP server
// But it should be a connection error, not a validation error
assert.Error(t, err)
assert.Contains(t, err.Error(), "dial")
}
func TestSendValidation(t *testing.T) {
tests := []struct {
name string
config *Config
message *Message
wantErr bool
errMsg string
}{
{
name: "nil config",
config: nil,
message: &Message{To: []string{"test@example.com"}, Subject: "Test", Body: "Test"},
wantErr: true,
errMsg: "configuration is required",
},
{
name: "nil message",
config: &Config{SMTPHost: "smtp.example.com", SMTPPort: 587, FromEmail: "from@example.com"},
message: nil,
wantErr: true,
errMsg: "message is required",
},
{
name: "invalid config",
config: &Config{
SMTPHost: "",
SMTPPort: 587,
},
message: &Message{To: []string{"test@example.com"}, Subject: "Test", Body: "Test"},
wantErr: true,
errMsg: "invalid email configuration",
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
err := Send(tt.config, tt.message)
if tt.wantErr {
assert.Error(t, err)
assert.Contains(t, err.Error(), tt.errMsg)
}
})
}
}
func TestSendAsync(t *testing.T) {
config := &Config{
SMTPHost: "smtp.example.com",
SMTPPort: 587,
FromEmail: "test@example.com",
}
message := &Message{
To: []string{"recipient@example.com"},
Subject: "Test Async",
Body: "Test async body",
}
// SendAsync should not block
start := time.Now()
SendAsync(config, message)
duration := time.Since(start)
// Should return almost immediately (< 100ms)
assert.Less(t, duration, 100*time.Millisecond)
// Give goroutine time to start
time.Sleep(50 * time.Millisecond)
}
func TestSendAsyncConcurrent(t *testing.T) {
config := &Config{
SMTPHost: "smtp.example.com",
SMTPPort: 587,
FromEmail: "test@example.com",
}
g := errgroup.Group{}
count := 5
for i := 0; i < count; i++ {
g.Go(func() error {
message := &Message{
To: []string{"recipient@example.com"},
Subject: "Concurrent Test",
Body: "Test body",
}
SendAsync(config, message)
return nil
})
}
if err := g.Wait(); err != nil {
t.Fatalf("SendAsync calls failed: %v", err)
}
}
+110
View File
@@ -0,0 +1,110 @@
package email
import (
"errors"
"fmt"
"strings"
"time"
)
// Message represents an email message to be sent.
type Message struct {
To []string // Required: recipient email addresses
Cc []string // Optional: carbon copy recipients
Bcc []string // Optional: blind carbon copy recipients
Subject string // Required: email subject
Body string // Required: email body content
IsHTML bool // Whether the body is HTML (default: false for plain text)
ReplyTo string // Optional: reply-to address
}
// Validate checks that the message has all required fields.
func (m *Message) Validate() error {
if len(m.To) == 0 {
return errors.New("at least one recipient is required")
}
if m.Subject == "" {
return errors.New("subject is required")
}
if m.Body == "" {
return errors.New("body is required")
}
return nil
}
// Format creates an RFC 5322 formatted email message.
func (m *Message) Format(fromEmail, fromName string) string {
var sb strings.Builder
fromEmail = sanitizeEmailHeaderValue(fromEmail)
fromName = sanitizeEmailHeaderValue(fromName)
to := sanitizeEmailHeaderValues(m.To)
cc := sanitizeEmailHeaderValues(m.Cc)
replyTo := sanitizeEmailHeaderValue(m.ReplyTo)
subject := sanitizeEmailHeaderValue(m.Subject)
// From header
if fromName != "" {
fmt.Fprintf(&sb, "From: %s <%s>\r\n", fromName, fromEmail)
} else {
fmt.Fprintf(&sb, "From: %s\r\n", fromEmail)
}
// To header
fmt.Fprintf(&sb, "To: %s\r\n", strings.Join(to, ", "))
// Cc header (optional)
if len(cc) > 0 {
fmt.Fprintf(&sb, "Cc: %s\r\n", strings.Join(cc, ", "))
}
// Reply-To header (optional)
if replyTo != "" {
fmt.Fprintf(&sb, "Reply-To: %s\r\n", replyTo)
}
// Subject header
fmt.Fprintf(&sb, "Subject: %s\r\n", subject)
// Date header (RFC 5322 format)
fmt.Fprintf(&sb, "Date: %s\r\n", time.Now().Format(time.RFC1123Z))
// MIME headers
sb.WriteString("MIME-Version: 1.0\r\n")
// Content-Type header
if m.IsHTML {
sb.WriteString("Content-Type: text/html; charset=utf-8\r\n")
} else {
sb.WriteString("Content-Type: text/plain; charset=utf-8\r\n")
}
// Empty line separating headers from body
sb.WriteString("\r\n")
// Body
sb.WriteString(m.Body)
return sb.String()
}
func sanitizeEmailHeaderValue(value string) string {
value = strings.NewReplacer("\r", " ", "\n", " ").Replace(value)
return strings.Join(strings.Fields(value), " ")
}
func sanitizeEmailHeaderValues(values []string) []string {
sanitized := make([]string, 0, len(values))
for _, value := range values {
sanitized = append(sanitized, sanitizeEmailHeaderValue(value))
}
return sanitized
}
// GetAllRecipients returns all recipients (To, Cc, Bcc) as a single slice.
func (m *Message) GetAllRecipients() []string {
var recipients []string
recipients = append(recipients, m.To...)
recipients = append(recipients, m.Cc...)
recipients = append(recipients, m.Bcc...)
return recipients
}
+204
View File
@@ -0,0 +1,204 @@
package email
import (
"strings"
"testing"
)
func TestMessageValidation(t *testing.T) {
tests := []struct {
name string
msg Message
wantErr bool
}{
{
name: "valid message",
msg: Message{
To: []string{"user@example.com"},
Subject: "Test Subject",
Body: "Test Body",
},
wantErr: false,
},
{
name: "no recipients",
msg: Message{
To: []string{},
Subject: "Test Subject",
Body: "Test Body",
},
wantErr: true,
},
{
name: "no subject",
msg: Message{
To: []string{"user@example.com"},
Subject: "",
Body: "Test Body",
},
wantErr: true,
},
{
name: "no body",
msg: Message{
To: []string{"user@example.com"},
Subject: "Test Subject",
Body: "",
},
wantErr: true,
},
{
name: "multiple recipients",
msg: Message{
To: []string{"user1@example.com", "user2@example.com"},
Cc: []string{"cc@example.com"},
Subject: "Test Subject",
Body: "Test Body",
},
wantErr: false,
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
err := tt.msg.Validate()
if (err != nil) != tt.wantErr {
t.Errorf("Validate() error = %v, wantErr %v", err, tt.wantErr)
}
})
}
}
func TestMessageFormatPlainText(t *testing.T) {
msg := Message{
To: []string{"user@example.com"},
Subject: "Test Subject",
Body: "Test Body",
IsHTML: false,
}
formatted := msg.Format("sender@example.com", "Sender Name")
// Check required headers
if !strings.Contains(formatted, "From: Sender Name <sender@example.com>") {
t.Error("Missing or incorrect From header")
}
if !strings.Contains(formatted, "To: user@example.com") {
t.Error("Missing or incorrect To header")
}
if !strings.Contains(formatted, "Subject: Test Subject") {
t.Error("Missing or incorrect Subject header")
}
if !strings.Contains(formatted, "Content-Type: text/plain; charset=utf-8") {
t.Error("Missing or incorrect Content-Type header for plain text")
}
if !strings.Contains(formatted, "Test Body") {
t.Error("Missing message body")
}
}
func TestMessageFormatHTML(t *testing.T) {
msg := Message{
To: []string{"user@example.com"},
Subject: "Test Subject",
Body: "<html><body>Test Body</body></html>",
IsHTML: true,
}
formatted := msg.Format("sender@example.com", "Sender Name")
// Check HTML content-type
if !strings.Contains(formatted, "Content-Type: text/html; charset=utf-8") {
t.Error("Missing or incorrect Content-Type header for HTML")
}
if !strings.Contains(formatted, "<html><body>Test Body</body></html>") {
t.Error("Missing HTML body")
}
}
func TestMessageFormatMultipleRecipients(t *testing.T) {
msg := Message{
To: []string{"user1@example.com", "user2@example.com"},
Cc: []string{"cc1@example.com", "cc2@example.com"},
Bcc: []string{"bcc@example.com"},
Subject: "Test Subject",
Body: "Test Body",
ReplyTo: "reply@example.com",
}
formatted := msg.Format("sender@example.com", "Sender Name")
// Check To header formatting
if !strings.Contains(formatted, "To: user1@example.com, user2@example.com") {
t.Error("Missing or incorrect To header with multiple recipients")
}
// Check Cc header formatting
if !strings.Contains(formatted, "Cc: cc1@example.com, cc2@example.com") {
t.Error("Missing or incorrect Cc header")
}
// Bcc should NOT appear in the formatted message
if strings.Contains(formatted, "Bcc:") {
t.Error("Bcc header should not appear in formatted message")
}
// Check Reply-To header
if !strings.Contains(formatted, "Reply-To: reply@example.com") {
t.Error("Missing or incorrect Reply-To header")
}
}
func TestMessageFormatSanitizesHeaderValues(t *testing.T) {
msg := Message{
To: []string{"user@example.com\r\nX-Injected-To: bad"},
Cc: []string{"cc@example.com\r\nX-Injected-Cc: bad"},
Subject: "Test\r\nX-Injected-Subject: bad",
Body: "Test Body",
ReplyTo: "reply@example.com\r\nX-Injected-Reply-To: bad",
}
formatted := msg.Format("sender@example.com\r\nX-Injected-From: bad", "Sender\r\nX-Injected-Name: bad")
headers := strings.SplitN(formatted, "\r\n\r\n", 2)[0]
if strings.Contains(headers, "\r\nX-Injected") {
t.Fatalf("header value injection was not sanitized:\n%s", headers)
}
if !strings.Contains(headers, "Subject: Test X-Injected-Subject: bad") {
t.Error("subject header was not normalized")
}
if !strings.Contains(headers, "From: Sender X-Injected-Name: bad <sender@example.com X-Injected-From: bad>") {
t.Error("from header was not normalized")
}
}
func TestGetAllRecipients(t *testing.T) {
msg := Message{
To: []string{"user1@example.com", "user2@example.com"},
Cc: []string{"cc@example.com"},
Bcc: []string{"bcc@example.com"},
}
recipients := msg.GetAllRecipients()
// Should have all 4 recipients
if len(recipients) != 4 {
t.Errorf("GetAllRecipients() returned %d recipients, want 4", len(recipients))
}
// Check all recipients are present
expectedRecipients := map[string]bool{
"user1@example.com": true,
"user2@example.com": true,
"cc@example.com": true,
"bcc@example.com": true,
}
for _, recipient := range recipients {
if !expectedRecipients[recipient] {
t.Errorf("Unexpected recipient: %s", recipient)
}
delete(expectedRecipients, recipient)
}
if len(expectedRecipients) > 0 {
t.Error("Not all expected recipients were returned")
}
}
+50
View File
@@ -0,0 +1,50 @@
# Maintaining the Memo Filter Engine
The engine is memo-specific; any future field or behavior changes must stay
consistent with the memo schema and store implementations. Use this guide when
extending or debugging the package.
## Adding a New Memo Field
1. **Update the schema**
- Add the field entry in `schema.go`.
- Define the backing column (`Column`), JSON path (if applicable), type, and
allowed operators.
- Include the CEL variable in `EnvOptions`.
2. **Adjust parser or renderer (if needed)**
- For non-scalar fields (JSON booleans, lists), add handling in
`parser.go` or extend the renderer helpers.
- Keep validation in the parser (e.g., reject unsupported operators).
3. **Write a golden test**
- Extend the dialect-specific memo filter tests under
`store/db/{sqlite,mysql,postgres}/memo_filter_test.go` with a case that
exercises the new field.
4. **Run `go test ./...`** to ensure the SQL output matches expectations across
all dialects.
## Supporting Dialect Nuances
- Centralize differences inside `render.go`. If a new dialect-specific behavior
emerges (e.g., JSON operators), add the logic there rather than leaking it
into store code.
- Use the renderer helpers (`jsonExtractExpr`, `jsonArrayExpr`, etc.) rather than
sprinkling ad-hoc SQL strings.
- When placeholders change, adjust `addArg` so that argument numbering stays in
sync with store queries.
## Debugging Tips
- **Parser errors** Most originate in `buildCondition` or schema validation.
Enable logging around `parser.go` when diagnosing unknown identifier/operator
messages.
- **Renderer output** Temporary printf/log statements in `renderCondition` help
identify which IR node produced unexpected SQL.
- **Store integration** Ensure drivers call `filter.DefaultEngine()` exactly once
per process; the singleton caches the parsed CEL environment.
## Testing Checklist
- `go test ./store/...` ensures all dialect tests consume the engine correctly.
- Add targeted unit tests whenever new IR nodes or renderer paths are introduced.
- When changing boolean or JSON handling, verify all three dialect test suites
(SQLite, MySQL, Postgres) to avoid regression.
+105
View File
@@ -0,0 +1,105 @@
# Memo Filter Engine
This package houses the memo-only filter engine that turns standard CEL syntax
into SQL fragments for the subset of expressions supported by the memo schema.
The engine follows a three phase pipeline inspired by systems
such as Calcite or Prisma:
1. **Parsing** CEL expressions are parsed with `cel-go` and validated against
the memo-specific environment declared in `schema.go`. Only fields that
exist in the schema can surface in the filter, and non-standard legacy
coercions are rejected.
2. **Normalization** the raw CEL AST is converted into an intermediate
representation (IR) defined in `ir.go`. The IR is a dialect-agnostic tree of
conditions (logical operators, comparisons, list membership, etc.). This
step enforces schema rules (e.g. operator compatibility, type checks).
3. **Rendering** the renderer in `render.go` walks the IR and produces a SQL
fragment plus placeholder arguments tailored to a target dialect
(`sqlite`, `mysql`, or `postgres`). Dialect differences such as JSON access,
boolean semantics, placeholders, and `LIKE` vs `ILIKE` are encapsulated in
renderer helpers.
The entry point is `filter.DefaultEngine()` from `engine.go`. It lazily constructs
an `Engine` configured with the memo schema and exposes:
```go
engine, _ := filter.DefaultEngine()
stmt, _ := engine.CompileToStatement(ctx, `has_task_list && visibility == "PUBLIC"`, filter.RenderOptions{
Dialect: filter.DialectPostgres,
})
// stmt.SQL -> "((memo.payload->'property'->>'hasTaskList')::boolean IS TRUE AND memo.visibility = $1)"
// stmt.Args -> ["PUBLIC"]
```
## Core Files
| File | Responsibility |
| ------------- | ------------------------------------------------------------------------------- |
| `schema.go` | Declares memo fields, their types, backing columns, CEL environment options |
| `ir.go` | IR node definitions used across the pipeline |
| `parser.go` | Converts CEL `Expr` into IR while applying schema validation |
| `render.go` | Translates IR into SQL, handling dialect-specific behavior |
| `engine.go` | Glue between the phases; exposes `Compile`, `CompileToStatement`, and `DefaultEngine` |
| `helpers.go` | Convenience helpers for store integration (appending conditions) |
## SQL Generation Notes
- **Placeholders** — `?` is used for SQLite/MySQL, `$n` for Postgres. The renderer
tracks offsets to compose queries with pre-existing arguments.
- **JSON Fields** — Memo metadata lives in `memo.payload`. The renderer handles
`JSON_EXTRACT`/`json_extract`/`->`/`->>` variations and boolean coercion.
- **Time Fields** — `created_ts`, `updated_ts`, and attachment `create_time` are
CEL `timestamp` values. Express instants with the `now` variable,
`duration("…")` (e.g. `created_ts >= now - duration("24h")`), or
`timestamp("2006-01-02T15:04:05Z")` / `timestamp(<epoch-seconds>)`. These fold
to epoch seconds at compile time — `now` is frozen once per compile (injectable
for tests via the engine clock) — so the backing columns stay unchanged.
- **Tag Operations** — `tag in [...]` and `"tag" in tags` become JSON array
predicates. SQLite uses `LIKE` patterns, MySQL uses `JSON_CONTAINS`, and
Postgres uses `@>`.
- **Boolean Flags** — Fields such as `has_task_list` render as `IS TRUE` equality
checks, or comparisons against `CAST('true' AS JSON)` depending on the dialect.
- **String Matching** — `content.contains(x)`, `content.startsWith(x)`, and
`content.endsWith(x)` render as case-insensitive `LIKE`/`ILIKE` with LIKE
metacharacters (`%`, `_`, `\`) escaped. Available on scalar string fields whose
schema sets `SupportsContains` (memo `content`; attachment `filename`,
`mime_type`).
- **Regex** — `field.matches("pattern")` renders to `~` (Postgres) or `REGEXP`
(MySQL/SQLite). SQLite uses a Go-backed `regexp` function registered in
`store/db/sqlite/functions.go`. Patterns are validated at compile time against
Go's RE2 via `cel.ValidateRegexLiterals()`. **Caveat:** regex *syntax* differs
per engine (Go RE2 on SQLite, POSIX ERE on Postgres, ICU on MySQL 8.0+), so
engine-specific patterns may not be portable.
- **Tag `all()` / `exists_one()`** — `tags.all(t, <pred>)` matches only non-empty
tag sets where every element satisfies the predicate; `tags.exists_one(t,
<pred>)` matches when exactly one element does (`COUNT(...) = 1`). Both iterate
per-element (`json_each` / `jsonb_array_elements_text` / `JSON_TABLE`).
- **Timestamp Accessors** — `created_ts.getFullYear()`, `getMonth()`, `getDate()`,
`getDayOfMonth()`, `getDayOfWeek()`, `getDayOfYear()`, `getHours()`,
`getMinutes()`, `getSeconds()` render to date-part extraction (`strftime` /
`EXTRACT` / `YEAR`/`MONTH`/…). Results are normalized to CEL's base (0-based
month, 0-based day-of-week with 0 = Sunday). The same accessors on `now` fold
to literal date parts of the frozen evaluation time (UTC), so saved filters
like `created_ts.getMonth() == now.getMonth() && created_ts.getDate() ==
now.getDate()` ("on this day") re-resolve on every compile. Extraction is UTC
on SQLite/Postgres (epoch columns); on MySQL the `TIMESTAMP` column is read in
the session time zone. A timezone argument is not supported.
- **Set Operations** — `ext.Sets()`: `sets.contains(tags, [...])`,
`sets.intersects(tags, [...])`, and `sets.equivalent(tags, [...])` desugar to
exact-membership checks (AND / OR of `"v" in tags`); `equivalent` adds a
`size(tags)` length check (relies on tags being a set).
- **`size()`** — `size(tags)` renders to JSON array length; `size(content)` (and
other string fields) render to `LENGTH` / `CHAR_LENGTH` (MySQL) for code-point
counts.
- **Arithmetic** — `+`, `-`, `*`, `/`, `%` constant-fold on literal/`now`/`duration`
operands (division and modulo guard against a zero divisor).
## Typical Integration
1. Fetch the engine with `filter.DefaultEngine()`.
2. Call `CompileToStatement` using the appropriate dialect enum.
3. Append the emitted SQL fragment/args to the existing `WHERE` clause.
4. Execute the resulting query through the store driver.
The `helpers.AppendConditions` helper encapsulates steps 23 when a driver needs
to process an array of filters.
+124
View File
@@ -0,0 +1,124 @@
package filter
import (
"context"
"strings"
"sync"
"time"
"github.com/google/cel-go/cel"
"github.com/pkg/errors"
)
// Engine parses CEL filters into a dialect-agnostic condition tree.
type Engine struct {
schema Schema
env *cel.Env
// nowFunc resolves the value of the `now` variable. It is frozen once per
// Compile so a single filter sees a single instant, and is overridable in
// tests for deterministic folding.
nowFunc func() time.Time
}
// NewEngine builds a new Engine for the provided schema.
func NewEngine(schema Schema) (*Engine, error) {
env, err := cel.NewEnv(schema.EnvOptions...)
if err != nil {
return nil, errors.Wrap(err, "failed to create CEL environment")
}
return &Engine{
schema: schema,
env: env,
nowFunc: time.Now,
}, nil
}
// Program stores a compiled filter condition.
type Program struct {
schema Schema
condition Condition
}
// ConditionTree exposes the underlying condition tree.
func (p *Program) ConditionTree() Condition {
return p.condition
}
// Compile parses the filter string into an executable program.
func (e *Engine) Compile(_ context.Context, filter string) (*Program, error) {
if strings.TrimSpace(filter) == "" {
return nil, errors.New("filter expression is empty")
}
ast, issues := e.env.Compile(filter)
if issues != nil && issues.Err() != nil {
return nil, errors.Wrap(issues.Err(), "failed to compile filter")
}
parsed, err := cel.AstToParsedExpr(ast)
if err != nil {
return nil, errors.Wrap(err, "failed to convert AST")
}
cond, err := buildCondition(parsed.GetExpr(), parseContext{schema: e.schema, now: e.nowFunc()})
if err != nil {
return nil, err
}
return &Program{
schema: e.schema,
condition: cond,
}, nil
}
// CompileToStatement compiles and renders the filter in a single step.
func (e *Engine) CompileToStatement(ctx context.Context, filter string, opts RenderOptions) (Statement, error) {
program, err := e.Compile(ctx, filter)
if err != nil {
return Statement{}, err
}
return program.Render(opts)
}
// RenderOptions configure SQL rendering.
type RenderOptions struct {
Dialect DialectName
PlaceholderOffset int
DisableNullChecks bool
}
// Statement contains the rendered SQL fragment and its args.
type Statement struct {
SQL string
Args []any
}
// Render converts the program into a dialect-specific SQL fragment.
func (p *Program) Render(opts RenderOptions) (Statement, error) {
renderer := newRenderer(p.schema, opts)
return renderer.Render(p.condition)
}
var (
defaultOnce sync.Once
defaultInst *Engine
defaultErr error
defaultAttachmentOnce sync.Once
defaultAttachmentInst *Engine
defaultAttachmentErr error
)
// DefaultEngine returns the process-wide memo filter engine.
func DefaultEngine() (*Engine, error) {
defaultOnce.Do(func() {
defaultInst, defaultErr = NewEngine(NewSchema())
})
return defaultInst, defaultErr
}
// DefaultAttachmentEngine returns the process-wide attachment filter engine.
func DefaultAttachmentEngine() (*Engine, error) {
defaultAttachmentOnce.Do(func() {
defaultAttachmentInst, defaultAttachmentErr = NewEngine(NewAttachmentSchema())
})
return defaultAttachmentInst, defaultAttachmentErr
}
+197
View File
@@ -0,0 +1,197 @@
package filter
import (
"context"
"testing"
"github.com/stretchr/testify/require"
)
func TestCompileAcceptsStandardTagEqualityPredicate(t *testing.T) {
t.Parallel()
engine, err := NewEngine(NewSchema())
require.NoError(t, err)
_, err = engine.Compile(context.Background(), `tags.exists(t, t == "1231")`)
require.NoError(t, err)
}
func TestCompileRejectsLegacyNumericLogicalOperand(t *testing.T) {
t.Parallel()
engine, err := NewEngine(NewSchema())
require.NoError(t, err)
_, err = engine.Compile(context.Background(), `pinned && 1`)
require.Error(t, err)
require.Contains(t, err.Error(), "failed to compile filter")
}
func TestCompileRejectsNonBooleanTopLevelConstant(t *testing.T) {
t.Parallel()
engine, err := NewEngine(NewSchema())
require.NoError(t, err)
_, err = engine.Compile(context.Background(), `1`)
require.EqualError(t, err, "filter must evaluate to a boolean value")
}
func TestCompileRejectsMalformedRegex(t *testing.T) {
t.Parallel()
engine, err := NewEngine(NewSchema())
require.NoError(t, err)
_, err = engine.Compile(context.Background(), `content.matches("(")`)
require.Error(t, err)
}
func TestCompileMatchesRendersRegexOperator(t *testing.T) {
t.Parallel()
engine, err := NewEngine(NewSchema())
require.NoError(t, err)
stmt, err := engine.CompileToStatement(context.Background(), `content.matches("v[0-9]+")`, RenderOptions{Dialect: DialectPostgres})
require.NoError(t, err)
require.Contains(t, stmt.SQL, "~")
require.Equal(t, []any{"v[0-9]+"}, stmt.Args)
}
func TestCompileRejectsStartsWithOnUnsupportedField(t *testing.T) {
t.Parallel()
engine, err := NewEngine(NewSchema())
require.NoError(t, err)
_, err = engine.Compile(context.Background(), `visibility.startsWith("P")`)
require.Error(t, err)
require.Contains(t, err.Error(), "does not support text matching")
}
func TestCompileContainsEscapesLikeWildcards(t *testing.T) {
t.Parallel()
engine, err := NewEngine(NewSchema())
require.NoError(t, err)
stmt, err := engine.CompileToStatement(context.Background(), `content.contains("50%_off")`, RenderOptions{Dialect: DialectSQLite})
require.NoError(t, err)
// The % and _ in the value must be escaped so they are matched literally,
// and SQLite needs an explicit ESCAPE clause.
require.Contains(t, stmt.SQL, `ESCAPE '\'`)
require.Equal(t, []any{`%50\%\_off%`}, stmt.Args)
}
// =============================================================================
// Cross-dialect rendering tests (no DB required; complements the SQLite-only
// behavioral tests in store/test by asserting MySQL/Postgres SQL generation).
// =============================================================================
func TestRenderStartsWithPerDialect(t *testing.T) {
t.Parallel()
engine, err := NewEngine(NewSchema())
require.NoError(t, err)
cases := []struct {
dialect DialectName
fragments []string
}{
{DialectSQLite, []string{"memos_unicode_lower(", "`memo`.`content`", `ESCAPE '\'`}},
{DialectPostgres, []string{"memo.content ILIKE $1"}},
{DialectMySQL, []string{"`memo`.`content` LIKE ?"}},
}
for _, tc := range cases {
stmt, err := engine.CompileToStatement(context.Background(), `content.startsWith("TODO")`, RenderOptions{Dialect: tc.dialect})
require.NoError(t, err, tc.dialect)
for _, frag := range tc.fragments {
require.Contains(t, stmt.SQL, frag, "dialect %s", tc.dialect)
}
require.Equal(t, []any{"TODO%"}, stmt.Args, "dialect %s", tc.dialect)
}
}
func TestRenderEndsWithPerDialect(t *testing.T) {
t.Parallel()
engine, err := NewEngine(NewSchema())
require.NoError(t, err)
for _, dialect := range []DialectName{DialectSQLite, DialectPostgres, DialectMySQL} {
stmt, err := engine.CompileToStatement(context.Background(), `content.endsWith(".md")`, RenderOptions{Dialect: dialect})
require.NoError(t, err, dialect)
require.Equal(t, []any{"%.md"}, stmt.Args, "dialect %s", dialect)
}
}
func TestRenderMatchesPerDialect(t *testing.T) {
t.Parallel()
engine, err := NewEngine(NewSchema())
require.NoError(t, err)
cases := []struct {
dialect DialectName
fragment string
}{
{DialectSQLite, "`memo`.`content` REGEXP ?"},
{DialectMySQL, "`memo`.`content` REGEXP ?"},
{DialectPostgres, "memo.content ~ $1"},
}
for _, tc := range cases {
stmt, err := engine.CompileToStatement(context.Background(), `content.matches("v[0-9]+")`, RenderOptions{Dialect: tc.dialect})
require.NoError(t, err, tc.dialect)
require.Contains(t, stmt.SQL, tc.fragment, "dialect %s", tc.dialect)
require.Equal(t, []any{"v[0-9]+"}, stmt.Args, "dialect %s", tc.dialect)
}
}
func TestRenderTagsAllPerDialect(t *testing.T) {
t.Parallel()
engine, err := NewEngine(NewSchema())
require.NoError(t, err)
cases := []struct {
dialect DialectName
fragments []string
}{
{DialectSQLite, []string{"NOT EXISTS", "json_each(", "!= '[]'", "memos_unicode_lower(value)"}},
{DialectPostgres, []string{"NOT EXISTS", "jsonb_array_elements_text(", "jsonb_array_length(", "value ILIKE"}},
{DialectMySQL, []string{"NOT EXISTS", "JSON_TABLE(", "JSON_LENGTH(", "value LIKE"}},
}
for _, tc := range cases {
stmt, err := engine.CompileToStatement(context.Background(), `tags.all(t, t.startsWith("work/"))`, RenderOptions{Dialect: tc.dialect})
require.NoError(t, err, tc.dialect)
for _, frag := range tc.fragments {
require.Contains(t, stmt.SQL, frag, "dialect %s", tc.dialect)
}
require.Equal(t, []any{"work/%"}, stmt.Args, "dialect %s", tc.dialect)
}
}
func TestRenderTextMatchEscaping(t *testing.T) {
t.Parallel()
engine, err := NewEngine(NewSchema())
require.NoError(t, err)
// Both % and _ in the value must be escaped so they match literally.
stmt, err := engine.CompileToStatement(context.Background(), `content.contains("a%b_c")`, RenderOptions{Dialect: DialectSQLite})
require.NoError(t, err)
require.Equal(t, []any{`%a\%b\_c%`}, stmt.Args)
}
func TestRenderAllRejectsUnsupportedPredicate(t *testing.T) {
t.Parallel()
engine, err := NewEngine(NewSchema())
require.NoError(t, err)
// size() is not a valid per-element predicate inside all().
_, err = engine.CompileToStatement(context.Background(), `tags.all(t, size(t) > 2)`, RenderOptions{Dialect: DialectSQLite})
require.Error(t, err)
}
+377
View File
@@ -0,0 +1,377 @@
package filter
import (
"context"
"testing"
"time"
"github.com/stretchr/testify/require"
)
// ---------------------------------------------------------------------------
// Arithmetic folding: division and modulo
// ---------------------------------------------------------------------------
func TestCompileDivisionFolds(t *testing.T) {
t.Parallel()
engine, err := NewEngine(NewSchema())
require.NoError(t, err)
stmt, err := engine.CompileToStatement(context.Background(), `creator_id == 100 / 10`, RenderOptions{Dialect: DialectSQLite})
require.NoError(t, err)
require.Equal(t, []any{int64(10)}, stmt.Args)
}
func TestCompileModuloFolds(t *testing.T) {
t.Parallel()
engine, err := NewEngine(NewSchema())
require.NoError(t, err)
stmt, err := engine.CompileToStatement(context.Background(), `creator_id == 17 % 5`, RenderOptions{Dialect: DialectSQLite})
require.NoError(t, err)
require.Equal(t, []any{int64(2)}, stmt.Args)
}
func TestCompileDivisionByZeroErrors(t *testing.T) {
t.Parallel()
engine, err := NewEngine(NewSchema())
require.NoError(t, err)
_, err = engine.Compile(context.Background(), `creator_id == 10 / 0`)
require.Error(t, err)
}
// ---------------------------------------------------------------------------
// size() on scalar string fields -> SQL length
// ---------------------------------------------------------------------------
func TestCompileSizeOnContentRendersLengthPerDialect(t *testing.T) {
t.Parallel()
engine, err := NewEngine(NewSchema())
require.NoError(t, err)
cases := []struct {
dialect DialectName
fragment string
}{
{DialectSQLite, "LENGTH("},
{DialectPostgres, "LENGTH("},
{DialectMySQL, "CHAR_LENGTH("},
}
for _, tc := range cases {
stmt, err := engine.CompileToStatement(context.Background(), `size(content) > 5`, RenderOptions{Dialect: tc.dialect})
require.NoError(t, err, tc.dialect)
require.Contains(t, stmt.SQL, tc.fragment, "dialect %s", tc.dialect)
require.Equal(t, []any{int64(5)}, stmt.Args, "dialect %s", tc.dialect)
}
}
// ---------------------------------------------------------------------------
// Timestamp accessor methods (getFullYear, getMonth, ...)
// ---------------------------------------------------------------------------
func TestCompileTimestampAccessorsPerDialect(t *testing.T) {
t.Parallel()
engine, err := NewEngine(NewSchema())
require.NoError(t, err)
cases := []struct {
name string
filter string
dialect DialectName
fragments []string
arg int64
}{
// getFullYear == 2024
{"sqlite year", `created_ts.getFullYear() == 2024`, DialectSQLite, []string{"strftime('%Y'", "'unixepoch'"}, 2024},
{"pg year", `created_ts.getFullYear() == 2024`, DialectPostgres, []string{"EXTRACT(YEAR FROM to_timestamp(", "AT TIME ZONE 'UTC'"}, 2024},
{"mysql year", `created_ts.getFullYear() == 2024`, DialectMySQL, []string{"YEAR(`memo`.`created_ts`)"}, 2024},
// getMonth is 0-based -> SQL must subtract 1
{"sqlite month", `created_ts.getMonth() == 5`, DialectSQLite, []string{"strftime('%m'", "- 1)"}, 5},
{"pg month", `created_ts.getMonth() == 5`, DialectPostgres, []string{"EXTRACT(MONTH FROM", "- 1)"}, 5},
{"mysql month", `created_ts.getMonth() == 5`, DialectMySQL, []string{"MONTH(`memo`.`created_ts`)", "- 1)"}, 5},
// getDayOfWeek 0=Sunday -> MySQL DAYOFWEEK is 1-based and must subtract 1
{"mysql dow", `created_ts.getDayOfWeek() == 0`, DialectMySQL, []string{"DAYOFWEEK(`memo`.`created_ts`)", "- 1)"}, 0},
{"sqlite dow", `created_ts.getDayOfWeek() == 0`, DialectSQLite, []string{"strftime('%w'"}, 0},
// getDate is 1-based -> no offset
{"sqlite date", `created_ts.getDate() == 22`, DialectSQLite, []string{"strftime('%d'"}, 22},
}
for _, tc := range cases {
stmt, err := engine.CompileToStatement(context.Background(), tc.filter, RenderOptions{Dialect: tc.dialect})
require.NoError(t, err, tc.name)
for _, frag := range tc.fragments {
require.Contains(t, stmt.SQL, frag, tc.name)
}
require.Equal(t, []any{tc.arg}, stmt.Args, tc.name)
}
}
func TestCompileNowAccessorsFoldToInjectedClock(t *testing.T) {
t.Parallel()
// 2026-07-07T10:30:45Z, a Tuesday (year day 188).
engine := memoEngineAt(t, time.Date(2026, time.July, 7, 10, 30, 45, 0, time.UTC).Unix())
cases := []struct {
name string
filter string
args []any
}{
{"on this day", `created_ts.getMonth() == now.getMonth() && created_ts.getDate() == now.getDate()`, []any{int64(6), int64(7)}},
{"year", `created_ts.getFullYear() < now.getFullYear()`, []any{int64(2026)}},
{"day of month", `created_ts.getDayOfMonth() == now.getDayOfMonth()`, []any{int64(6)}},
{"day of week", `created_ts.getDayOfWeek() == now.getDayOfWeek()`, []any{int64(2)}},
{"day of year", `created_ts.getDayOfYear() == now.getDayOfYear()`, []any{int64(187)}},
{"clock parts", `created_ts.getHours() == now.getHours() || created_ts.getMinutes() == now.getMinutes() || created_ts.getSeconds() == now.getSeconds()`, []any{int64(10), int64(30), int64(45)}},
}
for _, tc := range cases {
stmt, err := engine.CompileToStatement(context.Background(), tc.filter, RenderOptions{Dialect: DialectSQLite})
require.NoError(t, err, tc.name)
require.Equal(t, tc.args, stmt.Args, tc.name)
}
}
func TestCompileNowAccessorRejectsTimezoneArg(t *testing.T) {
t.Parallel()
engine, err := NewEngine(NewSchema())
require.NoError(t, err)
_, err = engine.Compile(context.Background(), `created_ts.getMonth() == now.getMonth("America/New_York")`)
require.Error(t, err)
}
func TestCompileOnThisDayPerDialect(t *testing.T) {
t.Parallel()
const onThisDay = `created_ts.getMonth() == now.getMonth() && created_ts.getDate() == now.getDate()`
engine := memoEngineAt(t, time.Date(2026, time.July, 7, 10, 30, 45, 0, time.UTC).Unix())
cases := []struct {
dialect DialectName
fragments []string
}{
{DialectSQLite, []string{"strftime('%m'", "strftime('%d'", "'unixepoch'"}},
{DialectPostgres, []string{"EXTRACT(MONTH FROM", "EXTRACT(DAY FROM"}},
{DialectMySQL, []string{"MONTH(`memo`.`created_ts`)", "DAYOFMONTH(`memo`.`created_ts`)"}},
}
for _, tc := range cases {
stmt, err := engine.CompileToStatement(context.Background(), onThisDay, RenderOptions{Dialect: tc.dialect})
require.NoError(t, err, tc.dialect)
for _, frag := range tc.fragments {
require.Contains(t, stmt.SQL, frag, "dialect %s", tc.dialect)
}
require.Equal(t, []any{int64(6), int64(7)}, stmt.Args, "dialect %s", tc.dialect)
}
}
func TestCompileNowAccessorsAtBoundaryDates(t *testing.T) {
t.Parallel()
// Every accessor folded at calendar edges: year rollover, leap day, Sunday.
allAccessors := `created_ts.getFullYear() == now.getFullYear() ` +
`&& created_ts.getMonth() == now.getMonth() ` +
`&& created_ts.getDate() == now.getDate() ` +
`&& created_ts.getDayOfMonth() == now.getDayOfMonth() ` +
`&& created_ts.getDayOfWeek() == now.getDayOfWeek() ` +
`&& created_ts.getDayOfYear() == now.getDayOfYear() ` +
`&& created_ts.getHours() == now.getHours() ` +
`&& created_ts.getMinutes() == now.getMinutes() ` +
`&& created_ts.getSeconds() == now.getSeconds()`
cases := []struct {
name string
clock time.Time
// year, month, date, dayOfMonth, dayOfWeek, dayOfYear, hours, minutes, seconds
args []any
}{
{
"new year's eve",
time.Date(2026, time.December, 31, 23, 59, 59, 0, time.UTC),
[]any{int64(2026), int64(11), int64(31), int64(30), int64(4), int64(364), int64(23), int64(59), int64(59)},
},
{
"new year's day",
time.Date(2027, time.January, 1, 0, 0, 0, 0, time.UTC),
[]any{int64(2027), int64(0), int64(1), int64(0), int64(5), int64(0), int64(0), int64(0), int64(0)},
},
{
"leap day",
time.Date(2028, time.February, 29, 12, 0, 0, 0, time.UTC),
[]any{int64(2028), int64(1), int64(29), int64(28), int64(2), int64(59), int64(12), int64(0), int64(0)},
},
{
"sunday",
time.Date(2026, time.July, 5, 8, 15, 30, 0, time.UTC),
[]any{int64(2026), int64(6), int64(5), int64(4), int64(0), int64(185), int64(8), int64(15), int64(30)},
},
}
for _, tc := range cases {
engine := memoEngineAt(t, tc.clock.Unix())
stmt, err := engine.CompileToStatement(context.Background(), allAccessors, RenderOptions{Dialect: DialectSQLite})
require.NoError(t, err, tc.name)
require.Equal(t, tc.args, stmt.Args, tc.name)
}
}
func TestCompileNowAccessorFoldsInUTC(t *testing.T) {
t.Parallel()
// A clock in UTC+8 at 05:00 on July 8 is still July 7, 21:00 in UTC;
// folding must not leak the clock's zone.
engine, err := NewEngine(NewSchema())
require.NoError(t, err)
engine.nowFunc = func() time.Time {
return time.Date(2026, time.July, 8, 5, 0, 0, 0, time.FixedZone("UTC+8", 8*3600))
}
stmt, err := engine.CompileToStatement(
context.Background(),
`created_ts.getDate() == now.getDate() && created_ts.getHours() == now.getHours()`,
RenderOptions{Dialect: DialectSQLite},
)
require.NoError(t, err)
require.Equal(t, []any{int64(7), int64(21)}, stmt.Args)
}
func TestCompileNowAccessorOnLeftSide(t *testing.T) {
t.Parallel()
engine := memoEngineAt(t, time.Date(2026, time.July, 7, 10, 30, 45, 0, time.UTC).Unix())
stmt, err := engine.CompileToStatement(context.Background(), `now.getMonth() == created_ts.getMonth()`, RenderOptions{Dialect: DialectSQLite})
require.NoError(t, err)
require.Contains(t, stmt.SQL, "strftime('%m'")
require.Equal(t, []any{int64(6)}, stmt.Args)
}
func TestCompileNowAccessorMirrorsOrderingOperator(t *testing.T) {
t.Parallel()
// Swapping the literal to the right must flip < to > to keep the meaning.
engine := memoEngineAt(t, time.Date(2026, time.July, 7, 10, 30, 45, 0, time.UTC).Unix())
stmt, err := engine.CompileToStatement(context.Background(), `now.getHours() < created_ts.getHours()`, RenderOptions{Dialect: DialectSQLite})
require.NoError(t, err)
require.Contains(t, stmt.SQL, "> ?")
require.Equal(t, []any{int64(10)}, stmt.Args)
}
func TestCompileNowAccessorAgainstLiteralFoldsToConstant(t *testing.T) {
t.Parallel()
// Both sides fold to literals; the comparison folds to a constant condition.
engine := memoEngineAt(t, time.Date(2026, time.July, 7, 10, 30, 45, 0, time.UTC).Unix())
// True → trivial filter (empty SQL, matches everything).
stmt, err := engine.CompileToStatement(context.Background(), `now.getFullYear() >= 2026`, RenderOptions{Dialect: DialectSQLite})
require.NoError(t, err)
require.Empty(t, stmt.SQL)
require.Empty(t, stmt.Args)
// False → unsatisfiable filter.
stmt, err = engine.CompileToStatement(context.Background(), `now.getFullYear() < 2026`, RenderOptions{Dialect: DialectSQLite})
require.NoError(t, err)
require.Equal(t, "1 = 0", stmt.SQL)
require.Empty(t, stmt.Args)
}
func TestCompileAttachmentNowAccessors(t *testing.T) {
t.Parallel()
engine, err := NewEngine(NewAttachmentSchema())
require.NoError(t, err)
engine.nowFunc = fixedClock(time.Date(2026, time.July, 7, 10, 30, 45, 0, time.UTC).Unix())
stmt, err := engine.CompileToStatement(
context.Background(),
`create_time.getMonth() == now.getMonth() && create_time.getDate() == now.getDate()`,
RenderOptions{Dialect: DialectSQLite},
)
require.NoError(t, err)
require.Contains(t, stmt.SQL, "`attachment`.`created_ts`")
require.Equal(t, []any{int64(6), int64(7)}, stmt.Args)
}
func TestCompileTimestampAccessorRejectsTimezoneArg(t *testing.T) {
t.Parallel()
engine, err := NewEngine(NewSchema())
require.NoError(t, err)
_, err = engine.Compile(context.Background(), `created_ts.getHours("America/New_York") == 9`)
require.Error(t, err)
}
func TestCompileTimestampAccessorRejectsNonTimestampField(t *testing.T) {
t.Parallel()
engine, err := NewEngine(NewSchema())
require.NoError(t, err)
// content is a string, not a timestamp.
_, err = engine.Compile(context.Background(), `content.getFullYear() == 2024`)
require.Error(t, err)
}
// ---------------------------------------------------------------------------
// ext.Sets(): sets.contains / sets.intersects / sets.equivalent over tags
// ---------------------------------------------------------------------------
func TestCompileSetsContainsRendersAndOfMemberships(t *testing.T) {
t.Parallel()
engine, err := NewEngine(NewSchema())
require.NoError(t, err)
stmt, err := engine.CompileToStatement(context.Background(), `sets.contains(tags, ["a", "b"])`, RenderOptions{Dialect: DialectSQLite})
require.NoError(t, err)
require.Contains(t, stmt.SQL, " AND ")
require.Len(t, stmt.Args, 2)
}
func TestCompileSetsIntersectsRendersOrOfMemberships(t *testing.T) {
t.Parallel()
engine, err := NewEngine(NewSchema())
require.NoError(t, err)
stmt, err := engine.CompileToStatement(context.Background(), `sets.intersects(tags, ["a", "b"])`, RenderOptions{Dialect: DialectSQLite})
require.NoError(t, err)
require.Contains(t, stmt.SQL, " OR ")
require.Len(t, stmt.Args, 2)
}
func TestCompileSetsEquivalentAddsLengthCheck(t *testing.T) {
t.Parallel()
engine, err := NewEngine(NewSchema())
require.NoError(t, err)
stmt, err := engine.CompileToStatement(context.Background(), `sets.equivalent(tags, ["a", "b"])`, RenderOptions{Dialect: DialectSQLite})
require.NoError(t, err)
require.Contains(t, stmt.SQL, "JSON_ARRAY_LENGTH")
require.Contains(t, stmt.Args, int64(2))
}
// ---------------------------------------------------------------------------
// exists_one() comprehension on tags
// ---------------------------------------------------------------------------
func TestCompileExistsOnePerDialect(t *testing.T) {
t.Parallel()
engine, err := NewEngine(NewSchema())
require.NoError(t, err)
cases := []struct {
dialect DialectName
fragments []string
}{
{DialectSQLite, []string{"COUNT(", "json_each(", ") = 1"}},
{DialectPostgres, []string{"COUNT(", "jsonb_array_elements_text(", ") = 1"}},
{DialectMySQL, []string{"COUNT(", "JSON_TABLE(", ") = 1"}},
}
for _, tc := range cases {
stmt, err := engine.CompileToStatement(context.Background(), `tags.exists_one(t, t == "urgent")`, RenderOptions{Dialect: tc.dialect})
require.NoError(t, err, tc.dialect)
for _, frag := range tc.fragments {
require.Contains(t, stmt.SQL, frag, "dialect %s", tc.dialect)
}
require.Equal(t, []any{"urgent"}, stmt.Args, "dialect %s", tc.dialect)
}
}
+25
View File
@@ -0,0 +1,25 @@
package filter
import (
"context"
"fmt"
)
// AppendConditions compiles the provided filters and appends the resulting SQL fragments and args.
func AppendConditions(ctx context.Context, engine *Engine, filters []string, dialect DialectName, where *[]string, args *[]any) error {
for _, filterStr := range filters {
stmt, err := engine.CompileToStatement(ctx, filterStr, RenderOptions{
Dialect: dialect,
PlaceholderOffset: len(*args),
})
if err != nil {
return err
}
if stmt.SQL == "" {
continue
}
*where = append(*where, fmt.Sprintf("(%s)", stmt.SQL))
*args = append(*args, stmt.Args...)
}
return nil
}
+196
View File
@@ -0,0 +1,196 @@
package filter
// Condition represents a boolean expression derived from the CEL filter.
type Condition interface {
isCondition()
}
// LogicalOperator enumerates the supported logical operators.
type LogicalOperator string
const (
LogicalAnd LogicalOperator = "AND"
LogicalOr LogicalOperator = "OR"
)
// LogicalCondition composes two conditions with a logical operator.
type LogicalCondition struct {
Operator LogicalOperator
Left Condition
Right Condition
}
func (*LogicalCondition) isCondition() {}
// NotCondition negates a child condition.
type NotCondition struct {
Expr Condition
}
func (*NotCondition) isCondition() {}
// FieldPredicateCondition asserts that a field evaluates to true.
type FieldPredicateCondition struct {
Field string
}
func (*FieldPredicateCondition) isCondition() {}
// ComparisonOperator lists supported comparison operators.
type ComparisonOperator string
const (
CompareEq ComparisonOperator = "="
CompareNeq ComparisonOperator = "!="
CompareLt ComparisonOperator = "<"
CompareLte ComparisonOperator = "<="
CompareGt ComparisonOperator = ">"
CompareGte ComparisonOperator = ">="
)
// ComparisonCondition represents a binary comparison.
type ComparisonCondition struct {
Left ValueExpr
Operator ComparisonOperator
Right ValueExpr
}
func (*ComparisonCondition) isCondition() {}
// InCondition represents an IN predicate with literal list values.
type InCondition struct {
Left ValueExpr
Values []ValueExpr
}
func (*InCondition) isCondition() {}
// ElementInCondition represents the CEL syntax `"value" in field`.
type ElementInCondition struct {
Element ValueExpr
Field string
}
func (*ElementInCondition) isCondition() {}
// TextMatchMode enumerates LIKE-based string match modes.
type TextMatchMode string
const (
TextMatchContains TextMatchMode = "contains"
TextMatchPrefix TextMatchMode = "prefix"
TextMatchSuffix TextMatchMode = "suffix"
)
// TextMatchCondition models a case-insensitive LIKE match on a scalar string field
// (content.contains/startsWith/endsWith).
type TextMatchCondition struct {
Field string
Mode TextMatchMode
Value string
}
func (*TextMatchCondition) isCondition() {}
// RegexCondition models field.matches("pattern") on a string field.
type RegexCondition struct {
Field string
Pattern string
}
func (*RegexCondition) isCondition() {}
// ConstantCondition captures a literal boolean outcome.
type ConstantCondition struct {
Value bool
}
func (*ConstantCondition) isCondition() {}
// ValueExpr models arithmetic or scalar expressions whose result feeds a comparison.
type ValueExpr interface {
isValueExpr()
}
// FieldRef references a named schema field.
type FieldRef struct {
Name string
}
func (*FieldRef) isValueExpr() {}
// LiteralValue holds a literal scalar.
type LiteralValue struct {
Value interface{}
}
func (*LiteralValue) isValueExpr() {}
// FunctionValue captures simple function calls like size(tags).
type FunctionValue struct {
Name string
Args []ValueExpr
}
func (*FunctionValue) isValueExpr() {}
// FieldAccessorValue captures a CEL timestamp accessor on a field, such as
// created_ts.getMonth(). It renders to a dialect-specific date-part extraction.
type FieldAccessorValue struct {
Field string
Accessor string // e.g. "getFullYear", "getMonth"
}
func (*FieldAccessorValue) isValueExpr() {}
// ListComprehensionCondition represents CEL macros like exists(), all(), filter().
type ListComprehensionCondition struct {
Kind ComprehensionKind
Field string // The list field to iterate over (e.g., "tags")
IterVar string // The iteration variable name (e.g., "t")
Predicate PredicateExpr // The predicate to evaluate for each element
}
func (*ListComprehensionCondition) isCondition() {}
// ComprehensionKind enumerates the types of list comprehensions.
type ComprehensionKind string
const (
ComprehensionExists ComprehensionKind = "exists"
ComprehensionAll ComprehensionKind = "all"
ComprehensionExistsOne ComprehensionKind = "exists_one"
)
// PredicateExpr represents predicates used in comprehensions.
type PredicateExpr interface {
isPredicateExpr()
}
// StartsWithPredicate represents t.startsWith("prefix").
type StartsWithPredicate struct {
Prefix string
}
func (*StartsWithPredicate) isPredicateExpr() {}
// EndsWithPredicate represents t.endsWith("suffix").
type EndsWithPredicate struct {
Suffix string
}
func (*EndsWithPredicate) isPredicateExpr() {}
// ContainsPredicate represents t.contains("substring").
type ContainsPredicate struct {
Substring string
}
func (*ContainsPredicate) isPredicateExpr() {}
// EqualsPredicate represents t == "value".
type EqualsPredicate struct {
Value string
}
func (*EqualsPredicate) isPredicateExpr() {}

Some files were not shown because too many files have changed in this diff Show More