chore: import upstream snapshot with attribution
Backend Tests / Tests (other) (push) Failing after 1s
Backend Tests / Static Checks (push) Failing after 2s
Backend Tests / Tests (internal) (push) Failing after 2s
Backend Tests / Tests (server) (push) Failing after 1s
Backend Tests / Tests (store) (push) Failing after 1s
Build Canary Image / build-frontend (push) Failing after 1s
Frontend Tests / Lint (push) Failing after 1s
Build Canary Image / build-push (linux/amd64) (push) Has been skipped
Build Canary Image / build-push (linux/arm64) (push) Has been skipped
Build Canary Image / merge (push) Has been skipped
Frontend Tests / Build (push) Failing after 1s
Release Please / release-please (push) Failing after 0s
Proto Linter / Lint Protos (push) Failing after 2s

This commit is contained in:
wehub-resource-sync
2026-07-13 12:02:24 +08:00
commit 4cddfcf2f3
1040 changed files with 200957 additions and 0 deletions
+50
View File
@@ -0,0 +1,50 @@
# Maintaining the Memo Filter Engine
The engine is memo-specific; any future field or behavior changes must stay
consistent with the memo schema and store implementations. Use this guide when
extending or debugging the package.
## Adding a New Memo Field
1. **Update the schema**
- Add the field entry in `schema.go`.
- Define the backing column (`Column`), JSON path (if applicable), type, and
allowed operators.
- Include the CEL variable in `EnvOptions`.
2. **Adjust parser or renderer (if needed)**
- For non-scalar fields (JSON booleans, lists), add handling in
`parser.go` or extend the renderer helpers.
- Keep validation in the parser (e.g., reject unsupported operators).
3. **Write a golden test**
- Extend the dialect-specific memo filter tests under
`store/db/{sqlite,mysql,postgres}/memo_filter_test.go` with a case that
exercises the new field.
4. **Run `go test ./...`** to ensure the SQL output matches expectations across
all dialects.
## Supporting Dialect Nuances
- Centralize differences inside `render.go`. If a new dialect-specific behavior
emerges (e.g., JSON operators), add the logic there rather than leaking it
into store code.
- Use the renderer helpers (`jsonExtractExpr`, `jsonArrayExpr`, etc.) rather than
sprinkling ad-hoc SQL strings.
- When placeholders change, adjust `addArg` so that argument numbering stays in
sync with store queries.
## Debugging Tips
- **Parser errors** Most originate in `buildCondition` or schema validation.
Enable logging around `parser.go` when diagnosing unknown identifier/operator
messages.
- **Renderer output** Temporary printf/log statements in `renderCondition` help
identify which IR node produced unexpected SQL.
- **Store integration** Ensure drivers call `filter.DefaultEngine()` exactly once
per process; the singleton caches the parsed CEL environment.
## Testing Checklist
- `go test ./store/...` ensures all dialect tests consume the engine correctly.
- Add targeted unit tests whenever new IR nodes or renderer paths are introduced.
- When changing boolean or JSON handling, verify all three dialect test suites
(SQLite, MySQL, Postgres) to avoid regression.
+105
View File
@@ -0,0 +1,105 @@
# Memo Filter Engine
This package houses the memo-only filter engine that turns standard CEL syntax
into SQL fragments for the subset of expressions supported by the memo schema.
The engine follows a three phase pipeline inspired by systems
such as Calcite or Prisma:
1. **Parsing** CEL expressions are parsed with `cel-go` and validated against
the memo-specific environment declared in `schema.go`. Only fields that
exist in the schema can surface in the filter, and non-standard legacy
coercions are rejected.
2. **Normalization** the raw CEL AST is converted into an intermediate
representation (IR) defined in `ir.go`. The IR is a dialect-agnostic tree of
conditions (logical operators, comparisons, list membership, etc.). This
step enforces schema rules (e.g. operator compatibility, type checks).
3. **Rendering** the renderer in `render.go` walks the IR and produces a SQL
fragment plus placeholder arguments tailored to a target dialect
(`sqlite`, `mysql`, or `postgres`). Dialect differences such as JSON access,
boolean semantics, placeholders, and `LIKE` vs `ILIKE` are encapsulated in
renderer helpers.
The entry point is `filter.DefaultEngine()` from `engine.go`. It lazily constructs
an `Engine` configured with the memo schema and exposes:
```go
engine, _ := filter.DefaultEngine()
stmt, _ := engine.CompileToStatement(ctx, `has_task_list && visibility == "PUBLIC"`, filter.RenderOptions{
Dialect: filter.DialectPostgres,
})
// stmt.SQL -> "((memo.payload->'property'->>'hasTaskList')::boolean IS TRUE AND memo.visibility = $1)"
// stmt.Args -> ["PUBLIC"]
```
## Core Files
| File | Responsibility |
| ------------- | ------------------------------------------------------------------------------- |
| `schema.go` | Declares memo fields, their types, backing columns, CEL environment options |
| `ir.go` | IR node definitions used across the pipeline |
| `parser.go` | Converts CEL `Expr` into IR while applying schema validation |
| `render.go` | Translates IR into SQL, handling dialect-specific behavior |
| `engine.go` | Glue between the phases; exposes `Compile`, `CompileToStatement`, and `DefaultEngine` |
| `helpers.go` | Convenience helpers for store integration (appending conditions) |
## SQL Generation Notes
- **Placeholders** — `?` is used for SQLite/MySQL, `$n` for Postgres. The renderer
tracks offsets to compose queries with pre-existing arguments.
- **JSON Fields** — Memo metadata lives in `memo.payload`. The renderer handles
`JSON_EXTRACT`/`json_extract`/`->`/`->>` variations and boolean coercion.
- **Time Fields** — `created_ts`, `updated_ts`, and attachment `create_time` are
CEL `timestamp` values. Express instants with the `now` variable,
`duration("…")` (e.g. `created_ts >= now - duration("24h")`), or
`timestamp("2006-01-02T15:04:05Z")` / `timestamp(<epoch-seconds>)`. These fold
to epoch seconds at compile time — `now` is frozen once per compile (injectable
for tests via the engine clock) — so the backing columns stay unchanged.
- **Tag Operations** — `tag in [...]` and `"tag" in tags` become JSON array
predicates. SQLite uses `LIKE` patterns, MySQL uses `JSON_CONTAINS`, and
Postgres uses `@>`.
- **Boolean Flags** — Fields such as `has_task_list` render as `IS TRUE` equality
checks, or comparisons against `CAST('true' AS JSON)` depending on the dialect.
- **String Matching** — `content.contains(x)`, `content.startsWith(x)`, and
`content.endsWith(x)` render as case-insensitive `LIKE`/`ILIKE` with LIKE
metacharacters (`%`, `_`, `\`) escaped. Available on scalar string fields whose
schema sets `SupportsContains` (memo `content`; attachment `filename`,
`mime_type`).
- **Regex** — `field.matches("pattern")` renders to `~` (Postgres) or `REGEXP`
(MySQL/SQLite). SQLite uses a Go-backed `regexp` function registered in
`store/db/sqlite/functions.go`. Patterns are validated at compile time against
Go's RE2 via `cel.ValidateRegexLiterals()`. **Caveat:** regex *syntax* differs
per engine (Go RE2 on SQLite, POSIX ERE on Postgres, ICU on MySQL 8.0+), so
engine-specific patterns may not be portable.
- **Tag `all()` / `exists_one()`** — `tags.all(t, <pred>)` matches only non-empty
tag sets where every element satisfies the predicate; `tags.exists_one(t,
<pred>)` matches when exactly one element does (`COUNT(...) = 1`). Both iterate
per-element (`json_each` / `jsonb_array_elements_text` / `JSON_TABLE`).
- **Timestamp Accessors** — `created_ts.getFullYear()`, `getMonth()`, `getDate()`,
`getDayOfMonth()`, `getDayOfWeek()`, `getDayOfYear()`, `getHours()`,
`getMinutes()`, `getSeconds()` render to date-part extraction (`strftime` /
`EXTRACT` / `YEAR`/`MONTH`/…). Results are normalized to CEL's base (0-based
month, 0-based day-of-week with 0 = Sunday). The same accessors on `now` fold
to literal date parts of the frozen evaluation time (UTC), so saved filters
like `created_ts.getMonth() == now.getMonth() && created_ts.getDate() ==
now.getDate()` ("on this day") re-resolve on every compile. Extraction is UTC
on SQLite/Postgres (epoch columns); on MySQL the `TIMESTAMP` column is read in
the session time zone. A timezone argument is not supported.
- **Set Operations** — `ext.Sets()`: `sets.contains(tags, [...])`,
`sets.intersects(tags, [...])`, and `sets.equivalent(tags, [...])` desugar to
exact-membership checks (AND / OR of `"v" in tags`); `equivalent` adds a
`size(tags)` length check (relies on tags being a set).
- **`size()`** — `size(tags)` renders to JSON array length; `size(content)` (and
other string fields) render to `LENGTH` / `CHAR_LENGTH` (MySQL) for code-point
counts.
- **Arithmetic** — `+`, `-`, `*`, `/`, `%` constant-fold on literal/`now`/`duration`
operands (division and modulo guard against a zero divisor).
## Typical Integration
1. Fetch the engine with `filter.DefaultEngine()`.
2. Call `CompileToStatement` using the appropriate dialect enum.
3. Append the emitted SQL fragment/args to the existing `WHERE` clause.
4. Execute the resulting query through the store driver.
The `helpers.AppendConditions` helper encapsulates steps 23 when a driver needs
to process an array of filters.
+124
View File
@@ -0,0 +1,124 @@
package filter
import (
"context"
"strings"
"sync"
"time"
"github.com/google/cel-go/cel"
"github.com/pkg/errors"
)
// Engine parses CEL filters into a dialect-agnostic condition tree.
type Engine struct {
schema Schema
env *cel.Env
// nowFunc resolves the value of the `now` variable. It is frozen once per
// Compile so a single filter sees a single instant, and is overridable in
// tests for deterministic folding.
nowFunc func() time.Time
}
// NewEngine builds a new Engine for the provided schema.
func NewEngine(schema Schema) (*Engine, error) {
env, err := cel.NewEnv(schema.EnvOptions...)
if err != nil {
return nil, errors.Wrap(err, "failed to create CEL environment")
}
return &Engine{
schema: schema,
env: env,
nowFunc: time.Now,
}, nil
}
// Program stores a compiled filter condition.
type Program struct {
schema Schema
condition Condition
}
// ConditionTree exposes the underlying condition tree.
func (p *Program) ConditionTree() Condition {
return p.condition
}
// Compile parses the filter string into an executable program.
func (e *Engine) Compile(_ context.Context, filter string) (*Program, error) {
if strings.TrimSpace(filter) == "" {
return nil, errors.New("filter expression is empty")
}
ast, issues := e.env.Compile(filter)
if issues != nil && issues.Err() != nil {
return nil, errors.Wrap(issues.Err(), "failed to compile filter")
}
parsed, err := cel.AstToParsedExpr(ast)
if err != nil {
return nil, errors.Wrap(err, "failed to convert AST")
}
cond, err := buildCondition(parsed.GetExpr(), parseContext{schema: e.schema, now: e.nowFunc()})
if err != nil {
return nil, err
}
return &Program{
schema: e.schema,
condition: cond,
}, nil
}
// CompileToStatement compiles and renders the filter in a single step.
func (e *Engine) CompileToStatement(ctx context.Context, filter string, opts RenderOptions) (Statement, error) {
program, err := e.Compile(ctx, filter)
if err != nil {
return Statement{}, err
}
return program.Render(opts)
}
// RenderOptions configure SQL rendering.
type RenderOptions struct {
Dialect DialectName
PlaceholderOffset int
DisableNullChecks bool
}
// Statement contains the rendered SQL fragment and its args.
type Statement struct {
SQL string
Args []any
}
// Render converts the program into a dialect-specific SQL fragment.
func (p *Program) Render(opts RenderOptions) (Statement, error) {
renderer := newRenderer(p.schema, opts)
return renderer.Render(p.condition)
}
var (
defaultOnce sync.Once
defaultInst *Engine
defaultErr error
defaultAttachmentOnce sync.Once
defaultAttachmentInst *Engine
defaultAttachmentErr error
)
// DefaultEngine returns the process-wide memo filter engine.
func DefaultEngine() (*Engine, error) {
defaultOnce.Do(func() {
defaultInst, defaultErr = NewEngine(NewSchema())
})
return defaultInst, defaultErr
}
// DefaultAttachmentEngine returns the process-wide attachment filter engine.
func DefaultAttachmentEngine() (*Engine, error) {
defaultAttachmentOnce.Do(func() {
defaultAttachmentInst, defaultAttachmentErr = NewEngine(NewAttachmentSchema())
})
return defaultAttachmentInst, defaultAttachmentErr
}
+197
View File
@@ -0,0 +1,197 @@
package filter
import (
"context"
"testing"
"github.com/stretchr/testify/require"
)
func TestCompileAcceptsStandardTagEqualityPredicate(t *testing.T) {
t.Parallel()
engine, err := NewEngine(NewSchema())
require.NoError(t, err)
_, err = engine.Compile(context.Background(), `tags.exists(t, t == "1231")`)
require.NoError(t, err)
}
func TestCompileRejectsLegacyNumericLogicalOperand(t *testing.T) {
t.Parallel()
engine, err := NewEngine(NewSchema())
require.NoError(t, err)
_, err = engine.Compile(context.Background(), `pinned && 1`)
require.Error(t, err)
require.Contains(t, err.Error(), "failed to compile filter")
}
func TestCompileRejectsNonBooleanTopLevelConstant(t *testing.T) {
t.Parallel()
engine, err := NewEngine(NewSchema())
require.NoError(t, err)
_, err = engine.Compile(context.Background(), `1`)
require.EqualError(t, err, "filter must evaluate to a boolean value")
}
func TestCompileRejectsMalformedRegex(t *testing.T) {
t.Parallel()
engine, err := NewEngine(NewSchema())
require.NoError(t, err)
_, err = engine.Compile(context.Background(), `content.matches("(")`)
require.Error(t, err)
}
func TestCompileMatchesRendersRegexOperator(t *testing.T) {
t.Parallel()
engine, err := NewEngine(NewSchema())
require.NoError(t, err)
stmt, err := engine.CompileToStatement(context.Background(), `content.matches("v[0-9]+")`, RenderOptions{Dialect: DialectPostgres})
require.NoError(t, err)
require.Contains(t, stmt.SQL, "~")
require.Equal(t, []any{"v[0-9]+"}, stmt.Args)
}
func TestCompileRejectsStartsWithOnUnsupportedField(t *testing.T) {
t.Parallel()
engine, err := NewEngine(NewSchema())
require.NoError(t, err)
_, err = engine.Compile(context.Background(), `visibility.startsWith("P")`)
require.Error(t, err)
require.Contains(t, err.Error(), "does not support text matching")
}
func TestCompileContainsEscapesLikeWildcards(t *testing.T) {
t.Parallel()
engine, err := NewEngine(NewSchema())
require.NoError(t, err)
stmt, err := engine.CompileToStatement(context.Background(), `content.contains("50%_off")`, RenderOptions{Dialect: DialectSQLite})
require.NoError(t, err)
// The % and _ in the value must be escaped so they are matched literally,
// and SQLite needs an explicit ESCAPE clause.
require.Contains(t, stmt.SQL, `ESCAPE '\'`)
require.Equal(t, []any{`%50\%\_off%`}, stmt.Args)
}
// =============================================================================
// Cross-dialect rendering tests (no DB required; complements the SQLite-only
// behavioral tests in store/test by asserting MySQL/Postgres SQL generation).
// =============================================================================
func TestRenderStartsWithPerDialect(t *testing.T) {
t.Parallel()
engine, err := NewEngine(NewSchema())
require.NoError(t, err)
cases := []struct {
dialect DialectName
fragments []string
}{
{DialectSQLite, []string{"memos_unicode_lower(", "`memo`.`content`", `ESCAPE '\'`}},
{DialectPostgres, []string{"memo.content ILIKE $1"}},
{DialectMySQL, []string{"`memo`.`content` LIKE ?"}},
}
for _, tc := range cases {
stmt, err := engine.CompileToStatement(context.Background(), `content.startsWith("TODO")`, RenderOptions{Dialect: tc.dialect})
require.NoError(t, err, tc.dialect)
for _, frag := range tc.fragments {
require.Contains(t, stmt.SQL, frag, "dialect %s", tc.dialect)
}
require.Equal(t, []any{"TODO%"}, stmt.Args, "dialect %s", tc.dialect)
}
}
func TestRenderEndsWithPerDialect(t *testing.T) {
t.Parallel()
engine, err := NewEngine(NewSchema())
require.NoError(t, err)
for _, dialect := range []DialectName{DialectSQLite, DialectPostgres, DialectMySQL} {
stmt, err := engine.CompileToStatement(context.Background(), `content.endsWith(".md")`, RenderOptions{Dialect: dialect})
require.NoError(t, err, dialect)
require.Equal(t, []any{"%.md"}, stmt.Args, "dialect %s", dialect)
}
}
func TestRenderMatchesPerDialect(t *testing.T) {
t.Parallel()
engine, err := NewEngine(NewSchema())
require.NoError(t, err)
cases := []struct {
dialect DialectName
fragment string
}{
{DialectSQLite, "`memo`.`content` REGEXP ?"},
{DialectMySQL, "`memo`.`content` REGEXP ?"},
{DialectPostgres, "memo.content ~ $1"},
}
for _, tc := range cases {
stmt, err := engine.CompileToStatement(context.Background(), `content.matches("v[0-9]+")`, RenderOptions{Dialect: tc.dialect})
require.NoError(t, err, tc.dialect)
require.Contains(t, stmt.SQL, tc.fragment, "dialect %s", tc.dialect)
require.Equal(t, []any{"v[0-9]+"}, stmt.Args, "dialect %s", tc.dialect)
}
}
func TestRenderTagsAllPerDialect(t *testing.T) {
t.Parallel()
engine, err := NewEngine(NewSchema())
require.NoError(t, err)
cases := []struct {
dialect DialectName
fragments []string
}{
{DialectSQLite, []string{"NOT EXISTS", "json_each(", "!= '[]'", "memos_unicode_lower(value)"}},
{DialectPostgres, []string{"NOT EXISTS", "jsonb_array_elements_text(", "jsonb_array_length(", "value ILIKE"}},
{DialectMySQL, []string{"NOT EXISTS", "JSON_TABLE(", "JSON_LENGTH(", "value LIKE"}},
}
for _, tc := range cases {
stmt, err := engine.CompileToStatement(context.Background(), `tags.all(t, t.startsWith("work/"))`, RenderOptions{Dialect: tc.dialect})
require.NoError(t, err, tc.dialect)
for _, frag := range tc.fragments {
require.Contains(t, stmt.SQL, frag, "dialect %s", tc.dialect)
}
require.Equal(t, []any{"work/%"}, stmt.Args, "dialect %s", tc.dialect)
}
}
func TestRenderTextMatchEscaping(t *testing.T) {
t.Parallel()
engine, err := NewEngine(NewSchema())
require.NoError(t, err)
// Both % and _ in the value must be escaped so they match literally.
stmt, err := engine.CompileToStatement(context.Background(), `content.contains("a%b_c")`, RenderOptions{Dialect: DialectSQLite})
require.NoError(t, err)
require.Equal(t, []any{`%a\%b\_c%`}, stmt.Args)
}
func TestRenderAllRejectsUnsupportedPredicate(t *testing.T) {
t.Parallel()
engine, err := NewEngine(NewSchema())
require.NoError(t, err)
// size() is not a valid per-element predicate inside all().
_, err = engine.CompileToStatement(context.Background(), `tags.all(t, size(t) > 2)`, RenderOptions{Dialect: DialectSQLite})
require.Error(t, err)
}
+377
View File
@@ -0,0 +1,377 @@
package filter
import (
"context"
"testing"
"time"
"github.com/stretchr/testify/require"
)
// ---------------------------------------------------------------------------
// Arithmetic folding: division and modulo
// ---------------------------------------------------------------------------
func TestCompileDivisionFolds(t *testing.T) {
t.Parallel()
engine, err := NewEngine(NewSchema())
require.NoError(t, err)
stmt, err := engine.CompileToStatement(context.Background(), `creator_id == 100 / 10`, RenderOptions{Dialect: DialectSQLite})
require.NoError(t, err)
require.Equal(t, []any{int64(10)}, stmt.Args)
}
func TestCompileModuloFolds(t *testing.T) {
t.Parallel()
engine, err := NewEngine(NewSchema())
require.NoError(t, err)
stmt, err := engine.CompileToStatement(context.Background(), `creator_id == 17 % 5`, RenderOptions{Dialect: DialectSQLite})
require.NoError(t, err)
require.Equal(t, []any{int64(2)}, stmt.Args)
}
func TestCompileDivisionByZeroErrors(t *testing.T) {
t.Parallel()
engine, err := NewEngine(NewSchema())
require.NoError(t, err)
_, err = engine.Compile(context.Background(), `creator_id == 10 / 0`)
require.Error(t, err)
}
// ---------------------------------------------------------------------------
// size() on scalar string fields -> SQL length
// ---------------------------------------------------------------------------
func TestCompileSizeOnContentRendersLengthPerDialect(t *testing.T) {
t.Parallel()
engine, err := NewEngine(NewSchema())
require.NoError(t, err)
cases := []struct {
dialect DialectName
fragment string
}{
{DialectSQLite, "LENGTH("},
{DialectPostgres, "LENGTH("},
{DialectMySQL, "CHAR_LENGTH("},
}
for _, tc := range cases {
stmt, err := engine.CompileToStatement(context.Background(), `size(content) > 5`, RenderOptions{Dialect: tc.dialect})
require.NoError(t, err, tc.dialect)
require.Contains(t, stmt.SQL, tc.fragment, "dialect %s", tc.dialect)
require.Equal(t, []any{int64(5)}, stmt.Args, "dialect %s", tc.dialect)
}
}
// ---------------------------------------------------------------------------
// Timestamp accessor methods (getFullYear, getMonth, ...)
// ---------------------------------------------------------------------------
func TestCompileTimestampAccessorsPerDialect(t *testing.T) {
t.Parallel()
engine, err := NewEngine(NewSchema())
require.NoError(t, err)
cases := []struct {
name string
filter string
dialect DialectName
fragments []string
arg int64
}{
// getFullYear == 2024
{"sqlite year", `created_ts.getFullYear() == 2024`, DialectSQLite, []string{"strftime('%Y'", "'unixepoch'"}, 2024},
{"pg year", `created_ts.getFullYear() == 2024`, DialectPostgres, []string{"EXTRACT(YEAR FROM to_timestamp(", "AT TIME ZONE 'UTC'"}, 2024},
{"mysql year", `created_ts.getFullYear() == 2024`, DialectMySQL, []string{"YEAR(`memo`.`created_ts`)"}, 2024},
// getMonth is 0-based -> SQL must subtract 1
{"sqlite month", `created_ts.getMonth() == 5`, DialectSQLite, []string{"strftime('%m'", "- 1)"}, 5},
{"pg month", `created_ts.getMonth() == 5`, DialectPostgres, []string{"EXTRACT(MONTH FROM", "- 1)"}, 5},
{"mysql month", `created_ts.getMonth() == 5`, DialectMySQL, []string{"MONTH(`memo`.`created_ts`)", "- 1)"}, 5},
// getDayOfWeek 0=Sunday -> MySQL DAYOFWEEK is 1-based and must subtract 1
{"mysql dow", `created_ts.getDayOfWeek() == 0`, DialectMySQL, []string{"DAYOFWEEK(`memo`.`created_ts`)", "- 1)"}, 0},
{"sqlite dow", `created_ts.getDayOfWeek() == 0`, DialectSQLite, []string{"strftime('%w'"}, 0},
// getDate is 1-based -> no offset
{"sqlite date", `created_ts.getDate() == 22`, DialectSQLite, []string{"strftime('%d'"}, 22},
}
for _, tc := range cases {
stmt, err := engine.CompileToStatement(context.Background(), tc.filter, RenderOptions{Dialect: tc.dialect})
require.NoError(t, err, tc.name)
for _, frag := range tc.fragments {
require.Contains(t, stmt.SQL, frag, tc.name)
}
require.Equal(t, []any{tc.arg}, stmt.Args, tc.name)
}
}
func TestCompileNowAccessorsFoldToInjectedClock(t *testing.T) {
t.Parallel()
// 2026-07-07T10:30:45Z, a Tuesday (year day 188).
engine := memoEngineAt(t, time.Date(2026, time.July, 7, 10, 30, 45, 0, time.UTC).Unix())
cases := []struct {
name string
filter string
args []any
}{
{"on this day", `created_ts.getMonth() == now.getMonth() && created_ts.getDate() == now.getDate()`, []any{int64(6), int64(7)}},
{"year", `created_ts.getFullYear() < now.getFullYear()`, []any{int64(2026)}},
{"day of month", `created_ts.getDayOfMonth() == now.getDayOfMonth()`, []any{int64(6)}},
{"day of week", `created_ts.getDayOfWeek() == now.getDayOfWeek()`, []any{int64(2)}},
{"day of year", `created_ts.getDayOfYear() == now.getDayOfYear()`, []any{int64(187)}},
{"clock parts", `created_ts.getHours() == now.getHours() || created_ts.getMinutes() == now.getMinutes() || created_ts.getSeconds() == now.getSeconds()`, []any{int64(10), int64(30), int64(45)}},
}
for _, tc := range cases {
stmt, err := engine.CompileToStatement(context.Background(), tc.filter, RenderOptions{Dialect: DialectSQLite})
require.NoError(t, err, tc.name)
require.Equal(t, tc.args, stmt.Args, tc.name)
}
}
func TestCompileNowAccessorRejectsTimezoneArg(t *testing.T) {
t.Parallel()
engine, err := NewEngine(NewSchema())
require.NoError(t, err)
_, err = engine.Compile(context.Background(), `created_ts.getMonth() == now.getMonth("America/New_York")`)
require.Error(t, err)
}
func TestCompileOnThisDayPerDialect(t *testing.T) {
t.Parallel()
const onThisDay = `created_ts.getMonth() == now.getMonth() && created_ts.getDate() == now.getDate()`
engine := memoEngineAt(t, time.Date(2026, time.July, 7, 10, 30, 45, 0, time.UTC).Unix())
cases := []struct {
dialect DialectName
fragments []string
}{
{DialectSQLite, []string{"strftime('%m'", "strftime('%d'", "'unixepoch'"}},
{DialectPostgres, []string{"EXTRACT(MONTH FROM", "EXTRACT(DAY FROM"}},
{DialectMySQL, []string{"MONTH(`memo`.`created_ts`)", "DAYOFMONTH(`memo`.`created_ts`)"}},
}
for _, tc := range cases {
stmt, err := engine.CompileToStatement(context.Background(), onThisDay, RenderOptions{Dialect: tc.dialect})
require.NoError(t, err, tc.dialect)
for _, frag := range tc.fragments {
require.Contains(t, stmt.SQL, frag, "dialect %s", tc.dialect)
}
require.Equal(t, []any{int64(6), int64(7)}, stmt.Args, "dialect %s", tc.dialect)
}
}
func TestCompileNowAccessorsAtBoundaryDates(t *testing.T) {
t.Parallel()
// Every accessor folded at calendar edges: year rollover, leap day, Sunday.
allAccessors := `created_ts.getFullYear() == now.getFullYear() ` +
`&& created_ts.getMonth() == now.getMonth() ` +
`&& created_ts.getDate() == now.getDate() ` +
`&& created_ts.getDayOfMonth() == now.getDayOfMonth() ` +
`&& created_ts.getDayOfWeek() == now.getDayOfWeek() ` +
`&& created_ts.getDayOfYear() == now.getDayOfYear() ` +
`&& created_ts.getHours() == now.getHours() ` +
`&& created_ts.getMinutes() == now.getMinutes() ` +
`&& created_ts.getSeconds() == now.getSeconds()`
cases := []struct {
name string
clock time.Time
// year, month, date, dayOfMonth, dayOfWeek, dayOfYear, hours, minutes, seconds
args []any
}{
{
"new year's eve",
time.Date(2026, time.December, 31, 23, 59, 59, 0, time.UTC),
[]any{int64(2026), int64(11), int64(31), int64(30), int64(4), int64(364), int64(23), int64(59), int64(59)},
},
{
"new year's day",
time.Date(2027, time.January, 1, 0, 0, 0, 0, time.UTC),
[]any{int64(2027), int64(0), int64(1), int64(0), int64(5), int64(0), int64(0), int64(0), int64(0)},
},
{
"leap day",
time.Date(2028, time.February, 29, 12, 0, 0, 0, time.UTC),
[]any{int64(2028), int64(1), int64(29), int64(28), int64(2), int64(59), int64(12), int64(0), int64(0)},
},
{
"sunday",
time.Date(2026, time.July, 5, 8, 15, 30, 0, time.UTC),
[]any{int64(2026), int64(6), int64(5), int64(4), int64(0), int64(185), int64(8), int64(15), int64(30)},
},
}
for _, tc := range cases {
engine := memoEngineAt(t, tc.clock.Unix())
stmt, err := engine.CompileToStatement(context.Background(), allAccessors, RenderOptions{Dialect: DialectSQLite})
require.NoError(t, err, tc.name)
require.Equal(t, tc.args, stmt.Args, tc.name)
}
}
func TestCompileNowAccessorFoldsInUTC(t *testing.T) {
t.Parallel()
// A clock in UTC+8 at 05:00 on July 8 is still July 7, 21:00 in UTC;
// folding must not leak the clock's zone.
engine, err := NewEngine(NewSchema())
require.NoError(t, err)
engine.nowFunc = func() time.Time {
return time.Date(2026, time.July, 8, 5, 0, 0, 0, time.FixedZone("UTC+8", 8*3600))
}
stmt, err := engine.CompileToStatement(
context.Background(),
`created_ts.getDate() == now.getDate() && created_ts.getHours() == now.getHours()`,
RenderOptions{Dialect: DialectSQLite},
)
require.NoError(t, err)
require.Equal(t, []any{int64(7), int64(21)}, stmt.Args)
}
func TestCompileNowAccessorOnLeftSide(t *testing.T) {
t.Parallel()
engine := memoEngineAt(t, time.Date(2026, time.July, 7, 10, 30, 45, 0, time.UTC).Unix())
stmt, err := engine.CompileToStatement(context.Background(), `now.getMonth() == created_ts.getMonth()`, RenderOptions{Dialect: DialectSQLite})
require.NoError(t, err)
require.Contains(t, stmt.SQL, "strftime('%m'")
require.Equal(t, []any{int64(6)}, stmt.Args)
}
func TestCompileNowAccessorMirrorsOrderingOperator(t *testing.T) {
t.Parallel()
// Swapping the literal to the right must flip < to > to keep the meaning.
engine := memoEngineAt(t, time.Date(2026, time.July, 7, 10, 30, 45, 0, time.UTC).Unix())
stmt, err := engine.CompileToStatement(context.Background(), `now.getHours() < created_ts.getHours()`, RenderOptions{Dialect: DialectSQLite})
require.NoError(t, err)
require.Contains(t, stmt.SQL, "> ?")
require.Equal(t, []any{int64(10)}, stmt.Args)
}
func TestCompileNowAccessorAgainstLiteralFoldsToConstant(t *testing.T) {
t.Parallel()
// Both sides fold to literals; the comparison folds to a constant condition.
engine := memoEngineAt(t, time.Date(2026, time.July, 7, 10, 30, 45, 0, time.UTC).Unix())
// True → trivial filter (empty SQL, matches everything).
stmt, err := engine.CompileToStatement(context.Background(), `now.getFullYear() >= 2026`, RenderOptions{Dialect: DialectSQLite})
require.NoError(t, err)
require.Empty(t, stmt.SQL)
require.Empty(t, stmt.Args)
// False → unsatisfiable filter.
stmt, err = engine.CompileToStatement(context.Background(), `now.getFullYear() < 2026`, RenderOptions{Dialect: DialectSQLite})
require.NoError(t, err)
require.Equal(t, "1 = 0", stmt.SQL)
require.Empty(t, stmt.Args)
}
func TestCompileAttachmentNowAccessors(t *testing.T) {
t.Parallel()
engine, err := NewEngine(NewAttachmentSchema())
require.NoError(t, err)
engine.nowFunc = fixedClock(time.Date(2026, time.July, 7, 10, 30, 45, 0, time.UTC).Unix())
stmt, err := engine.CompileToStatement(
context.Background(),
`create_time.getMonth() == now.getMonth() && create_time.getDate() == now.getDate()`,
RenderOptions{Dialect: DialectSQLite},
)
require.NoError(t, err)
require.Contains(t, stmt.SQL, "`attachment`.`created_ts`")
require.Equal(t, []any{int64(6), int64(7)}, stmt.Args)
}
func TestCompileTimestampAccessorRejectsTimezoneArg(t *testing.T) {
t.Parallel()
engine, err := NewEngine(NewSchema())
require.NoError(t, err)
_, err = engine.Compile(context.Background(), `created_ts.getHours("America/New_York") == 9`)
require.Error(t, err)
}
func TestCompileTimestampAccessorRejectsNonTimestampField(t *testing.T) {
t.Parallel()
engine, err := NewEngine(NewSchema())
require.NoError(t, err)
// content is a string, not a timestamp.
_, err = engine.Compile(context.Background(), `content.getFullYear() == 2024`)
require.Error(t, err)
}
// ---------------------------------------------------------------------------
// ext.Sets(): sets.contains / sets.intersects / sets.equivalent over tags
// ---------------------------------------------------------------------------
func TestCompileSetsContainsRendersAndOfMemberships(t *testing.T) {
t.Parallel()
engine, err := NewEngine(NewSchema())
require.NoError(t, err)
stmt, err := engine.CompileToStatement(context.Background(), `sets.contains(tags, ["a", "b"])`, RenderOptions{Dialect: DialectSQLite})
require.NoError(t, err)
require.Contains(t, stmt.SQL, " AND ")
require.Len(t, stmt.Args, 2)
}
func TestCompileSetsIntersectsRendersOrOfMemberships(t *testing.T) {
t.Parallel()
engine, err := NewEngine(NewSchema())
require.NoError(t, err)
stmt, err := engine.CompileToStatement(context.Background(), `sets.intersects(tags, ["a", "b"])`, RenderOptions{Dialect: DialectSQLite})
require.NoError(t, err)
require.Contains(t, stmt.SQL, " OR ")
require.Len(t, stmt.Args, 2)
}
func TestCompileSetsEquivalentAddsLengthCheck(t *testing.T) {
t.Parallel()
engine, err := NewEngine(NewSchema())
require.NoError(t, err)
stmt, err := engine.CompileToStatement(context.Background(), `sets.equivalent(tags, ["a", "b"])`, RenderOptions{Dialect: DialectSQLite})
require.NoError(t, err)
require.Contains(t, stmt.SQL, "JSON_ARRAY_LENGTH")
require.Contains(t, stmt.Args, int64(2))
}
// ---------------------------------------------------------------------------
// exists_one() comprehension on tags
// ---------------------------------------------------------------------------
func TestCompileExistsOnePerDialect(t *testing.T) {
t.Parallel()
engine, err := NewEngine(NewSchema())
require.NoError(t, err)
cases := []struct {
dialect DialectName
fragments []string
}{
{DialectSQLite, []string{"COUNT(", "json_each(", ") = 1"}},
{DialectPostgres, []string{"COUNT(", "jsonb_array_elements_text(", ") = 1"}},
{DialectMySQL, []string{"COUNT(", "JSON_TABLE(", ") = 1"}},
}
for _, tc := range cases {
stmt, err := engine.CompileToStatement(context.Background(), `tags.exists_one(t, t == "urgent")`, RenderOptions{Dialect: tc.dialect})
require.NoError(t, err, tc.dialect)
for _, frag := range tc.fragments {
require.Contains(t, stmt.SQL, frag, "dialect %s", tc.dialect)
}
require.Equal(t, []any{"urgent"}, stmt.Args, "dialect %s", tc.dialect)
}
}
+25
View File
@@ -0,0 +1,25 @@
package filter
import (
"context"
"fmt"
)
// AppendConditions compiles the provided filters and appends the resulting SQL fragments and args.
func AppendConditions(ctx context.Context, engine *Engine, filters []string, dialect DialectName, where *[]string, args *[]any) error {
for _, filterStr := range filters {
stmt, err := engine.CompileToStatement(ctx, filterStr, RenderOptions{
Dialect: dialect,
PlaceholderOffset: len(*args),
})
if err != nil {
return err
}
if stmt.SQL == "" {
continue
}
*where = append(*where, fmt.Sprintf("(%s)", stmt.SQL))
*args = append(*args, stmt.Args...)
}
return nil
}
+196
View File
@@ -0,0 +1,196 @@
package filter
// Condition represents a boolean expression derived from the CEL filter.
type Condition interface {
isCondition()
}
// LogicalOperator enumerates the supported logical operators.
type LogicalOperator string
const (
LogicalAnd LogicalOperator = "AND"
LogicalOr LogicalOperator = "OR"
)
// LogicalCondition composes two conditions with a logical operator.
type LogicalCondition struct {
Operator LogicalOperator
Left Condition
Right Condition
}
func (*LogicalCondition) isCondition() {}
// NotCondition negates a child condition.
type NotCondition struct {
Expr Condition
}
func (*NotCondition) isCondition() {}
// FieldPredicateCondition asserts that a field evaluates to true.
type FieldPredicateCondition struct {
Field string
}
func (*FieldPredicateCondition) isCondition() {}
// ComparisonOperator lists supported comparison operators.
type ComparisonOperator string
const (
CompareEq ComparisonOperator = "="
CompareNeq ComparisonOperator = "!="
CompareLt ComparisonOperator = "<"
CompareLte ComparisonOperator = "<="
CompareGt ComparisonOperator = ">"
CompareGte ComparisonOperator = ">="
)
// ComparisonCondition represents a binary comparison.
type ComparisonCondition struct {
Left ValueExpr
Operator ComparisonOperator
Right ValueExpr
}
func (*ComparisonCondition) isCondition() {}
// InCondition represents an IN predicate with literal list values.
type InCondition struct {
Left ValueExpr
Values []ValueExpr
}
func (*InCondition) isCondition() {}
// ElementInCondition represents the CEL syntax `"value" in field`.
type ElementInCondition struct {
Element ValueExpr
Field string
}
func (*ElementInCondition) isCondition() {}
// TextMatchMode enumerates LIKE-based string match modes.
type TextMatchMode string
const (
TextMatchContains TextMatchMode = "contains"
TextMatchPrefix TextMatchMode = "prefix"
TextMatchSuffix TextMatchMode = "suffix"
)
// TextMatchCondition models a case-insensitive LIKE match on a scalar string field
// (content.contains/startsWith/endsWith).
type TextMatchCondition struct {
Field string
Mode TextMatchMode
Value string
}
func (*TextMatchCondition) isCondition() {}
// RegexCondition models field.matches("pattern") on a string field.
type RegexCondition struct {
Field string
Pattern string
}
func (*RegexCondition) isCondition() {}
// ConstantCondition captures a literal boolean outcome.
type ConstantCondition struct {
Value bool
}
func (*ConstantCondition) isCondition() {}
// ValueExpr models arithmetic or scalar expressions whose result feeds a comparison.
type ValueExpr interface {
isValueExpr()
}
// FieldRef references a named schema field.
type FieldRef struct {
Name string
}
func (*FieldRef) isValueExpr() {}
// LiteralValue holds a literal scalar.
type LiteralValue struct {
Value interface{}
}
func (*LiteralValue) isValueExpr() {}
// FunctionValue captures simple function calls like size(tags).
type FunctionValue struct {
Name string
Args []ValueExpr
}
func (*FunctionValue) isValueExpr() {}
// FieldAccessorValue captures a CEL timestamp accessor on a field, such as
// created_ts.getMonth(). It renders to a dialect-specific date-part extraction.
type FieldAccessorValue struct {
Field string
Accessor string // e.g. "getFullYear", "getMonth"
}
func (*FieldAccessorValue) isValueExpr() {}
// ListComprehensionCondition represents CEL macros like exists(), all(), filter().
type ListComprehensionCondition struct {
Kind ComprehensionKind
Field string // The list field to iterate over (e.g., "tags")
IterVar string // The iteration variable name (e.g., "t")
Predicate PredicateExpr // The predicate to evaluate for each element
}
func (*ListComprehensionCondition) isCondition() {}
// ComprehensionKind enumerates the types of list comprehensions.
type ComprehensionKind string
const (
ComprehensionExists ComprehensionKind = "exists"
ComprehensionAll ComprehensionKind = "all"
ComprehensionExistsOne ComprehensionKind = "exists_one"
)
// PredicateExpr represents predicates used in comprehensions.
type PredicateExpr interface {
isPredicateExpr()
}
// StartsWithPredicate represents t.startsWith("prefix").
type StartsWithPredicate struct {
Prefix string
}
func (*StartsWithPredicate) isPredicateExpr() {}
// EndsWithPredicate represents t.endsWith("suffix").
type EndsWithPredicate struct {
Suffix string
}
func (*EndsWithPredicate) isPredicateExpr() {}
// ContainsPredicate represents t.contains("substring").
type ContainsPredicate struct {
Substring string
}
func (*ContainsPredicate) isPredicateExpr() {}
// EqualsPredicate represents t == "value".
type EqualsPredicate struct {
Value string
}
func (*EqualsPredicate) isPredicateExpr() {}
File diff suppressed because it is too large Load Diff
+980
View File
@@ -0,0 +1,980 @@
package filter
import (
"fmt"
"strings"
"github.com/pkg/errors"
)
type renderer struct {
schema Schema
dialect DialectName
placeholderOffset int
placeholderCounter int
args []any
}
type renderResult struct {
sql string
trivial bool
unsatisfiable bool
}
func newRenderer(schema Schema, opts RenderOptions) *renderer {
return &renderer{
schema: schema,
dialect: opts.Dialect,
placeholderOffset: opts.PlaceholderOffset,
}
}
func (r *renderer) Render(cond Condition) (Statement, error) {
result, err := r.renderCondition(cond)
if err != nil {
return Statement{}, err
}
args := r.args
if args == nil {
args = []any{}
}
switch {
case result.unsatisfiable:
return Statement{
SQL: "1 = 0",
Args: args,
}, nil
case result.trivial:
return Statement{
SQL: "",
Args: args,
}, nil
default:
return Statement{
SQL: result.sql,
Args: args,
}, nil
}
}
func (r *renderer) renderCondition(cond Condition) (renderResult, error) {
switch c := cond.(type) {
case *LogicalCondition:
return r.renderLogicalCondition(c)
case *NotCondition:
return r.renderNotCondition(c)
case *FieldPredicateCondition:
return r.renderFieldPredicate(c)
case *ComparisonCondition:
return r.renderComparison(c)
case *InCondition:
return r.renderInCondition(c)
case *ElementInCondition:
return r.renderElementInCondition(c)
case *TextMatchCondition:
return r.renderTextMatch(c)
case *RegexCondition:
return r.renderRegex(c)
case *ListComprehensionCondition:
return r.renderListComprehension(c)
case *ConstantCondition:
if c.Value {
return renderResult{trivial: true}, nil
}
return renderResult{sql: "1 = 0", unsatisfiable: true}, nil
default:
return renderResult{}, errors.Errorf("unsupported condition type %T", c)
}
}
func (r *renderer) renderLogicalCondition(cond *LogicalCondition) (renderResult, error) {
left, err := r.renderCondition(cond.Left)
if err != nil {
return renderResult{}, err
}
right, err := r.renderCondition(cond.Right)
if err != nil {
return renderResult{}, err
}
switch cond.Operator {
case LogicalAnd:
return combineAnd(left, right), nil
case LogicalOr:
return combineOr(left, right), nil
default:
return renderResult{}, errors.Errorf("unsupported logical operator %s", cond.Operator)
}
}
func (r *renderer) renderNotCondition(cond *NotCondition) (renderResult, error) {
child, err := r.renderCondition(cond.Expr)
if err != nil {
return renderResult{}, err
}
if child.trivial {
return renderResult{sql: "1 = 0", unsatisfiable: true}, nil
}
if child.unsatisfiable {
return renderResult{trivial: true}, nil
}
return renderResult{
sql: fmt.Sprintf("NOT (%s)", child.sql),
}, nil
}
func (r *renderer) renderFieldPredicate(cond *FieldPredicateCondition) (renderResult, error) {
field, ok := r.schema.Field(cond.Field)
if !ok {
return renderResult{}, errors.Errorf("unknown field %q", cond.Field)
}
switch field.Kind {
case FieldKindBoolColumn:
column := qualifyColumn(r.dialect, field.Column)
return renderResult{
sql: fmt.Sprintf("%s IS TRUE", column),
}, nil
case FieldKindJSONBool:
sql, err := r.jsonBoolPredicate(field)
if err != nil {
return renderResult{}, err
}
return renderResult{sql: sql}, nil
default:
return renderResult{}, errors.Errorf("field %q cannot be used as a predicate", cond.Field)
}
}
func (r *renderer) renderComparison(cond *ComparisonCondition) (renderResult, error) {
switch left := cond.Left.(type) {
case *FieldRef:
field, ok := r.schema.Field(left.Name)
if !ok {
return renderResult{}, errors.Errorf("unknown field %q", left.Name)
}
switch field.Kind {
case FieldKindBoolColumn:
return r.renderBoolColumnComparison(field, cond.Operator, cond.Right)
case FieldKindJSONBool:
return r.renderJSONBoolComparison(field, cond.Operator, cond.Right)
case FieldKindScalar:
return r.renderScalarComparison(field, cond.Operator, cond.Right)
default:
return renderResult{}, errors.Errorf("field %q does not support comparison", field.Name)
}
case *FunctionValue:
return r.renderFunctionComparison(left, cond.Operator, cond.Right)
case *FieldAccessorValue:
return r.renderAccessorComparison(left, cond.Operator, cond.Right)
default:
return renderResult{}, errors.New("comparison must start with a field reference or supported function")
}
}
// accessorSpec maps a CEL timestamp accessor to per-dialect SQL date-part tokens
// and the offset to subtract so the result matches CEL's base (e.g. CEL months
// are 0-based but every dialect reports 1-based, so off=1). off is indexed
// [sqlite, postgres, mysql].
type accessorSpec struct {
sqlite string // strftime format specifier
pg string // EXTRACT field
mysql string // function name
off [3]int
}
var accessorSpecs = map[string]accessorSpec{
"getFullYear": {"%Y", "YEAR", "YEAR", [3]int{0, 0, 0}},
"getMonth": {"%m", "MONTH", "MONTH", [3]int{1, 1, 1}},
"getDate": {"%d", "DAY", "DAYOFMONTH", [3]int{0, 0, 0}},
"getDayOfMonth": {"%d", "DAY", "DAYOFMONTH", [3]int{1, 1, 1}},
"getDayOfWeek": {"%w", "DOW", "DAYOFWEEK", [3]int{0, 0, 1}},
"getDayOfYear": {"%j", "DOY", "DAYOFYEAR", [3]int{1, 1, 1}},
"getHours": {"%H", "HOUR", "HOUR", [3]int{0, 0, 0}},
"getMinutes": {"%M", "MINUTE", "MINUTE", [3]int{0, 0, 0}},
"getSeconds": {"%S", "SECOND", "SECOND", [3]int{0, 0, 0}},
}
func (r *renderer) renderAccessorComparison(acc *FieldAccessorValue, op ComparisonOperator, right ValueExpr) (renderResult, error) {
field, ok := r.schema.Field(acc.Field)
if !ok {
return renderResult{}, errors.Errorf("unknown field %q", acc.Field)
}
value, err := expectNumericLiteral(right)
if err != nil {
return renderResult{}, err
}
expr, err := r.timestampAccessorExpr(field, acc.Accessor)
if err != nil {
return renderResult{}, err
}
placeholder := r.addArg(value)
return renderResult{
sql: fmt.Sprintf("%s %s %s", expr, sqlOperator(op), placeholder),
}, nil
}
// timestampAccessorExpr builds a dialect-specific integer expression for a CEL
// timestamp accessor. Extraction is UTC on SQLite/Postgres (epoch columns); on
// MySQL the TIMESTAMP column is read in the server session time zone.
func (r *renderer) timestampAccessorExpr(field Field, accessor string) (string, error) {
spec, ok := accessorSpecs[accessor]
if !ok {
return "", errors.Errorf("unsupported timestamp accessor %q", accessor)
}
col := qualifyColumn(r.dialect, field.Column)
var base string
var off int
switch r.dialect {
case DialectSQLite:
base = fmt.Sprintf("CAST(strftime('%s', %s, 'unixepoch') AS INTEGER)", spec.sqlite, col)
off = spec.off[0]
case DialectPostgres:
base = fmt.Sprintf("CAST(EXTRACT(%s FROM to_timestamp(%s) AT TIME ZONE 'UTC') AS INTEGER)", spec.pg, col)
off = spec.off[1]
case DialectMySQL:
base = fmt.Sprintf("%s(%s)", spec.mysql, col)
off = spec.off[2]
default:
return "", errors.Errorf("unsupported dialect %q", r.dialect)
}
if off != 0 {
base = fmt.Sprintf("(%s - %d)", base, off)
}
return base, nil
}
func (r *renderer) renderFunctionComparison(fn *FunctionValue, op ComparisonOperator, right ValueExpr) (renderResult, error) {
if fn.Name != "size" {
return renderResult{}, errors.Errorf("unsupported function %s in comparison", fn.Name)
}
if len(fn.Args) != 1 {
return renderResult{}, errors.New("size() expects one argument")
}
fieldArg, ok := fn.Args[0].(*FieldRef)
if !ok {
return renderResult{}, errors.New("size() argument must be a field")
}
field, ok := r.schema.Field(fieldArg.Name)
if !ok {
return renderResult{}, errors.Errorf("unknown field %q", fieldArg.Name)
}
if field.Kind == FieldKindVirtualAlias {
field, ok = r.schema.ResolveAlias(fieldArg.Name)
if !ok {
return renderResult{}, errors.Errorf("invalid alias %q", fieldArg.Name)
}
}
value, err := expectNumericLiteral(right)
if err != nil {
return renderResult{}, err
}
var expr string
switch {
case field.Kind == FieldKindJSONList:
expr = jsonArrayLengthExpr(r.dialect, field)
case field.Kind == FieldKindScalar && field.Type == FieldTypeString:
expr = stringLengthExpr(r.dialect, field.columnExpr(r.dialect))
default:
return renderResult{}, errors.Errorf("size() does not support field %q", field.Name)
}
placeholder := r.addArg(value)
return renderResult{
sql: fmt.Sprintf("%s %s %s", expr, sqlOperator(op), placeholder),
}, nil
}
// stringLengthExpr returns the character-count expression for a string column.
// MySQL's LENGTH counts bytes, so CHAR_LENGTH is used to count characters and
// match CEL's size() code-point semantics; SQLite/Postgres LENGTH already counts
// characters.
func stringLengthExpr(d DialectName, colExpr string) string {
if d == DialectMySQL {
return fmt.Sprintf("CHAR_LENGTH(%s)", colExpr)
}
return fmt.Sprintf("LENGTH(%s)", colExpr)
}
func (r *renderer) renderScalarComparison(field Field, op ComparisonOperator, right ValueExpr) (renderResult, error) {
lit, err := expectLiteral(right)
if err != nil {
return renderResult{}, err
}
columnExpr := field.columnExpr(r.dialect)
if lit == nil {
switch op {
case CompareEq:
return renderResult{sql: fmt.Sprintf("%s IS NULL", columnExpr)}, nil
case CompareNeq:
return renderResult{sql: fmt.Sprintf("%s IS NOT NULL", columnExpr)}, nil
default:
return renderResult{}, errors.Errorf("operator %s not supported for null comparison", op)
}
}
placeholder := ""
switch field.Type {
case FieldTypeString:
value, ok := lit.(string)
if !ok {
return renderResult{}, errors.Errorf("field %q expects string value", field.Name)
}
placeholder = r.addArg(value)
case FieldTypeInt, FieldTypeTimestamp:
num, err := toInt64(lit)
if err != nil {
return renderResult{}, errors.Wrapf(err, "field %q expects integer value", field.Name)
}
placeholder = r.addArg(num)
default:
return renderResult{}, errors.Errorf("unsupported data type %q for field %s", field.Type, field.Name)
}
return renderResult{
sql: fmt.Sprintf("%s %s %s", columnExpr, sqlOperator(op), placeholder),
}, nil
}
func (r *renderer) renderBoolColumnComparison(field Field, op ComparisonOperator, right ValueExpr) (renderResult, error) {
value, err := expectBool(right)
if err != nil {
return renderResult{}, err
}
placeholder := r.addBoolArg(value)
column := qualifyColumn(r.dialect, field.Column)
return renderResult{
sql: fmt.Sprintf("%s %s %s", column, sqlOperator(op), placeholder),
}, nil
}
func (r *renderer) renderJSONBoolComparison(field Field, op ComparisonOperator, right ValueExpr) (renderResult, error) {
value, err := expectBool(right)
if err != nil {
return renderResult{}, err
}
jsonExpr := jsonExtractExpr(r.dialect, field)
switch r.dialect {
case DialectSQLite:
switch op {
case CompareEq:
if field.Name == "has_task_list" {
target := "0"
if value {
target = "1"
}
return renderResult{sql: fmt.Sprintf("%s = %s", jsonExpr, target)}, nil
}
if value {
return renderResult{sql: fmt.Sprintf("%s IS TRUE", jsonExpr)}, nil
}
return renderResult{sql: fmt.Sprintf("NOT(%s IS TRUE)", jsonExpr)}, nil
case CompareNeq:
if field.Name == "has_task_list" {
target := "0"
if value {
target = "1"
}
return renderResult{sql: fmt.Sprintf("%s != %s", jsonExpr, target)}, nil
}
if value {
return renderResult{sql: fmt.Sprintf("NOT(%s IS TRUE)", jsonExpr)}, nil
}
return renderResult{sql: fmt.Sprintf("%s IS TRUE", jsonExpr)}, nil
default:
return renderResult{}, errors.Errorf("operator %s not supported for boolean JSON field", op)
}
case DialectMySQL:
boolStr := "false"
if value {
boolStr = "true"
}
return renderResult{
sql: fmt.Sprintf("%s %s CAST('%s' AS JSON)", jsonExpr, sqlOperator(op), boolStr),
}, nil
case DialectPostgres:
placeholder := r.addArg(value)
return renderResult{
sql: fmt.Sprintf("(%s)::boolean %s %s", jsonExpr, sqlOperator(op), placeholder),
}, nil
default:
return renderResult{}, errors.Errorf("unsupported dialect %s", r.dialect)
}
}
func (r *renderer) renderInCondition(cond *InCondition) (renderResult, error) {
fieldRef, ok := cond.Left.(*FieldRef)
if !ok {
return renderResult{}, errors.New("IN operator requires a field on the left-hand side")
}
if fieldRef.Name == "tag" {
return r.renderTagInList(cond.Values)
}
field, ok := r.schema.Field(fieldRef.Name)
if !ok {
return renderResult{}, errors.Errorf("unknown field %q", fieldRef.Name)
}
if field.Kind != FieldKindScalar {
return renderResult{}, errors.Errorf("field %q does not support IN()", fieldRef.Name)
}
return r.renderScalarInCondition(field, cond.Values)
}
func (r *renderer) renderTagInList(values []ValueExpr) (renderResult, error) {
field, ok := r.schema.ResolveAlias("tag")
if !ok {
return renderResult{}, errors.New("tag attribute is not configured")
}
conditions := make([]string, 0, len(values))
for _, v := range values {
lit, err := expectLiteral(v)
if err != nil {
return renderResult{}, err
}
str, ok := lit.(string)
if !ok {
return renderResult{}, errors.New("tags must be compared with string literals")
}
switch r.dialect {
case DialectSQLite:
// Support hierarchical tags: match exact tag OR tags with this prefix (e.g., "book" matches "book" and "book/something")
exactMatch := fmt.Sprintf("%s LIKE %s", jsonArrayExpr(r.dialect, field), r.addArg(fmt.Sprintf(`%%"%s"%%`, str)))
prefixMatch := fmt.Sprintf("%s LIKE %s", jsonArrayExpr(r.dialect, field), r.addArg(fmt.Sprintf(`%%"%s/%%`, str)))
expr := fmt.Sprintf("(%s OR %s)", exactMatch, prefixMatch)
conditions = append(conditions, expr)
case DialectMySQL:
// Support hierarchical tags: match exact tag OR tags with this prefix
exactMatch := fmt.Sprintf("JSON_CONTAINS(%s, %s)", jsonArrayExpr(r.dialect, field), r.addArg(fmt.Sprintf(`"%s"`, str)))
prefixMatch := fmt.Sprintf("%s LIKE %s", jsonArrayExpr(r.dialect, field), r.addArg(fmt.Sprintf(`%%"%s/%%`, str)))
expr := fmt.Sprintf("(%s OR %s)", exactMatch, prefixMatch)
conditions = append(conditions, expr)
case DialectPostgres:
// Support hierarchical tags: match exact tag OR tags with this prefix
exactMatch := fmt.Sprintf("%s @> jsonb_build_array(%s::json)", jsonArrayExpr(r.dialect, field), r.addArg(fmt.Sprintf(`"%s"`, str)))
prefixMatch := fmt.Sprintf("(%s)::text LIKE %s", jsonArrayExpr(r.dialect, field), r.addArg(fmt.Sprintf(`%%"%s/%%`, str)))
expr := fmt.Sprintf("(%s OR %s)", exactMatch, prefixMatch)
conditions = append(conditions, expr)
default:
return renderResult{}, errors.Errorf("unsupported dialect %s", r.dialect)
}
}
if len(conditions) == 1 {
return renderResult{sql: conditions[0]}, nil
}
return renderResult{
sql: fmt.Sprintf("(%s)", strings.Join(conditions, " OR ")),
}, nil
}
func (r *renderer) renderElementInCondition(cond *ElementInCondition) (renderResult, error) {
field, ok := r.schema.Field(cond.Field)
if !ok {
return renderResult{}, errors.Errorf("unknown field %q", cond.Field)
}
if field.Kind != FieldKindJSONList {
return renderResult{}, errors.Errorf("field %q is not a tag list", cond.Field)
}
lit, err := expectLiteral(cond.Element)
if err != nil {
return renderResult{}, err
}
str, ok := lit.(string)
if !ok {
return renderResult{}, errors.New("tags membership requires string literal")
}
switch r.dialect {
case DialectSQLite:
sql := fmt.Sprintf("%s LIKE %s", jsonArrayExpr(r.dialect, field), r.addArg(fmt.Sprintf(`%%"%s"%%`, str)))
return renderResult{sql: sql}, nil
case DialectMySQL:
sql := fmt.Sprintf("JSON_CONTAINS(%s, %s)", jsonArrayExpr(r.dialect, field), r.addArg(fmt.Sprintf(`"%s"`, str)))
return renderResult{sql: sql}, nil
case DialectPostgres:
sql := fmt.Sprintf("%s @> jsonb_build_array(%s::json)", jsonArrayExpr(r.dialect, field), r.addArg(fmt.Sprintf(`"%s"`, str)))
return renderResult{sql: sql}, nil
default:
return renderResult{}, errors.Errorf("unsupported dialect %s", r.dialect)
}
}
func (r *renderer) renderScalarInCondition(field Field, values []ValueExpr) (renderResult, error) {
placeholders := make([]string, 0, len(values))
for _, v := range values {
lit, err := expectLiteral(v)
if err != nil {
return renderResult{}, err
}
switch field.Type {
case FieldTypeString:
str, ok := lit.(string)
if !ok {
return renderResult{}, errors.Errorf("field %q expects string values", field.Name)
}
placeholders = append(placeholders, r.addArg(str))
case FieldTypeInt:
num, err := toInt64(lit)
if err != nil {
return renderResult{}, err
}
placeholders = append(placeholders, r.addArg(num))
default:
return renderResult{}, errors.Errorf("field %q does not support IN() comparisons", field.Name)
}
}
column := field.columnExpr(r.dialect)
return renderResult{
sql: fmt.Sprintf("%s IN (%s)", column, strings.Join(placeholders, ",")),
}, nil
}
func (r *renderer) renderTextMatch(cond *TextMatchCondition) (renderResult, error) {
field, ok := r.schema.Field(cond.Field)
if !ok {
return renderResult{}, errors.Errorf("unknown field %q", cond.Field)
}
column := field.columnExpr(r.dialect)
pattern := likePattern(cond.Mode, cond.Value)
return renderResult{sql: r.foldedLike(column, pattern)}, nil
}
func (r *renderer) renderRegex(cond *RegexCondition) (renderResult, error) {
field, ok := r.schema.Field(cond.Field)
if !ok {
return renderResult{}, errors.Errorf("unknown field %q", cond.Field)
}
column := field.columnExpr(r.dialect)
switch r.dialect {
case DialectPostgres:
// POSIX regex match operator.
return renderResult{sql: fmt.Sprintf("%s ~ %s", column, r.addArg(cond.Pattern))}, nil
case DialectMySQL, DialectSQLite:
// MySQL has a native REGEXP operator; SQLite uses the registered regexp() function.
return renderResult{sql: fmt.Sprintf("%s REGEXP %s", column, r.addArg(cond.Pattern))}, nil
default:
return renderResult{}, errors.Errorf("unsupported dialect %s", r.dialect)
}
}
// foldedLike renders a case-insensitive LIKE comparison of colExpr against a
// (already metacharacter-escaped) pattern, using each dialect's case-folding.
func (r *renderer) foldedLike(colExpr, pattern string) string {
switch r.dialect {
case DialectSQLite:
// memos_unicode_lower gives Unicode-aware folding; ESCAPE '\' is required
// because SQLite has no default LIKE escape character.
return fmt.Sprintf(`memos_unicode_lower(%s) LIKE memos_unicode_lower(%s) ESCAPE '\'`, colExpr, r.addArg(pattern))
case DialectPostgres:
// ILIKE is case-insensitive; backslash is the default escape character.
return fmt.Sprintf("%s ILIKE %s", colExpr, r.addArg(pattern))
default: // MySQL: default collation is case-insensitive; backslash is the default escape.
return fmt.Sprintf("%s LIKE %s", colExpr, r.addArg(pattern))
}
}
// likePattern escapes LIKE metacharacters in value and wraps it for the mode.
func likePattern(mode TextMatchMode, value string) string {
escaped := escapeLikeLiteral(value)
switch mode {
case TextMatchPrefix:
return escaped + "%"
case TextMatchSuffix:
return "%" + escaped
default:
return "%" + escaped + "%"
}
}
// escapeLikeLiteral escapes the LIKE metacharacters \, %, and _ so user input
// is matched literally. Backslash is the escape character on all three dialects.
func escapeLikeLiteral(s string) string {
return strings.NewReplacer(`\`, `\\`, `%`, `\%`, `_`, `\_`).Replace(s)
}
func (r *renderer) renderListComprehension(cond *ListComprehensionCondition) (renderResult, error) {
field, ok := r.schema.Field(cond.Field)
if !ok {
return renderResult{}, errors.Errorf("unknown field %q", cond.Field)
}
if field.Kind != FieldKindJSONList {
return renderResult{}, errors.Errorf("field %q is not a JSON list", cond.Field)
}
if cond.Kind == ComprehensionAll {
return r.renderTagAll(field, cond.Predicate)
}
if cond.Kind == ComprehensionExistsOne {
return r.renderTagExistsOne(field, cond.Predicate)
}
// Render based on predicate type
switch pred := cond.Predicate.(type) {
case *EqualsPredicate:
return r.renderTagEquals(field, pred.Value, cond.Kind)
case *StartsWithPredicate:
return r.renderTagStartsWith(field, pred.Prefix, cond.Kind)
case *EndsWithPredicate:
return r.renderTagEndsWith(field, pred.Suffix, cond.Kind)
case *ContainsPredicate:
return r.renderTagContains(field, pred.Substring, cond.Kind)
default:
return renderResult{}, errors.Errorf("unsupported predicate type %T in comprehension", pred)
}
}
// renderTagAll renders tags.all(t, <pred>): the array is non-empty AND no element
// fails the predicate. Element predicates use plain CEL semantics (case-insensitive
// for startsWith/endsWith/contains, case-sensitive for ==), evaluated per element.
func (r *renderer) renderTagAll(field Field, pred PredicateExpr) (renderResult, error) {
arrayExpr := jsonArrayExpr(r.dialect, field)
elemCond, err := r.elementPredicateSQL(pred)
if err != nil {
return renderResult{}, err
}
switch r.dialect {
case DialectSQLite:
nonEmpty := fmt.Sprintf("%s IS NOT NULL AND %s != '[]'", arrayExpr, arrayExpr)
sub := fmt.Sprintf("NOT EXISTS (SELECT 1 FROM json_each(%s) WHERE NOT (%s))", arrayExpr, elemCond)
return renderResult{sql: fmt.Sprintf("(%s AND %s)", nonEmpty, sub)}, nil
case DialectMySQL:
nonEmpty := fmt.Sprintf("%s IS NOT NULL AND JSON_LENGTH(%s) > 0", arrayExpr, arrayExpr)
sub := fmt.Sprintf("NOT EXISTS (SELECT 1 FROM JSON_TABLE(%s, '$[*]' COLUMNS (value VARCHAR(512) PATH '$')) AS elem WHERE NOT (%s))", arrayExpr, elemCond)
return renderResult{sql: fmt.Sprintf("(%s AND %s)", nonEmpty, sub)}, nil
case DialectPostgres:
nonEmpty := fmt.Sprintf("%s IS NOT NULL AND jsonb_array_length(%s) > 0", arrayExpr, arrayExpr)
sub := fmt.Sprintf("NOT EXISTS (SELECT 1 FROM jsonb_array_elements_text(%s) AS elem(value) WHERE NOT (%s))", arrayExpr, elemCond)
return renderResult{sql: fmt.Sprintf("(%s AND %s)", nonEmpty, sub)}, nil
default:
return renderResult{}, errors.Errorf("unsupported dialect %s", r.dialect)
}
}
// renderTagExistsOne renders tags.exists_one(t, <pred>): exactly one element
// satisfies the predicate, via a COUNT(...) = 1 subquery. A null or empty array
// yields COUNT 0, which is correctly not equal to 1.
func (r *renderer) renderTagExistsOne(field Field, pred PredicateExpr) (renderResult, error) {
arrayExpr := jsonArrayExpr(r.dialect, field)
elemCond, err := r.elementPredicateSQL(pred)
if err != nil {
return renderResult{}, err
}
switch r.dialect {
case DialectSQLite:
return renderResult{sql: fmt.Sprintf("(SELECT COUNT(*) FROM json_each(%s) WHERE %s) = 1", arrayExpr, elemCond)}, nil
case DialectMySQL:
return renderResult{sql: fmt.Sprintf("(SELECT COUNT(*) FROM JSON_TABLE(%s, '$[*]' COLUMNS (value VARCHAR(512) PATH '$')) AS elem WHERE %s) = 1", arrayExpr, elemCond)}, nil
case DialectPostgres:
return renderResult{sql: fmt.Sprintf("(SELECT COUNT(*) FROM jsonb_array_elements_text(%s) AS elem(value) WHERE %s) = 1", arrayExpr, elemCond)}, nil
default:
return renderResult{}, errors.Errorf("unsupported dialect %s", r.dialect)
}
}
// elementPredicateSQL builds the per-element SQL condition for an all() predicate.
// The iterated element is exposed as the unqualified column `value` on all dialects
// (json_each.value / JSON_TABLE column / elem(value)).
func (r *renderer) elementPredicateSQL(pred PredicateExpr) (string, error) {
switch p := pred.(type) {
case *EqualsPredicate:
return fmt.Sprintf("value = %s", r.addArg(p.Value)), nil
case *StartsWithPredicate:
return r.foldedLike("value", likePattern(TextMatchPrefix, p.Prefix)), nil
case *EndsWithPredicate:
return r.foldedLike("value", likePattern(TextMatchSuffix, p.Suffix)), nil
case *ContainsPredicate:
return r.foldedLike("value", likePattern(TextMatchContains, p.Substring)), nil
default:
return "", errors.Errorf("unsupported predicate %T in all()", pred)
}
}
// renderTagEquals generates SQL for tags.exists(t, t == "value").
func (r *renderer) renderTagEquals(field Field, value string, _ ComprehensionKind) (renderResult, error) {
arrayExpr := jsonArrayExpr(r.dialect, field)
switch r.dialect {
case DialectSQLite, DialectMySQL:
exactMatch := r.buildJSONArrayLike(arrayExpr, fmt.Sprintf(`%%"%s"%%`, value))
return renderResult{sql: r.wrapWithNullCheck(arrayExpr, exactMatch)}, nil
case DialectPostgres:
exactMatch := fmt.Sprintf("%s @> jsonb_build_array(%s::json)", arrayExpr, r.addArg(fmt.Sprintf(`"%s"`, value)))
return renderResult{sql: r.wrapWithNullCheck(arrayExpr, exactMatch)}, nil
default:
return renderResult{}, errors.Errorf("unsupported dialect %s", r.dialect)
}
}
// renderTagStartsWith generates SQL for tags.exists(t, t.startsWith("prefix")).
func (r *renderer) renderTagStartsWith(field Field, prefix string, _ ComprehensionKind) (renderResult, error) {
arrayExpr := jsonArrayExpr(r.dialect, field)
switch r.dialect {
case DialectSQLite, DialectMySQL:
// Match exact tag or tags with this prefix (hierarchical support)
exactMatch := r.buildJSONArrayLike(arrayExpr, fmt.Sprintf(`%%"%s"%%`, prefix))
prefixMatch := r.buildJSONArrayLike(arrayExpr, fmt.Sprintf(`%%"%s%%`, prefix))
condition := fmt.Sprintf("(%s OR %s)", exactMatch, prefixMatch)
return renderResult{sql: r.wrapWithNullCheck(arrayExpr, condition)}, nil
case DialectPostgres:
// Use PostgreSQL's powerful JSON operators
exactMatch := fmt.Sprintf("%s @> jsonb_build_array(%s::json)", arrayExpr, r.addArg(fmt.Sprintf(`"%s"`, prefix)))
prefixMatch := fmt.Sprintf("(%s)::text LIKE %s", arrayExpr, r.addArg(fmt.Sprintf(`%%"%s%%`, prefix)))
condition := fmt.Sprintf("(%s OR %s)", exactMatch, prefixMatch)
return renderResult{sql: r.wrapWithNullCheck(arrayExpr, condition)}, nil
default:
return renderResult{}, errors.Errorf("unsupported dialect %s", r.dialect)
}
}
// renderTagEndsWith generates SQL for tags.exists(t, t.endsWith("suffix")).
func (r *renderer) renderTagEndsWith(field Field, suffix string, _ ComprehensionKind) (renderResult, error) {
arrayExpr := jsonArrayExpr(r.dialect, field)
pattern := fmt.Sprintf(`%%%s"%%`, suffix)
likeExpr := r.buildJSONArrayLike(arrayExpr, pattern)
return renderResult{sql: r.wrapWithNullCheck(arrayExpr, likeExpr)}, nil
}
// renderTagContains generates SQL for tags.exists(t, t.contains("substring")).
func (r *renderer) renderTagContains(field Field, substring string, _ ComprehensionKind) (renderResult, error) {
arrayExpr := jsonArrayExpr(r.dialect, field)
pattern := fmt.Sprintf(`%%%s%%`, substring)
likeExpr := r.buildJSONArrayLike(arrayExpr, pattern)
return renderResult{sql: r.wrapWithNullCheck(arrayExpr, likeExpr)}, nil
}
// buildJSONArrayLike builds a LIKE expression for matching within a JSON array.
// Returns the LIKE clause without NULL/empty checks.
func (r *renderer) buildJSONArrayLike(arrayExpr, pattern string) string {
switch r.dialect {
case DialectSQLite, DialectMySQL:
return fmt.Sprintf("%s LIKE %s", arrayExpr, r.addArg(pattern))
case DialectPostgres:
return fmt.Sprintf("(%s)::text LIKE %s", arrayExpr, r.addArg(pattern))
default:
return ""
}
}
// wrapWithNullCheck wraps a condition with NULL and empty array checks.
// This ensures we don't match against NULL or empty JSON arrays.
func (r *renderer) wrapWithNullCheck(arrayExpr, condition string) string {
var nullCheck string
switch r.dialect {
case DialectSQLite:
nullCheck = fmt.Sprintf("%s IS NOT NULL AND %s != '[]'", arrayExpr, arrayExpr)
case DialectMySQL:
nullCheck = fmt.Sprintf("%s IS NOT NULL AND JSON_LENGTH(%s) > 0", arrayExpr, arrayExpr)
case DialectPostgres:
nullCheck = fmt.Sprintf("%s IS NOT NULL AND jsonb_array_length(%s) > 0", arrayExpr, arrayExpr)
default:
return condition
}
return fmt.Sprintf("(%s AND %s)", condition, nullCheck)
}
func (r *renderer) jsonBoolPredicate(field Field) (string, error) {
expr := jsonExtractExpr(r.dialect, field)
switch r.dialect {
case DialectSQLite:
return fmt.Sprintf("%s IS TRUE", expr), nil
case DialectMySQL:
return fmt.Sprintf("COALESCE(%s, CAST('false' AS JSON)) = CAST('true' AS JSON)", expr), nil
case DialectPostgres:
return fmt.Sprintf("(%s)::boolean IS TRUE", expr), nil
default:
return "", errors.Errorf("unsupported dialect %s", r.dialect)
}
}
func combineAnd(left, right renderResult) renderResult {
if left.unsatisfiable || right.unsatisfiable {
return renderResult{sql: "1 = 0", unsatisfiable: true}
}
if left.trivial {
return right
}
if right.trivial {
return left
}
return renderResult{
sql: fmt.Sprintf("(%s AND %s)", left.sql, right.sql),
}
}
func combineOr(left, right renderResult) renderResult {
if left.trivial || right.trivial {
return renderResult{trivial: true}
}
if left.unsatisfiable {
return right
}
if right.unsatisfiable {
return left
}
return renderResult{
sql: fmt.Sprintf("(%s OR %s)", left.sql, right.sql),
}
}
func (r *renderer) addArg(value any) string {
r.placeholderCounter++
r.args = append(r.args, value)
if r.dialect == DialectPostgres {
return fmt.Sprintf("$%d", r.placeholderOffset+r.placeholderCounter)
}
return "?"
}
func (r *renderer) addBoolArg(value bool) string {
var v any
switch r.dialect {
case DialectSQLite:
if value {
v = 1
} else {
v = 0
}
default:
v = value
}
return r.addArg(v)
}
func expectLiteral(expr ValueExpr) (any, error) {
lit, ok := expr.(*LiteralValue)
if !ok {
return nil, errors.New("expression must be a literal")
}
return lit.Value, nil
}
func expectBool(expr ValueExpr) (bool, error) {
lit, err := expectLiteral(expr)
if err != nil {
return false, err
}
value, ok := lit.(bool)
if !ok {
return false, errors.New("boolean literal required")
}
return value, nil
}
func expectNumericLiteral(expr ValueExpr) (int64, error) {
lit, err := expectLiteral(expr)
if err != nil {
return 0, err
}
return toInt64(lit)
}
func toInt64(value any) (int64, error) {
switch v := value.(type) {
case int:
return int64(v), nil
case int32:
return int64(v), nil
case int64:
return v, nil
case uint32:
return int64(v), nil
case uint64:
return int64(v), nil
case float32:
return int64(v), nil
case float64:
return int64(v), nil
default:
return 0, errors.Errorf("cannot convert %T to int64", value)
}
}
func sqlOperator(op ComparisonOperator) string {
return string(op)
}
func qualifyColumn(d DialectName, col Column) string {
switch d {
case DialectPostgres:
return fmt.Sprintf("%s.%s", col.Table, col.Name)
default:
return fmt.Sprintf("`%s`.`%s`", col.Table, col.Name)
}
}
func jsonPath(field Field) string {
return "$." + strings.Join(field.JSONPath, ".")
}
func jsonExtractExpr(d DialectName, field Field) string {
column := qualifyColumn(d, field.Column)
switch d {
case DialectSQLite, DialectMySQL:
return fmt.Sprintf("JSON_EXTRACT(%s, '%s')", column, jsonPath(field))
case DialectPostgres:
return buildPostgresJSONAccessor(column, field.JSONPath, true)
default:
return ""
}
}
func jsonArrayExpr(d DialectName, field Field) string {
column := qualifyColumn(d, field.Column)
switch d {
case DialectSQLite, DialectMySQL:
return fmt.Sprintf("JSON_EXTRACT(%s, '%s')", column, jsonPath(field))
case DialectPostgres:
return buildPostgresJSONAccessor(column, field.JSONPath, false)
default:
return ""
}
}
func jsonArrayLengthExpr(d DialectName, field Field) string {
arrayExpr := jsonArrayExpr(d, field)
switch d {
case DialectSQLite:
return fmt.Sprintf("JSON_ARRAY_LENGTH(COALESCE(%s, JSON_ARRAY()))", arrayExpr)
case DialectMySQL:
return fmt.Sprintf("JSON_LENGTH(COALESCE(%s, JSON_ARRAY()))", arrayExpr)
case DialectPostgres:
return fmt.Sprintf("jsonb_array_length(COALESCE(%s, '[]'::jsonb))", arrayExpr)
default:
return ""
}
}
func buildPostgresJSONAccessor(base string, path []string, terminalText bool) string {
expr := base
for idx, part := range path {
if idx == len(path)-1 && terminalText {
expr = fmt.Sprintf("%s->>'%s'", expr, part)
} else {
expr = fmt.Sprintf("%s->'%s'", expr, part)
}
}
return expr
}
+327
View File
@@ -0,0 +1,327 @@
package filter
import (
"fmt"
"github.com/google/cel-go/cel"
"github.com/google/cel-go/ext"
)
// DialectName enumerates supported SQL dialects.
type DialectName string
const (
DialectSQLite DialectName = "sqlite"
DialectMySQL DialectName = "mysql"
DialectPostgres DialectName = "postgres"
)
// FieldType represents the logical type of a field.
type FieldType string
const (
FieldTypeString FieldType = "string"
FieldTypeInt FieldType = "int"
FieldTypeBool FieldType = "bool"
FieldTypeTimestamp FieldType = "timestamp"
)
// FieldKind describes how a field is stored.
type FieldKind string
const (
FieldKindScalar FieldKind = "scalar"
FieldKindBoolColumn FieldKind = "bool_column"
FieldKindJSONBool FieldKind = "json_bool"
FieldKindJSONList FieldKind = "json_list"
FieldKindVirtualAlias FieldKind = "virtual_alias"
)
// Column identifies the backing table column.
type Column struct {
Table string
Name string
}
// Field captures the schema metadata for an exposed CEL identifier.
type Field struct {
Name string
Kind FieldKind
Type FieldType
Column Column
JSONPath []string
AliasFor string
SupportsContains bool
Expressions map[DialectName]string
AllowedComparisonOps map[ComparisonOperator]bool
}
// Schema collects CEL environment options and field metadata.
type Schema struct {
Name string
Fields map[string]Field
EnvOptions []cel.EnvOption
}
// Field returns the field metadata if present.
func (s Schema) Field(name string) (Field, bool) {
f, ok := s.Fields[name]
return f, ok
}
// ResolveAlias resolves a virtual alias to its target field.
func (s Schema) ResolveAlias(name string) (Field, bool) {
field, ok := s.Fields[name]
if !ok {
return Field{}, false
}
if field.Kind == FieldKindVirtualAlias {
target, ok := s.Fields[field.AliasFor]
if !ok {
return Field{}, false
}
return target, true
}
return field, true
}
// NewSchema constructs the memo filter schema and CEL environment.
func NewSchema() Schema {
fields := map[string]Field{
"content": {
Name: "content",
Kind: FieldKindScalar,
Type: FieldTypeString,
Column: Column{Table: "memo", Name: "content"},
SupportsContains: true,
Expressions: map[DialectName]string{},
},
"creator": {
Name: "creator",
Kind: FieldKindScalar,
Type: FieldTypeString,
Column: Column{Table: "memo_creator", Name: "username"},
Expressions: map[DialectName]string{
DialectSQLite: "('users/' || %s)",
DialectMySQL: "CONCAT('users/', %s)",
DialectPostgres: "('users/' || %s)",
},
AllowedComparisonOps: map[ComparisonOperator]bool{
CompareEq: true,
CompareNeq: true,
},
},
"creator_id": {
Name: "creator_id",
Kind: FieldKindScalar,
Type: FieldTypeInt,
Column: Column{Table: "memo", Name: "creator_id"},
Expressions: map[DialectName]string{},
AllowedComparisonOps: map[ComparisonOperator]bool{
CompareEq: true,
CompareNeq: true,
},
},
"created_ts": {
Name: "created_ts",
Kind: FieldKindScalar,
Type: FieldTypeTimestamp,
Column: Column{Table: "memo", Name: "created_ts"},
Expressions: map[DialectName]string{
// MySQL stores created_ts as TIMESTAMP, needs conversion to epoch
DialectMySQL: "UNIX_TIMESTAMP(%s)",
// PostgreSQL and SQLite store created_ts as BIGINT (epoch), no conversion needed
DialectPostgres: "%s",
DialectSQLite: "%s",
},
},
"updated_ts": {
Name: "updated_ts",
Kind: FieldKindScalar,
Type: FieldTypeTimestamp,
Column: Column{Table: "memo", Name: "updated_ts"},
Expressions: map[DialectName]string{
// MySQL stores updated_ts as TIMESTAMP, needs conversion to epoch
DialectMySQL: "UNIX_TIMESTAMP(%s)",
// PostgreSQL and SQLite store updated_ts as BIGINT (epoch), no conversion needed
DialectPostgres: "%s",
DialectSQLite: "%s",
},
},
"pinned": {
Name: "pinned",
Kind: FieldKindBoolColumn,
Type: FieldTypeBool,
Column: Column{Table: "memo", Name: "pinned"},
Expressions: map[DialectName]string{},
AllowedComparisonOps: map[ComparisonOperator]bool{
CompareEq: true,
CompareNeq: true,
},
},
"visibility": {
Name: "visibility",
Kind: FieldKindScalar,
Type: FieldTypeString,
Column: Column{Table: "memo", Name: "visibility"},
Expressions: map[DialectName]string{},
AllowedComparisonOps: map[ComparisonOperator]bool{
CompareEq: true,
CompareNeq: true,
},
},
"tags": {
Name: "tags",
Kind: FieldKindJSONList,
Type: FieldTypeString,
Column: Column{Table: "memo", Name: "payload"},
JSONPath: []string{"tags"},
},
"tag": {
Name: "tag",
Kind: FieldKindVirtualAlias,
Type: FieldTypeString,
AliasFor: "tags",
},
"has_task_list": {
Name: "has_task_list",
Kind: FieldKindJSONBool,
Type: FieldTypeBool,
Column: Column{Table: "memo", Name: "payload"},
JSONPath: []string{"property", "hasTaskList"},
AllowedComparisonOps: map[ComparisonOperator]bool{
CompareEq: true,
CompareNeq: true,
},
},
"has_link": {
Name: "has_link",
Kind: FieldKindJSONBool,
Type: FieldTypeBool,
Column: Column{Table: "memo", Name: "payload"},
JSONPath: []string{"property", "hasLink"},
AllowedComparisonOps: map[ComparisonOperator]bool{
CompareEq: true,
CompareNeq: true,
},
},
"has_code": {
Name: "has_code",
Kind: FieldKindJSONBool,
Type: FieldTypeBool,
Column: Column{Table: "memo", Name: "payload"},
JSONPath: []string{"property", "hasCode"},
AllowedComparisonOps: map[ComparisonOperator]bool{
CompareEq: true,
CompareNeq: true,
},
},
"has_incomplete_tasks": {
Name: "has_incomplete_tasks",
Kind: FieldKindJSONBool,
Type: FieldTypeBool,
Column: Column{Table: "memo", Name: "payload"},
JSONPath: []string{"property", "hasIncompleteTasks"},
AllowedComparisonOps: map[ComparisonOperator]bool{
CompareEq: true,
CompareNeq: true,
},
},
}
envOptions := []cel.EnvOption{
cel.Variable("content", cel.StringType),
cel.Variable("creator", cel.StringType),
cel.Variable("creator_id", cel.IntType),
cel.Variable("created_ts", cel.TimestampType),
cel.Variable("updated_ts", cel.TimestampType),
cel.Variable("pinned", cel.BoolType),
cel.Variable("tag", cel.StringType),
cel.Variable("tags", cel.ListType(cel.StringType)),
cel.Variable("visibility", cel.StringType),
cel.Variable("has_task_list", cel.BoolType),
cel.Variable("has_link", cel.BoolType),
cel.Variable("has_code", cel.BoolType),
cel.Variable("has_incomplete_tasks", cel.BoolType),
cel.Variable("now", cel.TimestampType),
ext.Sets(),
cel.ASTValidators(cel.ValidateRegexLiterals()),
}
return Schema{
Name: "memo",
Fields: fields,
EnvOptions: envOptions,
}
}
// NewAttachmentSchema constructs the attachment filter schema and CEL environment.
func NewAttachmentSchema() Schema {
fields := map[string]Field{
"filename": {
Name: "filename",
Kind: FieldKindScalar,
Type: FieldTypeString,
Column: Column{Table: "attachment", Name: "filename"},
SupportsContains: true,
Expressions: map[DialectName]string{},
},
"mime_type": {
Name: "mime_type",
Kind: FieldKindScalar,
Type: FieldTypeString,
Column: Column{Table: "attachment", Name: "type"},
SupportsContains: true,
Expressions: map[DialectName]string{},
},
"create_time": {
Name: "create_time",
Kind: FieldKindScalar,
Type: FieldTypeTimestamp,
Column: Column{Table: "attachment", Name: "created_ts"},
Expressions: map[DialectName]string{
// MySQL stores created_ts as TIMESTAMP, needs conversion to epoch
DialectMySQL: "UNIX_TIMESTAMP(%s)",
// PostgreSQL and SQLite store created_ts as BIGINT (epoch), no conversion needed
DialectPostgres: "%s",
DialectSQLite: "%s",
},
},
"memo_id": {
Name: "memo_id",
Kind: FieldKindScalar,
Type: FieldTypeInt,
Column: Column{Table: "attachment", Name: "memo_id"},
Expressions: map[DialectName]string{},
AllowedComparisonOps: map[ComparisonOperator]bool{
CompareEq: true,
CompareNeq: true,
},
},
}
envOptions := []cel.EnvOption{
cel.Variable("filename", cel.StringType),
cel.Variable("mime_type", cel.StringType),
cel.Variable("create_time", cel.TimestampType),
cel.Variable("memo_id", cel.AnyType),
cel.Variable("now", cel.TimestampType),
cel.ASTValidators(cel.ValidateRegexLiterals()),
}
return Schema{
Name: "attachment",
Fields: fields,
EnvOptions: envOptions,
}
}
// columnExpr returns the field expression for the given dialect, applying
// any schema-specific overrides (e.g. UNIX timestamp conversions).
func (f Field) columnExpr(d DialectName) string {
base := qualifyColumn(d, f.Column)
if expr, ok := f.Expressions[d]; ok && expr != "" {
return fmt.Sprintf(expr, base)
}
return base
}
+100
View File
@@ -0,0 +1,100 @@
package filter
import (
"context"
"testing"
"time"
"github.com/stretchr/testify/require"
)
// fixedClock returns a deterministic clock for asserting folded `now` values.
func fixedClock(epoch int64) func() time.Time {
return func() time.Time { return time.Unix(epoch, 0) }
}
func memoEngineAt(t *testing.T, epoch int64) *Engine {
t.Helper()
engine, err := NewEngine(NewSchema())
require.NoError(t, err)
engine.nowFunc = fixedClock(epoch)
return engine
}
func TestCompileNowVariableFoldsToInjectedClock(t *testing.T) {
t.Parallel()
engine := memoEngineAt(t, 1750000000)
stmt, err := engine.CompileToStatement(context.Background(), `created_ts >= now`, RenderOptions{Dialect: DialectSQLite})
require.NoError(t, err)
require.Equal(t, []any{int64(1750000000)}, stmt.Args)
}
func TestCompileNowFunctionIsRemoved(t *testing.T) {
t.Parallel()
engine, err := NewEngine(NewSchema())
require.NoError(t, err)
// now() was the legacy custom function; it is replaced by the `now` variable.
_, err = engine.Compile(context.Background(), `created_ts >= now()`)
require.Error(t, err)
}
func TestCompileNowMinusDurationFolds(t *testing.T) {
t.Parallel()
engine := memoEngineAt(t, 1750000000)
stmt, err := engine.CompileToStatement(context.Background(), `created_ts >= now - duration("1h")`, RenderOptions{Dialect: DialectSQLite})
require.NoError(t, err)
require.Equal(t, []any{int64(1750000000 - 3600)}, stmt.Args)
}
func TestCompileNowPlusDurationFolds(t *testing.T) {
t.Parallel()
engine := memoEngineAt(t, 1750000000)
stmt, err := engine.CompileToStatement(context.Background(), `updated_ts < now + duration("24h")`, RenderOptions{Dialect: DialectSQLite})
require.NoError(t, err)
require.Equal(t, []any{int64(1750000000 + 86400)}, stmt.Args)
}
func TestCompileAbsoluteTimestampStringFolds(t *testing.T) {
t.Parallel()
engine := memoEngineAt(t, 1750000000)
stmt, err := engine.CompileToStatement(context.Background(), `created_ts >= timestamp("2025-01-01T00:00:00Z")`, RenderOptions{Dialect: DialectSQLite})
require.NoError(t, err)
require.Equal(t, []any{int64(1735689600)}, stmt.Args)
}
func TestCompileTimestampFromEpochIntFolds(t *testing.T) {
t.Parallel()
// This is the shape the frontend date-range filter emits.
engine := memoEngineAt(t, 1750000000)
stmt, err := engine.CompileToStatement(context.Background(), `created_ts >= timestamp(1730000000)`, RenderOptions{Dialect: DialectSQLite})
require.NoError(t, err)
require.Equal(t, []any{int64(1730000000)}, stmt.Args)
}
func TestCompileInvalidDurationLiteralErrors(t *testing.T) {
t.Parallel()
engine := memoEngineAt(t, 1750000000)
_, err := engine.Compile(context.Background(), `created_ts >= now - duration("garbage")`)
require.Error(t, err)
require.Contains(t, err.Error(), "duration")
}
func TestCompileAttachmentCreateTimeUsesNow(t *testing.T) {
t.Parallel()
engine, err := NewEngine(NewAttachmentSchema())
require.NoError(t, err)
engine.nowFunc = fixedClock(1750000000)
stmt, err := engine.CompileToStatement(context.Background(), `create_time >= now - duration("24h")`, RenderOptions{Dialect: DialectSQLite})
require.NoError(t, err)
require.Equal(t, []any{int64(1750000000 - 86400)}, stmt.Args)
}