chore: import upstream snapshot with attribution
CI / Run CI (push) Has been cancelled
CI / check-backend (push) Has been cancelled
CI / check-frontend (push) Has been cancelled
CI / tests (push) Has been cancelled
CI / e2e-tests (push) Has been cancelled
Copilot Setup Steps / copilot-setup-steps (push) Has been cancelled
CI / Run CI (push) Has been cancelled
CI / check-backend (push) Has been cancelled
CI / check-frontend (push) Has been cancelled
CI / tests (push) Has been cancelled
CI / e2e-tests (push) Has been cancelled
Copilot Setup Steps / copilot-setup-steps (push) Has been cancelled
This commit is contained in:
wehub-resource-sync
@@ -0,0 +1,14 @@
|
||||
# Documentation for Chainlit developers, contributors, and AI coding assistants.
|
||||
|
||||
See also:
|
||||
|
||||
- [CONTRIBUTING.md](../CONTRIBUTING.md) — contribution guidelines
|
||||
- [CHANGELOG.md](../CHANGELOG.md) — release history
|
||||
|
||||
## Research
|
||||
|
||||
Internal notes on CI, tooling, and test layout:
|
||||
|
||||
- [CI cache optimization](research/ci-cache-optimization.md)
|
||||
- [Copilot type-checking](research/copilot-type-checking.md)
|
||||
- [E2E parallel execution](research/e2e-parallel-execution.md)
|
||||
@@ -0,0 +1,117 @@
|
||||
# Context7 Library IDs
|
||||
|
||||
Pre-resolved library IDs for [Context7 MCP](https://context7.com). Use these when looking up documentation via the Context7 MCP `query-docs` tool.
|
||||
|
||||
## Language
|
||||
|
||||
| Library/Framework | Context7 Library ID | Notes |
|
||||
| ----------------- | ------------------------ | ----------------------------------- |
|
||||
| Python 3.13 | /websites/python_3_13 | Standard library, language features |
|
||||
| TypeScript | /websites/typescriptlang | Type system, compiler options |
|
||||
|
||||
## Backend
|
||||
|
||||
| Library/Framework | Context7 Library ID | Notes |
|
||||
| ----------------- | ------------------------------- | ------------------------------------------ |
|
||||
| Pydantic | /websites/pydantic_dev | Models, validation, serialization |
|
||||
| FastAPI | /websites/fastapi_tiangolo | Web framework, async endpoints |
|
||||
| HTTPX | /encode/httpx | Async HTTP client (core dependency) |
|
||||
| python-socketio | /miguelgrinberg/python-socketio | SocketIO server — core real-time transport |
|
||||
|
||||
## LLM integrations
|
||||
|
||||
| Library/Framework | Context7 Library ID | Notes |
|
||||
| -------------------- | ----------------------------------------------- | ---------------------------------- |
|
||||
| MCP Python SDK | /modelcontextprotocol/python-sdk | Official MCP SDK (core dependency) |
|
||||
| OpenAI Python | /openai/openai-python | OpenAI API client |
|
||||
| Anthropic Python | /anthropics/anthropic-sdk-python | Anthropic API client |
|
||||
| LangChain | /websites/langchain_oss_python_langchain | LLM orchestration, chains, agents |
|
||||
| LlamaIndex | /websites/developers_llamaindex_ai_python | RAG / LLM data framework |
|
||||
| Semantic Kernel | /microsoft/semantic-kernel | Microsoft AI orchestration SDK |
|
||||
| Semantic Kernel docs | /websites/learn_microsoft_en-us_semantic-kernel | Official Microsoft docs |
|
||||
| MistralAI | /mistralai/client-python | MistralAI API client |
|
||||
|
||||
## Persistence
|
||||
|
||||
| Library/Framework | Context7 Library ID | Notes |
|
||||
| -------------------- | ---------------------------------------------- | ------------------------------ |
|
||||
| SQLAlchemy | /websites/sqlalchemy_en_20 | ORM for PostgreSQL/SQLite |
|
||||
| asyncpg | /websites/magicstack_github_io_asyncpg_current | Async PostgreSQL driver |
|
||||
| boto3 | /boto/boto3 | AWS SDK — DynamoDB + S3 |
|
||||
| Azure SDK | /azure/azure-sdk-for-python | Azure SDK for Python |
|
||||
| Azure Storage | /websites/learn_microsoft_en-us_azure_storage | Azure Blob Storage + Data Lake |
|
||||
| Google Cloud Storage | /googleapis/python-storage | GCS persistence |
|
||||
|
||||
## Platform integrations
|
||||
|
||||
| Library/Framework | Context7 Library ID | Notes |
|
||||
| ----------------- | ------------------------- | ----------------------------------- |
|
||||
| Slack Bolt | /slackapi/bolt-python | Slack app framework |
|
||||
| Discord | /llmstxt/discord_llms_txt | Discord bot API |
|
||||
| pandas | /websites/pandas_pydata | Data analysis (optional dependency) |
|
||||
|
||||
## Frontend — core
|
||||
|
||||
| Library/Framework | Context7 Library ID | Notes |
|
||||
| ----------------- | ---------------------------- | ------------------------- |
|
||||
| React | /websites/react_dev | Core UI library |
|
||||
| React Router | /websites/reactrouter_6_30_3 | Client-side routing (v6) |
|
||||
| Vite | /websites/vite_dev | Build tool and dev server |
|
||||
|
||||
## Frontend — UI
|
||||
|
||||
| Library/Framework | Context7 Library ID | Notes |
|
||||
| ----------------- | ----------------------------- | -------------------------------- |
|
||||
| Tailwind CSS | /websites/v3_tailwindcss | Utility-first CSS framework (v3) |
|
||||
| Radix UI | /websites/radix-ui_primitives | Unstyled UI primitives |
|
||||
| lucide-react | /websites/lucide_dev_guide | Icon library |
|
||||
|
||||
## Frontend — state & data
|
||||
|
||||
| Library/Framework | Context7 Library ID | Notes |
|
||||
| ----------------- | --------------------------------- | --------------------------- |
|
||||
| Zustand | /websites/zustand_pmnd_rs | For future Recoil migration |
|
||||
| SWR | /vercel/swr-site | Data fetching hooks |
|
||||
| react-hook-form | /websites/react-hook-form | Form state management |
|
||||
| Zod | /llmstxt/zod_dev_llms_txt | Schema validation |
|
||||
| socket.io-client | /websites/socket_io_v4_client-api | WebSocket client |
|
||||
|
||||
## Frontend — rendering
|
||||
|
||||
| Library/Framework | Context7 Library ID | Notes |
|
||||
| ----------------- | ------------------------- | ------------------------------- |
|
||||
| react-markdown | /remarkjs/react-markdown | Markdown renderer for React |
|
||||
| remark | /remarkjs/remark | Markdown processor |
|
||||
| react-remark | /remarkjs/react-remark | React hooks for remark |
|
||||
| remark-gfm | /remarkjs/remark-gfm | GitHub Flavored Markdown plugin |
|
||||
| remark-math | /remarkjs/remark-math | Math syntax plugin |
|
||||
| highlight.js | /highlightjs/highlight.js | Syntax highlighting |
|
||||
| Plotly.js | /plotly/plotly.js | Charting library |
|
||||
| react-plotly.js | /plotly/react-plotly.js | React wrapper for Plotly |
|
||||
|
||||
## Frontend — i18n
|
||||
|
||||
| Library/Framework | Context7 Library ID | Notes |
|
||||
| ----------------- | ------------------------------ | ------------------------------ |
|
||||
| i18next | /llmstxt/i18next_llms-full_txt | Internationalization framework |
|
||||
| react-i18next | /i18next/react-i18next | React bindings for i18next |
|
||||
|
||||
## Testing
|
||||
|
||||
| Library/Framework | Context7 Library ID | Notes |
|
||||
| ----------------- | --------------------------------- | --------------------- |
|
||||
| pytest | /websites/pytest_en_stable | Python test framework |
|
||||
| Cypress | /cypress-io/cypress-documentation | E2E testing framework |
|
||||
|
||||
## Lint & format
|
||||
|
||||
| Library/Framework | Context7 Library ID | Notes |
|
||||
| ----------------- | --------------------- | -------------- |
|
||||
| ESLint | /websites/eslint | JS/TS linter |
|
||||
| Prettier | /websites/prettier_io | Code formatter |
|
||||
|
||||
## Infra
|
||||
|
||||
| Library/Framework | Context7 Library ID | Notes |
|
||||
| ----------------- | --------------------------- | --------------- |
|
||||
| GitHub Actions | /websites/github_en_actions | CI/CD workflows |
|
||||
@@ -0,0 +1,426 @@
|
||||
# ESLint v9 Flat Config Migration + lint-staged Unification
|
||||
|
||||
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
|
||||
|
||||
**Goal:** Migrate ESLint from v8 (`.eslintrc`) to v9 (flat config) so that `eslint` with no args defaults to `.` (all files), while explicit file args override this — enabling lint-staged to pass staged files via `pnpm --filter` without linting the entire package.
|
||||
|
||||
**Architecture:** Replace the JSON `.eslintrc` + `.eslintignore` + `cypress/.eslintrc` with a single `eslint.config.mjs` at the repo root. Update all package `lint` scripts to bare `eslint` (no `--ext`, no directory). Update `lint-staged.config.js` to use package-scoped globs with `pnpm --filter` for ESLint and type-check.
|
||||
|
||||
**Tech Stack:** ESLint 9, typescript-eslint (unified package), @eslint/js, eslint-plugin-cypress 6, eslint-plugin-chai-friendly
|
||||
|
||||
**Key behavior (verified empirically on this repo):**
|
||||
|
||||
- ESLint v9 flat config: `eslint` (no args) = lints `.` (current directory)
|
||||
- ESLint v9 flat config: `eslint file1.ts file2.ts` = lints **only** those files
|
||||
- ESLint v8: no such behavior exists (no args = 0 files, silent exit 0)
|
||||
|
||||
---
|
||||
|
||||
## File Map
|
||||
|
||||
| Action | File | Responsibility |
|
||||
| ------ | -------------------------------- | ------------------------------------------------------------------------------ |
|
||||
| Create | `eslint.config.mjs` | Single flat config replacing `.eslintrc`, `.eslintignore`, `cypress/.eslintrc` |
|
||||
| Delete | `.eslintrc` | Replaced by `eslint.config.mjs` |
|
||||
| Delete | `.eslintignore` | Merged into `ignores` array in `eslint.config.mjs` |
|
||||
| Delete | `cypress/.eslintrc` | Merged into `eslint.config.mjs` with `files: ['cypress/**/*.ts']` |
|
||||
| Modify | `package.json` | Update ESLint deps |
|
||||
| Modify | `frontend/package.json` | Remove `--ext` and `./src` from lint script |
|
||||
| Modify | `libs/react-client/package.json` | Remove `--ext` and `./src` from lint script |
|
||||
| Modify | `libs/copilot/package.json` | Remove `--ext` and `./src` from lint script |
|
||||
| Modify | `lint-staged.config.js` | Package-scoped globs with `pnpm --filter` |
|
||||
|
||||
---
|
||||
|
||||
## Task 1: Update Dependencies
|
||||
|
||||
**Files:**
|
||||
|
||||
- Modify: `package.json` (devDependencies)
|
||||
- **Step 1: Remove old ESLint packages and add new ones**
|
||||
|
||||
```bash
|
||||
pnpm remove @typescript-eslint/eslint-plugin @typescript-eslint/parser eslint eslint-plugin-cypress
|
||||
```
|
||||
|
||||
- **Step 2: Install ESLint v9 and new packages**
|
||||
|
||||
```bash
|
||||
pnpm add -Dw eslint@^9 @eslint/js typescript-eslint eslint-plugin-cypress@^6
|
||||
```
|
||||
|
||||
> `eslint-plugin-chai-friendly@^1.1.1` stays — its peer dep is `eslint>=3.0.0` and it exports `configs.recommendedFlat` (verified at runtime).
|
||||
|
||||
- **Step 3: Verify installation**
|
||||
|
||||
```bash
|
||||
npx eslint --version
|
||||
```
|
||||
|
||||
Expected: `v9.x.x`
|
||||
|
||||
- **Step 4: Commit**
|
||||
|
||||
```bash
|
||||
git add package.json pnpm-lock.yaml
|
||||
git commit -m "chore: upgrade eslint to v9, add typescript-eslint unified package"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Task 2: Create Flat Config
|
||||
|
||||
**Files:**
|
||||
|
||||
- Create: `eslint.config.mjs`
|
||||
|
||||
This replaces three files: `.eslintrc`, `.eslintignore`, `cypress/.eslintrc`.
|
||||
|
||||
- **Step 1: Create `eslint.config.mjs`**
|
||||
|
||||
```js
|
||||
import eslint from '@eslint/js';
|
||||
import chaiFriendly from 'eslint-plugin-chai-friendly';
|
||||
import cypressPlugin from 'eslint-plugin-cypress/flat';
|
||||
import tseslint from 'typescript-eslint';
|
||||
|
||||
export default tseslint.config(
|
||||
// Global ignores (replaces .eslintignore + ignorePatterns)
|
||||
{
|
||||
ignores: ['**/node_modules/', '**/dist/', '**/*.jsx']
|
||||
},
|
||||
|
||||
// Base configs (replaces "extends" in .eslintrc)
|
||||
eslint.configs.recommended,
|
||||
...tseslint.configs.recommended,
|
||||
|
||||
// Project rules (replaces "rules" in .eslintrc)
|
||||
{
|
||||
linterOptions: {
|
||||
reportUnusedDisableDirectives: true
|
||||
},
|
||||
rules: {
|
||||
'@typescript-eslint/no-non-null-assertion': 'off',
|
||||
'@typescript-eslint/no-explicit-any': 'off',
|
||||
'no-unused-vars': 'off',
|
||||
'@typescript-eslint/no-unused-vars': [
|
||||
'error',
|
||||
{
|
||||
argsIgnorePattern: '^_',
|
||||
varsIgnorePattern: '^_',
|
||||
caughtErrorsIgnorePattern: '^_',
|
||||
ignoreRestSiblings: true
|
||||
}
|
||||
]
|
||||
}
|
||||
},
|
||||
|
||||
// Cypress overrides (replaces cypress/.eslintrc)
|
||||
{
|
||||
files: ['cypress/**/*.ts'],
|
||||
...cypressPlugin.configs.recommended,
|
||||
plugins: {
|
||||
'chai-friendly': chaiFriendly
|
||||
},
|
||||
rules: {
|
||||
...chaiFriendly.configs.recommendedFlat.rules
|
||||
}
|
||||
}
|
||||
);
|
||||
```
|
||||
|
||||
> **Why `.mjs`:** Root `package.json` has no `"type": "module"`, so ESM requires the `.mjs` extension.
|
||||
>
|
||||
> **What's gone:**
|
||||
>
|
||||
> - `"root": true` — flat config doesn't cascade from parent dirs
|
||||
> - `"parser"` — `tseslint.configs.recommended` sets the parser automatically
|
||||
> - `--ext .ts,.tsx` — flat config uses `files` patterns in config objects
|
||||
> - `.eslintignore` — replaced by `ignores` array
|
||||
> - `cypress/.eslintrc` — merged with `files: ['cypress/**/*.ts']`
|
||||
|
||||
- **Step 2: Verify the config loads**
|
||||
|
||||
```bash
|
||||
npx eslint --print-config frontend/src/App.tsx
|
||||
```
|
||||
|
||||
Expected: JSON output showing merged config with `@typescript-eslint/*` rules.
|
||||
|
||||
- **Step 3: Commit**
|
||||
|
||||
```bash
|
||||
git add eslint.config.mjs
|
||||
git commit -m "chore: add eslint v9 flat config"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Task 3: Delete Old Config Files
|
||||
|
||||
**Files:**
|
||||
|
||||
- Delete: `.eslintrc`
|
||||
- Delete: `.eslintignore`
|
||||
- Delete: `cypress/.eslintrc`
|
||||
- **Step 1: Delete the old files**
|
||||
|
||||
```bash
|
||||
rm .eslintrc .eslintignore cypress/.eslintrc
|
||||
```
|
||||
|
||||
- **Step 2: Verify ESLint still works with flat config only**
|
||||
|
||||
```bash
|
||||
npx eslint frontend/src/App.tsx
|
||||
```
|
||||
|
||||
Expected: no errors (or only pre-existing lint errors, not config errors).
|
||||
|
||||
- **Step 3: Commit**
|
||||
|
||||
```bash
|
||||
git add .eslintrc .eslintignore cypress/.eslintrc
|
||||
git commit -m "chore: remove legacy eslintrc config files"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Task 4: Update Package Lint Scripts
|
||||
|
||||
**Files:**
|
||||
|
||||
- Modify: `frontend/package.json:10` — `"lint"` script
|
||||
- Modify: `libs/react-client/package.json:9` — `"lint"` script
|
||||
- Modify: `libs/copilot/package.json:10` — `"lint"` script
|
||||
|
||||
In flat config, `eslint` with no args defaults to `.` and file type matching is handled by config objects (no `--ext` needed).
|
||||
|
||||
- **Step 1: Update frontend lint script**
|
||||
|
||||
In `frontend/package.json`, change:
|
||||
|
||||
```diff
|
||||
- "lint": "eslint ./src --ext .ts,.tsx",
|
||||
+ "lint": "eslint",
|
||||
```
|
||||
|
||||
- **Step 2: Update react-client lint script**
|
||||
|
||||
In `libs/react-client/package.json`, change:
|
||||
|
||||
```diff
|
||||
- "lint": "eslint ./src --ext .ts,.tsx",
|
||||
+ "lint": "eslint",
|
||||
```
|
||||
|
||||
- **Step 3: Update copilot lint script**
|
||||
|
||||
In `libs/copilot/package.json`, change:
|
||||
|
||||
```diff
|
||||
- "lint": "eslint ./src --ext .ts,.tsx",
|
||||
+ "lint": "eslint",
|
||||
```
|
||||
|
||||
- **Step 4: Verify each package lints correctly**
|
||||
|
||||
```bash
|
||||
pnpm --filter @chainlit/app lint
|
||||
pnpm --filter @chainlit/react-client lint
|
||||
pnpm --filter @chainlit/copilot lint
|
||||
```
|
||||
|
||||
Expected: each lints its own directory (`.` = package root). No config errors.
|
||||
|
||||
> **Note:** This now also lints files outside `src/` (e.g. `vite.config.ts`, test files, storybook files). This is intentional and improves coverage. If any new lint errors surface in those files, fix them before proceeding.
|
||||
|
||||
- **Step 5: Verify root lint script still works**
|
||||
|
||||
```bash
|
||||
pnpm lint
|
||||
```
|
||||
|
||||
Expected: runs all three packages' lint in parallel, same as before.
|
||||
|
||||
- **Step 6: Commit**
|
||||
|
||||
```bash
|
||||
git add frontend/package.json libs/react-client/package.json libs/copilot/package.json
|
||||
git commit -m "chore: simplify package lint scripts for eslint v9 flat config"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Task 5: Update lint-staged Config
|
||||
|
||||
**Files:**
|
||||
|
||||
- Modify: `lint-staged.config.js`
|
||||
|
||||
This is the payoff: package-scoped globs with `pnpm --filter` for ESLint. Prettier stays as direct invocation (it already uses the root `.prettierrc` and lint-staged passes staged files directly).
|
||||
|
||||
- **Step 1: Replace `lint-staged.config.js` contents**
|
||||
|
||||
```js
|
||||
export default {
|
||||
// ── Frontend ──────────────────────────────────────
|
||||
'frontend/**/*.{ts,tsx}': [
|
||||
'pnpm --filter @chainlit/app lint -- --fix',
|
||||
'prettier --write',
|
||||
() => 'pnpm --filter @chainlit/app type-check'
|
||||
],
|
||||
|
||||
// ── React Client ──────────────────────────────────
|
||||
'libs/react-client/**/*.{ts,tsx}': [
|
||||
'pnpm --filter @chainlit/react-client lint -- --fix',
|
||||
'prettier --write',
|
||||
() => 'pnpm --filter @chainlit/react-client type-check'
|
||||
],
|
||||
|
||||
// ── Copilot ───────────────────────────────────────
|
||||
'libs/copilot/**/*.{ts,tsx}': [
|
||||
'pnpm --filter @chainlit/copilot lint -- --fix',
|
||||
'prettier --write',
|
||||
() => 'pnpm --filter @chainlit/copilot type-check'
|
||||
],
|
||||
|
||||
// ── Cypress (not a workspace package) ─────────────
|
||||
'cypress/**/*.ts': ['eslint --fix', 'prettier --write'],
|
||||
|
||||
// ── Root config files ─────────────────────────────
|
||||
'*.{js,cjs,mjs}': ['eslint --fix', 'prettier --write'],
|
||||
|
||||
// ── Python ────────────────────────────────────────
|
||||
'backend/**/*.py': [
|
||||
'uv run ruff check',
|
||||
'uv run ruff format --check',
|
||||
() => 'uv run dmypy run -- backend/'
|
||||
],
|
||||
|
||||
// ── GitHub Actions ────────────────────────────────
|
||||
'.github/workflows/**': ['actionlint']
|
||||
};
|
||||
```
|
||||
|
||||
> **How `pnpm --filter @chainlit/app lint -- --fix` works with lint-staged:**
|
||||
>
|
||||
> 1. lint-staged matches staged files against the glob (e.g. `frontend/**/*.{ts,tsx}`)
|
||||
> 2. For string commands, lint-staged appends matched file paths as arguments
|
||||
> 3. Command becomes: `pnpm --filter @chainlit/app lint -- --fix /abs/path/to/file1.tsx /abs/path/to/file2.tsx`
|
||||
> 4. pnpm passes everything after `--` to the package's `lint` script
|
||||
> 5. Script expands to: `eslint --fix /abs/path/to/file1.tsx /abs/path/to/file2.tsx`
|
||||
> 6. ESLint v9: explicit file args override the default `.` — lints **only** the staged files
|
||||
>
|
||||
> **Why prettier stays as direct invocation:** Prettier doesn't have a "default to `.`" mode. The root `.prettierrc` is already the single source of config. lint-staged passes staged files directly — no benefit from `pnpm --filter` indirection.
|
||||
>
|
||||
> **Why** `backend/**/*.py` **instead of** `**/*.py`: Scopes Python linting to backend only, consistent with the package-scoped approach for JS/TS.
|
||||
|
||||
- **Step 2: Verify lint-staged config is valid**
|
||||
|
||||
```bash
|
||||
npx lint-staged --debug 2>&1 | head -20
|
||||
```
|
||||
|
||||
Expected: no syntax/parse errors.
|
||||
|
||||
- **Step 3: Test with a staged file**
|
||||
|
||||
```bash
|
||||
# Stage a minor whitespace change in a frontend file, then:
|
||||
npx lint-staged --verbose
|
||||
```
|
||||
|
||||
Expected: runs `pnpm --filter @chainlit/app lint -- --fix` with only the staged file, NOT the entire package.
|
||||
|
||||
- **Step 4: Commit**
|
||||
|
||||
```bash
|
||||
git add lint-staged.config.js
|
||||
git commit -m "chore: scope lint-staged to packages with pnpm --filter"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Task 6: Fix Lint Errors in Newly Covered Files
|
||||
|
||||
ESLint now covers files outside `src/` (config files, tests, storybook). Some may have pre-existing lint issues.
|
||||
|
||||
**Files:**
|
||||
|
||||
- Varies — depends on what ESLint reports
|
||||
- **Step 1: Run full lint to find new errors**
|
||||
|
||||
```bash
|
||||
pnpm lint 2>&1
|
||||
```
|
||||
|
||||
- **Step 2: Fix any errors**
|
||||
|
||||
Most likely candidates:
|
||||
|
||||
- `vite.config.ts` files — may use `require()` or have unused imports
|
||||
- `frontend/tests/*.spec.tsx` — may have unused variables in test helpers
|
||||
- `libs/copilot/.storybook/*.ts` — may have `any` types (but rule is off)
|
||||
- `libs/copilot/stories/*.ts` — storybook conventions
|
||||
|
||||
Fix each file. If a file genuinely shouldn't be linted (e.g. auto-generated), add it to the `ignores` array in `eslint.config.mjs`.
|
||||
|
||||
- **Step 3: Verify clean lint**
|
||||
|
||||
```bash
|
||||
pnpm lint
|
||||
```
|
||||
|
||||
Expected: exit 0, no errors.
|
||||
|
||||
- **Step 4: Commit**
|
||||
|
||||
```bash
|
||||
git add -A
|
||||
git commit -m "fix: resolve lint errors in newly covered files"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Task 7: Final Verification
|
||||
|
||||
- **Step 1: Run full CI-equivalent checks**
|
||||
|
||||
```bash
|
||||
pnpm lint && pnpm type-check && pnpm --filter @chainlit/app test
|
||||
```
|
||||
|
||||
Expected: all pass.
|
||||
|
||||
- **Step 2: Test lint-staged end-to-end**
|
||||
|
||||
Make a trivial change to a file in each package, stage it, and run:
|
||||
|
||||
```bash
|
||||
npx lint-staged --verbose
|
||||
```
|
||||
|
||||
Verify:
|
||||
|
||||
- Frontend file change triggers only `@chainlit/app` lint + type-check
|
||||
- react-client file change triggers only `@chainlit/react-client` lint + type-check
|
||||
- A Python file change triggers only ruff + dmypy
|
||||
- No cross-package lint runs
|
||||
- **Step 3: Commit any remaining changes**
|
||||
|
||||
```bash
|
||||
git add -A
|
||||
git commit -m "chore: complete eslint v9 migration and lint-staged unification"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Risk Checklist
|
||||
|
||||
| Risk | Mitigation |
|
||||
| --------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------ |
|
||||
| `eslint-plugin-chai-friendly` doesn't work in flat config | It exports `configs.recommendedFlat` (verified at runtime). If it fails, wrap with `FlatCompat` from `@eslint/eslintrc`. |
|
||||
| `eslint-plugin-cypress` v6 breaking changes | The v6 `/flat` export is documented. The plugin only adds 2 rules (`no-assigning-return-values`, `no-unnecessary-waiting`). Verify they still trigger. |
|
||||
| New lint errors in previously unlinted files | Task 6 handles this. Scope is small: ~10 files outside `src/` across all packages. |
|
||||
| `pnpm --filter` + lint-staged file passing | Verified empirically: lint-staged appends absolute paths, pnpm passes them after `--`, ESLint v9 lints only those files. |
|
||||
@@ -0,0 +1,308 @@
|
||||
# CI Cache Optimization P2 Implementation Plan
|
||||
|
||||
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
|
||||
|
||||
**Goal:** Build the frontend artifact that CI still duplicates today exactly once per commit and restore it in downstream jobs, cutting repeated `@chainlit/react-client` builds from the `check-frontend` matrix.
|
||||
|
||||
**Architecture:** On the current `feat/refactor-scripts` branch, P1 is already implemented and P3 has already collapsed the old `prepare`/`validate` split in `e2e-tests`. The original P2 write-up in `docs/research/ci-cache-optimization.md` assumes `pnpm run buildUi` still runs in `tests` and `e2e-tests`, but that is no longer true in the current workflows. The remaining explicit duplicated frontend build is `pnpm --filter @chainlit/react-client build` inside the three-leg `check-frontend` matrix, so this plan seeds `libs/react-client/dist` once in `ci.yaml`, restores it in `check-frontend.yaml`, and leaves `tests`, `e2e-tests`, and backend checks untouched.
|
||||
|
||||
**Tech Stack:** GitHub Actions reusable workflows, `actions/cache@v5`, pnpm 9, TypeScript 5, tsup
|
||||
|
||||
## Assumptions
|
||||
|
||||
- P1 is already merged on this branch: `e2e-tests.yaml` uses `actions/cache@v5` for the Cypress binary and no longer uses `cypress-io/github-action`.
|
||||
- P3 is already merged on this branch: `e2e-tests.yaml` has a single `prepare` job and no separate `validate` job.
|
||||
- Backward compatibility matters more than theoretical optimality. Do not rename existing top-level CI jobs that branch protection may already depend on.
|
||||
- The cache boundary for this iteration is only `libs/react-client/dist`. Do not broaden scope to `frontend/dist` or `libs/copilot/dist` unless a fresh measurement proves they are still rebuilt in CI.
|
||||
|
||||
## Success Criteria
|
||||
|
||||
- `check-frontend` no longer runs `pnpm --filter @chainlit/react-client build` in each matrix job.
|
||||
- A new cache-seeding job restores or builds `libs/react-client/dist` once per `github.sha`.
|
||||
- `check-frontend` fails fast if the expected cache is missing, instead of silently rebuilding.
|
||||
- `check-backend`, `tests`, and `e2e-tests` keep their current behavior.
|
||||
- A second CI run for the same commit shows a cache hit in the seed job and no rebuild.
|
||||
|
||||
---
|
||||
|
||||
## File Map
|
||||
|
||||
| Action | File | Responsibility |
|
||||
| ------ | ---------------------------------------- | ---------------------------------------------------------------------------------- |
|
||||
| Modify | `.github/workflows/ci.yaml` | Add a one-time cache-seeding job and wire existing jobs to depend on it |
|
||||
| Modify | `.github/workflows/check-frontend.yaml` | Restore `libs/react-client/dist` from cache and remove the duplicated inline build |
|
||||
| Modify | `docs/research/ci-cache-optimization.md` | Update P2 notes so the research doc matches the current branch reality |
|
||||
|
||||
---
|
||||
|
||||
## Task 1: Seed `@chainlit/react-client` Build Cache Once
|
||||
|
||||
**Files:**
|
||||
|
||||
- Modify: `.github/workflows/ci.yaml`
|
||||
|
||||
- [ ] **Step 1: Add a dedicated cache-seeding job**
|
||||
|
||||
Insert this job above `check-frontend` in `.github/workflows/ci.yaml`:
|
||||
|
||||
```yaml
|
||||
build-react-client:
|
||||
name: Build @chainlit/react-client once
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- uses: actions/checkout@v6
|
||||
- uses: ./.github/actions/pnpm-node-install
|
||||
name: Install Node, pnpm and dependencies.
|
||||
- name: Cache @chainlit/react-client dist
|
||||
id: react-client-dist
|
||||
uses: actions/cache@v5
|
||||
with:
|
||||
path: libs/react-client/dist
|
||||
key: react-client-build-${{ github.sha }}
|
||||
- name: Build @chainlit/react-client
|
||||
if: steps.react-client-dist.outputs.cache-hit != 'true'
|
||||
run: pnpm --filter @chainlit/react-client build
|
||||
```
|
||||
|
||||
Why this exact shape:
|
||||
|
||||
- `github.sha` is the safest cache key for compiled output because it matches the exact source tree used by downstream jobs.
|
||||
- `actions/cache@v5` restores immediately and saves at job completion, so downstream jobs can rely on the cache after `needs: build-react-client`.
|
||||
- `ubuntu-latest` matches the current `check-frontend` runner, so no cross-OS cache settings are needed.
|
||||
|
||||
- [ ] **Step 2: Wire `check-frontend` and the CI aggregator to the new job**
|
||||
|
||||
In `.github/workflows/ci.yaml`, change the existing jobs to:
|
||||
|
||||
```yaml
|
||||
check-frontend:
|
||||
needs: [build-react-client]
|
||||
uses: ./.github/workflows/check-frontend.yaml
|
||||
secrets: inherit
|
||||
|
||||
ci:
|
||||
runs-on: ubuntu-slim
|
||||
name: Run CI
|
||||
if: always()
|
||||
needs: [check-backend, build-react-client, check-frontend, tests, e2e-tests]
|
||||
```
|
||||
|
||||
The extra `build-react-client` entry in `ci.needs` is required. Without it, a failed seed job can cause `check-frontend` to be skipped and the final `ci` job would not treat that upstream failure as a hard failure.
|
||||
|
||||
- [ ] **Step 3: Verify the workflow file still formats cleanly**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
pnpm format-check:files .github/workflows/ci.yaml
|
||||
```
|
||||
|
||||
Expected: exit code `0` and no output complaining about YAML formatting.
|
||||
|
||||
- [ ] **Step 4: Commit**
|
||||
|
||||
```bash
|
||||
git add .github/workflows/ci.yaml
|
||||
git commit -m "ci: seed react-client build cache once per commit"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Task 2: Restore the Cache in `check-frontend`
|
||||
|
||||
**Files:**
|
||||
|
||||
- Modify: `.github/workflows/check-frontend.yaml`
|
||||
|
||||
- [ ] **Step 1: Replace the inline build step with cache restore + explicit failure**
|
||||
|
||||
In `.github/workflows/check-frontend.yaml`, replace:
|
||||
|
||||
```yaml
|
||||
- name: Build @chainlit/react-client
|
||||
run: pnpm --filter @chainlit/react-client build
|
||||
```
|
||||
|
||||
with:
|
||||
|
||||
```yaml
|
||||
- name: Restore @chainlit/react-client dist
|
||||
id: react-client-dist
|
||||
uses: actions/cache@v5
|
||||
with:
|
||||
path: libs/react-client/dist
|
||||
key: react-client-build-${{ github.sha }}
|
||||
|
||||
- name: Fail if @chainlit/react-client dist cache is missing
|
||||
if: steps.react-client-dist.outputs.cache-hit != 'true'
|
||||
run: |
|
||||
echo "❌ Error: expected react-client build cache for ${GITHUB_SHA}"
|
||||
exit 1
|
||||
```
|
||||
|
||||
Do **not** add a fallback rebuild here. The point of P2 is to prove the seed job is the single source of truth for this build artifact. Silent rebuilds would hide broken cache wiring.
|
||||
|
||||
- [ ] **Step 2: Keep the matrix commands unchanged**
|
||||
|
||||
After the cache restore, the job should still run the existing commands exactly as they are today:
|
||||
|
||||
```yaml
|
||||
- name: ${{ matrix.name }}
|
||||
run: ${{ matrix.command }}
|
||||
```
|
||||
|
||||
This preserves the existing job names:
|
||||
|
||||
- `Linting: frontend`
|
||||
- `Formatting: frontend`
|
||||
- `Type checking: frontend`
|
||||
|
||||
and avoids unnecessary branch-protection churn.
|
||||
|
||||
- [ ] **Step 3: Run the local smoke checks in the same order CI depends on**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
pnpm --filter @chainlit/react-client build
|
||||
pnpm lint
|
||||
pnpm format-check
|
||||
pnpm type-check
|
||||
```
|
||||
|
||||
Expected:
|
||||
|
||||
- `pnpm --filter @chainlit/react-client build` writes `libs/react-client/dist`
|
||||
- `pnpm lint` passes
|
||||
- `pnpm format-check` passes
|
||||
- `pnpm type-check` passes
|
||||
|
||||
- [ ] **Step 4: Commit**
|
||||
|
||||
```bash
|
||||
git add .github/workflows/check-frontend.yaml
|
||||
git commit -m "ci: restore react-client build output in frontend checks"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Task 3: Update the Research Doc So P2 Is No Longer Stale
|
||||
|
||||
**Files:**
|
||||
|
||||
- Modify: `docs/research/ci-cache-optimization.md`
|
||||
|
||||
- [ ] **Step 1: Rewrite the P2 problem statement to match the current branch**
|
||||
|
||||
Update the P2 section so it says the following, in substance:
|
||||
|
||||
```md
|
||||
### P2: Build `@chainlit/react-client` once at CI startup, share via cache
|
||||
|
||||
**Status**: Still useful on `feat/refactor-scripts`, but the scope is narrower than the original draft.
|
||||
|
||||
**Current reality on this branch**:
|
||||
|
||||
- `tests.yaml` no longer runs `pnpm run buildUi`
|
||||
- `e2e-tests.yaml` no longer runs `pnpm run buildUi`
|
||||
- `check-frontend.yaml` still runs `pnpm --filter @chainlit/react-client build` once per matrix leg
|
||||
|
||||
**Approach**: Seed `libs/react-client/dist` once in `ci.yaml` with `actions/cache`, then restore it in `check-frontend.yaml` and fail on cache miss instead of rebuilding.
|
||||
```
|
||||
|
||||
Keep the rest of the research document intact. This is a surgical doc correction, not a fresh rewrite.
|
||||
|
||||
- [ ] **Step 2: Update the expected impact table**
|
||||
|
||||
Change the P2 estimate from the old “buildUi in 7 jobs” numbers to the narrower current branch estimate:
|
||||
|
||||
```md
|
||||
| Change | Current | After | Saving |
|
||||
| ------------------------------------------------------------ | ---------------------- | ----------- | ----------------------------- |
|
||||
| P2: cache `@chainlit/react-client` build in `check-frontend` | ~20-40s per matrix leg | ~5s restore | ~45-90s cumulative per CI run |
|
||||
```
|
||||
|
||||
Use the final measured numbers from the first successful CI run if they differ materially from this estimate.
|
||||
|
||||
- [ ] **Step 3: Verify markdown formatting**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
pnpm format-check:files docs/research/ci-cache-optimization.md
|
||||
```
|
||||
|
||||
Expected: exit code `0`.
|
||||
|
||||
- [ ] **Step 4: Commit**
|
||||
|
||||
```bash
|
||||
git add docs/research/ci-cache-optimization.md
|
||||
git commit -m "docs: narrow P2 cache optimization scope to current workflows"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Task 4: Verify the New Workflow Behavior in CI
|
||||
|
||||
**Files:**
|
||||
|
||||
- No file changes
|
||||
|
||||
- [ ] **Step 1: Push the branch and wait for the `CI` workflow to finish**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
git push
|
||||
```
|
||||
|
||||
Expected: a new `CI` workflow run starts for the current commit.
|
||||
|
||||
- [ ] **Step 2: Inspect the first run**
|
||||
|
||||
Verify in the GitHub Actions UI:
|
||||
|
||||
- `build-react-client` shows either:
|
||||
- cache miss + one `pnpm --filter @chainlit/react-client build`, or
|
||||
- cache hit + skipped build on a rerun of the same commit
|
||||
- each `check-frontend` matrix job restores `react-client-build-${{ github.sha }}`
|
||||
- no `check-frontend` matrix job logs an inline `pnpm --filter @chainlit/react-client build`
|
||||
|
||||
- [ ] **Step 3: Re-run the same commit once**
|
||||
|
||||
Use the GitHub Actions UI to re-run the workflow for the same SHA.
|
||||
|
||||
Expected:
|
||||
|
||||
- `build-react-client` shows `cache-hit: true`
|
||||
- the build step is skipped
|
||||
- all `check-frontend` matrix jobs still restore the cache and pass
|
||||
|
||||
- [ ] **Step 4: Record the measured timings in the research doc**
|
||||
|
||||
If the measured timing differs from the estimate in Task 3, update the numbers in `docs/research/ci-cache-optimization.md` before merging.
|
||||
|
||||
- [ ] **Step 5: Commit**
|
||||
|
||||
```bash
|
||||
git add docs/research/ci-cache-optimization.md
|
||||
git commit -m "docs: record measured P2 CI cache results"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Non-Goals
|
||||
|
||||
- Do not reintroduce a full `buildUi` job unless fresh evidence shows `tests` or `e2e-tests` need it again.
|
||||
- Do not cache `frontend/dist` or `libs/copilot/dist` in this change.
|
||||
- Do not change `pnpm-node-install` or `uv-python-install`; P2 does not require action-level abstraction.
|
||||
- Do not change publish workflows as part of this task.
|
||||
|
||||
## Self-Review
|
||||
|
||||
**Spec coverage:** This plan covers P2 only and explicitly adapts it to the current branch state after P1 and P3. It does not assume the stale “buildUi in 7 jobs” model from the research draft.
|
||||
|
||||
**Placeholder scan:** No `TODO`, `TBD`, or “handle appropriately” placeholders remain. Each workflow edit has exact YAML and each verification step has an exact command or a concrete UI check.
|
||||
|
||||
**Consistency:** The cache key is consistently `react-client-build-${{ github.sha }}` and the cached path is consistently `libs/react-client/dist` in both producer and consumer jobs.
|
||||
@@ -0,0 +1,309 @@
|
||||
# CI Cache Optimization Research
|
||||
|
||||
> Last CI run analyzed: [#5989](https://github.com/Chainlit/chainlit/actions/runs/24017813922) on `feat/parallel-cypress` (2026-04-06, success, 13m 36s wall clock)
|
||||
|
||||
## Timing Findings
|
||||
|
||||
### Critical Path
|
||||
|
||||
The pipeline bottleneck is **Windows E2E shards** (13m 12s worst case). 60% of that time is setup, not tests.
|
||||
|
||||
```
|
||||
e2e-tests / windows-latest-2 (13m 12s) ← CRITICAL PATH
|
||||
|
||||
┌─ Install Node/pnpm ──── 1m 57s ─┐
|
||||
│ Install Cypress ─────── 1m 33s │
|
||||
│ Install Python/uv ──── 2m 50s │ 7m 49s setup (60%)
|
||||
│ Build UI ────────────── 1m 29s │
|
||||
└──────────────────────────────────┘
|
||||
┌─ Run tests ───────────── 4m 19s ─┐ 4m 19s actual work
|
||||
└──────────────────────────────────┘
|
||||
```
|
||||
|
||||
### All Jobs (wall clock)
|
||||
|
||||
| Job | Total | pnpm install | Cypress | uv sync | Build UI | Tests |
|
||||
| ---------------------- | ---------- | ------------ | ------- | ---------- | -------- | -------- |
|
||||
| lint-backend | 1m 21s | — | — | 11s | — | mypy 58s |
|
||||
| lint-ui | 1m 52s | 30s | — | — | 1m 07s | lint 10s |
|
||||
| pytest (3.10) | 3m 09s | 33s | — | 58s | 1m 07s | 24s |
|
||||
| pytest (3.11) | 3m 04s | 33s | — | 56s | 1m 06s | 23s |
|
||||
| pytest (3.12) | 3m 03s | 30s | — | 53s | 1m 07s | 29s |
|
||||
| **pytest (3.13)** | **5m 45s** | 30s | — | **3m 40s** | 1m 06s | 25s |
|
||||
| e2e ubuntu (avg of 5) | ~5m 50s | ~31s | ~11s | ~55s | ~1m 07s | ~3m |
|
||||
| e2e windows (avg of 5) | ~11m 56s | ~2m 02s | ~1m 30s | ~2m 28s | ~1m 22s | ~3m 50s |
|
||||
|
||||
### Anomalies
|
||||
|
||||
- **pytest (3.13) uv sync: 3m 40s** — vs ~55s for 3.10–3.12. Verified from logs: uv cache save **failed** during this run due to a transient GitHub cache service outage (`"Our services aren't available right now"`). The 3.13 slowness is likely source-building wheels that lack pre-built binaries for Python 3.13. This is a one-time cost that gets cached naturally once the service recovers — not a config issue.
|
||||
- **Windows pnpm install: ~2m** — vs ~30s on Ubuntu. Disk I/O and process spawning overhead.
|
||||
- **uv cache save failed across all jobs** in this run — transient outage, not a config problem.
|
||||
|
||||
### Redundant Work
|
||||
|
||||
`pnpm run buildUi` (builds `libs/react-client` → `libs/copilot` → `frontend`) runs independently in **7 jobs**:
|
||||
|
||||
- 2 E2E jobs (ubuntu + windows) × ~1m 10s = ~2m 20s
|
||||
- 4 pytest jobs × ~1m 07s = ~4m 28s
|
||||
- 1 lint-ui job × ~1m 07s = ~1m 07s
|
||||
- **Total: ~8 min of cumulative runner time** rebuilding identical, OS-independent output
|
||||
|
||||
Note: On `feat/refactor-scripts`, the Hatch build hook is already bypassed — `uv-python-install` uses `--no-install-project --no-editable`, and `pnpm run buildUi` is an explicit step. The `lint-backend` workflow does **not** need the build.
|
||||
|
||||
---
|
||||
|
||||
## Current Cache Configuration
|
||||
|
||||
| Tool | Action | Cache mechanism | Status |
|
||||
| ------------------- | --------------------------------------------------------------------------------- | ------------------------------------------------------------- | ------------------------------------------------- |
|
||||
| pnpm store | `actions/setup-node@v6.3.0` | `cache: 'pnpm'`, `cache-dependency-path: '**/pnpm-lock.yaml'` | ✅ Working (30s installs on Ubuntu) |
|
||||
| uv download cache | `astral-sh/setup-uv@v8.0.0` | `enable-cache: true` | ✅ Working (55s on Ubuntu), ⚠️ cache miss on 3.13 |
|
||||
| Cypress binary | `cypress-io/github-action@v6` (run used v6 despite local file referencing v7.1.9) | Internal (built-in to the action) | ✅ Ubuntu (11s), ⚠️ Windows (90s) |
|
||||
| Python installation | `actions/setup-python@v6.2.0` | Built-in (pre-installed on runners) | ✅ Fast on Ubuntu |
|
||||
| UI build output | — | **Not cached** | ❌ Rebuilt 14× per run |
|
||||
|
||||
---
|
||||
|
||||
## Research: Recommended Approaches per Tool
|
||||
|
||||
### 1. pnpm — `actions/setup-node` and `pnpm/action-setup`
|
||||
|
||||
**Source**: [actions/setup-node README](https://github.com/actions/setup-node), [pnpm/action-setup README](https://github.com/pnpm/action-setup)
|
||||
|
||||
**What's recommended**:
|
||||
|
||||
- `actions/setup-node` with `cache: 'pnpm'` — caches pnpm's content-addressable store. This is **already in use** and is the recommended approach.
|
||||
- `pnpm/action-setup` also offers `cache: true` but using both is redundant.
|
||||
- Docs explicitly recommend committing `pnpm-lock.yaml` and using `--frozen-lockfile` in CI.
|
||||
|
||||
**What's NOT recommended**:
|
||||
|
||||
- Caching `node_modules/` directly. pnpm uses hard-linked content-addressable storage; caching `node_modules` breaks this model and can cause integrity issues.
|
||||
|
||||
**Assessment**: Current config is correct. No change needed for pnpm caching itself.
|
||||
|
||||
### 2. Cypress binary — `cypress-io/github-action`
|
||||
|
||||
**Source**: [cypress-io/github-action README — "Install Cypress only"](https://github.com/cypress-io/github-action#install-cypress-only)
|
||||
|
||||
**Recommended pattern** (from the README, section "Install Cypress only"):
|
||||
|
||||
```yaml
|
||||
- uses: actions/cache@v5
|
||||
with:
|
||||
path: |
|
||||
~/.cache/Cypress
|
||||
node_modules
|
||||
key: my-cache-${{ runner.os }}-${{ hashFiles('package-lock.json') }}
|
||||
- run: npm i cypress
|
||||
- uses: cypress-io/github-action@v7
|
||||
with:
|
||||
install: false
|
||||
```
|
||||
|
||||
**Key observations**:
|
||||
|
||||
1. The README shows explicit `actions/cache` for `~/.cache/Cypress` as the recommended approach when splitting install from test execution.
|
||||
2. The `cypress-io/github-action` has a `cache-key` input for custom cache keys.
|
||||
3. For pnpm projects, the README shows: install pnpm → setup-node with `cache: 'pnpm'` → let the Cypress action handle installation.
|
||||
|
||||
**Verified from CI logs (Ubuntu shard 1)**:
|
||||
|
||||
1. `pnpm install` (03:46:41–03:47:00) runs Cypress's `postinstall` hook, which **fully installs the binary** in ~9s:
|
||||
```
|
||||
cypress@14.5.3 postinstall: Installing Cypress (version: 14.5.3)
|
||||
cypress@14.5.3 postinstall: Done
|
||||
```
|
||||
2. `cypress-io/github-action@v6` with `runTests: false` (03:47:00–03:47:11, ~11s) then runs again. Its only log output: `"Skipping running tests: runTests parameter is false"`. It re-runs the install/verify (redundant) and saves the Cypress binary cache in its post-step.
|
||||
|
||||
**Conclusion**: The action is **redundant for installation**. Its only value is the **post-step cache save** of `~/.cache/Cypress`. This can be replaced with a plain `actions/cache` step that:
|
||||
|
||||
1. Restores `~/.cache/Cypress` BEFORE `pnpm install` (so postinstall skips the download)
|
||||
2. Saves it after the job completes
|
||||
|
||||
This follows the "Install Cypress only" pattern from the action's own README.
|
||||
|
||||
**Paths for Windows**: The `~/.cache/Cypress` path is Linux-only. Windows uses `~/AppData/Local/Cypress/Cache`. Both must be listed in the `actions/cache` path, OR use the `CYPRESS_CACHE_FOLDER` env var to standardize.
|
||||
|
||||
### 3. uv — `astral-sh/setup-uv`
|
||||
|
||||
**Source**: [setup-uv README](https://github.com/astral-sh/setup-uv), [uv GitHub Actions guide](https://github.com/astral-sh/uv/blob/main/docs/guides/integration/github.md), [uv caching concepts](https://github.com/astral-sh/uv/blob/main/docs/concepts/cache.md)
|
||||
|
||||
**What's recommended**:
|
||||
|
||||
- `enable-cache: true` — caches uv's download cache (wheels, sdists). **Already in use.**
|
||||
- `cache-python: true` — caches Python installations managed by uv. **Not currently used.** This would help when uv installs Python (vs `actions/setup-python`).
|
||||
- `cache-dependency-glob` — controls cache key. Defaults to `**/uv.lock` etc. Current config uses defaults.
|
||||
- Manual approach: `actions/cache` with `UV_CACHE_DIR` + `uv cache prune --ci` for more control.
|
||||
|
||||
**What's NOT recommended**:
|
||||
|
||||
- Caching `.venv/` directly. The uv docs don't mention or recommend this. uv is designed to be fast at resolving+linking from its cache.
|
||||
- Caching pre-built wheels. From the uv docs: _"it's often faster to omit pre-built wheels from the cache (and instead re-download them from the registry on each run)."_ Only source-built wheels benefit from caching.
|
||||
|
||||
**Verified from CI logs**: The 3.13 outlier coincided with a **transient GitHub cache service outage** (`"Failed to save: Our services aren't available right now"` in the post-step). All pytest jobs in this run failed to save their uv cache. The 3m40s `uv sync` is likely source-building wheels that don't have pre-built binaries for Python 3.13 yet, combined with the cache miss.
|
||||
|
||||
**Assessment**: `cache-python: true` would NOT help here — it caches the Python installation binary, not package wheels. The uv download cache (`enable-cache: true`, already in use) is the correct mechanism; it just failed to save in this run due to the outage. No config change needed — this resolves on the next successful cache save.
|
||||
|
||||
**Important from uv docs**: _"it's often faster to omit pre-built wheels from the cache (and instead re-download them from the registry on each run)."_ Only source-built wheels benefit from caching. The current `enable-cache: true` already handles this correctly.
|
||||
|
||||
### 4. UI build output — no standard tool
|
||||
|
||||
**Source**: No specific recommendation from any action. This is a custom optimization.
|
||||
|
||||
**Standard GitHub Actions patterns**:
|
||||
|
||||
1. **`actions/cache`** — Cache build output keyed on source hash. Pros: simple. Cons: 10GB cache limit shared across all branches; large build outputs can evict other caches.
|
||||
2. **`actions/upload-artifact` + `actions/download-artifact`** — Build once in a dedicated job, share artifacts with downstream jobs. Pros: no cache eviction issues, guaranteed fresh builds. Cons: adds a sequential dependency (build job must complete before shards start), artifact upload/download overhead.
|
||||
|
||||
**Assessment**: A dedicated build job with artifact sharing is the cleaner pattern for this case (14 consumers, deterministic output). But it changes the dependency graph — E2E and pytest jobs would need to wait for the build job.
|
||||
|
||||
---
|
||||
|
||||
## Proposed Changes
|
||||
|
||||
### P1: Replace `cypress-io/github-action` with `actions/cache` for Cypress binary
|
||||
|
||||
**Status**: Verified from logs — the action is redundant for installation.
|
||||
|
||||
**Why `install: false` + `runTests: false` doesn't help**: The action's cache restore happens in its **main step**, which runs AFTER `pnpm-node-install`. By then, `pnpm install` has already triggered Cypress's `postinstall` hook and downloaded the binary. The cache restore is too late. With both flags off, the action is effectively a no-op during the main step — it only saves the cache in its post-step for future runs.
|
||||
|
||||
The README's [`install: false` pattern](https://github.com/cypress-io/github-action#install-cypress-only) assumes the action is still used to **run tests**. We don't — we use `pnpm test:e2e` with `cypress-split`.
|
||||
|
||||
**Approach**: Place `actions/cache` **before** `pnpm install` so the Cypress binary is restored before the postinstall hook runs:
|
||||
|
||||
1. Add `actions/cache` for Cypress binary dir BEFORE `pnpm-node-install`
|
||||
2. Remove the `cypress-io/github-action` step entirely
|
||||
3. Cypress's `postinstall` hook skips download when it finds the binary pre-cached
|
||||
|
||||
```yaml
|
||||
# In e2e-tests.yaml, BEFORE pnpm-node-install:
|
||||
- name: Cache Cypress binary
|
||||
uses: actions/cache@v4
|
||||
with:
|
||||
path: |
|
||||
~/.cache/Cypress
|
||||
~/AppData/Local/Cypress/Cache
|
||||
key: cypress-${{ runner.os }}-${{ hashFiles('pnpm-lock.yaml') }}
|
||||
# Then remove the cypress-io/github-action step entirely.
|
||||
```
|
||||
|
||||
**Cypress version**: `package.json` specifies `^14.5.3`, `pnpm-lock.yaml` resolves to `14.5.4`. Cypress is installed by `pnpm install` (postinstall hook) in both the current and proposed flow — no version change.
|
||||
|
||||
**Expected critical-path saving**: ~1m 30s per Windows shard (eliminates the redundant action step AND speeds up postinstall via cache hit on warm cache).
|
||||
|
||||
### P2: Build UI once at CI startup, share via cache
|
||||
|
||||
**Status**: Primary optimization — eliminates ~1m build per downstream job (×7 jobs).
|
||||
|
||||
**Problem**: `pnpm run buildUi` runs independently in 7 jobs, producing identical OS-independent output each time. Without a dedicated build job, all parallel jobs miss the cache simultaneously on the first run and all rebuild.
|
||||
|
||||
**Build command**: `pnpm run buildUi` = `pnpm build:libs && cd frontend && pnpm run build`
|
||||
|
||||
**Build output directories**:
|
||||
|
||||
- `libs/react-client/dist/`
|
||||
- `libs/copilot/dist/`
|
||||
- `frontend/dist/`
|
||||
|
||||
**Approach**: Add a `build-ui` job in `ci.yaml` that builds once and saves to `actions/cache`. Downstream workflows restore from cache and skip the build.
|
||||
|
||||
**Cache key**: `ui-build-${{ github.sha }}`
|
||||
|
||||
Using the commit SHA is the simplest correct key. Enumerating source files in `hashFiles()` would be fragile — the build depends on `package.json`, `pnpm-lock.yaml`, `vite.config.*`, `tsconfig.*`, `index.html`, plus all source in `frontend/src/`, `libs/react-client/src/`, `libs/copilot/src/`. Too many inputs to track reliably. The trade-off: `github.sha` rebuilds on every new commit even if only backend files changed. But on cache hit the `build-ui` job is ~15s (restore only), so the wasted work is minimal. `github.run_id` would NOT work — it's unique per run, so the cache would never hit across runs (e.g. retries).
|
||||
|
||||
```yaml
|
||||
# ci.yaml
|
||||
jobs:
|
||||
build-ui:
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
- uses: ./.github/actions/pnpm-node-install
|
||||
- uses: actions/cache@v4
|
||||
id: ui-cache
|
||||
with:
|
||||
path: |
|
||||
frontend/dist
|
||||
libs/react-client/dist
|
||||
libs/copilot/dist
|
||||
key: ui-build-${{ github.sha }}
|
||||
- name: Build UI
|
||||
if: steps.ui-cache.outputs.cache-hit != 'true'
|
||||
run: pnpm run buildUi
|
||||
|
||||
pytest:
|
||||
needs: build-ui # wait for build
|
||||
uses: ./.github/workflows/pytest.yaml
|
||||
e2e-tests:
|
||||
needs: build-ui # wait for build
|
||||
uses: ./.github/workflows/e2e-tests.yaml
|
||||
lint-ui:
|
||||
needs: build-ui # wait for build
|
||||
uses: ./.github/workflows/lint-ui.yaml
|
||||
lint-backend: # no build needed — runs in parallel with build-ui
|
||||
uses: ./.github/workflows/lint-backend.yaml
|
||||
```
|
||||
|
||||
Downstream workflows restore from cache and skip `pnpm run buildUi`:
|
||||
|
||||
```yaml
|
||||
# In pytest.yaml, e2e-tests.yaml, lint-ui.yaml — replace the buildUi step:
|
||||
- uses: actions/cache@v4
|
||||
with:
|
||||
path: |
|
||||
frontend/dist
|
||||
libs/react-client/dist
|
||||
libs/copilot/dist
|
||||
key: ui-build-${{ github.sha }}
|
||||
fail-on-cache-miss: true # build-ui already ran, cache must exist
|
||||
```
|
||||
|
||||
**Why cache over artifacts?**:
|
||||
|
||||
- **Artifacts**: free for public repos (no storage cap), but re-upload every run even when nothing changed. Each run pays upload+download overhead.
|
||||
- **Cache**: 10 GB per repo (shared with pnpm/uv/Cypress caches), but on repeat runs with the same SHA (retries, re-triggered workflows) the `build-ui` job itself becomes ~15s (cache hit, skip build). Build output is ~50-100 MB, well within limits alongside other caches.
|
||||
|
||||
**Why `build-ui` is still needed**: Without it, all parallel downstream jobs would miss the cache simultaneously on the first run and all build independently. `needs: build-ui` guarantees the cache is populated before they start.
|
||||
|
||||
**Trade-off**: `build-ui` adds a sequential dependency. On cache miss: ~1m 30s (pnpm install ~30s + build ~1m). On cache hit: ~15s. `lint-backend` runs in parallel and takes ~1m 20s, so `build-ui` doesn't extend the critical path.
|
||||
|
||||
**Expected saving**: ~1m per downstream job × 7 jobs = **~7 min cumulative runner time**. Critical path saving: ~1m per Windows E2E job (build eliminated, ~5s cache restore).
|
||||
|
||||
### P3: Merge `validate` + `prepare` e2e jobs
|
||||
|
||||
**Approach**: Combine the two trivial `ubuntu-slim` jobs into one to eliminate a job startup cycle.
|
||||
|
||||
**Expected saving**: ~10-15s.
|
||||
|
||||
### ~~P4: Add `cache-python: true` to setup-uv~~ (dropped)
|
||||
|
||||
**Reason**: Verified from logs that the Python 3.13 outlier was caused by a transient GitHub cache service outage, not a config issue. `cache-python: true` caches the Python installation binary, which is irrelevant — the slowness came from source-building wheels. The existing `enable-cache: true` already caches built wheels correctly. No change needed.
|
||||
|
||||
---
|
||||
|
||||
## Estimated Impact on Critical Path
|
||||
|
||||
| Change | Current (win E2E) | After | Saving |
|
||||
| --------------------------------- | ------------------------------------ | ------------------- | ----------- |
|
||||
| P1: Cypress cache + remove action | 1m 33s (action) + ~30s (postinstall) | ~5s | ~2m |
|
||||
| P2: Build UI once + cache | ~1m 22s (build per job) | ~5s (cache restore) | ~1m 17s |
|
||||
| P3: merge validate+prepare | ~15s overhead | 0s | ~15s |
|
||||
| **Cumulative (critical path)** | **13m 12s** | **~9m 40s** | **~3m 32s** |
|
||||
|
||||
Notes:
|
||||
|
||||
- P1 and P2 are the most reliable wins. P2 eliminates redundant builds across all 7 consuming jobs (E2E, pytest, lint-ui) with a single approach.
|
||||
- P2's `build-ui` job adds a sequential dependency (~1m 30s on Ubuntu) but `lint-backend` runs in parallel and takes ~1m 20s, so `build-ui` doesn't extend the critical path.
|
||||
- Cumulative runner time saved (not just critical path): ~9m across all 7 jobs that currently build UI.
|
||||
|
||||
---
|
||||
|
||||
## Discarded Ideas (with reasons)
|
||||
|
||||
| Idea | Why discarded | Source |
|
||||
| -------------------------------- | -------------------------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| Cache `node_modules/` directly | pnpm uses hard-linked content-addressable storage; caching `node_modules` breaks this model | [pnpm docs](https://pnpm.io/faq#why-is-it-recommended-to-not-use-npm-ci-for-pnpm), [actions/setup-node](https://github.com/actions/setup-node) |
|
||||
| Cache `.venv/` directly | uv docs don't recommend this; uv is designed to resolve+link fast from its cache | [uv caching docs](https://github.com/astral-sh/uv/blob/main/docs/concepts/cache.md) |
|
||||
| `cache-python: true` in setup-uv | Wouldn't help — 3.13 outlier was source-building wheels (not Python install), and was caused by a transient cache service outage | Verified from CI logs |
|
||||
| Cache pre-built wheels | uv docs explicitly say _"it's often faster to omit pre-built wheels from the cache"_ — only source-built wheels benefit | [uv caching docs](https://github.com/astral-sh/uv/blob/main/docs/concepts/cache.md) |
|
||||
@@ -0,0 +1,916 @@
|
||||
# Copilot Type Checking — Research & Proposed Solutions
|
||||
|
||||
## Problem Statement
|
||||
|
||||
Running `tsc --noemit` in `libs/copilot/` fails because copilot **directly imports source files from `@chainlit/app` (the frontend)**, and those source files rely on path aliases/baseUrl settings defined in the frontend's own `tsconfig.json` — settings that are **not available** when TypeScript runs under copilot's `tsconfig.json`.
|
||||
|
||||
The goal is to make `pnpm --filter @chainlit/copilot type-check` work correctly with minimal changes to copilot source code and, if possible, `tsconfig.json`.
|
||||
|
||||
---
|
||||
|
||||
## Architecture Summary
|
||||
|
||||
### Monorepo Workspace (pnpm)
|
||||
|
||||
```
|
||||
pnpm-workspace.yaml:
|
||||
- frontend/ → @chainlit/app (private, no exports/main/types)
|
||||
- libs/react-client/ → @chainlit/react-client (published, has main/types → dist/)
|
||||
- libs/copilot/ → @chainlit/copilot (private)
|
||||
```
|
||||
|
||||
### Dependency Graph
|
||||
|
||||
```
|
||||
@chainlit/copilot
|
||||
├── @chainlit/app (workspace:^) ← imports SOURCE files
|
||||
└── @chainlit/react-client (workspace:^) ← imports from dist/ (proper library)
|
||||
```
|
||||
|
||||
### How Copilot Imports from `@chainlit/app`
|
||||
|
||||
Copilot reaches **directly into frontend source** via deep path imports:
|
||||
|
||||
```typescript
|
||||
// libs/copilot/src/chat/body.tsx
|
||||
import Alert from '@chainlit/app/src/components/Alert';
|
||||
import ChatSettingsModal from '@chainlit/app/src/components/ChatSettings';
|
||||
import { ErrorBoundary } from '@chainlit/app/src/components/ErrorBoundary';
|
||||
import { TaskList } from '@chainlit/app/src/components/Tasklist';
|
||||
import ChatFooter from '@chainlit/app/src/components/chat/Footer';
|
||||
import MessagesContainer from '@chainlit/app/src/components/chat/MessagesContainer';
|
||||
import ScrollContainer from '@chainlit/app/src/components/chat/ScrollContainer';
|
||||
import Translator from '@chainlit/app/src/components/i18n/Translator';
|
||||
import { useLayoutMaxWidth } from '@chainlit/app/src/hooks/useLayoutMaxWidth';
|
||||
import { useUpload } from '@chainlit/app/src/hooks/useUpload';
|
||||
import { IAttachment, attachmentsState } from '@chainlit/app/src/state/chat';
|
||||
```
|
||||
|
||||
Full list of imported modules across all copilot files (25 import lines, 22 unique paths, 7 files):
|
||||
|
||||
| Copilot File | Imported from `@chainlit/app/src/...` |
|
||||
| -------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| `widget.tsx` | `components/Alert`, `components/ui/button`, `components/ui/popover` |
|
||||
| `components/WelcomeScreen.tsx` | `components/chat/Starters`, `lib/utils` |
|
||||
| `components/Header.tsx` | `components/AudioPresence`, `components/Logo`, `components/header/ChatProfiles`, `components/header/NewChat`, `components/ui/button` |
|
||||
| `components/ElementSideView.tsx` | `components/Elements`, `components/ui/dialog` |
|
||||
| `appWrapper.tsx` | `i18n` |
|
||||
| `app.tsx` | `components/i18n/Translator` |
|
||||
| `chat/body.tsx` | `components/Alert`, `components/ChatSettings`, `components/ErrorBoundary`, `components/Tasklist`, `components/chat/Footer`, `components/chat/MessagesContainer`, `components/chat/ScrollContainer`, `components/i18n/Translator`, `hooks/useLayoutMaxWidth`, `hooks/useUpload`, `state/chat` |
|
||||
|
||||
### Import Categories
|
||||
|
||||
Not just UI components — copilot imports 6 distinct categories:
|
||||
|
||||
| Category | Count | Imports |
|
||||
| ------------------------ | ----- | --------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| **UI components** | 13 | `Alert`, `AudioPresence`, `ChatSettings`, `Elements`, `ErrorBoundary`, `Logo`, `Tasklist`, `Starters`, `button`, `dialog`, `popover`, `ChatProfiles`, `NewChat` |
|
||||
| **Composite chat views** | 4 | `Footer`, `MessagesContainer`, `ScrollContainer`, `Translator` |
|
||||
| **Hooks** | 2 | `useLayoutMaxWidth`, `useUpload` |
|
||||
| **Recoil state** | 1 | `state/chat` (`IAttachment`, `attachmentsState`) |
|
||||
| **Utilities** | 1 | `lib/utils` (`cn`, `hasMessage`) |
|
||||
| **i18n config** | 1 | `i18n` (`i18nSetupLocalization`) |
|
||||
|
||||
### Contrasted with `@chainlit/react-client` (works correctly)
|
||||
|
||||
`react-client` is a properly published library:
|
||||
|
||||
- `package.json` has `"main": "dist/index.js"` and `"types": "dist/index.d.ts"`
|
||||
- Built with `tsup`, emits `.d.ts` declarations
|
||||
- Copilot imports from the package root: `import { useConfig } from '@chainlit/react-client'`
|
||||
|
||||
---
|
||||
|
||||
## How Copilot Is Used
|
||||
|
||||
Copilot is **NOT a published npm package**. It is a **self-contained IIFE bundle** that gets embedded into the Python backend and served as a drop-in embeddable widget.
|
||||
|
||||
### Build & Distribution Flow
|
||||
|
||||
```
|
||||
libs/copilot/
|
||||
index.tsx ← Vite entry point
|
||||
vite.config.ts ← builds as IIFE format (not ESM/CJS library)
|
||||
↓ vite build (rollup, inlineDynamicImports: true)
|
||||
dist/
|
||||
index.js ← 8.5 MB single IIFE bundle (everything inlined)
|
||||
assets/ ← static assets (pdf.worker.min.mjs ~1 MB)
|
||||
↓ backend/build.py:copy_copilot()
|
||||
backend/chainlit/copilot/dist/
|
||||
index.js ← copied into the Python package
|
||||
↓ FastAPI serves at /copilot/*
|
||||
End user loads <script src="https://your-chainlit-app/copilot/index.js">
|
||||
↓ IIFE self-executes
|
||||
window.mountChainlitWidget(config) ← injects Shadow DOM chat widget
|
||||
```
|
||||
|
||||
### Key Properties
|
||||
|
||||
- **Format**: IIFE with `inlineDynamicImports: true` — no code splitting possible
|
||||
- **Rendering**: Uses Shadow DOM (`attachShadow({ mode: 'open' })`) with inlined CSS
|
||||
- **Global API**: Exposes `window.mountChainlitWidget()`, `window.unmountChainlitWidget()`, `window.sendChainlitMessage()`, etc.
|
||||
- **No TypeScript consumers**: Nobody imports from `@chainlit/copilot` — the type-check is purely a development-time quality gate
|
||||
- **Vite resolves successfully**: The build uses `vite-tsconfig-paths` plugin + explicit `resolve.alias` entries, so the IIFE builds correctly despite tsc failing
|
||||
|
||||
### Vite Config Resolution
|
||||
|
||||
```typescript
|
||||
// libs/copilot/vite.config.ts
|
||||
resolve: {
|
||||
alias: {
|
||||
react: path.resolve(__dirname, './node_modules/react'),
|
||||
'@chainlit': path.resolve(__dirname, './node_modules/@chainlit'),
|
||||
postcss: path.resolve(__dirname, './node_modules/postcss'),
|
||||
tailwindcss: path.resolve(__dirname, './node_modules/tailwindcss'),
|
||||
i18next: path.resolve(__dirname, './node_modules/i18next'),
|
||||
sonner: path.resolve(__dirname, './node_modules/sonner'),
|
||||
'highlight.js': path.resolve(__dirname, './node_modules/highlight.js'),
|
||||
'react-i18next': path.resolve(__dirname, './node_modules/react-i18next'),
|
||||
'usehooks-ts': path.resolve(__dirname, './node_modules/usehooks-ts'),
|
||||
lodash: path.resolve(__dirname, './node_modules/lodash'),
|
||||
recoil: path.resolve(__dirname, './node_modules/recoil')
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
The `@chainlit` alias points to `node_modules/@chainlit`, which pnpm symlinks:
|
||||
|
||||
```
|
||||
libs/copilot/node_modules/@chainlit/app → ../../../../frontend
|
||||
libs/copilot/node_modules/@chainlit/react-client → ../../../react-client
|
||||
```
|
||||
|
||||
This means Vite follows symlinks into the actual source directories — `@chainlit/app/src/components/Alert` resolves to `frontend/src/components/Alert.tsx` via filesystem. Vite resolves each file's imports independently using the `vite-tsconfig-paths` plugin, which can apply different tsconfig settings per file. **tsc cannot do this** — it uses a single tsconfig for all files in its program.
|
||||
|
||||
---
|
||||
|
||||
## Actual Error Analysis (Verified)
|
||||
|
||||
Running `pnpm --filter @chainlit/copilot type-check` produces **243 errors** across **35 unique frontend source files**.
|
||||
|
||||
### Error Categories
|
||||
|
||||
| Category | Example Error | Count | Root Cause |
|
||||
| --------------------------------- | -------------------------------------------------------------- | ----- | ------------------------------------------------------------------------------- |
|
||||
| **`@/` alias misresolution** | `Cannot find module '@/components/ui/button'` | ~180 | Copilot's `@/*` → `./src/*` resolves to `libs/copilot/src/` not `frontend/src/` |
|
||||
| **`@/` false match** | `Module '"@/types"' has no exported member 'IInput'` | ~20 | Same-named file exists in copilot's src with different exports |
|
||||
| **Bare `baseUrl` imports** | `Cannot find module 'components/i18n/Translator'` | ~40 | Frontend's `baseUrl: "./src"` doesn't apply under copilot's tsconfig |
|
||||
| **`client-types/` alias missing** | `Cannot find module 'client-types/*'` | 3 | Copilot has no mapping for frontend's `client-types/*` path |
|
||||
| **Vite-specific import** | `Cannot find module 'pdfjs-dist/build/pdf.worker.min.mjs?url'` | 1 | tsc doesn't understand Vite's `?url` import suffix |
|
||||
|
||||
### Top Missing Modules (by frequency)
|
||||
|
||||
```
|
||||
19× @/components/ui/button
|
||||
13× @/components/ui/tooltip
|
||||
6× @/components/ui/dialog
|
||||
6× @/components/ui/command
|
||||
5× @/components/ui/skeleton
|
||||
5× @/components/ui/badge
|
||||
5× components/i18n (bare import)
|
||||
4× @/components/ui/select
|
||||
4× @/components/ui/input
|
||||
4× @/components/Markdown
|
||||
4× @/components/Icon
|
||||
4× components/i18n/Translator (bare import)
|
||||
3× types/Input (bare import)
|
||||
3× @/state/chat
|
||||
3× client-types/ (missing alias)
|
||||
2× hooks/useFetch (bare import)
|
||||
```
|
||||
|
||||
### All Bare Module Imports (non-`@/`, non-`client-types/`)
|
||||
|
||||
These rely on `baseUrl` resolution and **cannot** be fixed via `paths` mappings alone:
|
||||
|
||||
```
|
||||
components/i18n
|
||||
components/i18n/Translator
|
||||
contexts/MessageContext
|
||||
hooks/useFetch
|
||||
state/chat
|
||||
state/project
|
||||
types/Input
|
||||
```
|
||||
|
||||
### Transitive Fan-Out
|
||||
|
||||
The 22 direct imports from copilot fan out into 35 unique frontend files with errors. The heaviest fan-out comes from three entry points:
|
||||
|
||||
1. **`ChatSettings`** → pulls in 12 sub-components (`CheckboxInput`, `SelectInput`, `DatePickerInput`, `MultiSelectInput`, `RadioButtonGroup`, `SliderInput`, `SwitchInput`, `TagsInput`, `TextInput`, `FormInput`, `InputStateHandler`) each importing 3-6 UI primitives
|
||||
2. **`Elements`** → pulls in `Plotly.tsx`, `PDF.tsx`, `LazyDataframe.tsx`, `CustomElement/`, `File.tsx`, `Image.tsx`, `Text.tsx`
|
||||
3. **`MessagesContainer`** → pulls in `MessageComposer/` (10+ subfiles: `Input`, `SubmitButton`, `UploadButton`, `VoiceButton`, `ModePicker`, `CommandButtons`, `Mcp/`, etc.)
|
||||
|
||||
---
|
||||
|
||||
## Root Cause Analysis
|
||||
|
||||
### The Core Problem: Cross-Project Path Alias Mismatch
|
||||
|
||||
When `tsc` runs with `libs/copilot/tsconfig.json`, it follows imports into `frontend/src/...` files. Those files are type-checked using copilot's path mappings, not frontend's. TypeScript has **no mechanism to apply different tsconfig settings to different source files within a single compilation**.
|
||||
|
||||
#### Copilot's tsconfig.json
|
||||
|
||||
```json
|
||||
{
|
||||
"compilerOptions": {
|
||||
"baseUrl": "./src",
|
||||
"paths": { "@/*": ["./*"] },
|
||||
"moduleResolution": "Node",
|
||||
"skipLibCheck": true,
|
||||
"noEmit": true
|
||||
},
|
||||
"include": ["./src"]
|
||||
}
|
||||
```
|
||||
|
||||
#### Frontend's tsconfig.json
|
||||
|
||||
```json
|
||||
{
|
||||
"compilerOptions": {
|
||||
"composite": true,
|
||||
"baseUrl": "./src",
|
||||
"paths": {
|
||||
"client-types/*": ["../../libs/react-client/dist"],
|
||||
"@/*": ["./*"]
|
||||
},
|
||||
"moduleResolution": "Node",
|
||||
"noEmit": true
|
||||
},
|
||||
"include": ["./src"]
|
||||
}
|
||||
```
|
||||
|
||||
#### Three categories of internal imports that fail:
|
||||
|
||||
1. **`@/` Path Alias** (130+ frontend files use this)
|
||||
- Frontend: `@/lib/utils` → `frontend/src/lib/utils`
|
||||
- Under copilot's tsconfig: `@/lib/utils` → `libs/copilot/src/lib/utils` (WRONG)
|
||||
|
||||
2. **`client-types/` Path Alias** (8 frontend files)
|
||||
- Frontend: `client-types/*` → `../../libs/react-client/dist`
|
||||
- Under copilot's tsconfig: not mapped → module not found
|
||||
|
||||
3. **Bare Module Imports via `baseUrl`** (33+ frontend files)
|
||||
- Frontend: `components/Alert` → `frontend/src/components/Alert` (via `baseUrl: "./src"`)
|
||||
- Under copilot's tsconfig: `components/Alert` → `libs/copilot/src/components/Alert` (WRONG)
|
||||
|
||||
### Why Frontend Type-Check Passes
|
||||
|
||||
`pnpm --filter @chainlit/app type-check` succeeds because tsc uses the frontend's own tsconfig where all aliases resolve correctly. The problem only manifests when **copilot's** tsc follows imports into frontend files.
|
||||
|
||||
### CI Impact
|
||||
|
||||
```yaml
|
||||
# .github/workflows/check-frontend.yaml
|
||||
- name: 'Type checking: frontend'
|
||||
run: pnpm type-check # runs: pnpm run --parallel type-check
|
||||
```
|
||||
|
||||
Root `package.json`: `"type-check": "pnpm run --parallel type-check"` — runs all three packages in parallel. Frontend and react-client pass; copilot fails. **CI is currently broken for copilot type-checking.**
|
||||
|
||||
---
|
||||
|
||||
## Bundle Size Analysis
|
||||
|
||||
### Methodology
|
||||
|
||||
Bundle analyzed using `rollup-plugin-visualizer` (v7.0.1, March 2026) with treemap template and gzip size calculation, integrated as a Vite plugin during a one-off build.
|
||||
|
||||
### Overview
|
||||
|
||||
| Metric | Size |
|
||||
| --------------------------- | --------------------------------- |
|
||||
| Rendered (pre-minification) | 20.0 MB |
|
||||
| Minified | 8,551.6 KB (8.3 MB) |
|
||||
| Gzipped | 2,518.7 KB (2.5 MB) |
|
||||
| Module count | 3,053 |
|
||||
| Additional asset | `pdf.worker.min.mjs` — 1,046.2 KB |
|
||||
|
||||
### Bundle Composition by Package
|
||||
|
||||
```
|
||||
10900.4 KB (53.2%) plotly.js
|
||||
1672.1 KB ( 8.2%) highlight.js
|
||||
938.5 KB ( 4.6%) lucide-react
|
||||
780.9 KB ( 3.8%) pdfjs-dist
|
||||
673.3 KB ( 3.3%) lodash
|
||||
612.9 KB ( 3.0%) katex
|
||||
459.6 KB ( 2.2%) sucrase
|
||||
331.6 KB ( 1.6%) date-fns
|
||||
327.4 KB ( 1.6%) @chainlit/app (frontend source)
|
||||
281.9 KB ( 1.4%) parse5
|
||||
268.2 KB ( 1.3%) react-dom
|
||||
193.2 KB ( 0.9%) recoil
|
||||
162.0 KB ( 0.8%) other
|
||||
154.6 KB ( 0.8%) zod
|
||||
145.9 KB ( 0.7%) tailwind-merge
|
||||
134.0 KB ( 0.7%) react-day-picker
|
||||
129.6 KB ( 0.6%) react-player
|
||||
118.2 KB ( 0.6%) micromark-core-commonmark
|
||||
112.9 KB ( 0.6%) @tanstack/table-core
|
||||
99.3 KB ( 0.5%) react-hook-form
|
||||
94.8 KB ( 0.5%) swr
|
||||
89.4 KB ( 0.4%) i18next
|
||||
66.9 KB ( 0.3%) entities
|
||||
57.3 KB ( 0.3%) react-pdf
|
||||
49.6 KB ( 0.2%) react-dropzone
|
||||
48.8 KB ( 0.2%) @radix-ui/react-select
|
||||
48.4 KB ( 0.2%) embla-carousel
|
||||
48.2 KB ( 0.2%) engine.io-client
|
||||
42.0 KB ( 0.2%) @chainlit/react-client
|
||||
```
|
||||
|
||||
### Top 30 Individual Modules
|
||||
|
||||
```
|
||||
1. 10900.3 KB plotly.js/dist/plotly.js
|
||||
2. 780.9 KB pdfjs-dist/build/pdf.mjs
|
||||
3. 612.9 KB katex/dist/katex.mjs
|
||||
4. 562.8 KB lodash/lodash.js
|
||||
5. 193.2 KB recoil/es/index.js
|
||||
6. 159.0 KB highlight.js/lib/languages/mathematica.js
|
||||
7. 154.6 KB zod/lib/index.mjs
|
||||
8. 136.2 KB react-dom/cjs/react-dom.production.min.js (copilot copy)
|
||||
9. 130.8 KB react-dom/cjs/react-dom.production.min.js (frontend copy)
|
||||
10. 112.9 KB @tanstack/table-core/build/lib/index.mjs
|
||||
11. 111.8 KB highlight.js/lib/languages/isbl.js
|
||||
12. 109.6 KB parse5/dist/tokenizer/index.js
|
||||
13. 107.1 KB parse5/dist/parser/index.js
|
||||
14. 99.3 KB react-hook-form/dist/index.esm.mjs
|
||||
15. 89.4 KB i18next/dist/esm/i18next.js
|
||||
16. 82.2 KB highlight.js/lib/languages/gml.js
|
||||
17. 80.4 KB copilot src/index.css (inlined Tailwind CSS)
|
||||
18. 76.4 KB highlight.js/lib/core.js
|
||||
19. 73.2 KB tailwind-merge (copilot copy)
|
||||
20. 72.8 KB tailwind-merge (frontend copy)
|
||||
21. 64.5 KB highlight.js/lib/languages/1c.js
|
||||
22. 62.7 KB highlight.js/lib/languages/sqf.js
|
||||
23. 48.8 KB @radix-ui/react-select/dist/index.mjs
|
||||
24. 48.4 KB embla-carousel/esm/embla-carousel.esm.js
|
||||
25. 47.1 KB sucrase/dist/esm/parser/plugins/typescript.js
|
||||
26. 46.6 KB entities/lib/esm/generated/decode-data-html.js
|
||||
27. 43.0 KB sucrase/dist/esm/parser/tokenizer/readWordTree.js
|
||||
28. 42.0 KB @chainlit/react-client/dist/index.mjs
|
||||
29. 37.2 KB react-dropzone/dist/es/index.js
|
||||
30. 34.1 KB @radix-ui/react-menu/dist/index.mjs
|
||||
```
|
||||
|
||||
### Dependency Origin Split
|
||||
|
||||
| Source | Size | % |
|
||||
| -------------------------------------- | ----------- | ----- |
|
||||
| Frontend's `node_modules` (transitive) | 16,952.9 KB | 82.7% |
|
||||
| Copilot's own `node_modules` (direct) | 3,554.7 KB | 17.3% |
|
||||
|
||||
**82.7% of the copilot bundle comes from frontend's transitive dependencies.**
|
||||
|
||||
### Duplicate Packages
|
||||
|
||||
Several packages are bundled **twice** — once resolved from `frontend/node_modules` and once from `libs/copilot/node_modules` — because pnpm's strict module isolation creates separate copies:
|
||||
|
||||
| Package | Frontend copy | Copilot copy | Waste |
|
||||
| ---------------------------- | ------------- | ------------ | ----------- |
|
||||
| react-dom | 131.4 KB | 136.9 KB | 131.4 KB |
|
||||
| tailwind-merge | 72.8 KB | 73.2 KB | 72.8 KB |
|
||||
| swr | 46.4 KB | 48.5 KB | 46.4 KB |
|
||||
| lucide-react | 934.6 KB | 3.9 KB | 3.9 KB |
|
||||
| scheduler | 4.4 KB | 4.4 KB | 4.4 KB |
|
||||
| uuid | 2.3 KB | 4.8 KB | 2.3 KB |
|
||||
| **Total duplicate overhead** | | | **~263 KB** |
|
||||
|
||||
Note: The `vite.config.ts` `resolve.alias` entries try to deduplicate `react`, `react-dom`, `recoil`, etc. but don't cover all packages (notably `tailwind-merge`, `swr` are missing).
|
||||
|
||||
### Packages Copilot Probably Doesn't Need
|
||||
|
||||
These are pulled in transitively via `Elements`, `ChatSettings`, and `MessagesContainer` and may not be necessary for a widget:
|
||||
|
||||
| Package | Size | Pulled in by |
|
||||
| -------------------- | -------------- | ------------------------------------------------------- |
|
||||
| plotly.js | 10,900 KB | `Elements` → `Plotly.tsx` → `react-plotly.js` |
|
||||
| pdfjs-dist | 781 KB | `Elements` → `PDF.tsx` → `react-pdf` |
|
||||
| katex | 613 KB | `MessagesContainer` → Markdown → `rehype-katex` |
|
||||
| sucrase | 460 KB | Tailwind CSS (runtime processor?) |
|
||||
| date-fns | 332 KB | `ChatSettings` → `DatePickerInput` → `react-day-picker` |
|
||||
| parse5 | 282 KB | Markdown → `rehype-raw` (HTML parser) |
|
||||
| zod | 155 KB | Frontend form validation |
|
||||
| @tanstack/table-core | 113 KB | `Elements` → `LazyDataframe` |
|
||||
| react-hook-form | 99 KB | `ChatSettings` form management |
|
||||
| react-player | 130 KB | `Elements` → video player |
|
||||
| embla-carousel | 48 KB | `ChatSettings` → carousel component |
|
||||
| **Subtotal** | **~13,913 KB** | **67.9% of total bundle** |
|
||||
|
||||
---
|
||||
|
||||
## Proposed Solutions
|
||||
|
||||
### Solution 1: Add Frontend Path Mappings to Copilot's tsconfig
|
||||
|
||||
**Approach**: Duplicate the frontend's path mappings in copilot's tsconfig, adjusted for the relative directory difference.
|
||||
|
||||
```jsonc
|
||||
// libs/copilot/tsconfig.json
|
||||
{
|
||||
"compilerOptions": {
|
||||
"baseUrl": "./src",
|
||||
"paths": {
|
||||
"@/*": [
|
||||
"./*", // copilot's own @/ imports
|
||||
"../../frontend/src/*" // frontend's @/ imports
|
||||
],
|
||||
"client-types/*": [
|
||||
"../../libs/react-client/dist/*" // frontend's client-types alias
|
||||
]
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
**Pros**:
|
||||
|
||||
- Minimal change (only `tsconfig.json`)
|
||||
- No source code changes
|
||||
- Quick to implement
|
||||
|
||||
**Cons**:
|
||||
|
||||
- `@/*` becomes ambiguous — TypeScript tries copilot path first, then frontend path; potential false resolution if both have a file with the same name
|
||||
- **Does NOT solve bare module imports** (`components/...`, `hooks/...`) because those rely on `baseUrl` and there's only one `baseUrl` per tsconfig (~40 errors remain)
|
||||
- Fragile: any new path alias in frontend must be duplicated in copilot
|
||||
- Couples copilot's tsconfig to frontend's internal structure
|
||||
|
||||
**Verdict**: Partial fix (~75% of errors). Handles `@/` and `client-types/` but not bare imports.
|
||||
|
||||
---
|
||||
|
||||
### Solution 2: TypeScript Project References (`tsc -b`)
|
||||
|
||||
**Approach**: Use TypeScript's composite project references so that copilot references the frontend project. The frontend would emit `.d.ts` declarations that copilot consumes.
|
||||
|
||||
```jsonc
|
||||
// libs/copilot/tsconfig.json
|
||||
{
|
||||
"references": [
|
||||
{ "path": "../../frontend" },
|
||||
{ "path": "./tsconfig.node.json" }
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
Frontend tsconfig already has `"composite": true`. Would need to remove `"noEmit": true` from frontend and configure `declarationDir`.
|
||||
|
||||
**Pros**:
|
||||
|
||||
- Architecturally correct TypeScript solution
|
||||
- Each project uses its own tsconfig
|
||||
- Incremental builds
|
||||
|
||||
**Cons**:
|
||||
|
||||
- **Does not help here** because copilot doesn't import from `@chainlit/app` (the package root) — it imports from `@chainlit/app/src/components/...` (deep source paths). Project references resolve the _package root_ to declaration output, not source sub-paths.
|
||||
- Requires frontend to emit declarations (non-trivial for a complex app)
|
||||
- Would need the frontend to define an `exports` map or copilot to change all import paths
|
||||
|
||||
**Verdict**: Only works if combined with changing import paths (see Solution 5 or 6).
|
||||
|
||||
---
|
||||
|
||||
### Solution 3: Separate Type-Check tsconfig for Copilot
|
||||
|
||||
**Approach**: Create a `tsconfig.typecheck.json` in copilot that extends the base config but adjusts resolution specifically for type-checking. Use `rootDirs` to create a virtual merged directory.
|
||||
|
||||
```jsonc
|
||||
// libs/copilot/tsconfig.typecheck.json
|
||||
{
|
||||
"extends": "./tsconfig.json",
|
||||
"compilerOptions": {
|
||||
"rootDirs": ["./src", "../../frontend/src"],
|
||||
"paths": {
|
||||
"@/*": ["./*", "../../frontend/src/*"],
|
||||
"client-types/*": ["../../libs/react-client/dist/*"]
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
Update `package.json`:
|
||||
|
||||
```json
|
||||
"type-check": "tsc --noemit -p tsconfig.typecheck.json"
|
||||
```
|
||||
|
||||
**Pros**:
|
||||
|
||||
- No changes to production tsconfig (IDE, Vite still use `tsconfig.json`)
|
||||
- Can be as hacky as needed without affecting build
|
||||
- `rootDirs` can help with bare module resolution
|
||||
|
||||
**Cons**:
|
||||
|
||||
- `rootDirs` doesn't change module resolution — it only affects output directory structure and relative import resolution between the listed roots. It won't make `components/Alert` resolve to `frontend/src/components/Alert`.
|
||||
- Still can't solve bare imports via `baseUrl` (~40 errors remain)
|
||||
- Extra config file to maintain
|
||||
|
||||
**Verdict**: Marginal improvement over Solution 1. Keeps production config clean but same fundamental limits.
|
||||
|
||||
---
|
||||
|
||||
### Solution 4: Frontend Barrel Exports (Source-Level)
|
||||
|
||||
**Approach**: Have the frontend expose an explicit public API that copilot imports, instead of reaching into source files.
|
||||
|
||||
Create `frontend/src/copilot-exports.ts`:
|
||||
|
||||
```typescript
|
||||
// Explicit re-exports for copilot consumption
|
||||
export { default as Alert } from './components/Alert';
|
||||
export { Button } from './components/ui/button';
|
||||
export {
|
||||
Popover,
|
||||
PopoverContent,
|
||||
PopoverTrigger
|
||||
} from './components/ui/popover';
|
||||
export { default as Starters } from './components/chat/Starters';
|
||||
export { cn, hasMessage } from './lib/utils';
|
||||
// ... all other components copilot needs
|
||||
```
|
||||
|
||||
Add to `frontend/package.json`:
|
||||
|
||||
```json
|
||||
{
|
||||
"exports": {
|
||||
"./copilot": "./src/copilot-exports.ts"
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
Then copilot imports:
|
||||
|
||||
```typescript
|
||||
import { Alert, Button, cn } from '@chainlit/app/copilot';
|
||||
```
|
||||
|
||||
**Pros**:
|
||||
|
||||
- Clean API boundary
|
||||
- Frontend controls what's public
|
||||
- Single file to maintain
|
||||
- tsc resolves it through one entry point
|
||||
|
||||
**Cons**:
|
||||
|
||||
- **Does NOT solve the transitive resolution problem**: tsc still follows imports inside `copilot-exports.ts` into the rest of the frontend source, hitting the same path alias issues in all 35 downstream files
|
||||
- Requires changing copilot import paths (moderate source changes)
|
||||
- Must keep the barrel file in sync
|
||||
|
||||
**Verdict**: Doesn't solve the root cause unless combined with declaration generation.
|
||||
|
||||
---
|
||||
|
||||
### Solution 5: Build Frontend Declarations (tsup) + Barrel Export
|
||||
|
||||
**Approach**: Build the frontend components that copilot needs into a declaration file (`.d.ts`) using tsup. Copilot consumes built type artifacts rather than source. This follows the same proven pattern as `@chainlit/react-client`.
|
||||
|
||||
#### Implementation
|
||||
|
||||
**1. Barrel file** — `frontend/src/copilot-exports.ts` (22 re-export lines):
|
||||
|
||||
```typescript
|
||||
export { default as Alert } from './components/Alert';
|
||||
export { default as ChatSettingsModal } from './components/ChatSettings';
|
||||
export { ErrorBoundary } from './components/ErrorBoundary';
|
||||
export { TaskList } from './components/Tasklist';
|
||||
export { default as ChatFooter } from './components/chat/Footer';
|
||||
export { default as MessagesContainer } from './components/chat/MessagesContainer';
|
||||
export { default as ScrollContainer } from './components/chat/ScrollContainer';
|
||||
export {
|
||||
default as Translator,
|
||||
useTranslation
|
||||
} from './components/i18n/Translator';
|
||||
export { default as Starters } from './components/chat/Starters';
|
||||
export { Element } from './components/Elements';
|
||||
export { default as AudioPresence } from './components/AudioPresence';
|
||||
export { Logo } from './components/Logo';
|
||||
export { default as ChatProfiles } from './components/header/ChatProfiles';
|
||||
export { default as NewChatButton } from './components/header/NewChat';
|
||||
export { Button } from './components/ui/button';
|
||||
export {
|
||||
Dialog,
|
||||
DialogContent,
|
||||
DialogHeader,
|
||||
DialogTitle
|
||||
} from './components/ui/dialog';
|
||||
export {
|
||||
Popover,
|
||||
PopoverContent,
|
||||
PopoverTrigger
|
||||
} from './components/ui/popover';
|
||||
export { useLayoutMaxWidth } from './hooks/useLayoutMaxWidth';
|
||||
export { useUpload } from './hooks/useUpload';
|
||||
export { type IAttachment, attachmentsState } from './state/chat';
|
||||
export { cn, hasMessage } from './lib/utils';
|
||||
export { i18nSetupLocalization } from './i18n';
|
||||
```
|
||||
|
||||
**2. Build script** — `frontend/package.json`:
|
||||
|
||||
```json
|
||||
{
|
||||
"scripts": {
|
||||
"build:copilot-types": "tsup src/copilot-exports.ts --dts-only --outDir copilot-dist"
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
**3. Path redirect** — `libs/copilot/tsconfig.json`:
|
||||
|
||||
```jsonc
|
||||
{
|
||||
"compilerOptions": {
|
||||
"paths": {
|
||||
"@/*": ["./*"],
|
||||
"@chainlit/app/copilot": [
|
||||
"../../frontend/copilot-dist/copilot-exports.d.ts"
|
||||
]
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
**4. Copilot imports** — 25 import lines across 7 files become ~7 consolidated imports:
|
||||
|
||||
```typescript
|
||||
// Before (body.tsx — 11 lines):
|
||||
import Alert from '@chainlit/app/src/components/Alert';
|
||||
import ChatSettingsModal from '@chainlit/app/src/components/ChatSettings';
|
||||
// ... 9 more
|
||||
|
||||
// After (body.tsx — 1 line):
|
||||
import { Alert, ChatSettingsModal, ErrorBoundary, TaskList, ChatFooter,
|
||||
MessagesContainer, ScrollContainer, Translator, useLayoutMaxWidth,
|
||||
useUpload, IAttachment, attachmentsState } from '@chainlit/app/copilot';
|
||||
```
|
||||
|
||||
#### Why Barrel + tsup (not multi-entry, not raw tsc)
|
||||
|
||||
- **TypeScript has no glob re-export syntax** — `export * from './**/*'` doesn't exist. The barrel must be an explicit list.
|
||||
- **tsup bundles declarations** (via `rollup-plugin-dts`) — the output `.d.ts` is **self-contained**, all types inlined, no `@/` or `client-types/` imports leak through. Raw `tsc --emitDeclarationOnly` preserves original import paths, so the same alias mismatch would reappear.
|
||||
- **`moduleResolution: "Node"`** (used by both projects) **ignores `exports` field** in `package.json`. Multi-entry approaches relying on `exports` maps would not work without upgrading `moduleResolution` to `"Node16"`/`"NodeNext"`/`"Bundler"` across the entire monorepo.
|
||||
- **Single barrel = 1 entry point, 1 output file, 1 path mapping**. Multi-entry would need 22 tsup entry points + 22 path mappings.
|
||||
|
||||
**Pros**:
|
||||
|
||||
- **Fully solves the problem**: copilot sees `.d.ts` files, no transitive source resolution. `skipLibCheck: true` (already set) skips checking inside declarations.
|
||||
- Follows the same pattern as `@chainlit/react-client` (proven in this repo)
|
||||
- Clean architecture with explicit API boundary
|
||||
- Frontend controls what's public
|
||||
|
||||
**Cons**:
|
||||
|
||||
- Requires a build step: `pnpm --filter @chainlit/app build:copilot-types` must run before copilot type-check
|
||||
- ~25 import line changes in 7 copilot source files
|
||||
- CI pipeline needs ordering
|
||||
- tsup must handle the transitive dependency tree for `.d.ts` generation — may need config tuning for path aliases
|
||||
- Must keep barrel file in sync when adding new imports
|
||||
|
||||
**Verdict**: Architecturally clean and fully solves the problem. Moderate effort. **Recommended for proper fix.**
|
||||
|
||||
---
|
||||
|
||||
### Solution 6: Extract `@chainlit/ui` Shared Package
|
||||
|
||||
**Approach**: Create a new workspace package `libs/ui/` containing the shared components (Alert, Button, Popover, etc.). Both frontend and copilot depend on it.
|
||||
|
||||
```
|
||||
libs/ui/ → @chainlit/ui
|
||||
src/
|
||||
components/
|
||||
Alert.tsx
|
||||
ui/button.tsx
|
||||
ui/popover.tsx
|
||||
ui/dialog.tsx
|
||||
...
|
||||
lib/utils.ts
|
||||
hooks/useLayoutMaxWidth.ts
|
||||
hooks/useUpload.tsx
|
||||
...
|
||||
package.json (with main/types pointing to dist/)
|
||||
tsconfig.json
|
||||
```
|
||||
|
||||
**Pros**:
|
||||
|
||||
- Architecturally the "right" solution
|
||||
- Clean dependency graph: `copilot → ui`, `frontend → ui`
|
||||
- Each package has its own tsconfig, fully independent type checking
|
||||
- Reusable for future packages
|
||||
|
||||
**Cons**:
|
||||
|
||||
- **Significant refactoring effort**: need to extract ~25 components/hooks, resolve their dependencies (some depend on `@chainlit/react-client` state atoms, Recoil, etc.)
|
||||
- Some components (e.g., `MessagesContainer`, `ChatFooter`, `ChatSettings`) are deeply coupled to frontend state — may not be easily extractable
|
||||
- Would also need to extract hooks (`useUpload`), state atoms (`attachmentsState`), and i18n config
|
||||
- Risk of circular dependencies
|
||||
- High risk of breaking changes
|
||||
|
||||
**Verdict**: Best long-term architecture but highest implementation cost. Not suitable for a quick fix.
|
||||
|
||||
---
|
||||
|
||||
### Solution 7: Use `paths` to Redirect `@chainlit/app/src/*` + Eliminate Conflicting Aliases
|
||||
|
||||
**Approach**: Instead of copilot's imports going through node_modules, map the entire `@chainlit/app/src/*` prefix in copilot's tsconfig, AND change frontend's internal imports to not conflict with copilot's aliases.
|
||||
|
||||
Step 1 — In copilot's tsconfig, map:
|
||||
|
||||
```jsonc
|
||||
{
|
||||
"paths": {
|
||||
"@chainlit/app/src/*": ["../../frontend/src/*"],
|
||||
"@/*": ["./*"]
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
Step 2 — Change frontend files imported by copilot to use **relative imports** instead of `@/` aliases.
|
||||
|
||||
**Pros**:
|
||||
|
||||
- Works with existing import structure in copilot (no copilot source changes)
|
||||
- Relative imports in frontend are more portable
|
||||
|
||||
**Cons**:
|
||||
|
||||
- Must change 35+ frontend files to use relative imports
|
||||
- `client-types/` and bare imports still need addressing
|
||||
- Goes against the frontend's established code style
|
||||
|
||||
**Verdict**: Partial solution with significant frontend churn.
|
||||
|
||||
---
|
||||
|
||||
### Solution 8: Vite-Based Type Checking (`vite-plugin-checker`)
|
||||
|
||||
**Approach**: Replace `tsc --noemit` with Vite-integrated type checking.
|
||||
|
||||
**Verdict**: **Does not work** — `vite-plugin-checker` runs `tsc` under the hood in a worker. It does NOT use Vite's resolver for tsc. Same errors.
|
||||
|
||||
---
|
||||
|
||||
### Solution 9: Skip Copilot Type-Checking Entirely
|
||||
|
||||
**Approach**: Remove copilot from `type-check` or set it to always pass.
|
||||
|
||||
```json
|
||||
"type-check": "echo 'Skipped: cross-project path alias conflict'"
|
||||
```
|
||||
|
||||
**Pros**:
|
||||
|
||||
- Zero effort, zero risk
|
||||
- Vite build already validates copilot compiles correctly
|
||||
- Copilot has **no downstream TypeScript consumers** — it's an IIFE bundle
|
||||
- IDE still provides type hints via `vite-tsconfig-paths`
|
||||
|
||||
**Cons**:
|
||||
|
||||
- Loses compile-time type safety for copilot
|
||||
- CI won't catch type errors until build step
|
||||
- Technical debt accumulates silently
|
||||
|
||||
**Verdict**: Pragmatic stopgap. More defensible than usual since copilot is a build artifact, not a library.
|
||||
|
||||
---
|
||||
|
||||
### Solution 10: Frontend Switches to Relative Imports Everywhere
|
||||
|
||||
**Approach**: Remove all `@/`, `client-types/`, and bare `baseUrl` imports from the frontend source, replacing with relative imports.
|
||||
|
||||
**Pros**:
|
||||
|
||||
- **Completely solves the problem** with zero changes to copilot
|
||||
- Most portable and tool-agnostic approach
|
||||
|
||||
**Cons**:
|
||||
|
||||
- **170+ files** need changing in frontend
|
||||
- Long relative paths hurt readability (`../../../../lib/utils`)
|
||||
- Goes against modern frontend conventions
|
||||
- High merge conflict risk
|
||||
|
||||
**Verdict**: Solves the problem completely but very high cost and DX regression.
|
||||
|
||||
---
|
||||
|
||||
### Solution 11: Aggressive Path Mapping (All Categories)
|
||||
|
||||
**Approach**: Map every import pattern the frontend uses — `@/`, `client-types/`, and all bare module prefixes — in copilot's tsconfig.
|
||||
|
||||
```jsonc
|
||||
// libs/copilot/tsconfig.json
|
||||
{
|
||||
"compilerOptions": {
|
||||
"baseUrl": "./src",
|
||||
"paths": {
|
||||
"@/*": ["./*", "../../frontend/src/*"],
|
||||
"client-types/*": ["../../libs/react-client/dist/*"],
|
||||
"components/*": ["../../frontend/src/components/*"],
|
||||
"hooks/*": ["../../frontend/src/hooks/*"],
|
||||
"state/*": ["../../frontend/src/state/*"],
|
||||
"types/*": ["../../frontend/src/types/*"],
|
||||
"contexts/*": ["../../frontend/src/contexts/*"]
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
**Pros**:
|
||||
|
||||
- Could resolve **all** error categories through tsconfig alone
|
||||
- No source code changes in either project
|
||||
- Quick to try
|
||||
|
||||
**Cons**:
|
||||
|
||||
- **Extremely fragile**: every bare import pattern in frontend must be mapped
|
||||
- Copilot's own files could be shadowed if names collide
|
||||
- `@/` dual resolution has silent false-match risk
|
||||
- Vite-specific `?url` import still fails
|
||||
- Any new import pattern in frontend breaks copilot's type-check
|
||||
|
||||
**Verdict**: Might work for ~95% of errors. Hacky but zero-source-change. Best option for "make CI green fast."
|
||||
|
||||
---
|
||||
|
||||
## Solution Comparison Matrix
|
||||
|
||||
| # | Solution | Copilot Source Changes | Copilot tsconfig Changes | Frontend Changes | Fully Solves? | Effort | Fragility |
|
||||
| --- | ---------------------------------- | ---------------------- | ------------------------ | ------------------ | --------------------------------- | ----------- | --------- |
|
||||
| 1 | Add paths to copilot tsconfig | None | Moderate | None | **Partial** (~75%) | Low | Medium |
|
||||
| 2 | Project references | None | Small | Moderate | **No** (deep imports bypass refs) | Medium | Low |
|
||||
| 3 | Separate typecheck tsconfig | None | New file | None | **Partial** (~75%) | Low | Medium |
|
||||
| 4 | Frontend barrel exports (source) | ~15 import changes | None | New barrel file | **No** (transitive issue) | Medium | Low |
|
||||
| 5 | **Compiled declarations + barrel** | ~25 import lines | Small | Barrel + tsup step | **Yes** | Medium | Low |
|
||||
| 6 | **Extract `@chainlit/ui`** | ~25 import lines | Small | Major refactor | **Yes** | High | Very Low |
|
||||
| 7 | Redirect paths + relative imports | None | Small | 35+ files | **Partial** | Medium-High | Medium |
|
||||
| 8 | Vite-based type checking | None | None | None | **No** | Low | N/A |
|
||||
| 9 | Skip type-check | Script change | None | None | **N/A** (bypass) | None | N/A |
|
||||
| 10 | All-relative imports in frontend | None | None | 170+ files | **Yes** | High | Low |
|
||||
| 11 | **Aggressive path mapping** | None | tsconfig only | None | **~95%** | Low | High |
|
||||
|
||||
---
|
||||
|
||||
## Recommended Approaches (Ranked)
|
||||
|
||||
### Tier 1 — Immediate (make CI green now)
|
||||
|
||||
**Solution 11** (aggressive path mapping) or **Solution 9** (skip type-check). These buy time with minimal risk.
|
||||
|
||||
### Tier 2 — Proper Fix, Moderate Effort
|
||||
|
||||
**Solution 5** (compiled declarations + barrel). Follows the `@chainlit/react-client` pattern already proven in this repo. Creates a clean API boundary. ~25 import line changes in copilot, one barrel file in frontend, one tsup build script.
|
||||
|
||||
### Tier 3 — Long-Term Architecture
|
||||
|
||||
**Solution 6** (extract `@chainlit/ui` shared package). Do this when the monorepo grows or when copilot needs more independence.
|
||||
|
||||
---
|
||||
|
||||
## Bundle Optimization Opportunities
|
||||
|
||||
Independent of type-checking, the bundle analysis reveals significant optimization potential:
|
||||
|
||||
### High Impact (if copilot doesn't need these features)
|
||||
|
||||
| Action | Savings (rendered) | Approach |
|
||||
| ----------------------------------- | ------------------ | -------------------------------------------------------- |
|
||||
| Exclude plotly.js | ~10,900 KB (53%) | Lazy-load or exclude `Plotly.tsx` from `Elements` barrel |
|
||||
| Exclude pdfjs-dist | ~781 KB (3.8%) | Lazy-load or exclude `PDF.tsx` from `Elements` barrel |
|
||||
| Exclude katex | ~613 KB (3.0%) | Use plain markdown without math rendering |
|
||||
| Exclude date-fns + react-day-picker | ~466 KB (2.3%) | Simplify ChatSettings (no date picker) |
|
||||
| Exclude sucrase + parse5 | ~742 KB (3.6%) | Review if needed at runtime |
|
||||
|
||||
### Medium Impact
|
||||
|
||||
| Action | Savings | Approach |
|
||||
| ----------------------- | -------- | --------------------------------------------------------------------------------- |
|
||||
| Tree-shake lodash | ~500 KB | `import pick from 'lodash/pick'` instead of `import { pick } from 'lodash'` |
|
||||
| Deduplicate packages | ~263 KB | Add missing entries to `vite.config.ts` `resolve.alias` (`tailwind-merge`, `swr`) |
|
||||
| Tree-shake highlight.js | Variable | Import only needed languages instead of all |
|
||||
|
||||
### Relationship to Type-Checking Solutions
|
||||
|
||||
**Solution 5 (barrel export)** naturally enables bundle optimization: the barrel controls exactly what copilot imports at the type level, and a corresponding change at the Vite level (e.g., re-exporting a lightweight `Element` component that lazy-loads heavy renderers) would reduce bundle size without changing copilot's source code.
|
||||
|
||||
**Solution 6 (extract `@chainlit/ui`)** would inherently solve both problems — shared UI components wouldn't carry heavyweight dependencies like plotly.js.
|
||||
|
||||
---
|
||||
|
||||
## Open Questions / Next Steps
|
||||
|
||||
1. **Which tier to pursue?** Quick fix (Solution 11/9), proper fix (Solution 5), or long-term (Solution 6)?
|
||||
2. **Bundle optimization scope**: Should copilot support Plotly/PDF/video rendering, or should those be frontend-only? This determines whether `Elements` needs to be included as-is or refactored.
|
||||
3. **CI pipeline ordering**: If Solution 5 is chosen, the `build:copilot-types` step needs to run before `type-check`. Can this be integrated into the existing `pnpm type-check` script or does it need a separate CI step?
|
||||
4. **`moduleResolution` upgrade**: Is upgrading from `"Node"` to `"Node16"` or `"Bundler"` on the roadmap? This would unlock `exports` map support and simplify cross-package resolution.
|
||||
@@ -0,0 +1,110 @@
|
||||
# E2E Test Parallel Execution Research
|
||||
|
||||
## Current Architecture: Strictly Sequential
|
||||
|
||||
The e2e tests **cannot run in parallel** with the current setup. There are four structural reasons.
|
||||
|
||||
### 1. Single Hardcoded Port
|
||||
|
||||
Every test backend spawns on port `8000` (`backend/chainlit/config.py`):
|
||||
|
||||
```python
|
||||
DEFAULT_PORT = 8000
|
||||
```
|
||||
|
||||
`runChainlit` in `cypress/support/run.ts` invokes `uv run chainlit run <entryPoint> -h --ci` without passing `--port`, so every backend binds to the default.
|
||||
|
||||
### 2. Kill-Restart Lifecycle Per Spec File
|
||||
|
||||
`cypress.config.ts` orchestrates a strict **kill → start → test → kill** cycle:
|
||||
|
||||
```ts
|
||||
on('before:spec', async (spec) => {
|
||||
await killChainlit();
|
||||
await runChainlit(spec);
|
||||
});
|
||||
|
||||
on('after:spec', async () => {
|
||||
await killChainlit();
|
||||
});
|
||||
```
|
||||
|
||||
`killChainlit()` uses `fkill(:8000)` — a port-based kill. If two test runners were active, one would kill the other's backend.
|
||||
|
||||
### 3. Each Spec Has Its Own Chainlit App
|
||||
|
||||
Each test directory (`cypress/e2e/<test_name>/`) contains:
|
||||
|
||||
- `main.py` — the Chainlit application for that test
|
||||
- `.chainlit/config.toml` — per-test configuration
|
||||
|
||||
`runChainlit` uses `spec.absolute` to find the test directory and spawns Chainlit with `CHAINLIT_APP_ROOT` set to that directory. Since each test has a completely different backend app, the server must be restarted between specs.
|
||||
|
||||
### 4. Cypress Runs Specs Sequentially
|
||||
|
||||
Standard `cypress run` processes spec files one at a time in a single browser. Cypress Cloud offers parallelization across multiple CI machines, but the project doesn't use it (no `--parallel` or `--record` flags in `pnpm test:e2e`).
|
||||
|
||||
### Special Case: Mid-Test Restarts
|
||||
|
||||
The `data_layer` spec calls `cy.task('restartChainlit', Cypress.spec)` to kill and re-launch the backend within a single test to verify thread persistence across server restarts.
|
||||
|
||||
---
|
||||
|
||||
## Cypress Parallelization Capabilities
|
||||
|
||||
### Built-in `--parallel` Flag Requires Cypress Cloud
|
||||
|
||||
Cypress does **not** support parallelization out of the box for free.
|
||||
|
||||
```bash
|
||||
cypress run --record --parallel
|
||||
```
|
||||
|
||||
- `--parallel` **must** be used with `--record`, which sends results to Cypress Cloud (paid).
|
||||
- Cypress Cloud acts as the orchestrator: it dynamically assigns spec files to available CI machines using a load-balancing strategy based on historical run durations.
|
||||
- There is no local multi-process parallelism — it is **multi-machine** parallelism coordinated by the cloud service.
|
||||
|
||||
### Cypress vs Playwright
|
||||
|
||||
| Feature | Cypress | Playwright |
|
||||
| ----------------- | ------------------------- | ------------------------ |
|
||||
| Local parallelism | Not supported | Built-in (`--workers=4`) |
|
||||
| CI sharding | Via Cypress Cloud (paid) | Built-in (`--shard=1/4`) |
|
||||
| Orchestration | Dynamic, based on history | Manual split |
|
||||
|
||||
### Free Alternatives for Cypress Parallelization
|
||||
|
||||
| Tool | Description |
|
||||
| -------------------------------------------------------------- | ------------------------------------------------------------------------------------- |
|
||||
| **Manual CI matrix sharding** | Split spec files across GitHub Actions matrix jobs using `--spec` |
|
||||
| **[sorry-cypress](https://sorry-cypress.dev/)** | Open-source, self-hosted drop-in replacement for Cypress Cloud; supports `--parallel` |
|
||||
| **[cypress-split](https://github.com/bahmutov/cypress-split)** | Plugin that splits specs across CI machines using `SPLIT`/`SPLIT_INDEX` env vars |
|
||||
| **[currents.dev](https://currents.dev/)** | Cypress Cloud alternative with a free tier |
|
||||
|
||||
---
|
||||
|
||||
## What Would It Take to Parallelize
|
||||
|
||||
### Bottlenecks and Solutions
|
||||
|
||||
| Bottleneck | Solution |
|
||||
| ------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| Hardcoded port `8000` | Assign a unique port per worker (e.g. `8000 + workerIndex`). Pass `--port` to `chainlit run` and configure `baseUrl` dynamically. |
|
||||
| Port-based kill (`fkill(:8000)`) | Switch to PID-based process management — store the child PID from `spawn()` and kill it directly. |
|
||||
| Single Cypress browser | Use Cypress Cloud `--parallel`, sorry-cypress, cypress-split, or shard tests across CI matrix jobs. |
|
||||
| Shared filesystem (`.chainlit/` dirs) | Already isolated per test. Some tests (e.g. `data_layer`) write temp files like `thread_history.pickle` that must remain isolated per worker. |
|
||||
|
||||
### Strategy A: CI Matrix Sharding (Simplest)
|
||||
|
||||
Split the ~50 spec files into N groups across separate GitHub Actions runners:
|
||||
|
||||
1. Add a matrix dimension to the workflow that assigns each group to a separate runner.
|
||||
2. Each runner uses port `8000` (since they are on separate machines).
|
||||
3. No changes to Cypress config or `run.ts` — pass `--spec` with the subset.
|
||||
|
||||
### Strategy B: Same-Machine Parallelism (More Work)
|
||||
|
||||
1. Make `CHAINLIT_APP_PORT` dynamic — read from an env var per worker.
|
||||
2. Replace `fkill(:port)` with PID tracking (`child.pid` from `spawn`).
|
||||
3. Use `cypress-split` or `sorry-cypress` to distribute specs across multiple Cypress processes, each with its own `baseUrl`.
|
||||
4. Each Cypress process gets its own `CYPRESS_BASE_URL` pointing to its dedicated backend port.
|
||||
Reference in New Issue
Block a user