chore: import upstream snapshot with attribution
AI Engine CI / engine (push) Failing after 0s
Check generated models / generated-models (push) Failing after 1s
Enterprise E2E (Playwright) / pick (push) Failing after 8s
Enterprise E2E (Playwright) / playwright-e2e-enterprise (push) Has been cancelled

This commit is contained in:
wehub-resource-sync
2026-07-13 11:59:14 +08:00
commit bf122cd71b
5779 changed files with 1480999 additions and 0 deletions
+5
View File
@@ -0,0 +1,5 @@
# Developer Guide
This guide has moved to the repository root for easier discovery.
**See [DeveloperGuide.md](../DeveloperGuide.md) for the current developer guide.**
+66
View File
@@ -0,0 +1,66 @@
# Exception Handling Guide
This guide outlines the common error handling patterns used within Stirling-PDF and provides tips for internationalising error messages. The examples cover the main languages found in the project: Java, JavaScript, HTML/CSS, and a small amount of Python.
## General Principles
- **Fail fast and log clearly.** Exceptions should provide enough information for debugging without exposing sensitive data.
- **Use consistent user messages.** Text shown to users must be pulled from the localisation files so that translations are centrally managed.
- **Avoid silent failures.** Always log unexpected errors and provide the user with a helpful message.
## Java
Java forms the core of Stirling-PDF. When adding new features or handling errors:
1. **Create custom exceptions** to represent specific failure cases. This keeps the code self-documenting and easier to handle at higher levels.
2. **Use `try-with-resources`** when working with streams or other closable resources to ensure clean-up even on failure.
3. **Return meaningful HTTP status codes** in controllers by throwing `ResponseStatusException` or using `@ExceptionHandler` methods.
4. **Log with context** using the projects logging framework. Include identifiers or IDs that help trace the issue.
5. **Internationalise messages** by placing user-facing text in `messages_en_US.properties` and referencing them with message keys.
## JavaScript
On the client side, JavaScript handles form validation and user interactions.
- Use `try`/`catch` around asynchronous operations (e.g., `fetch`) and display a translated error notice if the call fails.
- Validate input before sending it to the server and provide inline feedback with messages from the translation files.
- Log unexpected errors to the browser console for easier debugging, but avoid revealing sensitive information.
## HTML & CSS
HTML templates should reserve a space for displaying error messages. Example pattern:
```html
<div class="error" role="alert" th:text="${errorMessage}"></div>
```
Use CSS classes (e.g., `.error`) to style the message so it is clearly visible and accessible. Keep the markup simple to ensure screen readers can announce the error correctly.
## Python
Python scripts in this project are mainly for utility tasks. Follow these guidelines:
- Wrap file operations or external calls in `try`/`except` blocks.
- Print or log errors in a consistent format. If the script outputs messages to end users, ensure they are translatable.
Example:
```python
try:
perform_task()
except Exception as err:
logger.error("Task failed: %s", err)
print(gettext("task.error"))
```
## Internationalisation (i18n)
All user-visible error strings should be defined in the main translation file (`messages_en_US.properties`). Other language files will use the same keys. Refer to messages in code rather than hard-coding text.
When creating new messages:
1. Add the English phrase to `messages_en_US.properties`.
2. Reference the message key in your Java, JavaScript, or Python code.
3. Update other localisation files as needed.
Following these patterns helps keep Stirling-PDF stable, easier to debug, and friendly to users in all supported languages.
+319
View File
@@ -0,0 +1,319 @@
# Stirling PDF File History Specification
## Overview
Stirling PDF implements a client-side file history system using IndexedDB storage. File metadata, including version history and tool chains, are stored as `StirlingFileStub` objects that travel alongside the actual file data. This enables comprehensive version tracking, tool history, and file lineage management without modifying PDF content.
## Storage Architecture
### IndexedDB-Based Storage
File history is stored in the browser's IndexedDB using the `fileStorage` service, providing:
- **Persistent storage**: Survives browser sessions and page reloads
- **Large capacity**: Supports files up to 100GB+ with full metadata
- **Fast queries**: Optimized for file browsing and history lookups
- **Type safety**: Structured TypeScript interfaces
### Core Data Structures
```typescript
interface StirlingFileStub extends BaseFileMetadata {
id: FileId; // Unique file identifier (UUID)
quickKey: string; // Deduplication key: name|size|lastModified
thumbnailUrl?: string; // Generated thumbnail blob URL
processedFile?: ProcessedFileMetadata; // PDF page data and processing results
// File Metadata
name: string;
size: number;
type: string;
lastModified: number;
createdAt: number;
// Version Control
isLeaf: boolean; // True if this is the latest version
versionNumber?: number; // Version number (1, 2, 3, etc.)
originalFileId?: string; // UUID of the root file in version chain
parentFileId?: string; // UUID of immediate parent file
// Tool History
toolHistory?: ToolOperation[]; // Complete sequence of applied tools
}
interface ToolOperation {
toolName: string; // Tool identifier (e.g., 'compress', 'sanitize')
timestamp: number; // When the tool was applied
}
interface StoredStirlingFileRecord extends StirlingFileStub {
data: ArrayBuffer; // Actual file content
fileId: FileId; // Duplicate for indexing
}
```
## Version Management System
### Version Progression
- **v1**: Original uploaded file (first version)
- **v2**: First tool applied to original
- **v3**: Second tool applied (inherits from v2)
- **v4**: Third tool applied (inherits from v3)
- **etc.**
### Leaf Node System
Only the latest version of each file family is marked as `isLeaf: true`:
- **Leaf files**: Show in default file list, available for tool processing
- **History files**: Hidden by default, accessible via history expansion
### File Relationships
```
document.pdf (v1, isLeaf: false)
↓ compress
document.pdf (v2, isLeaf: false)
↓ sanitize
document.pdf (v3, isLeaf: true) ← Current active version
```
## Implementation Architecture
### 1. FileStorage Service (`fileStorage.ts`)
**Core Methods:**
```typescript
// Store file with complete metadata
async storeStirlingFile(stirlingFile: StirlingFile, stub: StirlingFileStub): Promise<void>
// Load file with metadata
async getStirlingFile(id: FileId): Promise<StirlingFile | null>
async getStirlingFileStub(id: FileId): Promise<StirlingFileStub | null>
// Query operations
async getLeafStirlingFileStubs(): Promise<StirlingFileStub[]>
async getAllStirlingFileStubs(): Promise<StirlingFileStub[]>
// Version management
async markFileAsProcessed(fileId: FileId): Promise<boolean> // Set isLeaf = false
async markFileAsLeaf(fileId: FileId): Promise<boolean> // Set isLeaf = true
```
### 2. File Context Integration
**FileContext** manages runtime state with `StirlingFileStub[]` in memory:
```typescript
interface FileContextState {
files: {
ids: FileId[];
byId: Record<FileId, StirlingFileStub>;
};
}
```
**Key Operations:**
- `addFiles()`: Stores new files with initial metadata
- `addStirlingFileStubs()`: Loads existing files from storage with preserved metadata
- `consumeFiles()`: Processes files through tools, creating new versions
### 3. Tool Operation Integration
**Tool Processing Flow:**
1. **Input**: User selects files (marked as `isLeaf: true`)
2. **Processing**: Backend processes files and returns results
3. **History Creation**: New `StirlingFileStub` created with:
- Incremented version number
- Updated tool history
- Parent file reference
4. **Storage**: Both parent (marked `isLeaf: false`) and child (marked `isLeaf: true`) stored
5. **UI Update**: FileContext updated with new file state
**Child Stub Creation:**
```typescript
export function createChildStub(
parentStub: StirlingFileStub,
operation: { toolName: string; timestamp: number },
resultingFile: File,
thumbnail?: string
): StirlingFileStub {
return {
id: createFileId(),
name: resultingFile.name,
size: resultingFile.size,
type: resultingFile.type,
lastModified: resultingFile.lastModified,
quickKey: createQuickKey(resultingFile),
createdAt: Date.now(),
isLeaf: true,
// Version Control
versionNumber: (parentStub.versionNumber || 1) + 1,
originalFileId: parentStub.originalFileId || parentStub.id,
parentFileId: parentStub.id,
// Tool History
toolHistory: [...(parentStub.toolHistory || []), operation],
thumbnailUrl: thumbnail
};
}
```
## UI Integration
### File Manager History Display
**FileManager** (`FileManager.tsx`) provides:
- **Default View**: Shows only leaf files (`isLeaf: true`)
- **History Expansion**: Click to show all versions of a file family
- **History Groups**: Nested display using `FileHistoryGroup.tsx`
**FileListItem** (`FileListItem.tsx`) displays:
- **Version Badges**: v1, v2, v3 indicators
- **Tool Chain**: Complete processing history in tooltips
- **History Actions**: "Show/Hide History" toggle, "Restore" for history files
### FileManagerContext Integration
**File Selection Flow:**
```typescript
// Recent files (from storage)
onRecentFileSelect: (stirlingFileStubs: StirlingFileStub[]) => void
// Calls: actions.addStirlingFileStubs(stirlingFileStubs, options)
// New uploads
onFileUpload: (files: File[]) => void
// Calls: actions.addFiles(files, options)
```
**History Management:**
```typescript
// Toggle history visibility
const { expandedFileIds, onToggleExpansion } = useFileManagerContext();
// Restore history file to current
const handleAddToRecents = (file: StirlingFileStub) => {
fileStorage.markFileAsLeaf(file.id); // Make this version current
};
```
## Data Flow
### New File Upload
```
1. User uploads files → addFiles()
2. Generate thumbnails and page count
3. Create StirlingFileStub with isLeaf: true, versionNumber: 1
4. Store both StirlingFile + StirlingFileStub in IndexedDB
5. Dispatch to FileContext state
```
### Tool Processing
```
1. User selects tool + files → useToolOperation()
2. API processes files → returns processed File objects
3. createChildStub() for each result:
- Parent marked isLeaf: false
- Child created with isLeaf: true, incremented version
4. Store all files with updated metadata
5. Update FileContext with new state
```
### File Loading (Recent Files)
```
1. User selects from FileManager → onRecentFileSelect()
2. addStirlingFileStubs() with preserved metadata
3. Load actual StirlingFile data from storage
4. Files appear in workbench with complete history intact
```
## Performance Optimizations
### Metadata Regeneration
When loading files from storage, missing `processedFile` data is regenerated:
```typescript
// In addStirlingFileStubs()
const needsProcessing = !record.processedFile ||
!record.processedFile.pages ||
record.processedFile.pages.length === 0;
if (needsProcessing) {
const result = await generateThumbnailWithMetadata(stirlingFile);
record.processedFile = createProcessedFile(result.pageCount, result.thumbnail);
}
```
### Memory Management
- **Blob URL Tracking**: Automatic cleanup of thumbnail URLs
- **Lazy Loading**: Files loaded from storage only when needed
- **LRU Caching**: File objects cached in memory with size limits
## File Deduplication
### QuickKey System
Files are deduplicated using `quickKey` format:
```typescript
const quickKey = `${file.name}|${file.size}|${file.lastModified}`;
```
This prevents duplicate uploads while allowing different versions of the same logical file.
## Error Handling
### Graceful Degradation
- **Storage Failures**: Files continue to work without persistence
- **Metadata Issues**: Missing metadata regenerated on demand
- **Version Conflicts**: Automatic version number resolution
### Recovery Scenarios
- **Corrupted Storage**: Automatic cleanup and re-initialization
- **Missing Files**: Stubs cleaned up automatically
- **Version Mismatches**: Automatic version chain reconstruction
## Developer Guidelines
### Adding File History to New Components
1. **Use FileContext Actions**:
```typescript
const { actions } = useFileActions();
await actions.addFiles(files); // For new uploads
await actions.addStirlingFileStubs(stubs); // For existing files
```
2. **Preserve Metadata When Processing**:
```typescript
const childStub = createChildStub(parentStub, {
toolName: 'compress',
timestamp: Date.now()
}, processedFile, thumbnail);
```
3. **Handle Storage Operations**:
```typescript
await fileStorage.storeStirlingFile(stirlingFile, stirlingFileStub);
const stub = await fileStorage.getStirlingFileStub(fileId);
```
### Testing File History
1. **Upload files**: Should show v1, marked as leaf
2. **Apply tool**: Should create v2, mark v1 as non-leaf
3. **Check FileManager**: History should show both versions
4. **Restore old version**: Should mark old version as leaf
5. **Check storage**: Both versions should persist in IndexedDB
## Future Enhancements
### Potential Improvements
- **Branch History**: Support for parallel processing branches
- **History Export**: Export complete version history as JSON
- **Conflict Resolution**: Handle concurrent modifications
- **Cloud Sync**: Sync history across devices
- **Compression**: Compress historical file data
### API Extensions
- **Batch Operations**: Process multiple version chains simultaneously
- **Search Integration**: Search within tool history and file metadata
- **Analytics**: Track usage patterns and tool effectiveness
---
**Last Updated**: January 2025
**Implementation**: Stirling PDF Frontend v2
**Storage Version**: IndexedDB with fileStorage service
+73
View File
@@ -0,0 +1,73 @@
<p align="center">
<img src="https://raw.githubusercontent.com/Stirling-Tools/Stirling-PDF/main/docs/stirling.png" width="80">
<br>
<h1 align="center">Stirling-PDF</h1>
</p>
# How to add new languages to Stirling-PDF
Fork Stirling-PDF and create a new branch out of `main`.
## Frontend Translation Files (TOML Format)
### Add Language Directory and Translation File
1. Create a new language directory in `frontend/editor/public/locales/`
- Use hyphenated format: `pl-PL` (not underscore)
2. Copy the reference translation file:
- Source: `frontend/editor/public/locales/en-US/translation.toml`
- Destination: `frontend/editor/public/locales/pl-PL/translation.toml`
3. Translate all entries in the TOML file
- Keep the TOML structure intact
- Preserve all placeholders like `{n}`, `{total}`, `{filename}`, `{{variable}}`
- See `scripts/translations/README.md` for translation tools and workflows
4. Update the language selector in the frontend to include your new language
Then make a Pull Request (PR) into `main` for others to use!
## Handling Untranslatable Strings
Sometimes, certain strings may not require translation because they are the same in the target language or are universal (like names of protocols, certain terminologies, etc.). To ensure accurate statistics for language progress, these strings should be added to the `ignore_translation.toml` file located in the `scripts` directory. This will exclude them from the translation progress calculations.
For example, if the English string `error` does not need translation in Polish, add it to the `ignore_translation.toml` under the Polish section:
**Note**: Use underscores in `ignore_translation.toml` even though frontend uses hyphens (e.g., `pl_PL` not `pl-PL`)
```toml
[pl_PL]
ignore = [
"language.direction", # Existing entries
"error" # Add new entries here
]
```
## Add New Translation Tags
> [!IMPORTANT]
> If you add any new translation tags, they must first be added to the `en-US/translation.toml` file. This ensures consistency across all language files.
- New translation tags **must be added** to `frontend/editor/public/locales/en-US/translation.toml` to maintain a reference for other languages.
- After adding the new tags to `en-US/translation.toml`, add and translate them in the respective language file (e.g., `pl-PL/translation.toml`).
- Use the scripts in `scripts/translations/` to validate and manage translations (see `scripts/translations/README.md`)
Make sure to place the entry under the correct language section. This helps maintain the accuracy of translation progress statistics and ensures that the translation tool or scripts do not misinterpret the completion rate.
### Validation Commands
Use the translation scripts in `scripts/translations/` directory:
```bash
# Analyze translation progress
python3 scripts/translations/translation_analyzer.py --language pl-PL
# Validate TOML structure
python3 scripts/translations/validate_json_structure.py --language pl-PL
# Validate placeholders
python3 scripts/translations/validate_placeholders.py --language pl-PL
```
See `scripts/translations/README.md` for complete documentation.
+30
View File
@@ -0,0 +1,30 @@
# Developer Guide Directory
This directory contains all development-related documentation for Stirling PDF.
## 📚 Documentation Index
### Core Development
- **[DeveloperGuide.md](../DeveloperGuide.md)** - Main developer setup and architecture guide (in repo root)
- **[Taskfile.yml](../Taskfile.yml)** - Unified task runner for all build/dev/test/lint commands
- **[EXCEPTION_HANDLING_GUIDE.md](./EXCEPTION_HANDLING_GUIDE.md)** - Exception handling patterns and i18n best practices
- **[HowToAddNewLanguage.md](./HowToAddNewLanguage.md)** - Internationalization and translation guide
### Features & Documentation
- **[AGENTS.md](./AGENTS.md)** - Agent-based functionality documentation
- **[USERS.md](./USERS.md)** - User-focused documentation and guides
## 🔗 Related Files in Root
- **[README.md](../README.md)** - Project overview and quick start
- **[CONTRIBUTING.md](../CONTRIBUTING.md)** - Contribution guidelines
- **[SECURITY.md](../SECURITY.md)** - Security policies and reporting
- **[DATABASE.md](../DATABASE.md)** - Database setup and configuration (usage guide)
- **[HowToUseOCR.md](../HowToUseOCR.md)** - OCR setup and configuration (usage guide)
## 📝 Contributing to Documentation
When adding new development documentation:
1. Place technical guides in this `devGuide/` directory
2. Update this index file with a brief description
3. Keep user-facing docs (README, CONTRIBUTING, SECURITY) in the root
4. Follow existing naming conventions (PascalCase for guides)
+47
View File
@@ -0,0 +1,47 @@
# STYLELINT.md
## Usage
Apply Stylelint to your project's CSS with the following steps:
1. **NPM Script**
- Go to directory: `devTools/`
- Add Stylelint & stylistic/stylelint-plugin
```bash
npm install --save-dev stylelint stylelint-config-standard
npm install --save-dev @stylistic/stylelint-plugin
```
- Add a script entry to your `package.json`:
```jsonc
{
"scripts": {
"lint:css:check": "stylelint \"../app/core/src/main/**/*.css\" \"../app/proprietary/src/main/resources/static/css/*.css\" --config .stylelintrc.json",
"lint:css:fix": "stylelint \"../app/core/src/main/**/*.css\" \"../app/proprietary/src/main/resources/static/css/*.css\" --config .stylelintrc.json --fix"
}
}
```
- Run the linter:
```bash
npm run lint:css:check
npm run lint:css:fix
```
2. **CLI Usage**
- Lint all CSS files:
```bash
npx stylelint ../app/core/src/main/**/*.css ../app/proprietary/src/main/resources/static/css/*.css
```
- Lint a single file:
```bash
npx stylelint ../app/proprietary/src/main/resources/static/css/audit-dashboard.css
```
- Apply automatic fixes:
```bash
npx stylelint "../app/core/src/main/**/*.css" "../app/proprietary/src/main/resources/static/css/*.css" --fix
```
For full configuration options and rule customization, refer to the official documentation: [https://stylelint.io](https://stylelint.io)
+45
View File
@@ -0,0 +1,45 @@
# Who is using Stirling-PDF?
Understanding the diverse applications of Stirling-PDF can be an invaluable resource for collaboration and learning. This page provides a directory of users and use cases. If you are using Stirling-PDF, consider sharing your experiences to help others and foster a community of best practices.
## Adding Yourself as a User
If you're using Stirling-PDF or have integrated it into your platform or workflow, please consider contributing to this list by describing your use case. You can do this by opening a pull request to this file and adding your details in the format below:
- **N**: Name of the organization or individual.
- **D**: A brief description of your usage.
- **U**: Specific features or capabilities utilized.
- **L**: Optional link for further information (e.g., website, blog post).
- **Q**: Contact information for sharing insights (optional).
Example entry:
```
* N: Example Corp
D: Using Stirling-PDF for automated document processing in our SaaS platform focusing on compression.
U: OCR, merging PDFs, metadata editing, encryption, compression.
L: https://example.com/stirling-pdf
Q: @example-user on Discord/email
```
---
## Requirements for Listing
- You must represent the entity you're listing and ensure the details are accurate.
- Trial deployments are welcome if they represent a realistic evaluation of Stirling-PDF in action.
- Community contributions, including home-lab setups or non-commercial uses, are encouraged.
---
## Users (Alphabetically)
* N:
D:
U:
L:
* N:
D:
U:
L: