run-llama--liteparse/CONTRIBUTING.md

Contributing to LiteParse

Thank you for your interest in contributing to LiteParse! This document provides guidelines and information for contributors.

Getting Started

1. Fork the repository
2. Clone your fork:

   git clone https://github.com/YOUR_USERNAME/liteparse.git
   cd liteparse
   

3. Install prerequisites (see Development Prerequisites)

What to Contribute?

In this project, we welcome a wide range of contributions, but we do want to maintain the spirit of the project. We are primarily focused on:

• Core algorithms for PDF parsing and text extraction
• OCR integrations and improvements
• Different types or modifications to output formats

We are less interested in:

• Markdown output
• Any LLM integration or agent code
• Anything that doesn't directly relate to improving the core parsing and extraction capabilities

Architecture Overview

LiteParse is written in Rust with bindings for multiple platforms:

crates/
├── liteparse/           # Core Rust library (parsing, grid projection, OCR, output)
├── pdfium-sys/          # Raw FFI bindings to PDFium (auto-downloads pdfium)
├── pdfium/              # Safe Rust wrapper around pdfium-sys
├── liteparse-napi/      # Node.js native addon (napi-rs)
├── liteparse-python/    # Python extension module (PyO3 + maturin)
└── liteparse-wasm/      # WebAssembly bindings (wasm-bindgen + wasm-pack)

packages/
├── node/                # Node.js package (@llamaindex/liteparse)
├── python/              # Python package (liteparse)
└── wasm/                # WASM package (@llamaindex/liteparse-wasm)

Development Prerequisites

You'll need the following tools installed:

Tool	Purpose	Install
Rust toolchain	Core library and all bindings	rustup.rs
napi-rs CLI	Node.js native addon builds	npm i -g @napi-rs/cli
maturin	Python extension builds	pip install maturin
wasm-pack	WebAssembly builds	cargo install wasm-pack

PDFium is auto-downloaded by the pdfium-sys build script — no manual setup needed. For WASM, a static libpdfium.a is downloaded and linked into the .wasm binary. For native targets, a shared library (.dylib/.so/.dll) is downloaded and copied to the build output.

Building

Important: Workspace-wide cargo build will fail

The binding crates (liteparse-python, liteparse-napi, liteparse-wasm) each require their own specialized toolchain to link correctly. A plain cargo build at the workspace root will fail because, for example, the Python bindings need a Python interpreter to resolve _Py* symbols.

Core Rust library only

cargo build -p liteparse

Node.js bindings (napi-rs)

cd packages/node
npm install
npm run build          # Builds Rust → .node addon, copies pdfium, compiles TS

Individual steps:

npm run build:rs       # napi build (compiles liteparse-napi crate)
npm run build:pdfium   # Copies pdfium shared library alongside the addon
npm run build:ts       # Compiles TypeScript wrapper

To test locally, import from the package directly or use npm link:

import { LiteParse } from './packages/node/dist/lib.js';

Python bindings (maturin + PyO3)

cd packages/python
maturin develop        # Builds Rust and installs into active virtualenv

maturin develop compiles the liteparse-python crate and installs the resulting package into your current Python virtual environment. Then test with:

import liteparse

WASM bindings (wasm-pack)

cd packages/wasm
npm run build          # Browser target (--target web)
npm run build:bundler  # Bundler target (webpack/vite)
npm run build:nodejs   # Node.js target

PDFium is statically linked into the .wasm binary — the output in packages/wasm/pkg/ is fully self-contained.

Development Workflow

Testing Local Changes

# Parse a document (Node.js CLI)
cd packages/node && npm run build
node dist/cli.js parse document.pdf

# Python CLI
cd packages/python && maturin develop
lit parse document.pdf

Linting & Formatting

cargo fmt              # Format Rust code
cargo clippy           # Lint Rust code

Debugging Grid Projection

When working on the grid projection algorithm, you can enable built-in debug logging and visual output instead of adding ad-hoc console.log statements.

Debug logging traces every decision the projection makes — block detection, anchor extraction, snap assignment, rendering, and flowing text classification:

lit parse document.pdf --debug
lit parse document.pdf --debug --debug-page 3
lit parse document.pdf --debug --debug-text-filter "Total" "Revenue"
lit parse document.pdf --debug --debug-region "0,100,300,200"
lit parse document.pdf --debug --debug-output ./debug-output

Visual grid export generates PNG images showing text boxes color-coded by snap type (blue=left, red=right, green=center, gray=floating, yellow=flowing) with anchor lines overlaid:

lit parse document.pdf --debug-visualize
lit parse document.pdf --debug-visualize --debug-output ./my-debug

Pull Requests

1. Fork and create a feature branch from main
2. Make your changes
3. Ensure linting passes (cargo fmt --check && cargo clippy)
4. Submit a pull request

When you submit a PR, a number of CICD checks will run. Among these, your code will be tested against a regression suite of documents to ensure that your changes don't break existing parsing capabilities. It will be up to the maintainers discretion to determine if any changes to the regression set are expected/positive or unexpected/negative.

PR Guidelines

• Keep PRs focused on a single change
• Update documentation if needed
• Add tests for new functionality
• For parsing issues, include a test document if possible

Reporting Issues

Parsing Issues

If you're reporting a problem with document parsing:

1. You must attach the document or provide a way to reproduce the issue
2. Include the command you ran
3. Show the expected vs actual output
4. Include your LiteParse version (lit --version)

Issues without reproducible examples will be closed.

Bug Reports

For other bugs:

1. Describe what you expected vs what happened
2. Include steps to reproduce
3. Include error messages/stack traces
4. Include version information

Questions?

• Open a Discussion for questions
• Check existing issues before opening new ones
• Read the README for usage documentation

License

By contributing, you agree that your contributions will be licensed under the Apache 2.0 License.