MLflow TypeScript SDK

MLflow Typescript SDK is a variant of the [MLflow Python SDK](https://github.com/mlflow/mlflow) that provides a TypeScript API for MLflow. > [!IMPORTANT] > MLflow Typescript SDK is catching up with the Python SDK. Currently only support [Tracing]() and [Feedback Collection]() features. Please raise an issue in Github if you need a feature that is not supported. ## Packages | Package | NPM | Description | | --------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------- | | [@mlflow/core](./core) | [![npm package](https://img.shields.io/npm/v/%40mlflow%2Fcore?style=flat-square)](https://www.npmjs.com/package/@mlflow/core) | The core tracing functionality and manual instrumentation. | | [@mlflow/openai](./integrations/openai) | [![npm package](https://img.shields.io/npm/v/%40mlflow%2Fopenai?style=flat-square)](https://www.npmjs.com/package/@mlflow/openai) | Auto-instrumentation integration for OpenAI. | ## Installation ```bash npm install @mlflow/core ``` > [!NOTE] > MLflow Typescript SDK requires Node.js 20 or higher. ## Quickstart Start MLflow Tracking Server if you don't have one already: ```bash pip install mlflow mlflow server --backend-store-uri sqlite:///mlruns.db --port 5000 ``` Self-hosting MLflow server requires Python 3.10 or higher. If you don't have one, you can also use [managed MLflow service](https://mlflow.org/#get-started) for free to get started quickly. Instantiate MLflow SDK in your application: ```typescript import * as mlflow from '@mlflow/core'; mlflow.init({ trackingUri: 'http://localhost:5000', experimentId: '', }); ``` ### Configure with environment variables The SDK can also read configuration from environment variables so you can avoid hard-coding connection details. If `MLFLOW_TRACKING_URI` and `MLFLOW_EXPERIMENT_ID` are set, you can initialize the client without passing any arguments: ```bash export MLFLOW_TRACKING_URI=http://localhost:5000 export MLFLOW_EXPERIMENT_ID=123456789 ``` ```typescript import * as mlflow from '@mlflow/core'; mlflow.init(); // Uses the values from the environment ``` ### Authentication For MLflow tracking servers that require authentication, the SDK supports: 1. **Basic Auth** (username/password): ```typescript mlflow.init({ trackingUri: 'http://localhost:5000', experimentId: '123456789', trackingServerUsername: 'user', trackingServerPassword: 'pass', }); ``` Or via environment variables: ```bash export MLFLOW_TRACKING_USERNAME=user export MLFLOW_TRACKING_PASSWORD=pass ``` 2. **Bearer Token**: ```typescript mlflow.init({ trackingUri: 'http://localhost:5000', experimentId: '123456789', trackingServerToken: 'my-token', }); ``` Or via environment variable: ```bash export MLFLOW_TRACKING_TOKEN=my-token ``` 3. **No authentication** (default for local development) Create a trace: ```typescript // Wrap a function with mlflow.trace to generate a span when the function is called. // MLflow will automatically record the function name, arguments, return value, // latency, and exception information to the span. const getWeather = mlflow.trace( (city: string) => { return `The weather in ${city} is sunny`; }, // Pass options to set span name. See https://mlflow.org/docs/latest/genai/tracing/quickstart // for the full list of options. { name: 'get-weather' }, ); getWeather('San Francisco'); // Alternatively, start and end span manually const span = mlflow.startSpan({ name: 'my-span' }); span.end(); ``` Tag spans with a severity level so users (or you) can filter by **Minimum log level** in the trace UI: ```typescript import { SpanLogLevel } from '@mlflow/core'; const tracedAnswer = mlflow.trace((query: string) => llm.generate(query), { name: 'answer', spanType: mlflow.SpanType.CHAT_MODEL, logLevel: SpanLogLevel.INFO, }); // The string form works too: mlflow.startSpan({ name: 'plumbing', logLevel: 'DEBUG' }); ``` When you use one of the autolog integrations (`@mlflow/openai`, `@mlflow/anthropic`, `@mlflow/gemini`, etc.), MLflow stamps a sensible default level on every span based on its type — you don't need to annotate manually. View traces in MLflow UI: ![MLflow Tracing UI](https://github.com/mlflow/mlflow/blob/891fed9a746477f808dd2b82d3abb2382293c564/docs/static/images/llms/tracing/quickstart/openai-tool-calling-trace-detail.png?raw=true) ## Publishing 1. Run `yarn bump-version --version ` from this directory to bump the package versions appropriately 2. `cd` into `core` and run `npm publish`, and repeat for `integrations/openai` ## Adding New Integrations The TypeScript SDK supports pluggable auto-instrumentation packages under [`integrations/`](./integrations). To add a new integration: 1. Create a new workspace package (for example, `integrations/`), modeled after the [OpenAI integration](./integrations/openai). 2. Implement the instrumentation entry points in `src/`, exporting a `register()` helper that configures tracing for the target client library. 3. Add package metadata (`package.json`, `tsconfig.json`, and optional `README.md`) so the integration can be built and published. 4. Add unit and/or integration tests under `tests/` that exercise the new instrumentation. 5. Update the root [`package.json`](./package.json) `build:integrations` and `test:integrations` scripts if your package requires additional build or test commands. Once your integration package is ready, run the local workflow outlined in [Running the SDK after changes](#running-the-sdk-after-changes) and open a pull request that describes the new provider support. ## Contributing We welcome contributions of new features, bug fixes, and documentation improvements. To contribute: 1. Review the project-wide [contribution guidelines](../../CONTRIBUTING.md) and follow the MLflow [Code of Conduct](../../CODE_OF_CONDUCT.rst). 2. Discuss larger proposals in a GitHub issue or the MLflow community channels before investing significant effort. 3. Fork the repository (or use a feature branch) and make your changes with clear, well-structured commits. 4. Ensure your code includes tests and documentation updates where appropriate. 5. Submit a pull request that summarizes the motivation, implementation details, and validation steps. The MLflow team will review and provide feedback. ## Running the SDK after Changes The TypeScript workspace uses npm workspaces. After modifying the core SDK or any integration: ```bash npm install # Install or update workspace dependencies npm run build # Build the core package and all integrations npm run test # Execute the test suites for the core SDK and integrations ``` You can run package-specific scripts from their respective directories (for example, `cd core && npm run test`) when iterating on a particular feature. Remember to rebuild before consuming the SDK from another project so that the latest TypeScript output is emitted to `dist/`. ## Trace Usage MLflow Tracing empowers you throughout the end-to-end lifecycle of your application. Here's how it helps you at each step of the workflow, click on each section to learn more:

🔍 Build & Debug

#### Smooth Debugging Experience MLflow's tracing capabilities provide deep insights into what happens beneath the abstractions of your application, helping you precisely identify where issues occur. [Learn more →](https://mlflow.org/docs/latest/genai/tracing/observe-with-traces/ui)

![Trace Debug](https://raw.githubusercontent.com/mlflow/mlflow/master/docs/static/images/llms/tracing/genai-trace-debug.png)

💬 Human Feedback

#### Track Annotation and User Feedback Attached to Traces Collecting and managing feedback is essential for improving your application. MLflow Tracing allows you to attach user feedback and annotations directly to traces, creating a rich dataset for analysis. This feedback data helps you understand user satisfaction, identify areas for improvement, and build better evaluation datasets based on real user interactions. [Learn more →](https://mlflow.org/docs/latest/genai/assessments/feedback)

![Human Feedback](https://raw.githubusercontent.com/mlflow/mlflow/master/docs/static/images/llms/tracing/genai-human-feedback.png)

📊 Evaluation

#### Systematic Quality Assessment Throughout Your Application Evaluating the performance of your application is crucial, but creating a reliable evaluation process can be challenging. Traces serve as a rich data source, helping you assess quality with precise metrics for all components. When combined with MLflow's evaluation capabilities, you get a seamless experience for assessing and improving your application's performance. [Learn more →](https://mlflow.org/docs/latest/genai/eval-monitor)

![Evaluation](https://raw.githubusercontent.com/mlflow/mlflow/master/docs/static/images/llms/tracing/genai-trace-evaluation.png)

🚀 Production Monitoring

#### Monitor Applications with Your Favorite Observability Stack Machine learning projects don't end with the first launch. Continuous monitoring and incremental improvement are critical to long-term success. Integrated with various observability platforms such as Databricks, Datadog, Grafana, and Prometheus, MLflow Tracing provides a comprehensive solution for monitoring your applications in production. [Learn more →](https://mlflow.org/docs/latest/genai/tracing/prod-tracing)

![Monitoring](https://raw.githubusercontent.com/mlflow/mlflow/master/docs/static/images/llms/tracing/genai-monitoring.png)

📦 Dataset Collection

#### Create High-Quality Evaluation Datasets from Production Traces Traces from production are invaluable for building comprehensive evaluation datasets. By capturing real user interactions and their outcomes, you can create test cases that truly represent your application's usage patterns. This comprehensive data capture enables you to create realistic test scenarios, validate model performance on actual usage patterns, and continuously improve your evaluation datasets. [Learn more →](https://mlflow.org/docs/latest/genai/tracing/search-traces#creating-evaluation-datasets)

![Dataset Collection](https://raw.githubusercontent.com/mlflow/mlflow/master/docs/static/images/llms/tracing/genai-trace-dataset.png)

## Documentation 📘 Official documentation for MLflow Typescript SDK can be found [here](https://mlflow.org/docs/latest/genai/tracing/quickstart). ## License This project is licensed under the [Apache License 2.0](https://github.com/mlflow/mlflow/blob/master/LICENSE.txt).