mlflow--mlflow/examples/llms/RAG/mlflow_docs_scraped.csv

1	source	text
2	getting-started/index.html	Documentation Getting Started with MLflow Getting Started with MLflow For those new to MLflow or seeking a refresher on its core functionalities, the quickstart tutorials here are the perfect starting point. They will guide you step-by-step through fundamental concepts, focusing purely on a task that will maximize your understanding of how to use MLflow to solve a particular task. 5-minute Quickstart - MLflow Tracking In this brief introductory quickstart on MLflow Tracking, you will learn how to leverage MLflow to: Log training statistics (loss, accuracy, etc.) and hyperparameters for a model Log (save) a model for later retrieval Register a model to enable state transitions for deployment Load the model and use it for inference In the process of learning these key concepts, you will be exposed to the MLflow fluent API, the MLflow Tracking UI, and learn how to add metadata associated with a model training event to an MLflow run. If you would like to get started immediately by interactively running the notebook, you can: Download the Notebook Quickstart elements You can read through the quickstart as a guide, or navigate directly to the notebook example to get started. MLflow Tracking Quickstart Guide Learn the basics of MLflow Tracking in a fast-paced guide with a focus on seeing your first model in the MLflow UI MLflow Tracking Quickstart Notebook See an example of using the MLflow fluent API to log, load, and register a model for inference. Great for code-focused learners! Logging your first MLflow Model In this lengthy tutorial, you will walk through the basics of MLflow in a sequential and guided manner. With each subsequent step, you will increase your familiarity with the primary functionality around MLflow Tracking and how to navigate the MLflow UI. If you would like to get started immediately by interactively running the notebook, you can: Download the Notebook Guide sections Interested in navigating directly to the content that you're curious about? Select the section from each tutorial below! Setting up the MLflow Tracking Server Learn how to start an MLflow Tracking Server and the MLflow UI Server locally Using the MLflow Client Connect to the Tracking Server with the MLflow Client and learn how to search for experiments Create your first Experiment Explore the MLflow UI and create your first MLflow experiment with a unique name and identifying tags Search Experiments by tags Learn how to use tags for MLflow experiments and how to leverage them for searching Creating a Dataset for Testing Build a synthetic dataset to use while exploring the features of MLflow Logging your first MLflow run Train a model using the synthetic dataset and log the trained model, metrics, and parameters View the full Notebook See the tutorial notebook in its entirety. If you prefer just reading code, this is the best place to look. 15 minute Quickstart - Autologging in MLflow In this rapid-pace quickstart, you will be exposed to the autologging feature in MLflow to simplify the logging of models, metrics, and parameters. After training and viewing the logged run data, we'll load the logged model to perform inference, showing core features of MLflow Tracking in the most time-efficient manner possible. Introduction to autologging Train a model and use MLflow autologging to automatically record model artifacts, metrics, and parameters View autologged data in the MLflow UI See what autologging will autonomously log for you during model training with only a single line of code Loading a model for inference Load the autologged model in its native format and use it to generate predictions 15 minute Quickstart - Comparing Runs and Deploying your Best Model This quickstart tutorial focuses on the MLflow UI's run comparison feature, provides a brief overview of MLflow Projects, and shows how to register a model. After locally serving the registered model, a brief example of preparing a model for remote deployment via containerizing the model via Docker is covered. Generate runs Run an MLflow Project that will perform hyperparameter tuning to generate a large volume of runs Run comparison Use the MLflow UI Runs Compare functionality to evaluate the hyperparameter tuning run and select the best model Register the best model Learn to register models with the MLflow UI and perform stage transitions from within the UI Start a local ML inference server Use the integrated inference server in MLflow to serve your registered model locally Build a deployable container for your model Learn how to generate a docker container that houses your model for deployment to external services 5 Minute Tracking Server Overview This quickstart tutorial walks through different types of MLflow Tracking Servers and how to use them to log your MLflow experiments. 5 Minute Tracking Server Overview Learn how to log MLflow experiments with different tracking servers Previous Next © MLflow Project, a Series of LF Projects, LLC. All rights reserved.
3	new-features/index.html	Documentation New Features New Features Looking to learn about new significant releases in MLflow? Find out about the details of major features, changes, and deprecations below. MLflow Docs Overhaul The MLflow docs are getting a facelift with added content, tutorials, and guides. Stay tuned for further improvements to the site! released in 2.8.0 New Features for LLM Evaluation The functionality provided for LLM evaluation in MLflow is getting greatly expanded. Check out all of the new features in the guide and the tutorials. Learn more released in 2.8.0 Updated Model Registry UI A new opt-in Model Registry UI has been built that uses Aliases and Tags for managing model development. See more about the new UI workflow in the docs. Learn more released in 2.8.0 Spark Connect support You can now log, save, and load models trained using Spark Connect. Try out Spark 3.5 and the MLflow integration today! released in 2.8.0 AI21 Labs added as an MLflow Gateway provider You can now use the MLflow AI Gateway to connect to LLMs hosted by AI21 Labs. Learn more released in 2.8.0 AWS Bedrock added as an MLflow Gateway provider You can now use the MLflow AI Gateway to connect to LLMs hosted by AWS's Bedrock service. Learn more released in 2.8.0 PaLM 2 added as an MLflow Gateway provider You can now use the MLflow AI Gateway to connect to LLMs hosted by Google's PaLM 2 service. Learn more released in 2.8.0 Hugging Face TGI added as an MLflow Gateway provider You can self-host your own transformers-based models from the Hugging Face Hub and directly connect to the models with the AI Gateway with TGI. Learn more released in 2.8.0 LLM evaluation viewer added to MLflow UI You can view your LLM evaluation results directly from the MLflow UI. Learn more released in 2.7.0 Introducting the Prompt Engineering UI Link your MLflow Tracking Server with your MLflow AI Gateway Server to experiment, evaluate, and construct prompts that can be compared amongst different providers without writing a single line of code. Learn more released in 2.7.0 MosaicML Support in AI Gateway MosaicML has now been added to the supported providers in MLflow AI Gateway. You can now seamlessly interface with managed popular models like MPT-30B and other models in the MPT family. Try it out today with our example. Learn more released in 2.7.0 Cloudflare R2 now supported as an artifact store Cloudflare's R2 storage backend is now supported for use as an artifact store. To learn more about R2, read the Cloudflare docs to get more information and to explore what is possible. released in 2.7.0 Params support for PyFunc Models PyFunc models now support passing parameters at inference time. With this new feature, you can define the allowable keys, with default values, for any parameters that you would like consumers of your model to be able to override. This is particularly useful for LLMs, where you might want to let users adjust commonly modified parameters for a model, such as token counts and temperature. Learn more released in 2.6.0 MLflow Serving support added to MLflow AI Gateway The MLflow AI Gateway now supports defining an MLflow serving endpoint as provider. With this new feature, you can serve any OSS transformers model that conforms to the completions or embeddings route type definitions. Try it out today with our end-to-end example. Learn more released in 2.6.0 Introducing the MLflow AI Gateway We're excited to announce the newest top-level component in the MLflow ecosystem: The AI Gateway. With this new feature, you can create a single access point to many of the most popular LLM SaaS services available now, simplifying interfaces, managing credentials, and providing a unified standard set of APIs to reduce the complexity of building products and services around LLMs. Learn more released in 2.5.0 MLflow Evaluate now supports LLMs You can now use MLflow evaluate to compare results from your favorite LLMs on a fixed prompt. With support for many of the standard evaluation metrics for LLMs built in directly to the API, the featured LLM modeling tasks of text summarization, text classification, question answering, and text generation allows you to view the results of submitted text to multiple models in a single UI element. Learn more released in 2.4.0 Chart View added to the MLflow UI You can now visualize parameters and metrics across multiple runs as a chart on the runs table. Learn more released in 2.2.0 Previous Next © MLflow Project, a Series of LF Projects, LLC. All rights reserved.
4	llms/index.html	Documentation LLMs LLMs LLMs, or Large Language Models, have rapidly become a cornerstone in the machine learning domain, offering immense capabilities ranging from natural language understanding to code generation and more. However, harnessing the full potential of LLMs often involves intricate processes, from interfacing with multiple providers to fine-tuning specific models to achieve desired outcomes. Such complexities can easily become a bottleneck for developers and data scientists aiming to integrate LLM capabilities into their applications. MLflow's Support for LLMs aims to alleviate these challenges by introducing a suite of features and tools designed with the end-user in mind: MLflow AI Gateway Serving as a unified interface, the MLflow AI Gateway simplifies interactions with multiple LLM providers, such as OpenAI, MosaicML, Cohere, Anthropic, PaLM 2, AWS Bedrock, and AI21 Labs. In addition to supporting the most popular SaaS LLM providers, the AI Gateway provides an integration to MLflow model serving, allowing you to serve your own LLM or a fine-tuned foundation model within your own serving infrastructure. Note The MLflow AI Gateway is in active development and has been marked as Experimental. APIs may change as this new feature is refined and its functionality is expanded based on feedback. Benefits of the MLflow AI Gateway Unified Endpoint: No more juggling between multiple provider APIs. Simplified Integrations: One-time setup, no repeated complex integrations. Secure Credential Management: Centralized storage prevents scattered API keys. No hardcoding or user-handled keys. Consistent API Experience: Uniform API across all providers. Easy-to-use REST endpoints and Client API. Seamless Provider Swapping: Swap providers without touching your code. Zero downtime provider, model, or route swapping. Explore the Native Provider integrations The MLflow AI Gateway supports a large range of foundational models from popular SaaS model vendors, as well as providing a means of self-hosting your own open source model via an integration with MLflow model serving. To learn more about how to get started using the MLflow AI Gateway to simplify the configuration and management of your LLM serving needs, select the provider that you're interested in below: Getting Started Examples for each Provider If you're interested in learning about how to set up the MLflow AI Gateway for a specific provider, follow the links below for our up-to-date documentation on GitHub. Each link will take you to a README file that will explain how to set up a route for the provider. In the same directory as the README, you will find a runnable example of how to query the routes that the example creates, providing you with a quick reference for getting started with your favorite provider! OpenAI quickstart MosaicML quickstart Anthropic quickstart Cohere quickstart MLflow quickstart AWS Bedrock quickstart AI21 Labs quickstart PaLM 2 quickstart Azure OpenAI quickstart Hugging Face Text Generation Interface (TGI) quickstart Note The MLflow and Hugging Face TGI providers are for self-hosted LLM serving of either foundation open-source LLM models, fine-tuned open-source LLM models, or your own custom LLM. The example documentation for these providers will show you how to get started with these, using free-to-use open-source models from the Hugging Face Hub. LLM Evaluation Navigating the vast landscape of Large Language Models (LLMs) can be daunting. Determining the right model, prompt, or service that aligns with a project's needs is no small feat. Traditional machine learning evaluation metrics often fall short when it comes to assessing the nuanced performance of generative models. Enter MLflow LLM Evaluation. This feature is designed to simplify the evaluation process, offering a streamlined approach to compare foundational models, providers, and prompts. Benefits of MLflow's LLM Evaluation Simplified Evaluation: Navigate the LLM space with ease, ensuring the best fit for your project with standard metrics that can be used to compare generated text. Use-Case Specific Metrics: Leverage MLflow's mlflow.evaluate() API for a high-level, frictionless evaluation experience. Customizable Metrics: Beyond the provided metrics, MLflow supports a plugin-style for custom scoring, enhancing the evaluation's flexibility. Comparative Analysis: Effortlessly compare foundational models, providers, and prompts to make informed decisions. Deep Insights: Dive into the intricacies of generative models with a comprehensive suite of LLM-relevant metrics. MLflow's LLM Evaluation is designed to bridge the gap between traditional machine learning evaluation and the unique challenges posed by LLMs. Prompt Engineering UI Effective utilization of LLMs often hinges on crafting the right prompts. The development of a high-quality prompt is an iterative process of trial and error, where subsequent experimentation is not guaranteed to result in cumulative quality improvements. With the volume and speed of iteration through prompt experimentation, it can quickly become very overwhelming to remember or keep a history of the state of different prompts that were tried. Serving as a powerful tool for prompt engineering, the MLflow Prompt Engineering UI revolutionizes the way developers interact with and refine LLM prompts. Benefits of the MLflow Prompt Engineering UI Iterative Development: Streamlined process for trial and error without the overwhelming complexity. UI-Based Prototyping: Prototype, iterate, and refine prompts without diving deep into code. Accessible Engineering: Makes prompt engineering more user-friendly, speeding up experimentation. Optimized Configurations: Quickly hone in on the best model configurations for tasks like question answering or document summarization. Transparent Tracking: Every model iteration and configuration is meticulously tracked. Ensures reproducibility and transparency in your development process. Note The MLflow Prompt Engineering UI is in active development and has been marked as Experimental. Features and interfaces may evolve as feedback is gathered and the tool is refined. Native MLflow Flavors for LLMs Harnessing the power of LLMs becomes effortless with flavors designed specifically for working with LLM libraries and frameworks. Benefits of MLflow's Native Flavors for LLMs Support for Popular Packages: Native integration with packages like transformers, sentence-transformers, open-ai , and langchain. Standardized interfaces for tasks like saving, logging, and managing inference configurations. PyFunc Compatibility: Load models as PyFuncs for broad compatibility across serving infrastructures. Strengthens the MLOps process for LLMs, ensuring smooth deployments. Cohesive Ecosystem: All essential tools and functionalities consolidated under MLflow. Focus on deriving value from LLMs without getting bogged down by interfacing and optimization intricacies. Explore the Native LLM Flavors Select the integration below to read the documentation on how to leverage MLflow's native integration with these popular libraries: Native Integration Examples If you'd like to directly explore code examples for how to get started with using our official library integrations, you can navigate directly to our up-to-date examples on GitHub below: transformers Simple Text Generation Example A Conversational Model Example Component Logging with Transformers Audio Transcription with Whisper Fine-tuning a Text Classification Model sentence-transformers Text Encoding Example langchain Logging and using a Chain Logging and using an Agent Logging and using a Retriever Chain 1 Logging and using a Retrieval QA Chain 1 1 Demonstrates the use of Retrieval Augmented Generation (RAG) using a Vector Store openai Using a Completions endpoint Using a Chat endpoint Performing Embeddings Generation Using OpenAI on a Spark DataFrame for Batch Processing Using Azure OpenAI LLM Tracking in MLflow Empowering developers with advanced tracking capabilities, the MLflow LLM Tracking System stands out as the premier solution for managing and analyzing interactions with Large Language Models (LLMs). Benefits of the MLflow LLM Tracking System Robust Interaction Management: Comprehensive tracking of every LLM interaction for maximum insight. Tailor-Made for LLMs: Unique features specifically designed for LLMs. From logging prompts to tracking dynamic data, MLflow has it covered. Deep Model Insight: Introduces 'predictions' as a core entity, alongside the existing artifacts, parameters, and metrics. Gain unparalleled understanding of text-generating model behavior and performance. Clarity and Repeatability: Ensures consistent and transparent tracking across all LLM interactions. Facilitates informed decision-making and optimization in LLM deployment and utilization. Tutorials and Use Case Guides for LLMs in MLflow Interested in learning how to leverage MLflow for your LLM projects? Look in the tutorials and guides below to learn more about interesting use cases that could help to make your journey into leveraging LLMs a bit easier! Evaluating LLMs Learn how to evaluate LLMs with MLflow. Using Custom PyFunc with LLMs Explore the nuances of packaging, customizing, and deploying advanced LLMs in MLflow using custom PyFuncs. Question Generation for RAG Learn how to leverage LLMs to generate a question dataset for use in Retrieval Augmented Generation applications. Previous Next © MLflow Project, a Series of LF Projects, LLC. All rights reserved.
5	deep-learning/index.html	Documentation Deep Learning Deep Learning The realm of deep learning has witnessed an unprecedented surge, revolutionizing numerous sectors with its ability to process vast amounts of data and capture intricate patterns. From the real-time object detection in autonomous vehicles to the generation of art through Generative Adversarial Networks, and from natural language processing applications in chatbots to predictive analytics in e-commerce, deep learning models are at the forefront of today's AI-driven innovations. MLflow acknowledges the profound impact and complexity of deep learning. With a keen focus on the unique challenges posed by deep learning workflows, such as iterative model training and hyperparameter tuning, MLflow introduces a robust suite of tools specifically designed for these advanced models. MLflow helps to facilitate seamless model development, ensuring reproducibility, and provides enhanced monitoring capabilities with the concept of 'steps' for recording metrics at various training iterations, with integrated UI features that enable you to easily visualize the iterative improvements of key metrics during training epochs. Key Benefits: Iterative Model Training: With the concept of 'steps', MLflow allows users to log metrics at various training iterations, offering a granular view of the model's progress. Reproducibility: Ensure that every model training run can be replicated with the exact same conditions. Scalability: Handle projects ranging from small-scale models to enterprise-level deployments with ease. Traceability: Keep track of every detail, from hyperparameters to the final model output. Deep Autologging Integrations One of the standout features of MLflow's deep learning support is its deep autologging integrations. These integrations automatically capture and log intricate details during the training of deep learning models, ensuring that every nuance, from model parameters to evaluation metrics, is meticulously recorded. This is especially prominent in frameworks like TensorFlow, PyTorch Lightning, base PyTorch, and Keras, making the iterative training process more insightful and manageable. Native Library Support Deep learning in MLflow is enriched by its native support for a number of the most popular libraries. The native integration with each of these libraries within MLflow help to streamline and simplify the training process, as well as saving, logging, loading, and representing models as generic Python functions for inference use anywhere. Opting for these native integrations brings forth a myriad of advantages: Auto-logging Capabilities: Automatically capture details without manual intervention. Custom Serialization: Streamline the model saving and loading process with custom methods tailored for each library. Unified Interface: Regardless of the underlying library, interact with a consistent MLflow interface. The officially supported integrations for deep learning libraries in MLflow encompass: Harness the power of these integrations and elevate your deep learning projects with MLflow's comprehensive support. MLflow Tracking for Deep Learning Tracking remains a cornerstone of the MLflow ecosystem, especially vital for the iterative nature of deep learning: Experiments and Runs: Organize your deep learning projects into experiments, with each experiment containing multiple runs. Each run captures essential data like metrics at various training steps, hyperparameters, and the code state. Artifacts: Store vital outputs such as deep learning models, visualizations, or even tensorboard logs. This artifact repository ensures traceability and easy access. Metrics at Steps: With deep learning's iterative nature, MLflow allows logging metrics at various training steps, offering a granular view of the model's progress. Dependencies and Environment: Capture the computational environment, including deep learning frameworks' versions, ensuring reproducibility. Input Examples and Model Signatures: Define the expected format of the model's inputs, crucial for complex data like images or sequences. UI Integration: The enhanced UI provides a visual overview of deep learning runs, facilitating comparison and insights into training progress. Search Functionality: Efficiently navigate through your deep learning experiments using robust search capabilities. APIs: Interact with the tracking system programmatically, integrating deep learning workflows seamlessly. Model Registry A centralized repository for your deep learning models: Versioning: Handle multiple iterations and versions of deep learning models, facilitating comparison or reversion. Annotations: Attach notes, training datasets, or other relevant metadata to models. Lifecycle Stages: Clearly define the stage of each model version, ensuring clarity in deployment and further fine-tuning. Deployment for Deep Learning Models Transition deep learning models from training to real-world applications: Consistency: Ensure models, especially those with GPU dependencies, behave consistently across different deployment environments. Docker and GPU Support: Deploy in containerized environments, ensuring all dependencies, including GPU support, are encapsulated. Scalability: From deploying a single model to serving multiple distributed deep learning models, MLflow scales as per your requirements. Previous Next © MLflow Project, a Series of LF Projects, LLC. All rights reserved.
6	deployment/index.html	Documentation Deployment Deployment In the modern age of machine learning, deploying models effectively and consistently plays a pivotal role. The capability to serve predictions at scale, manage dependencies, and ensure reproducibility is paramount for businesses to derive actionable insights from their ML models. Whether it's for real-time predictions, batch processing, or interactive analyses, a robust model serving framework is essential. MLflow offers a comprehensive suite tailored for seamless model deployment. With its focus on ease of use, consistency, and adaptability, MLflow simplifies the model serving process, ensuring models are not just artifacts but actionable tools for decision-making. The Power of MLflow in Model Serving Dependency and Environment Management: MLflow ensures that the deployment environment mirrors the training environment, capturing all dependencies. This guarantees that models run consistently, regardless of where they're deployed. Packaging Models and Code: With MLflow, not just the model, but any supplementary code and configurations are packaged along with the deployment container. This ensures that the model can be executed seamlessly without any missing components. Deployment Avenues MLflow offers multiple ways to deploy your models based on your needs: Local Flask Server: Quickly deploy your model in a containerized local environment. This server runs a model container with the dependencies defined during model saving. Local Flask Server with MLServer: Easily deploy a containerized model along with MLServer and KServe. This powerful alternative to a base Flask server leverages all of the benefits of Seldon's framework to enhance your inference capabilities. Remote Container Serving: Once a model container has been defined and built, it can be deployed to a remote serving environment. This is especially useful for cloud deployments where providers offer elastic container execution capabilities. AzureML AWS Sagemaker Kubernetes: For those invested in the Kubernetes ecosystem, MLflow supports model deployment to Kubernetes clusters, ensuring scalability and resilience. Databricks Model Serving: Directly deploy models in a Databricks environment, taking advantage of Databricks' performance optimizations and integrations. Tutorials and Guides Deploying a Model to Kubernetes with MLflow Explore an end-to-end guide on using MLflow to train a linear regression model, package it, and deploy it to a Kubernetes cluster. Understand how MLflow simplifies the entire process, from training to serving. Conclusion Model serving is an intricate process, and MLflow is designed to make it as intuitive and reliable as possible. With its myriad deployment options and focus on consistency, MLflow ensures that models are ready for action, be it in a local environment, the cloud, or on a large-scale Kubernetes cluster. Dive into the provided tutorials, explore the functionalities, and streamline your model deployment journey with MLflow. Previous Next © MLflow Project, a Series of LF Projects, LLC. All rights reserved.
7	projects.html	Documentation MLflow Projects MLflow Projects An MLflow Project is a format for packaging data science code in a reusable and reproducible way, based primarily on conventions. In addition, the Projects component includes an API and command-line tools for running projects, making it possible to chain together projects into workflows. Table of Contents Overview Specifying Projects Running Projects Iterating Quickly Building Multistep Workflows Overview At the core, MLflow Projects are just a convention for organizing and describing your code to let other data scientists (or automated tools) run it. Each project is simply a directory of files, or a Git repository, containing your code. MLflow can run some projects based on a convention for placing files in this directory (for example, a conda.yaml file is treated as a Conda environment), but you can describe your project in more detail by adding a MLproject file, which is a YAML formatted text file. Each project can specify several properties: NameA human-readable name for the project. Entry PointsCommands that can be run within the project, and information about their parameters. Most projects contain at least one entry point that you want other users to call. Some projects can also contain more than one entry point: for example, you might have a single Git repository containing multiple featurization algorithms. You can also call any .py or .sh file in the project as an entry point. If you list your entry points in a MLproject file, however, you can also specify parameters for them, including data types and default values. EnvironmentThe software environment that should be used to execute project entry points. This includes all library dependencies required by the project code. See Project Environments for more information about the software environments supported by MLflow Projects, including Conda environments, Virtualenv environments, and Docker containers. You can run any project from a Git URI or from a local directory using the mlflow run command-line tool, or the mlflow.projects.run() Python API. These APIs also allow submitting the project for remote execution on Databricks and Kubernetes. Important By default, MLflow uses a new, temporary working directory for Git projects. This means that you should generally pass any file arguments to MLflow project using absolute, not relative, paths. If your project declares its parameters, MLflow automatically makes paths absolute for parameters of type path. Specifying Projects By default, any Git repository or local directory can be treated as an MLflow project; you can invoke any bash or Python script contained in the directory as a project entry point. The Project Directories section describes how MLflow interprets directories as projects. To provide additional control over a project's attributes, you can also include an MLproject file in your project's repository or directory. Finally, MLflow projects allow you to specify the software environment that is used to execute project entry points. Project Environments MLflow currently supports the following project environments: Virtualenv environment, conda environment, Docker container environment, and system environment. Virtualenv environment (preferred)Virtualenv environments support Python packages available on PyPI. When an MLflow Project specifies a Virtualenv environment, MLflow will download the specified version of Python by using pyenv and create an isolated environment that contains the project dependencies using virtualenv, activating it as the execution environment prior to running the project code. You can specify a Virtualenv environment for your MLflow Project by including a python_env entry in your MLproject file. For details, see the Project Directories and Specifying an Environment sections. Docker container environmentDocker containers allow you to capture non-Python dependencies such as Java libraries. When you run an MLflow project that specifies a Docker image, MLflow runs your image as is with the parameters specified in your MLproject file. In this case you'll need to pre build your images with both environment and code to run it. To run the project with a new image that's based on your image and contains the project's contents in the /mlflow/projects/code directory, use the --build-image flag when running mlflow run. Environment variables, such as MLFLOW_TRACKING_URI, are propagated inside the Docker container during project execution. Additionally, runs and experiments created by the project are saved to the tracking server specified by your tracking URI. When running against a local tracking URI, MLflow mounts the host system's tracking directory (e.g., a local mlruns directory) inside the container so that metrics, parameters, and artifacts logged during project execution are accessible afterwards. See Dockerized Model Training with MLflow for an example of an MLflow project with a Docker environment. To specify a Docker container environment, you must add an MLproject file to your project. For information about specifying a Docker container environment in an MLproject file, see Specifying an Environment. Conda environmentConda environments support both Python packages and native libraries (e.g, CuDNN or Intel MKL). When an MLflow Project specifies a Conda environment, it is activated before project code is run. Warning By using conda, you're responsible for adhering to Anaconda's terms of service. By default, MLflow uses the system path to find and run the conda binary. You can use a different Conda installation by setting the MLFLOW_CONDA_HOME environment variable; in this case, MLflow attempts to run the binary at $MLFLOW_CONDA_HOME/bin/conda. You can specify a Conda environment for your MLflow project by including a conda.yaml file in the root of the project directory or by including a conda_env entry in your MLproject file. For details, see the Project Directories and Specifying an Environment sections. The mlflow run command supports running a conda environment project as a virtualenv environment project. To do this, run mlflow run with --env-manager virtualenv: mlflow run /path/to/conda/project --env-manager virtualenv Warning When a conda environment project is executed as a virtualenv environment project, conda dependencies will be ignored and only pip dependencies will be installed. System environmentYou can also run MLflow Projects directly in your current system environment. All of the project's dependencies must be installed on your system prior to project execution. The system environment is supplied at runtime. It is not part of the MLflow Project's directory contents or MLproject file. For information about using the system environment when running a project, see the Environment parameter description in the Running Projects section. Project Directories When running an MLflow Project directory or repository that does not contain an MLproject file, MLflow uses the following conventions to determine the project's attributes: The project's name is the name of the directory. The Conda environment is specified in conda.yaml, if present. If no conda.yaml file is present, MLflow uses a Conda environment containing only Python (specifically, the latest Python available to Conda) when running the project. Any .py and .sh file in the project can be an entry point. MLflow uses Python to execute entry points with the .py extension, and it uses bash to execute entry points with the .sh extension. For more information about specifying project entrypoints at runtime, see Running Projects. By default, entry points do not have any parameters when an MLproject file is not included. Parameters can be supplied at runtime via the mlflow run CLI or the mlflow.projects.run() Python API. Runtime parameters are passed to the entry point on the command line using --key value syntax. For more information about running projects and with runtime parameters, see Running Projects. MLproject File You can get more control over an MLflow Project by adding an MLproject file, which is a text file in YAML syntax, to the project's root directory. The following is an example of an MLproject file: name: My Project python_env: python_env.yaml # or # conda_env: my_env.yaml # or # docker_env: # image: mlflow-docker-example entry_points: main: parameters: data_file: path regularization: {type: float, default: 0.1} command: "python train.py -r {regularization} {data_file}" validate: parameters: data_file: path command: "python validate.py {data_file}" The file can specify a name and a Conda or Docker environment, as well as more detailed information about each entry point. Specifically, each entry point defines a command to run and parameters to pass to the command (including data types). Specifying an Environment This section describes how to specify Conda and Docker container environments in an MLproject file. MLproject files cannot specify both a Conda environment and a Docker environment. Virtualenv environmentInclude a top-level python_env entry in the MLproject file. The value of this entry must be a relative path to a python_env YAML file within the MLflow project's directory. The following is an example MLProject file with a python_env definition: python_env: files/config/python_env.yaml python_env refers to an environment file located at <MLFLOW_PROJECT_DIRECTORY>/files/config/python_env.yaml, where <MLFLOW_PROJECT_DIRECTORY> is the path to the MLflow project's root directory. The following is an example of a python_env.yaml file: # Python version required to run the project. python: "3.8.15" # Dependencies required to build packages. This field is optional. build_dependencies: - pip - setuptools - wheel==0.37.1 # Dependencies required to run the project. dependencies: - mlflow==2.3 - scikit-learn==1.0.2 Conda environmentInclude a top-level conda_env entry in the MLproject file. The value of this entry must be a relative path to a Conda environment YAML file within the MLflow project's directory. In the following example: conda_env: files/config/conda_environment.yaml conda_env refers to an environment file located at <MLFLOW_PROJECT_DIRECTORY>/files/config/conda_environment.yaml, where <MLFLOW_PROJECT_DIRECTORY> is the path to the MLflow project's root directory. Docker container environmentInclude a top-level docker_env entry in the MLproject file. The value of this entry must be the name of a Docker image that is accessible on the system executing the project; this image name may include a registry path and tags. Here are a couple of examples. Example 1: Image without a registry path docker_env: image: mlflow-docker-example-environment In this example, docker_env refers to the Docker image with name mlflow-docker-example-environment and default tag latest. Because no registry path is specified, Docker searches for this image on the system that runs the MLflow project. If the image is not found, Docker attempts to pull it from DockerHub. Example 2: Mounting volumes and specifying environment variables You can also specify local volumes to mount in the docker image (as you normally would with Docker's -v option), and additional environment variables (as per Docker's -e option). Environment variables can either be copied from the host system's environment variables, or specified as new variables for the Docker environment. The environment field should be a list. Elements in this list can either be lists of two strings (for defining a new variable) or single strings (for copying variables from the host system). For example: docker_env: image: mlflow-docker-example-environment volumes: ["/local/path:/container/mount/path"] environment: [["NEW_ENV_VAR", "new_var_value"], "VAR_TO_COPY_FROM_HOST_ENVIRONMENT"] In this example our docker container will have one additional local volume mounted, and two additional environment variables: one newly-defined, and one copied from the host system. Example 3: Image in a remote registry docker_env: image: 012345678910.dkr.ecr.us-west-2.amazonaws.com/mlflow-docker-example-environment:7.0 In this example, docker_env refers to the Docker image with name mlflow-docker-example-environment and tag 7.0 in the Docker registry with path 012345678910.dkr.ecr.us-west-2.amazonaws.com, which corresponds to an Amazon ECR registry. When the MLflow project is run, Docker attempts to pull the image from the specified registry. The system executing the MLflow project must have credentials to pull this image from the specified registry. Example 4: Build a new image docker_env: image: python:3.8 mlflow run ... --build-image To build a new image that's based on the specified image and files contained in the project directory, use the --build-image argument. In the above example, the image python:3.8 is pulled from Docker Hub if it's not present locally, and a new image is built based on it. The project is executed in a container created from this image. Command Syntax When specifying an entry point in an MLproject file, the command can be any string in Python format string syntax. All of the parameters declared in the entry point's parameters field are passed into this string for substitution. If you call the project with additional parameters not listed in the parameters field, MLflow passes them using --key value syntax, so you can use the MLproject file to declare types and defaults for just a subset of your parameters. Before substituting parameters in the command, MLflow escapes them using the Python shlex.quote function, so you don't need to worry about adding quotes inside your command field. Specifying Parameters MLflow allows specifying a data type and default value for each parameter. You can specify just the data type by writing: parameter_name: data_type in your YAML file, or add a default value as well using one of the following syntaxes (which are equivalent in YAML): parameter_name: {type: data_type, default: value} # Short syntax parameter_name: # Long syntax type: data_type default: value MLflow supports four parameter types, some of which it treats specially (for example, downloading data to local files). Any undeclared parameters are treated as string. The parameter types are: stringA text string. floatA real number. MLflow validates that the parameter is a number. pathA path on the local file system. MLflow converts any relative path parameters to absolute paths. MLflow also downloads any paths passed as distributed storage URIs (s3://, dbfs://, gs://, etc.) to local files. Use this type for programs that can only read local files. uriA URI for data either in a local or distributed storage system. MLflow converts relative paths to absolute paths, as in the path type. Use this type for programs that know how to read from distributed storage (e.g., programs that use Spark). Running Projects MLflow provides two ways to run projects: the mlflow run command-line tool, or the mlflow.projects.run() Python API. Both tools take the following parameters: Project URIA directory on the local file system or a Git repository path, specified as a URI of the form https://<repo> (to use HTTPS) or user@host:path (to use Git over SSH). To run against an MLproject file located in a subdirectory of the project, add a '#' to the end of the URI argument, followed by the relative path from the project's root directory to the subdirectory containing the desired project. Project VersionFor Git-based projects, the commit hash or branch name in the Git repository. Entry PointThe name of the entry point, which defaults to main. You can use any entry point named in the MLproject file, or any .py or .sh file in the project, given as a path from the project root (for example, src/test.py). ParametersKey-value parameters. Any parameters with declared types are validated and transformed if needed. Deployment Mode Both the command-line and API let you launch projects remotely in a Databricks environment. This includes setting cluster parameters such as a VM type. Of course, you can also run projects on any other computing infrastructure of your choice using the local version of the mlflow run command (for example, submit a script that does mlflow run to a standard job queueing system). You can also launch projects remotely on Kubernetes clusters using the mlflow run CLI (see Run an MLflow Project on Kubernetes). EnvironmentBy default, MLflow Projects are run in the environment specified by the project directory or the MLproject file (see Specifying Project Environments). You can ignore a project's specified environment and run the project in the current system environment by supplying the --env-manager=local flag, but this can lead to unexpected results if there are dependency mismatches between the project environment and the current system environment. For example, the tutorial creates and publishes an MLflow Project that trains a linear model. The project is also published on GitHub at https://github.com/mlflow/mlflow-example. To run this project: mlflow run git@github.com:mlflow/mlflow-example.git -P alpha=0.5 There are also additional options for disabling the creation of a Conda environment, which can be useful if you quickly want to test a project in your existing shell environment. Run an MLflow Project on Databricks You can run MLflow Projects remotely on Databricks. To use this feature, you must have an enterprise Databricks account (Community Edition is not supported) and you must have set up the Databricks CLI. Find detailed instructions in the Databricks docs (Azure Databricks, Databricks on AWS). Run an MLflow Project on Kubernetes You can run MLflow Projects with Docker environments on Kubernetes. The following sections provide an overview of the feature, including a simple Project execution guide with examples. To see this feature in action, you can also refer to the Docker example, which includes the required Kubernetes backend configuration (kubernetes_backend.json) and Kubernetes Job Spec (kubernetes_job_template.yaml) files. How it works When you run an MLflow Project on Kubernetes, MLflow constructs a new Docker image containing the Project's contents; this image inherits from the Project's Docker environment. MLflow then pushes the new Project image to your specified Docker registry and starts a Kubernetes Job on your specified Kubernetes cluster. This Kubernetes Job downloads the Project image and starts a corresponding Docker container. Finally, the container invokes your Project's entry point, logging parameters, tags, metrics, and artifacts to your MLflow tracking server. Execution guide You can run your MLflow Project on Kubernetes by following these steps: Add a Docker environment to your MLflow Project, if one does not already exist. For reference, see Specifying an Environment. Create a backend configuration JSON file with the following entries: kube-context The Kubernetes context where MLflow will run the job. If not provided, MLflow will use the current context. If no context is available, MLflow will assume it is running in a Kubernetes cluster and it will use the Kubernetes service account running the current pod ('in-cluster' configuration). repository-uri The URI of the docker repository where the Project execution Docker image will be uploaded (pushed). Your Kubernetes cluster must have access to this repository in order to run your MLflow Project. kube-job-template-path The path to a YAML configuration file for your Kubernetes Job - a Kubernetes Job Spec. MLflow reads the Job Spec and replaces certain fields to facilitate job execution and monitoring; MLflow does not modify the original template file. For more information about writing Kubernetes Job Spec templates for use with MLflow, see the Job Templates section. Example Kubernetes backend configuration { "kube-context": "docker-for-desktop", "repository-uri": "username/mlflow-kubernetes-example", "kube-job-template-path": "/Users/username/path/to/kubernetes_job_template.yaml" } If necessary, obtain credentials to access your Project's Docker and Kubernetes resources, including: The Docker environment image specified in the MLproject file. The Docker repository referenced by repository-uri in your backend configuration file. The Kubernetes context referenced by kube-context in your backend configuration file. MLflow expects these resources to be accessible via the docker and kubectl CLIs before running the Project. Run the Project using the MLflow Projects CLI or Python API, specifying your Project URI and the path to your backend configuration file. For example: mlflow run <project_uri> --backend kubernetes --backend-config examples/docker/kubernetes_config.json where <project_uri> is a Git repository URI or a folder. Job Templates MLflow executes Projects on Kubernetes by creating Kubernetes Job resources. MLflow creates a Kubernetes Job for an MLflow Project by reading a user-specified Job Spec. When MLflow reads a Job Spec, it formats the following fields: metadata.name Replaced with a string containing the name of the MLflow Project and the time of Project execution spec.template.spec.container[0].name Replaced with the name of the MLflow Project spec.template.spec.container[0].image Replaced with the URI of the Docker image created during Project execution. This URI includes the Docker image's digest hash. spec.template.spec.container[0].command Replaced with the Project entry point command specified when executing the MLflow Project. The following example shows a simple Kubernetes Job Spec that is compatible with MLflow Project execution. Replaced fields are indicated using bracketed text. Example Kubernetes Job Spec apiVersion: batch/v1 kind: Job metadata: name: "{replaced with MLflow Project name}" namespace: mlflow spec: ttlSecondsAfterFinished: 100 backoffLimit: 0 template: spec: containers: - name: "{replaced with MLflow Project name}" image: "{replaced with URI of Docker image created during Project execution}" command: ["{replaced with MLflow Project entry point command}"] env: ["{appended with MLFLOW_TRACKING_URI, MLFLOW_RUN_ID and MLFLOW_EXPERIMENT_ID}"] resources: limits: memory: 512Mi requests: memory: 256Mi restartPolicy: Never The container.name, container.image, and container.command fields are only replaced for the first container defined in the Job Spec. Further, the MLFLOW_TRACKING_URI, MLFLOW_RUN_ID and MLFLOW_EXPERIMENT_ID are appended to container.env. Use KUBE_MLFLOW_TRACKING_URI to pass a different tracking URI to the job container from the standard MLFLOW_TRACKING_URI. All subsequent container definitions are applied without modification. Iterating Quickly If you want to rapidly develop a project, we recommend creating an MLproject file with your main program specified as the main entry point, and running it with mlflow run .. To avoid having to write parameters repeatedly, you can add default parameters in your MLproject file. Building Multistep Workflows The mlflow.projects.run() API, combined with mlflow.client, makes it possible to build multi-step workflows with separate projects (or entry points in the same project) as the individual steps. Each call to mlflow.projects.run() returns a run object, that you can use with mlflow.client to determine when the run has ended and get its output artifacts. These artifacts can then be passed into another step that takes path or uri parameters. You can coordinate all of the workflow in a single Python program that looks at the results of each step and decides what to submit next using custom code. Some example use cases for multi-step workflows include: Modularizing Your Data Science CodeDifferent users can publish reusable steps for data featurization, training, validation, and so on, that other users or team can run in their workflows. Because MLflow supports Git versioning, another team can lock their workflow to a specific version of a project, or upgrade to a new one on their own schedule. Hyperparameter TuningUsing mlflow.projects.run() you can launch multiple runs in parallel either on the local machine or on a cloud platform like Databricks. Your driver program can then inspect the metrics from each run in real time to cancel runs, launch new ones, or select the best performing run on a target metric. Cross-validationSometimes you want to run the same training code on different random splits of training and validation data. With MLflow Projects, you can package the project in a way that allows this, for example, by taking a random seed for the train/validation split as a parameter, or by calling another project first that can split the input data. For an example of how to construct such a multistep workflow, see the MLflow Multistep Workflow Example project. Previous Next © MLflow Project, a Series of LF Projects, LLC. All rights reserved.
8	auth/index.html	Documentation MLflow Authentication MLflow Authentication Note This feature is still experimental and may change in a future release without warning. MLflow supports basic HTTP authentication to enable access control over experiments and registered models. Once enabled, any visitor will be required to login before they can view any resource from the Tracking Server. Table of Contents Overview How It Works Permissions Permissions Database Admin Users Managing Permissions Authenticating to MLflow Using MLflow UI Using Environment Variables Using Credentials File Using REST API Creating a New User Using MLflow UI Using REST API Using MLflow AuthServiceClient Configuration Custom Authentication MLflow Authentication provides Python and REST API for managing users and permissions. MLflow Authentication Python API MLflow Authentication REST API Overview To enable MLflow authentication, launch the MLflow UI with the following command: mlflow server --app-name basic-auth Server admin can choose to disable this feature anytime by restarting the server without the app-name flag. Any users and permissions created will be persisted on a SQL database and will be back in service once the feature is re-enabled. Due to the nature of HTTP authentication, it is only supported on a remote Tracking Server, where users send requests to the server via REST APIs. How It Works Permissions The available permissions are: Permission Can read Can update Can delete Can manage READ Yes No No No EDIT Yes Yes No No MANAGE Yes Yes Yes Yes NO_PERMISSIONS No No No No The default permission for all users is READ. It can be changed in the configuration file. Permissions can be granted on individual resources for each user. Supported resources include Experiment and Registered Model. To access an API endpoint, an user must have the required permission. Otherwise, a 403 Forbidden response will be returned. Required Permissions for accessing experiments: API Endpoint Method Required permission Create Experiment 2.0/mlflow/experiments/create POST None Get Experiment 2.0/mlflow/experiments/get GET can_read Get Experiment By Name 2.0/mlflow/experiments/get-by-name GET can_read Delete Experiment 2.0/mlflow/experiments/delete POST can_delete Restore Experiment 2.0/mlflow/experiments/restore POST can_delete Update Experiment 2.0/mlflow/experiments/update POST can_update Search Experiments 2.0/mlflow/experiments/search POST None Search Experiments 2.0/mlflow/experiments/search GET None Set Experiment Tag 2.0/mlflow/experiments/set-experiment-tag POST can_update Create Run 2.0/mlflow/runs/create POST can_update Get Run 2.0/mlflow/runs/get GET can_read Update Run 2.0/mlflow/runs/update POST can_update Delete Run 2.0/mlflow/runs/delete POST can_delete Restore Run 2.0/mlflow/runs/restore POST can_delete Search Runs 2.0/mlflow/runs/search POST None Set Tag 2.0/mlflow/runs/set-tag POST can_update Delete Tag 2.0/mlflow/runs/delete-tag POST can_update Log Metric 2.0/mlflow/runs/log-metric POST can_update Log Param 2.0/mlflow/runs/log-parameter POST can_update Log Batch 2.0/mlflow/runs/log-batch POST can_update Log Model 2.0/mlflow/runs/log-model POST can_update List Artifacts 2.0/mlflow/artifacts/list GET can_read Get Metric History 2.0/mlflow/metrics/get-history GET can_read Required Permissions for accessing registered models: API Endpoint Method Required permission Create Registered Model 2.0/mlflow/registered-models/create POST None Rename Registered Model 2.0/mlflow/registered-models/rename POST can_update Update Registered Model 2.0/mlflow/registered-models/update PATCH can_update Delete Registered Model 2.0/mlflow/registered-models/delete DELETE can_delete Get Registered Model 2.0/mlflow/registered-models/get GET can_read Search Registered Models 2.0/mlflow/registered-models/search GET None Get Latest Versions 2.0/mlflow/registered-models/get-latest-versions POST can_read Get Latest Versions 2.0/mlflow/registered-models/get-latest-versions GET can_read Set Registered Model Tag 2.0/mlflow/registered-models/set-tag POST can_update Delete Registered Model Tag 2.0/mlflow/registered-models/delete-tag DELETE can_update Set Registered Model Alias 2.0/mlflow/registered-models/alias POST can_update Delete Registered Model Alias 2.0/mlflow/registered-models/alias DELETE can_delete Get Model Version By Alias 2.0/mlflow/registered-models/alias GET can_read Create Model Version 2.0/mlflow/model-versions/create POST can_update Update Model Version 2.0/mlflow/model-versions/update PATCH can_update Transition Model Version Stage 2.0/mlflow/model-versions/transition-stage POST can_update Delete Model Version 2.0/mlflow/model-versions/delete DELETE can_delete Get Model Version 2.0/mlflow/model-versions/get GET can_read Search Model Versions 2.0/mlflow/model-versions/search GET None Get Model Version Download Uri 2.0/mlflow/model-versions/get-download-uri GET can_read Set Model Version Tag 2.0/mlflow/model-versions/set-tag POST can_update Delete Model Version Tag 2.0/mlflow/model-versions/delete-tag DELETE can_delete MLflow Authentication introduces several new API endpoints to manage users and permissions. API Endpoint Method Required permission Create User 2.0/mlflow/users/create POST None Get User 2.0/mlflow/users/get GET Only readable by that user Update User Password 2.0/mlflow/users/update-password PATCH Only updatable by that user Update User Admin 2.0/mlflow/users/update-admin PATCH Only admin Delete User 2.0/mlflow/users/delete DELETE Only admin Create Experiment Permission 2.0/mlflow/experiments/permissions/create POST can_manage Get Experiment Permission 2.0/mlflow/experiments/permissions/get GET can_manage Update Experiment Permission 2.0/mlflow/experiments/permissions/update PATCH can_manage Delete Experiment Permission 2.0/mlflow/experiments/permissions/delete DELETE can_manage Create Registered Model Permission 2.0/mlflow/registered-models/permissions/create POST can_manage Get Registered Model Permission 2.0/mlflow/registered-models/permissions/get GET can_manage Update Registered Model Permission 2.0/mlflow/registered-models/permissions/update PATCH can_manage Delete Registered Model Permission 2.0/mlflow/registered-models/permissions/delete DELETE can_manage Some APIs will also have their behaviour modified. For example, the creator of an experiment will automatically be granted MANAGE permission on that experiment, so that the creator can grant or revoke other users' access to that experiment. API Endpoint Method Effect Create Experiment 2.0/mlflow/experiments/create POST Automatically grants MANAGE permission to the creator. Create Registered Model 2.0/mlflow/registered-models/create POST Automatically grants MANAGE permission to the creator. Search Experiments 2.0/mlflow/experiments/search POST Only returns experiments which the user has READ permission on. Search Experiments 2.0/mlflow/experiments/search GET Only returns experiments which the user has READ permission on. Search Runs 2.0/mlflow/runs/search POST Only returns experiments which the user has READ permission on. Search Registered Models 2.0/mlflow/registered-models/search GET Only returns registered models which the user has READ permission on. Search Model Versions 2.0/mlflow/model-versions/search GET Only returns registered models which the user has READ permission on. Permissions Database All users and permissions are stored in a database in basic_auth.db, relative to the directory where MLflow server is launched. The location can be changed in the configuration file. To run migrations, use the following command: python -m mlflow.server.auth db upgrade --url <database_url> Admin Users Admin users have unrestricted access to all MLflow resources, including creating or deleting users, updating password and admin status of other users, granting or revoking permissions from other users, and managing permissions for all MLflow resources, even if NO_PERMISSIONS is explicitly set to that admin account. MLflow has a built-in admin user that will be created the first time that the MLflow authentication feature is enabled. Note It is recommended that you update the default admin password as soon as possible after creation. The default admin user credentials are as follows: Username Password admin password Multiple admin users can exist by promoting other users to admin, using the 2.0/mlflow/users/update-admin endpoint. Example # authenticate as built-in admin user export MLFLOW_TRACKING_USERNAME=admin export MLFLOW_TRACKING_PASSWORD=password from mlflow.server import get_app_client tracking_uri = "http://localhost:5000/" auth_client = get_app_client("basic-auth", tracking_uri=tracking_uri) auth_client.create_user(username="user1", password="pw1") auth_client.update_user_admin(username="user1", is_admin=True) Managing Permissions MLflow provides REST APIs and a client class AuthServiceClient to manage users and permissions. To instantiate AuthServiceClient, it is recommended that you use mlflow.server.get_app_client(). Example export MLFLOW_TRACKING_USERNAME=admin export MLFLOW_TRACKING_PASSWORD=password from mlflow import MlflowClient from mlflow.server import get_app_client tracking_uri = "http://localhost:5000/" auth_client = get_app_client("basic-auth", tracking_uri=tracking_uri) auth_client.create_user(username="user1", password="pw1") auth_client.create_user(username="user2", password="pw2") client = MlflowClient(tracking_uri=tracking_uri) experiment_id = client.create_experiment(name="experiment") auth_client.create_experiment_permission( experiment_id=experiment_id, username="user2", permission="MANAGE" ) Authenticating to MLflow Using MLflow UI When a user first visits the MLflow UI on a browser, they will be prompted to login. There is no limit to how many login attempts can be made. Currently, MLflow UI does not display any information about the current user. Once a user is logged in, the only way to log out is to close the browser. Using Environment Variables MLflow provides two environment variables for authentication: MLFLOW_TRACKING_USERNAME and MLFLOW_TRACKING_PASSWORD. To use basic authentication, you must set both environment variables. export MLFLOW_TRACKING_USERNAME=username export MLFLOW_TRACKING_PASSWORD=password import mlflow mlflow.set_tracking_uri("https://<mlflow_tracking_uri>/") with mlflow.start_run(): ... Using Credentials File You can save your credentials in a file to remove the need for setting environment variables every time. The credentials should be saved in ~/.mlflow/credentials using INI format. Note that the password will be stored unencrypted on disk, and is protected only by filesystem permissions. If the environment variables MLFLOW_TRACKING_USERNAME and MLFLOW_TRACKING_PASSWORD are configured, they override any credentials provided in the credentials file. Credentials file format [mlflow] mlflow_tracking_username = username mlflow_tracking_password = password Using REST API A user can authenticate using the HTTP Authorization request header. See https://developer.mozilla.org/en-US/docs/Web/HTTP/Authentication for more information. In Python, you can use the requests library: import requests response = requests.get( "https://<mlflow_tracking_uri>/", auth=("username", "password"), ) Creating a New User Important To create a new user, you are required to authenticate with admin privileges. Using MLflow UI MLflow UI provides a simple page for creating new users at <tracking_uri>/signup. Using REST API Alternatively, you can send POST requests to the Tracking Server endpoint 2.0/users/create. In Python, you can use the requests library: import requests response = requests.post( "https://<mlflow_tracking_uri>/api/2.0/mlflow/users/create", json={ "username": "username", "password": "password", }, ) Using MLflow AuthServiceClient MLflow AuthServiceClient provides a function to create new users easily. import mlflow auth_client = mlflow.server.get_app_client( "basic-auth", tracking_uri="https://<mlflow_tracking_uri>/" ) auth_client.create_user(username="username", password="password") Configuration Authentication configuration is located at mlflow/server/auth/basic_auth.ini: Variable Description default_permission Default permission on all resources database_uri Database location to store permission and user data admin_username Default admin username if the admin is not already created admin_password Default admin password if the admin is not already created authorization_function Function to authenticate requests The authorization_function setting supports pluggable authentication methods if you want to use another authentication method than HTTP basic auth. The value specifies module_name:function_name. The function has the following signature: def authenticate_request() -> Union[Authorization, Response]: ... The function should return a werkzeug.datastructures.Authorization object if the request is authenticated, or a Response object (typically 401: Unauthorized) if the request is not authenticated. For an example of how to implement a custom authentication method, see tests/server/auth/jwt_auth.py. NOTE: This example is not intended for production use. Custom Authentication MLflow authentication is designed to be extensible. If your organization desires more advanced authentication logic (e.g., token-based authentication), it is possible to install a third party plugin or to create your own plugin. Your plugin should be an installable Python package. It should include an app factory that extends the MLflow app and, optionally, implement a client to manage permissions. The app factory function name will be passed to the --app argument in Flask CLI. See https://flask.palletsprojects.com/en/latest/cli/#application-discovery for more information. Example: my_auth/__init__.py from flask import Flask from mlflow.server import app def create_app(app: Flask = app): app.add_url_rule(...) return app class MyAuthClient: ... Then, the plugin should be installed in your Python environment: pip install my_auth Then, register your plugin in mlflow/setup.py: setup( ..., entry_points=""" ... [mlflow.app] my-auth=my_auth:create_app [mlflow.app.client] my-auth=my_auth:MyAuthClient """, ) Then, you can start the MLflow server: mlflow server --app-name my-auth Previous Next © MLflow Project, a Series of LF Projects, LLC. All rights reserved.
9	search-experiments.html	Documentation Search Experiments Search Experiments mlflow.search_experiments() and MlflowClient.search_experiments() support the same filter string syntax as mlflow.search_runs() and MlflowClient.search_runs(), but the supported identifiers and comparators are different. Table of Contents Syntax Identifier Comparator Examples Syntax See Search Runs Syntax for more information. Identifier The following identifiers are supported: attributes.name: Experiment name attributes.creation_time: Experiment creation time attributes.last_update_time: Experiment last update time Note attributes can be omitted. name is equivalent to attributes.name. tags.<tag key>: Tag Comparator Comparators for string attributes and tags: =: Equal !=: Not equal LIKE: Case-sensitive pattern match ILIKE: Case-insensitive pattern match Comparators for numeric attributes: =: Equal !=: Not equal <: Less than <=: Less than or equal to >: Greater than >=: Greater than or equal to Examples # Matches experiments with name equal to 'x' "attributes.name = 'x'" # or "name = 'x'" # Matches experiments with name starting with 'x' "attributes.name LIKE 'x%'" # Matches experiments with 'group' tag value not equal to 'x' "tags.group != 'x'" # Matches experiments with 'group' tag value containing 'x' or 'X' "tags.group ILIKE '%x%'" # Matches experiments with name starting with 'x' and 'group' tag value equal to 'y' "attributes.name LIKE 'x%' AND tags.group = 'y'" Previous Next © MLflow Project, a Series of LF Projects, LLC. All rights reserved.
10	python_api/index.html	Documentation Python API Python API The MLflow Python API is organized into the following modules. The most common functions are exposed in the mlflow module, so we recommend starting there. mlflow mlflow.artifacts mlflow.catboost mlflow.client mlflow.data mlflow.deployments mlflow.diviner mlflow.entities mlflow.environment_variables mlflow.fastai mlflow.gateway mlflow.gluon mlflow.h2o mlflow.johnsnowlabs mlflow.keras_core mlflow.langchain mlflow.lightgbm mlflow.metrics mlflow.mleap mlflow.models mlflow.onnx mlflow.paddle mlflow.pmdarima mlflow.projects mlflow.prophet mlflow.pyfunc mlflow.pyspark.ml mlflow.pytorch mlflow.recipes mlflow.sagemaker mlflow.sentence_transformers mlflow.server mlflow.shap mlflow.sklearn mlflow.spacy mlflow.spark mlflow.statsmodels mlflow.system_metrics mlflow.tensorflow mlflow.transformers mlflow.types mlflow.utils mlflow.xgboost mlflow.openai See also the index of all functions and classes. Log Levels MLflow Python APIs log information during execution using the Python Logging API. You can configure the log level for MLflow logs using the following code snippet. Learn more about Python log levels at the Python language logging guide. import logging logger = logging.getLogger("mlflow") # Set log level to debugging logger.setLevel(logging.DEBUG) Previous Next © MLflow Project, a Series of LF Projects, LLC. All rights reserved.

The file is too large to be shown. View Raw