chore: import upstream snapshot with attribution

docs/concepts/concept-connections.md

@@ -0,0 +1,25 @@
+In prompt flow, you can utilize connections to securely manage credentials or secrets for external services.
+
+# Connections
+
+Connections are for storing information about how to access external services like LLMs: endpoint, api keys etc.
+
+- In your local development environment, the connections are persisted in your local machine with keys encrypted.
+- In Azure AI, connections can be configured to be shared across the entire workspace. Secrets associated with connections are securely persisted in the corresponding Azure Key Vault, adhering to robust security and compliance standards.
+
+Prompt flow provides a variety of pre-built connections, including Azure OpenAI, OpenAI, etc. These pre-built connections enable seamless integration with these resources within the built-in tools. Additionally, you have the flexibility to create custom connection types using key-value pairs, empowering them to tailor the connections to their specific requirements, particularly in Python tools.
+
+| Connection type                                              | Built-in tools                  |
+| ------------------------------------------------------------ | ------------------------------- |
+| [Azure OpenAI](https://azure.microsoft.com/products/cognitive-services/openai-service) | LLM or Python                   |
+| [OpenAI](https://openai.com/)                               | LLM or Python                   |
+| [Cognitive Search](https://azure.microsoft.com/products/search) | Vector DB Lookup or Python      |
+| [Serp](https://serpapi.com/)                                 | Serp API or Python              |
+| [Serverless](https://learn.microsoft.com/azure/ai-studio/concepts/deployments-overview)                                               | LLM or Python                   |
+| Custom                                                       | Python                          |
+
+By leveraging connections in prompt flow, you can easily establish and manage connections to external APIs and data sources, facilitating efficient data exchange and interaction within their AI applications.
+
+## Next steps
+
+- [Create connections](../how-to-guides/manage-connections.md)

docs/concepts/concept-flows.md

@@ -0,0 +1,58 @@
+While how LLMs work may be elusive to many developers, how LLM apps work is not - they essentially involve a series of calls to external services such as LLMs/databases/search engines, or intermediate data processing, all glued together.
+
+# Flows
+
+## Flex flow
+
+You can create LLM apps using a Python function or class as the entry point, which encapsulating your app logic. You can directly test or run these with pure code experience. Or you can define a `flow.flex.yaml` that points to these entries, which enables testing, running, or viewing traces via the [Promptflow VS Code Extension](https://marketplace.visualstudio.com/items?itemName=prompt-flow.prompt-flow).
+
+Our [examples](https://github.com/microsoft/promptflow/tree/main/examples/flex-flows) should also give you an idea how to write `flex flows`.
+
+## DAG flow
+
+Thus LLM apps can be defined as Directed Acyclic Graphs (DAGs) of function calls. These DAGs are flows in prompt flow.
+
+A `DAG flow` in prompt flow is a DAG of functions (we call them [tools](./concept-tools.md)). These functions/tools connected via input/output dependencies and executed based on the topology by prompt flow executor.
+
+A flow is represented as a YAML file and can be visualized with our [Prompt flow for VS Code extension](https://marketplace.visualstudio.com/items?itemName=prompt-flow.prompt-flow). Here is an example `flow.dag.yaml`:
+
+![flow_dag](../media/how-to-guides/quick-start/flow_dag.png)
+
+Please refer to our [examples](https://github.com/microsoft/promptflow/tree/main/examples/flows) to learn how to write a `DAG flow`. 
+
+## When to use Flex or DAG flow
+
+`Dag flow` provides a UI-friendly way to develop your LLM app, which has the following benefits:
+- **Low code**: user can drag-and-drop in UI to create a LLM app.
+- **DAG Visualization**: user can easily understand the logic structure of the app with DAG view.
+
+`Flex flow` provides a code-friendly way to develop your LLM app, which has the following benefits:
+- **Quick start**: Users can quickly test with a simple prompt, then customize with python code with Tracing visualization UI.
+- **More advanced orchestration**: Users can write complex flow with Python built-in control operators (if-else, foreach) or other 3rd party / open-source library. 
+- **Easy onboard from other platforms**: user might already onboard platforms like `langchain` and `sematic kernel` with existing code. User can easily onboard promptflow with a few code changes.
+
+## Flow types
+
+Prompt flow examples organize flows by three categories:
+
+- **Standard flow** or **Chat flow**: these two are for you to develop your LLM application. The primary difference between the two lies in the additional support provided by the "Chat Flow" for chat applications. For instance, you can define `chat_history`, `chat_input`, and `chat_output` for your flow. The prompt flow, in turn, will offer a chat-like experience (including conversation history) during the development of the flow. Moreover, it also provides a sample chat application for deployment purposes.
+- **Evaluation flow** is for you to test/evaluate the quality of your LLM application (standard/chat flow). It usually run on the outputs of standard/chat flow, and compute some metrics that can be used to determine whether the standard/chat flow performs well. E.g. is the answer accurate? is the answer fact-based?
+
+
+Flex flow [examples](https://github.com/microsoft/promptflow/tree/main/examples/flex-flows):
+- [basic](https://github.com/microsoft/promptflow/tree/main/examples/flex-flows/basic): A basic standard flow define using function entry that calls Azure OpenAI with connection info stored in environment variables.
+- [chat-basic](https://github.com/microsoft/promptflow/tree/main/examples/flex-flows/chat-basic): A basic chat flow defined using class entry. It demonstrates how to create a chatbot that can remember previous interactions and use the conversation history to generate next message.
+- [eval-code-quality](https://github.com/microsoft/promptflow/tree/main/examples/flex-flows/eval-code-quality): A example flow defined using function entry which shows how to evaluate the quality of code snippet.
+
+DAG flow [examples](https://github.com/microsoft/promptflow/tree/main/examples/flows):
+- [basic](https://github.com/microsoft/promptflow/tree/main/examples/flows/standard/basic): A basic standard flow using custom python tool that calls Azure OpenAI with connection info stored in environment variables.
+- [chat-basic](https://github.com/microsoft/promptflow/tree/main/examples/flows/chat/chat-basic): This example shows how to create a basic chat flow. It demonstrates how to create a chatbot that can remember previous interactions and use the conversation history to generate next message.
+- [eval-basic](https://github.com/microsoft/promptflow/tree/main/examples/flows/evaluation/eval-basic): 
+
+
+## Next steps
+
+- [Quick start](../how-to-guides/quick-start.md)
+- [Initialize and test a flow](../how-to-guides/develop-a-dag-flow/init-and-test-a-flow.md)
+- [Run and evaluate a flow](../how-to-guides/run-and-evaluate-a-flow/index.md)
+- [Tune prompts using variants](../how-to-guides/tune-prompts-with-variants.md)

docs/concepts/concept-tools.md

@@ -0,0 +1,29 @@
+Tools are the fundamental building blocks of a [flow](./concept-flows.md).
+
+Each tool is an executable unit, basically a function to performs various tasks including but not limited to:
+- Accessing LLMs for various purposes
+- Querying databases
+- Getting information from search engines
+- Pre/post processing of data
+
+# Tools
+
+Prompt flow provides 3 basic tools:
+- [LLM](../reference/tools-reference/llm-tool.md): The LLM tool allows you to write custom prompts and leverage large language models to achieve specific goals, such as summarizing articles, generating customer support responses, and more.
+- [Python](../reference/tools-reference/python-tool.md): The Python tool enables you to write custom Python functions to perform various tasks, such as fetching web pages, processing intermediate data, calling third-party APIs, and more.
+- [Prompt](../reference/tools-reference/prompt-tool.md): The Prompt tool allows you to prepare a prompt as a string for more complex use cases or for use in conjunction with other prompt tools or python tools.
+
+## More tools
+
+Our partners also contributes other useful tools for advanced scenarios, here are some links:
+- [Vector DB Lookup](../reference/tools-reference/vector_db_lookup_tool.md): vector search tool that allows users to search top k similar vectors from vector database.
+- [Faiss Index Lookup](../reference/tools-reference/faiss_index_lookup_tool.md): querying within a user-provided Faiss-based vector store.
+
+## Custom tools
+
+You can create your own tools that can be shared with your team or anyone in the world. 
+Learn more on [Create and Use Tool Package](../how-to-guides/develop-a-tool/create-and-use-tool-package.md)
+
+## Next steps
+
+For more information on the available tools and their usage, visit the our [reference doc](../reference/index.md).

docs/concepts/concept-variants.md

@@ -0,0 +1,27 @@
+With prompt flow, you can use variants to tune your prompt. In this article, you'll learn the prompt flow variants concept.
+
+# Variants
+
+A variant refers to a specific version of a tool node that has distinct settings. Currently, variants are supported only in the LLM tool. For example, in the LLM tool, a new variant can represent either a different prompt content or different connection settings.
+
+Suppose you want to generate a summary of a news article. You can set different variants of prompts and settings like this:
+
+| Variants  | Prompt                                                       | Connection settings |
+| --------- | ------------------------------------------------------------ | ------------------- |
+| Variant 0 | `Summary: {{input sentences}}`                               | Temperature = 1     |
+| Variant 1 | `Summary: {{input sentences}}`                               | Temperature = 0.7   |
+| Variant 2 | `What is the main point of this article? {{input sentences}}` | Temperature = 1     |
+| Variant 3 | `What is the main point of this article? {{input sentences}}` | Temperature = 0.7   |
+
+By utilizing different variants of prompts and settings, you can explore how the model responds to various inputs and outputs, enabling you to discover the most suitable combination for your requirements.
+
+## Benefits of using variants 
+
+- **Enhance the quality of your LLM generation**: By creating multiple variants of the same LLM node with diverse prompts and configurations, you can identify the optimal combination that produces high-quality content aligned with your needs.
+- **Save time and effort**: Even slight modifications to a prompt can yield significantly different results. It's crucial to track and compare the performance of each prompt version. With variants, you can easily manage the historical versions of your LLM nodes, facilitating updates based on any variant without the risk of forgetting previous iterations. This saves you time and effort in managing prompt tuning history.
+- **Boost productivity**: Variants streamline the optimization process for LLM nodes, making it simpler to create and manage multiple variations. You can achieve improved results in less time, thereby increasing your overall productivity.
+- **Facilitate easy comparison**: You can effortlessly compare the results obtained from different variants side by side, enabling you to make data-driven decisions regarding the variant that generates the best outcomes.
+
+## Next steps
+
+- [Tune prompts with variants](../how-to-guides/tune-prompts-with-variants.md)

docs/concepts/design-principles.md

@@ -0,0 +1,51 @@
+# Design principles
+
+When we started this project, [LangChain](https://www.langchain.com/) already became popular esp. after the ChatGPT launch. One of the questions we’ve been asked is what’s the difference between prompt flow and LangChain. This article is to elucidate the reasons for building prompt flow and the deliberate design choices we have made. To put it succinctly, prompt flow is a suite of development tools for you to build LLM apps with a strong emphasis of quality through experimentations, not a framework - which LangChain is.
+
+While LLM apps are mostly in exploration stage, Microsoft started in this area a bit earlier and we’ve had the opportunity to observe how developers are integrating LLMs into existing systems or build new applications. These invaluable insights have shaped the fundamental design principles of prompt flow. 
+
+## 1. Expose the prompts vs. hiding them
+
+The core essence of LLM applications lies in the prompts themselves, at least for today. When developing a reasonably complex LLM application, the majority of development work should be “tuning” the prompts (note the intentional use of the term "tuning," which we will delve into further later on). Any framework or tool trying to help in this space should focus on making prompt tuning easier and more straightforward. On the other hand, prompts are very volatile, it's unlikely to write a single prompt that can work across different models or even different version of same models. Building a successful LLM-based application, you have to understand every prompt introduced, so that you can tune it when necessary. LLM is simply not powerful or deterministic enough that you can use a prompt written by others like you use libraries in traditional programming languages.
+
+In this context, any design that tries to provide a smart function or agent by encapsulating a few prompts in a library is unlikely to yield favorable results in real-world scenarios. And hiding prompts inside a library’s code base only makes it’s hard for people to improve or tailor the prompts to suit their specific needs.
+
+Prompt flow, being positioned as a tool, refrains from wrapping any prompts within its core codebase. The only place you will see prompts are our sample flows, which are, of course, available for adoption and utilization. Every prompt should be authored and controlled by the developers themselves, rather than relying on us.
+
+## 2. A new way of work
+
+LLMs possess remarkable capabilities that enable developers to enhance their applications without delving deep into the intricacies of machine learning. In the meantime, LLMs make these apps more stochastic, which pose new challenges to application development. Merely asserting "no exception" or "result == x" in gated tests is no longer sufficient. Adopting a new methodology and employing new tools becomes imperative to ensure the quality of LLM applications — an entirely novel way of working is required.
+
+At the center of this paradigm shift is evaluation, a term frequently used in machine learning space, refers to the process of assessing the performance and quality of a trained model. It involves measuring how well the model performs on a given task or dataset, which plays a pivotal role in understanding the model's strengths, weaknesses, and overall effectiveness. Evaluation metrics and techniques vary depending on the specific task and problem domain. Some common metrics include accuracy, precision and recall, you probably already familiar with. Now the LLM apps share similarities with machine learning models, they requires an evaluation-centric approach integrated into the development workflow, with a robust set of metrics and evaluation forming the foundation for ensuring the quality of LLM applications.
+
+Prompt flow offers a range of tools to streamline the new way of work:
+
+* Develop your evaluation program as Evaluation flow to calculate metrics for your app/flow, learn from our sample evaluation flows.
+* Iterate on your application flow and run evaluation flows via the SDK/CLI, allowing you to compare metrics and choose the optimal candidate for release. These iterations include trying different prompts, different LLM parameters like temperature etc. - this is referred as “tuning” process earlier, or sometime referred as experimentation.
+* Integrate the evaluation into your CI/CD pipeline, aligning the assertions in your gated tests with the selected metrics.
+
+
+Prompt flow introduces two conceptual components to facilitate this workflow:
+
+* Evaluation flow: a flow type that indicates this flow is not for deploy or integrate into your app, it’s for evaluating an app/flow performance.
+* Run: every time you run your flow with data, or run an evaluation on the output of a flow, a Run object is created to manage the history and allow for comparison and additional analysis.
+
+While new concepts introduce additional cognitive load, we firmly believe they hold greater importance compared to abstracting different LLM APIs or vector database APIs.
+
+## 3. Optimize for “visibility”
+
+There are quite some interesting application patterns emerging because of LLMs, like Retrieval Augmented Generation (RAG), ReAct and more. Though how LLMs work may remain enigmatic to many developers, how LLM apps work is not - they essentially involve a series of calls to external services such as LLMs, databases, and search engines, all glued together. Architecturally there isn’t much new, patterns like RAG and ReAct are both straightforward to implement once a developer understands what they are - plain Python programs with API calls to external services can totally serve the purpose effectively.
+
+By observing many internal use cases, we learned that deeper insight into the detail of the execution is critical. Establishing a systematic method for tracking interactions with external systems is one of design priority. Consequently, We adopted an unconventional approach - prompt flow has a YAML file describing how function calls (we call them [Tools](../concepts/concept-tools.md)) are executed and connected into a Directed Acyclic Graph (DAG). 
+
+This approach offers several key benefits, primarily centered around **enhanced visibility**:
+1) During development, your flow can be visualized in an intelligible manner, enabling clear identification of any faulty components. As a byproduct, you obtain an architecturally descriptive diagram that can be shared with others.
+2) Each node in the flow has it’s internal detail visualized in a consistent way.
+3) Single nodes can be individually run or debugged without the need to rerun previous nodes.
+</b>
+
+![promptflow-dag](../media/promptflow-dag.png)
+
+The emphasis on visibility in prompt flow's design helps developers to gain a comprehensive understanding of the intricate details of their applications. This, in turn, empowers developers to engage in effective troubleshooting and optimization.
+
+Despite there're some control flow features like "activate-when" to serve the needs of branches/switch-case, we do not intend to make Flow itself Turing-complete. If you want to develop an agent which is fully dynamic and guided by LLM, leveraging [Semantic Kernel](https://github.com/microsoft/semantic-kernel) together with prompt flow would be a favorable option.

docs/concepts/index.md

@@ -0,0 +1,13 @@
+# Concepts
+
+In this section, you will learn the basic concepts of prompt flow.
+
+```{toctree}
+:maxdepth: 1
+
+concept-flows
+concept-tools
+concept-connections
+concept-variants
+design-principles
+```