chore: import upstream snapshot with attribution
This commit is contained in:
wehub-resource-sync
+21
@@ -0,0 +1,21 @@
|
||||
MIT License
|
||||
|
||||
Copyright (c) 2024 Aishwarya Naresh Reganti
|
||||
|
||||
Permission is hereby granted, free of charge, to any person obtaining a copy
|
||||
of this software and associated documentation files (the "Software"), to deal
|
||||
in the Software without restriction, including without limitation the rights
|
||||
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
||||
copies of the Software, and to permit persons to whom the Software is
|
||||
furnished to do so, subject to the following conditions:
|
||||
|
||||
The above copyright notice and this permission notice shall be included in all
|
||||
copies or substantial portions of the Software.
|
||||
|
||||
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
||||
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
||||
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
||||
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
||||
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
||||
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
||||
SOFTWARE.
|
||||
@@ -0,0 +1,351 @@
|
||||
# :star: :bookmark: awesome-generative-ai-guide
|
||||
|
||||
Generative AI is experiencing rapid growth, and this repository serves as a comprehensive hub for updates on generative AI research, interview materials, notebooks, and more!
|
||||
|
||||
<a href="https://trendshift.io/repositories/7663" target="_blank"><img src="https://trendshift.io/api/badge/repositories/7663" alt="aishwaryanr%2Fawesome-generative-ai-guide | Trendshift" style="width: 250px; height: 55px;" width="250" height="55"/></a>
|
||||
|
||||
Explore the following resources:
|
||||
|
||||
1. [Monthly Best GenAI Papers List](https://github.com/aishwaryanr/awesome-generative-ai-guide?tab=readme-ov-file#star-best-genai-papers-list-january-2024)
|
||||
2. [GenAI Interview Resources](https://github.com/aishwaryanr/awesome-generative-ai-guide?tab=readme-ov-file#computer-interview-prep)
|
||||
3. [Applied LLMs Mastery 2024 (created by Aishwarya Naresh Reganti) course material](https://github.com/aishwaryanr/awesome-generative-ai-guide?tab=readme-ov-file#ongoing-applied-llms-mastery-2024)
|
||||
4. [Generative AI Genius 2024 (created by Aishwarya Naresh Reganti) course material](https://github.com/aishwaryanr/awesome-generative-ai-guide/blob/main/free_courses/generative_ai_genius/README.md)
|
||||
5. [AI Evals for Everyone (created by Aishwarya Naresh Reganti & Kiriti Badam) - Get Certified!](https://github.com/aishwaryanr/awesome-generative-ai-guide/blob/main/free_courses/ai_evals_for_everyone/README.md)
|
||||
6. **[NEW] [OpenClaw Mastery for Everyone (created by Aishwarya Reganti & Kiriti Badam) - Get Certified!](https://github.com/aishwaryanr/awesome-generative-ai-guide/blob/main/free_courses/openclaw_mastery_for_everyone/README.md)**
|
||||
7. [List of all GenAI-related free courses (over 90 listed)](https://github.com/aishwaryanr/awesome-generative-ai-guide?tab=readme-ov-file#book-list-of-free-genai-courses)
|
||||
8. [List of code repositories/notebooks for developing generative AI applications](https://github.com/aishwaryanr/awesome-generative-ai-guide?tab=readme-ov-file#notebook-code-notebooks)
|
||||
|
||||
We'll be updating this repository regularly, so keep an eye out for the latest additions!
|
||||
|
||||
Happy Learning!
|
||||
|
||||
---
|
||||
## :star: Top AI Tools List
|
||||
|
||||
Discover our favorite AI tools spanning every layer of AI application development. Click [here](https://github.com/aishwaryanr/awesome-generative-ai-guide/blob/main/resources/our_favourite_ai_tools.md) to learn more.
|
||||
|
||||
---
|
||||
|
||||
## :speaker: Announcements
|
||||
|
||||
- **NEW: OpenClaw Mastery for Everyone is now live with certification!** ([Click Here](https://github.com/aishwaryanr/awesome-generative-ai-guide/blob/main/free_courses/openclaw_mastery_for_everyone/README.md))
|
||||
- AI Evals for Everyone course is now live with certification! ([Click Here](https://github.com/aishwaryanr/awesome-generative-ai-guide/blob/main/free_courses/ai_evals_for_everyone/README.md))
|
||||
- Applied LLMs Mastery full course content has been released!!! ([Click Here](https://github.com/aishwaryanr/awesome-generative-ai-guide/blob/main/free_courses/Applied_LLMs_Mastery_2024))
|
||||
- 5-day roadmap to learn LLM foundations out now! ([Click Here](https://github.com/aishwaryanr/awesome-generative-ai-guide/blob/main/resources/genai_roadmap.md))
|
||||
- 60 Common GenAI Interview Questions out now! ([Click Here](https://github.com/aishwaryanr/awesome-generative-ai-guide/blob/main/interview_prep/60_gen_ai_questions.md))
|
||||
- ICLR 2024 paper summaries ([Click Here](https://areganti.notion.site/06f0d4fe46a94d62bff2ae001cfec22c?v=d501ca62e4b745768385d698f173ae14))
|
||||
- List of free GenAI courses ([Click Here](https://github.com/aishwaryanr/awesome-generative-ai-guide#book-list-of-free-genai-courses))
|
||||
- Generative AI resources and roadmaps
|
||||
- [3-day RAG roadmap](https://github.com/aishwaryanr/awesome-generative-ai-guide/blob/main/resources/RAG_roadmap.md)
|
||||
- [5-day LLM foundations roadmap](https://github.com/aishwaryanr/awesome-generative-ai-guide/blob/main/resources/genai_roadmap.md)
|
||||
- [5-day LLM agents roadmap](https://github.com/aishwaryanr/awesome-generative-ai-guide/blob/main/resources/agents_roadmap.md)
|
||||
- [Agents 101 guide](https://github.com/aishwaryanr/awesome-generative-ai-guide/blob/main/resources/agents_101_guide.md)
|
||||
- [Introduction to MM LLMs](https://github.com/aishwaryanr/awesome-generative-ai-guide/blob/main/resources/mm_llms_guide.md)
|
||||
- [LLM Lingo Series: Commonly used LLM terms and their easy-to-understand definitions](https://github.com/aishwaryanr/awesome-generative-ai-guide/blob/main/resources/llm_lingo)
|
||||
|
||||
---
|
||||
|
||||
|
||||
## :mortar_board: Courses
|
||||
|
||||
#### [Ongoing] Applied LLMs Mastery 2024
|
||||
|
||||
Join 1000+ students on this 10-week adventure as we delve into the application of LLMs across a variety of use cases
|
||||
|
||||
#### [Link](https://areganti.notion.site/Applied-LLMs-Mastery-2024-562ddaa27791463e9a1286199325045c) to the course website
|
||||
|
||||
##### [Feb 2024] Registrations are still open [click here](https://forms.gle/353sQMRvS951jDYu7) to register
|
||||
|
||||
🗓️\*Week 1 [Jan 15 2024]**\*: [Practical Introduction to LLMs](https://github.com/aishwaryanr/awesome-generative-ai-guide/blob/main/free_courses/Applied_LLMs_Mastery_2024/week1_part1_foundations.md)**
|
||||
|
||||
- Applied LLM Foundations
|
||||
- Real World LLM Use Cases
|
||||
- Domain and Task Adaptation Methods
|
||||
|
||||
🗓️\*Week 2 [Jan 22 2024]**\*: [Prompting and Prompt
|
||||
Engineering](https://github.com/aishwaryanr/awesome-generative-ai-guide/blob/main/free_courses/Applied_LLMs_Mastery_2024/week2_prompting.md)**
|
||||
|
||||
- Basic Prompting Principles
|
||||
- Types of Prompting
|
||||
- Applications, Risks and Advanced Prompting
|
||||
|
||||
🗓️\*Week 3 [Jan 29 2024]**\*: [LLM Fine-tuning](https://github.com/aishwaryanr/awesome-generative-ai-guide/blob/main/free_courses/Applied_LLMs_Mastery_2024/week3_finetuning_llms.md)**
|
||||
|
||||
- Basics of Fine-Tuning
|
||||
- Types of Fine-Tuning
|
||||
- Fine-Tuning Challenges
|
||||
|
||||
🗓️\*Week 4 [Feb 5 2024]**\*: [RAG (Retrieval-Augmented Generation)](https://github.com/aishwaryanr/awesome-generative-ai-guide/blob/main/free_courses/Applied_LLMs_Mastery_2024/week4_RAG.md)**
|
||||
|
||||
- Understanding the concept of RAG in LLMs
|
||||
- Key components of RAG
|
||||
- Advanced RAG Methods
|
||||
|
||||
🗓️\*Week 5 [ Feb 12 2024]**\*: [Tools for building LLM Apps](https://github.com/aishwaryanr/awesome-generative-ai-guide/blob/main/free_courses/Applied_LLMs_Mastery_2024/week5_tools_for_LLM_apps.md)**
|
||||
|
||||
- Fine-tuning Tools
|
||||
- RAG Tools
|
||||
- Tools for observability, prompting, serving, vector search etc.
|
||||
|
||||
🗓️\*Week 6 [Feb 19 2024]**\*: [Evaluation Techniques](https://github.com/aishwaryanr/awesome-generative-ai-guide/blob/main/free_courses/Applied_LLMs_Mastery_2024/week6_llm_evaluation.md)**
|
||||
|
||||
- Types of Evaluation
|
||||
- Common Evaluation Benchmarks
|
||||
- Common Metrics
|
||||
|
||||
🗓️\*Week 7 [Feb 26 2024]**\*: [Building Your Own LLM Application](https://github.com/aishwaryanr/awesome-generative-ai-guide/blob/main/free_courses/Applied_LLMs_Mastery_2024/week7_build_llm_app.md)**
|
||||
|
||||
- Components of LLM application
|
||||
- Build your own LLM App end to end
|
||||
|
||||
🗓️\*Week 8 [March 4 2024]**\*: [Advanced Features and Deployment](https://github.com/aishwaryanr/awesome-generative-ai-guide/blob/main/free_courses/Applied_LLMs_Mastery_2024/week8_advanced_features.md)**
|
||||
|
||||
- LLM lifecycle and LLMOps
|
||||
- LLM Monitoring and Observability
|
||||
- Deployment strategies
|
||||
|
||||
🗓️\*Week 9 [March 11 2024]**\*: [Challenges with LLMs](https://github.com/aishwaryanr/awesome-generative-ai-guide/blob/main/free_courses/Applied_LLMs_Mastery_2024/week9_challenges_with_llms.md)**
|
||||
|
||||
- Scaling Challenges
|
||||
- Behavioral Challenges
|
||||
- Future directions
|
||||
|
||||
🗓️\*Week 10 [March 18 2024]**\*: [Emerging Research Trends](https://github.com/aishwaryanr/awesome-generative-ai-guide/blob/main/free_courses/Applied_LLMs_Mastery_2024/week10_research_trends.md)**
|
||||
|
||||
- Smaller and more performant models
|
||||
- Multimodal models
|
||||
- LLM Alignment
|
||||
|
||||
🗓️*Week 11 *Bonus\* [March 25 2024]**\*: [Foundations](https://github.com/aishwaryanr/awesome-generative-ai-guide/blob/main/free_courses/Applied_LLMs_Mastery_2024/week11_foundations.md)**
|
||||
|
||||
- Generative Models Foundations
|
||||
- Self-Attention and Transformers
|
||||
- Neural Networks for Language
|
||||
|
||||
---
|
||||
|
||||
#### :book: List of Free GenAI Courses
|
||||
|
||||
##### LLM Basics and Foundations
|
||||
|
||||
1. [Large Language Models](https://rycolab.io/classes/llm-s23/) by ETH Zurich
|
||||
|
||||
2. [Understanding Large Language Models](https://www.cs.princeton.edu/courses/archive/fall22/cos597G/) by Princeton
|
||||
|
||||
3. [Transformers course](https://huggingface.co/learn/nlp-course/chapter1/1) by Huggingface
|
||||
|
||||
4. [NLP course](https://huggingface.co/learn/nlp-course/chapter1/1) by Huggingface
|
||||
|
||||
5. [CS324 - Large Language Models](https://stanford-cs324.github.io/winter2022/) by Stanford
|
||||
|
||||
6. [Generative AI with Large Language Models](https://www.coursera.org/learn/generative-ai-with-llms) by Coursera
|
||||
|
||||
7. [Introduction to Generative AI](https://www.coursera.org/learn/introduction-to-generative-ai) by Coursera
|
||||
|
||||
8. [Generative AI Fundamentals](https://www.cloudskillsboost.google/paths/118/course_templates/556) by Google Cloud
|
||||
9. [5-Day Gen AI Intensive Course](https://www.youtube.com/watch?v=kpRyiJUUFxY&list=PLqFaTIg4myu-b1PlxitQdY0UYIbys-2es) by Google & Kaggle
|
||||
|
||||
10. [Introduction to Large Language Models](https://www.cloudskillsboost.google/paths/118/course_templates/539) by Google Cloud
|
||||
11. [Introduction to Generative AI](https://www.cloudskillsboost.google/paths/118/course_templates/536) by Google Cloud
|
||||
12. [Generative AI Concepts](https://www.datacamp.com/courses/generative-ai-concepts) by DataCamp (Daniel Tedesco Data Lead @ Google)
|
||||
13. [1 Hour Introduction to LLM (Large Language Models)](https://www.youtube.com/watch?v=xu5_kka-suc) by WeCloudData
|
||||
14. [LLM Foundation Models from the Ground Up | Primer](https://www.youtube.com/watch?v=W0c7jQezTDw&list=PLTPXxbhUt-YWjMCDahwdVye8HW69p5NYS) by Databricks
|
||||
15. [Generative AI Explained](https://courses.nvidia.com/courses/course-v1:DLI+S-FX-07+V1/) by Nvidia
|
||||
16. [Transformer Models and BERT Model](https://www.cloudskillsboost.google/course_templates/538) by Google Cloud
|
||||
17. [Generative AI Learning Plan for Decision Makers](https://explore.skillbuilder.aws/learn/public/learning_plan/view/1909/generative-ai-learning-plan-for-decision-makers) by AWS
|
||||
18. [Introduction to Responsible AI](https://www.cloudskillsboost.google/course_templates/554) by Google Cloud
|
||||
19. [Fundamentals of Generative AI](https://learn.microsoft.com/en-us/training/modules/fundamentals-generative-ai/) by Microsoft Azure
|
||||
20. [Generative AI for Beginners](https://github.com/microsoft/generative-ai-for-beginners?WT.mc_id=academic-122979-leestott) by Microsoft
|
||||
21. [ChatGPT for Beginners: The Ultimate Use Cases for Everyone](https://www.udemy.com/course/chatgpt-for-beginners-the-ultimate-use-cases-for-everyone/) by Udemy
|
||||
22. [[1hr Talk] Intro to Large Language Models](https://www.youtube.com/watch?v=zjkBMFhNj_g) by Andrej Karpathy
|
||||
23. [ChatGPT for Everyone](https://learnprompting.org/courses/chatgpt-for-everyone) by Learn Prompting
|
||||
24. [Large Language Models (LLMs) (In English)](https://www.youtube.com/playlist?list=PLxlkzujLkmQ9vMaqfvqyfvZV_o8EqjAk7) by Kshitiz Verma (JK Lakshmipat University, Jaipur, India)
|
||||
25. [Generative AI for Beginners](https://codekidz.ai/lesson-intro/generative-a-362093) By CodeKidz, based on Microsoft's open sourced course.
|
||||
|
||||
##### Building LLM Applications
|
||||
|
||||
1. [LLMOps: Building Real-World Applications With Large Language Models](https://www.udacity.com/course/building-real-world-applications-with-large-language-models--cd13455) by Udacity
|
||||
|
||||
2. [Full Stack LLM Bootcamp](https://fullstackdeeplearning.com/llm-bootcamp/) by FSDL
|
||||
|
||||
3. [Generative AI for beginners](https://github.com/microsoft/generative-ai-for-beginners/tree/main) by Microsoft
|
||||
|
||||
4. [Large Language Models: Application through Production](https://www.edx.org/learn/computer-science/databricks-large-language-models-application-through-production) by Databricks
|
||||
|
||||
5. [Generative AI Foundations](https://www.youtube.com/watch?v=oYm66fHqHUM&list=PLhr1KZpdzukf-xb0lmiU3G89GJXaDbAIF) by AWS
|
||||
|
||||
6. [Introduction to Generative AI Community Course](https://www.youtube.com/watch?v=ajWheP8ZD70&list=PLmQAMKHKeLZ-iTT-E2kK9uePrJ1Xua9VL) by ineuron
|
||||
|
||||
7. [LLM University](https://docs.cohere.com/docs/llmu) by Cohere
|
||||
8. [LLM Learning Lab](https://lightning.ai/pages/llm-learning-lab/) by Lightning AI
|
||||
9. [LangChain for LLM Application Development](https://learn.deeplearning.ai/login?redirect_course=langchain&callbackUrl=https%3A%2F%2Flearn.deeplearning.ai%2Fcourses%2Flangchain) by Deeplearning.AI
|
||||
10. [LLMOps](https://learn.deeplearning.ai/llmops) by DeepLearning.AI
|
||||
11. [Automated Testing for LLMOps](https://learn.deeplearning.ai/automated-testing-llmops) by DeepLearning.AI
|
||||
12. [Building Generative AI Applications Using Amazon Bedrock](https://explore.skillbuilder.aws/learn/course/external/view/elearning/17904/building-generative-ai-applications-using-amazon-bedrock-aws-digital-training) by AWS
|
||||
13. [Efficiently Serving LLMs](https://learn.deeplearning.ai/courses/efficiently-serving-llms/lesson/1/introduction) by DeepLearning.AI
|
||||
14. [Building Systems with the ChatGPT API](https://www.deeplearning.ai/short-courses/building-systems-with-chatgpt/) by DeepLearning.AI
|
||||
15. [Serverless LLM apps with Amazon Bedrock](https://www.deeplearning.ai/short-courses/serverless-llm-apps-amazon-bedrock/) by DeepLearning.AI
|
||||
16. [Building Applications with Vector Databases](https://www.deeplearning.ai/short-courses/building-applications-vector-databases/) by DeepLearning.AI
|
||||
17. [Automated Testing for LLMOps](https://www.deeplearning.ai/short-courses/automated-testing-llmops/) by DeepLearning.AI
|
||||
18. [Build LLM Apps with LangChain.js](https://www.deeplearning.ai/short-courses/build-llm-apps-with-langchain-js/) by DeepLearning.AI
|
||||
19. [Advanced Retrieval for AI with Chroma](https://www.deeplearning.ai/short-courses/advanced-retrieval-for-ai/) by DeepLearning.AI
|
||||
20. [Operationalizing LLMs on Azure](https://www.coursera.org/learn/llmops-azure) by Coursera
|
||||
21. [Generative AI Full Course – Gemini Pro, OpenAI, Llama, Langchain, Pinecone, Vector Databases & More](https://www.youtube.com/watch?v=mEsleV16qdo) by freeCodeCamp.org
|
||||
22. [Training & Fine-Tuning LLMs for Production](https://learn.activeloop.ai/courses/llms) by Activeloop
|
||||
|
||||
|
||||
##### Prompt Engineering, RAG and Fine-Tuning
|
||||
|
||||
1. [LangChain & Vector Databases in Production](https://www.youtube.com/redirect?event=video_description&redir_token=QUFFLUhqbVhnQW8xNDdhSU9IUDVLXzFhV2N0UkNRMkZrQXxBQ3Jtc0traUxHMzZJcGJQYjlyckYxaGxYVWlsOFNGUFlFVEdhNzdjTWpPUlQ2TF9XczRqNkxMVGpJTnd5YmYzV0prQ0IwZURNcHhIZ3h1Z051VTl5MXBBLUN0dkM0NHRkQTFua1Jpc0VCRFJUb0ZQZG95b0JqMA&q=https%3A%2F%2Flearn.activeloop.ai%2Fcourses%2Flangchain&v=gKUTDC13jys) by Activeloop
|
||||
|
||||
2. [Reinforcement Learning from Human Feedback](https://learn.deeplearning.ai/reinforcement-learning-from-human-feedback) by DeepLearning.AI
|
||||
|
||||
3. [Building Applications with Vector Databases](https://learn.deeplearning.ai/building-applications-vector-databases) by DeepLearning.AI
|
||||
|
||||
4. [Finetuning Large Language Models](https://learn.deeplearning.ai/finetuning-large-language-models) by Deeplearning.AI
|
||||
5. [LangChain: Chat with Your Data](https://learn.deeplearning.ai/langchain-chat-with-your-data/) by Deeplearning.AI
|
||||
|
||||
6. [Building Systems with the ChatGPT API](https://learn.deeplearning.ai/chatgpt-building-system) by Deeplearning.AI
|
||||
7. [Prompt Engineering with Llama 2](https://www.deeplearning.ai/short-courses/prompt-engineering-with-llama-2/) by Deeplearning.AI
|
||||
8. [Building Applications with Vector Databases](https://learn.deeplearning.ai/building-applications-vector-databases) by Deeplearning.AI
|
||||
9. [ChatGPT Prompt Engineering for Developers](https://learn.deeplearning.ai/chatgpt-prompt-eng/lesson/1/introduction) by Deeplearning.AI
|
||||
10. [Advanced RAG Orchestration series](https://www.youtube.com/watch?v=CeDS1yvw9E4) by LlamaIndex
|
||||
11. [Prompt Engineering Specialization](https://www.coursera.org/specializations/prompt-engineering) by Coursera
|
||||
12. [Augment your LLM Using Retrieval Augmented Generation](https://courses.nvidia.com/courses/course-v1:NVIDIA+S-FX-16+v1/) by Nvidia
|
||||
13. [Knowledge Graphs for RAG](https://www.deeplearning.ai/short-courses/knowledge-graphs-rag/) by Deeplearning.AI
|
||||
14. [Open Source Models with Hugging Face](https://www.deeplearning.ai/short-courses/open-source-models-hugging-face/) by Deeplearning.AI
|
||||
15. [Vector Databases: from Embeddings to Applications](https://www.deeplearning.ai/short-courses/vector-databases-embeddings-applications/) by Deeplearning.AI
|
||||
16. [Understanding and Applying Text Embeddings](https://www.deeplearning.ai/short-courses/google-cloud-vertex-ai/) by Deeplearning.AI
|
||||
17. [JavaScript RAG Web Apps with LlamaIndex](https://www.deeplearning.ai/short-courses/javascript-rag-web-apps-with-llamaindex/) by Deeplearning.AI
|
||||
18. [Quantization Fundamentals with Hugging Face](https://www.deeplearning.ai/short-courses/quantization-fundamentals-with-hugging-face/) by Deeplearning.AI
|
||||
19. [Preprocessing Unstructured Data for LLM Applications](https://www.deeplearning.ai/short-courses/preprocessing-unstructured-data-for-llm-applications/) by Deeplearning.AI
|
||||
20. [Retrieval Augmented Generation for Production with LangChain & LlamaIndex](https://learn.activeloop.ai/courses/rag) by Activeloop
|
||||
21. [Quantization in Depth](https://www.deeplearning.ai/short-courses/quantization-in-depth/) by Deeplearning.AI
|
||||
|
||||
##### Evaluation
|
||||
|
||||
1. [Building and Evaluating Advanced RAG Applications](https://learn.deeplearning.ai/building-evaluating-advanced-rag) by DeepLearning.AI
|
||||
2. [Evaluating and Debugging Generative AI Models Using Weights and Biases](https://learn.deeplearning.ai/evaluating-debugging-generative-ai) by Deeplearning.AI
|
||||
3. [Quality and Safety for LLM Applications](https://www.deeplearning.ai/short-courses/quality-safety-llm-applications/) by Deeplearning.AI
|
||||
4. [Red Teaming LLM Applications](https://www.deeplearning.ai/short-courses/red-teaming-llm-applications/?utm_campaign=giskard-launch&utm_medium=headband&utm_source=dlai-homepage) by Deeplearning.AI
|
||||
|
||||
##### Multimodal
|
||||
|
||||
1. [How Diffusion Models Work](https://www.deeplearning.ai/short-courses/how-diffusion-models-work/) by DeepLearning.AI
|
||||
2. [How to Use Midjourney, AI Art and ChatGPT to Create an Amazing Website](https://www.youtube.com/watch?v=5wdCev86RYE) by Brad Hussey
|
||||
3. [Build AI Apps with ChatGPT, DALL-E and GPT-4](https://scrimba.com/learn/buildaiapps) by Scrimba
|
||||
4. [11-777: Multimodal Machine Learning](https://www.youtube.com/playlist?list=PL-Fhd_vrvisNM7pbbevXKAbT_Xmub37fA) by Carnegie Mellon University
|
||||
5. [Prompt Engineering for Vision Models](https://www.deeplearning.ai/short-courses/prompt-engineering-for-vision-models/) by Deeplearning.AI
|
||||
|
||||
##### Agents
|
||||
1. [Building RAG Agents with LLMs](https://courses.nvidia.com/courses/course-v1:DLI+S-FX-15+V1/) by Nvidia
|
||||
2. [Functions, Tools and Agents with LangChain](https://learn.deeplearning.ai/functions-tools-agents-langchain) by Deeplearning.AI
|
||||
3. [AI Agents in LangGraph](https://www.deeplearning.ai/short-courses/ai-agents-in-langgraph/) by Deeplearning.AI
|
||||
4. [AI Agentic Design Patterns with AutoGen](https://www.deeplearning.ai/short-courses/ai-agentic-design-patterns-with-autogen/) by Deeplearning.AI
|
||||
5. [Multi AI Agent Systems with crewAI](https://www.deeplearning.ai/short-courses/multi-ai-agent-systems-with-crewai/) by Deeplearning.AI
|
||||
6. [Building Agentic RAG with LlamaIndex](https://www.deeplearning.ai/short-courses/building-agentic-rag-with-llamaindex/) by Deeplearning.AI
|
||||
7. [LLM Observability: Agents, Tools, and Chains](https://courses.arize.com/p/agents-tools-and-chains) by Arize AI
|
||||
8. [Building Agentic RAG with LlamaIndex](https://www.deeplearning.ai/short-courses/building-agentic-rag-with-llamaindex/) by Deeplearning.AI
|
||||
9. [Agents Tools & Function Calling with Amazon Bedrock (How-to)](https://www.youtube.com/watch?app=desktop&v=2L_XE6g3atI) by AWS Developers
|
||||
10. [ChatGPT & Zapier: Agentic AI for Everyone](https://www.coursera.org/learn/agentic-ai-chatgpt-zapier) by Coursera
|
||||
11. [Multi-Agent Systems with AutoGen](https://www.manning.com/books/multi-agent-systems-with-autogen-cx) by Victor Dibia [Book]
|
||||
12. [Large Language Model Agents MOOC, Fall 2024](https://llmagents-learning.org/f24) by Dawn Song & Xinyun Chen – A comprehensive course covering foundational and advanced topics on LLM agents.
|
||||
13. [CS294/194-196 Large Language Model Agents](https://rdi.berkeley.edu/llm-agents/f24) by UC Berkeley
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
#### Miscellaneous
|
||||
|
||||
1. [Avoiding AI Harm](https://www.coursera.org/learn/avoiding-ai-harm) by Coursera
|
||||
2. [Developing AI Policy](https://www.coursera.org/learn/developing-ai-policy) by Coursera
|
||||
|
||||
---
|
||||
|
||||
## :paperclip: Resources
|
||||
|
||||
- [ICLR 2024 Paper Summaries](https://areganti.notion.site/06f0d4fe46a94d62bff2ae001cfec22c?v=d501ca62e4b745768385d698f173ae14)
|
||||
|
||||
---
|
||||
|
||||
## :computer: Interview Prep
|
||||
|
||||
#### Topic wise Questions:
|
||||
|
||||
1. [Common GenAI Interview Questions](https://github.com/aishwaryanr/awesome-generative-ai-guide/blob/main/interview_prep/60_gen_ai_questions.md)
|
||||
2. Prompting and Prompt Engineering
|
||||
3. Model Fine-Tuning
|
||||
4. Model Evaluation
|
||||
5. MLOps for GenAI
|
||||
6. Generative Models Foundations
|
||||
7. Latest Research Trends
|
||||
|
||||
#### GenAI System Design (Coming Soon):
|
||||
|
||||
1. Designing an LLM-Powered Search Engine
|
||||
2. Building a Customer Support Chatbot
|
||||
3. Building a system for natural language interaction with your data.
|
||||
4. Building an AI Co-pilot
|
||||
5. Designing a Custom Chatbot for Q/A on Multimodal Data (Text, Images, Tables, CSV Files)
|
||||
6. Building an Automated Product Description and Image Generation System for E-commerce
|
||||
|
||||
---
|
||||
|
||||
## :notebook: Code Notebooks
|
||||
|
||||
#### RAG Tutorials
|
||||
|
||||
- [AWS Bedrock Workshop Tutorials](https://github.com/aws-samples/amazon-bedrock-workshop) by Amazon Web Services
|
||||
- [Langchain Tutorials](https://github.com/gkamradt/langchain-tutorials) by gkamradt
|
||||
- [LLM Applications for production](https://github.com/ray-project/llm-applications/tree/main) by ray-project
|
||||
- [LLM tutorials](https://github.com/ollama/ollama/tree/main/examples) by Ollama
|
||||
- [LLM Hub](https://github.com/mallahyari/llm-hub) by mallahyari
|
||||
- [RAG cookbook](https://docs.camel-ai.org/cookbooks/agents_with_rag.html) by CAMEL-AI
|
||||
|
||||
#### Fine-Tuning Tutorials
|
||||
|
||||
- [LLM Fine-tuning tutorials](https://github.com/ashishpatel26/LLM-Finetuning) by ashishpatel26
|
||||
- [PEFT](https://github.com/huggingface/peft/tree/main/examples) example notebooks by Huggingface
|
||||
- [Free LLM Fine-Tuning Notebooks](https://levelup.gitconnected.com/14-free-large-language-models-fine-tuning-notebooks-532055717cb7) by Youssef Hosni
|
||||
|
||||
|
||||
#### Comprehensive LLM Code Repositories
|
||||
- [LLM-PlayLab](https://github.com/Sakil786/LLM-PlayLab) This playlab encompasses a multitude of projects crafted through the utilization of Transformer Models
|
||||
- [RAG Techniques](https://github.com/NirDiamant/RAG_Techniques) by Nir Diamant — 35+ runnable Jupyter notebooks covering advanced RAG techniques (chunking, query transformation/HyDE, reranking, self-RAG, graph RAG, evaluation)
|
||||
- [GenAI Agents](https://github.com/NirDiamant/GenAI_Agents) by Nir Diamant — 50+ tutorials and reference implementations for building GenAI agents, from simple bots to multi-agent systems
|
||||
|
||||
|
||||
---
|
||||
|
||||
## :black_nib: Contributing
|
||||
|
||||
If you want to add to the repository or find any issues, please feel free to raise a PR and ensure correct placement within the relevant section or category.
|
||||
|
||||
---
|
||||
|
||||
## :pushpin: Cite Us
|
||||
|
||||
To cite this guide, use the below format:
|
||||
|
||||
```
|
||||
@article{areganti_generative_ai_guide,
|
||||
author = {Reganti, Aishwarya Naresh},
|
||||
journal = {https://github.com/aishwaryanr/awesome-generative-ai-resources},
|
||||
month = {01},
|
||||
title = {{Generative AI Guide}},
|
||||
year = {2024}
|
||||
}
|
||||
```
|
||||
|
||||
## License
|
||||
|
||||
[MIT License]
|
||||
|
||||
|
||||
|
||||
<sup>**</sup> This section is sponsored. We do not endorse or guarantee the product/service and are not responsible for any issues arising from its use. Please evaluate and use at your discretion.
|
||||
|
||||
## Security & Safety Tools
|
||||
|
||||
- **[OWASP Agent Memory Guard](https://github.com/OWASP/www-project-agent-memory-guard)** - Official OWASP reference implementation for AI agent memory poisoning defense (ASI06 from OWASP Top 10 for Agentic AI Systems). Provides pre-write scanning, pre-read validation, and audit logging for agent memory.
|
||||
@@ -0,0 +1,7 @@
|
||||
# WeHub 来源说明
|
||||
|
||||
- 原始项目：`aishwaryanr/awesome-generative-ai-guide`
|
||||
- 原始仓库：https://github.com/aishwaryanr/awesome-generative-ai-guide
|
||||
- 导入方式：上游默认分支的最新快照
|
||||
- 原作者、版权和许可证信息以原始仓库及本仓库 LICENSE 为准
|
||||
- 本文件仅用于记录来源，不代表 WeHub 是原项目作者
|
||||
@@ -0,0 +1,63 @@
|
||||
# Applied LLMs Mastery 2024
|
||||
|
||||
|
||||
|
||||
# About This Course
|
||||
|
||||
|
||||
**[Official Course Website](https://areganti.notion.site/The-LevelUp-Org-Applied-LLMs-562ddaa27791463e9a1286199325045c)**
|
||||
|
||||
Welcome to an exciting 10-week journey into the world of large language models!
|
||||
|
||||
LLMs are currently experiencing a substantial surge in popularity. Their significance has notably increased in diverse applications, including natural language processing, machine translation, and code and text generation. This rise in prominence is driven by a growing trend among both companies and individuals to leverage LLMs for automating a wide range of tasks. Understanding and learning about LLMs is highly valuable in light of their growing usage and transformative impact across various domains.
|
||||
|
||||
If you're eager to dive into this trend, you'll discover plenty of resources on the internet. But here's the catch – many of them are all over the place, missing a step-by-step guide from basics to real-world use. This can be overwhelming, and you might feel a bit lost.
|
||||
|
||||
Imagine this course as your comprehensive guide, exploring every aspect of using LLMs in real-world scenarios. It serves as the crucial link that brings everything together. Each week, we'll delve into the above topics, providing in-depth insights and hands-on experiences. This approach ensures you gain a thorough and well-rounded understanding of every facet within the topic.
|
||||
|
||||
We've organized the content into four key pillars –
|
||||
|
||||
- **Fundamentals** (Week 1)
|
||||
- **Tools and Techniques** (Weeks 2-5)
|
||||
- **Deployment and Evaluation** (Weeks 6-9)
|
||||
- **Challenges and Future trends** (Weeks 9-10)
|
||||
|
||||
This course caters to a diverse audience, including business leaders, professionals, computer science enthusiasts, or students looking to enhance their knowledge in LLMs. While we aim to keep mathematical foundations relatively light, we'll touch on LLM architectural basics in week 11 as bonus content for those interested in delving into LLM research.
|
||||
|
||||
# Course Format
|
||||
|
||||
To make this course accessible to a wide audience, we've designed it as a self-paced audit course. You can register for the course here and course material will be released weekly, featuring mind maps, "ETMI5: Explain to Me in 5" sections for a quick overview, relevant resources, and comprehensive content to ensure your understanding of each topic. Additionally, we'll provide research papers and distilled summaries to keep you updated on the latest research. This page serves as your central hub for all resources.
|
||||
|
||||
Stay informed by registering for email notifications whenever new content is uploaded, or follow our updates on [LinkedIn](https://www.linkedin.com/in/areganti/). For any queries, feel free to contact the instructor at ***aish@levelup4all.org*** or on LinkedIn. At the end of each week, we'll address frequently asked questions. To maximize your learning experience, allocate 2-3 hours weekly for reading content and engaging in suggested hands-on experiments.
|
||||
|
||||
|
||||
# Key Takeaways
|
||||
|
||||
- Understanding the practical fundamentals of LLMs, including its capabilities and limitations
|
||||
- Hands-on experience with end-to-end execution of LLM use cases
|
||||
- Learning best practices for exploring and evaluating the usefulness of LLMs in specific scenarios
|
||||
- Proficiency in integrating and comprehending new updates in LLMs, effectively fitting each piece into the larger puzzle and understanding its relevance.
|
||||
|
||||
|
||||
|
||||
# Disclaimer
|
||||
|
||||
This course content is developed by [Aishwarya Naresh Reganti](https://www.linkedin.com/in/areganti/). The course is offered independently, for **free** and is not affiliated with her professional responsibilities or employer. The content presented in this course is intended for educational purposes only and does not reflect the views or policies of any associated organizations.
|
||||
|
||||
|
||||
|
||||
To cite this guide, use the below format:
|
||||
|
||||
```
|
||||
@article{areganti_generative_ai_guide,
|
||||
author = {Reganti, Aishwarya},
|
||||
journal = {https://github.com/aishwaryanr/awesome-generative-ai-resources},
|
||||
month = {01},
|
||||
title = {{Generative AI Guide}},
|
||||
year = {2024}
|
||||
}
|
||||
```
|
||||
|
||||
## License
|
||||
|
||||
[MIT License]
|
||||
BIN
Binary file not shown.
|
After Width: | Height: | Size: 385 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 471 KiB |
Binary file not shown.
Binary file not shown.
|
After Width: | Height: | Size: 211 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 193 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 106 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 212 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 86 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 86 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 87 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 298 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 50 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 34 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 22 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 14 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 64 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 102 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 55 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 159 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 278 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 295 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 33 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 134 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 190 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 192 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 27 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 47 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 64 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 147 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 1013 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 343 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 52 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 273 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 285 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 361 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 755 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 199 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 103 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 172 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 1.3 MiB |
Binary file not shown.
|
After Width: | Height: | Size: 150 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 186 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 486 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 357 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 233 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 1.0 MiB |
Binary file not shown.
|
After Width: | Height: | Size: 420 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 421 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 503 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 142 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 848 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 285 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 747 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 292 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 422 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 35 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 369 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 321 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 1.8 MiB |
Binary file not shown.
|
After Width: | Height: | Size: 199 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 574 KiB |
@@ -0,0 +1,279 @@
|
||||
# [Week 10] Emerging Research Trends
|
||||
|
||||
## ETMI5: Explain to Me in 5
|
||||
|
||||
Within this segment of our course, we will delve into the latest research developments surrounding LLMs. Kicking off with an examination of MultiModal Large Language Models (MM-LLMs), we'll explore how this particular area is advancing swiftly. Following that, our discussion will extend to popular open-source models, focusing on their construction and contributions. Subsequently, we'll tackle the concept of agents that possess the capability to carry out tasks autonomously from inception to completion. Additionally, we'll understand the role of domain-specific models in enriching specialized knowledge across various sectors and take a closer look at groundbreaking architectures such as the Mixture of Experts and RWKV, which are set to improve the scalability and efficiency of LLMs.
|
||||
|
||||
## Multimodal LLMs (MM-LLMs)
|
||||
|
||||
In the past year, there have been notable advancements in MultiModal Large Language Models (MM-LLMs). Specifically, MM-LLMs represent a significant evolution in the space of language models, as they incorporate multimodal components alongside their text processing capabilities. While progress has also been made in multimodal models in general, MM-LLMs have experienced particularly substantial improvements, largely due to the remarkable enhancements in LLMs over the year, upon which they heavily rely.
|
||||
|
||||
Moreover, the development of MM-LLMs has been greatly aided by the adoption of cost-effective training strategies. These strategies have enabled these models to efficiently manage inputs and outputs across multiple modalities. Unlike conventional models, MM-LLMs not only retain the impressive reasoning and decision-making capabilities inherent in Large Language Models but also expand their utility to address a diverse array of tasks spanning various modalities.
|
||||
|
||||
To understand how MM-LLMs function, we can go over some common architectural components. Most MM-LLMs can be divided in 5 main components as shown in the image below. The components explained below are adapted from the paper “[MM-LLMs: Recent Advances in MultiModal Large Language Models](https://arxiv.org/pdf/2401.13601.pdf)”. Let’s understand each of the components in detail.
|
||||
|
||||
|
||||
|
||||
Image Source: [https://arxiv.org/pdf/2401.13601.pdf](https://arxiv.org/pdf/2401.13601.pdf)
|
||||
|
||||
**1. Modality Encoder:** The Modality Encoder (ME) plays a pivotal role in encoding inputs from diverse modalities $I_X$ to extract corresponding features $F_X$ Various pre-trained encoder options exist for different modalities, including visual, audio, and 3D inputs. For visual inputs, options like NFNet-F6, ViT, CLIP ViT, and Eva-CLIP ViT are commonly employed. Similarly, for audio inputs, frameworks such as CFormer, HuBERT, BEATs, and Whisper are utilized. Point cloud inputs are encoded using ULIP-2 with a PointBERT backbone. Some MM-LLMs leverage ImageBind, a unified encoder covering multiple modalities, including image, video, text, audio, and heat maps.
|
||||
|
||||
**2. Input Projector:** The Input Projector $Θ_(X→T)$ aligns the encoded features of other modalities $F_X$ with the text feature space $T$. This alignment is crucial for effectively integrating multimodal information into the LLM Backbone. The Input Projector can be implemented through various methods such as Linear Projectors, Multi-Layer Perceptrons (MLPs), Cross-attention, Q-Former, or P-Former, each with its unique approach to aligning features across modalities.
|
||||
|
||||
**3. LLM Backbone:** The LLM Backbone serves as the core agent in MM-LLMs, inheriting notable properties from LLMs such as zero-shot generalization, few-shot In-Context Learning (ICL), Chain-of-Thought (CoT), and instruction following. The backbone processes representations from various modalities, engaging in semantic understanding, reasoning, and decision-making regarding the inputs. Additionally, some MM-LLMs incorporate Parameter-Efficient Fine-Tuning (PEFT) methods like Prefix-tuning, Adapter, or LoRA to minimize the number of additional trainable parameters.
|
||||
|
||||
**4. Output Projector:** The Output Projector $Θ_(T→X)$ maps signal token representations $S_X$from the LLM Backbone into features $H_X$ understandable to the Modality Generator $MG_X$. This projection facilitates the generation of multimodal content. The Output Projector is typically implemented using a Tiny Transformer or MLP, and its optimization focuses on minimizing the distance between the mapped features $H_X$ and the conditional text representations of $MG_X$ .
|
||||
|
||||
**5. Modality Generator:** The Modality Generator $MG_X$ is responsible for producing outputs in distinct modalities such as images, videos, or audio. Commonly, existing works leverage off-the-shelf Latent Diffusion Models (LDMs) for image, video, and audio synthesis. During training, ground truth content is transformed into latent features, which are then de-noised to generate multimodal content using LDMs conditioned on the mapped features $H_X$ from the Output Projector.
|
||||
|
||||
### Training
|
||||
|
||||
MM-LLMs are trained in two main stages: MultiModal Pre-Training (MM PT) and MultiModal Instruction-Tuning (MM IT).
|
||||
|
||||
**MM PT:**
|
||||
During MM PT, MM-LLMs are trained to understand and generate content from different types of data like images, videos, and text. They learn to align these different kinds of information to work together. For example, they learn to associate a picture of a cat with the word "cat" and vice versa. This stage focuses on teaching the model to handle different types of input and output.
|
||||
|
||||
**MM IT:**
|
||||
In MM IT, the model is fine-tuned based on specific instructions. This helps the model adapt to new tasks and perform better on them. There are two main methods used in MM IT:
|
||||
|
||||
- **Supervised Fine-Tuning (SFT):** The model is trained on examples that are structured in a way that includes instructions. For instance, in a question-answer task, each question is paired with the correct answer. This helps the model learn to follow instructions and generate appropriate responses.
|
||||
- **Reinforcement Learning from Human Feedback (RLHF):** The model receives feedback on its responses, usually in the form of human-generated feedback. This feedback helps the model improve its performance over time by learning from its mistakes.
|
||||
|
||||
Therefore MM-LLMs are trained to understand and generate content from multiple sources of information, and they can be fine-tuned to perform specific tasks better based on instructions and feedback.
|
||||
|
||||
The below diagram summarizes popular MM-LLMs and models used for each of their components.
|
||||
|
||||
|
||||
|
||||
Image Source: [https://arxiv.org/pdf/2401.13601.pdf](https://arxiv.org/pdf/2401.13601.pdf)
|
||||
|
||||
### Emerging Research Directions
|
||||
|
||||
Some potential future directions for MM-LLMs involve extending their capabilities through various avenues:
|
||||
|
||||
1. **More Powerful Models**:
|
||||
- Extend MM-LLMs to accommodate additional modalities beyond the current ones like image, video, audio, 3D, and text, such as web pages, heat maps, and figures/tables.
|
||||
- Incorporate various types and sizes of LLMs to provide practitioners with flexibility in selecting the most suitable one for their specific requirements.
|
||||
- Enhance MM IT datasets by diversifying the range of instructions to improve MM-LLMs' understanding and execution of user commands.
|
||||
- Explore integrating retrieval-based approaches to complement generative processes in MM-LLMs, potentially enhancing overall performance.
|
||||
2. **More Challenging Benchmarks**:
|
||||
- Develop larger-scale benchmarks that include a wider range of modalities and use unified evaluation standards to adequately challenge the capabilities of MM-LLMs.
|
||||
- Tailor benchmarks to assess MM-LLMs' proficiency in practical applications, such as evaluating their ability to discern and respond to nuanced aspects of social abuse presented in memes.
|
||||
3. **Mobile/Lightweight Deployment**:
|
||||
- Develop lightweight implementations to deploy MM-LLMs on resource-constrained platforms like low-power mobile and IoT devices, ensuring optimal performance.
|
||||
4. **Embodied Intelligence**:
|
||||
- Explore embodied intelligence to replicate human-like perception and interaction with the surroundings, enabling robots to autonomously implement extended plans based on real-time observations.
|
||||
- Further enhance MM-LLM-based embodied intelligence to improve the autonomy of robots, building on existing advancements like PaLM-E and EmbodiedGPT.
|
||||
5. **Continual IT**:
|
||||
- Develop approaches for MM-LLMs to continually adapt to new MM tasks while maintaining superior performance on previously learned tasks, addressing challenges such as catastrophic forgetting and negative forward transfer.
|
||||
- Establish benchmarks and develop methods to overcome challenges in continual IT for MM-LLMs, ensuring efficient adaptation to emerging requirements without substantial retraining costs.
|
||||
|
||||
## Open-Source Models
|
||||
|
||||
Recent developments in open-source LLMs have been pivotal in democratizing access to advanced AI technologies. Open-source LLMs offer several advantages over closed-source models, enhancing transparency, customizability, and collaboration. They allow for a deeper understanding of model workings, enable modifications to suit specific needs, and encourage improvements through community contributions. They also serve as educational tools and support a diverse AI ecosystem, preventing monopolies. However, challenges such as computational demands and potential misuse exist, but the benefits of open-source models often outweigh these issues, especially for those valuing openness and adaptability in AI development.
|
||||
|
||||
A few popular Open-Source LLMs are listed below:
|
||||
|
||||
### **LLaMA by Meta**
|
||||
|
||||
- **LLaMA** (13B parameters) was released by Meta in February 2023, outperforming GPT-3 on many NLP benchmarks despite having fewer parameters. **LLaMA-2**, an enhanced version with 40% more data and doubled context length, was released in July 2023 along with specialized versions for conversations (**LLaMA 2-Chat**) and code generation (**LLaMA Code**).
|
||||
|
||||
### **Mistral**
|
||||
|
||||
- Developed by a Paris-based startup, **Mistral 7B** set new benchmarks by outperforming all existing open-source LLMs up to 13B parameters in English and code benchmarks. Mistral AI later also released **Mixtral 8x7B**, a Sparse Mixture of Experts (SMoE) model. This model marks a departure from traditional AI architectures and training methods, aiming to provide the developer community with innovative tools that can inspire new applications and technologies. We’ll learn more about the Mixture of Experts paradigm in the next serction
|
||||
|
||||
### **Open Language Model (OLMo)**
|
||||
|
||||
- **OLMo** is part of the AI2 LLM framework aimed at encouraging open research by providing access to training data, code, models, and evaluation tools. It includes the **Dolma dataset**, comprehensive training and inference code, model weights for four 7B scale variants, and an extensive evaluation suite under the Catwalk project.
|
||||
|
||||
### **LLM360 Initiative**
|
||||
|
||||
- **LLM360** proposes a fully open-source approach to LLM development, advocating for the release of training code, data, model checkpoints, and intermediate results. It released two 7B parameter LLMs, **AMBER** and **CRYSTALCODER**, complete with resources for transparency and reproducibility in LLM training.
|
||||
|
||||
While Llama and Mistral only release their models, OLMo and LLM360 go further by providing checkpoints, datasets, and more, ensuring their offerings are fully open and capable of being reproduced.
|
||||
|
||||
## Agents
|
||||
|
||||
LLM Agents have been gaining significant momentum in recent months and represent the future and expansion of LLM capabilities. An LLM agent is an AI system that employs a large language model at its core to perform a wide range of tasks, not limited to text generation. These tasks include conducting conversations, reasoning, completing various tasks, and exhibiting autonomous behaviors based on the context and instructions provided. LLM agents operate through sophisticated prompt engineering, where instructions, context, and permissions are encoded to guide the agent's actions and responses.
|
||||
|
||||
### **Capabilities of LLM Agents**
|
||||
|
||||
- **Autonomy**: LLM agents can operate with varying degrees of autonomy, from reactive to proactive behaviors, based on their design and the prompts they receive.
|
||||
- **Task Completion**: With access to external knowledge bases, tools, and reasoning capabilities, LLM agents can assist in or independently handle a variety of applications, from chatbots to complex workflow automation.
|
||||
- **Adaptability**: Their language modeling strength allows them to understand and follow natural language prompts, making them versatile and capable of customizing their responses and actions.
|
||||
- **Advanced Skills**: Through prompt engineering, LLM agents can be equipped with advanced analytical, planning, and execution skills. They can manage tasks with minimal human intervention, relying on their ability to access and process information.
|
||||
- **Collaboration**: They enable seamless collaboration between humans and AI by responding to interactive prompts and integrating feedback into their operations.
|
||||
|
||||
LLM agents combine the core language processing capabilities of LLMs with additional modules like planning, memory, and tool usage, effectively becoming the "brain" that directs a series of operations to fulfill tasks or respond to queries. This architecture allows them to break down complex questions into manageable parts, retrieve and analyze relevant information, and generate comprehensive responses or visual representations as needed.
|
||||
|
||||
Example:
|
||||
|
||||
Suppose we're interested in organizing an international conference on sustainable energy solutions, aiming to cover topics such as renewable energy technologies, sustainability practices in energy production, and innovative policies for promoting green energy. The task involves complex planning and information gathering, including identifying key speakers, understanding current trends in sustainable energy, and engaging with stakeholders.
|
||||
|
||||
To tackle this multifaceted project, an LLM agent could be employed to:
|
||||
|
||||
1. **Research and Summarization**: Break down the task into sub-tasks such as identifying emerging trends in sustainable energy, locating leading experts in the field, and summarizing recent research findings. The agent would use its access to a vast range of digital resources to compile comprehensive reports.
|
||||
2. **Speaker Engagement**: Draft personalized invitations to potential speakers, incorporating details about the conference's aims and how their expertise aligns with its goals. The agent can generate these communications based on profiles and previous works of the experts.
|
||||
3. **Logistics Planning**: Create a detailed plan for the conference, including a timeline of activities leading up to the event, a checklist for logistical arrangements (venue, virtual platform setup for hybrid participation, etc.), and a strategy for participant engagement. The agent can outline these plans by accessing databases of event planning resources and best practices.
|
||||
4. **Stakeholder Communication**: Draft updates and newsletters for stakeholders, providing insights into the conference's progress, highlights of the agenda, and key speakers confirmed. The agent tailors each communication piece to its audience, whether it's sponsors, participants, or the general public.
|
||||
5. **Interactive Q&A Session Planning**: Develop a framework for an interactive Q&A session, including pre-gathering questions from potential attendees, categorizing them, and preparing briefing documents for speakers. The agent can facilitate this by analyzing registration data and submitted queries.
|
||||
|
||||
In this scenario, the LLM agent not only aids in the execution of complex and time-consuming tasks but also ensures that the planning process is thorough, informed by the latest developments in sustainable energy, and tailored to the specific goals of the conference. By leveraging external databases, tools for data analysis and visualization, and its innate language processing capabilities, the LLM agent acts as a comprehensive assistant, streamlining the organization of a large-scale event with numerous moving parts.
|
||||
|
||||
The framework for LLM agents can be conceptualized through various lenses, and one such perspective is offered by the paper “[A Survey on Large Language Model based Autonomous Agents](https://arxiv.org/pdf/2308.11432.pdf)”, through its distinctive components. This architecture is composed of four key modules: the Profiling Module, Memory Module, Planning Module, and Action Module. Each of these modules plays a crucial role in enabling the LLM agent to act autonomously and effectively in various scenarios.
|
||||
|
||||
|
||||
|
||||
Image Source : [https://arxiv.org/pdf/2308.11432.pdf](https://arxiv.org/pdf/2308.11432.pdf)
|
||||
|
||||
### **Components of LLM Agents**
|
||||
|
||||
1. **Profiling Module**
|
||||
|
||||
The Profiling Module is responsible for defining the agent's identity and role. It incorporates information such as age, gender, career, personality traits, and social relationships to shape the agent's behavior. This module uses various methods to create profiles, including handcrafting for precise control, LLM-generation for scalability, and dataset alignment for real-world accuracy. The agent's profile significantly influences its interactions, decision-making processes, and the way it executes tasks, making this module foundational to the agent's design.
|
||||
|
||||
**2. Memory Module**
|
||||
|
||||
The Memory Module stores information the agent perceives from its environment and uses this stored knowledge to inform future actions. It mimics human memory processes, with structures inspired by sensory, short-term, and long-term memory. This module enables the agent to accumulate experiences, evolve based on past interactions, and behave in a consistent and effective manner. It ensures that the agent can recall past behaviors, learn from them, and adapt its strategies over time.
|
||||
|
||||
**3. Planning Module**
|
||||
|
||||
The Planning Module empowers the agent with the ability to decompose complex tasks into simpler subtasks and address them individually, mirroring human problem-solving strategies. It includes planning both with and without feedback, allowing for flexible adaptation to changing environments and requirements. Strategies such as single-path reasoning and Chain of Thought (CoT) are used to guide the agent in a step-by-step manner towards achieving its goals, making the planning process critical for the agent's effectiveness and reliability.
|
||||
|
||||
**4. Action Module**
|
||||
|
||||
The Action Module translates the agent's decisions into specific outcomes, directly interacting with the environment. It considers the goals of the actions, how actions are generated, the range of possible actions (action space), and the consequences of these actions. This module integrates inputs from the profiling, memory, and planning modules to execute decisions that align with the agent's objectives and capabilities. It is essential for the practical application of the agent's strategies, enabling it to produce tangible results in the real world.
|
||||
|
||||
Together, these modules form a comprehensive framework for LLM agent architecture, allowing for the creation of agents that can assume specific roles, perceive and learn from their environment, and autonomously execute tasks with a degree of sophistication and flexibility that mimics human behavior.
|
||||
|
||||
### Future Research Directions
|
||||
|
||||
1. Most LLM Agent research has been confined to text-based interactions. Expanding into multi-modal environments, where agents can process and generate outputs across various formats like images, audio, and video, introduces complexities in data processing and requires agents to interpret and respond to a broader range of sensory inputs.
|
||||
2. Hallucination, where models generate factually incorrect text, becomes more problematic in LLM agent systems due to the potential for cascading misinformation. Developing strategies to detect and mitigate hallucinations involves managing information flow to prevent inaccuracies from spreading across the network.
|
||||
3. While LLM agents learn from instant feedback, creating reliable interactive environments for scalable learning poses challenges. Furthermore, current methods focus on adjusting agents individually, not fully leveraging the collective intelligence that could emerge from coordinated interactions among multiple agents.
|
||||
4. Scaling the number of agents (multi-agent systems) for a use-case raises significant computational demands and complexities in coordination and communication among agents. Developing efficient orchestration methodologies is essential for optimizing workflows and ensuring effective multi-agent cooperation.
|
||||
5. Current benchmarks may not adequately capture the emergent behaviors critical to agents or span across diverse research domains. Developing comprehensive benchmarks is crucial for assessing agents’ capabilities in various fields, including science, economics, and healthcare.
|
||||
|
||||
## Domain Specific LLMs
|
||||
|
||||
While general LLMs are versatile and perform well on a broad range of tasks, they often fall short when it comes to handling specialized or niche tasks due to a lack of training on domain-specific data. Additionally, running these generic models can be costly. In these scenarios, domain-specific LLMs emerge as a superior alternative. Their training is focused on data from specific fields, which enhances their accuracy and provides them with a deeper understanding of the relevant terminology and concepts. This tailored approach not only improves their performance on tasks specific to a certain domain but also minimizes the chances of generating irrelevant or incorrect information.
|
||||
|
||||
Designed to adhere to the regulatory and ethical standards of their respective domains, these models ensure the appropriate handling of sensitive data. They also communicate more effectively with domain experts, thanks to their command of professional language. From an economic standpoint, domain-specific LLMs offer more efficient solutions by eliminating the need for significant manual adjustments. Furthermore, their specialized knowledge base enables the identification of unique insights and patterns, driving innovation in their respective fields.
|
||||
|
||||
Some popular domain specific LLMs are listed below
|
||||
|
||||
### Popular Domain Specific LLMs
|
||||
|
||||
**Clinical and Biomedical LLMs**
|
||||
|
||||
- **BioBERT**: A domain-specific model pre-trained on large-scale biomedical corpora, designed to mine biomedical text effectively.
|
||||
- **Hi-BEHRT**: Offers a hierarchical Transformer-based structure for analyzing extended sequences in electronic health records, showcasing the model's ability to handle complex medical data.
|
||||
|
||||
**LLMs for Finance**
|
||||
|
||||
- **BloombergGPT**: A finance-specific model with 50 billion parameters, trained on a vast array of financial data, showing excellence in financial tasks.
|
||||
- **FinGPT**: A financial model fine-tuned with specific applications in mind, leveraging pre-existing LLMs for enhanced financial data understanding.
|
||||
|
||||
**Code-Specific LLMs**
|
||||
|
||||
- **WizardCoder**: Empowers Code LLMs with complex instruction fine-tuning, showcasing adaptability to coding domain challenges.
|
||||
- **CodeT5**: A unified pre-trained model focusing on the semantics conveyed in code, highlighting the importance of developer-assigned identifiers in understanding programming tasks.
|
||||
|
||||
These domain-specific LLMs illustrate the vast potential and adaptability of AI across different fields, from understanding multilingual content and processing clinical data to financial analysis and code generation. By honing in on the unique challenges and data types of each domain, these models open up new avenues for innovation, efficiency, and accuracy in AI applications.
|
||||
|
||||
### Future Trends for domain specific LLMs
|
||||
|
||||
1. Domain-specific LLMs will likely evolve to handle not just text but also images, audio, and other data types, enabling more comprehensive understanding and interaction capabilities across various formats.
|
||||
2. Future models may incorporate advanced interactive learning techniques, enabling them to update their knowledge base in real-time based on user feedback and new data, ensuring their outputs remain relevant and accurate.
|
||||
3. We might see an increase in systems where domain-specific LLMs work in concert with other AI technologies, such as decision-making algorithms and predictive models, to provide holistic solutions (Agents, like we discussed in the previous section)
|
||||
4. With growing awareness of AI's societal impact, the development of domain-specific LLMs will likely emphasize ethical considerations, fairness, and transparency, particularly in sensitive areas like healthcare and finance.
|
||||
|
||||
## New LLM Architectures
|
||||
|
||||
### Mixture of Experts
|
||||
|
||||
Mixture of Experts (MoEs) represents a sophisticated architecture within the realm of transformer models, focusing on enhancing model scalability and computational efficiency. Here's a breakdown of what MoEs are and their significance:
|
||||
|
||||
**Definition and Components**
|
||||
|
||||
- **MoEs in Transformers**: In transformer models, MoEs replace traditional dense feed-forward network (FFN) layers with sparse MoE layers. These layers comprise a number of "experts," each being a neural network—typically FFNs, but potentially more complex structures or even hierarchical MoEs.
|
||||
- **Experts**: These are specialized neural networks (often FFNs) that handle specific portions of the data. An MoE layer may contain several experts, such as 8, allowing for a diverse range of data processing capabilities within the same model layer.
|
||||
- **Gate Network/Router**: This is a critical component that directs input tokens to the appropriate experts based on learned parameters. The router decides, for instance, which expert is best suited to process a given input token, thus enabling a dynamic allocation of computational resources.
|
||||
|
||||
**Advantages**
|
||||
|
||||
- **Efficient Pretraining**: By utilizing MoEs, models can be pretrained with significantly less computational resources, allowing for larger model or dataset scales within the same compute budget as a dense model.
|
||||
- **Faster Inference**: Despite having a large number of parameters, MoEs only use a subset for inference, leading to quicker processing times compared to dense models with a similar parameter count. However, this efficiency comes with the caveat of high memory requirements due to the need to load all parameters into RAM.
|
||||
|
||||
**Challenges**
|
||||
|
||||
- **Training Generalization**: While MoEs are more compute-efficient during pretraining, they have historically faced challenges in generalizing well during fine-tuning, often leading to overfitting.
|
||||
- **Memory Requirements**: The efficient inference process of MoEs requires substantial memory to load the entire model's parameters, even though only a fraction are actively used during any given inference task.
|
||||
|
||||
**Implementation Details**
|
||||
|
||||
- **Parameter Sharing**: Not all parameters in a MoE model are exclusive to individual experts. Many are shared across the model, contributing to its efficiency. For instance, in a MoE model like Mixtral 8x7B, the dense equivalent parameter count might be less than the sum total of all experts due to shared components.
|
||||
- **Inference Speed**: The inference speed benefits stem from the model only engaging a subset of experts for each token, effectively reducing the computational load to that of a much smaller model, while maintaining the benefits of a large parameter space.
|
||||
|
||||
### Mamba Models
|
||||
|
||||
Mamba is an innovative recurrent neural network architecture that stands out for its efficiency in handling long sequences, potentially up to 1 million elements. This model has garnered attention for being a strong competitor to the well-known Transformer models due to its impressive scalability and faster processing capabilities. Here's a simplified overview of what Mamba is and why it's significant:
|
||||
|
||||
**Core Features of Mamba:**
|
||||
|
||||
- **Linear Time Processing**: Unlike Transformers, which suffer from computational and memory costs that scale quadratically with sequence length, Mamba operates in linear time. This makes it much more efficient, especially for very long sequences.
|
||||
- **Selective State Spaces**: Mamba employs selective state spaces, allowing it to manage and process lengthy sequences effectively by focusing on relevant parts of the data at any given time.
|
||||
|
||||
Selective State Spaces (SSS) in the context of models like Mamba refer to a sophisticated approach in neural network architecture that enables the model to efficiently handle and process very long sequences of data. This approach is particularly designed to improve upon the limitations of traditional models like Transformers and Recurrent Neural Networks (RNNs) when dealing with sequences of significant length. Here’s a breakdown of the key concepts behind Selective State Spaces:
|
||||
|
||||
**Basis of Selective State Spaces:**
|
||||
|
||||
- **State Space Models (SSMs)**: At the core, SSS builds upon the concept of State Space Models. SSMs are a class of models used for describing systems that evolve over time, capturing dynamics through state variables that change in response to external inputs. SSMs have been used in various fields, such as signal processing, control systems, and now, in sequence modeling for AI.
|
||||
- **Selectivity Mechanism**: The "selective" aspect introduces a mechanism that allows the model to determine which parts of the input sequence are relevant at any given time. This is achieved through a gating or routing function that dynamically selects which state space (or subset of the model's parameters) should be activated based on the input. This selective activation helps the model to focus its computational resources on the most pertinent parts of the data, enhancing efficiency.
|
||||
|
||||
**Advantages Over Traditional Models:**
|
||||
|
||||
- **Efficiency with Long Sequences**: Mamba's architecture is optimized for speed, offering up to five times faster throughput than Transformers while handling long sequences more effectively.
|
||||
- **Versatility**: While its prowess is evident in text-based applications like chatbots and summarization, Mamba also shows potential in other areas requiring the analysis of long sequences, such as audio generation, genomics, and time series data.
|
||||
- **Innovative Design**: The model builds on state space models (S4) but introduces a novel approach by incorporating selective structured state space sequence models, which enhance its processing capabilities.
|
||||
|
||||
Mamba represents a significant advancement in sequence modeling, offering a more efficient alternative to Transformers for tasks involving long sequences. Its ability to scale linearly with sequence length without a corresponding increase in computational and memory requirements makes it a promising tool for a wide range of applications beyond just natural language processing.
|
||||
|
||||
In essence, Mamba is redefining what's possible in AI sequence modeling, combining the best of RNNs and state space models with innovative techniques to achieve high efficiency and performance across various domains.
|
||||
|
||||
### **RWKV: Reinventing RNNs for the Transformer Era**
|
||||
|
||||
The RWKV architecture represents a novel approach in the realm of neural network models, integrating the strengths of Recurrent Neural Networks (RNNs) with the transformative capabilities of transformers. This hybrid architecture, spearheaded by Bo Peng and supported by a vibrant community, aims to address specific challenges in processing long sequences of data, making it particularly intriguing for various applications in Natural Language Processing (NLP) and beyond.
|
||||
|
||||
**Key Features of RWKV:**
|
||||
|
||||
- **Efficiency in Handling Long Sequences**: Unlike traditional transformers that struggle with quadratic computational and memory costs as sequence lengths increase, RWKV is designed to scale linearly. This makes it adept at efficiently processing sequences that are significantly longer than those manageable by conventional models.
|
||||
- **RNN and Transformer Hybrid**: RWKV combines RNNs' ability to handle sequential data with the transformer's powerful self-attention mechanism. This fusion aims to leverage the best of both worlds: the sequential data processing capability of RNNs and the context-aware, parallel processing strengths of transformers.
|
||||
- **Innovative Architecture**: RWKV introduces a simplified and optimized design that allows it to operate effectively as an RNN. It incorporates additional features such as TokenShift and SmallInitEmb to enhance performance, enabling it to achieve results comparable to those of GPT models.
|
||||
- **Scalability and Performance**: With the infrastructure to support training models up to 14B parameters and optimizations to overcome issues like numerical instability, RWKV presents a scalable and robust framework for developing advanced AI models.
|
||||
|
||||
**Advantages over Traditional Models:**
|
||||
|
||||
- **Handling Very Long Contexts**: RWKV can utilize contexts of thousands of tokens and beyond, surpassing traditional RNN limitations and enabling more comprehensive understanding and generation of text.
|
||||
- **Parallelized Training**: Unlike conventional RNNs that are challenging to parallelize, RWKV's architecture allows for faster training, akin to "linearized GPT," providing both speed and efficiency.
|
||||
- **Memory and Speed Efficiency**: RWKV models can be trained and run with long contexts without the significant RAM requirements of large transformers, offering a balance between computational resource use and model performance.
|
||||
|
||||
**Applications and Integration:**
|
||||
|
||||
RWKV's architecture makes it suitable for a wide range of applications, from pure language models to multi-modal tasks. Its integration into the Hugging Face Transformers library facilitates easy access and utilization by the AI community, supporting a variety of tasks including text generation, chatbots, and more.
|
||||
|
||||
In summary, RWKV represents an exciting development in AI research, combining RNNs' sequential processing advantages with the contextual awareness and efficiency of transformers. Its design addresses key challenges in long sequence modeling, offering a promising tool for advancing NLP and related fields.
|
||||
|
||||
## Read/Watch These Resources (Optional)
|
||||
|
||||
1. LLM Agents: [https://www.promptingguide.ai/research/llm-agents](https://www.promptingguide.ai/research/llm-agents)
|
||||
2. LLM Powered Autonomous Agents: [https://lilianweng.github.io/posts/2023-06-23-agent/](https://lilianweng.github.io/posts/2023-06-23-agent/)
|
||||
3. Emerging Trends in LLM Architecture- [https://medium.com/@bijit211987/emerging-trends-in-llm-architecture-a8897d9d987b](https://medium.com/@bijit211987/emerging-trends-in-llm-architecture-a8897d9d987b)
|
||||
4. Four LLM trends since ChatGPT and their implications for AI builders: [https://towardsdatascience.com/four-llm-trends-since-chatgpt-and-their-implications-for-ai-builders-a140329fc0d2](https://towardsdatascience.com/four-llm-trends-since-chatgpt-and-their-implications-for-ai-builders-a140329fc0d2)
|
||||
|
||||
## Read These Papers (Optional)
|
||||
|
||||
1. [https://arxiv.org/abs/2401.13601](https://arxiv.org/abs/2401.13601)
|
||||
2. [https://arxiv.org/abs/2312.00752](https://arxiv.org/abs/2312.00752)
|
||||
3. [https://arxiv.org/abs/2310.14724](https://arxiv.org/abs/2310.14724)
|
||||
4. [https://arxiv.org/abs/2307.06435](https://arxiv.org/abs/2307.06435)
|
||||
@@ -0,0 +1,155 @@
|
||||
# [Week 11] LLM Foundations
|
||||
|
||||
## ETMI5: Explain to Me in 5
|
||||
|
||||
In the first week of our course, we looked at the difference between two types of machine learning models: generative models, which LLMs are a part of, and discriminative models. Generative models are good at learning from data and creating new things. This week, we'll learn about how LLMs were developed by looking at the history of neural networks used in language processing. We start with the basics of Recurrent Neural Networks (RNNs) and move to more advanced architectures like sequence-to-sequence models, attention mechanisms, and transformers We'll also review some of the earlier language models that used transformers, like BERT and GPT. Finally, we'll talk about how the LLMs we use today were built on these earlier developments.
|
||||
|
||||
## Generative vs Discriminative models
|
||||
|
||||
In the first week, we briefly covered the idea of Generative AI. It's essential to note that all machine learning models fall into one of two categories: generative or discriminative. LLMs belong to the generative category, meaning they learn text features and produce them for various applications. While we won't delve deeply into the mathematical intricacies, it's important to grasp the distinctions between generative and discriminative models to gain a general understanding of how LLMs operate:
|
||||
|
||||
### **Generative Models**
|
||||
|
||||
Generative models try to understand how data is generated. They learn the patterns and structures in the data so they can create new similar data points.
|
||||
|
||||
For example, if you have a generative model for images of dogs, it learns what features and characteristics make up a dog (like fur, ears, and tails), and then it can generate new images of dogs that look realistic, even though they've never been seen before.
|
||||
|
||||
### **Discriminative Models**
|
||||
|
||||
Discriminative models, on the other hand, are focused on making decisions or predictions based on the input they receive.
|
||||
|
||||
Using the same example of images of dogs, a discriminative model would look at an image and decide whether it contains a dog or not. It doesn't worry about how the data was generated; it's just concerned with making the right decision based on the input it's given.
|
||||
|
||||
Therefore, Generative models learn the underlying patterns in the data to create new samples, while discriminative models focus on making decisions or predictions based on the input data without worrying about how the data was generated.
|
||||
|
||||
**Essentially, generative models create, while discriminative models classify or predict.**
|
||||
|
||||
## Neural Networks for Language
|
||||
|
||||
For several years, neural networks have been integral to machine learning. Among these, a prominent class of models heavily reliant on neural networks is referred to as deep learning models. The initial neural network type introduced for text generation was termed as a Recurrent Neural Network (RNN). Subsequent iterations with improvements emerged later, such as Long Short-Term Memory networks (LSTMs), Bidirectional LSTMs, and Gated Recurrent Units (GRUs). Now, let's explore how RNNs generate text.
|
||||
|
||||
### Recurrent Neural Network (RNN)
|
||||
|
||||
Recurrent Neural Networks (RNNs) are a type of artificial neural network designed to handle sequential data by allowing information to persist through loops within the network architecture. Traditional neural networks lack the ability to retain information over time, which can be a major limitation when dealing with sequential data like text, audio, or time-series data.
|
||||
|
||||
The basic principle behind RNNs is that they have connections that form a directed cycle, allowing information to be passed from one step of the network to the next. This means that the output of the network at a particular time step depends not only on the current input but also on the previous inputs and the internal state of the network, which captures information from earlier time steps.
|
||||
|
||||
|
||||
|
||||
Image Source: [https://colah.github.io/posts/2015-08-Understanding-LSTMs/](https://colah.github.io/posts/2015-08-Understanding-LSTMs/)
|
||||
|
||||
Here's a simplified explanation of how RNNs work:
|
||||
|
||||
1. **Input Processing**: At each time step $t$, the RNN receives an input $x_t$. This input could be a single element of a sequence (e.g., a word in a sentence) or a feature vector representing some aspect of the input data.
|
||||
2. **State Update**: The input $x_t$ is combined with the internal state $h_{t-1}$ of the network from the previous time step to produce a new state $h_t$ using a set of weighted connections (parameters) within the network. This update process allows the network to retain information from previous time steps.
|
||||
3. **Output Generation**: The current state $h_t$ is used to generate an output $y_t$ at the current time step. This output can be used for various tasks, such as classification, prediction, or sequence generation.
|
||||
4. **Recurrent Connections**: The key feature of RNNs is the presence of recurrent connections, which allow information to flow through the network over time. These connections create a form of memory within the network, enabling it to capture dependencies and patterns in sequential data.
|
||||
|
||||
While RNNs are powerful models for handling sequential data, they can suffer from certain limitations, such as difficulties in learning long-range dependencies and vanishing/exploding gradient problems during training. To address these issues, more advanced variants of RNNs, such as Long Short-Term Memory (LSTM) networks and Gated Recurrent Units (GRUs), have been developed. These architectures incorporate mechanisms for better handling long-term dependencies and mitigating gradient-related problems, leading to improved performance on a wide range of sequential data tasks.
|
||||
|
||||
### Long Short-Term Memory (LSTM)
|
||||
|
||||
LSTM networks are thus an enhanced version of RNNs designed to better handle sequences of data like text just like RNNs, but with the below improvements:
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
Image Source: [https://colah.github.io/posts/2015-08-Understanding-LSTMs/](https://colah.github.io/posts/2015-08-Understanding-LSTMs/)
|
||||
|
||||
1. **Memory Cell**: LSTMs have a special memory cell that can store information over time.
|
||||
2. **Gating Mechanism**: LSTMs use gates to control the flow of information into and out of the memory cell:
|
||||
- Input Gate: Decides how much new information to keep.
|
||||
- Forget Gate: Decides how much old information to forget.
|
||||
- Output Gate: Decides how much of the current cell state to output.
|
||||
3. **Gradient Flow**: LSTMs help gradients flow better during training, which helps in learning from long sequences of data.
|
||||
4. **Learning Long-Term Dependencies**: LSTMs are good at remembering important information from earlier in the sequence, making them useful for tasks where understanding context over long distances is crucial.
|
||||
|
||||
Therefore LSTMs are better at handling sequences by remembering important information and forgetting what's not needed, which makes them more effective than traditional RNNs for tasks like language processing.
|
||||
|
||||
Both RNNs and LSTMs (and their variants) are widely used for language modeling tasks, where the goal is to predict the next word in a sequence of words. They can learn the underlying structure of language and generate coherent text. However, they struggle to handle input sequences of variable lengths and generate output sequences of variable lengths because their fixed-size hidden states limit their ability to capture long-range dependencies and maintain context over time.
|
||||
|
||||
### Sequence-to-Sequence (Seq2Seq) models
|
||||
|
||||
That's where Sequence-to-Sequence (Seq2Seq) models come in; they work by employing an encoder-decoder architecture, where the input sequence is encoded into a fixed-size representation (context vector) by the encoder, and then decoded into an output sequence by the decoder. This architecture allows Seq2Seq models to handle sequences of variable lengths and effectively capture the semantic meaning and structure of the input sequence while generating the corresponding output sequence. A simple Seq2Seq model is depicted below. Each unit in the Seq2Seq is still an RNN type of architecture.
|
||||
|
||||
We won’t dive too deep into the workings here for brevity, [this](https://www.analyticsvidhya.com/blog/2020/08/a-simple-introduction-to-sequence-to-sequence-models/#:~:text=Sequence%20to%20Sequence%20(often%20abbreviated,Chatbots%2C%20Text%20Summarization%2C%20etc.) article is a great read for those interested:
|
||||
|
||||
|
||||
|
||||
Image Source: [https://towardsdatascience.com/sequence-to-sequence-model-introduction-and-concepts-44d9b41cd42d](https://towardsdatascience.com/sequence-to-sequence-model-introduction-and-concepts-44d9b41cd42d)
|
||||
|
||||
### Seq2Seq models + Attention
|
||||
|
||||
|
||||
|
||||
Image Source: [https://lena-voita.github.io/nlp_course/seq2seq_and_attention.html](https://lena-voita.github.io/nlp_course/seq2seq_and_attention.html)
|
||||
|
||||
The problem with traditional Seq2Seq models lies in their inability to effectively handle long input sequences, especially when generating output sequences of variable lengths. In standard Seq2Seq models, a fixed-length context vector is used to summarize the entire input sequence, which can lead to information loss, particularly for long sequences. Additionally, when generating output sequences, the decoder may struggle to focus on relevant parts of the input sequence, resulting in suboptimal translations or predictions.
|
||||
|
||||
To address these issues, attention mechanisms were introduced. Attention mechanisms allow Seq2Seq models to dynamically focus on different parts of the input sequence during the decoding process.
|
||||
|
||||
**Here's how attention works:**
|
||||
|
||||
1. **Encoder Representation**: First, the input sequence is processed by an encoder. The encoder converts each word or element of the input sequence into a hidden state. These hidden states represent different parts of the input sequence and contain information about the sequence's content and structure.
|
||||
2. **Calculating Attention Weights**: During decoding, the decoder needs to decide which parts of the input sequence to focus on. To do this, it calculates attention weights. These weights indicate the relevance or importance of each encoder hidden state to the current decoding step. Essentially, the model is trying to determine which parts of the input sequence are most relevant for generating the next output token.
|
||||
3. **Softmax Normalization**: After calculating the attention weights, the model normalizes them using a softmax function. This ensures that the attention weights sum up to one, effectively turning them into a probability distribution. By doing this, the model can ensure that it allocates its attention appropriately across different parts of the input sequence.
|
||||
4. **Weighted Sum**: With the attention weights calculated and normalized, the model then takes a weighted sum of the encoder hidden states. Essentially, it combines information from different parts of the input sequence based on their importance or relevance as determined by the attention weights. This weighted sum represents the "attended" information from the input sequence, focusing on the parts that are most relevant for the current decoding step.
|
||||
5. **Combining Context with Decoder State**: Finally, the context vector obtained from the weighted sum is combined with the current state of the decoder. This combined representation contains information from both the input sequence (through the context vector) and the decoder's previous state. It serves as the basis for generating the output of the decoder for the current decoding step.
|
||||
6. **Repeating for Each Decoding Step**: Steps 2 to 5 are repeated for each decoding step until the end-of-sequence token is generated or a maximum length is reached. At each step, the attention mechanism helps the model decide where to focus its attention in the input sequence, enabling it to generate accurate and contextually relevant output sequences.
|
||||
|
||||
### Transformer Models
|
||||
|
||||
The problem with Seq2Seq models with attention lies in their computational inefficiency and inability to capture dependencies effectively across long sequences. While attention mechanisms significantly improve the model's ability to focus on relevant parts of the input sequence during decoding, they also introduce computational overhead due to the need to compute attention weights for each decoder step. Additionally, like we mentioned before, traditional Seq2Seq models with attention still rely on RNN or LSTM networks, which have limitations in capturing long-range dependencies.
|
||||
|
||||
The Transformer model was introduced to address these limitations and improve the efficiency and effectiveness of sequence-to-sequence tasks. Here's how the Transformer model solves the problems of Seq2Seq models with attention:
|
||||
|
||||
|
||||
|
||||
Image Source: [https://arxiv.org/pdf/1706.03762.pdf](https://arxiv.org/pdf/1706.03762.pdf)
|
||||
|
||||
1. **Self-Attention Mechanism**: Instead of relying solely on attention mechanisms between the encoder and decoder, the Transformer model introduces a self-attention mechanism. This mechanism allows each position in the input sequence to attend to all other positions, capturing dependencies across the entire input sequence simultaneously. Self-attention enables the model to capture long-range dependencies more effectively compared to traditional Seq2Seq models with attention.
|
||||
2. **Parallelization**: The Transformer model relies on self-attention layers that can be computed in parallel for each position in the input sequence. This parallelization greatly improves the model's computational efficiency compared to traditional Seq2Seq models with recurrent layers, which process sequences sequentially. As a result, the Transformer model can process sequences much faster, making it more suitable for handling long sequences and large-scale datasets.
|
||||
3. **Positional Encoding**: Since the Transformer model does not use recurrent layers, it lacks inherent information about the order of elements in the input sequence. To address this, positional encoding is added to the input embeddings to provide information about the position of each element in the sequence. Positional encoding allows the model to distinguish between elements based on their position, ensuring that the model can effectively process sequences with ordered elements.
|
||||
4. **Transformer Architecture**: The Transformer model consists of an encoder-decoder architecture, similar to traditional Seq2Seq models. However, it replaces recurrent layers with self-attention layers, which enables the model to capture dependencies across long sequences more efficiently. Additionally, the Transformer architecture allows for greater flexibility and scalability, making it easier to train and deploy on various tasks and datasets.
|
||||
|
||||
In summary, the Transformer model addresses the limitations of Seq2Seq models with attention by introducing self-attention mechanisms, parallelization, positional encoding, and a flexible architecture. These advancements improve the model's ability to capture long-range dependencies, process sequences efficiently, and achieve state-of-the-art performance on various sequence-to-sequence tasks.
|
||||
|
||||
### Older Language Models
|
||||
|
||||
Although LLMs have gained significant attention recently, especially with models like GPT from OpenAI, it's important to recognize that the groundwork for this architecture was laid by earlier models such as BERT, GPT (older versions) and T5 explained below.
|
||||
|
||||
LLMs like BERT (Bidirectional Encoder Representations from Transformers), GPT (Generative Pre-trained Transformer), and T5 (Text-To-Text Transfer Transformer) build on top of the concepts introduced by the Transformer model (described in the previous sections) using the following steps:
|
||||
|
||||
1. **Pre-training and Fine-Tuning**: These models utilize a pre-training and fine-tuning approach. During pre-training, the model is trained on large-scale corpora using unsupervised learning objectives, such as masked language modeling (BERT), autoregressive language modeling (GPT), or text-to-text pre-training (T5). This pre-training phase allows the model to learn rich representations of language and general knowledge from large amounts of text data. After pre-training, the model can be fine-tuned on specific downstream tasks with labeled data, enabling it to adapt its learned representations to perform various NLP tasks such as text classification, question answering, and machine translation.
|
||||
2. **Bidirectional Context**: BERT introduced bidirectional context modeling by utilizing a masked language modeling objective. Instead of processing text in a left-to-right or right-to-left manner, BERT is able to consider context from both directions by masking some of the input tokens and predicting them based on the surrounding context. This bidirectional context modeling enables BERT to capture deeper semantic relationships and dependencies within text, leading to improved performance on a wide range of NLP tasks.
|
||||
3. **Autoregressive Generation**: GPT models leverage autoregressive generation, where the model predicts the next token in a sequence based on the previously generated tokens. This approach allows GPT models to generate coherent and contextually relevant text by considering the entire history of the generated sequence. GPT models are particularly effective for tasks that involve generating natural language, such as text generation, dialogue generation, and summarization.
|
||||
4. **Text-to-Text Approach**: T5 introduces a unified text-to-text framework, where all NLP tasks are framed as text-to-text mapping problems. This approach unifies various NLP tasks, such as translation, classification, summarization, and question answering, under a single framework, simplifying the training and deployment process. T5 achieves this by representing both the input and output of each task as textual strings, enabling the model to learn a single mapping function that can be applied across different tasks.
|
||||
5. **Large-Scale Training**: These models are trained on large-scale datasets containing billions of tokens, leveraging massive computational resources and distributed training techniques. By training on extensive data and with powerful hardware, these models can capture rich linguistic patterns and semantic relationships, leading to significant improvements in performance across a wide range of NLP tasks.
|
||||
|
||||
### Large Language Models
|
||||
|
||||
The latest Llama such as Llama and ChatGPT represent significant advancements over earlier models like BERT and GPT in several key ways:
|
||||
|
||||
1. **Task Specialization**: While earlier LLMs like BERT and GPT were designed to perform a wide range of NLP tasks, including text classification, language generation, and question answering, newer models like Llama and ChatGPT are more specialized. For example, Llama is specifically tailored for multimodal tasks, such as image captioning and visual question answering, while ChatGPT is optimized for conversational applications, such as dialogue generation and chatbots.
|
||||
2. **Multimodal Capabilities**: Llama and other recent LLMs integrate multimodal capabilities, allowing them to process and generate text in conjunction with other modalities such as images, audio, and video. This enables LLMs to perform tasks that require understanding and generating content across multiple modalities, opening up new possibilities for applications like image captioning, video summarization, and multimodal dialogue systems.
|
||||
3. **Improved Efficiency**: Recent advancements in LLM architecture and training methodologies have led to improvements in efficiency, allowing models like Llama and ChatGPT to achieve comparable performance to their predecessors with fewer parameters and computational resources. This increased efficiency makes it more practical to deploy these models in real-world applications and reduces the environmental impact associated with training large models.
|
||||
4. **Fine-Tuning and Transfer Learning**: LLMs like ChatGPT are often fine-tuned on specific datasets or tasks to further improve performance in targeted domains. By fine-tuning on domain-specific data, these models can adapt their pre-trained knowledge to better suit the requirements of particular applications, leading to enhanced performance and generalization.
|
||||
5. **Interactive and Dynamic Responses**: ChatGPT and similar conversational models are designed to generate interactive and dynamic responses in natural language conversations. These models leverage context from previous turns in the conversation to generate more coherent and contextually relevant responses, making them more suitable for human-like interaction in chatbot applications and dialogue systems.
|
||||
|
||||
## Read/Watch These Resources (Optional)
|
||||
|
||||
1. Understanding LSTM Networks: [https://colah.github.io/posts/2015-08-Understanding-LSTMs/](https://colah.github.io/posts/2015-08-Understanding-LSTMs/)
|
||||
2. Sequence to Sequence (seq2seq) and Attention: [https://lena-voita.github.io/nlp_course/seq2seq_and_attention.html](https://lena-voita.github.io/nlp_course/seq2seq_and_attention.html)
|
||||
3. Sequence to Sequence models: [https://www.youtube.com/watch?v=kklo05So99U](https://www.youtube.com/watch?v=kklo05So99U)
|
||||
4. How Attention works in Deep Learning: understanding the attention mechanism in sequence models**:** [https://theaisummer.com/attention/](https://theaisummer.com/attention/)
|
||||
5. Intro to LLMs:
|
||||
1. [https://www.youtube.com/watch?v=zjkBMFhNj_g&t=1845s](https://www.youtube.com/watch?v=zjkBMFhNj_g&t=1845s)
|
||||
2. [https://www.youtube.com/watch?v=zizonToFXDs](https://www.youtube.com/watch?v=zizonToFXDs)
|
||||
6. Transformers: [https://www.youtube.com/watch?v=wl3mbqOtlmM](https://www.youtube.com/watch?v=wl3mbqOtlmM)
|
||||
|
||||
## Read These Papers (Optional)
|
||||
|
||||
1. [https://arxiv.org/abs/1706.03762](https://arxiv.org/abs/1706.03762)
|
||||
2. [https://arxiv.org/abs/2005.14165](https://arxiv.org/abs/2005.14165)
|
||||
3. [https://arxiv.org/abs/1910.10683](https://arxiv.org/abs/1910.10683)
|
||||
@@ -0,0 +1,195 @@
|
||||
# [Week 1, Part 1] Applied LLM Foundations and Real World Use Cases
|
||||
|
||||
[Jan 15 2024] You can register [here](https://forms.gle/353sQMRvS951jDYu7) to receive course content and other resources
|
||||
|
||||
## ETMI5: Explain to Me in 5
|
||||
|
||||
In this part of the course, we delve into the intricacies of Large Language Models (LLMs). We start off by exploring the historical context and fundamental concepts of artificial intelligence (AI), machine learning (ML), neural networks (NNs), and generative AI (GenAI). We then examine the core attributes of LLMs, focusing on their scale, extensive training on diverse datasets, and the role of model parameters. Then we go over the types of challenges associated with using LLMs.
|
||||
|
||||
In the next section, we explore practical applications of LLMs across various domains, emphasizing their versatility in areas like content generation, language translation, text summarization, question answering etc. The section concludes with an analysis of the challenges encountered in deploying LLMs, covering essential aspects such as scalability, latency, monitoring etc.
|
||||
|
||||
In summary, this part of the course provides a practical and informative exploration of Large Language Models, offering insights into their evolution, functionality, applications, challenges, and real-world impact.
|
||||
|
||||
## History and Background
|
||||
|
||||
|
||||
|
||||
Image Source: [https://medium.com/womenintechnology/ai-c3412c5aa0ac](https://medium.com/womenintechnology/ai-c3412c5aa0ac)
|
||||
|
||||
The terms mentioned in the image above have likely come up in conversations about ChatGPT. The visual representation offers a broad overview of how they fit into a hierarchy. AI is a comprehensive domain, where LLMs constitute a specific subdomain, and ChatGPT exemplifies an LLM in this context.
|
||||
|
||||
In summary, **Artificial Intelligence (AI)** is a branch of computer science that involves creating machines with human-like thinking and behavior. **Machine Learning(ML)**, a subfield of AI, allows computers to learn patterns from data and make predictions without explicit programming. **Neural Networks (NNs)**, a subset of ML, mimic the human brain's structure and are crucial in deep learning algorithms. Deep Learning (DL), a subset of NN, is effective for complex problem-solving, as seen in image recognition and language translation technologies. **Generative AI (GenAI)**, a subset of DL, can create diverse content based on learned patterns. **Large Language Models (LLMs)**, a form of GenAI, specialize in generating human-like text by learning from extensive textual data.
|
||||
|
||||
Generative AI and Large Language Models (LLMs) have revolutionized the field of artificial intelligence, allowing machines to create diverse content such as text, images, music, audio, and videos. Unlike discriminative models that classify, generative AI models generate new content by learning patterns and relationships from human-created datasets.
|
||||
|
||||
At the core of generative AI are foundation models which essentially refer to large AI models capable of multi-tasking, performing tasks like summarization, Q&A, and classification out-of-the-box. These models, like the popular one that everyone’s heard of-ChatGPT, can adapt to specific use cases with minimal training and generate content with minimal example data.
|
||||
|
||||
The training of generative AI often involves supervised learning, where the model is provided with human-created content and corresponding labels. By learning from this data, the model becomes proficient in generating content similar to the training set.
|
||||
|
||||
Generative AI is not a new concept. One notable example of early generative AI is the Markov chain, a statistical model introduced by Russian mathematician Andrey Markov in 1906. Markov models were initially used for tasks like next-word prediction, but their simplicity limited their ability to generate plausible text.
|
||||
|
||||
The landscape has significantly changed over the years with the advent of more powerful architectures and larger datasets. In 2014, generative adversarial networks (GANs) emerged, using two models working together—one generating output and the other discriminating real data from the generated output. This approach, exemplified by models like StyleGAN, significantly improved the realism of generated content.
|
||||
|
||||
A year later, diffusion models were introduced, refining their output iteratively to generate new data samples resembling the training dataset. This innovation, as seen in Stable Diffusion, contributed to the creation of realistic-looking images.
|
||||
|
||||
In 2017, Google introduced the transformer architecture, a breakthrough in natural language processing. Transformers encode each word as a token, generating an attention map that captures relationships between tokens. This attention to context enhances the model's ability to generate coherent text, exemplified by large language models like ChatGPT.
|
||||
|
||||
The generative AI boom owes its momentum not only to larger datasets but also to diverse research advances. These approaches, including GANs, diffusion models, and transformers, showcase the breadth of methods contributing to the exciting field of generative AI.
|
||||
|
||||
## Enter LLMs
|
||||
|
||||
The term "Large" in Large Language Models (LLMs) refers to the sheer scale of these models—both in terms of the size of their architecture and the vast amount of data they are trained on. The size matters because it allows them to capture more complex patterns and relationships within language. Popular LLMs like GPT-3, Gemini, Claude etc. have thousands of billion model parameters. In the context of machine learning, model parameters are like the knobs and switches that the algorithm tunes during training to make accurate predictions or generate meaningful outputs.
|
||||
|
||||
Now, let's break down what "Language Models" mean in this context. Language models are essentially algorithms or systems that are trained to understand and generate human-like text. They serve as a representation of how language works, learning from diverse datasets to predict what words or sequences of words are likely to come next in a given context.
|
||||
|
||||
The "Large" aspect amplifies their capabilities. Traditional language models, especially those from the past, were smaller in scale and couldn't capture the intricacies of language as effectively. With advancements in technology and the availability of massive computing power, we've been able to build much larger models. These Large Language Models, like ChatGPT, have billions of parameters, which are essentially the variables the model uses to make sense of language.
|
||||
|
||||
Take a look at the infographic from “Information is beautiful” below to see how many parameters recent LLMs have. You can view the live visualization [here](https://informationisbeautiful.net/visualizations/the-rise-of-generative-ai-large-language-models-llms-like-chatgpt/)
|
||||
|
||||
|
||||
|
||||
Image source: [https://informationisbeautiful.net/visualizations/the-rise-of-generative-ai-large-language-models-llms-like-chatgpt/](https://informationisbeautiful.net/visualizations/the-rise-of-generative-ai-large-language-models-llms-like-chatgpt/)
|
||||
|
||||
## Training LLMs
|
||||
|
||||
Training LLMs is a complex process that involves instructing the model to comprehend and produce human-like text. Here's a simplified breakdown of how LLM training works:
|
||||
|
||||
1. **Providing Input Text:**
|
||||
- LLMs are initially exposed to extensive text data, encompassing various sources such as books, articles, and websites.
|
||||
- The model's task during training is to predict the next word or token in a sequence based on the context provided. It learns patterns and relationships within the text data.
|
||||
2. **Optimizing Model Weights:**
|
||||
- The model comprises different weights associated with its parameters, reflecting the significance of various features.
|
||||
- Throughout training, these weights are fine-tuned to minimize the error rate. The objective is to enhance the model's accuracy in predicting the next word.
|
||||
3. **Fine-tuning Parameter Values:**
|
||||
- LLMs continuously adjust parameter values based on error feedback received during predictions.
|
||||
- The model refines its grasp of language by iteratively adjusting parameters, improving accuracy in predicting subsequent tokens.
|
||||
|
||||
The training process may vary depending on the specific type of LLM being developed, such as those optimized for continuous text or dialogue.
|
||||
|
||||
LLM performance is heavily influenced by two key factors:
|
||||
|
||||
- **Model Architecture:** The design and intricacy of the LLM architecture impact its ability to capture language nuances.
|
||||
- **Dataset:** The quality and diversity of the dataset utilized for training are crucial in shaping the model's language understanding.
|
||||
|
||||
Training a private LLM demands substantial computational resources and expertise. The duration of the process can range from several days to weeks, contingent on the model's complexity and dataset size. Commonly, cloud-based solutions and high-performance GPUs are employed to expedite the training process, making it more efficient. Overall, LLM training is a meticulous and resource-intensive undertaking that lays the groundwork for the model's language comprehension and generation capabilities.
|
||||
|
||||
After the initial training, LLMs can be easily customized for various tasks using relatively small sets of supervised data, a procedure referred to as fine-tuning.
|
||||
|
||||
There are three prevalent learning models:
|
||||
|
||||
1. **Zero-shot learning:** The base LLMs can handle a wide range of requests without explicit training, often by using prompts, though the accuracy of responses may vary.
|
||||
2. **Few-shot learning:** By providing a small number of pertinent training examples, the performance of the base model significantly improves in a specific domain.
|
||||
3. **Domain Adaptation:** This extends from few-shot learning, where practitioners train a base model to adjust its parameters using additional data relevant to the particular application or domain.
|
||||
|
||||
We will be diving deep into each of these methods during the course.
|
||||
|
||||
## LLM Real World Use Cases
|
||||
|
||||
LLMs are already being leveraged in various applications showcasing their versatility and power of these models in transforming several domains. Here's how LLMs can be applied to specific cases:
|
||||
|
||||
|
||||
|
||||
1. **Content Generation:**
|
||||
- LLMs excel in content generation by understanding context and generating coherent and contextually relevant text. They can be employed to automatically generate creative content for marketing, social media posts, and other communication materials, ensuring a high level of quality and relevance.
|
||||
- **Real World Applications:** Marketing platforms, social media management tools, content creation platforms, advertising agencies
|
||||
2. **Language Translation:**
|
||||
- LLMs can significantly improve language translation tasks by understanding the nuances of different languages. They can provide accurate and context-aware translations, making them valuable tools for businesses operating in multilingual environments. This can enhance global communication and outreach.
|
||||
- **Real World Applications**: Translation services, global communication platforms, international business applications
|
||||
3. **Text Summarization:**
|
||||
- LLMs are adept at summarizing lengthy documents by identifying key information and maintaining the core message. This capability is valuable for content creators, researchers, and businesses looking to quickly extract essential insights from large volumes of text, improving efficiency in information consumption.
|
||||
- **Real World Applications**: Research tools, news aggregators, content curation platforms
|
||||
4. **Question Answering and Chatbots:**
|
||||
- LLMs can be employed for question answering tasks, where they comprehend the context of a question and generate relevant and accurate responses. They enable these systems to engage in more natural and context-aware conversations, understanding user queries and providing relevant responses.
|
||||
- **Real World Applications***:* Customer support systems, chatbots, virtual assistants, educational platforms
|
||||
5. **Content Moderation:**
|
||||
- LLMs can be utilized for content moderation by analyzing text and identifying potentially inappropriate or harmful content. This helps in maintaining a safe and respectful online environment by automatically flagging or filtering out content that violates guidelines, ensuring user safety.
|
||||
- **Real World Applications**: Social media platforms, online forums, community management tools.
|
||||
6. **Information Retrieval:**
|
||||
- LLMs can enhance information retrieval systems by understanding user queries and retrieving relevant information from large datasets. This is particularly useful in search engines, databases, and knowledge management systems, where LLMs can improve the accuracy of search results.
|
||||
- **Real World Applications**: Search engines, database systems, knowledge management platforms
|
||||
7. **Educational Tools:**
|
||||
- LLMs contribute to educational tools by providing natural language interfaces for learning platforms. They can assist students in generating summaries, answering questions, and engaging in interactive learning conversations. This facilitates personalized and efficient learning experiences.
|
||||
- **Real World Applications**: E-learning platforms, educational chatbots, interactive learning applications
|
||||
|
||||
Summary of popular LLM use-cases
|
||||
|
||||
| No. | Use case | Description |
|
||||
| --- | --- | --- |
|
||||
| 1 | Content Generation | Craft human-like text, videos, code and images when provided with instructions |
|
||||
| 2 | Language Translation | Translate languages from one to another |
|
||||
| 3 | Text Summarization | Summarize lengthy texts, simplifying comprehension by highlighting key points. |
|
||||
| 4 | Question Answering and Chatbots | LLMs can provide relevant answers to queries, leveraging their vast knowledge |
|
||||
| 5 | Content Moderation | Assist in content moderation by identifying and filtering inappropriate or harmful language |
|
||||
| 6 | Information Retrieval | Retrieve relevant information from large datasets or documents. |
|
||||
| 7 | Educational Tools | Tutor, provide explanations, and generate learning materials. |
|
||||
|
||||
Understanding the utilization of generative AI models, especially LLMs, can also be gleaned from the extensive array of startups operating in this domain. An [infographic](https://www.sequoiacap.com/article/generative-ai-act-two/) presented by Sequoia Capital highlighted these companies across diverse sectors, illustrating the versatile applications and the significant presence of numerous players in the generative AI space.
|
||||
|
||||
|
||||
|
||||
Image Source: [https://markovate.com/blog/applications-and-use-cases-of-llm/](https://markovate.com/blog/applications-and-use-cases-of-llm/)
|
||||
|
||||
## LLM Challenges
|
||||
|
||||
|
||||
|
||||
Although LLMs have undoubtedly revolutionized various applications, numerous challenges persist. These challenges are categorized into different themes:
|
||||
|
||||
- **Data Challenges:** This pertains to the data used for training and how the model addresses gaps or missing data.
|
||||
- **Ethical Challenges:** This involves addressing issues such as mitigating biases, ensuring privacy, and preventing the generation of harmful content in the deployment of LLMs.
|
||||
- **Technical Challenges:** These challenges focus on the practical implementation of LLMs.
|
||||
- **Deployment Challenges:** Concerned with the specific processes involved in transitioning fully-functional LLMs into real-world use-cases (productionization)
|
||||
|
||||
**Data Challenges:**
|
||||
|
||||
1. **Data Bias:** The presence of prejudices and imbalances in the training data leading to biased model outputs.
|
||||
2. **Limited World Knowledge and Hallucination:** LLMs may lack comprehensive understanding of real-world events and information and tend to hallucinate information. Note that training them on new data is a long and expensive process.
|
||||
3. **Dependency on Training Data Quality:** LLM performance is heavily influenced by the quality and representativeness of the training data.
|
||||
|
||||
**Ethical and Social Challenges:**
|
||||
|
||||
1. **Ethical Concerns:** Concerns regarding the responsible and ethical use of language models, especially in sensitive contexts.
|
||||
2. **Bias Amplification:** Biases present in the training data may be exacerbated, resulting in unfair or discriminatory outputs.
|
||||
3. **Legal and Copyright Issues:** Potential legal complications arising from generated content that infringes copyrights or violates laws.
|
||||
4. **User Privacy Concerns:** Risks associated with generating text based on user inputs, especially when dealing with private or sensitive information.
|
||||
|
||||
**Technical Challenges:**
|
||||
|
||||
1. **Computational Resources:** Significant computing power required for training and deploying large language models.
|
||||
2. **Interpretability:** Challenges in understanding and explaining the decision-making process of complex models.
|
||||
3. **Evaluation**: Evaluation presents a notable challenge as assessing models across diverse tasks and domains is inadequately designed, particularly due to the challenges posed by freely generated content.
|
||||
4. **Fine-tuning Challenges:** Difficulties in adapting pre-trained models to specific tasks or domains.
|
||||
5. **Contextual Understanding:** LLMs may face challenges in maintaining coherent context over longer passages or conversations.
|
||||
6. **Robustness to Adversarial Attacks:** Vulnerability to intentional manipulations of input data leading to incorrect outputs.
|
||||
7. **Long-Term Context:** Struggles in maintaining context and coherence over extended pieces of text or discussions.
|
||||
|
||||
**Deployment Challenges:**
|
||||
|
||||
1. **Scalability:** Ensuring that the model can scale efficiently to handle increased workloads and demand in production environments.
|
||||
2. **Latency:** Minimizing the response time or latency of the model to provide quick and efficient interactions, especially in real-time applications.
|
||||
3. **Monitoring and Maintenance:** Implementing robust monitoring systems to track model performance, detect issues, and perform regular maintenance to avoid downtime.
|
||||
4. **Integration with Existing Systems:** Ensuring smooth integration of LLMs with existing software, databases, and infrastructure within an organization.
|
||||
5. **Cost Management:** Optimizing the cost of deploying and maintaining large language models, as they can be resource-intensive in terms of both computation and storage.
|
||||
6. **Security Concerns:** Addressing potential security vulnerabilities and risks associated with deploying language models in production, including safeguarding against malicious attacks.
|
||||
7. **Interoperability:** Ensuring compatibility with other tools, frameworks, or systems that may be part of the overall production pipeline.
|
||||
8. **User Feedback Incorporation:** Developing mechanisms to incorporate user feedback to continuously improve and update the model in a production environment.
|
||||
9. **Regulatory Compliance:** Adhering to regulatory requirements and compliance standards, especially in industries with strict data protection and privacy regulations.
|
||||
10. **Dynamic Content Handling:** Managing the generation of text in dynamic environments where content and user interactions change frequently.
|
||||
|
||||
## Read/Watch These Resources (Optional)
|
||||
|
||||
1. [https://www.nvidia.com/en-us/glossary/generative-ai/](https://www.nvidia.com/en-us/glossary/generative-ai/)
|
||||
2. [https://markovate.com/blog/applications-and-use-cases-of-llm/](https://markovate.com/blog/applications-and-use-cases-of-llm/)
|
||||
3. [https://www.sequoiacap.com/article/generative-ai-act-two/](https://www.sequoiacap.com/article/generative-ai-act-two/)
|
||||
4. [https://datasciencedojo.com/blog/challenges-of-large-language-models/](https://datasciencedojo.com/blog/challenges-of-large-language-models/)
|
||||
5. [https://snorkel.ai/enterprise-llm-challenges-and-how-to-overcome-them/](https://snorkel.ai/enterprise-llm-challenges-and-how-to-overcome-them/)
|
||||
6. [https://www.youtube.com/watch?v=MyFrMFab6bo](https://www.youtube.com/watch?v=MyFrMFab6bo)
|
||||
7. [https://www.youtube.com/watch?v=cEyHsMzbZBs](https://www.youtube.com/watch?v=cEyHsMzbZBs)
|
||||
|
||||
## Read These Papers (Optional)
|
||||
|
||||
1. [https://dl.acm.org/doi/abs/10.1145/3605943](https://dl.acm.org/doi/abs/10.1145/3605943)
|
||||
2. [https://www.sciencedirect.com/science/article/pii/S2950162823000176](https://www.sciencedirect.com/science/article/pii/S2950162823000176)
|
||||
3. [https://arxiv.org/pdf/2303.13379.pdf](https://arxiv.org/pdf/2303.13379.pdf)
|
||||
4. [https://proceedings.mlr.press/v202/kandpal23a/kandpal23a.pdf](https://proceedings.mlr.press/v202/kandpal23a/kandpal23a.pdf)
|
||||
5. [https://link.springer.com/article/10.1007/s12599-023-00795-x](https://link.springer.com/article/10.1007/s12599-023-00795-x)
|
||||
@@ -0,0 +1,156 @@
|
||||
# [Week 1, Part 2] Domain and Task Adaptation Methods
|
||||
|
||||
## ETMI5: Explain to Me in 5
|
||||
|
||||
In this section, we delve into the limitations of general AI models in specialized domains, underscoring the significance of domain-adapted LLMs. We explore the advantages of these models, including depth, precision, improved user experiences, and addressing privacy concerns.
|
||||
|
||||
We introduce three types of domain adaptation methods: Domain-Specific Pre-Training, Domain-Specific Fine-Tuning, and Retrieval Augmented Generation (RAG). Each method is outlined, providing details on types, training durations, and quick summaries. We then explain each of these methods in further detail with real-world examples. In the end, we provide an overview of when RAG should be used as opposed to model updating methods.
|
||||
|
||||
## Using LLMs Effectively
|
||||
|
||||
While general AI models such as ChatGPT demonstrate impressive text generation abilities across various subjects, they may lack the depth and nuanced understanding required for specific domains. Additionally, these models are more prone to generating inaccurate or contextually inappropriate content, referred to as hallucinations. For instance, in healthcare, specific terms like "electronic health record interoperability" or "patient-centered medical home" hold significant importance, but a generic language model may struggle to fully comprehend their relevance due to a lack of specific training on healthcare data. This is where task-specific and domain-specific LLMs play a crucial role. These models need to possess specialized knowledge of industry-specific terminology and practices to ensure accurate interpretation of domain-specific concepts. Throughout the remainder of this course, we will refer to these specialized LLMs as **domain-specific LLM**s, a commonly used term for such models.
|
||||
|
||||
Here are some benefits of using domain-specific LLMs:
|
||||
|
||||
1. **Depth and Precision**: General LLMs, while proficient in generating text across diverse topics, may lack the depth and nuance required for specialized domains. Domain-specific LLMs are tailored to understand and interpret industry-specific terminology, ensuring precision in comprehension.
|
||||
2. **Overcoming Limitations**: General LLMs have limitations, including potential inaccuracies, lack of context, and susceptibility to hallucinations. In domains like finance or medicine, where specific terminology is crucial, domain-specific LLMs excel in providing accurate and contextually relevant information.
|
||||
3. **Enhanced User Experiences**: Domain-specific LLMs contribute to enhanced user experiences by offering tailored and personalized responses. In applications such as customer service chatbots or dynamic AI agents, these models leverage specialized knowledge to provide more accurate and insightful information.
|
||||
4. **Improved Efficiency and Productivity**: Businesses can benefit from the improved efficiency of domain-specific LLMs. By automating tasks, generating content aligned with industry-specific terminology, and streamlining operations, these models free up human resources for higher-level tasks, ultimately boosting productivity.
|
||||
5. **Addressing Privacy Concerns**: In industries dealing with sensitive data, such as healthcare, using general LLMs may pose privacy challenges. Domain-specific LLMs can provide a closed framework, ensuring the protection of confidential data and adherence to privacy agreements.
|
||||
|
||||
If you recall from the [previous section](https://www.notion.so/Week-1-Applied-LLM-Foundations-369ae7cf630d467cbfeedd3b9b3bfc46?pvs=21), we had multiple ways to use LLMs in specific use cases, namely
|
||||
|
||||
1. **Zero-shot learning**
|
||||
2. **Few-shot learning**
|
||||
3. **Domain Adaptation**
|
||||
|
||||
Zero-shot learning and few-shot learning involve instructing the general model either through examples or by prompting it with specific questions of interest. Another concept introduced is domain adaptation, which will be the primary focus in this section. More details about the first two methods will be explored when we delve into the topic of prompting.
|
||||
|
||||
## Types of Domain Adaptation Methods
|
||||
|
||||
There are several methods to incorporate domain-specific knowledge into LLMs, each with its own advantages and limitations. Here are three classes of approaches:
|
||||
|
||||
1. **Domain-Specific Pre-Training:**
|
||||
- ***Training Duration**:* Days to weeks to months
|
||||
- ***Summary**:* Requires a large amount of domain training data; can customize model architecture, size, tokenizer, etc.
|
||||
|
||||
In this method, LLMs are pre-trained on extensive datasets representing various natural language use cases. For instance, models like PaLM 540B, GPT-3, and LLaMA 2 have been pre-trained on datasets with sizes ranging from 499 billion to 2 trillion tokens. Examples of domain-specific pre-training include models like ESMFold, ProGen2 for protein sequences, Galactica for science, BloombergGPT for finance, and StarCoder for code. These models outperform generalist models within their domains but still face limitations in terms of accuracy and potential hallucinations.
|
||||
|
||||
2. **Domain-Specific Fine-Tuning:**
|
||||
- ***Training Duration**:* Minutes to hours
|
||||
- ***Summary**:* Adds domain-specific data; tunes for specific tasks; updates LLM model
|
||||
|
||||
Fine-tuning involves training a pre-trained LLM on a specific task or domain, adapting its knowledge to a narrower context. Examples include Alpaca (fine-tuned LLaMA-7B model for general tasks), xFinance (fine-tuned LLaMA-13B model for financial-specific tasks), and ChatDoctor (fine-tuned LLaMA-7B model for medical chat). The costs for fine-tuning are significantly smaller compared to pre-training.
|
||||
|
||||
3. **Retrieval Augmented Generation (RAG):**
|
||||
- ***Training Duration**:* Not required
|
||||
- ***Summary**:* No model weights; external information retrieval system can be tuned
|
||||
|
||||
RAG involves grounding the LLM's parametric knowledge with external or non-parametric knowledge from an information retrieval system. This external knowledge is provided as additional context in the prompt to the LLM. The advantages of RAG include no training costs, low expertise requirement, and the ability to cite sources for human verification. This approach addresses limitations such as hallucinations and allows for precise manipulation of knowledge. The knowledge base is easily updatable without changing the LLM. Strategies to combine non-parametric knowledge with an LLM's parametric knowledge are actively researched.
|
||||
|
||||
|
||||
## **Domain-Specific Pre-Training**
|
||||
|
||||
|
||||
|
||||
Image Source [https://www.analyticsvidhya.com/blog/2023/08/domain-specific-llms/](https://www.analyticsvidhya.com/blog/2023/08/domain-specific-llms/)
|
||||
|
||||
Domain-specific pre-training involves training large language models on extensive datasets that specifically represent the language and characteristics of a particular domain or field. This process aims to enhance the model's understanding and performance within a defined subject area. Let’s understand domain specific pretraining through the example of [BloombergGPT,](https://arxiv.org/pdf/2303.17564.pdf) a large language model for finance.
|
||||
|
||||
BloombergGPT is a 50 billion parameter language model designed to excel in various tasks within the financial industry. While general models are versatile and perform well across diverse tasks, they may not outperform domain-specific models in specialized areas. At Bloomberg, where a significant majority of applications are within the financial domain, there is a need for a model that excels in financial tasks while maintaining competitive performance on general benchmarks. BloombergGPT can perform the following tasks:
|
||||
|
||||
1. **Financial Sentiment Analysis:** Analyzing and determining sentiment in financial texts, such as news articles, social media posts, or financial reports. This helps in understanding market sentiment and making informed investment decisions.
|
||||
2. **Named Entity Recognition:** Identifying and classifying entities (such as companies, individuals, and financial instruments) mentioned in financial documents. This is crucial for extracting relevant information from large datasets.
|
||||
3. **News Classification:** Categorizing financial news articles into different topics or classes. This can aid in organizing and prioritizing news updates based on their relevance to specific financial areas.
|
||||
4. **Question Answering in Finance:** Answering questions related to financial topics. Users can pose queries about market trends, financial instruments, or economic indicators, and BloombergGPT can provide relevant answers.
|
||||
5. **Conversational Systems for Finance:** Engaging in natural language conversations related to finance. Users can interact with BloombergGPT to seek information, clarify doubts, or discuss financial concepts.
|
||||
|
||||
To achieve this, BloombergGPT undergoes domain-specific pre-training using a large dataset that combines domain-specific financial language documents from Bloomberg's extensive archives with public datasets. This dataset, named FinPile, consists of diverse English financial documents, including news, filings, press releases, web-scraped financial documents, and social media content. The training corpus is roughly divided into half domain-specific text and half general-purpose text. The aim is to leverage the advantages of both domain-specific and general data sources.
|
||||
|
||||
The model architecture is based on guidelines from previous research efforts, containing 70 layers of transformer decoder blocks (read more in the [paper](https://arxiv.org/pdf/2303.17564.pdf))
|
||||
|
||||
## **Domain-Specific Fine-Tuning**
|
||||
|
||||
Domain-specific fine-tuning is the process of refining a pre-existing language model for a particular task or within a specific domain to enhance its performance and tailor it to the unique context of that domain. This method involves taking an LLM that has undergone pre-training on a diverse dataset encompassing various language use cases and subsequently fine-tuning it on a narrower dataset specifically related to a particular domain or task.
|
||||
|
||||
💡Note that the previous method, i.e., domain-specific pre-training involves training a language model exclusively on data from a specific domain, creating a specialized model for that domain. On the other hand, domain-specific fine-tuning takes a pre-trained general model and further trains it on domain-specific data, adapting it for tasks within that domain without starting from scratch. Pre-training is domain-exclusive from the beginning, while fine-tuning adapts a more versatile model to a specific domain.
|
||||
|
||||
The key steps in domain-specific fine-tuning include:
|
||||
|
||||
1. **Pre-training:** Initially, a large language model is pre-trained on an extensive dataset, allowing it to grasp general language patterns, grammar, and contextual understanding (A general LLM).
|
||||
2. **Fine-tuning Dataset:** A more focused dataset, tailored to the desired domain or task, is collected or prepared. This dataset contains relevant examples and instances related to the target domain, potentially including labeled examples for supervised learning.
|
||||
3. **Fine-tuning Process:** The pre-trained language model undergoes further training on this domain-specific dataset. During fine-tuning, the model's parameters are adjusted based on the new dataset, while retaining the general language understanding acquired during pre-training.
|
||||
4. **Task Optimization:** The fine-tuned model is optimized for specific tasks within the chosen domain. This optimization may involve adjusting parameters related to the task, such as the model architecture, size, or tokenizer, to achieve optimal performance.
|
||||
|
||||
Domain-specific fine-tuning offers several advantages:
|
||||
|
||||
- It enables the model to specialize in a particular domain, enhancing its effectiveness for tasks within that domain.
|
||||
- It saves time and computational resources compared to training a model from scratch, leveraging the knowledge gained during pre-training.
|
||||
- The model can adapt to the specific requirements and nuances of the target domain, leading to improved performance on domain-specific tasks.
|
||||
|
||||
A popular example for domain-specific fine-tuning is the ChatDoctor LLM which is a specialized language model fine-tuned on Meta-AI's large language model meta-AI (LLaMA) using a dataset of 100,000 patient-doctor dialogues from an online medical consultation platform. The model undergoes fine-tuning on real-world patient interactions, significantly improving its understanding of patient needs and providing more accurate medical advice. ChatDoctor uses real-time information from online sources like Wikipedia and curated offline medical databases, enhancing the accuracy of its responses to medical queries. The model's contributions include a methodology for fine-tuning LLMs in the medical field, a publicly shared dataset, and an autonomous ChatDoctor model capable of retrieving updated medical knowledge. Read more about ChatDoctor in the paper [here](https://arxiv.org/pdf/2303.14070.pdf).
|
||||
|
||||
## Retrieval Augmented Generation (RAG)
|
||||
|
||||
Retrieval Augmented Generation (RAG) is an AI framework that enhances the quality of responses generated by LLMs by incorporating up-to-date and contextually relevant information from external sources during the generation process. It addresses the inconsistency and lack of domain-specific knowledge in LLMs, reducing the chances of hallucinations or incorrect responses. RAG involves two phases: retrieval, where relevant information is searched and retrieved, and content generation, where the LLM synthesizes an answer based on the retrieved information and its internal training data. This approach improves accuracy, allows source verification, and reduces the need for continuous model retraining.
|
||||
|
||||
|
||||
|
||||
Image Source: [https://www.deeplearning.ai/short-courses/langchain-for-llm-application-development/](https://www.deeplearning.ai/short-courses/langchain-for-llm-application-development/)
|
||||
|
||||
The diagram above outlines the fundamental RAG pipeline, consisting of three key components:
|
||||
|
||||
1. **Ingestion:**
|
||||
- Documents undergo segmentation into chunks, and embeddings are generated from these chunks, subsequently stored in an index.
|
||||
- Chunks are essential for pinpointing the relevant information in response to a given query, resembling a standard retrieval approach.
|
||||
2. **Retrieval:**
|
||||
- Leveraging the index of embeddings, the system retrieves the top-k documents when a query is received, based on the similarity of embeddings.
|
||||
3. **Synthesis:**
|
||||
- Examining the chunks as contextual information, the LLM utilizes this knowledge to formulate accurate responses.
|
||||
|
||||
💡Unlike previous methods for domain adaptation, it's important to highlight that RAG doesn't necessitate any model training whatsoever. It can be readily applied without the need for training when specific domain data is provided.
|
||||
|
||||
In contrast to earlier approaches for model updates (pre-training and fine-tuning), RAG comes with specific advantages and disadvantages. The decision to employ or refrain from using RAG depends on an evaluation of these factors.
|
||||
|
||||
| Advantages of RAG | Disadvantages of RAG |
|
||||
| --- | --- |
|
||||
| Information Freshness: RAG addresses the static nature of LLMs by providing up-to-date or context-specific data from an external database. | Complex Implementation (Multiple moving parts): Implementing RAG may involve creating a vector database, embedding models, search index etc. The performance of RAG depends on the individual performance of all these components |
|
||||
| Domain-Specific Knowledge: RAG supplements LLMs with domain-specific knowledge by fetching relevant results from a vector database | Increased Latency: The retrieval step in RAG involves searching through databases, which may introduce latency in generating responses compared to models that don't rely on external sources. |
|
||||
| Reduced Hallucination and Citations: RAG reduces the likelihood of hallucinations by grounding LLMs with external, verifiable facts and can also cite sources | |
|
||||
| Cost-Efficiency: RAG is a cost-effective solution, avoiding the need for extensive model training or fine-tuning | |
|
||||
|
||||
## **Choosing Between RAG, Domain-Specific Fine-Tuning, and Domain-Specific Pre-Training**
|
||||
|
||||
|
||||
|
||||
### **Use Domain-Specific Pre-Training When:**
|
||||
|
||||
- **Exclusive Domain Focus:** Pre-training is suitable when you require a model exclusively trained on data from a specific domain, creating a specialized language model for that domain.
|
||||
- **Customizing Model Architecture:** It allows you to customize various aspects of the model architecture, size, tokenizer, etc., based on the specific requirements of the domain.
|
||||
- **Extensive Training Data Available:** Effective pre-training often requires a large amount of domain-specific training data to ensure the model captures the intricacies of the chosen domain.
|
||||
|
||||
### **Use Domain-Specific Fine-Tuning When:**
|
||||
|
||||
- **Specialization Needed:** Fine-tuning is suitable when you already have a pre-trained LLM, and you want to adapt it for specific tasks or within a particular domain.
|
||||
- **Task Optimization:** It allows you to adjust the model's parameters related to the task, such as architecture, size, or tokenizer, for optimal performance in the chosen domain.
|
||||
- **Time and Resource Efficiency:** Fine-tuning saves time and computational resources compared to training a model from scratch since it leverages the knowledge gained during the pre-training phase.
|
||||
|
||||
### **Use RAG When:**
|
||||
|
||||
- **Information Freshness Matters:** RAG provides up-to-date, context-specific data from external sources.
|
||||
- **Reducing Hallucination is Crucial:** Ground LLMs with verifiable facts and citations from an external knowledge base.
|
||||
- **Cost-Efficiency is a Priority:** Avoid extensive model training or fine-tuning; implement without the need for training.
|
||||
|
||||
## Read/Watch These Resources (Optional)
|
||||
|
||||
1. [https://www.deeplearning.ai/short-courses/langchain-for-llm-application-development/](https://www.deeplearning.ai/short-courses/langchain-for-llm-application-development/)
|
||||
2. [https://www.superannotate.com/blog/llm-fine-tuning#what-is-llm-fine-tuning](https://www.superannotate.com/blog/llm-fine-tuning#what-is-llm-fine-tuning)
|
||||
3. [https://aws.amazon.com/what-is/retrieval-augmented-generation/#:~:text=Retrieval-Augmented Generation (RAG),sources before generating a response](https://aws.amazon.com/what-is/retrieval-augmented-generation/#:~:text=Retrieval%2DAugmented%20Generation%20(RAG),sources%20before%20generating%20a%20response).
|
||||
4. [https://www.youtube.com/watch?v=cXPYtkosXG4](https://www.youtube.com/watch?v=cXPYtkosXG4)
|
||||
5. [https://gradientflow.substack.com/p/best-practices-in-retrieval-augmented](https://gradientflow.substack.com/p/best-practices-in-retrieval-augmented)
|
||||
|
||||
## Read These Papers (Optional)
|
||||
|
||||
1. [https://proceedings.neurips.cc/paper_files/paper/2020/file/6b493230205f780e1bc26945df7481e5-Paper.pdf](https://proceedings.neurips.cc/paper_files/paper/2020/file/6b493230205f780e1bc26945df7481e5-Paper.pdf)
|
||||
2. [https://arxiv.org/abs/2202.01110](https://arxiv.org/abs/2202.01110)
|
||||
3. [https://arxiv.org/abs/1801.06146](https://arxiv.org/abs/1801.06146)
|
||||
@@ -0,0 +1,301 @@
|
||||
# [Week 2] Prompting and Prompt Engineering
|
||||
|
||||
## ETMI5: Explain to Me in 5
|
||||
|
||||
In the section on prompting, you will learn the basics of formulating effective prompts to guide language models in generating desired outputs. You will explore prompt engineering techniques to refine these prompts for improved performance in various applications. We'll cover the importance of contextual understanding, leveraging training data patterns, and utilizing transfer learning. Advanced prompting methods like Chain-of-Thought and Tree-of-Thought will be introduced, highlighting their roles in enhancing reasoning capabilities. Additionally, the section will address the risks associated with prompting, such as bias and prompt hacking, and provide strategies to mitigate these risks.
|
||||
|
||||
## Introduction
|
||||
|
||||
### Prompting
|
||||
|
||||
In the realm of language models, "**prompting**" refers to the art and science of formulating precise instructions or queries provided to the model to generate desired outputs. It's the input—typically in the form of text—that users present to the language model to elicit specific responses. The effectiveness of a prompt lies in its ability to guide the model's understanding and generate outputs aligned with user expectations.
|
||||
|
||||
### Prompt Engineering
|
||||
|
||||
- Prompt engineering, a rapidly growing field, revolves around refining prompts to unleash the full potential of Language Models in various applications.
|
||||
- In research, prompt engineering is a powerful tool, enhancing LLMs' performance across tasks like question answering and arithmetic reasoning. Users need to leverage these skills to create effective prompting techniques that seamlessly interact with LLMs and other tools.
|
||||
- Beyond crafting prompts, prompt engineering is a rich set of skills essential for interacting and developing with LLMs. It's not just about design; it's a crucial skill for understanding and exploiting LLM capabilities, ensuring safety, and introducing novel features like domain knowledge integration.
|
||||
- This proficiency is vital in aligning AI behavior with human intent. While professional prompt engineers delve into the complexities of AI, the skill isn't exclusive to specialists. Anyone refining prompts for models like ChatGPT is engaging in prompt engineering, making it accessible to users exploring language model potentials.
|
||||
|
||||
|
||||
|
||||
Image Source: [https://zapier.com/blog/prompt-engineering/](https://zapier.com/blog/prompt-engineering/)
|
||||
|
||||
## Why Prompting?
|
||||
|
||||
Large language models are trained through a process called unsupervised learning on vast amounts of diverse text data. During training, the model learns to predict the next word in a sentence based on the context provided by the preceding words. This process allows the model to capture grammar, facts, reasoning abilities, and even some aspects of common sense.
|
||||
|
||||
Prompting is a crucial aspect of using these models effectively. Here's why prompting LLMs the right way is essential:
|
||||
|
||||
1. **Contextual Understanding:** LLMs are trained to understand context and generate responses based on the patterns learned from diverse text data. When you provide a prompt, it's crucial to structure it in a way that aligns with the context the model is familiar with. This helps the model make relevant associations and produce coherent responses.
|
||||
2. **Training Data Patterns:** During training, the model learns from a wide range of text, capturing the linguistic nuances and patterns present in the data. Effective prompts leverage this training by incorporating similar language and structures that the model has encountered in its training data. This enables the model to generate responses that are consistent with its learned patterns.
|
||||
3. **Transfer Learning:** LLMs utilize transfer learning. The knowledge gained during training on diverse datasets is transferred to the task at hand when prompted. A well-crafted prompt acts as a bridge, connecting the general knowledge acquired during training to the specific information or action desired by the user.
|
||||
4. **Contextual Prompts for Contextual Responses:** By using prompts that resemble the language and context the model was trained on, users tap into the model's ability to understand and generate content within similar contexts. This leads to more accurate and contextually appropriate responses.
|
||||
5. **Mitigating Bias:** The model may inherit biases present in its training data. Thoughtful prompts can help mitigate bias by providing additional context or framing questions in a way that encourages unbiased responses. This is crucial for aligning model outputs with ethical standards.
|
||||
|
||||
To summarize, the training of LLMs involves learning from massive datasets, and prompting is the means by which users guide these models to produce useful, relevant, and policy-compliant responses. It's a collaborative process where users and models work together to achieve the desired outcome. There’s also a growing field called adversarial prompting which involves intentionally crafting prompts to exploit weaknesses or biases in a language model, with the goal of generating responses that may be misleading, inappropriate, or showcase the model's limitations. Safeguarding models from providing harmful responses is a challenge that needs to be solved and is an active research area.
|
||||
|
||||
## Prompting Basics
|
||||
|
||||
The basic principles of prompting involve the inclusion of specific elements tailored to the task at hand. These elements include:
|
||||
|
||||
1. **Instruction:** Clearly specify the task or action you want the model to perform. This sets the context for the model's response and guides its behavior.
|
||||
2. **Context:** Provide external information or additional context that helps the model better understand the task and generate more accurate responses. Context can be crucial in steering the model towards the desired outcome.
|
||||
3. **Input Data:** Include the input or question for which you seek a response. This is the information on which you want the model to act or provide insights.
|
||||
4. **Output Indicator:** Define the type or format of the desired output. This guides the model in presenting the information in a way that aligns with your expectations.
|
||||
|
||||
Here's an example prompt for a text classification task:
|
||||
|
||||
**Prompt:**
|
||||
|
||||
```python
|
||||
Classify the text into neutral, negative, or positive
|
||||
Text: I think the food was okay.
|
||||
Sentiment:
|
||||
```
|
||||
|
||||
In this example:
|
||||
|
||||
- **Instruction:** "Classify the text into neutral, negative, or positive."
|
||||
- **Input Data:** "I think the food was okay."
|
||||
- **Output Indicator:** "Sentiment."
|
||||
|
||||
Note that this example doesn't explicitly use context, but context can also be incorporated into the prompt to provide additional information that aids the model in understanding the task better.
|
||||
|
||||
It's important to highlight that **not** all four elements are always necessary for a prompt, and the format can vary based on the specific task. The key is to structure prompts in a way that effectively communicates the user's intent and guides the model to produce relevant and accurate responses.
|
||||
|
||||
OpenAI has recently provided guidelines on best practices for prompt engineering using the OpenAI API. For a detailed understanding, you can explore the guidelines [here](https://help.openai.com/en/articles/6654000-best-practices-for-prompt-engineering-with-openai-api), the below points gives a brief summary:
|
||||
|
||||
1. **Use the Latest Model:** For optimal results, it is recommended to use the latest and most capable models.
|
||||
2. **Structure Instructions:** Place instructions at the beginning of the prompt and use ### or """ to separate the instruction and context for clarity and effectiveness.
|
||||
3. **Be Specific and Descriptive:** Clearly articulate the desired context, outcome, length, format, style, etc., in a specific and detailed manner.
|
||||
4. **Specify Output Format with Examples:** Clearly express the desired output format through examples, making it easier for the model to understand and respond accurately.
|
||||
5. **Use Zero-shot, Few-shot, and Fine-tune Approach:** Begin with a zero-shot approach, followed by a few-shot approach (providing examples). If neither works, consider fine-tuning the model.
|
||||
6. **Avoid Fluffy Descriptions:** Reduce vague and imprecise descriptions. Instead, use clear instructions and avoid unnecessary verbosity.
|
||||
7. **Provide Positive Guidance:** Instead of stating what not to do, clearly state what actions should be taken in a given situation, offering positive guidance.
|
||||
8. **Code Generation Specific - Use "Leading Words":** When generating code, utilize "leading words" to guide the model toward a specific pattern or language, improving the accuracy of code generation.
|
||||
|
||||
💡It’s also important to note that crafting effective prompts is an iterative process, and you may need to experiment to find the most suitable approach for your specific use case. Prompt patterns may be specific to models and how they were trained (architecture, datasets used etc.)
|
||||
|
||||
Explore these [examples](https://www.promptingguide.ai/introduction/examples) of prompts to gain a better understanding of how to craft effective prompts in different use-cases.
|
||||
|
||||
## Advanced Prompting Techniques
|
||||
|
||||
Prompting techniques constitute a rapidly evolving area of research, with researchers continually exploring novel methods to effectively prompt models for optimal performance. The simplest forms of prompting include zero-shot, where only instructions are provided, and few-shot, where examples are given, and the language model (LLM) is tasked with replication. More intricate techniques are elucidated in various research papers. While the provided list is not exhaustive, existing prompting methods can be tentatively classified into high-level categories. It's crucial to note that these classes are derived from current techniques and are not exhaustive or definitive; they are subject to evolution and modification, reflecting the dynamic nature of advancements in this field. It's important to highlight that numerous methods may fall into one or more of these classes, exhibiting overlapping characteristics to get the benefits offered by multiple categories.
|
||||
|
||||
|
||||
|
||||
### A. Step**-by-Step Modular Decomposition**
|
||||
|
||||
These methods involve breaking down complex problems into smaller, manageable steps, facilitating a structured approach to problem-solving. These methods guide the LLM through a sequence of intermediate steps, allowing it to focus on solving one step at a time rather than tackling the entire problem in a single step. This approach enhances the reasoning abilities of LLMs and is particularly useful for tasks requiring multi-step thinking.
|
||||
|
||||
Examples of methods falling under this category include:
|
||||
|
||||
1. **Chain-of-Thought (CoT) Prompting:**
|
||||
|
||||
Chain-of-Thought (CoT) Prompting is a technique to enhance complex reasoning capabilities through intermediate reasoning steps. This method involves providing a sequence of reasoning steps that guide a large language model (LLM) through a problem, allowing it to focus on solving one step at a time.
|
||||
|
||||
In the provided example below, the prompt involves evaluating whether the sum of odd numbers in a given group is an even number. The LLM is guided to reason through each example step by step, providing intermediate reasoning before arriving at the final answer. The output shows that the model successfully solves the problem by considering the odd numbers and their sums.
|
||||
|
||||
|
||||
|
||||
Image Source: [Wei et al. (2022)](https://arxiv.org/abs/2201.11903)
|
||||
|
||||
1a. **Zero-shot/Few-Shot CoT Prompting:**
|
||||
|
||||
Zero-shot involves adding the prompt "Let's think step by step" to the original question to guide the LLM through a systematic reasoning process. Few-shot prompting provides the model with a few examples of similar problems to enhance reasoning abilities. These CoT methods prompt significantly improves the model's performance by explicitly instructing it to think through the problem step by step. In contrast, without the special prompt, the model fails to provide the correct answer.
|
||||
|
||||
|
||||
|
||||
Image Source: [Kojima et al. (2022)](https://arxiv.org/abs/2205.11916)
|
||||
|
||||
1b. **Automatic Chain-of-Thought (Auto-CoT):**
|
||||
|
||||
Automatic Chain-of-Thought (Auto-CoT) was designed to automate the generation of reasoning chains for demonstrations. Instead of manually crafting examples, Auto-CoT leverages LLMs with a "Let's think step by step" prompt to automatically generate reasoning chains one by one.
|
||||
|
||||
|
||||
|
||||
Image Source: [Zhang et al. (2022)](https://arxiv.org/abs/2210.03493)
|
||||
|
||||
The Auto-CoT process involves two main stages:
|
||||
|
||||
1. **Question Clustering:** Partition questions into clusters based on similarity.
|
||||
2. **Demonstration Sampling:** Select a representative question from each cluster and generate its reasoning chain using Zero-Shot-CoT with simple heuristics.
|
||||
|
||||
The goal is to eliminate manual efforts in creating diverse and effective examples. Auto-CoT ensures diversity in demonstrations, and the heuristic-based approach encourages the model to generate simple yet accurate reasoning chains.
|
||||
|
||||
Overall, these CoT prompting techniques showcase the effectiveness of guiding LLMs through step-by-step reasoning for improved problem-solving and demonstration generation.
|
||||
|
||||
1. **Tree-of-Thoughts (ToT) Prompting**
|
||||
|
||||
Tree-of-Thoughts (ToT) Prompting is a technique that extends the Chain-of-Thought approach. It allows language models to explore coherent units of text ("thoughts") as intermediate steps towards problem-solving. ToT enables models to make deliberate decisions, consider multiple reasoning paths, and self-evaluate choices. It introduces a structured framework where models can look ahead or backtrack as needed during the reasoning process. ToT Prompting provides a more structured and dynamic approach to reasoning, allowing language models to navigate complex problems with greater flexibility and strategic decision-making. It is particularly beneficial for tasks that require comprehensive and adaptive reasoning capabilities.
|
||||
|
||||
**Key Characteristics:**
|
||||
|
||||
- **Coherent Units ("Thoughts"):** ToT prompts LLMs to consider coherent units of text as intermediate reasoning steps.
|
||||
- **Deliberate Decision-Making:** Enables models to make decisions intentionally and evaluate different reasoning paths.
|
||||
- **Backtracking and Looking Ahead:** Allows models to backtrack or look ahead during the reasoning process, providing flexibility in problem-solving.
|
||||
|
||||
|
||||
|
||||
Image Source: [Yao et el. (2023)](https://arxiv.org/abs/2305.10601)
|
||||
|
||||
1. **Graph of Thought Prompting**
|
||||
|
||||
This work arises from the fact that human thought processes often follow non-linear patterns, deviating from simple sequential chains. In response, the authors propose Graph-of-Thought (GoT) reasoning, a novel approach that models thoughts not just as chains but as graphs, capturing the intricacies of non-sequential thinking.
|
||||
|
||||
This extension introduces a paradigm shift in representing thought units. Nodes in the graph symbolize these thought units, and edges depict connections, presenting a more realistic portrayal of the complexities inherent in human cognition. Unlike traditional trees, GoT employs Directed Acyclic Graphs (DAGs), allowing the modeling of paths that fork and converge. This divergence provides GoT with a significant advantage over conventional linear approaches.
|
||||
|
||||
The GoT reasoning model operates in a two-stage framework. Initially, it generates rationales, and subsequently, it produces the final answer. To facilitate this, the model leverages a Graph-of-Thoughts encoder for representation learning. The integration of GoT representations with the original input occurs through a gated fusion mechanism, enabling the model to combine both linear and non-linear aspects of thought processes.
|
||||
|
||||
|
||||
|
||||
Image Source: [Yao et el. (2023)](https://arxiv.org/abs/2305.16582)
|
||||
|
||||
### B. Comprehensive **Reasoning and Verification**
|
||||
|
||||
Comprehensive Reasoning and Verification methods in prompting entail a more sophisticated approach where reasoning is not just confined to providing a final answer but involves generating detailed intermediate steps. The distinctive aspect of these techniques is the integration of a self-verification mechanism within the framework. As the LLM generates intermediate answers or reasoning traces, it autonomously verifies their consistency and correctness. If the internal verification yields a false result, the model iteratively refines its responses, ensuring that the generated reasoning aligns with the expected logical coherence. These checks contributes to a more robust and reliable reasoning process, allowing the model to adapt and refine its outputs based on internal validation
|
||||
|
||||
1. **Automatic Prompt Engineer**
|
||||
|
||||
Automatic Prompt Engineer (APE) is a technique that treats instructions as programmable elements and seeks to optimize them by conducting a search across a pool of instruction candidates proposed by an LLM. Drawing inspiration from classical program synthesis and human prompt engineering, APE employs a scoring function to evaluate the effectiveness of candidate instructions. The selected instruction, determined by the highest score, is then utilized as the prompt for the LLM. This automated approach aims to enhance the efficiency of prompt generation, aligning with classical program synthesis principles and leveraging the knowledge embedded in large language models to improve overall performance in producing desired outputs.
|
||||
|
||||
|
||||
|
||||
Image Source: [Zhou et al., (2022)](https://arxiv.org/abs/2211.01910)
|
||||
|
||||
1. **Chain of Verification (CoVe)**
|
||||
|
||||
The Chain-of-Verification (CoVe) method addresses the challenge of hallucination in large language models by introducing a systematic verification process. It begins with the model drafting an initial response to a user query, potentially containing inaccuracies. CoVe then plans and poses independent verification questions, aiming to fact-check the initial response without bias. The model answers these questions, and based on the verification outcomes, generates a final response, incorporating corrections and improvements identified through the verification process. CoVe ensures unbiased verification, leading to enhanced factual accuracy in the final response, and contributes to improved overall model performance by mitigating the generation of inaccurate information.
|
||||
|
||||
|
||||
|
||||
Image Source: [Dhuliawala et al.2023](https://arxiv.org/abs/2309.11495)
|
||||
|
||||
1. **Self Consistency**
|
||||
|
||||
Self Consistency represents a refinement in prompt engineering, specifically targeting the limitations of naive greedy decoding in chain-of-thought prompting. The core concept involves sampling multiple diverse reasoning paths using few-shot CoT and leveraging the generated responses to identify the most consistent answer. This method aims to enhance the performance of CoT prompting, particularly in tasks that demand arithmetic and commonsense reasoning. By introducing diversity in reasoning paths and prioritizing consistency, Self Consistency contributes to more robust and accurate language model responses within the CoT framework.
|
||||
|
||||
|
||||
|
||||
Image Source: [Wang et al. (2022)](https://arxiv.org/pdf/2203.11171.pdf)
|
||||
|
||||
1. **ReACT**
|
||||
|
||||
The ReAct framework combines reasoning and action in LLMs to enhance their capabilities in dynamic tasks. The framework involves generating both verbal reasoning traces and task-specific actions in an interleaved manner. ReAct aims to address the limitations of models, like chain-of-thought , that lack access to the external world and can encounter issues such as fact hallucination and error propagation. Inspired by the synergy between "acting" and "reasoning" in human learning and decision-making, ReAct prompts LLMs to create, maintain, and adjust plans for acting dynamically. The model can interact with external environments, such as knowledge bases, to retrieve additional information, leading to more reliable and factual responses.
|
||||
|
||||
|
||||
|
||||
Image Source: [Yao et al., 2022](https://arxiv.org/abs/2210.03629)
|
||||
|
||||
**How ReAct Works:**
|
||||
|
||||
1. **Dynamic Reasoning and Acting:** ReAct generates both verbal reasoning traces and actions, allowing for dynamic reasoning in response to complex tasks.
|
||||
2. **Interaction with External Environments: T**he action step enables interaction with external sources, like search engines or knowledge bases, to gather information and refine reasoning.
|
||||
3. **Improved Task Performance:** The framework's integration of reasoning and action contributes to outperforming state-of-the-art baselines on language and decision-making tasks.
|
||||
4. **Enhanced Human Interpretability:** ReAct leads to improved human interpretability and trustworthiness of LLMs, making their responses more understandable and reliable.
|
||||
|
||||
### C. Usage of External Tools/Knowledge or Aggregation
|
||||
|
||||
This category of prompting methods encompasses techniques that leverage external sources, tools, or aggregated information to enhance the performance of LLMs. These methods recognize the importance of accessing external knowledge or tools for more informed and contextually rich responses. Aggregation techniques involve harnessing the power of multiple responses to enhance the robustness. This approach recognizes that diverse perspectives and reasoning paths can contribute to more reliable and comprehensive answers. Here's an overview:
|
||||
|
||||
1. **Active Prompting (Aggregation)**
|
||||
|
||||
Active Prompting was designed to enhance the adaptability LLMs to various tasks by dynamically selecting task-specific example prompts. Chain-of-Thought methods typically rely on a fixed set of human-annotated exemplars, which may not always be the most effective for diverse tasks. Here's how Active Prompting addresses this challenge:
|
||||
|
||||
1. **Dynamic Querying:**
|
||||
- The process begins by querying the LLM with or without a few CoT examples for a set of training questions.
|
||||
- The model generates k possible answers, introducing an element of uncertainty in its responses.
|
||||
2. **Uncertainty Metric:**
|
||||
- An uncertainty metric is calculated based on the disagreement among the k generated answers. This metric reflects the model's uncertainty about the most appropriate response.
|
||||
3. **Selective Annotation:**
|
||||
- The questions with the highest uncertainty, indicating a lack of consensus in the model's responses, are selected for annotation by humans.
|
||||
- Humans provide new annotated exemplars specifically tailored to address the uncertainties identified by the LLM.
|
||||
4. **Adaptive Learning:**
|
||||
- The newly annotated exemplars are incorporated into the training data, enriching the model's understanding and adaptability for those specific questions.
|
||||
- The model learns from the newly annotated examples, adjusting its responses based on the task-specific guidance provided.
|
||||
|
||||
Active Prompting's dynamic adaptation mechanism enables LLMs to actively seek and incorporate task-specific examples that align with the challenges posed by different tasks. By leveraging human-annotated exemplars for uncertain cases, this approach contributes to a more contextually aware and effective performance across diverse tasks.
|
||||
|
||||
|
||||
|
||||
Image Source: [Diao et al., (2023)](https://arxiv.org/pdf/2302.12246.pdf)
|
||||
|
||||
1. **Automatic Multi-step Reasoning and Tool-use (ART) (External Tools)**
|
||||
|
||||
ART emphasizes on task handling with LLMs. This framework integrates Chain-of-Thought prompting and tool usage by employing a frozen LLM. Instead of manually crafting demonstrations, ART selects task-specific examples from a library and enables the model to automatically generate intermediate reasoning steps. During test time, it integrates external tools into the reasoning process, fostering zero-shot generalization for new tasks. ART is not only extensible, allowing for human updates to task and tool libraries, but also promotes adaptability and versatility in addressing a variety of tasks with LLMs.
|
||||
|
||||
|
||||
|
||||
Image Source: [Paranjape et al., (2023)](https://arxiv.org/abs/2303.09014)
|
||||
|
||||
1. **Chain-of-Knowledge (CoK)**
|
||||
|
||||
This framework aims to bolster LLMs by dynamically integrating grounding information from diverse sources, fostering more factual rationales and mitigating the risk of hallucination during generation. CoK operates through three key stages: reasoning preparation, dynamic knowledge adapting, and answer consolidation. It starts by formulating initial rationales and answers while identifying relevant knowledge domains. Subsequently, it refines these rationales incrementally by adapting knowledge from the identified domains, ultimately providing a robust foundation for the final answer. The figure below illustrates a comparison with other methods, highlighting CoK's incorporation of heterogeneous sources for knowledge retrieval and dynamic knowledge adapting.
|
||||
|
||||
|
||||
|
||||
Image Source: [Li et al. 2024](https://arxiv.org/abs/2401.04398)
|
||||
|
||||
## Risks
|
||||
|
||||
Prompting comes with various risks, and prompt hacking is a notable concern that exploits vulnerabilities in LLMs. The risks associated with prompting include:
|
||||
|
||||
1. **Prompt Injection:**
|
||||
- *Risk:* Malicious actors can inject harmful or misleading content into prompts, leading LLMs to generate inappropriate, biased, or false outputs.
|
||||
- *Context:* Untrusted text used in prompts can be manipulated to make the model say anything the attacker desires, compromising the integrity of generated content.
|
||||
2. **Prompt Leaking:**
|
||||
- *Risk:* Attackers may extract sensitive information from LLM responses, posing privacy and security concerns.
|
||||
- *Context:* Changing the user_input to attempt to leak the prompt itself is a form of prompt leaking, potentially revealing internal information.
|
||||
3. **Jailbreaking:**
|
||||
- *Risk:* Jailbreaking allows users to bypass safety and moderation features, leading to the generation of controversial, harmful, or inappropriate responses.
|
||||
- *Context:* Prompt hacking methodologies, such as pretending, can exploit the model's difficulty in rejecting harmful prompts, enabling users to ask any question they desire.
|
||||
4. **Bias and Misinformation:**
|
||||
- *Risk:* Prompts that introduce biased or misleading information can result in outputs that perpetuate or amplify existing biases and spread misinformation.
|
||||
- *Context:* Crafted prompts can manipulate LLMs into producing biased or inaccurate responses, contributing to the reinforcement of societal biases.
|
||||
5. **Security Concerns:**
|
||||
- *Risk:* Prompt hacking poses a broader security threat, allowing attackers to compromise the integrity of LLM-generated content and potentially exploit models for malicious purposes.
|
||||
- *Context:* Defensive measures, including prompt-based defenses and continuous monitoring, are essential to mitigate security risks associated with prompt hacking.
|
||||
|
||||
To address these risks, it is crucial to implement robust defensive strategies, conduct regular audits of model behavior, and stay vigilant against potential vulnerabilities introduced through prompts. Additionally, ongoing research and development are necessary to enhance the resilience of LLMs against prompt-based attacks and mitigate biases in generated content.
|
||||
|
||||
## Popular Tools
|
||||
|
||||
Here is a collection of well-known tools for prompt engineering. While some function as end-to-end app development frameworks, others are tailored for prompt generation and maintenance or evaluation purposes. The listed tools are predominantly open source or free to use and have demonstrated good adaptability. It's important to note that there are additional tools available, although they might be less widely recognized or require payment.
|
||||
|
||||
1. **[PromptAppGPT](https://github.com/mleoking/PromptAppGPT):**
|
||||
- *Description:* A low-code prompt-based rapid app development framework.
|
||||
- *Features:* Low-code prompt-based development, GPT text and DALLE image generation, online prompt editor/compiler/runner, automatic UI generation, support for plug-in extensions.
|
||||
- *Objective:* Enables natural language app development based on GPT, lowering the barrier to GPT application development.
|
||||
2. **[PromptBench](https://github.com/microsoft/promptbench):**
|
||||
- *Description:* A PyTorch-based Python package for the evaluation of LLMs.
|
||||
- *Features:* User-friendly APIs for quick model performance assessment, prompt engineering methods (Few-shot Chain-of-Thought, Emotion Prompt, Expert Prompting), evaluation of adversarial prompts, dynamic evaluation to mitigate potential test data contamination.
|
||||
- *Objective:* Facilitates the evaluation and assessment of LLMs with various capabilities, including prompt engineering and adversarial prompt evaluation.
|
||||
3. **[Prompt Engine](https://github.com/microsoft/prompt-engine):**
|
||||
- *Description:* An NPM utility library for creating and maintaining prompts for LLMs.
|
||||
- *Background:* Aims to simplify prompt engineering for LLMs like GPT-3 and Codex, providing utilities for crafting inputs that coax specific outputs from the models.
|
||||
- *Objective:* Facilitates the creation and maintenance of prompts, codifying patterns and practices around prompt engineering.
|
||||
4. **[Prompts AI](https://github.com/sevazhidkov/prompts-ai):**
|
||||
- *Description:* An advanced GPT-3 playground with a focus on helping users discover GPT-3 capabilities and assisting developers in prompt engineering for specific use cases.
|
||||
- *Goals:* Aid first-time GPT-3 users, experiment with prompt engineering, optimize the product for use cases like creative writing, classification, and chat bots.
|
||||
5. **[OpenPrompt](https://github.com/thunlp/OpenPrompt):**
|
||||
- *Description:* A library built upon PyTorch for prompt-learning, adapting LLMs to downstream NLP tasks.
|
||||
- *Features:* Standard, flexible, and extensible framework for deploying prompt-learning pipelines, supporting loading PLMs from huggingface transformers.
|
||||
- *Objective:* Provides a standardized approach to prompt-learning, making it easier to adapt PLMs for specific NLP tasks.
|
||||
6. **[Promptify](https://github.com/promptslab/Promptify):**
|
||||
- *Features:* Test suite for LLM prompts, perform NLP tasks in a few lines of code, handle out-of-bounds predictions, output provided as Python objects for easy parsing, support for custom examples and samples, run inference on models from the Huggingface Hub.
|
||||
- *Objective:* Aims to facilitate prompt testing for LLMs, simplify NLP tasks, and optimize prompts to reduce OpenAI token costs.
|
||||
|
||||
## Read/Watch These Resources (Optional)
|
||||
|
||||
1. [https://www.promptingguide.ai/](https://www.promptingguide.ai/)
|
||||
2. [https://aman.ai/primers/ai/prompt-engineering/](https://aman.ai/primers/ai/prompt-engineering/)
|
||||
3. [https://www.deeplearning.ai/short-courses/chatgpt-prompt-engineering-for-developers/](https://www.deeplearning.ai/short-courses/chatgpt-prompt-engineering-for-developers/)
|
||||
4. [https://learnprompting.org/courses](https://learnprompting.org/courses)
|
||||
|
||||
## Read These Papers (Optional)
|
||||
|
||||
1. [https://arxiv.org/abs/2304.05970](https://arxiv.org/abs/2304.05970)
|
||||
2. [https://arxiv.org/abs/2309.11495](https://arxiv.org/abs/2309.11495)
|
||||
3. [https://arxiv.org/abs/2310.08123](https://arxiv.org/abs/2310.08123)
|
||||
4. [https://arxiv.org/abs/2305.13626](https://arxiv.org/abs/2305.13626)
|
||||
@@ -0,0 +1,161 @@
|
||||
# [Week 3] Fine Tuning LLMs
|
||||
|
||||
## ETMI5: Explain to Me in 5
|
||||
|
||||
In this section, we will go over the Fine-Tuning domain adaptation method for LLMs. Fine-tuning involves further training pre-trained models for specific tasks or domains, adapting them to new data distributions, and enhancing efficiency by leveraging pre-existing knowledge. It is crucial for tasks where generic models may not excel. Two main types of fine-tuning include unsupervised (updating models without modifying behavior) and supervised (updating models with labeled data). We emphasize on the popular supervised method-Instruction fine-tuning which augments input-output examples with explicit instructions for better generalization. We’ll dig deeper into Reinforcement Learning from Human Feedback (RLHF) which incorporates human feedback for model fine-tuning and Direct Preference Optimization (DPO) that directly optimizes models based on user preferences. We provide an overview of Parameter-Efficient Fine-Tuning (PEFT) approaches as well where selective updates are made to model parameters, addressing computational challenges, memory efficiency, and allowing versatility across modalities.
|
||||
|
||||
## Introduction
|
||||
|
||||
Fine-tuning is the process of taking pre-trained models and further training them on smaller, domain-specific datasets. The aim is to refine their capabilities and enhance performance in a specific task or domain. This process transforms general-purpose models into specialized ones, bridging the gap between generic pre-trained models and the unique requirements of particular applications.
|
||||
|
||||
Consider OpenAI's GPT-3, a state-of-the-art LLM designed for a broad range of NLP tasks. To illustrate the need for fine-tuning, imagine a healthcare organization wanting to use GPT-3 to assist doctors in generating patient reports from textual notes. While GPT-3 is proficient in general text understanding, it may not be optimized for intricate medical terms and specific healthcare jargon.
|
||||
|
||||
In this scenario, the organization engages in fine-tuning GPT-3 on a dataset filled with medical reports and patient notes. The model becomes more familiar with medical terminologies, nuances of clinical language, and typical report structures. As a result, after fine-tuning, GPT-3 is better suited to assist doctors in generating accurate and coherent patient reports, showcasing its adaptability for specific tasks.
|
||||
|
||||
Fine-tuning is not exclusive to language models; any machine learning model may require retraining under certain circumstances. It involves adjusting model parameters to align with the distribution of new, specific datasets. This process is illustrated with the example of a convolutional neural network trained to identify images of automobiles and the challenges it faces when applied to detecting trucks on highways.
|
||||
|
||||
The key principle behind fine-tuning is to leverage pre-trained models and recalibrate their parameters using novel data, adapting them to new contexts or applications. It is particularly beneficial when the distribution of training data significantly differs from the requirements of a specific application.The choice of the base general model model depends on the nature of the task, such as text generation or text classification.
|
||||
|
||||
## Why Fine-Tuning?
|
||||
|
||||
While large language models are indeed trained on a diverse set of tasks, the need for fine-tuning arises because these large generic models are designed to perform reasonably well across various applications, but not necessarily excel in a specific task. The optimization of generic models is aimed at achieving decent performance across a range of tasks, making them versatile but not specialized.
|
||||
|
||||
Fine-tuning becomes essential to ensure that a model attains exceptional proficiency in a particular task or domain of interest. The emphasis shifts from achieving general competence to achieving mastery in a specific application. This is particularly crucial when the model is intended for a focused use case, and overall general performance is not the primary concern.
|
||||
|
||||
In essence, generic large language models can be considered as being proficient in multiple tasks but not reaching the level of mastery in any. Fine-tuned models, on the other hand, undergo a tailored optimization process to become masters of a specific task or domain. Therefore, the decision to fine-tune models is driven by the necessity to achieve superior performance in targeted applications, making them highly effective specialists in their designated areas.
|
||||
|
||||
For a deeper understanding, explore why fine-tuning models for tasks in new domains is deemed crucial for several compelling reasons.
|
||||
|
||||
1. **Domain-Specific Adaptation:** Pre-trained LLMs may not be optimized for specific tasks or domains. Fine-tuning allows adaptation to the nuances and characteristics of a new domain, enhancing performance in domain-specific tasks. For instance, large generic LLMs might not be sufficiently trained on tasks like document analysis in the legal domain. Fine-tuning can allow the model to understand legal terminology and nuances for tasks like contract review.
|
||||
2. **Shifts in Data Distribution:** Models trained on one dataset may not generalize well to out-of-distribution examples. Fine-tuning helps align the model with the distribution of new data, addressing shifts in data characteristics and improving performance on specific tasks. For example: Fine-tuning a sentiment analysis model for social media comments. The distribution of language and sentiments on social media may differ significantly from the original training data, requiring adaptation for accurate sentiment classification.
|
||||
3. **Cost and Resource Efficiency:** Training a model from scratch on a new task often requires a large labeled dataset, which can be costly and time-consuming. Fine-tuning allows leveraging a pre-trained model's knowledge and adapting it to the new task with a smaller dataset, making the process more efficient. For example: Adapting a pre-trained model for a small e-commerce platform to recommend products based on user preferences. Fine-tuning is more resource-efficient than training a model from scratch with a limited dataset.
|
||||
4. **Out-of-Distribution Data Handling:**
|
||||
- Fine-tuning mitigates the suboptimal performance of pre-trained models when dealing with out-of-distribution examples. Instead of starting training anew, fine-tuning allows building upon the existing model's foundation with a relatively modest dataset. For example: Fine-tuning a speech recognition model for a new regional accent. The model can be adapted to recognize speech patterns specific to the new accent without extensive retraining.
|
||||
5. **Knowledge Transfer:**
|
||||
- Pre-trained models capture general patterns and knowledge from vast amounts of data during pre-training. Fine-tuning facilitates the transfer of this general knowledge to specific tasks, making it a valuable tool for leveraging pre-existing knowledge in new applications. For example: Transferring medical knowledge from a pre-trained model to a new healthcare chatbot. Fine-tuning with medical literature enables the model to provide accurate and contextually relevant responses in healthcare conversations.
|
||||
6. **Task-Specific Optimization:**
|
||||
- Fine-tuning enables the optimization of model parameters for task-specific objectives. For example, in the medical domain, fine-tuning an LLM with medical literature can enhance its performance in medical applications. For example: Optimizing a pre-trained model for code generation in a software development environment. Fine-tuning with code examples allows the model to better understand and generate code snippets.
|
||||
7. **Adaptation to User Preferences:** Fine-tuning allows adapting the model to user preferences and specific task requirements. It enables the model to generate more contextually relevant and task-specific responses. For example: Fine-tuning a virtual assistant model to align with user preferences in language and tone. This ensures that the assistant generates responses that match the user's communication style.
|
||||
8. **Continual Learning:** Fine-tuning supports continual learning by allowing models to adapt to evolving data and user requirements over time. It enables models to stay relevant and effective in dynamic environments. For instance: Continually updating a news summarization model to adapt to evolving news topics and user preferences. Fine-tuning enables the model to stay relevant and provide timely summaries.
|
||||
|
||||
In summary, fine-tuning is a powerful technique that enables organizations to adapt pre-trained models to specific tasks, domains, and user requirements, providing a practical and efficient solution for deploying models in real-world applications.
|
||||
|
||||
## Types of Fine-Tuning
|
||||
|
||||
At a high level, fine-tuning methods for language models can be categorized into two main approaches: supervised and unsupervised. In machine learning, supervised methods involve having labeled data, where the model is trained on examples with corresponding desired outputs. On the other hand, unsupervised methods operate with unlabeled data, focusing on extracting patterns and structures without explicit labels.
|
||||
|
||||
### **Unsupervised Fine-Tuning Methods:**
|
||||
|
||||
1. **Unsupervised Full Fine-Tuning:** Unsupervised fine-tuning becomes relevant when there is a need to update the knowledge base of an LLM without modifying its existing behavior. For instance, if the goal is to fine-tune the model on legal literature or adapt it to a new language, an unstructured dataset containing legal documents or texts in the desired language can be utilized. In such cases, the unstructured dataset comprises articles, legal papers, or relevant content from authoritative sources in the legal domain. This approach allows the model to effectively refine its understanding and adapt to the nuances of legal language without relying on labeled examples, showcasing the versatility of unsupervised fine-tuning across various domains.
|
||||
2. **Contrastive Learning:** Contrastive learning is a method employed in fine-tuning language models, emphasizing the training of the model to discern between similar and dissimilar examples in the latent space. The objective is to optimize the model's ability to distinguish subtle nuances and patterns within the data. This is achieved by encouraging the model to bring similar examples closer together in the latent space while pushing dissimilar examples apart. The resulting learned representations enable the model to capture intricate relationships and differences in the input data. Contrastive learning is particularly beneficial in tasks where a nuanced understanding of similarities and distinctions is crucial, making it a valuable technique for refining language models for specific applications that require fine-grained discrimination.
|
||||
|
||||
### **Supervised Fine-Tuning Methods:**
|
||||
|
||||
1. **Parameter-Efficient Fine-Tuning: It** is a fine-tuning strategy that aims to reduce the computational expenses associated with updating the parameters of a language model. Instead of updating all parameters during fine-tuning, PEFT focuses on selectively updating a small set of parameters, often referred to as a low-dimensional matrix. One prominent example of PEFT is the low-rank adaptation (LoRA) technique. LoRA operates on the premise that fine-tuning a foundational model for downstream tasks only requires updates across certain parameters. The low-rank matrix effectively represents the relevant space related to the target task, and training this matrix is performed instead of adjusting the entire model's parameters. PEFT, and specifically techniques like LoRA, can significantly decrease the costs associated with fine-tuning, making it a more efficient process.
|
||||
2. **Supervised Full Fine-Tuning**: It involves updating all parameters of the language model during the training process. Unlike PEFT, where only a subset of parameters is modified, full fine-tuning requires sufficient memory and computational resources to store and process all components being updated. This comprehensive approach results in a new version of the model with updated weights across all layers. While full fine-tuning is more resource-intensive, it ensures that the entire model is adapted to the specific task or domain, making it suitable for situations where a thorough adjustment of the language model is desired.
|
||||
3. **Instruction Fine-Tuning:** Instruction Fine-Tuning involves the process of training a language model using examples that explicitly demonstrate how it should respond to specific queries or tasks. This method aims to enhance the model's performance on targeted tasks by providing explicit instructions within the training data. For instance, if the task involves summarization or translation, the dataset is curated to include examples with clear instructions like "summarize this text" or "translate this phrase." Instruction fine-tuning ensures that the model becomes adept at understanding and executing specific instructions, making it suitable for applications where precise task execution is essential.
|
||||
4. **Reinforcement Learning from Human Feedback (RLHF):** RLHF takes the concept of supervised fine-tuning a step further by incorporating reinforcement learning principles. In RLHF, human evaluators are enlisted to rate the model's outputs based on specific prompts. These ratings serve as a form of reward, guiding the model to optimize its parameters to maximize positive feedback. RLHF is a resource-intensive process that leverages human preferences to refine the model's behavior. Human feedback contributes to training a reward model that guides the subsequent reinforcement learning phase, resulting in improved model performance aligned with human preferences.
|
||||
|
||||
Techniques such as contrastive learning, as well as supervised and unsupervised fine-tuning, are not exclusive to LLMs and have been employed for domain adaptation even before the advent of LLMs. However, following the rise of LLMs, there has been a notable increase in the prominence of techniques such as RLHF, instruction fine-tuning, and PEFT. In the upcoming sections, we will explore these methodologies in greater detail to comprehend their applications and significance.
|
||||
|
||||
## Instruction Fine-Tuning
|
||||
|
||||
Instruction fine-tuning is a method that has gained prominence in making LLMs more practical for real-world applications. In contrast to standard supervised fine-tuning, where models are trained on input examples and corresponding outputs, instruction tuning involves augmenting input-output examples with explicit instructions. This unique approach enables instruction-tuned models to generalize more effectively to new tasks. The data for instruction tuning is constructed differently, with instructions providing additional context for the model.
|
||||
|
||||
|
||||
|
||||
Image Source: [Wei et al., 2022](https://openreview.net/forum?id=gEZrGCozdqR)
|
||||
|
||||
One notable dataset for instruction tuning is "Natural Instructions". This dataset consists of 193,000 instruction-output examples sourced from 61 existing English NLP tasks. The uniqueness of this dataset lies in its structured approach, where crowd-sourced instructions from each task are aligned to a common schema. Each instruction is associated with a task, providing explicit guidance on how the model should respond. The instructions cover various fields, including a definition, things to avoid, and positive and negative examples. This structured nature makes the dataset valuable for fine-tuning models, as it provides clear and detailed instructions for the desired task. However, it's worth noting that the outputs in this dataset are relatively short, which might make the data less suitable for generating long-form content. Despite this limitation, Natural Instructions serves as a rich resource for training models through instruction tuning, enhancing their adaptability to specific NLP tasks. The below image contains an example instruction format
|
||||
|
||||
|
||||
|
||||
Image Source: **[Mishra et al., 2022](https://aclanthology.org/2022.acl-long.244/)**
|
||||
|
||||
Instruction fine-tuning has become a valuable tool in the evolving landscape of natural language processing and machine learning, enabling LLMs to adapt to specific tasks with nuanced instructions.
|
||||
|
||||
## **Reinforcement Learning from Human Feedback (RLHF)**
|
||||
|
||||
Reinforcement Learning from Human Feedback is a methodology designed to enhance language models by incorporating human feedback, aligning them more closely with intricate human values. The RLHF process comprises three fundamental steps:
|
||||
|
||||
**1. Pretraining Language Models (LMs):**
|
||||
RLHF initiates with a pretrained LM, typically achieved through classical pretraining objectives. The initial LM, which can vary in size, is flexible in choice. While optional, the initial LM can undergo fine-tuning on additional data. The crucial aspect is to have a model that exhibits a positive response to diverse instructions.
|
||||
|
||||
**2. Reward Model Training:**
|
||||
The subsequent step involves generating a reward model (RM) calibrated with human preferences. This model assigns scalar rewards to sequences of text, reflecting human preferences. The dataset for training the reward model is generated by sampling prompts and passing them through the initial LM to produce text. Human annotators rank the generated text outputs, and these rankings are used to create a regularized dataset for training the reward model. The reward function combines the preference model and a penalty on the difference between the RL policy and the initial model.
|
||||
|
||||
**3. Fine-Tuning with RL:**
|
||||
The final step entails fine-tuning the initial LLM using reinforcement learning. Proximal Policy Optimization (PPO) is a commonly used RL algorithm for this task. The RL policy is the LM that takes in a prompt and produces text, with actions corresponding to tokens in the LM's vocabulary. The reward function, derived from the preference model and a constraint on policy shift, guides the fine-tuning. PPO updates the LM's parameters to maximize the reward metrics in the current batch of prompt-generation pairs. Some parameters of the LM are frozen due to computational constraints, and the fine-tuning aims to align the model with human preferences.
|
||||
|
||||
|
||||
|
||||
Image Source: [https://openai.com/research/instruction-following](https://openai.com/research/instruction-following)
|
||||
|
||||
💡If you’re lost understanding RL terms like PPO, policy etc. Think of this analogy- Fine-Tuning with RL, specifically using Proximal Policy Optimization (PPO), is similar to refining instructions to train a pet, such as teaching a dog tricks. Think of the dog initially learning with general guidance (policy) and receiving treats (rewards) for correct actions. Now, imagine the dog mastering a new trick but not quite perfectly. Fine-tuning, with PPO, involves adjusting your instructions slightly based on how well the dog performs, similar to tweaking the model's behavior (policy) in Reinforcement Learning. It's like refining the instructions to optimize the learning process, much like perfecting your pet's tricks through gradual adjustments and treats for better performance.
|
||||
|
||||
## Direct Preference Optimization DPO (*Bonus Topic)*
|
||||
|
||||
Direct Preference Optimization (DPO) is an equivalent of RLHF and has been gaining significant traction these days. DPO offers a straightforward method for fine-tuning large language models based on human preferences. It eliminates the need for a complex reward model and directly incorporates user feedback into the optimization process. In DPO, users simply compare two model-generated outputs and express their preferences, allowing the LLM to adjust its behavior accordingly. This user-friendly approach comes with several advantages, including ease of implementation, computational efficiency, and greater control over the LLM's behavior.
|
||||
|
||||
|
||||
|
||||
Image Source: [Rafailov, Rafael, et al.](https://arxiv.org/html/2305.18290v2)
|
||||
|
||||
💡In the context of LLMs, maximum likelihood is a principle used during the training of the model. Imagine the model is like a writer trying to predict the next word in a sentence. Maximum likelihood training involves adjusting the model's parameters (the factors that influence its predictions) to maximize the likelihood of generating the actual sequences of words observed in the training data. It's like tuning the writer's skills to make the sentences they create most closely resemble the sentences they've seen before. So, maximum likelihood helps the LLM learn to generate text that is most similar to the examples it was trained on.
|
||||
|
||||
***DPO**:* DPO takes a straightforward approach by directly optimizing the LM based on user preferences without the need for a separate reward model. Users compare two model-generated outputs, expressing their preferences to guide the optimization process.
|
||||
|
||||
***RLHF**:* RLHF follows a more structured path, leveraging reinforcement learning principles. It involves training a reward model that learns to identify and reward desirable LM outputs. The reward model then guides the LM's training process, shaping its behavior towards achieving positive outcomes.
|
||||
|
||||
### **DPO (Direct Policy Optimization) vs. RLHF (Reinforcement Learning from Human Feedback): Understanding the Differences**
|
||||
|
||||
**DPO - A Simpler Approach:**
|
||||
Direct Policy Optimization (DPO) takes a straightforward path, sidestepping the need for a complex reward model. It directly optimizes the Large Language Model (LLM) based on user preferences, where users compare two outputs and indicate their preference. This simplicity results in key advantages:
|
||||
|
||||
1. **Ease of Implementation:** DPO is more user-friendly as it eliminates the need for designing and training a separate reward model, making it accessible to a broader audience.
|
||||
2. **Computational Efficiency:** Operating directly on the LLM, DPO leads to faster training times and lower computational costs compared to RLHF, which involves multiple phases.
|
||||
3. **Greater Control:** Users have direct control over the LLM's behavior, guiding it toward specific goals and preferences without the complexities of RLHF.
|
||||
4. **Faster Convergence:** Due to its simpler structure and direct optimization, DPO often achieves desired results faster, making it suitable for tasks with rapid iteration needs.
|
||||
5. **Improved Performance:** Recent research suggests that DPO can outperform RLHF in scenarios like sentiment control and response quality, particularly in summarization and dialogue tasks.
|
||||
|
||||
**RLHF - A More Structured Approach:**
|
||||
It follows a more structured path, leveraging reinforcement learning principles. It includes three training phases: pre-training, reward model training, and fine-tuning with reinforcement learning. While flexible, RLHF comes with complexities:
|
||||
|
||||
1. **Complexity:** RLHF can be more complex and sometimes unstable, demanding more computational resources and dealing with challenges like convergence, drift, or uncorrelated distribution problems.
|
||||
2. **Flexibility in Defining Rewards:** RLHF allows for more nuanced reward structures, beneficial for tasks requiring precise control over the LLM's output.
|
||||
3. **Handling Diverse Feedback Formats:** RLHF can handle various forms of human feedback, including numerical ratings or textual corrections, whereas DPO primarily relies on binary preferences.
|
||||
4. **Handling Large Datasets:** RLHF can be more efficient in handling massive datasets, especially with distributed training techniques.
|
||||
|
||||
In summary, the choice depends on the specific task, available resources, and the desired level of control, with both methods offering strengths and weaknesses in different contexts. As advancements continue, these methods contribute to evolving and enhancing fine-tuning processes for LLMs.
|
||||
|
||||
## Parameter Efficient Fine-Tuning (PEFT)
|
||||
|
||||
Parameter-Efficient Fine-Tuning (PEFT) addresses the resource-intensive nature of fine-tuning LLMs. Unlike full fine-tuning that modifies all parameters, PEFT fine-tunes only a small subset of additional parameters while keeping the majority of pretrained model weights frozen. This selective approach minimizes computational requirements, mitigates catastrophic forgetting, and facilitates fine-tuning even with limited computational resources. PEFT, as a whole, offers a more efficient and practical method for adapting LLMs to specific downstream tasks without the need for extensive computational power and memory.
|
||||
|
||||
**Advantages of Parameter-Efficient Fine-Tuning (PEFT)**
|
||||
|
||||
1. **Computational Efficiency:** PEFT fine-tunes LLMs with significantly fewer parameters than full fine-tuning. This reduces the computational demands, making it feasible to fine-tune on less powerful hardware or in resource-constrained environments.
|
||||
2. **Memory Efficiency:** By freezing the majority of pretrained model weights, PEFT avoids excessive memory usage associated with modifying all parameters. This makes PEFT particularly suitable for tasks where memory constraints are a concern.
|
||||
3. **Catastrophic Forgetting Mitigation:** PEFT prevents catastrophic forgetting, a phenomenon observed during full fine-tuning where the model loses knowledge from its pre-trained state. This ensures that the LLM retains valuable information while adapting to new tasks.
|
||||
4. **Versatility Across Modalities:** PEFT extends beyond natural language processing tasks, proving effective in various modalities such as computer vision and audio. Its versatility makes it applicable to a wide range of downstream tasks.
|
||||
5. **Modular Adaptation for Multiple Tasks:** The modular nature of PEFT allows the same pretrained model to be adapted for multiple tasks by adding small task-specific weights. This avoids the need to store full copies for different applications, enhancing flexibility and efficiency.
|
||||
6. **INT8 Tuning:** PEFT's capabilities include INT8 (8-bit integer) tuning, showcasing its adaptability to different quantization techniques. This enables fine-tuning even on platforms with limited computational resources.
|
||||
|
||||
In summary, PEFT offers a practical and efficient solution for fine-tuning large language models, addressing computational and memory challenges while maintaining performance on downstream tasks.
|
||||
|
||||
A summary of the most popular PEFT methods are in the chart below. Please download for improved visibility.
|
||||
|
||||
[PEFT (1).pdf](https://github.com/aishwaryanr/awesome-generative-ai-resources/blob/main/free_courses/Applied_LLMs_Mastery_2024/img/PEFT_(1).pdf)
|
||||
|
||||
## Read/Watch These Resources (Optional)
|
||||
|
||||
1. [https://www.superannotate.com/blog/llm-fine-tuning](https://www.superannotate.com/blog/llm-fine-tuning)
|
||||
2. [https://www.deeplearning.ai/short-courses/finetuning-large-language-models/](https://www.deeplearning.ai/short-courses/finetuning-large-language-models/)
|
||||
3. [https://www.youtube.com/watch?v=eC6Hd1hFvos](https://www.youtube.com/watch?v=eC6Hd1hFvos)
|
||||
4. [https://www.labellerr.com/blog/comprehensive-guide-for-fine-tuning-of-llms/](https://www.labellerr.com/blog/comprehensive-guide-for-fine-tuning-of-llms/)
|
||||
|
||||
## Read These Papers (Optional)
|
||||
|
||||
1. [https://arxiv.org/abs/2303.15647](https://arxiv.org/abs/2303.15647)
|
||||
2. [https://arxiv.org/abs/2109.10686](https://arxiv.org/abs/2109.10686)
|
||||
3. [https://arxiv.org/abs/2304.01933](https://arxiv.org/abs/2304.01933)
|
||||
@@ -0,0 +1,201 @@
|
||||
# [Week 4] Retrieval Augmented Generation
|
||||
|
||||
## ETMI5: Explain to Me in 5
|
||||
|
||||
In this week’s content, we will do an in-depth exploration of Retrieval Augmented Generation (RAG), an AI framework that enhances the capabilities of Large Language Models by integrating real-time, contextually relevant information from external sources during the response generation process. It addresses the limitations of LLMs, such as inconsistency and lack of domain-specific knowledge, hence reducing the risk of generating incorrect or hallucinated responses.
|
||||
|
||||
RAG operates in three key phases: ingestion, retrieval, and synthesis. In the ingestion phase, documents are segmented into smaller, manageable chunks, which are then transformed into embeddings and stored in an index for efficient retrieval. The retrieval phase involves leveraging the index to retrieve the top-k relevant documents based on similarity metrics when a user query is received. Finally, in the synthesis phase, the LLM utilizes the retrieved information along with its internal training data to formulate accurate responses to user queries.
|
||||
|
||||
We will discuss the history of RAG and then delve into the key components of RAG, including ingestion, retrieval, and synthesis, providing detailed insights into each phase's processes and strategies for improvement. We will also go over various challenges associated with RAG, such as data ingestion complexity, efficient embedding, and fine-tuning for generalization and propose solutions to each of them.
|
||||
|
||||
## What is RAG? (Recap)
|
||||
|
||||
Retrieval Augmented Generation (RAG) is an AI framework that enhances the quality of responses generated by LLMs by incorporating up-to-date and contextually relevant information from external sources during the generation process. It addresses the inconsistency and lack of domain-specific knowledge in LLMs, reducing the chances of hallucinations or incorrect responses. RAG involves two phases: retrieval, where relevant information is searched and retrieved, and content generation, where the LLM synthesizes an answer based on the retrieved information and its internal training data. This approach improves accuracy, allows source verification, and reduces the need for continuous model retraining.
|
||||
|
||||
|
||||
|
||||
Image Source: [https://www.deeplearning.ai/short-courses/langchain-for-llm-application-development/](https://www.deeplearning.ai/short-courses/langchain-for-llm-application-development/)
|
||||
|
||||
The diagram above outlines the fundamental RAG pipeline, consisting of three key components:
|
||||
|
||||
1. **Ingestion:**
|
||||
- Documents undergo segmentation into chunks, and embeddings are generated from these chunks, subsequently stored in an index.
|
||||
- Chunks are essential for pinpointing the relevant information in response to a given query, resembling a standard retrieval approach.
|
||||
2. **Retrieval:**
|
||||
- Leveraging the index of embeddings, the system retrieves the top-k documents when a query is received, based on the similarity of embeddings.
|
||||
3. **Synthesis:**
|
||||
- Examining the chunks as contextual information, the LLM utilizes this knowledge to formulate accurate responses.
|
||||
|
||||
💡Unlike previous methods for domain adaptation, it's important to highlight that RAG doesn't necessitate any model training whatsoever. It can be readily applied without the need for training when specific domain data is provided.
|
||||
|
||||
## History
|
||||
|
||||
RAG, or Retrieval-Augmented Generation, made its debut in [this](https://arxiv.org/pdf/2005.11401.pdf) paper by Meta. The idea came about in response to the limitations observed in large pre-trained language models regarding their ability to access and manipulate knowledge effectively.
|
||||
|
||||
|
||||
|
||||
Image Source: [https://arxiv.org/pdf/2005.11401.pdf](https://arxiv.org/pdf/2005.11401.pdf)
|
||||
|
||||
Below is a short summary of how the authors introduce the problem and provide a solution:
|
||||
|
||||
RAG came about because, even though big language models were good at remembering facts and performing specific tasks, they struggled when it came to precisely using and manipulating that knowledge. This became evident in tasks heavy on knowledge, where other specialized models outperformed them. The authors identified challenges in existing models, such as difficulty explaining decisions and keeping up with real-world changes. Before RAG, there were promising results with hybrid models that mixed both parametric and non-parametric memories. Examples like REALM and ORQA combined masked language models with a retriever, showing positive outcomes in this direction.
|
||||
|
||||
Then, along came RAG, a game-changer in the form of a flexible fine-tuning method for retrieval-augmented generation. RAG combined pre-trained parametric memory (like a seq2seq model) with non-parametric memory from a dense vector index of Wikipedia, accessed through a pre-trained neural retriever like Dense Passage Retriever (DPR). RAG models aimed to enhance pre-trained, parametric-memory generation models by combining them with non-parametric memory through fine-tuning. The seq2seq model in RAG used latent documents retrieved by the neural retriever, creating a model trained end-to-end. Training involved fine-tuning on any seq2seq task, learning both the generator and retriever. Latent documents were then handled using a top-K approximation, either per output or per token.
|
||||
|
||||
RAG's main significance was moving away from past approaches that proposed adding non-parametric memory to systems. Instead, RAG explored a new approach where both parametric and non-parametric memory components were pre-trained and filled with lots of knowledge. In experiments, RAG proved its worth by achieving top-notch results in open-domain question answering and surpassing previous models in fact verification and knowledge-intensive generation. Another win for RAG was showing it could adapt, allowing the non-parametric memory to be swapped out and updated to keep the model's knowledge fresh in a changing world.
|
||||
|
||||
## Key Components
|
||||
|
||||
As mentioned earlier, the key elements of RAG involve the processes of ingestion, retrieval, and synthesis. Now, let's delve deeper into each of these components.
|
||||
|
||||
### Ingestion
|
||||
|
||||
In RAG, the ingestion process refers to the handling and preparation of data before it is utilized by the model for generating responses.
|
||||
|
||||
|
||||
|
||||
This process involves 3 key steps:
|
||||
|
||||
1. **Chunking: B**reaking down input text into smaller, more manageable segments or chunks. This can be based on size, sentences, or other natural divisions within the text. We will dig deeper into chunking strategies in the next sections. As an example, consider a comprehensive article on the Renaissance. The chunking process involves breaking down the article into manageable segments based on natural breaks, such as paragraphs or distinct historical periods (e.g., Early Renaissance, High Renaissance). Each of these segments becomes a chunk, enabling focused analysis by the language model.
|
||||
2. **Embedding**: Transforming the text or chunks into a vector format that captures essential qualities in a computationally friendly way. This step is crucial for efficient processing by the language model. Following from the previous example- once the article segments are identified, the embedding process transforms the content of each chunk into a vector format. For instance, the section on the High Renaissance could be embedded into a vector that captures key artistic, cultural, and historical aspects. This vector representation enhances the model's ability to understand and process the nuanced information within the chunk.
|
||||
3. **Indexing:** Organizing the embedded data in a structured format optimized for quick and efficient retrieval. This often involves creating a vector representation for each document and storing these vectors in a searchable format, such as a vector database or search engine. In the example we discussed- The indexed database is created by organizing these vector representations of historical events. Each chunk, now represented as a vector, is indexed for efficient retrieval. When a user queries about a specific aspect of the Renaissance, the indexing enables the quick identification and retrieval of the most relevant chunks, providing contextually rich responses.
|
||||
|
||||
### Retrieval
|
||||
|
||||
The retrieval component involves the following steps:
|
||||
|
||||
|
||||
|
||||
1. **User Query:** A user poses a natural language query to the LLM. For instance, let’s say we’ve completed the ingestion process for renaissance articles as explained in the above method and a user poses a query, "Tell me about the Renaissance period.”
|
||||
2. **Query Conversion:** The query is sent to an embedding model, which converts the natural language query into a numeric format, creating an embedding or vector representation. The embedding model is the same as the model used to embed articles in the ingestion phase.
|
||||
3. **Vector Comparison:** The numeric vectors of the query are compared to vectors in a index of a knowledge base created in the previous phase. This involves measuring similarity or distance metrics between the query vector and vectors stored in the index (often cosine similarity).
|
||||
4. **Top-K Retrieval:** The system then retrieves the top-K documents or passages from the knowledge base that have the highest similarity to the query vector. This step involves selecting a predefined number (K) of the most relevant documents based on the vector similarities. These embeddings may include information about different aspects of the Renaissance.
|
||||
5. **Data Retrieval:** The system retrieves the actual content or data from the selected top-K documents in the knowledge base. This content is typically in human-readable form, representing relevant information related to the user's query.
|
||||
|
||||
Therefore, at the end of the retrieval phase, the LLM has access to relevant context regarding the segments of the knowledge base that hold utmost relevance to the user's query. In this example, the retrieval process ensures that the user receives a well-informed response about the Renaissance, drawing on historical documents stored in the knowledge base to provide contextually rich information.
|
||||
|
||||
### Synthesis
|
||||
|
||||
The Synthesis phase is very similar to regular LLM generation, except that now the LLM has access to additional context from the knowledge base. The LLM presents the final answer to the user, combining its own language generation with information retrieved from the knowledge base. The response may include references to specific documents or historical sources.
|
||||
|
||||
|
||||
|
||||
## RAG Challenges
|
||||
|
||||
Although RAG seems to be a very straightforward way to integrate LLMs with knowledge, there are still the below mentioned open research and application challenges with RAG.
|
||||
|
||||
1. **Data Ingestion Complexity:** Dealing with the complexity of ingesting extensive knowledge bases involves overcoming engineering challenges. For instance, parallelizing requests effectively, managing retry mechanisms, and scaling infrastructure are critical considerations. Imagine ingesting large volumes of diverse data sources, such as scientific articles, and ensuring efficient processing for subsequent retrieval and generation tasks.
|
||||
2. **Efficient Embedding:** Ensuring the efficient embedding of large datasets poses challenges like addressing rate limits, implementing robust retry logic, and managing self-hosted models. Consider the scenario where an AI system needs to embed a vast collection of news articles, requiring strategies to handle changing data, syncing mechanisms, and optimizing embedding costs.
|
||||
3. **Vector Database Considerations:** Storing data in a vector database introduces considerations such as understanding compute resources, monitoring, sharding, and addressing potential bottlenecks. Think about the challenges involved in maintaining a vector database for a diverse range of documents, each with varying levels of complexity and importance.
|
||||
4. **Fine-Tuning and Generalization:** Fine-tuning RAG models for specific tasks while ensuring generalization across diverse knowledge-intensive NLP tasks is challenging. For instance, achieving optimal performance in question-answering tasks might require different fine-tuning approaches compared to tasks involving creative language generation, requiring careful balance.
|
||||
5. **Hybrid Parametric and Non-Parametric Memory: I**ntegrating parametric and non-parametric memory components in models like RAG presents challenges related to knowledge revision, interpretability, and avoiding hallucinations. Consider the difficulty in ensuring that a language model combines its pre-trained knowledge with dynamically retrieved information, avoiding inaccuracies and maintaining coherence.
|
||||
6. **Knowledge Update Mechanisms:** Developing mechanisms to update non-parametric memory as real-world knowledge evolves is crucial. Imagine a scenario where RAG models need to adapt to changing information in domains like medicine, where new research findings and treatments continually emerge, requiring timely updates for accurate responses.
|
||||
|
||||
## Improving RAG components (Ingestion)
|
||||
|
||||
### 1. **Better Chunking Strategies**
|
||||
|
||||
In the context of enhancing the Ingestion process for the RAG components, adopting advanced chunking strategies is necessary for efficient handling of textual data. In a simple RAG pipeline, a fixed strategy is adopted, i.e., a fixed number of words or characters form a single chunk.
|
||||
|
||||
Considering the complexities involved in large datasets, the following strategies are being used recently:
|
||||
|
||||
1. **Content-Based Chunking:** Breaks down text based on meaning and sentence structure using techniques like part-of-speech tagging or syntactic parsing. This preserves the sense and coherence of the text. However, one consideration of this chunking is it requires additional computational resources and algorithmic complexity.
|
||||
2. **Sentence Chunking:** Involves breaking text into complete and grammatically correct sentences using sentence boundary recognition or speech segment. Maintains the unity and completeness of the text but can generate chunks of varying sizes, lacking homogeneity.
|
||||
3. **Recursive Chunking:** Splits text into chunks of different levels, creating a hierarchical and flexible structure. Offers greater granularity and variety in text, but managing and indexing these chunks involves increased complexity.
|
||||
|
||||
### **2. Better Indexing Strategies**
|
||||
|
||||
Improved indexing allows for more efficient search and retrieval of information. When chunks of data are properly indexed, it becomes easier to locate and retrieve specific pieces of information quickly. Some improved strategies include:
|
||||
|
||||
1. **Detailed Indexing:** Chunks through sub-parts (e.g., sentences) and assigns each chunk an identifier based on its position and a feature vector based on content. Provides specific context and accuracy but requires more memory and processing time.
|
||||
2. **Question-Based Indexing:** Chunks through knowledge domains (e.g., topics) and assigns each chunk an identifier based on its category and a vector of characteristics based on relevance. Aligns directly with user requests, enhancing efficiency, but may result in information loss and lower accuracy.
|
||||
3. **Optimized Indexing with Chunk Summaries:** Generates a summary for each chunk using extraction or compression techniques. Assigns an identifier based on the summary and a feature vector based on similarity. Provides greater synthesis and variety but demands complexity in generating and comparing summaries.
|
||||
|
||||
## Improving RAG components (Retrieval)
|
||||
|
||||
### 1. **Hypothetical Questions and HyDE:**
|
||||
|
||||
The introduction of hypothetical questions involves generating a question for each chunk, embedding these questions in vectors, and performing a query search against this index of question vectors. This enhances search quality due to higher semantic similarity between queries and hypothetical questions compared to actual chunks. Conversely, HyDE (Hypothetical Response Extraction) involves generating a hypothetical response given the query, enhancing search quality by leveraging the vector representation of the query and its hypothetical response.
|
||||
|
||||
|
||||
|
||||
Image Source: [https://arxiv.org/pdf/2212.10496.pdf](https://arxiv.org/pdf/2212.10496.pdf)
|
||||
|
||||
### 2. **Context Enrichment:**
|
||||
|
||||
The strategy here aims for smaller chunk retrieval for improved search quality while incorporating surrounding context for reasoning by the Language Model. Two options can explored:
|
||||
|
||||
1. Sentence Window Retrieval: Embedding each sentence in a document separately to achieve high accuracy in the cosine distance search between the query and the context. After retrieving the most relevant single sentence, a context window is extended by including a specified number of sentences before and after the retrieved sentence. This extended context is then sent to the LLM for reasoning upon the provided query. The goal is to enhance the LLM's understanding of the context surrounding the retrieved sentence, enabling more informed responses.
|
||||
|
||||
|
||||
|
||||
Image Source: [https://medium.com/@shivansh.kaushik/advanced-text-retrieval-with-elasticsearch-llamaindex-sentence-window-retrieval-cb5ea720aa44](https://medium.com/@shivansh.kaushik/advanced-text-retrieval-with-elasticsearch-llamaindex-sentence-window-retrieval-cb5ea720aa44)
|
||||
|
||||
1. Auto-Merging Retriever: In this approach, documents are initially split into smaller child chunks, each referring to a larger parent chunk. During retrieval, smaller chunks are fetched first. If, among the top retrieved chunks, more than a specified number are linked to the same parent node (larger chunk), the context fed to the LLM is replaced by this parent node. This process can be likened to automatically merging several retrieved chunks into a larger parent chunk, hence the name "auto-merging retriever." The method aims to capture both granularity and context, contributing to more comprehensive and coherent responses from the LLM.
|
||||
|
||||
|
||||
|
||||
Image Source: [https://twitter.com/clusteredbytes](https://twitter.com/clusteredbytes)
|
||||
|
||||
### 3. **Fusion Retrieval or Hybrid Search:**
|
||||
|
||||
This strategy integrates conventional keyword-based search approaches with contemporary semantic search techniques. By incorporating diverse algorithms like tf-idf (term frequency–inverse document frequency) or BM25 alongside vector-based search, RAG systems can harness the benefits of both semantic relevance and keyword matching, resulting in more thorough and inclusive search outcomes.
|
||||
|
||||
|
||||
|
||||
Image Source: [https://towardsdatascience.com/improving-retrieval-performance-in-rag-pipelines-with-hybrid-search-c75203c2f2f5](https://towardsdatascience.com/improving-retrieval-performance-in-rag-pipelines-with-hybrid-search-c75203c2f2f5)
|
||||
|
||||
### 4. **Reranking & Filtering:**
|
||||
|
||||
Post-retrieval refinement is performed through filtering, reranking, or transformations. LlamaIndex provides various Postprocessors, allowing the filtering of results based on similarity score, keywords, metadata, or reranking with models like LLMs or sentence-transformer cross-encoders. This step precedes the final presentation of retrieved context to the LLM for answer generation.
|
||||
|
||||
|
||||
|
||||
Image Source: [https://www.pinecone.io/learn/series/rag/rerankers/](https://www.pinecone.io/learn/series/rag/rerankers/)
|
||||
|
||||
### 4. **Query Transformations and Routing [[Source](https://blog.langchain.dev/deconstructing-rag/)]**
|
||||
|
||||
Query transformation methods enhance retrieval by breaking down complex queries into sub-questions (Expansion) and improving poorly worded queries through re-writing. While dynamic Query Routing optimizes data retrieval in diverse sources. The below are popular approaches
|
||||
|
||||
### **Query Transformations**
|
||||
|
||||
1. **Query Expansion***:* Query expansion decomposes the input into sub-questions, each of which is a more narrow retrieval challenge. For example, a question about physics can be stepped-back into a question (and LLM-generated answer) about the physical principles behind the user query.
|
||||
2. **Query Re-writing**: Addressing poorly framed or worded user queries, the [Rewrite-Retrieve-Read](https://arxiv.org/pdf/2305.14283.pdf?ref=blog.langchain.dev) approach involves rephrasing questions to enhance retrieval effectiveness. The method is explained in detail in the paper.
|
||||
3. Query Compression: In scenarios where a user question follows a broader chat conversation, the full conversational context may be necessary to answer the question. Query compression is utilized to condense chat history into a final question for retrieval.
|
||||
|
||||
### **Query Routing**
|
||||
|
||||
1. **Dynamic Query Routing**: The question of where the data resides is crucial in RAG, especially in production settings with diverse data-stores. Dynamic query routing, supported by LLMs, efficiently directs incoming queries to the appropriate datastores. This dynamic routing adapts to different sources and optimizes the retrieval process.
|
||||
|
||||
## Improving RAG components (Generation)
|
||||
|
||||
The most straightforward method for LLM generation involves concatenating all the relevant context pieces, surpassing a predefined relevance threshold, and presenting them along with the query to the LLM in a single instance. However, more advanced alternatives exist, necessitating multiple calls to the LLM to iteratively enhance the retrieved context, ultimately leading to the generation of a more refined and improved answer. Some methods are illustrated below.
|
||||
|
||||
### 1. **Response Synthesis Approaches:**
|
||||
|
||||
Involves 3 steps
|
||||
|
||||
1. **Iterative Refinement:** Refine the answer by sending retrieved context to the Language Model chunk by chunk.
|
||||
2. **Summarization:** Summarize the retrieved context to fit into the prompt and generate a concise answer.
|
||||
3. **Multiple Answers and Concatenation:** Generate multiple answers based on different context chunks and then concatenate or summarize them.
|
||||
|
||||
### 2. **Encoder and LLM Fine-Tuning:**
|
||||
|
||||
This approach involves the fine-tuning the LLM models within our RAG pipeline.
|
||||
|
||||
1. **Encoder Fine-Tuning:** Fine-tune the Transformer Encoder for better embeddings quality and context retrieval.
|
||||
2. **Ranker Fine-Tuning:** Use a cross-encoder for reranking retrieved results, especially if there's a lack of trust in the base Encoder.
|
||||
3. **RA-DIT Technique:** Use a technique like RA-DIT to tune both the LLM and the Retriever on triplets of query, context, and answer.
|
||||
|
||||
## Read/Watch These Resources (Optional)
|
||||
|
||||
1. Building Production Ready RAG Applications: [https://www.youtube.com/watch?v=TRjq7t2Ms5I](https://www.youtube.com/watch?v=TRjq7t2Ms5I)
|
||||
2. Amazon article on RAG- [https://docs.aws.amazon.com/sagemaker/latest/dg/jumpstart-foundation-models-customize-rag.html](https://docs.aws.amazon.com/sagemaker/latest/dg/jumpstart-foundation-models-customize-rag.html)
|
||||
3. Huggingface tools for RAG- [https://huggingface.co/docs/transformers/model_doc/rag](https://huggingface.co/docs/transformers/model_doc/rag)
|
||||
4. 12 RAG Pain Points and Proposed Solutions- [https://towardsdatascience.com/12-rag-pain-points-and-proposed-solutions-43709939a28c](https://towardsdatascience.com/12-rag-pain-points-and-proposed-solutions-43709939a28c)
|
||||
|
||||
## Read These Papers (Optional)
|
||||
|
||||
1. [Retrieval-Augmented Generation for Large Language Models: A Survey](https://arxiv.org/pdf/2312.10997.pdf)
|
||||
|
||||
2. [Seven Failure Points When Engineering a Retrieval Augmented Generation System](https://arxiv.org/abs/2401.05856)
|
||||
@@ -0,0 +1,188 @@
|
||||
# [Week 5] Tools for Building LLM Applications
|
||||
|
||||
## ETMI5: Explain to Me in 5
|
||||
|
||||
In this section of our course, we explore the essential technologies and tools that facilitate the creation and enhancement of LLM applications. This includes Custom Model Adaptation for bespoke solutions, RAG-based Applications for contextually rich responses, and an extensive range of tools for input processing, development, application management, and output analysis. Through this comprehensive overview, we aim to equip you with the knowledge to leverage both proprietary and open-source models, alongside advanced development, hosting, and monitoring tools.
|
||||
|
||||
## Types of LLM Applications
|
||||
|
||||
LLM applications are gaining momentum, with an increasing number of startups and companies integrating them into their operations for various purposes. These applications can be categorized into three main types, based on how LLMs are utilized
|
||||
|
||||
1. **Custom Model Adaptation**: This encompasses both the development of custom models from scratch and fine-tuning pre-existing models. While custom model development demands skilled ML scientists and substantial resources, fine-tuning involves updating pre-trained models with additional data. Though fine-tuning is increasingly accessible due to open-source innovations, it still requires a sophisticated team and may result in unintended consequences. Despite its challenges, both approaches are witnessing rapid adoption across industries.
|
||||
2. **RAG based Applications**: The Retrieval Augmented Generation (RAG) method, likely the simplest and most widely adopted approach currently, utilizes a foundational model supplemented with contextual information. This involves retrieving embeddings, which represent words or phrases in a multidimensional vector space, from dedicated vector databases. Through the conversion of unstructured data into embeddings and their storage in these databases, RAG enables efficient retrieval of pertinent context during queries. This facilitates natural language comprehension and timely insights extraction without the need for extensive model customization or training. A notable advantage of RAG is its ability to bypass traditional model limitations like context window constraints. Moreover, it offers cost-effectiveness and scalability, catering to diverse developers and organizations. Furthermore, by harnessing embeddings retrieval, RAG effectively addresses concerns regarding data currency and seamlessly integrates into various applications and systems.
|
||||
|
||||
In the previous weeks’ [content](https://www.notion.so/Week-1-Part-2-Domain-and-Task-Adaptation-Methods-6ad3284a96a241f3bd2318f4f502a1da?pvs=21), we covered the distinctions between these methodologies and discussed the criteria for selecting the most appropriate one based on your specific needs. Please review the materials for further details.
|
||||
|
||||
In the upcoming sections, we'll explore the tool options available for both of these methodologies. There's certainly some overlap between them, which we'll address.
|
||||
|
||||
## Types of Tools
|
||||
|
||||
We can broadly categorize tools into four major groups:
|
||||
|
||||
1. **Input Processing Tools**: These are tools designed to ingest data and various inputs for the application.
|
||||
2. **LLM Development Tools**: These tools facilitate interaction with the Large Language Model, including calling, fine-tuning, conducting experiments, and orchestration.
|
||||
3. **Output Tools**: These tools are utilized for managing the output from the LLM application, essentially focusing on post-output processes.
|
||||
4. **Application Tools**: These tools oversee the comprehensive management of the aforementioned three components, including application hosting, monitoring, and more.
|
||||
|
||||
|
||||
|
||||
If you're remember from the previous content how RAG operates, an application typically follows these steps:
|
||||
|
||||
1. Receives a query from the user (user's input to the application).
|
||||
2. Utilizes an embedding search to find pertinent data (this involves an embedding LLM, data sources and a vector database for storing data embeddings).
|
||||
3. Forwards the retrieved documents along with the query to the LLM for processing.
|
||||
4. Delivers the LLM's output back to the user.
|
||||
|
||||
Hosting and monitoring LLM responses are integrated into the overall application architecture, as depicted in the image below. For fine-tuning applications, much of this workflow is maintained. However, there's a need for a framework and computing resources dedicated to model fine-tuning. Additionally, the application may or may not utilize external data, in which case the vector database component might not be necessary. In the figure below, each of these components and their category association is depicted. Now that we know how each of the tools are utilized, let’s dig deeper into each of these tool types.
|
||||
|
||||
💡If you’re still unsure why each of these tool categories are required, please review the previous weeks’ content to understand how RAG and Fine-Tuning applications work
|
||||
|
||||
|
||||
|
||||
Summary of tools available to build LLM Apps
|
||||
|
||||
## Input Processing Tools
|
||||
|
||||
### 1. Data Pipelines/Sources
|
||||
|
||||
In LLM applications, the effective management and processing of data are key to boosting performance and functionality. The types of data these applications work with are diverse, encompassing text documents, PDFs, and structured formats like CSV files or SQL tables. To navigate this variety, a range of data pipelines and source tools are chosen for loading and transforming data.
|
||||
|
||||
**A. Data Loading and ETL (Extract, Transform, Load) Tools**
|
||||
|
||||
- **Traditional ETL Tools**: Established ETL solutions are widely used to manage data workflows. **[Databricks](http://databricks.com)** is chosen for its robust data processing capabilities, emphasizing machine learning and analytics, while **[Apache Airflow](https://airflow.apache.org/)** is preferred for its ability to programmatically author, schedule, and monitor workflows.
|
||||
- **Document Loaders and Orchestration Frameworks**: Applications that predominantly deal with unstructured data often utilize document loaders integrated within orchestration frameworks. Notable examples include:
|
||||
- **[LangChain](https://www.langchain.com/)**, powered by Unstructured, aids in processing unstructured data for LLM applications.
|
||||
- **[LlamaIndex](https://www.llamaindex.ai/)**, a component of the Llama Hub ecosystem, offers indexing and retrieval functions for efficient data management.
|
||||
|
||||
Further details on LlamaIndex and LangChain will be provided in the orchestration section.
|
||||
|
||||
**B. Specialized Data-Replication Solutions**
|
||||
|
||||
Although the existing stack for data management in LLM applications is operational, there is potential for enhancement, especially in developing data-replication solutions specifically tailored for LLM apps. Such innovations could make the integration and operationalization of data more streamlined, improving both efficiency and the scope of possible applications.
|
||||
|
||||
**Data Loaders for Structured and Unstructured Data**
|
||||
|
||||
The capability to integrate data from a variety of sources is enabled by data loaders that can handle both structured and unstructured inputs. For instance:
|
||||
|
||||
- **Unstructured Data**: Solutions provided by **Unstructured.io** allow for the creation of complex ETL pipelines. These are vital for applications aimed at generating personalized content or conducting semantic searches with data stored in formats like PDFs, documents, and presentations.
|
||||
- **Structured Data Sources**: Loaders that directly connect to databases and other structured data repositories are used, facilitating seamless data integration and manipulation.
|
||||
|
||||
### 2. Vector Databases
|
||||
|
||||
Referring back to the content on RAG, we explored how the most relevant documents are identified through embedding similarity. This is the role where vector databases come into play.
|
||||
|
||||
The primary role of a vector database is to store, compare, and retrieve embeddings (i.e., vectors) efficiently, often scaling up to billions. Among the various options available, **[Pinecone](https://www.pinecone.io/)** stands out as a prevalent choice due to its cloud-hosted nature, making it readily accessible and equipped with features that cater to the demands of large enterprises, such as scalability, Single Sign-On, and Service Level Agreements on uptime.
|
||||
|
||||
The spectrum of vector databases is broad, encompassing:
|
||||
|
||||
- **Open Source Systems** like [Weaviate](https://weaviate.io/), [Vespa](https://vespa.ai/), and [Qdrant](https://qdrant.tech/): These platforms offer exceptional performance on a single-node basis and can be customized for particular applications, making them favored choices among AI teams with the expertise to develop tailored platforms.
|
||||
- **Local Vector Management Libraries** such as [Chroma](https://www.trychroma.com/) and [Faiss](https://github.com/facebookresearch/faiss): Known for their excellent developer experience, these libraries are straightforward to implement for small-scale applications and development experiments. However, they may not serve as complete substitutes for a full-fledged database at scale.
|
||||
- **OLTP Extensions like [pgvector](https://supabase.com/docs/guides/database/extensions/pgvector)**: This option is suited for those who tend to use Postgres for various database requirements or enterprises that procure most of their data infrastructure from a single cloud provider, offering a viable solution for vector support. The long-term viability of closely integrating vector and scalar workloads remains to be seen.
|
||||
|
||||
With the evolution of technology, many open-source vector database providers are venturing into cloud services. Achieving high performance in the cloud, catering to a wide array of use cases, presents a significant challenge. While the immediate future may not witness drastic changes in the offerings available, the long-term landscape is expected to evolve.
|
||||
|
||||
## LLM Development Tools
|
||||
|
||||
### 1. Models
|
||||
|
||||
Developers have a variety of model options to choose from, each with its own set of advantages depending on the project's requirements. The starting point for many is the OpenAI API, with GPT-4 or GPT-4-32k models being popular choices due to their wide-ranging compatibility and minimal need for fine-tuning.
|
||||
|
||||
As applications move from development to production, the focus often shifts towards balancing cost and performance.
|
||||
|
||||
Beyond proprietary models, there's a growing interest in open-source alternatives, most of which are available on **[Huggingface](https://huggingface.co/).** Open-source models provide a flexible and cost-effective solution, especially useful in high-volume, consumer-facing applications like search or chat functions. While traditionally seen as lagging behind their proprietary counterparts in terms of accuracy and performance, the gap is closing. Initiatives like Meta's LLaMa models have showcased the potential for open-source models to reach high levels of accuracy, spurring the development of various alternatives aimed at matching or even surpassing proprietary model performance.
|
||||
|
||||
The choice between proprietary and open-source models doesn't just hinge on cost. Considerations include the specific needs of the application, such as accuracy, inference speed, customization options, and the potential need for fine-tuning to meet particular requirements. Users may also weigh the benefits of hosting models themselves against using cloud-based solutions, which can simplify deployment but may involve different cost structures and scalability considerations.
|
||||
|
||||
💡Note that many proprietary models cannot be fine-tuned by the application developers.
|
||||
|
||||
### 2. Orchestration
|
||||
|
||||
Orchestration tools in the context of LLM applications are software frameworks designed to streamline and manage complex processes involving multiple components and interactions with LLMs. Here's a breakdown of what these tools do:
|
||||
|
||||
1. **Automate Prompt Engineering**: Orchestration tools automate the creation and management of prompts, which are queries or instructions sent to LLMs. These tools use advanced strategies to construct prompts that effectively communicate the task at hand to the model, improving the relevance and accuracy of the model's responses.
|
||||
2. **Integrate External Data**: They facilitate the incorporation of external data into prompts, enhancing the model's responses with context that it wasn't originally trained on. This could involve pulling information from databases, web services, or other data sources to provide the LLM with the most current or relevant data for generating its responses.
|
||||
3. **Manage API Interactions**: Orchestration tools handle the complexities of interfacing with LLM APIs, including making calls to the model, managing API keys, and handling the data returned by the model. This allows developers to focus on higher-level application logic rather than the intricacies of API communication.
|
||||
4. **Prompt Chaining and Memory Management**: They enable prompt chaining, where the output of one LLM interaction is used as input for another, allowing for more sophisticated dialogues or data processing sequences. Additionally, they can maintain a "memory" of previous interactions, helping the model build on past responses for more coherent and contextually relevant outputs.
|
||||
5. **Simplify Application Development**: By abstracting away the complexity of working directly with LLMs, orchestration tools make it easier for developers to build applications. They provide templates and frameworks for common use cases like chatbots, content generation, and information retrieval, speeding up the development process.
|
||||
6. **Avoid Vendor Lock-in**: These tools often design their systems to be model-agnostic, meaning they can work with different LLMs from various providers. This flexibility allows developers to switch between models as needed without rewriting large portions of their application code.
|
||||
|
||||
Frameworks like **LangChain** and **LlamaIndex** work by simplifying complex processes such as prompt chaining, interfacing with external APIs, integrating contextual data from vector databases, and maintaining consistency across multiple LLM interactions. They offer templates for a wide range of applications, making them particularly popular among hobbyists and startups eager to launch their applications quickly, with LangChain leading in usage.
|
||||
|
||||
|
||||
|
||||
Image Source: [https://stackoverflow.com/questions/76990736/differences-between-langchain-llamaindex](https://stackoverflow.com/questions/76990736/differences-between-langchain-llamaindex)
|
||||
|
||||
Retrieval-augmented generation techniques, which personalize model outputs by embedding specific data within prompts, demonstrate how personalization can be achieved without altering the model's weights through fine-tuning. Tools like LangChain and LlamaIndex offer structures for weaving data into the model's context, facilitating this process.
|
||||
|
||||
The availability of language model APIs democratizes access to powerful models, extending their use beyond specialized machine learning teams to the broader developer community. This expansion is likely to spur the development of more developer-oriented tools. LangChain, for instance, assists developers in overcoming common challenges by abstracting complexities such as model integration, data connection, and avoiding vendor lock-in. Its utility ranges from prototyping to full-scale production use, indicating a significant shift towards more accessible and versatile tooling in the LLM application development ecosystem.
|
||||
|
||||
### 3. Compute/Training Frameworks
|
||||
|
||||
Compute and training frameworks play essential roles in the development and deployment of LLM applications, particularly when it comes to fine-tuning models to suit specific needs or developing entirely new models. These frameworks and services provide the necessary infrastructure and tools required for handling the substantial computational demands of working with LLMs.
|
||||
|
||||
**Compute Frameworks**
|
||||
|
||||
Compute frameworks and cloud services offer scalable resources needed to run LLM applications efficiently. Examples include:
|
||||
|
||||
- **Cloud Providers**: Services like **[AWS](https://aws.amazon.com/) (Amazon Web Services)** provide a wide range of computing resources, including GPU and CPU instances, which are critical for both training and inference phases of LLM applications. These platforms offer flexibility and scalability, allowing developers to adjust resources according to their project's requirements.
|
||||
- **LLM Infrastructure Companies**: Companies like **[Fireworks.ai](https://fireworks.ai/)** and **[Anyscale](https://www.anyscale.com/)** specialize in providing infrastructure solutions tailored for LLMs. These services are designed to optimize the performance of LLM applications, offering specialized hardware and software configurations that can significantly reduce training and inference times.
|
||||
|
||||
**Training Frameworks**
|
||||
|
||||
For the development and fine-tuning of LLMs, deep learning frameworks are used. These include:
|
||||
|
||||
- **PyTorch**: A popular choice among researchers and developers for training LLMs due to its flexibility, ease of use, and dynamic computational graph. PyTorch supports a wide range of LLM architectures and provides tools for efficient model training and fine-tuning.
|
||||
- **TensorFlow**: Another widely used framework that offers robust support for LLM training and deployment. TensorFlow is known for its scalability and is suited for both research prototypes and production deployments.
|
||||
|
||||
💡Note that LLM API applications, such as those leveraging RAG, typically do not require direct access to computational resources for training since they use pre-trained models provided via an API. In these cases, the focus is more on integrating the API into the application and possibly using orchestration tools to manage interactions with the model.
|
||||
|
||||
### 4. Experimentation Tools
|
||||
|
||||
Experimentation tools are pivotal for LLM applications, as they facilitate the exploration and optimization of hyperparameters, fine-tuning techniques, and the models themselves. These tools help track and manage the multitude of experiments that are part of developing and refining LLM applications, enabling more systematic and data-driven approaches to model improvement.
|
||||
|
||||
💡 It's important to note that the mentioned tools are primarily beneficial for scenarios involving the fine-tuning or training of models, where experimentation is key. If you're working on applications, these tools might not hold the same level of utility since the LLM operates as a black box. In such cases, the LLM's inner workings and training processes are managed externally, and the focus shifts towards optimizing the use of the model through APIs rather than directly manipulating its training or fine-tuning parameters.
|
||||
|
||||
The below are some experimentation tools
|
||||
|
||||
- **Experiment Tracking**: Tools like **[Weights & Biases](https://wandb.ai/site)** provide platforms for tracking experiments, including changes in hyperparameters, model architectures, and performance metrics over time. This facilitates a more organized approach to experimentation, helping developers to identify the most effective configurations.
|
||||
- **Model Development and Hosting**: Platforms like **Hugging Face** and **[MLFlow](https://mlflow.org/)** offer ecosystems for developing, sharing, and deploying ML models, including custom LLMs. These services simplify access to model repositories (model hubs), computing resources, and deployment capabilities, streamlining the development cycle.
|
||||
- **Performance Evaluation**: Tools like **[Statsig](https://www.statsig.com/)** offer capabilities for evaluating model performance in a live production environment, allowing developers to conduct A/B tests and gather real-world feedback on model behavior.
|
||||
|
||||
## Application Tools
|
||||
|
||||
### 1. Hosting
|
||||
|
||||
Developers leveraging open-source models have a range of hosting services at their disposal. Innovations from companies like [OctoML](https://octo.ai/) have expanded hosting capabilities beyond traditional server setups, enabling deployment on edge devices and directly within browsers. This shift not only enhances privacy and security but also serves to reduce latency and costs. Hosting platforms like [Replicate](https://replicate.com/) are incorporating tools designed to simplify the integration and utilization of these models for software developers, reflecting a belief in the potential of smaller, finely tuned models to achieve top-tier accuracy within specific domains.
|
||||
|
||||
Beyond the LLM components, the static elements of LLM applications—essentially, everything excluding the model itself—also require hosting solutions. Common choices include platforms like [Vercel](https://vercel.com/) and services provided by major cloud providers. Yet, the landscape is evolving with the emergence of startups like [Steamship](https://www.steamship.com/) and [Streamlit](https://streamlit.io/), which offer end-to-end hosting solutions tailored for LLM applications, indicating a broadening of hosting options to support the diverse needs of developers.
|
||||
|
||||
### 2. Monitoring
|
||||
|
||||
Monitoring and observability tools are essential for maintaining and improving applications, especially after deployment in production. These tools enable developers to track key metrics such as the model's performance, cost, latency, and overall behavior. Insights gained from these metrics are invaluable for guiding the iteration of prompts and further experimentation with models, ensuring that the application remains efficient, cost-effective, and aligned with user needs.
|
||||
|
||||
One notable development in this area is the launch of **[LangKit](https://github.com/whylabs/langkit) by WhyLabs**. LangKit is specifically designed to offer developers enhanced visibility into the quality of model outputs.
|
||||
|
||||
Some other examples:
|
||||
|
||||
**[Gantry](https://www.gantry.io/)** offers a holistic approach to understanding model performance by tracking inputs and outputs alongside relevant metadata and user feedback. It assists in uncovering how models function in real-world scenarios, identifying errors, and spotting underperforming cohorts or use cases.
|
||||
|
||||
**[Helicone](https://www.helicone.ai/)** is designed to offer actionable insights into application performance with minimal setup. It enables real-time monitoring of model interactions, helping developers understand how their models are performing across different metrics. By logging inputs, outputs, and enriching them with metadata and user feedback, Helicone provides a comprehensive view of model behavior.
|
||||
|
||||
## Output Tools
|
||||
|
||||
### 1. Evaluation
|
||||
|
||||
When developing applications with LLMs, developers often navigate a complex balance among model performance, inference cost, and latency. Strategies to enhance one aspect, such as iterating on prompts, fine-tuning the model, or switching model providers, can impact the others. Given the probabilistic nature of LLMs and the variability in tasks they perform, assessing performance becomes a critical challenge. To aid in this process, a range of evaluation tools have been developed. These tools assist in refining prompts, tracking experimentation, and monitoring model performance, both offline and online. Here's an overview of the types of tools available:
|
||||
|
||||
For those looking to optimize the interaction with LLMs, No Code / Low Code prompt engineering tools are invaluable. They allow developers and prompt engineers to experiment with different prompts and compare outputs across various models without deep coding requirements. Some examples of such tools include [Humanloop](https://humanloop.com/), [PromptLayer](https://promptlayer.com/) etc.
|
||||
|
||||
Once deployed, it's important to continually monitor an LLM application's performance in the real world. Performance monitoring tools offer insights into how well the model is performing against key metrics, identify potential degradation over time, and highlight areas for improvement. These tools can alert developers to issues that may affect user experience or operational costs, enabling timely adjustments to maintain or enhance the application's effectiveness. Some performance monitoring tools include [Honeyhive](https://www.honeyhive.ai/) and [Scale AI](https://scale.com/).
|
||||
|
||||
The infographic below provides a summary of the tools available for each component of the LLM application process.
|
||||
|
||||
## Read/Watch These Resources (Optional)
|
||||
|
||||
1. [https://www.secopsolution.com/blog/top-10-llm-tools-in-2024](https://www.secopsolution.com/blog/top-10-llm-tools-in-2024)
|
||||
2. [https://www.sequoiacap.com/article/llm-stack-perspective/](https://www.sequoiacap.com/article/llm-stack-perspective/)
|
||||
3. [https://www.codesmith.io/blog/introducing-the-emerging-llm-tech-stack](https://www.codesmith.io/blog/introducing-the-emerging-llm-tech-stack)
|
||||
4. [https://stackshare.io/index/llm-tools](https://stackshare.io/index/llm-tools)
|
||||
@@ -0,0 +1,210 @@
|
||||
# [Week 6] LLM Evaluation Techniques
|
||||
|
||||
## ETMI5: Explain to Me in 5
|
||||
|
||||
In this section of the content, we dive deep into the evaluation techniques applied to LLMs, focusing on two dimensions- pipeline and model evaluations. We examine how prompts are assessed for their effectiveness, leveraging tools like Prompt Registry and Playground. Additionally, we explore the importance of evaluating the quality of retrieved documents in RAG pipelines, utilizing metrics such as Context Precision and Relevancy. We then discuss the relevance metrics used to gauge response pertinence, including Perplexity and Human Evaluation, along with specialized RAG-specific metrics like Faithfulness and Answer Relevance. Additionally, we emphasize the significance of alignment metrics in ensuring LLMs adhere to human standards, covering dimensions such as Truthfulness and Safety. Lastly, we highlight the role of task-specific benchmarks like GLUE and SQuAD in assessing LLM performance across diverse real-world applications.
|
||||
|
||||
## Evaluating Large Language Models (Dimensions)
|
||||
|
||||
Understanding whether LLMs meet our specific needs is crucial. We must establish clear metrics to gauge the value added by LLM applications. When we refer to "LLM evaluation" in this section, we encompass assessing the entire pipeline, including the LLM itself, all input sources, and the content processed by it. This includes the prompts used for the LLM and, in the case of RAG use-cases, the quality of retrieved documents. To evaluate systems effectively, we'll break down LLM evaluation into dimensions:
|
||||
|
||||
A. **Pipeline Evaluation**: Assessing the effectiveness of individual components within the LLM pipeline, including prompts and retrieved documents.
|
||||
B. **Model Evaluation**: Evaluating the performance of the LLM model itself, focusing on the quality and relevance of its generated output.
|
||||
|
||||
Now we’ll dig deeper into each of these two dimensions
|
||||
|
||||
## A. LLM Pipeline Evaluation
|
||||
|
||||
In this section, we’ll look at 2 types of evaluation:
|
||||
|
||||
1. **Evaluating Prompts**: Given the significant impact prompts have on the output of LLM pipelines, we will delve into various methods for assessing and experimenting with prompts.
|
||||
2. **Evaluating the Retrieval Pipeline**: Essential for LLM pipelines incorporating RAG, this involves retrieving the top-k documents to assess the LLM's performance.
|
||||
|
||||
### A1. Evaluating Prompts
|
||||
|
||||
The effectiveness of prompts can be evaluated by experimenting with various prompts and observing the changes in LLM performance. This process is facilitated by prompt testing frameworks, which generally include:
|
||||
|
||||
- Prompt Registry: A space for users to list prompts they wish to evaluate on the LLM.
|
||||
- Prompt Playground: A feature to experiment with different prompts, observe the responses generated, and log them. This function calls the LLM API to get responses.
|
||||
- Evaluation: A section with a user-defined function for evaluating how various prompts perform.
|
||||
- Analytics and Logging: Features providing additional information such as logging and resource usage, aiding in the selection of the most effective prompts.
|
||||
|
||||
Commonly used tools for prompt testing include Promptfoo, PromptLayer, and others.
|
||||
|
||||
**Automatic Prompt Generation**
|
||||
|
||||
More recently there have also been methods to optimize prompts in an automatic manner, for instance- [Zhou et al., (2022)](https://arxiv.org/abs/2211.01910) introduced Automatic Prompt EngineerAPE, a framework for automatically generating and selecting instructions. It treats prompt generation as a language synthesis problem and uses the LLM itself to generate and explore candidate solutions. First, an LLM generates prompt candidates based on output demonstrations. These candidates guide the search process. Then, the prompts are executed using a target model, and the best instruction is chosen based on evaluation scores.
|
||||
|
||||
|
||||
|
||||
### A2. Evaluating Retrieval Pipeline
|
||||
|
||||
In RAG use-cases, solely assessing the end outcome doesn't capture the complete picture. Essentially, the LLM responds to queries based on the context provided. It's crucial to evaluate intermediate results, including the quality of retrieved documents. If the term RAG is unfamiliar to you, please refer to the Week 4 content explaining how RAG operates. Throughout this discussion, we'll refer to the top-k retrieved documents as "context" for the LLM, which requires evaluation. Below are some typical metrics to evaluate the quality of RAG context.
|
||||
|
||||
The below mentioned metrics are sourced from [RAGas](https://docs.ragas.io/en/stable/concepts/metrics/faithfulness.html) an open-source library for RAG pipeline evaluations
|
||||
|
||||
1. **Context Precision (From RAGas [documentation](https://docs.ragas.io/en/stable/concepts/metrics/context_precision.html)):**
|
||||
|
||||
Context Precision is a metric that evaluates whether all of the ground-truth relevant items present in the contexts are ranked higher or not. Ideally all the relevant chunks must appear at the top ranks. This metric is computed using the question and the contexts, with values ranging between 0 and 1, where higher scores indicate better precision.
|
||||
|
||||
$$
|
||||
\text{Context Precision@k} = {\sum {\text{precision@k}} \over \text{total number of relevant items in the top K results}}
|
||||
$$
|
||||
|
||||
$$
|
||||
\text{Precision@k} = {\text{true positives@k} \over (\text{true positives@k} + \text{false positives@k})}
|
||||
$$
|
||||
|
||||
Where k is the total number of chunks in contexts
|
||||
|
||||
2. **Context Relevancy(From RAGas [documentation](https://docs.ragas.io/en/stable/concepts/metrics/context_precision.html))**
|
||||
|
||||
This metric gauges the relevancy of the retrieved context, calculated based on both the question and contexts. The values fall within the range of (0, 1), with higher values indicating better relevancy. Ideally, the retrieved context should exclusively contain essential information to address the provided query. To compute this, we initially estimate the value of
|
||||
by identifying sentences within the retrieved context that are relevant for answering the given question. The final score is determined by the following formula:
|
||||
|
||||
$$
|
||||
\text{context relevancy} = {|S| \over |\text{Total number of sentences in retrived context}|}
|
||||
$$
|
||||
|
||||
```python
|
||||
Hint
|
||||
|
||||
Question: What is the capital of France?
|
||||
|
||||
High context relevancy: France, in Western Europe, encompasses medieval cities, alpine villages and Mediterranean beaches. Paris, its capital, is famed for its fashion houses, classical art museums including the Louvre and monuments like the Eiffel Tower.
|
||||
|
||||
Low context relevancy: France, in Western Europe, encompasses medieval cities, alpine villages and Mediterranean beaches. Paris, its capital, is famed for its fashion houses, classical art museums including the Louvre and monuments like the Eiffel Tower. The country is also renowned for its wines and sophisticated cuisine. Lascaux’s ancient cave drawings, Lyon’s Roman theater and the vast Palace of Versailles attest to its rich history.
|
||||
```
|
||||
|
||||
3. **Context Recall(From RAGas [documentation](https://docs.ragas.io/en/stable/concepts/metrics/context_precision.html)):** Context recall measures the extent to which the retrieved context aligns with the annotated answer, treated as the ground truth. It is computed based on the ground truth and the retrieved context, and the values range between 0 and 1, with higher values indicating better performance. To estimate context recall from the ground truth answer, each sentence in the ground truth answer is analyzed to determine whether it can be attributed to the retrieved context or not. In an ideal scenario, all sentences in the ground truth answer should be attributable to the retrieved context.
|
||||
|
||||
The formula for calculating context recall is as follows:
|
||||
|
||||
$$
|
||||
\text{context recall} = {|\text{GT sentences that can be attributed to context}| \over |\text{Number of sentences in GT}|}
|
||||
$$
|
||||
|
||||
|
||||
General retrieval metrics can also be used to evaluate the quality of retrieved documents or context, however, note that these metrics provide a lot more weight to the ranks of retrieved documents which might not be super crucial for RAG use-cases:
|
||||
|
||||
1. **Mean Average Precision (MAP)**: Averages the precision scores after each relevant document is retrieved, considering the order of the documents. It is particularly useful when the order of retrieval is important.
|
||||
2. **Normalized Discounted Cumulative Gain (nDCG)**: Measures the gain of a document based on its position in the result list. The gain is accumulated from the top of the result list to the bottom, with the gain of each result discounted at lower ranks.
|
||||
3. **Reciprocal Rank**: Focuses on the rank of the first relevant document, with higher scores for cases where the first relevant document is ranked higher.
|
||||
4. **Mean Reciprocal Rank (MRR)**: Averages the reciprocal ranks of results for a sample of queries. It is particularly used when the interest is in the rank of the first correct answer.
|
||||
|
||||
## B. LLM Model Evaluation
|
||||
|
||||
Now that we've discussed evaluating LLM pipeline components, let's delve into the heart of the pipeline: the LLM model itself. Assessing LLM models isn't straightforward due to their broad applicability and versatility. Different use cases may require focusing on certain dimensions more than others. For instance, in applications where accuracy is paramount, evaluating whether the model avoids hallucinations (generating responses that are not factual) can be crucial. Conversely, in other scenarios where maintaining impartiality across different populations is essential, adherence to principles to avoid bias is paramount. LLM evaluation can be broadly categorized into these dimensions:
|
||||
|
||||
- **Relevance Metrics**: Assess the pertinence of the response to the user's query and context.
|
||||
- **Alignment Metrics**: Evaluate how well the model aligns with human preferences in the given use-case, in aspects such as fairness, robustness, and privacy.
|
||||
- **Task-Specific Metrics**: Gauge the performance of LLMs across different downstream tasks, such as multihop reasoning, mathematical reasoning, and more.
|
||||
|
||||
### B1. Relevance Metrics
|
||||
|
||||
Some common response relevance metrics include:
|
||||
|
||||
1. Perplexity: Measures how well the LLM predicts a sample of text. Lower perplexity values indicate better performance. [Formula and mathematical explanation](https://huggingface.co/docs/transformers/en/perplexity)
|
||||
2. Human Evaluation: Involves human evaluators assessing the quality of the model's output based on criteria such as relevance, fluency, coherence, and overall quality.
|
||||
3. BLEU (Bilingual Evaluation Understudy): Compares the LLM generated output with reference answer to measure similarity. Higher BLEU scores signify better performance. [Formula](https://www.youtube.com/watch?v=M05L1DhFqcw)
|
||||
4. Diversity: Measures the variety and uniqueness of generated LLM responses, including metrics like n-gram diversity or semantic similarity. Higher diversity scores indicate more diverse and unique outputs.
|
||||
5. ROUGE (Recall-Oriented Understudy for Gisting Evaluation) is a metric used to evaluate the quality of LLM generated text by comparing it with reference text. It assesses how well the generated text captures the key information present in the reference text. ROUGE calculates precision, recall, and F1-score, providing insights into the similarity between the generated and reference texts. [Formula](https://www.youtube.com/watch?v=TMshhnrEXlg)
|
||||
|
||||
**RAG specific relevance metrics**
|
||||
|
||||
Apart from the above mentioned generic relevance metrics, RAG pipelines use additional metrics to judge if the answer is relevant to the context provided and to the query posed. Some metrics as defined by [RAGas](https://docs.ragas.io/en/stable/concepts/metrics/faithfulness.html) are:
|
||||
|
||||
1. **Faithfulness(From RAGas [documentation](https://docs.ragas.io/en/stable/concepts/metrics/context_precision.html))**
|
||||
|
||||
This measures the factual consistency of the generated answer against the given context. It is calculated from answer and retrieved context. The answer is scaled to (0,1) range. Higher the better.
|
||||
|
||||
The generated answer is regarded as faithful if all the claims that are made in the answer can be inferred from the given context. To calculate this a set of claims from the generated answer is first identified. Then each one of these claims are cross checked with given context to determine if it can be inferred from given context or not. The faithfulness score is given by:
|
||||
|
||||
$$
|
||||
{|\text{Number of claims in the generated answer that can be inferred from given context}| \over |\text{Total number of claims in the generated answer}|}
|
||||
$$
|
||||
|
||||
```markdown
|
||||
Hint
|
||||
|
||||
Question: Where and when was Einstein born?
|
||||
|
||||
Context: Albert Einstein (born 14 March 1879) was a German-born theoretical physicist, widely held to be one of the greatest and most influential scientists of all time
|
||||
|
||||
High faithfulness answer: Einstein was born in Germany on 14th March 1879.
|
||||
|
||||
Low faithfulness answer: Einstein was born in Germany on 20th March 1879.
|
||||
```
|
||||
|
||||
2. **Answer Relevance(From RAGas [documentation](https://docs.ragas.io/en/stable/concepts/metrics/context_precision.html))**
|
||||
|
||||
The evaluation metric, Answer Relevancy, focuses on assessing how pertinent the generated answer is to the given prompt. A lower score is assigned to answers that are incomplete or contain redundant information. This metric is computed using the question and the answer with values ranging between 0 and 1, where higher scores indicate better relevancy.
|
||||
|
||||
An answer is deemed relevant when it directly and appropriately addresses the original question. Importantly, our assessment of answer relevance does not consider factuality but instead penalizes cases where the answer lacks completeness or contains redundant details. To calculate this score, the LLM is prompted to generate an appropriate question for the generated answer multiple times, and the mean cosine similarity between these generated questions and the original question is measured. The underlying idea is that if the generated answer accurately addresses the initial question, the LLM should be able to generate questions from the answer that align with the original question.
|
||||
|
||||
3. **Answer semantic similarity(From RAGas [documentation](https://docs.ragas.io/en/stable/concepts/metrics/context_precision.html))**
|
||||
|
||||
The concept of Answer Semantic Similarity pertains to the assessment of the semantic resemblance between the generated answer and the ground truth. This evaluation is based on the ground truth answer and the generated LLM answer , with values falling within the range of 0 to 1. A higher score signifies a better alignment between the generated answer and the ground truth.
|
||||
|
||||
Measuring the semantic similarity between answers can offer valuable insights into the quality of the generated response. This evaluation utilizes a cross-encoder model to calculate the semantic similarity score.
|
||||
|
||||
### B2. Alignment Metrics
|
||||
|
||||
Metrics of this type are crucial, especially when LLMs are utilized in applications that interact directly with people, to ensure they conform to acceptable human standards. The challenge with these metrics is their difficulty to quantify mathematically. Instead, the assessment of LLM alignment involves conducting specific tests on benchmarks designed to evaluate alignment, using the results as an indirect measure. For instance, to evaluate a model's fairness, datasets are employed where the model must recognize stereotypes, and its performance in this regard serves as an indirect indicator of the LLM's fairness alignment. Thus, there's no universally correct method for this evaluation. In our course, we will adopt the approaches outlined in the influential study “[TRUSTLLM: Trustworthiness in Large Language Models](https://arxiv.org/pdf/2401.05561.pdf)” to explore alignment dimensions and the proxy tasks that help gauge LLM alignment.
|
||||
|
||||
There is no single definition for Alignment, but here are some dimensions to quantify alignment, we use definitions from the paper mentioned above:
|
||||
|
||||
1. **Truthfulness**-Pertains to the accurate representation of information by LLMs. It encompasses evaluations of their tendency to generate misinformation, hallucinate, exhibit sycophantic behavior, and correct adversarial facts.
|
||||
2. **Safety**: Entails ability of LLMs avoiding unsafe or illegal outputs and promoting healthy conversations.
|
||||
3. **Fairness**: Entails preventing biased or discriminatory outcomes from LLMs, with assessing stereotypes, disparagement, and preference biases.
|
||||
4. **Robustness:** Refers to LLM’s stability and performance across various input conditions, distinct from resilience against attacks.
|
||||
5. **Privacy**: Emphasizes preserving human and data autonomy, focusing on evaluating LLMs' privacy awareness and potential leakage.
|
||||
6. **Machine Ethics**: Defining machine ethics for LLMs remains challenging due to the lack of a comprehensive ethical theory. Instead, we can divide it into three segments: implicit ethics, explicit ethics, and emotional awareness. E
|
||||
7. **Transparency**: Concerns the availability of information about LLMs and their outputs to users.
|
||||
8. **Accountability**: The LLMs ability to autonomously provide explanations and justifications for their behavior.
|
||||
9. **Regulations and Laws**: Ability of LLMs to abide by rules and regulations posed by nations and organizations.
|
||||
|
||||
In the paper, the authors further dissect each of these dimensions into more specific categories, as illustrated in the image below. For instance, Truthfulness is segmented into aspects such as misinformation, hallucination, sycophancy, and adversarial factuality. Moreover, each of these sub-dimensions is accompanied by corresponding datasets and metrics designed to quantify them.
|
||||
|
||||
💡This serves as a basic illustration of utilizing proxy tasks, datasets, and metrics to evaluate an LLM's performance within a specific dimension. The choice of which dimensions are relevant will vary based on your specific task, requiring you to select the most applicable ones for your needs.
|
||||
|
||||
|
||||
|
||||
### B3. Task-Specific Metrics
|
||||
|
||||
Often, it's necessary to create tailored benchmarks, including datasets and metrics, to evaluate an LLM's performance in a specific task. For example, if developing a chatbot requiring strong reasoning abilities, utilizing common-sense reasoning benchmarks can be beneficial. Similarly, for multilingual understanding, machine translation benchmarks are valuable.
|
||||
|
||||
Below, we outline some popular examples.
|
||||
|
||||
1. **GLUE (General Language Understanding Evaluation)**: A collection of nine tasks designed to measure a model's ability to understand English text. Tasks include sentiment analysis, question answering, and textual entailment.
|
||||
2. **SuperGLUE**: An extension of GLUE with more challenging tasks, aimed at pushing the limits of models' comprehension capabilities. It includes tasks like word sense disambiguation, more complex question answering, and reasoning.
|
||||
3. **SQuAD (Stanford Question Answering Dataset)**: A benchmark for models on reading comprehension, where the model must predict the answer to a question based on a given passage of text.
|
||||
4. **Commonsense Reasoning Benchmarks**:
|
||||
- **Winograd Schema Challenge**: Tests models on commonsense reasoning and understanding by asking them to resolve pronoun references in sentences.
|
||||
- **SWAG (Situations With Adversarial Generations)**: Evaluates a model's ability to predict the most likely ending to a given sentence based on commonsense knowledge.
|
||||
5. **Natural Language Inference (NLI) Benchmarks**:
|
||||
- **MultiNLI**: Tests a model's ability to predict whether a given hypothesis is true (entailment), false (contradiction), or undetermined (neutral) based on a given premise.
|
||||
- **SNLI (Stanford Natural Language Inference)**: Similar to MultiNLI but with a different dataset for evaluation.
|
||||
6. **Machine Translation Benchmarks**:
|
||||
- **WMT (Workshop on Machine Translation)**: Annual competition with datasets for evaluating translation quality across various language pairs.
|
||||
7. **Task-Oriented Dialogue Benchmarks**:
|
||||
- **MultiWOZ**: A dataset for evaluating dialogue systems in task-oriented conversations, like booking a hotel or finding a restaurant.
|
||||
8. **Code Generation and Understanding Benchmarks**:
|
||||
- MBPP Dataset: The benchmark consists of around 1,000 crowd-sourced Python programming problems, designed to be solvable by entry level programmers.
|
||||
9. **Chart Understanding Benchmarks**:
|
||||
1. ChartQA: Contains machine-generated questions based on chart summaries, focusing on complex reasoning tasks that existing datasets often overlook due to their reliance on template-based questions and fixed vocabularies.
|
||||
|
||||
The [Hugging Face OpenLLM Leaderboard](https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard) features an array of datasets and tasks used to assess foundational models and chatbots
|
||||
|
||||
|
||||
|
||||
## Read/Watch These Resources (Optional)
|
||||
|
||||
1. LLM Evaluation by Klu.ai: [https://klu.ai/glossary/llm-evaluation](https://klu.ai/glossary/llm-evaluation)
|
||||
2. Microsoft LLM Evaluation Leaderboard: [https://llm-eval.github.io/](https://llm-eval.github.io/)
|
||||
3. Evaluating and Debugging Generative AI Models Using Weights and Biases course: [https://www.deeplearning.ai/short-courses/evaluating-debugging-generative-ai/](https://www.deeplearning.ai/short-courses/evaluating-debugging-generative-ai/)
|
||||
|
||||
## Read These Papers (Optional)
|
||||
|
||||
1. [https://arxiv.org/abs/2310.19736](https://arxiv.org/abs/2310.19736)
|
||||
2. [https://arxiv.org/abs/2401.05561](https://arxiv.org/abs/2401.05561)
|
||||
@@ -0,0 +1,198 @@
|
||||
# [Week 7] Building Your Own LLM Application
|
||||
|
||||
## ETMI5: Explain to Me in 5
|
||||
|
||||
In the previous parts of the course we covered techniques such as prompting, RAG, and fine-tuning, this section will adopt a practical, hands-on approach to showcase how LLMs can be employed in application development. We'll start with basic examples and progressively incorporate more advanced functionalities like chaining, memory management, and tool integration. Additionally, we'll explore implementations of RAG and fine-tuning. Finally, by integrating these concepts, we'll learn how to construct LLM agents effectively.
|
||||
|
||||
## Introduction
|
||||
|
||||
As LLMs have become increasingly prevalent, there are now multiple ways to utilize them. We'll start with basic examples and gradually introduce more advanced features, allowing you to build upon your understanding step by step.
|
||||
|
||||
This guide is designed to cover the basics, aiming to familiarize you with the foundational elements through simple applications. These examples serve as starting points and are not intended for production environments. For insights into deploying applications at scale, including discussions on LLM tools, evaluation, and more, refer to our content from previous weeks. As we progress through each section, we'll gradually move from basic to more advanced components.
|
||||
|
||||
In every section, we'll not only describe the component but also provide resources where you can find code samples to help you develop your own implementations. There are several frameworks available for developing your application, with some of the most well-known being LangChain, LlamaIndex, Hugging Face, and Amazon Bedrock, among others. Our goal is to supply resources from a broad array of these frameworks, enabling you to select the one that best fits the needs of your specific application.
|
||||
|
||||
As you explore each section, select a few resources to help build the app with the component and proceed further.
|
||||
|
||||
|
||||
|
||||
## 1. Simple LLM App (Prompt + LLM)
|
||||
|
||||
**Prompt:** A prompt, in this context, is essentially a carefully constructed request or instruction that guides the model in generating a response. It's the initial input given to the LLM that outlines the task you want it to perform or the question you need answered. In the second week's content, we delved extensively into prompt engineering, please head back to older content to learn more.
|
||||
|
||||
The foundational aspect of LLM application development is the interaction between a user-defined prompt and the LLM itself. This process involves crafting a prompt that clearly communicates the user's request or question, which is then processed by the LLM to generate a response. For example:
|
||||
|
||||
```python
|
||||
# Define the prompt template with placeholders
|
||||
prompt_template = "Provide expert advice on the following topic: {topic}."
|
||||
# Fill in the template with the actual topic
|
||||
prompt = prompt_template.replace("{topic}", topic)
|
||||
# API call to an LLM
|
||||
llm_response = call_llm_api(topic)
|
||||
|
||||
```
|
||||
|
||||
Observe that the prompt functions as a template rather than a fixed string, improving its reusability and flexibility for modifications at run-time. The complexity of the prompt can vary; it can be crafted with simplicity or detailed intricacy depending on the requirement.
|
||||
|
||||
### Resources/Code
|
||||
|
||||
1. [**Documentation/Code**] LangChain cookbook for simple LLM Application ([link](https://python.langchain.com/docs/expression_language/cookbook/prompt_llm_parser))
|
||||
2. [**Video**] Hugging Face + LangChain in 5 mins by AI Jason ([link](https://www.youtube.com/watch?v=_j7JEDWuqLE))
|
||||
3. [**Documentation/Code**] Using LLMs with LlamaIndex ([link](https://docs.llamaindex.ai/en/stable/understanding/using_llms/using_llms.html))
|
||||
4. [**Blog**] Getting Started with LangChain by Leonie Monigatti ([link](https://towardsdatascience.com/getting-started-with-langchain-a-beginners-guide-to-building-llm-powered-applications-95fc8898732c))
|
||||
5. [**Notebook**] Running an LLM on your own laptop by LearnDataWithMark ([link](https://github.com/mneedham/LearnDataWithMark/blob/main/llm-own-laptop/notebooks/LLMOwnLaptop.ipynb))
|
||||
|
||||
---
|
||||
|
||||
## 2. Chaining Prompts (Prompt Chains + LLM)
|
||||
|
||||
Although utilizing prompt templates and invoking LLMs is effective, sometimes, you might need to ask the LLM several questions, one after the other, using the answers you got before to ask the next question. Imagine this: first, you ask the LLM to figure out what topic your question is about. Then, using that information, you ask it to give you an expert answer on that topic. This step-by-step process, where one answer leads to the next question, is called "chaining." Prompt Chains are essentially this sequence of chains used for executing a series of LLM actions.
|
||||
|
||||
LangChain has emerged as a widely-used library for creating LLM applications, enabling the chaining of multiple questions and answers with the LLM to produce a singular final response. This approach is particularly beneficial for larger projects requiring multiple steps to achieve the desired outcome. The example discussed illustrates a basic method of chaining. LangChain's [documentation](https://js.langchain.com/docs/modules/chains/) offers guidance on more complex chaining techniques.
|
||||
|
||||
```python
|
||||
prompt1 ="what topic is the following question about-{question}?"
|
||||
prompt2 = "Provide expert advice on the following topic: {topic}."
|
||||
```
|
||||
|
||||
### Resources/Code
|
||||
|
||||
1. **[Article] ****Prompt Chaining Article on Prompt Engineering Guide([link](https://www.promptingguide.ai/techniques/prompt_chaining))
|
||||
2. [**Video**] LLM Chains using GPT 3.5 and other LLMs — LangChain #3 James Briggs ([link](https://www.youtube.com/watch?v=S8j9Tk0lZHU))
|
||||
3. [**Video**] LangChain Basics Tutorial #2 Tools and Chains by Sam Witteveen ([link](https://www.youtube.com/watch?v=hI2BY7yl_Ac))
|
||||
4. [**Code**] LangChain tools and Chains Colab notebook by Sam Witteveen ([link](https://colab.research.google.com/drive/1zTTPYk51WvPV8GqFRO18kDe60clKW8VV?usp=sharing))
|
||||
|
||||
---
|
||||
|
||||
## **3. Adding External Knowledge Base: Retrieval-Augmented Generation (RAG)**
|
||||
|
||||
Next, we'll explore a different type of application. If you've followed our previous discussions, you're aware that although LLMs excel at providing information, their knowledge is limited to what was available up until their last training session. To generate meaningful outputs beyond this point, they require access to an external knowledge base. This is the role that Retrieval-Augmented Generation (RAG) plays.
|
||||
|
||||
Retrieval-Augmented Generation, or RAG, is like giving your LLM a personal library to check before answering. Before the LLM comes up with something new, it looks through a bunch of information (like articles, books, or the web) to find stuff related to your question. Then, it combines what it finds with its own knowledge to give you a better answer. This is super handy when you need your app to pull in the latest information or deep dive into specific topics.
|
||||
|
||||
To implement RAG (Retrieval-Augmented Generation) beyond the LLM and prompts, you'll need the following technical elements:
|
||||
|
||||
**A knowledge base, specifically a vector database**
|
||||
|
||||
A comprehensive collection of documents, articles, or data entries that the system can draw upon to find information. This database isn't just a simple collection of texts; it's often transformed into a vector database. Here, each item in the knowledge base is converted into a high-dimensional vector representing the semantic meaning of the text. This transformation is done using models similar to the LLM but focused on encoding texts into vectors.
|
||||
|
||||
The purpose of having a vectorized knowledge base is to enable efficient similarity searches. When the system is trying to find information relevant to a user's query, it converts the query into a vector using the same encoding process. Then, it searches the vector database for vectors (i.e., pieces of information) that are closest to the query vector, often using measures like cosine similarity. This process quickly identifies the most relevant pieces of information within a vast database, something that would be impractical with traditional text search methods.
|
||||
|
||||
**Retrieval Component**
|
||||
|
||||
The retrieval component is the engine that performs the actual search of the knowledge base to find information relevant to the user's query. It's responsible for several key tasks:
|
||||
|
||||
1. **Query Encoding:** It converts the user's query into a vector using the same model or method used to vectorize the knowledge base. This ensures that the query and the database entries are in the same vector space, making similarity comparison possible.
|
||||
2. **Similarity Search:** Once the query is vectorized, the retrieval component searches the vector database for the closest vectors. This search can be based on various algorithms designed to efficiently handle high-dimensional data, ensuring that the process is both fast and accurate.
|
||||
3. **Information Retrieval:** After identifying the closest vectors, the retrieval component fetches the corresponding entries from the knowledge base. These entries are the pieces of information deemed most relevant to the user's query.
|
||||
4. **Aggregation (Optional):** In some implementations, the retrieval component may also aggregate or summarize the information from multiple sources to provide a consolidated response. This step is more common in advanced RAG systems that aim to synthesize information rather than citing sources directly.
|
||||
|
||||
In the RAG framework, the retrieval component's output (i.e., the retrieved information) is then fed into the LLM along with the original query. This enables the LLM to generate responses that are not only contextually relevant but also enriched with the specificity and accuracy of the retrieved information. The result is a hybrid model that leverages the best of both worlds: the generative flexibility of LLMs and the factual precision of dedicated knowledge bases.
|
||||
|
||||
By combining a vectorized knowledge base with an efficient retrieval mechanism, RAG systems can provide answers that are both highly relevant and deeply informed by a wide array of sources. This approach is particularly useful in applications requiring up-to-date information, domain-specific knowledge, or detailed explanations that go beyond the pre-existing knowledge of an LLM.
|
||||
|
||||
Frameworks like LangChain already have good abstractions in place to build RAG frameworks
|
||||
|
||||
A simple example from LangChain is shown [here](https://python.langchain.com/docs/expression_language/cookbook/retrieval)
|
||||
|
||||
### Resources/Code
|
||||
|
||||
1. [**Article**] All You Need to Know to Build Your First LLM App by Dominik Polzer ([link](https://towardsdatascience.com/all-you-need-to-know-to-build-your-first-llm-app-eb982c78ffac))
|
||||
2. [**Video**] RAG from Scratch series by LangChain ([link](https://www.youtube.com/watch?v=wd7TZ4w1mSw&list=PLfaIDFEXuae2LXbO1_PKyVJiQ23ZztA0x))
|
||||
3. [**Video**] A deep dive into Retrieval-Augmented Generation with LlamaIndex ([link](https://www.youtube.com/watch?v=Y0FL7BcSigI&t=3s))
|
||||
4. [**Notebook**] RAG using LangChain with Amazon Bedrock Titan text, and embedding, using OpenSearch vector engine notebook ([link](https://github.com/aws-samples/rag-using-langchain-amazon-bedrock-and-opensearch))
|
||||
5. [**Video**] LangChain - Advanced RAG Techniques for better Retrieval Performance by Coding Crashcourses ([link](https://www.youtube.com/watch?v=KQjZ68mToWo))
|
||||
6. [**Video**] Chatbots with RAG: LangChain Full Walkthrough by James Briggs ([link](https://www.youtube.com/watch?v=LhnCsygAvzY&t=11s))
|
||||
|
||||
---
|
||||
|
||||
## **4. Adding** Memory to LLMs
|
||||
|
||||
We've explored chaining and incorporating knowledge. Now, consider the scenario where we need to remember past interactions in lengthy conversations with the LLM, where previous dialogues play a role.
|
||||
|
||||
This is where the concept of Memory comes into play as a vital component. Memory mechanisms, such as those available on platforms like LangChain, enable the storage of conversation history. For example, LangChain's ConversationBufferMemory feature allows for the preservation of messages, which can then be retrieved and used as context in subsequent interactions. You can discover more about these memory abstractions and their applications on LangChain's [documentation](https://python.langchain.com/docs/modules/memory/types/).
|
||||
|
||||
### Resources/Code
|
||||
|
||||
1. [**Article**] Conversational Memory for LLMs with LangChain by Pinecone([link](https://www.pinecone.io/learn/series/langchain/langchain-conversational-memory/))
|
||||
2. [**Blog**] How to add memory to a chat LLM model by Nikolay Penkov ([link](https://medium.com/@penkow/how-to-add-memory-to-a-chat-llm-model-34e024b63e0c))
|
||||
3. [**Documentation**] Memory in LlamaIndex documentation ([link](https://docs.llamaindex.ai/en/latest/api_reference/memory.html))
|
||||
4. [**Video**] LangChain: Giving Memory to LLMs by Prompt Engineering ([link](https://www.youtube.com/watch?v=dxO6pzlgJiY))
|
||||
5. [**Video**] Building a LangChain Custom Medical Agent with Memory by ([link](https://www.youtube.com/watch?v=6UFtRwWnHws))
|
||||
|
||||
---
|
||||
|
||||
## **5. Using External Tools with LLMs**
|
||||
|
||||
Consider a scenario within an LLM application, such as a travel planner, where the availability of destinations or attractions depends on seasonal openings. Imagine we have access to an API that provides this specific information. In this case, the application must query the API to determine if a location is open. If the location is closed, the LLM should adjust its recommendations accordingly, suggesting alternative options. This illustrates a crucial instance where integrating external tools can significantly enhance the functionality of LLMs, enabling them to provide more accurate and contextually relevant responses. Such integrations are not limited to travel planning; there are numerous other situations where external data sources, APIs, and tools can enrich LLM applications. Examples include weather forecasts for event planning, stock market data for financial advice, or real-time news for content generation, each adding a layer of dynamism and specificity to the LLM's capabilities.
|
||||
|
||||
In frameworks like LangChain, integrating these external tools is streamlined through its chaining framework, which allows for the seamless incorporation of new elements such as APIs, data sources, and other tools.
|
||||
|
||||
### Resources/Code
|
||||
|
||||
1. [**Documentation/Code**] List of LLM tools by LangChain ([link](https://python.langchain.com/docs/integrations/tools))
|
||||
2. [**Documentation/Code**]Tools in LlamaIndex ([link](https://docs.llamaindex.ai/en/stable/module_guides/deploying/agents/tools/root.html))
|
||||
3. [**Video**] Building Custom Tools and Agents with LangChain by Sam Witteveen ([link](https://www.youtube.com/watch?v=biS8G8x8DdA))
|
||||
|
||||
---
|
||||
|
||||
## **6. LLMs Making Decisions: Agents**
|
||||
|
||||
In the preceding sections, we explored complex LLM components like tools and memory. Now, let's say we want our LLM to effectively utilize these elements to make decisions on our behalf.
|
||||
|
||||
LLM agents do exactly this, they are systems designed to perform complex tasks by combining LLMs with other modules such as planning, memory, and tool usage. These agents leverage the capabilities of LLMs to understand and generate human-like language, enabling them to interact with users and process information effectively.
|
||||
|
||||
For instance, consider a scenario where we want our LLM agent to assist in financial planning. The task is to analyze an individual's spending habits over the past year and provide recommendations for budget optimization.
|
||||
|
||||
To accomplish this task, the agent first utilizes its memory module to access stored data regarding the individual's expenditures, income sources, and financial goals. It then employs a planning mechanism to break down the task into several steps:
|
||||
|
||||
1. **Data Analysis**: The agent uses external tools to process the financial data, categorizing expenses, identifying trends, and calculating key metrics such as total spending, savings rate, and expenditure distribution.
|
||||
2. **Budget Evaluation**: Based on the analyzed data, the LLM agent evaluates the current budget's effectiveness in meeting the individual's financial objectives. It considers factors such as discretionary spending, essential expenses, and potential areas for cost reduction.
|
||||
3. **Recommendation Generation**: Leveraging its understanding of financial principles and optimization strategies, the agent formulates personalized recommendations to improve the individual's financial health. These recommendations may include reallocating funds towards savings, reducing non-essential expenses, or exploring investment opportunities.
|
||||
4. **Communication**: Finally, the LLM agent communicates the recommendations to the user in a clear and understandable manner, using natural language generation capabilities to explain the rationale behind each suggestion and potential benefits.
|
||||
|
||||
Throughout this process, the LLM agent seamlessly integrates its decision-making abilities with external tools, memory storage, and planning mechanisms to deliver actionable insights tailored to the user's financial situation.
|
||||
|
||||
Here's how LLM agents combine various components to make decisions:
|
||||
|
||||
1. **Language Model (LLM)**: The LLM serves as the central controller or "brain" of the agent. It interprets user queries, generates responses, and orchestrates the overall flow of operations required to complete tasks.
|
||||
2. **Key Modules**:
|
||||
- **Planning**: This module helps the agent break down complex tasks into manageable subparts. It formulates a plan of action to achieve the desired goal efficiently.
|
||||
- **Memory**: The memory module allows the agent to store and retrieve information relevant to the task at hand. It helps maintain the state of operations, track progress, and make informed decisions based on past observations.
|
||||
- **Tool Usage**: The agent may utilize external tools or APIs to gather data, perform computations, or generate outputs. Integration with these tools enhances the agent's capabilities to address a wide range of tasks.
|
||||
|
||||
Existing frameworks offer built-in modules and abstractions for constructing agents. Please refer to the resources provided below for implementing your own agent.
|
||||
|
||||
### Resources/Code
|
||||
|
||||
1. [**Documentation/Code**] Agents in LangChain ([link](https://python.langchain.com/docs/modules/agents/))
|
||||
2. [**Documentation/Code**] Agents in LlamaIndex ([link](https://docs.llamaindex.ai/en/stable/module_guides/deploying/agents/root.html))
|
||||
3. [**Video**] LangChain Agents - Joining Tools and Chains with Decisions by Sam Witteveen ([link](https://www.youtube.com/watch?v=ziu87EXZVUE&t=59s))
|
||||
4. [**Article**] Building Your First LLM Agent Application by Nvidia ([link](https://developer.nvidia.com/blog/building-your-first-llm-agent-application))
|
||||
5. [**Video**] OpenAI Functions + LangChain : Building a Multi Tool Agent by Sam Witteveen ([link](https://www.youtube.com/watch?v=4KXK6c6TVXQ))
|
||||
|
||||
---
|
||||
|
||||
## **7. Fine-Tuning**
|
||||
|
||||
In earlier sections, we explored using pre-trained LLMs with additional components. However, there are scenarios where the LLM must be updated with relevant information before usage, particularly when LLMs lack specific knowledge on a subject. In such instances, it's necessary to first fine-tune the LLM before applying the strategies outlined in sections 1-5 to build an application around it.
|
||||
|
||||
Various platforms offer fine-tuning capabilities, but it's important to note that fine-tuning demands more resources than simply eliciting responses from an LLM, as it involves training the model to understand and generate information on the desired topics.
|
||||
|
||||
### Resources/Code
|
||||
|
||||
1. [**Article**] How to Fine-Tune LLMs in 2024 with Hugging Face by philschmid ([link](https://www.philschmid.de/fine-tune-llms-in-2024-with-trl))
|
||||
2. [**Video**] Fine-tuning Large Language Models (LLMs) | w/ Example Code by Shaw Talebi ([link](https://www.youtube.com/watch?v=eC6Hd1hFvos))
|
||||
3. [**Video**] Fine-tuning LLMs with PEFT and LoRA by Sam Witteveen ([link](https://www.youtube.com/watch?v=Us5ZFp16PaU&t=261s))
|
||||
4. [**Video**] LLM Fine Tuning Crash Course: 1 Hour End-to-End Guide by AI Anytime ([link](https://www.youtube.com/watch?v=mrKuDK9dGlg))
|
||||
5. [**Article**] How to Fine-Tune an LLM series by Weights and Biases ([link](https://wandb.ai/capecape/alpaca_ft/reports/How-to-Fine-Tune-an-LLM-Part-1-Preparing-a-Dataset-for-Instruction-Tuning--Vmlldzo1NTcxNzE2))
|
||||
|
||||
---
|
||||
|
||||
## Read/Watch These Resources (Optional)
|
||||
|
||||
1. List of LLM notebooks by aishwaryanr ([link](https://github.com/aishwaryanr/awesome-generative-ai-guide?tab=readme-ov-file#notebook-code-notebooks))
|
||||
2. LangChain How to and Guides by Sam Witteveen ([link](https://www.youtube.com/watch?v=J_0qvRt4LNk&list=PL8motc6AQftk1Bs42EW45kwYbyJ4jOdiZ))
|
||||
3. LangChain Crash Course For Beginners | LangChain Tutorial by codebasics ([link](https://www.youtube.com/watch?v=nAmC7SoVLd8))
|
||||
4. Build with LangChain Series ([link](https://www.youtube.com/watch?v=mmBo8nlu2j0&list=PLfaIDFEXuae06tclDATrMYY0idsTdLg9v))
|
||||
5. LLM hands on course by Maxime Labonne ([link](https://github.com/mlabonne/llm-course))
|
||||
@@ -0,0 +1,233 @@
|
||||
# [Week 8] Advanced Features and Deployment
|
||||
|
||||
## ETMI5: Explain to Me in 5
|
||||
|
||||
In this section of our content, we will delve into the complexities of deploying LLMs and managing them effectively throughout their lifecycle. We will first discuss LLMOps which involves specialized practices, techniques, and tools tailored to the operational management of LLMs in production environments. We will explore the deployment lifecycle of LLMs, examining areas where operational efficiency is important.We will then proceed to discuss in more depth the crucial components for deployment, namely Monitoring and Observability for LLMs, as well as Security and Compliance for LLMs.
|
||||
|
||||
## LLM Application Stages
|
||||
|
||||
When deploying LLMs, it's essential to establish a layer of abstraction to manage tasks surrounding them effectively, ensuring smooth operation and optimal performance. This layer is generally referred to as LLMOps, a more formal definition is given below:
|
||||
|
||||
LLMOps, or Large Language Model Operations, refers to the specialized practices, techniques, and tools used for the operational management of LLMs in production environments. This field focuses on managing and automating the lifecycle of LLMs from development, deployment, to maintenance, ensuring efficient deployment, monitoring, and maintenance of these models.
|
||||
|
||||
In the upcoming sections, we'll initially explore the deployment lifecycle of LLMs, followed by an examination of critical areas where operational efficiency is crucial.
|
||||
|
||||
Here’s an outline that follows the chronological sequence of the LLM lifecycle:
|
||||
|
||||
### **1. Pre-Development and Planning**
|
||||
|
||||
This phase sets the foundation for a successful LLM project by emphasizing early engagement with the broader AI and ML community and incorporating ethical considerations into the model development strategy. It involves understanding the landscape of LLM technology, including trends, opportunities, and challenges, as well as preemptively addressing potential ethical and bias issues. This stage is critical for aligning the project with best practices, legal and ethical standards, and ensuring that the development team is equipped with the latest knowledge and tools. It includes components like:
|
||||
|
||||
- **Literature Survey**: Engaging with the AI and ML community early on to understand current trends, challenges, and best practices.
|
||||
- **Ethical Model Development**: Considering ethical implications, potential biases, and privacy concerns at the planning stage to guide the development process.
|
||||
|
||||
### **2. Data Preparation and Analysis**
|
||||
|
||||
Data is at the heart of LLMs, and this superclass focuses on the collection, cleaning, labeling, and preparation of data, followed by exploratory analysis to understand its characteristics and inform subsequent modeling decisions. This stage is crucial for ensuring that the data is of high quality, representative, and free of biases as much as possible, laying a solid foundation for training effective and reliable models. This phase can be divided into:
|
||||
|
||||
- **Data Management**: The initial step involves collecting, cleaning, labeling, and preparing data, which is foundational for training LLMs.
|
||||
- **Exploratory Data Analysis**: Analyzing the data to understand its characteristics, which informs the model training strategy and prompt design.
|
||||
|
||||
### **3. Model Development and Training**
|
||||
|
||||
At this stage, the focus shifts to the actual construction and optimization of the LLM, involving training and fine-tuning on the prepared data, as well as prompt engineering to guide the model towards generating desired outputs. This phase is where the model's ability to perform specific tasks is developed and refined, making it a critical period for setting up the model's eventual performance and applicability to real-world tasks. This phase can be divided into:
|
||||
|
||||
- **Model Training and Fine-tuning**: Utilizing pre-trained models and adjusting them with specific datasets to improve performance for targeted tasks.
|
||||
- **Prompt Engineering**: Developing inputs that guide the model to generate desired outputs, essential for effective model training and task performance.
|
||||
|
||||
### **4. Optimization for Deployment**
|
||||
|
||||
Before deployment, models undergo optimization processes such as hyperparameter tuning, pruning, and quantization to balance performance with computational efficiency. This superclass is about making the model ready for production by ensuring it operates efficiently, can be deployed on the required platforms, and meets the necessary performance benchmarks, thus preparing the model for real-world application. This phase can be divided into:
|
||||
|
||||
- **Hyperparameter Tuning**: Fine-tuning model parameters to balance between performance and computational efficiency, crucial before deployment.
|
||||
- **Model Pruning and Quantization**: Techniques employed to make models lighter and faster, facilitating easier deployment, especially in resource-constrained environments.
|
||||
|
||||
### **5. Deployment and Integration**
|
||||
|
||||
This phase involves making the trained and optimized model accessible for real-world application, typically through APIs or web services, and integrating it into existing systems or workflows. It includes automating the deployment process to facilitate smooth updates and scalability. This stage is key to translating the model's capabilities into practical, usable tools or services. It can be divided into:
|
||||
|
||||
- **Deployment Process**: Making the model available for use in production through suitable interfaces such as APIs or web services.
|
||||
- **Continuous Integration and Delivery (CI/CD)**: Automating the model development, testing, and deployment process to ensure a smooth transition from development to production.
|
||||
|
||||
### **6. Post-Deployment Monitoring and Maintenance**
|
||||
|
||||
After deployment, ongoing monitoring and maintenance are essential to ensure the model continues to perform well over time, remains secure, and adheres to compliance requirements. This involves tracking performance, identifying and correcting drift or degradation, and updating the model as necessary. This phase ensures the long-term reliability and effectiveness of the LLM in production environments. It can be divided into:
|
||||
|
||||
- **Monitoring and Observability**: Continuously tracking the model’s performance to detect and address issues like model drift.
|
||||
- **Model Review and Governance**: Managing the lifecycle of models including updates, version control, and ensuring they meet performance benchmarks.
|
||||
- **Security and Compliance**: Ensuring ongoing compliance with legal and ethical standards, including data privacy and security protocols.
|
||||
|
||||
### **7. Continuous Improvement and Compliance**
|
||||
|
||||
This overarching class emphasizes the importance of regularly revisiting and refining the model and its deployment strategy to adapt to new data, feedback, and evolving regulatory landscapes. It underscores the need for a proactive, iterative approach to managing LLMs, ensuring they remain state-of-the-art, compliant, and aligned with user needs and ethical standards. It can be divided into
|
||||
|
||||
- **Privacy and Regulatory Compliance**: Regularly reviewing and updating practices to adhere to evolving regulations such as GDPR and CCPA.
|
||||
- **Best Practices Adoption**: Implementing the latest methodologies and tools for data science and software engineering to refine and enhance the model development and deployment processes.
|
||||
|
||||
Now that we understand the necessary steps for deploying and managing LLMs, let's dive further into the aspects that hold greater relevance for deployment i.e., in this section of our course, go over the post-deployment process, building on the groundwork laid in our discussions over the past weeks.
|
||||
|
||||
While phases 1-5 have been outlined previously, and certain elements such as data preparation and model development are universal across machine learning models, our focus now shifts exclusively to nuances involved in deploying LLMs.
|
||||
|
||||
We will explore in greater detail the areas of:
|
||||
|
||||
- **Deployment of LLMs**: Understanding the intricacies of deploying large language models and the mechanisms for facilitating ongoing learning and adaptation.
|
||||
- **Monitoring and Observability for LLMs**: Examining the strategies and technologies for keeping a vigilant eye on LLM performance and ensuring operational transparency.
|
||||
- **Security and Compliance for LLMs**: Addressing the safeguarding of LLMs against threats and ensuring adherence to ethical standards and practices.
|
||||
|
||||
## **Deployment of LLMs**
|
||||
|
||||
Deploying LLMs into production environments entails a good understanding of both the technical landscape and the specific requirements of the application at hand. Here are some key considerations to keep in mind when deploying LLM applications:
|
||||
|
||||
### **1. Choice Between External Providers and Self-hosting**
|
||||
|
||||
- **External Providers**: Leveraging services like OpenAI or Anthropic can simplify deployment by outsourcing computational tasks but may involve higher costs and data privacy concerns.
|
||||
- **Self-hosting**: Opting for open-source models offers greater control over data and costs but requires more effort in setting up and managing infrastructure.
|
||||
|
||||
### **2. System Design and Scalability**
|
||||
|
||||
- A robust LLM application service must ensure seamless user experiences and 24/7 availability, necessitating fault tolerance, zero downtime upgrades, and efficient load balancing.
|
||||
- Scalability must be planned, considering both the current needs and potential growth, to handle varying loads without degrading performance.
|
||||
|
||||
### **3. Monitoring and Observability**
|
||||
|
||||
- **Performance Metrics**: Such as Queries per Second (QPS), Latency, and Tokens Per Second (TPS), are crucial for understanding the system's efficiency and capacity.
|
||||
- **Quality Metrics**: Customized to the application's use case, these metrics help assess the LLM's output quality and relevance.
|
||||
|
||||
We will go over this more deeply in the next section
|
||||
|
||||
### **4. Cost Management**
|
||||
|
||||
- Deploying LLMs, especially at scale, can be costly. Strategies for cost management include careful resource allocation, utilizing cost-efficient computational resources (e.g., spot instances), and optimizing model inference costs through techniques like request batching.
|
||||
|
||||
### **5. Data Privacy and Security**
|
||||
|
||||
- Ensuring data privacy and compliance with regulations (e.g., GDPR) is paramount, especially when using LLMs for processing sensitive information.
|
||||
- Security measures should be in place to protect both the data being processed and the application itself from unauthorized access and attacks.
|
||||
|
||||
### **6. Rapid Iteration and Flexibility**
|
||||
|
||||
- The ability to quickly iterate and adapt the LLM application is crucial due to the fast-paced development in the field. Infrastructure should support rapid deployment, testing, and rollback procedures.
|
||||
- Flexibility in the deployment strategy allows for adjustments based on performance feedback, emerging best practices, and evolving business requirements.
|
||||
|
||||
### **7. Infrastructure as Code (IaC)**
|
||||
|
||||
- Employing IaC for defining and managing infrastructure can greatly enhance the reproducibility, consistency, and speed of deployment processes, facilitating easier scaling and management of LLM applications.
|
||||
|
||||
### **8. Model Composition and Task Composability**
|
||||
|
||||
- Many applications require composing multiple models or tasks, necessitating a system design that supports such compositions efficiently.
|
||||
- Tools and frameworks that facilitate the integration and orchestration of different LLM components are essential for building complex applications.
|
||||
|
||||
### **9. Hardware and Resource Optimization**
|
||||
|
||||
- Choosing the right hardware (GPUs, TPUs) based on the application's latency and throughput requirements is critical for performance optimization.
|
||||
- Effective resource management strategies, such as auto-scaling and load balancing, ensure that computational resources are used efficiently, balancing cost and performance.
|
||||
|
||||
### **10. Legal and Ethical Considerations**
|
||||
|
||||
- Beyond technical and operational considerations, deploying LLMs also involves ethical considerations around the model's impact, potential biases, and the fairness of its outputs.
|
||||
- Legal obligations regarding the use of AI and data must be carefully reviewed and adhered to, ensuring that the deployment of LLMs aligns with societal norms and regulations.
|
||||
|
||||
## **Monitoring and Observability for LLMs**
|
||||
|
||||
Monitoring and observability refer to the processes and tools used to track, analyze, and understand the behavior and performance of these models during deployment and operation.
|
||||
|
||||
Monitoring is crucial for LLMs to ensure optimal performance, detect faults, plan capacity, maintain security and compliance, govern models, and drive continuous improvement.
|
||||
|
||||
Here are some key metrics that should be monitored for LLMs, we’ve already discussed tools for monitoring in the previous parts of our course
|
||||
|
||||
### Basic Monitoring Strategies
|
||||
|
||||
**1. User-Facing Performance Metrics**
|
||||
|
||||
- **Latency**: The time it takes for the LLM to respond to a query, critical for user satisfaction.
|
||||
- **Availability**: The percentage of time the LLM service is operational and accessible to users, reflecting its reliability.
|
||||
- **Error Rates**: The frequency of unsuccessful requests or responses, indicating potential issues in the LLM or its integration points.
|
||||
|
||||
**2. Model Outputs**
|
||||
|
||||
- **Accuracy**: Measuring how often the LLM provides correct or useful responses, fundamental to its value.
|
||||
- **Confidence Scores**: The LLM's own assessment of its response accuracy, useful for filtering or prioritizing outputs.
|
||||
- **Aggregate Metrics**: Compilation of performance indicators such as precision, recall, and F1 score to evaluate overall model efficacy.
|
||||
|
||||
**3. Data Inputs**
|
||||
|
||||
- **Logging Queries**: Recording user inputs to the LLM for later analysis, troubleshooting, and understanding user interaction patterns.
|
||||
- **Traceability**: Ensuring a clear path from input to output, aiding in debugging and improving model responses.
|
||||
|
||||
**4. Resource Utilization**
|
||||
|
||||
- **Compute Usage**: Tracking CPU/GPU consumption to optimize computational resource allocation and cost.
|
||||
- **Memory Usage**: Monitoring the amount of memory utilized by the LLM, important for managing large models and preventing system overload.
|
||||
|
||||
**5. Training Data Drift**
|
||||
|
||||
- **Statistical Analysis**: Employing statistical tests to compare current input data distributions with those of the training dataset, identifying significant variances.
|
||||
- **Detection Mechanisms**: Implementing automated systems to alert on detected drifts, ensuring the LLM remains accurate over time.
|
||||
|
||||
**6. Custom Metrics**
|
||||
|
||||
- **Application-Specific KPIs**: Developing unique metrics that directly relate to the application's goals, such as user engagement or content generation quality.
|
||||
- **Innovation Tracking**: Continuously evolving metrics to capture new insights and improve LLM performance and user experience.
|
||||
|
||||
### Advanced Monitoring Strategies
|
||||
|
||||
**1. Real-Time Monitoring**
|
||||
|
||||
- **Immediate Insights**: Offering a live view into the LLM's operation, enabling quick detection and response to issues.
|
||||
- **System Performance**: Understanding the dynamic behavior of the LLM in various conditions, adjusting resources in real-time.
|
||||
|
||||
**2. Data Drift Detection**
|
||||
|
||||
- **Maintaining Model Accuracy**: Regularly comparing incoming data against the model's training data to ensure consistency and relevance.
|
||||
- **Adaptive Strategies**: Implementing mechanisms to adjust the model or its inputs in response to detected drifts, preserving performance.
|
||||
|
||||
**3. Scalability and Performance**
|
||||
|
||||
- **Demand Management**: Architecting the LLM system to expand resources in response to user demand, ensuring responsiveness.
|
||||
- **Efficiency Optimization**: Fine-tuning the deployment architecture for optimal performance, balancing speed with cost.
|
||||
|
||||
**4. Interpretability and Debugging**
|
||||
|
||||
- **Model Understanding**: Applying techniques like feature importance, attention mechanisms, and example-based explanations to decipher model decisions.
|
||||
- **Debugging Tools**: Utilizing logs, metrics, and model internals to diagnose and resolve issues, enhancing model reliability.
|
||||
|
||||
**5. Bias Detection and Fairness**
|
||||
|
||||
- **Proactive Bias Monitoring**: Regularly assessing model outputs for unintentional biases, ensuring equitable responses across diverse user groups.
|
||||
- **Fairness Metrics**: Developing and tracking measures of fairness, correcting biases through model adjustments or retraining.
|
||||
|
||||
**6. Compliance Practices**
|
||||
|
||||
- **Regulatory Adherence**: Ensuring the LLM meets legal and ethical standards, incorporating data protection, privacy, and transparency measures.
|
||||
- **Audit and Reporting**: Maintaining records of LLM operations, decisions, and adjustments to comply with regulatory requirements and facilitate audits.
|
||||
|
||||
## **Security and Compliance for LLMs**
|
||||
|
||||
### Security
|
||||
|
||||
Maintaining security in LLM deployments is crucial due to the advanced capabilities of these models in text generation, problem-solving, and interpreting complex instructions. As LLMs increasingly integrate with external tools, APIs, and applications, they open new avenues for potential misuse by malicious actors, raising concerns about social engineering, data exfiltration, and the safe handling of sensitive information. To safeguard against these risks, businesses must develop comprehensive strategies to regulate LLM outputs and mitigate security vulnerabilities.
|
||||
|
||||
Security plays a crucial role in preventing their misuse for generating misleading content or facilitating malicious activities, such as social engineering attacks. By implementing robust security measures, organizations can protect sensitive data processed by LLMs, ensuring confidentiality and privacy. Furthermore, maintaining stringent security practices helps uphold user trust and ensures compliance with legal and ethical standards, fostering responsible deployment and usage of LLM technologies. In essence, prioritizing LLM security is essential for safeguarding both the integrity of the models and the trust of the users who interact with them.
|
||||
|
||||
**How to Ensure LLM Security?**
|
||||
|
||||
- **Data Security**: Implement Reinforcement Learning from Human Feedback (RLHF) and external censorship mechanisms to align LLM outputs with human values and filter out impermissible content.
|
||||
- **Model Security**: Secure the model against tampering by employing validation processes, checksums, and measures to prevent unauthorized modifications to the model’s architecture and parameters.
|
||||
- **Infrastructure Security**: Protect hosting environments through stringent security protocols, including firewalls, intrusion detection systems, and encryption, to prevent unauthorized access and threats.
|
||||
- **Ethical Considerations**: Integrate ethical guidelines to prevent the generation of harmful, biased, or misleading outputs, ensuring LLMs contribute positively and responsibly to users and society.
|
||||
|
||||
### Compliance
|
||||
|
||||
Compliance in the context of LLMs refers to adhering to legal, regulatory, and ethical standards governing their development, deployment, and usage. It encompasses various aspects such as data privacy regulations, intellectual property rights, fairness and bias mitigation, transparency, and accountability.
|
||||
|
||||
Below are some considerations to bear in mind to guarantee adherence to compliance standards when deploying LLMs.
|
||||
|
||||
- **Familiarize with GDPR and EU AI Act**: Gain a comprehensive understanding of regulations like the GDPR in the EU, which governs data protection and privacy, and stay updated on the progress and requirements of the proposed EU AI Act, particularly concerning AI systems.
|
||||
- **International Data Protection Laws**: For global operations, be aware of and comply with data protection laws in other jurisdictions, ensuring LLM deployments meet all applicable international standards.
|
||||
|
||||
## Read/Watch These Resources (Optional)
|
||||
|
||||
1. LLM Monitoring and Observability — A Summary of Techniques and Approaches for Responsible AI -[https://towardsdatascience.com/llm-monitoring-and-observability-c28121e75c2f](https://towardsdatascience.com/llm-monitoring-and-observability-c28121e75c2f)
|
||||
2. LLM Observability- [https://www.tasq.ai/glossary/llm-observability/](https://www.tasq.ai/glossary/llm-observability/)
|
||||
3. LLMs — Observability and Monitoring**-** [https://medium.com/@bijit211987/llm-observability-and-monitoring-925f93242ccf](https://medium.com/@bijit211987/llm-observability-and-monitoring-925f93242ccf)
|
||||
@@ -0,0 +1,235 @@
|
||||
# [Week 9] Challenges with LLMs
|
||||
|
||||
## ETMI5: Explain to Me in 5
|
||||
|
||||
In this section of the course on LLM Challenges, we've identified two main areas of concern with LLMs: behavioral challenges and deployment challenges. Behavioral challenges include issues like hallucination, where LLMs generate fictitious information, and adversarial attacks, where inputs are crafted to manipulate model behavior. Deployment challenges encompass memory and scalability issues, as well as security and privacy concerns. LLMs demand significant computational resources for deployment and face risks of privacy breaches due to their ability to process vast datasets and generate text. To mitigate these challenges, we discuss various strategies such as robust defenses against adversarial attacks, efficient memory management, and privacy-preserving training algorithms. Additionally we will go over techniques like differential privacy, model stacking, and preprocessing methods that are being employed to safeguard user privacy and ensure the reliable and ethical use of LLMs across different applications.
|
||||
|
||||
## Types of Challenges
|
||||
|
||||
We categorize the challenges into two main areas: managing the behavior of LLMs and the technical difficulties encountered during their deployment. Given the evolving nature of this technology, it's likely that current challenges will be mitigated, and new ones may emerge over time. However, as of February 15, 2024, these are the prominently discussed challenges associated with LLMs:
|
||||
|
||||
## **A. Behavioral Challenges**
|
||||
|
||||
### 1. Hallucination
|
||||
|
||||
LLMs sometimes generate plausible but entirely fictitious information or responses, known as "hallucinations." This challenge is particularly harmful in applications requiring high factual accuracy, such as news generation, educational content, or medical advice as hallucinations can erode trust in LLM outputs, leading to misinformation or potentially harmful advice being followed.
|
||||
|
||||
### 2. Adversarial Attacks
|
||||
|
||||
LLMs can be vulnerable to adversarial attacks, where inputs are specially crafted to trick the model into making errors or revealing sensitive information. These attacks can compromise the integrity and reliability of LLM applications, posing significant security risks.
|
||||
|
||||
### 3. Alignment
|
||||
|
||||
Ensuring LLMs align with human values and intentions is a complex task. Misalignment can result from the model pursuing objectives that don't fully encapsulate the user's goals or ethical standards. Misalignment can lead to undesirable outcomes, such as generating content that is offensive, biased, or ethically questionable.
|
||||
|
||||
### 4. Prompt Brittleness
|
||||
|
||||
LLMs can be overly sensitive to the exact wording of prompts, leading to inconsistent or unpredictable outputs. Small changes in prompt structure can yield vastly different responses. This brittleness complicates the development of reliable applications and requires users to have a deep understanding of how to effectively interact with LLMs.
|
||||
|
||||
## **B. Deployment Challenges**
|
||||
|
||||
### 1. Memory and Scalability Challenges
|
||||
|
||||
Deploying LLMs at scale involves significant memory and computational resource demands. Managing these resources efficiently while maintaining high performance and low latency is a technical hurdle. Scalability challenges can limit the ability of LLMs to be integrated into real-time or resource-constrained applications, affecting their accessibility and utility.
|
||||
|
||||
### 2. Security & Privacy
|
||||
|
||||
Protecting the data used by and generated from LLMs is critical, especially when dealing with personal or sensitive information. LLMs need robust security measures to prevent unauthorized access and ensure privacy. Without adequate security and privacy protections, there is a risk of data breaches, unauthorized data usage, and loss of user trust.
|
||||
|
||||
Let’s dig a little deeper into each of issues and existing solutions for them
|
||||
|
||||
## A1. Hallucinations
|
||||
|
||||
Hallucination refers to the model generating information that seems plausible but is actually false or made up. This happens because LLMs are designed to create text that mimics the patterns they've seen in their training data, regardless of whether those patterns reflect real, accurate information. Hallucination is particularly harmful in RAG based applications where the model can generate content that is not supported by data but it is very hard to decipher.
|
||||
|
||||
Hallucination can arise from several factors:
|
||||
|
||||
- **Biases in Training Data:** If the data used to train the model contains inaccuracies or biases, the model might reproduce these errors or skewed perspectives in its outputs.
|
||||
- **Lack of Real-Time Information:** Since LLMs are trained on data that becomes outdated, they can't access or incorporate the latest information, leading to responses based on no longer accurate data. This is the most common cause for hallucinations.
|
||||
- **Model's Limitations:** LLMs don't actually understand the content they generate; they just follow data patterns. This can result in outputs that are grammatically correct and sound logical but are disconnected from actual facts.
|
||||
- **Overgeneralization:** Sometimes, LLMs might apply broad patterns to specific situations where those patterns don't fit, creating convincing but incorrect information.
|
||||
|
||||
**How to detect and mitigate hallucinations?**
|
||||
|
||||
There's a need for automated methods to identify hallucinations in order to understand the model's performance without constant manual checks. Below, we explore various popular research efforts focused on detecting such hallucinations and some of them also propose methods to mitigate hallucinations.
|
||||
|
||||
These are only two of the popular methods, the list is not comprehensive by any means:
|
||||
|
||||
1. **SELFCHECKGPT: Zero-Resource Black-Box Hallucination Detection
|
||||
for Generative Large Language Models ([link](https://arxiv.org/pdf/2303.08896.pdf))**
|
||||
|
||||
✅Hallucination Detection
|
||||
|
||||
❌Hallucination Mitigation
|
||||
|
||||
|
||||
|
||||
|
||||
Image Source: [https://arxiv.org/pdf/2303.08896.pdf](https://arxiv.org/pdf/2303.08896.pdf)
|
||||
|
||||
SelfCheckGPT uses the following steps to detect hallucinations
|
||||
|
||||
1. **Generate Multiple Responses:** SelfCheckGPT begins by prompting the LLM to generate multiple responses to the same question or statement. This step leverages the model's ability to produce varied outputs based on the same input, exploiting the stochastic nature of its response generation mechanism.
|
||||
2. **Analyze Consistency Among Responses:** The key hypothesis is that factual information will lead to consistent responses across different samples, as the model relies on its training on real-world data. In contrast, hallucinated (fabricated) content will result in inconsistent responses, as the model doesn't have a factual basis to generate them and thus "guesses" differently each time.
|
||||
3. **Apply Metrics for Consistency Measurement:** SelfCheckGPT employs five different metrics to assess the consistency among the generated responses. Some of them are popular semantic similarity metrics like BERTScore, N-Gram Overlap etc.
|
||||
4. **Determine Factual vs. Hallucinated Content:** By evaluating the consistency of information across the sampled responses using the above metrics, SelfCheckGPT can infer whether the content is likely factual or hallucinated. High consistency across metrics suggests factual content, while significant variance indicates hallucination.
|
||||
|
||||
A significant advantage of this method is that it operates without the need for external knowledge bases or databases, making it especially useful for black-box models where the internal data or processing mechanisms are inaccessible.
|
||||
|
||||
1. **Self-Contradictory Hallucinations of LLMs: Evaluation, Detection, and Mitigation** ([link](https://arxiv.org/pdf/2305.15852.pdf))
|
||||
|
||||
✅Hallucination Detection
|
||||
|
||||
✅Hallucination Mitigation
|
||||
|
||||
|
||||
|
||||
Image Source: [https://arxiv.org/pdf/2305.15852.pdf](https://arxiv.org/pdf/2305.15852.pdf)
|
||||
|
||||
This research presents a three-step pipeline to detect and mitigate hallucinations, specifically self-contradictions, in LLMs.
|
||||
|
||||
💡 Self-contradiction refers to a scenario where a statement or series of statements within the same context logically conflict with each other, making them mutually incompatible. In the context of LLMs, self-contradiction occurs when the model generates two or more sentences that present opposing facts, ideas, or claims, such that if one sentence is true, the other must be false, given the same context.
|
||||
|
||||
Here's a breakdown of the process:
|
||||
|
||||
1. **Triggering Self-Contradictions:** The process begins by generating sentence pairs that are likely to contain self-contradictions. This is done by applying constraints designed to elicit responses from the LLM that may logically conflict with each other within the same context.
|
||||
2. **Detecting Self-Contradictions:** Various existing prompting strategies are explored to detect these self-contradictions. The authors examine different methods that have been previously developed, applying them to identify when an LLM has produced two sentences that cannot both be true.
|
||||
3. **Mitigating Self-Contradictions:** Once self-contradictions are detected, an iterative mitigation procedure is employed. This involves making local text edits to remove the contradictory information while ensuring that the text remains fluent and informative. This step is crucial for improving the trustworthiness of the LLM's output.
|
||||
|
||||
The framework is extensively evaluated across four modern LLMs, revealing a significant prevalence of self-contradictions in their outputs. For instance, 17.7% of all sentences generated by ChatGPT contained self-contradictions, many of which could not be verified using external knowledge bases like Wikipedia.
|
||||
|
||||
## A2. Adversarial Attacks
|
||||
|
||||
Adversarial attacks involve manipulating the LLM’s behavior by providing crafted inputs or prompts, with the goal of causing unintended or malicious outcomes. There are many types of adversarial attacks, we discuss a few here:
|
||||
|
||||
1. Prompt Injection (PI): Injecting prompts to manipulate the behavior of the model, overriding original instructions and controls.
|
||||
2. Jailbreaking: Circumventing filtering or restrictions by simulating scenarios where the model has no constraints or accessing a developer mode that can bypass restrictions.
|
||||
3. Data Poisoning: Injecting malicious data into the training set to manipulate the model's behavior during training or inference.
|
||||
4. Model Inversion: Exploiting the model's output to infer sensitive information about the training data or the model's parameters.
|
||||
5. Backdoor Attacks: Embedding hidden patterns or triggers into the model, which can be exploited to achieve certain outcomes when specific conditions are met.
|
||||
6. Membership Inference: Determining whether a particular sample was used in the training data of the model, potentially revealing sensitive information about individuals.
|
||||
|
||||
Adversarial attacks pose a significant challenge to LLMs by compromising model integrity and security. These attacks enable adversaries to remotely control the model, steal data, and propagate disinformation. Furthermore, LLMs' adaptability and autonomy make them potent tools for user manipulation, increasing the risk of societal harm.
|
||||
|
||||
Effectively addressing these challenges requires robust defenses and proactive measures to safeguard against adversarial manipulation of AI systems.
|
||||
|
||||
Several efforts have been made to develop robust LLMs and evaluate them against adversarial attacks. One approach to mitigating such attacks involves training the LLM to become accustomed to adversarial inputs, instructing it not to respond to them. An example of this is presented in the paper [SmoothLLM](https://arxiv.org/pdf/2310.03684.pdf), which functions by perturbing multiple copies of a given input prompt at the character level and then consolidating the resulting predictions to identify adversarial inputs. Leveraging the fragility of prompts generated adversarially to changes at the character level, SmoothLLM notably decreases the success rate of jailbreaking attacks on various widely-used LLMs to less than one percent. Critically, this defensive strategy avoids unnecessary caution and provides demonstrable assurances regarding the mitigation of attacks.
|
||||
|
||||
Another mechanism to defend LLMs against adversarial attacks involves the use of a perplexity filter as presented in [this](https://arxiv.org/pdf/2309.00614v2.pdf) paper. This filter operates on the principle that unconstrained attacks on LLMs often result in gibberish strings with high perplexity, indicating a lack of fluency, grammar mistakes, or illogical sequences. In this approach, two variations of the perplexity filter are considered. The first is a simple threshold-based filter, where a prompt passes the filter if its log perplexity is below a predefined threshold. The second variation involves checking perplexity in windows, treating the text as a sequence of contiguous chunks and flagging the text as suspicious if any window has high perplexity.
|
||||
|
||||
A good starting point to read about Adversarial techniques is [Greshake et al. 2023](https://arxiv.org/abs/2302.12173). The paper proposes a classification of attacks and potential causes, as depicted in the image below.
|
||||
|
||||
|
||||
|
||||
## A3. Alignment
|
||||
|
||||
Alignment refers to the ability of LLMs to understand instructions and generate outputs that align with human expectations. Foundational LLMs, like GPT-3, are trained on massive textual datasets to predict subsequent tokens, giving them extensive world knowledge. However, they may still struggle with accurately interpreting instructions and producing outputs that match human expectations. This can lead to biased or incorrect content generation, limiting their practical usefulness.
|
||||
|
||||
Alignment is a broad concept that can be explained in various dimensions, one such categorization is done in [this](https://arxiv.org/pdf/2308.05374.pdf) paper. The paper proposes multiple dimensions and sub-classes for ensuring LLM alignment. For instance, harmful content generated by LLMs can be categorized into harms incurred to individual users (e.g., emotional harm, offensiveness, discrimination), society (e.g., instructions for creating violent or dangerous behaviors), or stakeholders (e.g., providing misinformation leading to wrong business decisions).
|
||||
|
||||
|
||||
|
||||
In broad terms, LLM Alignment can be improved through the following process:
|
||||
|
||||
- Determine the most crucial dimensions for alignment depending on the specific use-case.
|
||||
- Identify suitable benchmarks for evaluation purposes.
|
||||
- Employ Supervised Fine-Tuning (SFT) methods to enhance the benchmarks.
|
||||
|
||||
Some popular aligned LLMs and benchmarks are listed in the image below
|
||||
|
||||
|
||||
|
||||
Image Source: [https://arxiv.org/pdf/2307.12966.pdf](https://arxiv.org/pdf/2307.12966.pdf)
|
||||
|
||||
## A4. Prompt Brittleness
|
||||
|
||||
During the prompt engineering segment of our course, we explored various techniques for prompting LLMs. These sophisticated methods are essential because providing instructions similar to humans isn't suitable for LLMs. An overview of commonly used prompting methods is shown in the image below.
|
||||
|
||||
LLMs require precise prompting, and even slight alterations can impact LLMs, altering their responses. This poses a challenge during deployment, as individuals unfamiliar with prompting methods may struggle to obtain accurate answers from LLMs.
|
||||
|
||||
|
||||
|
||||
Image Source: [https://arxiv.org/pdf/2307.10169.pdf](https://arxiv.org/pdf/2307.10169.pdf)
|
||||
|
||||
In general, prompt brittleness in LLMs can be reduced by adopting the following high level strategies:
|
||||
|
||||
1. **Standardized Prompts:** Establishing standardized prompt formats and syntax guidelines can help ensure consistency and reduce the risk of unexpected variations.
|
||||
2. **Robust Prompt Engineering:** Invest in thorough prompt engineering, considering various prompt formulations and their potential impacts on model outputs. This may involve testing different prompt styles and formats to identify the most effective ones.
|
||||
3. **Human-in-the-Loop Validation:** Incorporate human validation or feedback loops to assess the effectiveness of prompts and identify potential brittleness issues before deployment.
|
||||
4. **Diverse Prompt Testing:** Test prompts across diverse datasets and scenarios to evaluate their robustness and generalizability. This can help uncover any brittleness issues that may arise in different contexts.
|
||||
5. **Adaptive Prompting:** Develop adaptive prompting techniques that allow the model to dynamically adjust its behavior based on user input or contextual cues, reducing reliance on fixed prompt structures.
|
||||
6. **Regular Monitoring and Maintenance:** Continuously monitor model performance and prompt effectiveness in real-world applications, updating prompts as needed to address any brittleness issues that may arise over time.
|
||||
|
||||
## B1. Memory and Scalability Challenges
|
||||
|
||||
In this section, we delve into the specific challenges related to memory and scalability when deploying LLMs, rather than focusing on their development.
|
||||
|
||||
Let's explore these challenges and potential solutions in detail:
|
||||
|
||||
1. **Fine-tuning LLMs:** Continuous fine-tuning of LLMs is crucial to ensure they stay updated with the latest knowledge or adapt to specific domains. Fine-tuning involves adjusting pre-trained model parameters on smaller, task-specific datasets to enhance performance. However, fine-tuning entire LLMs requires substantial memory, making it impractical for many users and leading to computational inefficiencies during deployment.
|
||||
|
||||
**Solutions:** One approach is to leverage systems like RAG, where information can be utilized as context, enabling the model to learn from any knowledge base. Another solution is Parameter-efficient Fine-tuning (PEFT), such as adapters, which update only a subset of model parameters, reducing memory requirements while maintaining task performance. Methods like prefix-tuning and prompt-tuning prepend learnable token embeddings to inputs, facilitating efficient adaptation to specific datasets without the need to store and load individual fine-tuned models for each task. All these methods have been discussed in our previous weeks’ content. Please read through for deeper insights.
|
||||
|
||||
2. **Inference Latency:** LLMs often suffer from high inference latencies due to low parallelizability and large memory footprints. This results from processing tokens sequentially during inference and the extensive memory needed for decoding.
|
||||
|
||||
**Solution:** Various techniques address these challenges, including efficient attention mechanisms. These mechanisms aim to accelerate attention computations by reducing memory bandwidth bottlenecks and introducing sparsity patterns to the attention matrix. [Multi-query attention](https://blog.fireworks.ai/multi-query-attention-is-all-you-need-db072e758055) and [FlashAttention](https://arxiv.org/abs/2205.14135) optimize memory bandwidth usage, while [quantization](https://www.tensorops.ai/post/what-are-quantized-llms) and [pruning](https://medium.com/@bnjmn_marie/freeze-and-prune-to-fine-tune-your-llm-with-apt-dc750b7bfbae) techniques reduce memory footprint and computational complexity without sacrificing performance.
|
||||
|
||||
3. **Limited Context Length:** Limited context length refers to the constraint on the amount of contextual information an LLM can effectively process during computations. This limitation stems from practical considerations such as computational resources and memory constraints, posing challenges for tasks requiring understanding longer contexts, such as novel writing or summarization.
|
||||
|
||||
**Solution:** Researchers propose several solutions to address limited context length. Efficient attention mechanisms, like [Luna](https://arxiv.org/abs/2106.01540) and [dilated attention](https://arxiv.org/abs/2209.15001), handle longer sequences efficiently by reducing computational requirements. Length generalization methods aim to enable LLMs trained on short sequences to perform well on longer sequences during inference. This involves exploring different positional embedding schemes, such as Absolute Positional Embeddings and [ALiBi,](https://arxiv.org/abs/2108.12409) to inject positional information effectively.
|
||||
|
||||
|
||||
## B2. Privacy
|
||||
|
||||
Privacy risks stem from their ability to process and generate text based on vast and varied training datasets. Models like GPT-3 have the potential to inadvertently capture and replicate sensitive information present in their training data, leading to potential privacy concerns during text generation. Issues such as unintentional data memorization, data leakage, and the possibility of disclosing confidential or personally identifiable information (PII) are significant challenges.
|
||||
|
||||
Moreover, when LLMs are fine-tuned for specific tasks, additional privacy considerations arise. Striking a balance between harnessing the utility of these powerful language models and safeguarding user privacy is crucial for ensuring their reliable and ethical use across various applications.
|
||||
|
||||
We review key privacy risks and attacks, along with possible mitigation strategies. The classification provided below is adapted from [this](https://arxiv.org/pdf/2402.00888.pdf) paper, which categorizes privacy attacks as:
|
||||
|
||||
|
||||
|
||||
Image Source: [https://arxiv.org/pdf/2402.00888.pdf](https://arxiv.org/pdf/2402.00888.pdf)
|
||||
|
||||
1. **Gradient Leakage Attack:**
|
||||
|
||||
In this attack, adversaries exploit access to gradients or gradient information to compromise the privacy and safety of deep learning models. Gradients, which indicate the direction of the steepest increase in a function, are crucial for optimizing model parameters during training to minimize the loss function.
|
||||
|
||||
To mitigate gradient-based attacks, several strategies can be employed:
|
||||
|
||||
1. **Random Noise Insertion**: Injecting random noise into gradients can disrupt the adversary's ability to infer sensitive information accurately.
|
||||
2. **Differential Privacy**: Applying differential privacy techniques helps to add noise to the gradients, thereby obscuring any sensitive information contained within them.
|
||||
3. **Homomorphic Encryption**: Using homomorphic encryption allows for computations on encrypted data, preventing adversaries from accessing gradients directly.
|
||||
4. **Defense Mechanisms**: Techniques like adding Gaussian or Laplacian noise to gradients, coupled with differential privacy and additional clipping, can effectively defend against gradient leakage attacks. However, these methods may slightly reduce the model's utility.
|
||||
|
||||
**2. Membership Inference Attack**
|
||||
|
||||
A Membership Inference Attack (MIA) aims to determine if a particular data sample was part of a machine learning model's training data, even without direct access to the model's parameters. Attackers exploit the model's tendency to overfit its training data, leading to lower loss values for training samples. These attacks raise serious privacy concerns, especially when models are trained on sensitive data like medical records or financial information.
|
||||
|
||||
Mitigating MIA in language models involves various mechanisms:
|
||||
|
||||
1. **Dropout and Model Stacking**: Dropout randomly deletes neuron connections during training to mitigate overfitting. Model stacking involves training different parts of the model with different subsets of data to reduce overall overfitting tendencies.
|
||||
2. **Differential Privacy (DP)**: DP-based techniques involve data perturbation and output perturbation to prevent privacy leakage. Models equipped with DP and trained using stochastic gradient descent can reduce privacy leakages while maintaining model utility.
|
||||
3. **Regularization**: Regularization techniques prevent overfitting and improve model generalization. Label smoothing is one such method that prevents overfitting, thus contributing to MIA prevention.
|
||||
|
||||
**3. Personally Identifiable Information (PII) attack**
|
||||
|
||||
This attack involves the exposure of data that can uniquely identify individuals, either alone or in combination with other information. This includes direct identifiers like passport details and indirect identifiers such as race and date of birth. Sensitive PII encompasses information like name, phone number, address, social security number (SSN), financial, and medical records, while non-sensitive PII includes data like zip code, race, and gender. Attackers may acquire PII through various means such as phishing, social engineering, or exploiting vulnerabilities in systems.
|
||||
|
||||
To mitigate PII leakage in LLMs, several strategies can be employed:
|
||||
|
||||
1. **Preprocessing Techniques**: Deduplication during the preprocessing phase can significantly reduce the amount of memorized text in LLMs, thus decreasing the stored personal information. Additionally, personal information or content identifying and filtering with restrictive terms of use can limit the presence of sensitive content in training data.
|
||||
2. **Privacy-Preserving Training Algorithms**: Techniques like differentially private stochastic gradient descent [(DP-SGD)](https://assets.amazon.science/01/6e/4f6c2b1046d4b9b8651166bbcd93/differentially-private-decoding-in-large-language-models.pdf#:~:text=While%20the%20intersection%20of%20DP%20and%20LLMs%20is%20fairly%20novel%2C%20the%20prominent%20approach&text=vate%20Stochastic%20Gradient%20Descent%20(DP%2DSGD)%20(Song%20et%20al.%2C%202013;) can be used during training to ensure the privacy of training data. However, DP-SGD may incur a significant computational cost and decrease model utility.
|
||||
3. **PII Scrubbing**: This involves filtering datasets to eliminate PII from text, often leveraging Named Entity Recognition (NER) to tag PII. However, PII scrubbing methods may face challenges in preserving dataset utility and accurately removing all PII.
|
||||
4. **Fine-Tuning Considerations**: During fine-tuning on task-specific data, it's crucial to ensure that the data doesn't contain sensitive information to prevent privacy leaks. While fine-tuning may help the LM "forget" some memorized data from pretraining, it can still introduce privacy risks if the task-specific data contains PII.
|
||||
|
||||
## Read/Watch These Resources (Optional)
|
||||
|
||||
1. The Unspoken Challenges of Large Language Models - [https://deeperinsights.com/ai-blog/the-unspoken-challenges-of-large-language-models](https://deeperinsights.com/ai-blog/the-unspoken-challenges-of-large-language-models)
|
||||
2. 15 Challenges With Large Language Models (LLMs)- [https://www.predinfer.com/blog/15-challenges-with-large-language-models-llms/](https://www.predinfer.com/blog/15-challenges-with-large-language-models-llms/)
|
||||
|
||||
## Read These Papers (Optional)
|
||||
|
||||
1. [https://arxiv.org/abs/2307.10169](https://arxiv.org/abs/2307.10169)
|
||||
2. [https://www.techrxiv.org/doi/full/10.36227/techrxiv.23589741.v1](https://www.techrxiv.org/doi/full/10.36227/techrxiv.23589741.v1)
|
||||
3. [https://arxiv.org/abs/2311.05656](https://arxiv.org/abs/2311.05656)
|
||||
@@ -0,0 +1,95 @@
|
||||
# Agentic AI Crash Course
|
||||
|
||||
|
||||
|
||||
**Everything you need to know about agentic AI in the real world**
|
||||
|
||||
*Created by [Aishwarya Reganti](https://www.linkedin.com/in/areganti/) & [Kiriti Badam](https://www.linkedin.com/in/sai-kiriti-badam/)*
|
||||
|
||||
---
|
||||
|
||||
## ❗❗Please read before engaging with the course since many influencers have shared incorrect details
|
||||
|
||||
|
||||
**Claim: This course was taught at MIT and Oxford**<br/>
|
||||
Truth: The instructors have taught professional AI programs at MIT and Oxford, but this specific course was never offered there.
|
||||
|
||||
**Claim: This course costs USD 2500 and is now free**<br/>
|
||||
Truth: This is a short intro course, the heading clearly says "crash course". We would never price this at 2500, it has always been free.
|
||||
|
||||
**Claim: This course will change your life and guarantee jobs**<br/>
|
||||
Truth: This course gives you a solid way to enter the topic, build confidence, and feel good about the space. It is not a magic ticket and we never promise outcomes like that.
|
||||
|
||||
We put a lot of care into keeping it simple, useful, and a genuinely good starter. We love that it is getting attention, but we want people to engage with it for the right reasons, not for promises we never made.
|
||||
|
||||
---
|
||||
## 📚 Course Parts
|
||||
|
||||
### [Part 1: What Are AI Agents Anyway?](./part1_what_are_ai_agents_anyway.md)
|
||||
Understanding the fundamental differences between generative AI and agentic AI, core capabilities, and real-world applications.
|
||||
|
||||
### [Part 2: The 4 Types of Agentic Systems (and When to Use What)](./part2_the_4_types_of_agentic_systems.md)
|
||||
Explore workflow agents, semi-autonomous agents, rule-based systems, and autonomous agents with decision frameworks.
|
||||
|
||||
### [Part 3: What Are Tools in AI?](./part3_what_are_tools_in_ai.md)
|
||||
Learn about AI model integration, API connections, tool ecosystems, and custom tool development.
|
||||
|
||||
### [Part 4: What Is RAG, and What Does It Mean to Make It Agentic?](./part4_what_is_rag_and_agentic.md)
|
||||
Deep dive into Retrieval-Augmented Generation, traditional vs agentic RAG, and implementation patterns.
|
||||
|
||||
### [Part 5: What Is MCP and Why Should You Care?](./part5_what_is_mcp_and_why_care.md)
|
||||
Understanding Model Context Protocol, AI model integration strategies, and enterprise implementation.
|
||||
|
||||
### [Part 6: Planning in Agents + Reasoning Models](./part6_planning_in_agents_reasoning_models.md)
|
||||
Agent planning strategies, reasoning model integration, and advanced reasoning capabilities.
|
||||
|
||||
### [Part 7: Memory in Agents](./part7_memory_in_agents.md)
|
||||
Short-term and long-term memory systems, architecture patterns, and performance optimization.
|
||||
|
||||
### [Part 8: Multi-Agent Systems](./part8_multi_agent_systems.md)
|
||||
Multi-agent architecture, hierarchical patterns, coordination strategies, and scalability considerations.
|
||||
|
||||
### [Part 9: Real-World Agentic Systems (Under the Hood)](./part9_real_world_agentic_systems.md)
|
||||
Case studies of production systems, architecture patterns, and lessons from enterprise implementations.
|
||||
|
||||
### [Part 10: AI Agent Lessons and What's Ahead](./part10_ai_agent_lessons_whats_ahead.md)
|
||||
Latest developments, future trends, industry roadmap, and emerging technologies in agentic AI.
|
||||
|
||||
---
|
||||
|
||||
## 🎥 Advanced Video Lectures
|
||||
|
||||
### Core System Design & Applications
|
||||
- [Master Generative AI System Design](https://maven.com/p/8c3221/master-generative-ai-system-design)
|
||||
- [Why AI Agents Aren't Enough for Real-World Applications](https://maven.com/p/20f0ed/why-ai-agents-aren-t-enough-for-real-world-applications)
|
||||
- [Designing Agentic AI Applications for Enterprise Use Cases](https://maven.com/p/497d05/designing-agentic-ai-applications-for-enterprise-use-cases)
|
||||
- [Building Agentic AI Applications in 2025](https://maven.com/p/82345a/building-agentic-ai-applications-in-2025)
|
||||
- [Evaluating Agentic AI Applications: Beyond Vibe Checks](https://maven.com/p/6f0e97/evaluating-agentic-ai-applications-beyond-vibe-checks)
|
||||
|
||||
### Product & Enterprise Implementation
|
||||
- [AI Native Products: What Every PM Needs to Know and Do](https://maven.com/p/9a34b0/ai-native-products-what-every-pm-needs-to-know-and-do)
|
||||
- [Designing Agentic AI Systems for Enterprise Use Cases - Part 1](https://maven.com/p/466e22/1-designing-agentic-ai-systems-for-enterprise-use-cases)
|
||||
- [Designing Agentic AI Systems for Enterprise Use Cases - Part 2](https://maven.com/p/a0cdf1/2-designing-agentic-ai-systems-for-enterprise-use-cases)
|
||||
|
||||
### Advanced Topics & Q2 2025 Updates
|
||||
- [Building Agentic AI Applications: 2025 Q2 Updates - Part 1](https://maven.com/p/b8470c/1-building-agentic-ai-applications-2025-q2-updates)
|
||||
- [AI Protocols 101: What You Should Know About MCP, A2A, etc.](https://maven.com/p/e2b5db/2-ai-protocols-101-what-you-should-know-about-mcp-a2a-etc)
|
||||
- [Single vs Multi-Agent AI Systems](https://maven.com/p/0e0e15/3-single-vs-multi-agent-ai-systems)
|
||||
- [Don't Build AI Products Like Traditional Software](https://maven.com/p/88a325/don-t-build-ai-products-like-traditional-software)
|
||||
|
||||
---
|
||||
|
||||
## 🚀 Getting Started
|
||||
|
||||
Navigate to **Part 1** to begin your journey into the world of agentic AI!
|
||||
|
||||
[🎯 Start with Part 1: What Are AI Agents Anyway?](./part1_what_are_ai_agents_anyway.md)
|
||||
|
||||
---
|
||||
|
||||
|
||||
**Happy Learning!** 🎉
|
||||
|
||||
|
||||
|
||||
|
||||
Binary file not shown.
|
After Width: | Height: | Size: 643 KiB |
@@ -0,0 +1,129 @@
|
||||
# Part 10: AI Agent Lessons and What's Ahead
|
||||
|
||||
|
||||
## A Quick Recap
|
||||
|
||||
Here’s what we covered over the last 9 parts:
|
||||
|
||||
- **Part 1 — What agents are:** Not just chatbots that generate text, but systems that can decide and act.
|
||||
- **Part 2 — Types of agents:** From tightly controlled workflow agents to fully autonomous ones, depending on how much decision-making you hand over.
|
||||
- **Part 3–4 — Tools and RAG:** The bread and butter of agent action and knowledge grounding.
|
||||
- **Part 5 — MCP:** A clean way to structure everything an agent needs (tools, memory, prior messages) into one payload.
|
||||
- **Part 6 — Planning and reasoning models:** Why plain LLMs aren’t enough for complex decisions, and how newer models are built for multi-step tasks.
|
||||
- **Part 7 — Memory:** Short-term vs. long-term memory, what to store, how to retrieve, and why it matters for continuity.
|
||||
- **Part 8 — Multi-agent systems:** Orchestration, peer-to-peer collaboration, and the messiness of coordination.
|
||||
- **Part 9 — Real-world systems:** How Perplexity, NotebookLM, and DeepResearch likely use these patterns in different ways.
|
||||
|
||||
We’ve covered the **moving parts** that show up in real-world systems.
|
||||
But all of it falls apart if you’re not thinking about two things: **observability** and **evaluation**.
|
||||
|
||||
---
|
||||
|
||||
## What’s Still Hard
|
||||
|
||||
### Observability
|
||||
Observability means tracking what your agent is doing — at every step. You’ll want:
|
||||
- Logs of tool calls, decisions, retries
|
||||
- Metrics to spot bottlenecks in latency and cost
|
||||
- Visibility into when things go off-rail
|
||||
- Step-wise traceability for debugging
|
||||
|
||||
Tools like **Comet Opik** help with this.
|
||||
Design observability **from day one**, especially for high-autonomy agents.
|
||||
|
||||
---
|
||||
|
||||
### Evaluation
|
||||
Agents are **non-deterministic**.
|
||||
You need **continuous evaluation**, not just manual testing.
|
||||
|
||||
At a minimum, track:
|
||||
- Goal or task completion rates
|
||||
- Tool call success/failure
|
||||
- RAG quality and hallucination metrics
|
||||
- Model overthinking or inefficiency
|
||||
- Latency and token usage at each step
|
||||
|
||||
Evaluation is how you **understand** and **improve** your system.
|
||||
Too many teams do *vibe checks* instead of real evals — and get stuck in **PoC purgatory**.
|
||||
|
||||
Think of evals + observability as your **testing pipeline** — the agentic equivalent of software QA.
|
||||
Metrics will vary by use case, but the discipline is the same.
|
||||
|
||||
---
|
||||
|
||||
## Where Things Are Headed in Agentic AI
|
||||
|
||||
This space is early, but here are clear trends:
|
||||
|
||||
---
|
||||
|
||||
### 1. Protocols > Prompts
|
||||
<img width="800" height="800" alt="image" src="https://github.com/user-attachments/assets/13b78c2e-fc2b-41fd-a0e5-aa88c60187ea" />
|
||||
|
||||
_Image Source: Reuven’s LinkedIn post_
|
||||
|
||||
As systems grow, we’ll move away from handcrafted prompts toward shared **standards**.
|
||||
|
||||
- **MCP** (Model Context Protocol) standardizes how we package structured context — tools, memory, RAG, prior instructions.
|
||||
- **A2A** (Agent-to-Agent), released by Google, focuses on cross-platform agent communication with a shared schema.
|
||||
|
||||
Expect cleaner abstractions over time — though it’ll take a while before anything becomes as standard as HTTP.
|
||||
|
||||
---
|
||||
|
||||
### 2. Hybrid Reasoning Models
|
||||
Reasoning models will evolve toward **selective planning** — knowing when to plan vs. act fast.
|
||||
|
||||
We’re already seeing this with **Claude 3.7** and others.
|
||||
The aim: balance intelligence with efficiency — without overthinking every task.
|
||||
|
||||
---
|
||||
|
||||
### 3. Better Memory Systems
|
||||
Today’s memory is mostly **duct-taped in**.
|
||||
The future: memory that knows **what to recall, when, and why**.
|
||||
Expect:
|
||||
- Task-scoped memory
|
||||
- Session-based memory
|
||||
- Persona-specific memory
|
||||
|
||||
And **easier management**.
|
||||
|
||||
---
|
||||
|
||||
### 4. Tool Ecosystem Maturity
|
||||
Right now, everyone’s building custom tools/wrappers. Over time:
|
||||
- Trusted, plug-and-play APIs
|
||||
- Better abstraction layers
|
||||
- Shared security practices
|
||||
|
||||
Just like microservices matured in traditional software, tools will mature in the **agentic stack**.
|
||||
|
||||
---
|
||||
|
||||
## A Final Word
|
||||
|
||||
If you’ve followed along, you’ve seen the theme:
|
||||
|
||||
We didn’t start with **architecture**.
|
||||
We started with **problems**.
|
||||
|
||||
That’s the real mindset shift:
|
||||
> Don’t chase agents for the hype.
|
||||
> Build them when they make solving a problem easier, faster, or smarter.
|
||||
|
||||
**Start simple. Measure everything. Scale when needed.**
|
||||
Agent-first thinking breaks. Problem-first thinking scales.
|
||||
|
||||
---
|
||||
|
||||
Thanks for reading, sharing, and thinking along during these 10 parts.
|
||||
If you take away one thing from this series — let it be this:
|
||||
|
||||
> **Problem first, always.**
|
||||
|
||||
Check out the readme for more lectures and advanced topics. If this was helpful, feel free to forward it to someone looking to learn in this space. And if you’d like to go deeper, our full 6-week course covers system design, applied agentic concepts, and real evaluation workflows, the kind that support production-grade applications. The course is built for everyone, whether you’re a Product Manager, Architect, Director, C-suite leader, or someone seriously exploring agentic AI.
|
||||
|
||||
Our next cohort starts soon. Our next cohort starts soon. Early bird pricing is live: use the code "GITHUB" to get $300 off (Valid only for August 2025) to [register here](https://maven.com/aishwarya-kiriti/genai-system-design)!!
|
||||
|
||||
@@ -0,0 +1,119 @@
|
||||
# Part 1: What Are Agents Anyway?
|
||||
|
||||
Hi there,
|
||||
|
||||
These days, everyone seems to be racing to “build agents”, but pause for a second.
|
||||
What even is an AI agent? And why is the whole world suddenly obsessed?
|
||||
|
||||
To be honest, there’s no widely accepted definition.
|
||||
But here’s a simple and useful one for our purposes:
|
||||
|
||||
> Generative AI is great at understanding and generating content.
|
||||
> **Agentic AI goes a step further — it understands, generates content, and performs actions.**
|
||||
|
||||
<img width="1024" height="1024" alt="image" src="https://github.com/user-attachments/assets/3bf03a13-eb32-487f-9ddb-2c97919f1e80" />
|
||||
|
||||
---
|
||||
|
||||
## A Quick Rewind
|
||||
|
||||
In 2022, ChatGPT blew up because, for the first time, AI felt conversational.
|
||||
You didn’t need to write code or train models — you could just talk to it.
|
||||
|
||||
Let’s compare:
|
||||
- **Traditional programming** → Needed code to operate
|
||||
- **Traditional ML** → Needed feature engineering
|
||||
- **Deep learning** → Needed task-specific training
|
||||
- **ChatGPT** → Could reason across tasks and respond without training
|
||||
|
||||
This is known as **zero-shot learning** (no examples needed) or **in-context learning** (understands tasks just from instructions).
|
||||
|
||||
---
|
||||
|
||||
## By 2024, People Wanted More
|
||||
|
||||
Talking was cool — but what if the AI could actually do things?
|
||||
|
||||
For example:
|
||||
- Instead of just giving you a list of leads, could it email them?
|
||||
- Instead of summarizing a doc, could it file it in the right folder and create a task in your workflow?
|
||||
- Instead of suggesting a product to a user, could it automatically customize the landing page?
|
||||
|
||||
That’s where **agents** came in.
|
||||
|
||||
---
|
||||
|
||||
## How Do Agents Take Action?
|
||||
|
||||
The magic lies in the **tools**.
|
||||
|
||||
Most agents are paired with APIs, function calls, or plugins that let them interact with external systems.
|
||||
The LLM doesn’t just respond with text — it outputs structured commands like:
|
||||
- `Call the send_email() function with the following inputs…`
|
||||
- `Fetch records from the CRM using this query…`
|
||||
- `Schedule a meeting for Tuesday at 2PM…`
|
||||
|
||||
This works because of a mechanism called **tool use** (or **function calling**).
|
||||
The agent is told what tools are available, and it figures out when and how to use them — either directly or through some planning mechanism.
|
||||
|
||||
---
|
||||
|
||||
## More Advanced Agents Include:
|
||||
- **Memory** → To remember past steps or context
|
||||
- **Planning modules** → To decide what to do next, especially for multi-step tasks
|
||||
- **State management** → So the agent can track progress and avoid loops or failures
|
||||
|
||||
Think of the LLM as the **brain**, and tools as the **hands**.
|
||||
Without tools, an agent just talks. With tools, it acts.
|
||||
|
||||
<img width="1216" height="413" alt="image" src="https://github.com/user-attachments/assets/9cd7fa10-21a0-42a3-95fb-3bf081e10af1" />
|
||||
|
||||
---
|
||||
|
||||
## Two Ways to Define Agents
|
||||
|
||||
**Technical view** → Agents = LLM + Tools + Planning + Memory (and the components above)
|
||||
**Business view** → Agents = Systems that complete tasks end-to-end
|
||||
|
||||
**Important:** Today's agents are not AI innovations.
|
||||
They are **engineering wrappers** around AI models. The underlying intelligence still comes from the AI models — the agent just helps act on that intelligence.
|
||||
|
||||
---
|
||||
|
||||
## How to Actually Build Agentic AI Applications
|
||||
|
||||
Here’s where most people go wrong:
|
||||
They start with “Let’s build an agent!” instead of “What real-world problem are we solving?”
|
||||
|
||||
Flip the narrative.
|
||||
Start with **real-world/enterprise pain points**, like:
|
||||
- A support team drowning in repetitive queries
|
||||
- An analyst switching between dashboards to find insights
|
||||
- A sales team manually logging and tracking customer activity
|
||||
|
||||
This course is focused on building agents that work in the real world — not just demos.
|
||||
Sure, you can spin up quick personal agents or prototypes without much structure, but when you're building for the enterprise, **design choices matter**.
|
||||
|
||||
---
|
||||
|
||||
## A Useful Mental Model: Autonomy vs. Control
|
||||
|
||||
Once you've identified the problem, the next decision is:
|
||||
**How autonomous should your agent be?**
|
||||
|
||||
Think of it as a tradeoff:
|
||||
- How much autonomy are you giving the agent
|
||||
- vs.
|
||||
- How much control do you want to retain on the human side
|
||||
|
||||
This isn't a one-size-fits-all decision — it's contextual.
|
||||
Different problems demand different levels of agent involvement.
|
||||
|
||||
---
|
||||
|
||||
In the next part, we’ll go deeper into this autonomy-control tradeoff and walk through how to design agents based on the level of autonomy your use case actually needs.
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
@@ -0,0 +1,161 @@
|
||||
# Part 2: The 4 Types of Agentic Systems (and When to Use What)
|
||||
|
||||
Hi there,
|
||||
|
||||
In the previous part, we looked at what makes AI agentic — it’s not just about understanding or generating content, it’s about performing actions and handling tasks end-to-end.
|
||||
|
||||
But as teams rush to “add agents” to their stack, here’s the catch:
|
||||
Not all agents are built the same, and not all problems need highly autonomous systems.
|
||||
|
||||
In this lesson, we’ll walk through four types of agentic systems (as discussed yesterday), using a simple but powerful lens:
|
||||
|
||||
- How much autonomy does the agent have?
|
||||
- How much control does the human or system retain?
|
||||
|
||||
This balance impacts how the system behaves, how you evaluate it, and what infrastructure you need to build.
|
||||
|
||||
---
|
||||
<img width="1216" height="413" alt="image" src="https://github.com/user-attachments/assets/58097651-e6d0-4835-9c13-042f647cf437" />
|
||||
|
||||
|
||||
## The Tool-Augmented LLM
|
||||
|
||||
At the core of most modern agents is an **LLM (Large Language Model)** acting as the brain of the system.
|
||||
Throughout this course, we use the term LLM to refer broadly to generative AI models — not just text-only models.
|
||||
|
||||
On its own, it can generate content, but to turn it into an agent, you augment it with:
|
||||
- **Tools** → APIs, functions, databases it can call
|
||||
- **Planning** → The ability to break a goal into multiple steps
|
||||
- **Memory** → So it can track past actions and outcomes
|
||||
- **State and Control Logic** → To know what’s done, what failed, and what to do next
|
||||
|
||||
When connected to these components, the LLM becomes more than a chatbot.
|
||||
It becomes a goal-driven system that can reason, take action, and adapt.
|
||||
|
||||
But depending on how much you trust it to act without supervision, you end up with different types of agents.
|
||||
Let’s walk through them, starting from the least autonomous.
|
||||
|
||||
---
|
||||
|
||||
## 1. Rule-Based Systems/Agents
|
||||
**Low Autonomy, Low Control**
|
||||
|
||||
These systems don’t use LLMs at all. They’re built with traditional *if-this-then-that* logic. Every decision path is manually scripted. There’s no reasoning or learning. Rule-based agents have existed long before the LLM era.
|
||||
|
||||
> Wait, aren’t we talking about AI agents?
|
||||
> Yes — but not every problem needs an AI model. Start with the problem, not the AI. If you can solve it without AI, don’t overcomplicate it.
|
||||
|
||||
**What problems do they solve?**
|
||||
Well-structured, repetitive tasks with fixed inputs and outputs.
|
||||
|
||||
**Examples:**
|
||||
- Automatically approve reimbursements under a fixed amount
|
||||
- Rename files in a folder based on filename patterns
|
||||
- Copy data from Excel sheets into form fields
|
||||
|
||||
**Pros:** Fast, auditable, predictable
|
||||
**Cons:** Brittle to change, can’t handle ambiguity
|
||||
**Best used when:** You know all the conditions ahead of time and there’s no need for flexibility.
|
||||
|
||||
---
|
||||
|
||||
## 2. Workflow Agents
|
||||
**Low Autonomy, High Control**
|
||||
|
||||
This is often the first step for enterprises introducing LLMs into their workflows.
|
||||
Here, the LLM enhances an existing workflow but doesn’t execute actions independently. A human stays in control.
|
||||
|
||||
**What problems do they solve?**
|
||||
Repetitive tasks that benefit from natural language understanding, summarization, or generation, but still need human decision-making.
|
||||
|
||||
**Examples:**
|
||||
- Suggesting first-draft responses in a support tool like Zendesk
|
||||
- Generating summaries of meeting transcripts
|
||||
- Translating natural language queries into structured search inputs for BI dashboards
|
||||
|
||||
**How the LLM is used:**
|
||||
It reads input (text, tickets, documents), understands context, and generates useful content, but doesn’t act on it.
|
||||
A human still decides what to do.
|
||||
|
||||
**Pros:** Easy to deploy, low risk, quick value
|
||||
**Cons:** Can’t execute or plan, limited end-to-end value
|
||||
**Best used when:** You want to augment your team’s productivity without giving up oversight.
|
||||
|
||||
---
|
||||
|
||||
## 3. Semi-Autonomous Agents
|
||||
**Moderate to High Autonomy, Moderate Control**
|
||||
|
||||
These are true agentic systems. They not only understand tasks but can plan multi-step actions, invoke tools, and complete goals with minimal supervision. However, they often operate with some constraints or monitoring built in.
|
||||
|
||||
**What problems do they solve?**
|
||||
Multi-step workflows that are well-understood but too tedious or time-consuming for humans.
|
||||
|
||||
**Examples:**
|
||||
- A lead follow-up agent that drafts, personalizes, and sends emails based on CRM data, while logging results
|
||||
- A document automation agent that extracts details from contracts and updates internal systems
|
||||
- A research agent that pulls data from multiple sources, compares findings, and sends a structured report
|
||||
|
||||
**How the LLM is used:**
|
||||
The LLM plans the steps, calls APIs to fetch or push data, keeps track of progress, and adapts if something goes wrong.
|
||||
It often includes fallback paths or checkpoints for human review.
|
||||
|
||||
**Pros:** Automates complex workflows, saves time, higher ROI
|
||||
**Cons:** Needs infrastructure (planning, memory, tool calling), harder to test
|
||||
**Best used when:** You want to automate well-bounded business workflows while retaining some control.
|
||||
|
||||
---
|
||||
|
||||
## 4. Autonomous Agents
|
||||
**High Autonomy, Low Control**
|
||||
|
||||
These agents are fully goal-driven. You give them a broad objective, and they figure out what to do, how to do it, when to retry, and when to escalate. They act independently, often across systems and over time.
|
||||
|
||||
**What problems do they solve?**
|
||||
High-effort, async, or long-running tasks that span multiple systems or steps and don’t need constant human input.
|
||||
|
||||
**Examples:**
|
||||
- A competitive research agent that pulls data over days, summarizes updates, and generates weekly insight briefs
|
||||
- An ops automation agent that detects issues in pipelines, diagnoses root causes, and files tickets with suggested fixes
|
||||
- A testing agent that autonomously runs product flows, logs results, and suggests new edge-case scenarios
|
||||
|
||||
**How the LLM is used:**
|
||||
The LLM is the planner, decision-maker, tool-user, memory tracker, and communicator. It manages retries, evaluates whether goals are met, and decides when to stop or adapt.
|
||||
|
||||
**Pros:** Extremely scalable, can handle complex tasks
|
||||
**Cons:** High risk if not monitored, hard to evaluate or trace, infra-heavy
|
||||
**Best used when:** The task is high-leverage, async, and doesn’t require human feedback at every step.
|
||||
|
||||
---
|
||||
<img width="683" height="316" alt="image" src="https://github.com/user-attachments/assets/cbea3f8f-b2c5-4dbc-8d0f-5b102430d675" />
|
||||
|
||||
|
||||
## How to Decide What to Build
|
||||
|
||||
Not by picking your favorite architecture.
|
||||
You start with the **problem**.
|
||||
|
||||
Ask yourself:
|
||||
- Is it repetitive and structured?
|
||||
- Does it involve language understanding or generation?
|
||||
- Is it a multi-step task that needs decision-making?
|
||||
- Do you trust an AI system to execute the entire task, or do you want a human in the loop?
|
||||
|
||||
Here’s the key:
|
||||
- These approaches aren’t mutually exclusive.
|
||||
- A single system can mix them — some parts might require high control, others can benefit from high autonomy.
|
||||
- Each problem type can be tackled by either a single agent or a group of collaborating agents.
|
||||
|
||||
We’ll dive deeper into **single-agent vs. multi-agent design** later in the course.
|
||||
For now, remember:
|
||||
> Don’t start with “How do I build a multi-agent system?”
|
||||
> Start with “What’s the problem I’m solving, and what kind of autonomy does it require?”
|
||||
|
||||
Let the problem shape the agentic design, not the other way around.
|
||||
|
||||
---
|
||||
|
||||
In the next part, we’ll dive deeper into the **role of tools** in agentic systems. They’re the reason AI has become far more usable — and we’ll break down exactly how and why in our deep dive.
|
||||
|
||||
|
||||
|
||||
@@ -0,0 +1,152 @@
|
||||
# Part 3: What are Tools in AI?
|
||||
|
||||
In the previous part, we talked about different types of agents, from rule-based to fully autonomous, and how the right level of autonomy depends on the problem you're solving.
|
||||
|
||||
But here's a shared trait across all agent types, no matter how simple or complex:
|
||||
|
||||
> **They rely on tools to perform actions.**
|
||||
|
||||
---
|
||||
|
||||
## What Are “Tools” in AI?
|
||||
|
||||
In the context of agentic AI, tools are external capabilities the LLM can invoke, things like:
|
||||
- APIs
|
||||
- Database queries
|
||||
- Internal services
|
||||
- Third-party systems
|
||||
- Internal functions written in code
|
||||
|
||||
They turn the LLM from something that just **talks** into something that can **act**.
|
||||
|
||||
Remember, LLMs on their own are **stateless**, have **no access to real-time systems**, and **can’t take action**.
|
||||
|
||||
---
|
||||
|
||||
## But Give Them Tools, and They Can:
|
||||
- Fetch data from your internal systems
|
||||
- Trigger events (e.g., send an email, create a JIRA ticket)
|
||||
- Access structured data like calendars, dashboards, or CRMs
|
||||
- Run pre-written logic based on business rules
|
||||
|
||||
This is how **generation turns into execution**.
|
||||
|
||||
---
|
||||
|
||||
## Why Tools Matter
|
||||
|
||||
1. **They unlock execution**
|
||||
Without tools, your agent is just an assistant that makes suggestions.
|
||||
With tools, it can complete workflows end-to-end.
|
||||
|
||||
2. **They increase precision**
|
||||
Rather than hallucinating, the LLM can ask the right system directly —
|
||||
“What’s the actual order status?” instead of making up a delay reason.
|
||||
|
||||
3. **They let you control risk**
|
||||
You define what’s exposed. The LLM can’t do anything outside of the tools you register.
|
||||
|
||||
4. **They enable composability**
|
||||
If you want to combine your CRM, calendar, and email stack into one assistant,
|
||||
you can expose each of those as tools and let the LLM orchestrate them.
|
||||
|
||||
---
|
||||
|
||||
## Step-by-Step Example: End-to-End Agent Task Using Tools
|
||||
|
||||
**Task:**
|
||||
> “Inform a customer that their order is delayed and offer a new delivery time.”
|
||||
|
||||
**Here’s how the system works with tools:**
|
||||
|
||||
**Input** — A human types:
|
||||
_“Hey, can you let John know his order is delayed and reschedule it for tomorrow?”_
|
||||
|
||||
**Planning** — The LLM breaks it down:
|
||||
- Check the order status
|
||||
- If delayed, check delivery slots
|
||||
- Draft an email
|
||||
- Send the email
|
||||
- Log the interaction
|
||||
|
||||
**Tool calls:**
|
||||
```text
|
||||
get_order_status(order_id=12345)
|
||||
get_available_slots(date=today+1)
|
||||
send_email(to=john@example.com, content=...)
|
||||
log_event(event_type="reschedule", status="completed")
|
||||
```
|
||||
|
||||
**Text generation** — The LLM composes the message:
|
||||
_“Hi John, just letting you know your order has been delayed. We’ve rescheduled it for tomorrow. Thanks for your patience.”_
|
||||
|
||||
**Execution** — The system runs the actions, logs the output, and optionally sends a status update to a dashboard.
|
||||
|
||||
---
|
||||
|
||||
## How This Works (Visual)
|
||||
<img width="808" height="357" alt="image" src="https://github.com/user-attachments/assets/f7ce3097-873f-4519-b7ba-30b80785deae" />
|
||||
|
||||
|
||||
Here’s what’s happening:
|
||||
|
||||
1. The user asks a question or gives a task.
|
||||
2. The LLM understands what needs to be done and plans its next step.
|
||||
3. A parser converts the LLM’s idea into a structured format (like `get_order_status(order_id=12345)`).
|
||||
4. The agent calls the right tool — API, database query, or internal function.
|
||||
5. The tool returns a result — this is called an **observation**.
|
||||
6. The LLM looks at the result, decides what’s missing or what comes next.
|
||||
7. This loop continues until it has enough to generate the final answer or complete the task.
|
||||
|
||||
The LLM is using each tool’s result to guide its next decision.
|
||||
|
||||
---
|
||||
|
||||
**Key reminder:**
|
||||
The LLM itself is still just generating text.
|
||||
That text is structured into tool calls, executed externally, and the results are fed back into the LLM — creating a loop of reasoning, action, and reflection (**a.k.a. an agent**).
|
||||
|
||||
This structure is used by frameworks like **LangChain**, **CrewAI**, **AutoGen**, and even custom orchestration setups in production teams.
|
||||
|
||||
---
|
||||
|
||||
## What Makes a Tool Usable by an LLM?
|
||||
|
||||
To register a tool with an agent system, you typically define:
|
||||
- **Name** (e.g., `create_meeting`)
|
||||
- **Description** (so the model knows when to use it)
|
||||
- **Input parameters** (and types)
|
||||
- **Output structure** (so the model can use the result)
|
||||
|
||||
This metadata is what allows the LLM to reason about which tool to use and how.
|
||||
|
||||
---
|
||||
|
||||
## A Note on Parsing and Structured Outputs
|
||||
|
||||
The parser plays a key role in converting the LLM’s response into a structured tool call — something the system can reliably execute (like `get_order_status(order_id=12345)`).
|
||||
|
||||
But in many modern setups, you don’t always need a separate parser.
|
||||
Most popular LLMs, especially those designed for tool use, can directly produce structured outputs — like JSON or function calls — that can be consumed by your backend as-is.
|
||||
|
||||
Similarly, well-designed tools return structured data, making it easier for the LLM to reason about what to do next.
|
||||
|
||||
**The structure on both sides** (input and output) is what makes agent loops **robust, traceable, and production-grade**.
|
||||
|
||||
---
|
||||
|
||||
## The Takeaway
|
||||
|
||||
A lot of this will feel familiar if you've built or worked with APIs before.
|
||||
But if you're not from that world, don’t overthink the wiring.
|
||||
|
||||
Just remember this:
|
||||
> AI models on their own can **understand** and **generate**.
|
||||
> When they’re connected to software, tools, APIs, and internal systems — they can actually **do things**.
|
||||
|
||||
---
|
||||
|
||||
In the next part, we’ll learn about **Retrieval-Augmented Generation (RAG)** — what it is, when to use it, and how it fits naturally into agentic pipelines as a memory or context layer.
|
||||
|
||||
|
||||
|
||||
@@ -0,0 +1,178 @@
|
||||
# Part 4: Retrieval-Augmented Generation (RAG) and the Rise of Agentic RAG
|
||||
|
||||
In the previous part, we looked at how tools help AI agents interact with real-world systems — send emails, file tickets, trigger APIs.
|
||||
|
||||
But what if the model doesn’t need to act?
|
||||
What if it just needs access to the right information?
|
||||
|
||||
That’s the case in many enterprise settings:
|
||||
- Internal docs spread across teams
|
||||
- Policy PDFs no one remembers writing
|
||||
- Customer insights buried in CRM notes
|
||||
- Dashboards and emails with useful context
|
||||
|
||||
Tools won’t help here. The model needs to think with your data.
|
||||
That’s where **RAG** comes in.
|
||||
|
||||
---
|
||||
|
||||
## What Is RAG?
|
||||
|
||||
RAG stands for **Retrieval-Augmented Generation**.
|
||||
It’s a system design where the model retrieves relevant information from your own data — just before generating a response.
|
||||
|
||||
Instead of relying only on what the model was trained on, RAG gives it access to **live, contextual information** from your enterprise systems. This makes answers more accurate, grounded, and auditable.
|
||||
|
||||
You might be wondering:
|
||||
> “Why not just give all the data to the model directly?”
|
||||
|
||||
The problem is:
|
||||
- Models can only process a limited amount of text at a time.
|
||||
- Even within that limit, they struggle when too much irrelevant or noisy information is included.
|
||||
- This makes responses less focused and more error-prone.
|
||||
|
||||
---
|
||||
|
||||
## The RAG Process (at a Glance)
|
||||
|
||||
<img width="1024" height="356" alt="image" src="https://github.com/user-attachments/assets/2e7c2384-a564-45c1-9a08-57cfb02ee435" />
|
||||
|
||||
|
||||
Here’s what it looks like in practice:
|
||||
|
||||
1. **Data** – Your internal content (PDFs, emails, notes, wikis)
|
||||
2. **Chunking** – Broken into smaller parts for better indexing
|
||||
3. **Prompt + Context** – At query time, the system retrieves relevant pieces (retrieval phase)
|
||||
4. **LLM** – The model uses that context to generate a response
|
||||
5. **Output** – The result is based on your data, not just what the model “knows”
|
||||
|
||||
_Image Source: https://hyperight.com/7-practical-applications-of-rag-models-and-their-impact-on-society/_
|
||||
|
||||
---
|
||||
|
||||
## Why RAG Is Everywhere in Enterprise AI
|
||||
|
||||
You’ll often hear this number:
|
||||
> From what I’ve seen across clients and systems, **70% of enterprise GenAI use-cases use RAG**.
|
||||
|
||||
Why RAG is invaluable to enterprises:
|
||||
- Enterprise knowledge changes frequently
|
||||
- Fine-tuning models is expensive and slow
|
||||
- Retrieval is faster, safer, and easier to control
|
||||
- It brings structure and traceability into LLM systems
|
||||
- It works on both unstructured (docs) and semi-structured (dashboards, notes) data
|
||||
|
||||
So instead of asking:
|
||||
> “How do I teach the model everything we know?”
|
||||
Most teams ask:
|
||||
> “How do I let the model fetch what we already have?”
|
||||
|
||||
---
|
||||
|
||||
## RAG = LLM + Additional Retrieved Data
|
||||
|
||||
RAG became the dominant pattern in 2024 for a reason:
|
||||
It bridged the gap between general-purpose LLMs and private, task-specific enterprise knowledge.
|
||||
|
||||
At its core, RAG is simple:
|
||||
- You take an LLM
|
||||
- You feed it additional, retrieved information right before generation
|
||||
|
||||
This makes the model more accurate, more context-aware, and less reliant on memorized facts.
|
||||
It’s especially useful for tasks like **Q&A, summarization, and policy lookups** — particularly in data-rich environments like **legal, finance, and support**.
|
||||
|
||||
No wonder 2024 was dubbed **“the year of RAG.”**
|
||||
|
||||
---
|
||||
|
||||
## But Now We’re Moving Into the Agentic Era
|
||||
|
||||
RAG isn’t going away, but it’s evolving.
|
||||
|
||||
Today’s systems don’t just retrieve once and generate an answer.
|
||||
In **agentic workflows**, retrieval becomes part of a broader, dynamic reasoning loop.
|
||||
|
||||
Agents plan, retrieve, reflect, and retrieve again — not just once, but as many times as needed throughout a task.
|
||||
|
||||
That’s where **Agentic RAG** comes in.
|
||||
|
||||
---
|
||||
|
||||
## What Is Agentic RAG?
|
||||
|
||||
<img width="1456" height="971" alt="image" src="https://github.com/user-attachments/assets/8a7347b0-2ead-4d56-9f33-ef57667d0f00" />
|
||||
|
||||
|
||||
Traditional RAG:
|
||||
- One query
|
||||
- One retrieval
|
||||
- One response
|
||||
|
||||
It works well for standalone questions like:
|
||||
> “What’s our policy on PTO rollover?”
|
||||
|
||||
But most real-world enterprise workflows aren’t one-shot.
|
||||
|
||||
---
|
||||
|
||||
**Example:**
|
||||
Let’s say you’re building a deal assistant for your sales team.
|
||||
In a single task, the agent may need to:
|
||||
- Pull the customer’s CRM history
|
||||
- Retrieve current pricing for their segment
|
||||
- Look up regional legal terms
|
||||
- Reference past contract clauses
|
||||
- Generate a custom proposal
|
||||
- Double-check facts
|
||||
- Log the interaction
|
||||
|
||||
---
|
||||
|
||||
In **agentic systems**, retrieval isn’t just a setup step.
|
||||
It’s how the agent:
|
||||
- Gathers missing context
|
||||
- Checks its assumptions
|
||||
- Adapts mid-task
|
||||
|
||||
That means RAG becomes:
|
||||
- A tool for in-task learning
|
||||
- A method for reducing hallucinations
|
||||
- A mechanism for handling dynamic workflows
|
||||
- A bridge between reasoning and grounded enterprise knowledge
|
||||
|
||||
Agentic RAG turns retrieval into a **first-class decision-making loop** by using retrieval as part of the model’s thinking process.
|
||||
|
||||
---
|
||||
|
||||
## RAG as a Tool
|
||||
|
||||
If you think about it, RAG is also a kind of **tool**.
|
||||
But instead of triggering an action, it helps the agent pull the right information from a large volume of data.
|
||||
|
||||
In practice, agents often combine:
|
||||
- **RAG**
|
||||
- **Tools**
|
||||
- **Planning**
|
||||
|
||||
…to complete complex tasks **reliably and contextually**.
|
||||
|
||||
---
|
||||
|
||||
## A Note on Scope
|
||||
|
||||
RAG is a deep and rapidly evolving space — honestly, it could be its own course.
|
||||
If you're curious to explore further:
|
||||
- I’ve curated a **GitHub repo** of key RAG papers that covers the landscape well
|
||||
- I have a **101 guide on Agentic RAG** too
|
||||
|
||||
That said, not every RAG optimization is necessary for every use-case.
|
||||
In our 6-week course, we focus on helping you understand **when and where** each technique makes sense, rather than applying them blindly.
|
||||
|
||||
---
|
||||
|
||||
|
||||
In the next part, we’ll dive into one of the most talked-about concepts lately: **Model Context Protocol (MCP)**.
|
||||
|
||||
To get the most out of it, we’d recommend revisiting **Part 3 on tools**, since MCP builds directly on that concept!
|
||||
|
||||
|
||||
@@ -0,0 +1,116 @@
|
||||
# Part 5: What Is MCP and Why Should You Care?
|
||||
|
||||
---
|
||||
|
||||
## First, a Quick Recap
|
||||
|
||||
- **Part 3:** We learned that tools let models **do things**.
|
||||
- **Part 4:** We saw that RAG helps models **find relevant info** before answering.
|
||||
|
||||
These are **external supports** — they help the model act smarter, but the coordination still sits outside the model.
|
||||
|
||||
But what if you could pass **all the context a model needs** — tools, retrieved data, memory, instructions — in one clean, structured format?
|
||||
|
||||
That’s what **Model Context Protocol (MCP)** is trying to solve.
|
||||
|
||||
---
|
||||
|
||||
## So What Is MCP?
|
||||
<img width="737" height="452" alt="image" src="https://github.com/user-attachments/assets/2e17fcb6-ab35-4a4c-88b4-c74db165e4b8" />
|
||||
|
||||
|
||||
At its core, **Model Context Protocol** is a standardized way to give an LLM everything it needs to reason and respond.
|
||||
|
||||
Think of it like packaging up:
|
||||
|
||||
- The task you want the model to do
|
||||
- The tools/APIs it can use
|
||||
- The documents or memory it might need
|
||||
- The prior messages in the conversation
|
||||
|
||||
…and then handing all of that over in one go.
|
||||
|
||||
It’s **not** a tool, library, or product.
|
||||
It’s a **protocol** — a structure for communication between the model and the outside world.
|
||||
|
||||
If you’re from the tech world, equivalents would be: **HTTP**, **TCP/IP**, or **SMTP**.
|
||||
If you’re not, just remember: tech folks love standardization — it makes things easier to reuse and plug together.
|
||||
|
||||
---
|
||||
|
||||
## Why Does This Matter?
|
||||
|
||||
Let’s say you’re building an agent.
|
||||
Right now, you’re probably juggling:
|
||||
|
||||
- Sending a prompt
|
||||
- Passing retrieved documents
|
||||
- Registering tools
|
||||
- Managing state
|
||||
- Keeping track of what happened before
|
||||
|
||||
MCP says:
|
||||
> “Let’s standardize how we give all of that to the model, so we don’t reinvent the wheel for every use case.”
|
||||
|
||||
And for **enterprises**, this matters a lot.
|
||||
As agents get more complex, coordinating **tools**, **RAG**, **memory**, and **outputs** becomes messy.
|
||||
|
||||
MCP makes that orchestration **composable**, **modular**, and easier to plug into other systems.
|
||||
|
||||
If you’ve ever worked with APIs, think of MCP like a **well-defined request schema**.
|
||||
Instead of tossing everything into one long string and hoping the model figures it out, the model still sees text — but it’s **structured**, with **clear context, options, and grounding**.
|
||||
|
||||
---
|
||||
|
||||
## Why Did MCP Catch On So Fast?
|
||||
|
||||
Given that MCP is just a protocol, you might be wondering:
|
||||
> What makes it better, and why did everyone jump on board?
|
||||
|
||||
Here’s what helped:
|
||||
|
||||
1. **AI-Native** — MCP was built for AI agents. It makes space for everything agents use today: tools, prompts, memory, documents, and more.
|
||||
2. **Strong docs and examples** — Anthropic (creators of MCP) released not just the spec but also clients, SDKs, testing tools, and real-world demos.
|
||||
3. **Network effect** — Released quietly in Nov 2024, most people slept on it… until 2025, when it exploded. Tools, startups, and even OpenAI began supporting it.
|
||||
|
||||
---
|
||||
|
||||
## Common Misunderstandings
|
||||
|
||||
- **MCP isn’t a new API or product** — It’s just a pattern, a clean way to frame what you send to the model.
|
||||
- **It doesn’t make models smarter** — It just gives them better, more structured context.
|
||||
- **It’s not just for agents** — Even simple assistants benefit from better context management.
|
||||
|
||||
---
|
||||
|
||||
## So… Should You Care?
|
||||
|
||||
If you’re building toy prompts or quick demos — probably not (yet).
|
||||
|
||||
But if you’re working on:
|
||||
|
||||
- Enterprise-grade agents
|
||||
- Multi-tool workflows
|
||||
- LLMs that need to access **memory + RAG + planning**
|
||||
- Systems where **context management** is a bottleneck
|
||||
|
||||
…then **yes**, you should care. MCP is about getting better at passing evolving, structured context into models.
|
||||
|
||||
But keep in mind: MCP is just a protocol.
|
||||
Like all standards, it only works if it’s widely adopted.
|
||||
If something better comes along before MCP becomes “the HTTP of agents,” the ecosystem might shift again.
|
||||
|
||||
---
|
||||
|
||||
## Further Reading & Resources
|
||||
|
||||
- We did a **[full deep-dive article](https://thenuancedperspective.substack.com/p/mcp-overhyped-misunderstood-and-actually)** on MCP, including clients, servers, and real-world use cases (written by Kiriti Badam, OpenAI).
|
||||
- We also ran a **free live session** — you can catch the [recording](https://maven.com/p/82345a) here.
|
||||
|
||||
---
|
||||
|
||||
In the next part, we’ll learn about the **planning** component of agentic systems and why it matters.
|
||||
|
||||
PS: We also teach a widely loved course on how to actually build AI systems in this fast-changing environment, using a problem-first approach. It’s designed for PMs, leaders, engineers, decision-makers etc. who are working within real-world constraints. Alumni come from Google, Meta, Apple, Netflix, AWS, Spotify, Snapchat, Deloitte, and more. Our next cohort starts soon. Our next cohort starts soon. Early bird pricing is live: use the code "GITHUB" to get $300 off (Valid only for August 2025) to [register here](https://maven.com/aishwarya-kiriti/genai-system-design)!!
|
||||
|
||||
|
||||
@@ -0,0 +1,143 @@
|
||||
# Part 6: Planning in Agents + Reasoning Models
|
||||
|
||||
|
||||
---
|
||||
|
||||
## Woah! We’re more than halfway through our course!
|
||||
|
||||
Over the past few parts, we talked about what agents can do:
|
||||
- Use tools
|
||||
- Retrieve information through RAG
|
||||
- Pass everything in a clean format using MCP
|
||||
|
||||
But all of that assumes something fundamental:
|
||||
**That the agent actually knows what to do next.**
|
||||
And that’s where things often break.
|
||||
|
||||
Today, we shift focus from tools and inputs to **how agents think** — more specifically, how modern models are starting to plan and why that changes how we design real-world systems.
|
||||
|
||||
---
|
||||
|
||||
## Why Planning Matters in Agentic Systems
|
||||
|
||||
Here are a few examples to start with.
|
||||
|
||||
If you ask an agent:
|
||||
> “What’s 13 multiplied by 47?”
|
||||
…it can either solve it directly or call a calculator. This is a one-step task — no real planning needed.
|
||||
|
||||
Now imagine asking:
|
||||
> “Find all our Q1 clients in the healthcare sector, check which ones are overdue on payments, and draft personalized emails with new payment links.”
|
||||
|
||||
In this case, the agent needs to:
|
||||
- Understand the instruction
|
||||
- Break it into manageable parts
|
||||
- Retrieve the right data
|
||||
- Choose tools
|
||||
- Perform steps in order
|
||||
- Handle exceptions
|
||||
- Know when the task is done
|
||||
|
||||
That loop of interpreting, sequencing, and acting is **planning**.
|
||||
|
||||
The agent (meaning the model) is expected to figure this out on its own — including which tools to use and how to apply the information it has.
|
||||
|
||||
---
|
||||
|
||||
## Why Traditional LLMs Struggle With Planning
|
||||
|
||||
Most general-purpose LLMs were never trained to do this.
|
||||
|
||||
They are trained to **predict the next token** based on the previous context — nothing more.
|
||||
They excel at:
|
||||
- Continuing sentences
|
||||
- Generating summaries
|
||||
- Answering direct questions
|
||||
|
||||
…but they behave more like **short-sighted generators**.
|
||||
They complete what’s in front of them but aren’t wired to think ahead.
|
||||
|
||||
When asked to act as agents in multi-step, decision-making tasks, they tend to:
|
||||
- Skip steps
|
||||
- Repeat actions
|
||||
- Overcomplicate simple things
|
||||
- Lose the plot halfway through
|
||||
|
||||
---
|
||||
|
||||
## Early Attempts to Improve Reasoning
|
||||
|
||||
To patch this gap, builders experimented with prompting techniques to nudge planning behavior.
|
||||
|
||||
A popular example: **Chain-of-Thought prompting** — adding “Let’s think step by step” to break tasks into stages.
|
||||
|
||||
This worked for logic puzzles and structured Q&A, but fell short for **real agents** working with:
|
||||
- Tools
|
||||
- Unpredictable inputs
|
||||
- Changing state
|
||||
|
||||
Because underneath, these models still weren’t trained for planning — they were just responding to **prompt tricks**.
|
||||
|
||||
---
|
||||
|
||||
## Then Came Reasoning Models
|
||||
|
||||
The next shift: train models to plan **by design**.
|
||||
|
||||
This gave rise to **Large Reasoning Models (LRMs)**.
|
||||
<img width="743" height="663" alt="image" src="https://github.com/user-attachments/assets/ce4d8d91-b539-4003-adfc-1fa6dcfd3631" />
|
||||
|
||||
**LLMs:**
|
||||
input → LLM → output statement
|
||||
|
||||
**LRMs:**
|
||||
input → LRM → plan step + output statement
|
||||
|
||||
|
||||
|
||||
All still text, but LRMs are nudged during training to **think before acting**.
|
||||
|
||||
---
|
||||
|
||||
**Examples:**
|
||||
- OpenAI’s **o-series** (o1, o3) — first public examples
|
||||
- DeepSeek’s **DeepSeek-R1** — tuned for tool-augmented reasoning and planning
|
||||
- Google’s **Gemini thinking models**
|
||||
- Anthropic’s **Claude 3.7 reasoning mode**
|
||||
|
||||
Some even activate reasoning **only when needed**.
|
||||
|
||||
---
|
||||
|
||||
## How They Fit in Agentic Design
|
||||
|
||||
The main value of reasoning models is in improving the **planning component** — the part that asks:
|
||||
> “What should I do next, and why?”
|
||||
|
||||
In enterprise use cases, **planning is where agents often fail**.
|
||||
Reasoning models can help, but they aren’t magic.
|
||||
|
||||
---
|
||||
|
||||
## Use Them With Caution
|
||||
|
||||
Reasoning models are still **new** and come with tradeoffs:
|
||||
- Overthink simple tasks
|
||||
- Generate longer outputs
|
||||
- Increase latency and cost
|
||||
- Can hallucinate logical-sounding but incorrect plans
|
||||
|
||||
**Rule of thumb:**
|
||||
- Don’t start with a reasoning model.
|
||||
- Begin with a mid-size base model.
|
||||
- Only switch if you see clear planning failures — and even then, evaluate the real impact.
|
||||
|
||||
---
|
||||
|
||||
## Up Next
|
||||
|
||||
In the next part, we’ll shift to another **core component of agents**: **memory** — how agents can remember effectively and why it matters.
|
||||
|
||||
|
||||
|
||||
|
||||
@@ -0,0 +1,178 @@
|
||||
# Part 7: Memory in Agents
|
||||
|
||||
|
||||
---
|
||||
|
||||
Over the past few parts, we’ve explored what makes agents act — from **tools** and **RAG**, to **MCP** and **reasoning models**.
|
||||
|
||||
Today, we shift gears to something that determines **how well** they act over time: **memory**.
|
||||
|
||||
Because here’s the baseline:
|
||||
AI models **don’t have memory inherently**. They’re **stateless** by design. Every input is treated independently unless you **architect memory into the system**.
|
||||
|
||||
---
|
||||
|
||||
## Why Memory Matters
|
||||
<img width="438" height="164" alt="image" src="https://github.com/user-attachments/assets/1d3fccea-8f9a-42b8-baa4-b3fe9f57dad1" />
|
||||
|
||||
|
||||
_Image Source: https://arxiv.org/html/2502.12110v1_
|
||||
|
||||
If an agent is helping you draft emails, summarize long threads, or manage workflows over days or weeks — it needs to remember:
|
||||
- The email format
|
||||
- The user's name
|
||||
- The tone to use
|
||||
|
||||
Sure, you could pass that information again and again with every prompt…
|
||||
But wouldn’t it be better if the agent could retrieve the right information **on its own**, at the right time, from an **external database**?
|
||||
|
||||
That’s exactly where **memory** comes in.
|
||||
|
||||
---
|
||||
|
||||
## “Wait… isn’t this just like Agentic RAG (Day 4)?”
|
||||
|
||||
Fair question — and you’re not wrong. Managing memory often looks a lot like doing Agentic RAG.
|
||||
|
||||
You:
|
||||
1. Write structured or unstructured memories (facts, logs, past outputs)
|
||||
2. Store them with metadata, tags, or embeddings
|
||||
3. Retrieve the relevant slice when needed
|
||||
4. Ground the model’s next action using that context
|
||||
|
||||
**The difference:**
|
||||
- **RAG** → Helps answer questions with knowledge.
|
||||
- **Memory** → Helps agents behave coherently over time.
|
||||
|
||||
---
|
||||
|
||||
## Two Types of Memory in Agents
|
||||
|
||||
When designing real-world agent systems, you typically deal with **two kinds of memory**.
|
||||
|
||||
<img width="571" height="372" alt="image" src="https://github.com/user-attachments/assets/7c2d9e58-a219-4cea-b9f0-6de091298d66" />
|
||||
|
||||
|
||||
_Image Source: https://langchain-ai.github.io/langgraph/concepts/memory/#what-is-memory_
|
||||
|
||||
---
|
||||
|
||||
### 1. Short-Term Memory
|
||||
|
||||
Scoped to a single session or task.
|
||||
|
||||
**Includes:**
|
||||
- The conversation so far
|
||||
- Tools used
|
||||
- Responses generated
|
||||
- Documents retrieved
|
||||
|
||||
Think of it as a raw log of user–agent conversations.
|
||||
|
||||
LangGraph, Autogen, and similar frameworks treat this as part of the agent’s **state**.
|
||||
But state grows fast, and most agents perform poorly when buried under irrelevant history.
|
||||
|
||||
**Strategies to manage short-term memory:**
|
||||
- Trim stale messages
|
||||
- Summarize the past into key points
|
||||
- Filter based on what’s still relevant
|
||||
|
||||
It’s a balancing act: **context length vs clarity vs cost**.
|
||||
|
||||
---
|
||||
|
||||
### 2. Long-Term Memory
|
||||
|
||||
Lives across sessions, days, weeks — even forever.
|
||||
|
||||
**Helps agents remember:**
|
||||
- Who the user is
|
||||
- How they prefer to interact
|
||||
- What’s already been done
|
||||
- Important past context
|
||||
|
||||
**Examples:**
|
||||
- “User prefers neutral tone”
|
||||
- “User name is X and stays in city Y”
|
||||
- “Invoice #123 has already been escalated”
|
||||
|
||||
More data ≠ better by default — it’s about retrieving the right thing at the right time.
|
||||
|
||||
---
|
||||
|
||||
## Types of Long-Term Memory to Consider
|
||||
|
||||
Borrowing from cognitive science:
|
||||
|
||||
- **Semantic Memory** → Facts and info (objective)
|
||||
_“User speaks English and prefers Excel files.”_
|
||||
|
||||
- **Episodic Memory** → Past actions
|
||||
_“Agent already generated a summary yesterday.”_
|
||||
|
||||
- **Procedural Memory** → Preferences (subjective)
|
||||
_“Avoid passive voice. Prioritize action items.”_
|
||||
|
||||
---
|
||||
|
||||
**Examples by use case:**
|
||||
|
||||
- **User-facing chatbots** → Semantic memory for personalization
|
||||
- **Process automation agents** → Episodic memory to avoid retries or loops
|
||||
- **Adaptive assistants** → Procedural memory to adjust prompts based on feedback
|
||||
|
||||
---
|
||||
|
||||
## Key Design Questions
|
||||
|
||||
Before saying “we need memory,” ask:
|
||||
- **What kind?**
|
||||
- **Why is it needed?**
|
||||
- **How will it be stored, retrieved, and kept fresh?**
|
||||
|
||||
---
|
||||
|
||||
## Managing Memory in Practice
|
||||
|
||||
Managing memory often feels like managing RAG.
|
||||
The hard part? Deciding **what to store** and **what to retrieve**.
|
||||
|
||||
Stuffing more text into the agent input rarely helps — it often **hurts performance**.
|
||||
|
||||
You need to design memory intentionally, based on:
|
||||
- The agent’s job
|
||||
- What it needs to recall
|
||||
- When it should recall it
|
||||
- How to keep it useful over time
|
||||
|
||||
---
|
||||
|
||||
## A Few Enterprise Examples
|
||||
|
||||
**Customer Support Agent**
|
||||
- Needs: recent support history, known bugs, user sentiment
|
||||
- Memory types: episodic + semantic
|
||||
|
||||
**Sales Copilot**
|
||||
- Needs: previous pitches, user objections, close status
|
||||
- Memory types: semantic + procedural
|
||||
|
||||
**Compliance Auditor Agent**
|
||||
- Needs: flagged items, prior exceptions, policy changes
|
||||
- Memory types: episodic
|
||||
|
||||
---
|
||||
|
||||
In all cases, it’s not about **how much** data you store — it’s about **how relevant and structured** it is.
|
||||
|
||||
And yes, I’ve said this painfully many times, but I’ll say it again:
|
||||
> **Problem-first, always.** The memory strategy, like tools or planning, depends entirely on the problem you’re solving.
|
||||
|
||||
---
|
||||
|
||||
## Up Next
|
||||
|
||||
In the next part, we’ll talk about **multi-agent systems** — what they are, how they coordinate, and whether you actually need more than one agent at all.
|
||||
|
||||
|
||||
|
||||
@@ -0,0 +1,134 @@
|
||||
# Part 8: Multi-Agent Systems
|
||||
|
||||
---
|
||||
|
||||
So far, we’ve talked a lot about what makes a **single agent** act — from **tools** and **RAG** to **memory** and **planning**.
|
||||
|
||||
But what if your agentic pipeline needs to:
|
||||
- Parallelize tasks to speed things up
|
||||
- Use different agent personas for different parts of a task
|
||||
- Break up complexity across specialized units, like in a team
|
||||
|
||||
That’s where **multi-agent systems** come in.
|
||||
|
||||
---
|
||||
|
||||
## Why Use Multi-Agent Systems?
|
||||
|
||||
Sometimes, a single agent just can’t cut it because the problem demands **scale**, **specialization**, or **parallel thinking**.
|
||||
|
||||
**Examples:**
|
||||
- Generating a marketing strategy that needs **market insights**, **legal review**, and **creative suggestions**.
|
||||
- Building a compliance assistant that needs to **extract information**, **flag risks**, and **cross-check policies**.
|
||||
- Automating a sales process where **one agent** talks to the user, **another** enriches data, and **a third** handles follow-ups.
|
||||
|
||||
Could you do this with one beefy agent?
|
||||
**Maybe.**
|
||||
|
||||
But splitting it into **multiple, specialized agents** can enable:
|
||||
- **Parallelization** → Agents work on parts of a task simultaneously
|
||||
- **Specialization** → One agent is great at legalese, another at writing emails
|
||||
- **Tooling independence** → Each agent can have its own tools and memory
|
||||
|
||||
---
|
||||
|
||||
## Flat vs Hierarchical Agent Coordination
|
||||
|
||||
All multi-agent systems need some way to **coordinate**.
|
||||
Two common communication patterns:
|
||||
<img width="1144" height="626" alt="image" src="https://github.com/user-attachments/assets/c6e2c1c0-bb92-48c8-ba53-dfbfdc6e3926" />
|
||||
|
||||
|
||||
---
|
||||
|
||||
### 1. Hierarchical Patterns (More Controllable)
|
||||
|
||||
An **orchestrator agent** delegates subtasks to others.
|
||||
It sees the full picture and controls the flow.
|
||||
|
||||
**Use when:**
|
||||
- Tasks can be clearly decomposed
|
||||
- You want tight control
|
||||
- You have known agent roles (e.g., summarizer, generator, checker)
|
||||
|
||||
**Think:** enterprise workflows, tool suites, parallel pipelines.
|
||||
|
||||
---
|
||||
|
||||
### 2. Flat Patterns (More Dynamic)
|
||||
|
||||
Agents talk to each other as **peers** — no boss.
|
||||
|
||||
**Use when:**
|
||||
- Tasks need creativity or debate
|
||||
- You want agents to evaluate each other
|
||||
- There’s no one “correct” answer path
|
||||
|
||||
**Think:** brainstorming, ranking options, multi-view reasoning.
|
||||
|
||||
---
|
||||
|
||||
## What Nobody Tells You: Multi-Agent Systems Are a Pain
|
||||
|
||||
On paper, this sounds great.
|
||||
And sure, you can build quick multi-agent prototypes and have fun with them.
|
||||
|
||||
But for **customer/enterprise** use cases… it’s painful.
|
||||
|
||||
Most people read a blog on multi-agent systems and get excited about modularity —
|
||||
> “It’s like microservices!” they say.
|
||||
|
||||
But **AI agents are not microservices**.
|
||||
|
||||
Unlike code, AI models are **non-deterministic**. They don’t always behave the same way.
|
||||
Adding more agents means:
|
||||
- More **non-determinism** (variation across agents, not just within one)
|
||||
- More **memory and state complexity** (who knows what, and when?)
|
||||
- Higher **latency** and **cost**
|
||||
- More **coordination bugs** and failure points
|
||||
- More **collusion**, where agents agree when they shouldn’t (happens more than you think)
|
||||
|
||||
Honestly, I could write a book on how painful it is to get multi-agent systems working reliably.
|
||||
|
||||
---
|
||||
|
||||
## So… Should You Use Them?
|
||||
|
||||
My personal rule:
|
||||
> **In the enterprise, don’t start with multi-agents. Start with one.**
|
||||
|
||||
Let that **one agent** fail — empirically (via eval metrics) or operationally — before you scale.
|
||||
|
||||
From my experience, **70%+ of enterprise use cases** work just fine with a single well-designed agent — one that uses **tools**, **memory**, **RAG**, and **planning**.
|
||||
|
||||
---
|
||||
|
||||
### Multi-agent systems shine when:
|
||||
- The task is big enough to need **parallel execution**
|
||||
- You need **clear specialization**
|
||||
- You want **creative debate**, evaluation, or distributed decision-making
|
||||
|
||||
Even then, you need **strong design** — especially around **memory**, **state**, and **communication protocols**.
|
||||
|
||||
---
|
||||
|
||||
## Final Word: Problem First, Always
|
||||
|
||||
This has been our mantra from Day 1:
|
||||
> Don’t build a multi-agent system because it sounds “agentic.”
|
||||
> Build it if — and only if — your problem needs it.
|
||||
|
||||
The only way to know?
|
||||
- Have the right **metrics**
|
||||
- Test
|
||||
- Let simpler systems fail first
|
||||
|
||||
---
|
||||
|
||||
## Up Next
|
||||
|
||||
In the next part, we’ll talk about **real-world agents** and how they function.
|
||||
|
||||
|
||||
|
||||
|
||||
@@ -0,0 +1,119 @@
|
||||
# Part 9: Real-world Agentic Systems (Under the hood)
|
||||
|
||||
|
||||
|
||||
---
|
||||
|
||||
So far, we’ve covered all the ingredients that make up an agent:
|
||||
**tools**, **planning**, **RAG**, **memory**, **structure**, and **coordination** in multi-agent setups.
|
||||
|
||||
But you might be thinking:
|
||||
> “Where does all this actually show up in the real world?”
|
||||
|
||||
Let’s walk through a few public-facing systems that exhibit **agentic behavior** — as far as we can tell.
|
||||
|
||||
⚠️ **Note:**
|
||||
These aren’t open source. We don’t know their exact internals.
|
||||
What follows is an informed simplification based on how they behave externally — just enough to understand how the agentic stack might show up in practice.
|
||||
|
||||
---
|
||||
|
||||
## **NotebookLM (Google): Agentic Search on Your Own Data**
|
||||
|
||||
Google’s NotebookLM acts like a personal research assistant. You upload your files, and it helps you work with them — summarizing, answering questions, even generating audio versions or study guides.
|
||||
|
||||
**Core focus:** Q&A over your content — essentially a scaled-up, personal RAG system.
|
||||
|
||||
**How it likely works:**
|
||||
1. **User uploads files** (PDFs, notes, slides, etc.)
|
||||
2. **Preprocessing** — Stores them for retrieval later.
|
||||
3. **User asks a question** — e.g., _“What were the key insights from my Q2 strategy deck?”_
|
||||
4. **Planning** — Interprets task type (summary, Q&A, comparison?), identifies relevant docs/sections.
|
||||
5. **RAG** — Retrieves the most relevant document chunks.
|
||||
6. **LLM Generation** — Responds clearly, grounded in your content.
|
||||
7. **Memory** —
|
||||
- Short-term: Tracks the conversation.
|
||||
- Long-term: Likely minimal or none.
|
||||
8. **Tools** — Possibly file viewers, summarization modules.
|
||||
|
||||
**What makes it agentic:** Interprets goals, searches across your data, and composes responses — not just static outputs.
|
||||
|
||||
---
|
||||
|
||||
## **Perplexity: Agentic Search on the Open Web**
|
||||
|
||||
Perplexity gives you a direct, answer-like response with sources — instead of a page of links.
|
||||
|
||||
**How it likely works:**
|
||||
1. **User asks a question** — e.g., _“What’s the latest research on Alzheimer’s treatments?”_
|
||||
2. **Planning** — Interprets intent (“latest,” “credible”), decides search approach.
|
||||
3. **Tool Use** — Issues queries via web APIs.
|
||||
4. **RAG** — Retrieves relevant page snippets.
|
||||
5. **LLM Response** — Synthesizes an answer with citations.
|
||||
6. **Memory** —
|
||||
- Short-term: Session context.
|
||||
- Long-term: May store preferences (e.g., “always use WSJ for news”).
|
||||
|
||||
**What makes it agentic:** Fetches info, decides what to use, and constructs an answer in a multi-step loop.
|
||||
|
||||
---
|
||||
|
||||
## **DeepResearch (OpenAI): Deep Agentic Workflows**
|
||||
|
||||
DeepResearch tackles **open-ended, complex research tasks** — e.g., market analysis, competitive landscapes, technical deep dives.
|
||||
|
||||
**How it likely works:**
|
||||
1. **User asks a broad task** — e.g., _“Analyze the generative AI landscape for education startups.”_
|
||||
2. **Planning** — Breaks into subtasks (funding, trends, companies, risks), forms an execution plan.
|
||||
3. **Tools** — Likely includes:
|
||||
- Web search
|
||||
- Document readers (PDFs)
|
||||
- Data tools (spreadsheets, graphs)
|
||||
- Report generation modules
|
||||
4. **Agentic RAG** — Not one-shot retrieval — fetches, reflects, re-fetches as task evolves.
|
||||
5. **Memory** —
|
||||
- Episodic: Tracks which parts are done.
|
||||
- Semantic: Stores key facts/names.
|
||||
6. **Multi-step Reasoning** — Loops: plan → retrieve → read → rethink → generate → refine → repeat.
|
||||
|
||||
**What makes it agentic:** Heavy planning, iterative tool use, self-directed progress.
|
||||
|
||||
---
|
||||
|
||||
## **Connecting to Day 2: Levels of Autonomy**
|
||||
<img width="694" height="370" alt="image" src="https://github.com/user-attachments/assets/48812496-309d-42cd-9c86-8ef3cb345ec2" />
|
||||
<img width="969" height="231" alt="image" src="https://github.com/user-attachments/assets/12489209-a75b-4a31-853e-33dda02e1aaa" />
|
||||
|
||||
|
||||
|
||||
**NotebookLM** — Between Level 2 and Level 3.
|
||||
- High-control workflow agent.
|
||||
- Strong retrieval, limited autonomous decision-making.
|
||||
|
||||
**Perplexity** — Level 3 (maybe touching Level 4).
|
||||
- Plans queries, organizes sources, crafts answers.
|
||||
|
||||
**DeepResearch** — Strong Level 4.
|
||||
- Takes high-level goals, breaks down tasks, works iteratively with minimal guidance.
|
||||
|
||||
---
|
||||
|
||||
## Try It Yourself
|
||||
|
||||
They all have free versions — experiment and watch for:
|
||||
- How much **control** you have
|
||||
- How much the **system decides** on its own
|
||||
|
||||
It’s a great way to sharpen your instinct for agent design.
|
||||
|
||||
---
|
||||
|
||||
## Up Next
|
||||
|
||||
In the next part, we’ll wrap up the series:
|
||||
- Summarize what we’ve learned
|
||||
- Share best practices
|
||||
- Take a quick look at where **agentic AI** is headed
|
||||
|
||||
|
||||
|
||||
@@ -0,0 +1,101 @@
|
||||
# AI Evals for Everyone - Free Course 🎯
|
||||
|
||||
|
||||
|
||||
Welcome to **AI Evals for Everyone**, a beginner-friendly 101 course that clears up all the confusion around AI evaluation. No matter your background, this course will equip you with practical knowledge to build evaluations that actually work.
|
||||
|
||||
## 🎬 NEW: Watch on YouTube!
|
||||
|
||||
**The complete course is now available as a video series!**
|
||||
|
||||
[](https://www.youtube.com/playlist?list=PLZoalK-hTD4VPIkRXNdSEwcTCt2QUgEPR)
|
||||
|
||||
**Bonus Content**: The YouTube series includes **3 additional hands-on chapters** on **Building Evals with Arize AI** - practical tutorials to implement everything you've learned!
|
||||
|
||||
[**Watch the Full Playlist →**](https://www.youtube.com/playlist?list=PLZoalK-hTD4VPIkRXNdSEwcTCt2QUgEPR)
|
||||
|
||||
## 🎓 Get Certified!
|
||||
|
||||
**Follow these simple steps to earn your AI Evals certification:**
|
||||
|
||||
1. **📚 Read all 10 chapters or watch the videos on YouTube** - Complete the course content at your own pace
|
||||
2. **📝 Take the final assessment** - Test your knowledge with our [certification quiz](https://ai-evals-course-website-2025.vercel.app/quiz-google.html)
|
||||
3. **🏆 Get your certificate** - Receive a personalized certificate upon completion
|
||||
|
||||
|
||||
|
||||
**[Start Your Certification Journey →](https://ai-evals-course-website-2025.vercel.app/quiz-google.html)**
|
||||
|
||||
## 💬 What Students Are Saying
|
||||
|
||||
See what others who completed the course have to say: [Student Testimonials](https://testimonial.to/evals-for-everyone/all)
|
||||
|
||||
## 📚 Course Overview
|
||||
|
||||
Start from zero and learn step-by-step how to build AI evaluation systems. This 101 course cuts through the hype and confusion to give you clear, practical guidance you can implement immediately.
|
||||
|
||||
**Created by:** [Aishwarya Naresh Reganti](https://www.linkedin.com/in/areganti/) & [Kiriti Badam](https://www.linkedin.com/in/sai-kiriti-badam/)
|
||||
|
||||
## 📖 Course Chapters
|
||||
|
||||
1. **[WTH are AI Evals?](./chapters/01_wth_are_ai_evals.md)** - Understanding why AI evaluation is different and unavoidable
|
||||
2. **[Model Evaluations vs Product Evaluations](./chapters/02_model_vs_product_evaluations.md)** - Learning the crucial distinction that trips up most teams
|
||||
3. **[The Evaluation Framework](./chapters/03_evaluation_building_blocks.md)** - Core components for systematic evaluation
|
||||
4. **[Building Reference Datasets](./chapters/04_building_reference_datasets.md)** - Creating reference datasets before you launch your product
|
||||
5. **[How to Build Evaluation Metrics](./chapters/05_building_evaluation_metrics.md)** - Practical approaches from code-based metrics to LLM judges and more
|
||||
6. **[Production Challenges](./chapters/06_production_challenge.md)** - Why production breaks all your assumptions (and evals sometimes)
|
||||
7. **[Production Monitoring Strategies](./chapters/07_production_monitoring_strategies.md)** - Real-world monitoring to understand emerging patterns
|
||||
8. **[The Complete Evaluation Process](./chapters/08_evaluation_process.md)** - Building confidence incrementally through iterations
|
||||
9. **[Common Misconceptions About AI Evaluation](./chapters/09_case_studies.md)** - Real examples from AI products at scale
|
||||
10. **[Glossary of Terms](./chapters/10_common_pitfalls.md)** - Summary of terms generally used in evaluation process
|
||||
|
||||
|
||||
## 🚀 Who Should Take This Course?
|
||||
|
||||
- **AI Engineers** building production systems
|
||||
- **Product Managers** responsible for AI products
|
||||
- **Data Scientists** transitioning to production
|
||||
- **Engineering Leaders** making evaluation strategy decisions
|
||||
- **Quality Engineers** expanding into AI testing
|
||||
|
||||
## 💡 What You'll Learn
|
||||
|
||||
- Why AI systems need evaluation (it's simpler than you think!)
|
||||
- The difference between testing models vs testing your actual product
|
||||
- How to build your first evaluation dataset in just a few hours
|
||||
- Three straightforward approaches to measuring AI quality
|
||||
- How to monitor your AI system once it's live
|
||||
- Common mistakes everyone makes (and how to avoid them)
|
||||
|
||||
## 🏆 Course Features
|
||||
|
||||
- **Beginner-Friendly** - No prior evaluation experience needed
|
||||
- **Practical & Hands-On** - Build real evaluation systems as you learn
|
||||
- **Clear Examples** - Every concept explained with concrete examples
|
||||
- **Get Certified** - Earn your AI Evals certification
|
||||
- **Self-Paced** - Learn at your own speed
|
||||
|
||||
|
||||
## 🔗 Additional Resources
|
||||
|
||||
### 🎬 YouTube Video Series
|
||||
**[Watch the Complete Video Course](https://www.youtube.com/playlist?list=PLZoalK-hTD4VPIkRXNdSEwcTCt2QUgEPR)** - All chapters available as videos, plus 3 bonus hands-on chapters on building evals with Arize AI!
|
||||
|
||||
### 🎯 Our Maven Courses
|
||||
|
||||
**Choose the course that fits your learning journey:**
|
||||
|
||||
- **[#1 Rated Enterprise AI Course](https://maven.com/aishwarya-kiriti/genai-system-design)** - New to AI? Start here! A comprehensive program for building timeless enterprise AI systems from scratch.
|
||||
|
||||
- **[Advanced Evals Course](https://maven.com/aishwarya-kiriti/evals-problem-first)** - Already building AI? Take our newly launched course focused on systematically improving your AI products through advanced evaluation techniques.
|
||||
|
||||
*📝 Note: Use code **GITHUB15** for a limited 15% off on Maven courses (valid until August 15th, 2026)*
|
||||
|
||||
### 📱 Stay Connected
|
||||
- **Follow [Aishwarya on LinkedIn](https://www.linkedin.com/in/areganti/)** for AI evaluation insights and updates
|
||||
- **Follow [Kiriti on LinkedIn](https://www.linkedin.com/in/sai-kiriti-badam/)** for production AI learnings
|
||||
- Get the latest resources, tips, and industry updates directly in your feed!
|
||||
|
||||
## 📄 License
|
||||
|
||||
This course is released under the MIT License. Feel free to use, share, and adapt the content with attribution.
|
||||
@@ -0,0 +1,119 @@
|
||||
# Chapter 1: WTH are AI Evals?
|
||||
|
||||
|
||||
|
||||
## What Are Evals and Why Do They Suddenly Matter?
|
||||
|
||||
If you have been following recent AI updates, especially in the product space, you have probably heard the term *evals* come up repeatedly. It shows up in conversations, blog posts, product reviews, and conference talks. Everyone seems to use it casually, yet everyone also seems to mean something different by it.
|
||||
|
||||
The usefulness of evals is debated heavily, often without clarity on what people are actually referring to or where they are coming from. This lack of precision has led to a growing number of misconceptions as teams build AI solutions in the real world.
|
||||
|
||||
|
||||
|
||||
We put together this guide as a practical starting point. Think of it as a 101. The goal is to explain *why* evaluation is needed, *what* it actually refers to in practice, and *where* people commonly misunderstand it when building AI products.
|
||||
|
||||
Our hope is that by the end of this course, you are able to separate noise from signal, understand how practitioners on the ground think about evaluation, and start building evaluations using a first principles approach.
|
||||
|
||||
## The Shift That Makes Evals Unavoidable
|
||||
|
||||
|
||||
|
||||
Before getting into evaluation itself, we need to build intuition for a much larger shift. AI systems and AI products are fundamentally *non-deterministic*. We will use this term often throughout this chapter and the rest of the course, because it is the single biggest reason evals exist in the first place.
|
||||
|
||||
Most of us have spent our careers working with traditional software products. In those systems, the number of actions a user can perform is usually limited. Users click buttons, fill out forms, upload a photo, submit a request, or complete a predefined flow. In most cases, both the input and the expected output are known ahead of time. If a user uploads a photo, the system should store it. If they submit a form, the backend should validate and process it. The code is written to explicitly enforce these expectations.
|
||||
|
||||
To make sure the product behaves as intended, teams rely on unit tests and integration tests. The core assumption is simple. Given an input x, the system should reliably produce output y. If you can verify this offline, you can be reasonably confident the product will behave the same way in production.
|
||||
|
||||
The standard software lifecycle follows a familiar pattern. You build version one of the product, add tests to ensure it works, test it with different users, fix bugs, and then continue building on top of that foundation.
|
||||
|
||||
The expectation is that if you have tested x to y thoroughly before shipping, production issues will be relatively rare and manageable. When problems do occur, they are often clear outliers that can be debugged and fixed.
|
||||
|
||||
## How AI Products Break Classical Assumptions
|
||||
|
||||
|
||||
|
||||
AI products break these assumptions in two fundamental ways.
|
||||
|
||||
**First, the input space becomes effectively unbounded.** Most AI products accept text, voice, images, or video as input. Users are no longer selecting from predefined flows or filling structured forms. They are expressing intent in natural language, often ambiguously, incompletely, or in ways the product team never anticipated. You no longer control how users frame their requests. You only control how the system attempts to respond.
|
||||
|
||||
**Second, the output is no longer guaranteed.** The same input, or even small changes in phrasing, can yield different responses across runs. This is a property of the models powering these products. They are highly sensitive to context and phrasing, and they produce probabilistic outputs rather than fixed answers.
|
||||
|
||||
Traditional unit and integration tests answer a narrow question. *Did the system do exactly what we expected?* In AI products, there are two unknowns that make this question insufficient.
|
||||
|
||||
First, you do not fully know how end users will interact with your system.
|
||||
|
||||
Second, you do not have direct visibility into how the model arrives at its answers.
|
||||
|
||||
Large language models are black boxes. There is no simple pass or fail signal.
|
||||
|
||||
## So How Do Teams Build with Confidence?
|
||||
|
||||
In practice, teams start by estimating how users might interact with the system. For example, if you are building an agent to help answer customer queries for a large retail company like Amazon or Walmart, you can look at historical customer support data to understand commonly asked questions. You can then test how your system responds to those questions before launch.
|
||||
|
||||
A simple way to think about this is a table like the following:
|
||||
|
||||
| User Question | Expected Correct Answer | Agent Generated Answer |
|
||||
|--------------|------------------------|----------------------|
|
||||
| I can't seem to refund my shoes, it's been 45 days | Explain return policy and escalate | [System Response] |
|
||||
| I requested a refund a week ago and haven't gotten it | Check status and provide update | [System Response] |
|
||||
|
||||
This is often where people first encounter the idea of evaluation. It is also where confusion usually starts.
|
||||
|
||||
## Why the Word "Evals" Causes Confusion
|
||||
|
||||
|
||||
|
||||
A major source of confusion is that the term *evals* is used loosely to refer to very different things. In this course, we will use the word *evaluation* intentionally, because *evals* has become a catch-all term that hides important distinctions.
|
||||
|
||||
Broadly, there are two kinds of evaluations.
|
||||
|
||||
### Model Evaluations
|
||||
|
||||
**Model evaluations** are primarily conducted by frontier labs and research teams. Their goal is to answer a specific question. *How capable is this model in general compared to others?*
|
||||
|
||||
These evaluations rely on standardized benchmarks that test reasoning, factual recall, coding ability, or performance on academic style tasks. They are usually run on fixed datasets with predefined expected answers, and the model's outputs are scored across multiple dimensions using evaluation metrics.
|
||||
|
||||
Model evaluations are valuable. They help researchers measure progress, help infrastructure teams choose base models, and help vendors communicate improvements.
|
||||
|
||||
However, they are intentionally broad and domain agnostic. They are not designed to tell you whether a model will work well inside a specific product, workflow, or business context.
|
||||
|
||||
### AI Product Evaluations
|
||||
|
||||
**AI product evaluations** are what practitioners should care about most when building real products. Product evaluations focus on whether a system behaves acceptably in a specific domain, for a specific workflow, and for real users.
|
||||
|
||||
Real world data is far more nuanced than benchmark datasets. Domain rules, edge cases, risk tolerance, and downstream consequences matter deeply. A model that performs well in general may still fail in ways that are unacceptable for your product.
|
||||
|
||||
Even if frontier labs have done extensive model evaluations, product teams still need their own evaluation process. Model evaluations tell you what a model can do in general. Product evaluations tell you whether it should be used in your system.
|
||||
|
||||
In the rest of this course, we focus on AI product evaluations and product evaluation metrics, not model evaluations. This is the level at which product teams actually make decisions and manage risk.
|
||||
|
||||
## Clearing Up the Terminology
|
||||
|
||||
Before moving on, let us clearly define the terms we will use throughout this course. These words are often used interchangeably in conversations, which is one of the main reasons people get confused about evaluation.
|
||||
|
||||
**Evaluation** refers to the overall process of assessing how an AI system behaves. It is not a single test, score, or dashboard. Evaluation is the act of checking whether a system's outputs meet certain expectations under specific conditions. This can happen before launch, after launch, or continuously in production.
|
||||
|
||||
**A benchmark or evaluation harness** is the setup used to run evaluations in a repeatable way. This usually includes a dataset of example inputs, any required context, and a defined execution process. Benchmarks exist to ensure consistency across different evaluation runs and make results comparable.
|
||||
|
||||
**Evaluation metrics** are the dimensions along which system behavior is judged. A metric answers the question, *what does good mean in this context?* Common examples include correctness, relevance, completeness, safety, tone, or helpfulness. Metrics can be objective or subjective, but they are always context dependent.
|
||||
|
||||
The same metric can mean very different things in different domains. Take *helpfulness* as an example. In a real estate product, helpfulness might mean summarizing listings clearly, surfacing relevant comparables, or asking clarifying questions when user intent is vague. Over explaining or speculating would be harmful, even if the response sounds articulate.
|
||||
|
||||
In an insurance or healthcare workflow, helpfulness might mean knowing when *not* to answer. Escalating uncertainty, flagging missing information, or deferring to a human can be more helpful than attempting to provide a complete answer.
|
||||
|
||||
Because of this, evaluation metrics must almost always be guided by explicit *rubrics*. A rubric defines what good looks like and what failure looks like in a given context. Without rubrics, metrics like helpfulness, correctness, or safety become vague labels that different people interpret differently.
|
||||
|
||||
**Model evaluations** assess general model capability independent of any specific product.
|
||||
|
||||
**AI product evaluations** assess whether a model behaves acceptably inside a real product.
|
||||
|
||||
Throughout this course, when we talk about evaluation, we are referring to AI product evaluations unless stated otherwise. Our goal is not to measure how intelligent a model is in the abstract, but to understand whether a system is behaving well enough for the product we are trying to build.
|
||||
|
||||
## Key Takeaways
|
||||
|
||||
As you can see, *evals* is an overloaded term that means different things to different stakeholders. When someone says PMs should do evals, they often mean defining rubrics and evaluation metrics and expectations for product behavior. When someone says a model's evals look good, they usually mean benchmark scores on popular datasets. When data labeling companies talk about writing evals, they typically mean creating training datasets and annotation guidelines.
|
||||
|
||||
Understanding these distinctions is crucial for building effective evaluation systems that actually help you ship better AI products.
|
||||
|
||||
At this point, you should have a clearer mental map. In the next chapter, we will look at the different ways evaluations are implemented in practice, and the tradeoffs each approach introduces.
|
||||
|
||||
@@ -0,0 +1,155 @@
|
||||
# Chapter 2: Model vs Product Evaluations
|
||||
|
||||
|
||||
|
||||
## From Model Creation to Product Implementation
|
||||
|
||||
In the previous chapter, we established that AI evaluation is unavoidable. Now we need to understand how evaluation works across the AI ecosystem.
|
||||
|
||||
When AI companies build models, they evaluate them to understand their general capabilities. But these same models get used in thousands of different applications, from customer support chatbots to legal document analysis to medical diagnosis tools. Each application has its own requirements, constraints, and success criteria.
|
||||
|
||||
This creates a natural progression: models are evaluated for their general abilities, then they need additional evaluation when you use them for specific purposes. The first tells you what a model can do in theory. The second tells you whether it actually works for your particular use case.
|
||||
|
||||
Model creators have their own perspective on evaluation worth understanding first.
|
||||
|
||||
## Model Evaluations: Measuring General Capability
|
||||
|
||||
|
||||
|
||||
When AI companies develop models, they need to understand and communicate what their models can do. **Model evaluations** serve this purpose, measuring general capability across broad domains. Their goal is straightforward: *How capable is this model compared to others?*
|
||||
|
||||
You see these evaluations in research papers, vendor marketing materials, and leaderboards.
|
||||
|
||||
These evaluations use standardized benchmarks that test different aspects of model capability:
|
||||
|
||||
- **MMLU (Massive Multitask Language Understanding)**: Tests knowledge across 57 academic subjects from elementary math to professional law
|
||||
- **HumanEval**: Measures coding ability by testing whether models can write Python functions that pass unit tests
|
||||
- **GSM8K**: Tests grade-school level mathematical reasoning
|
||||
- **GPQA**: Tests graduate-level reasoning in physics, chemistry, and biology
|
||||
|
||||
Model evaluations serve as a competitive landscape for AI providers. When companies release new models, they publish benchmark scores to demonstrate improvements and establish market positioning. A model that scores 85% on MMLU versus 78% on the previous version signals meaningful progress to potential customers and the research community.
|
||||
|
||||
The industry benefits from this standardization in several ways. Teams can track scientific progress over time, organizations can compare and choose between different models, and everyone gets objective measures that cut through marketing claims.
|
||||
|
||||
The benchmarks are carefully designed to be objective, repeatable, and comparable across different models. Model evaluations can test both general capabilities and domain-specific knowledge, but they're designed to assess what models can do in standardized conditions, not how they'll perform in your specific business context with your particular constraints and requirements.
|
||||
|
||||
## Why Model Evaluations Don't Predict Product Success
|
||||
|
||||
|
||||
|
||||
Here's a concrete example. Suppose you're building an AI system to help insurance agents process claims. You're choosing between two models:
|
||||
|
||||
- **Model A** scores 92% on MMLU and 85% on HumanEval
|
||||
- **Model B** scores 87% on MMLU and 79% on HumanEval
|
||||
|
||||
Based on benchmark scores, Model A looks clearly superior. When you test them on actual insurance claims with your specific data, workflows, and business constraints, Model B might perform significantly better.
|
||||
|
||||
Why? Because your insurance use case has specific requirements that general benchmarks don't capture:
|
||||
|
||||
- **Domain knowledge**: Understanding insurance terminology, regulations, and claim types
|
||||
- **Risk tolerance**: The cost of approving a fraudulent claim versus denying a legitimate one
|
||||
- **Business constraints**: Processing time requirements, escalation policies, compliance needs
|
||||
- **Real-world messiness**: Incomplete forms, ambiguous language, edge cases specific to insurance
|
||||
|
||||
Model B might have seen more insurance-related data during training, or its architecture might be better suited to the structured reasoning required for claims processing. The benchmark scores can't tell you this.
|
||||
|
||||
## Real-World Context Is Far More Complicated
|
||||
|
||||
The data that powers your business lives in industry-specific silos that benchmark creators never see. Healthcare data has different patterns than financial data. Legal documents follow different structures than customer support conversations. Manufacturing quality reports contain domain knowledge that doesn't exist in academic datasets.
|
||||
|
||||
This means benchmark performance often fails to predict real-world behavior. Consider a customer support AI that scores well on standard helpfulness benchmarks. When a frustrated customer types "this is the third time I'm contacting you about my broken order and nobody seems to care," the AI needs to:
|
||||
|
||||
- Recognize the emotional context and escalation history
|
||||
- Know when to apologize versus when to escalate immediately
|
||||
- Understand your company's specific policies and capabilities
|
||||
- Balance being helpful with managing expectations appropriately
|
||||
|
||||
These nuanced requirements emerge from your specific business context, customer base, and operational constraints. They don't appear in general benchmarks, but they're critical for your product's success.
|
||||
|
||||
## AI Product Evaluations: What Actually Matters for Your Business
|
||||
|
||||
**AI product evaluations** focus on a different question: *Does this system behave acceptably for our specific use case, with our users, in our domain?*
|
||||
|
||||
Product evaluations are context-dependent by design. They test whether the AI system:
|
||||
- Handles your specific user inputs appropriately
|
||||
- Follows your business rules and constraints
|
||||
- Escalates correctly when uncertain
|
||||
- Maintains appropriate tone and style for your brand
|
||||
- Manages risk according to your tolerance levels
|
||||
|
||||
The metrics you track in product evaluation often look very different from model evaluation metrics. Instead of general correctness, you might measure:
|
||||
|
||||
- **Escalation accuracy**: Does the system correctly identify when it should hand off to a human?
|
||||
- **Policy compliance**: Does it follow your company's specific guidelines and constraints?
|
||||
- **Risk management**: How often does it make decisions you later have to reverse?
|
||||
- **User experience**: Are users able to complete their tasks efficiently?
|
||||
|
||||
## A Practical Example: Legal Document Analysis
|
||||
|
||||
Imagine you're building an AI system to help lawyers review contracts. Two different evaluation approaches would look completely different:
|
||||
|
||||
### Model Evaluation Approach
|
||||
- Test general reading comprehension on legal text
|
||||
- Measure accuracy on standardized legal reasoning benchmarks
|
||||
- Compare performance to other models on academic legal datasets
|
||||
|
||||
### Product Evaluation Approach
|
||||
- Test on your firm's actual contract types and templates
|
||||
- Measure how often it catches the specific risk patterns your lawyers care about
|
||||
- Evaluate whether it flags clauses that your legal team would want to review
|
||||
- Test escalation behavior when it encounters unusual or high-risk terms
|
||||
- Measure time savings for your lawyers while maintaining quality standards
|
||||
|
||||
The model evaluation tells you the AI can understand legal language in general. The product evaluation tells you whether it can actually help your lawyers do their job better.
|
||||
|
||||
## Focus on Product Evaluation for Builders
|
||||
|
||||
Now that we understand both approaches, it becomes clear that model evaluation alone isn't really useful for builders. While model evaluations help with initial model selection, the real work happens at the product evaluation level.
|
||||
|
||||
**Baseline capability assessment**: If a model performs poorly on relevant general benchmarks, it's unlikely to work well in your specific domain. Model evaluations can help you eliminate obviously unsuitable options.
|
||||
|
||||
**Comparative analysis**: When choosing between models with similar architectures, benchmark scores can provide useful signals about relative capability, especially when combined with product-specific testing.
|
||||
|
||||
**Progress tracking**: If you're fine-tuning or customizing a model, general benchmarks can help you verify that you're not degrading core capabilities while adding domain-specific knowledge.
|
||||
|
||||
But model evaluations should be just the starting point, not the endpoint, of your evaluation process.
|
||||
|
||||
## Building Your Product Evaluation Strategy
|
||||
|
||||
Given this distinction, how should you approach evaluation for your AI product?
|
||||
|
||||
**Start with model evaluations as a filter**. Use benchmark scores to eliminate models that lack the basic capabilities your application requires. If you need strong reasoning ability, look for models that perform well on reasoning benchmarks. If you need multilingual support, check language-specific evaluations.
|
||||
|
||||
**But invest your time in product evaluations**. This is where you'll discover whether the AI actually works for your use case. Design evaluations that test:
|
||||
- Your specific user inputs and edge cases
|
||||
- Your business constraints and requirements
|
||||
- Your risk tolerance and escalation needs
|
||||
- Your quality standards and success metrics
|
||||
|
||||
**Use real data whenever possible**. Synthetic test cases are useful for getting started, but nothing beats evaluating on actual examples from your domain with real user inputs and expected outputs.
|
||||
|
||||
**Make it an ongoing process**. Unlike model evaluations, which are typically done once during model selection, product evaluation should be continuous. User behavior evolves, business requirements change, and your AI system needs to adapt.
|
||||
|
||||
## The Evaluation Hierarchy
|
||||
|
||||
|
||||
|
||||
Think of evaluation as a hierarchy:
|
||||
|
||||
1. **Model capability**: Can this model handle the type of task I need? (Model evaluation)
|
||||
2. **Domain fit**: Does it work well with my specific data and requirements? (Basic product evaluation)
|
||||
3. **Production readiness**: Does it behave safely and reliably with real users? (Comprehensive product evaluation)
|
||||
4. **Continuous improvement**: How do I maintain and improve performance over time? (Ongoing product evaluation)
|
||||
|
||||
Most teams spend too much time on level 1 and not enough on levels 2-4. The companies that succeed with AI products flip this priority.
|
||||
|
||||
## Key Takeaways
|
||||
|
||||
Model evaluations and product evaluations serve fundamentally different purposes. Model evaluations help you understand general capability and compare different models. Product evaluations tell you whether an AI system will actually work for your business.
|
||||
|
||||
The benchmark illusion (assuming strong model evaluations guarantee product success) is one of the most common reasons AI projects fail to translate from demos to production.
|
||||
|
||||
Your evaluation strategy should use model evaluations as an initial filter but invest most of your effort in product-specific evaluation that tests real use cases, real data, and real business requirements.
|
||||
|
||||
In the next chapter, we'll dive into a systematic framework for thinking about AI system behavior that will help you design product evaluations that actually predict real-world performance.
|
||||
|
||||
@@ -0,0 +1,186 @@
|
||||
# Chapter 3: The Evaluation Framework
|
||||
|
||||
## Setting Up Evaluation for Your AI Product
|
||||
|
||||
In the previous chapters, we covered why evaluation matters and the difference between model and product evaluation. Now you understand you need to evaluate your AI products.
|
||||
|
||||
If you want to evaluate a new product you're building, where do you start?
|
||||
|
||||
We'll cover the basic concepts that help you approach evaluation strategically. This will set you up to build datasets and evaluation systems that actually help you improve your product.
|
||||
|
||||
## What You're Actually Evaluating
|
||||
|
||||
|
||||
|
||||
When you evaluate any AI system, you're looking at three things - **Input** (what goes into the system), **Expected** (what should happen), and **Actual** (what actually happens).
|
||||
|
||||
Sounds simple, but each piece is more complex than it appears.
|
||||
|
||||
### Input: Everything That Affects Your System
|
||||
|
||||
"Input" isn't just the user's question. It includes everything that influences how your system behaves - the user's actual question or request, previous conversation history and context, data your system retrieves (documents, database entries, API calls), and system configuration (prompts, parameters, business rules).
|
||||
|
||||
This matters because many evaluation problems happen when teams only test the obvious user inputs but ignore how context and configuration changes affect behavior.
|
||||
|
||||
### Expected: What Good Looks Like
|
||||
|
||||
Defining what should happen is often the hardest part. What should your system actually do?
|
||||
|
||||
Expected behavior depends on your specific requirements:
|
||||
- Accuracy of information
|
||||
- Completeness of the response
|
||||
- Appropriate tone and style
|
||||
- Safety and compliance
|
||||
- Following your business rules
|
||||
|
||||
Take a healthcare AI. When someone asks "Is this medication safe for children?", good behavior isn't just giving accurate information. It includes:
|
||||
- Noting that medical advice should come from doctors
|
||||
- Suggesting they talk to their pediatrician
|
||||
- Providing general information without specific medical recommendations
|
||||
- Escalating if the situation seems urgent
|
||||
|
||||
Defining expected behavior requires input from both technical teams and people who understand the domain and business context.
|
||||
|
||||
### Actual: What Your System Really Does
|
||||
|
||||
This is what your system produces: the response, the actions, the decisions.
|
||||
|
||||
But "actual" includes more than just the final output:
|
||||
- The content and quality of responses
|
||||
- Which tools or data sources were used
|
||||
- Reasoning and decision-making process
|
||||
- Performance metrics like response time
|
||||
|
||||
Understanding what actually happens often requires logging different parts of your system.
|
||||
|
||||
## Why Generic Metrics Don't Work
|
||||
|
||||
Once you understand these three pieces, you can see why simple metrics like "helpfulness" or "correctness" don't work for real AI products.
|
||||
|
||||
The same metric means completely different things depending on your context and requirements.
|
||||
|
||||
### Context Changes Everything
|
||||
|
||||
|
||||
|
||||
Take "helpfulness." What's helpful depends entirely on the situation:
|
||||
|
||||
**Customer service**: Helpful means solving problems quickly and escalating when needed. Explaining too much when someone just wants a refund isn't helpful.
|
||||
|
||||
**Education**: Helpful means guiding students to understanding, not just giving answers. A direct solution without explanation isn't helpful even if it's correct.
|
||||
|
||||
**Medical information**: Helpful means providing accurate general information while being clear about limitations. Being too specific about medical advice would be harmful.
|
||||
|
||||
This is why you can't just copy evaluation metrics from other applications. You need to define what quality means for your specific situation.
|
||||
|
||||
### Multiple Dimensions Matter
|
||||
|
||||
Real systems need evaluation across several specific areas. Here's why:
|
||||
|
||||
Say a customer asks "Can I return my shoes after 45 days?" and your system responds:
|
||||
|
||||
*"Unfortunately, our return policy only allows returns within 30 days of purchase. However, since you're clearly frustrated about this situation and have been a loyal customer, I understand your disappointment. While I cannot process the return myself, I recommend contacting our customer care team who may be able to offer alternative solutions or exceptions based on your purchase history and the specific circumstances of your case."*
|
||||
|
||||
You need to evaluate this across several areas:
|
||||
- **Policy accuracy**: Does it correctly state the 30-day policy?
|
||||
- **Escalation**: Does it properly refer to the right team?
|
||||
- **Tone**: Is it professional and empathetic without over-apologizing?
|
||||
- **Business risk**: Does it avoid making unauthorized promises?
|
||||
|
||||
A single "correctness" score would miss important problems. The response could be accurate about the policy but fail to escalate properly, or escalate correctly but use the wrong tone.
|
||||
|
||||
The key is finding the minimum set of areas that give you the most signal about what matters for your specific product.
|
||||
|
||||
These areas are what we call **evaluation metrics**. Some people call these "evals," but we prefer to be more precise. We use "evaluation" for the overall process and "evaluation metrics" for the specific measures you use to judge quality.
|
||||
|
||||
## Making Subjective Assessment Consistent
|
||||
|
||||
Many important quality areas require subjective judgment. Different people might evaluate the same response differently when looking at tone, appropriateness, or escalation decisions.
|
||||
|
||||
Rubrics solve this by providing explicit criteria for judgment.
|
||||
|
||||
### Building Simple Rubrics
|
||||
|
||||
A good rubric defines:
|
||||
- What counts as acceptable versus not acceptable performance
|
||||
- Specific things to look for
|
||||
- Examples of responses in each category
|
||||
- How to handle edge cases
|
||||
|
||||
For example, a rubric for "appropriate escalation" might specify:
|
||||
|
||||
**Acceptable**: Correctly identifies situations that need human intervention (policy exceptions, billing disputes, complex technical issues) and provides appropriate context when escalating
|
||||
|
||||
**Not Acceptable**: Fails to escalate when human intervention is needed, escalates unnecessarily for routine questions, or escalates without sufficient context
|
||||
|
||||
Rubrics make subjective evaluation more consistent and help different team members align on quality standards.
|
||||
|
||||
## Why Evaluation Requires Team Collaboration
|
||||
|
||||
Effective AI evaluation isn't just a technical problem. It requires collaboration between different roles:
|
||||
|
||||
**Subject matter experts** understand what good behavior looks like in the domain. They know the edge cases, risks, and nuances that technical metrics might miss.
|
||||
|
||||
**Product teams** understand user needs and business priorities. They know what trade-offs matter and how evaluation connects to user experience.
|
||||
|
||||
**Engineers** understand system capabilities and constraints. They know what's measurable, what's technically feasible, and how to implement evaluation systems.
|
||||
|
||||
This collaboration matters because evaluation decisions affect every aspect of your AI product. The metrics you choose influence what behaviors you optimize for and how you measure success.
|
||||
|
||||
## Building Team Alignment
|
||||
|
||||
One of the most valuable outcomes of systematic evaluation is getting everyone aligned on quality. When engineering, product, and domain expert teams can look at the same examples and agree on good versus poor performance, you can move much faster.
|
||||
|
||||
This process often reveals hidden assumptions and disagreements. The product team might prioritize user satisfaction while the legal team prioritizes risk management. Working through evaluation examples helps surface and resolve these tensions before they affect the product.
|
||||
|
||||
## How Do You Identify the Right Dimensions?
|
||||
|
||||
So far we've talked about why you need specific evaluation metrics and why collaboration matters. But how do you actually figure out which dimensions to focus on?
|
||||
|
||||
The process usually starts with understanding your specific failure modes. What could go wrong with your AI system that would be unacceptable for your users or business? What behaviors would make you pull the system offline immediately?
|
||||
|
||||
Different stakeholders will have different answers:
|
||||
- **Domain experts** worry about accuracy, compliance, and safety risks specific to their field
|
||||
- **Product teams** focus on user experience, completion rates, and satisfaction
|
||||
- **Business stakeholders** care about liability, brand risk, and operational costs
|
||||
|
||||
The key is starting with these concerns and translating them into observable, measurable behaviors. Instead of "the system should be safe," you might define "the system should escalate medical questions to qualified professionals" or "the system should not provide financial advice without appropriate disclaimers."
|
||||
|
||||
You also need to consider your specific user context. A chatbot for customer service has different quality requirements than one for technical support or educational tutoring. The same AI technology needs completely different evaluation approaches depending on who's using it and for what purpose.
|
||||
|
||||
## The Pre-Deployment Validation Process
|
||||
|
||||
The approach we're describing helps you validate your AI system before you put it in front of real users. This pre-deployment validation is essential because it's much easier to catch and fix issues in controlled testing than after users start depending on your system.
|
||||
|
||||
Think of this as building confidence in your system's behavior before the stakes get high. When you're working with reference datasets and controlled examples, you can iterate quickly, test edge cases thoroughly, and refine your approach without worrying about user impact. You can have domain experts review outputs carefully, engineers can debug issues systematically, and product teams can ensure the behavior aligns with user needs.
|
||||
|
||||
This validation process involves several key activities:
|
||||
|
||||
**Building comprehensive test scenarios**: You'll create examples that represent the full range of situations your system needs to handle, from common user requests to edge cases that could cause problems.
|
||||
|
||||
**Establishing clear quality criteria**: You'll work with stakeholders to define exactly what good behavior looks like in your specific context, creating rubrics that everyone can agree on.
|
||||
|
||||
**Testing system behavior systematically**: You'll run your AI system against your test scenarios and evaluate whether it meets your quality standards across different dimensions.
|
||||
|
||||
**Iterating based on findings**: When you discover issues, you'll fix them and re-test to ensure the problems are resolved without creating new ones.
|
||||
|
||||
Once you deploy and real users start interacting with your AI, you'll need to adapt these same concepts for ongoing monitoring. Real-world conditions introduce new challenges like unpredictable user behavior, scale issues, and evolving requirements that require different approaches while building on the same evaluation foundation.
|
||||
|
||||
## Where This Leads
|
||||
|
||||
Understanding what goes into your system, what should happen, and what actually happens helps you see why AI evaluation is more complex than traditional software testing. The challenge isn't just technical - it's about getting alignment across different perspectives on quality.
|
||||
|
||||
Generic metrics like "helpfulness" mean different things in different contexts. Effective evaluation requires specific metrics that reflect your domain, users, and business requirements.
|
||||
|
||||
AI evaluation is inherently collaborative. Subject matter experts, product teams, and engineers each bring essential perspectives to defining what good performance looks like.
|
||||
|
||||
But this raises an important question: how do you actually come up with all these metrics? How do you know which ones matter most for your specific situation? How do you balance different team perspectives to create evaluation criteria everyone can agree on?
|
||||
|
||||
**Want to go deeper?** Choose the course that fits your journey:
|
||||
- **New to AI?** Check out our **[#1 rated Enterprise AI Course on Maven](https://maven.com/aishwarya-kiriti/genai-system-design)** for comprehensive guidance on building production-ready AI systems from scratch.
|
||||
- **Already building AI?** Take our newly launched **[Advanced Evals course](https://maven.com/aishwarya-kiriti/evals-problem-first)** for systematically improving your AI products through advanced evaluation techniques.
|
||||
|
||||
*📝 Note: Use code **GITHUB15** for 15% off on Maven courses (valid until January 15th, 2025)*
|
||||
|
||||
In the next chapter, we'll talk about building a reference dataset so you can understand how to apply this framework and improve your system once you've built a version of your product. We'll cover how you could set this up in a more systematic way.
|
||||
|
||||
@@ -0,0 +1,198 @@
|
||||
# Chapter 4: Building Reference Datasets
|
||||
|
||||
|
||||
|
||||
## Getting Started with Systematic Evaluation
|
||||
|
||||
In the previous chapter, we covered why you need specific evaluation metrics and how different stakeholders bring different perspectives to defining quality. Now let's talk about how to set this up systematically.
|
||||
|
||||
You've built a version of your AI product and you want to evaluate it properly before putting it in front of real users. Where do you start?
|
||||
|
||||
The most practical approach is building a reference dataset. Think of this as a small, carefully chosen set of examples that represent the scenarios you care most about. It's not meant to be comprehensive - it's meant to be useful for validating your system's behavior in a controlled environment before deployment.
|
||||
|
||||
**A note on system complexity**: In this guide, we'll focus on single-step AI interactions (where the user asks something and the system responds). Many complex AI systems involve multiple steps like calling tools, multi-turn conversations, or reasoning chains, but the core ideas we'll cover can be translated to those more complex scenarios as well.
|
||||
|
||||
## What Is a Reference Dataset?
|
||||
|
||||
A reference dataset is your first concrete representation of how the system should behave when deployed. It's a collection of realistic inputs paired with what you expect the system to do in those situations.
|
||||
|
||||
The key word here is "realistic." These aren't made-up test cases. They're examples that reflect how real users will actually interact with your system, including the messy, ambiguous, and edge-case scenarios that always happen in production.
|
||||
|
||||
Each example in your dataset typically includes:
|
||||
- **Input**: A realistic user request or scenario
|
||||
- **Expected output**: What the system should do (written in plain language)
|
||||
- **Context**: Any additional information the system needs
|
||||
|
||||
The expected output doesn't have to be a perfect response. It can be a description of the right behavior, like "escalate to human agent" or "ask for clarification about the user's budget range."
|
||||
|
||||
## Why Start Small and Specific
|
||||
|
||||
|
||||
|
||||
Teams often make the mistake of trying to build comprehensive test coverage from day one. This doesn't work well for AI systems.
|
||||
|
||||
Instead, start with a small set of examples that represent scenarios you absolutely cannot get wrong. These are usually:
|
||||
- High-risk situations where failure would be unacceptable
|
||||
- Common user workflows that need to work smoothly
|
||||
- Edge cases that reveal important system limitations
|
||||
- Examples that expose different evaluation dimensions you care about
|
||||
|
||||
For a customer support AI, this might include:
|
||||
- A billing dispute that requires human escalation
|
||||
- A simple return request that should be handled automatically
|
||||
- An angry customer message that needs careful tone handling
|
||||
- A request that's outside your company's service scope
|
||||
|
||||
Starting small lets you focus on quality over quantity. It's better to have 20 well-chosen examples with clear expected behaviors than 200 generic test cases.
|
||||
|
||||
## Step 1: Generate Your Initial Examples
|
||||
|
||||
The best source for examples is usually existing data from your domain. If you have historical customer support tickets, user queries, or domain-specific scenarios, start there.
|
||||
|
||||
If you don't have existing data, this is where collaboration becomes essential:
|
||||
|
||||
**Subject matter experts** should contribute the majority of initial examples. They know the edge cases, the high-risk scenarios, and the subtle requirements that technical teams might miss. Don't rely on engineers to generate domain-specific examples because they'll miss important nuances.
|
||||
|
||||
**Product teams** can contribute examples based on user research, feature requirements, and common user journeys they've observed.
|
||||
|
||||
**Engineers** can help identify technical edge cases and system boundary conditions.
|
||||
|
||||
We recommend avoiding AI-generated synthetic examples at this stage. AI can help with formatting or expanding on human-written examples, but it tends to create shallow scenarios that miss real-world complexity.
|
||||
|
||||
### Example: Customer Support Dataset
|
||||
|
||||
Here's what an initial reference dataset might look like for a customer support system:
|
||||
|
||||
| Input | Expected Behavior |
|
||||
|-------|-------------------|
|
||||
| "I want to return my shoes but I lost the receipt" | Ask for order number or email, explain receipt alternatives, process if sufficient info available |
|
||||
| "Your service is terrible and I'm switching to a competitor" | Acknowledge frustration, apologize professionally, escalate to retention team |
|
||||
| "How do I track my order?" | Ask for order number, provide tracking information, explain delivery timeline |
|
||||
| "I was charged twice for the same order" | Apologize, escalate immediately to billing team with all available details |
|
||||
|
||||
Notice these examples cover different scenarios (returns, complaints, tracking, billing) and different required behaviors (self-service, escalation, information gathering).
|
||||
|
||||
## Step 2: Run Your System on the Examples
|
||||
|
||||
Once you have your initial set of examples, run your AI system on these inputs. Keep the system simple at this stage. Don't add complexity before you understand baseline behavior.
|
||||
|
||||
This step often reveals immediate problems with your system design, prompts, or configuration. That's valuable information.
|
||||
|
||||
Document both what the system outputs and any intermediate steps (which tools it used, what data it retrieved, how confident it was). This helps you understand not just what went wrong, but why.
|
||||
|
||||
## Step 3: Evaluate Alignment with Domain Experts
|
||||
|
||||
Now you have three columns: input, expected behavior, and actual system output. The next step is getting domain experts to evaluate the alignment.
|
||||
|
||||
Keep this process simple. Ask domain experts to review each example and answer: "Was this response satisfactory? If not, why not?"
|
||||
|
||||
Avoid asking for numerical scores at this stage. Scores are harder to interpret and don't give you actionable feedback. Simple yes/no judgments with explanations are more useful.
|
||||
|
||||
**Make it easy for domain experts to participate**. Don't ask them to learn complex rubrics or scoring systems. Frame questions in their language and focus on their expertise.
|
||||
|
||||
### Example Evaluation Session
|
||||
|
||||
Continuing with our customer support example:
|
||||
|
||||
| Input | System Output | Expected | Satisfactory | If not, why? |
|
||||
|-------|---------------|----------|--------------|--------------|
|
||||
| "Lost receipt for shoe return" | "I can help you process a return. Please provide your order confirmation email." | Ask for order number or email, explain alternatives | Yes | Handled appropriately |
|
||||
| "Service is terrible, switching" | "I understand you're frustrated. Let me see how I can help resolve your issue." | Acknowledge, apologize, escalate to retention | No | Didn't escalate to retention team |
|
||||
|
||||
This gives you specific, actionable feedback about where your system is failing.
|
||||
|
||||
## Step 4: Identify Error Patterns
|
||||
|
||||
Now bring the engineering perspective back in. Look at the annotations from domain experts and identify patterns in the failures.
|
||||
|
||||
Many issues that look different on the surface come from the same root cause. The goal is clustering errors into a small number of underlying problems that you can actually fix.
|
||||
|
||||
Add two more columns to your analysis:
|
||||
- **Error category**: What type of failure is this?
|
||||
- **Potential cause**: Why might this be happening?
|
||||
|
||||
Common error patterns include:
|
||||
- **Missing context**: System doesn't have access to information it needs
|
||||
- **Prompt issues**: Instructions aren't clear or specific enough
|
||||
- **Business rule failures**: System doesn't follow domain-specific policies
|
||||
- **Escalation problems**: Doesn't recognize when human intervention is needed
|
||||
|
||||
### Example Error Analysis
|
||||
|
||||
| Input | System Output | Satisfactory | Error Category | Potential Cause |
|
||||
|-------|---------------|--------------|----------------|-----------------|
|
||||
| "Service terrible, switching" | Generic help offer | No | Missing escalation | No escalation logic for retention cases |
|
||||
| "Charged twice" | "Let me help with that" | No | Missing urgency | Billing issues not flagged as high-priority |
|
||||
|
||||
This helps you prioritize fixes and understand whether issues are implementation problems or deeper design issues.
|
||||
|
||||
## Step 5: Decide Which Metrics You Need
|
||||
|
||||
Here's a key insight: if an issue can be fixed once and is unlikely to return, fix it and move on. If an issue represents a behavior that can reappear in different forms, you need an ongoing metric to track it.
|
||||
|
||||
For example:
|
||||
- A missing instruction in a prompt is usually a one-time fix
|
||||
- Appropriate escalation behavior is an ongoing concern that needs monitoring
|
||||
|
||||
Create metrics for recurring risks, not one-off bugs.
|
||||
|
||||
**Be ruthless about what you measure**. You want the minimum number of metrics that give you the maximum amount of signal. If there are issues in your use case that you're not really worried about (small things that don't significantly impact your users or business), don't add a metric for them.
|
||||
|
||||
Only create metrics for behaviors you actually care about and can take action on. If you wouldn't change your system based on a metric, don't track it.
|
||||
|
||||
Based on your error analysis, identify 2-4 key behaviors that need ongoing measurement. These become your evaluation metrics. More than that becomes difficult to manage and act upon effectively.
|
||||
|
||||
For our customer support example, you might end up with:
|
||||
- **Escalation accuracy**: Does the system correctly identify when human intervention is needed?
|
||||
- **Information gathering**: Does it ask for the right information to resolve requests?
|
||||
- **Tone appropriateness**: Does it match the professional, helpful brand voice?
|
||||
|
||||
**Focus on what matters, not implementation**. At this stage, don't worry about how you'll measure these behaviors. Just identify which behaviors are most important for your use case. We'll cover implementation approaches in the next chapter.
|
||||
|
||||
For now, examples of metrics you might track include:
|
||||
- Response time stays under acceptable limits
|
||||
- Required legal disclaimers appear in financial advice responses
|
||||
- Billing-related queries get properly flagged for escalation
|
||||
- System outputs maintain valid structure for downstream processing
|
||||
- Appropriate tone and empathy in customer interactions
|
||||
- Accurate assessment of query complexity for escalation decisions
|
||||
- Relevant information gathering without being repetitive
|
||||
|
||||
What matters most is identifying which failure modes are critical for your specific system and user needs.
|
||||
|
||||
## Step 6: Iterate and Expand
|
||||
|
||||
Your reference dataset isn't static. As you fix issues and learn more about system behavior, add new examples that represent:
|
||||
- Edge cases discovered in production
|
||||
- New failure modes that emerge
|
||||
- Additional scenarios your system needs to handle
|
||||
|
||||
The dataset grows into a record of hard-won understanding about what good behavior looks like in your domain.
|
||||
|
||||
## Common Pitfalls to Avoid
|
||||
|
||||
**Don't make it too big too fast**: Start with 10-20 high-quality examples rather than 100 mediocre ones.
|
||||
|
||||
**Don't rely entirely on synthetic data**: AI-generated examples often miss real-world complexity and edge cases.
|
||||
|
||||
**Don't skip domain expert involvement**: Technical teams alone cannot define what good behavior looks like in specialized domains.
|
||||
|
||||
**Don't create metrics for every issue**: Focus on recurring risks that need ongoing monitoring.
|
||||
|
||||
**Don't make rubrics too complex**: Simple "acceptable/not acceptable" categories work better than elaborate scoring systems.
|
||||
|
||||
## What You End Up With
|
||||
|
||||
After following this process, you'll have:
|
||||
- A reference dataset that represents scenarios you care about
|
||||
- Clear definitions of what good behavior looks like
|
||||
- Specific metrics that track the most important behavioral dimensions
|
||||
- Rubrics that make subjective evaluation consistent
|
||||
- A process for expanding and refining your evaluation over time
|
||||
|
||||
This becomes the foundation for ongoing evaluation and improvement. Every time you make changes to your system, you can run it against your reference dataset to check for regressions. Every time you discover new edge cases in production, you can add them to improve your evaluation coverage.
|
||||
|
||||
The goal isn't perfect evaluation - it's systematic improvement. Your reference dataset helps you move from vague concerns about system behavior to concrete, measurable criteria you can act on.
|
||||
|
||||
In this chapter, we've identified what metrics are important to track based on your specific failure modes and business requirements. In the next chapter, we'll talk about how to implement these metrics, from simple code-based checks to more sophisticated evaluation approaches.
|
||||
|
||||
@@ -0,0 +1,242 @@
|
||||
# Chapter 5: Implementing Evaluation Metrics
|
||||
|
||||
|
||||
|
||||
## From What to How
|
||||
|
||||
In the previous chapter, we walked through building reference datasets and identifying which metrics matter for your system. You now have a clear list of behaviors you want to track, such as escalation accuracy, response time, or tone appropriateness.
|
||||
|
||||
Now comes the practical question: how do you actually measure these behaviors?
|
||||
|
||||
This chapter covers the different approaches you can use to implement your metrics. We'll explore when each approach works well, their trade-offs, and how to choose the right mix for your specific situation.
|
||||
|
||||
## Three Ways to Measure AI Behavior
|
||||
|
||||
|
||||
|
||||
There are three main approaches to implementing evaluation metrics:
|
||||
|
||||
**Human evaluation**: Having people assess AI system behavior based on their expertise and judgment
|
||||
**Code-based metrics**: Deterministic checks written in code that look for specific patterns or properties
|
||||
**LLM judges**: Using one model to evaluate another model's behavior
|
||||
|
||||
Each approach has strengths and weaknesses. Most effective evaluation systems use a combination of multiple approaches.
|
||||
|
||||
## Human Evaluation: The Gold Standard
|
||||
|
||||
Human evaluation is exactly what we did when building the reference dataset in the previous chapter. You show examples to domain experts, product managers, or other stakeholders and ask them to judge whether the AI system's behavior was acceptable.
|
||||
|
||||
This approach has major advantages:
|
||||
- **Nuanced judgment**: Humans can assess complex, subjective qualities like appropriateness, empathy, and contextual correctness
|
||||
- **Domain expertise**: Subject matter experts understand subtleties that automated systems miss
|
||||
- **Flexibility**: Humans can adapt their evaluation criteria on the fly when they encounter edge cases
|
||||
- **Ground truth**: Human judgment often serves as the standard that other metrics try to approximate
|
||||
|
||||
### Why Human Evaluation Doesn't Scale
|
||||
|
||||
The problem with human evaluation is obvious: it's slow and expensive. If you had to have a human evaluate every single conversation your AI system has in production, you'd need an army of evaluators working around the clock.
|
||||
|
||||
Imagine a customer support AI that handles 10,000 interactions per day. Having domain experts review each one would be impractical and cost-prohibitive. Even sampling 1% would require evaluating 100 interactions daily.
|
||||
|
||||
This is why we need the other automated approaches. They're attempts to capture human-like judgment at scale. The goal is finding automated methods that correlate well with human evaluation while being fast and cost-effective enough to run in production.
|
||||
|
||||
### When to Use Human Evaluation
|
||||
|
||||
Human evaluation still has important roles:
|
||||
- **Calibrating automated metrics**: Use human judgment to test whether your LLM judges or other metrics align with expert assessment
|
||||
- **Edge case analysis**: When automated metrics flag something as problematic, humans can investigate whether it's a real issue
|
||||
- **Periodic sampling**: Regularly evaluate a small sample of interactions to ensure your automated systems are working correctly
|
||||
- **High-stakes decisions**: For critical interactions or when the cost of errors is very high
|
||||
|
||||
## Code-Based Metrics: When Rules Work
|
||||
|
||||
Code-based metrics are deterministic checks you can implement with regular programming. They're fast, reliable, and easy to understand.
|
||||
|
||||
These work well when you can define success clearly and objectively:
|
||||
|
||||
**Structure validation**: Check if the response contains required fields, follows JSON format, or includes mandatory disclaimers
|
||||
|
||||
**Performance metrics**: Measure response time, token count, or API call frequency
|
||||
|
||||
**Content detection**: Verify specific phrases appear (like "please consult your doctor" in medical responses) or don't appear (like specific banned words)
|
||||
|
||||
**Classification flags**: Check if the system correctly tagged a query as "billing," "technical support," or "escalation needed"
|
||||
|
||||
### Example: Structured Output Validation
|
||||
|
||||
Say you're building an AI system that helps sales teams qualify leads. The system needs to extract key information from customer conversations and output it in a structured format for the CRM system.
|
||||
|
||||
Your AI system should output JSON like this:
|
||||
```json
|
||||
{
|
||||
"customer_name": "John Smith",
|
||||
"company": "TechCorp",
|
||||
"budget_range": "50000-100000",
|
||||
"timeline": "Q2 2024",
|
||||
"decision_maker": true,
|
||||
"contact_email": "john@techcorp.com"
|
||||
}
|
||||
```
|
||||
|
||||
A code-based metric can easily verify:
|
||||
- Is the output valid JSON?
|
||||
- Are all required fields present (customer_name, company, budget_range)?
|
||||
- Is the budget_range in the expected format?
|
||||
- Is the decision_maker field a boolean?
|
||||
- Is the contact_email field a valid email format?
|
||||
|
||||
This kind of check is perfect for code-based metrics because the requirements are objective and well-defined.
|
||||
|
||||
### When Code-Based Metrics Fall Short
|
||||
|
||||
Code-based metrics struggle with subjective qualities like tone, appropriateness, or nuanced decision-making. You can't easily write code to detect whether a customer service response shows appropriate empathy or whether an escalation decision was justified.
|
||||
|
||||
They also miss nuanced meaning and context. A response might pass all the structural checks but still be unhelpful, inappropriate, or incorrect in ways that matter to users.
|
||||
|
||||
## LLM Judges: Automating Human-Like Evaluation
|
||||
|
||||
LLM judges use one model to evaluate another model's behavior. The idea is to replace the manual human evaluation process we used when building reference datasets with an automated system that can make similar judgments at scale.
|
||||
|
||||
Instead of having domain experts review every response, you give an LLM the same criteria and ask it to assess whether the behavior was appropriate. This lets you evaluate thousands of interactions with the same standards a human expert would apply.
|
||||
|
||||
This approach works for subjective or complex evaluations:
|
||||
|
||||
**Tone assessment**: Is the response professional and empathetic?
|
||||
**Escalation decisions**: Should this query have been escalated to a human?
|
||||
**Reasoning quality**: Does the explanation make logical sense?
|
||||
**Safety evaluation**: Does the response avoid harmful content?
|
||||
|
||||
### Example: Customer Service Tone
|
||||
|
||||
For evaluating whether a customer service response shows appropriate empathy, an LLM judge can assess nuanced qualities like tone, professionalism, and contextual appropriateness that would be difficult to capture with code-based metrics.
|
||||
|
||||
Whether you're using LLM judges or human evaluators, you need clear criteria for what constitutes good and poor performance. This is where rubrics become essential.
|
||||
|
||||
A good rubric defines:
|
||||
- **Acceptable performance**: Specific characteristics of good behavior
|
||||
- **Not acceptable performance**: Clear failure criteria
|
||||
- **Examples**: Concrete instances of each category
|
||||
- **Edge case guidance**: How to handle ambiguous situations
|
||||
|
||||
### Example: From Error Pattern to LLM Judge Rubric
|
||||
|
||||
Here's how you'd build an LLM judge based on the customer support example from Chapter 4.
|
||||
|
||||
**The Problem**: In your reference dataset evaluation, you found that when customers expressed frustration and mentioned switching providers (like "Service is terrible, switching"), your system gave generic help offers instead of escalating to the retention team.
|
||||
|
||||
**The Pattern**: Analysis revealed this was part of a broader "escalation accuracy" issue. The system wasn't recognizing when situations required specialized human intervention.
|
||||
|
||||
**The Metric**: You decided to track "escalation accuracy" as an ongoing metric since this behavior could reappear in many different forms.
|
||||
|
||||
**The LLM Judge Rubric**:
|
||||
|
||||
**Acceptable**:
|
||||
- Correctly identifies customer retention situations (mentions switching, canceling, competitor comparisons, dissatisfaction with service)
|
||||
- Escalates billing disputes over significant amounts ($100+)
|
||||
- Recognizes technical issues beyond basic troubleshooting scope
|
||||
- Provides relevant context when escalating (customer sentiment, issue details, urgency level)
|
||||
|
||||
**Not Acceptable**:
|
||||
- Misses clear retention signals and attempts generic problem-solving
|
||||
- Fails to escalate high-value billing disputes
|
||||
- Tries to handle complex technical issues that require specialized expertise
|
||||
- Escalates routine questions that could be resolved automatically
|
||||
- Escalates without sufficient context for the human agent
|
||||
|
||||
**Examples**:
|
||||
- **Acceptable**: "Your service is terrible and I'm switching to CompetitorX" → Escalates to retention team noting customer dissatisfaction and competitor mention
|
||||
- **Not Acceptable**: "I want to cancel my subscription to save money" → Provides generic retention offer instead of escalating to retention specialists
|
||||
- **Acceptable**: "I was charged $500 for services I never ordered" → Escalates to billing team with charge amount and dispute details
|
||||
- **Not Acceptable**: "How do I reset my password?" → Escalates to technical support instead of providing standard reset instructions
|
||||
|
||||
This rubric now gives you a measurable way to track the escalation behavior that was failing in your reference dataset, turning the discovered error pattern into an ongoing monitoring capability.
|
||||
|
||||
**LLM Judge Prompt Example**:
|
||||
|
||||
Here's how you might structure a prompt for an LLM judge using this rubric:
|
||||
|
||||
```
|
||||
You are evaluating customer service responses for escalation accuracy. Your job is to determine if the AI system correctly identified when human intervention was needed.
|
||||
|
||||
EVALUATION CRITERIA:
|
||||
|
||||
Acceptable Performance:
|
||||
- Correctly identifies customer retention situations (mentions switching, canceling, competitors)
|
||||
- Escalates billing disputes over $100
|
||||
- Recognizes complex technical issues beyond basic troubleshooting
|
||||
- Provides relevant context when escalating (sentiment, details, urgency)
|
||||
|
||||
Not Acceptable Performance:
|
||||
- Misses clear retention signals and tries generic problem-solving
|
||||
- Fails to escalate high-value billing disputes
|
||||
- Attempts to handle complex technical issues requiring specialized expertise
|
||||
- Escalates routine questions that could be resolved automatically
|
||||
- Escalates without sufficient context for human agents
|
||||
|
||||
EXAMPLES:
|
||||
- Customer: "Your service is terrible and I'm switching to CompetitorX"
|
||||
Acceptable Response: Escalates to retention team noting dissatisfaction and competitor mention
|
||||
Not Acceptable: Offers generic troubleshooting help
|
||||
|
||||
- Customer: "I was charged $500 for services I never ordered"
|
||||
Acceptable Response: Escalates to billing team with charge details
|
||||
Not Acceptable: Asks customer to verify their account information
|
||||
|
||||
TASK:
|
||||
Review the customer input and AI response below. Determine if the escalation decision was:
|
||||
- Acceptable
|
||||
- Not Acceptable
|
||||
|
||||
Provide a brief explanation for your judgment.
|
||||
|
||||
Customer Input: [INPUT]
|
||||
AI Response: [RESPONSE]
|
||||
|
||||
Your Evaluation:
|
||||
```
|
||||
|
||||
This prompt gives the LLM judge the same detailed criteria that human evaluators would use, allowing it to make consistent assessments at scale.
|
||||
|
||||
## A Note on LLM Judge Calibration
|
||||
|
||||
While we've shown you how to build an LLM judge with clear criteria and rubrics, remember that in practice, calibrating an LLM judge is a much longer and more data-driven process than what we've demonstrated here. LLM judges are powerful but challenging to implement well. They can be inconsistent, biased, or misaligned with human judgment. They're also more expensive and slower than other approaches.
|
||||
|
||||
Effective LLM judge calibration requires extensive testing against human judgment across hundreds of examples, not just a few. You need to systematically identify where the LLM judge disagrees with human evaluators, understand why those disagreements happen, and iteratively refine your prompts and criteria until alignment is acceptable for your specific use case.
|
||||
|
||||
**Calibration is Essential**
|
||||
|
||||
The biggest challenge with LLM judges is ensuring they actually align with human judgment. Just because you write detailed criteria doesn't mean the LLM will interpret them the same way a human expert would. In fact, if not calibrated properly they can add more problems to your system because they add another layer of non-determinism.
|
||||
|
||||
You need to test your LLM judge against human evaluations:
|
||||
- Have humans evaluate a sample of examples using your rubric
|
||||
- Run your LLM judge on the same examples
|
||||
- Compare the results to see where they agree and disagree
|
||||
- Refine your prompt and criteria based on the differences
|
||||
- Repeat until alignment is acceptable
|
||||
|
||||
We leave you here since this is a 101 course, but building reliable LLM judges can be a course on its own. Remember to dig deeper to learn these concepts well.
|
||||
|
||||
**Want to go deeper?** Choose the course that fits your journey:
|
||||
- **New to AI?** Check out our **[#1 rated Enterprise AI Course on Maven](https://maven.com/aishwarya-kiriti/genai-system-design)** for comprehensive guidance on building production-ready AI systems from scratch.
|
||||
- **Already building AI?** Take our newly launched **[Advanced Evals course](https://maven.com/aishwarya-kiriti/evals-problem-first)** for systematically improving your AI products through advanced evaluation techniques.
|
||||
|
||||
*📝 Note: Use code **GITHUB15** for 15% off on Maven courses (valid until January 15th, 2025)*
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
## What You End Up With
|
||||
|
||||
After implementing your metrics, you'll have a measurement system that can:
|
||||
- Automatically track the behaviors you care about most
|
||||
- Run consistently across different examples
|
||||
- Provide actionable feedback for system improvement
|
||||
- Scale with your evaluation needs
|
||||
|
||||
This system becomes the foundation for continuous improvement. You can run it on new examples, track performance over time, and identify areas where your AI system needs work. However, you probably have questions on how to deploy these metrics in a production setup. Should you be running them on all your production inputs and outputs or just samples? Are these metrics enough or should you keep reinventing? We'll talk about all this in the next chapter.
|
||||
|
||||
|
||||
|
||||
In the next chapter, we'll explore how to use these metrics in an improvement loop that helps your system get better over time.
|
||||
|
||||
@@ -0,0 +1,104 @@
|
||||
# Chapter 6: Production Deployment and Real User Behavior
|
||||
|
||||
|
||||
|
||||
## From Lab to Real World
|
||||
|
||||
So far in this course, we've covered the essential building blocks of AI evaluation. We started by understanding why evaluation matters for AI systems and distinguished between model evaluations and product evaluations. We explored the conceptual foundation of input, expected, and actual behavior. We walked through building reference datasets to systematically identify what matters for your specific use case. And we covered three approaches to implementing evaluation metrics: human evaluation, code-based metrics, and LLM judges.
|
||||
|
||||
At this point, you have a solid evaluation framework. You've built reference datasets that represent important scenarios for your system. You've identified the key metrics that track behaviors you actually care about. You've implemented ways to measure those behaviors, whether through human judgment, deterministic code checks, or calibrated LLM judges.
|
||||
|
||||
But here's where things get interesting and more complex.
|
||||
|
||||
Everything we've discussed so far happens in controlled conditions. You're testing with carefully chosen examples, evaluating against clear expected behaviors, and working with stakeholders who understand your system's goals. You're essentially working in a lab environment where you control the inputs and can predict most of the scenarios.
|
||||
|
||||
Production is different. When real users start interacting with your AI system, several things happen that change the evaluation game entirely.
|
||||
|
||||
## The Reality of Real Users
|
||||
|
||||
|
||||
|
||||
Real users don't behave like your reference datasets. They don't ask questions the way you expect, they don't provide complete information, and they often try to use your system for purposes you never intended.
|
||||
|
||||
**Users bring unexpected context**: Your customer service AI might be designed for product questions, but users will ask about competitor products, share personal stories, or try to use it for technical support issues outside your scope.
|
||||
|
||||
**Users test edge cases you missed**: No matter how thorough your reference dataset, real users will find scenarios you didn't anticipate. They'll phrase requests in ways that confuse your system, combine multiple intents in a single message, or operate under assumptions that don't match your business model.
|
||||
|
||||
**User evolution**: As users get comfortable with your system, their behavior evolves. They develop new ways to phrase requests, discover shortcuts, and use your system in increasingly sophisticated ways. Think about how people use ChatGPT today compared to when it first launched - the questions become more complex, the use cases expand, and the expectations change. This natural evolution means the distribution of inputs your system receives will shift over time.
|
||||
|
||||
**Volume changes everything**: When you test with 50 carefully chosen examples, you can review each interaction manually. When your system handles 10,000 interactions per day, you need fundamentally different approaches to understanding what's happening.
|
||||
|
||||
## The Scale Challenge
|
||||
|
||||
In controlled testing, you can review every example and understand every failure. In production, this becomes impossible.
|
||||
|
||||
Consider a customer support AI that handles 5,000 conversations daily. Even if 95% of interactions go perfectly, you still have 250 potentially problematic conversations every day. Manual review of each one would require dedicated staff just for evaluation.
|
||||
|
||||
The challenge isn't just volume - it's also about detection. In your reference dataset, you know which examples should pass or fail your evaluation metrics. In production, you don't know ahead of time which conversations will be problematic.
|
||||
|
||||
This shifts the evaluation question from "How did we do on this specific set of examples?" to "How are we doing overall, and where should we focus our attention?"
|
||||
|
||||
## From Evaluation to Monitoring
|
||||
|
||||
|
||||
|
||||
Moving to production fundamentally changes your relationship with evaluation. During development, evaluation was about validation (testing whether your system works as intended). In production, evaluation becomes monitoring (continuously checking whether your system continues to work well as conditions change).
|
||||
|
||||
This affects how you think about measurement, response, and improvement:
|
||||
|
||||
**Evaluation builds confidence before deployment**: You test thoroughly to gain confidence that your system is ready for users.
|
||||
|
||||
**Monitoring maintains quality during deployment**: You track performance to catch problems early and guide improvements.
|
||||
|
||||
|
||||
|
||||
**The flywheel of improvement**: Good production monitoring feeds back into your evaluation process. Issues discovered in production become new test cases in your reference datasets. Patterns identified in monitoring inform better pre-deployment validation. The two work together in a continuous improvement cycle.
|
||||
|
||||
This creates a natural progression: strong evaluation gives you confidence to deploy, effective monitoring helps you improve, and improved systems perform better in evaluation.
|
||||
|
||||
## Four Core Challenges in Production
|
||||
|
||||
|
||||
|
||||
When you move from controlled evaluation to production monitoring, four key challenges emerge that require careful planning:
|
||||
|
||||
### 1. Log Filtering
|
||||
|
||||
With thousands of events happening daily, you can't manually review everything. You need systematic approaches to identify which logs deserve attention. This means developing filtering and sampling strategies that help you focus on the data most likely to reveal problems or insights.
|
||||
|
||||
### 2. Metric Selection
|
||||
|
||||
Remember that evaluation metrics aren't free. LLM judges cost money to run, human evaluation requires time and expertise, and even code-based metrics might not always be as trivial or cheap as running unit tests in traditional software setups. At scale, these costs add up quickly. You need to be strategic about which metrics provide the most valuable insights relative to their cost.
|
||||
|
||||
### 3. Online vs. Offline Evaluation
|
||||
|
||||
This is where we introduce an important distinction that will shape your production monitoring strategy:
|
||||
|
||||
**Online evaluation** happens in real-time as users interact with your system. These metrics run immediately and can trigger alerts or interventions. For example, you might have an online safety filter that flags inappropriate content before it reaches users.
|
||||
|
||||
**Offline evaluation** happens after the fact, often in batch processes. These metrics analyze interactions that already occurred to identify trends, assess quality over time, or conduct detailed investigations. For example, you might run expensive LLM judges overnight to assess the previous day's customer service interactions.
|
||||
|
||||
The choice between online and offline evaluation affects cost, complexity, and responsiveness. Online evaluation gives you immediate feedback but needs to be fast and lightweight. Offline evaluation can be more thorough and sophisticated but only helps you improve future interactions.
|
||||
|
||||
### 4. Emerging Issue Discovery
|
||||
|
||||
Despite doing all of this systematically, it's possible that we have not anticipated some issues at all. What do we do about that?
|
||||
|
||||
Even the most thorough offline evaluation process can't predict every problem that will emerge in production. Users will find new ways to confuse your system, edge cases you never considered will surface, and changing business requirements will create new failure modes.
|
||||
|
||||
This means you need strategies for discovering issues that your existing evaluation framework doesn't catch. How do you identify problems you weren't looking for? How do you evolve your evaluation approach as new patterns emerge?
|
||||
|
||||
These four challenges form the foundation of production monitoring strategy. Getting them right determines whether your monitoring system provides actionable insights or becomes an expensive distraction.
|
||||
|
||||
## What Comes Next
|
||||
|
||||
The transition from controlled evaluation to production monitoring requires addressing these four core challenges systematically. The goal isn't to replicate your reference dataset evaluation at production scale (that would be impractical and expensive). Instead, you need smart strategies for each challenge.
|
||||
|
||||
In the next chapter, we'll cover practical approaches to:
|
||||
- **Log filtering**: Strategies for identifying which data needs attention without drowning in information
|
||||
- **Metric selection**: Frameworks for choosing the right mix of evaluation approaches based on value and cost
|
||||
- **Online vs offline evaluation**: Designing systems that balance immediate responsiveness with thorough analysis
|
||||
- **Emerging issue discovery**: Methods for identifying problems that your existing evaluation framework doesn't catch
|
||||
|
||||
These approaches will help you build a monitoring system that provides actionable insights while remaining sustainable and cost-effective as your AI system scales.
|
||||
|
||||
@@ -0,0 +1,295 @@
|
||||
# Chapter 7: Production Monitoring Strategies
|
||||
|
||||
|
||||
|
||||
## From Challenges to Solutions
|
||||
|
||||
In the previous chapter, we identified four core challenges that emerge when you move from controlled evaluation to production monitoring:
|
||||
|
||||
1. **Log filtering**: How to identify which data deserves attention
|
||||
2. **Metric selection**: How to choose the right evaluation approaches
|
||||
3. **Online vs offline evaluation**: How to balance real-time needs with thorough analysis
|
||||
4. **Emerging issue discovery**: How to find problems you weren't looking for
|
||||
|
||||
Now we'll address each challenge with practical strategies you can implement. The goal is building a sustainable monitoring system that provides actionable insights without overwhelming your team or budget.
|
||||
|
||||
## Log Filtering: Finding Signal in the Noise
|
||||
|
||||
|
||||
|
||||
When your AI system handles thousands of events daily, you need systematic approaches to identify what requires attention. Random sampling might miss critical issues, while trying to review everything is impossible.
|
||||
|
||||
### Priority-Based Filtering
|
||||
|
||||
Start by defining what matters most for your specific business context. Not all events are equally important, and what deserves attention varies significantly based on your use case and risk tolerance.
|
||||
|
||||
For example, you might consider categorizing events like this:
|
||||
|
||||
**Potential high-priority signals** could include safety violations, system errors, or high-value interactions - but you need to define what "high-value" means for your business.
|
||||
|
||||
**Potential medium-priority signals** might be routine interactions that show unusual patterns - though you'll need to determine what constitutes "unusual" in your domain.
|
||||
|
||||
**Potential low-priority signals** could be simple, standard interactions - but again, "simple" and "standard" depend entirely on your system's purpose and user base.
|
||||
|
||||
### Signal-Based Sampling
|
||||
|
||||
Beyond basic priority filtering, you can look for implicit and explicit signals that users give you about interaction quality. You need to identify which signals matter most for your specific system and users.
|
||||
|
||||
Some examples of signals you might consider:
|
||||
|
||||
**Conversation patterns** like unusual length (much shorter or longer than typical), repetition (users rephrasing questions), explicit escalation requests, or confusion indicators. But what counts as "unusual" length depends entirely on your domain - a financial advisory conversation naturally runs longer than a weather query.
|
||||
|
||||
**User behavior patterns** such as extensive editing of generated content, retry behavior, frustration indicators, or abandonment patterns. For a content generation system, whether users copy-paste or heavily modify outputs tells you something about quality - but you need to decide what level of modification indicates a problem versus normal customization.
|
||||
|
||||
**Content quality indicators** including response completeness, format consistency, or context matching. The thresholds that matter depend on your system's purpose and user expectations.
|
||||
|
||||
The critical decision is determining which of these signals are most indicative of problems in your specific context.
|
||||
|
||||
### Example: Customer Support Filtering Considerations
|
||||
|
||||
A customer support AI team might consider various approaches, but the specific choices depend on their business priorities and risk tolerance:
|
||||
|
||||
They might choose to always examine interactions with explicit escalation requests or safety concerns, but the definition of "safety concern" varies by industry. They could focus on conversations mentioning competitors or billing disputes, but whether a $50 or $500 dispute deserves attention depends on their business model.
|
||||
|
||||
They might sample more heavily from unusually long conversations, but "unusual" for a simple password reset differs from "unusual" for a complex technical issue. They could prioritize first-time users or interactions that show signs of confusion, but the thresholds that matter depend on their user base and system design.
|
||||
|
||||
### Signal Considerations for Different AI Systems
|
||||
|
||||
**Content Generation AI** teams might care about extensive user editing of outputs, but they need to decide whether 50% editing indicates a problem or normal creative refinement.
|
||||
|
||||
**Financial Advisory AI** teams might monitor for repeated clarification requests, but they must determine whether two follow-ups indicate confusion or appropriate due diligence.
|
||||
|
||||
**E-commerce Recommendation AI** teams might track ignored recommendations, but they need to consider whether this indicates poor recommendations or users with specific preferences.
|
||||
|
||||
In each case, the team must define their own thresholds and priorities based on their specific context, users, and business goals.
|
||||
|
||||
### Dynamic Filtering Based on Production Signals
|
||||
|
||||
Your filtering strategy should adapt based on observable changes in your production environment, but you need to decide which signals matter most for your business.
|
||||
|
||||
**Consider increasing sampling when you observe** production changes like error rate spikes, new product launches that might confuse users, increases in human support tickets, shifts in user behavior patterns, seasonal events that change user needs, or marketing campaigns that influence how users phrase requests.
|
||||
|
||||
**Consider decreasing sampling when you see** stable performance metrics, mature interaction patterns, or stable behavior in specific system components - though you must balance this against resource constraints and the risk of missing emerging issues.
|
||||
|
||||
**Examples of production signals you might track** include support ticket volume and categories, user session abandonment rates, conversation length trends, system performance metrics, business metrics like conversion rates, and external events like product launches or competitor actions.
|
||||
|
||||
The key decisions are which signals to monitor, what changes are significant enough to trigger sampling adjustments, and how quickly to respond to different types of changes. These choices depend entirely on your business context, user base, and risk tolerance.
|
||||
|
||||
## Metric Selection: Choosing Your Evaluation Mix
|
||||
|
||||
|
||||
|
||||
Not all metrics are equally valuable, and running everything is expensive. You need frameworks for choosing the right mix of evaluation approaches.
|
||||
|
||||
### The Metric Value Framework
|
||||
|
||||
|
||||
|
||||
Evaluate each potential metric across three dimensions:
|
||||
|
||||
**Impact**: How much does this metric help you improve your system?
|
||||
- High impact: Metrics that reveal actionable problems
|
||||
- Medium impact: Metrics that provide useful trends
|
||||
- Low impact: Metrics that are interesting but don't drive decisions
|
||||
|
||||
**Reliability**: How consistent and accurate is this metric?
|
||||
- High reliability: Human expert evaluation, well-validated code checks
|
||||
- Medium reliability: Calibrated LLM judges, statistical measures
|
||||
- Low reliability: Uncalibrated automated assessments, proxy metrics
|
||||
|
||||
**Cost**: What does it cost to run this metric at scale?
|
||||
- Low cost: Simple code-based checks, existing system metrics
|
||||
- Medium cost: Fast LLM judge calls, periodic human spot-checks
|
||||
- High cost: Detailed human evaluation, expensive model calls, complex analysis
|
||||
|
||||
### Prioritization Matrix
|
||||
|
||||
|
||||
|
||||
Plot your potential metrics on a simple matrix:
|
||||
|
||||
**High Impact + Low Cost = Must Have**
|
||||
- Simple safety filters
|
||||
- Basic structure validation
|
||||
- Performance metrics (response time, success rate)
|
||||
- Clear policy violation detection
|
||||
|
||||
**High Impact + High Cost = Strategic Investment**
|
||||
- Calibrated LLM judges for subjective quality
|
||||
- Expert human evaluation for critical interactions
|
||||
- Detailed escalation accuracy assessment
|
||||
|
||||
**Low Impact + Low Cost = Nice to Have**
|
||||
- Basic statistical trends
|
||||
- Simple response length tracking
|
||||
- Automated sentiment detection
|
||||
|
||||
**Low Impact + High Cost = Avoid**
|
||||
- Elaborate scoring systems that don't drive decisions
|
||||
- Expensive metrics that duplicate existing insights
|
||||
- Over-detailed measurement of stable system behaviors
|
||||
|
||||
|
||||
## Online vs Offline Evaluation: Guardrails vs Improvement Flywheel
|
||||
|
||||
|
||||
|
||||
The choice between real-time and batch evaluation comes down to a fundamental question: What behaviors, if they go wrong, would be huge for your business?
|
||||
|
||||
### Online Evaluation: Business-Critical Guardrails
|
||||
|
||||
|
||||
|
||||
Online evaluation serves as guardrails - metrics that must run in real-time because the behaviors they monitor are so critical that failure would significantly impact your business.
|
||||
|
||||
These are metrics where you need immediate intervention, not just later analysis. When these guardrails trigger, your system should take immediate action like handing off to a human agent, blocking harmful content, or escalating to specialists.
|
||||
|
||||
**Think of guardrails for behaviors like**:
|
||||
- Safety violations that could harm users or your business
|
||||
- Compliance failures that could create legal liability
|
||||
- High-value customer situations that require immediate attention
|
||||
- System failures that impact user experience
|
||||
- Critical business rule violations
|
||||
|
||||
**Guardrail characteristics**:
|
||||
- Must be fast and reliable (failures cascade quickly)
|
||||
- Should trigger immediate actions (handoffs, blocks, escalations)
|
||||
- Focus on preventing catastrophic outcomes, not optimization
|
||||
- Need to work even when other systems are stressed
|
||||
|
||||
**Examples of potential guardrail metrics**:
|
||||
- Safety filters blocking harmful content before it reaches users
|
||||
- Compliance checks ensuring required disclaimers in financial advice
|
||||
- Uncertainty detection triggering immediate human handoff
|
||||
- High-value customer detection routing to premium support
|
||||
- System error detection triggering failover procedures
|
||||
|
||||
### Offline Evaluation: Improvement Flywheel
|
||||
|
||||
|
||||
|
||||
Offline evaluation powers your improvement flywheel - analyzing data after the fact to understand trends, assess quality, and guide system improvements.
|
||||
|
||||
These metrics help you get better over time rather than preventing immediate disasters. They're often more sophisticated, expensive, or time-consuming than guardrails, but they provide the insights needed to evolve your system.
|
||||
|
||||
**Offline evaluation focuses on**:
|
||||
- Understanding quality trends over time
|
||||
- Identifying patterns that inform system improvements
|
||||
- Conducting detailed analysis of complex behaviors
|
||||
- Assessing the effectiveness of your guardrails and other systems
|
||||
- Discovering opportunities for optimization
|
||||
|
||||
**Examples of potential offline metrics**:
|
||||
- LLM judge assessment of conversation quality trends
|
||||
- Human expert review of escalated cases to improve escalation logic
|
||||
- Analysis of user satisfaction patterns to guide product development
|
||||
- Evaluation of edge cases to expand training data
|
||||
- Assessment of guardrail effectiveness and calibration
|
||||
|
||||
### Making the Guardrail Decision
|
||||
|
||||
The key decision is identifying which behaviors are guardrail-worthy - meaning failure would have immediate, significant business impact.
|
||||
|
||||
For a healthcare AI, incorrect medication information might be a guardrail issue requiring immediate intervention. For an e-commerce chatbot, product recommendation accuracy might be important for improvement but not guardrail-critical.
|
||||
|
||||
For a financial advisory AI, compliance violations are clearly guardrail territory, while response tone optimization belongs in the improvement flywheel.
|
||||
|
||||
The cost and complexity of guardrails mean you should be selective about what requires real-time intervention versus what can wait for batch analysis and gradual improvement.
|
||||
|
||||
|
||||
|
||||
## Emerging Issue Discovery: When Your Signals Don't Match Your Metrics
|
||||
|
||||
|
||||
|
||||
Remember the log filtering approach we discussed earlier - you're already sampling based on implicit and explicit user signals like conversation length anomalies, retry behavior, editing patterns, and frustration indicators. But what happens when these signals are telling you something your current metrics aren't capturing?
|
||||
|
||||
This is where emerging issue discovery becomes critical. You might find that your existing evaluation metrics show everything is working well, but the user behavior signals you're sampling suggest otherwise.
|
||||
|
||||
### When Signals and Metrics Diverge
|
||||
|
||||
Consider this scenario: You're monitoring a content generation AI, and you've been sampling interactions where users heavily edit the generated outputs (one of your implicit signals). Your current metrics - like content relevance and grammar correctness - show these interactions are scoring well. But the signal persists: users keep making extensive edits.
|
||||
|
||||
This divergence suggests there might be a quality dimension you're not measuring. Perhaps users are editing for tone, brand voice, or subtle contextual appropriateness that your current metrics don't capture. The user behavior signal is revealing a hidden issue that your evaluation framework missed.
|
||||
|
||||
### Systematic Investigation of Signal-Metric Gaps
|
||||
|
||||
When you notice this pattern - where your sampling signals flag interactions but your metrics show no actionable improvements - it's time for manual investigation, which means you'll need to look at these traces manually, just like we did initially when building reference datasets:
|
||||
|
||||
**Analyze the filtered logs differently**: Instead of applying your existing metrics, look at the interactions your signals flagged with fresh eyes. What patterns do you see that your metrics might be missing?
|
||||
|
||||
**Qualitative review**: Have domain experts or users review the flagged interactions without knowing the metric scores. What do they notice that your metrics don't capture?
|
||||
|
||||
**Signal correlation analysis**: Look at which combinations of signals tend to appear together. Multiple signals pointing to the same interactions might indicate a systematic issue.
|
||||
|
||||
### Example: E-commerce Recommendation Discovery
|
||||
|
||||
An e-commerce AI notices high rates of users ignoring recommendations (a signal they're sampling). But their existing metrics show the recommendations are relevant and properly formatted. Investigation reveals users are ignoring recommendations during certain seasonal periods or for specific product categories - suggesting the system lacks awareness of temporal context or category-specific preferences that existing relevance metrics don't measure.
|
||||
|
||||
### Building New Metrics from Signal Patterns
|
||||
|
||||
When signal-metric divergence reveals hidden issues, you need to develop new evaluation approaches:
|
||||
|
||||
**Pattern documentation**: Systematically document what the expert review reveals about the flagged interactions.
|
||||
|
||||
**New metric development**: Create evaluation approaches that can capture the quality dimensions you discovered.
|
||||
|
||||
**Validation against signals**: Test whether your new metrics correlate with the user behavior signals that originally flagged the issue.
|
||||
|
||||
**Integration into your framework**: Add the new metrics to your offline evaluation for trend monitoring, and consider whether any need to become online guardrails.
|
||||
|
||||
### The Discovery Loop
|
||||
|
||||
|
||||
|
||||
This creates a continuous discovery loop:
|
||||
|
||||
1. **User signals** indicate potential issues through behavior patterns
|
||||
2. **Log filtering** samples these concerning interactions
|
||||
3. **Metric analysis** may show existing metrics aren't capturing the problem
|
||||
4. **Investigation** reveals hidden quality dimensions or failure modes
|
||||
5. **New metrics** are developed to monitor these newly discovered issues
|
||||
6. **Updated sampling** incorporates lessons learned to catch similar issues earlier
|
||||
|
||||
This loop ensures your evaluation framework evolves as you discover new ways your system can fail or as user expectations change over time.
|
||||
|
||||
The key insight is that user behavior signals often reveal problems before your metrics do - they're an early warning system that helps you discover evaluation gaps before they become major issues.
|
||||
|
||||
## Building Your Production Monitoring Strategy
|
||||
|
||||
Combining these four strategies creates a comprehensive production monitoring approach:
|
||||
|
||||
### Start Simple and Evolve
|
||||
|
||||
Begin with basic filtering, essential metrics, simple online checks, and manual discovery processes. Add complexity as you understand your system's behavior patterns and your team's capacity.
|
||||
|
||||
### Balance Cost and Value
|
||||
|
||||
Continuously evaluate whether your monitoring provides enough insight to justify its cost. Expensive evaluation that doesn't drive improvements should be reconsidered.
|
||||
|
||||
### Plan for Scale
|
||||
|
||||
Design your monitoring to grow with your system. Approaches that work for thousands of daily interactions need to adapt when you reach hundreds of thousands.
|
||||
|
||||
### Close the Feedback Loop
|
||||
|
||||
The goal of monitoring is improvement. Ensure that insights from your monitoring system feed back into better evaluation, system refinements, and updated business processes.
|
||||
|
||||
## The Complete Evaluation Journey: From Concepts to Production
|
||||
|
||||
We've now covered the full spectrum of AI evaluation - from understanding why evaluation matters (Chapter 1) to building systematic evaluation frameworks (Chapters 2-3), creating reference datasets and implementing metrics (Chapters 4-5), and finally deploying robust production monitoring (Chapters 6-7).
|
||||
|
||||
The key insight is that evaluation is never complete: you start by building evaluation for anticipated behaviors and failure modes, but real users will always find new ways to interact with your system that you haven't seen before. This is why production monitoring becomes a continuous cycle of discovering new patterns through user signals, manually investigating when your current metrics don't capture emerging issues, developing new evaluation approaches, and feeding these insights back into your evaluation framework.
|
||||
|
||||
Think of it as building evaluation for the patterns you can anticipate, then using monitoring to discover and evaluate the patterns you couldn't predict.
|
||||
|
||||
|
||||
|
||||
**Want to go deeper?** Choose the course that fits your journey:
|
||||
- **New to AI?** Check out our **[#1 rated Enterprise AI Course on Maven](https://maven.com/aishwarya-kiriti/genai-system-design)** for comprehensive guidance on building production-ready AI systems from scratch.
|
||||
- **Already building AI?** Take our newly launched **[Advanced Evals course](https://maven.com/aishwarya-kiriti/evals-problem-first)** for systematically improving your AI products through advanced evaluation techniques.
|
||||
|
||||
*📝 Note: Use code **GITHUB15** for 15% off on Maven courses (valid until January 15th, 2025)*
|
||||
|
||||
In the next chapter, we'll explore how to use these monitoring insights to create continuous improvement cycles that help your AI system get better over time.
|
||||
|
||||
@@ -0,0 +1,209 @@
|
||||
# Chapter 8: The Complete Evaluation Process
|
||||
|
||||
|
||||
|
||||
## From Concept to Production: Your Step-by-Step Guide
|
||||
|
||||
In the previous seven chapters, we've covered the complete landscape of AI evaluation - from understanding why it matters to deploying production monitoring systems. Now let's consolidate everything into a clear, step-by-step process you can follow to build robust evaluation for your AI system.
|
||||
|
||||
This chapter serves as your practical roadmap, connecting all the concepts we've discussed into actionable steps you can implement.
|
||||
|
||||
## The Two-Phase Approach
|
||||
|
||||
|
||||
|
||||
AI evaluation follows two distinct phases:
|
||||
|
||||
**Phase 1: Pre-Deployment Validation** (Chapters 1-5)
|
||||
- Build confidence that your system works as intended before users interact with it
|
||||
- Create systematic evaluation frameworks and metrics
|
||||
- Test thoroughly in controlled conditions
|
||||
|
||||
**Phase 2: Production Monitoring** (Chapters 6-7)
|
||||
- Monitor system performance with real users at scale
|
||||
- Discover new issues and evolving user behaviors
|
||||
- Continuously improve your system and evaluation approach
|
||||
|
||||
Here's how to work through each phase.
|
||||
|
||||
---
|
||||
|
||||
## Phase 1: Pre-Deployment Validation
|
||||
|
||||
### Step 1: Understand Your Evaluation Context
|
||||
*Based on Chapters 1-3*
|
||||
|
||||
**What you're doing**: Establish the foundation for your evaluation approach by understanding what makes AI evaluation unique and what you need to measure.
|
||||
|
||||
**Key decisions**: Recognize that your AI system is non-deterministic, focus on product evaluation (how your system behaves in your specific use case) rather than model evaluation, and identify the three components you're evaluating - Input, Expected, and Actual.
|
||||
|
||||
**What to do**: Start by mapping out your specific use case and domain requirements. Identify stakeholders who need to be involved - domain experts, product teams, and engineers. Remember that generic metrics like "helpfulness" mean different things in different contexts, so prepare for collaborative evaluation design across different team perspectives.
|
||||
|
||||
**Output**: Clear understanding that you're building evaluation for your specific context, not just testing general AI capabilities.
|
||||
|
||||
### Step 2: Build Your Reference Dataset
|
||||
*Based on Chapter 4*
|
||||
|
||||
**What you're doing**: Create a systematic collection of examples that represent the scenarios you care about most, with clear expectations for how your system should behave.
|
||||
|
||||
**Key decisions**:
|
||||
- Start small and specific (10-20 high-quality examples) rather than trying to be comprehensive
|
||||
- Focus on scenarios you absolutely cannot get wrong
|
||||
- Include realistic inputs that represent actual user behavior
|
||||
|
||||
**Action items**:
|
||||
1. **Generate initial examples**: Work with domain experts to create realistic scenarios based on historical data or domain knowledge
|
||||
2. **Run your system**: Test your AI system on these examples and document both outputs and any intermediate steps
|
||||
3. **Evaluate with experts**: Have domain experts review each example and answer "Was this response satisfactory? If not, why not?"
|
||||
4. **Identify error patterns**: Analyze failures to cluster them into underlying problems you can actually fix
|
||||
5. **Decide on ongoing metrics**: Determine which behaviors need continuous monitoring (recurring risks) versus one-time fixes
|
||||
|
||||
**Output**: A reference dataset with examples, system outputs, expert evaluations, and identified metrics for ongoing measurement.
|
||||
|
||||
### Step 3: Implement Your Evaluation Metrics
|
||||
*Based on Chapter 5*
|
||||
|
||||
|
||||
|
||||
**What you're doing**: Build the actual measurement systems that can assess your identified metrics using three possible approaches.
|
||||
|
||||
**Key decisions**:
|
||||
- Choose the right mix of human evaluation, code-based metrics, and LLM judges
|
||||
- Start simple and add complexity only when needed
|
||||
- Remember that LLM judges require careful calibration against human judgment
|
||||
|
||||
**Action items**:
|
||||
1. **For objective, measurable properties**: Implement code-based metrics (structure validation, performance checks, required content)
|
||||
2. **For subjective qualities**: Consider LLM judges with detailed rubrics and examples
|
||||
3. **For critical quality assessment**: Plan for human evaluation, at least for calibration and spot-checking
|
||||
4. **Build rubrics**: Create clear criteria defining acceptable vs. not acceptable performance with specific examples
|
||||
5. **Test your metrics**: Validate that your evaluation approaches actually catch the issues you care about
|
||||
6. **Calibrate LLM judges**: If using them, extensively test against human judgment and iteratively refine
|
||||
|
||||
**Output**: Implemented evaluation metrics that can reliably assess the behaviors you identified in Step 2.
|
||||
|
||||
---
|
||||
|
||||
## Phase 2: Production Monitoring
|
||||
|
||||
### Step 4: Deploy Smart Log Filtering
|
||||
*Based on Chapter 7 - Log Filtering*
|
||||
|
||||
**What you're doing**: Create systematic approaches to identify which production data deserves attention, since you can't manually review everything at scale.
|
||||
|
||||
**Key decisions**:
|
||||
- Define what matters most for your business context (high/medium/low priority events)
|
||||
- Choose which implicit and explicit user signals to monitor
|
||||
- Set up dynamic filtering that adapts to production changes
|
||||
|
||||
**Action items**:
|
||||
1. **Establish priority categories**: Define which events always need attention vs. which can be sampled
|
||||
2. **Identify user signals**: Look for patterns like unusual conversation length, retry behavior, editing patterns, frustration indicators
|
||||
3. **Set up signal-based sampling**: Sample more heavily from interactions showing concerning signals
|
||||
4. **Monitor production changes**: Increase sampling during new product launches, error rate spikes, or business requirement changes
|
||||
5. **Adapt over time**: Adjust your filtering strategy based on what you learn
|
||||
|
||||
**Output**: A filtering system that efficiently identifies the most important production data to examine.
|
||||
|
||||
### Step 5: Select and Deploy Your Production Metrics
|
||||
*Based on Chapter 7 - Metric Selection*
|
||||
|
||||
**What you're doing**: Choose which evaluation metrics to run in production based on their impact, reliability, and cost.
|
||||
|
||||
**Key decisions**:
|
||||
- Prioritize high-impact metrics that drive actionable improvements
|
||||
- Balance metric value against computational and financial costs
|
||||
- Focus resources on metrics that actually help you make better decisions
|
||||
|
||||
**Action items**:
|
||||
1. **Evaluate each metric**: Assess impact (how much it helps improve your system), reliability (how consistent it is), and cost (computational/financial expense)
|
||||
2. **Prioritize systematically**: Focus on high-impact, low-cost metrics first; carefully consider high-impact, high-cost metrics; avoid low-impact approaches regardless of cost
|
||||
3. **Start essential**: Implement must-have metrics that provide basic system health and safety monitoring
|
||||
4. **Add strategically**: Gradually incorporate more sophisticated metrics based on demonstrated value
|
||||
|
||||
**Output**: A cost-effective mix of evaluation metrics running in production.
|
||||
|
||||
### Step 6: Implement Guardrails and Improvement Loops
|
||||
*Based on Chapter 7 - Online vs Offline Evaluation*
|
||||
|
||||
**What you're doing**: Distinguish between metrics that need immediate intervention (guardrails) versus those that guide longer-term improvement.
|
||||
|
||||
**Key decisions**:
|
||||
- Identify which behaviors, if they go wrong, would be huge for your business (guardrails)
|
||||
- Design offline evaluation for trend analysis and system improvement
|
||||
- Balance real-time intervention needs with batch analysis efficiency
|
||||
|
||||
**Action items**:
|
||||
1. **Design guardrails**: Implement fast, reliable online metrics for business-critical behaviors that trigger immediate actions (handoffs, escalations, blocks)
|
||||
2. **Set up improvement loops**: Create offline evaluation processes that analyze trends, assess quality over time, and guide system improvements
|
||||
3. **Define trigger actions**: Establish clear procedures for what happens when guardrails activate
|
||||
4. **Plan feedback cycles**: Ensure offline analysis insights feed back into system improvements and evaluation refinements
|
||||
|
||||
**Output**: A two-tier system with real-time guardrails for critical issues and batch analysis for continuous improvement.
|
||||
|
||||
### Step 7: Build Emerging Issue Discovery
|
||||
*Based on Chapter 7 - Emerging Issue Discovery*
|
||||
|
||||
**What you're doing**: Create processes to discover problems your existing evaluation framework doesn't capture, using the same manual investigation techniques from reference dataset building.
|
||||
|
||||
**Key decisions**:
|
||||
- Recognize that user signals often reveal problems before metrics do
|
||||
- Plan for manual investigation when signals and metrics diverge
|
||||
- Build systematic processes to evolve your evaluation framework over time
|
||||
|
||||
**Action items**:
|
||||
1. **Monitor signal-metric divergence**: Watch for cases where user behavior signals flag issues but your metrics show no problems
|
||||
2. **Conduct manual investigation**: When divergence occurs, manually review the flagged interactions just like you did when building reference datasets
|
||||
3. **Identify hidden issues**: Look for quality dimensions or failure modes your current metrics don't capture
|
||||
4. **Develop new metrics**: Create evaluation approaches for newly discovered issues
|
||||
5. **Update your framework**: Add new metrics to your evaluation system and refine your filtering approach
|
||||
6. **Close the discovery loop**: Ensure insights from investigation feed back into better evaluation and system improvements
|
||||
|
||||
**Output**: A continuously evolving evaluation framework that adapts as you discover new issues and user behaviors.
|
||||
|
||||
---
|
||||
|
||||
## The Complete Process Flow
|
||||
|
||||
|
||||
|
||||
Here's how all these steps connect:
|
||||
|
||||
1. **Foundation** → Understand your specific evaluation needs and context
|
||||
2. **Reference Dataset** → Build systematic examples with clear quality expectations
|
||||
3. **Metrics Implementation** → Create reliable measurement systems for your quality criteria
|
||||
4. **Production Filtering** → Efficiently identify important production data to examine
|
||||
5. **Metric Deployment** → Run cost-effective evaluation at scale
|
||||
6. **Guardrails + Improvement** → Handle critical issues immediately while building long-term improvement
|
||||
7. **Discovery Loop** → Continuously evolve your evaluation as you learn new failure modes
|
||||
|
||||
## Key Principles Throughout
|
||||
|
||||
**Start Simple**: Begin with basic approaches and add complexity only when justified by clear value.
|
||||
|
||||
**Focus on Context**: Generic evaluation approaches don't work - everything must be tailored to your specific use case, users, and business requirements.
|
||||
|
||||
**Collaborate Across Teams**: Effective evaluation requires input from domain experts, product teams, and engineers working together.
|
||||
|
||||
**Embrace Evolution**: Your evaluation framework should continuously improve as you discover new ways your system can fail or as user expectations change.
|
||||
|
||||
**Connect Evaluation to Improvement**: The goal is better AI systems, not perfect measurement. Focus on evaluation that drives actionable improvements.
|
||||
|
||||
## What You End Up With
|
||||
|
||||
Following this complete process gives you:
|
||||
|
||||
- **Confidence before deployment**: Systematic validation that your system works as intended
|
||||
- **Effective production monitoring**: Smart filtering and evaluation that scales with your system
|
||||
- **Proactive issue detection**: Early warning systems that catch problems before they become major issues
|
||||
- **Continuous improvement**: Feedback loops that help your system get better over time
|
||||
- **Sustainable evaluation**: Cost-effective approaches that provide value without overwhelming your team
|
||||
|
||||
## The Ongoing Journey
|
||||
|
||||
Remember that evaluation is never complete. You start by building evaluation for patterns you can anticipate, then use production monitoring to discover and evaluate patterns you couldn't predict. User behavior evolves, business requirements change, and new failure modes emerge.
|
||||
|
||||
The framework we've built gives you the tools to adapt your evaluation approach as your understanding deepens and your system grows. The key is maintaining the discipline of systematic evaluation while staying flexible enough to learn and evolve.
|
||||
|
||||
This complete process transforms evaluation from an afterthought into a core capability that helps you build more reliable, useful, and trustworthy AI systems.
|
||||
|
||||
@@ -0,0 +1,173 @@
|
||||
# Chapter 9: Common Misconceptions About AI Evaluation
|
||||
|
||||
|
||||
|
||||
## Clearing Up the Confusion
|
||||
|
||||
Now that you've worked through this complete evaluation course, you're equipped to recognize common misconceptions that trip up many teams building AI systems. This chapter addresses the most frequent misunderstandings we encounter, explaining why they're problematic and pointing you to the right approaches.
|
||||
|
||||
Each misconception below includes a reference to the chapters where we covered the correct approach in detail.
|
||||
|
||||
---
|
||||
|
||||
## Foundation Misconceptions
|
||||
|
||||
|
||||
|
||||
### 1. "Model evaluations (benchmarks) predict my product success"
|
||||
|
||||
**Why this is wrong**: Model evaluations test general capabilities on standardized tasks, but your product operates in a specific domain with unique requirements, constraints, and user behaviors. A model that scores 92% on general benchmarks might perform poorly for your insurance claims processing system if it hasn't seen domain-specific patterns.
|
||||
|
||||
**The reality**: Product evaluation in your specific context is what matters. You need to test how the model behaves with your data, your users, your business rules, and your risk tolerance.
|
||||
|
||||
**Where we covered this**: Chapter 2 explains the crucial distinction between model and product evaluations, showing why benchmark performance often fails to predict real-world success in your specific use case.
|
||||
|
||||
### 2. "Engineers can design evaluation metrics alone"
|
||||
|
||||
**Why this is wrong**: Engineers understand technical implementation but may miss domain-specific quality requirements, business risks, and subtle user expectations. What looks technically correct might be completely inappropriate for the domain.
|
||||
|
||||
**The reality**: Effective evaluation requires collaboration between domain experts (who understand quality), product teams (who understand user needs), and engineers (who understand technical constraints). Each brings essential perspectives.
|
||||
|
||||
**Where we covered this**: Chapter 3 emphasizes that evaluation is inherently collaborative and explains how different stakeholders contribute to defining quality standards and building rubrics.
|
||||
|
||||
### 3. "Evaluation is a one-time setup before launch"
|
||||
|
||||
**Why this is wrong**: This treats evaluation like traditional software testing, where you can validate everything upfront and expect it to stay valid. AI systems are non-deterministic, user behavior evolves, and business requirements change.
|
||||
|
||||
**The reality**: Evaluation is a continuous process that evolves with your system. You start with pre-deployment validation, then monitor in production, discover new issues, and continuously refine your evaluation approach.
|
||||
|
||||
**Where we covered this**: Chapter 1 explains why AI systems require ongoing evaluation, and Chapter 6 details how production monitoring differs from pre-deployment testing.
|
||||
|
||||
|
||||
|
||||
---
|
||||
|
||||
## Pre-Deployment Misconceptions
|
||||
|
||||
|
||||
|
||||
### 4. "I need comprehensive evaluation coverage from day one"
|
||||
|
||||
**Why this is wrong**: Trying to build comprehensive evaluation upfront leads to analysis paralysis and often misses the most important issues. You can't predict every failure mode, and attempting comprehensive coverage dilutes effort from high-impact scenarios.
|
||||
|
||||
**The reality**: Start small with 10-20 high-quality examples representing scenarios you absolutely cannot get wrong. Focus on quality over quantity and expand as you learn more about your system's behavior patterns.
|
||||
|
||||
**Where we covered this**: Chapter 4 walks through building reference datasets, emphasizing starting small and specific rather than trying to be comprehensive from the beginning.
|
||||
|
||||
### 5. "Code-based metrics aren't sophisticated enough for AI systems"
|
||||
|
||||
**Why this is wrong**: This assumes you need complex evaluation for complex systems. In practice, simple code-based checks often provide the most reliable signal for many important behaviors like structure validation, compliance requirements, and performance monitoring.
|
||||
|
||||
**The reality**: Simple code checks are fast, reliable, and easy to understand. Use them for objective, measurable properties before adding complexity with LLM judges or human evaluation.
|
||||
|
||||
**Where we covered this**: Chapter 5 details the three evaluation approaches, showing when code-based metrics are most effective and why they should often be your first choice.
|
||||
|
||||
### 6. "LLM judges are the best way to evaluate AI systems"
|
||||
|
||||
**Why this is wrong**: LLM judges seem appealing because they can assess subjective qualities at scale, but they're expensive, slow, and can be inconsistent or misaligned with human judgment. Uncalibrated LLM judges often create more problems than they solve.
|
||||
|
||||
**The reality**: LLM judges are powerful tools when properly calibrated, but they require extensive validation against human judgment. Start with simpler approaches and add LLM judges only when justified by clear value.
|
||||
|
||||
**Where we covered this**: Chapter 5 explains the challenges with LLM judges and emphasizes that calibration is essential for reliable results.
|
||||
|
||||
### 7. "If I write detailed criteria, LLM judges will work correctly"
|
||||
|
||||
**Why this is wrong**: Detailed criteria help, but don't guarantee that an LLM will interpret them the same way human experts would. LLMs can be overly strict, overly lenient, or miss subtle contextual cues that humans notice.
|
||||
|
||||
**The reality**: LLM judge calibration requires extensive testing against human evaluations across hundreds of examples, statistical analysis of agreement rates, and iterative prompt refinement. This process often takes weeks or months.
|
||||
|
||||
**Where we covered this**: Chapter 5 includes detailed guidance on LLM judge calibration and why detailed criteria alone are insufficient for reliable evaluation.
|
||||
|
||||
---
|
||||
|
||||
## Production Misconceptions
|
||||
|
||||
|
||||
|
||||
### 8. "I need to evaluate every production interaction"
|
||||
|
||||
**Why this is wrong**: At scale, evaluating every interaction is impossible and unnecessary. It would require enormous computational resources and human effort while providing diminishing returns from analyzing routine, successful interactions.
|
||||
|
||||
**The reality**: Smart sampling based on user signals is more effective. Focus evaluation on interactions showing concerning patterns like unusual length, retry behavior, or frustration indicators.
|
||||
|
||||
**Where we covered this**: Chapter 7's log filtering section explains how to identify which production data deserves attention through priority-based filtering and signal-based sampling.
|
||||
|
||||
### 9. "I need a sophisticated dashboard with dozens of metrics"
|
||||
|
||||
**Why this is wrong**: More metrics don't automatically mean better insights. Too many metrics create noise, make it hard to focus on what matters, and often lead to analysis paralysis rather than actionable improvements.
|
||||
|
||||
**The reality**: Focus on a minimum set of actionable metrics that drive real improvements. It's better to have 3-5 metrics that consistently guide decisions than 20 metrics that no one acts on.
|
||||
|
||||
**Where we covered this**: Chapter 7's metric selection section provides frameworks for choosing metrics based on impact, reliability, and cost rather than trying to measure everything.
|
||||
|
||||
|
||||
|
||||
### 10. "Online evaluation is always better than offline"
|
||||
|
||||
**Why this is wrong**: Online evaluation seems superior because it provides immediate feedback, but it must be fast and simple to avoid adding latency. Complex analysis that requires expensive computation or sophisticated reasoning belongs in offline evaluation.
|
||||
|
||||
**The reality**: Use online evaluation for business-critical guardrails that need immediate intervention. Use offline evaluation for detailed analysis, trend identification, and system improvement insights.
|
||||
|
||||
**Where we covered this**: Chapter 7 distinguishes between online guardrails (preventing immediate problems) and offline improvement loops (driving long-term system enhancement).
|
||||
|
||||
|
||||
|
||||
### 11. "Evals vs A/B testing - I need to pick one approach"
|
||||
|
||||
**Why this is wrong**: This creates a false dichotomy between two complementary approaches. Each serves different purposes and they work better together than in isolation.
|
||||
|
||||
**The reality**: Use evaluation metrics to monitor known patterns and behaviors you understand. Use A/B testing to discover new patterns through explicit user signals (ratings, conversions) and implicit signals (behavior changes, engagement).
|
||||
|
||||
**Where we covered this**: Chapter 7's emerging issue discovery section explains how user signals can reveal problems your evaluation metrics don't capture, leading to new evaluation approaches.
|
||||
|
||||
|
||||
|
||||
### 12. "Evaluation metrics are fixed once implemented"
|
||||
|
||||
**Why this is wrong**: This assumes your system, users, and business requirements remain static. In reality, user behavior evolves, business priorities change, and you discover new failure modes that require different evaluation approaches.
|
||||
|
||||
**The reality**: Metrics retire and update over time as you learn. A metric that was critical during early deployment might become less useful as your system matures. Meanwhile, new user behaviors might require entirely new metrics.
|
||||
|
||||
**Where we covered this**: Chapter 7's emerging issue discovery explains the continuous loop of discovering new patterns, developing new metrics, and retiring outdated approaches.
|
||||
|
||||
**Examples of metric evolution**:
|
||||
- **Retiring**: A "response format validation" metric becomes less important as your system matures and format errors become rare
|
||||
- **Adding**: A "seasonal context awareness" metric becomes important after discovering users ask different questions during holidays
|
||||
- **Updating**: An "escalation accuracy" metric needs refinement after business policy changes affect when human handoffs are appropriate
|
||||
|
||||
|
||||
|
||||
---
|
||||
|
||||
## Why These Misconceptions Persist
|
||||
|
||||
Understanding why these misconceptions are common helps you avoid them:
|
||||
|
||||
**AI evaluation is relatively new**: Unlike traditional software testing, systematic AI evaluation is still emerging, leading to borrowed assumptions from other domains.
|
||||
|
||||
**Complexity creates uncertainty**: AI systems are complex, making simple approaches seem inadequate even when they're often the most effective.
|
||||
|
||||
**Tool marketing influences thinking**: Vendors promote sophisticated solutions that may be overkill for many practical needs.
|
||||
|
||||
**Success stories lack context**: Case studies often don't include the failures and iterations that led to successful evaluation approaches.
|
||||
|
||||
## The Right Mindset
|
||||
|
||||
Instead of falling into these misconceptions, approach AI evaluation with these principles:
|
||||
|
||||
**Start simple and evolve**: Begin with basic approaches that provide clear value, then add complexity only when justified.
|
||||
|
||||
**Focus on your context**: Generic solutions rarely work - everything must be tailored to your specific use case, users, and business requirements.
|
||||
|
||||
**Embrace collaboration**: Combine technical, domain, and business perspectives rather than trying to solve evaluation in isolation.
|
||||
|
||||
**Expect continuous evolution**: Build evaluation systems that can adapt as you learn more about your system and users.
|
||||
|
||||
**Prioritize actionable insights**: Measure things that drive real improvements rather than pursuing measurement for its own sake.
|
||||
|
||||
## Moving Forward
|
||||
|
||||
Now that you understand these common misconceptions and have worked through the complete evaluation methodology, you're equipped to build effective evaluation systems that avoid these pitfalls.
|
||||
|
||||
Remember: the goal isn't perfect measurement - it's building better AI systems through systematic, thoughtful evaluation that evolves with your understanding and needs.
|
||||
|
||||
@@ -0,0 +1,187 @@
|
||||
# Chapter 10: Glossary of Terms
|
||||
|
||||
|
||||
|
||||
## Making Sense of the Evaluation Vocabulary
|
||||
|
||||
Throughout this course, we've used specific terms to describe different aspects of AI evaluation. This glossary clarifies what we mean by each term, helping you navigate the sometimes confusing world of evaluation terminology.
|
||||
|
||||
|
||||
|
||||
---
|
||||
|
||||
### Evals
|
||||
The catch-all term that everyone uses for everything evaluation-related, which is exactly why it causes so much confusion. Someone might say "we need better evals" and mean anything from benchmark scores to production monitoring dashboards. We intentionally avoid this term in favor of more precise language.
|
||||
|
||||
### Evaluation
|
||||
The overall process of assessing how an AI system behaves. This includes everything from designing metrics to running tests to analyzing results. Evaluation answers the question: "Is this system behaving the way we want it to?"
|
||||
|
||||
### Evaluation Metrics
|
||||
The specific dimensions along which system behavior is judged. These answer "what does good mean in this context?" Examples include escalation accuracy, response time, or compliance adherence. Always context-dependent and require clear rubrics.
|
||||
|
||||
### Expected Behavior
|
||||
What your system should do in a given situation. Part of the Input-Expected-Actual framework. Often requires collaboration between domain experts and product teams to define clearly.
|
||||
|
||||
### Explicit Signals
|
||||
Direct indicators users give about their experience, such as ratings, explicit escalation requests ("let me talk to a human"), or direct complaints. Easier to interpret than implicit signals but less common.
|
||||
|
||||
### Actual Behavior
|
||||
What your system actually does when given specific inputs. This includes not just the final output, but intermediate steps and any actions taken.
|
||||
|
||||
### Benchmark
|
||||
A standardized test used to measure model capabilities across different systems. Examples include MMLU, HumanEval, or GSM8K. Useful for comparing models but don't predict performance in your specific use case.
|
||||
|
||||
### Code-Based Metrics
|
||||
Deterministic checks written in programming code that look for specific patterns or properties. Fast, reliable, and perfect for objective measurements like structure validation, required content presence, or performance monitoring.
|
||||
|
||||
|
||||
|
||||
### Guardrails
|
||||
Real-time evaluation metrics that monitor business-critical behaviors and trigger immediate interventions when problems occur. These are online metrics for situations where failure would have immediate, significant business impact. Examples include safety filters or compliance checks.
|
||||
|
||||
### Implicit Signals
|
||||
Indirect indicators of user satisfaction or system problems, revealed through user behavior rather than explicit feedback. Examples include conversation length anomalies, retry behavior, extensive editing of generated content, or abandonment patterns.
|
||||
|
||||
### Improvement Flywheel
|
||||
The offline evaluation process that powers long-term system enhancement through trend analysis, quality assessment, and systematic investigation of issues discovered in production.
|
||||
|
||||
### Input
|
||||
Everything that influences how your AI system behaves, including the user's request, conversation history, retrieved data, and system configuration. Part of the Input-Expected-Actual evaluation framework.
|
||||
|
||||
### LLM Judge
|
||||
Using one language model to evaluate another model's behavior. Powerful for assessing subjective qualities like tone or appropriateness, but requires extensive calibration against human judgment to be reliable.
|
||||
|
||||
### Log Filtering
|
||||
Systematic approaches to identify which production data deserves evaluation attention. Uses priority-based filtering and signal-based sampling since you can't review everything at scale.
|
||||
|
||||
### Model Evaluation
|
||||
Assessment of general AI model capabilities, typically using standardized benchmarks. Helps with model selection but doesn't predict performance in your specific product context.
|
||||
|
||||
### Metric Selection
|
||||
The process of choosing which evaluation approaches to implement based on their impact, reliability, and cost. Requires balancing value against computational and financial expenses.
|
||||
|
||||
### Non-Deterministic
|
||||
A key characteristic of AI systems where the same input can produce different outputs across runs. This breaks traditional software testing assumptions and makes evaluation more complex but essential.
|
||||
|
||||
### Offline Evaluation
|
||||
Evaluation that happens after interactions occur, often in batch processes. Used for trend analysis, detailed quality assessment, and system improvement insights. Allows for sophisticated, expensive analysis that would be impractical in real-time.
|
||||
|
||||
### Online Evaluation
|
||||
Real-time evaluation that runs as interactions happen and can trigger immediate responses. Must be fast and lightweight. Used for guardrails and situations requiring immediate intervention.
|
||||
|
||||
|
||||
|
||||
### Product Evaluation
|
||||
Assessment of how an AI system behaves in your specific use case, with your users, data, and business context. This is what actually matters for building successful AI products, as opposed to general model capabilities.
|
||||
|
||||
|
||||
|
||||
### Production Monitoring
|
||||
Continuous evaluation of AI system performance with real users at scale. Includes log filtering, metric deployment, guardrails, and emerging issue discovery.
|
||||
|
||||
### Reference Dataset
|
||||
A carefully chosen collection of realistic examples that represent scenarios you care most about. Includes inputs, expected behaviors, and serves as the foundation for systematic evaluation. Start small (10-20 examples) and expand based on learning.
|
||||
|
||||
|
||||
|
||||
### Rubric
|
||||
Explicit criteria that define what constitutes acceptable versus unacceptable performance. Essential for making subjective evaluation consistent. Should include specific examples and edge case guidance.
|
||||
|
||||
### Signal-Based Sampling
|
||||
Sampling production data based on implicit and explicit user signals rather than random selection. More effective for catching problems than uniform sampling across all interactions.
|
||||
|
||||
### Signal-Metric Divergence
|
||||
When user behavior signals indicate problems but your current evaluation metrics show no issues. This pattern suggests hidden quality dimensions that your existing evaluation framework doesn't capture.
|
||||
|
||||
### User Evolution
|
||||
The natural progression of how users interact with AI systems over time. As users become comfortable, they develop new interaction patterns, push boundaries, and use systems in increasingly sophisticated ways. This changes the distribution of inputs your system receives.
|
||||
|
||||
---
|
||||
|
||||
## Framework Concepts
|
||||
|
||||
### Input-Expected-Actual Framework
|
||||
The conceptual foundation for thinking about AI system behavior:
|
||||
- **Input**: Everything that goes into your system
|
||||
- **Expected**: What should happen given your requirements
|
||||
- **Actual**: What your system really does
|
||||
|
||||
This framework helps structure evaluation by making explicit what you're comparing.
|
||||
|
||||
### Guardrails vs. Improvement Flywheel
|
||||
The two-tier approach to production evaluation:
|
||||
- **Guardrails**: Online metrics for immediate intervention on business-critical issues
|
||||
- **Improvement Flywheel**: Offline analysis for long-term system enhancement
|
||||
|
||||
### Discovery Loop
|
||||
The continuous cycle of emerging issue discovery:
|
||||
1. User signals indicate potential problems
|
||||
2. Log filtering samples concerning interactions
|
||||
3. Existing metrics may not capture the issues
|
||||
4. Manual investigation reveals hidden problems
|
||||
5. New metrics are developed
|
||||
6. Updated framework catches similar issues earlier
|
||||
|
||||
---
|
||||
|
||||
## Process Terms
|
||||
|
||||
### Pre-Deployment Validation
|
||||
The systematic evaluation work done before real users interact with your system. Includes building reference datasets, implementing metrics, and testing in controlled conditions to build confidence.
|
||||
|
||||
### Calibration
|
||||
The process of ensuring LLM judges align with human judgment through extensive testing, comparison analysis, and iterative refinement. Often takes weeks or months and is essential for reliable automated evaluation.
|
||||
|
||||
### Emerging Issue Discovery
|
||||
Systematic approaches to find problems your existing evaluation framework doesn't capture. Uses signal-metric divergence analysis and manual investigation to evolve evaluation as new failure modes emerge.
|
||||
|
||||
---
|
||||
|
||||
## Common Anti-Patterns (What NOT to Do)
|
||||
|
||||
### Evaluation Drift
|
||||
When your evaluation metrics become disconnected from actual user needs or business goals. Happens when you measure things because they're easy to measure rather than because they matter.
|
||||
|
||||
### Metric Overload
|
||||
Having too many evaluation metrics, making it impossible to focus on what actually drives improvements. More metrics don't automatically mean better insights.
|
||||
|
||||
### Calibration Neglect
|
||||
Deploying LLM judges without proper validation against human judgment, leading to evaluation that's worse than having no evaluation at all.
|
||||
|
||||
### Coverage Obsession
|
||||
Trying to evaluate everything comprehensively rather than focusing on high-impact scenarios. Leads to analysis paralysis and diluted effort.
|
||||
|
||||
---
|
||||
|
||||
## Key Principles
|
||||
|
||||
Throughout this course, we've emphasized these core principles:
|
||||
|
||||
**Context is King**: Everything must be tailored to your specific use case, users, and business requirements. Generic approaches rarely work.
|
||||
|
||||
**Start Simple, Evolve**: Begin with basic approaches and add complexity only when justified by clear value.
|
||||
|
||||
**Collaboration is Essential**: Combine technical, domain, and business perspectives rather than trying to solve evaluation in isolation.
|
||||
|
||||
**Continuous Learning**: Evaluation systems must adapt as you discover new failure modes and as user behavior evolves.
|
||||
|
||||
**Action Over Measurement**: The goal is better AI systems, not perfect measurement. Focus on evaluation that drives real improvements.
|
||||
|
||||
---
|
||||
|
||||
## Using This Glossary
|
||||
|
||||
This glossary reflects the specific way we use these terms in this course. You might encounter different definitions elsewhere - the AI evaluation field is still developing standard terminology. When working with others, it's always worth clarifying what specific terms mean in your context.
|
||||
|
||||
Remember: the vocabulary matters less than the underlying concepts. Focus on building systematic, thoughtful evaluation that helps you create better AI systems for your users.
|
||||
|
||||
**Ready to get certified?** You've completed all 10 chapters of this AI evaluation course! **[Take the certification assessment now](https://ai-evals-course-website-2025.vercel.app/quiz-google.html)** to earn your AI Evals for Everyone certificate and test your knowledge.
|
||||
|
||||
---
|
||||
|
||||
**Want to go deeper?** Choose the course that fits your journey:
|
||||
- **New to AI?** Check out our **[#1 rated Enterprise AI Course on Maven](https://maven.com/aishwarya-kiriti/genai-system-design)** for comprehensive guidance on building production-ready AI systems from scratch.
|
||||
- **Already building AI?** Take our newly launched **[Advanced Evals course](https://maven.com/aishwarya-kiriti/evals-problem-first)** for systematically improving your AI products through advanced evaluation techniques.
|
||||
|
||||
*📝 Note: Use code **GITHUB15** for 15% off on Maven courses (valid until January 15th, 2025)*
|
||||
|
||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user