chore: import upstream snapshot with attribution

.setup/README.md

@@ -0,0 +1,29 @@
+# Setup Instructions
+
+Here you will find several methods of running the code found in the book. There are two preferred methods, one for local usage and one for cloud-based:
+
+* **Local**: Using a [Conda](../.setup/conda) environment
+* **Cloud**: Using [Google Colab Notebooks](https://github.com/HandsOnLLM/Hands-On-Large-Language-Models/tree/main?tab=readme-ov-file#table-of-contents)
+
+## Quick start
+
+If you already have a local Python (3.10.*) environment and [Microsoft Visual C++ 14.0](https://visualstudio.microsoft.com/visual-cpp-build-tools/) or greater installed, you can install the complete environment as follows at the root of this repository:
+
+```bash
+pip install -r requirements.txt
+```
+
+> [!WARNING]
+> If you get the following error `error: Microsoft Visual C++ 14.0 or greater is required.` then you will need to install C++. 
+> Follow the instructions [here](../.setup/conda/common_issues.md) for an installation guide before you can install your environment.
+
+If you have conda installed (this **does not** require an additional installation of C++), you can also install the complete environment as follows at the root of this repository:
+
+```bash
+conda env create -f environment.yml
+```
+
+> [!TIP]
+> After preparing your environment, it is recommended to install the GPU version of PyTorch following the instructions [here](https://pytorch.org/) as most examples will require a GPU.
+
+For a complete tutorial on setting up your environment, visit the [conda example](../.setup/conda).

LICENSE

@@ -0,0 +1,201 @@
+                                 Apache License
+                           Version 2.0, January 2004
+                        http://www.apache.org/licenses/
+
+   TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
+
+   1. Definitions.
+
+      "License" shall mean the terms and conditions for use, reproduction,
+      and distribution as defined by Sections 1 through 9 of this document.
+
+      "Licensor" shall mean the copyright owner or entity authorized by
+      the copyright owner that is granting the License.
+
+      "Legal Entity" shall mean the union of the acting entity and all
+      other entities that control, are controlled by, or are under common
+      control with that entity. For the purposes of this definition,
+      "control" means (i) the power, direct or indirect, to cause the
+      direction or management of such entity, whether by contract or
+      otherwise, or (ii) ownership of fifty percent (50%) or more of the
+      outstanding shares, or (iii) beneficial ownership of such entity.
+
+      "You" (or "Your") shall mean an individual or Legal Entity
+      exercising permissions granted by this License.
+
+      "Source" form shall mean the preferred form for making modifications,
+      including but not limited to software source code, documentation
+      source, and configuration files.
+
+      "Object" form shall mean any form resulting from mechanical
+      transformation or translation of a Source form, including but
+      not limited to compiled object code, generated documentation,
+      and conversions to other media types.
+
+      "Work" shall mean the work of authorship, whether in Source or
+      Object form, made available under the License, as indicated by a
+      copyright notice that is included in or attached to the work
+      (an example is provided in the Appendix below).
+
+      "Derivative Works" shall mean any work, whether in Source or Object
+      form, that is based on (or derived from) the Work and for which the
+      editorial revisions, annotations, elaborations, or other modifications
+      represent, as a whole, an original work of authorship. For the purposes
+      of this License, Derivative Works shall not include works that remain
+      separable from, or merely link (or bind by name) to the interfaces of,
+      the Work and Derivative Works thereof.
+
+      "Contribution" shall mean any work of authorship, including
+      the original version of the Work and any modifications or additions
+      to that Work or Derivative Works thereof, that is intentionally
+      submitted to Licensor for inclusion in the Work by the copyright owner
+      or by an individual or Legal Entity authorized to submit on behalf of
+      the copyright owner. For the purposes of this definition, "submitted"
+      means any form of electronic, verbal, or written communication sent
+      to the Licensor or its representatives, including but not limited to
+      communication on electronic mailing lists, source code control systems,
+      and issue tracking systems that are managed by, or on behalf of, the
+      Licensor for the purpose of discussing and improving the Work, but
+      excluding communication that is conspicuously marked or otherwise
+      designated in writing by the copyright owner as "Not a Contribution."
+
+      "Contributor" shall mean Licensor and any individual or Legal Entity
+      on behalf of whom a Contribution has been received by Licensor and
+      subsequently incorporated within the Work.
+
+   2. Grant of Copyright License. Subject to the terms and conditions of
+      this License, each Contributor hereby grants to You a perpetual,
+      worldwide, non-exclusive, no-charge, royalty-free, irrevocable
+      copyright license to reproduce, prepare Derivative Works of,
+      publicly display, publicly perform, sublicense, and distribute the
+      Work and such Derivative Works in Source or Object form.
+
+   3. Grant of Patent License. Subject to the terms and conditions of
+      this License, each Contributor hereby grants to You a perpetual,
+      worldwide, non-exclusive, no-charge, royalty-free, irrevocable
+      (except as stated in this section) patent license to make, have made,
+      use, offer to sell, sell, import, and otherwise transfer the Work,
+      where such license applies only to those patent claims licensable
+      by such Contributor that are necessarily infringed by their
+      Contribution(s) alone or by combination of their Contribution(s)
+      with the Work to which such Contribution(s) was submitted. If You
+      institute patent litigation against any entity (including a
+      cross-claim or counterclaim in a lawsuit) alleging that the Work
+      or a Contribution incorporated within the Work constitutes direct
+      or contributory patent infringement, then any patent licenses
+      granted to You under this License for that Work shall terminate
+      as of the date such litigation is filed.
+
+   4. Redistribution. You may reproduce and distribute copies of the
+      Work or Derivative Works thereof in any medium, with or without
+      modifications, and in Source or Object form, provided that You
+      meet the following conditions:
+
+      (a) You must give any other recipients of the Work or
+          Derivative Works a copy of this License; and
+
+      (b) You must cause any modified files to carry prominent notices
+          stating that You changed the files; and
+
+      (c) You must retain, in the Source form of any Derivative Works
+          that You distribute, all copyright, patent, trademark, and
+          attribution notices from the Source form of the Work,
+          excluding those notices that do not pertain to any part of
+          the Derivative Works; and
+
+      (d) If the Work includes a "NOTICE" text file as part of its
+          distribution, then any Derivative Works that You distribute must
+          include a readable copy of the attribution notices contained
+          within such NOTICE file, excluding those notices that do not
+          pertain to any part of the Derivative Works, in at least one
+          of the following places: within a NOTICE text file distributed
+          as part of the Derivative Works; within the Source form or
+          documentation, if provided along with the Derivative Works; or,
+          within a display generated by the Derivative Works, if and
+          wherever such third-party notices normally appear. The contents
+          of the NOTICE file are for informational purposes only and
+          do not modify the License. You may add Your own attribution
+          notices within Derivative Works that You distribute, alongside
+          or as an addendum to the NOTICE text from the Work, provided
+          that such additional attribution notices cannot be construed
+          as modifying the License.
+
+      You may add Your own copyright statement to Your modifications and
+      may provide additional or different license terms and conditions
+      for use, reproduction, or distribution of Your modifications, or
+      for any such Derivative Works as a whole, provided Your use,
+      reproduction, and distribution of the Work otherwise complies with
+      the conditions stated in this License.
+
+   5. Submission of Contributions. Unless You explicitly state otherwise,
+      any Contribution intentionally submitted for inclusion in the Work
+      by You to the Licensor shall be under the terms and conditions of
+      this License, without any additional terms or conditions.
+      Notwithstanding the above, nothing herein shall supersede or modify
+      the terms of any separate license agreement you may have executed
+      with Licensor regarding such Contributions.
+
+   6. Trademarks. This License does not grant permission to use the trade
+      names, trademarks, service marks, or product names of the Licensor,
+      except as required for reasonable and customary use in describing the
+      origin of the Work and reproducing the content of the NOTICE file.
+
+   7. Disclaimer of Warranty. Unless required by applicable law or
+      agreed to in writing, Licensor provides the Work (and each
+      Contributor provides its Contributions) on an "AS IS" BASIS,
+      WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
+      implied, including, without limitation, any warranties or conditions
+      of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
+      PARTICULAR PURPOSE. You are solely responsible for determining the
+      appropriateness of using or redistributing the Work and assume any
+      risks associated with Your exercise of permissions under this License.
+
+   8. Limitation of Liability. In no event and under no legal theory,
+      whether in tort (including negligence), contract, or otherwise,
+      unless required by applicable law (such as deliberate and grossly
+      negligent acts) or agreed to in writing, shall any Contributor be
+      liable to You for damages, including any direct, indirect, special,
+      incidental, or consequential damages of any character arising as a
+      result of this License or out of the use or inability to use the
+      Work (including but not limited to damages for loss of goodwill,
+      work stoppage, computer failure or malfunction, or any and all
+      other commercial damages or losses), even if such Contributor
+      has been advised of the possibility of such damages.
+
+   9. Accepting Warranty or Additional Liability. While redistributing
+      the Work or Derivative Works thereof, You may choose to offer,
+      and charge a fee for, acceptance of support, warranty, indemnity,
+      or other liability obligations and/or rights consistent with this
+      License. However, in accepting such obligations, You may act only
+      on Your own behalf and on Your sole responsibility, not on behalf
+      of any other Contributor, and only if You agree to indemnify,
+      defend, and hold each Contributor harmless for any liability
+      incurred by, or claims asserted against, such Contributor by reason
+      of your accepting any such warranty or additional liability.
+
+   END OF TERMS AND CONDITIONS
+
+   APPENDIX: How to apply the Apache License to your work.
+
+      To apply the Apache License to your work, attach the following
+      boilerplate notice, with the fields enclosed by brackets "[]"
+      replaced with your own identifying information. (Don't include
+      the brackets!)  The text should be enclosed in the appropriate
+      comment syntax for the file format. We also recommend that a
+      file or class name and description of purpose be included on the
+      same "printed page" as the copyright notice for easier
+      identification within third-party archives.
+
+   Copyright 2024 Jay Alammar & Maarten Grootendorst
+
+   Licensed under the Apache License, Version 2.0 (the "License");
+   you may not use this file except in compliance with the License.
+   You may obtain a copy of the License at
+
+       http://www.apache.org/licenses/LICENSE-2.0
+
+   Unless required by applicable law or agreed to in writing, software
+   distributed under the License is distributed on an "AS IS" BASIS,
+   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+   See the License for the specific language governing permissions and
+   limitations under the License.

README.md

@@ -0,0 +1,109 @@
+# Hands-On Large Language Models
+
+<a href="https://www.linkedin.com/in/jalammar/"><img src="https://img.shields.io/badge/Follow%20Jay-blue.svg?logo=linkedin"></a>
+<a href="https://www.linkedin.com/in/mgrootendorst/"><img src="https://img.shields.io/badge/Follow%20Maarten-blue.svg?logo=linkedin"></a>
+<a href="https://www.deeplearning.ai/short-courses/how-transformer-llms-work/?utm_campaign=handsonllm-launch&utm_medium=partner"><img src="https://img.shields.io/badge/DeepLearning.AI%20Course-NEW!-&labelColor=black&color=red.svg?logo=data:image/svg%2bxml;base64,PHN2ZyB4bWxucz0iaHR0cDovL3d3dy53My5vcmcvMjAwMC9zdmciIHZpZXdCb3g9IjAuMDAwMzY1MjgxIC0wLjAwMDE0MDE0MiAzMy4yOSAzMy4xNSI+Cgk8cGF0aCBkPSJNMTYuNjQzIDMzLjE0NWMtMy4yOTIgMC02LjUxLS45NzItOS4yNDYtMi43OTNhMTYuNTg4IDE2LjU4OCAwIDAxLTYuMTMtNy40MzhBMTYuNTA3IDE2LjUwNyAwIDAxLjMyIDEzLjM0YTE2LjU1IDE2LjU1IDAgMDE0LjU1NS04LjQ4NUExNi42NjUgMTYuNjY1IDAgMDExMy4zOTYuMzE4YTE2LjcxIDE2LjcxIDAgMDE5LjYxNi45NDQgMTYuNjI4IDE2LjYyOCAwIDAxNy40NyA2LjEwMyAxNi41MjIgMTYuNTIyIDAgMDEyLjgwNCA5LjIwN2MwIDQuMzk2LTEuNzUzIDguNjEtNC44NzQgMTEuNzE5YTE2LjY4IDE2LjY4IDAgMDEtMTEuNzY5IDQuODU0em0uMTI1LTYuNjI4YzYuOTA2IDAgMTIuNTE3LTUuNjk4IDEyLjUxNy0xMi43MyAwLTcuMDMtNS42MS0xMi43MjUtMTIuNTE3LTEyLjcyNS02LjkwNiAwLTEyLjUxNyA1LjY5OC0xMi41MTcgMTIuNzI1IDAgNy4wMjcgNS42MTEgMTIuNzMgMTIuNTE3IDEyLjczem0tLjEyNS0yLjkxOGMtNi4yODkgMC0xMS4zODYtNC45MjUtMTEuMzg2LTExLjAwMkM1LjI1NyA2LjUyIDEwLjM2IDEuNTkgMTYuNjQzIDEuNTljNi4yODQgMCAxMS4zODYgNC45MyAxMS4zODYgMTEuMDA3cy01LjA5NyAxMS4wMDItMTEuMzg2IDExLjAwMnptLS4yNDItNC41MDhjNC43NyAwIDguNjMzLTMuNjc5IDguNjMzLTguMjE4IDAtNC41MzgtMy44ODUtOC4yMjEtOC42MzMtOC4yMjEtNC43NDcgMC04LjYzMiAzLjY3OS04LjYzMiA4LjIyMSAwIDQuNTQzIDMuODg1IDguMjE4IDguNjMyIDguMjE4eiIgZmlsbD0iI0ZENEE2MSIvPgo8L3N2Zz4="></a>
+
+Welcome! In this repository you will find the code for all examples throughout the book [Hands-On Large Language Models](https://www.amazon.com/Hands-Large-Language-Models-Understanding/dp/1098150961) written by [Jay Alammar](https://www.linkedin.com/in/jalammar/) and [Maarten Grootendorst](https://www.linkedin.com/in/mgrootendorst/) which we playfully dubbed: <br> 
+
+<p align="center"><b><i>"The Illustrated LLM Book"</i></b></p>
+
+Through the visually educational nature of this book and with **almost 300 custom made figures**, learn the practical tools and concepts you need to use Large Language Models today!
+
+<a href="https://www.amazon.com/Hands-Large-Language-Models-Understanding/dp/1098150961"><img src="images/book_cover.png" width="50%" ></a>
+
+<br>
+
+The book is available on:
+
+* [Amazon](https://www.amazon.com/Hands-Large-Language-Models-Understanding/dp/1098150961)
+* [Shroff Publishers (India)](https://www.shroffpublishers.com/books/computer-science/large-language-models/9789355425522/)
+* [O'Reilly](https://www.oreilly.com/library/view/hands-on-large-language/9781098150952/)
+* [Kindle](https://www.amazon.com/Hands-Large-Language-Models-Alammar-ebook/dp/B0DGZ46G88/ref=tmm_kin_swatch_0?_encoding=UTF8&qid=&sr=)
+* [Barnes and Noble](https://www.barnesandnoble.com/w/hands-on-large-language-models-jay-alammar/1145185960)
+* [Goodreads](https://www.goodreads.com/book/show/210408850-hands-on-large-language-models)
+
+## Table of Contents
+
+We advise to run all examples through Google Colab for the easiest setup. Google Colab allows you to use a T4 GPU with 16GB of VRAM for free. All examples were mainly built and tested using Google Colab, so it should be the most stable platform. However, any other cloud provider should work. 
+
+| Chapter  | Notebook  |
+|---|---|
+| Chapter 1: Introduction to Language Models  | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/HandsOnLLM/Hands-On-Large-Language-Models/blob/main/chapter01/Chapter%201%20-%20Introduction%20to%20Language%20Models.ipynb)   |
+| Chapter 2: Tokens and Embeddings  | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/HandsOnLLM/Hands-On-Large-Language-Models/blob/main/chapter02/Chapter%202%20-%20Tokens%20and%20Token%20Embeddings.ipynb)  |
+| Chapter 3: Looking Inside Transformer LLMs  | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/HandsOnLLM/Hands-On-Large-Language-Models/blob/main/chapter03/Chapter%203%20-%20Looking%20Inside%20LLMs.ipynb)  |
+| Chapter 4: Text Classification  | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/HandsOnLLM/Hands-On-Large-Language-Models/blob/main/chapter04/Chapter%204%20-%20Text%20Classification.ipynb)  |
+| Chapter 5: Text Clustering and Topic Modeling  | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/HandsOnLLM/Hands-On-Large-Language-Models/blob/main/chapter05/Chapter%205%20-%20Text%20Clustering%20and%20Topic%20Modeling.ipynb)  |
+| Chapter 6: Prompt Engineering  | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/HandsOnLLM/Hands-On-Large-Language-Models/blob/main/chapter06/Chapter%206%20-%20Prompt%20Engineering.ipynb)  |
+| Chapter 7: Advanced Text Generation Techniques and Tools  | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/HandsOnLLM/Hands-On-Large-Language-Models/blob/main/chapter07/Chapter%207%20-%20Advanced%20Text%20Generation%20Techniques%20and%20Tools.ipynb)  |
+| Chapter 8: Semantic Search and Retrieval-Augmented Generation  | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/HandsOnLLM/Hands-On-Large-Language-Models/blob/main/chapter08/Chapter%208%20-%20Semantic%20Search.ipynb)  |
+| Chapter 9: Multimodal Large Language Models  | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/HandsOnLLM/Hands-On-Large-Language-Models/blob/main/chapter09/Chapter%209%20-%20Multimodal%20Large%20Language%20Models.ipynb)  |
+| Chapter 10: Creating Text Embedding Models  | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/HandsOnLLM/Hands-On-Large-Language-Models/blob/main/chapter10/Chapter%2010%20-%20Creating%20Text%20Embedding%20Models.ipynb)  |
+| Chapter 11: Fine-tuning Representation Models for Classification  | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/HandsOnLLM/Hands-On-Large-Language-Models/blob/main/chapter11/Chapter%2011%20-%20Fine-Tuning%20BERT.ipynb)  |
+| Chapter 12: Fine-tuning Generation Models  | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/HandsOnLLM/Hands-On-Large-Language-Models/blob/main/chapter12/Chapter%2012%20-%20Fine-tuning%20Generation%20Models.ipynb)  |
+
+> [!TIP]
+> You can check the [setup](.setup/) folder for a quick-start guide to install all packages locally and you can check the [conda](.setup/conda/) folder for a complete guide on how to setup your environment, including conda and PyTorch installation.
+> Note that the depending on your OS, Python version, and dependencies your results might be slightly differ. However, they
+> should this be similar to the examples in the book. 
+
+
+## Reviews
+
+> "*Jay and Maarten have continued their tradition of providing beautifully illustrated and insightful descriptions of complex topics in their new book. Bolstered with working code, timelines, and references to key papers, their book is a valuable resource for anyone looking to understand the main techniques behind how Large Language Models are built.*"
+>    
+> **Andrew Ng** - founder of [DeepLearning.AI](https://www.deeplearning.ai/)
+
+---
+
+> "*This is an exceptional guide to the world of language models and their practical applications in industry. Its highly-visual coverage of generative, representational, and retrieval applications of language models empowers readers to quickly understand, use, and refine LLMs. Highly recommended!*"
+>
+> **Nils Reimers** - Director of Machine Learning at Cohere | creator of [sentence-transformers](https://github.com/UKPLab/sentence-transformers)
+
+---
+
+> "*I can’t think of another book that is more important to read right now. On every single page, I learned something that is critical to success in this era of language models.*"
+> 
+> **Josh Starmer** - [StatQuest](https://www.youtube.com/channel/UCtYLUTtgS3k1Fg4y5tAhLbw)
+
+---
+
+> "*If you’re looking to get up to speed in everything regarding LLMs, look no further! In this wonderful book, Jay and Maarten will take you from zero to expert in the history and latest advances in large language models. With very intuitive explanations, great real-life examples, clear illustrations, and comprehensive code labs, this book lifts the curtain on the complexities of transformer models, tokenizers, semantic search, RAG, and many other cutting-edge technologies. A must read for anyone interested in the latest AI technology!*"
+> 
+> **Luis Serrano, PhD** - Founder and CEO of [Serrano Academy](https://www.youtube.com/@SerranoAcademy)
+
+---
+
+> "*Hands-On Large Language Models brings clarity and practical examples to cut through the hype of AI. It provides a wealth of great diagrams and visual aids to supplement the clear explanations. The worked examples and code make concrete what other books leave abstract. The book starts with simple introductory beginnings, and steadily builds in scope. By the final chapters, you will be fine-tuning and building your own large language models with confidence.*"
+>
+> **Leland McInnes** - Researcher at the Tutte Institute for Mathematics and Computing | creator of [UMAP](https://github.com/lmcinnes/umap) and [HDBSCAN](https://github.com/scikit-learn-contrib/hdbscan)
+
+---
+
+## [Bonus content!](bonus/)
+
+We attempted to put as much information into the book without it being overwhelming. However, even with a 400-page book there is still much to discover! 
+
+We continue to create more guides that compliment the book and go more in-depth into new and [exciting topics]((bonus/)):
+
+| [A Visual Guide to Mamba](https://newsletter.maartengrootendorst.com/p/a-visual-guide-to-mamba-and-state)             |  [A Visual Guide to Quantization](https://newsletter.maartengrootendorst.com/p/a-visual-guide-to-quantization) | [The Illustrated Stable Diffusion](https://jalammar.github.io/illustrated-stable-diffusion/) |
+:-------------------------:|:-------------------------:|:-------------------------:
+![](images/mamba.png)  |  ![](images/quant.png) |  ![](images/diffusion.png)
+**[A Visual Guide to Mixture of Experts](https://newsletter.maartengrootendorst.com/p/a-visual-guide-to-mixture-of-experts)**  | **[A Visual Guide to Reasoning LLMs](https://newsletter.maartengrootendorst.com/p/a-visual-guide-to-reasoning-llms)**  |  **[The Illustrated DeepSeek-R1](https://newsletter.languagemodels.co/p/the-illustrated-deepseek-r1)**
+![](images/moe.png)  |  ![](images/reasoning.png) |  ![](images/deepseek.png)
+
+## Citation
+
+Please consider citing the book if you consider it useful for your research:
+
+```
+@book{hands-on-llms-book,
+  author       = {Jay Alammar and Maarten Grootendorst},
+  title        = {Hands-On Large Language Models},
+  publisher    = {O'Reilly},
+  year         = {2024},
+  isbn         = {978-1098150969},
+  url          = {https://www.oreilly.com/library/view/hands-on-large-language/9781098150952/},
+  github       = {https://github.com/HandsOnLLM/Hands-On-Large-Language-Models}
+}
+```

README.wehub.md

@@ -0,0 +1,7 @@
+# WeHub 来源说明
+
+- 原始项目：`HandsOnLLM/Hands-On-Large-Language-Models`
+- 原始仓库：https://github.com/HandsOnLLM/Hands-On-Large-Language-Models
+- 导入方式：上游默认分支的最新快照
+- 原作者、版权和许可证信息以原始仓库及本仓库 LICENSE 为准
+- 本文件仅用于记录来源，不代表 WeHub 是原项目作者

bonus/2_deeplearningai.md

@@ -0,0 +1,14 @@
+# [How Transformer LLMs Work](https://www.deeplearning.ai/short-courses/how-transformer-llms-work/?utm_campaign=handsonllm-launch&utm_medium=partner)
+
+In this course, we take the content from the book and enhance it through a short course on the main components of what makes a Transformer LLM. This **highly animated** course, will further enhance the intuition you get from the book's content.
+ 
+<p align="center">
+    <a href="https://www.deeplearning.ai/short-courses/how-transformer-llms-work/?utm_campaign=handsonllm-launch&utm_medium=partner"><img src="../images/dlai.png" width="50%" ></a>
+</p>
+
+## What you'll learn
+
+* Gain an understanding of the key components of transformers, including tokenization, embeddings, self-attention, and transformer blocks, to build a strong technical foundation.
+* Understand recent transformer improvements to the attention mechanism such as KV cache, multi-query attention, grouped query attention, and sparse attention.
+* Compare tokenization strategies used in modern LLMS and explore transformers in the Hugging Face Transformers library.
+

bonus/README.md

@@ -0,0 +1,28 @@
+# Bonus Material
+
+With the incredible growth of Language AI in recent years, capturing everything in a single book (even with 400 pages!) 
+is near impossible. That does not mean one should not try and come up with a creative solution to this problem of size and growth.
+
+We decided to cover the fundamentals of LLMs in the book which left us with an interesting opportunity. 
+Using the book as a starting point, we could continue creating visual/illustrative content that explore 
+certain topics in-depth that could not be covered that way in the book.
+
+<p align="center"><i>All bonus materials enhance the book through the same <b>visual</b> and <b>illustrative</b> style of the book</i></p>
+
+After reading the book, you are ready to go through these more complex topics through highly visual, detailed, and in-depth guides. Because you are already familiar with our illustrative styles, reading through these advanced concepts should be a breeze!
+
+<p align="center">
+<a href="https://www.amazon.com/Hands-Large-Language-Models-Understanding/dp/1098150961"><img src="../images/bonus_overview.png" width="100%"></a>
+</p>
+
+To get a feeling of each piece of additional content, click any of the markdown files in the folder to get a short description of what it is about, or you can go directly to these visual educative articles:
+
+1. [**Hands-On Large Language Models**](https://www.amazon.com/Hands-Large-Language-Models-Understanding/dp/1098150961)
+2. [How **Transformer LLMs** Work](https://www.deeplearning.ai/short-courses/how-transformer-llms-work/?utm_campaign=handsonllm-launch&utm_medium=partner)
+3. [A Visual Guide to **Quantization**](https://newsletter.maartengrootendorst.com/p/a-visual-guide-to-quantization)
+4. [A Visual Guide to **Mamba and State Space Models**](https://newsletter.maartengrootendorst.com/p/a-visual-guide-to-mamba-and-state)
+5. [A Visual Guide to **Mixture of Experts** (MoE)](https://newsletter.maartengrootendorst.com/p/a-visual-guide-to-mixture-of-experts)
+6. [The Illustrated **Stable Diffusion**](https://jalammar.github.io/illustrated-stable-diffusion/)
+7. [A Visual Guide to **Reasoning LLMs**](https://newsletter.maartengrootendorst.com/p/a-visual-guide-to-reasoning-llms)
+8. [The Illustrated **DeepSeek-R1**](https://newsletter.languagemodels.co/p/the-illustrated-deepseek-r1)
+9. [A Visual Guide to **LLM Agents**](https://newsletter.maartengrootendorst.com/p/a-visual-guide-to-llm-agents)

chapter01/Chapter 1 - Introduction to Language Models.ipynb

@@ -0,0 +1,183 @@
+{
+ "cells": [
+  {
+   "cell_type": "markdown",
+   "metadata": {
+    "id": "EDe7DsPWmEBV"
+   },
+   "source": [
+    "<h1>Chapter 1 - Introduction to Language Models</h1>\n",
+    "<i>Exploring the exciting field of Language AI</i>\n",
+    "\n",
+    "\n",
+    "<a href=\"https://www.amazon.com/Hands-Large-Language-Models-Understanding/dp/1098150961\"><img src=\"https://img.shields.io/badge/Buy%20the%20Book!-grey?logo=amazon\"></a>\n",
+    "<a href=\"https://www.oreilly.com/library/view/hands-on-large-language/9781098150952/\"><img src=\"https://img.shields.io/badge/O'Reilly-white.svg?logo=data:image/svg%2bxml;base64,PHN2ZyB3aWR0aD0iMzQiIGhlaWdodD0iMjciIHZpZXdCb3g9IjAgMCAzNCAyNyIgZmlsbD0ibm9uZSIgeG1sbnM9Imh0dHA6Ly93d3cudzMub3JnLzIwMDAvc3ZnIj4KPGNpcmNsZSBjeD0iMTMiIGN5PSIxNCIgcj0iMTEiIHN0cm9rZT0iI0Q0MDEwMSIgc3Ryb2tlLXdpZHRoPSI0Ii8+CjxjaXJjbGUgY3g9IjMwLjUiIGN5PSIzLjUiIHI9IjMuNSIgZmlsbD0iI0Q0MDEwMSIvPgo8L3N2Zz4K\"></a>\n",
+    "<a href=\"https://github.com/HandsOnLLM/Hands-On-Large-Language-Models\"><img src=\"https://img.shields.io/badge/GitHub%20Repository-black?logo=github\"></a>\n",
+    "[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/HandsOnLLM/Hands-On-Large-Language-Models/blob/main/chapter01/Chapter%201%20-%20Introduction%20to%20Language%20Models.ipynb)\n",
+    "\n",
+    "---\n",
+    "\n",
+    "This notebook is for Chapter 1 of the [Hands-On Large Language Models](https://www.amazon.com/Hands-Large-Language-Models-Understanding/dp/1098150961) book by [Jay Alammar](https://www.linkedin.com/in/jalammar) and [Maarten Grootendorst](https://www.linkedin.com/in/mgrootendorst/).\n",
+    "\n",
+    "---\n",
+    "\n",
+    "<a href=\"https://www.amazon.com/Hands-Large-Language-Models-Understanding/dp/1098150961\">\n",
+    "<img src=\"https://raw.githubusercontent.com/HandsOnLLM/Hands-On-Large-Language-Models/main/images/book_cover.png\" width=\"350\"/></a>\n"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "### [OPTIONAL] - Installing Packages on <img src=\"https://colab.google/static/images/icons/colab.png\" width=100>\n",
+    "\n",
+    "If you are viewing this notebook on Google Colab (or any other cloud vendor), you need to **uncomment and run** the following codeblock to install the dependencies for this chapter:"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "---\n",
+    "\n",
+    "💡 **NOTE**: We will want to use a GPU to run the examples in this notebook. In Google Colab, go to\n",
+    "**Runtime > Change runtime type > Hardware accelerator > GPU > GPU type > T4**.\n",
+    "\n",
+    "---"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "# %%capture\n",
+    "# !pip install transformers==4.41.2 accelerate==0.31.0"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {
+    "id": "hXp09JFsFBXi"
+   },
+   "source": [
+    "# Phi-3\n",
+    "\n",
+    "The first step is to load our model onto the GPU for faster inference. Note that we load the model and tokenizer separately (although that isn't always necessary)."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {
+    "id": "RSNalRXZyTTk"
+   },
+   "outputs": [],
+   "source": [
+    "from transformers import AutoModelForCausalLM, AutoTokenizer\n",
+    "\n",
+    "# Load model and tokenizer\n",
+    "model = AutoModelForCausalLM.from_pretrained(\n",
+    "    \"microsoft/Phi-3-mini-4k-instruct\",\n",
+    "    device_map=\"cuda\",\n",
+    "    torch_dtype=\"auto\",\n",
+    "    trust_remote_code=False,\n",
+    ")\n",
+    "tokenizer = AutoTokenizer.from_pretrained(\"microsoft/Phi-3-mini-4k-instruct\")"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {
+    "id": "qdyYYS0E5fEU"
+   },
+   "source": [
+    "Although we can now use the model and tokenizer directly, it's much easier to wrap it in a `pipeline` object:"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 2,
+   "metadata": {
+    "id": "DiUi4Wu1FCyN"
+   },
+   "outputs": [],
+   "source": [
+    "from transformers import pipeline\n",
+    "\n",
+    "# Create a pipeline\n",
+    "generator = pipeline(\n",
+    "    \"text-generation\",\n",
+    "    model=model,\n",
+    "    tokenizer=tokenizer,\n",
+    "    return_full_text=False,\n",
+    "    max_new_tokens=500,\n",
+    "    do_sample=False\n",
+    ")"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {
+    "id": "mD49kysT5mMY"
+   },
+   "source": [
+    "Finally, we create our prompt as a user and give it to the model:"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 4,
+   "metadata": {
+    "id": "hkR7LBmiyXmY"
+   },
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      " Why did the chicken join the band? Because it had the drumsticks!\n"
+     ]
+    }
+   ],
+   "source": [
+    "# The prompt (user input / query)\n",
+    "messages = [\n",
+    "    {\"role\": \"user\", \"content\": \"Create a funny joke about chickens.\"}\n",
+    "]\n",
+    "\n",
+    "# Generate output\n",
+    "output = generator(messages)\n",
+    "print(output[0][\"generated_text\"])"
+   ]
+  }
+ ],
+ "metadata": {
+  "accelerator": "GPU",
+  "colab": {
+   "authorship_tag": "ABX9TyPCWg08aO4e8NWQuYCK5ppF",
+   "gpuType": "T4",
+   "provenance": []
+  },
+  "kernelspec": {
+   "display_name": "Python 3 (ipykernel)",
+   "language": "python",
+   "name": "python3"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 3
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython3",
+   "version": "3.12.9"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 4
+}

chapter03/Chapter 3 - Looking Inside LLMs.ipynb

@@ -0,0 +1,896 @@
+{
+ "cells": [
+  {
+   "cell_type": "markdown",
+   "metadata": {
+    "id": "adFzzFsB-Ofl"
+   },
+   "source": [
+    "<h1>Chapter 3 - Looking Inside Transformer LLMs</h1>\n",
+    "<i>An extensive look into the transformer architecture of generative LLMs</i>\n",
+    "\n",
+    "<a href=\"https://www.amazon.com/Hands-Large-Language-Models-Understanding/dp/1098150961\"><img src=\"https://img.shields.io/badge/Buy%20the%20Book!-grey?logo=amazon\"></a>\n",
+    "<a href=\"https://www.oreilly.com/library/view/hands-on-large-language/9781098150952/\"><img src=\"https://img.shields.io/badge/O'Reilly-white.svg?logo=data:image/svg%2bxml;base64,PHN2ZyB3aWR0aD0iMzQiIGhlaWdodD0iMjciIHZpZXdCb3g9IjAgMCAzNCAyNyIgZmlsbD0ibm9uZSIgeG1sbnM9Imh0dHA6Ly93d3cudzMub3JnLzIwMDAvc3ZnIj4KPGNpcmNsZSBjeD0iMTMiIGN5PSIxNCIgcj0iMTEiIHN0cm9rZT0iI0Q0MDEwMSIgc3Ryb2tlLXdpZHRoPSI0Ii8+CjxjaXJjbGUgY3g9IjMwLjUiIGN5PSIzLjUiIHI9IjMuNSIgZmlsbD0iI0Q0MDEwMSIvPgo8L3N2Zz4K\"></a>\n",
+    "<a href=\"https://github.com/HandsOnLLM/Hands-On-Large-Language-Models\"><img src=\"https://img.shields.io/badge/GitHub%20Repository-black?logo=github\"></a>\n",
+    "[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/HandsOnLLM/Hands-On-Large-Language-Models/blob/main/chapter03/Chapter%203%20-%20Looking%20Inside%20LLMs.ipynb)\n",
+    "\n",
+    "---\n",
+    "\n",
+    "This notebook is for Chapter 3 of the [Hands-On Large Language Models](https://www.amazon.com/Hands-Large-Language-Models-Understanding/dp/1098150961) book by [Jay Alammar](https://www.linkedin.com/in/jalammar) and [Maarten Grootendorst](https://www.linkedin.com/in/mgrootendorst/).\n",
+    "\n",
+    "---\n",
+    "\n",
+    "<a href=\"https://www.amazon.com/Hands-Large-Language-Models-Understanding/dp/1098150961\">\n",
+    "<img src=\"https://raw.githubusercontent.com/HandsOnLLM/Hands-On-Large-Language-Models/main/images/book_cover.png\" width=\"350\"/></a>"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "### [OPTIONAL] - Installing Packages on <img src=\"https://colab.google/static/images/icons/colab.png\" width=100>\n",
+    "\n",
+    "If you are viewing this notebook on Google Colab (or any other cloud vendor), you need to **uncomment and run** the following codeblock to install the dependencies for this chapter:\n",
+    "\n",
+    "---\n",
+    "\n",
+    "💡 **NOTE**: We will want to use a GPU to run the examples in this notebook. In Google Colab, go to\n",
+    "**Runtime > Change runtime type > Hardware accelerator > GPU > GPU type > T4**.\n",
+    "\n",
+    "---\n"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "# %%capture\n",
+    "# !pip install transformers>=4.41.2 accelerate>=0.31.0"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {
+    "id": "W_23Z_do-faF"
+   },
+   "source": [
+    "# Loading the LLM"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {
+    "colab": {
+     "base_uri": "https://localhost:8080/",
+     "height": 759,
+     "referenced_widgets": [
+      "5ecafba0f8f04685a56d2b1495baea24",
+      "53945d03a26044878ddb7fc6eadfd8db",
+      "99db408011ea43c79adbd4a880839484",
+      "fd181746067945febc55f94e2dcf6f67",
+      "c5acd3e22bbe4a1897f7a12051e8eae9",
+      "446d084fde5f422a8c0525e8c5b47f93",
+      "6a0dabe02c874ecabd257c5da5f4a7c5",
+      "1ee8996717864f79af6cf0314cd27c59",
+      "7c719ce3694845b384ad4ae7207d31cc",
+      "989fc06dfb25420eaf155bfc0174c692",
+      "443850bc37d94a90aa037e73a77f9369",
+      "184bbd6daf424ddeba7f8cbe0b5b34d9",
+      "a6ee07658a234a54b38d849ff2017d6c",
+      "bf63e0a23fa147778dd88b48075611e3",
+      "acf8b6868c484885871964cc538359c6",
+      "32cc3b668de84e8a9f8cde25d3846822",
+      "b1115e5097c4404694e5a7c228e25234",
+      "6ca20198bc144510a14b795aaf813940",
+      "982a35743f51448cba583e54e7d3987d",
+      "17250e2791ba42099c50efa594e229ce",
+      "a4c438b7029f47d9a54d5a0ce16541c0",
+      "fafd3d7714d4466ea00f354b20dc954a",
+      "0a18c83d2645496797d74aef5e84dafa",
+      "bfbbea9912104cb1803c7c827c1cea7d",
+      "18c23f29824e4cb58e461bf89811a32a",
+      "74105769e6a541098769921b84d1f1cd",
+      "52109f76852e4388b6df4e5441c43a48",
+      "4f39386aea7c4ac9abf2d66291cbcb4e",
+      "4893546c2d6f4eec9cb5b7f6853013f0",
+      "88616761593a4b2b9365294427a3136d",
+      "d3ab3ad192f54d5b9f9e08db62884f0c",
+      "6be54d3b31864134bc36d3b9d997530a",
+      "27a1df02460948e19cbd40b69a89bec6",
+      "6b81bad9c639454980de2a67b414b988",
+      "1e8224a73a724058b28a13db2c2197c2",
+      "3cbbd1ae4e4d4bf1a34c9d960ee7d26d",
+      "e9c0309e4af44726b180c977692e2469",
+      "97421a19ce25438e846a84b45321f9d0",
+      "c6aa0ad56ce44b1eab1356d3bc56706b",
+      "87d0758fdaaa4d90b21b1291f4d20039",
+      "8c85660c9e9f4027bd64d596160a7d7f",
+      "fce8b8595211452f8a759a6e98410f6a",
+      "4249df3055d4427493e2a2e775dc1a93",
+      "e42e39002e66410b9d714656a632adea",
+      "3cf79ac9541c4b7bbf686b604ff73b81",
+      "b709537463c3485dbbcef93e3636af2d",
+      "cf7c24d7629b4362ae0905f8e4bbd997",
+      "d9f9f98cb9dd465daed1da06d7d4084b",
+      "cb20a4ccd9ea4548bc7f694068bb30f6",
+      "b91d534a7a5c4863a9e858cedbda9fd5",
+      "a65601c4918c4c2cb2020837cd1e1f85",
+      "99505d538b894a359bc3791cd95423d5",
+      "1a48245eddfc4923a5f267cd799ba9c7",
+      "6c50ddb227b4421da5cd391e4d6ec94c",
+      "7d62b5c9c90644378b0ca96cee430419",
+      "9ec56d025f8446a08184b055fe11598e",
+      "54894d44acce4fe6b0ef749d5c02e3cd",
+      "e2dd956536a0407fb9b9c4a01c01ba9c",
+      "54370883565644c5a2529e092db7f259",
+      "7ec24f1bff5f4ca78d7c36a637cfa294",
+      "36ff492b5c7c4ff3bcf72cc574d03a38",
+      "8279cee867884166bb09df4e02635e2a",
+      "a2644dee82b14cfbab069a484f3841e2",
+      "e761a9afe1b847579fb51eb0eddd4488",
+      "c59d9e04e5964a8891cc3ddce33d6f86",
+      "a9007e7552ab4634ac44577279b242ac",
+      "2cff15834149418c81eee5239a5e275a",
+      "6623f077d95f4faf883fa2ca4397169d",
+      "98529c941229460da8563b94d3419c37",
+      "3bef1bf002c94340a3323592d616e7ac",
+      "47c8fd6b7c9844d983014e07f1999cd2",
+      "aa24ef9c6e1d49708b7bb4755a9adb78",
+      "28261270cd0449f8ac85dc4b0efdac57",
+      "a231962de8584f69a6a107c275d1cda5",
+      "2d35dfe75a744987ab210f5fb0118301",
+      "1cbaa3bcf20b4af099f6d4310dd071dd",
+      "4f7088db853e47e6b6a6ccd654e379dc",
+      "7e0710dc5b5c4002b4bcc189bf5514cf",
+      "66dd13ca8234409eab15ecbf7a009914",
+      "8f021e2bfb7246248baa49b08f4d3358",
+      "0a91788829fc49ef95a69efd3256a8e3",
+      "b42a56f9b2e44b6992413e023ed44b0e",
+      "f9fc98e3d8ed4338bd763d152f8cc5f9",
+      "f786e0117562476697092ef828ceb1b2",
+      "07fed042c0894ca5aebe717eca6f3018",
+      "3bea08fef3ef456e9f180d8a23ded5bc",
+      "35f3727f37c44640986d8416141f8069",
+      "0dd1ce8a2306431c9b0412e5992f4f84",
+      "dcea31213c9f421abc7ffabc3499ecb9",
+      "61643fa2fcb54ebaad8da5106dec9ea0",
+      "0c4f8c58d213493494120c070d86ac76",
+      "e15e9978db794891b9fa0d8ce096c983",
+      "b15098d69b5f418da3f81aba8fb79de0",
+      "f7cf40042a2e4c8cb8b87444893d8ec1",
+      "6dcbc6ec46d44a55a1b38267201926b8",
+      "6de4dd6e36b444638b2f65e0ca80bc9a",
+      "60febb04a12447c192e1b8eb2aa5ba28",
+      "6d200e6b1bdc4e918093670df8e37dc8",
+      "934e950a3c0c48f08802d551ee1bd429",
+      "34058533e3cf46a88db8927372102b9f",
+      "2a7ba0f87814436386e66f4ef7f1111d",
+      "d3e41286c5b747a8bb5cf326f9f80ad3",
+      "fe92dc2d5e8c449bbb36dafbd6c9935f",
+      "615d4ec0fb194688a392b71b99cf3621",
+      "e2056e3dba884242807a98b9b3837843",
+      "6ca50fb814b64fccaf0e1c6c11d8f4d8",
+      "9ec1d173921748b2af19a3a21df9ed40",
+      "f8c3d566e98d47239ef2b823544b75a5",
+      "3618dcec35a740c485ecafa5589e0c91",
+      "f66dd730f3364e35974a3918d12ff51d",
+      "dcb9240335394bfa8d3949ef1cdbcdf8",
+      "2df88b2fd5e242eca8b1d3f6cea1349b",
+      "1b103c69baf74fceb551ebcb5a0ac5e8",
+      "6847b7b6b3854d6e9b72f40040f84c8f",
+      "ac3e67f03f604883ab4787930cd316f3",
+      "6d7e012a7fbf4d788054ead5020e9314",
+      "a432e2e32c7c4e56b138ebaefad76c93",
+      "352fe4ea215240149d73478e34cd9b66",
+      "bf9bc31d7f99477bb391f69df41b8dbe",
+      "41b8e463309e4013a50268677f44d4b5",
+      "989326b74cd146e3b5b2f2d5f19bdf41",
+      "36b7269d8eb849b084694fd1f3b177b9",
+      "2d09f9ed15fa452ba8d9ce9aba9f61ac",
+      "53818082d62749a28552a1eebd304d88",
+      "766f67cb6a8a4a58965edb671ce624e8",
+      "a86d22308dbf4c11bf3d6f6515aef561",
+      "e9e7d944715b402ab149f86862b92259",
+      "4aaa732bb1b94b4895ca3f00f93cd762",
+      "9fb02b3bbe79434a93f32291c208aaad",
+      "dd9b3a5e84ba44cb9717ded470c258b4",
+      "fa89524d446b480aa50d203d01ec7bb7",
+      "c8b77256d5fc436fbfdcc150843a6b5b",
+      "f5f5b592768048169676e09cca453645",
+      "77f40b8bf30c437ba987b71178d0e9f6",
+      "3f39ec300bd84852a2388dadaafd8c4b",
+      "250cac43e6da47dd8d732ea57d8c50ec",
+      "f929d12aad68458b98c21e0669da3d8e",
+      "44ddbcadcc4c477c80daf278122de46d",
+      "eb606db4125e4eb097d5b7d3cdb90976",
+      "b6c177de60b54edd887d1ca983ea7546",
+      "6db56d7c52244a3984a0638e060a81cb",
+      "11d17ae63dd44ecc8813d482ee17dd95",
+      "ff3733c6a1f34580b037e296e3abed6b",
+      "2d31d51641e945f695f7315b68e0ad2e",
+      "1b694930328e46bd9e0d61063b9141d4",
+      "b544b6f2c2bb4f36bf9a983005a8bdb8",
+      "afb8b0e602b649fcb92634c8aab4caf7",
+      "24fc703c916f43aab5900288b8aa5aca",
+      "16cffa93ab234718a8ede1044596e8b2",
+      "2d7999217424413d988cd29d41ed5ace",
+      "0d1546581c90418fa1cfc37491339134",
+      "0e233853a3b74211a0b65dcdd001feed",
+      "a1d2163af40a4aaab8a765b148845807",
+      "2aaa722b303b4e1f823cee828fc7958c"
+     ]
+    },
+    "executionInfo": {
+     "elapsed": 130259,
+     "status": "ok",
+     "timestamp": 1718959891215,
+     "user": {
+      "displayName": "Maarten Grootendorst",
+      "userId": "11015108362723620659"
+     },
+     "user_tz": -120
+    },
+    "id": "-5RLd6dI-Ytm",
+    "outputId": "fb085ff7-e06f-4142-8e95-5ff98b212e37"
+   },
+   "outputs": [
+    {
+     "name": "stderr",
+     "output_type": "stream",
+     "text": [
+      "/usr/local/lib/python3.10/dist-packages/huggingface_hub/utils/_token.py:89: UserWarning: \n",
+      "The secret `HF_TOKEN` does not exist in your Colab secrets.\n",
+      "To authenticate with the Hugging Face Hub, create a token in your settings tab (https://huggingface.co/settings/tokens), set it as secret in your Google Colab and restart your session.\n",
+      "You will be able to reuse this secret in all of your notebooks.\n",
+      "Please note that authentication is recommended but still optional to access public models or datasets.\n",
+      "  warnings.warn(\n"
+     ]
+    },
+    {
+     "data": {
+      "application/vnd.jupyter.widget-view+json": {
+       "model_id": "5ecafba0f8f04685a56d2b1495baea24",
+       "version_major": 2,
+       "version_minor": 0
+      },
+      "text/plain": [
+       "tokenizer_config.json:   0%|          | 0.00/3.17k [00:00<?, ?B/s]"
+      ]
+     },
+     "metadata": {},
+     "output_type": "display_data"
+    },
+    {
+     "data": {
+      "application/vnd.jupyter.widget-view+json": {
+       "model_id": "184bbd6daf424ddeba7f8cbe0b5b34d9",
+       "version_major": 2,
+       "version_minor": 0
+      },
+      "text/plain": [
+       "tokenizer.model:   0%|          | 0.00/500k [00:00<?, ?B/s]"
+      ]
+     },
+     "metadata": {},
+     "output_type": "display_data"
+    },
+    {
+     "data": {
+      "application/vnd.jupyter.widget-view+json": {
+       "model_id": "0a18c83d2645496797d74aef5e84dafa",
+       "version_major": 2,
+       "version_minor": 0
+      },
+      "text/plain": [
+       "tokenizer.json:   0%|          | 0.00/1.84M [00:00<?, ?B/s]"
+      ]
+     },
+     "metadata": {},
+     "output_type": "display_data"
+    },
+    {
+     "data": {
+      "application/vnd.jupyter.widget-view+json": {
+       "model_id": "6b81bad9c639454980de2a67b414b988",
+       "version_major": 2,
+       "version_minor": 0
+      },
+      "text/plain": [
+       "added_tokens.json:   0%|          | 0.00/293 [00:00<?, ?B/s]"
+      ]
+     },
+     "metadata": {},
+     "output_type": "display_data"
+    },
+    {
+     "data": {
+      "application/vnd.jupyter.widget-view+json": {
+       "model_id": "3cf79ac9541c4b7bbf686b604ff73b81",
+       "version_major": 2,
+       "version_minor": 0
+      },
+      "text/plain": [
+       "special_tokens_map.json:   0%|          | 0.00/568 [00:00<?, ?B/s]"
+      ]
+     },
+     "metadata": {},
+     "output_type": "display_data"
+    },
+    {
+     "name": "stderr",
+     "output_type": "stream",
+     "text": [
+      "Special tokens have been added in the vocabulary, make sure the associated word embeddings are fine-tuned or trained.\n"
+     ]
+    },
+    {
+     "data": {
+      "application/vnd.jupyter.widget-view+json": {
+       "model_id": "9ec56d025f8446a08184b055fe11598e",
+       "version_major": 2,
+       "version_minor": 0
+      },
+      "text/plain": [
+       "config.json:   0%|          | 0.00/931 [00:00<?, ?B/s]"
+      ]
+     },
+     "metadata": {},
+     "output_type": "display_data"
+    },
+    {
+     "data": {
+      "application/vnd.jupyter.widget-view+json": {
+       "model_id": "2cff15834149418c81eee5239a5e275a",
+       "version_major": 2,
+       "version_minor": 0
+      },
+      "text/plain": [
+       "configuration_phi3.py:   0%|          | 0.00/10.4k [00:00<?, ?B/s]"
+      ]
+     },
+     "metadata": {},
+     "output_type": "display_data"
+    },
+    {
+     "name": "stderr",
+     "output_type": "stream",
+     "text": [
+      "A new version of the following files was downloaded from https://huggingface.co/microsoft/Phi-3-mini-4k-instruct:\n",
+      "- configuration_phi3.py\n",
+      ". Make sure to double-check they do not contain any added malicious code. To avoid downloading new versions of the code file, you can pin a revision.\n"
+     ]
+    },
+    {
+     "data": {
+      "application/vnd.jupyter.widget-view+json": {
+       "model_id": "7e0710dc5b5c4002b4bcc189bf5514cf",
+       "version_major": 2,
+       "version_minor": 0
+      },
+      "text/plain": [
+       "modeling_phi3.py:   0%|          | 0.00/73.8k [00:00<?, ?B/s]"
+      ]
+     },
+     "metadata": {},
+     "output_type": "display_data"
+    },
+    {
+     "name": "stderr",
+     "output_type": "stream",
+     "text": [
+      "A new version of the following files was downloaded from https://huggingface.co/microsoft/Phi-3-mini-4k-instruct:\n",
+      "- modeling_phi3.py\n",
+      ". Make sure to double-check they do not contain any added malicious code. To avoid downloading new versions of the code file, you can pin a revision.\n",
+      "WARNING:transformers_modules.microsoft.Phi-3-mini-4k-instruct.ff07dc01615f8113924aed013115ab2abd32115b.modeling_phi3:`flash-attention` package not found, consider installing for better performance: No module named 'flash_attn'.\n",
+      "WARNING:transformers_modules.microsoft.Phi-3-mini-4k-instruct.ff07dc01615f8113924aed013115ab2abd32115b.modeling_phi3:Current `flash-attention` does not support `window_size`. Either upgrade or use `attn_implementation='eager'`.\n"
+     ]
+    },
+    {
+     "data": {
+      "application/vnd.jupyter.widget-view+json": {
+       "model_id": "dcea31213c9f421abc7ffabc3499ecb9",
+       "version_major": 2,
+       "version_minor": 0
+      },
+      "text/plain": [
+       "model.safetensors.index.json:   0%|          | 0.00/16.3k [00:00<?, ?B/s]"
+      ]
+     },
+     "metadata": {},
+     "output_type": "display_data"
+    },
+    {
+     "data": {
+      "application/vnd.jupyter.widget-view+json": {
+       "model_id": "34058533e3cf46a88db8927372102b9f",
+       "version_major": 2,
+       "version_minor": 0
+      },
+      "text/plain": [
+       "Downloading shards:   0%|          | 0/2 [00:00<?, ?it/s]"
+      ]
+     },
+     "metadata": {},
+     "output_type": "display_data"
+    },
+    {
+     "data": {
+      "application/vnd.jupyter.widget-view+json": {
+       "model_id": "dcb9240335394bfa8d3949ef1cdbcdf8",
+       "version_major": 2,
+       "version_minor": 0
+      },
+      "text/plain": [
+       "model-00001-of-00002.safetensors:   0%|          | 0.00/4.97G [00:00<?, ?B/s]"
+      ]
+     },
+     "metadata": {},
+     "output_type": "display_data"
+    },
+    {
+     "data": {
+      "application/vnd.jupyter.widget-view+json": {
+       "model_id": "36b7269d8eb849b084694fd1f3b177b9",
+       "version_major": 2,
+       "version_minor": 0
+      },
+      "text/plain": [
+       "model-00002-of-00002.safetensors:   0%|          | 0.00/2.67G [00:00<?, ?B/s]"
+      ]
+     },
+     "metadata": {},
+     "output_type": "display_data"
+    },
+    {
+     "data": {
+      "application/vnd.jupyter.widget-view+json": {
+       "model_id": "f5f5b592768048169676e09cca453645",
+       "version_major": 2,
+       "version_minor": 0
+      },
+      "text/plain": [
+       "Loading checkpoint shards:   0%|          | 0/2 [00:00<?, ?it/s]"
+      ]
+     },
+     "metadata": {},
+     "output_type": "display_data"
+    },
+    {
+     "data": {
+      "application/vnd.jupyter.widget-view+json": {
+       "model_id": "2d31d51641e945f695f7315b68e0ad2e",
+       "version_major": 2,
+       "version_minor": 0
+      },
+      "text/plain": [
+       "generation_config.json:   0%|          | 0.00/172 [00:00<?, ?B/s]"
+      ]
+     },
+     "metadata": {},
+     "output_type": "display_data"
+    }
+   ],
+   "source": [
+    "from transformers import AutoModelForCausalLM, AutoTokenizer, pipeline\n",
+    "\n",
+    "# Load model and tokenizer\n",
+    "tokenizer = AutoTokenizer.from_pretrained(\"microsoft/Phi-3-mini-4k-instruct\")\n",
+    "\n",
+    "model = AutoModelForCausalLM.from_pretrained(\n",
+    "    \"microsoft/Phi-3-mini-4k-instruct\",\n",
+    "    device_map=\"cuda\",\n",
+    "    torch_dtype=\"auto\",\n",
+    "    trust_remote_code=False,\n",
+    ")\n",
+    "\n",
+    "# Create a pipeline\n",
+    "generator = pipeline(\n",
+    "    \"text-generation\",\n",
+    "    model=model,\n",
+    "    tokenizer=tokenizer,\n",
+    "    return_full_text=False,\n",
+    "    max_new_tokens=50,\n",
+    "    do_sample=False,\n",
+    ")"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {
+    "id": "REqcz-ID_XgV"
+   },
+   "source": [
+    "# The Inputs and Outputs of a Trained Transformer LLM\n"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {
+    "colab": {
+     "base_uri": "https://localhost:8080/"
+    },
+    "executionInfo": {
+     "elapsed": 4955,
+     "status": "ok",
+     "timestamp": 1718959896168,
+     "user": {
+      "displayName": "Maarten Grootendorst",
+      "userId": "11015108362723620659"
+     },
+     "user_tz": -120
+    },
+    "id": "17h6TPHluJ-i",
+    "outputId": "18727eeb-ccd6-40f8-aab1-25c8d9a03cbe"
+   },
+   "outputs": [
+    {
+     "name": "stderr",
+     "output_type": "stream",
+     "text": [
+      "WARNING:transformers_modules.microsoft.Phi-3-mini-4k-instruct.ff07dc01615f8113924aed013115ab2abd32115b.modeling_phi3:You are not running the flash-attention implementation, expect numerical differences.\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "\n",
+      "\n",
+      "\n",
+      "Solution 1:\n",
+      "\n",
+      "Subject: My Sincere Apologies for the Gardening Mishap\n",
+      "\n",
+      "\n",
+      "Dear Sarah,\n",
+      "\n",
+      "\n",
+      "I hope this message finds you well. I am writing to express my deep\n"
+     ]
+    }
+   ],
+   "source": [
+    "prompt = \"Write an email apologizing to Sarah for the tragic gardening mishap. Explain how it happened.\"\n",
+    "\n",
+    "output = generator(prompt)\n",
+    "\n",
+    "print(output[0]['generated_text'])"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {
+    "colab": {
+     "base_uri": "https://localhost:8080/"
+    },
+    "executionInfo": {
+     "elapsed": 1,
+     "status": "ok",
+     "timestamp": 1718959898745,
+     "user": {
+      "displayName": "Maarten Grootendorst",
+      "userId": "11015108362723620659"
+     },
+     "user_tz": -120
+    },
+    "id": "eoFkdTd6_g5o",
+    "outputId": "bdcfde9f-28b7-4f43-ec0c-32c16677a776"
+   },
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Phi3ForCausalLM(\n",
+      "  (model): Phi3Model(\n",
+      "    (embed_tokens): Embedding(32064, 3072, padding_idx=32000)\n",
+      "    (embed_dropout): Dropout(p=0.0, inplace=False)\n",
+      "    (layers): ModuleList(\n",
+      "      (0-31): 32 x Phi3DecoderLayer(\n",
+      "        (self_attn): Phi3Attention(\n",
+      "          (o_proj): Linear(in_features=3072, out_features=3072, bias=False)\n",
+      "          (qkv_proj): Linear(in_features=3072, out_features=9216, bias=False)\n",
+      "          (rotary_emb): Phi3RotaryEmbedding()\n",
+      "        )\n",
+      "        (mlp): Phi3MLP(\n",
+      "          (gate_up_proj): Linear(in_features=3072, out_features=16384, bias=False)\n",
+      "          (down_proj): Linear(in_features=8192, out_features=3072, bias=False)\n",
+      "          (activation_fn): SiLU()\n",
+      "        )\n",
+      "        (input_layernorm): Phi3RMSNorm()\n",
+      "        (resid_attn_dropout): Dropout(p=0.0, inplace=False)\n",
+      "        (resid_mlp_dropout): Dropout(p=0.0, inplace=False)\n",
+      "        (post_attention_layernorm): Phi3RMSNorm()\n",
+      "      )\n",
+      "    )\n",
+      "    (norm): Phi3RMSNorm()\n",
+      "  )\n",
+      "  (lm_head): Linear(in_features=3072, out_features=32064, bias=False)\n",
+      ")\n"
+     ]
+    }
+   ],
+   "source": [
+    "print(model)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {
+    "id": "RTrwzB67BYVY"
+   },
+   "source": [
+    "# Choosing a single token from the probability distribution (sampling / decoding)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {
+    "id": "sEcxYgJxBYbJ"
+   },
+   "outputs": [],
+   "source": [
+    "prompt = \"The capital of France is\"\n",
+    "\n",
+    "# Tokenize the input prompt\n",
+    "input_ids = tokenizer(prompt, return_tensors=\"pt\").input_ids\n",
+    "\n",
+    "# Tokenize the input prompt\n",
+    "input_ids = input_ids.to(\"cuda\")\n",
+    "\n",
+    "# Get the output of the model before the lm_head\n",
+    "model_output = model.model(input_ids)\n",
+    "\n",
+    "# Get the output of the lm_head\n",
+    "lm_head_output = model.lm_head(model_output[0])"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {
+    "colab": {
+     "base_uri": "https://localhost:8080/",
+     "height": 36
+    },
+    "executionInfo": {
+     "elapsed": 421,
+     "status": "ok",
+     "timestamp": 1718960391623,
+     "user": {
+      "displayName": "Maarten Grootendorst",
+      "userId": "11015108362723620659"
+     },
+     "user_tz": -120
+    },
+    "id": "68YUSS4GBf9Q",
+    "outputId": "2dc25e8d-03b6-4bca-b46c-fec3e3a4a492"
+   },
+   "outputs": [
+    {
+     "data": {
+      "application/vnd.google.colaboratory.intrinsic+json": {
+       "type": "string"
+      },
+      "text/plain": [
+       "'Paris'"
+      ]
+     },
+     "execution_count": 6,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "token_id = lm_head_output[0,-1].argmax(-1)\n",
+    "tokenizer.decode(token_id)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {
+    "colab": {
+     "base_uri": "https://localhost:8080/"
+    },
+    "executionInfo": {
+     "elapsed": 901,
+     "status": "ok",
+     "timestamp": 1718960415287,
+     "user": {
+      "displayName": "Maarten Grootendorst",
+      "userId": "11015108362723620659"
+     },
+     "user_tz": -120
+    },
+    "id": "cWWrfC5oBjwp",
+    "outputId": "c2fdeab7-e787-466f-88f4-988cd5f939a6"
+   },
+   "outputs": [
+    {
+     "data": {
+      "text/plain": [
+       "torch.Size([1, 6, 3072])"
+      ]
+     },
+     "execution_count": 7,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "model_output[0].shape"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {
+    "colab": {
+     "base_uri": "https://localhost:8080/"
+    },
+    "executionInfo": {
+     "elapsed": 1079,
+     "status": "ok",
+     "timestamp": 1718960424560,
+     "user": {
+      "displayName": "Maarten Grootendorst",
+      "userId": "11015108362723620659"
+     },
+     "user_tz": -120
+    },
+    "id": "nC1PdOnTBnxZ",
+    "outputId": "1fd5f482-7046-4536-b745-4e681d6ecdaf"
+   },
+   "outputs": [
+    {
+     "data": {
+      "text/plain": [
+       "torch.Size([1, 6, 32064])"
+      ]
+     },
+     "execution_count": 8,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "lm_head_output.shape"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {
+    "id": "Of2_rP4QBqrZ"
+   },
+   "source": [
+    "# Speeding up generation by caching keys and values\n"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {
+    "id": "B0n6JhNHBrin"
+   },
+   "outputs": [],
+   "source": [
+    "prompt = \"Write a very long email apologizing to Sarah for the tragic gardening mishap. Explain how it happened.\"\n",
+    "\n",
+    "# Tokenize the input prompt\n",
+    "input_ids = tokenizer(prompt, return_tensors=\"pt\").input_ids\n",
+    "input_ids = input_ids.to(\"cuda\")"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {
+    "colab": {
+     "base_uri": "https://localhost:8080/"
+    },
+    "executionInfo": {
+     "elapsed": 47155,
+     "status": "ok",
+     "timestamp": 1718960517928,
+     "user": {
+      "displayName": "Maarten Grootendorst",
+      "userId": "11015108362723620659"
+     },
+     "user_tz": -120
+    },
+    "id": "BwIvt6jSByAF",
+    "outputId": "e71c4141-2ca3-488a-fdfb-8d9357af0125"
+   },
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "6.66 s ± 2.22 s per loop (mean ± std. dev. of 7 runs, 1 loop each)\n"
+     ]
+    }
+   ],
+   "source": [
+    "%%timeit -n 1\n",
+    "# Generate the text\n",
+    "generation_output = model.generate(\n",
+    "  input_ids=input_ids,\n",
+    "  max_new_tokens=100,\n",
+    "  use_cache=True\n",
+    ")"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {
+    "colab": {
+     "base_uri": "https://localhost:8080/"
+    },
+    "executionInfo": {
+     "elapsed": 152674,
+     "status": "ok",
+     "timestamp": 1718960670601,
+     "user": {
+      "displayName": "Maarten Grootendorst",
+      "userId": "11015108362723620659"
+     },
+     "user_tz": -120
+    },
+    "id": "dFb1dcvJByCW",
+    "outputId": "0aba6a01-9bc7-40b7-e2e1-e064f13b4c88"
+   },
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "21.9 s ± 94.6 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)\n"
+     ]
+    }
+   ],
+   "source": [
+    "%%timeit -n 1\n",
+    "# Generate the text\n",
+    "generation_output = model.generate(\n",
+    "  input_ids=input_ids,\n",
+    "  max_new_tokens=100,\n",
+    "  use_cache=False\n",
+    ")"
+   ]
+  }
+ ],
+ "metadata": {
+  "accelerator": "GPU",
+  "colab": {
+   "gpuType": "T4",
+   "provenance": []
+  },
+  "kernelspec": {
+   "display_name": "Python 3 (ipykernel)",
+   "language": "python",
+   "name": "python3"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 3
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython3",
+   "version": "3.12.9"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 4
+}

environment.yml

@@ -0,0 +1,248 @@
+name: thellmbook
+channels:
+  - conda-forge
+  - defaults
+dependencies:
+  - bzip2=1.0.8
+  - ca-certificates=2024.8.30
+  - libffi=3.4.2
+  - libsqlite=3.46.1
+  - libzlib=1.3.1
+  - openssl=3.3.2
+  - pip=24.2
+  - python=3.10.14
+  - setuptools=73.0.1
+  - tk=8.6.13
+  - ucrt=10.0.22621.0
+  - vc=14.3
+  - vc14_runtime=14.40.33810
+  - vs2015_runtime=14.40.33810
+  - wheel=0.44.0
+  - xz=5.2.6
+  - pip:
+      - accelerate==0.31.0
+      - aiohappyeyeballs==2.4.0
+      - aiohttp==3.10.5
+      - aiosignal==1.3.1
+      - annotated-types==0.7.0
+      - annoy==1.17.3
+      - anyio==4.4.0
+      - argon2-cffi==23.1.0
+      - argon2-cffi-bindings==21.2.0
+      - arrow==1.3.0
+      - asttokens==2.4.1
+      - async-lru==2.0.4
+      - async-timeout==4.0.3
+      - attrs==24.2.0
+      - babel==2.16.0
+      - beautifulsoup4==4.12.3
+      - bertopic==0.16.3
+      - bitsandbytes==0.43.1
+      - bleach==6.1.0
+      - boto3==1.35.15
+      - botocore==1.35.15
+      - certifi==2024.8.30
+      - cffi==1.17.1
+      - charset-normalizer==3.3.2
+      - click==8.1.7
+      - cloudpickle==3.0.0
+      - cohere==5.5.8
+      - colorama==0.4.6
+      - colorcet==3.1.0
+      - colorspacious==1.1.2
+      - comm==0.2.2
+      - contourpy==1.3.0
+      - cycler==0.12.1
+      - dask==2024.8.2
+      - dataclasses-json==0.6.7
+      - datamapplot==0.3.0
+      - datasets==2.20.0
+      - datashader==0.16.3
+      - debugpy==1.8.5
+      - decorator==5.1.1
+      - defusedxml==0.7.1
+      - dill==0.3.8
+      - diskcache==5.6.3
+      - distro==1.9.0
+      - docstring-parser==0.16
+      - duckduckgo-search==6.1.6
+      - eval-type-backport==0.2.0
+      - evaluate==0.4.2
+      - exceptiongroup==1.2.2
+      - executing==2.1.0
+      - fastavro==1.9.7
+      - fastjsonschema==2.20.0
+      - filelock==3.16.0
+      - fonttools==4.53.1
+      - fqdn==1.5.1
+      - frozenlist==1.4.1
+      - fsspec==2024.5.0
+      - gensim==4.3.2
+      - greenlet==3.0.3
+      - h11==0.14.0
+      - hdbscan==0.8.38.post1
+      - httpcore==1.0.5
+      - httpx==0.27.2
+      - httpx-sse==0.4.0
+      - huggingface-hub==0.24.6
+      - idna==3.8
+      - imageio==2.35.1
+      - importlib-metadata==8.4.0
+      - intel-openmp==2021.4.0
+      - ipykernel==6.29.5
+      - ipython==8.27.0
+      - ipywidgets==8.1.3
+      - isoduration==20.11.0
+      - jedi==0.19.1
+      - jinja2==3.1.4
+      - jmespath==1.0.1
+      - joblib==1.4.2
+      - json5==0.9.25
+      - jsonpatch==1.33
+      - jsonpointer==3.0.0
+      - jsonschema==4.23.0
+      - jsonschema-specifications==2023.12.1
+      - jupyter-client==8.6.2
+      - jupyter-core==5.7.2
+      - jupyter-events==0.10.0
+      - jupyter-lsp==2.2.5
+      - jupyter-server==2.14.2
+      - jupyter-server-terminals==0.5.3
+      - jupyterlab==4.2.2
+      - jupyterlab-pygments==0.3.0
+      - jupyterlab-server==2.27.3
+      - jupyterlab-widgets==3.0.13
+      - kiwisolver==1.4.7
+      - langchain==0.2.5
+      - langchain-community==0.2.5
+      - langchain-core==0.2.38
+      - langchain-openai==0.1.8
+      - langchain-text-splitters==0.2.4
+      - langsmith==0.1.117
+      - lazy-loader==0.4
+      - llama-cpp-python==0.2.78
+      - llvmlite==0.43.0
+      - locket==1.0.0
+      - markdown-it-py==3.0.0
+      - markupsafe==2.1.5
+      - marshmallow==3.22.0
+      - matplotlib==3.9.0
+      - matplotlib-inline==0.1.7
+      - mdurl==0.1.2
+      - mistune==3.0.2
+      - mkl==2021.4.0
+      - mpmath==1.3.0
+      - mteb==1.12.39
+      - multidict==6.1.0
+      - multipledispatch==1.0.0
+      - multiprocess==0.70.16
+      - mypy-extensions==1.0.0
+      - nbclient==0.10.0
+      - nbconvert==7.16.4
+      - nbformat==5.10.4
+      - nest-asyncio==1.6.0
+      - networkx==3.3
+      - nltk==3.8.1
+      - notebook-shim==0.2.4
+      - numba==0.60.0
+      - numexpr==2.10.0
+      - numpy==1.26.4
+      - openai==1.34.0
+      - orjson==3.10.7
+      - overrides==7.7.0
+      - packaging==24.1
+      - pandas==2.2.2
+      - pandocfilters==1.5.1
+      - param==2.1.1
+      - parameterized==0.9.0
+      - parso==0.8.4
+      - partd==1.4.2
+      - peft==0.11.1
+      - pillow==10.4.0
+      - platformdirs==4.3.2
+      - plotly==5.24.0
+      - polars==1.6.0
+      - prometheus-client==0.20.0
+      - prompt-toolkit==3.0.47
+      - psutil==6.0.0
+      - pure-eval==0.2.3
+      - pyarrow==17.0.0
+      - pyarrow-hotfix==0.6
+      - pycparser==2.22
+      - pyct==0.5.0
+      - pydantic==2.9.1
+      - pydantic-core==2.23.3
+      - pygments==2.18.0
+      - pylabeladjust==0.1.13
+      - pynndescent==0.5.13
+      - pyparsing==3.1.4
+      - pyqtree==1.0.0
+      - pyreqwest-impersonate==0.5.3
+      - python-dateutil==2.9.0.post0
+      - python-json-logger==2.0.7
+      - pytrec-eval-terrier==0.5.6
+      - pytz==2024.1
+      - pywin32==306
+      - pywinpty==2.0.13
+      - pyyaml==6.0.2
+      - pyzmq==26.2.0
+      - referencing==0.35.1
+      - regex==2024.7.24
+      - requests==2.32.3
+      - rfc3339-validator==0.1.4
+      - rfc3986-validator==0.1.1
+      - rich==13.8.1
+      - rpds-py==0.20.0
+      - s3transfer==0.10.2
+      - safetensors==0.4.5
+      - scikit-image==0.24.0
+      - scikit-learn==1.5.0
+      - send2trash==1.8.3
+      - sentence-transformers==3.0.1
+      - sentencepiece==0.2.0
+      - seqeval==1.2.2
+      - setfit==1.0.3
+      - shtab==1.7.1
+      - six==1.16.0
+      - smart-open==7.0.4
+      - sniffio==1.3.1
+      - soupsieve==2.6
+      - sqlalchemy==2.0.34
+      - stack-data==0.6.3
+      - sympy==1.13.2
+      - tbb==2021.13.1
+      - tenacity==8.5.0
+      - terminado==0.18.1
+      - threadpoolctl==3.5.0
+      - tifffile==2024.8.30
+      - tiktoken==0.7.0
+      - tinycss2==1.3.0
+      - tokenizers==0.19.1
+      - tomli==2.0.1
+      - toolz==0.12.1
+      - torch==2.3.1
+      - tornado==6.4.1
+      - tqdm==4.66.5
+      - traitlets==5.14.3
+      - transformers==4.41.2
+      - trl==0.9.4
+      - types-python-dateutil==2.9.0.20240906
+      - types-requests==2.32.0.20240907
+      - typing-extensions==4.12.2
+      - typing-inspect==0.9.0
+      - tyro==0.8.10
+      - tzdata==2024.1
+      - umap-learn==0.5.6
+      - uri-template==1.3.0
+      - urllib3==2.2.2
+      - wcwidth==0.2.13
+      - webcolors==24.8.0
+      - webencodings==0.5.1
+      - websocket-client==1.8.0
+      - widgetsnbextension==4.0.13
+      - wrapt==1.16.0
+      - xarray==2024.7.0
+      - xxhash==3.5.0
+      - yarl==1.11.1
+      - zipp==3.20.1
+prefix: C:\ProgramData\miniconda3\envs\thellmbook

requirements.txt

@@ -0,0 +1,43 @@
+# Data handling
+numpy == 1.26.4
+pandas == 2.2.2
+datasets == 2.20.0
+
+# Environment
+jupyterlab == 4.2.2
+ipywidgets == 8.1.3
+
+# Hard dependencies throughout the entire book
+torch == 2.3.1
+transformers == 4.41.2
+sentence-transformers == 3.0.1
+matplotlib == 3.9.0
+scikit-learn == 1.5.0
+sentencepiece == 0.2.0
+nltk == 3.8.1
+evaluate == 0.4.2
+scipy >= 1.15.0
+
+# Cloud providers
+openai == 1.34.0
+cohere == 5.5.8
+
+# Chapter-specific dependencies
+datamapplot == 0.3.0
+faiss-cpu==1.8.0
+bertopic == 0.16.3
+annoy == 1.17.3
+llama_cpp_python == 0.2.78 -C cmake.args="-DLLAMA_BLAS=ON"
+numexpr == 2.10.0
+langchain == 0.2.5
+langchain-community == 0.2.5
+langchain-openai==0.1.8
+duckduckgo-search == 7.1.1
+gensim == 4.3.2
+setfit == 1.0.3
+seqeval == 1.2.2
+trl == 0.9.4
+peft == 0.11.1
+accelerate == 0.31.0
+bitsandbytes == 0.43.1
+mteb == 1.12.39

requirements_min.txt

@@ -0,0 +1,43 @@
+# Data handling
+numpy >= 1.26.4
+pandas >= 2.2.2
+datasets >= 2.20.0
+
+# Environment
+jupyterlab >= 4.2.2
+ipywidgets >= 8.1.3
+
+# Hard dependencies throughout the entire book
+torch >= 2.3.1
+transformers >= 4.41.2
+sentence-transformers >= 3.0.1
+matplotlib >= 3.9.0
+scikit-learn >= 1.5.0
+sentencepiece >= 0.2.0
+nltk >= 3.8.1
+evaluate >= 0.4.2
+scipy >= 1.15.0
+
+# Cloud providers
+openai >= 1.34.0
+cohere >= 5.5.8
+
+# Chapter-specific dependencies
+datamapplot >= 0.3.0
+faiss-cpu >= 1.8.0
+bertopic >= 0.16.3       
+annoy >= 1.17.3              
+llama_cpp_python >= 0.2.78 -C cmake.args="-DLLAMA_BLAS=ON"
+numexpr >= 2.10.0
+langchain >= 0.2.5
+langchain-community >= 0.2.5
+langchain-openai >= 0.1.8
+duckduckgo-search >= 6.1.6
+gensim >= 4.3.2             
+setfit >= 1.0.3              
+seqeval >= 1.2.2           
+trl >= 0.9.4                
+peft >= 0.11.1               
+accelerate >= 0.31.0       
+bitsandbytes >= 0.43.1       
+mteb >= 1.12.39