{
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "StkY5oHGU-iN"
      },
      "source": [
        "# LLMWare Model Exploration\n",
        "\n",
        "## This is the 'entrypoint' example that provides a general introduction of llmware models.\n",
        "\n",
        "This notebook provides an introduction to LLMWare Agentic AI models and demonstrates their usage."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "collapsed": true,
        "id": "KyaEnPzOVTJe"
      },
      "outputs": [],
      "source": [
        "# install dependencies\n",
        "!pip3 install llmware"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "mcOxXgs1XTjD"
      },
      "source": [
        "If you have any dependency install issues, please review the README, docs link, or raise an Issue.\n",
        "\n",
        "Usually, if there is a missing dependency, the code will give the warning - and a clear direction like `pip install transformers'` required for this example, etc.\n",
        "\n",
        "As an alternative to pip install ... if you prefer, you can also clone the repo from github which provides a benefit of having access to 100+ examples.\n",
        "\n",
        "To clone the repo:\n",
        "```\n",
        "git clone \"https://www.github.com/llmware-ai/llmware.git\"\n",
        "sh \"welcome_to_llmware.sh\"\n",
        "```\n",
        "\n",
        "The second script `\"welcome_to_llmware.sh\"` will install all of the dependencies.\n",
        "\n",
        "If using Windows, then use the `\"welcome_to_llmware_windows.sh\"` script."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "n4aKjcEiVjYE"
      },
      "outputs": [],
      "source": [
        "# Import Library\n",
        "from llmware.models import ModelCatalog"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ePtRGBIlZEkP"
      },
      "source": [
        "## GETTING STARTED WITH AGENTIC AI\n",
        "All LLMWare models are accessible through the ModelCatalog generally consisting of two steps to access any model\n",
        "\n",
        "- Step 1 - load the model - pulls from global repo the first time, and then automatically caches locally\n",
        "- Step 2 - use the model with inference or function call"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "collapsed": true,
        "id": "D0xL5WOgVlGX"
      },
      "outputs": [],
      "source": [
        "# 'Standard' Models use 'inference' and take a general text input and provide a general text output\n",
        "\n",
        "model = ModelCatalog().load_model(\"bling-answer-tool\")\n",
        "response = model.inference(\"My son is 21 years old.\\nHow old is my son?\")\n",
        "\n",
        "print(\"\\nresponse: \", response)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "collapsed": true,
        "id": "1AkSZ3Z_VqWt"
      },
      "outputs": [],
      "source": [
        "#   Optional parameters can improve results\n",
        "model = ModelCatalog().load_model(\"bling-phi-3-gguf\", temperature=0.0,sample=False, max_output=200)\n",
        "\n",
        "#   all LLMWare models have been fine-tuned to assume that the input will include a text passage, and that the\n",
        "#   model's main job is to 'read' the passage, and then 'answer' a question based on that information\n",
        "\n",
        "text_passage = \"The company's stock price increased by $3 after reporting positive earnings.\"\n",
        "prompt = \"What was the increase in the stock price?\"\n",
        "\n",
        "response = model.inference(prompt,add_context=text_passage)\n",
        "\n",
        "print(\"\\nresponse: \", response)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "OuNEktB-aPVw"
      },
      "source": [
        "## Models we have and support\n",
        "Inference models can also be integrated into Prompts - which provide advanced handling for integrating with knowledge retrieval, managing source information, and providing fact-checking\n",
        "\n",
        "Discovering other models is easy -> to invoke a model, simply use the `'model_name'` and pass in `.load_model()`.\n",
        "\n",
        "***note***: *model_names starting with `'bling'`, `'dragon'`, and `'slim'` are llmware models.*\n",
        "- we do **include other popular models** such as `phi-3`, `qwen-2`, `yi`, `llama-3`, `mistral`\n",
        "- it is easy to extend the model catalog to **include other 3rd party models**, including `ollama` and `lm studio`.\n",
        "- we do **support** `open ai`, `anthropic`, `cohere` and `google api` models as well."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "collapsed": true,
        "id": "erEHenbjaYqi"
      },
      "outputs": [],
      "source": [
        "all_generative_models = ModelCatalog().list_generative_local_models()\n",
        "print(\"\\n\\nModel Catalog - load model with ModelCatalog().load_model(model_name)\")\n",
        "for i, model in enumerate(all_generative_models):\n",
        "\n",
        "    model_name = model[\"model_name\"]\n",
        "    model_family = model[\"model_family\"]\n",
        "\n",
        "    print(\"model: \", i, model)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "tLCuxZcYdTHn"
      },
      "source": [
        "## Slim Models\n",
        "Slim models are 'Function Calling' Models that perform a specialized task and output python dictionaries\n",
        "- by design, slim models are specialists that **perform single function**.\n",
        "- by design, slim models generally **do not require any specific** `'prompt instructions'`, but will often accept a `\"parameter\"` which is passed to the function."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "collapsed": true,
        "id": "1ZS2wo8zdDOd"
      },
      "outputs": [],
      "source": [
        "model = ModelCatalog().load_model(\"slim-sentiment-tool\")\n",
        "response = model.function_call(\"That was the worst earnings call ever - what a disaster.\")\n",
        "\n",
        "#   the 'overall' model response is just a python dictionary\n",
        "print(\"\\nresponse: \", response)\n",
        "print(\"llm_response: \", response['llm_response'])\n",
        "print(\"sentiment: \", response['llm_response']['sentiment'])"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "Ohp-shGjkDkz"
      },
      "outputs": [],
      "source": [
        "#   here is one of the slim model applied against a common earnings extract\n",
        "\n",
        "text_passage = (\"Here’s what Costco reported for its fiscal second quarter of 2024 compared with what Wall Street \"\n",
        "                \"was expecting, based on a survey of analysts by LSEG, formerly known as Refinitiv: Earnings \"\n",
        "                \"per share: $3.92 vs. $3.62 expected.  Revenue: $58.44 billion vs. $59.16 billion expected \"\n",
        "                \"In the three-month period that ended Feb. 18, Costco’s net income rose to $1.74 billion, or \"\n",
        "                \"$3.92 per share, compared with $1.47 billion, or $3.30 per share, a year earlier. \")\n",
        "\n",
        "#   extract model takes a 'key' for a parameter, and looks for the 'value' in the text\n",
        "model = ModelCatalog().load_model(\"slim-extract-tool\")\n",
        "\n",
        "#   the general structure of a function call includes a text passage input, a function and parameters\n",
        "response = model.function_call(text_passage,function=\"extract\",params=[\"revenue\"])\n",
        "\n",
        "print(\"\\nextract response: \", response)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "w13LLW_OdxCm"
      },
      "outputs": [],
      "source": [
        "#   Function calling models generally come with a test set that is a great way to learn how they work\n",
        "#   please note that each test can take a few minutes with 20-40 test questions\n",
        "\n",
        "# You can try rest of them yourself by removing comment(s) of the below catalog.\n",
        "ModelCatalog().tool_test_run(\"slim-topics-tool\")\n",
        "ModelCatalog().tool_test_run(\"slim-tags-tool\")\n",
        "ModelCatalog().tool_test_run(\"slim-emotions-tool\")\n",
        "ModelCatalog().tool_test_run(\"slim-summary-tool\")\n",
        "ModelCatalog().tool_test_run(\"slim-xsum-tool\")\n",
        "ModelCatalog().tool_test_run(\"slim-boolean-tool\")"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "kOPly8bfdnan"
      },
      "source": [
        "## Agentic AI\n",
        "Function calling models can be integrated into Agent processes which can orchestrate processes comprising multiple models and steps - most of our use cases will use the function calling models in that context\n",
        "\n",
        "## Last note:\n",
        "Most of the models are packaged as `\"gguf\"` usually identified as GGUFGenerativeModel, or with `'-gguf'` or `'-tool` at the end of their name.   These models are optimized to run most efficiently on a CPU-based laptop (especially Mac OS).   You can also try the standard Pytorch versions of these models, which should yield virtually identical results, but will be slower."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "rvLVgWYMe6RO"
      },
      "source": [
        "## Journey is yet to start!\n",
        "Loved it?? This is just an example of our models. Please check out our other Agentic AI examples with every model in detail here: https://github.com/llmware-ai/llmware/tree/main/fast_start/agents\n",
        "\n",
        "Also, if you have more interest in RAG, then please go with our RAG examples, which you can find here: https://github.com/llmware-ai/llmware/tree/main/fast_start/rag\n",
        "\n",
        "If you liked it, then please **star our repo https://github.com/llmware-ai/llmware** ⭐\n",
        "\n",
        "Any doubts?? Join our **discord server: https://discord.gg/GN49aWx2H3** 🫂"
      ]
    }
  ],
  "metadata": {
    "colab": {
      "provenance": []
    },
    "kernelspec": {
      "display_name": "Python 3",
      "name": "python3"
    },
    "language_info": {
      "name": "python"
    }
  },
  "nbformat": 4,
  "nbformat_minor": 0
}