wehub-resource-sync
67 lines
22 KiB
Plaintext
67 lines
22 KiB
Plaintext
{
|
|
"cells": [
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
""
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"# **Using externally-hosted LLMs**\n",
|
|
"Use llama_cookbook.inference.llm to perform inference using Llama and other models using third party services. At the moment, three services have been incorporated:\n",
|
|
"- Together.ai\n",
|
|
"- Anyscale\n",
|
|
"- OpenAI\n",
|
|
"\n",
|
|
"An API token for each service must be obtained and provided to the method before running. "
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"from llama_cookbook.inference.llm import TOGETHER, OPENAI, ANYSCALE\n",
|
|
"\n",
|
|
"together_example = TOGETHER(\"togethercomputer/llama-2-7b-chat\",\"09e45...\")\n",
|
|
"print( together_example.query(prompt=\"Why is the sky blue?\"))\n",
|
|
"\n",
|
|
"\n",
|
|
"openai_example = OPENAI(\"gpt-3.5-turbo\",\"sk-LIz9zL3cYp...\")\n",
|
|
"print( openai_example.query(prompt=\"Why is the sky blue?\"))\n",
|
|
"\n",
|
|
"\n",
|
|
"anyscale_example = ANYSCALE(\"meta-llama/Llama-2-7b-chat-hf\",\"esecret_c3u4x7...\")\n",
|
|
"print( anyscale_example.query(prompt=\"Why is the sky blue?\"))"
|
|
]
|
|
}
|
|
],
|
|
"metadata": {
|
|
"custom": {
|
|
"cells": [],
|
|
"metadata": {
|
|
"fileHeader": "",
|
|
"fileUid": "9af50647-0f34-423b-936e-6950218a612f",
|
|
"isAdHoc": false,
|
|
"language_info": {
|
|
"name": "plaintext"
|
|
},
|
|
"orig_nbformat": 4
|
|
},
|
|
"nbformat": 4,
|
|
"nbformat_minor": 2
|
|
},
|
|
"indentAmount": 2,
|
|
"language_info": {
|
|
"name": "python"
|
|
}
|
|
},
|
|
"nbformat": 4,
|
|
"nbformat_minor": 2
|
|
}
|