wehub/run-llama--llama_index
1
0
Fork 0
Code Issues Pull Requests Actions Deployments Models Agent Notes Packages Projects Releases Wiki Activity
Files
main
run-llama--llama_index/docs/examples/property_graph/Dynamic_KG_Extraction.ipynb
T
wehub-resource-sync a0c8464e58
Build Package / build (ubuntu-latest) (push) Failing after 1s
Details
CodeQL / Analyze (python) (push) Failing after 1s
Details
Core Typecheck / core-typecheck (push) Failing after 1s
Details
Linting / lint (push) Failing after 1s
Details
llama-dev tests / test-llama-dev (push) Failing after 1s
Details
Publish Sub-Package to PyPI if Needed / publish_subpackage_if_needed (push) Has been skipped
Details
Sync Docs to Developer Hub / sync-docs (push) Failing after 0s
Details
Build Package / build (windows-latest) (push) Has been cancelled
Details
chore: import upstream snapshot with attribution
2026-07-13 12:26:52 +08:00

543 lines
26 KiB
Plaintext
Raw Permalink Blame History

{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Comparing LLM Path Extractors for Knowledge Graph Construction\n",
"\n",
"In this notebook, we'll compare three different LLM Path Extractors from llama_index:\n",
"1. SimpleLLMPathExtractor\n",
"2. SchemaLLMPathExtractor\n",
"3. DynamicLLMPathExtractor (New)\n",
"\n",
"We'll use a Wikipedia page as our test data and visualize the resulting knowledge graphs using Pyvis."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Setup and Imports"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"!pip install llama_index pyvis wikipedia"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"from llama_index.core import Document, PropertyGraphIndex\n",
"from llama_index.core.indices.property_graph import (\n",
" SimpleLLMPathExtractor,\n",
" SchemaLLMPathExtractor,\n",
" DynamicLLMPathExtractor,\n",
")\n",
"from llama_index.llms.openai import OpenAI\n",
"from llama_index.core import Settings\n",
"\n",
"import wikipedia\n",
"\n",
"import os"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"import nest_asyncio\n",
"\n",
"nest_asyncio.apply()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Set up LLM Backend"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"os.environ[\"OPENAI_API_KEY\"] = \"sk-proj-...\"\n",
"\n",
"# Set up global configurations\n",
"llm = OpenAI(temperature=0.0, model=\"gpt-3.5-turbo\")\n",
"\n",
"Settings.llm = llm\n",
"Settings.chunk_size = 2048\n",
"Settings.chunk_overlap = 20"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Fetch Some Raw Text from Wikipedia"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"def get_wikipedia_content(title):\n",
" try:\n",
" page = wikipedia.page(title)\n",
" return page.content\n",
" except wikipedia.exceptions.DisambiguationError as e:\n",
" print(f\"Disambiguation page. Options: {e.options}\")\n",
" except wikipedia.exceptions.PageError:\n",
" print(f\"Page '{title}' does not exist.\")\n",
" return None"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Fetched content for 'Barack Obama' (length: 83977 characters)\n"
]
}
],
"source": [
"wiki_title = \"Barack Obama\"\n",
"content = get_wikipedia_content(wiki_title)\n",
"\n",
"if content:\n",
" document = Document(text=content, metadata={\"title\": wiki_title})\n",
" print(\n",
" f\"Fetched content for '{wiki_title}' (length: {len(content)} characters)\"\n",
" )\n",
"else:\n",
" print(\"Failed to fetch Wikipedia content.\")"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## 1. SimpleLLMPathExtractor"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [
{
"data": {
"application/vnd.jupyter.widget-view+json": {
"model_id": "26d142b755d94e109cf1dfcbebb4e689",
"version_major": 2,
"version_minor": 0
},
"text/plain": [
"Parsing nodes: 0%| | 0/1 [00:00<?, ?it/s]"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"Extracting paths from text: 100%|██████████| 11/11 [00:09<00:00, 1.19it/s]\n"
]
},
{
"data": {
"text/plain": [
"[(EntityNode(label='entity', embedding=None, properties={'title': 'Barack Obama', 'triplet_source_id': 'c4bbe9b8-ccd0-464c-b34c-37ede77f2717'}, name='Obama'),\n",
" Relation(label='Has', source_id='Obama', target_id='Half-sister', properties={'title': 'Barack Obama', 'triplet_source_id': 'bd93d2e0-ab20-4f4c-a412-bb42f93ae56f'}),\n",
" EntityNode(label='entity', embedding=None, properties={'title': 'Barack Obama', 'triplet_source_id': 'bd93d2e0-ab20-4f4c-a412-bb42f93ae56f'}, name='Half-sister')),\n",
" (EntityNode(label='entity', embedding=None, properties={'title': 'Barack Obama', 'triplet_source_id': 'c4bbe9b8-ccd0-464c-b34c-37ede77f2717'}, name='Obama'),\n",
" Relation(label='Selected', source_id='Obama', target_id='Joe biden as his vice presidential running mate', properties={'title': 'Barack Obama', 'triplet_source_id': 'bc18ad10-3040-41a8-b595-4dd8ddb31a0b'}),\n",
" EntityNode(label='entity', embedding=None, properties={'title': 'Barack Obama', 'triplet_source_id': 'bc18ad10-3040-41a8-b595-4dd8ddb31a0b'}, name='Joe biden as his vice presidential running mate')),\n",
" (EntityNode(label='entity', embedding=None, properties={'title': 'Barack Obama', 'triplet_source_id': 'c4bbe9b8-ccd0-464c-b34c-37ede77f2717'}, name='Obama'),\n",
" Relation(label='Made', source_id='Obama', target_id='First public speech', properties={'title': 'Barack Obama', 'triplet_source_id': '6c89e860-215d-4f5b-8b1c-3183fe71bb6c'}),\n",
" EntityNode(label='entity', embedding=None, properties={'title': 'Barack Obama', 'triplet_source_id': '6c89e860-215d-4f5b-8b1c-3183fe71bb6c'}, name='First public speech')),\n",
" (EntityNode(label='entity', embedding=None, properties={'title': 'Barack Obama', 'triplet_source_id': 'c4bbe9b8-ccd0-464c-b34c-37ede77f2717'}, name='Obama'),\n",
" Relation(label='Banned', source_id='Obama', target_id='New offshore oil and gas drilling', properties={'title': 'Barack Obama', 'triplet_source_id': '62942a1e-18ae-4f45-9c73-ea39934f5519'}),\n",
" EntityNode(label='entity', embedding=None, properties={'title': 'Barack Obama', 'triplet_source_id': '62942a1e-18ae-4f45-9c73-ea39934f5519'}, name='New offshore oil and gas drilling')),\n",
" (EntityNode(label='entity', embedding=None, properties={'title': 'Barack Obama', 'triplet_source_id': 'c4bbe9b8-ccd0-464c-b34c-37ede77f2717'}, name='Obama'),\n",
" Relation(label='Met with', source_id='Obama', target_id='Australian prime minister', properties={'title': 'Barack Obama', 'triplet_source_id': 'c4bbe9b8-ccd0-464c-b34c-37ede77f2717'}),\n",
" EntityNode(label='entity', embedding=None, properties={'title': 'Barack Obama', 'triplet_source_id': 'c4bbe9b8-ccd0-464c-b34c-37ede77f2717'}, name='Australian prime minister'))]"
]
},
"execution_count": null,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"kg_extractor = SimpleLLMPathExtractor(\n",
" llm=llm, max_paths_per_chunk=20, num_workers=4\n",
")\n",
"\n",
"simple_index = PropertyGraphIndex.from_documents(\n",
" [document],\n",
" llm=llm,\n",
" embed_kg_nodes=False,\n",
" kg_extractors=[kg_extractor],\n",
" show_progress=True,\n",
")\n",
"\n",
"simple_index.property_graph_store.save_networkx_graph(\n",
" name=\"./SimpleGraph.html\"\n",
")\n",
"simple_index.property_graph_store.get_triplets(\n",
" entity_names=[\"Barack Obama\", \"Obama\"]\n",
")[:5]"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## 2. DynamicLLMPathExtractor"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Without intial ontology :\n",
"Here, we let the LLM define the ontology on the fly, giving it full freedom to label the nodes as it best sees fit."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [
{
"data": {
"application/vnd.jupyter.widget-view+json": {
"model_id": "df697bb50670468d94e4a9c223774cc2",
"version_major": 2,
"version_minor": 0
},
"text/plain": [
"Parsing nodes: 0%| | 0/1 [00:00<?, ?it/s]"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"Extracting and inferring knowledge graph from text: 100%|██████████| 11/11 [00:50<00:00, 4.59s/it]\n"
]
},
{
"data": {
"text/plain": [
"[(EntityNode(label='PERSON', embedding=None, properties={'approval_rating': '63 percent', 'title': 'Barack Obama', 'triplet_source_id': '425eced4-ff34-49c2-b4ce-64ac96bf8d43'}, name='Obama'),\n",
" Relation(label='MOVED_TO', source_id='Obama', target_id='Afghanistan', properties={'action': 'moved to bolster', 'quantity': 'U.S. troop strength in Afghanistan', 'title': 'Barack Obama', 'triplet_source_id': 'ff7b416e-2885-4296-b7e2-156cb3578bb1'}),\n",
" EntityNode(label='COUNTRY', embedding=None, properties={'title': 'Barack Obama', 'triplet_source_id': 'ff7b416e-2885-4296-b7e2-156cb3578bb1'}, name='Afghanistan')),\n",
" (EntityNode(label='PERSON', embedding=None, properties={'title': 'Barack Obama', 'triplet_source_id': '5137cb5e-04a8-4a71-bc1d-200783ec4628'}, name='Barack Obama'),\n",
" Relation(label='RECEIVED', source_id='Barack Obama', target_id='Our Great National Parks', properties={'award': 'Primetime Emmy Award', 'category': 'Outstanding Narrator', 'title': 'Barack Obama', 'triplet_source_id': '5137cb5e-04a8-4a71-bc1d-200783ec4628'}),\n",
" EntityNode(label='TV SHOW', embedding=None, properties={'title': 'Barack Obama', 'triplet_source_id': '5137cb5e-04a8-4a71-bc1d-200783ec4628'}, name='Our Great National Parks')),\n",
" (EntityNode(label='PERSON', embedding=None, properties={'title': 'Barack Obama', 'triplet_source_id': '5137cb5e-04a8-4a71-bc1d-200783ec4628'}, name='Barack Obama'),\n",
" Relation(label='PUBLISHED', source_id='Barack Obama', target_id='A Promised Land', properties={'title': 'Barack Obama', 'triplet_source_id': '43848a0a-858e-4552-b820-b8831931f63f'}),\n",
" EntityNode(label='BOOK', embedding=None, properties={'release_date': 'November 17', 'title': 'Barack Obama', 'triplet_source_id': 'caf64843-39ce-4992-9c40-e7b1166af804'}, name='A Promised Land')),\n",
" (EntityNode(label='PERSON', embedding=None, properties={'title': 'Barack Obama', 'triplet_source_id': '5137cb5e-04a8-4a71-bc1d-200783ec4628'}, name='Barack Obama'),\n",
" Relation(label='RECEIVED', source_id='Barack Obama', target_id='Shoah Foundation Institute for Visual History and Education', properties={'award': 'Ambassador of Humanity Award', 'title': 'Barack Obama', 'triplet_source_id': '5137cb5e-04a8-4a71-bc1d-200783ec4628'}),\n",
" EntityNode(label='ORGANIZATION', embedding=None, properties={'title': 'Barack Obama', 'triplet_source_id': '5137cb5e-04a8-4a71-bc1d-200783ec4628'}, name='Shoah Foundation Institute for Visual History and Education')),\n",
" (EntityNode(label='PERSON', embedding=None, properties={'title': 'Barack Obama', 'triplet_source_id': '5137cb5e-04a8-4a71-bc1d-200783ec4628'}, name='Barack Obama'),\n",
" Relation(label='SUPPORTED', source_id='Barack Obama', target_id='payday loan regulations', properties={'title': 'Barack Obama', 'triplet_source_id': '13073b9d-68e7-4973-9f70-bd65912d9604'}),\n",
" EntityNode(label='POLICY', embedding=None, properties={'target': 'low-income workers', 'title': 'Barack Obama', 'triplet_source_id': '13073b9d-68e7-4973-9f70-bd65912d9604'}, name='payday loan regulations'))]"
]
},
"execution_count": null,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"kg_extractor = DynamicLLMPathExtractor(\n",
" llm=llm,\n",
" max_triplets_per_chunk=20,\n",
" num_workers=4,\n",
" # Let the LLM infer entities and their labels (types) on the fly\n",
" allowed_entity_types=None,\n",
" # Let the LLM infer relationships on the fly\n",
" allowed_relation_types=None,\n",
" # LLM will generate any entity properties, set `None` to skip property generation (will be faster without)\n",
" allowed_relation_props=[],\n",
" # LLM will generate any relation properties, set `None` to skip property generation (will be faster without)\n",
" allowed_entity_props=[],\n",
")\n",
"\n",
"dynamic_index = PropertyGraphIndex.from_documents(\n",
" [document],\n",
" llm=llm,\n",
" embed_kg_nodes=False,\n",
" kg_extractors=[kg_extractor],\n",
" show_progress=True,\n",
")\n",
"\n",
"dynamic_index.property_graph_store.save_networkx_graph(\n",
" name=\"./DynamicGraph.html\"\n",
")\n",
"\n",
"dynamic_index.property_graph_store.get_triplets(\n",
" entity_names=[\"Barack Obama\", \"Obama\"]\n",
")[:5]"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### With initial ontology for guided KG extraction : \n",
"Here, we have partial knowledge of what we want to detect, we know the article is about Barack Obama, so we define some entities and relations that could help guide the LLM in the labeling process as it detects the entities and relations. This doesn't guarantee that the LLM will use them, it simply guides it and gives it some ideas. It will still be up to the LLM to decide whether it uses the entities and relations we provide or not. "
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [
{
"data": {
"application/vnd.jupyter.widget-view+json": {
"model_id": "54cf07121c7d4d71ac3072a457fdb808",
"version_major": 2,
"version_minor": 0
},
"text/plain": [
"Parsing nodes: 0%| | 0/1 [00:00<?, ?it/s]"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"Extracting and inferring knowledge graph from text: 100%|██████████| 11/11 [00:47<00:00, 4.29s/it]\n"
]
},
{
"data": {
"text/plain": [
"[(EntityNode(label='PERSON', embedding=None, properties={'description': '44th President of the United States', 'title': 'Barack Obama', 'triplet_source_id': 'd286a836-a5ad-43af-b6de-bd43f072512c'}, name='Obama'),\n",
" Relation(label='MOVED_TO', source_id='Obama', target_id='Afghanistan', properties={'description': 'moved to bolster U.S. troop strength', 'title': 'Barack Obama', 'triplet_source_id': '23c1750d-de01-4a75-814e-b56b81b9bbb4'}),\n",
" EntityNode(label='COUNTRY', embedding=None, properties={'title': 'Barack Obama', 'triplet_source_id': '23c1750d-de01-4a75-814e-b56b81b9bbb4'}, name='Afghanistan')),\n",
" (EntityNode(label='POLITICIAN', embedding=None, properties={'title': 'Barack Obama', 'triplet_source_id': '8f9dc0b3-ff33-46e9-ad3f-040755d33fc7'}, name='Barack Obama'),\n",
" Relation(label='ESTABLISHED', source_id='Barack Obama', target_id='White House Task Force to Protect Students from Sexual Assault', properties={'title': 'Barack Obama', 'triplet_source_id': '8af352da-b50d-4043-8002-870991473cf6'}),\n",
" EntityNode(label='ORGANIZATION', embedding=None, properties={'title': 'Barack Obama', 'triplet_source_id': '8af352da-b50d-4043-8002-870991473cf6'}, name='White House Task Force to Protect Students from Sexual Assault')),\n",
" (EntityNode(label='POLITICIAN', embedding=None, properties={'title': 'Barack Obama', 'triplet_source_id': '8f9dc0b3-ff33-46e9-ad3f-040755d33fc7'}, name='Barack Obama'),\n",
" Relation(label='BECAME_CHAIRMAN_OF', source_id='Barack Obama', target_id=\"Illinois Senate\\\\'s Health and Human Services Committee\", properties={'title': 'Barack Obama', 'triplet_source_id': '5bf11d65-0078-48bb-97b5-109b4469d46a'}),\n",
" EntityNode(label='COMMITTEE', embedding=None, properties={'title': 'Barack Obama', 'triplet_source_id': '5bf11d65-0078-48bb-97b5-109b4469d46a'}, name=\"Illinois Senate\\\\'s Health and Human Services Committee\")),\n",
" (EntityNode(label='PERSON', embedding=None, properties={'description': '44th President of the United States', 'title': 'Barack Obama', 'triplet_source_id': 'd286a836-a5ad-43af-b6de-bd43f072512c'}, name='Obama'),\n",
" Relation(label='USED', source_id='Obama', target_id='last day in office', properties={'description': 'used phrase \"thanks, Obama\"', 'title': 'Barack Obama', 'triplet_source_id': 'd286a836-a5ad-43af-b6de-bd43f072512c'}),\n",
" EntityNode(label='EVENT', embedding=None, properties={'description': 'final day in office', 'title': 'Barack Obama', 'triplet_source_id': 'd286a836-a5ad-43af-b6de-bd43f072512c'}, name='last day in office')),\n",
" (EntityNode(label='PERSON', embedding=None, properties={'description': '44th President of the United States', 'title': 'Barack Obama', 'triplet_source_id': 'd286a836-a5ad-43af-b6de-bd43f072512c'}, name='Obama'),\n",
" Relation(label='SAID', source_id='Obama', target_id='34,000 U.S. troops', properties={'description': 'said the U.S. military would reduce the troop level in Afghanistan', 'title': 'Barack Obama', 'triplet_source_id': '23c1750d-de01-4a75-814e-b56b81b9bbb4'}),\n",
" EntityNode(label='MILITARY_FORCE', embedding=None, properties={'title': 'Barack Obama', 'triplet_source_id': '23c1750d-de01-4a75-814e-b56b81b9bbb4'}, name='34,000 U.S. troops'))]"
]
},
"execution_count": null,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"kg_extractor = DynamicLLMPathExtractor(\n",
" llm=llm,\n",
" max_triplets_per_chunk=20,\n",
" num_workers=4,\n",
" allowed_entity_types=[\"POLITICIAN\", \"POLITICAL_PARTY\"],\n",
" allowed_relation_types=[\"PRESIDENT_OF\", \"MEMBER_OF\"],\n",
" allowed_relation_props=[\"description\"],\n",
" allowed_entity_props=[\"description\"],\n",
")\n",
"\n",
"dynamic_index_2 = PropertyGraphIndex.from_documents(\n",
" [document],\n",
" llm=llm,\n",
" embed_kg_nodes=False,\n",
" kg_extractors=[kg_extractor],\n",
" show_progress=True,\n",
")\n",
"\n",
"dynamic_index_2.property_graph_store.save_networkx_graph(\n",
" name=\"./DynamicGraph_2.html\"\n",
")\n",
"dynamic_index_2.property_graph_store.get_triplets(\n",
" entity_names=[\"Barack Obama\", \"Obama\"]\n",
")[:5]"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# 3 - SchemaLLMPathExtractor"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [
{
"data": {
"application/vnd.jupyter.widget-view+json": {
"model_id": "14addfbbee2b4d6784b6383f37c909ad",
"version_major": 2,
"version_minor": 0
},
"text/plain": [
"Parsing nodes: 0%| | 0/1 [00:00<?, ?it/s]"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"Extracting paths from text with schema: 100%|██████████| 11/11 [00:52<00:00, 4.81s/it]\n"
]
},
{
"data": {
"text/plain": [
"[(EntityNode(label='PERSON', embedding=None, properties={'title': 'Barack Obama', 'triplet_source_id': '87af3360-fa63-40c2-8440-f4114a7093fd'}, name='Barack Obama'),\n",
" Relation(label='HAS', source_id='Barack Obama', target_id='References', properties={'title': 'Barack Obama', 'triplet_source_id': '87af3360-fa63-40c2-8440-f4114a7093fd'}),\n",
" EntityNode(label='CONCEPT', embedding=None, properties={'title': 'Barack Obama', 'triplet_source_id': '87af3360-fa63-40c2-8440-f4114a7093fd'}, name='References')),\n",
" (EntityNode(label='PERSON', embedding=None, properties={'title': 'Barack Obama', 'triplet_source_id': '87af3360-fa63-40c2-8440-f4114a7093fd'}, name='Barack Obama'),\n",
" Relation(label='INTERCEPTED', source_id='Barack Obama', target_id='pipe bomb', properties={'title': 'Barack Obama', 'triplet_source_id': 'ada0abff-9671-4156-b06c-bf5067e6d54c'}),\n",
" EntityNode(label='PRODUCT', embedding=None, properties={'title': 'Barack Obama', 'triplet_source_id': 'ada0abff-9671-4156-b06c-bf5067e6d54c'}, name='pipe bomb')),\n",
" (EntityNode(label='PERSON', embedding=None, properties={'title': 'Barack Obama', 'triplet_source_id': '87af3360-fa63-40c2-8440-f4114a7093fd'}, name='Barack Obama'),\n",
" Relation(label='HAS', source_id='Barack Obama', target_id='end of 2015', properties={'title': 'Barack Obama', 'triplet_source_id': '2b64d219-d19b-4346-a6a0-4369599af5d1'}),\n",
" EntityNode(label='TIME', embedding=None, properties={'title': 'Barack Obama', 'triplet_source_id': '2b64d219-d19b-4346-a6a0-4369599af5d1'}, name='end of 2015')),\n",
" (EntityNode(label='PERSON', embedding=None, properties={'title': 'Barack Obama', 'triplet_source_id': '87af3360-fa63-40c2-8440-f4114a7093fd'}, name='Barack Obama'),\n",
" Relation(label='GRADUATED_FROM', source_id='Barack Obama', target_id='Columbia University', properties={'title': 'Barack Obama', 'triplet_source_id': '65be5ae1-bc74-43ee-9655-855daf81f74f'}),\n",
" EntityNode(label='ORGANIZATION', embedding=None, properties={'title': 'Barack Obama', 'triplet_source_id': '65be5ae1-bc74-43ee-9655-855daf81f74f'}, name='Columbia University')),\n",
" (EntityNode(label='PERSON', embedding=None, properties={'title': 'Barack Obama', 'triplet_source_id': '87af3360-fa63-40c2-8440-f4114a7093fd'}, name='Barack Obama'),\n",
" Relation(label='EDUCATION', source_id='Barack Obama', target_id='Schools and Universities', properties={'extra_description': 'Attended schools and universities', 'title': 'Barack Obama', 'triplet_source_id': '1f495d28-7df4-44dc-a3e3-bfc6161d3d2d'}),\n",
" EntityNode(label='ORGANIZATION', embedding=None, properties={'title': 'Barack Obama', 'triplet_source_id': '1f495d28-7df4-44dc-a3e3-bfc6161d3d2d'}, name='Schools and Universities'))]"
]
},
"execution_count": null,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"kg_extractor = SchemaLLMPathExtractor(\n",
" llm=llm,\n",
" max_triplets_per_chunk=20,\n",
" strict=False, # Set to False to showcase why it's not going to be the same as DynamicLLMPathExtractor\n",
" possible_entities=None, # USE DEFAULT ENTITIES (PERSON, ORGANIZATION... etc)\n",
" possible_relations=None, # USE DEFAULT RELATIONSHIPS\n",
" possible_relation_props=[\n",
" \"extra_description\"\n",
" ], # Set to `None` to skip property generation\n",
" possible_entity_props=[\n",
" \"extra_description\"\n",
" ], # Set to `None` to skip property generation\n",
" num_workers=4,\n",
")\n",
"\n",
"schema_index = PropertyGraphIndex.from_documents(\n",
" [document],\n",
" llm=llm,\n",
" embed_kg_nodes=False,\n",
" kg_extractors=[kg_extractor],\n",
" show_progress=True,\n",
")\n",
"\n",
"schema_index.property_graph_store.save_networkx_graph(\n",
" name=\"./SchemaGraph.html\"\n",
")\n",
"schema_index.property_graph_store.get_triplets(\n",
" entity_names=[\"Barack Obama\", \"Obama\"]\n",
")[:5]"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Comparison and Analysis\n",
"\n",
"Let's compare the results of the three extractors:\n",
"\n",
"1. **SimpleLLMPathExtractor**: This extractor creates a basic knowledge graph without any predefined schema. It may produce a larger number of diverse relationships but might lack consistency in entity and relation naming.\n",
"\n",
"\n",
"3. **DynamicLLMPathExtractor**: \n",
" - This new extractor combines the flexibility of SimpleLLMPathExtractor with some initial guidance from a schema. It can expand beyond the initial entity and relation types, potentially producing a rich and diverse graph while maintaining some level of consistency. \n",
" - Not giving it any entities or relations to start with in the input gives the LLM complete freedom to infer the schema on the fly as it best sees fit. This is going to vary based on the LLM and the temperature used.\n",
"\n",
"3. **SchemaLLMPathExtractor**: With a predefined schema, this extractor produces a more structured graph. The entities and relations are limited to those specified in the schema, which can lead to a more consistent but potentially less comprehensive graph. Even if we set \"strict\" to false, the extracted KG Graph doesn't reflect the LLM's pursuit of trying to find new entities and types that fall outside of the input schema's scope.\n",
"\n",
"\n",
"## Key observations:\n",
"\n",
"- The SimpleLLMPathExtractor graph might have the most diverse set of entities and relations.\n",
"- The SchemaLLMPathExtractor graph should be the most consistent but might miss a lot of relationships that don't fit the predefined schema, even if we don't impose a strict validation of the schema.\n",
"- The DynamicLLMPathExtractor graph should show a balance between diversity and consistency, potentially capturing important relationships that the schema-based approach might miss while still maintaining some structure.\n",
"\n",
"## The choice between these extractors depends on the specific use case:\n",
"\n",
"- Use SimpleLLMPathExtractor for exploratory analysis where you want to capture a wide range of potential relationships for RAG applications, without caring about the entity types.\n",
"- Use SchemaLLMPathExtractor when you have a well-defined domain and want to ensure consistency in the extracted knowledge.\n",
"- Use DynamicLLMPathExtractor when you want a balance between structure and flexibility, allowing the model to discover new entity and relation types while still providing some initial guidance. This one is especially useful if you want a KG with labeled (typed) entities but don't have an input Schema (or you've partially defined the schema as a starting base)."
]
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3"
}
},
"nbformat": 4,
"nbformat_minor": 4
}
Reference in New Issue View Git Blame Copy Permalink