This application implements a Retrieval-Augmented Generation (RAG) system that combines audio transcription, vector database storage, and CrewAI Flows for orchestrated processing. The system allows users to ingest multimodal data (audio, text) into a vector database and then query it using voice input.

We use:

CrewAI Flows for orchestrated processing
AssemblyAI for audio transcription
Milvus for vector database storage
OpenAI for embeddings and LLM

Setup and Installation

Ensure you have Python 3.10 or later installed on your system.

Install dependencies:

pip install -r requirements.txt

Start Milvus using Docker:

docker-compose up -d

Copy .env.example to .env and configure the following environment variables:

ASSEMBLYAI_API_KEY=your_assemblyai_api_key_here
OPENAI_API_KEY=your_openai_api_key_here

Run the application:

python main.py

Usage

Data Ingestion: Place your data (audio, text files) in the data/ directory
System Setup: The system automatically processes and stores data in the vector database
Voice Input: Record your voice query using the microphone
Audio Transcription: AssemblyAI transcribes your voice to text
Vector Search: OpenAI generates embeddings and searches Milvus vector database
Research Agent: CrewAI Research Agent analyzes search results and finds relevant information
Response Agent: CrewAI Response Agent synthesizes information into comprehensive answer
Final Response: View the agent-generated response based on your knowledge base

Get a FREE Data Science eBook 📖 with 150+ essential lessons in Data Science when you subscribe to our newsletter! Stay in the loop with the latest tutorials, insights, and exclusive resources. Subscribe now!

Contribution

Contributions are welcome! Please fork the repository and submit a pull request with your improvements.

README.md

Multimodal Agentic RAG System

Setup and Installation

Usage

📬 Stay Updated with Our Newsletter!

Contribution