wehub-resource-syncMultiModal RAG with ColiVara and DeepSeek-Janus-Pro
This project implements a MultiModal RAG with DeepSeek's latest model Janus-Pro and ColiVara.
We use the following tools
- DeepSeek-Janus-Pro as the multi-modal LLM.
- ColiVara for SOTA document understanding and retrieval.
- Firecrawl for web scraping.
- Streamlit as the web interface.
Demo
A demo of the project is available below:
Setup and installations
Setup Janus:
git clone https://github.com/deepseek-ai/Janus.git
pip install -e ./Janus
Get the API keys:
Create a .env file and store them as follows:
COLIVARA_API_KEY="<COLIVARA-API-KEY>"
FIRECRAWL_API_KEY="<FIRECRAWL-API-KEY>"
Install Dependencies: Ensure you have Python 3.11 or later installed.
pip install streamlit-pdf-viewer colivara-py streamlit fastembed flash-attn transformers
Run the project
Finally, run the project by running the following command:
streamlit run app.py
📬 Stay Updated with Our Newsletter!
Get a FREE Data Science eBook 📖 with 150+ essential lessons in Data Science when you subscribe to our newsletter! Stay in the loop with the latest tutorials, insights, and exclusive resources. Subscribe now!
Contribution
Contributions are welcome! Please fork the repository and submit a pull request with your improvements.