ai4finance-foundation--fingpt/SETUP.md

Setup Guide: Running FinGPT Locally and on Replit

This guide provides step-by-step instructions for running FinGPT both locally and on Replit, addressing the requirements for different use cases and hardware configurations.

Table of Contents

• Hardware Requirements
• Local Setup
• Replit Setup
• Quick Start Examples
• Troubleshooting

Hardware Requirements

Minimum Requirements (For Inference Only)

• CPU: Any modern multi-core processor
• RAM: 8GB minimum, 16GB recommended
• Storage: 20GB free space
• GPU: Not required for cloud API usage, recommended for local models

Recommended Requirements (For Training/Fine-tuning)

• CPU: Modern multi-core processor (Intel i7+/AMD Ryzen 7+)
• RAM: 32GB minimum, 64GB recommended
• Storage: 50GB+ free space (SSD recommended)
• GPU: NVIDIA GPU with 12GB+ VRAM (RTX 3090, A100, etc.)
• CUDA: 11.8+ for GPU acceleration

Cloud GPU Options

If you don't have a powerful GPU, consider these cloud platforms:

• Google Colab: Free tier with GPU access
• Kaggle Kernels: Free GPU access
• RunPod: Affordable GPU rentals
• Vast.ai: Low-cost GPU marketplace
• Lambda Labs: GPU cloud for ML

Local Setup

Prerequisites

• Python 3.8 or higher
• Git
• Virtual environment (recommended)

Step 1: Clone the Repository

git clone https://github.com/AI4Finance-Foundation/FinGPT.git
cd FinGPT

Step 2: Create Virtual Environment (Recommended)

# Using venv
python -m venv fingpt_env
source fingpt_env/bin/activate  # On Windows: fingpt_env\Scripts\activate

# Using conda
conda create -n fingpt python=3.8
conda activate fingpt

Step 3: Install Dependencies

Basic Installation

pip install -r requirements.txt
pip install -e .

For Inference with Local Models

pip install transformers==4.32.0 peft==0.5.0
pip install sentencepiece accelerate torch
pip install datasets bitsandbytes

For Training/Fine-tuning

pip install transformers==4.32.0 peft==0.5.0
pip install sentencepiece accelerate torch
pip install datasets bitsandbytes
pip install deepspeed wandb  # Optional for advanced training

For FinGPT-Forecaster

pip install yfinance finnhub-python
pip install gradio beautifulsoup4 requests

Step 4: Verify Installation

python -c "import transformers; import torch; print('Transformers:', transformers.__version__); print('PyTorch:', torch.__version__); print('CUDA available:', torch.cuda.is_available())"

Replit Setup

Step 1: Create a New Replit

1. Go to replit.com
2. Click "Create Repl"
3. Select "Python" as the template
4. Name your repl (e.g., "FinGPT")

Step 2: Import the Repository

1. In your Replit, click the "Shell" tab
2. Run the following commands:

git clone https://github.com/AI4Finance-Foundation/FinGPT.git
mv FinGPT/* .
mv FinGPT/.* . 2>/dev/null || true
rmdir FinGPT

Step 3: Configure Replit for FinGPT

Update .replit file

Create or update the .replit file:

[run]
command = "python main.py"

[env]
PYTHONPATH = "."

Update pyproject.toml (if needed)

Ensure your dependencies are listed:

[project]
name = "fingpt"
requires-python = ">=3.8"
dependencies = [
    "transformers==4.32.0",
    "peft==0.5.0",
    "torch",
    "accelerate",
    "sentencepiece",
    "datasets",
    "bitsandbytes",
    "numpy",
    "pandas",
]

Step 4: Install Dependencies

pip install -r requirements.txt
pip install transformers==4.32.0 peft==0.5.0
pip install sentencepiece accelerate torch
pip install datasets bitsandbytes

Step 5: Handle GPU on Replit

Replit offers GPU access on paid plans. To use GPU:

1. Upgrade to a Replit plan with GPU access
2. Enable GPU in your Replit settings
3. The PyTorch installation will automatically detect CUDA

Step 6: Run FinGPT

# Run a simple inference script
python -c "from transformers import AutoTokenizer; print('FinGPT ready!')"

Quick Start Examples

Example 1: Running Inference with Pre-trained Models

Using FinGPT-Sentiment Model (Local)

from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel
import torch

# Load base model
base_model = AutoModelForCausalLM.from_pretrained(
    'meta-llama/Llama-2-7b-chat-hf',
    trust_remote_code=True,
    device_map="auto",
    torch_dtype=torch.float16,
)
tokenizer = AutoTokenizer.from_pretrained('meta-llama/Llama-2-7b-chat-hf')

# Load FinGPT model
model = PeftModel.from_pretrained(
    base_model, 
    'FinGPT/fingpt-sentiment_llama2-13b_lora'
)
model = model.eval()

# Prepare input
text = "Glaxo's ViiV Healthcare Signs China Manufacturing Deal With Desano"
prompt = f"What is the sentiment of this news? Please choose an answer from {{negative/neutral/positive}}.\n\n{text}"

# Generate response
inputs = tokenizer(prompt, return_tensors='pt')
inputs = {key: value.to(model.device) for key, value in inputs.items()}

with torch.no_grad():
    outputs = model.generate(
        **inputs, 
        max_new_tokens=100,
        do_sample=True,
        temperature=0.7
    )

response = tokenizer.decode(outputs[0], skip_special_tokens=True)
print(response)

Using Cloud API (No GPU Required)

import os

# Set your API key
os.environ['OPENAI_API_KEY'] = 'your-api-key-here'
os.environ['FINGPT_LLM_PROVIDER'] = 'openai'

# Use FinGPT with OpenAI
from fingpt.Forecaster import FinGPTForecaster

forecaster = FinGPTForecaster()
result = forecaster.predict(ticker="AAPL", date="2024-01-15")
print(result)

Example 2: Running FinGPT-Forecaster Demo

Local Setup

cd fingpt/FinGPT_Forecaster
pip install -r requirements.txt

Run the demo notebook

jupyter notebook demo.ipynb

Or run the Gradio app:

import gradio as gr
from fingpt.Forecaster import FinGPTForecaster

forecaster = FinGPTForecaster()

def predict(ticker, date, weeks, add_financials):
    result = forecaster.predict(
        ticker=ticker, 
        date=date, 
        weeks=weeks,
        add_financials=add_financials
    )
    return result

iface = gr.Interface(
    fn=predict,
    inputs=[
        gr.Textbox(label="Ticker Symbol"),
        gr.Textbox(label="Date (YYYY-MM-DD)"),
        gr.Slider(1, 12, value=4, label="Number of Weeks"),
        gr.Checkbox(label="Add Basic Financials")
    ],
    outputs="text",
    title="FinGPT-Forecaster"
)

iface.launch()

Example 3: Training with LoRA (Requires GPU)

Use the provided Jupyter notebooks:

• FinGPT_Training_LoRA_with_ChatGLM2_6B_for_Beginners.ipynb
• FinGPT_ Training with LoRA and Meta-Llama-3-8B.ipynb

# Start Jupyter
jupyter notebook

# Open and run the training notebook cell by cell

Running Different FinGPT Components

FinGPT-Sentiment Analysis

cd fingpt/FinGPT_Sentiment_Analysis_v3
# Run benchmark notebooks
jupyter notebook benchmark/benchmarks.ipynb

FinGPT-Forecaster

cd fingpt/FinGPT_Forecaster
# Run demo
jupyter notebook demo.ipynb

FinGPT-RAG

cd fingpt/FinGPT_RAG
# Check the README for specific setup instructions

FinGPT-Benchmark

cd fingpt/FinGPT_Benchmark
# Run demo
jupyter notebook demo.ipynb

Troubleshooting

Common Issues and Solutions

Issue 1: CUDA Out of Memory

Problem: RuntimeError: CUDA out of memory

Solutions:

• Use a smaller model (7B instead of 13B)
• Enable quantization (8-bit or 4-bit)
• Reduce batch size
• Use gradient checkpointing

# Enable 8-bit quantization
model = AutoModelForCausalLM.from_pretrained(
    model_name,
    load_in_8bit=True,
    device_map="auto"
)

Issue 2: Import Errors

Problem: ModuleNotFoundError: No module named 'transformers'

Solution:

pip install transformers==4.32.0 peft==0.5.0
pip install sentencepiece accelerate torch

Issue 3: HuggingFace Authentication

Problem: OSError: meta-llama/Llama-2-7b-chat-hf is a gated model

Solution:

1. Go to HuggingFace Llama 2 page
2. Accept the user agreement
3. Generate an access token in your HuggingFace settings
4. Login in your terminal:

huggingface-cli login

Issue 4: Replit GPU Not Available

Problem: GPU not detected on Replit

Solution:

• Upgrade to a Replit plan with GPU access
• Enable GPU in Replit settings
• Use cloud APIs instead of local models

Issue 5: Slow Performance on CPU

Problem: Inference is very slow on CPU

Solutions:

• Use cloud APIs (OpenAI, MiniMax) instead of local models
• Use smaller models
• Enable CPU optimizations:

import torch
model = AutoModelForCausalLM.from_pretrained(
    model_name,
    torch_dtype=torch.float32,
    device_map="cpu"
)

Issue 6: Dependency Conflicts

Problem: Version conflicts between packages

Solution:

# Create fresh environment
python -m venv fresh_env
source fresh_env/bin/activate
pip install --upgrade pip
pip install -r requirements.txt --force-reinstall

Getting Help

If you encounter issues not covered here:

1. Check the GitHub Issues
2. Join the Discord community
3. Refer to specific component READMEs in the fingpt/ directory
4. Check the FinGPT documentation

Additional Resources

• FinGPT Research Paper
• HuggingFace Models
• FinGPT Demos
• Medium Blog Series

Disclaimer

Nothing herein is financial advice, and NOT a recommendation to trade real money. Please use common sense and always first consult a professional before trading or investing.