WebRAG – Retrieval-Augmented Generation Platform

WebRAG is a full-stack AI application that transforms multiple websites into a searchable knowledge base using Retrieval-Augmented Generation (RAG).

Users can crawl and index multiple websites simultaneously, then ask natural language questions across all indexed content. Every response is grounded in retrieved documents and includes source citations for transparency.

🚀 Live Links

🔗 Live Demo: https://webrag-frontend.onrender.com

⚙️ API: https://webrag-zkg6.onrender.com (~50s cold start on free tier)

💻 GitHub: https://github.com/Priyanshu-Priya/WebRAG

Features

• 🌐 Crawl and index multiple websites simultaneously
• 🤖 Semantic question answering using Retrieval-Augmented Generation (RAG)
• 📖 Perplexity-style source citations for every response
• 🔍 Cross-website semantic search
• ⚡ Hybrid crawler using HTTPX with Playwright fallback for JavaScript-heavy websites
• 🧹 Intelligent HTML cleaning with BeautifulSoup
• 🧩 Automatic text chunking and embedding generation
• 📚 ChromaDB vector database for semantic retrieval
• 🔄 SHA-256 based change detection with incremental re-indexing
• 📊 Content diff reports for updated pages
• 📡 Real-time indexing progress using WebSockets
• 📱 Responsive React dashboard

Tech Stack

Frontend

• React
• Vite
• Tailwind CSS
• Framer Motion
• Axios

Backend

• FastAPI
• LangChain
• SQLAlchemy
• APScheduler

AI

• Groq (Llama 3.3)
• BAAI/bge-base-en-v1.5
• Sentence Transformers

Storage

• SQLite
• ChromaDB

Outcome

Built a production-ready AI platform capable of maintaining continuously updated knowledge bases from multiple websites while providing accurate, source-grounded answers with automatic freshness handling.