GenAI-WebMind-Agent

WebMind AI Agent is a Generative AI-powered web content question-answering application built using Streamlit, LangChain, Google Gemini, and FAISS.

The application extracts content from web URLs, converts the content into vector embeddings, stores them in a FAISS vector database, and allows users to ask questions about the webpage using AI.

---

Features

• Load content from any website URL
• AI-powered question answering using Gemini
• Retrieval-Augmented Generation (RAG)
• Semantic search using FAISS
• Sentence Transformer embeddings
• Interactive Streamlit UI
• Displays source URLs used for answers

---

Technologies Used

• Python
• Streamlit
• LangChain
• Google Gemini API
• FAISS
• HuggingFace Sentence Transformers

---

Project Structure

├── Main.py
├── env.txt
├── faiss_store.pkl
├── requirements.txt
└── README.md

---

Required Libraries

Install the following libraries before running the project:

pip install streamlit
pip install langchain
pip install langchain-community
pip install langchain-google-genai
pip install langchain-text-splitters
pip install faiss-cpu
pip install sentence-transformers
pip install python-dotenv
pip install beautifulsoup4
pip install lxml

Or install everything at once:

pip install -r requirements.txt

---

requirements.txt

streamlit
langchain
langchain-community
langchain-google-genai
langchain-text-splitters
faiss-cpu
sentence-transformers
python-dotenv
beautifulsoup4
lxml

---

Setup Environment Variables

Create a .env file in the project directory.

GOOGLE_API_KEY=your_google_api_key

Get your Gemini API key from:

https://aistudio.google.com/app/apikey

---

How to Run the Project

Run the Streamlit app using:

streamlit run Main.py

---

How the Project Works

Step 1 — Enter Website URL

The user enters a website URL in the Streamlit sidebar.

Step 2 — Load Web Content

The application uses WebBaseLoader to extract webpage content.

Step 3 — Split Text

The webpage content is divided into smaller chunks using RecursiveCharacterTextSplitter.

Step 4 — Create Embeddings

Text chunks are converted into embeddings using:

sentence-transformers/all-MiniLM-L6-v2

Step 5 — Store in FAISS

Embeddings are stored in a FAISS vector database for semantic retrieval.

Step 6 — Ask Questions

Users can ask questions related to the webpage content.

Step 7 — AI Response Generation

Relevant content is retrieved from FAISS and sent to Gemini for answer generation.

---

Example Workflow

1. Paste a website URL
2. Click "Process URLs"
3. Ask questions about the webpage
4. Get AI-generated answers instantly

---

Example Questions

• What is this article about?
• Summarize the webpage
• What are the key points?
• Who is the author?
• Explain the main topic

---

Future Improvements

• Multiple URL support
• PDF document support
• Chat history memory
• Advanced RAG pipeline
• Conversational AI interface
• Source citation improvements

---

Author

Santhosh S S

---