Yara — Agentic Docs Support Assistant

Persian-first support chatbot that answers how-to questions from your documentation and runs a multi-step troubleshooting agent when something is broken. Built as a hybrid RAG system with evidence gates, claim verification, JWT auth, and cost-aware model roles.

---

What it does

Mode	When	Behavior
Chatbot	How-to / docs questions	Intent → hybrid retrieve → relevance gate → grounded answer → claim verifier
Agent	Errors, failures, “it doesn’t work”, 502, etc.	Gather slots → hypothesize → retrieve & propose → ask the user if it’s fixed

• Answers are written in Persian
• Sources are returned as real doc URLs (IDs mapped on the backend — the model does not invent links)
• If evidence is insufficient → no answer (no_relevant_docs / clarification), not a best-effort hallucination
• Topic switches mid-troubleshoot reset agent state so the previous loop does not continue

Response statuses

answered · awaiting_clarification · awaiting_verification · resolved · needs_clarification · off_topic · no_relevant_docs

---

Architecture

User (RTL UI)
    │  JWT Bearer
    ▼
FastAPI  ──► Router (intent / troubleshooting)
    │              │
    │              ├─ chatbot path ──► multi-query hybrid RAG
    │              │                      Qdrant (vector) + BM25 (keyword)
    │              │                      → RRF merge → rerank → relevance gate
    │              │                      → answer LLM → claim verifier
    │              │
    │              └─ agent path ──► gather → hypotheses → propose → verify
    │
    ├── Redis   (sessions, FAQ cache, rate limits, users, prefs)
    └── Qdrant  (doc embeddings)

Stack

Layer	Choice
Backend	FastAPI + Uvicorn
Frontend	Static RTL chat UI (frontend/)
Agents	Google ADK + LiteLLM
Vector DB	Qdrant
Keyword	BM25 (rank-bm25)
Reranker	AvalAI qwen3-rerank
Embeddings	AvalAI Alibaba text-embedding-v4 (1024-d)
Cache / sessions	Redis
LLM gateway	AvalAI (OpenAI-compatible)

Agent model roles (config-driven)

Role	Env var	Default	Job
Router	ROUTER_MODEL	deepseek-v4-flash	Intent + troubleshooting slots / topic switch
Relevance	RELEVANCE_MODEL	qwen3.7-plus	Evidence full / partial / none
Answer / rewrite / verifier	LLM_MODEL	gpt-4.1-mini	Final answer, query rewrite, claim check, hypotheses

Feature flags:

Flag	Effect
USE_AGENTIC_RAG	Full agent loop vs simple RAG
USE_CLAIM_VERIFIER	Post-answer claim check against retrieved chunks
USE_TROUBLESHOOTING_AGENT	Multi-step diagnose loop vs chatbot-only
USE_INJECTION_GUARD	Prompt-injection gate before RAG / agents
TROUBLESHOOT_MAX_GATHER_ROUNDS	Max clarifying rounds before diagnosing (default 2)

---

Project layout

.
├── backend/           # FastAPI app, agents, RAG, auth, memory
│   └── app/
│       ├── agents/    # Orchestrator, troubleshooting, streaming
│       ├── api/       # /api/chat, /api/auth
│       ├── cache/     # FAQ answer cache
│       ├── memory/    # Sessions, agent state, user prefs
│       ├── rag/       # Retrieve, rerank, select, pipeline
│       ├── security/  # JWT, rate limit, injection guard
│       └── observability/
├── frontend/          # Login + chat UI (Persian RTL)
├── ingestion/         # MDX parse → chunk → embed → Qdrant + BM25
├── eval/              # Question sets + eval runners / results
├── data/              # BM25 pickle (shipped in Docker image)
├── docs/              # Source documentation (MDX) used for ingestion
├── docker-compose.yml # Local Redis + Qdrant (+ optional Postgres)
├── Dockerfile
└── .env.example

---

Prerequisites

• Python 3.12+
• Docker (for Redis + Qdrant locally)
• An AvalAI API key

---

Quick start

1. Clone & env

git clone <your-repo-url> yara-docs-agent
cd yara-docs-agent

cp .env.example .env
# Set at least: AVALAI_API_KEY, JWT_SECRET

2. Infrastructure

docker compose up -d

This starts Redis and Qdrant (and optionally Postgres if defined in compose).

3. Install & ingest

Ingest once locally (parses MDX under docs/, embeds, builds BM25, upserts Qdrant):

python -m venv .venv
# Windows: .venv\Scripts\activate
# macOS/Linux: source .venv/bin/activate

pip install -r requirements.txt
python ingestion/ingest.py

4. Run the API

cd backend
uvicorn main:app --reload --host 0.0.0.0 --port 8000

Open http://127.0.0.1:8000/ for the chat UI.

---

Auth

JWT replaces static API keys.

Endpoint	Auth	Purpose
POST /api/auth/register	public	{username, password}
POST /api/auth/login	public	{access_token, token_type, expires_in}
GET /api/auth/me	optional Bearer	current user
POST /api/chat	Bearer when required	support chat

• JWT_SECRET — strong secret in production
• JWT_EXPIRE_MINUTES — default 720; set 0 for never-expire demos
• AUTH_REQUIRED — optional override; defaults on in production, off in development

# register
curl -sS -X POST http://127.0.0.1:8000/api/auth/register \
  -H "Content-Type: application/json" \
  -d "{\"username\":\"demo\",\"password\":\"secret12\"}"

# login → access_token
curl -sS -X POST http://127.0.0.1:8000/api/auth/login \
  -H "Content-Type: application/json" \
  -d "{\"username\":\"demo\",\"password\":\"secret12\"}"

# chat
curl -sS -X POST http://127.0.0.1:8000/api/chat \
  -H "Authorization: Bearer <access_token>" \
  -H "Content-Type: application/json" \
  -d "{\"message\":\"چطور لاگ ببینم؟\"}"

---

Useful endpoints

Path	Notes
POST /api/chat	Support chat (Bearer when auth required)
GET /health	Redis + Qdrant ping + config (public)
GET /metrics	Counters — Bearer JWT
GET /api/usage	Usage JSON — Bearer JWT
GET /usage	HTML dashboard (API needs login)
DELETE /api/cache/faq	Flush FAQ cache — Bearer JWT

---

RAG pipeline (chatbot)

Query
  → intent / expand
  → multi-query retrieve (vector + BM25)
  → RRF merge (by chunk_id)
  → rerank (optional skip when BM25 ∩ vector agree)
  → relevance gate (full / partial / none)
  → answer (Persian) + source IDs → URLs
  → claim verifier

Troubleshooting agent:

Classify → gather slots → hypothesize → retrieve / propose → ask user if fixed
         ↖ topic switch resets ↙              ↓ no → next hypothesis

UI extras from the API: mode, mode_label, status_badge, banner, slots, suggestions, hypothesis_chips.

---

Security & cost highlights

• Prompt injection guard on raw user text and as an ADK before_model_callback
• Rate limits (Redis sliding window, in-memory fallback): IP + session per minute → HTTP 429 + Retry-After
• Stable errors: {status, error, request_id} — no stack traces to clients
• FAQ cache: normalized question hash → answered payload (single-turn, TTL configurable)
• Cost controls: cheap router, mid relevance, mid answer model; early exit on clarify / off-topic; capped retrieval attempts and context size
• Structured JSON logs with request_id, latency, models, approx tokens — never log secrets or full doc payloads

See docs/SECURITY_AND_COST.md for the full checklist (note: that doc may still mention older deploy names).

---

Deploy

1. Set APP_ENV=production and a strong JWT_SECRET
2. Point REDIS_URL / QDRANT_URL (and optional QDRANT_API_KEY) at managed services
3. Reuse indexes when possible:
   • BM25 is baked into the Docker image (data/bm25.pkl)
   • Copy Qdrant once (no re-embed):

python ingestion/migrate_qdrant.py --source http://localhost:6333 --dest http://YOUR-QDRANT:6333

4. Build & run with the included Dockerfile / start.sh (serves API + static frontend)
5. Register at least one user via /api/auth/register
6. Prefer USE_CLAIM_VERIFIER=true in production

---

Evaluation

Question sets and runners live under eval/:

# example — see scripts in eval/ for the current runner
# python eval/<runner>.py

Results are stored as JSON under eval/results_*.json for regression comparison.

---

Try it (agent UX)

1. Start the API and open http://127.0.0.1:8000/
2. Log in (UI can auto-register a demo user in development)
3. Examples:
   • دیپلوی → clarification + suggestion chips
   • برنامه‌ام بعد از دیپلوی بالا نمیاد → troubleshooting banner + gather flow
   • After a proposal → verify chips «بله، حل شد» / «هنوز حل نشد»

---

License

Specify your license here (e.g. MIT). Documentation under docs/ may follow its own upstream license — check before redistributing.