Resume Evaluation Engine

A state-driven, multi-agent reasoning system built with LangGraph + FastAPI for evaluating resumes against job descriptions.

---

Architecture

POST /evaluate
      │
      ▼
 FastAPI (validation + routing only)
      │
      ▼
 LangGraph Execution (one per request)
      │
      ├── [1] JD Interpreter Agent         ← LLM | Structures JD into hiring intent
      │
      ├── [2] Resume Interpreter Agent      ← LLM | Normalizes resume into profile
      │
      └── [3] Eligibility Gate Agent        ← DETERMINISTIC | Mandatory skills + experience check
                    │
            ┌───────┴────────┐
          [FAIL]           [PASS]
            │                 │
            ▼                 ├── [4] Hybrid Skill Matching    ← Zilliz + LLM
     Early Rejection          │
            │                 ├── [5] Experience Reasoning     ← LLM
            ▼                 │
           END                ├── [6] LLM Judge                ← LLM (senior panelist)
                              │
                              ├── [7] Critic Agent             ← LLM (devil's advocate)
                              │
                              └── [8] Final Decision Agent     ← Weighted scoring + LLM
                                            │
                                            ▼
                                           END

---

Agent Responsibilities

#	Agent	Type	Reads	Writes
1	JD Interpreter	LLM	jd_text	jd_structured
2	Resume Interpreter	LLM	resume_text	resume_structured
3	Eligibility Gate	Deterministic	jd_structured, resume_structured	eligibility_passed
4	Hybrid Skill Matching	Zilliz + LLM	jd_structured, resume_structured	keyword_match_score, semantic_similarity_score, skill_score
5	Experience Reasoning	LLM	jd_structured, resume_structured	experience_score
6	LLM Judge	LLM	All scores + structures	llm_judgement
7	Critic	LLM	llm_judgement + scores	critic_feedback
8	Final Decision	LLM + Math	Everything	final_decision

---

Scoring Formula

weighted_score = skill_score × 0.35
               + experience_score × 0.30
               + keyword_match_score × 0.20
               + semantic_similarity_score × 0.15

blended_score = 0.60 × weighted_score
              + 0.40 × (judge_base_score × judge_confidence)

adjusted_score = clamp(blended_score + critic_adjustment, 0.0, 1.0)

decision:
  adjusted_score >= 0.65  →  shortlist
  0.40 ≤ score < 0.65     →  hold
  score < 0.40            →  reject

---

API

POST /evaluate

Request:

{
  "jd_text": "We are looking for a Senior Python Engineer...",
  "resume_text": "John Doe | 8 years experience | Python, FastAPI..."
}

Response:

{
  "decision": "shortlist",
  "final_score": 0.782,
  "explanation": "Candidate demonstrates strong Python and FastAPI expertise with 8 years of relevant backend engineering experience. Domain background in cloud infrastructure aligns well with the role's requirements. Minor gap in Kubernetes experience is offset by transferable container orchestration skills.",
  "trace_id": "3f7a1c2e-8b4d-4e6f-a321-9c0b5e7d2a18"
}

GET /trace/{trace_id}

Retrieve full reasoning trace for auditability.

GET /health

Liveness check.

---

Setup

1. Clone and install

cd resume_evaluator
pip install -r requirements.txt

2. Configure environment

cp .env.example .env
# Edit .env with your credentials:
# OPENAI_API_KEY=sk-...
# ZILLIZ_URI=https://your-cluster.zillizcloud.com
# ZILLIZ_TOKEN=your_token
# MONGODB_URI=mongodb://localhost:27017

3. Run

python run.py
# or
uvicorn api.main:app --reload --host 0.0.0.0 --port 8000

4. Test

curl -X POST http://localhost:8000/evaluate \
  -H "Content-Type: application/json" \
  -d '{
    "jd_text": "Senior Python Engineer role requiring 5+ years Python, FastAPI, PostgreSQL, Docker, AWS experience. Team lead responsibilities.",
    "resume_text": "Jane Smith. 7 years software engineering. Python (Expert), FastAPI (Expert), Django (Intermediate), PostgreSQL, Docker, Kubernetes. Led team of 4 engineers. AWS certified. Domain: B2B SaaS, fintech."
  }'

---

Project Structure

resume_evaluator/
├── run.py                          # Server entry point
├── requirements.txt
├── .env.example
├── api/
│   └── main.py                     # FastAPI app (routing only, no reasoning)
├── core/
│   ├── state.py                    # Shared EvaluationState TypedDict
│   ├── llm.py                      # LLM + embedding client singletons
│   └── graph.py                    # LangGraph workflow definition
├── agents/
│   ├── jd_interpreter.py           # Agent 1
│   ├── resume_interpreter.py       # Agent 2
│   ├── eligibility_gate.py         # Agent 3 (non-LLM)
│   ├── hybrid_skill_matching.py    # Agent 4
│   ├── experience_reasoning.py     # Agent 5
│   ├── llm_judge.py                # Agent 6
│   ├── critic.py                   # Agent 7
│   └── final_decision.py           # Agent 8
└── db/
    ├── mongo.py                    # MongoDB persistence
    └── zilliz.py                   # Zilliz vector DB client

---

Design Principles

• State-isolated: Each agent reads only what it needs; writes only its owned fields
• Deterministic thresholds: Final decision is math-first, LLM-explained
• Early rejection: Non-LLM gate stops processing immediately on hard failures
• Hybrid matching: Keyword + semantic + LLM for nuanced skill evaluation
• Self-correction: Critic agent challenges every Judge output
• Full auditability: Every agent step logged to MongoDB with timestamps
• Graceful degradation: Zilliz failure falls back to set-intersection scoring