deepeval

Python toolkit for Chinese LLMs, with flexible batch capacity, structured real-time visulization and automated accumulation for streaming, and explicit feedback on vendor-native parameter validation.

Learn GenAI and Agentic AI from Zero to Production

26-framework SDET monorepo: Playwright, Selenium, Cypress, Cucumber, Postman, FastAPI, Pact, AI/LLM eval (DeepEval), agentic AI (LangChain, LangGraph, DSPy, Claude), k6, flakiness detection, site monitoring, failure triage, QMS evidence (ISO 9001/SOC 2), dependency audit · GitHub Actions · DataDog · Terraform IaC

Advanced RAG pipelines for medical (HealthBench, MedCaseReasoning, MetaMedQA, PubMedQA) and financial (FinanceBench, Earnings Calls) QA. LangGraph orchestration + BAML structructed generation, Milvus Hybrid search (Dense + BM25 + RRF), three-layer Metadata Enrichment, Contextual AI instruction-following reranker, and DeepEval evaluation.

Advanced RAG pipeline optimization framework using DSPy. Implements modular RAG pipelines with Query-Rewriting, Sub-Query Decomposition, and Hybrid Search via Weaviate. Automates prompt tuning and few-shot selection using GEPA, SIMBA, MIPRO, COPRO, and BootstrapFewShot optimizers on datasets like FreshQA, HotpotQA, TriviaQA, Wikipedia and PubMedQA.

An example that combines Behave (BDD testing) with DeepEval (LLM evaluation) to create human-readable, stakeholder-friendly tests for AI Agents / chatbots.

🚀 Production-ready modular RAG monorepo: Local LLM inference (vLLM) • Hybrid retrieval with Qdrant • Semantic caching • Docling document parsing • Cross-encoder reranking • DeepEval evaluation • Full observability with Langfuse • Open WebUI chat interface • OpenAI-compatible API • Fully Dockerized

Self-Reflective Question Answering for Biomedical Reasoning. GRPO fine-tuning via QLoRA & Unsloth with rewards for correctness, relevance, groundness, utility & XML structure. Structured think → answer → self-reflection with context grading, relevance assessment & groundness evaluation. DeepEval LLM-as-a-Judge (GEval, Faithfulness, Relevancy).

Multi-agent LangGraph app turning raw requirements into backlog-ready user stories, ACs, and test cases. Live Gemini-Flash CI eval gate + nightly N=10 adversarial regression.

面向开发者的中文 Eval Harness 源码教材：解析 lm-evaluation-harness、Inspect AI、OpenAI Evals、Promptfoo、DeepEval 与 Harbor，覆盖任务、运行、评分、统计与发布门禁。

Production-grade prompt engineering curriculum — 7 modules covering foundations, reasoning, RAG, security, and evaluation. Model-agnostic (OpenAI/Anthropic/Ollama), runnable notebooks, mini-projects, Mermaid diagrams. Not prompt tips — prompt engineering as a discipline.

AI research assistant for prosthetics, orthotics & rehabilitation robotics papers. RAG pipeline with streaming citations, structured summaries, terminology extraction, and paper comparison. Built with Next.js 15, Claude, pgvector. Includes observability tracing and DeepEval evaluation pipeline.

A runtime-native AI intelligence platform for portfolio analysis

BNS-LexAI is an AI-powered legal information and case understanding assistant.

Automated LLM red-teaming framework testing 520+ adversarial prompts across 6 failure categories using DeepEval, LangSmith & Groq

A hybrid RAG system that answers complex questions about the UI GreenMetric Sustainable University Rankings by combining unstructured narrative guidelines with structured tabular data.

An intelligent research paper assistant that uses Retrieval-Augmented Generation (RAG) to help users search, understand, and interact with research papers through contextual question answering

Production RAG system in Python: Haystack pipelines, FastAPI SSE streaming, Qdrant hybrid retrieval, OpenAI embeddings, DeepEval golden-set evaluation, and Langfuse tracing. Includes latency benchmarks (P50/P95 TTFT), retrieval failure-mode analysis, and chunking-strategy decision logs.

Provider-agnostic coding-agent harness + DeepEval evaluation suite