Where to see the agent eval source code?

[Daisy Lin (ruqianq)]

Hi I am new to autogen, and interested in how autogen evaluate my agent workflow. I found the documentation here talks about agent_evals but I don see that in the source code, can anyone point me to the right place, thanks!
https://autogenhub.github.io/autogen/docs/reference/agentchat/contrib/agent_eval/

[Luis Cruz (artur282)]

The evals aren't under the docs tree — the documentation page describes the concept, the implementation lives in two places in the repository:

1. AutoGen Studio agent evals — python/packages/autogen-studio/autogenstudio/eval/

• judges.py — the judges that score a run
• runners.py — executes the test cases and collects the results
• orchestrator.py — drives the whole eval run
• data model for the evals: python/packages/autogen-studio/autogenstudio/datamodel/eval.py

2. AutoGenBench / agbench — python/packages/agbench/

• benchmarks/GAIA/ and benchmarks/HumanEval/ — each benchmark ships Templates/, Scripts/ and a config.yaml declaring its tasks
• the harness itself is in src/, packaged via pyproject.toml / setup.py (console script: agbench = "agbench.cli:main")

To run it locally, from a clone of the repo:

pip install -e autogen/python/packages/agbench
cd python/packages/agbench/benchmarks/HumanEval
python Scripts/init_tasks.py                      # materialises Tasks/*.jsonl
agbench run Tasks/human_eval_MagenticOne.jsonl
agbench tabulate Results/human_eval_MagenticOne

Note that agbench runs each repetition inside a fresh Docker container by default (--native exists but the README discourages it), and the README currently marks the plain pip install agbench path as deprecated in favour of installing from source.

If what you actually want is to score your own workflow without Studio, the smallest working pattern in the repo is python/samples/task_centric_memory/eval_self_teaching.py — its siblings eval_retrieval.py and eval_learning_from_demonstration.py show the same structure: define the tasks, run the agent, tabulate results. That is the shortest path from "here is my workflow" to "here is a number for it".