Issues

• match(numeric=True) reads 一万亿 as 10,000 and scores the wrong numeric target CORRECT

  UKGovernmentBEIS/inspect_ai#5928 · Filter by author XCODESSS opened on Oct 18, 2026

• frontier() marks lower-scoring models as frontier when dataframe index labels repeat

  UKGovernmentBEIS/inspect_ai#5926 · Filter by author XCODESSS opened on Oct 18, 2026

• samples_df() rejects valid time-only ISO strings in type=time columns

  UKGovernmentBEIS/inspect_ai#5925 · Filter by author XCODESSS opened on Oct 18, 2026

• log_viewer() generates incorrect sample links for IDs containing URL delimiters

  UKGovernmentBEIS/inspect_ai#5924 · Filter by author XCODESSS opened on Oct 18, 2026

• Fireworks provider rejects the stream model arg (-M stream=true)

  UKGovernmentBEIS/inspect_ai#5923 · Filter by author SunnySoldier357 opened on Oct 18, 2026

• S3 JSON log reads fail under Trio

  UKGovernmentBEIS/inspect_ai#5921 · Filter by author yinli-systems opened on Oct 17, 2026
  1

• Typed metadata discards Pydantic alias keys before validation

  UKGovernmentBEIS/inspect_ai#5920 · Filter by author j45856021-dev opened on Oct 17, 2026

• JSON log single-sample reads return stale contents after same-path replacement

  UKGovernmentBEIS/inspect_ai#5919 · Filter by author yinli-systems opened on Oct 17, 2026
  1

• read_eval_log_samples omits persisted SampleSource-generated samples

  UKGovernmentBEIS/inspect_ai#5918 · Filter by author yinli-systems opened on Oct 17, 2026
  1

• Bridged host-tool approval can be bypassed by re-declaring the tool under a different name

  UKGovernmentBEIS/inspect_ai#5917 · Filter by author AUTHENSOR opened on Oct 17, 2026

• inspect score --stream N --overwrite can replace the input log with a partial log

  UKGovernmentBEIS/inspect_ai#5914 · Filter by author tivris opened on Oct 17, 2026

• read_file() silently returns empty content for filenames such as report=2026.txt

  UKGovernmentBEIS/inspect_ai#5907 · Filter by author XCODESSS opened on Oct 17, 2026

• Anthropic native compaction under-counts usage: streamed responses drop usage.iterations, and cache tokens are not summed over iterations

  UKGovernmentBEIS/inspect_ai#5904 · Filter by author tivris opened on Oct 16, 2026

• tool_with(): if/elif chain silently drops viewer/model_input when combined with parallel (or with each other)

  UKGovernmentBEIS/inspect_ai#5896 · Filter by author BlueX888 opened on Oct 16, 2026
  1

• sandbox exec: a command timeout and an unresponsive sandbox raise the same SandboxTimeoutError

  UKGovernmentBEIS/inspect_ai#5894 · Filter by author MattFisher opened on Oct 16, 2026
  2

• Gemini API agent bridge ignores snake_case function_declarations (LiteLLM), so the model gets no tools

  UKGovernmentBEIS/inspect_ai#5891 · Filter by author taxfree-python opened on Oct 16, 2026

• Chat completions agent bridge drops Gemini thought signatures: 400 "Function call is missing a thought_signature" on the second turn

  UKGovernmentBEIS/inspect_ai#5890 · Filter by author taxfree-python opened on Oct 16, 2026

• Metrics dict silently drops a second variant of the same metric (e.g. stderr() and stderr(cluster=...))

  acceptedbug
  UKGovernmentBEIS/inspect_ai#5887 · Filter by author hzmsh opened on Oct 15, 2026
  1

• multi-turn agent runs with openrouter/anthropic/* models do not correctly cache input tokens

  acceptedbug
  UKGovernmentBEIS/inspect_ai#5873 · Filter by author mileswhiticker opened on Oct 15, 2026

• model_graded_fact() and model_graded_qa() show the grader multiple targets run together ("ParisCity of Light")

  UKGovernmentBEIS/inspect_ai#5864 · Filter by author wguo7 opened on Oct 14, 2026

• score(action="append") names the new score differently on errored samples

  UKGovernmentBEIS/inspect_ai#5863 · Filter by author hzmsh opened on Oct 14, 2026

• Eval header shows a bare ... for any string value over 50 characters without whitespace

  UKGovernmentBEIS/inspect_ai#5857 · Filter by author ynaamane opened on Oct 14, 2026

• Google: Inspect's HTTP hooks see no requests on asyncio

  UKGovernmentBEIS/inspect_ai#5842 · Filter by author petetronic opened on Oct 13, 2026

• Model output cache never hits on Windows for model names containing ":" (Ollama tags, Bedrock IDs)

  UKGovernmentBEIS/inspect_ai#5840 · Filter by author wguo7 opened on Oct 13, 2026
  2

• Score multiple choice by the probability of the correct answer

  UKGovernmentBEIS/inspect_ai#5832 · Filter by author taziksh opened on Oct 12, 2026
  2