Repository navigation
Ask cloud models for a list in one {"items": [...]} tool call; add scripts/compare_extraction.py - #67
Merged
Merged
Conversation
…dels on the same articles and diff them Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FomNXd75yhujHo6xyWtGGp
…s return the list in one tool call Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FomNXd75yhujHo6xyWtGGp
…one parallel call per item GPT-6 Luna in parallel-tools mode returned 1 event per article on 20 of 20 articles (both runs) and sometimes a single person. The one-object request returned 2-14 events and the missing people. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FomNXd75yhujHo6xyWtGGp
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 0926308405
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
…nged article sample in `compare_extraction`
Cached extractions from the one-call-per-item format were missing items, so
they must not be reused after the switch to one `{"items": [...]}` call.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FomNXd75yhujHo6xyWtGGp
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Requested by Alex · project thread
The bug
Cloud list requests (all four extraction calls) went out in Instructor's parallel-tools mode: the model gets a
Persontool and has to call it once per person, all in one reply. Nothing makes it call 12 times instead of once, and GPT-6 Luna often stopped early.On 20 Guantánamo articles, run twice:
Asking for the list as one
{"items": [...]}object, the way local calls already do, gave 2 to 14 events per article and the missing people. The experiment calledcloud_generation()with that wrapper model, which is the same request this PR now makes. Wall-clock time for 20 articles went from 34 s to about 90 s, because Luna now writes the full lists.Changes
src/utils/llm.py: anyList[Model]request is wrapped as{"items": [...]}and unwrapped (_unwrap_list()). Cloud calls useTOOLS/ANTHROPIC_TOOLSwithparallel_tool_calls=False. The parallel-tools modes and their fallback branch are gone. Multi-tool-call recovery now also accepts a list split across several{"items": [...]}calls.src/utils/extraction_cache.py: the cache key now includesLIST_REQUEST_FORMAT. Extractions cached under the old format were the under-extracted ones, so they miss and get redone instead of being reused.scripts/compare_extraction.py: runs the pipeline's extraction step (no merging) per model on the same articles, then diffs the names (run/compare). It refuses to reuse an output folder whose article sample differs. This is the script behind the numbers above. Its outputs contain article-derived text, so keep them off GitHub.CLAUDE.md: documents both.Full write-up: https://claude.ai/artifact/XVuWG6tLT3jvBRdNtA6BUi
Checks
The local equivalents of
just cipass: ruff check, ruff format check and the file-length check. 570 tests passed (-m "not asyncio").🤖 Generated with Claude Code
https://claude.ai/code/session_01FomNXd75yhujHo6xyWtGGp