Repository navigation
Optimize review summaries for consistency - #24854
Conversation
 [Medium impact] The changes affect Frigate's AI review-summary workflow and its local testing tool. This PR appears safe to merge, with a non-blocking improvement to the new length guidance. Findings
|
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
| "scene": 'Describe how the sequence begins, then the progression of events: all significant movements and actions in order. For named subjects (those with a `←` separator in "Objects in Scene"), always use their name; do not replace them with generic terms. For unnamed objects (e.g., "person", "car"), refer to them naturally with articles (e.g., "a person", "the car"). Your description should align with and support the threat level you assign. Use no more than five sentences.', | ||
| "title": "Name the primary activity across the observations, together with the location. An activity is what is being done with objects, tools, or surfaces; locomotion through the scene qualifies as the activity only when no other interaction is observed. For named subjects, always use their name. For unnamed objects, refer to them naturally with articles.", | ||
| "shortSummary": "Briefly summarize the primary activity across the observations.", | ||
| "shortSummary": "Briefly summarize the primary activity across the observations in one sentence. For named subjects, always use their name.", |
There was a problem hiding this comment.
The new guidance limits sentences, but scene allows only 600 characters and shortSummary allows 140. Five sentences, or even one detailed sentence, can exceed those limits. Providers that enforce character limits can therefore still cut descriptions off mid-sentence. Include the character budgets in each style’s guidance and check them in the new test.
Knowledge Base Used: Classification and generative AI
Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!
There was a problem hiding this comment.
This is entirely expected, sentences is an easier metric for a model to target instead of characters. A cut off sentence is non-fatal for the fields.
There was a problem hiding this comment.
That makes sense. Sentence count is a more practical target for the models, and a cutoff in these fields is non-fatal. I’ll withdraw the character-budget suggestion.
Please read the contributing guidelines and the AI policy before submitting a PR. Every PR must be read and submitted by a person, and PRs that appear to be unreviewed AI output will be closed without review.
Proposed change
Ran through on all of the recommended models with a handful of clips with variations and found they were inconsistent. The annotated frames used different formatting and this caused issues.
Type of change
Additional information
For new features
AI disclosure
AI tool(s) used (e.g., Claude, Copilot, ChatGPT, Cursor):
How AI was used (e.g., code generation, code review, debugging, documentation):
Extent of AI involvement (e.g., generated entire implementation, assisted with specific functions, suggested fixes):
Human oversight: Describe what manual review, testing, and validation you performed on the AI-generated portions.
Checklist
enlocale.ruff format frigate)