Skip to content

feat: silently ignore unknown /-shorthand comments (foreign-bot friendly)#273

Open
eg-ayoub wants to merge 1 commit into
mainfrom
improvement/PTFE-3408-add-command-ignore-list
Open

feat: silently ignore unknown /-shorthand comments (foreign-bot friendly)#273
eg-ayoub wants to merge 1 commit into
mainfrom
improvement/PTFE-3408-add-command-ignore-list

Conversation

@eg-ayoub

@eg-ayoub eg-ayoub commented Jul 15, 2026

Copy link
Copy Markdown

Bert-E's reactor treats any comment starting with /word as a shorthand for one of its own commands, and posts an "UnknownCommand" reply when the keyword doesn't match a registered command. On pull requests shared with bots such as CodeRabbit, Gemini or GitHub Copilot, that behaviour produces noisy "unknown command" replies every time a user runs /coderabbit review, /gemini …, /copilot …, etc.

Approach

Use the reactor's own command/option registry as an implicit allowlist. When an unknown keyword arrives on the / shorthand path, silently drop it unless it looks like a typo of a registered keyword. "Looks like a typo" is determined with difflib.get_close_matches (default cutoff 0.6).

No new configuration surface — the behaviour is derived from the commands and options bert-e already registers.

Behaviour matrix

/ shorthand path only; comments that explicitly address Bert-E via its @<robot> mention are unaffected and always dispatch as before.

comment outcome
/help registered command fires
/hlep close match to help → UnknownCommand reply (typo hint)
/apporve close match to approve → UnknownCommand reply (typo hint)
/coderabbit review no close match → silently ignored
/gemini … no close match → silently ignored
/copilot … no close match → silently ignored
@bert-e coderabbit still raises UnknownCommand (explicit prefix)

Cutoff choice

Empirically validated against the reactor's actual registry (Reactor.get_commands() + Reactor.get_options() after commands.setup()):

  • Foreign-bot names (coderabbit, copilot, gemini, devin, claude, other-bot-name, …) → no close match at cutoffs 0.6, 0.7 or 0.75.
  • Realistic typos (apporve, hlep, wiat, staus, resset, …) → match at all three cutoffs.

Sticking with the Python default (0.6).

Ticket: PTFE-3408

@eg-ayoub
eg-ayoub requested a review from a team as a code owner July 15, 2026 13:22
@eg-ayoub
eg-ayoub force-pushed the improvement/PTFE-3408-add-command-ignore-list branch from 7f60778 to 813e7a2 Compare July 15, 2026 13:48
@codecov

codecov Bot commented Jul 15, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 94.20290% with 4 lines in your changes missing coverage. Please review.
✅ Project coverage is 89.65%. Comparing base (6ddd9a5) to head (a2014a1).

Files with missing lines Patch % Lines
bert_e/tests/unit/test_reactor.py 92.30% 4 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main     #273      +/-   ##
==========================================
+ Coverage   89.62%   89.65%   +0.02%     
==========================================
  Files          81       81              
  Lines       10614    10681      +67     
==========================================
+ Hits         9513     9576      +63     
- Misses       1101     1105       +4     
Flag Coverage Δ
integration 87.20% <76.47%> (-0.03%) ⬇️
tests 87.17% <76.47%> (-0.03%) ⬇️
tests-BuildFailedTest 26.62% <17.64%> (-0.02%) ⬇️
tests-QuickTest 34.13% <17.64%> (-0.03%) ⬇️
tests-RepositoryTests 26.29% <17.64%> (-0.02%) ⬇️
tests-TaskQueueTests 51.32% <41.17%> (-0.04%) ⬇️
tests-TestBertE 65.51% <76.47%> (+0.01%) ⬆️
tests-TestQueueing 53.52% <29.41%> (-0.05%) ⬇️
tests-api-mock 15.23% <0.00%> (-0.10%) ⬇️
tests-noqueue 77.42% <76.47%> (-0.01%) ⬇️
tests-noqueue-BuildFailedTest 26.62% <17.64%> (-0.02%) ⬇️
tests-noqueue-QuickTest 34.13% <17.64%> (-0.03%) ⬇️
tests-noqueue-RepositoryTests 26.29% <17.64%> (-0.02%) ⬇️
tests-noqueue-TaskQueueTests 51.32% <41.17%> (-0.04%) ⬇️
tests-noqueue-TestBertE 61.91% <76.47%> (+0.02%) ⬆️
tests-noqueue-TestQueueing 26.31% <17.64%> (-0.02%) ⬇️
tests-server 28.09% <4.34%> (-0.15%) ⬇️
unittests 43.69% <91.30%> (+0.35%) ⬆️
utests 28.55% <91.30%> (+0.45%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@matthiasL-scality

Copy link
Copy Markdown
Contributor

Design feedback

ignored_slash_commands: blocklist vs allowlist

The current approach requires administrators to maintain a blocklist of every foreign bot keyword — a list that will inevitably grow stale as new AI tools enter the workflow (Devin, new Copilot commands, etc.).

An allowlist feels more maintainable and logically sounder: Bert-E already has an internal registry of its own recognised commands (via @reactor.command). Any /keyword shorthand that does not match a registered command could simply be dropped silently, with zero configuration required.

The main tradeoff to weigh: a user who mistypes /apporve would no longer receive an "unknown command" reply. That regression can be mitigated by convention — the explicit @<robot> approve form always goes through the full dispatch path and would still produce feedback.

Worth discussing whether that trade is acceptable before the setting lands in production configs.


bypass_author_approval should be a separate PR

The change in integration.py (making bypass_author_approval trigger integration-branch creation) and the corresponding test additions are independent of the slash-command filtering feature. Bundling them means both changes ship in the same release, which makes changelogs harder to read and rollbacks harder to scope. Please consider splitting it into its own PR so the two fixes can be released and reverted independently.

@eg-ayoub
eg-ayoub force-pushed the improvement/PTFE-3408-add-command-ignore-list branch from 813e7a2 to 3d677d4 Compare July 15, 2026 14:40
@eg-ayoub

Copy link
Copy Markdown
Author

ok @matthiasL-scality I will first split this into two PRs and we'll look at them separately.

@eg-ayoub
eg-ayoub force-pushed the improvement/PTFE-3408-add-command-ignore-list branch 2 times, most recently from e9fdd63 to 7422ee2 Compare July 16, 2026 09:43
@eg-ayoub

Copy link
Copy Markdown
Author

@matthiasL-scality this is the offshoot PR #274 for the approve issue

…mmand

Bert-E's reactor treats any comment starting with /word as a shorthand for
one of its own commands, and posts an "UnknownCommand" reply when the
keyword doesn't match a registered command. On pull requests shared with
bots such as CodeRabbit, Gemini or GitHub Copilot, that behaviour produces
noisy "unknown command" replies every time a user runs /coderabbit review,
/gemini ..., /copilot ..., etc.

Use the reactor's own command/option registry as an implicit allowlist:
when an unknown keyword arrives on the / shorthand path, silently drop it
unless it looks like a typo of a registered keyword. "Looks like a typo"
is determined with difflib.get_close_matches (default cutoff 0.6,
empirically clean against realistic bot names).

Behaviour matrix (/ shorthand path only; @<robot> mentions are unaffected
and always dispatch as before):

  /help               -> registered command fires
  /hlep               -> close match to `help` -> UnknownCommand
  /apporve            -> close match to `approve` -> UnknownCommand
  /coderabbit review  -> no close match -> silently ignored
  /gemini ...         -> no close match -> silently ignored

PTFE-3408

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@eg-ayoub
eg-ayoub force-pushed the improvement/PTFE-3408-add-command-ignore-list branch from 7422ee2 to a2014a1 Compare July 17, 2026 07:29
@eg-ayoub eg-ayoub changed the title feat: add ignored_slash_commands setting to skip foreign-bot comments feat: silently ignore unknown /-shorthand comments (foreign-bot friendly) Jul 17, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants