Skip to content

DEV-205663: Add rule template create, update and delete tools - #134

Open
regmimridul wants to merge 10 commits into
mainfrom
feature/DEV-205663
Open

DEV-205663: Add rule template create, update and delete tools#134
regmimridul wants to merge 10 commits into
mainfrom
feature/DEV-205663

Conversation

@regmimridul

Copy link
Copy Markdown
Contributor

🎯 What does this PR do?

DEV-205663. Adds three write tools over the public DQ rule template API, so the template library can be managed programmatically instead of through the UI.

  • create_data_quality_rule_templatePOST /rest/dq/1.0/ruleTemplates. Returns the created template with its assigned id; a duplicate name (409) is reported as such rather than overwriting.
  • update_data_quality_rule_templatePUT /rest/dq/1.0/ruleTemplates/{name}. Partial update by name; returns the per-deployment cascade outcomes and reports partial when some deployed rules were skipped.
  • delete_data_quality_rule_templateDELETE /rest/dq/1.0/ruleTemplates/{name}?deleteDeployments=. cascade=false (default) refuses while deployments are live and reports the count.

New client code in pkg/clients/dq_template_write_client.go (create/update/delete, reusing the existing dqDo helper) and pkg/clients/business_rule_assets_client.go (Business Rule asset name → UUID resolution). Two small additions to dq_templates_client.go: a businessRuleAssetIds field on DQRuleTemplate, and a ErrDQRuleTemplateNotFound sentinel wrapped around the existing 404 so the write tools can distinguish "no such template" without matching on message text.

All three carry the confirm checkpoint from docs/TOOL_CONTRIBUTION_STANDARDS.md §5 — confirm=false (default) previews and writes nothing, confirm=true performs the write — and the preview echoes every field that will be sent (§5.2).

Where the API does not do what the ticket assumed

Three acceptance criteria describe behaviour the public API does not provide. Each is implemented in the tool instead; flagging so the ticket can be corrected.

Ticket AC API reality What this PR does
Update is PATCH, partial It is a PUT and the payload requires ruleTemplateName, sql, dialect, description, dimensions Reads the template and merges, so omitted fields keep their stored values. tolerance is carried through even though the tool cannot set it — otherwise a PUT would silently wipe it
Update takes cascade (default false); false leaves deployments untouched No such flag. The update and its cascade are one transaction No cascade switch is exposed. The preview reports how many deployed rules will be affected, and the result reports per-rule DEPLOYED/SKIPPED/FAILED
Delete errors if the template is missing Delete is idempotent204 even when nothing matched Existence is checked with a GET first
Delete errors if deployments exist and cascade=false No such API error deployedRuleCount is checked first and the delete is refused client-side, with the count in the message

Two smaller mismatches:

  • description and dimensions are required, not optional. The API's write payload requires both (dimensions with minItems: 1). Per §6.1 the tools reject a missing one up front with a self-correcting message rather than letting it surface as a raw 400.
  • dialect is not an enum. The ticket describes it as one; the spec has a free-form string (1–255 chars) validated server-side. Documented that way, and the service's rejection is surfaced with context.

Decisions taken against the ticket

  • Tool names spell out the domaincreate_data_quality_rule_template, not dq_create_rule_template — per §4.1, and matching the three sibling template tools already on main (list_/get_/deploy_data_quality_rule_template, §6.5). Worth noting the newer job tools (dq_get_job, dq_get_job_run) use the short form, so the repo currently has both conventions live; that is a wider cleanup, not one for this PR.
  • Delete has a confirm checkpoint, although the ticket says confirmation is the caller's responsibility. §5.1 is explicit that a checkpoint the model can skip is not a checkpoint, and this is the only destructive tool in the set. Happy to drop it if the team disagrees.
  • Registered ungated, matching the other DQ tools on main today. §3.1 says write tools should sit behind an experimental flag, but the data-quality flag was removed in feat(dq): remove data-quality experimental flag DEV-215225 #126 and Revert #126: restore the data-quality experimental flag #129 proposes restoring it — if Revert #126: restore the data-quality experimental flag #129 lands, these should join that block.
  • tolerance is not exposed as an input, since the ticket's field list omits it (§1.2, narrowest surface). It is preserved through updates rather than being settable.
  • No rename. The API supports renaming via the update payload, but it changes the template's URL and leaves no alias, and the ticket does not ask for it.

Impact Analysis

Medium — these are the first tools that mutate the rule template library, and delete with cascade=true removes rules from live jobs, which stops those checks running on future job runs. That path is guarded three ways: the delete is refused outright unless cascade=true is passed explicitly, the confirm checkpoint means the first call always previews and writes nothing, and the preview states the exact number of deployed rules that would go. delete is the only tool marked DestructiveHint: true.

Otherwise additive: three new tool packages, two new client files, three lines in RegisterAll. The only changes to existing code are the two small additions to dq_templates_client.go described above; the 404 sentinel keeps the same error text ordering and get_data_quality_rule_template's tests are unaffected.

Out-of-the-box (system) templates are rejected before any write in both update and delete, matching the ticket's "read/apply only" scope.

Per §8.2 the client was written against udq-app-client/oas/dq-v1-public-oas-spec.yaml in the dq repo, not from guesswork. §8.3 contract tests on the DQ side are not part of this PR.

Verified with gofmt -l, go build ./..., go vet ./... and go test — new package suites plus pkg/tools and pkg/clients, all passing. Tests cover the confirm checkpoint writing nothing, the merge preserving untouched fields, cascade refusal with a live deployment count, name-vs-UUID business rule resolution including the ambiguous case, and 409/400 surfacing.

✅ Checklist

  • My code follows the style guidelines of this project.
  • I have performed a self-review of my own code.
  • I have commented my code, particularly in hard-to-understand areas.
  • I have made corresponding changes to the documentation (if needed).
  • My commit messages follow the Conventional Commits standard.

🤖 Generated with Claude Code

regmimridul and others added 3 commits September 3, 2026 15:24
…nitors

Two read-only MCP tools over the public DQ job-run API, both keyed by run_id:

- dq_get_job_run_profile reads a run's column-level profiling statistics via a
  new clients.GetDqJobRunProfile over GET /rest/dq/1.0/jobRuns/{id}/profile,
  paginated with limit/offset and a derived hasMore.
- dq_get_job_run_monitors reads a run's adaptive and custom monitor results via
  the existing clients.GetDqJobRunMonitors, adding each monitor's tolerance and
  a summary counting monitors by state.

A run with no profile or no monitor results reports why instead of an empty
success, and HTTP 400/401/403/404/500 and transport failures map to
status/message/guidance as in dq_get_job_run.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
# Conflicts:
#	README.md
#	pkg/tools/register.go
docs/TOOL_CONTRIBUTION_STANDARDS.md section 4.1 requires the MCP tool name
to spell out domain abbreviations, and section 4.2 requires qualifying nouns
that collide across domains — "run" and "profile" mean different things to
data quality, lineage and classification.

  dq_get_job_run_profile  -> get_data_quality_job_run_profile
  dq_get_job_run_monitors -> get_data_quality_job_run_monitors

Also aligns these two with their nearest siblings, which already use the
long form: get_data_quality_rule, get_data_quality_rule_results,
list_data_quality_rule_templates (section 6.5).

Only the Name strings and LLM-facing prose change. Section 4.1 allows Go
package directories to keep the short form, so pkg/tools/get_dq_job_run_*
is unchanged, as are references to main's own dq_get_job_run tool.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@regmimridul
regmimridul marked this pull request as ready for review September 6, 2026 00:19
@regmimridul
regmimridul requested a review from a team as a code owner September 6, 2026 00:19
@regmimridul
regmimridul marked this pull request as draft September 8, 2026 13:05
regmimridul and others added 6 commits September 8, 2026 14:59
Introduces the data-quality experimental feature and puts
get_data_quality_job_run_profile and get_data_quality_job_run_monitors
behind it, so the DQ surface is opt-in rather than on by default.

The flag identifier lives next to ContextSpecificationsFeature and is
registered in knownExperimentalFeatures, so --experimental=data-quality,
COLLIBRA_MCP_EXPERIMENTAL and mcp.experimental all accept it and it shows
up in --help. Its description is deliberately generic: the rule template
write tools on feature/DEV-205663 join the same gate without having to
touch that entry.

The annotation test now enables the new feature, keeping its "every gate
on" contract intact, and a hidden/visible pair proves the gate actually
gates - matching the existing debug-tool tests.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Three write tools over the public DQ rule template API, each behind the
confirm checkpoint required by docs/TOOL_CONTRIBUTION_STANDARDS.md section 5:

- create_data_quality_rule_template (POST /rest/dq/1.0/ruleTemplates)
- update_data_quality_rule_template (PUT  /rest/dq/1.0/ruleTemplates/{name})
- delete_data_quality_rule_template (DELETE .../{name}?deleteDeployments=)

Three behaviours the ticket asks for are not in the API and are implemented
in the tools instead: partial-update semantics (the API's update is a
full-replacement PUT, so the template is read and merged), "template not
found" on delete (the API's delete is idempotent, so existence is checked
first), and refusing a delete while deployments are live (no such API error,
so deployedRuleCount is checked first).

The API has no way to update a template without cascading to its deployed
rules, so no cascade switch is exposed on update; the preview reports the
affected-rule count and the result reports per-deployment outcomes.

businessRuleLinks accepts Business Rule asset names or UUIDs and resolves
names to UUIDs before the write, reporting an unknown or ambiguous name
rather than guessing.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The create tool's sql field gave 'select * from @dataset where {{column}}
is null' as its example. @dataset is not a rule template placeholder — it
belongs to freeform SQL rules — so an agent following the example would
author a template the service cannot resolve. Rule templates use
{{dq-jobname}} for the job's table, per RuleTemplateMapperTest and
RuleTemplatesBllTest in the dq repo and every custom template in the udq
environment.

Names both placeholders in the create and update tools' descriptions and
sql field tags, and corrects the test fixtures to SQL the service would
accept. generate_dq_rule_sql keeps @dataset, which is correct there.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Tolerance
---------
Neither tool could set tolerance, so every template they created got the
service default of 0 — one failing record fails the rule. Adds tolerance to
create and update as *int, so an explicit 0 is distinguishable from "not
supplied": omitted on create leaves the default to the service, and omitted
on update keeps the stored value through the full-replacement PUT. Negative
values are rejected before the network call. The create preview echoes it,
per section 5.2.

Dialect
-------
The dialect field offered 'postgres' as an example and described the value
as "not a fixed list in the API". SqlDialectTranslationService defines
exactly ten supported dialects — snowflake, bigquery, oracle, sqlserver,
spark, redshift, databricks, sap, athena, trino — matched with
equalsIgnoreCase, and its own comment records that postgres and mysql are
excluded because neither is a deployment target for DQ rules. Both tools now
name the real set and say so.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Omitting tolerance on update keeps the stored value; the service default only
applies on create. The field description asserted both.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…flag

Moves create_, update_ and delete_data_quality_rule_template into the
data-quality experimental gate introduced on feature/DEV-205661, so all
five new DQ tools are opt-in through one flag rather than shipping
enabled by default.

No new machinery: the flag identifier, its knownExperimentalFeatures
entry and the startup log line already come from the parent branch, and
its description was written generically so this change does not touch
them. The gated-tool list in the register test grows by the three names,
which is what keeps the hidden/visible pair honest.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…flag

Extends the data-quality gate to the remaining 18 DQ tools, so all 23 are
opt-in through one flag instead of five being gated and the rest shipping
enabled. This makes the README's existing annotations true: eight tools
already advertised a data-quality feature flag that nothing implemented.

Registration order is unchanged, so the diff is the wrapper plus one
indent level. search_catalog_columns stays ungated - it is a catalog
Column search over the Knowledge Graph API and has nothing to do with DQ
jobs or rules, despite sitting next to them in the list.

Note this is a behaviour change for anyone running without the flag: the
DQ tools disappear from tools/list until they opt in. create_data_quality_job
is gated with the rest but has no README entry to annotate - a pre-existing
documentation gap, left alone here rather than filled in with a guess.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant