DEV-205663: Add rule template create, update and delete tools - #134
Open
regmimridul wants to merge 10 commits into
Open
DEV-205663: Add rule template create, update and delete tools#134regmimridul wants to merge 10 commits into
regmimridul wants to merge 10 commits into
Conversation
…nitors
Two read-only MCP tools over the public DQ job-run API, both keyed by run_id:
- dq_get_job_run_profile reads a run's column-level profiling statistics via a
new clients.GetDqJobRunProfile over GET /rest/dq/1.0/jobRuns/{id}/profile,
paginated with limit/offset and a derived hasMore.
- dq_get_job_run_monitors reads a run's adaptive and custom monitor results via
the existing clients.GetDqJobRunMonitors, adding each monitor's tolerance and
a summary counting monitors by state.
A run with no profile or no monitor results reports why instead of an empty
success, and HTTP 400/401/403/404/500 and transport failures map to
status/message/guidance as in dq_get_job_run.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
# Conflicts: # README.md # pkg/tools/register.go
docs/TOOL_CONTRIBUTION_STANDARDS.md section 4.1 requires the MCP tool name to spell out domain abbreviations, and section 4.2 requires qualifying nouns that collide across domains — "run" and "profile" mean different things to data quality, lineage and classification. dq_get_job_run_profile -> get_data_quality_job_run_profile dq_get_job_run_monitors -> get_data_quality_job_run_monitors Also aligns these two with their nearest siblings, which already use the long form: get_data_quality_rule, get_data_quality_rule_results, list_data_quality_rule_templates (section 6.5). Only the Name strings and LLM-facing prose change. Section 4.1 allows Go package directories to keep the short form, so pkg/tools/get_dq_job_run_* is unchanged, as are references to main's own dq_get_job_run tool. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
regmimridul
marked this pull request as ready for review
September 6, 2026 00:19
regmimridul
marked this pull request as draft
September 8, 2026 13:05
5 tasks
Introduces the data-quality experimental feature and puts get_data_quality_job_run_profile and get_data_quality_job_run_monitors behind it, so the DQ surface is opt-in rather than on by default. The flag identifier lives next to ContextSpecificationsFeature and is registered in knownExperimentalFeatures, so --experimental=data-quality, COLLIBRA_MCP_EXPERIMENTAL and mcp.experimental all accept it and it shows up in --help. Its description is deliberately generic: the rule template write tools on feature/DEV-205663 join the same gate without having to touch that entry. The annotation test now enables the new feature, keeping its "every gate on" contract intact, and a hidden/visible pair proves the gate actually gates - matching the existing debug-tool tests. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Three write tools over the public DQ rule template API, each behind the
confirm checkpoint required by docs/TOOL_CONTRIBUTION_STANDARDS.md section 5:
- create_data_quality_rule_template (POST /rest/dq/1.0/ruleTemplates)
- update_data_quality_rule_template (PUT /rest/dq/1.0/ruleTemplates/{name})
- delete_data_quality_rule_template (DELETE .../{name}?deleteDeployments=)
Three behaviours the ticket asks for are not in the API and are implemented
in the tools instead: partial-update semantics (the API's update is a
full-replacement PUT, so the template is read and merged), "template not
found" on delete (the API's delete is idempotent, so existence is checked
first), and refusing a delete while deployments are live (no such API error,
so deployedRuleCount is checked first).
The API has no way to update a template without cascading to its deployed
rules, so no cascade switch is exposed on update; the preview reports the
affected-rule count and the result reports per-deployment outcomes.
businessRuleLinks accepts Business Rule asset names or UUIDs and resolves
names to UUIDs before the write, reporting an unknown or ambiguous name
rather than guessing.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The create tool's sql field gave 'select * from @dataset where {{column}} is null' as its example. @dataset is not a rule template placeholder — it belongs to freeform SQL rules — so an agent following the example would author a template the service cannot resolve. Rule templates use {{dq-jobname}} for the job's table, per RuleTemplateMapperTest and RuleTemplatesBllTest in the dq repo and every custom template in the udq environment. Names both placeholders in the create and update tools' descriptions and sql field tags, and corrects the test fixtures to SQL the service would accept. generate_dq_rule_sql keeps @dataset, which is correct there. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Tolerance --------- Neither tool could set tolerance, so every template they created got the service default of 0 — one failing record fails the rule. Adds tolerance to create and update as *int, so an explicit 0 is distinguishable from "not supplied": omitted on create leaves the default to the service, and omitted on update keeps the stored value through the full-replacement PUT. Negative values are rejected before the network call. The create preview echoes it, per section 5.2. Dialect ------- The dialect field offered 'postgres' as an example and described the value as "not a fixed list in the API". SqlDialectTranslationService defines exactly ten supported dialects — snowflake, bigquery, oracle, sqlserver, spark, redshift, databricks, sap, athena, trino — matched with equalsIgnoreCase, and its own comment records that postgres and mysql are excluded because neither is a deployment target for DQ rules. Both tools now name the real set and say so. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Omitting tolerance on update keeps the stored value; the service default only applies on create. The field description asserted both. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…flag Moves create_, update_ and delete_data_quality_rule_template into the data-quality experimental gate introduced on feature/DEV-205661, so all five new DQ tools are opt-in through one flag rather than shipping enabled by default. No new machinery: the flag identifier, its knownExperimentalFeatures entry and the startup log line already come from the parent branch, and its description was written generically so this change does not touch them. The gated-tool list in the register test grows by the three names, which is what keeps the hidden/visible pair honest. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
regmimridul
force-pushed
the
feature/DEV-205663
branch
from
September 8, 2026 19:02
f8968c9 to
c42a233
Compare
…flag Extends the data-quality gate to the remaining 18 DQ tools, so all 23 are opt-in through one flag instead of five being gated and the rest shipping enabled. This makes the README's existing annotations true: eight tools already advertised a data-quality feature flag that nothing implemented. Registration order is unchanged, so the diff is the wrapper plus one indent level. search_catalog_columns stays ungated - it is a catalog Column search over the Knowledge Graph API and has nothing to do with DQ jobs or rules, despite sitting next to them in the list. Note this is a behaviour change for anyone running without the flag: the DQ tools disappear from tools/list until they opt in. create_data_quality_job is gated with the rest but has no README entry to annotate - a pre-existing documentation gap, left alone here rather than filled in with a guess. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
regmimridul
marked this pull request as ready for review
September 8, 2026 20:27
5 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
🎯 What does this PR do?
DEV-205663. Adds three write tools over the public DQ rule template API, so the template library can be managed programmatically instead of through the UI.
create_data_quality_rule_template—POST /rest/dq/1.0/ruleTemplates. Returns the created template with its assigned id; a duplicate name (409) is reported as such rather than overwriting.update_data_quality_rule_template—PUT /rest/dq/1.0/ruleTemplates/{name}. Partial update by name; returns the per-deployment cascade outcomes and reportspartialwhen some deployed rules were skipped.delete_data_quality_rule_template—DELETE /rest/dq/1.0/ruleTemplates/{name}?deleteDeployments=.cascade=false(default) refuses while deployments are live and reports the count.New client code in
pkg/clients/dq_template_write_client.go(create/update/delete, reusing the existingdqDohelper) andpkg/clients/business_rule_assets_client.go(Business Rule asset name → UUID resolution). Two small additions todq_templates_client.go: abusinessRuleAssetIdsfield onDQRuleTemplate, and aErrDQRuleTemplateNotFoundsentinel wrapped around the existing 404 so the write tools can distinguish "no such template" without matching on message text.All three carry the confirm checkpoint from
docs/TOOL_CONTRIBUTION_STANDARDS.md§5 —confirm=false(default) previews and writes nothing,confirm=trueperforms the write — and the preview echoes every field that will be sent (§5.2).Where the API does not do what the ticket assumed
Three acceptance criteria describe behaviour the public API does not provide. Each is implemented in the tool instead; flagging so the ticket can be corrected.
PUTand the payload requiresruleTemplateName,sql,dialect,description,dimensionstoleranceis carried through even though the tool cannot set it — otherwise a PUT would silently wipe itcascade(default false);falseleaves deployments untouchedDEPLOYED/SKIPPED/FAILED204even when nothing matchedGETfirstcascade=falsedeployedRuleCountis checked first and the delete is refused client-side, with the count in the messageTwo smaller mismatches:
descriptionanddimensionsare required, not optional. The API's write payload requires both (dimensionswithminItems: 1). Per §6.1 the tools reject a missing one up front with a self-correcting message rather than letting it surface as a raw 400.dialectis not an enum. The ticket describes it as one; the spec has a free-form string (1–255 chars) validated server-side. Documented that way, and the service's rejection is surfaced with context.Decisions taken against the ticket
create_data_quality_rule_template, notdq_create_rule_template— per §4.1, and matching the three sibling template tools already onmain(list_/get_/deploy_data_quality_rule_template, §6.5). Worth noting the newer job tools (dq_get_job,dq_get_job_run) use the short form, so the repo currently has both conventions live; that is a wider cleanup, not one for this PR.maintoday. §3.1 says write tools should sit behind an experimental flag, but thedata-qualityflag was removed in feat(dq): remove data-quality experimental flag DEV-215225 #126 and Revert #126: restore the data-quality experimental flag #129 proposes restoring it — if Revert #126: restore the data-quality experimental flag #129 lands, these should join that block.toleranceis not exposed as an input, since the ticket's field list omits it (§1.2, narrowest surface). It is preserved through updates rather than being settable.Impact Analysis
Medium — these are the first tools that mutate the rule template library, and
deletewithcascade=trueremoves rules from live jobs, which stops those checks running on future job runs. That path is guarded three ways: the delete is refused outright unlesscascade=trueis passed explicitly, the confirm checkpoint means the first call always previews and writes nothing, and the preview states the exact number of deployed rules that would go.deleteis the only tool markedDestructiveHint: true.Otherwise additive: three new tool packages, two new client files, three lines in
RegisterAll. The only changes to existing code are the two small additions todq_templates_client.godescribed above; the 404 sentinel keeps the same error text ordering andget_data_quality_rule_template's tests are unaffected.Out-of-the-box (system) templates are rejected before any write in both update and delete, matching the ticket's "read/apply only" scope.
Per §8.2 the client was written against
udq-app-client/oas/dq-v1-public-oas-spec.yamlin thedqrepo, not from guesswork. §8.3 contract tests on the DQ side are not part of this PR.Verified with
gofmt -l,go build ./...,go vet ./...andgo test— new package suites pluspkg/toolsandpkg/clients, all passing. Tests cover the confirm checkpoint writing nothing, the merge preserving untouched fields, cascade refusal with a live deployment count, name-vs-UUID business rule resolution including the ambiguous case, and 409/400 surfacing.✅ Checklist
🤖 Generated with Claude Code