Repository navigation
feat(ai project): add model deployment Azure Policy pre-checks and warnings - #10902
Open
Zhijie Huang (hund030) wants to merge 2 commits into
Open
Zhijie Huang (hund030) wants to merge 2 commits into
Zhijie Huang (hund030) wants to merge 2 commits into
Conversation
…rnings - Pre-check model deployments via ARM \�valuateDeploymentPolicies\ before writing config in \�zd ai project deployment add\ or submitting ARM deployments in \�zd provision\. - Emit non-blocking policy warnings with Foundry Govern links when reconciling adopted deployments in \�zd ai project add\ and enrich runtime ARM policy denial errors.
Zhijie Huang (hund030)
requested review from
a team,
JerryYang (JerryYangKai),
FanH (Siglud),
Yimin-Jin,
Eric Chen (eriolchan),
Glenn Harper (glharper),
Hui Miao (huimiu),
Wei Meng (m5i-work),
John Miller (therealjohn),
Travis Angevine (trangevi) and
JianW (v1212)
as code owners
November 11, 2026 07:55
|
Azure Pipelines: Successfully started running 2 pipeline(s). 25 pipeline(s) were filtered out due to trigger conditions. There may be pipelines that require an authorized user to comment /azp run to run. |
Contributor
There was a problem hiding this comment.
🟡 Changes recommended
Policy checks can evaluate the wrong project or stale models, miss existing accounts and nested denials, and leave configuration changes behind after a rejected deployment.
7 open issues
What changed in this PR
Adds Azure Policy pre-checks and denial diagnostics to model deployment workflows in the azure.ai.projects extension.
Changes:
- Evaluate model deployments before submission, blocking on
Denywhile allowingAudit, with opt-out settings and Foundry Govern links. - Integrate blocking checks into deployment authoring and provisioning, plus non-blocking warnings during existing-project adoption.
- Add structured policy errors, runtime ARM denial guidance, and unit tests.
Extension user-contract review: blocked (10 files inspected; 0 new contract comments; 0 matched threads; prior review history unavailable).
| File | Description |
|---|---|
cli/azd/extensions/azure.ai.projects/internal/provisioning/model_policy.go |
Implements policy evaluation, metadata lookup, opt-outs, links, and messages. |
cli/azd/extensions/azure.ai.projects/internal/provisioning/model_policy_test.go |
Tests evaluation, batching, metadata resolution, and provisioning checks. |
cli/azd/extensions/azure.ai.projects/internal/provisioning/foundry_provisioning_provider.go |
Invokes policy checks before ARM deployment submission. |
cli/azd/extensions/azure.ai.projects/internal/exterrors/errors.go |
Adds runtime policy-denial guidance and assignment extraction. |
cli/azd/extensions/azure.ai.projects/internal/exterrors/errors_test.go |
Tests guidance for ARM policy-denial responses. |
cli/azd/extensions/azure.ai.projects/internal/exterrors/codes.go |
Defines the model-policy blocking error code. |
cli/azd/extensions/azure.ai.projects/internal/cmd/project_deployment_reconcile.go |
Adds non-blocking policy warnings during deployment reconciliation. |
cli/azd/extensions/azure.ai.projects/internal/cmd/project_deployment_reconcile_test.go |
Tests reconciliation warnings and partial evaluation failures. |
cli/azd/extensions/azure.ai.projects/internal/cmd/project_deployment_add.go |
Checks the selected model before deployment-specific writes. |
cli/azd/extensions/azure.ai.projects/internal/cmd/project_deployment_add_test.go |
Tests policy rejection without deployment-specific mutations. |
Review effort: Max
- Route reconciliation policy warnings through injected \stderr\ writer (suppressed for \json\/\ one\), unwrap \properties.error\ ARM polling envelopes, and resolve \projectResourceId\/\deployments\ authoritatively from resolved template parameters. - Reject trailing empty segments on resource IDs and document model policy checks and \AZD_AI_AGENT_SKIP_POLICY_CHECKS\ in \README.md\.
Contributor
There was a problem hiding this comment.
🟡 Changes recommended
Policy evaluation can falsely block valid deployments by reusing mismatched live model metadata and treating unverified template parameters as proof of deployed model resources.
3 open issues
7 resolved since last review
Review effort: Max
Comment on lines
+298
to
+299
| if src != nil && src.parameters != nil { | ||
| return projectID, decodeModelDeploymentsValue(resolveTemplateParameterValue(src, "deployments")) |
| log.Printf("[FoundryPolicy] Could not resolve model metadata for deployment %q: %v", dep.Name, err) | ||
| continue | ||
| } | ||
| if fetched != nil { |
Comment on lines
+439
to
+440
| if target == nil || strings.TrimSpace(target.ResourceId) == "" || | ||
| provisioning.ShouldSkipModelPolicyChecks(environmentValues) { |
Contributor
azure.ai.projects PR buildNote This is an unsigned development build. Install it only if you trust this PR. Install the extension: azd ext install "https://azuresdkartifacts.z5.web.core.windows.net/azd/extensions/pr/10902/azure-ai-projects.zip"
|
JerryYang (JerryYangKai)
approved these changes
Nov 11, 2026
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Microsoft Foundry model deployments added or provisioned against an existing project with active Azure Policy
Denyassignments fail late during ARM template execution without Foundry Govern remediation links. This PR adds ARMevaluateDeploymentPoliciespre-checks, brownfield reconciliation warnings, and runtime ARM policy denial guidance inazure.ai.projects.Changes
internal/provisioning/model_policy.go: Evaluate model deployments viaPOST <accountId>/evaluateDeploymentPolicies?api-version=2026-03-15-preview(resolving missing metadata viaGET <accountId>/deployments/<name>), block only onDeny(Auditis non-blocking), honorAZD_AI_AGENT_SKIP_POLICY_CHECKS, and generate Foundry Govern portal deep links.internal/cmd/project_deployment_add.go,internal/provisioning/foundry_provisioning_provider.go: Hard-block policy-denied models (CodeModelBlockedByPolicy) before mutatingazure.yaml/.envinazd ai project deployment addor submitting ARM deployments inazd provision.internal/cmd/project_deployment_reconcile.go,internal/exterrors/errors.go: Emit non-blocking policy warnings duringazd ai project addbrownfield deployment reconciliation and enrich runtime ARMRequestDisallowedByPolicy/PolicyViolationerrors.Manual Validation
azure.ai.projectslocally viaazd x buildand tested againstxiaofhua-tool-policy-project(which enforcesxh-model-deny-policywithDenyandxh-model-audit-policywithAudit).azd ai project deployment addandazd provision --no-promptwith denied modelOpenAI/text-embedding-3-small; verified both hard-blocked withmodel_blocked_by_policyand the Foundry Govern link, while compliant/auditedOpenAI/gpt-4.1andAZD_AI_AGENT_SKIP_POLICY_CHECKS=truesucceeded.azd ai project add --project-id <id> --no-promptwith existing denied deploymenttext-embedding-3-smallinazure.yaml; verified project adoption succeeded and printed the non-blocking Azure Policy warning with the Govern link.Screen Shots