Skip to content

fix(spec): hold the identifier check to declared names, not to metadata.extensions - #2707

Open
uny wants to merge 4 commits into
a2ui-project:mainfrom
uny:spec/identifier-check-subschema-only
Open

uny wants to merge 4 commits into
a2ui-project:mainfrom
uny:spec/identifier-check-subschema-only

Conversation

@uny

@uny uny commented Sep 20, 2026

Copy link
Copy Markdown

Description

validate_catalogs_identifiers in specification/v1_0/test/run_tests.py walks every dict and list under a component or function definition and checks the keys of any properties object it meets against UAX #31 (str.isidentifier()). That reaches into metadata.extensions, which common_types.json's Extensions types as opaque vendor JSON — keys are identifiers, values are unconstrained (patternProperties: {…: {}}). The naming rule itself (docs/a2ui_protocol.md line 529) covers "component names, function names, and argument/property names": the names a catalog declares.

So a catalog component carrying

"metadata": {
  "extensions": {
    "com_example_legacy": { "properties": { "x-legacy-name": { "type": "string" } } }
  }
}

passes the structural (ajv) check against catalog_definition.json and is refused by the identifier check with Invalid argument/property name: 'x-legacy-name', although no stated rule constrains what is inside an extension value.

This PR:

  1. factors the walk into iter_subschemas, which yields the direct subschemas of a schema through JSON Schema subschema positions only (properties, items, prefixItems, allOf/anyOf/oneOf, not, if/then/else, $defs, patternProperties, additionalProperties, dependentSchemas, contentSchema, … plus the legacy definitions/dependencies that ajv still evaluates under --spec=draft2020) and skips annotation values such as metadata, description, examples; validate_catalogs_identifiers checks properties keys through it;
  2. adds exactly such an extension to test/testing_catalog.json as the fixture;
  3. adds validate_identifier_walk, a harness step that places an invalid name under every keyword the walk must follow (and inside metadata.extensions / examples, where it must not look) and checks the exact names reported — the shipped catalogs only ever carry valid names, so without this a keyword dropped from the tables would go unnoticed.

Fixes #2705.

Reproduction

No workflow runs run_tests.py, so:

cd specification/v1_0/test && yarn install && python3 run_tests.py
  • main + the fixture alone: structural check passes, identifier check fails with the message above (166 passed / 1 failed).
  • This branch: 193 passed (167 + 26 synthetic cases).
  • Non-regression is part of the run: a hyphenated key under allOf → items → properties, under a function's args.properties, under a declared property named metadata, and under each of the 20 subschema keywords is still refused, and dropping a keyword from the tables fails its case.

Pre-launch Checklist

One time:

For this PR:

  • I have updated the relevant CHANGELOG.md file. — none under specification/
  • I updated/added relevant documentation. — none needed; behaviour now matches the prose
  • My code changes (if any) have tests. — the fixture in testing_catalog.json and the synthetic cases in validate_identifier_walk
  • If my branch is on a fork, I have verified that scripts/e2e_test.sh passes. — N/A, specification/v1_0/test only; python3 run_tests.py, pyink --check and prettier --check pass

🤖 Generated with Claude Code

uny added 3 commits September 20, 2026 13:09
…ta.extensions

validate_catalogs_identifiers walked every dict and list under a component
or function definition and checked the keys of any `properties` object it
met against UAX a2ui-project#31. That reaches into `metadata.extensions`, which
common_types.json's `Extensions` types as opaque vendor JSON, so an
extension whose value carries a JSON Schema fragment with a hyphenated key
was refused with "Invalid argument/property name" although the naming rule
covers only the names a catalog declares.

Descend through JSON Schema subschema positions only (`properties`, `items`,
`allOf`/`anyOf`/`oneOf`, `$defs`, `additionalProperties`, ...), and add such
an extension to testing_catalog.json: it passes the structural check
already and now passes the identifier check too, while a hyphenated key in
a real subschema position is still refused.
…lk with synthetic cases

Review of the subschema-only walk found two positions it no longer reached
that ajv still evaluates under `--spec=draft2020`: schema-valued
`dependencies` (legacy, but honoured) and `contentSchema`. Add both.

Factor the walk into `iter_subschemas`, which yields `(pointer, subschema)`
for every JSON Schema subschema position and nothing else, so other checks
over catalog schemas can share the one definition of "what is schema and
what is annotation".

The shipped catalogs only ever carry valid names, so a keyword dropped from
the SUBSCHEMA_* tuples went unnoticed. Add a harness step that runs the walk
over synthetic schemas and checks the exact names it reports: hyphens under
`allOf -> items -> properties`, function `args`, a declared property named
`metadata` and `dependencies` are refused; the same key inside
`metadata.extensions` or `examples` is not.
…the walk

Under `--spec=draft2020 --strict=false` ajv ignores `additionalItems`, so
walking it claimed a rationale that does not hold; `definitions` and
`dependencies` are still evaluated and stay.

The synthetic cases covered only a few keywords, so most of the SUBSCHEMA_*
tuples could lose an entry unnoticed. Place an invalid name under every
keyword the walk must follow, with the keywords spelled out in the case
block rather than derived from the tuples, so a dropped keyword still has a
failing case.
@google-cla

google-cla Bot commented Sep 20, 2026

Copy link
Copy Markdown

Thanks for your pull request! It looks like this may be your first contribution to a Google open source project. Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA).

View this failed invocation of the CLA check for more information.

For the most up to date status, view the checks section at the bottom of the pull request.

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces a utility to recursively validate property names against UAX #31 identifiers within JSON schemas, including comprehensive synthetic test cases to ensure the identifier walk correctly traverses subschemas. The reviewer suggested an optimization for the iter_subschemas function to improve performance and readability by using a single dictionary mapping for keyword categories instead of multiple tuples. No review comments were removed.

Comment thread specification/v1_0/test/run_tests.py Outdated
Review suggestion: one keyword -> shape dictionary, iterated over the keys
the schema actually has, instead of three tuples looked up on every node.
@Varun-S10

Copy link
Copy Markdown
Collaborator

Hi @uny, I have reviewed and verified the changes. Thank you for your contribution.

Hi @gspencergoog, I would like to escalate this PR. Could you please take a look?

@Varun-S10 Varun-S10 added status: needs review and removed status: needs-triage auto-managed: https://github.com/a2ui-project/a2ui/blob/main/scripts/triage.mjs labels Sep 22, 2026

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[BUG]: run_tests.py's identifier check walks metadata.extensions, which the schema and prose treat as opaque vendor data

2 participants