`schema`-type rule leaks the validated document via process-global validator cache

[nd-jharn]

Summary

vacuum keeps a process-global schema validator, parser.globalValidator (parser/json_schema.go), created once via sync.Once and never reset. Every rule that uses the built-in schema function validates through it (parser.ValidateNodeAgainstSchema).

That validator owns a SchemaCache that is never cleared, and each cache entry retains the *base.Schema it compiled. Through GoLow() that schema holds the validated document's index.SpecIndex -> index.Rolodex -> root *yaml.Node -> the entire parsed YAML tree.

In a long lived process (e.g. validation service/daemon), every document validated with a ruleset that uses the schema function stays pinned in memory for the lifetime of the process, even after the caller Release()s the Document and after RuleSetExecutionResult.Release() (which calls libopenapi.ClearAllCaches()) has run. (Which we had attempted previously)
ClearAllCaches() does not help either because the global validator's cache is owned by vacuum/libopenapi-validator, not by the libopenapi caches that call clears.

Environment

• vacuum v0.30.0 (initially reproduced on v0.29.10)
• Go 1.24

Minimal reproduction

mkdir vacuumleak && cd vacuumleak
go mod init vacuumleak
go get github.com/daveshanley/vacuum@v0.30.0
go run .

main.go:

// Minimal reproduction of a memory leak in vacuum's `schema` function.
package main

import (
	"fmt"
	"runtime"
	"runtime/debug"
	"strings"

	"github.com/daveshanley/vacuum/motor"
	"github.com/daveshanley/vacuum/rulesets"
	"github.com/pb33f/libopenapi"
)

// a self-contained OpenAPI 3.0 document with many component schemas.
func makeSpec(n int) []byte {
	var b strings.Builder
	b.WriteString("openapi: 3.0.3\ninfo:\n  title: leak-repro\n  version: \"1.0.0\"\npaths: {}\ncomponents:\n  schemas:\n")
	for i := 0; i < n; i++ {
		fmt.Fprintf(&b, "    Thing%d:\n      type: object\n      properties:\n        id: { type: string }\n        name: { type: string }\n        nested: { $ref: '#/components/schemas/Nested%d' }\n", i, i)
		fmt.Fprintf(&b, "    Nested%d:\n      type: object\n      properties:\n        value: { type: integer }\n        tags: { type: array, items: { type: string } }\n", i)
	}
	return []byte(b.String())
}

// a ruleset that does not trigger the leak:
const controlRuleset = `
rules:
  name-is-string:
    given: $.components.schemas[*].properties.name.type
    severity: error
    then:
      function: pattern
      functionOptions: { match: "^string$" }
`

// trigger using the `schema` function:
const schemaRuleset = `
rules:
  schema-is-object:
    given: $.components.schemas[*]
    severity: error
    then:
      function: schema
      functionOptions:
        forceValidationOnCurrentNode: true
        schema: { type: object, required: [type] }
`

func liveHeap() (uint64, uint64) {
	for i := 0; i < 4; i++ { // full GC + return to OS, so we measure only live memory
		debug.FreeOSMemory()
	}
	var m runtime.MemStats
	runtime.ReadMemStats(&m)
	return m.HeapAlloc / (1024 * 1024), m.HeapObjects
}

func loadRuleSet(y string) *rulesets.RuleSet {
	parsed, err := rulesets.CreateRuleSetFromData([]byte(y))
	if err != nil {
		panic(err)
	}
	return rulesets.BuildDefaultRuleSets().GenerateRuleSetFromSuppliedRuleSet(parsed)
}

func loop(label string, rs *rulesets.RuleSet, spec []byte) {
	h, o := liveHeap()
	fmt.Printf("[%-7s] before        heap=%3dMiB liveObjects=%d\n", label, h, o)
	for i := 1; i <= 5; i++ {
		doc, err := libopenapi.NewDocument(spec) // a fresh, caller-owned document each run
		if err != nil {
			panic(err)
		}
		res := motor.ApplyRulesToRuleSet(&motor.RuleSetExecution{
			RuleSet: rs, Document: doc, SpecInfo: doc.GetSpecInfo(), SpecFileName: "openapi.yaml",
		})
		res.Release() // releases vacuum-owned resources AND calls libopenapi.ClearAllCaches()
		doc.Release() // caller releases its own document
		h, o := liveHeap()
		fmt.Printf("[%-7s] after run #%d  heap=%3dMiB liveObjects=%d\n", label, i, h, o)
	}
}

func main() {
	spec := makeSpec(500)
	fmt.Printf("spec size: %d KiB\n", len(spec)/1024)
	loop("control", loadRuleSet(controlRuleset), spec) // stays flat: document is reclaimed
	loop("schema", loadRuleSet(schemaRuleset), spec)   // jumps and stays: one document pinned forever
}

Output

spec size: 151 KiB
[control] before        heap=  2MiB liveObjects=10535
[control] after run #1  heap=  2MiB liveObjects=11890
[control] after run #5  heap=  2MiB liveObjects=11990     <- flat; each document is fully reclaimed
[schema ] before        heap=  2MiB liveObjects=12001
[schema ] after run #1  heap=  8MiB liveObjects=74892     <- +6MiB / +63k live objects retained...
[schema ] after run #5  heap=  8MiB liveObjects=74889     <- ...and never freed (a full document tree)

The control ruleset (pattern function) leaves memory flat (the document is reclaimed after each run).
The schema ruleset permanently retains a whole parsed document (here ~6mb from a 151kb spec; with a multi-megabyte spec like we initially saw this happen on it is 100+ MiB). It never drops, no matter how many GC cycles you (force to) run or how many fresh documents are validated afterwards.

Root cause

A heap reference-chain analysis (via goref) of the retained objects shows the GC root is the global validator:

github.com/daveshanley/vacuum/parser.globalValidator          (process-global, sync.Once, never reset)
 - options            [*libopenapi-validator/config.ValidationOptions]
    - SchemaCache      [cache.SchemaCache] (sync.Map, never cleared)
       - SchemaCacheEntry.Schema   [*high/base.Schema]       (this retains the source schema)
          - GoLow -> [*low/base.SchemaProxy]
             - idx    [*index.SpecIndex]                                             (the validated documents index)
                - rolodex -> rootNode -> the entire parsed YAML tree

Relevant code:

• vacuum -> parser/json_schema.go -> globalValidator / getGlobalValidator is a sync.Once singleton created with the default, caching-enabled options (in schema_validation.NewSchemaValidator()) and never reset.
• libopenapi-validator -> schemaValidator.validateSchemaWithVersion stores compiled.ToCacheEntry(schema) into options.SchemaCache. cache.SchemaCacheEntry holds Schema *base.Schema (schema_validation/schema_resources.go -> ToCacheEntry), and that field is what drags in the document's SpecIndex/root node in the cache.

Why the usual cleanup we tried doesn't help

So we already tried some other cleanup methods before committing to finding why some specs were leaking pretty large amounts of memory:

• RuleSetExecutionResult.Release() releases vacuum-owned documents/indexes and calls libopenapi.ClearAllCaches(), but that clears the libopenapi caches — not the SchemaCache living inside vacuum's (schema rule) parser.globalValidator.
• The caller releasing its own Document (and even Rolodex) does nothing, because the global cache still holds a strong reference to the schema -> index -> tree.

Impact

Any long-lived process (a validation server, a watcher, a CLI validating many files in one run) that lints with a ruleset containing schema-function rules retains full parsed documents in memory permanently. For large specs this is 100s of MiB and eventually OOMs the process. Rulesets without schema-function rules are unaffected (see control).

Suggested fixes

• In vacuum itself, some options I tried/thought of which fixed it/could fix it:
  1. Create parser.globalValidator with caching disabled for this cross-document use schema_validation.NewSchemaValidator(config.WithSchemaCache(nil))
  2. Expose a reset for the global validator and call it from RuleSetExecutionResult.Release() (next to libopenapi.ClearAllCaches())
  3. Make the validator passable to the schema rule
  4. Scope the validator per RuleSetExecution instead of a process global.
• In libopenapi-validator: Looking at how the cache is used *in vacuum and that library itself (so I have not looked at all usages, so this might be wrong): drop the Schema *base.Schema field from cache.SchemaCacheEntry (and stop setting it in ToCacheEntry).
  At the only place a cache entry is read back (schema_validation/validate_schema.go, the SchemaCache.Load hit path) only RenderedInline, RenderedNode, ResourceNodes and CompiledSchema are used. Since Schema seems write-only, removing it severs the -> SpecIndex -> tree retention while keeping the compiled-schema cache intact. (Verified locally: with this one change the reproduction above stays flat, findings unchanged. Our own testcase that surfaced this is a 3.5MB OpenAPI spec with custom ruleset, among which are 8 schema rules, the memory usage (before->after validation) went from 6mb->~130mb to 6mb->6mb)

[nd-jharn]

I can create a PR on libopenapi-validator or vacuum if you'd like; if so please provide some feedback on what your preference for a solution direction would be.
I think the libopenapi-validator would be the most conclusive fix, but I dont know about the impact of it. Alternatively making the global validator runtime(as in RuleSetExecution) scoped, and providing a global default without a cache would be the most compatible, most versatile solution probably. (Again IMO, lemme know what you think)