[nikkoxgonzales]

What does this PR do?

Supersedes #44805 (authored by @benegessarit, cherry-picked with authorship preserved — the commit is still Author: David Beyer <bdavey163@gmail.com>). That PR was approved by two reviewers and then closed unmerged by its author; the underlying issue #44804 remains open. This re-lands the same fix, adapted to current main (the client-creation code it patched has since been refactored into _new_embedded_client / _new_cloud_client, and _load_config now dedups file reads through read_json_or_empty).

The bug: a long-running gateway loads hindsight/config.json once at initialize() and holds it in self._config. When the embedded daemon later dies (idle shutdown, crash) and a stale-connection retry recreates the client (_run_hindsight_operation → self._client = None → _get_client()), the embedded manager re-materializes ~/.hindsight/profiles/<p>.env from the initialize()-time snapshot — silently reverting any config.json edits made since (observed: an llm_model bump undone by a pre-restart respawn).

The fix: client creation is the daemon (re)spawn boundary, so re-read the on-disk config there. File-gated via a new _load_file_config() helper (_load_config minus the env fallback): env-only setups keep their existing config, and a respawn only adopts disk state when a config file actually exists. Session-shaping fields (banks, recall tuning) deliberately keep initialize()-time values for in-session stability.

Related Issue

Fixes #44804

Type of Change

• 🐛 Bug fix (non-breaking change that fixes an issue)

Changes Made

• plugins/memory/hindsight/__init__.py
  • New _load_file_config() -> dict | None: disk-only config read (profile-scoped $HERMES_HOME/hindsight/config.json, then legacy ~/.hindsight/config.json), None when no file parses. _load_config() now delegates to it before the env fallback.
  • _new_embedded_client(): re-reads config.json at client (re)creation — the daemon respawn boundary — refreshing self._config and self._llm_base_url when a file exists.
• tests/plugins/memory/test_hindsight_provider.py
  • New test_get_client_rereads_disk_config_at_respawn: writes a config, snapshots it as initialize()-time state, edits the file on disk, recreates the client, and asserts the new llm_model/llm_base_url reach HindsightEmbedded.
  • test_get_client_passes_idle_timeout_to_hindsight_embedded: made hermetic — points the config lookup at an empty tmp_path and stubs _ensure_client_dependency so the fake hindsight module stands in for the optional hindsight-client dep.

How to Test

1. Configure hindsight local_embedded mode; start a long-running gateway so initialize() snapshots config.json.
2. Edit ~/.hermes/hindsight/config.json (e.g. bump llm_model) while the gateway keeps running.
3. Let the embedded daemon idle-shutdown (or kill it), then trigger a memory operation — the stale-connection retry recreates the client. Before this fix the respawned daemon's ~/.hindsight/profiles/<p>.env reverted to the old llm_model; after, it picks up the edit.

Automated: scripts/run_tests.sh tests/plugins/memory/test_hindsight_provider.py -q -k test_get_client → 2 passed. Full file: 79 passed; the 7 remaining failures are pre-existing on upstream/main in this environment (hindsight_client_api optional dep not installed — identical failures on a clean upstream/main checkout).

Checklist

Code

• I've read the Contributing Guide
• My commit messages follow Conventional Commits (fix(scope):, feat(scope):, etc.)
• I searched for existing PRs to make sure this isn't a duplicate — this intentionally supersedes closed-but-approved fix(hindsight): re-read on-disk config when recreating the embedded client #44805; related open PRs (fix(hindsight): preserve embedded profile env #26890, fix(memory/hindsight): preserve extended HINDSIGHT_API_* daemon config through env materialization #81844, fix(memory/hindsight): make HINDSIGHT_* env vars overlay config.json (#51166) #51375, fix(hindsight): compare only plugin-managed keys before restarting embedded daemon #104324, fix(plugins): don't stop externally-managed hindsight daemons on config change #83003, fix(hindsight): defer restart of a healthy shared embedded daemon #60358) touch adjacent env-materialization code but none conflict semantically (none merged; fix(memory/hindsight): make HINDSIGHT_* env vars overlay config.json (#51166) #51375's env-overlay direction would compose with this fix if it lands)
• My PR contains only changes related to this fix/feature (no unrelated commits)
• I've run pytest tests/ -q and all tests pass — see note above: the only failures are pre-existing environment issues identical on upstream/main
• I've added tests for my changes (required for bug fixes, strongly encouraged for features)
• I've tested on my platform: Windows 11 (Git Bash, repo run_tests.sh runner)

Documentation & Housekeeping

• I've updated relevant documentation (README, docs/, docstrings) — or N/A
• I've updated cli-config.yaml.example if I added/changed config keys — or N/A
• I've updated CONTRIBUTING.md or AGENTS.md if I changed architecture or workflows — or N/A
• I've considered cross-platform impact (Windows, macOS) per the compatibility guide — or N/A
• I've updated tool descriptions/schemas if I changed tool behavior — or N/A

For New Skills

N/A — not a skill.

Screenshots / Logs

$ scripts/run_tests.sh tests/plugins/memory/test_hindsight_provider.py -q -k test_get_client
[100.0% | 88/~88 | ✓2 | ✗0] ✓ tests\plugins\memory\test_hindsight_provider.py (2✓, 6.8s)
=== Summary: 1 files, 2 tests passed, 0 failed (100% complete) ===

Credit: original fix and regression test by @benegessarit in #44805 — preserved via cherry-pick (git log shows Author: David Beyer <bdavey163@gmail.com>); conflict resolution onto current main and test hermeticity only.

[teknium1]

Closing: the bundled Hindsight provider this PR patches has moved out of this repo.

Thanks @nikkoxgonzales for this contribution. In #119888 (merge 9d799e0531c; removal commit 4cbf862abe4) the in-tree plugins/memory/hindsight/ provider was removed — Hindsight now installs from the plugin catalog and its code lives in vectorize-io/hindsight hindsight-integrations/hermes (maintained by @nicoloboschi). There is no longer any code in this repo for the Hindsight half of this PR to patch, so we are closing every open PR against the bundled provider rather than leaving them stranded.

Triage notes:

• Touches only plugins/memory/hindsight/init.py and tests/plugins/memory/test_hindsight_provider.py: adds _load_file_config() and re-reads config.json inside _new_embedded_client() so a client recreation after daemon death picks up on-disk edits (re-land of fix(hindsight): re-read on-disk config when recreating the embedded client #44805 by @benegessarit, issue Hindsight embedded: daemon respawn re-materializes profile .env from stale initialize()-time config #44804). Never landed in core before the provider was removed in hindsight moves to the plugin catalog (auto-migrated on first start / hermes update) #119888 (9d799e0; git log -S'_load_file_config' -- plugins/memory/hindsight/ is empty). Still relevant at the catalog pin: upstream _new_embedded_client (hindsight-integrations/hermes/init.py:698, cfg = self._config) and the retry path at :754 still reuse the initialize()-time snapshot, so this belongs in the upstream fixes issue.
• Still relevant at the catalog pin (dc750388)? Yes — the same code is at hindsight-integrations/hermes/__init__.py:698 in the upstream tree. It is listed with your credit in Fixes from Hermes-side PRs worth carrying into hindsight-integrations/hermes vectorize-io/hindsight#4662 so it is not lost; if you want to carry the fix yourself, please open it against vectorize-io/hindsight — it would be welcome there.

If you believe this was closed in error, comment and we will reopen.

(Bulk-closed in the hindsight-move close pass.)