ExporterExporter improvements#426
Open
inqrphl wants to merge 13 commits into
Open
Conversation
added 2 commits
July 21, 2026 16:34
the prefix is set using the "url prefix" line in the configuration section /settings/ExporterExporter/server HandlerExporterExporter.GetMappings were returning prefixed paths already, But HandlerExporterExporter.ServeHTTP was not checking for them
…ctory instead of failing returning an error would propagate up and stop snclient from starting decision: ignore exporterexporter config files that fail to be parsed. even if all config files are failing, do not return an error
inqrphl
marked this pull request as draft
July 21, 2026 15:29
inqrphl
marked this pull request as ready for review
July 21, 2026 15:29
inqrphl
marked this pull request as draft
July 21, 2026 15:29
…roperly restart exporter exporter
… restarts and remove the stop(), init(), start() loop when a request to exporter_exporter comes, it dynamically picks which module to use from Handler.ExporterExporter.modules the mappings it passes to listener are always the same, l.urlPrefix + "/list" and l.urlPrefix + "/proxy" the listener does not need to be restarted on a file watcher cahnge. only the loaded modules need to be changed, which can be done using reloadModules() reload the modules, and protect modules using a mutex during its reload
…estricts expoter modules based on the method they want to use mainly to restrict usage of the 'exec' method so that someone can not just write a file in the modules directory and then execute something malicious. that would not even need a restart of snclient due to newly added file watcher
inqrphl
force-pushed
the
exporter-improvements
branch
from
July 22, 2026 11:35
b92c5b2 to
2d0b441
Compare
added 3 commits
July 22, 2026 14:09
added both in HTML and in JSON outputs
…unctionality, including newly added functions - checks if bad module configuration files of different reasons are skipped, and valid module configuration files are added - check if 'allowed methods' are working properly, including comma separation of allowed modules - check if snclient agent and exporter_exporter starts, even if no valid module files exist - check if url prefix is used, and is correctly listed under /list endpoints - check if adding a new file in the module config directory triggers a reload, and the new file is listed in /list - check if /list works without a module dir also on the /list endpoint, always append the urlPrefix to links of the modules and change the HTTP status code to OK.
Contributor
Author
|
I did implement the file watcher now, and it has a debounce period of 5 seconds. The file watcher sends events through a channel. To not reload modules for every single event, I implemented a debounce mechanism. Meaning it ignores events for 5 seconds after a reload of modules was triggered. Is that ok? Secondly, the watcher does not care if the event added/modified/deleted a file that ends with .yml or .yaml extension. I did this so that even writing to a "timestamp" file the current time can trigger a reload of the modules. Thought this would help out due script. I can either leave it out like that, or throw away events that dont work on files that arent YAML |
… the modules dir watcher and by default it is off it is better to set it default off, so only users wanting this toggle it on.
…ConfigDirectoryNoReloadWhenWatcherDisabled - very nice name, checks if the file watcher functionality is toggled off
inqrphl
marked this pull request as ready for review
July 22, 2026 13:57
inqrphl
marked this pull request as draft
July 22, 2026 14:15
….yml and .yaml extension files. it ignores other file changes ai change - add another test TestExporterExporterFileWatcherIgnoresNonYaml that checks for this behavior - .yml file is added while .txt file is ignored after refresh
inqrphl
marked this pull request as ready for review
July 22, 2026 14:26
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fix 'url prefix' not working. ExporterExporter was registering the prefixes correctly so that listener routed them to the ExporterExporter. But it was not removing the prefix in ServeHTTP before checking if the remaining portion is "/list" or "/proxy"
Read modules: The ExporterExporter can specify a module configuration path, in which multiple module config files can be put inside. If opening/parsing/validating one of the configs failed, it would return an error that would propage upwards to the snclient module initialization system. This would make snclient module agent exit, due to ExporterExporter module not being able to start.
In this PR, it just skips over the modules that cause errors, this can be either due to bad YAML syntax, file being unparsable as a module config file, file not being readable etc.
Even if all of the module config files are wrong, it does not return an error and cause snclient to stop.
File watcher: The module directory is now watched for changes using fsnotify. This uses a new library: fsnotify , which uses inodes on Linux and 'ReadDirectoryChangesW' on windows. Seems to be more robust than manually checking files.
If a file is written into, deleted or renamed an event is fired. The events that are related to non .yml / .yaml files are ignored. If so, the modules are regenerated from the module config folder. Until the regeneration is complete, the exporter_exporter uses the old parsed config, but once the new one is ready it is switched to new version. The change is protected with a mutex. After a change new events that come for 5 seconds are ignored in a debounce period.
I checked this both by hand in Linux and Windows, seems to be notificing and firing events, and reloading the modules index without problems.
the file watcher is toggleable using the 'modules dir watcher' option. By default it is false, so only users needing this may turn it on knowing its possible side effects.
Add 'allowed methods' configuration point.
Exporter_exporter has multiple ways of getting results from a prometheus conforming exporter. Https does a query and gets the output, file reads from a file (which is presumably refreshed from time to time) and exec calls an exporter directly.
Add a whitelist in the exporter_exporter configuration section called 'allowed methods', if its populated it will reject module definitions that dont use its methods. Is meant to protect against 'exec' types of modules.