Skip to content

Redact sensitive headers from debug logs#2279

Open
pavel-ptashyts wants to merge 2 commits into
AsyncHttpClient:mainfrom
maygemdev:perf/redact-sensitive-log-headers
Open

Redact sensitive headers from debug logs#2279
pavel-ptashyts wants to merge 2 commits into
AsyncHttpClient:mainfrom
maygemdev:perf/redact-sensitive-log-headers

Conversation

@pavel-ptashyts

@pavel-ptashyts pavel-ptashyts commented Jul 20, 2026

Copy link
Copy Markdown
Contributor

Summary

  • redact authentication and cookie values in HTTP and WebSocket debug logs
  • expose a shared sensitive-header predicate and redaction value for DefaultRequest
  • allow sensitive logging through an explicit startup-only JVM property or environment variable
  • avoid rendering complete request objects in retry and replay logs by default
  • add regression coverage for Netty message formatting and request string output

Root cause

Debug logging passed complete Netty request and response objects to SLF4J. Their
string representations included every header value, so user-supplied
Authorization headers and other credentials could be written to application
logs.

Sensitive logging

Sensitive values remain redacted by default. Full header values can be enabled
before HttpMessageFormatter is initialized with either:

  • JVM property: -Dorg.asynchttpclient.enableSensitiveLogging=true
  • environment variable: AHC_ENABLE_SENSITIVE_LOGGING=true

The JVM property takes precedence over the environment variable. The setting is
read once and remains immutable for the process. Enabling it can expose
credentials and session data in logs.

Validation

  • JDK 11 default-redaction focused suite: 32 tests passed, 2 startup-mode tests skipped
  • JDK 11 system-property, environment-variable, and precedence startup cases passed
  • JDK 11 ./mvnw clean verify: 1,307 tests passed, 17 skipped; Javadocs and Revapi passed

Closes #1739
Closes #1740

Codex on behalf of Pavel Ptashyts

Request and response debug logging rendered complete Netty messages,
which exposed Authorization values and other credentials.

Format logged HTTP messages with shared redaction for authentication and
cookie headers. Reuse the same public predicate in DefaultRequest, and
avoid rendering complete requests from retry and replay paths.

Add regression coverage for HTTP message formatting and request string
representations.

Fixes AsyncHttpClient#1739
Fixes AsyncHttpClient#1740

Codex on behalf of Pavel Ptashyts

Co-Authored-By: Codex <codex@openai.com>
@hyperxpro

Copy link
Copy Markdown
Member

I would like to have a system property / environment variable to enable non-redacted logs. It is helpful in debugging at CI pipeline.

Probably like AHC_ENABLE_SENSITIVE_LOGGING=true?

Of course feel free to choose a better key.

Redaction is secure by default, but it can prevent diagnosis of
authentication failures that only occur in CI.

Read a JVM property or environment variable once when the HTTP message
formatter is initialized. Fold that immutable setting into
isSensitiveHeader so both Netty logging and DefaultRequest use the same
decision without repeated configuration lookups.

Refs AsyncHttpClient#1739
Refs AsyncHttpClient#1740

Codex on behalf of Pavel Ptashyts

Co-Authored-By: Codex <codex@openai.com>
@pavel-ptashyts

Copy link
Copy Markdown
Contributor Author

@hyperxpro Implemented in 2a7f11b84.

Sensitive values remain redacted by default. Unredacted logging can be enabled before HttpMessageFormatter is initialized with either:

  • JVM property: -Dorg.asynchttpclient.enableSensitiveLogging=true
  • environment variable: AHC_ENABLE_SENSITIVE_LOGGING=true

The value is calculated once and remains immutable for the process. The JVM property takes precedence over the environment variable, including allowing an explicit false to keep redaction enabled. Full JDK 11 verification passes.

Thanks for the suggestion.


private static final String ENABLE_SENSITIVE_LOGGING_PROPERTY = "org.asynchttpclient.enableSensitiveLogging";
private static final String ENABLE_SENSITIVE_LOGGING_ENVIRONMENT_VARIABLE = "AHC_ENABLE_SENSITIVE_LOGGING";
private static final boolean SENSITIVE_LOGGING_ENABLED = isSensitiveLoggingEnabled();

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is read once when the class loads and never rechecked. Surefire in this repo reuses a single JVM across the whole client module by default, so whichever test touches this class first locks the value for the rest of the run. That means the EnabledIfSystemProperty and EnabledIfEnvironmentVariable tests further down just get skipped in a normal build instead of actually running, so the enabled path has no real coverage right now. Might be worth resolving this per call or behind a seam that a test can override.

private static StringBuilder appendHeaders(StringBuilder value, HttpHeaders headers) {
for (Map.Entry<String, String> header : headers) {
value.append('\n').append(header.getKey()).append(": ")
.append(isSensitiveHeader(header.getKey()) ? REDACTED : header.getValue());

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This uses a colon followed by a space between header name and value, but DefaultRequest toString uses a colon with no space for the same kind of dump. Small thing, but the two renderings will drift if they stay separate.

* @param name the header name
* @return {@code true} for authentication and cookie headers when sensitive logging is disabled
*/
public static boolean isSensitiveHeader(CharSequence name) {

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

DefaultRequest in the base package now depends on this method and REDACTED. Since the base Request model has not reached into netty handler before, it might be cleaner to move this predicate and the redacted constant into a neutral utility package and have this class consume it instead.

return value;
}

private static boolean isSensitiveLoggingEnabled() {

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This reads from an environment variable that can be inherited from a container or orchestration setup. Worth considering a one time warn log when this ends up enabled, so it shows up in the logs instead of only being discoverable after something has already leaked.

import java.util.List;
import java.util.Map;

import static org.asynchttpclient.netty.handler.HttpMessageFormatter.REDACTED;

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This pulls the base Request model into a dependency on the netty handler package, which has been treated as transport level internal up to now. See the note on HttpMessageFormatter about moving the shared predicate somewhere neutral instead.

sb.append(header.getKey());
sb.append(':');
sb.append(header.getValue());
sb.append(isSensitiveHeader(header.getKey()) ? REDACTED : header.getValue());

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Headers are redacted here but formParams a bit further down in this same method still get appended in full. If a password ever comes through as a form field it would still show up in this output untouched, worth checking if that is in scope for the issues this closes.

future.touch();

LOGGER.debug("\n\nReplaying Request {}\n for Future {}\n", newRequest, future);
LOGGER.debug("\n\nReplaying request '{}' to '{}'\n for Future {}\n", newRequest.getMethod(), newRequest.getUri(), future);

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Checked this one, NettyResponseFuture toString does not render the underlying NettyRequest headers since NettyRequest has no toString override, so this falls back to identity hash and does not leak anything today.

Might be worth a short comment here noting that this is load bearing, so a future toString added to NettyRequest does not quietly reopen this.

}

@Test
@EnabledIfSystemProperty(named = "org.asynchttpclient.enableSensitiveLogging", matches = "(?i)true")

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same concern as on HttpMessageFormatter, this test and the one below it get silently skipped in a normal run rather than exercised, so the enabled path is effectively untested in CI as configured.

@hyperxpro

Copy link
Copy Markdown
Member

Also please don't rebase your PRs as it spams my inbox :)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

authorization headers sensitive data leaks in debug logs authorization headers exposed in log

2 participants