Skip to content

Only ignore whitespace between adjacent encoded words - #231

Open
decius7bc wants to merge 1 commit into
DockYard:masterfrom
decius7bc:fix-encoded-word-whitespace
Open

decius7bc wants to merge 1 commit into
DockYard:masterfrom
decius7bc:fix-encoded-word-whitespace

Conversation

@decius7bc

Copy link
Copy Markdown

#168 added whitespace stripping between adjacent RFC 2047 encoded-words, but the regex wasn't anchored. It removed whitespace before every =? in the rest of the value, so a plain word followed by an encoded-word lost its separating space:

raw =
  "Subject: =?utf-8?Q?Gr=C3=BC=C3=9Fe?= from the\r\n" <>
    " =?utf-8?Q?M=C3=BCller?= family\r\n" <>
    "From: a@example.com\r\n\r\nbody\r\n"

Mail.Parsers.RFC2822.parse(raw).headers["subject"]
# before: "Grüße from theMüller family"
# after:  "Grüße from the Müller family"

Folding isn't the cause. A one-line value like =?utf-8?Q?a?= x =?utf-8?Q?b?= decoded to "a xb" too.

RFC 2047 §6.2 says only linear white space between two adjacent encoded-words is ignored. Whitespace next to ordinary text must stay.

Changes proposed in this pull request

The regex in parse_encoded_word/2 now:

  • only matches at the start of the remainder (\A), i.e. directly after the encoded-word that was just decoded;
  • only matches linear white space, 1*([CRLF] (SP / HTAB)), instead of \s. \s also matches a bare CR, LF, VT or FF;
  • only strips it when a complete encoded-word (=?charset?[BbQq]?text?=) follows, so a stray =? that isn't an encoded-word keeps the space before it.

Tests

  • Adjacent encoded-words still decode to "ab": on one line, and folded with a space or a tab (the Ignore space between two encoded words (as per RFC 2047) #168 behaviour).
  • Bare CR, LF, VT and FF between encoded-words are kept.
  • Whitespace between an encoded-word and plain text is kept, including the folded Subject above and a stray =?.

🤖 Generated with Claude Code

The whitespace stripping added in DockYard#168 was unanchored and removed
whitespace before every `=?` in the remainder, so plain text followed
by an encoded word lost its separating space (e.g. "a x =?..?=" became
"a xb"). Per RFC 2047 §6.2, only linear white space between two
adjacent encoded words is ignored.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@bcardarella

Copy link
Copy Markdown
Member

@andrewtimberlake thoughts?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants