Skip to content

[fix][fn] Ack late tuples in window functions under ATLEAST_ONCE - #26651

Open
Dream95 wants to merge 1 commit into
apache:masterfrom
Dream95:fix_fn_late_msg_unack
Open

Dream95 wants to merge 1 commit into
apache:masterfrom
Dream95:fix_fn_late_msg_unack

Conversation

@Dream95

@Dream95 Dream95 commented Sep 19, 2026

Copy link
Copy Markdown
Contributor

Fixes #26577

Motivation

Window functions ack ATLEAST_ONCE records in processWindow, after the window fires. A late tuple (timestamp behind the watermark) is not added to WindowManager, so that ack never happens.
The message stays unacked, gets redelivered, and can push the consumer toward maxUnackedMessages.
ATMOST_ONCE already acks at the start of process(), so it is unaffected.

Modifications

WindowFunctionExecutor.process() acks late tuples when the guarantee is ATLEAST_ONCE.
If lateDataTopic is not set, ack after the existing "will not be processed" log. If it is set, ack in thenAccept after the late-data send succeeds. A failed send leaves the record unacked.

Verifying this change

  • Make sure that the change passes the CI checks.
    This change added tests and can be verified as follows:
  • testLateTupleAckedWithAtleastOnce covers the no-late-data-topic case
  • testLateTupleAckedAfterLateDataSend checks that ack waits for a successful send
  • testLateTupleNotAckedWhenLateDataSendFails checks that a failed send does not ack

Does this pull request potentially affect one of the following parts:

If the box was checked, please highlight the changes

  • Dependencies (add or upgrade a dependency)
  • The public API
  • The schema
  • The default values of configurations
  • The threading model
  • The binary protocol
  • The REST endpoints
  • The admin CLI options
  • The metrics
  • Anything that affects deployment

Signed-off-by: Dream95 <zhou_8621@163.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug][Functions] Event-time window function never acks late messages under ATLEAST_ONCE

1 participant