Skip to content

perf(pd): reduce disaggregated serving latency - #1481

Open
sufubao wants to merge 2 commits into
ModelTC:mainfrom
sufubao:perf/pd-reduce-disaggregated-latency
Open

perf(pd): reduce disaggregated serving latency#1481
sufubao wants to merge 2 commits into
ModelTC:mainfrom
sufubao:perf/pd-reduce-disaggregated-latency

Conversation

@sufubao

@sufubao sufubao commented Aug 24, 2026

Copy link
Copy Markdown
Collaborator

Summary

  • make detokenization and PD KV polling intervals configurable through environment variables
  • reduce router-loop overhead by making metric gauge cadence configurable and avoiding uvloop busy-yields for sub-millisecond intervals

The request shared-memory handle cleanup was extracted into #1482.

Tests

  • pre-commit run black --files
  • pre-commit run flake8 --files

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant