Skip to content

feat(rpent): add event-driven ActiveVLN navigation - #15

Draft
yufoo1 wants to merge 9 commits into
BUAA-CI-LAB:mainfrom
yufoo1:feat/rpent-activevln-navigation
Draft

yufoo1 wants to merge 9 commits into
BUAA-CI-LAB:mainfrom
yufoo1:feat/rpent-activevln-navigation

Conversation

@yufoo1

@yufoo1 yufoo1 commented Nov 2, 2026 •

Copy link
Copy Markdown
Collaborator

Summary

Add RPent planning above ActiveVLN navigation: event-driven local skills, compact visual feedback, persistent task progress, bounded tool execution, cancellation and a paired Habitat evaluator. Go2 releases its motion lease before planner waits. Local STOP is feedback to the planner, not automatic task success.

Preserve both the explicit Codex planner backend and the experimental shared-model backend. The latter borrows the navigator's resident model, checks model-instance identity, validates tool arguments, and rejects success while saved goals remain. Optional ordered goals and literal task-span tools are included; they do not infer visual completion themselves.

Dependencies and review scope

This PR targets main and includes its Go2 dependency until that PR merges. The original mixed development branch and uncommitted research workspaces are preserved. The vLLM integration is independent and not included.

Verification

  • After rebasing onto the minimal Go2 PR: retained RPent navigation/event/shared-model/task-source/interactive checks, 96 passed.
  • Ruff lint/format, git diff --check and commit policy: passed. RPent implementation and model-service pin are unchanged by the dependency update.

No robot motion, planner API calls or fresh GPU experiments were performed while preparing this PR. Shared-model planning quality/latency, optional text CUDA graphs and physical Go2 use remain experimental; this PR does not claim improved task success or a speedup. Existing paired-evaluation results retain their protocol and limitations.

Process boundaries

RPent stays in optional agents/rpent/navigation, with its dependencies declared there. Habitat metrics remain evaluator-only; inference receives RGB/instructions. Infer owns model execution; Run owns observations, sessions and action execution. No raw videos, checkpoints, credentials or private deployment configuration are included.

Signed-off-by: yufoo1 <yufoo1.cs@gmail.com>
Signed-off-by: yufoo1 <yufoo1.cs@gmail.com>
Signed-off-by: yufoo1 <yufoo1.cs@gmail.com>
Signed-off-by: yufoo1 <yufoo1.cs@gmail.com>
Signed-off-by: yufoo1 <yufoo1.cs@gmail.com>
Signed-off-by: yufoo1 <yufoo1.cs@gmail.com>
Signed-off-by: yufoo1 <yufoo1.cs@gmail.com>
Signed-off-by: yufoo1 <yufoo1.cs@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant