Dual Arm Real Life Setup

This repository contains modular controllers and test scripts for a real-world robotic pipeline involving an xArm7 and an xArm6, Dynamixel Grippers, RealSense RGB-D cameras, and remote SAM segmentation.

Hardware & 3D Printing

The gripper used in this project consists of custom 3D printed parts designed for Dynamixel X-series motors.

Printing Instructions

The STL files are located in gripper/3d_printed_parts/. For a complete gripper, you need to print:

• Base (base.stl): The main chassis that mounts to the motor.
• Gear (gear.stl): The internal drive gear.
• Left Tip (left_tip.stl): The left-side finger.
• Right Tip (right_tip.stl): The right-side finger.
• Cover (cover.stl or cover.3mf): The protective housing.

Setup Instructions

Prerequisites

• Python 3.10+
• uv (recommended for package management)

Installation

1. Clone the repository and navigate to the root:

   git clone https://github.com/wig-nesh/dual-arm-setup-RRC.git
   cd dual-arm-setup-RRC

2. Create environment and install dependencies:

   uv sync

Usage

Full Pipeline

The end-to-end pipeline is split into two stages, each run via a thin entry point in scripts/.

1. Grasp generation - capture RGB-D, segment with SAM, build a pointcloud, and query the grasp-model server. Results are saved under run_data/<timestamp>/.

   uv run scripts/generation.py \
     --sam-url http://dualarm@orion.rrcx.tk:8000 \
     --model-url http://localhost:8000 \
     [--ffs-url http://<ffs-host>:<port>]

   Omit --ffs-url to use the RealSense's native depth.

2. Grasp execution - load a run, build per-arm poses, and execute the grasp on both xArms.

   uv run scripts/execution.py --run-dir run_data/<timestamp> [--direct] [--grasp-idx 0]

   Flags: --direct (skip manual jog), --grasp-idx N, --dry-run, --record-realsense, --manual-xyzrpy.

Hardware Tests

xArm Control

Run the single arm test with PyBullet visualization:

uv run xarm/xarm_test.py

Run the dual arm simultaneous movement test (no PyBullet yet):

uv run xarm/dual_arm_test.py

Gripper Control

Test the dual gripper setup (Left: ID 0, Right: ID 1):

uv run gripper/dual_gripper_test.py

• L: Toggle Left
• R: Toggle Right

RealSense Capture

Capture RGB-D data to the data/ folder:

uv run realsense/realsense_test.py

• SPACE: Capture Frame
• Q: Quit

SAM Client Test

Run interactive segmentation on your webcam feed:

uv run client/webcam_sam_test.py --url http://dualarm@orion.rrcx.tk:8000

• Left Click: Select point to segment.
• Q: Quit.

Grasp Model Client Test

To run the Grasp Model client, first setup SSH Port Forwarding in a separate terminal:

ssh -L 8000:gnode117:8000 ayushk02@ada.iiit.ac.in

Then, run the prediction test with interactive Open3D visualization:

uv run client/model_predict_test.py --url http://localhost:8000

Data Storage

Each generation run writes to its own timestamped folder under run_data/ (git-ignored):

• rgb.png, depth.npy, intrinsics.json - raw capture and camera calibration.
• sam_mask.png / sam_mask_vis.png - SAM segmentation mask.
• pcd_scaled.ply - processed, scaled pointcloud sent to the model.
• marker_transform.npy - camera-to-world ArUco transform (if markers were detected).
• grasps.npz - grasp pairs and scores returned by the model server (consumed by scripts/execution.py).